The controller does not have regulated voltage and I'm not aware of any ebike that does. Someone will correct me if I'm wrong. The behavior you describe is typical of bikes at low voltage and this is the result you'll get every time.
If I am not mistaken high quality controllers from ASI do have it. I should check their specs though.