CyberZHG / keras-radam / 47

99%

push

travis-ci

Change lr decay logic (#26)

* Update optimizers.py

Changed `lr` decay logic

Once `t > warmup_steps`, the movement toward `min_lr` is abrupt. _Ex:_ `self.total_steps = 200`, `warmup_steps = 100`, `self.lr = 1e-3`, `self.min_lr = 1e-4`:

- `t = 99 `  --> `lr = 9.9e-4`
- `t = 100` --> `lr = 1.0e-3` 
- `t = 101` --> `lr = 5.455e-4`

Doubt this is intended behavior. Plot ([code](https://puu.sh/E8HuR/a07ccd9e6f.txt)) comparison below:

<hr>

![image](https://user-images.githubusercontent.com/16495490/63619016-96bc7480-c5bb-11e9-9e91-e32a2a8ebde4.png)

* Fixed to account for decay_steps = 0

* Fixed styling to appease Travis

* Update optimizers.py

2 of 2 new or added lines in 1 file covered. (100.0%)

377 of 381 relevant lines covered (98.95%)

2.89 hits per line