OSSInnovation
/
mindspore
1
0
Fork 26
Code Issues Pull Requests 7 Releases Wiki Activity

modify lr comment

This commit is contained in:
Jiaqi 2020-09-19 21:02:30 +08:00
parent 4e105dabf4
commit 8dd4517af3
1 changed files with 6 additions and 0 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

6
mindspore/nn/learning_rate_schedule.py
View File

@ -62,10 +62,12 @@ class ExponentialDecayLR(LearningRateSchedule):
decayed\_learning\_rate[i] = learning\_rate * decay\_rate^{p} decayed\_learning\_rate[i] = learning\_rate * decay\_rate^{p}
Where : Where :
.. math:: .. math::
p = \frac{current\_step}{decay\_steps} p = \frac{current\_step}{decay\_steps}
If `is_stair` is True, the formula is : If `is_stair` is True, the formula is :
.. math:: .. math::
p = floor(\frac{current\_step}{decay\_steps}) p = floor(\frac{current\_step}{decay\_steps})
@ -116,10 +118,12 @@ class NaturalExpDecayLR(LearningRateSchedule):
decayed\_learning\_rate[i] = learning\_rate * e^{-decay\_rate * p} decayed\_learning\_rate[i] = learning\_rate * e^{-decay\_rate * p}
Where : Where :
.. math:: .. math::
p = \frac{current\_step}{decay\_steps} p = \frac{current\_step}{decay\_steps}
If `is_stair` is True, the formula is : If `is_stair` is True, the formula is :
.. math:: .. math::
p = floor(\frac{current\_step}{decay\_steps}) p = floor(\frac{current\_step}{decay\_steps})
@ -171,10 +175,12 @@ class InverseDecayLR(LearningRateSchedule):
decayed\_learning\_rate[i] = learning\_rate / (1 + decay\_rate * p) decayed\_learning\_rate[i] = learning\_rate / (1 + decay\_rate * p)
Where : Where :
.. math:: .. math::
p = \frac{current\_step}{decay\_steps} p = \frac{current\_step}{decay\_steps}
If `is_stair` is True, The formula is : If `is_stair` is True, The formula is :
.. math:: .. math::
p = floor(\frac{current\_step}{decay\_steps}) p = floor(\frac{current\_step}{decay\_steps})