mindspore-Ecosystem
/
mindspore
21
0
Fork 27
Code Issues Pull Requests 6 Projects Releases Wiki Activity

!46355 [2.0.0-alpha][docs] Update Lamb formula 

Merge pull request !46355 from shaojunsong/code_docs_lamb_2.0
This commit is contained in:
i-robot 2022-12-02 07:50:49 +00:00 committed by Gitee
parent 72c4202ebd 52e70c14a5
commit 1ab3010df2
No known key found for this signature in database
GPG Key ID: 173E9B9CA92EEF8F
2 changed files with 2 additions and 2 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

2
docs/api/api_python/nn/mindspore.nn.Lamb.rst
View File

@ -37,7 +37,7 @@ mindspore.nn.Lamb
/(\sqrt{\hat{\boldsymbol{v}}_{t}}+\epsilon) \\
&\hspace{5mm}\boldsymbol{w}_{t+1}^{(i)}=\boldsymbol{w}_{t}^{(i)}- \gamma_{t}
\frac{\boldsymbol{\phi}\left(\left\|\boldsymbol{w}_{t}^{(i)}\right\|\right)}
{\left\|\boldsymbol{w}_{t}^{(i)}+\lambda \boldsymbol{w}_{t}^{(i)}\right\|}\left(\boldsymbol{r}_{t}^{(i)}+
{\left\|\boldsymbol{r}_{t}^{(i)}+\lambda \boldsymbol{w}_{t}^{(i)}\right\|}\left(\boldsymbol{r}_{t}^{(i)}+
\lambda \boldsymbol{w}_{t}^{(i)}\right) \\
&\textbf{end for} \\[-1.ex]
&\newline

2
mindspore/python/mindspore/nn/optim/lamb.py
View File

@ -113,7 +113,7 @@ class Lamb(Optimizer):
/(\sqrt{\hat{\boldsymbol{v}}_{t}}+\epsilon) \\
&\hspace{5mm}\boldsymbol{w}_{t+1}^{(i)}=\boldsymbol{w}_{t}^{(i)}- \gamma_{t}
\frac{\boldsymbol{\phi}\left(\left\|\boldsymbol{w}_{t}^{(i)}\right\|\right)}
{\left\|\boldsymbol{w}_{t}^{(i)}+\lambda \boldsymbol{w}_{t}^{(i)}\right\|}\left(\boldsymbol{r}_{t}^{(i)}+
{\left\|\boldsymbol{r}_{t}^{(i)}+\lambda \boldsymbol{w}_{t}^{(i)}\right\|}\left(\boldsymbol{r}_{t}^{(i)}+
\lambda \boldsymbol{w}_{t}^{(i)}\right) \\
&\textbf{end for} \\[-1.ex]
&\newline