forked from mindspore-Ecosystem/mindspore
40 lines
1.6 KiB
ReStructuredText
40 lines
1.6 KiB
ReStructuredText
|
mindspore.ops.ApplyAdagrad
|
|||
|
|
===========================
|
|||
|
|
|
|||
|
|
.. py:class:: mindspore.ops.ApplyAdagrad(update_slots=True)
|
|||
|
|
|
|||
|
|
<20><><EFBFBD><EFBFBD>Adagrad<61>㷨<EFBFBD><E3B7A8><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD>ز<EFBFBD><D8B2><EFBFBD><EFBFBD><EFBFBD>
|
|||
|
|
|
|||
|
|
Adagrad<61>㷨<EFBFBD><E3B7A8><EFBFBD><EFBFBD><EFBFBD><EFBFBD> `Adaptive Subgradient Methods for Online Learning and Stochastic Optimization <http://www.jmlr.org/papers/volume12/duchi11a/duchi11a.pdf>`_ <20><><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD>Բ<EFBFBD>ͬ<EFBFBD><CDAC><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD>ȵ<EFBFBD><C8B5><EFBFBD><EFBFBD>⣬<EFBFBD><E2A3AC><EFBFBD><EFBFBD>Ӧ<EFBFBD><D3A6>Ϊ<EFBFBD><CEAA><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD>䲻ͬ<E4B2BB><CDAC>ѧϰ<D1A7>ʡ<EFBFBD>
|
|||
|
|
|
|||
|
|
.. math::
|
|||
|
|
\begin{array}{ll} \\
|
|||
|
|
accum += grad * grad \\
|
|||
|
|
var -= lr * grad * \frac{1}{\sqrt{accum}}
|
|||
|
|
\end{array}
|
|||
|
|
|
|||
|
|
`var` <20><> `accum` <20><> `grad` <20><><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD>ѭ<EFBFBD><D1AD>ʽ<EFBFBD><CABD><EFBFBD><EFBFBD>ת<EFBFBD><D7AA><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD>ʹ<EFBFBD><CAB9><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD>һ<EFBFBD>¡<EFBFBD><C2A1><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD>Ǿ<EFBFBD><C7BE>в<EFBFBD>ͬ<EFBFBD><CDAC><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD>ͣ<EFBFBD><CDA3>ϵ;<CFB5><CDBE>ȵ<EFBFBD><C8B5><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD>ͽ<EFBFBD>ת<EFBFBD><D7AA>Ϊ<EFBFBD><CEAA><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD>߾<EFBFBD><DFBE>ȵ<EFBFBD><C8B5><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD>͡<EFBFBD>
|
|||
|
|
|
|||
|
|
**<2A><><EFBFBD><EFBFBD><EFBFBD><EFBFBD>**
|
|||
|
|
|
|||
|
|
- **update_slots** (bool) - <20>Ƿ<EFBFBD><C7B7><EFBFBD><EFBFBD><EFBFBD> `accum` <20><><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD>ΪTrue<75><65> `accum` <20><><EFBFBD><EFBFBD><EFBFBD>¡<EFBFBD>Ĭ<EFBFBD><C4AC>ֵΪ<D6B5><CEAA>True<75><65>
|
|||
|
|
|
|||
|
|
**<2A><><EFBFBD>룺**
|
|||
|
|
|
|||
|
|
- **var** (Parameter) - Ҫ<><D2AA><EFBFBD>µ<EFBFBD>Ȩ<EFBFBD>ء<EFBFBD><D8A1><EFBFBD><EFBFBD><EFBFBD>ά<EFBFBD>ȣ<EFBFBD><C8A3><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD>Ϊfloat32<33><32>float16<31><36>
|
|||
|
|
- **accum** (Parameter) - Ҫ<><D2AA><EFBFBD>µ<EFBFBD><C2B5>ۻ<EFBFBD><DBBB><EFBFBD>shape<70><65><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD>ͱ<EFBFBD><CDB1><EFBFBD><EFBFBD><EFBFBD> `var` <20><>ͬ<EFBFBD><CDAC>
|
|||
|
|
- **lr** (Union[Number, Tensor]) - ѧϰ<D1A7>ʣ<EFBFBD><CAA3><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD>Scalar<61><72><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD>Ϊfloat32<33><32>float16<31><36>
|
|||
|
|
- **grad** (Tensor) - <20>ݶȣ<DDB6>Ϊһ<CEAA><D2BB>Tensor<6F><72>shape<70><65><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD>ͱ<EFBFBD><CDB1><EFBFBD><EFBFBD><EFBFBD> `var` <20><>ͬ<EFBFBD><CDAC>
|
|||
|
|
|
|||
|
|
**<2A><><EFBFBD><EFBFBD><EFBFBD><EFBFBD>**
|
|||
|
|
|
|||
|
|
2<><32>Tensor<6F><72><EFBFBD>ɵ<EFBFBD>tuple<6C><65><EFBFBD><EFBFBD><EFBFBD>º<EFBFBD><C2BA><EFBFBD><EFBFBD><EFBFBD><EFBFBD>ݡ<EFBFBD>
|
|||
|
|
|
|||
|
|
- **var** (Tensor) - shape<70><65><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD> `var` <20><>ͬ<EFBFBD><CDAC>
|
|||
|
|
- **accum** (Tensor) - shape<70><65><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD> `accum` <20><>ͬ.
|
|||
|
|
|
|||
|
|
**<2A>쳣<EFBFBD><ECB3A3>**
|
|||
|
|
|
|||
|
|
- **TypeError** - <20><><EFBFBD><EFBFBD> `var` <20><> `accum` <20><> `lr` <20><> `grad` <20><><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD>ͼȲ<CDBC><C8B2><EFBFBD>float16Ҳ<36><D2B2><EFBFBD><EFBFBD>float32<33><32>
|
|||
|
|
- **TypeError** - <20><><EFBFBD><EFBFBD> `lr` <20>Ȳ<EFBFBD><C8B2><EFBFBD><EFBFBD><EFBFBD>ֵ<EFBFBD><D6B5>Ҳ<EFBFBD><D2B2><EFBFBD><EFBFBD>Tensor<6F><72>
|
|||
|
|
- **RuntimeError** - <20><><EFBFBD><EFBFBD> `var` <20><> `accum` <20><> `grad` <20><>֧<EFBFBD><D6A7><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD>ת<EFBFBD><D7AA><EFBFBD><EFBFBD>
|