forked from mindspore-Ecosystem/mindspore
143 lines
6.1 KiB
ReStructuredText
143 lines
6.1 KiB
ReStructuredText
|
mindspore.nn.Optimizer
|
|||
|
|
======================
|
|||
|
|
|
|||
|
|
.. py:class:: mindspore.nn.Optimizer(learning_rate, parameters, weight_decay=0.0, loss_scale=1.0)
|
|||
|
|
|
|||
|
|
<20><><EFBFBD>ڲ<EFBFBD><DAB2><EFBFBD><EFBFBD><EFBFBD><EFBFBD>µ<EFBFBD><C2B5>Ż<EFBFBD><C5BB><EFBFBD><EFBFBD><EFBFBD><EFBFBD>ࡣ<EFBFBD><E0A1A3>Ҫֱ<D2AA><D6B1>ʹ<EFBFBD><CAB9><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD>࣬<EFBFBD><E0A3AC>ʵ<EFBFBD><CAB5><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD>һ<EFBFBD><D2BB><EFBFBD><EFBFBD><EFBFBD>ࡣ
|
|||
|
|
|
|||
|
|
<20>Ż<EFBFBD><C5BB><EFBFBD>֧<EFBFBD>ֲ<EFBFBD><D6B2><EFBFBD><EFBFBD><EFBFBD><EFBFBD>顣<EFBFBD><E9A1A3><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD>ʱ<EFBFBD><CAB1>ÿ<EFBFBD><C3BF><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD>ò<EFBFBD>ͬ<EFBFBD><CDAC>ѧϰ<D1A7>ʣ<EFBFBD>`lr` <20><><EFBFBD><EFBFBD>Ȩ<EFBFBD><C8A8>˥<EFBFBD><CBA5><EFBFBD><EFBFBD>`weight_decay`<EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD>ݶ<EFBFBD><EFBFBD><EFBFBD><EFBFBD>Ļ<EFBFBD><EFBFBD><EFBFBD>`grad_centralization`<EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD>ԡ<EFBFBD>
|
|||
|
|
|
|||
|
|
.. note::
|
|||
|
|
<20>ڲ<EFBFBD><DAB2><EFBFBD>δ<EFBFBD><CEB4><EFBFBD><EFBFBD>ʱ<EFBFBD><CAB1><EFBFBD>Ż<EFBFBD><C5BB><EFBFBD><EFBFBD><EFBFBD><EFBFBD>õ<EFBFBD> `weight_decay` Ӧ<><D3A6><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD>ƺ<EFBFBD><C6BA><EFBFBD>"beta"<22><>"gamma"<22><><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD>ͨ<EFBFBD><CDA8><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD>ɵ<EFBFBD><C9B5><EFBFBD>Ȩ<EFBFBD><C8A8>˥<EFBFBD><CBA5><EFBFBD><EFBFBD><EFBFBD>ԡ<EFBFBD><D4A1><EFBFBD><EFBFBD><EFBFBD>ʱ<EFBFBD><CAB1>ÿ<EFBFBD><C3BF><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD> `weight_decay` <20><><EFBFBD><EFBFBD>δ<EFBFBD><CEB4><EFBFBD>ã<EFBFBD><C3A3><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD>ʹ<EFBFBD><CAB9><EFBFBD>Ż<EFBFBD><C5BB><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD>õ<EFBFBD> `weight_decay`<EFBFBD><EFBFBD>
|
|||
|
|
|
|||
|
|
**<2A><><EFBFBD><EFBFBD><EFBFBD><EFBFBD>**
|
|||
|
|
|
|||
|
|
- **learning_rate (Union[float, int, Tensor, Iterable, LearningRateSchedule]):
|
|||
|
|
|
|||
|
|
- **float** - <20>̶<EFBFBD><CCB6><EFBFBD>ѧϰ<D1A7>ʡ<EFBFBD><CAA1><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD>ڵ<EFBFBD><DAB5><EFBFBD><EFBFBD>㡣
|
|||
|
|
- **int** - <20>̶<EFBFBD><CCB6><EFBFBD>ѧϰ<D1A7>ʡ<EFBFBD><CAA1><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD>ڵ<EFBFBD><DAB5><EFBFBD><EFBFBD>㡣<EFBFBD><E3A1A3><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD>ͻᱻת<E1B1BB><D7AA>Ϊ<EFBFBD><CEAA><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD>
|
|||
|
|
- **Tensor** - <20><><EFBFBD><EFBFBD><EFBFBD>DZ<EFBFBD><C7B1><EFBFBD><EFBFBD><EFBFBD>һά<D2BB><CEAC><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD>ǹ̶<C7B9><CCB6><EFBFBD>ѧϰ<D1A7>ʡ<EFBFBD>һά<D2BB><CEAC><EFBFBD><EFBFBD><EFBFBD>Ƕ<EFBFBD>̬<EFBFBD><CCAC>ѧϰ<D1A7>ʣ<EFBFBD><CAA3><EFBFBD>i<EFBFBD><69><EFBFBD><EFBFBD>ȡ<EFBFBD><C8A1><EFBFBD><EFBFBD><EFBFBD>е<EFBFBD>i<EFBFBD><69>ֵ<EFBFBD><D6B5>Ϊѧϰ<D1A7>ʡ<EFBFBD>
|
|||
|
|
- **Iterable** - <20><>̬<EFBFBD><CCAC>ѧϰ<D1A7>ʡ<EFBFBD><CAA1><EFBFBD>i<EFBFBD><69><EFBFBD><EFBFBD>ȡ<EFBFBD><C8A1><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD>i<EFBFBD><69>ֵ<EFBFBD><D6B5>Ϊѧϰ<D1A7>ʡ<EFBFBD>
|
|||
|
|
- **LearningRateSchedule** - <20><>̬<EFBFBD><CCAC>ѧϰ<D1A7>ʡ<EFBFBD><CAA1><EFBFBD>ѵ<EFBFBD><D1B5><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD>У<EFBFBD><D0A3>Ż<EFBFBD><C5BB><EFBFBD><EFBFBD><EFBFBD>ʹ<EFBFBD>ò<EFBFBD><C3B2><EFBFBD><EFBFBD><EFBFBD>step<65><70><EFBFBD><EFBFBD>Ϊ<EFBFBD><CEAA><EFBFBD>룬<EFBFBD><EBA3AC><EFBFBD><EFBFBD> `LearningRateSchedule` ʵ<><CAB5><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD>㵱ǰѧϰ<D1A7>ʡ<EFBFBD>
|
|||
|
|
|
|||
|
|
- **parameters (Union[list[Parameter], list[dict]])** - <20><><EFBFBD><EFBFBD><EFBFBD><EFBFBD> `Parameter` <20><><EFBFBD>ɵ<EFBFBD><C9B5>б<EFBFBD><D0B1><EFBFBD><EFBFBD>ֵ<EFBFBD><D6B5><EFBFBD><EFBFBD>ɵ<EFBFBD><C9B5>б<EFBFBD><D0B1><EFBFBD><EFBFBD><EFBFBD><EFBFBD>б<EFBFBD>Ԫ<EFBFBD><D4AA><EFBFBD><EFBFBD><EFBFBD>ֵ<EFBFBD>ʱ<EFBFBD><CAB1><EFBFBD>ֵ<EFBFBD><D6B5>ļ<EFBFBD><C4BC><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD>"params"<22><>"lr"<22><>"weight_decay"<22><>"grad_centralization"<22><>"order_params"<22><>
|
|||
|
|
|
|||
|
|
- **params** - <20><><EFBFBD><EFBFBD><EEA1A3>ǰ<EFBFBD><C7B0><EFBFBD><EFBFBD><EFBFBD><EFBFBD>Ȩ<EFBFBD>أ<EFBFBD><D8A3><EFBFBD>ֵ<EFBFBD><D6B5><EFBFBD><EFBFBD><EFBFBD><EFBFBD> `Parameter` <20>б<EFBFBD><D0B1><EFBFBD>
|
|||
|
|
- **lr** - <20><>ѡ<EFBFBD><D1A1><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD>д<EFBFBD><D0B4><EFBFBD>"lr"<22><><EFBFBD><EFBFBD>ʹ<EFBFBD>ö<EFBFBD>Ӧ<EFBFBD><D3A6>ֵ<EFBFBD><D6B5>Ϊѧϰ<D1A7>ʡ<EFBFBD><CAA1><EFBFBD><EFBFBD><EFBFBD>û<EFBFBD>У<EFBFBD><D0A3><EFBFBD>ʹ<EFBFBD><CAB9><EFBFBD>Ż<EFBFBD><C5BB><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD>õ<EFBFBD> `learning_rate` <20><>Ϊѧϰ<D1A7>ʡ<EFBFBD>
|
|||
|
|
- **weight_decay** - <20><>ѡ<EFBFBD><D1A1><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD>д<EFBFBD><D0B4><EFBFBD>"weight_decay<61><79><EFBFBD><EFBFBD><EFBFBD><EFBFBD>ʹ<EFBFBD>ö<EFBFBD>Ӧ<EFBFBD><D3A6>ֵ<EFBFBD><D6B5>ΪȨ<CEAA><C8A8>˥<EFBFBD><CBA5>ֵ<EFBFBD><D6B5><EFBFBD><EFBFBD><EFBFBD><EFBFBD>û<EFBFBD>У<EFBFBD><D0A3><EFBFBD>ʹ<EFBFBD><CAB9><EFBFBD>Ż<EFBFBD><C5BB><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD>õ<EFBFBD> `weight_decay` <20><>ΪȨ<CEAA><C8A8>˥<EFBFBD><CBA5>ֵ<EFBFBD><D6B5>
|
|||
|
|
- **grad_centralization** - <20><>ѡ<EFBFBD><D1A1><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD>д<EFBFBD><D0B4><EFBFBD>"grad_centralization"<22><><EFBFBD><EFBFBD>ʹ<EFBFBD>ö<EFBFBD>Ӧ<EFBFBD><D3A6>ֵ<EFBFBD><D6B5><EFBFBD><EFBFBD>ֵ<EFBFBD><D6B5><EFBFBD><EFBFBD>Ϊ<EFBFBD><CEAA><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD>͡<EFBFBD><CDA1><EFBFBD><EFBFBD><EFBFBD>û<EFBFBD>У<EFBFBD><D0A3><EFBFBD><EFBFBD><EFBFBD>Ϊ `grad_centralization` ΪFalse<73><65><EFBFBD>ò<EFBFBD><C3B2><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD>ھ<EFBFBD><DABE><EFBFBD><EFBFBD>㡣
|
|||
|
|
- **order_params** - <20><>ѡ<EFBFBD><D1A1><EFBFBD><EFBFBD>Ӧֵ<D3A6><D6B5>Ԥ<EFBFBD>ڵIJ<DAB5><C4B2><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD>˳<EFBFBD><EFBFBD>ʹ<EFBFBD>ò<EFBFBD><C3B2><EFBFBD><EFBFBD><EFBFBD><EFBFBD>鹦<EFBFBD><E9B9A6>ʱ<EFBFBD><CAB1>ͨ<EFBFBD><CDA8>ʹ<EFBFBD>ø<EFBFBD><C3B8><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EEB1A3> `parameters` <20><>˳<EFBFBD><CBB3><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD>ܡ<EFBFBD><DCA1><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD>д<EFBFBD><D0B4><EFBFBD>"order_params"<22><><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD>Ը<EFBFBD><D4B8><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD>е<EFBFBD><D0B5><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD>"order_params"<22>еIJ<D0B5><C4B2><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD>ijһ<C4B3><D2BB> `params` <20><><EFBFBD><EFBFBD><EFBFBD>С<EFBFBD>
|
|||
|
|
|
|||
|
|
- **weight_decay** (Union[float, int]) - Ȩ<><C8A8>˥<EFBFBD><CBA5><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD>ֵ<EFBFBD><D6B5><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD>ڻ<EFBFBD><DABB><EFBFBD><EFBFBD><EFBFBD>0<EFBFBD><30><EFBFBD><EFBFBD><EFBFBD><EFBFBD> `weight_decay` <20><><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD>ת<EFBFBD><D7AA>Ϊ<EFBFBD><CEAA><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD>Ĭ<EFBFBD><C4AC>ֵ<EFBFBD><D6B5>0.0<EFBFBD><EFBFBD>
|
|||
|
|
- **loss_scale** (float) - <20>ݶ<EFBFBD><DDB6><EFBFBD><EFBFBD><EFBFBD>ϵ<EFBFBD><CFB5><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD>0<EFBFBD><30><EFBFBD><EFBFBD><EFBFBD><EFBFBD> `loss_scale` <20><><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD>ת<EFBFBD><D7AA>Ϊ<EFBFBD><CEAA><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD>ͨ<EFBFBD><CDA8>ʹ<EFBFBD><CAB9>Ĭ<EFBFBD><C4AC>ֵ<EFBFBD><D6B5><EFBFBD><EFBFBD><EFBFBD><EFBFBD>ѵ<EFBFBD><D1B5>ʱʹ<CAB1><CAB9><EFBFBD><EFBFBD> `FixedLossScaleManager`<EFBFBD><EFBFBD><EFBFBD><EFBFBD> `FixedLossScaleManager `<EFBFBD><EFBFBD> `drop_overflow_update` <20><><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD>ΪFalseʱ<65><CAB1><EFBFBD><EFBFBD>ֵ<EFBFBD><D6B5>Ҫ<EFBFBD><D2AA> `FixedLossScaleManager` <20>е<EFBFBD> `loss_scale` <20><>ͬ<EFBFBD><CDAC><EFBFBD>йظ<D0B9><D8B8><EFBFBD><EFBFBD><EFBFBD>ϸ<EFBFBD><CFB8>Ϣ<EFBFBD><CFA2><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD>class<73><73>`mindspore.FixedLossScaleManager`<EFBFBD><EFBFBD>Ĭ<EFBFBD><EFBFBD>ֵ<EFBFBD><EFBFBD>1.0<EFBFBD><EFBFBD>
|
|||
|
|
|
|||
|
|
**<2A>쳣<EFBFBD><ECB3A3>**
|
|||
|
|
|
|||
|
|
- **TypeError** - `learning_rate` <20><><EFBFBD><EFBFBD>int<6E><74>float<61><74>Tensor<6F><72>Iterable<6C><65>LearningRateSchedule<6C><65>
|
|||
|
|
- **TypeError** - `parameters` <20><>Ԫ<EFBFBD>ز<EFBFBD><D8B2><EFBFBD>Parameter<65><72><EFBFBD>ֵ䡣
|
|||
|
|
- **TypeError** - `loss_scale` <20><><EFBFBD><EFBFBD>float<61><74>
|
|||
|
|
- **TypeError** - `weight_decay` <20><><EFBFBD><EFBFBD>float<61><74>int<6E><74>
|
|||
|
|
- **ValueError** - `loss_scale` С<>ڻ<EFBFBD><DABB><EFBFBD><EFBFBD><EFBFBD>0<EFBFBD><30>
|
|||
|
|
- **ValueError** - `weight_decay` С<><D0A1>0<EFBFBD><30>
|
|||
|
|
- **ValueError** - `learning_rate` <20><>һ<EFBFBD><D2BB>Tensor<6F><72><EFBFBD><EFBFBD><EFBFBD><EFBFBD>Tensor<6F><72>ά<EFBFBD>ȴ<EFBFBD><C8B4><EFBFBD>1<EFBFBD><31>
|
|||
|
|
|
|||
|
|
**֧<><D6A7>ƽ̨<C6BD><CCA8>**
|
|||
|
|
|
|||
|
|
``Ascend`` ``GPU`` ``CPU``
|
|||
|
|
|
|||
|
|
.. py:method:: broadcast_params(optim_result)
|
|||
|
|
|
|||
|
|
<20><><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD>˳<EFBFBD><CBB3><EFBFBD><EFBFBD><EFBFBD>в<EFBFBD><D0B2><EFBFBD><EFBFBD>㲥<EFBFBD><E3B2A5>
|
|||
|
|
|
|||
|
|
**<2A><><EFBFBD><EFBFBD><EFBFBD><EFBFBD>**
|
|||
|
|
|
|||
|
|
**optim_result** (bool) - <20><><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD>½<EFBFBD><C2BD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD>֤<EFBFBD><D6A4><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD>ɺ<EFBFBD><C9BA><EFBFBD>ִ<EFBFBD>в<EFBFBD><D0B2><EFBFBD><EFBFBD>㲥<EFBFBD><E3B2A5>
|
|||
|
|
|
|||
|
|
**<2A><><EFBFBD>أ<EFBFBD>**
|
|||
|
|
|
|||
|
|
bool<6F><6C>״̬<D7B4><CCAC>־<EFBFBD><D6BE>
|
|||
|
|
|
|||
|
|
.. py:method:: decay_weight(gradients)
|
|||
|
|
|
|||
|
|
˥<><CBA5>Ȩ<EFBFBD>ء<EFBFBD>
|
|||
|
|
|
|||
|
|
һ<>ּ<EFBFBD><D6BC><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD>ѧϰ<D1A7><CFB0><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD>ģ<EFBFBD><EFBFBD><CDB9><EFBFBD><EFBFBD>ϵķ<CFB5><C4B7><EFBFBD><EFBFBD><EFBFBD><EFBFBD>̳<EFBFBD> :class:`mindspore.nn.Optimizer` <20>Զ<EFBFBD><D4B6><EFBFBD><EFBFBD>Ż<EFBFBD><C5BB><EFBFBD>ʱ<EFBFBD><CAB1><EFBFBD>ɵ<EFBFBD><C9B5>øýӿڽ<D3BF><DABD><EFBFBD>Ȩ<EFBFBD><C8A8>˥<EFBFBD><CBA5><EFBFBD><EFBFBD>
|
|||
|
|
|
|||
|
|
**<2A><><EFBFBD><EFBFBD><EFBFBD><EFBFBD>**
|
|||
|
|
|
|||
|
|
**gradients** (tuple[Tensor]) - <20><><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD>ݶȣ<DDB6><C8A3><EFBFBD>״<EFBFBD><D7B4>shape<70><65><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD>ͬ<EFBFBD><CDAC>
|
|||
|
|
|
|||
|
|
**<2A><><EFBFBD>أ<EFBFBD>**
|
|||
|
|
|
|||
|
|
tuple[Tensor]<5D><>˥<EFBFBD><CBA5>Ȩ<EFBFBD>غ<EFBFBD><D8BA><EFBFBD><EFBFBD>ݶȡ<DDB6>
|
|||
|
|
|
|||
|
|
.. py:method:: get_lr()
|
|||
|
|
|
|||
|
|
<20>Ż<EFBFBD><C5BB><EFBFBD><EFBFBD><EFBFBD><EFBFBD>øýӿڻ<D3BF>ȡ<EFBFBD><C8A1>ǰ<EFBFBD><C7B0><EFBFBD>裨step<65><70><EFBFBD><EFBFBD>ѧϰ<D1A7>ʡ<EFBFBD><CAA1>̳<EFBFBD> :class:`mindspore.nn.Optimizer` <20>Զ<EFBFBD><D4B6><EFBFBD><EFBFBD>Ż<EFBFBD><C5BB><EFBFBD>ʱ<EFBFBD><CAB1><EFBFBD><EFBFBD><EFBFBD>ڲ<EFBFBD><DAB2><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD>ǰ<EFBFBD><C7B0><EFBFBD>øýӿڻ<D3BF>ȡѧϰ<D1A7>ʡ<EFBFBD>
|
|||
|
|
|
|||
|
|
**<2A><><EFBFBD>أ<EFBFBD>**
|
|||
|
|
|
|||
|
|
float<61><74><EFBFBD><EFBFBD>ǰ<EFBFBD><C7B0><EFBFBD><EFBFBD><EFBFBD><EFBFBD>ѧϰ<D1A7>ʡ<EFBFBD>
|
|||
|
|
|
|||
|
|
.. py:method:: get_lr_parameter(param)
|
|||
|
|
|
|||
|
|
<20><><EFBFBD><EFBFBD><EFBFBD><EFBFBD>ʹ<EFBFBD><CAB9><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD>鹦<EFBFBD>ܣ<EFBFBD><DCA3><EFBFBD>Ϊ<EFBFBD><CEAA>ͬ<EFBFBD><CDAC><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD>ò<EFBFBD>ͬ<EFBFBD><CDAC>ѧϰ<D1A7><CFB0>ʱ<EFBFBD><CAB1><EFBFBD><EFBFBD>ȡָ<C8A1><D6B8><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD>ѧϰ<D1A7>ʡ<EFBFBD>
|
|||
|
|
|
|||
|
|
**<2A><><EFBFBD><EFBFBD><EFBFBD><EFBFBD>**
|
|||
|
|
|
|||
|
|
**param** (Union[Parameter, list[Parameter]]) - `Parameter` <20><> `Parameter` <20>б<EFBFBD><D0B1><EFBFBD>
|
|||
|
|
|
|||
|
|
**<2A><><EFBFBD>أ<EFBFBD>**
|
|||
|
|
|
|||
|
|
Parameter<65><72><EFBFBD><EFBFBD><EFBFBD><EFBFBD> `Parameter` <20><> `Parameter` <20>б<EFBFBD><D0B1><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD>ʹ<EFBFBD><CAB9><EFBFBD>˶<EFBFBD>̬ѧϰ<D1A7>ʣ<EFBFBD><CAA3><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD>ڼ<EFBFBD><DABC><EFBFBD>ѧϰ<D1A7>ʵ<EFBFBD> `LearningRateSchedule` <20><> `LearningRateSchedule` <20>б<EFBFBD><D0B1><EFBFBD>
|
|||
|
|
|
|||
|
|
**<2A><><EFBFBD><EFBFBD><EFBFBD><EFBFBD>**
|
|||
|
|
|
|||
|
|
>>> from mindspore import nn
|
|||
|
|
>>> net = Net()
|
|||
|
|
>>> conv_params = list(filter(lambda x: 'conv' in x.name, net.trainable_params()))
|
|||
|
|
>>> no_conv_params = list(filter(lambda x: 'conv' not in x.name, net.trainable_params()))
|
|||
|
|
>>> group_params = [{'params': conv_params, 'lr': 0.05},
|
|||
|
|
... {'params': no_conv_params, 'lr': 0.01}]
|
|||
|
|
>>> optim = nn.Momentum(group_params, learning_rate=0.1, momentum=0.9, weight_decay=0.0)
|
|||
|
|
>>> conv_lr = optim.get_lr_parameter(conv_params)
|
|||
|
|
>>> print(conv_lr[0].asnumpy())
|
|||
|
|
0.05
|
|||
|
|
|
|||
|
|
.. py:method:: gradients_centralization(gradients)
|
|||
|
|
|
|||
|
|
<20>ݶ<EFBFBD><DDB6><EFBFBD><EFBFBD>Ļ<EFBFBD><C4BB><EFBFBD>
|
|||
|
|
|
|||
|
|
һ<><D2BB><EFBFBD>Ż<EFBFBD><C5BB><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD>ѧϰ<D1A7><CFB0><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD>ģ<EFBFBD><C4A3>ѵ<EFBFBD><D1B5><EFBFBD>ٶȵķ<C8B5><C4B7><EFBFBD><EFBFBD><EFBFBD><EFBFBD>̳<EFBFBD> :class:`mindspore.nn.Optimizer` <20>Զ<EFBFBD><D4B6><EFBFBD><EFBFBD>Ż<EFBFBD><C5BB><EFBFBD>ʱ<EFBFBD><CAB1><EFBFBD>ɵ<EFBFBD><C9B5>øýӿڽ<D3BF><DABD><EFBFBD><EFBFBD>ݶ<EFBFBD><DDB6><EFBFBD><EFBFBD>Ļ<EFBFBD><C4BB><EFBFBD>
|
|||
|
|
|
|||
|
|
**<2A><><EFBFBD><EFBFBD><EFBFBD><EFBFBD>**
|
|||
|
|
|
|||
|
|
**gradients** (tuple[Tensor]) - <20><><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD>ݶȣ<DDB6><C8A3><EFBFBD>״<EFBFBD><D7B4>shape<70><65><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD>ͬ<EFBFBD><CDAC>
|
|||
|
|
|
|||
|
|
**<2A><><EFBFBD>أ<EFBFBD>**
|
|||
|
|
|
|||
|
|
tuple[Tensor]<5D><><EFBFBD>ݶ<EFBFBD><DDB6><EFBFBD><EFBFBD>Ļ<EFBFBD><C4BB><EFBFBD><EFBFBD><EFBFBD><EFBFBD>ݶȡ<DDB6>
|
|||
|
|
|
|||
|
|
.. py:method:: scale_grad(gradients)
|
|||
|
|
|
|||
|
|
<20><><EFBFBD><EFBFBD><EFBFBD>ڻ<EFBFBD><DABB>Ͼ<EFBFBD><CFBE>ȳ<EFBFBD><C8B3><EFBFBD><EFBFBD><EFBFBD>ԭ<EFBFBD>ݶȡ<DDB6>
|
|||
|
|
|
|||
|
|
<20>̳<EFBFBD> :class:`mindspore.nn.Optimizer` <20>Զ<EFBFBD><D4B6><EFBFBD><EFBFBD>Ż<EFBFBD><C5BB><EFBFBD>ʱ<EFBFBD><CAB1><EFBFBD>ɵ<EFBFBD><C9B5>øýӿڻ<D3BF>ԭ<EFBFBD>ݶȡ<DDB6>
|
|||
|
|
|
|||
|
|
**<2A><><EFBFBD><EFBFBD><EFBFBD><EFBFBD>**
|
|||
|
|
|
|||
|
|
**gradients** (tuple[Tensor]) - <20><><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD>ݶȣ<DDB6><C8A3><EFBFBD>״<EFBFBD><D7B4>shape<70><65><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD>ͬ<EFBFBD><CDAC>
|
|||
|
|
|
|||
|
|
**<2A><><EFBFBD>أ<EFBFBD>**
|
|||
|
|
|
|||
|
|
tuple[Tensor]<5D><><EFBFBD><EFBFBD>ԭ<EFBFBD><D4AD><EFBFBD><EFBFBD><EFBFBD>ݶȡ<DDB6>
|
|||
|
|
|
|||
|
|
.. py:method:: target
|
|||
|
|
:property:
|
|||
|
|
|
|||
|
|
<20><><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD>ָ<EFBFBD><D6B8><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD>host<73><74><EFBFBD>ϻ<EFBFBD><CFBB><EFBFBD><EFBFBD>豸<EFBFBD><E8B1B8>device<63><65><EFBFBD>ϸ<EFBFBD><CFB8>²<EFBFBD><C2B2><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD>Ϊstr<74><72>ֻ<EFBFBD><D6BB><EFBFBD><EFBFBD>'CPU'<27><>'Ascend'<27><>'GPU'<27><>
|
|||
|
|
|
|||
|
|
.. py:method:: unique
|
|||
|
|
:property:
|
|||
|
|
|
|||
|
<20><><EFBFBD><EFBFBD><EFBFBD>Ա<EFBFBD>ʾ<EFBFBD>Ƿ<EFBFBD><C7B7><EFBFBD><EFBFBD>Ż<EFBFBD><C5BB><EFBFBD><EFBFBD>н<EFBFBD><D0BD><EFBFBD><EFBFBD>ݶ<EFBFBD>ȥ<EFBFBD>أ<EFBFBD>ͨ<EFBFBD><CDA8><EFBFBD><EFBFBD><EFBFBD><EFBFBD>ϡ<EFBFBD><CFA1><EFBFBD><EFBFBD><EFBFBD>硣<EFBFBD><E7A1A3><EFBFBD><EFBFBD><EFBFBD>ݶ<EFBFBD><DDB6><EFBFBD>ϡ<EFBFBD><CFA1><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD>ΪTrue<75><65><EFBFBD><EFBFBD><EFBFBD><EFBFBD>ǰ<EFBFBD><C7B0>ϡ<EFBFBD><CFA1><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD>Ѷ<EFBFBD>Ȩ<EFBFBD><C8A8>ȥ<EFBFBD>أ<EFBFBD><D8A3><EFBFBD><EFBFBD>ݶ<EFBFBD><DDB6>dz<EFBFBD><C7B3>ܵģ<DCB5><C4A3><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD>ΪFalse<73><65>δ<EFBFBD><CEB4><EFBFBD><EFBFBD>ʱĬ<CAB1><C4AC>ֵΪTrue<75><65>
|