forked from mindspore-Ecosystem/mindspore
code_docs_cnapi
This commit is contained in:
wanyiming
parent
6b6bdf5d1d
commit
822b11a9be
|
|
@ -0,0 +1,159 @@
|
|||
mindspore.common.initializer
|
||||
=============================
|
||||
|
||||
初始化神经元参数。
|
||||
|
||||
.. py:class:: mindspore.common.initializer.Initializer(**kwargs)
|
||||
|
||||
初始化器的抽象基类。
|
||||
|
||||
**参数：**
|
||||
|
||||
- **kwargs** (`dict`) – **Initializer** 的关键字参数。
|
||||
|
||||
|
||||
.. py:method:: mindspore.common.initializer.initializer(init, shape=None, dtype=mstype.float32)
|
||||
|
||||
创建并初始化一个Tensor。
|
||||
|
||||
**参数：**
|
||||
|
||||
- **init** (`Union[Tensor, str, Initializer, numbers.Number]`) – 初始化方式。
|
||||
|
||||
- *str*：`init` 是继承自 Initializer 的类的别名，实际使用时会调用相应的类。`init` 的值可以是"normal"、"ones"或"zeros"等。
|
||||
- *Initializer*：`init` 是继承自Initializer，用于初始化Tensor的类。
|
||||
- *numbers.Number*：调用常量来初始化张量。
|
||||
|
||||
- **shape** (`Union[[tuple, list, int]`) - 被初始化的Tensor的shape，默认值为None。
|
||||
|
||||
- **dtype** (`mindspore.dtype`) – 被初始化的Tensor的数据类型，默认值为 `mindspore.float32` 。
|
||||
|
||||
**返回：**
|
||||
|
||||
Tensor，返回一个张量对象。
|
||||
|
||||
**异常：**
|
||||
TypeError: 参数`init`的类型不正确。
|
||||
ValueError: 通过`init`传入的Tensor的shape和作为参数传入的shape不一致。
|
||||
|
||||
**样例：**
|
||||
|
||||
.. code-block::
|
||||
|
||||
>>> import mindspore
|
||||
>>> from mindspore.common.initializer import initializer, One
|
||||
>>> tensor = initializer('ones', [1, 2, 3], mindspore.float32)
|
||||
>>> tensor = initializer(One(), [1, 2, 3], mindspore.float32)
|
||||
>>> tensor = initializer(0, [1, 2, 3], mindspore.float32)
|
||||
|
||||
.. py:class:: mindspore.common.initializer.TruncatedNormal(sigma=0.01)
|
||||
|
||||
生成一个数组用于初始化Tensor，数组中的数值从截断正态分布中采样得到。
|
||||
|
||||
**参数：**
|
||||
|
||||
- **sigma** (`float`) - 截断正态分布的标准差，默认值为0.01。
|
||||
|
||||
|
||||
.. py:class:: mindspore.common.initializer.Normal(sigma=0.01, mean=0.0)
|
||||
|
||||
生成一个数组用于初始化Tensor，数组中的数值从正态分布N(sigma, mean)中采样得到。
|
||||
|
||||
.. math::
|
||||
f(x) = \frac{1} {\sqrt{2*π} * sigma}exp(-\frac{(x - mean)^2} {2*{sigma}^2})
|
||||
|
||||
**参数：**
|
||||
|
||||
- **sigma** (`float`) - 正态分布的标准差，默认值为0.01。
|
||||
|
||||
- **mean** (`float`) - 正态分布的均值，默认值为0.0。
|
||||
|
||||
|
||||
|
||||
.. py:class:: mindspore.common.initializer.Uniform(scale=0.07)
|
||||
|
||||
生成一个数组用于初始化Tensor，数组中的数值从均匀分布U(-scale, scale)中采样得到。
|
||||
|
||||
**参数：**
|
||||
|
||||
- **scale** (`float`) - 均匀分布的边界，默认值为0.07。
|
||||
|
||||
|
||||
.. py:class:: mindspore.common.initializer.HeUniform(negative_slope=0, mode="fan_in", nonlinearity="leaky_relu")
|
||||
|
||||
生成一个数组用于初始化Tensor，数组中的数值从HeKaiming均匀分布U[-boundary,boundary]中采样得到，其中
|
||||
|
||||
.. math::
|
||||
boundary = \sqrt{\frac{6}{(1 + a^2) \times \text{fan_in}}}
|
||||
|
||||
是HeUniform分布的边界。
|
||||
|
||||
**参数：**
|
||||
|
||||
- **negative_slope** (`int, float, bool`) - 本层激活函数的负数区间斜率（仅适用于非线性激活函数"leaky_relu"），默认值为0。
|
||||
|
||||
- **mode** (`str`) - 可选"fan_in"或"fan_out"，"fan_in"会保留前向传递中权重方差的量级，"fan_out"会保留反向传递的量级，默认为"fan_in"。
|
||||
|
||||
- **nonlinearity** (`str`) - 非线性激活函数，推荐使用"relu"或"leaky_relu"，默认为"leaky_relu"。
|
||||
|
||||
|
||||
|
||||
.. py:class:: mindspore.common.initializer.HeNormal(negative_slope=0, mode="fan_in", nonlinearity="leaky_relu")
|
||||
|
||||
生成一个数组用于初始化Tensor，数组中的数值从HeKaiming正态分布N(0, sigma^2)中采样得到，其中
|
||||
|
||||
.. math::
|
||||
sigma = \frac{gain} {\sqrt{N}}
|
||||
|
||||
其中，gain是一个可选的缩放因子。如果mode是"fan_in"， N是权重Tensor中输入单元的数量，如果mode是"fan_out"， N是权重Tensor中输出单元的数量。
|
||||
|
||||
HeUniform 算法的详细信息，请查看 https://arxiv.org/abs/1502.01852。
|
||||
|
||||
**参数：**
|
||||
|
||||
- **negative_slope** (`int, float, bool`) - 本层激活函数的负数区间斜率（仅适用于非线性激活函数"leaky_relu"），默认值为0。
|
||||
|
||||
- **mode** (`str`) - 可选"fan_in"或"fan_out"，"fan_in"会保留前向传递中权重方差的量级，"fan_out"会保留反向传递的量级，默认为"fan_in"。
|
||||
|
||||
- **nonlinearity** (`str`) - 非线性激活函数，推荐使用"relu"或"leaky_relu"，默认为"leaky_relu"。
|
||||
|
||||
|
||||
.. py:class:: mindspore.common.initializer.XavierUniform(gain=1)
|
||||
|
||||
生成一个数组用于初始化Tensor，数组中的数值从Xarvier均匀分布U[-boundary,boundary]中采样得到，其中
|
||||
|
||||
.. math::
|
||||
|
||||
boundary = gain * \sqrt{\frac{6}{n_{in} + n_{out}}}
|
||||
|
||||
- `gain` 是一个可选的缩放因子。
|
||||
- `n_{in}` 为权重Tensor中输入单元的数量。
|
||||
- `n_{out}` 为权重Tensor中输出单元的数量。
|
||||
|
||||
有关 XavierUniform 算法的详细信息，请查看 http://proceedings.mlr.press/v9/glorot10a.html。
|
||||
|
||||
**参数：**
|
||||
|
||||
- **gain** (`float`) - 可选的缩放因子，默认值为1。
|
||||
|
||||
|
||||
.. py:class:: mindspore.common.initializer.One(**kwargs)
|
||||
|
||||
生成一个值全为1的常量数组用于初始化Tensor。
|
||||
|
||||
|
||||
|
||||
.. py:class:: mindspore.common.initializer.Zero(**kwargs)
|
||||
|
||||
生成一个值全为0的常量数组用于初始化Tensor。
|
||||
|
||||
|
||||
|
||||
.. py:class:: mindspore.common.initializer.Constant(value)
|
||||
|
||||
生成一个常量数组用于初始化Tensor。
|
||||
|
||||
**参数：**
|
||||
|
||||
- **value** (`Union[int, numpy.ndarray]`) - 用于初始化的常数值或者数组。
|
||||
|
||||
|
|
@ -0,0 +1,52 @@
|
|||
mindspore.nn.CosineDecayLR
|
||||
===========================
|
||||
|
||||
.. py:class:: mindspore.nn.CosineDecayLR(min_lr, max_lr, decay_steps)
|
||||
|
||||
基于余弦衰减函数计算学习率。
|
||||
|
||||
对于当前step，decayed_learning_rate[current_step]的计算公式为：
|
||||
|
||||
.. math::
|
||||
decayed\_learning\_rate[current\_step] = min\_lr + 0.5 * (max\_lr - min\_lr) *
|
||||
(1 + cos(\frac{current\_step}{decay\_steps}\pi))
|
||||
|
||||
|
||||
**参数：**
|
||||
|
||||
- **min_lr** (float): 学习率的最小值。
|
||||
- **max_lr** (float): 学习率的最大值。
|
||||
- **decay_steps** (int): 用于计算衰减学习率的值。
|
||||
|
||||
**输入：**
|
||||
|
||||
- **global_step** (Tensor) - 当前step数。
|
||||
|
||||
**输出：**
|
||||
|
||||
Tensor。形状为 :math:`()` 的当前step的学习率值。
|
||||
|
||||
**异常：**
|
||||
|
||||
- **TypeError:** `min_lr` 或 `max_lr` 不是float。
|
||||
- **TypeError:** `decay_steps` 不是整数。
|
||||
- **ValueError:** `min_lr` 小于0或 `decay_steps` 小于1。
|
||||
- **ValueError:** `max_lr` 小于或等于0。
|
||||
|
||||
**支持平台：**
|
||||
|
||||
``Ascend`` ``GPU``
|
||||
|
||||
**样例：**
|
||||
|
||||
.. code-block::
|
||||
|
||||
>>> min_lr = 0.01
|
||||
>>> max_lr = 0.1
|
||||
>>> decay_steps = 4
|
||||
>>> global_steps = Tensor(2, mstype.int32)
|
||||
>>> cosine_decay_lr = nn.CosineDecayLR(min_lr, max_lr, decay_steps)
|
||||
>>> result = cosine_decay_lr(global_steps)
|
||||
>>> print(result)
|
||||
0.055
|
||||
|
||||
|
|
@ -0,0 +1,61 @@
|
|||
mindspore.nn.ExponentialDecayLR
|
||||
================================
|
||||
|
||||
.. py:class:: mindspore.nn.ExponentialDecayLR(learning_rate, decay_rate, decay_steps, is_stair=False)
|
||||
|
||||
基于指数衰减函数计算学习率。
|
||||
|
||||
对于当前step，decayed_learning_rate[current_step]的计算公式为：
|
||||
|
||||
.. math::
|
||||
decayed\_learning\_rate[current\_step] = learning\_rate * decay\_rate^{p}
|
||||
|
||||
其中，
|
||||
|
||||
.. math::
|
||||
p = \frac{current\_step}{decay\_steps}
|
||||
|
||||
如果 `is_stair` 为True，则公式为：
|
||||
|
||||
.. math::
|
||||
p = floor(\frac{current\_step}{decay\_steps})
|
||||
|
||||
**参数：**
|
||||
|
||||
- **learning_rate** (float): 学习率的初始值。
|
||||
- **decay_rate** (float): 衰减率。
|
||||
- **decay_steps** (int): 用于计算衰减学习率的值。
|
||||
- **is_stair** (bool): 如果为True，则学习率每 `decay_steps` 步衰减一次。默认值：False。
|
||||
|
||||
**输入：**
|
||||
|
||||
- **global_step** (Tensor) - 当前step数。
|
||||
|
||||
**输出：**
|
||||
|
||||
Tensor。形状为 :math:`()` 的当前step的学习率值。
|
||||
|
||||
**异常：**
|
||||
|
||||
- **TypeError:** `learning_rate` 或 `decay_rate` 不是float。
|
||||
- **TypeError:** `decay_steps` 不是int或 `is_stair` 不是bool。
|
||||
- **ValueError:** `decay_steps` 小于1。
|
||||
- **ValueError:** `learning_rate` 或 `decay_rate` 小于或等于0。
|
||||
|
||||
**支持平台：**
|
||||
|
||||
``Ascend`` ``GPU`` ``CPU``
|
||||
|
||||
**样例：**
|
||||
|
||||
.. code-block::
|
||||
|
||||
>>> learning_rate = 0.1
|
||||
>>> decay_rate = 0.9
|
||||
>>> decay_steps = 4
|
||||
>>> global_step = Tensor(2, mstype.int32)
|
||||
>>> exponential_decay_lr = nn.ExponentialDecayLR(learning_rate, decay_rate, decay_steps)
|
||||
>>> result = exponential_decay_lr(global_step)
|
||||
>>> print(result)
|
||||
0.09486833
|
||||
|
||||
|
|
@ -0,0 +1,53 @@
|
|||
mindspore.nn.WarmUpLR
|
||||
======================
|
||||
|
||||
.. py:class:: mindspore.nn.WarmUpLR(learning_rate, warmup_steps)
|
||||
|
||||
学习率热身。
|
||||
|
||||
对于当前step，计算warmup_learning_rate[current_step]的公式为：
|
||||
|
||||
.. math::
|
||||
warmup\_learning\_rate[current\_step] = learning\_rate * tmp\_step / warmup\_steps
|
||||
|
||||
其中，
|
||||
|
||||
.. math:
|
||||
tmp\_step=min(current\_step, warmup\_steps)
|
||||
|
||||
**参数：**
|
||||
|
||||
- **learning_rate** (float): 学习率的初始值。
|
||||
- **warmup_steps** (int): 学习率warmup的step数。
|
||||
|
||||
**输入：**
|
||||
|
||||
- **global_step** (Tensor)：当前step数。
|
||||
|
||||
**输出：**
|
||||
|
||||
Tensor。形状为 :math:`()` 的当前step的学习率值。
|
||||
|
||||
**异常：**
|
||||
|
||||
- **TypeError：** `learning_rate` 不是float。
|
||||
- **TypeError：** `warmup_steps` 不是int。
|
||||
- **ValueError：** `warmup_steps` 小于1。
|
||||
- **ValueError：** `learning_rate` 小于或等于0。
|
||||
|
||||
**支持平台：**
|
||||
|
||||
``Ascend`` ``GPU``
|
||||
|
||||
**样例：**
|
||||
|
||||
.. code-block::
|
||||
|
||||
>>> learning_rate = 0.1
|
||||
>>> warmup_steps = 2
|
||||
>>> global_step = Tensor(2, mstype.int32)
|
||||
>>> warmup_lr = nn.WarmUpLR(learning_rate, warmup_steps)
|
||||
>>> result = warmup_lr(global_step)
|
||||
>>> print(result)
|
||||
0.1
|
||||
|
||||
|
|
@ -29,8 +29,7 @@ _INITIALIZER_ALIAS = dict()
|
|||
|
||||
class Initializer:
|
||||
"""
|
||||
The base class of the initializer.
|
||||
Initialization of tensor basic attributes and model weight values.
|
||||
The abstract base class of the initializer.
|
||||
|
||||
Args:
|
||||
kwargs (dict): Keyword arguments for Initializer.
|
||||
|
|
@ -91,7 +90,7 @@ def _assignment(arr, num):
|
|||
@_register('zeros')
|
||||
class Zero(Initializer):
|
||||
"""
|
||||
Generates an array with constant value of zero in order to initialize the input tensor.
|
||||
Generates an array with constant value of zero in order to initialize a tensor.
|
||||
|
||||
Examples:
|
||||
>>> import mindspore
|
||||
|
|
@ -106,7 +105,7 @@ class Zero(Initializer):
|
|||
@_register('ones')
|
||||
class One(Initializer):
|
||||
"""
|
||||
Generates an array with constant value of one in order to initialize the input tensor.
|
||||
Generates an array with constant value of one in order to initialize a tensor.
|
||||
|
||||
Examples:
|
||||
>>> import mindspore
|
||||
|
|
@ -229,7 +228,7 @@ def _calculate_in_and_out(arr):
|
|||
class XavierUniform(Initializer):
|
||||
r"""
|
||||
Generates an array with values sampled from Xavier uniform distribution
|
||||
:math:`{U}(-\text{boundary}, \text{boundary})` in order to initialize the input tensor, where:
|
||||
:math:`{U}(-\text{boundary}, \text{boundary})` in order to initialize a tensor, where:
|
||||
|
||||
.. math::
|
||||
boundary = gain * \sqrt{\frac{6}{n_{in} + n_{out}}}
|
||||
|
|
@ -268,7 +267,7 @@ class XavierUniform(Initializer):
|
|||
class HeUniform(Initializer):
|
||||
r"""
|
||||
Generates an array with values sampled from HeKaiming Uniform distribution
|
||||
:math:`{U}(-\text{boundary}, \text{boundary})` in order to initialize the input tensor, where
|
||||
:math:`{U}(-\text{boundary}, \text{boundary})` in order to initialize a tensor, where
|
||||
|
||||
.. math::
|
||||
boundary = \sqrt{\frac{6}{(1 + a^2) \times \text{fan_in}}}
|
||||
|
|
@ -314,7 +313,7 @@ class HeUniform(Initializer):
|
|||
class HeNormal(Initializer):
|
||||
r"""
|
||||
Generates an array with values sampled from HeKaiming Normal distribution
|
||||
:math:`{N}(0, \text{sigma}^2)` in order to initialize the input tensor, where
|
||||
:math:`{N}(0, \text{sigma}^2)` in order to initialize a tensor, where
|
||||
|
||||
.. math::
|
||||
sigma = \frac{gain} {\sqrt{N}}
|
||||
|
|
@ -357,7 +356,7 @@ class HeNormal(Initializer):
|
|||
|
||||
class Constant(Initializer):
|
||||
"""
|
||||
Generates an array with constant value in order to initialize the input tensor.
|
||||
Generates an array with constant value in order to initialize a tensor.
|
||||
|
||||
Args:
|
||||
value (Union[int, numpy.ndarray]): The value to initialize.
|
||||
|
|
@ -381,7 +380,7 @@ class Constant(Initializer):
|
|||
class Uniform(Initializer):
|
||||
r"""
|
||||
Generates an array with values sampled from Uniform distribution :math:`{U}(-\text{scale}, \text{scale})` in order
|
||||
to initialize the input tensor.
|
||||
to initialize a tensor.
|
||||
|
||||
Args:
|
||||
scale (float): The bound of the Uniform distribution. Default: 0.07.
|
||||
|
|
@ -406,7 +405,7 @@ class Uniform(Initializer):
|
|||
class Normal(Initializer):
|
||||
r"""
|
||||
Generates an array with values sampled from Normal distribution :math:`{N}(\text{sigma}, \text{mean})` in order to
|
||||
initialize the input tensor.
|
||||
initialize a tensor.
|
||||
|
||||
.. math::
|
||||
f(x) = \frac{1} {\sqrt{2*π} * sigma}exp(-\frac{(x - mean)^2} {2*{sigma}^2})
|
||||
|
|
@ -438,7 +437,7 @@ class Normal(Initializer):
|
|||
@_register()
|
||||
class TruncatedNormal(Initializer):
|
||||
r"""
|
||||
Generates an array with values sampled from Truncated Normal distribution in order to initialize the input tensor.
|
||||
Generates an array with values sampled from Truncated Normal distribution in order to initialize a tensor.
|
||||
|
||||
Args:
|
||||
sigma (float): The standard deviation of Truncated Normal distribution. Default: 0.01.
|
||||
|
|
@ -477,7 +476,7 @@ def initializer(init, shape=None, dtype=mstype.float32):
|
|||
dtype (:class:`mindspore.dtype`): The type of data in initialized tensor. Default: mindspore.float32.
|
||||
|
||||
Returns:
|
||||
Union[Tensor], return is Tensor object.
|
||||
Tensor, return is Tensor object.
|
||||
|
||||
Raises:
|
||||
TypeError: The type of the argument 'init' is not correct.
|
||||
|
|
|
|||
|
|
@ -58,10 +58,10 @@ class ExponentialDecayLR(LearningRateSchedule):
|
|||
r"""
|
||||
Calculates learning rate based on exponential decay function.
|
||||
|
||||
For the i-th step, the formula of computing decayed_learning_rate[i] is:
|
||||
For current step, the formula of computing decayed_learning_rate[current_step] is:
|
||||
|
||||
.. math::
|
||||
decayed\_learning\_rate[i] = learning\_rate * decay\_rate^{p}
|
||||
decayed\_learning\_rate[current\_step] = learning\_rate * decay\_rate^{p}
|
||||
|
||||
Where :
|
||||
|
||||
|
|
@ -259,10 +259,10 @@ class CosineDecayLR(LearningRateSchedule):
|
|||
r"""
|
||||
Calculates learning rate based on cosine decay function.
|
||||
|
||||
For the i-th step, the formula of computing decayed_learning_rate[i] is:
|
||||
For current step, the formula of computing decayed_learning_rate[current_step] is:
|
||||
|
||||
.. math::
|
||||
decayed\_learning\_rate[i] = min\_learning\_rate + 0.5 * (max\_learning\_rate - min\_learning\_rate) *
|
||||
decayed\_learning\_rate[current\_step] = min\_lr + 0.5 * (max\_lr - min\_lr) *
|
||||
(1 + cos(\frac{current\_step}{decay\_steps}\pi))
|
||||
|
||||
|
||||
|
|
@ -412,10 +412,10 @@ class WarmUpLR(LearningRateSchedule):
|
|||
r"""
|
||||
Gets learning rate warming up.
|
||||
|
||||
For the i-th step, the formula of computing warmup_learning_rate[i] is:
|
||||
For current step, the formula of computing warmup_learning_rate[i] is:
|
||||
|
||||
.. math::
|
||||
warmup\_learning\_rate[i] = learning\_rate * tmp\_step / warmup\_steps
|
||||
warmup\_learning\_rate[current\_step] = learning\_rate * tmp\_step / warmup\_steps
|
||||
|
||||
Where :
|
||||
|
||||
|
|
|
|||
Loading…
Reference in New Issue