forked from mindspore-Ecosystem/mindspore
Add document for clip by global norm when auto parallel mode
This commit is contained in:
parent
8b7d7a1e1a
commit
49a2f1cad7
|
@ -6,7 +6,10 @@
|
|||
通过权重梯度总和的比率来裁剪多个Tensor的值。
|
||||
|
||||
.. note::
|
||||
输入'x'应为Tensor的tuple或list。否则,将引发错误。
|
||||
输入 `x` 应为Tensor的tuple或list。否则,将引发错误。
|
||||
|
||||
.. note::
|
||||
在半自动并行模式或自动并行模式下,如果输入是梯度,那么将会自动汇聚所有设备上的梯度的平方和。
|
||||
|
||||
**参数:**
|
||||
|
||||
|
|
|
@ -197,7 +197,12 @@ def clip_by_global_norm(x, clip_norm=1.0, use_norm=None):
|
|||
Clips tensor values by the ratio of the sum of their norms.
|
||||
|
||||
Note:
|
||||
Input 'x' should be a tuple or list of tensors. Otherwise, it will raise an error.
|
||||
Input `x` should be a tuple or list of tensors. Otherwise, it will raise an error.
|
||||
|
||||
Note:
|
||||
On the SEMI_AUTO_PARALLEL mode or AUTO_PARALLEL mode, if the input `x` is the gradient,
|
||||
the gradient norm values on all devices will be automatically aggregated by allreduce inserted after the local
|
||||
square sum of the gradients.
|
||||
|
||||
Args:
|
||||
x (Union(tuple[Tensor], list[Tensor])): Input data to clip.
|
||||
|
|
Loading…
Reference in New Issue