Add document for clip by global norm when auto parallel mode

This commit is contained in:
huangxinjing 2022-03-11 10:06:36 +08:00
parent 8b7d7a1e1a
commit 49a2f1cad7
2 changed files with 10 additions and 2 deletions

View File

@ -6,7 +6,10 @@
通过权重梯度总和的比率来裁剪多个Tensor的值。
.. note::
输入'x'应为Tensor的tuple或list。否则将引发错误。
输入 `x` 应为Tensor的tuple或list。否则将引发错误。
.. note::
在半自动并行模式或自动并行模式下,如果输入是梯度,那么将会自动汇聚所有设备上的梯度的平方和。
**参数:**

View File

@ -197,7 +197,12 @@ def clip_by_global_norm(x, clip_norm=1.0, use_norm=None):
Clips tensor values by the ratio of the sum of their norms.
Note:
Input 'x' should be a tuple or list of tensors. Otherwise, it will raise an error.
Input `x` should be a tuple or list of tensors. Otherwise, it will raise an error.
Note:
On the SEMI_AUTO_PARALLEL mode or AUTO_PARALLEL mode, if the input `x` is the gradient,
the gradient norm values on all devices will be automatically aggregated by allreduce inserted after the local
square sum of the gradients.
Args:
x (Union(tuple[Tensor], list[Tensor])): Input data to clip.