!31150 Add document for clip by norm about the parallel mode

Merge pull request !31150 from huangxinjing/code_docs_add_clip_by_norm
This commit is contained in:
i-robot 2022-03-14 09:46:34 +00:00 committed by Gitee
commit fce00c509a
No known key found for this signature in database
GPG Key ID: 173E9B9CA92EEF8F
2 changed files with 10 additions and 2 deletions

View File

@ -6,7 +6,10 @@
通过权重梯度总和的比率来裁剪多个Tensor的值。
.. note::
输入'x'应为Tensor的tuple或list。否则将引发错误。
输入 `x` 应为Tensor的tuple或list。否则将引发错误。
.. note::
在半自动并行模式或自动并行模式下,如果输入是梯度,那么将会自动汇聚所有设备上的梯度的平方和。
**参数:**

View File

@ -197,7 +197,12 @@ def clip_by_global_norm(x, clip_norm=1.0, use_norm=None):
Clips tensor values by the ratio of the sum of their norms.
Note:
Input 'x' should be a tuple or list of tensors. Otherwise, it will raise an error.
Input `x` should be a tuple or list of tensors. Otherwise, it will raise an error.
Note:
On the SEMI_AUTO_PARALLEL mode or AUTO_PARALLEL mode, if the input `x` is the gradient,
the gradient norm values on all devices will be automatically aggregated by allreduce inserted after the local
square sum of the gradients.
Args:
x (Union(tuple[Tensor], list[Tensor])): Input data to clip.