forked from mindspore-Ecosystem/mindspore
!46447 [MD][DOC] Add Chinese doc for error_samples_mode APIs
Merge pull request !46447 from TinaMengtingZhang/code_docs_error_samples_mode
This commit is contained in:
commit
df2f50f734
|
@ -0,0 +1,13 @@
|
|||
mindspore.dataset.config.get_error_samples_mode
|
||||
===============================================
|
||||
|
||||
.. py:function:: mindspore.dataset.config.get_error_samples_mode()
|
||||
|
||||
获取当前数据管道中处理错误样本的方法。
|
||||
|
||||
返回:
|
||||
ErrorSamplesMode,当前数据管道处理错误样本的方法。
|
||||
|
||||
- ErrorSamplesMode.RETURN: 表示错误样本会导致产生错误并返回。
|
||||
- ErrorSamplesMode.REPLACE: 表示错误样本会被内部确定的样本替换。
|
||||
- ErrorSamplesMode.SKIP: 表示错误样本会被跳过。
|
|
@ -0,0 +1,21 @@
|
|||
mindspore.dataset.config.set_error_samples_mode
|
||||
===============================================
|
||||
|
||||
.. py:function:: mindspore.dataset.config.set_error_samples_mode(error_samples_mode)
|
||||
|
||||
设置在数据管道中处理错误样本的方法。
|
||||
|
||||
.. note::
|
||||
- 此错误样本特性仅适用于数据集管道中的Map操作。
|
||||
- 对于替换模式,缓存将会用于存放内部确定的样本。
|
||||
- 如果在多机设置中使用跳过模式,请手动确保每个分片的有效样本数是相同的(否则可能会导致挂起)。一种解决方法是手动Concat一个样本全有效的数据集,然后采用Take操作填补跳过的错误样本数。
|
||||
|
||||
参数:
|
||||
- **error_samples_mode** (ErrorSamplesMode) - 处理错误样本的方法。默认值:ErrorSamplesMode.RETURN。
|
||||
|
||||
- ErrorSamplesMode.RETURN:表示错误样本会导致产生错误并返回。
|
||||
- ErrorSamplesMode.REPLACE:表示错误样本会被内部确定的样本替换。
|
||||
- ErrorSamplesMode.SKIP:表示错误样本会被跳过。
|
||||
|
||||
异常:
|
||||
- **TypeError** - `error_samples_mode` 不是ErrorSamplesMode类型。
|
|
@ -259,6 +259,8 @@ config模块能够设置或获取数据处理的全局配置参数。
|
|||
mindspore.dataset.config.get_fast_recovery
|
||||
mindspore.dataset.config.set_multiprocessing_timeout_interval
|
||||
mindspore.dataset.config.get_multiprocessing_timeout_interval
|
||||
mindspore.dataset.config.set_error_samples_mode
|
||||
mindspore.dataset.config.get_error_samples_mode
|
||||
|
||||
其他
|
||||
-----
|
||||
|
|
|
@ -178,6 +178,8 @@ The configuration module provides various functions to set and get the supported
|
|||
mindspore.dataset.config.get_fast_recovery
|
||||
mindspore.dataset.config.set_multiprocessing_timeout_interval
|
||||
mindspore.dataset.config.get_multiprocessing_timeout_interval
|
||||
mindspore.dataset.config.set_error_samples_mode
|
||||
mindspore.dataset.config.get_error_samples_mode
|
||||
|
||||
Others
|
||||
-------
|
||||
|
|
|
@ -912,12 +912,12 @@ def set_error_samples_mode(error_samples_mode):
|
|||
Set the method in which erroneous samples should be processed in a dataset pipeline.
|
||||
|
||||
Note:
|
||||
1. This error samples feature is only applicable to the Map operation in a dataset pipeline.
|
||||
2. For replacement mode, a cache of internally determined samples will be used.
|
||||
3. If skip mode is used in a distributed setting, beware to manually ensure the
|
||||
number of valid samples are the same for each shard (otherwise one may encounter hangs).
|
||||
One technique is to manually concat a dataset of all valid samples plus a
|
||||
take operation for the number of skipped erroneous samples.
|
||||
- This error samples feature is only applicable to the Map operation in a dataset pipeline.
|
||||
- For replacement mode, a cache of internally determined samples will be used.
|
||||
- If skip mode is used in a distributed setting, beware to manually ensure the
|
||||
number of valid samples are the same for each shard (otherwise one may encounter hangs).
|
||||
One technique is to manually concat a dataset of all valid samples plus a
|
||||
take operation for the number of skipped erroneous samples.
|
||||
|
||||
Args:
|
||||
error_samples_mode (ErrorSamplesMode): The method in which erroneous samples should be processed in a dataset
|
||||
|
@ -946,9 +946,11 @@ def get_error_samples_mode():
|
|||
|
||||
Returns:
|
||||
ErrorSamplesMode, The method in which erroneous samples should be processed in a dataset pipeline.
|
||||
|
||||
- ErrorSamplesMode.RETURN: means erroneous sample results in error raised and returned.
|
||||
- ErrorSamplesMode.REPLACE: means erroneous sample is replaced with an internally determined sample.
|
||||
- ErrorSamplesMode.SKIP: means erroneous sample is skipped.
|
||||
|
||||
Examples:
|
||||
>>> error_samples_mode = ds.config.get_error_samples_mode()
|
||||
"""
|
||||
|
|
Loading…
Reference in New Issue