Commit Graph

2436 Commits

Author SHA1 Message Date
Jesse Lee 8a08d0c37b Phase 2 of CacheOp 2020-08-19 13:45:55 -04:00
Zirui Wu 78c1aa1d96 Implemented Callback for Dataset
implment pause in MapOp, added more to callback

add ds_callback

- Initial drop of Python DSCallback

- Pybind DSCallback

- Pybind DSCallback

added callback to mapOp

- de_pipeline DSCallback

- de_pipeline DSCallback

add test case, segfault for now

fix seg fault

- de_pipeline DSCallback

remove 1 line

update callback test case, now works

use builder class for mapOp callback

- de_pipeline DSCallback

- de_pipeline DSCallback

- de_pipeline DSCallback

better test case

minor fix

add comments and minor clean ups

get rid of nullptr in MapOp, use other flag instead

fix a bug ParseMapOp only takes 1 callback

- Added WaitedDSCalabck

refactor callback param

fix text case incorrect number

- added testing

fix cpp test case

- added testing

- revert back lenet changes

- cleanup test_callbacks.py

- cleanup test_callbacks.py

fix CI stage I

fix CI stage II

fix CI and update epoch counter

- add validation
- add more testing  test_callbacks.py

use random data op to do tests

adjust when to call EpochBegin/End

- add repeat with callback

- addressing reviewers' comments

- docstring and CI fixes

- docstring and CI fixes

- docstring and CI fixes

- rebase with upstream/master

fix cpp test case

fix review comments

addr review cmts, add test case
2020-08-19 12:31:04 -04:00
YangLuo b50ae27c72 !4597 [Dataset] C++ API Support for build_vocab 2020-08-19 22:47:47 +08:00
mindspore-ci-bot 748d7f1f1a !4685 Export server embedding table ckpts.
Merge pull request !4685 from ZPaC/master-supports-export-server-ckpt
2020-08-19 20:08:00 +08:00
ZPaC 5084557f67 Export server embedding table ckpt. 2020-08-19 17:09:57 +08:00
mindspore-ci-bot d541e261a0 !4696 C++ API Support for CSV Dataset
Merge pull request !4696 from jiangzhiwen/jzw/c_api_csv
2020-08-19 17:08:33 +08:00
mindspore-ci-bot 70a218123b !4722 modiy error type
Merge pull request !4722 from lijiaqi/modify_error_type
2020-08-19 16:18:50 +08:00
mindspore-ci-bot 2db4bda219 !4622 serving restful init
Merge pull request !4622 from hexia/http_init
2020-08-19 16:15:58 +08:00
mindspore-ci-bot 256dccc651 !4498 Gnn data processing supports distributed scenarios
Merge pull request !4498 from heleiwang/gnn_distributed
2020-08-19 16:09:05 +08:00
jiangzhiwen 39df25f803 c++ api for CSV Dataset 2020-08-19 16:02:11 +08:00
mindspore-ci-bot 2112cb91d5 !4690 add categorical distribution
Merge pull request !4690 from baihuawei/categorical
2020-08-19 14:31:14 +08:00
李嘉琪 c65ea1687b modify error type 2020-08-19 14:21:56 +08:00
mindspore-ci-bot 2d683234fb !4723 Support to concat more than 3 tensors in auto parallel mode
Merge pull request !4723 from yangzhenzhang/concat-more-than-3-tensors
2020-08-19 14:16:02 +08:00
yangzhenzhang cda08f6a52 concat 3 tensors in auto parallel mode 2020-08-19 11:27:35 +08:00
baihuawei 772e14d00d add categorical 2020-08-19 11:21:15 +08:00
heleiwang 8ee4d8e92d Gnn data processing supports distributed scenarios 2020-08-19 11:09:01 +08:00
mindspore-ci-bot 6868b9b6fa !4422 Minddata new dataset api ==> RandomData
Merge pull request !4422 from xiefangqi/xfq_c++api_randomdata
2020-08-19 10:48:53 +08:00
mindspore-ci-bot 50b5c8035b !4701 fix concat get_dataset_size error
Merge pull request !4701 from guozhijian/fix_concat_get_dataset_size_err
2020-08-19 09:14:30 +08:00
xiefangqi b91e56375e add randomdataset and schema 2020-08-19 09:13:43 +08:00
mindspore-ci-bot 2aab14242c !4669 add toolbox and dpn
Merge pull request !4669 from zhangxinfeng3/master
2020-08-19 09:13:29 +08:00
mindspore-ci-bot 505b6b5a9b !4699 NMS_GPU_Sorting_Mem Fix
Merge pull request !4699 from danishnxt/GPU_two
2020-08-19 09:08:20 +08:00
nhussain 738ae2c78d change api to use std_vector
try to display image
2020-08-18 15:41:21 -04:00
islam_amin 979111d0ef Added Posterize Op 2020-08-18 13:55:47 -04:00
jonyguo b0b370336f fix: concat get_dataset_size error 2020-08-18 23:40:17 +08:00
danish 97f08e74ec nms_sorting fix
lint py fix 2

nms_py_file test value fix

lint fix
2020-08-18 11:15:33 -04:00
jonyguo 5f294fb90d fix: random affine parameter check 2020-08-18 21:58:53 +08:00
panyifeng 22a9d02e9f switch_layer incorporate env_get and tuple_get 2020-08-18 20:23:23 +08:00
hexia 75d116b5db serving http init 2020-08-18 20:15:47 +08:00
zhangxinfeng3 d643c228d7 update variational and toolbox 2020-08-18 19:02:27 +08:00
mindspore-ci-bot 4f46c4277a !4614 Code compile performance optimization
Merge pull request !4614 from zhoufeng/code-compile-performance
2020-08-18 16:45:15 +08:00
mindspore-ci-bot cc077de9e3 !4425 fix if statement with no return coredump
Merge pull request !4425 from fary86/fix_if_no_return_coredump
2020-08-18 16:09:31 +08:00
mindspore-ci-bot 2ae6365d77 !4650 EmbeddingLookup support auto parallel
Merge pull request !4650 from yangzhenzhang/embedding-lookup-auto-parallel
2020-08-18 15:51:04 +08:00
zhoufeng 663278112f optimize code compile performance
Signed-off-by: zhoufeng <zhoufeng54@huawei.com>
2020-08-18 14:33:24 +08:00
mindspore-ci-bot 025e6f23f7 !4643 add test cases for parameter slice init using all kind of initializers
Merge pull request !4643 from yihuaijie/dev
2020-08-18 14:21:41 +08:00
mindspore-ci-bot 26de4540ef !4639 rename param_value to param_info
Merge pull request !4639 from vlne-v1/rename_param_value
2020-08-18 14:16:23 +08:00
fary86 38083e055a Fix coredump missing return statement after while loop 2020-08-18 14:16:21 +08:00
yangzhenzhang 6f6a8ae9f0 embedding lookup auto parallel 2020-08-18 11:40:09 +08:00
Wei Luning c1c30a44f1 rename param_value -> param_info 2020-08-18 11:16:28 +08:00
mindspore-ci-bot b92e776052 !4208 C++ API Support for Manifest Dataset
Merge pull request !4208 from jiangzhiwen/jzw/c_api_manifest
2020-08-18 10:50:41 +08:00
Yi Huaijie 0f7ead5f14 parameter slice init test all initializers 2020-08-18 10:34:15 +08:00
mindspore-ci-bot 3fb58fcbe4 !4585 add gpu nccl broadcast
Merge pull request !4585 from baihuawei/broadcast
2020-08-18 09:53:52 +08:00
jiangzhiwen a88273bd3f c++ api support for Manifest Dataset 2020-08-18 09:48:55 +08:00
mindspore-ci-bot 19192e75e3 !4609 change unsupport to unsupported
Merge pull request !4609 from chenzhongming/new_master
2020-08-18 09:11:46 +08:00
mindspore-ci-bot 113ff6ca24 !4367 CutMixBatch Augmentation Op
Merge pull request !4367 from MahdiRahmaniHanzaki/cutmix
2020-08-18 03:27:33 +08:00
mahdi 3ecc53fb4e Added CutMix 2020-08-17 13:01:47 -04:00
chenzomi bb125cb309 change unsupport to unsupported 2020-08-17 20:37:09 +08:00
baihuawei b9ebd9c280 add gpu nccl broadcast 2020-08-17 20:11:09 +08:00
mindspore-ci-bot d1ad33676b !4598 support axis is None for all and any interface in graph mode
Merge pull request !4598 from zhangbuxue/support_axis_is_None_for_all_and_any_in_graph_mode
2020-08-17 19:52:40 +08:00
mindspore-ci-bot 8c72d5b9d1 !4566 Add ScatterNdUpdate cpu kernel
Merge pull request !4566 from huanghui/scatter-nd-update-cpu-kernel
2020-08-17 19:33:04 +08:00
buxue 09f1a4bbaf support axis is None for all and any interface in graph mode. 2020-08-17 17:53:02 +08:00