Commit Graph

2375 Commits

Author SHA1 Message Date
mindspore-ci-bot 527d7f0afb !5580 [MD] remove finish in FileReader
Merge pull request !5580 from liyong126/rmv_filereader_finish
2020-09-03 21:18:35 +08:00
yao_yf 05c003ae6b origin/semi_auto_parallel_reshape_parameter_has_another_user 2020-09-03 20:27:46 +08:00
mindspore-ci-bot fc79997de5 !5502 Mod SoftmaxCrossEntropyWithlogits
Merge pull request !5502 from wanyiming/mod_SoftmaxCrossEntropyWithlogits
2020-09-03 20:24:43 +08:00
mindspore-ci-bot b961d47d40 !5642 UTest compiling performance optimization
Merge pull request !5642 from zhoufeng/ut-compile-performance
2020-09-03 20:21:13 +08:00
liyong ac39c20f41 del finish in FileReader 2020-09-03 20:05:38 +08:00
mindspore-ci-bot 01da7d2c1a !5713 Align num_samples of CSV with other dataset
Merge pull request !5713 from jiangzhiwen/fix/csv_num_samples
2020-09-03 19:52:43 +08:00
Xiaoda Zhang 42f1241270 remove 'multi-subgraphs' to internal 2020-09-03 19:07:08 +08:00
wanyiming 0ec70068ae mod_SoftmaxCrossEntropyWithLogits 2020-09-03 17:33:20 +08:00
jiangzhiwen 1a1a8893a4 Align num_samples of CSV with other dataset 2020-09-03 17:30:39 +08:00
zhoufeng 6d500223b2 utest compiling performance optimization
Signed-off-by: zhoufeng <zhoufeng54@huawei.com>
2020-09-03 16:57:19 +08:00
zhousiyi 301aeaaf1d remove gvar source file from compile as ut_tests will use gvar so 2020-09-03 06:49:30 +00:00
mindspore-ci-bot 0118930c6b !5671 modify voc doc & fix buildvocab bug
Merge pull request !5671 from luoyang/c-api-pyfunc
2020-09-03 09:25:13 +08:00
mindspore-ci-bot 290979853e !5663 add check for mod when y in (-1, 0)
Merge pull request !5663 from zhangbuxue/add_check_for_mod_for_y
2020-09-03 09:16:45 +08:00
mindspore-ci-bot 021ba724cf !5645 [bug][api]updata signature
Merge pull request !5645 from vlne-v1/ref_demo
2020-09-03 09:11:50 +08:00
Mahdi 6e7aab8c4f Added GetColumnNames for Datasets 2020-09-02 17:12:48 -04:00
Cathy Wong 4d4c11b133 dataset API docstring cleanup: Standard product terms NumPy, Python 2020-09-02 10:42:12 -04:00
tinazhang e430b4056a add tfrecord dataset to cpp api
fix to support schema=nullptr
2020-09-02 08:55:47 -04:00
mindspore-ci-bot 35e6cca1a3 !5634 wrap numpy random seed into an api
Merge pull request !5634 from yihuaijie/master
2020-09-02 19:28:33 +08:00
buxue 4d59fe9ee1 add check for mod when y in (-1, 0) 2020-09-02 18:10:00 +08:00
Wei Luning 879a519136 updata signature 2020-09-02 17:29:29 +08:00
Yi Huaijie 4a5d115a66 add get_seed() and set_seed() 2020-09-02 17:17:04 +08:00
luoyang 8aba39a71b Modify VOCDataset doc & Fix BuildVocab bug 2020-09-02 16:54:50 +08:00
He Wei c1469d61ce Fix get_py_obj_dtype() for mindspore type
Return mstype.type_type if input is mindspore type object,
for example:
```
get_py_obj_dtype(mstype.float32) ---> mstype.type_type
```
2020-09-02 16:14:25 +08:00
mindspore-ci-bot ccc0ea60ee !5661 fix auto parallel reshape strategy set when it is first operator
Merge pull request !5661 from yao_yf/auto_parallel_reshape_fix
2020-09-02 15:42:47 +08:00
mindspore-ci-bot 847fadc8b9 !5516 auto parallel context modify
Merge pull request !5516 from yao_yf/auto_parallel_context_collation
2020-09-02 15:41:05 +08:00
mindspore-ci-bot bb84f50407 !5473 optim pylint
Merge pull request !5473 from jinyaohui/master
2020-09-02 15:25:30 +08:00
mindspore-ci-bot 4ec343961e !5482 modify save_checkpoint
Merge pull request !5482 from liuyang/md_save_checkpoint
2020-09-02 14:20:48 +08:00
yao_yf 755f381406 fix auto parallel reshape strategy set when it is first operator 2020-09-02 14:15:54 +08:00
mindspore-ci-bot 5e3e583b47 !5615 Add check for duplicate column to a-cpi & add voc descriptions for class_index
Merge pull request !5615 from luoyang/c-api-pyfunc
2020-09-02 11:59:34 +08:00
mindspore-ci-bot 3048240f16 !5508 Add AdamApplyOneWithDecayAssign fusion pass
Merge pull request !5508 from YuJianfeng/adam_assign
2020-09-02 11:23:34 +08:00
luoyang ce419404ce Add check for duplicate column for a-cpi 2020-09-02 10:41:52 +08:00
mindspore-ci-bot 03093778df !5387 Set num_epochs=1 for MindData python testcases
Merge pull request !5387 from lixiachen/testcase_epoch
2020-09-02 09:45:13 +08:00
mindspore-ci-bot be3d79cb6b !4204 add dynamic shape support for GatherV2 and others
Merge pull request !4204 from fary86/adapt_primitive_dynamic_shape
2020-09-02 09:31:14 +08:00
mindspore-ci-bot 120ab80b23 !5641 check user define bprop when there is parameter in nested network
Merge pull request !5641 from zhangbuxue/check_user_define_bprop_when_there_is_parameter_in_nested_network
2020-09-02 09:04:05 +08:00
mindspore-ci-bot 0cb6d29f0c !4998 Shared memory pool clean up
Merge pull request !4998 from JesseKLee/mem_pool
2020-09-02 02:01:19 +08:00
Lixia Chen 27d5281641 Change epoch count to 1 for python testcases 2020-09-01 11:24:26 -04:00
buxue 359e1f236e check user define bprop when there is parameter in nested network 2020-09-01 21:24:27 +08:00
fary86 144a35b17e Adapt GatherV2 for dynamic shape 2020-09-01 19:24:21 +08:00
mindspore-ci-bot 8f3ebfd469 !5624 Optimize the performance of GraphData.get_neg_sampled_neighbors
Merge pull request !5624 from heleiwang/gnn_perf
2020-09-01 19:22:53 +08:00
heleiwang 857cf2f77f Optimize the negative sampling process 2020-09-01 16:29:59 +08:00
mindspore-ci-bot c2fddb56c8 !4922 Transform tuple parameter to multiple parameters
Merge pull request !4922 from amongo/TupleTransform
2020-09-01 15:15:56 +08:00
huangdongrun 0099da2c99 add support for tuple parameter transform
add support for pynative pass

add testcases
2020-09-01 14:17:52 +08:00
mindspore-ci-bot dff02cc282 !5188 rename image files in imagefolder
Merge pull request !5188 from tony_liu2/alt
2020-09-01 11:16:30 +08:00
mindspore-ci-bot 5453b40311 !4477 add dataset compare function to utils
Merge pull request !4477 from tony_liu2/staging
2020-09-01 11:15:18 +08:00
mindspore-ci-bot b99149dc37 !4901 Adding data_helper for minddata
Merge pull request !4901 from EricZ/md_helper
2020-09-01 11:14:11 +08:00
mindspore-ci-bot ab29dbf98b !5385 dataset fixes: Update OneHot API doc; fixup UTs
Merge pull request !5385 from cathwong/ckw_dataset_ut_cleanup8
2020-09-01 11:11:49 +08:00
mindspore-ci-bot 8ebc8ce9c6 !5438 support parameter tuple input in pynative mode
Merge pull request !5438 from wangqiuliang/support-parameter-tuple-input
2020-09-01 11:04:03 +08:00
mindspore-ci-bot 3be136f6f0 !5354 Fix randomdata issue
Merge pull request !5354 from xiefangqi/fix_randomdata_columnlist
2020-09-01 11:03:54 +08:00
mindspore-ci-bot 6cba0deb05 !4877 BucketBatchByLength for C++ API
Merge pull request !4877 from MahdiRahmaniHanzaki/bucket_batch_by_length_c
2020-09-01 10:58:51 +08:00
mindspore-ci-bot 1a4d3e351e !5586 Boost python pass compile and train performance
Merge pull request !5586 from BowenK/performance
2020-09-01 09:19:02 +08:00
mahdi 84efa0b96b Added BucketBatchByLength for C API 2020-08-31 17:06:16 -04:00
Eric 65bf3ecc3e Adding data_helper class 2020-08-31 16:29:14 -04:00
Cathy Wong 7f6782be2a dataset fixes: Update OneHot API docs; fixup Python UTs 2020-08-31 15:04:26 -04:00
mindspore-ci-bot 65819fb50c !5488 heck arg is tensor when graph output is a const or parameter
Merge pull request !5488 from zhangbuxue/add_check_tensor_before_compile_graph
2020-08-31 21:36:04 +08:00
yao_yf 8f7aa5bd5a auto parallel context modify 2020-08-31 21:19:49 +08:00
kingfo 7765d44b76 support parameter tuple input in pynative mode 2020-08-31 20:38:43 +08:00
BowenK d6fb7d2db1 Remove debug drawing and printing to boost compile performance; re-opt after python pass to boost training; fix NewParameter tensor clone 2020-08-31 17:42:00 +08:00
buxue 4168c51f0d add check tensor when graph output is a const or parameter 2020-08-31 17:17:24 +08:00
liuyang_655 4683de3443 modify save_checkpoint 2020-08-31 16:24:57 +08:00
jinyaohui a9972a7def optim pylint 2020-08-31 14:49:57 +08:00
mindspore-ci-bot 8d41931456 !5352 refactor ms_context implementation
Merge pull request !5352 from fary86/refactor_context_interface
2020-08-31 09:27:33 +08:00
mindspore-ci-bot 4d963d96f4 !5534 Fix Tensor.from_numpy() returns wrong type
Merge pull request !5534 from hewei/fix_tensor_from_numpy
2020-08-31 09:15:48 +08:00
Jesse Lee e89b4648f6 Memory pool clean up 2020-08-30 11:28:24 -04:00
fary86 fcbb3e0edc Refactor ms_context implementation 2020-08-30 22:51:40 +08:00
He Wei 27721ff4ea Fix Tensor.from_numpy() returns wrong type
Tensor.from_numpy() should return mindspore.Tensor not _c_expression.Tensor.
2020-08-29 19:05:44 +08:00
yujianfeng 4b77f6b53c Add AdamApplyOneWithDecayAssign fusion pass 2020-08-29 17:54:15 +08:00
mindspore-ci-bot e6a4d932b4 !5350 [AutoParallel]Rectification distributed init
Merge pull request !5350 from lichen/rectification_init
2020-08-29 14:55:18 +08:00
mindspore-ci-bot d81b30e6a0 !5312 make backend/optimizer free of pybind
Merge pull request !5312 from xychow/remove-backend-py-dependency-2
2020-08-29 14:42:33 +08:00
mindspore-ci-bot be606ba8f5 !5432 Mindspore parallel supports all elementary-wise operators
Merge pull request !5432 from yihuaijie/master
2020-08-29 14:16:48 +08:00
lichenever d3e55b543e rectification init 2020-08-29 09:34:58 +08:00
mindspore-ci-bot 1821e98e07 !4087 asyn run graph
Merge pull request !4087 from kisnwang/async-run-graph
2020-08-29 09:18:37 +08:00
Yi Huaijie 84948ca730 parallel supports more elementary-wise operators 2020-08-28 18:19:06 +08:00
kswang 756bb6d53f async run graph 2020-08-28 18:18:47 +08:00
zhousiyi c25e37e7bf make backend/optimizer pybind free 2020-08-28 08:56:58 +00:00
mindspore-ci-bot 414184c184 !5367 Check the parameter's split strategies if it has multiple users
Merge pull request !5367 from yangzhenzhang/check-parameter-split
2020-08-28 16:26:46 +08:00
mindspore-ci-bot 820f2cb4eb !5351 move ParalleMode to Context
Merge pull request !5351 from yao_yf/parallel_context_collation
2020-08-28 16:13:01 +08:00
yao_yf 07117e4dd4 mv ParallelMode to context 2020-08-28 19:13:55 +08:00
xiefangqi e2085a9505 fix randomdata issue 2020-08-28 11:55:49 +08:00
He Wei d7cdda1cb0 Provide Tensor.from_numpy() to avoid copy 2020-08-28 11:03:14 +08:00
yangzhenzhang fbda03bbcc check parameter split 2020-08-28 10:49:27 +08:00
tony_liu2 70bfd506a1 add function
pr comment fix

revert graphengine
2020-08-27 16:50:19 -04:00
tony_liu2 5e4c314130 rename dataset dir and image files
change references to use path original

revert folder name

revert graphengine
2020-08-27 16:49:23 -04:00
mindspore-ci-bot 716329df6c !5278 Change default value of sampler in c-api
Merge pull request !5278 from luoyang/c-api
2020-08-27 21:33:18 +08:00
mindspore-ci-bot 7aaf4118bf !5297 CutMixBatch and MixUpBatch fix for 2D one-hot labels
Merge pull request !5297 from MahdiRahmaniHanzaki/cutmix-fix
2020-08-27 21:32:09 +08:00
YangLuo 419478b410 Change default value of sampler to seperate behaviour of default sampler
& null sampler, add check for duplicate column name
2020-08-27 21:24:37 +08:00
mindspore-ci-bot 9392623066 !5181 Python pass pattern renaming and interface tweaking
Merge pull request !5181 from BowenK/new_parameter
2020-08-27 19:45:36 +08:00
mindspore-ci-bot a8317acee8 !5111 Support int64 for cpu sparse optimizers
Merge pull request !5111 from YuJianfeng/int64
2020-08-27 17:17:05 +08:00
mindspore-ci-bot e94416be0c !5283 Support setting operator io format in the frontend
Merge pull request !5283 from liangchenghui/io_format
2020-08-27 15:48:13 +08:00
mindspore-ci-bot fabebb2678 !5264 [bug]fix bugs when parameters updata
Merge pull request !5264 from vlne-v1/I1SP3I-return-value-not-the-exact-value
2020-08-27 14:13:20 +08:00
mindspore-ci-bot b346f0b3ec !5026 Fix the problem of resource clear
Merge pull request !5026 from Simson/enhancement-API
2020-08-27 11:19:02 +08:00
yujianfeng e949540062 Support int64 for cpu sparse optimizers 2020-08-27 10:57:53 +08:00
simson d4babf190c fix risk of memory leak 2020-08-27 10:54:06 +08:00
mindspore-ci-bot d1b95ed9bd !5240 Fix remove internal output for unique device target
Merge pull request !5240 from YuJianfeng/internal_output
2020-08-27 10:35:37 +08:00
BowenK 641d12d6d9 python pass pattern renaming and interface tweaking 2020-08-27 10:03:46 +08:00
mindspore-ci-bot 66d6320b21 !5224 Add test case about loss scale in parallel mode
Merge pull request !5224 from yangzhenzhang/add-split-sens-and-loss-scale-test-case
2020-08-27 09:06:01 +08:00
mahdi 7ad7024c45 Fixed 2D one-hot label problems in CutMix and MixUp 2020-08-26 16:17:21 -04:00
islam_amin 94a4a3f20b changing affine and posterize apis 2020-08-26 10:51:54 -04:00
Wei Luning 4cc8342b5b fix bug in Parameter init_data 2020-08-26 22:15:22 +08:00
梁成辉 34d433fd9a Set io format for old backend 2020-08-26 20:49:15 +08:00
kpy 44b738e8ac fix grad all ci bug 2020-08-26 18:28:35 +08:00
yujianfeng e688e1df32 Fix remove internal output for unique device target 2020-08-26 17:31:35 +08:00
mindspore-ci-bot 314c5ad055 !5172 remove name arg from gradoperation
Merge pull request !5172 from riemann_penn/remove_name_args_from_gradoperation
2020-08-26 16:55:57 +08:00
mindspore-ci-bot 95212b55a0 !3271 make reftype a subtype of MetaTensor and try to mark ref in node input
Merge pull request !3271 from vlne-v1/ref_demo
2020-08-26 15:45:48 +08:00
yangzhenzhang 6ae5893681 add test cases 2020-08-26 15:11:44 +08:00
Wei Luning 24a10225cf change base class of ref to tensor in cpp 2020-08-26 12:41:25 +08:00
mindspore-ci-bot e4d2f2fd3b !5030 Adding a BBox class and fixing Posterize C Op testing.
Merge pull request !5030 from islam_amin/bbox_class
2020-08-26 11:40:57 +08:00
panyifeng 1a54785fe2 remove name arg from gradoperation 2020-08-26 11:16:57 +08:00
mindspore-ci-bot cd1240180b !5187 C++ API: Lexicographical order support for CLUE, CSV & TextFile Datasets
Merge pull request !5187 from cathwong/ckw_c_api_fixes2
2020-08-26 10:48:05 +08:00
mindspore-ci-bot 20b3134785 !5184 fix: padded dataset when no div and with repeat op
Merge pull request !5184 from guozhijian/fix_padded_with_no_div_repeat
2020-08-26 10:47:22 +08:00
mindspore-ci-bot f3fd7a5578 !5073 Add checks and exception handling DS callback
Merge pull request !5073 from h.farahat/map_callback_end
2020-08-26 10:40:40 +08:00
jonyguo 6df1a43ec2 fix: padded dataset with non div & repeat 2020-08-26 08:56:37 +08:00
Cathy Wong 46e9d79a50 C++ API: Lexicographical order support for CLUE, CSV & TextFile Datasets 2020-08-25 14:11:19 -04:00
mindspore-ci-bot 971716f4b9 !5136 fix large for loop runtime error due to lacking of backend operators
Merge pull request !5136 from fary86/fix_large_for_runtime_error
2020-08-25 22:26:25 +08:00
mindspore-ci-bot 346b97acb8 !5156 fix softdvpp coredump
Merge pull request !5156 from qianlong21st/master_fix_softdvpp_core
2020-08-25 22:20:00 +08:00
mindspore-ci-bot 2e3f5cd41b !5114 fix bug in c-api: rename, concat, take
Merge pull request !5114 from luoyang/c-api-pyfunc
2020-08-25 20:57:41 +08:00
qianlong 12dfca71f1 fix softdvpp coredump 2020-08-25 18:56:40 +08:00
YangLuo 67f5c89cd6 Check input image type for random posterize 2020-08-25 16:26:17 +08:00
fary86 8d245497a4 Fix large for loop Runtime Error due to backend missing operators 2020-08-25 16:25:41 +08:00
luoyang b26f6b3d5e fix bug in c-api: rename, concat, take 2020-08-25 16:25:00 +08:00
huangdongrun 113641decd fix test cases 2020-08-25 14:38:08 +08:00
mindspore-ci-bot 7d4f481884 !5017 remove internal interface in wide&deep
Merge pull request !5017 from yao_yf/wide_and_deep_no_internal_interface
2020-08-25 14:11:51 +08:00
mindspore-ci-bot 957126375b !5095 Adding Float tensor support for CutMixBatch
Merge pull request !5095 from MahdiRahmaniHanzaki/cutmix-fix
2020-08-25 10:53:58 +08:00
mindspore-ci-bot afd16fbf0a !4963 fix bug of switch layer join
Merge pull request !4963 from fary86/fix_switch_layer_join_bug
2020-08-25 10:47:32 +08:00
mindspore-ci-bot abe6b82138 !5011 remove global grad ops
Merge pull request !5011 from riemann_penn/remove_global_grad_ops
2020-08-25 09:47:55 +08:00
fary86 947e19b839 Fix bug of switch layer join 2020-08-25 09:08:53 +08:00
islam_amin 641e751d5e adding bbox class 2020-08-24 18:22:58 -04:00
Mahdi a2c38d89f9 Added float32 support for CutMixBatch 2020-08-24 17:15:25 -04:00
hesham 8eeceb267b - release the wait if training is ended 2020-08-24 13:11:07 -04:00
yao_yf a9a8e323b2 remove internal interface in wide&deep 2020-08-24 19:18:47 +08:00
mindspore-ci-bot 4f6e63fcf8 !4576 Support if by if not inline
Merge pull request !4576 from amongo/SupportIfByIfNotInline
2020-08-24 19:15:39 +08:00
mindspore-ci-bot fc6eee3bda !5019 raise RuntimeError when set different mode after Initializer created
Merge pull request !5019 from yihuaijie/dev
2020-08-24 18:49:32 +08:00
huangdongrun 1bd9fefd84 support if by if not inline
add testcase of net of if by if
2020-08-24 17:11:26 +08:00
panyifeng 637e812347 remove global grad ops 2020-08-24 16:50:09 +08:00
mindspore-ci-bot 226f019e45 !4888 add parallel attention test
Merge pull request !4888 from hanyang/master
2020-08-24 15:50:56 +08:00
Yi Huaijie 394be43492 raise RuntimeError when set different mode after Initializer created 2020-08-24 14:08:09 +08:00
mindspore-ci-bot 8d693306f4 !4126 Add new parameter
Merge pull request !4126 from BowenK/new_parameter
2020-08-24 10:01:46 +08:00
mindspore-ci-bot 56835aaf88 !5005 Fix the bug in the formula in Bernoulli log_probs
Merge pull request !5005 from zichun_ye/fix_bernoulli_probs
2020-08-24 09:09:43 +08:00
mindspore-ci-bot 0bbce9367e !4772 Adding AlbumDataset for device training
Merge pull request !4772 from EricZ/ms-album
2020-08-24 08:27:14 +08:00
Zichun Ye 9e7d6e2397 fix bernoulli prob formula; fix some other minor bugs
update threshold of softplus computation

support fp for bernoulli and geometric distribution
2020-08-23 17:49:09 -04:00
Eric c79db93c48 Initial commit for album
Added linter fix for album dataset

Added testDataset

Adding signature

Added JsonDataset example API

Example dataset

Resolving format

More fixing

Refactor

Small fix

Added compiling album dataset

Running tests

Added linter fix #1

Passing UT

Added dataset API

Addressing clang

Clang part 2

Fixing pass

Fixed tree check

lint fix

Added lint fix part 2
2020-08-22 11:07:10 -04:00
mindspore-ci-bot 94a109f476 !4898 Fix coredump caused by function call depth too large
Merge pull request !4898 from fary86/fix_call_depth_too_large
2020-08-22 16:08:27 +08:00
fary86 04524b6bd3 Fix coredump caused by function call depth too large 2020-08-22 15:00:04 +08:00
mindspore-ci-bot ac81886328 !4916 fix generator_dataset hangs and test_graphdata_distributed.py failing randomly
Merge pull request !4916 from heleiwang/gnn_fix_bug
2020-08-22 10:51:01 +08:00
mindspore-ci-bot b366608a3f !4952 Fix errors in log calculation logics
Merge pull request !4952 from peixu_ren/custom_pp_ops
2020-08-22 10:40:40 +08:00
mindspore-ci-bot 9b503e4f38 !4955 Fixes for Dynamic Augmentation Ops
Merge pull request !4955 from MahdiRahmaniHanzaki/dynamic-ops-fix
2020-08-22 10:27:50 +08:00
peixu_ren 1c8eb9b15d Fix errors in log calculation logics 2020-08-21 21:28:10 -04:00
Mahdi a5f9b8f92e Added fix for MixUpBatch and CutMixBatch and for RandomAffine
updated c color op descriptions
2020-08-21 17:40:08 -04:00
mindspore-ci-bot 9d7250c483 !4776 Introduce 2 extra ctrl flags to DataBuffer in dataset, address remaining cmts to PR4632
Merge pull request !4776 from ZiruiWu/map_callback_follow_up
2020-08-22 03:31:31 +08:00
Zirui Wu 74c1e6da60 introducing pause and quit flags to DataBuffer
fix review cmts

fix ci

fix Ci

fixci

address ci

ci

- add timeout
- add more test cases

fix CI

address review cmts
2020-08-21 14:33:26 -04:00
mindspore-ci-bot 82c888f065 !4930 Fix CI cifar hang issue
Merge pull request !4930 from xiefangqi/fix_cifar_nofile_issue
2020-08-21 20:49:40 +08:00
xiefangqi e3e7820413 fix cifar stuck problem 2020-08-21 19:35:07 +08:00
mindspore-ci-bot 0d1a7ac654 !4909 [bug]support implicit type conversion for parameter
Merge pull request !4909 from vlne-v1/I1QM7L-implicit-type-conversion-parameter
2020-08-21 17:08:23 +08:00
BowenK e7c6b7e66a Add NewParameter and Imm patterns 2020-08-21 16:53:04 +08:00
mindspore-ci-bot c92f7c9170 !4911 add func type check for switch layer
Merge pull request !4911 from riemann_penn/add_func_type_check_for_switch_layer
2020-08-21 16:42:12 +08:00
panyifeng abab21ed57 add func type check for switch layer 2020-08-21 16:11:40 +08:00
mindspore-ci-bot 6e8d3a3b82 !4859 Add CTCGrerdyDecoder ops for old backend.
Merge pull request !4859 from liuxiao93/Add-ReversSqueuce-EditDistance-CTCGrerdyDecoder
2020-08-21 16:05:19 +08:00
heleiwang 4870abc848 1. fix generator_dataset hangs
2. fix test_graphdata_distributed.py failing randomly
2020-08-21 16:03:06 +08:00
Wei Luning 77dcdd89ec support parameter updata with implicit type conversion 2020-08-21 15:24:57 +08:00
panyifeng f9f3cd7ce0 fix switch_layer_issues 2020-08-21 14:17:51 +08:00
mindspore-ci-bot b2a5440548 !4883 Fix compilation warnings
Merge pull request !4883 from zhoufeng/xiu-ba-ge
2020-08-21 12:33:20 +08:00
mindspore-ci-bot bccb92adf7 !4868 fix: concat with none sample dataset
Merge pull request !4868 from guozhijian/fix_concat_with_zero_dataset
2020-08-21 12:02:30 +08:00
liuxiao93 9bc18ea2e5 Add CTCGrerdyDecoder ops for GE. 2020-08-21 10:12:01 +08:00
Su Teng e3ae23c939 add parallel attention test 2020-08-21 10:10:53 +08:00
mindspore-ci-bot 2ac410e90d !4789 Add EditDistance op for GE.
Merge pull request !4789 from liuxiao93/Add-EditDistance-op-for-GE
2020-08-21 09:49:38 +08:00
mindspore-ci-bot 3d06cbf987 !4801 Must set or change parallel mode before any Initializer created
Merge pull request !4801 from yihuaijie/dev
2020-08-21 09:47:32 +08:00
zhoufeng 22ec976ef7 fix compilation warnings
Signed-off-by: zhoufeng <zhoufeng54@huawei.com>
2020-08-21 09:44:04 +08:00
jonyguo 5b4b539751 fix: concat with none sample dataset 2020-08-21 09:19:50 +08:00
mindspore-ci-bot 9a3baf4f6c !4314 [MD] remove defaults used in binding code, and remove bilinear resize op
Merge pull request !4314 from nhussain/clean_up_bindings
2020-08-20 22:16:56 +08:00
mindspore-ci-bot bfa3cd900e !4843 remove ccsrc/common.h by explicit dependent header file
Merge pull request !4843 from xychow/remove-ccsrc-common-h
2020-08-20 22:02:13 +08:00
mindspore-ci-bot 98052f9d06 !4847 [MD] Add some data validation
Merge pull request !4847 from xiefangqi/xfq_add_numworker_validate
2020-08-20 20:22:51 +08:00
mindspore-ci-bot 2cb22a0f78 !4688 Fix bool operation parsing to be consistent with pynative
Merge pull request !4688 from amongo/FixBoolParse
2020-08-20 20:06:41 +08:00
liuxiao93 4c99f4f649 Add EditDistance op for GE. 2020-08-20 19:39:57 +08:00
Yi Huaijie 89a4ebf8a1 parallel mode must be set before create an initializer 2020-08-20 19:20:46 +08:00
zhousiyi d0e58dd765 remove ccsrc/common.h
replace frontend/operator/ops.h in backend with base/core_ops.h as
backend should not use any frontend-only primitive
2020-08-20 09:34:07 +00:00
xiefangqi 7dc7990d75 add NumWorkers validate 2020-08-20 17:25:30 +08:00
huangdongrun f30418991c refactor bool op parsing to be consistent with pynative mode
add testcase of st
2020-08-20 16:45:30 +08:00
mindspore-ci-bot 9ee144ea40 !4744 [AutoParallel]Support bert
Merge pull request !4744 from lichen/support_bert
2020-08-20 14:13:19 +08:00
mindspore-ci-bot 7b0a4e5929 !4796 Fix randomdata ut core dump problem
Merge pull request !4796 from xiefangqi/fix_randomdata_ut
2020-08-20 14:11:55 +08:00
xiefangqi c0d18d2517 fix randomdata ut problem 2020-08-20 11:13:02 +08:00
mindspore-ci-bot ed954dc407 !4782 Python UT fails on RandomPosterize
Merge pull request !4782 from islam_amin/bbox_class
2020-08-20 10:18:59 +08:00
mindspore-ci-bot 24700afa2d !4689 add check for user define bprop in Pynative mode
Merge pull request !4689 from zhangbuxue/add_check_for_user_define_bprop_in_Pynative_mode
2020-08-20 09:43:46 +08:00
islam_amin 85357238fe Fixing posterize python ut 2020-08-19 21:35:07 -04:00
Jesse Lee 8a08d0c37b Phase 2 of CacheOp 2020-08-19 13:45:55 -04:00
Zirui Wu 78c1aa1d96 Implemented Callback for Dataset
implment pause in MapOp, added more to callback

add ds_callback

- Initial drop of Python DSCallback

- Pybind DSCallback

- Pybind DSCallback

added callback to mapOp

- de_pipeline DSCallback

- de_pipeline DSCallback

add test case, segfault for now

fix seg fault

- de_pipeline DSCallback

remove 1 line

update callback test case, now works

use builder class for mapOp callback

- de_pipeline DSCallback

- de_pipeline DSCallback

- de_pipeline DSCallback

better test case

minor fix

add comments and minor clean ups

get rid of nullptr in MapOp, use other flag instead

fix a bug ParseMapOp only takes 1 callback

- Added WaitedDSCalabck

refactor callback param

fix text case incorrect number

- added testing

fix cpp test case

- added testing

- revert back lenet changes

- cleanup test_callbacks.py

- cleanup test_callbacks.py

fix CI stage I

fix CI stage II

fix CI and update epoch counter

- add validation
- add more testing  test_callbacks.py

use random data op to do tests

adjust when to call EpochBegin/End

- add repeat with callback

- addressing reviewers' comments

- docstring and CI fixes

- docstring and CI fixes

- docstring and CI fixes

- rebase with upstream/master

fix cpp test case

fix review comments

addr review cmts, add test case
2020-08-19 12:31:04 -04:00
YangLuo b50ae27c72 !4597 [Dataset] C++ API Support for build_vocab 2020-08-19 22:47:47 +08:00
nhussain eb9a611041 remove old defaults 2020-08-19 10:11:19 -04:00
buxue 855d6b8fed add check for user define bprop in Pynative mode. 2020-08-19 19:14:02 +08:00
lichenever 221a801395 auto parallel support bert 2020-08-19 17:47:59 +08:00
mindspore-ci-bot d541e261a0 !4696 C++ API Support for CSV Dataset
Merge pull request !4696 from jiangzhiwen/jzw/c_api_csv
2020-08-19 17:08:33 +08:00
mindspore-ci-bot 70a218123b !4722 modiy error type
Merge pull request !4722 from lijiaqi/modify_error_type
2020-08-19 16:18:50 +08:00
mindspore-ci-bot 2db4bda219 !4622 serving restful init
Merge pull request !4622 from hexia/http_init
2020-08-19 16:15:58 +08:00
mindspore-ci-bot 256dccc651 !4498 Gnn data processing supports distributed scenarios
Merge pull request !4498 from heleiwang/gnn_distributed
2020-08-19 16:09:05 +08:00
jiangzhiwen 39df25f803 c++ api for CSV Dataset 2020-08-19 16:02:11 +08:00
李嘉琪 c65ea1687b modify error type 2020-08-19 14:21:56 +08:00
mindspore-ci-bot 2d683234fb !4723 Support to concat more than 3 tensors in auto parallel mode
Merge pull request !4723 from yangzhenzhang/concat-more-than-3-tensors
2020-08-19 14:16:02 +08:00
yangzhenzhang cda08f6a52 concat 3 tensors in auto parallel mode 2020-08-19 11:27:35 +08:00
heleiwang 8ee4d8e92d Gnn data processing supports distributed scenarios 2020-08-19 11:09:01 +08:00
mindspore-ci-bot 6868b9b6fa !4422 Minddata new dataset api ==> RandomData
Merge pull request !4422 from xiefangqi/xfq_c++api_randomdata
2020-08-19 10:48:53 +08:00
mindspore-ci-bot 50b5c8035b !4701 fix concat get_dataset_size error
Merge pull request !4701 from guozhijian/fix_concat_get_dataset_size_err
2020-08-19 09:14:30 +08:00
xiefangqi b91e56375e add randomdataset and schema 2020-08-19 09:13:43 +08:00
nhussain 738ae2c78d change api to use std_vector
try to display image
2020-08-18 15:41:21 -04:00
islam_amin 979111d0ef Added Posterize Op 2020-08-18 13:55:47 -04:00
jonyguo b0b370336f fix: concat get_dataset_size error 2020-08-18 23:40:17 +08:00
jonyguo 5f294fb90d fix: random affine parameter check 2020-08-18 21:58:53 +08:00
panyifeng 22a9d02e9f switch_layer incorporate env_get and tuple_get 2020-08-18 20:23:23 +08:00
hexia 75d116b5db serving http init 2020-08-18 20:15:47 +08:00
mindspore-ci-bot 4f46c4277a !4614 Code compile performance optimization
Merge pull request !4614 from zhoufeng/code-compile-performance
2020-08-18 16:45:15 +08:00
mindspore-ci-bot cc077de9e3 !4425 fix if statement with no return coredump
Merge pull request !4425 from fary86/fix_if_no_return_coredump
2020-08-18 16:09:31 +08:00
mindspore-ci-bot 2ae6365d77 !4650 EmbeddingLookup support auto parallel
Merge pull request !4650 from yangzhenzhang/embedding-lookup-auto-parallel
2020-08-18 15:51:04 +08:00
zhoufeng 663278112f optimize code compile performance
Signed-off-by: zhoufeng <zhoufeng54@huawei.com>
2020-08-18 14:33:24 +08:00
mindspore-ci-bot 025e6f23f7 !4643 add test cases for parameter slice init using all kind of initializers
Merge pull request !4643 from yihuaijie/dev
2020-08-18 14:21:41 +08:00
mindspore-ci-bot 26de4540ef !4639 rename param_value to param_info
Merge pull request !4639 from vlne-v1/rename_param_value
2020-08-18 14:16:23 +08:00
fary86 38083e055a Fix coredump missing return statement after while loop 2020-08-18 14:16:21 +08:00
yangzhenzhang 6f6a8ae9f0 embedding lookup auto parallel 2020-08-18 11:40:09 +08:00
Wei Luning c1c30a44f1 rename param_value -> param_info 2020-08-18 11:16:28 +08:00
Yi Huaijie 0f7ead5f14 parameter slice init test all initializers 2020-08-18 10:34:15 +08:00
jiangzhiwen a88273bd3f c++ api support for Manifest Dataset 2020-08-18 09:48:55 +08:00
mindspore-ci-bot 113ff6ca24 !4367 CutMixBatch Augmentation Op
Merge pull request !4367 from MahdiRahmaniHanzaki/cutmix
2020-08-18 03:27:33 +08:00
mahdi 3ecc53fb4e Added CutMix 2020-08-17 13:01:47 -04:00
mindspore-ci-bot d1ad33676b !4598 support axis is None for all and any interface in graph mode
Merge pull request !4598 from zhangbuxue/support_axis_is_None_for_all_and_any_in_graph_mode
2020-08-17 19:52:40 +08:00
buxue 09f1a4bbaf support axis is None for all and any interface in graph mode. 2020-08-17 17:53:02 +08:00
jonyguo 8bb4449fa8 add testcase for padded dataset with decode op 2020-08-17 16:43:23 +08:00
mindspore-ci-bot f41ca6b5c6 !4558 soft_dvpp_decode_resize_jpeg_op issue resolved, input size can be an int, break the constraints of input size being sequence (height, width)
Merge pull request !4558 from magemomou/my/work
2020-08-17 16:40:43 +08:00
mayang c410c6a6e2 soft_dvpp_decode_resize_jpeg_op issue resolved, input size can be an int, break the constraints of input size being sequence (height, width) 2020-08-17 14:57:04 +08:00
mindspore-ci-bot dda4c26e7c !4423 add reshape type of tensor
Merge pull request !4423 from lianliguang/add-reshape-type-to-tensor
2020-08-17 11:34:11 +08:00
WilliamLian 6760d9976d add reshape type to tensor 2020-08-17 09:34:38 +08:00
mindspore-ci-bot f875bf21bc !2948 fix control flow
Merge pull request !2948 from amongo/FixControlFlow
2020-08-17 08:56:22 +08:00
huangdongrun 2a6d346d2f support if by if grad parameter
add join for ref

adjust env eliminate to eliminate all env ops

add partial app cache

resolve while endless

fix env eliminate

support for "for while" cases

fix join shape error
2020-08-15 09:47:10 +08:00
Cathy Wong 9c8af0d1cf C++ API: Provide validate param functions 2020-08-14 17:54:38 -04:00
mindspore-ci-bot 7d70fb4dc4 !3769 add RandomColorOp
Merge pull request !3769 from Alexey_Shevlyakov/ColorOp
2020-08-14 23:43:08 +08:00
mindspore-ci-bot 0e4d2c2535 !4188 fix switch layer sigle prim cell
Merge pull request !4188 from riemann_penn/fix_switch_layer_sigle_prim_cell
2020-08-14 19:54:37 +08:00
mindspore-ci-bot f41c21c5fa !4284 remove to_full_tensor and load_inputs in exexute stage
Merge pull request !4284 from yao_yf/remove_to_full_tensor_and_load_inputs_in_exexute_stage
2020-08-14 18:05:44 +08:00
mindspore-ci-bot 659ed37812 !4443 [bug]fix bugs in parameters and add ut cases
Merge pull request !4443 from vlne-v1/I1RDCY-slice_shape_update_initializer
2020-08-14 18:03:20 +08:00
mindspore-ci-bot ebdea65f70 !4402 [Dataset] Fix c_api bugs
Merge pull request !4402 from luoyang/c-api-pyfunc
2020-08-14 17:40:32 +08:00
panyifeng 1c296b96c9 fix switch layer sigle prim cell 2020-08-14 17:04:01 +08:00
Wei Luning d18ce95714 * add support to update by scalar* fix update error for by Initializer when slice_shape is* add ut for compute and update 2020-08-14 16:35:28 +08:00
yao_yf cbb4363fa7 remove to_full_tensor and load_inputs in exexute stage 2020-08-14 11:40:27 +08:00
mindspore-ci-bot 0f637fdc58 !4404 update field split
Merge pull request !4404 from yangzhenzhang/update-field-split-in-forward-communication
2020-08-14 10:16:24 +08:00
mindspore-ci-bot e9629f95e1 !4418 Add UniqueWithPad cpu kernel
Merge pull request !4418 from huanghui/unique-with-pad-cpu-kernel
2020-08-14 09:46:22 +08:00
mindspore-ci-bot 93f60269c3 !4369 Added distribution_specific_args checking to some functions and fixed minor issues in docs
Merge pull request !4369 from XunDeng/pp_poc_v3
2020-08-14 09:33:23 +08:00
yangzhenzhang 14c77c9f03 update field split 2020-08-14 09:00:56 +08:00
Alexey Shevlyakov 8526d5414d RandomColor 2020-08-13 16:10:21 -04:00
mindspore-ci-bot 2953720169 !4322 Adding RGBA color conversion
Merge pull request !4322 from EricZ/bitmap_support
2020-08-14 01:58:33 +08:00
mindspore-ci-bot 7a10e07fc6 !4383 Add distributedSampler to concatOP
Merge pull request !4383 from genglishuai/genglshOK
2020-08-14 00:06:06 +08:00
eric 6a41b44724 Added wrapper around color change function 2020-08-13 11:39:56 -04:00
genglishuai 4d4df68d1e add DistributedSampler for Concat op 2020-08-13 22:52:02 +08:00
luoyang 373adaa553 !4402 [Dataset] Fix c_api bugs 2020-08-13 22:38:55 +08:00
mindspore-ci-bot b4b6e5c8ed !4370 support kw and kwargs for cell in Pynative
Merge pull request !4370 from zhangbuxue/support_kw_and_kwargs_for_cell_in_pynative
2020-08-13 22:19:41 +08:00
mindspore-ci-bot ed1244558c !4366 Fixing param type in c++ api and fixing python validator for Repeat Op
Merge pull request !4366 from TinaMengtingZhang/cpp-api-repeat-count
2020-08-13 22:03:39 +08:00
huanghui 8f70af7300 add unique_with_pad cpu kernel 2020-08-13 21:55:00 +08:00
buxue 0be4114240 support kw and kwargs for cell in Pynative 2020-08-13 20:29:47 +08:00
mindspore-ci-bot 1b63c76c61 !4194 Add infer and dpn to nn/probability
Merge pull request !4194 from zhangxinfeng3/master
2020-08-13 17:32:50 +08:00
mindspore-ci-bot 2db0290c49 !4356 Add validation for field split
Merge pull request !4356 from yangzhenzhang/update-field-split
2020-08-13 16:43:44 +08:00
mindspore-ci-bot 58be11907c !4346 all dataset support get_dataset_size
Merge pull request !4346 from anzhengqi/I1PXKS-Dataset-support-getsize
2020-08-13 14:49:22 +08:00
mindspore-ci-bot 579e0ab925 !4354 modify python ut runtest.sh
Merge pull request !4354 from changzherui/mod_python_ut
2020-08-13 14:17:44 +08:00
zhangxinfeng3 b16b71b9e8 add infer and dpn 2020-08-13 11:46:07 +08:00
mindspore-ci-bot 1716beb037 !3942 embeddinglookup_wrap
Merge pull request !3942 from yao_yf/embeddinglookup_wrap
2020-08-13 11:21:22 +08:00
yangzhenzhang 4a0e6ff7fc update field split 2020-08-13 10:35:04 +08:00
Xun Deng c0a3b51bb8 fixed minor issues in docs, updated parameter checking in some functions in distributions 2020-08-12 22:05:58 -04:00
mindspore-ci-bot 03210ef166 !2557 Add UnsortedSegmentSum fission pass
Merge pull request !2557 from huanghui/unsorted-segment-sum-fission-pass
2020-08-13 08:57:01 +08:00
mindspore-ci-bot 265064733b !4340 move cc file that end with py to pybind
Merge pull request !4340 from liubuyu/master
2020-08-13 08:56:01 +08:00
tinazhang 0868720eb7 fix parameter type for repeat op in c++ api and added c++/python ut. 2020-08-12 17:23:42 -04:00
tony_liu2 deb1146474 reorganize headers and tests
remove headers

fix iterator file

fix pr comments

fix new file
2020-08-12 14:36:01 -04:00
huanghui b8d7f6d77f add UnsortedSegmentSum fission pass 2020-08-12 19:17:19 +08:00
changzherui d3f709be55 modify python ut runtest.sh 2020-08-12 19:06:06 +08:00
anzhengqi 3e31ac6d62 all Dataset support get_dataset_size 2020-08-12 15:35:47 +08:00
liubuyu 45b6600001 mv cc file that end with py to pybind 2020-08-12 14:50:18 +08:00
yao_yf e4de26d5bc embeddinglookup wrap 2020-08-12 14:24:13 +08:00
mindspore-ci-bot 5adba834d0 !3819 C++ API Support for CLUE Dataset
Merge pull request !3819 from jiangzhiwen/jzw/c_api_clue
2020-08-12 14:21:19 +08:00
mindspore-ci-bot 735fc98c78 !4265 support two mstypes do equal.
Merge pull request !4265 from zhangbuxue/support_two_mstypes_do_equal
2020-08-12 11:43:04 +08:00
mindspore-ci-bot 5b3f209e43 !4269 change export from geir to air
Merge pull request !4269 from fary86/change_export_interface
2020-08-12 10:25:00 +08:00
jiangzhiwen 5842a0212f c++ api support for CLUE Dataset 2020-08-12 09:55:43 +08:00
mindspore-ci-bot e509f87f7f !4219 Add Unique CPU kernel
Merge pull request !4219 from huanghui/add-op-unique
2020-08-12 09:39:14 +08:00
buxue 35677ac098 support two mstypes do equal. 2020-08-12 09:12:46 +08:00
mindspore-ci-bot 9276a9d2fb !3771 C++ API Support for TextFile Dataset
Merge pull request !3771 from cathwong/ckw_c_api_textfile_nosampler
2020-08-12 04:32:05 +08:00
avakh 477528de7f random sharpness cpp op support 2020-08-11 15:30:40 -04:00
mindspore-ci-bot 8ee68ae667 !4245 [MD] Add SolarizeOp
Merge pull request !4245 from nhussain/solarize
2020-08-12 02:27:42 +08:00
fary86 73325e0f01 Change export interface 2020-08-12 00:01:31 +08:00
nhussain 7766efd58e add files 2020-08-11 11:54:14 -04:00
Cathy Wong 7f39b5cfd7 C++ API Support for TextFile Dataset and Unit Tests 2020-08-11 11:53:53 -04:00
mindspore-ci-bot 4f75adb11a !4297 sync code incubator to master
Merge pull request !4297 from guozhijian/code_sync_incubator_f3c32baf_to_master_fcfc75a3_0811
2020-08-11 22:13:46 +08:00
huanghui fa081092c8 add Unique cpu kernel 2020-08-11 21:36:55 +08:00
mindspore-ci-bot c2997845c2 !4289 Fix pad op validate issue
Merge pull request !4289 from xiefangqi/fix_pad_validate
2020-08-11 20:20:35 +08:00
mindspore-ci-bot 67f5b0e916 !4258 change binary to mindir
Merge pull request !4258 from leopz/fix_doc
2020-08-11 20:17:41 +08:00
mindspore-ci-bot c6acafcbc6 !4268 fix glog level dont work
Merge pull request !4268 from liubuyu/master
2020-08-11 20:05:03 +08:00
jonyguo 4964f7703a Merge branch 'master' into code_sync_incubator_f3c32baf_to_master_fcfc75a3_0811 2020-08-11 19:40:04 +08:00
mindspore-ci-bot a2827eb657 !3874 add SoftDvppDecodeResizeJpegOp and SoftDvppDecodeRandomCropResizeJpeg op
Merge pull request !3874 from qianlong21st/soft_dvpp
2020-08-11 19:00:36 +08:00
mindspore-ci-bot 468e797e97 !4255 unify primitive
Merge pull request !4255 from lianliguang/unify-primitive
2020-08-11 18:41:21 +08:00
xiefangqi 7ee1e46455 fix pad op validate issue 2020-08-11 17:28:08 +08:00
liubuyu 25a67602b7 bug fix for glog level don't work 2020-08-11 16:44:50 +08:00
qianlong e9c4378ed8 add support of soft dvpp op 2020-08-11 15:46:07 +08:00
leopz eea597c92a change binary to mindir 2020-08-11 15:18:40 +08:00
WilliamLian 1f5441d73a remove inferfunction to core 2020-08-11 14:22:29 +08:00
mindspore-ci-bot b120cbcfb4 !4249 Add parameter checking in distributions
Merge pull request !4249 from XunDeng/pp_poc_v3
2020-08-11 14:16:29 +08:00
mindspore-ci-bot 6772bbde8d !4021 Add ScalarAffine and Softplus bijector
Merge pull request !4021 from XunDeng/scalar_affine_softplus
2020-08-11 14:14:35 +08:00
mindspore-ci-bot 0115876363 !4197 Add AdamApplyOneAssign fusion pass
Merge pull request !4197 from YuJianfeng/adam_assign
2020-08-11 11:26:00 +08:00
mindspore-ci-bot 266df56570 !4220 [MD] add num_samples support in minddataset pk sampler
Merge pull request !4220 from liyong126/minddataset_pk_num_samples
2020-08-11 11:22:28 +08:00
mindspore-ci-bot f21ee64fd0 !4247 Add ChannelSwap operation
Merge pull request !4247 from 章一智/channel_swap_op
2020-08-11 09:59:55 +08:00
mindspore-ci-bot 2cab58fdbc !3850 Implementing RandomAffine C Op and a CPP UT for AutoContrast & Equalize
Merge pull request !3850 from islam_amin/randomaffine_op
2020-08-11 09:29:09 +08:00
mindspore-ci-bot 0bb39830ae !4250 fix cpp api iterator build function
Merge pull request !4250 from anthonyaje/fix_cpp_api_iterator
2020-08-11 07:21:27 +08:00