yystopf
/
mindspore
forked from mindspore-Ecosystem/mindspore
0
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity
45,876 Commits 23 Branches 0 Tags 3.9 GiB
Commit Graph

4041 Commits

Author SHA1 Message Date
i-robot 12d8906e2e
!30253 Support more ops for dump flag 
Merge pull request !30253 from huanghui/enhance-dump-flag
 2022-03-15 16:49:11 +00:00
hesham 3d16a4dee2 Fix bug in terminating a failed pipeline 2022-03-15 11:35:28 -04:00
liuluobin f13d342986 Implementation of element wise parallel ops 2022-03-15 19:24:17 +08:00
i-robot 313a247ae8
!31178 upgrade Ascend package 11 Mar 22 
Merge pull request !31178 from shenwei41/upgrade_ascend_20220311
 2022-03-15 11:08:32 +00:00
i-robot 3be552ca39
!31156 test centos ut 
Merge pull request !31156 from luoyang/test_centos
 2022-03-15 09:38:15 +00:00
shenwei41 f1fde6a262 upgrade Ascend package 11 Mar 22 2022-03-15 16:31:27 +08:00
huangxinjing 31f55b6525 1. The main gol: Fix mixing inserting the AllReduce when where is no mirror appeared 
2. remove pattern match error as the origin pattern match will find no operator if there is only one parameter
 2022-03-15 15:54:06 +08:00
i-robot 01078763df
!31227 don't handle reshape when using adafactor optimizer in auto parallel 
Merge pull request !31227 from yangzhenzhang/adafactor-parallel-skip-handle-reshape
 2022-03-15 02:47:31 +00:00
huanghui 4482acf586 Support API set_dump for more type of ops 2022-03-15 10:24:15 +08:00
i-robot d4fc47ef0c
!31211 Clean iterators before forking 
Merge pull request !31211 from h.farahat/pytest_problem
 2022-03-15 01:47:28 +00:00
yangzhenzhang 1f98ffb79c adafactor parallel skip handle reshape 2022-03-15 09:18:19 +08:00
i-robot 6be7c42052
!31191 [MD] more tests for failover reset 
Merge pull request !31191 from Mohammad Motallebi/failover_more_tests
 2022-03-14 21:50:37 +00:00
h.farahat 318e6b8803 fix segfault 2022-03-14 14:55:21 -04:00
mohammad 583eb32723 [MD] Add more UT and a ST for failover reset 2022-03-14 13:52:09 -04:00
luoyang 74d8969b91 ut failed with pillow>=9.0.0, need more validation 2022-03-14 23:58:09 +08:00
i-robot dcb5cd670c
!30953 Dynamic Weight Decay 
Merge pull request !30953 from wanyiming/dynamic_wd
 2022-03-14 06:03:24 +00:00
lilei 690c58ebcf modify virtualdataset bug for master 2022-03-13 10:49:15 +08:00
i-robot 8b7d7a1e1a
!31141 [MD][Offload] Offload RandomSharpness Op Pure Sharpness Support & UT 
Merge pull request !31141 from alashkari/fix-random-sharpness
 2022-03-11 21:06:13 +00:00
Rescue face5e51a0 Added sharpness without randomness support and sharpnes op ut. 
fix pad issue
 2022-03-11 17:30:49 +00:00
i-robot 603ff25afb
!31177 Add ut for fallback python buildin function with tensor input 
Merge pull request !31177 from LiangZhibo/syntax
 2022-03-11 08:38:53 +00:00
i-robot 3c0e34ada0
!30855 Support user-defined classes by ms_class decorators 
Merge pull request !30855 from huangbingjian/ms_class_dev
 2022-03-11 08:37:12 +00:00
liangzhibo a3f183bdad Add test case for fallback buildin function with constant tensor 2022-03-11 15:35:29 +08:00
i-robot c2a5cc1486
!31040 Produce parallel operators for Argmin/max, SquareSumAll and UnsortedSegmentProd 
Merge pull request !31040 from Bert0108/reduce_operators_arg
 2022-03-11 06:15:56 +00:00
shenyu 443da2280d [FIX][BUG][MD][FUNC] RandomInvert 2022-03-11 11:53:04 +08:00
shenyu c14a8084c7 [FIX][BUG][MD][FUNC] RandomAuoContrast 2022-03-11 10:29:49 +08:00
i-robot a064d0855b
!30728 add ci test for unify backend 
Merge pull request !30728 from xiaoyao/master
 2022-03-11 01:50:03 +00:00
i-robot eecf6db426
!31121 [MD] revert OBSMindDataset 
Merge pull request !31121 from liyong126/revert_OBSMindDataset
 2022-03-11 01:21:10 +00:00
i-robot 216e7c6a92
!31041 add check for conv2d 
Merge pull request !31041 from yangzhenzhang/add-check-for-conv2d
 2022-03-11 01:20:52 +00:00
i-robot 67d10ce3be
!31066 Fix a bug where ROIAlign and CropAndResize distributed op do not support GPU 
Merge pull request !31066 from liuluobin/fix_roialign
 2022-03-11 01:16:46 +00:00
huangbingjian 4f7c9cb27f Support user-defined classes through ms_class decorators. 2022-03-10 20:12:54 +08:00
i-robot 99910ef27f
!31088 convert C++ data to python when run fallback function 
Merge pull request !31088 from lianliguang/add-fallback-builtin-function
 2022-03-10 11:56:01 +00:00
yangzhenzhang c00d29f223 rebase 2022-03-10 19:17:06 +08:00
liyong 50fc6f5736 Revert "!30810 [MD] add OBSMindDataset " 
This reverts commit 9fcab9184e, reversing
changes made to afb33fff62.
 2022-03-10 19:10:11 +08:00
liuluobin 8f045d02e3 Fix a bug where ROIAlign and CropAndResize distributed op do not support GPU 2022-03-10 17:31:36 +08:00
lianliguang d9390afad1 convert MS C++ dtype to MS python 2022-03-10 16:09:34 +08:00
xiao_yao1994 94df0869ad uni_backend: add ut ge test 2022-03-10 16:08:36 +08:00
Bert0108 bfc5e4345c add distributed operators for argmax/min sqauresumall and unsortedsetmentprod 2022-03-10 14:44:37 +08:00
liyong 10ada68fa9 add OBSMindDataset 2022-03-10 11:18:59 +08:00
i-robot 059ac45ae8
!30920 Call Draw function directly 
Merge pull request !30920 from huanghui/fix-draw
 2022-03-10 03:15:56 +00:00
wanyiming a124ec4de7 add dynamic_decay 2022-03-10 11:02:27 +08:00
i-robot b32daf3cb4
!31000 add fallback example of function 
Merge pull request !31000 from lianliguang/add-fallback-builtin-function
 2022-03-10 01:57:37 +00:00
i-robot 698b99aaa3
!18945 [assistant][ops]New operator implementation, include OmniglotDataset 
Merge pull request !18945 from Wangsong95/omniglot_dataset
 2022-03-09 06:51:15 +00:00
lianliguang f3b1e3291c add jit fallback example 2022-03-09 11:25:03 +08:00
qkeys ce7597f500 [feat][assistant][I3J6VB] add new data operator OmniglotDataset 2022-03-08 21:36:22 +08:00
i-robot 0b207fce31
!29735 unique support multi_dim tensor 
Merge pull request !29735 from jiangzhenguang/unique_multi_tensor
 2022-03-08 08:30:57 +00:00
i-robot f73987dcf2
!30949 Change the Parameter type name from 'ParameterTensor' to 'Parameter' 
Merge pull request !30949 from 张清华/opt
 2022-03-08 06:17:31 +00:00
jiangzhenguang 28f89f36b4 unique support multi-dim tensor 2022-03-08 14:09:19 +08:00
Zhang Qinghua 973008ebcf Change the Parameter type name from 'ParameterTensor' to 'Parameter' 2022-03-08 11:23:56 +08:00
i-robot 82bc2b722d
!30417 add history and lambda callbacks 
Merge pull request !30417 from liutongtong9/add_callbacks
 2022-03-08 02:17:46 +00:00
i-robot 75e1939f50
!30899 Enable python buildin function type for graph fallback and add some ut for python buildin function for graph fallback 
Merge pull request !30899 from LiangZhibo/syntax
 2022-03-08 01:35:47 +00:00
i-robot 2655d64720
!30865 [MD] Modify Batch number of workers with Python multiprocessing 
Merge pull request !30865 from cathwong/ckw_batch_workers_pymultip
 2022-03-07 18:21:16 +00:00
Cathy Wong bc29dd9d72 [MD] Modify Batch number of workers with Python multiprocessing 2022-03-07 10:39:05 -05:00
i-robot 335ef1c270
!30459 Add ut validate function for parallel 
Merge pull request !30459 from liuluobin/ut_master
 2022-03-07 15:23:14 +00:00
i-robot 4a9a65d2a6
!30919 Produce parallel operators for ReduceAll and ReduceProd 
Merge pull request !30919 from Bert0108/reduce_operators_merge
 2022-03-07 11:30:37 +00:00
liuluobin b797a410cc Add validate function for parallel ut 2022-03-07 16:19:52 +08:00
huanghui ce0f500a3f convert draw_ to call Draw function derectly 2022-03-07 16:12:52 +08:00
Bert0108 dfc92f1791 add distributed parallel operators for reduceall and reduceprod 2022-03-07 15:40:11 +08:00
liangzhibo 1a81031d92 Add testcases for fallback python buildin function 2022-03-07 14:20:24 +08:00
lianliguang 55e7f2d761 modify DynamicShape to TensorShape 2022-03-07 14:19:12 +08:00
i-robot a2303a92bd
!30741 [dataset][dfx] 易用性-Python多进程统一整改专项 
Merge pull request !30741 from xiefangqi/md_add_multiprocessing_optimization
 2022-03-07 01:29:21 +00:00
i-robot c4cc0e6f26
!30742 auto parallel support not ony power of 2 device numbers 
Merge pull request !30742 from yao_yf/auto_parallel_support_not_only_2_power
 2022-03-07 01:09:01 +00:00
yao_yf b60e54e0d5 support not only power of 2 2022-03-06 18:08:46 +08:00
xiefangqi 2de6c8908b add a clean up subprocess to clean subprocess when main process exit 2022-03-05 17:24:11 +08:00
liutongtong 7afcdfd211 add history and lambda callbacks 2022-03-04 17:24:57 +08:00
i-robot b90cf43562
!30553 Support dataset reset() to recover after failure 
Merge pull request !30553 from h.farahat/reset
 2022-03-04 02:19:45 +00:00
i-robot f01d841113
!27943 [MD][Autotune] Save/Load Autotune config 
Merge pull request !27943 from harshvardhangupta/save_load_at_config
 2022-03-03 20:42:23 +00:00
i-robot 7b26a32e98
!22608 [assistant][ops]New operator implementation, include KITTIDataset 
Merge pull request !22608 from Wangsong95/kitti_dataset
 2022-03-03 07:05:40 +00:00
h.farahat a3dec34833 Dataset failover reset 2022-03-02 23:53:56 -05:00
harshvardhangupta bd61adbb72 Implement save/load of autotune dataset pipeline configuration #27943 2022-03-02 15:08:08 -05:00
i-robot 70e61e9014
!30781 [MD] test_pyfunc_multiprocess_autotune.py - enable AutoTune 
Merge pull request !30781 from cathwong/ckw_ut_fixup_map_python_multiproc
 2022-03-02 19:50:29 +00:00
i-robot edcc6b790d
!30714 [MD][Offload] Add TypeCast op to offload 
Merge pull request !30714 from markuskunej/offload_typecast
 2022-03-02 19:48:58 +00:00
Cathy Wong eb931c7f88 [MD] test_pyfunc_multiprocess_autotune.py - enable AutoTune 2022-03-02 13:25:35 -05:00
i-robot 4aa82cc21e
!30679 [MD] Add tests for Python Multiprocessing with AutoTune 
Merge pull request !30679 from cathwong/ckw_ut_map_python_multiproc
 2022-03-02 14:26:09 +00:00
zx 93617ce91e [feat][assistant][I3J6VO] add new data operator KITTI 2022-03-02 19:29:17 +08:00
Cathy Wong b151f90b8d [MD] Add tests for Python Multiprocessing with AutoTune 2022-03-01 20:20:59 -05:00
markuskunej 6de37045ff Added TypeCast op in dataset offload. 2022-03-01 21:55:27 +00:00
wangjun 46612fabfb add st for shard 2022-03-01 15:32:37 +08:00
i-robot ad9757ccf0
!30661 [Auto parallel] [MoE] Fix an error of configuring MoE parallel 
Merge pull request !30661 from Xiaoda/124-moe-changes
 2022-03-01 02:26:37 +00:00
i-robot 789c1d6bd3
!30614 [AutoParallel] Fix Overflow As the cast is inserted before Mirror 
Merge pull request !30614 from huangxinjing/fix_cast_error
 2022-02-28 09:04:46 +00:00
Xiaoda Zhang 81e5abe580 fix an error of configuring parallel 2022-02-28 17:03:53 +08:00
i-robot a92c54b206
!30496 [Fallback] Suppport scipy module. 
Merge pull request !30496 from huangbingjian/support_scipy
 2022-02-28 08:33:34 +00:00
i-robot a8686ae3d9
!18827 [assistant][ops]New operator implementation, include LFWDataset 
Merge pull request !18827 from Wangsong95/lfw_dataset
 2022-02-28 08:31:11 +00:00
zx 2132f62d98 [feat][assistant][I3J6VQ] add new data operator LFW 2022-02-28 13:43:05 +08:00
huangxinjing 896daee845 [AutoParallel]Fix insert error for the mirror 2022-02-26 17:54:42 +08:00
yangzhenzhang 43e6e16da3 check platform for resizebilinear 2022-02-26 09:11:32 +08:00
i-robot 5deccfe64b
!30260 [MD][AutoTune] Re-enable AT for non-sink models 
Merge pull request !30260 from danishfarid/re-enable_AT
 2022-02-25 22:26:14 +00:00
i-robot 0341d96dd6
!30469 add shard function to support part of the graph executed in auto_parallel under pynative mode 
Merge pull request !30469 from wangjun/0223_pp
 2022-02-25 06:52:24 +00:00
huangbingjian a69d13bc44 [Fallback] Suppport scipy module. 2022-02-25 10:13:43 +08:00
i-robot cfe0f76d2b
!30491 ut for allgather fusion 
Merge pull request !30491 from jiahongQian/master
 2022-02-25 01:30:24 +00:00
danishfarid 6c4697fc8b reenable AT for non-sink 
tests re-enable

code check fix 1

remove self

lint fixing

lint fix 2

remove test as per req
 2022-02-25 00:02:58 +00:00
wangjun 24d448239c add pynative_parallel 2022-02-24 21:07:51 +08:00
i-robot 981eae461a
!30118 自动优化器并行特性 
Merge pull request !30118 from zhuyuxiao/I4S85V
 2022-02-24 11:09:44 +00:00
jiahongQian 25f57505bf ut for allgather fusion 2022-02-24 17:40:32 +08:00
i-robot bbcfbce9e0
!29997 [Auto parallel] [MoE] Support data_parallel + expert_parallel 
Merge pull request !29997 from Xiaoda/124-moe-changes
 2022-02-24 09:23:47 +00:00
zhuyuxiao d0e0e305d3 good 2022-02-24 16:10:18 +08:00
i-robot f2130e7434
!30483 [AutoParallel]Pipeline Automatic detection Opt 
Merge pull request !30483 from lichen/pipeline_opt_detection
 2022-02-24 03:40:15 +00:00
yao_yf e21f878e14 adasum ut fix 2022-02-24 09:31:04 +08:00
Xiaoda Zhang b714451937 implementing expert_parallel+data_parallel in MoE: 
1) extending _Linear's input as 4-dimension tensor: [outer_batch, expert_dim, -1, hidden], and _Liner's BatchMatMul becomes BatchMatMul(4_dim_tensor, 3_dim_tensor);
2) configuring the _Linear's BatchMatMul sharding strategy as [[dp, ep, 1, 1], [ep, 1, mp]];
3) introducing a new parameter 'expert_parallel' in TransformerOpParallelConfig, creating a new class MoEParallelConfig to include 'data_parallel', 'model_parallel' and 'expert_parallel';
4) changing parallel config for FeedForward, TransformerEncoderLayer, TransformerDecoderLayer.
 2022-02-24 09:24:27 +08:00
wangshengnan12@huawei.com acbefd80ea pipeline_opt_detection 2022-02-24 09:23:17 +08:00
i-robot 2a00ffd3b1
!30259 [MD]Update set_autotune_enable API to add save filepath 
Merge pull request !30259 from cathwong/ckw_at_save_api
 2022-02-24 01:00:59 +00:00
Cathy Wong 46e223e569 [MD] Update set_autotune_enable API to add save filepath 2022-02-23 14:50:55 -05:00
i-robot 81260a2319
!30466 takedown test_auto_parallel_adasum.py to ensure stability, again 
Merge pull request !30466 from yanghaoran/master
 2022-02-23 09:15:58 +00:00
i-robot 14393503b7
!30431 allreduce allgather fusion 
Merge pull request !30431 from jiahongQian/master
 2022-02-23 08:52:40 +00:00
yanghaoran 71d6b7d506 takedown test_auto_parallel_adasum.py to ensure stability, again 2022-02-23 16:42:13 +08:00
i-robot 2e8eac8341
!30367 auto_parallel_adasum_support_data_parallel 
Merge pull request !30367 from yao_yf/auto_parallel_adasum_support_data_parallel
 2022-02-23 06:50:26 +00:00
jiahongQian 8a2151d8bb allgather reducescatter fusion 2022-02-23 10:47:30 +08:00
i-robot eeb731ae3e
!18738 [assistant][ops]New operator implementation, include LSUNDataset 
Merge pull request !18738 from Wangsong95/lsun_dataset
 2022-02-23 02:28:40 +00:00
i-robot 5bee7156b9
!30369 add_virtualdataset_ut 
Merge pull request !30369 from lilei/add_virtualdataset_ut
 2022-02-23 02:21:34 +00:00
i-robot 0f24b679ec
!29819 Add GlobalNorm Search 
Merge pull request !29819 from huangxinjing/add_global_norm_search
 2022-02-23 01:15:08 +00:00
i-robot abb89d3f06
!29705 [MD][Offload] Support Multi-Column Datasets and Map Column Names to Index 
Merge pull request !29705 from markuskunej/offload_multi_col_ds
 2022-02-22 22:48:38 +00:00
i-robot 6edc6fccee
!30189 Optimize error message while outmost network input is wrong 
Merge pull request !30189 from zhangzhaoju/master_outmost
 2022-02-22 13:05:23 +00:00
yao_yf 19236b1a70 auto parallel adasum support data parallel and hybrid parallel 2022-02-22 20:14:00 +08:00
huangxinjing 092ba035e3 Add global norm parallel support 2022-02-22 16:08:53 +08:00
liyong 3e1c668247 fix mnist dataset 2022-02-22 15:25:13 +08:00
yanghaoran bfe139b662 takedonw test_auto_parallel_adasum.py to ensure gate stability 2022-02-22 14:51:03 +08:00
lilei bc62e24d94 add_virtualdataset_ut 2022-02-22 11:38:10 +08:00
i-robot ce5e93593d
!30301 modify error log 
Merge pull request !30301 from lianliguang/change-error-log
 2022-02-22 02:29:48 +00:00
i-robot 94c8c6355c
!30294 auto_parallel_adasum_checks_and_ut. 
Merge pull request !30294 from yao_yf/auto_parallel_adasum_checks_and_ut
 2022-02-22 01:10:24 +00:00
i-robot 926fb6f5ae
!25559 [feat] [assistant] [I48O7X, I48O6R] Add Asin, AsinGrad 
Merge pull request !25559 from 桂宁馨/Asin
 2022-02-21 14:00:58 +00:00
zhangzhaoju e000889265 Fix issue I4KVTZ 
The description of "outmost network" in error message instead by
method/function name
 2022-02-21 19:52:53 +08:00
i-robot 95f5075371
!30190 [AUTO_PARALLEL]Fix CI error 
Merge pull request !30190 from huangxinjing/fix_ci_error
 2022-02-21 11:50:28 +00:00
lianliguang efabace125 change error log 2022-02-21 17:48:16 +08:00
i-robot 908beb6b2f
!26420 [feat] [assistant] [I48O90, I48O4P] Add AsinhGrad,Asinh 
Merge pull request !26420 from 桂宁馨/Asinh
 2022-02-21 09:33:18 +00:00
i-robot f6215c614c
!30064 modify ckpt type error 
Merge pull request !30064 from changzherui/mod_ckpt_type_err
 2022-02-21 08:59:00 +00:00
huangxinjing 5e325ac336 [AUTO_PARALLEL]Fix insert nodes error 2022-02-21 16:39:50 +08:00
i-robot a5242d33e1
!30181 support TensorShape ops 
Merge pull request !30181 from lianliguang/change-dynshape-to-tensorshape
 2022-02-21 07:36:40 +00:00
i-robot 7386612515
!29820 moe_topk routing 
Merge pull request !29820 from wangshengnan123/moe_topk_routing
 2022-02-21 06:21:55 +00:00
yao_yf 4b79d4c425 auto parallel adasum uts and checks 2022-02-21 10:41:09 +08:00
lianliguang 5c8af66e53 support TensorShape ops to instead of DynamicShape ops 2022-02-21 09:33:32 +08:00
markuskunej 87a45b239b Add offload support for multi-column datasets and map offload operations 
to specific columns.
 2022-02-18 20:09:49 +00:00
zhixinaa 4ce6104ff9 [feat][assistant][I3J6V9] add new data operator LSUN 2022-02-18 15:40:32 +08:00
桂宁馨 caf54a52f0 [feat] [assistant] [I48O90, I48O4P] Add Asinh, AsinhGrad 2022-02-18 15:08:55 +08:00
桂宁馨 911c6eb144 [feat] [assistant] [I48O7X, I48O6R] Add Asin, AsinGrad 2022-02-18 14:11:41 +08:00
wangshengnan123 7322426648 top_k routing 2022-02-18 14:10:22 +08:00
changzherui 05dca07cf6 modify ckpt type error 2022-02-18 00:42:55 +08:00
i-robot 48d4f34576
!30167 Add UT case for 'Convolution+Transformer' structure 
Merge pull request !30167 from Bert0108/ut_conformer
 2022-02-17 07:43:58 +00:00
i-robot 7c32002e0b
!29930 Support List Slice 
Merge pull request !29930 from lianliguang/support-list-slice
 2022-02-17 07:24:40 +00:00
Bert0108 25a9c73a08 add ut case for monitoring the conformer structure 2022-02-17 15:08:01 +08:00
i-robot 465f21a3ea
!30133 Add backend check for RandomChoiceWithMask 
Merge pull request !30133 from liuluobin/rnd_fix_master
 2022-02-17 02:11:54 +00:00
lianliguang 49608835c1 support list slice 2022-02-17 09:47:26 +08:00
i-robot c4c5a1c6da
!29862 MD Autotune supporting MindRecordOp 
Merge pull request !29862 from Mohammad Motallebi/autotune_addworker_mindrecord
 2022-02-16 23:34:11 +00:00
mohammad ddd3b07648 MD autotune supports mindrecord 2022-02-16 14:00:31 -05:00
liuluobin b0b79ef8fb Add backend check for RandomChoiceWithMask 2022-02-16 18:56:57 +08:00
i-robot 1ceecd9f58
!24412 [assistant][ops] Add LogicalXor 
Merge pull request !24412 from 李定维/LogicalXor
 2022-02-16 06:21:39 +00:00
i-robot 7d356fbee8
!30088 [MD] Disable shared memory in Python Multiprocessing UTs for CI 
Merge pull request !30088 from cathwong/ckw_ut_fixup_python_multiproc
 2022-02-15 21:36:34 +00:00
Cathy Wong 6c5dbf937f [MD] Disable shared memory in Python Multiprocessing UTs for CI 2022-02-15 11:00:44 -05:00
i-robot d5100190c9
!29612 Impementatin of CropAndResizeInfo and ROIAlignInfo. 
Merge pull request !29612 from liuluobin/crop_resize_master
 2022-02-15 11:36:06 +00:00
lvxinyu 28a08b3cf6 LogicalXor 2022-02-15 13:31:49 +08:00
i-robot c45e8b5c9b
!29797 [MD]Enhance Python Multiprocessing test coverage 
Merge pull request !29797 from cathwong/ckw_ut_python_multiproc
 2022-02-14 18:41:22 +00:00
liuluobin 80fd640819 Implementation of ROIAlignInfo and CropAndResizeInfo 2022-02-14 10:46:09 +08:00
Margaret_wangrui 462c38813d [ME] Add parameter name check. 2022-02-14 10:07:43 +08:00
Cathy Wong befe8f4837 [MD]Enhance Python Multiprocessing test coverage 2022-02-11 12:31:21 -05:00
i-robot 81bcff32ad
!29825 support pad mode for conv2d_transpose 
Merge pull request !29825 from yangzhenzhang/support-pad-mode-for-conv2d-transpose
 2022-02-11 01:34:30 +00:00
i-robot 5858a06689
!29700 [MD] Tidy up UT and device queue info messages 
Merge pull request !29700 from cathwong/ckw_device_queue_probes
 2022-02-09 12:47:27 +00:00
yangzhenzhang 4061c11e18 support pad mode for conv2d transpose 2022-02-09 11:52:47 +08:00
Cathy Wong fb469f1ad2 [MD] Tidy up UT and device queue info messages 2022-02-08 17:51:11 -05:00
许志杰 ab9a32ad0d [feat] [assistant] [#I40GYE] add new loader SQuADDataset 2022-02-08 03:00:09 -08:00
i-robot f17b96ac68
!29719 fix vocab validation 
Merge pull request !29719 from luoyang/fix_vocab
 2022-02-08 01:10:56 +00:00
luoyang c248274d91 fix vocab validation 2022-02-07 15:55:59 +08:00
i-robot a55b1b5e05
!29453 support pad mode for conv2d parallel operator 
Merge pull request !29453 from yangzhenzhang/support-pad-mode-for-conv2d
 2022-02-07 06:58:00 +00:00
i-robot e1c2c4c268
!29699 Fix duplicate wp hit bug in offline debugger 
Merge pull request !29699 from TinaMengtingZhang/bugfix_dbg
 2022-02-01 21:43:24 +00:00
i-robot a1b5f06407
!29535 [MD] Profiler Memory Tooling Support 
Merge pull request !29535 from cathwong/ckw_memory_tool
 2022-02-01 21:39:05 +00:00
TinaMengtingZhang 23eb886d11 bugfix: duplicate wp hit results in offline dbg 2022-02-01 11:58:42 -05:00
Cathy Wong 3fe9a05341 [MD] Profiler Memory Tool Support 2022-01-31 17:46:17 -05:00
i-robot e12d6af257
!29358 MD Child Sampler Code and UT Issues 
Merge pull request !29358 from zetongzhao/sampler_AddChild
 2022-01-31 16:55:22 +00:00
zetongzhao ea9783e18c fix sampler add_child 2022-01-31 07:45:42 -05:00
i-robot 48f4fd2f52
!29639 build core.so 
Merge pull request !29639 from zhoufeng/dynamic-core
 2022-01-30 02:00:21 +00:00
i-robot 02d63d7ad9
!23008 [assistant][ops][FilterWikipediaXML] 
Merge pull request !23008 from Isaac/FilterWikipediaXML
 2022-01-29 08:35:29 +00:00
i-robot ea391f6eb1
!29505 Fix AllGather Cast when the parameters are shared 
Merge pull request !29505 from huangxinjing/fx_allgather_cast
 2022-01-29 07:46:10 +00:00
i-robot f5392f2460
!22443 [assistant][ops] Add new data operator MaskAlongAxis 
Merge pull request !22443 from TR-nbu/MaskAlongAxis
 2022-01-29 07:19:20 +00:00
zhoufeng 8e97ce5cd2 build core.so 
Signed-off-by: zhoufeng <zhoufeng54@huawei.com>
 2022-01-29 14:11:02 +08:00
tophand 797cd52c96 [feat][assistant][I40GXK]add new data operator Filter_Wikipedia_XML 2022-01-28 02:26:20 -08:00
QRICK 52c73ff1b3 [assistant][ops][I3T96S] Add new data operator MaskAlongAxis 2022-01-28 14:53:38 +08:00
i-robot 47a4860043
!27184 [assistant][ops]Add Cummax 
Merge pull request !27184 from 陈钧/cummax
 2022-01-28 06:40:25 +00:00
i-robot 92c59c3401
!25289 [assistant][ops] Add Coalesce 
Merge pull request !25289 from 李定维/Coalesce
 2022-01-27 12:31:30 +00:00
i-robot a24709b46a
!22446 [assistant][ops] Add new data operator MaskAlongAxisIID 
Merge pull request !22446 from TR-nbu/MaskAlongAxisIID
 2022-01-27 08:45:24 +00:00
王泰格 cc8d3e6279 [assistant][ops][I40FKE] add operator Coalesce 2022-01-27 16:08:31 +08:00
QRICK 9ed73eae94 [assistant][ops][I3T96P] Add new data operator MaskAlongAxisIID 2022-01-27 12:20:26 +08:00
chen-jun0211 5343a83f79 Add Cummax 2022-01-27 10:53:51 +08:00
i-robot 4861e9abab
!24098 [assistant][ops] Add UpperBound 
Merge pull request !24098 from 王刚克/UpperBound
 2022-01-26 10:46:26 +00:00
i-robot 2b82ba1e49
!26244 [assistant][ops] Add new data PhaseVocoder 
Merge pull request !26244 from TR-nbu/PhaseVocoder
 2022-01-26 06:46:36 +00:00
i-robot b0b3d7ede7
!29421 Implementation of BoundingBoxEncodeInfo, IOUInfo and RandomChoickWithMaskInfo 
Merge pull request !29421 from liuluobin/parallel_ops_master
 2022-01-26 06:40:32 +00:00
i-robot 6fac9c5d1d
!28505 Change SparseTensor to COOTensor 
Merge pull request !28505 from wangrao124/COOTensor
 2022-01-26 06:22:28 +00:00
i-robot 55ba926a04
!29087 optimize sharding propagation&add ut on pangu 
Merge pull request !29087 from bichaoyang/master
 2022-01-25 11:59:25 +00:00
huangxinjing 34101d748d Add cast ut and fix juge condition 2022-01-25 18:14:11 +08:00
yangzhenzhang 6df5e9aa5d support pad mode for conv2d 2022-01-25 17:26:47 +08:00
liuluobin fb79c508f2 Implementation of BoundingBoxEncodeInfo, IOUInfo, RandomChoiceWithMaskInfo. 2022-01-25 15:49:48 +08:00
wang-gangke e912bc8a24 UpperBound 2022-01-25 15:45:33 +08:00
wangrao124 9dfc703f7a change sparsetensor to cootensor 2022-01-25 15:39:37 +08:00
b00518648 2ff3425c76 1.consider comm cost inside a op when select a strategy;2.deal with the ops that share same param;3.add ut to fix the perfermence at pangu_alpha 2022-01-25 15:05:37 +08:00
i-robot bff5afe2bb
!25078 [assistant][ops] Add Cross 
Merge pull request !25078 from 冯钰/cross
 2022-01-25 02:16:41 +00:00
donrichnx eee07047a0 [feat] [assistant] [I40GYJ] add Captioning task of CocoDataset 2022-01-24 17:24:16 +08:00
minatofy 7832d77718 cross 2022-01-24 17:09:36 +08:00
QRICK 859f90cb85 [assistant][ops][I3T96R] Add new data operator PhaseVocoder 2022-01-24 16:00:24 +08:00
i-robot 8284a5590d
!29427 clean code 
Merge pull request !29427 from huangbingjian/clean_code
 2022-01-24 01:35:29 +00:00
i-robot 85ac17a8c6
!22736 [assistant][ops] Add new dataset operator LibriTTSDataset 
Merge pull request !22736 from TR-nbu/LibriTTSDataset
 2022-01-22 10:03:40 +00:00
i-robot ee8fba498a
!29376 support config allreduce group size for batch norm 
Merge pull request !29376 from yangzhenzhang/support-config-group-for-batchnorm
 2022-01-22 06:20:20 +00:00
huangbingjian c36a85e00d clean code 2022-01-22 11:39:12 +08:00
i-robot fe2a557901
!23919 [assistant][ops] Add HSVToRGB 
Merge pull request !23919 from 苏家毅/HSVToRGB
 2022-01-22 02:11:13 +00:00
yangzhenzhang 4d0b1a47ee support config group for batchnorm 2022-01-22 09:29:52 +08:00
davilsu af1333e1cc HSVToRGB 2022-01-21 20:57:30 +08:00
i-robot 690a660919
!17918 [assistant][MelScale] 
Merge pull request !17918 from QingfengLi/MelScale
 2022-01-21 09:52:56 +00:00
ZJM 8d339ccfea [assistant][ops][22736] Add new dataset operator LibriTTSDataset 2022-01-21 17:34:42 +08:00
i-robot b1213abe7d
!23937 [assistant][ops]Add GridSampler3D and GridSampler3DGrad 
Merge pull request !23937 from 梁天凯/GridSampler3D
 2022-01-21 07:09:17 +00:00
chenx2ovo bbba4fb940 [feat][assistant][I3CEG0] add op melscale 2022-01-21 11:30:34 +08:00
i-robot 4d81f9737f
!29324 support dilation for conv2d 
Merge pull request !29324 from yangzhenzhang/support-dilation-for-conv2d
 2022-01-21 02:57:15 +00:00
bubtltk f5ffb282ee Add GridSampler3D and GridSampler3DGrad 2022-01-20 22:23:30 +08:00
yangzhenzhang 000c5b5394 support dilation for conv2d 2022-01-20 19:04:52 +08:00
ZJM fff47f3886 [assistant][ops][I3T96T] Add new dataset operator CMUArcticDataset 2022-01-20 17:40:35 +08:00
i-robot 7bb5819889
!29265 fix resize_bilinear infer 
Merge pull request !29265 from jiangzhenguang/resize_bilinear
 2022-01-20 06:51:03 +00:00
zy 7543172bd9 [feat] [assistant] [I40GJE] Add new AICPU operator CholeskyInverse 2022-01-20 11:01:45 +08:00
i-robot 501614a61f
!22514 [assistant][ops] Add new data operator Multi30kDataset 
Merge pull request !22514 from 杨旭华/Multi30kDataset
 2022-01-20 02:13:17 +00:00
i-robot 46fdab3848
!23958 [assistant][ops] Add LuSolve 
Merge pull request !23958 from 张渝/lu_solve
 2022-01-20 01:42:24 +00:00
i-robot 6b6b001a78
!23443 [assistant][ops] Add LowerBound 
Merge pull request !23443 from 梁天凯/LowerBound
 2022-01-20 01:40:44 +00:00
i-robot f4bd87380b
!29195 Update API docs for Audio and Callback to the previous modifications 
Merge pull request !29195 from xiaotianci/fix_chinese_api
 2022-01-20 01:17:02 +00:00
bubtltk e514fc3feb Add LowerBound 2022-01-20 03:07:48 +08:00
unknown c4ee548905 [feat][assistant][I40GY6]add new data operator Multi30kDataset 2022-01-19 18:33:13 +08:00
i-robot 10c8b83068
!22421 [assistant][ops] Add new dataset operator GTZANDataset 
Merge pull request !22421 from TR-nbu/GTZANDataset
 2022-01-19 09:08:12 +00:00
zy 37bb297bc7 [feat] [assistant] [I40GI8] Add new operation LuSolve 2022-01-19 15:45:58 +08:00
Xiao Tianci 1d628bfb85 Fix API docs of Audio and Callback 2022-01-19 15:23:50 +08:00
ZJM 21973bbbec [assistant][ops][I3T96W] Add new dataset operator GTZANDataset 2022-01-19 13:48:50 +08:00
Weijun 85a7a994fe [feat] [assistant] [I40GJ3] Add new AICPU operator Lstsq 2022-01-19 13:19:46 +08:00
jiangzhenguang abddc993ea fix resize_bilinear 2022-01-18 20:33:44 +08:00
yangzhenzhang 5514189257 support group for conv2d 2022-01-18 17:28:13 +08:00
i-robot 19a9ad6f38
!22893 [assistant][ops] Add new data operator CreateFbMatrix 
Merge pull request !22893 from TR-nbu/CreateFbMatrix
 2022-01-17 08:33:48 +00:00
1462492739@qq.com bf5f24d62f [assistant][ops][I3T96S] Add new data operator CreateFbMatrix 2022-01-14 21:24:17 +08:00
yangzhenzhang 6dd7333c0b fix bug for conv2d transpose 2022-01-14 16:36:03 +08:00
i-robot fd5d5f1d21
!22754 [assistant][ops] Add math operators MatrixInverse, MatrixDeterminant and LogMatrixDeterminant 
Merge pull request !22754 from 孟权令/MatrixInverse
 2022-01-14 06:56:30 +00:00
i-robot 9dae085c46
!29054 Delete testcase for ops 
Merge pull request !29054 from zhanlijun/addtest1
 2022-01-14 06:08:51 +00:00
i-robot cfe5669de9
!28975 [MD] Chained Sampler Python UT fixes and updates 
Merge pull request !28975 from cathwong/ckw_ut_sampler_chain_fixes2
 2022-01-14 03:49:05 +00:00
Sawyer f335c44340 [feat][assistant][I3T92S][I3T92Q][I3T92R] add MatrixInverse, MatrixDeterminant and LogMatrixDeterminant 2022-01-14 09:20:55 +08:00
Cathy Wong 9d9013ba79 [MD] Chained Sampler UT fixes and updates 2022-01-13 17:39:33 -05:00
zhanlijun 5a5042e385 delete testcase for ops 2022-01-13 21:41:03 +08:00
i-robot a1b7b030c2
!29003 fix minddata api doc 
Merge pull request !29003 from luoyang/fix_api_doc
 2022-01-13 13:23:19 +00:00
i-robot e3ff16dac6
!28941 support single director exchange for conv2d 
Merge pull request !28941 from yangzhenzhang/support-single-direction-exchange-for-conv2d
 2022-01-13 11:43:28 +00:00
luoyang e5f2c75233 fix minddata api doc 2022-01-13 16:39:08 +08:00
yangzhenzhang a9d2e255f5 support single direction exchange for conv2d 2022-01-13 09:15:49 +08:00
markuskunej 54e584c300 Added input column validation for offloaded map operations. 
Moved check_add_offload from dataset_helper.py to offload.py.
 2022-01-12 16:16:49 +00:00
i-robot c5b650069f
!28662 Add input type check for instance Create in Cell.construct 
Merge pull request !28662 from zhangzhaoju/master
 2022-01-12 09:46:29 +00:00
i-robot fc4abc99f5
!28595 Fix some code review probelms 
Merge pull request !28595 from xiaotianci/update_code_review
 2022-01-12 09:30:28 +00:00
i-robot 5c961f9fb2
!28812 [MD UT] Misc AutoTune and Profiling UT updates 
Merge pull request !28812 from cathwong/ckw_prof_misc
 2022-01-12 03:03:13 +00:00
i-robot 017cb5f3ad
!27980 auto insert VirtualDataset node for master 
Merge pull request !27980 from lilei/insert_virtualdataset_for_master
 2022-01-12 01:14:44 +00:00
Cathy Wong 12b4e81f55 MD UT: Misc AutoTune and Profiling UT updates 2022-01-11 11:56:06 -05:00
i-robot 15bd3af3e3
!28821 Adapt to the old data format of aicpu 
Merge pull request !28821 from zangqx/zangqx_5
 2022-01-11 11:48:13 +00:00
i-robot d33f1c7e48
!28757 fix greater op infervalue issue & err massage 
Merge pull request !28757 from Simson/op_infer
 2022-01-11 11:03:42 +00:00
i-robot 8a6ca25539
!28641 unify ge and vm backend 
Merge pull request !28641 from xiaoyao/master
 2022-01-11 07:53:33 +00:00
Xiao Tianci 96a2e38d01 update review problems 2022-01-11 15:31:53 +08:00
i-robot 1ac5261dfe
!28598 fix softmaxcrossentropywithlogits infer 
Merge pull request !28598 from jiangzhenguang/softmaxcrossentropywithlogits
 2022-01-11 07:27:21 +00:00
臧庆香 10d275114e Adapt to the old data format of aicpu 2022-01-11 14:37:44 +08:00
xiao_yao1994 0843910260 unify ge and vm backend 2022-01-11 11:53:58 +08:00
zhangzhaoju a54d1f33e7 Add input type check for instance Create in Cell.construct 
Adjust the order of resource release sequence

Save executor_info to cache after compile avoiding inconsistent between
python cache and c++ cache

Clean entire_costgraph at ClearResAtExit to avoid coredump.
 2022-01-11 10:17:39 +08:00
simson 0191832a1d fix greater op infervalue 2022-01-11 09:21:48 +08:00
yangzhenzhang e5df74e9e4 compute top bottom overlap for conv2d 2022-01-10 16:15:04 +08:00
jiangzhenguang d895eadd7b fix softmaxcrossentropywithlogits infer 2022-01-06 20:44:11 +08:00
Margaret_wangrui 7d1f3d71ba Optimize error message 2022-01-06 20:32:08 +08:00
i-robot 0823ec3e9a
!28535 Modify Error Info 
Merge pull request !28535 from liuyang/ms_error
 2022-01-06 08:45:12 +00:00
i-robot d7c41f3f5f
!28553 [Fallback] Deal with constants such as np.pi 
Merge pull request !28553 from huangbingjian/attr_const
 2022-01-06 07:13:24 +00:00
liuyang_655 15a153a40d Modify error info 2022-01-06 01:28:53 -05:00
huangbingjian 45e8b52f6e [Fallback] Do not interpret constant from third-partiy library. 2022-01-06 09:42:54 +08:00
i-robot b073af4b5d
!28559 [MD][Offload] Added dimension check to required Offload ops. 
Merge pull request !28559 from alashkari/offload-fix
 2022-01-05 07:38:31 +00:00
i-robot a1720c210d
!22348 [assistant][ops] Add new loader EnWik9Dataset 
Merge pull request !22348 from 杨旭华/EnWik9Dataset
 2022-01-05 07:36:48 +00:00
i-robot ec485cb436
!28307 convert EnvSetItem/EnvGetItem to primitive EnvironSet/EnvironGet 
Merge pull request !28307 from xychow/EnvGetItem-conversion
 2022-01-05 07:10:30 +00:00
Rescue fac68dbb46 Added dimension check to required Offload ops. 2022-01-05 04:26:31 +00:00
i-robot de030c8d41
!28204 Improved HwcToChw support multi-channel input. 
Merge pull request !28204 from unseenme/mindcon_coding_hwc2chw
 2022-01-04 11:50:04 +00:00
unseenme 755cb0adfa Improved HwcToChw support multi-channel input. 2022-01-04 15:47:04 +09:00
Eden 63ab65e3a4 [feat] [assistant] [#I40GYF] add new loader EnWik9Dataset 2022-01-03 21:45:43 -08:00
zhousiyi 19e9571149 rename env_getitem/setitem to EnvironGet/EnvironGet 2022-01-04 02:17:30 +00:00
i-robot dba67422f9 !28384 [Fallback] Support the use of self 
Merge pull request !28384 from huangbingjian/fallback_self
 2021-12-31 09:26:03 +00:00
i-robot e8b4e34f02 !28352 Optimize the exception message while the return number of brop is wrong 
Merge pull request !28352 from zhangzhaoju/master_I4NX1K
 2021-12-31 08:17:18 +00:00
i-robot 03b4829b86 !28431 [Auto parallel] MoE Changes 
Merge pull request !28431 from Xiaoda/122-moe-changes
 2021-12-31 08:06:45 +00:00
zhangzhaoju 1950b6a823 Optimize the exception message while the return number of brop is wrong 
Fix:I4OQM9 Bprop out check message optimize
 2021-12-31 15:01:57 +08:00
vddong 7449b8364b [feat][assistant][I40GYQ] add new dataset operator WIDERFace 2021-12-31 13:49:47 +08:00
huangbingjian 58bafb539a [Fallback] Support self attr/method. 2021-12-31 10:57:41 +08:00
Xiaoda Zhang 6d8320fa66 1) fix the exact division in moe; 
2) changing CumSum from composition to a single Operator;
3) add InferMirrorOps for CumSumInfo.
 2021-12-30 19:20:05 +08:00
jingyangxiang a7f8b3af1c [feat][assistant][ops]New operator implementation, include IMDBDataset 
Signed-off-by: zjuter0126 <977180923@qq.com>
 2021-12-30 11:16:23 +08:00
zjuter0126 3c2ca41886 [feat][assistant][ops] add new data operator WikiTextDataset 2021-12-29 18:54:54 +08:00
i-robot 17b3e3cd13 !28267 Resuse ConvertData of GenerateArgumentsKey 
Merge pull request !28267 from zhangzhaoju/master_res_base
 2021-12-29 08:20:10 +00:00
i-robot ae1278fa42 !28074 convert execute mode to graph mode when loading mindir 
Merge pull request !28074 from lianliguang/master
 2021-12-29 06:23:57 +00:00
i-robot ac9e545dc8 !22546 add new dataset loading operator CaltechDataset 
Merge pull request !22546 from 杨旭华/CaltechDataset
 2021-12-29 03:40:06 +00:00
zhangzhaoju c5b2996b3f Resuse ConvertData of GenerateArgumentsKey 2021-12-29 10:30:17 +08:00
lianliguang d5efef0ae6 convert pynative mode to graph mode when loading mindir 2021-12-29 09:46:08 +08:00
donrichnx 9a0fc0ffc1 [feat] [assistant] [I40GYH] add new dataset loading operator CaltechDataset 2021-12-28 19:04:18 +08:00
Xiaoda Zhang 1bdb610b34 changing default value of single-loop flag 2021-12-28 17:56:30 +08:00
i-robot 67e843ec7f !27487 [assistant] [ops][SpectralCentroid] 
Merge pull request !27487 from 杨旭华/SpectralCentroidOp
 2021-12-28 09:10:32 +00:00
i-robot 7faf774063 !22310 [assistant][ops] Add new loader SemeionDataset 
Merge pull request !22310 from 杨旭华/SemeionDataset
 2021-12-28 03:13:02 +00:00
i-robot bff8d6ae01 !28247 Fix Python Vocab validation 
Merge pull request !28247 from luoyang/vocab
 2021-12-28 02:58:25 +00:00
luqilin 896f10d6d5 [feat][assistant][I40GYK] add new dataset loading operator SemeionDataset 2021-12-27 19:49:06 +08:00
tongyue 9a1dcd60fc [fix] [assistant] [I40GXI] fix bug in the infer function of the operator SpectralCentroid 2021-12-27 01:41:31 -08:00
i-robot 69a6c7db98 !23380 [assistant][ops] Add new loader AmazonReview 
Merge pull request !23380 from 杨旭华/AmazonReview
 2021-12-27 09:36:32 +00:00
luoyang ca3de66559 Fix Python Vocab validation 2021-12-27 17:12:07 +08:00
i-robot c13f31861c !26398 [feat] [assistant] [I48OBC] add new LogicalNot 
Merge pull request !26398 from 桂胜楠/logicalnot
 2021-12-27 02:54:53 +00:00
i-robot eb7a78a998 !28134 Python Vocab refactoring and add ids_to_tokens & tokens_to_ids 
Merge pull request !28134 from luoyang/vocab
 2021-12-25 06:39:41 +00:00
i-robot dd90a56d68 !28073 fix code warning && remove save_graphs use in st/ut 
Merge pull request !28073 from huanghui/fix-warning
 2021-12-24 09:29:25 +00:00
i-robot 170e6ad857 !28052 modify profiler ai_cpu ut and st 
Merge pull request !28052 from zangqx/zangqx_04
 2021-12-24 08:14:25 +00:00
i-robot 22c25ec10e !27862 [Auto parallel] [Sharding propagation] dealing with cast 
Merge pull request !27862 from Xiaoda/119-adapting-sharding-propagation
 2021-12-24 07:33:47 +00:00
luoyang 0288480722 Python Vocab refactoring and add ids_to_tokens & tokens_to_ids 2021-12-24 15:26:15 +08:00
huanghui 74ca50e652 fix code warning && remove save_graphs use in st/ut 2021-12-24 14:45:37 +08:00
zheng_pengfei 09b5e545dd [feat] [assistant] [I48OBC] add new LogicalNot 2021-12-23 20:39:17 +08:00
臧庆香 fcd3fface0 modify profiler ai_cpu ut and st 2021-12-23 20:10:28 +08:00
Xiaoda Zhang 66c7474e5a remove CastInfo from CNODE 2021-12-23 19:05:02 +08:00
i-robot f9270b8e09 !28069 change API：cell.shard 
Merge pull request !28069 from zhuyuxiao/I4LE9J
 2021-12-23 10:58:43 +00:00
chengchuanxing 07aa66ee43 [feat][assistant][I40GY2] add new loader AmazonReview 2021-12-23 16:32:31 +08:00
i-robot 1ea8693c83 !27356 dataset: check whether operator in nn or ops is mixed into operations 
Merge pull request !27356 from ms_yan/operator_mixed
 2021-12-23 07:04:52 +00:00
i-robot 3b6c40ff1e !28028 [ME] Optimize the error message of the operator module. 
Merge pull request !28028 from Margaret_wangrui/operator
 2021-12-23 06:48:00 +00:00
zhuyuxiao dd7bbf92dd change API 2021-12-23 14:26:06 +08:00
i-robot 91434597c3 !25861 [assistant][ops]New operator implementation, include BesselI0 and BesselI1 
Merge pull request !25861 from 张璇/BesselI0
 2021-12-23 02:41:41 +00:00
ms_yan 192983b5da check operator mixed or not 2021-12-23 10:22:03 +08:00
Margaret_wangrui 38ddd04c83 [ME] Optimize the error message of the operator module. 2021-12-23 10:10:48 +08:00
lilei 017aa359a6 insert VirtualDataset node for master 2021-12-23 09:25:05 +08:00
zx e0a7a87a9c [feat][assistant][I4CRK5] add new Ascend operator BesselI0 and BesselI1 2021-12-22 16:19:28 +08:00
gaocongli b9187f96da parsing the profiling data on ai cpu in binary 2021-12-22 11:59:47 +08:00
i-robot 14df5d3984 !23670 [assistant][ops] add new dataset loading operator CoNLL2000ChunkingDataset 
Merge pull request !23670 from 杨旭华/CoNLL2000ChunkingDataset
 2021-12-22 01:50:12 +00:00
i-robot b1de53fac2 !27728 [Fallback] Support numpy augassign method, subscript method and binary operations 
Merge pull request !27728 from huangbingjian/fallback_parse
 2021-12-22 01:01:46 +00:00
i-robot 372065c821 !26325 [assistant][ops] add Addcdiv and Addcmul 
Merge pull request !26325 from 孟权令/Addcdiv
 2021-12-21 12:46:21 +00:00
i-robot 2fbec9a554 !27856 use neighbor-exchange-v2 for conv2d 
Merge pull request !27856 from yangzhenzhang/use-neighborexchangev2-for-conv2d
 2021-12-21 10:59:54 +00:00
i-robot bc235730ca !27957 parallel_ut_refactoring 
Merge pull request !27957 from 王志伟/parallel_ut_refactoring2
 2021-12-21 08:02:16 +00:00
huangbingjian fea6974c80 Support numpy augassign method, subscript method and binary operations. 2021-12-21 15:58:20 +08:00
fattycat 289a7997df [feat] [assistant] [#I40GYB] add new loader CoNLL2000ChunkingDataset 2021-12-21 15:31:25 +08:00
Sawyer 564d83c959 [feat][assistant][I3T921][I3T923] add Addcdiv and Addcmul 2021-12-21 14:56:04 +08:00
i-robot cfdae8e0fc !27925 [ME] Optimize the error message of the operator module. 
Merge pull request !27925 from Margaret_wangrui/operator
 2021-12-21 06:54:30 +00:00
yangzhenzhang 8a68577756 use neighbor-exchange-v2 for conv2d 2021-12-21 14:27:41 +08:00
wzw a9b78682d5 parallel ut refactor 3 2021-12-21 14:26:58 +08:00
i-robot f007b8a99f !27745 [MD][Offload] Move offload checks and split to the C pipeline 
Merge pull request !27745 from markuskunej/offload_column_pipeline
 2021-12-21 04:26:51 +00:00
Margaret_wangrui f9a88a01bc [ME] Optimize the error message of the operator module 2021-12-21 10:19:25 +08:00
i-robot 7331629ceb !22686 [assistant][ops][Dither] 
Merge pull request !22686 from Isaac/Dither
 2021-12-20 11:16:20 +00:00
i-robot 76aaf62088 !27860 The profiler framework data is changed to binary data 
Merge pull request !27860 from helloiSCSI/data_reduce
 2021-12-20 07:47:49 +00:00
i-robot 953920112c !22553 [assistant][ops] Add new loader UDPOSDataset 
Merge pull request !22553 from 杨旭华/UDPOSDataset
 2021-12-20 06:54:54 +00:00
i-robot cc3c980a07 !27874 Real、conj、imag 接口支持实数输入 
Merge pull request !27874 from zhouyaqiang0/master
 2021-12-20 03:53:16 +00:00
helloiSCSI 115ec28ea0 Profiler save framework data as binary data 
1. Reconstructing the profiler data reporting module
2. Rewrite the write framework data module, Detail: add a new profiler_reporter file to report binnary data, and delele all the old
profiler reporter diretcory.
3. Rewrite the parse framework data module
4. Fix the framework module ut and st
 2021-12-20 11:06:53 +08:00
i-robot f1b26ea205 !21339 [assistant][ops]New operator implementation, include SparseApplyAdadeltaD 
Merge pull request !21339 from 梁秀波/SparseApplyAdadeltaD
 2021-12-20 01:34:10 +00:00
zhouyaqiang a6b48e003d Adapt Real input of ops real imag conj 2021-12-19 17:26:09 +08:00
haohaonannan97 f480be7c48 [feat][assistant][I40GXF] add new data operator Dither 2021-12-18 08:24:48 -08:00
王天浩 11cb3e4ef7 [fear][assistant][I40GYA] add new loader UDPOSDataset 2021-12-18 23:28:05 +08:00
wild-fox 452dbd4551 [feat][assistant][I40FFU] add new Ascend operator SparseApplyAdadelta 2021-12-18 17:56:51 +08:00
zorax 031c792cf6 [assistant][ops]New operator implementation, include ApplyAdamWithAmsgradD 2021-12-18 16:52:47 +08:00
i-robot 87c7fc13bb !23042 [assistant][ops]New operator implementation, include NonMaxSuppressionV3 
Merge pull request !23042 from 梁秀波/NonMaxSuppressionV3
 2021-12-18 06:51:51 +00:00
i-robot 57bbfe6453 !27741 Add test util function to write object into json 
Merge pull request !27741 from sabrinasun_59ee/test
 2021-12-18 03:00:53 +00:00
sabrinasun 2fa9399231 add test function to write object to json 2021-12-17 16:11:05 -05:00
markuskunej 0c10f52eb4 Moved the offload operation check and map operation split from python (datasets.py) to c++ (NodeOffloadPass.cc). 
Updated check for offload in python (dataset_helper.py and
iterators.py).

Updated auto_offload test in test_map_offload.py.

Changed MapNode.offload type from bool->int in bindings.cc.

Changed manual offload flag from int->enum.

Update offload end of pipeline detection to be column-specific.
 2021-12-17 14:35:17 +00:00
i-robot 457d953565 !22471 [assistant][ops]New operator implementation, include IsClose 
Merge pull request !22471 from 梁秀波/IsClose
 2021-12-17 11:55:37 +00:00
韩峥嵘 efe0fce473 [feat][assistant][I40FG0] add new Ascend operator NonMaxSuppression 2021-12-17 17:08:39 +08:00
i-robot 9b27295d1e !27544 fixed st and landscape bugs 
Merge pull request !27544 from Songyuanwei/test
 2021-12-17 08:14:23 +00:00
i-robot 96b2dce300 !22248 [assistant][ops] Add new loader operator YahooAnswers 
Merge pull request !22248 from 杨旭华/YahooAnswers
 2021-12-17 06:40:13 +00:00
i-robot 3121a317ec !27775 Fix Slice node's location incorrect 
Merge pull request !27775 from huanghui/fix-Slice-no-location
 2021-12-17 06:35:26 +00:00
韩峥嵘 43a495f3fd [feat][assistant][I40FG5] add new Ascend operator IsClose 2021-12-17 12:03:38 +08:00
Carry955 2320bb7c70 [feat][assistant][I40GXW] add new loader YahooAnswers 2021-12-16 18:33:47 -08:00
i-robot a618bb8ab5 !26949 save dataset into mindir - phase 2 
Merge pull request !26949 from luoyang/mindir-stage2-2
 2021-12-17 01:14:39 +00:00
i-robot d1dbdeffcb !27684 Update Python UT removal of generated files 
Merge pull request !27684 from zetongzhao/update_python_remove
 2021-12-17 01:09:17 +00:00
zetongzhao 7cb6989efd replace python removal with pytest tmp_path 2021-12-16 14:38:49 -05:00
i-robot 2bf244cd66 !27757 Support variable index of getitem during handling cell list getattr. 
Merge pull request !27757 from 张清华/opt_cell_list_getattr2
 2021-12-16 18:13:36 +00:00
i-robot 0158db12e6 !27800 add testcase for ops 
Merge pull request !27800 from zhanlijun/addtest1
 2021-12-16 13:12:26 +00:00
songyuanwei 11d84058c3 fixed st and landscape bugs 2021-12-16 21:03:37 +08:00
Zhang Qinghua ea8c47e981 Support variable index of getitem during handling cell list getattr. 2021-12-16 20:35:45 +08:00
zhanlijun 82de6c7ebc add testcase 2021-12-16 20:25:38 +08:00
huanghui d549ec5af5 fix Slice cannot get location 2021-12-16 17:19:20 +08:00
YangLuo fa7ed720c8 Saving dataset into Mindir and load for inference phase 2 2021-12-16 16:46:51 +08:00
i-robot c8096bc416 !27701 add output parallel strategy for gather op 
Merge pull request !27701 from yangzhenzhang/add-output-strategy-for-gather-op
 2021-12-16 06:54:40 +00:00
i-robot 67fc03f623 !27694 [API] python frame rebuild 
Merge pull request !27694 from kingxian/master
 2021-12-16 06:49:56 +00:00
jinxiaoxian 01fc6efc7d rebuild python frame 2021-12-16 10:50:59 +08:00
yangzhenzhang 5f6477b022 add output strategy for gather op 2021-12-16 09:24:50 +08:00
i-robot 05336b03c5 !22332 [assistant][ops][SogouNews] 
Merge pull request !22332 from 杨旭华/SogouNews
 2021-12-16 01:07:21 +00:00
i-robot 8d648e0f1e !19673 Seperate constant from pb file 
Merge pull request !19673 from sabrinasun_59ee/constant
 2021-12-15 23:52:33 +00:00
i-robot f6fe56c20d !27447 [MD][Offload] Fix Offload Normalize op bug, Added extra tests to Offload UT. 
Merge pull request !27447 from alashkari/offload-fix
 2021-12-15 16:00:26 +00:00
li-qiyao f4bfcb8c25 [feat][assistant][I40GXN] add new loader SogouNews 2021-12-14 22:53:38 -08:00
i-robot 005d4bb742 !27590 Change the AllSwap to the Internal Usage 
Merge pull request !27590 from huangxinjing/mv_all_swap
 2021-12-15 03:50:46 +00:00
i-robot 83e9fe5c48 !27542 dataset: change dataset_file into dataset_files for MindDataset 
Merge pull request !27542 from ms_yan/datasetfiles_name
 2021-12-15 03:09:33 +00:00
i-robot dac636decd !22297 [assistant][ops] Add new loader IWSLTDataset 
Merge pull request !22297 from 杨旭华/IWSLTDataset
 2021-12-15 02:55:08 +00:00
i-robot a5ce9e9b9b !27570 [Fallback] Process the attributes/methods of Tensor, different from NumPy 
Merge pull request !27570 from huangbingjian/parse_tensor
 2021-12-15 02:13:21 +00:00
i-robot d49f5e6caf !27525 support optimizer parallel for adafactor 
Merge pull request !27525 from yangzhenzhang/support-opt-parallel-for-adafactor
 2021-12-15 01:25:12 +00:00
sabrinasun 542d715217 seperate constant from pb 2021-12-14 13:19:14 -05:00
Rescue dc5c388505 Fixed offload Normalize op, added offload Normalize and Rescale tests to offload UT. 2021-12-14 17:03:11 +00:00
i-robot d57bdfb4f6 !23261 [assistant][ops][GainOp] 
Merge pull request !23261 from 杨旭华/Gain
 2021-12-14 11:05:27 +00:00
huangbingjian db4567989a fallback parse Tensor, different from NumPy. 2021-12-14 12:59:47 +08:00
i-robot 21176d6c51 !27602 Support getattr by the item of cell list: Handle the DoSignaturePrimitive('getitem') between getattr and resolve. 
Merge pull request !27602 from 张清华/opt_cell_list_getattr
 2021-12-14 03:52:42 +00:00
ms_yan a85113c093 change dataset_file into dataset_files for MIndDataset 2021-12-14 10:52:41 +08:00
i-robot 19b7b8b2de !27577 fix reshape bool in auto parallel 
Merge pull request !27577 from yao_yf/reshape_bool_redistribution_fix
 2021-12-14 01:25:05 +00:00
Zhang Qinghua 3e7b73e6c7 Support getattr by the item of cell list: Handle the DoSignaturePrimitive('getitem') between getattr and resolve. 2021-12-14 09:06:34 +08:00
liannai a99b5461bd [feat][assistant][I40GXH]add new data operator Gain 2021-12-13 22:03:09 +08:00
lufei9026 30bc609ce6 [feat][assistant][I40GY8]add new loader IWSLTDataset 2021-12-13 21:47:55 +08:00
i-robot efa63b78c0 !27600 [ME][Fallback] Add tensor testcase. 
Merge pull request !27600 from Margaret_wangrui/fallback_tensor
 2021-12-13 13:33:41 +00:00
i-robot 0d3ddbdde4 !27580 [ME][Fallback] Add array property test cases 
Merge pull request !27580 from Margaret_wangrui/fallback
 2021-12-13 13:14:52 +00:00
yao_yf 30576c6a75 fix reshape bool type in auto parallel 2021-12-13 20:24:33 +08:00
i-robot 2a2cd5e4ca !22222 [assistant][ops] Add new loader PennTreebankDataset 
Merge pull request !22222 from 杨旭华/PennTreebankDataset
 2021-12-13 11:43:00 +00:00
Margaret_wangrui dafe31e2cd [ME][Fallback] Add tensor testcase. 2021-12-13 17:13:24 +08:00
i-robot 7d7ec3c0c6 !27449 [ME]Add `MS` header of env var 
Merge pull request !27449 from chenfei_mindspore/environment_variable_fix
 2021-12-13 08:03:45 +00:00
huangxinjing f5889750f4 Move the AllSwap to the _AllSwap 2021-12-13 14:19:29 +08:00
yangzhenzhang 2a0b528084 support opt parallel for adafactor 2021-12-13 11:21:44 +08:00
Margaret_wangrui 751763dcef [ME][Fallback] Add array property test cases 2021-12-13 10:17:35 +08:00
huangbingjian 038cbe43de add tesecases of Tensor 2021-12-11 10:14:41 +08:00
i-robot 22078f77ea !27522 [ME][Fallback] Add tensor testcase. 
Merge pull request !27522 from Margaret_wangrui/fallback
 2021-12-10 09:40:42 +00:00
i-robot 938dc8abd0 !27439 [Auto parallel] Add new operatorInfo for Parallel: CumSum 
Merge pull request !27439 from Xiaoda/117-add-cumsum-op
 2021-12-10 07:00:06 +00:00
Margaret_wangrui 2779760409 [ME][Fallback] Add tensor testcase 2021-12-10 14:53:27 +08:00
chenfei 0ae5eb4618 add ms header of env var 2021-12-10 09:35:22 +08:00
i-robot 093f6f7bf2 !27453 Handle the interpreted node for binary ops, and refactoring other modules. 
Merge pull request !27453 from 张清华/opt_jit_fallback2
 2021-12-09 10:59:34 +00:00
i-robot d46e047fe1 !27372 fix permission bug when remove read_only_file on windows 
Merge pull request !27372 from zhangbuxue/fix_permission_bug_when_remove_read_only_file_on_windows
 2021-12-09 06:11:13 +00:00
Zhang Qinghua 135db97692 Handle the interpreted node for binary ops. 
Refactoring specialize.
Adjust ir/dat/dot order.
Optimize DumpIR FV routine.
 2021-12-09 11:33:45 +08:00
i-robot 0e358f4cb3 !27428 revert insert VirtualDataset node for master 
Merge pull request !27428 from lilei/modify_virtualdataset_for_master
 2021-12-09 02:47:13 +00:00
buxue ae256a362b fix permission bug when remove read_only_file on windows 2021-12-09 09:32:24 +08:00
Xiaoda Zhang 8042c88223 add the new operatorInfo for parallel: CumSum 2021-12-09 09:21:52 +08:00
i-robot 66a0e7efa5 !27424 [MD][revert] revert offload modification code to fix pangu model problem 
Merge pull request !27424 from xiefangqi/md_revert_offload_pr
 2021-12-08 13:30:50 +00:00
lilei 2edf6ab33b revert insert VirtualDataset node for master 2021-12-08 17:39:42 +08:00
i-robot faaec746f7 !27401 add more ut tests for allreduce fusion 
Merge pull request !27401 from jiahongQian/master
 2021-12-08 09:38:37 +00:00
Megalomania 259854e040 [feat] [assistant] [I40GY5] add new loader PennTreebankDataset 2021-12-08 01:22:40 -08:00
xiefangqi bae1dd3bb0 Revert "Moved the offload operation check and map operation split from python (datasets.py) to c++ (NodeOffloadPass.cc)." 
This reverts commit 84d38e547a.
 2021-12-08 16:39:10 +08:00
i-robot 895256f69a !20043 [assistant][ops] Add new YelpReview 
Merge pull request !20043 from 杨旭华/YelpReview
 2021-12-08 08:13:34 +00:00
jiahongQian b03c8d18d3 add more ut tests 2021-12-08 14:17:02 +08:00
i-robot 75e35bded2 !27081 [MD][Offload] Move offload checks and split to the C pipeline 
Merge pull request !27081 from markuskunej/move_offload_check_to_c
 2021-12-07 15:55:16 +00:00
i-robot b80ed801d5 !27335 change log level for read not exist tensors 
Merge pull request !27335 from yelihua/dev
 2021-12-07 13:08:20 +00:00
lufei9026 2947f418b1 [feat][assistant][I3T96N]add new loader YelpReview 2021-12-07 17:09:41 +08:00
i-robot ffca7b08a5 !27237 auto insert VirtualDataset node for master 
Merge pull request !27237 from lilei/modify_virtualdataset_for_master
 2021-12-07 09:02:50 +00:00
i-robot 25ae2d620b !27271 modify c++ msg & conv2d 
Merge pull request !27271 from huchunmei/master
 2021-12-07 07:56:21 +00:00
yelihua 01823faed0 change error level to info as dir not exist is normal case 2021-12-07 15:27:37 +08:00
i-robot 7a8e35b4f1 !26456 [assistant][ops]Add new operator Spectrogram 
Merge pull request !26456 from YJfuel123/Spectrogram
 2021-12-07 06:39:01 +00:00
i-robot 3d0f8fcc99 !22552 [assistant][ops]New operator implementation, include STL10Dataset 
Merge pull request !22552 from 王睿/stl10_dataset
 2021-12-07 06:37:04 +00:00
i-robot 59184541c3 !27140 fix sync GraphCell params data from device bug 
Merge pull request !27140 from zhangbuxue/fix_sync_GraphCell_params_data_from_device_bug
 2021-12-07 04:33:21 +00:00
huchunmei 1023c7a5e2 modify c++ msg 2021-12-07 11:20:12 +08:00
i-robot f40668ef73 !27251 test_micro_batch_Interleaved 
Merge pull request !27251 from lilei/add_parallel_ut
 2021-12-07 02:18:09 +00:00
i-robot 9b2f4141c9 !27302 add summart ops params valid check ut 
Merge pull request !27302 from helloiSCSI/add_st
 2021-12-07 01:56:52 +00:00
RainWang6188 c4063f0f11 [feat][assistant][I3J6VI] add new data operator STL10 2021-12-07 01:21:11 +00:00
buxue d7dc539b6e fix sync GraphCell params data from device bug 2021-12-07 09:20:46 +08:00
i-robot d723cc417d !27151 [ME][Fallback] [ME][Fallback] Support built-in type: complex and reversed. 
Merge pull request !27151 from Margaret_wangrui/fallback
 2021-12-07 01:04:14 +00:00
i-robot 8bcd6e3a86 !27287 [ME][Fallback] Handle interpret nodes in list. 
Merge pull request !27287 from Margaret_wangrui/list_interpret
 2021-12-07 01:02:12 +00:00
i-robot d1aa8d6cb5 !26995 fix:unexpected eiminate of tile 
Merge pull request !26995 from zhangzhaoju/master_bug_fix
 2021-12-07 00:58:44 +00:00
lilei 05189459ab auto insert VirtualDataset node for master 2021-12-06 21:39:37 +08:00
Margaret_wangrui 6e5d625153 [ME][Fallback] Support built-in type: complex and reversed. 2021-12-06 21:15:30 +08:00
i-robot b1eae16a05 !22304 [assistant][ops]New operator implementation, include SVHNDataset 
Merge pull request !22304 from 王睿/svhn_dataset
 2021-12-06 12:30:13 +00:00
helloiSCSI 7cb5ed7676 add summart ops params valid check ut 2021-12-06 20:11:27 +08:00
郑昊 4ae41350a6 [feat][assistant][I3ALD1] add new method Spectrogram 2021-12-06 19:24:37 +08:00
lilei e933aa268b test_micro_batch_Interleaved 2021-12-06 18:37:21 +08:00
Margaret_wangrui fa8506c0a7 [ME][Fallback] Handle interpret nodes in list. 2021-12-06 16:58:21 +08:00
zhangzhaoju 722086435d 1. fix:unexpected eiminate of tile 
2. fix:delay memory allocat in NewTensorFilledWithData
 2021-12-06 14:49:48 +08:00
i-robot 2d23b698a6 !27024 add allreduce fusion by size 
Merge pull request !27024 from jiahongQian/master
 2021-12-06 05:32:12 +00:00
q00596439 de36fdc169 add allreduce fusion size and unify the interface 2021-12-05 21:35:11 +08:00
zhangyuqwer e934381b08 [feat][assistant][I3ZSQV] add new data operator CharNGram 2021-12-04 22:09:34 +08:00
RainWang6188 340b3b4d5d [feat][assistant][I3J6VJ] add new data operator SVHN 2021-12-04 08:19:47 +00:00
mohammad af6e587c28 MD Profiler: support multiple start/stop calls 2021-12-03 23:25:38 -05:00
danishfarid faaa5d5f0a autotune step 
fix testing script default

updates and changed sampling function

comments addressed

comments addressed

comments addressed

moving isSinkCheckAbove

update - move CheckSinkFunction

fix
 2021-12-03 20:29:35 +00:00
markuskunej 84d38e547a Moved the offload operation check and map operation split from python (datasets.py) to c++ (NodeOffloadPass.cc). 
Updated check for offload in python (dataset_helper.py and
iterators.py).

Updated auto_offload test in test_map_offload.py.

Changed MapNode.offload type from bool->int in bindings.cc.

Changed manual offload flag from int->enum.
 2021-12-03 14:40:01 +00:00
luoyang f3e1794a74 update log 2021-12-03 15:09:36 +08:00
i-robot 93db7f741f !20788 [assistant][ops] Add new dataset loading operator TedliumDataset 
Merge pull request !20788 from 杨旭华/TedliumDataset
 2021-12-03 06:29:29 +00:00
i-robot 8e4f3472d8 !27154 Enable the test for Parallel Optimizer 
Merge pull request !27154 from huangxinjing/enable_gradient_accumulation
 2021-12-03 06:23:53 +00:00
i-robot c0048bb117 !26953 Add some bprop mindir files 
Merge pull request !26953 from YuJianfeng/bprop_mindir
 2021-12-03 03:28:37 +00:00
i-robot 9cec03e3a8 !27033 close autotune in non-sink mode 
Merge pull request !27033 from luoyang/close_autotune_nonsink
 2021-12-03 01:26:06 +00:00
huangxinjing 8c9b2b93a8 Add transformer 2021-12-03 09:08:57 +08:00
luoyang 9cbb45bcc4 close autotune in non-sink mode 2021-12-03 08:46:14 +08:00
i-robot 9ddd3fb66f !27085 [MD][UT]: Use pytest.mark.forked for profiling tests 
Merge pull request !27085 from cathwong/ckw_ut_prof_forked
 2021-12-02 17:00:28 +00:00
Cathy Wong 367e5ed1dd MD UT: Use pytest.mark.forked for profiling tests 2021-12-02 10:55:05 -05:00
i-robot eb25c473d0 !27117 [ME][Fallback] Handle interpret nodes in tuple. 
Merge pull request !27117 from Margaret_wangrui/tuple_interpret
 2021-12-02 14:11:12 +00:00
i-robot fa457b2bfa !17832 [assistant][ops]New operator implementation, include KMnistDataset 
Merge pull request !17832 from 张璇/kmnist_dataset
 2021-12-02 14:00:36 +00:00
i-robot 3e73e124ac !23225 [assistant][ops] Add Glove 
Merge pull request !23225 from 张渝/Glove
 2021-12-02 12:22:17 +00:00
i-robot eab0699c6d !26986 change MD error log 
Merge pull request !26986 from shenwei41/master
 2021-12-02 11:35:07 +00:00
i-robot 7c9dc30e78 !27101 check args for shard 
Merge pull request !27101 from yangzhenzhang/fix-bug-for-out-strategy
 2021-12-02 08:24:47 +00:00
Margaret_wangrui f7fc7f4238 [ME][Fallback] Handle interpret nodes in tuple 2021-12-02 15:36:54 +08:00
i-robot 53f50c0d6b !26880 [MD] fix mindrecord log message part02 
Merge pull request !26880 from liyong126/fix_mindrecord_log_msg_2
 2021-12-02 06:48:15 +00:00
yangzhenzhang 7454b8f8f2 check args for shard 2021-12-02 11:51:29 +08:00
i-robot 3bff079969 !26896 Enhance error message of audio ops 
Merge pull request !26896 from xiaotianci/enhance_audio_err_message
 2021-12-02 03:42:35 +00:00
liyong d35e003def fix mindrecord log part02 2021-12-02 11:28:58 +08:00
i-robot ae94c51a97 !26930 fix bug of export mindir when using encrypt 
Merge pull request !26930 from lianliguang/mindir_load_2G
 2021-12-02 02:21:19 +00:00
luqilin 34bffbf768 [feat][assistant][I3T96H] add new dataset loading operator TedliumDataset 2021-12-02 09:53:39 +08:00
i-robot b00b1cc276 !26982 [ME][Fallback] Review the local dict and try to eval out the interpret node. 
Merge pull request !26982 from Margaret_wangrui/recheck_local_dict
 2021-12-02 01:09:12 +00:00
yujianfeng 7c808ee792 Add some bprop mindir files 2021-12-01 19:44:06 +08:00
zy 8f9a09f79c [fix] [assistant] [I3ZSQR] add new data operator GloVe 2021-12-01 17:12:20 +08:00
shenwei41 5d48409395 change err info 2021-12-01 16:48:14 +08:00
i-robot 6924866a87 !26746 [MD] minddata error msg optimize 
Merge pull request !26746 from xiefangqi/md_datasetops_error_msg_opt
 2021-12-01 07:18:18 +00:00
zy 5ca98a494a [fix] [assistant] [I3ZSQS] add new data operator FastText 2021-12-01 11:12:08 +08:00
“uccInf” 92d6d31750 [feat][assistant][I3J6V7] add new data operator KMnist 2021-12-01 03:05:05 +00:00
Margaret_wangrui ee1b4d4e6d [ME][Fallback] Review the local dict and try to eval out the interpret node 2021-12-01 10:02:26 +08:00
i-robot b6a6dbbe8f !27003 [Auto parallel] [Sharding propagation] Dealing with a parameter being used by mutiple ops 
Merge pull request !27003 from Xiaoda/114-adapting-sharding-propagation-for-common-parameter
 2021-12-01 02:01:34 +00:00
i-robot fcac7351aa !26801 add loss landscape ut and st 
Merge pull request !26801 from Songyuanwei/landscape_ci
 2021-12-01 01:49:43 +00:00
i-robot eb9537af11 !21690 [assistant][ops] Add data operator Vectors 
Merge pull request !21690 from 张渝/Vectors
 2021-12-01 01:29:45 +00:00
Xiao Tianci b883498c93 enhance error message of audio ops 2021-12-01 09:27:04 +08:00
i-robot 1905b228b4 !26609 Update minddata log message 
Merge pull request !26609 from luoyang/log
 2021-12-01 01:25:11 +00:00
lianliguang c7acd4b0ca fix bug of export mindir when using encrypt mode 2021-12-01 09:22:53 +08:00
i-robot bf59d75604 !26611 [MD] fix mindrecord log message part01 
Merge pull request !26611 from liyong126/fix_mindrecord_log_msg
 2021-12-01 01:12:48 +00:00
xiefangqi f74b5cea0c optimize datasetops error msg 2021-12-01 09:11:22 +08:00
Xiaoda Zhang 364858cbc9 In sharding propagation, to keep strategy consistent of parameter being used by multiple operators, we check the edge with one node of TmpIdentityInfo 2021-11-30 18:58:33 +08:00
i-robot 2fd00935ed !20964 [assistant][ops] Add new data operator SpeechCommandsDataset 
Merge pull request !20964 from 杨旭华/SpeechCommandsDataset
 2021-11-30 09:33:03 +00:00
i-robot 95c0cafef4 !26824 white list for syntax exception 
Merge pull request !26824 from zhangzhaoju/master_syntax
 2021-11-30 08:01:59 +00:00
i-robot 7554053c00 !26786 Add initializer and optimizer ut 
Merge pull request !26786 from wanyiming/add_init_opt_ut
 2021-11-30 07:23:27 +00:00
luoyang 1a6a42c083 Update minddata log message 2021-11-30 15:22:40 +08:00
zhangyuqwer 6a6e73ef82 [fix] [assistant] [I3ZSQM] add new data operator Vectors 2021-11-30 14:39:42 +08:00
songyuanwei a67ac908d8 add landscape ut and st 2021-11-30 11:28:47 +08:00
liyong 2869352cc8 fix mindrecord log part01 2021-11-30 10:57:58 +08:00
Xiaoda Zhang 04db51a528 In a previous PR (https://gitee.com/mindspore/mindspore/pulls/26807/), we replaced 'auto_parallel_search_mode' by 'search_mode' directly. 
However, to be forward compatitable, it is suitable to keep 'auto_parallel_search_mode' available. This PR recovers the 'auto_parallel_search_mode' interface and adds a warning when using this old interface.

This PR also deals with other codestyle things.
 2021-11-29 15:40:23 +08:00
zhangzhaoju d34f14e324 white list for syntax exception 2021-11-29 12:40:38 +08:00
vddong 91d3a7543f [feat][assistant][I3T96F] add new data operator SpeechCommands 2021-11-27 11:25:18 +08:00
i-robot d1e4e674ab !26690 Add supoort resolving outer lambda function for ops.Partial. 
Merge pull request !26690 from hezhenhao1/add_lambda
 2021-11-26 09:21:34 +00:00
hezhenhao1 4af312d17e Add supoort resolving outer lambda function for ops.Partial. 2021-11-26 15:05:50 +08:00
i-robot 9f8ec2c5ab !26807 [Auto parallel] [Sharding propagation] Interface change of sharding propagation 
Merge pull request !26807 from Xiaoda/113-auto-parallel-search-mode-changes-to-search-mode
 2021-11-26 01:48:58 +00:00
i-robot 6ecbc97fd6 !26804 virtual_dataset_avoid_auto_parallel 
Merge pull request !26804 from yao_yf/virtual_dataset_avoid_auto_parallel
 2021-11-26 01:48:53 +00:00
wanyiming ab65f4ac98 addut 2021-11-26 09:48:34 +08:00
i-robot b282414de7 !26619 arallel_ut_refactoring 
Merge pull request !26619 from 王志伟/parallel_ut_refactoring1
 2021-11-26 01:34:49 +00:00
i-robot bbe9606f1c !26805 Revert "Add the check of function return None." 
Merge pull request !26805 from Margaret_wangrui/return_none
 2021-11-26 01:28:46 +00:00
i-robot d8fbc26419 !26700 [MD][Profiler][Autotune] Add test, logs, clean state 
Merge pull request !26700 from harshvardhangupta/add_test_at_prof
 2021-11-25 19:38:33 +00:00
Margaret_wangrui 922678edef Revert "Add the check of function return None." 
This reverts commit f9a384456a.
 2021-11-25 21:18:24 +08:00
i-robot 4407210ae3 !26689 Modify error 
Merge pull request !26689 from liuyang/error_ms
 2021-11-25 12:18:11 +00:00
Xiaoda Zhang ad5ac77ae8 1) 'auto_parallel_search_mode' changes to 'search_mode'; 
2) 'sharding_propagation' moves to 'search_mode';
 2021-11-25 19:38:21 +08:00
yao_yf f29ce1fb60 virtual dataset avoid auto parallel 2021-11-25 18:47:35 +08:00
harshvardhangupta 02f4a8068b add tests, clean states, print proper warn logs 2021-11-25 00:03:38 -05:00
liuyang_655 12092c882f modify error 2021-11-24 22:40:23 -05:00