p60975123
/
mindspore
forked from mindspore-Ecosystem/mindspore
1
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity
43,567 Commits 23 Branches 0 Tags 3.9 GiB
Commit Graph

5490 Commits

Author SHA1 Message Date
i-robot a9cdbd5ae8
!32005 fix bugs of moe 
Merge pull request !32005 from bichaoyang/master
 2022-03-29 01:30:10 +00:00
larrygld aeb0b2090b [assistant][ops] Add array operators Triu 2022-03-28 21:25:29 +08:00
王超 0cf25e180f [assistant] [ops] [I48O8K] add new array operator RightShift 2022-03-28 18:51:44 +08:00
i-robot c2e0796bdb
!25339 [assistant][ops] Add new Tril 
Merge pull request !25339 from 张凯磊/Tril
 2022-03-28 09:26:13 +00:00
b00518648 93da6bab46 fix bugs of moe: only use a fewer dp in moe 2022-03-28 17:22:48 +08:00
i-robot 5c095e75fa
!32006 Fix exception when sending large messages 
Merge pull request !32006 from chengang/fix_send_large_msg
 2022-03-28 06:27:19 +00:00
Parallels f660b3c58e Add cluster topology state 2022-03-28 05:48:11 +00:00
i-robot 8add6a926c
!31925 modify interface of shard function 
Merge pull request !31925 from wangjun/interface_modify_0325
 2022-03-28 02:24:16 +00:00
i-robot a15f682803
!31603 support str.format 
Merge pull request !31603 from lianliguang/supported-str-format
 2022-03-28 01:33:58 +00:00
lilei 452362332e xmodify stridedslice for master 2022-03-28 09:24:13 +08:00
Parallels 7bcf3f7b03 Fix exception when sending large messages 2022-03-27 02:10:40 +00:00
Carry955 3d182380d8 [feat][assistant][I4862P] add new aicpu operator Tril 2022-03-26 19:36:29 +08:00
i-robot 5a4e25db92
!26226 [assistant][ops] Add NonDeterministicInts 
Merge pull request !26226 from yyxhgg/NonDeterministicInts
 2022-03-26 09:42:53 +00:00
i-robot d66af6eb86
!26399 [assistant][ops] Add CropAndResizeGradBoxes 
Merge pull request !26399 from wangyantao/CropAndResizeGradBoxes
 2022-03-26 09:41:38 +00:00
lianliguang 6304c2ef3d support str.format 2022-03-26 14:53:06 +08:00
lianliguang 424c16d116 fix abs(tensor) when tesnor data is a 0-dim number 2022-03-26 14:20:03 +08:00
wangjun 789539cbaa modify interface name for shard 2022-03-26 09:10:14 +08:00
i-robot 3f18180606
!31742 [MD][Autotune] Change autotune config 
Merge pull request !31742 from harshvardhangupta/at_config_suggestion
 2022-03-25 21:00:12 +00:00
wangyantao 668cdb936f [feat] [assistant] [I471CP] add new operator CropAndResizeGradBoxes 2022-03-26 00:30:34 +08:00
ZCX 0cb72a21f3 [feat] [assistant] [I471DI] add new Ascend operator NonDeterministicInts 2022-03-25 22:29:47 +08:00
harshvardhangupta 0086f4ee05 Modify the Dataset AutoTune config #31742 2022-03-25 09:03:38 -04:00
i-robot 40015e003f
!31896 fix isolated node error 
Merge pull request !31896 from lianliguang/fix-isolated-node-bug
 2022-03-25 11:52:21 +00:00
lizhenyu 812127940d add ut for recovery 2022-03-25 16:58:08 +08:00
lianliguang 3d7c56c4cc fix isolate node bug when using fallback interpret node 2022-03-25 16:41:16 +08:00
i-robot c6da65ecfa
!31879 Training support recovery 
Merge pull request !31879 from zyli2020/worker_failover_bp
 2022-03-25 07:06:55 +00:00
i-robot 1af8b365de
!31880 [MD] add OBSMindDataset 
Merge pull request !31880 from liyong126/add_obs_mindrecord_dataset
 2022-03-25 06:50:58 +00:00
i-robot 8589335879
!31889 Add some ut pipeline test case for C++ dataset 
Merge pull request !31889 from liucunwei/dataset-lcw-0324
 2022-03-25 03:27:37 +00:00
i-robot 5f343bcd09
!31824 add waring suggestion log and map op error check 
Merge pull request !31824 from guozhijian/fix_more_info
 2022-03-25 03:24:07 +00:00
lizhenyu 275f81c47b failover support exit process when timeout occur 2022-03-25 11:16:04 +08:00
liyong 419109cf0a add OBSMindDataset 2022-03-25 11:03:53 +08:00
liucunwei 03b32e4eab Add some ut pipeline test case for C++ dataset 2022-03-25 10:25:19 +08:00
i-robot 3c83d3f600
!31668 [Auto-Par] [D-Rec] Modify the name of the functions and variables of Parameter shared User strategy treatment & Add new treatment case of MatMul/Gather 
Merge pull request !31668 from 王浩然/D-Rec-deliver
 2022-03-25 02:11:41 +00:00
jonyguo 317c48a933 add log warning and op error check 2022-03-25 09:26:29 +08:00
7347157+joylvliang@user.noreply.gitee.com 39e70e8519 fix_bug_of_tensor_copy_canot_work_in_int64_with_D 2022-03-24 20:47:42 +08:00
i-robot 0b6f330d7e
!31666 [MS][LITE] new core ops api and lite adapter new api 
Merge pull request !31666 from luoyuan/core2
 2022-03-24 08:50:40 +00:00
i-robot aca631a73b
!31714 [MD] fix GIL acquire issue 
Merge pull request !31714 from liyong126/fix_two_level_pipeline
 2022-03-24 08:31:12 +00:00
yanghaoran e0ee78497d takedown tcptest.sendlargemessages to ensure gate stability 2022-03-24 15:20:30 +08:00
haoran.wang fdfbe2dedc Modify the name of the funtions and variables of Parameter shared User strategy treatment 2022-03-24 06:16:55 +00:00
luoyuan 3e6e4bfdd3 add core ops api and adapter new mindapi 2022-03-24 10:47:40 +08:00
liyong bff0fbc76f fix GIL acquire Misuse 2022-03-24 08:45:30 +08:00
i-robot 4110c3d6d6
!31646 [dataset][bugfix] 修复config 接口typeerror的问题 
Merge pull request !31646 from xiefangqi/md_fix_interface_comments
 2022-03-23 02:45:01 +00:00
i-robot f59de99ed6
!31654 [ME]Support raise in constant scenarios. 
Merge pull request !31654 from Margaret_wangrui/raise
 2022-03-23 02:28:37 +00:00
Margaret_wangrui 6d10adc9c5 [ME]Support raise in constant scenarios. 2022-03-22 19:33:47 +08:00
i-robot a50223f93a
!31680 [assistant][Fade] bug fix 
Merge pull request !31680 from QingfengLi/fade
 2022-03-22 07:58:59 +00:00
i-robot ecde0749bb
!31576 dynamic shape CPU & GPU 
Merge pull request !31576 from TuDouNi/dynamic_shape_gpu_cpu_2
 2022-03-22 07:27:50 +00:00
i-robot 32ecb8cbea
!31664 fix grpc not found in gpu serving 
Merge pull request !31664 from zhoufeng/xiu-ba-ge
 2022-03-22 06:10:24 +00:00
i-robot 200a77c3d1
!31393 fix csr/coo docs and checks 
Merge pull request !31393 from wangrao124/pr_0316
 2022-03-22 05:15:38 +00:00
ttudu adb18b82ca dynamic shape CPU & GPU 2022-03-22 13:03:31 +08:00
chenx2ovo c50a4611ec [fix][assistant][I3CEGF] fix can not compute bool tensor 2022-03-22 10:20:01 +08:00
xiefangqi f81c7967ec fix config interface comments 2022-03-22 10:04:12 +08:00
i-robot 3b24d825bf
!31578 Fix exception for sending big message using TCP client. 
Merge pull request !31578 from chengang/fix_rpc_big_msg
 2022-03-22 01:27:29 +00:00
i-robot 9bfa6ebc58
!31611 code check clean 
Merge pull request !31611 from yuchaojie/code-clean
 2022-03-22 01:20:44 +00:00
zhoufeng 55f6d075e8 fix grpc not found in serving 
Signed-off-by: zhoufeng <zhoufeng54@huawei.com>
 2022-03-21 22:06:46 +08:00
i-robot 264bed91a4
!31594 Add ps execution mode implementation 
Merge pull request !31594 from ZPaC/add-dist-execution-mode
 2022-03-21 13:07:24 +00:00
i-robot 4bb2fbdb5f
!30695 Support third-party modules 
Merge pull request !30695 from huangbingjian/import_module
 2022-03-21 08:36:39 +00:00
yuchaojie 42818b17f6 code check clean 2022-03-21 16:32:22 +08:00
i-robot 4dd3a94d27
!31571 clean code 
Merge pull request !31571 from kisnwang/clean_code
 2022-03-21 08:31:38 +00:00
ZPaC 506989abb8 Add ps execution mode implementation 2022-03-21 14:35:25 +08:00
huangbingjian a62484c5ad Added recognition of third-party libraries 2022-03-21 14:10:23 +08:00
i-robot 91d9585df7
!31346 add vmap ut and st 
Merge pull request !31346 from Erpim/vmap_v12
 2022-03-21 03:49:08 +00:00
kswang 70b85a16cb clean code 2022-03-21 09:54:06 +08:00
Erpim 0e7e1f8400 add vmap st and ut 2022-03-21 09:38:38 +08:00
i-robot af1f1b346d
!31509 [MD] fix mindrecord commit issue 
Merge pull request !31509 from liyong126/fix_mindrecord_issue
 2022-03-20 11:16:31 +00:00
i-robot 048d089f9a
!31520 test ut 
Merge pull request !31520 from gaoyong10/dynamic_shape_01
 2022-03-20 04:24:23 +00:00
Parallels d3bcc66d15 Fix sending large message using rpc. 2022-03-20 10:42:52 +08:00
gaoyong10 89c98f8b73 Add ut for dynamic shape and control flow. 2022-03-19 19:48:44 +08:00
zhoufeng da11e0f150 change debug.a to object 
Signed-off-by: zhoufeng <zhoufeng54@huawei.com>
 2022-03-19 15:39:49 +08:00
liyong 00371d523f fix mindrecord issue 2022-03-19 12:04:27 +08:00
i-robot dc5f77f585
!31298 build libmindspore_backend.so 
Merge pull request !31298 from zhoufeng/backend-frontend-decoupling
 2022-03-18 09:41:51 +00:00
wangrao124 5e5415f617 fix csr/coo 2022-03-18 16:46:38 +08:00
i-robot c0c5b8c143
!31480 Reduce UT CPP profiler tests time 
Merge pull request !31480 from zetongzhao/reduce_profiler_test_time
 2022-03-17 19:10:57 +00:00
zetongzhao 694ede05cf change set_dataset 2022-03-17 13:13:01 -04:00
i-robot d37ee6e729
!31173 Stream Assign Refactor 
Merge pull request !31173 from jiaorui/refactor-stream
 2022-03-17 11:39:11 +00:00
zhoufeng b7bb53ff73 build libmindspore_backend.so 
Signed-off-by: zhoufeng <zhoufeng54@huawei.com>
 2022-03-17 19:14:45 +08:00
i-robot 2e516c735b
!28640 [assistant] [ops] Add the FillV2 operator 
Merge pull request !28640 from 爱科学的小松鼠/OpFill
 2022-03-17 09:48:14 +00:00
i-robot c2a50f81df
!31398 fix code check for minddata 
Merge pull request !31398 from guozhijian/fix_code_check
 2022-03-17 06:49:43 +00:00
hu-daiwang 3b093696e7 [feat] [assistant] [I48O6L] add the fillV2 operator 2022-03-17 11:19:59 +08:00
jiaorui 91bd673538 refactor 2022-03-17 10:50:12 +08:00
i-robot e6dadfd999
!31159 Add st and ut testcase for pass tuple_transform 
Merge pull request !31159 from chenfei_mindspore/master-develop
 2022-03-17 02:01:44 +00:00
i-robot c3d15a079e
!31325 Training supports failover 
Merge pull request !31325 from zyli2020/worker_failover_bp
 2022-03-17 01:37:30 +00:00
jonyguo 11079eabbb fix: code check 2022-03-17 08:23:12 +08:00
i-robot 8e1f6d6b9b
!31268 Move UT CPP Helper Functions 
Merge pull request !31268 from zetongzhao/ut_cpp_helper
 2022-03-16 18:20:24 +00:00
i-robot 1d6834ce4d
!31334 [MD][Autotune] Modify the generated autotune config to improve readability 
Merge pull request !31334 from harshvardhangupta/modify_at_config
 2022-03-16 18:18:57 +00:00
lizhenyu 545707b80c Training supports failover 2022-03-16 23:34:29 +08:00
zetongzhao ab5f79f522 move common helper function 2022-03-16 10:30:43 -04:00
chenfei 3872e29c3b add ut for tuple trans 2022-03-16 21:33:54 +08:00
i-robot 88691d054e
!31321 Convert AbstractList to AbstractTuple recursively 
Merge pull request !31321 from hewei/fix_opt_perf
 2022-03-16 07:09:52 +00:00
i-robot c2212f88b4
!31164 Fix the global norm missing insert allreduce 
Merge pull request !31164 from huangxinjing/fx_global_norm_error
 2022-03-16 06:46:13 +00:00
i-robot 72ea797e4d
!30339 [BUG][MD][FUNC]RandomInvert 
Merge pull request !30339 from yangwm/randominvert
 2022-03-16 06:39:53 +00:00
i-robot 8e8391dadb
!30429 BUG][MD][FUNC]RandomAutoContrast 
Merge pull request !30429 from yangwm/autocontrast
 2022-03-16 06:38:54 +00:00
He Wei 51c147ab12 Convert AbstractList to AbstractTuple recursively 2022-03-16 09:59:27 +08:00
i-robot bf03f0e030
!31252 Implementation of element wise parallel ops 
Merge pull request !31252 from liuluobin/element_wise_ops
 2022-03-16 01:32:29 +00:00
i-robot 5f75c701d5
!31332 Fix bug in terminating a failed pipeline 
Merge pull request !31332 from h.farahat/fix_reset
 2022-03-15 20:17:05 +00:00
i-robot 272d25b291
!31137 Test SkipFirstEpochSampler 
Merge pull request !31137 from zetongzhao/test_skip_first_epoch_sampler
 2022-03-15 18:26:28 +00:00
harshvardhangupta accc32da6c modify the at config file 2022-03-15 13:49:00 -04:00
i-robot 12d8906e2e
!30253 Support more ops for dump flag 
Merge pull request !30253 from huanghui/enhance-dump-flag
 2022-03-15 16:49:11 +00:00
hesham 3d16a4dee2 Fix bug in terminating a failed pipeline 2022-03-15 11:35:28 -04:00
i-robot 059c939854
!31269 Fix ut debug mode compile 
Merge pull request !31269 from qujianwei/master
 2022-03-15 12:35:38 +00:00
liuluobin f13d342986 Implementation of element wise parallel ops 2022-03-15 19:24:17 +08:00
i-robot 313a247ae8
!31178 upgrade Ascend package 11 Mar 22 
Merge pull request !31178 from shenwei41/upgrade_ascend_20220311
 2022-03-15 11:08:32 +00:00
i-robot 3be552ca39
!31156 test centos ut 
Merge pull request !31156 from luoyang/test_centos
 2022-03-15 09:38:15 +00:00
shenwei41 f1fde6a262 upgrade Ascend package 11 Mar 22 2022-03-15 16:31:27 +08:00
huangxinjing 31f55b6525 1. The main gol: Fix mixing inserting the AllReduce when where is no mirror appeared 
2. remove pattern match error as the origin pattern match will find no operator if there is only one parameter
 2022-03-15 15:54:06 +08:00
qujianwei d5eb5090d9 fix ut compile in debug mode 2022-03-15 11:04:37 +08:00
i-robot 01078763df
!31227 don't handle reshape when using adafactor optimizer in auto parallel 
Merge pull request !31227 from yangzhenzhang/adafactor-parallel-skip-handle-reshape
 2022-03-15 02:47:31 +00:00
huanghui 4482acf586 Support API set_dump for more type of ops 2022-03-15 10:24:15 +08:00
i-robot d4fc47ef0c
!31211 Clean iterators before forking 
Merge pull request !31211 from h.farahat/pytest_problem
 2022-03-15 01:47:28 +00:00
i-robot 4b15a46a49
!30858 Refactor cpu kernel factory. 
Merge pull request !30858 from TronZhang/refactor_cpu_factory
 2022-03-15 01:33:14 +00:00
yangzhenzhang 1f98ffb79c adafactor parallel skip handle reshape 2022-03-15 09:18:19 +08:00
i-robot 6be7c42052
!31191 [MD] more tests for failover reset 
Merge pull request !31191 from Mohammad Motallebi/failover_more_tests
 2022-03-14 21:50:37 +00:00
zetongzhao 9d4c8e29f9 add test for skip first epoch sampler 2022-03-14 15:01:08 -04:00
h.farahat 318e6b8803 fix segfault 2022-03-14 14:55:21 -04:00
mohammad 583eb32723 [MD] Add more UT and a ST for failover reset 2022-03-14 13:52:09 -04:00
luoyang 74d8969b91 ut failed with pillow>=9.0.0, need more validation 2022-03-14 23:58:09 +08:00
i-robot a33611ce27
!31192 Support sync and async message sending mode for TCP client 
Merge pull request !31192 from chengang/add_send_sync
 2022-03-14 07:55:49 +00:00
i-robot dcb5cd670c
!30953 Dynamic Weight Decay 
Merge pull request !30953 from wanyiming/dynamic_wd
 2022-03-14 06:03:24 +00:00
i-robot c2c02b094a
!31204 modify virtualdataset bug for master 
Merge pull request !31204 from lilei/modify_virtualdataset_for_master
 2022-03-14 03:33:25 +00:00
TronZhang db2931516e refactor cpu factory 2022-03-14 10:05:56 +08:00
i-robot c62112ba89
!30938 mindspore support compile with gcc-9 and gcc-8 
Merge pull request !30938 from xulei/ms_gcc-9
 2022-03-14 01:12:25 +00:00
lilei 690c58ebcf modify virtualdataset bug for master 2022-03-13 10:49:15 +08:00
Parallels 6f31f270e9 support sync and async message sending mode for tcp client 2022-03-12 21:14:35 +08:00
i-robot 8b7d7a1e1a
!31141 [MD][Offload] Offload RandomSharpness Op Pure Sharpness Support & UT 
Merge pull request !31141 from alashkari/fix-random-sharpness
 2022-03-11 21:06:13 +00:00
Rescue face5e51a0 Added sharpness without randomness support and sharpnes op ut. 
fix pad issue
 2022-03-11 17:30:49 +00:00
i-robot 603ff25afb
!31177 Add ut for fallback python buildin function with tensor input 
Merge pull request !31177 from LiangZhibo/syntax
 2022-03-11 08:38:53 +00:00
i-robot 3c0e34ada0
!30855 Support user-defined classes by ms_class decorators 
Merge pull request !30855 from huangbingjian/ms_class_dev
 2022-03-11 08:37:12 +00:00
xulei bb10249d49 support gcc-8 and gcc-9 2022-03-11 16:12:41 +08:00
liangzhibo a3f183bdad Add test case for fallback buildin function with constant tensor 2022-03-11 15:35:29 +08:00
i-robot f3b753807b
!30929 Add AkgKernel for lite_adapter 
Merge pull request !30929 from DeshiChen/0307_akgkernel
 2022-03-11 07:01:10 +00:00
i-robot 3375003da9
!31158 Add registration and heartbeat message from compute graph node to meta server node 
Merge pull request !31158 from chengang/add_pb_to_node
 2022-03-11 06:55:03 +00:00
i-robot c2a5cc1486
!31040 Produce parallel operators for Argmin/max, SquareSumAll and UnsortedSegmentProd 
Merge pull request !31040 from Bert0108/reduce_operators_arg
 2022-03-11 06:15:56 +00:00
shenyu 443da2280d [FIX][BUG][MD][FUNC] RandomInvert 2022-03-11 11:53:04 +08:00
Parallels 5fe6d42fe1 Add registration and heartbeat message from compute graph node to meta server node 2022-03-11 11:42:08 +08:00
shenyu c14a8084c7 [FIX][BUG][MD][FUNC] RandomAuoContrast 2022-03-11 10:29:49 +08:00
i-robot a064d0855b
!30728 add ci test for unify backend 
Merge pull request !30728 from xiaoyao/master
 2022-03-11 01:50:03 +00:00
i-robot eecf6db426
!31121 [MD] revert OBSMindDataset 
Merge pull request !31121 from liyong126/revert_OBSMindDataset
 2022-03-11 01:21:10 +00:00
i-robot 216e7c6a92
!31041 add check for conv2d 
Merge pull request !31041 from yangzhenzhang/add-check-for-conv2d
 2022-03-11 01:20:52 +00:00
i-robot 67d10ce3be
!31066 Fix a bug where ROIAlign and CropAndResize distributed op do not support GPU 
Merge pull request !31066 from liuluobin/fix_roialign
 2022-03-11 01:16:46 +00:00
huangbingjian 4f7c9cb27f Support user-defined classes through ms_class decorators. 2022-03-10 20:12:54 +08:00
i-robot 99910ef27f
!31088 convert C++ data to python when run fallback function 
Merge pull request !31088 from lianliguang/add-fallback-builtin-function
 2022-03-10 11:56:01 +00:00
yangzhenzhang c00d29f223 rebase 2022-03-10 19:17:06 +08:00
liyong 50fc6f5736 Revert "!30810 [MD] add OBSMindDataset " 
This reverts commit 9fcab9184e, reversing
changes made to afb33fff62.
 2022-03-10 19:10:11 +08:00
dayschan 601fb25cb7 Add AkgKernel for lite_adapter 
Implements a temporary solution of InferShape for GraphKernel ops by
saving shape/type/format in custom attributes and decode them in InferShape function.
 2022-03-10 18:19:28 +08:00
liuluobin 8f045d02e3 Fix a bug where ROIAlign and CropAndResize distributed op do not support GPU 2022-03-10 17:31:36 +08:00
lianliguang d9390afad1 convert MS C++ dtype to MS python 2022-03-10 16:09:34 +08:00
xiao_yao1994 94df0869ad uni_backend: add ut ge test 2022-03-10 16:08:36 +08:00
i-robot 872cb74d3f
!30982 Vocab C++ Interface Alignment and SentencePieceVocab C++ interface alignment and Python interface refactoring 
Merge pull request !30982 from 刘勇琪/master-vocab-sentencepiecevocab
 2022-03-10 07:58:41 +00:00
i-robot 9fcab9184e
!30810 [MD] add OBSMindDataset 
Merge pull request !30810 from liyong126/add_obs_mindrecord_dataset
 2022-03-10 07:28:46 +00:00
Bert0108 bfc5e4345c add distributed operators for argmax/min sqauresumall and unsortedsetmentprod 2022-03-10 14:44:37 +08:00
liu-yongqi-63 457ee03082 Vocab C++ Interface Alignment and SentencePieceVocab C++ interface alignment and Python interface refactoring 2022-03-10 11:33:00 +08:00
i-robot 662a7f8408
!31044 add cpp ut for combine_like_graphs 
Merge pull request !31044 from huanghui/add-ut
 2022-03-10 03:28:56 +00:00
liyong 10ada68fa9 add OBSMindDataset 2022-03-10 11:18:59 +08:00
i-robot 059ac45ae8
!30920 Call Draw function directly 
Merge pull request !30920 from huanghui/fix-draw
 2022-03-10 03:15:56 +00:00
wanyiming a124ec4de7 add dynamic_decay 2022-03-10 11:02:27 +08:00
huanghui 1077601e26 add cpp ut for jit_config 2022-03-10 10:56:18 +08:00
i-robot b32daf3cb4
!31000 add fallback example of function 
Merge pull request !31000 from lianliguang/add-fallback-builtin-function
 2022-03-10 01:57:37 +00:00
i-robot 3f1a7f45bb
!30942 Add hwc_to_chw operation on lite 
Merge pull request !30942 from shenwei41/hwc_to_chw
 2022-03-10 01:21:54 +00:00
i-robot f57f862d7b
!30782 [MD] pushdown skip node in MD pipeline 
Merge pull request !30782 from Mohammad Motallebi/failover_pushdown_skipop
 2022-03-10 01:12:46 +00:00
mohammad 260cebf650 [MD] skip node pushdown optimization pass for Reset 2022-03-09 09:52:25 -05:00
shenwei41 7f7bae8bfe add hwc_to_chw operation on lite 2022-03-09 15:43:13 +08:00
i-robot 698b99aaa3
!18945 [assistant][ops]New operator implementation, include OmniglotDataset 
Merge pull request !18945 from Wangsong95/omniglot_dataset
 2022-03-09 06:51:15 +00:00
lianliguang f3b1e3291c add jit fallback example 2022-03-09 11:25:03 +08:00
i-robot cfda27a6c7
!30985 Add TCP communication module into compute graph nodes and meta server node 
Merge pull request !30985 from chengang/topology_2
 2022-03-09 01:51:23 +00:00
Parallels e2125108b8 Add tcp communication into compute graph nodes and meta server node 2022-03-09 00:02:25 +08:00
qkeys ce7597f500 [feat][assistant][I3J6VB] add new data operator OmniglotDataset 2022-03-08 21:36:22 +08:00
i-robot 0b207fce31
!29735 unique support multi_dim tensor 
Merge pull request !29735 from jiangzhenguang/unique_multi_tensor
 2022-03-08 08:30:57 +00:00
i-robot 63d7254bd3
!30952 add encryption to lite 
Merge pull request !30952 from yeyunpeng2020/cryption
 2022-03-08 07:30:41 +00:00
i-robot f73987dcf2
!30949 Change the Parameter type name from 'ParameterTensor' to 'Parameter' 
Merge pull request !30949 from 张清华/opt
 2022-03-08 06:17:31 +00:00
jiangzhenguang 28f89f36b4 unique support multi-dim tensor 2022-03-08 14:09:19 +08:00
Zhang Qinghua 973008ebcf Change the Parameter type name from 'ParameterTensor' to 'Parameter' 2022-03-08 11:23:56 +08:00
i-robot 82bc2b722d
!30417 add history and lambda callbacks 
Merge pull request !30417 from liutongtong9/add_callbacks
 2022-03-08 02:17:46 +00:00
i-robot 75e1939f50
!30899 Enable python buildin function type for graph fallback and add some ut for python buildin function for graph fallback 
Merge pull request !30899 from LiangZhibo/syntax
 2022-03-08 01:35:47 +00:00
i-robot 64544b142a
!30340 develop maxpool3d, avgpool3d and conv3d on CPU(including farward and backward) 
Merge pull request !30340 from zhangbuxue/develop_conv3d_maxpool3d_and_avgpool3d_on_cpu
 2022-03-08 01:31:54 +00:00
yeyunpeng2020 f670a635f0 add encryption to lite 2022-03-08 09:15:10 +08:00
i-robot 2655d64720
!30865 [MD] Modify Batch number of workers with Python multiprocessing 
Merge pull request !30865 from cathwong/ckw_batch_workers_pymultip
 2022-03-07 18:21:16 +00:00
Cathy Wong bc29dd9d72 [MD] Modify Batch number of workers with Python multiprocessing 2022-03-07 10:39:05 -05:00
i-robot 335ef1c270
!30459 Add ut validate function for parallel 
Merge pull request !30459 from liuluobin/ut_master
 2022-03-07 15:23:14 +00:00
buxue 50f3ad00ab develop maxpool3d, avgpool3d and conv3d on CPU(including forward and backward) 2022-03-07 20:44:38 +08:00
i-robot 4a9a65d2a6
!30919 Produce parallel operators for ReduceAll and ReduceProd 
Merge pull request !30919 from Bert0108/reduce_operators_merge
 2022-03-07 11:30:37 +00:00
liuluobin b797a410cc Add validate function for parallel ut 2022-03-07 16:19:52 +08:00
huanghui ce0f500a3f convert draw_ to call Draw function derectly 2022-03-07 16:12:52 +08:00
Bert0108 dfc92f1791 add distributed parallel operators for reduceall and reduceprod 2022-03-07 15:40:11 +08:00
liangzhibo 1a81031d92 Add testcases for fallback python buildin function 2022-03-07 14:20:24 +08:00
lianliguang 55e7f2d761 modify DynamicShape to TensorShape 2022-03-07 14:19:12 +08:00
i-robot 960d551b8f
!30889 parse core type from pre-build for tbe kernel compile 
Merge pull request !30889 from liubuyu/master
 2022-03-07 03:53:48 +00:00
i-robot a2303a92bd
!30741 [dataset][dfx] 易用性-Python多进程统一整改专项 
Merge pull request !30741 from xiefangqi/md_add_multiprocessing_optimization
 2022-03-07 01:29:21 +00:00
i-robot c4cc0e6f26
!30742 auto parallel support not ony power of 2 device numbers 
Merge pull request !30742 from yao_yf/auto_parallel_support_not_only_2_power
 2022-03-07 01:09:01 +00:00
yao_yf b60e54e0d5 support not only power of 2 2022-03-06 18:08:46 +08:00
lby 2b1b539b36 parse core type from pre-build for tbe kernel compile 2022-03-06 15:23:50 +08:00
xiefangqi 2de6c8908b add a clean up subprocess to clean subprocess when main process exit 2022-03-05 17:24:11 +08:00
liutongtong 7afcdfd211 add history and lambda callbacks 2022-03-04 17:24:57 +08:00
i-robot fdf7aebd78
!30746 Fix recv actor stuck issue. 
Merge pull request !30746 from ZPaC/sync-route-table
 2022-03-04 03:01:03 +00:00
i-robot b90cf43562
!30553 Support dataset reset() to recover after failure 
Merge pull request !30553 from h.farahat/reset
 2022-03-04 02:19:45 +00:00
i-robot f01d841113
!27943 [MD][Autotune] Save/Load Autotune config 
Merge pull request !27943 from harshvardhangupta/save_load_at_config
 2022-03-03 20:42:23 +00:00
i-robot 7b26a32e98
!22608 [assistant][ops]New operator implementation, include KITTIDataset 
Merge pull request !22608 from Wangsong95/kitti_dataset
 2022-03-03 07:05:40 +00:00
h.farahat a3dec34833 Dataset failover reset 2022-03-02 23:53:56 -05:00
i-robot 1eb2975ac2
!30753 Support random port for TCP server 
Merge pull request !30753 from chengang/support_random_port
 2022-03-03 02:08:28 +00:00
harshvardhangupta bd61adbb72 Implement save/load of autotune dataset pipeline configuration #27943 2022-03-02 15:08:08 -05:00
i-robot 70e61e9014
!30781 [MD] test_pyfunc_multiprocess_autotune.py - enable AutoTune 
Merge pull request !30781 from cathwong/ckw_ut_fixup_map_python_multiproc
 2022-03-02 19:50:29 +00:00
i-robot edcc6b790d
!30714 [MD][Offload] Add TypeCast op to offload 
Merge pull request !30714 from markuskunej/offload_typecast
 2022-03-02 19:48:58 +00:00
Cathy Wong eb931c7f88 [MD] test_pyfunc_multiprocess_autotune.py - enable AutoTune 2022-03-02 13:25:35 -05:00
i-robot 4aa82cc21e
!30679 [MD] Add tests for Python Multiprocessing with AutoTune 
Merge pull request !30679 from cathwong/ckw_ut_map_python_multiproc
 2022-03-02 14:26:09 +00:00
Parallels 5e09380eb3 add random port support for tcp server 2022-03-02 21:27:33 +08:00
zx 93617ce91e [feat][assistant][I3J6VO] add new data operator KITTI 2022-03-02 19:29:17 +08:00
ZPaC 4c74b600ea Fix recv actor stuck issue. 2022-03-02 19:26:45 +08:00
i-robot 6bc31df410
!30672 delete ut test libgraph and liberror_manager dependency 
Merge pull request !30672 from qujianwei/master
 2022-03-02 02:18:24 +00:00
Cathy Wong b151f90b8d [MD] Add tests for Python Multiprocessing with AutoTune 2022-03-01 20:20:59 -05:00
markuskunej 6de37045ff Added TypeCast op in dataset offload. 2022-03-01 21:55:27 +00:00
qujianwei 7e1aebc2c9 delete ut libgraph.so and liberror_manager.so dependency 2022-03-01 20:50:45 +08:00
wangjun 46612fabfb add st for shard 2022-03-01 15:32:37 +08:00
i-robot ad9757ccf0
!30661 [Auto parallel] [MoE] Fix an error of configuring MoE parallel 
Merge pull request !30661 from Xiaoda/124-moe-changes
 2022-03-01 02:26:37 +00:00
i-robot 4367377000
!30611 Fix rpc route bugs 
Merge pull request !30611 from ZPaC/sync-route-table
 2022-02-28 15:50:35 +00:00
ZPaC dc80f9b8d1 Fix rpc route bugs 2022-02-28 19:18:42 +08:00
i-robot 789c1d6bd3
!30614 [AutoParallel] Fix Overflow As the cast is inserted before Mirror 
Merge pull request !30614 from huangxinjing/fix_cast_error
 2022-02-28 09:04:46 +00:00
Xiaoda Zhang 81e5abe580 fix an error of configuring parallel 2022-02-28 17:03:53 +08:00
i-robot a92c54b206
!30496 [Fallback] Suppport scipy module. 
Merge pull request !30496 from huangbingjian/support_scipy
 2022-02-28 08:33:34 +00:00
i-robot a8686ae3d9
!18827 [assistant][ops]New operator implementation, include LFWDataset 
Merge pull request !18827 from Wangsong95/lfw_dataset
 2022-02-28 08:31:11 +00:00
zx 2132f62d98 [feat][assistant][I3J6VQ] add new data operator LFW 2022-02-28 13:43:05 +08:00
huangxinjing 896daee845 [AutoParallel]Fix insert error for the mirror 2022-02-26 17:54:42 +08:00
i-robot 330cacf905
!30573 Catch abnormals from sto functions in debug 
Merge pull request !30573 from maning202007/master
 2022-02-26 08:52:22 +00:00
i-robot de95400c52
!30587 check platform for resize bilinear 
Merge pull request !30587 from yangzhenzhang/check-platform-for-resize-bilinear
 2022-02-26 06:46:44 +00:00
i-robot 909183a2ef
!30317 extract common as an independent shared library 
Merge pull request !30317 from zhoufeng/backend-compile-independently
 2022-02-26 06:08:17 +00:00
i-robot 17accc12cf
!30238 [GraphKernel] replace nodes in lite and move const_input_to_attr pass. 
Merge pull request !30238 from chenlei_autodiff/lite_callback
 2022-02-26 03:56:57 +00:00
maning202007 9328041de4 Catch abnormals from sto functions in debug 
Fix type errors

Fix bugs

Fix cycle depth
 2022-02-26 11:03:52 +08:00
zhoufeng f49b195c39 extract common as an independent shared library 
Signed-off-by: zhoufeng <zhoufeng54@huawei.com>
 2022-02-26 10:55:51 +08:00
yangzhenzhang 43e6e16da3 check platform for resizebilinear 2022-02-26 09:11:32 +08:00
i-robot 5deccfe64b
!30260 [MD][AutoTune] Re-enable AT for non-sink models 
Merge pull request !30260 from danishfarid/re-enable_AT
 2022-02-25 22:26:14 +00:00
i-robot c4843c4085
!29958 upgrade ascend 20220211 
Merge pull request !29958 from shenwei41/upgrade_ascend_20220211
 2022-02-25 14:55:12 +00:00
shenwei41 ff75314719 upgrade ascend 20220211 2022-02-25 21:19:45 +08:00
chenlei_autodiff 9fabf8ae0d [GraphKernel] replace graph kernel node with custom in lite. 2022-02-25 17:09:24 +08:00
i-robot 0341d96dd6
!30469 add shard function to support part of the graph executed in auto_parallel under pynative mode 
Merge pull request !30469 from wangjun/0223_pp
 2022-02-25 06:52:24 +00:00
huangbingjian a69d13bc44 [Fallback] Suppport scipy module. 2022-02-25 10:13:43 +08:00
i-robot cfe0f76d2b
!30491 ut for allgather fusion 
Merge pull request !30491 from jiahongQian/master
 2022-02-25 01:30:24 +00:00
danishfarid 6c4697fc8b reenable AT for non-sink 
tests re-enable

code check fix 1

remove self

lint fixing

lint fix 2

remove test as per req
 2022-02-25 00:02:58 +00:00
wangjun 24d448239c add pynative_parallel 2022-02-24 21:07:51 +08:00
i-robot ac1463a192
!30298 Package the cuda operators as a dynamic link library 
Merge pull request !30298 from jinjiali-kali/cuda_ops
 2022-02-24 11:36:20 +00:00
i-robot 981eae461a
!30118 自动优化器并行特性 
Merge pull request !30118 from zhuyuxiao/I4S85V
 2022-02-24 11:09:44 +00:00
jiahongQian 25f57505bf ut for allgather fusion 2022-02-24 17:40:32 +08:00
i-robot bbcfbce9e0
!29997 [Auto parallel] [MoE] Support data_parallel + expert_parallel 
Merge pull request !29997 from Xiaoda/124-moe-changes
 2022-02-24 09:23:47 +00:00
zhuyuxiao d0e0e305d3 good 2022-02-24 16:10:18 +08:00
i-robot f2130e7434
!30483 [AutoParallel]Pipeline Automatic detection Opt 
Merge pull request !30483 from lichen/pipeline_opt_detection
 2022-02-24 03:40:15 +00:00
yao_yf e21f878e14 adasum ut fix 2022-02-24 09:31:04 +08:00
Xiaoda Zhang b714451937 implementing expert_parallel+data_parallel in MoE: 
1) extending _Linear's input as 4-dimension tensor: [outer_batch, expert_dim, -1, hidden], and _Liner's BatchMatMul becomes BatchMatMul(4_dim_tensor, 3_dim_tensor);
2) configuring the _Linear's BatchMatMul sharding strategy as [[dp, ep, 1, 1], [ep, 1, mp]];
3) introducing a new parameter 'expert_parallel' in TransformerOpParallelConfig, creating a new class MoEParallelConfig to include 'data_parallel', 'model_parallel' and 'expert_parallel';
4) changing parallel config for FeedForward, TransformerEncoderLayer, TransformerDecoderLayer.
 2022-02-24 09:24:27 +08:00
wangshengnan12@huawei.com acbefd80ea pipeline_opt_detection 2022-02-24 09:23:17 +08:00
i-robot 2a00ffd3b1
!30259 [MD]Update set_autotune_enable API to add save filepath 
Merge pull request !30259 from cathwong/ckw_at_save_api
 2022-02-24 01:00:59 +00:00
Cathy Wong 46e223e569 [MD] Update set_autotune_enable API to add save filepath 2022-02-23 14:50:55 -05:00
i-robot 81260a2319
!30466 takedown test_auto_parallel_adasum.py to ensure stability, again 
Merge pull request !30466 from yanghaoran/master
 2022-02-23 09:15:58 +00:00
i-robot 14393503b7
!30431 allreduce allgather fusion 
Merge pull request !30431 from jiahongQian/master
 2022-02-23 08:52:40 +00:00
yanghaoran 71d6b7d506 takedown test_auto_parallel_adasum.py to ensure stability, again 2022-02-23 16:42:13 +08:00
i-robot 2e8eac8341
!30367 auto_parallel_adasum_support_data_parallel 
Merge pull request !30367 from yao_yf/auto_parallel_adasum_support_data_parallel
 2022-02-23 06:50:26 +00:00
jiahongQian 8a2151d8bb allgather reducescatter fusion 2022-02-23 10:47:30 +08:00