Nigel
/
mindspore
forked from mindspore-Ecosystem/mindspore
0
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity
56,551 Commits 24 Branches 0 Tags 3.9 GiB
Commit Graph

651 Commits

Author SHA1 Message Date
Zhang Qinghua f90dcc963d Support isolated side-effect nodes in constant returned func graph. 2022-07-27 10:12:24 +08:00
wangshengnan12@huawei.com 937e65e81a clean_code_auto_parallel 2022-07-21 15:06:05 +08:00
zhangyinxia 3e1320b246 add lamb and clipbynorm pass for ge 2022-07-20 10:54:06 +08:00
yangzhenzhang 25009f9051 modify gather op 2022-07-07 14:59:50 +08:00
i-robot 58c901923a
!36767 modify the generated strategy for conv2d 
Merge pull request !36767 from yangzhenzhang/modify-conv2d-gen-strategy-func
 2022-06-29 01:39:29 +00:00
wzw d309dfd41e add transformer testcase 2022-06-28 20:01:29 +08:00
yangzhenzhang 9b50d2cfee modify conv2d gen strategy func 2022-06-28 16:32:15 +08:00
wangshengnan12@huawei.com bdf2004ca4 fix_pipeline_compile_error 2022-06-28 09:55:31 +08:00
huangxinjing 055ab431db [AutoParallel]Add virtual mul 
Fix OneHot Error
Add onehot test cast
 2022-06-25 09:06:40 +08:00
huangxinjing d0f2bf37c6 Add cross entory loss 
[AutoParallel]Add sub graph support for parallel training
Fix forward mark error and virtual dataset error
Add Test cases
Add virtual dataset inserts, forbiden the subgraphs as candidates
 2022-06-20 10:53:21 +08:00
i-robot e902c7c731
!35839 fix bug of moe 
Merge pull request !35839 from bichaoyang/master
 2022-06-17 08:15:25 +00:00
yangzhenzhang b63e70f6a9 gen data parallel strategy for conv2d auto parallel 2022-06-16 16:44:30 +08:00
bichaoyang 68398b054a fix bug of moe 2022-06-14 20:04:27 +08:00
yangzhenzhang e5b62b2714 modify gen strategy for matmul op 2022-06-09 19:50:31 +08:00
i-robot 6f18373b10
!35252 add python implement and st for random gamma 
Merge pull request !35252 from zhangqi/0522_2
 2022-06-02 09:26:10 +00:00
zhangqi d0afaf60f0 add python implement and st for random gamma 2022-06-01 11:20:36 +08:00
yangzhenzhang 3df7174d25 modify gen strategy for parallel op 2022-05-31 18:41:42 +08:00
i-robot 291b5faa98
!33925 parallel_initializer_seed 
Merge pull request !33925 from yao_yf/auto_parallel_initializer_seed
 2022-05-25 02:03:16 +00:00
i-robot 4f0b1dc60e
!34511 Add CPU support for KLDivLoss op 
Merge pull request !34511 from zhuyuxiao/I51VMV
 2022-05-23 11:06:37 +00:00
yao_yf 91dabdce17 parallel initializer seed set 2022-05-23 14:24:29 +08:00
zhuyuxiao ce16351d16 add CPU support for KLDivLoss op 2022-05-20 15:34:31 +08:00
i-robot 71f2ee7172
!34555 [AutoParallel]Parallel_adapt_simple_control_flow 
Merge pull request !34555 from lichen/adapt_control_flow
 2022-05-20 01:31:48 +00:00
i-robot 527539ef7c
!34409 unsorted segment prod gpu inference 
Merge pull request !34409 from ling/core
 2022-05-20 01:20:54 +00:00
i-robot 67033817ff
!34295 Optimiz lamb ascend kernel 
Merge pull request !34295 from wanghenchang/lamb-optimiz-ascend
 2022-05-20 01:20:25 +00:00
wangchangheng 6c879b5554 optimize lamb 2022-05-19 19:18:16 +08:00
ling 437deb9da2 unsorted segment prod gpu inference 2022-05-19 18:59:19 +08:00
Zhang Qinghua d7762f1c8d Add bprop_return_sparse flag for sparse bprop primitive and remove context option: 'enable_sparse'. 2022-05-19 17:31:57 +08:00
wangshengnan12@huawei.com 48065b72a1 adapt_control_flow 2022-05-19 15:50:29 +08:00
bichaoyang 98d4b75d1b reminder when missing 'pipeline_end' 2022-05-16 11:11:00 +08:00
i-robot 6c78153ef8
!33948 add softmax mul parallel tests 
Merge pull request !33948 from wanhanyang/new_softmax
 2022-05-06 09:00:41 +00:00
i-robot 3409f924bc
!33947 Add softmax gather parallel tests 
Merge pull request !33947 from wanhanyang/test_softmax_gather
 2022-05-06 08:31:38 +00:00
i-robot 8f565f098e
!33904 use _inner_ops shard instand of array_ops range 
Merge pull request !33904 from yanzhenxiang2020/range_shard_inner
 2022-05-06 08:10:46 +00:00
Wan Hanyang 2cf05811b8 add test softmax gather net tests 2022-05-06 14:59:27 +08:00
Wan Hanyang d16b2c15b1 Add mul softmax net tests 2022-05-06 14:53:30 +08:00
i-robot 1b01960935
!33923 回退 'Pull Request !33791 : Add gather with softmax parallel tests' 
Merge pull request !33923 from suteng/revert-merge-33791-master
 2022-05-06 05:42:18 +00:00
suteng 5c9f240735
回退 'Pull Request !33791 : Add gather with softmax parallel tests' 2022-05-06 03:20:35 +00:00
suteng 9c5d75d416
回退 'Pull Request !33790 : Add parallel fused softmax parallel tests' 2022-05-06 03:20:18 +00:00
yanzhenxiang2020 04c5a582da use _inner_ops shard instand of array_ops range 2022-05-06 10:36:57 +08:00
i-robot 43fd864c10
!31745 Enable if parallel call flag by default 
Merge pull request !31745 from xychow/enable-if-parallel-call-by-default
 2022-05-05 09:28:39 +00:00
i-robot 3d1060dcdb
!33790 Add parallel fused softmax parallel tests 
Merge pull request !33790 from wanhanyang/softmax2
 2022-04-29 13:02:50 +00:00
Su Teng 1686559b0d add gather with softmax parallel tests 2022-04-29 18:18:01 +08:00
Su Teng 1e1eaedec7 add parallel fused softmax tests 2022-04-29 18:14:43 +08:00
zhousiyi d4aba77d3d enable if parallel call transformation by default; fix test case ut/python/pipeline/parse/test_fix_bug.py::test_fix_expanddims_loss_scale 
Check only if all branch of middle graph's output are scalar, parallel_if transformation should be ignored

fix test_auto_monad.py::test_while_if geswitch input shape check failed

fix ut/python/pipeline/parse/test_super.py::test_super_cell core dump as varg and kwarg parameter are removed but those flags are not set, it will cause FuncGraph::GetPositionArgsCount be negative

skip test_control_flow in ut/python/paralle as Parallel team is working on contro flow adaption.

fix ut/python/ir/test_row_tensor.py::test_row_tensor_with_control_flow_if

fix for kwonlyargs when eliminate_parameter and add testdases

fix cannot join for st/control/inner/test_012_if_in_for.py

fix pclint errors

fix test/st/scipy_st/sparse/test_linalg.py, if the output cnode of former funcgraph is Depend CNode with input 1 as middle funcgraph call, then keep the output of former funcgraph as Depend other than middle funcgraph tail call.

log as exception

If the output of middle funcgraph contain PyInterpret CNode, then ignore
the transformation as the Value in Abstract will be kAnyValue after join
of the PyInterpret CNode value.

Backend cannot support for RealDiv with Int64, so change one operand to Float
 2022-04-29 08:22:56 +00:00
wangshengnan123 f7949289e6 rm constant sharding 2022-04-14 11:15:42 +08:00
i-robot e09a674bcc
!32529 Fix optimizer limit for the Parallel Optimizer 
Merge pull request !32529 from huangxinjing/fix_model_limit
 2022-04-06 03:39:36 +00:00
huangxinjing b16dbf2b5d 1. Fix optimizer check error, as the check is done by the class name, too naive 
2. Add virutal assign add to the operator
 2022-04-05 20:16:03 +08:00
i-robot acce047dfe
!32341 add parallel ops about Invert CheckValid PopulationCount 
Merge pull request !32341 from yangzhenzhang/add-parallel-operators
 2022-04-01 02:53:58 +00:00
i-robot 72414ff8d9
!31941 modify Strided_slice for master 
Merge pull request !31941 from lilei/modify_stridedslice_for_master
 2022-03-31 09:11:24 +00:00
yangzhenzhang c927da3b41 add parallel operators 2022-03-31 15:11:25 +08:00
i-robot 9810fa53cb
!31899 [Auto-Par] [D-Rec] Add Mem & Redis coefficient on D-Rec cost model for Pangu-alpha 
Merge pull request !31899 from FRHW-WANG/D-Rec-deliver
 2022-03-31 02:12:26 +00:00
haoran.wang 3674e1d713 Add Mem & Redis coefficient for PanGu-alpha 2022-03-29 12:00:34 +00:00
i-robot e897c98b1f
!31988 Implementation of SquaredDifferenceInfo, ErfinvInfo, MaskedFillInfo, SplitVInfo, GammaInfo, KLDivLossInfo and LinSpaceInfo. 
Merge pull request !31988 from liuluobin/ops_impl
 2022-03-29 08:08:32 +00:00
liuluobin 6f914b8b3c Implementation of SquaredDifferenceInfo, ErfinvInfo, MaskedFillInfo, SplitVInfo, GammaInfo, KLDivLossInfo and LinSpaceInfo 2022-03-29 09:30:21 +08:00
i-robot a9cdbd5ae8
!32005 fix bugs of moe 
Merge pull request !32005 from bichaoyang/master
 2022-03-29 01:30:10 +00:00
b00518648 93da6bab46 fix bugs of moe: only use a fewer dp in moe 2022-03-28 17:22:48 +08:00
lilei 452362332e xmodify stridedslice for master 2022-03-28 09:24:13 +08:00
wangjun 789539cbaa modify interface name for shard 2022-03-26 09:10:14 +08:00
haoran.wang fdfbe2dedc Modify the name of the funtions and variables of Parameter shared User strategy treatment 2022-03-24 06:16:55 +00:00
i-robot c2212f88b4
!31164 Fix the global norm missing insert allreduce 
Merge pull request !31164 from huangxinjing/fx_global_norm_error
 2022-03-16 06:46:13 +00:00
i-robot bf03f0e030
!31252 Implementation of element wise parallel ops 
Merge pull request !31252 from liuluobin/element_wise_ops
 2022-03-16 01:32:29 +00:00
liuluobin f13d342986 Implementation of element wise parallel ops 2022-03-15 19:24:17 +08:00
huangxinjing 31f55b6525 1. The main gol: Fix mixing inserting the AllReduce when where is no mirror appeared 
2. remove pattern match error as the origin pattern match will find no operator if there is only one parameter
 2022-03-15 15:54:06 +08:00
yangzhenzhang 1f98ffb79c adafactor parallel skip handle reshape 2022-03-15 09:18:19 +08:00
i-robot dcb5cd670c
!30953 Dynamic Weight Decay 
Merge pull request !30953 from wanyiming/dynamic_wd
 2022-03-14 06:03:24 +00:00
lilei 690c58ebcf modify virtualdataset bug for master 2022-03-13 10:49:15 +08:00
i-robot c2a5cc1486
!31040 Produce parallel operators for Argmin/max, SquareSumAll and UnsortedSegmentProd 
Merge pull request !31040 from Bert0108/reduce_operators_arg
 2022-03-11 06:15:56 +00:00
i-robot 216e7c6a92
!31041 add check for conv2d 
Merge pull request !31041 from yangzhenzhang/add-check-for-conv2d
 2022-03-11 01:20:52 +00:00
yangzhenzhang c00d29f223 rebase 2022-03-10 19:17:06 +08:00
liuluobin 8f045d02e3 Fix a bug where ROIAlign and CropAndResize distributed op do not support GPU 2022-03-10 17:31:36 +08:00
Bert0108 bfc5e4345c add distributed operators for argmax/min sqauresumall and unsortedsetmentprod 2022-03-10 14:44:37 +08:00
wanyiming a124ec4de7 add dynamic_decay 2022-03-10 11:02:27 +08:00
i-robot 335ef1c270
!30459 Add ut validate function for parallel 
Merge pull request !30459 from liuluobin/ut_master
 2022-03-07 15:23:14 +00:00
liuluobin b797a410cc Add validate function for parallel ut 2022-03-07 16:19:52 +08:00
Bert0108 dfc92f1791 add distributed parallel operators for reduceall and reduceprod 2022-03-07 15:40:11 +08:00
yao_yf b60e54e0d5 support not only power of 2 2022-03-06 18:08:46 +08:00
wangjun 46612fabfb add st for shard 2022-03-01 15:32:37 +08:00
i-robot ad9757ccf0
!30661 [Auto parallel] [MoE] Fix an error of configuring MoE parallel 
Merge pull request !30661 from Xiaoda/124-moe-changes
 2022-03-01 02:26:37 +00:00
Xiaoda Zhang 81e5abe580 fix an error of configuring parallel 2022-02-28 17:03:53 +08:00
huangxinjing 896daee845 [AutoParallel]Fix insert error for the mirror 2022-02-26 17:54:42 +08:00
yangzhenzhang 43e6e16da3 check platform for resizebilinear 2022-02-26 09:11:32 +08:00
i-robot 0341d96dd6
!30469 add shard function to support part of the graph executed in auto_parallel under pynative mode 
Merge pull request !30469 from wangjun/0223_pp
 2022-02-25 06:52:24 +00:00
i-robot cfe0f76d2b
!30491 ut for allgather fusion 
Merge pull request !30491 from jiahongQian/master
 2022-02-25 01:30:24 +00:00
wangjun 24d448239c add pynative_parallel 2022-02-24 21:07:51 +08:00
i-robot 981eae461a
!30118 自动优化器并行特性 
Merge pull request !30118 from zhuyuxiao/I4S85V
 2022-02-24 11:09:44 +00:00
jiahongQian 25f57505bf ut for allgather fusion 2022-02-24 17:40:32 +08:00
i-robot bbcfbce9e0
!29997 [Auto parallel] [MoE] Support data_parallel + expert_parallel 
Merge pull request !29997 from Xiaoda/124-moe-changes
 2022-02-24 09:23:47 +00:00
zhuyuxiao d0e0e305d3 good 2022-02-24 16:10:18 +08:00
i-robot f2130e7434
!30483 [AutoParallel]Pipeline Automatic detection Opt 
Merge pull request !30483 from lichen/pipeline_opt_detection
 2022-02-24 03:40:15 +00:00
yao_yf e21f878e14 adasum ut fix 2022-02-24 09:31:04 +08:00
Xiaoda Zhang b714451937 implementing expert_parallel+data_parallel in MoE: 
1) extending _Linear's input as 4-dimension tensor: [outer_batch, expert_dim, -1, hidden], and _Liner's BatchMatMul becomes BatchMatMul(4_dim_tensor, 3_dim_tensor);
2) configuring the _Linear's BatchMatMul sharding strategy as [[dp, ep, 1, 1], [ep, 1, mp]];
3) introducing a new parameter 'expert_parallel' in TransformerOpParallelConfig, creating a new class MoEParallelConfig to include 'data_parallel', 'model_parallel' and 'expert_parallel';
4) changing parallel config for FeedForward, TransformerEncoderLayer, TransformerDecoderLayer.
 2022-02-24 09:24:27 +08:00
wangshengnan12@huawei.com acbefd80ea pipeline_opt_detection 2022-02-24 09:23:17 +08:00
i-robot 81260a2319
!30466 takedown test_auto_parallel_adasum.py to ensure stability, again 
Merge pull request !30466 from yanghaoran/master
 2022-02-23 09:15:58 +00:00
i-robot 14393503b7
!30431 allreduce allgather fusion 
Merge pull request !30431 from jiahongQian/master
 2022-02-23 08:52:40 +00:00
yanghaoran 71d6b7d506 takedown test_auto_parallel_adasum.py to ensure stability, again 2022-02-23 16:42:13 +08:00
i-robot 2e8eac8341
!30367 auto_parallel_adasum_support_data_parallel 
Merge pull request !30367 from yao_yf/auto_parallel_adasum_support_data_parallel
 2022-02-23 06:50:26 +00:00
jiahongQian 8a2151d8bb allgather reducescatter fusion 2022-02-23 10:47:30 +08:00
i-robot 5bee7156b9
!30369 add_virtualdataset_ut 
Merge pull request !30369 from lilei/add_virtualdataset_ut
 2022-02-23 02:21:34 +00:00
yao_yf 19236b1a70 auto parallel adasum support data parallel and hybrid parallel 2022-02-22 20:14:00 +08:00
huangxinjing 092ba035e3 Add global norm parallel support 2022-02-22 16:08:53 +08:00
yanghaoran bfe139b662 takedonw test_auto_parallel_adasum.py to ensure gate stability 2022-02-22 14:51:03 +08:00