Nigel
/
mindspore
forked from mindspore-Ecosystem/mindspore
0
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity
35,316 Commits 24 Branches 0 Tags 3.9 GiB
Commit Graph

450 Commits

Author SHA1 Message Date
yangzhenzhang 1b8eb283e4 modify batch parallel info 2021-09-13 16:28:56 +08:00
huangxinjing e02f553010 Fix spell error and add mode check 2021-09-09 18:50:53 +08:00
Xiaoda Zhang 5613c0b974 add a moe implementation: 
1) extend the Liner cell for including BatchMatMul implementation, in
which the first dimension indicates the expert number;
2) implement a Switch (top1) router;
3) implement a MoE cell, which extends the FeedForward cell.
 2021-09-09 11:53:42 +08:00
i-robot 77424eaad5 !23004 Add args Check for Transformer 
Merge pull request !23004 from huangxinjing/args_check
 2021-09-08 08:24:06 +00:00
huangxinjing 6cea07f749 Add args check 2021-09-08 09:12:35 +08:00
yao_yf 3ef26288a2 parallel_sparse_attention_ops_fix_repeated_cal 2021-09-06 19:57:22 +08:00
i-robot fa12d62d4d !21776 set device_id master 0813 
Merge pull request !21776 from mindspore_ding/set_device_id_master_0813
 2021-09-06 03:02:58 +00:00
dingpeifei b4bc6000dc set device id master 0813 2021-09-02 20:51:50 +08:00
i-robot d87d0e07c2 !22255 recompute_interface_modify 
Merge pull request !22255 from yao_yf/recompute_interface_modify
 2021-09-01 08:09:36 +00:00
yao_yf 39055af6e4 recompute interface modify 2021-09-01 10:59:59 +08:00
i-robot cc8d614b25 !22650 fixed sparse attention modify 
Merge pull request !22650 from yao_yf/fixed_sparse_attention_modify
 2021-09-01 01:29:45 +00:00
i-robot 389f3a6b6c !21835 make alltoall and neighborexchange to be interface && revert pr 21395 
Merge pull request !21835 from zhoufeng/revert-same-input-to-comm-op
 2021-08-31 08:53:46 +00:00
yao_yf 82889ec56b fixed sparse attention 2021-08-31 16:11:56 +08:00
yao_yf 68dd138462 add parallel sparse attention ops: dsd_matmul 2021-08-30 18:46:52 +08:00
zhoufeng ecae690a19 Revert "fix same node is used by two comm op" 
This reverts commit b09d411dc4.

add AlltoAll and NeighborExchange as interface
 2021-08-30 16:54:38 +08:00
yao_yf b8a9cbe2a3 add cus_matmul_dds parallel ops 2021-08-30 09:44:48 +08:00
Zhang Qinghua a137fa1d0b Optimize the Executors routines. 
- Fix the key generating.
- Distinguish the executors.
 2021-08-28 17:36:12 +08:00
zhihenghu ce12c02343 Add Sparse Attention 
adjut the file structure and name

Deleted extra information

Do some formatting work

Add test case and fix some document

fix imports
 2021-08-28 11:50:58 +08:00
i-robot e6e1f37ae4 !22346 [Core] Fix the bug of scope setting when cloning nodes 
Merge pull request !22346 from Xiaoda/86-fix-the-fullname-scope-bug
 2021-08-26 13:04:15 +00:00
i-robot 8d00a8d803 !22360 Fix Transformer Mirror Error 
Merge pull request !22360 from huangxinjing/fix_transformer_mirror_error
 2021-08-26 08:16:33 +00:00
Xiaoda Zhang b2703879c6 fix the scope setting error when cloning nodes 2021-08-26 10:25:38 +08:00
i-robot edcbb68d71 !22386 fix neighborexchange empty input case 
Merge pull request !22386 from zhoufeng/fix-neighbor-empty-input-bak
 2021-08-26 01:52:10 +00:00
zhoufeng e5a1582e4b fix neighborexchange empty input case 
Signed-off-by: zhoufeng <zhoufeng54@huawei.com>
 2021-08-25 21:41:22 +08:00
huangxinjing 62496d75f3 less the interface exposed 2021-08-25 21:22:40 +08:00
yangzhenzhang 0b9b2a9458 add test cases 2021-08-25 17:22:20 +08:00
lichenever 5812076512 Refactor_part_of_pipeline 2021-08-24 21:37:41 +08:00
yangzhenzhang f1afaeac5a modify check strategy for conv2d 2021-08-24 16:06:16 +08:00
ms_yan 36a8886ca2 Revert "[feat] [assistant] [I3T96T] add new Dataset operator CMUARCTICDataset" 
This reverts commit b077aa1cab.

Revert "[feat] [assistant] [I3T96X] add new Dataset operator LibriSpeechDataset"

This reverts commit 4e6f7dc97d.

delete pass_registry_test.cc

comment  hiai_nlu_model_multi.pb related  line
 2021-08-23 01:46:38 +08:00
djc b077aa1cab [feat] [assistant] [I3T96T] add new Dataset operator CMUARCTICDataset 2021-08-22 16:26:45 +08:00
djc 4e6f7dc97d [feat] [assistant] [I3T96X] add new Dataset operator LibriSpeechDataset 2021-08-22 13:39:37 +08:00
huangxinjing d777742904 1. Move the class to mindspore.parallel, support activation sharding 2021-08-20 14:34:10 +08:00
i-robot dde05c36b8 !21551 auto_parallel_context interface dataset_strategy union 
Merge pull request !21551 from yao_yf/auto_parallel_context_dataset_strategy_union
 2021-08-18 09:03:18 +00:00
i-robot 415275ae17 !21805 support adafactor model parallel 
Merge pull request !21805 from yangzhenzhang/auto-parallel-support-adafactor-opt
 2021-08-18 07:20:39 +00:00
i-robot 0d839fa7c6 !21809 Improved Transformer Struture and Add Args Check 
Merge pull request !21809 from huangxinjing/transformer_improved
 2021-08-17 12:05:25 +00:00
i-robot a77a0b968d !21761 comm_recompute_interface. 
Merge pull request !21761 from yao_yf/comm_recompute_interface
 2021-08-17 09:14:29 +00:00
yangzhenzhang 7ca64d2235 auto parallel support adafactor opt 2021-08-17 10:36:42 +08:00
yao_yf 5277b229be add cell comm recompute interface 2021-08-17 09:23:21 +08:00
huangxinjing 18044aff0f 1. Add docstring, elimitate attention mask, tuple append the deocoder return layer past 
split attention inputs from two into three

Add input check and modify test case

Fix check error and add check to transformer inputs

add input check to the transformers
 2021-08-16 21:08:48 +08:00
yao_yf a83bf73298 union auto_parallel_context interface dataset_strategy 2021-08-16 10:54:46 +08:00
yao_yf e233880e41 fix reshape depend reshape in auto parallel 2021-08-12 11:45:28 +08:00
i-robot 63445ff6fd !21627 alltoall exception handle 
Merge pull request !21627 from zhoufeng/alltoall_exception
 2021-08-12 01:40:22 +00:00
yangzhenzhang d18c813ee4 check strategy for conv2d 2021-08-11 19:12:48 +08:00
zhoufeng 03a56f2bb0 alltoall exception handle 
Signed-off-by: zhoufeng <zhoufeng54@huawei.com>
 2021-08-11 17:30:49 +08:00
i-robot 4aaa8126a0 !21528 Add Parallel Print Support 
Merge pull request !21528 from huangxinjing/add_print
 2021-08-09 10:55:04 +00:00
huangxinjing 92bad162bd Add print 2021-08-09 11:14:25 +08:00
huangbingjian 53b31abf12 remove useless depend 2021-08-05 20:21:24 +08:00
yangzhenzhang ef0361a449 fix bugs for conv2d 2021-08-03 10:48:05 +08:00
Xiaoda Zhang 4b4b3cdaf4 add reduceany operator and extend onehot to multi-dimensions 2021-08-02 08:53:43 +08:00
huangxinjing 615d1a179d Add transformer layer 2021-07-29 11:10:43 +08:00
i-robot 9f296c58d6 !20960 [AutoParallel]Add replace graph for conv2d 
Merge pull request !20960 from lichen/add_replace_graph_for_conv2d
 2021-07-28 07:06:41 +00:00
lichenever a7f8024c29 add_replace_graph_for_conv2d 2021-07-28 11:02:21 +08:00
yao_yf dc7dc7d3fa dataset strategy set 2021-07-27 18:44:10 +08:00
yangzhenzhang 80e5cc0e52 add parallel op for gatherd 2021-07-26 11:14:25 +08:00
Xiaoda Zhang bb5d4212f7 enable All2All in infering redistribution ops 2021-07-24 09:09:01 +08:00
lichenever 3c7cfb7c08 auto_parallel_support_control_flow 2021-07-22 17:31:31 +08:00
i-robot a7d40fc220 !20520 [AutoParallel]Add op AllToAllv 
Merge pull request !20520 from lichen/add_op_AllToAllv
 2021-07-20 07:57:03 +00:00
lichenever 8c1998fd6b add_op_AllToAllv 2021-07-20 10:49:28 +08:00
i-robot c9d3c1d346 !20411 enable optimizer parallel for inference 
Merge pull request !20411 from gziyan/enable_opt_shard_predict
 2021-07-20 01:37:06 +00:00
yangzhenzhang b31cd27a08 update check strategy for conv2d 2021-07-19 15:03:26 +08:00
Ziyan 1c9166e0a6 remove restriction for opt shard in inference 2021-07-16 17:16:12 +08:00
Xiaoda Zhang 04381273b3 Add the sharding propagation function: 
1) users configure sharding strategies for operators;
2) framework will propagate the strategies from configured-ops to
non-configured ops using BFS;
3) the propagation goal is to minimize redistribution communication
cost;
 2021-07-14 10:54:18 +08:00
chenhaozhe 086a871975 Change Loss to LossBase 2021-07-02 16:41:02 +08:00
lichenever db8850a4a3 pipeline_support_predict_master 2021-06-26 09:25:59 +08:00
Ziyan be1f5a43d7 opt shard fit micro batch 2021-06-24 14:52:19 +08:00
yangzhenzhang 69acf757d0 add parallel op for conv2d backprop input 2021-06-24 10:01:49 +08:00
yangzhenzhang 24370b5613 add parallel op for maxpool 2021-06-21 09:00:56 +08:00
yangzhenzhang af0d28de48 add parallel op for batchnorm 2021-06-17 15:43:24 +08:00
i-robot 85d860e6a2 !16457 [AutoParallel]pipeline_split_adapt_master 
Merge pull request !16457 from lichen/pipeline_split_adapt_master
 2021-06-11 11:37:40 +08:00
lichenever db5d508356 pipeline_split_adapt_master 2021-06-10 20:17:33 +08:00
yangzhenzhang 7a40741048 add parallel operator for conv2d 2021-06-10 14:58:51 +08:00
Ziyan 95ac0f6d58 fix optimizer weight shard config 2021-06-08 10:44:17 +08:00
chenhaozhe 9da8534396 change _Loss to Loss 2021-06-03 15:26:59 +08:00
mindspore-ci-bot 1c8fda25ef !16478 handle load op in step parallel 
From: @gong_zi_yan
Reviewed-by: @yangzhenzhang,@stsuteng
Signed-off-by: @stsuteng
 2021-05-29 09:32:35 +08:00
mindspore-ci-bot b45b63fc58 !17239 add parallel gathernd test case 
From: @hanyang001
Reviewed-by: @stsuteng,@yangzhenzhang
Signed-off-by: @stsuteng
 2021-05-29 09:11:25 +08:00
Wan Hanyang c51dff2634 add parallel gathernd test case 2021-05-28 17:21:11 +08:00
Wan Hanyang 3ce521d78f add parallel layernorm test case 2021-05-28 17:20:04 +08:00
Ziyan 4b17493e52 handle load in step parallel 2021-05-28 09:16:40 +08:00
yangzhenzhang d711d98f07 clean duplicate code 2021-05-25 17:25:34 +08:00
yao_yf 732d13ccff parallel dropout support repeated compute 2021-05-20 19:42:13 +08:00
yangzhenzhang 6aa3859131 modify check strategy for scatter update 2021-05-10 11:22:00 +08:00
Ziyan 2a752f24bf enable not fully use opt shard 2021-05-07 15:32:48 +08:00
yao_yf e967f1939b parallel envs variable check 2021-05-07 09:12:53 +08:00
mindspore-ci-bot 78fcdbc7c9 !15790 modify scatter update op 
From: @yangzhenzhang
Reviewed-by: @kisnwang,@stsuteng
Signed-off-by: @stsuteng
 2021-04-28 10:48:19 +08:00
yangzhenzhang 075f680a42 modify scatter update op 2021-04-27 20:23:14 +08:00
Xiaoda Zhang aa52399200 Making the Tile operator to have more parallel strategies 2021-04-27 11:36:40 +08:00
yao_yf 093ef784de dont insert virtualoutput for scalar 2021-04-26 19:55:16 +08:00
mindspore-ci-bot 3cfd58e8e0 !15643 insert virtual div only for first input of dropout do mask 
From: @yangzhenzhang
Reviewed-by: @stsuteng,@kisnwang
Signed-off-by: @stsuteng
 2021-04-26 09:21:00 +08:00
mindspore-ci-bot 49d6c029a6 !15542 split axis and batch for gather 
From: @yangzhenzhang
Reviewed-by: @kisnwang,@stsuteng,@stsuteng
Signed-off-by: @stsuteng,@stsuteng
 2021-04-25 19:33:09 +08:00
yangzhenzhang 5828973978 fix bug for dropout do mask 2021-04-25 16:47:44 +08:00
yao_yf 21276408b8 parallel virtual_out_ops 2021-04-25 11:18:54 +08:00
yangzhenzhang 213922574e split axis and batch for gatherv2 2021-04-23 16:59:35 +08:00
yangzhenzhang c2ca2232c5 add select op 2021-04-20 09:23:22 +08:00
mindspore-ci-bot 1c9d3c0aa0 !15353 add parallel operator for scatter update 
From: @yangzhenzhang
Reviewed-by: @kisnwang,@stsuteng
Signed-off-by: @stsuteng
 2021-04-20 09:03:08 +08:00
mindspore-ci-bot 0fd1726e79 !15172 Clean GraphKernel's codes from frontend 
From: @dayschan
Reviewed-by: @gaoxiong1,@dylangeng,@gaoxiong1
Signed-off-by: @dylangeng,@dylangeng
 2021-04-19 09:34:35 +08:00
yangzhenzhang 9cdd70433f add scatterupdate op 2021-04-17 17:32:48 +08:00
yangzhenzhang d070af122f add topk op 2021-04-17 14:12:49 +08:00
dayschan 771e3f61f3 Clean GraphKernel's codes from frontend 
1. set class GraphKernel as deprecated, and treat it as Cell
2. set class InplaceAssign as deprecated, suggested using Assign instead.
3. set op_selector as deprecated, removed the _selected_ops and _selected_grad_ops, replaced with real operations
4. removed the two passes of GraphKernel from frontend
5. removed the GraphKernel's codes from other modules
 2021-04-17 11:03:34 +08:00
yangzhenzhang f9f5df368e add gathernd op 2021-04-16 14:29:03 +08:00
yangzhenzhang bcd2ecc403 check layouts for shared parameter 2021-04-15 10:39:05 +08:00
yao_yf a83fb3316b fix parallel timeout 2021-04-02 20:19:26 +08:00