p47068915
/
mindspore
forked from mindspore-Ecosystem/mindspore
1
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity
34,741 Commits 23 Branches 0 Tags 3.9 GiB
Commit Graph

4562 Commits

Author SHA1 Message Date
Cathy Wong b1094b9d81 MD Profiling - Add Start Stop Python UT 2021-11-17 13:49:43 -05:00
yanghaitao1 c94aa6b872 use profiler acl api instead 2021-11-17 09:33:31 -05:00
i-robot 4e88f2e5e8 !26345 add examples of function grad, jvp and vjp 
Merge pull request !26345 from chenzhuo/jvp
 2021-11-17 08:35:38 +00:00
linqingke bef2923acf MindSpore support load custom aicpu ops. 2021-11-17 14:58:50 +08:00
i-robot 36517de69c !24519 [assistant][ops] Add new audio operator SlidingWindowCmn. 
Merge pull request !24519 from Rainfor/SlidingWindowCmn
 2021-11-17 06:40:43 +00:00
i-robot 9f2e1edc00 !26279 neighborexchangev2 fix bug 
Merge pull request !26279 from TuDouNi/neighborexchangev2_bug
 2021-11-17 02:29:21 +00:00
wangkc123 706f9e2cbb [feat] [assistant] [I3CKEH] add new audio operator SlidingWindowCmn 2021-11-16 22:00:09 +08:00
yangzhenzhang ba99e4c505 add check for resize op 2021-11-16 15:39:52 +08:00
i-robot 58b69a05ee !20576 [assistant][ops] add new dataset loading operator LJSpeechDataset 
Merge pull request !20576 from 杨旭华/LJSpeechDataset
 2021-11-16 07:00:20 +00:00
chenzhuo d07f24462c add examples of grad, jvp and vjp 2021-11-16 14:15:26 +08:00
ttudu 33ac1de062 fix bug 2021-11-16 11:14:58 +08:00
i-robot 7cdef3da6e !26334 [MD] fix occasional fail in random rampler ut 
Merge pull request !26334 from liyong126/fix_random_sampler
 2021-11-16 02:49:59 +00:00
i-robot 6bdd38399a !25811 fault_recover_by_mirror_group 
Merge pull request !25811 from yao_yf/fault_recover_by_mirror_group
 2021-11-16 02:20:58 +00:00
liyong 9663884db3 fix occasional failed in random sampler 2021-11-16 09:53:38 +08:00
i-robot 5af8572cd7 !25488 add the set_dump API 
Merge pull request !25488 from wenkai/wk1027setdump1
 2021-11-16 01:47:32 +00:00
i-robot 0113b3eee1 !26236 Add MD Profiler save() 
Merge pull request !26236 from Mohammad Motallebi/add_profiler_save
 2021-11-16 01:04:53 +00:00
mohammad 5c8ab5f60c add MD Profiler Save() 2021-11-15 13:18:36 -05:00
wenkai d2eca80d40 add the set_dump API, see I4AUIR for details 2021-11-15 21:28:40 +08:00
i-robot 9dfe08b805 !25909 [Fallback] Add global Py param when processing global namespace info 
Merge pull request !25909 from huangbingjian/update_global_params
 2021-11-15 11:43:35 +00:00
i-robot 9522ee9686 !25703 [assistant][ops] Add math operator LpNorm 
Merge pull request !25703 from 孟权令/LpNorm
 2021-11-15 02:36:54 +00:00
vddong 6fe2d35ae2 [feat][assistant][I3T96D] add new dataset loading operator LJSpeechDataset 2021-11-14 14:52:15 +08:00
yao_yf 501b978d16 find data parallel common group in auto parallel 2021-11-13 10:23:45 +08:00
i-robot 04da5c2808 !26140 Add callback support to BatchOp Datasets to enable Add/Remove workers 
Merge pull request !26140 from h.farahat/cb_batch
 2021-11-12 17:23:19 +00:00
i-robot e154dca4b1 !22193 [assistant][ops] Add new loader operator DBpedia 
Merge pull request !22193 from 杨旭华/DBpedia
 2021-11-12 09:02:28 +00:00
i-robot 0f07408425 !26034 F.grad support sens_param and fix graph_mode bug 
Merge pull request !26034 from zhang_sss/grad
 2021-11-12 06:38:15 +00:00
i-robot e773f0621e !25918 add Function jvp and vjp 
Merge pull request !25918 from chenzhuo/jvp
 2021-11-12 06:37:46 +00:00
i-robot 5869e02fa1 !26135 Enable tuple input of optimizer as variable 
Merge pull request !26135 from LiangZhibo/broaden3
 2021-11-12 06:29:35 +00:00
i-robot 6c587dc2d3 !25391 Support to profiling parallel strategy 
Merge pull request !25391 from ougongchang/profiling_stategy
 2021-11-12 06:07:30 +00:00
Carry955 0d26c38693 [feat][assistant][I40GXT] add new loader DBpedia 2021-11-11 19:02:22 -08:00
i-robot 7a73bae5c3 !26036 add output strategy for matmul operator 
Merge pull request !26036 from yangzhenzhang/add-output-strategy-for-op-init
 2021-11-12 01:22:22 +00:00
i-robot e03544685b !26049 [Auto parallel] Support reshape in sharding propagation 
Merge pull request !26049 from Xiaoda/99-ad-reshape-adjusting-in-sharding-propagation
 2021-11-12 01:14:18 +00:00
chenzhuo f36edec3ff add function jvp 2021-11-12 08:44:41 +08:00
hesham 2662b6d5c3 Add callback to Batch op 2021-11-11 10:58:01 -05:00
zhang__sss 0dd68aa7eb grad 2021-11-11 23:18:35 +08:00
i-robot 1792d5f8a4 !26131 Offload Hardware Acceleration Computing 
Merge pull request !26131 from alashkari/hw_accel_move_copy
 2021-11-11 13:18:07 +00:00
ougongchang 9229f1c1ff profiler support to collect parallel strategy info 
If SetNodeOutputType functions forcibly splits into multiple functions, the readability decreases, so it blocks lizard scans
 2021-11-11 20:11:42 +08:00
i-robot b910870ecc !19986 [assistant][ops] Add new dataset loading operator AGNEWS 
Merge pull request !19986 from 杨旭华/AGNEWS
 2021-11-11 06:22:25 +00:00
Xiaoda Zhang a772767265 support reshape in sharding propagation: 
1) using 'swc index of strategy_cost_' as reshape's selected strategy;
2) when encountering reshape in BFS, select the 'swc index' with zero communication cost;
3) when encountering a reshape that is already visited, check whether there exists communication between reshape and current operator. It is OK if communication happens between two configured operators;
4) currently, two consecutive reshapes are not supported;
5) adjusting BFS structure in graph_costmodel.cc;
6) adjusting some code in step_auto_parallel.cc to avoid cyclomatic complexity.
 2021-11-11 11:35:43 +08:00
l00591931 9353bd10a4 Enable tuple as variable in optimizer 2021-11-11 11:31:05 +08:00
zetongzhao 2de8142c44 profiler test 2021-11-10 11:55:19 -05:00
Megalomania c34bbabed9 [feat] [assistant] [I3T96M] add new loader AGNEWS 2021-11-09 23:51:19 -08:00
yangzhenzhang 8431ba616c add output strategy for op init 2021-11-10 15:35:45 +08:00
rescue 22e03ab12e Adding offload hardware accelerator. 2021-11-10 07:03:43 +00:00
i-robot 8bf7e28fa6 !25410 add dump flag for fusion nodes 
Merge pull request !25410 from yuchaojie/ir_fusion3
 2021-11-10 02:25:08 +00:00
i-robot d4c8dde6d2 !25923 Move IsRealKernel and IsRealCNodeKernel from AnfAlgo to AnfUtils 
Merge pull request !25923 from DeshiChen/1101_anfalgo
 2021-11-10 01:56:07 +00:00
i-robot 2e174427c9 !25963 [MD][Autotune] Add WaitFor 
Merge pull request !25963 from harshvardhangupta/hesh_tree_mod
 2021-11-09 22:43:00 +00:00
i-robot b7244ac987 !25852 MD C++ UT: Remove old tests part 2 
Merge pull request !25852 from hetshah/hs_remove_old_tests
 2021-11-09 22:07:59 +00:00
hetshah 5c703015e4 remove old tests and move coverage 2021-11-09 21:27:37 +00:00
i-robot 61b53fee3a !25980 Add callback support to Mappable Datasets to enable Add/Remove workers 
Merge pull request !25980 from h.farahat/cb_mappable
 2021-11-09 20:05:01 +00:00
i-robot f1f7731fb2 !25964 Fix serdes multithreading bug 
Merge pull request !25964 from zetongzhao/serdes_multithreading
 2021-11-09 15:48:06 +00:00
hesham 17e338e909 Add WaitFor 2021-11-09 10:19:48 -05:00
i-robot 14c3517b9a !25452 Add Gradient Sharding 
Merge pull request !25452 from huangxinjing/pipeline_shard_mini_step
 2021-11-09 08:15:23 +00:00
dayschan cbb84ff580 Move IsRealKernel and IsRealCNodeKernel from AnfAlgo to AnfUtils 
the function IsOneOfPrimitive and IsOneOfPrimitiveCNode is useful,
we can move them into anf.cc
 2021-11-09 14:34:21 +08:00
yuchaojie 0c90aecae4 add dump flag for fusion nodes 2021-11-09 10:48:17 +08:00
huangbingjian d0ca18b921 Add global Py param when processing global namespace info. 2021-11-09 09:34:30 +08:00
hesham 040398a660 Add callback to Mappable ops 
Update mindspore/ccsrc/minddata/dataset/engine/datasetops/source/mappable_leaf_op.cc
 2021-11-08 20:17:34 -05:00
huangxinjing f354ab22a3 add pipeline shard interface 
Add support for no pipeline accugradient

Add delay tag for fusion op

Optimizer the visite order

add mirror for mini step control

Move the group to attributes

Add gradient_shard control for the mini step

Fix code stype

Fix ut description

Add interface
 2021-11-09 09:11:13 +08:00
i-robot d89a39c661 !25851 MD Profiling Start and Stop Support 
Merge pull request !25851 from cathwong/ckw_prof_startstop4
 2021-11-08 16:16:42 +00:00
i-robot 527abb8ede !26003 [MD] Added restrictions on shuffle and num_samples 
Merge pull request !26003 from liyong126/fix_shuffle_and_num_samples
 2021-11-08 11:58:08 +00:00
kman066 4ebec36ab2 add lpnorm 2021-11-08 19:36:13 +08:00
i-robot efc33a8225 !25745 F.grad support get gradient by position 
Merge pull request !25745 from zhang_sss/grad
 2021-11-08 08:59:23 +00:00
liyong 79891feadb fix shuffle and num_samples issue 2021-11-08 15:33:01 +08:00
zhang__sss 8bae4d9d50 grad 2021-11-08 14:12:58 +08:00
Cathy Wong cdf3126459 MD Profiler Start Stop Support 2021-11-05 19:46:39 -04:00
zetongzhao 5e8377ea5c fix multithreading bug 2021-11-05 16:49:18 -04:00
i-robot ded1c77bbf !25765 neighborExchangeV2 & grad 
Merge pull request !25765 from TuDouNi/neighborExchangeV2
 2021-11-05 09:31:03 +00:00
ttudu e953c15cd2 NeighborExchangeV2 & Grad 2021-11-05 11:00:46 +08:00
i-robot 7c14f984ed !25793 add output strategy for shard 
Merge pull request !25793 from yangzhenzhang/add-output-strategy
 2021-11-05 01:19:01 +00:00
i-robot 76f4f77cc2 !24520 Add graph run history to dump structure 
Merge pull request !24520 from parastooashtari/multi_root_graph_dump
 2021-11-04 12:55:01 +00:00
i-robot 92b81d3221 !23257 [assistant][ops][DBToAmplitudeOp] 
Merge pull request !23257 from 杨旭华/DBToAmplitude
 2021-11-04 10:32:32 +00:00
yangzhenzhang 6ad6304b77 add output strategy 2021-11-04 16:28:54 +08:00
i-robot 2828f317f7 !25718 optimize implicit convert error message 
Merge pull request !25718 from chujinjin/optimize_implicit_convert_error_msg
 2021-11-03 12:31:56 +00:00
i-robot 835e2e546e !25759 Fix the unloadable bprop mindir files 
Merge pull request !25759 from YuJianfeng/bprop_mindir
 2021-11-03 09:26:54 +00:00
i-robot 1a2a462c8f !25786 fix resnet pynative performance 
Merge pull request !25786 from baihuawei/fixpynativeperformance
 2021-11-03 08:58:09 +00:00
yujianfeng fc769a79fe Fix the unloadable bprop mindir files 2021-11-03 14:29:56 +08:00
liucunwei a86ac516fc add some of ut about dataset IteratorOneColumn 2021-11-03 12:02:00 +08:00
baihuawei e59d07899b fix reset8p pynative performance 2021-11-03 09:13:19 +08:00
Isaac 69f57bc5b1 [feat][assistant][I40GXD]add new data operator DBToAmplitude 2021-11-02 23:45:27 +08:00
parastooashtari 1a59dc37bf add graph execution order history to dump 2021-11-02 11:00:38 -04:00
i-robot 2224f75b47 !25696 add pass switch 
Merge pull request !25696 from zhoufeng/pass-switch
 2021-11-02 01:33:15 +00:00
i-robot 1edd91c974 !25329 MD C++ UT: Remove old tests part 1 
Merge pull request !25329 from hetshah/hs_remove_old_tests
 2021-11-01 19:35:58 +00:00
i-robot eff2318bb6 !25524 [Autotune] Add TreeModifier and Remove Worker logic 
Merge pull request !25524 from h.farahat/tree_modifier
 2021-11-01 13:35:04 +00:00
i-robot 30870cc133 !25443 [MD] mindrecord support overwriting 
Merge pull request !25443 from liyong126/add_mindrecord_overwrite
 2021-11-01 13:20:48 +00:00
chujinjin 26410ccf75 optimize implicit convert error message 2021-11-01 17:34:11 +08:00
zhoufeng fb17c448c2 add pass switch 
Signed-off-by: zhoufeng <zhoufeng54@huawei.com>
 2021-11-01 16:33:02 +08:00
i-robot 1462eb1f4e !25465 Ascend Device Context for MindRT 
Merge pull request !25465 from laiyongqiang/ascend_context
 2021-11-01 07:16:08 +00:00
LaiYongqiang 07e5ed9f16 ascend device context 2021-11-01 11:42:40 +08:00
liyong 41e8280f55 add mindrecord overwrite 2021-11-01 10:44:20 +08:00
i-robot 45882752ba !25644 [Fallback] Support builtin type 
Merge pull request !25644 from huangbingjian/fallback_set
 2021-10-30 10:01:52 +00:00
i-robot f16ed11237 !25651 Add some fallback testcases 
Merge pull request !25651 from Margaret_wangrui/fallback_tests
 2021-10-30 09:21:46 +00:00
huangbingjian 5063ee55bc [Fallback] Support builtin type: set, dict, slice, bool, int, float, str 2021-10-30 14:54:59 +08:00
yujianfeng 195563c67f Fix the random failure of ut tests in ci 2021-10-30 14:09:12 +08:00
i-robot b27520b6ae !25640 fix minddata ut core 
Merge pull request !25640 from luoyang/serving
 2021-10-30 01:35:32 +00:00
hesham 62d9004d1e TreeModifier and Remove worker 2021-10-29 17:11:19 -04:00
i-robot 7d50c3d5d7 !25551 [Fallback] Added test cases for python built-in functions 
Merge pull request !25551 from huangbingjian/add_fallback_testcases
 2021-10-29 09:15:19 +00:00
luoyang beba798e00 fix minddata ut core 2021-10-29 16:28:22 +08:00
i-robot 958c2a8e00 !25560 Support bytes input for decode op 
Merge pull request !25560 from luoyang/serving
 2021-10-29 06:47:02 +00:00
Margaret_wangrui 6275cdcf7a add fallback testcases 2021-10-29 12:02:16 +08:00
i-robot e43d0ee554 !25613 [MD] fix dataset ut part06 
Merge pull request !25613 from liyong126/fix_md_ut_part06
 2021-10-29 03:32:27 +00:00
i-robot 48699c939c !25544 add parallel op for resizenearestneighbor 
Merge pull request !25544 from yangzhenzhang/add-parallel-op-for-resizenearestneighbor
 2021-10-29 02:47:49 +00:00
liyong b9bad0bdbb fix md ut part06 2021-10-29 10:26:34 +08:00
huangbingjian 02f72ab3ae [Fallback] Added test cases for python built-in functions 2021-10-29 09:33:16 +08:00
i-robot 357ad46cb1 !25585 Enable JIT Fallback in default. 
Merge pull request !25585 from 张清华/opt_jit_fallback
 2021-10-28 17:30:50 +00:00
i-robot 036bc4329f !25447 Check Device Shape 
Merge pull request !25447 from hwjiaorui/check-support-json
 2021-10-28 12:38:06 +00:00
Zhang Qinghua e8b421fe6e Enable JIT Fallback in default. 2021-10-28 20:18:06 +08:00
liucunwei f2813b909e add some of ut cases about createIterator columns 2021-10-28 18:57:02 +08:00
i-robot 8758830723 !25479 delete mindquantum c++ simulator 
Merge pull request !25479 from donghufeng/master
 2021-10-28 08:14:18 +00:00
luoyang b47ac46e92 Support bytes input for decode op 2021-10-28 15:22:50 +08:00
yangzhenzhang c42081619e add parallel op for resizenearestneighbor 2021-10-28 11:31:00 +08:00
i-robot a7834c382f !25511 Add some fallback testcases 
Merge pull request !25511 from Margaret_wangrui/fallback_tests
 2021-10-28 02:28:20 +00:00
Margaret_wangrui 7738f55f44 Add some fallback testcases 2021-10-28 09:15:24 +08:00
i-robot 9e1370515a !25451 Make MD ProfilingManager a singleton class 
Merge pull request !25451 from h.farahat/profiling_manager_singlton
 2021-10-27 20:54:13 +00:00
hesham 843357a6d8 Make ProfilingManager a singleton 2021-10-27 11:37:02 -04:00
i-robot 418e1b425b !23490 [assistant][ops] Add New Math Operator IsNan 
Merge pull request !23490 from 孟权令/IsNan
 2021-10-27 12:15:41 +00:00
xuxusheng 7a9a4d0515 delete mindquantum simulator 2021-10-27 18:35:35 +08:00
i-robot 360e228f45 !25358 Fix fallback and cell kwargs issues. 
Merge pull request !25358 from 张清华/opt_fallback
 2021-10-27 09:07:55 +00:00
Sawyer c032d1f83c [feat][assistant][I3T92O]add new operator IsNan 2021-10-27 16:14:03 +08:00
i-robot 180b071122 !22149 [assistant][ops] Add New IsInf 
Merge pull request !22149 from 孟权令/IsInf
 2021-10-27 07:41:15 +00:00
i-robot 0d93793e80 !25463 log improvement 
Merge pull request !25463 from laiyongqiang/step_log
 2021-10-27 07:17:27 +00:00
Zhang Qinghua b43693be23 Fix fallback and cell kwargs issues. 2021-10-27 14:06:02 +08:00
LaiYongqiang dc7988f4bd log improvement 2021-10-27 12:36:47 +08:00
hwjiaorui 5f494aa61b check transfrom device shape 2021-10-27 09:12:41 +08:00
i-robot 74977e548d !24624 [Autotune] Modify MD callback and AddWorker logic to ParallelOps 
Merge pull request !24624 from h.farahat/add_worker
 2021-10-27 00:41:20 +00:00
hetshah 1be909c28e removed old tests and moved additonal coverage to new test files 2021-10-26 20:43:49 +00:00
i-robot 5a49db7a36 !25212 Reduce size of queue and test 
Merge pull request !25212 from zetongzhao/resize_smaller
 2021-10-26 19:36:00 +00:00
i-robot c0f6286fcb !25092 [MD] API for autotune 
Merge pull request !25092 from harshvardhangupta/auto_tune_api
 2021-10-26 18:14:43 +00:00
i-robot 704c490e80 !25449 MD UT - Add missing set and restore seed for parallel test execution 
Merge pull request !25449 from cathwong/ckw_ut_py_parallel
 2021-10-26 17:53:39 +00:00
Cathy Wong e66aba9ef5 MD UT - Add missing set and restore seed. 
test_places365_content_check - delete visualize_dataset
 2021-10-26 13:19:47 -04:00
harshvardhangupta 1bb45142c9 Implement autotune api 2021-10-26 11:26:13 -04:00
zetongzhao 9dad5f1f46 resize to a smaller size 2021-10-25 16:03:57 -04:00
doit cd9242eaa9 [feat] [assistant] [I3T92M] add new math operator IsInf 2021-10-25 14:40:40 +08:00
huangbingjian d7e97dd74a change EXCEPTION level to CRITICAL level 2021-10-25 14:25:50 +08:00
i-robot 9acbf3eb7a !25282 1、Optimize the error message in operator 2、add operator test cases 
Merge pull request !25282 from Margaret_wangrui/operator_cases
 2021-10-25 01:22:42 +00:00
i-robot cb307e24cf !25153 refactor device loop control 
Merge pull request !25153 from laiyongqiang/adjust_kernel_refactory
 2021-10-23 07:42:49 +00:00
i-robot 6f413ee29d !25155 Support ops, nn and numpy namespaces, and add test case. 
Merge pull request !25155 from 张清华/opt_fallback
 2021-10-23 03:50:53 +00:00
Margaret_wangrui 7b93e4e87c 1、Optimize the error message in operator 2、add operator test cases 2021-10-23 10:16:59 +08:00
LaiYongqiang 9bfb2d99fa refactor device loop control 2021-10-22 20:42:38 +08:00
i-robot e05a4393ef !24611 split compile and generate kernel mod 
Merge pull request !24611 from liubuyu/bug_fix2
 2021-10-22 09:02:55 +00:00
Zhang Qinghua 6a79e262ae Support ops, nn and numpy namespaces, and add test case. 2021-10-22 16:31:30 +08:00
i-robot c544af28b6 !25286 dataset: modify profiling ut for parallel running 
Merge pull request !25286 from ms_yan/ut_parallel
 2021-10-22 08:09:03 +00:00
ms_yan bb314ad07f make ut parallel 2021-10-22 13:17:43 +08:00
lby 6872e67131 split compile ang gen kernel mod 2021-10-22 10:13:50 +08:00
i-robot cf0029b33a !25277 [MD] fix mindrecord ut part05 
Merge pull request !25277 from liyong126/fix_md_ut_05
 2021-10-22 01:43:03 +00:00
i-robot 782874bf18 !21215 [MS][crowdfunding]New operator implementation, include phaser 
Merge pull request !21215 from Wangsong95/phaser
 2021-10-22 01:08:50 +00:00
hesham 85e077b60b Callback changes 
AddWorker logic to parallel ops
 2021-10-21 17:35:11 -04:00
liyong 7c9f6899a3 fix md ut part05 2021-10-21 20:40:09 +08:00
i-robot b7d8bcfe8d !25177 [MD] fix mindrecord ut part04 
Merge pull request !25177 from liyong126/fix_md_ut_part04
 2021-10-21 01:39:26 +00:00
i-robot 5d8be044c9 !18018 [assistant][ops]New operator implementation, include FashionMnistDataset 
Merge pull request !18018 from 张璇/fashionmnist_dataset
 2021-10-20 06:42:54 +00:00
i-robot f9bd640b09 !16269 [assistant][RandomLighting] 
Merge pull request !16269 from QingfengLi/randomlighting
 2021-10-20 06:42:12 +00:00
liyong fc5ea9cc1c fix dataset ut for running independently 2021-10-20 14:27:36 +08:00