Commit Graph

3528 Commits

Author SHA1 Message Date
Cathy Wong 3fe9a05341 [MD] Profiler Memory Tool Support 2022-01-31 17:46:17 -05:00
i-robot e12d6af257
!29358 MD Child Sampler Code and UT Issues
Merge pull request !29358 from zetongzhao/sampler_AddChild
2022-01-31 16:55:22 +00:00
zetongzhao ea9783e18c fix sampler add_child 2022-01-31 07:45:42 -05:00
i-robot 48f4fd2f52
!29639 build core.so
Merge pull request !29639 from zhoufeng/dynamic-core
2022-01-30 02:00:21 +00:00
i-robot 02d63d7ad9
!23008 [assistant][ops][FilterWikipediaXML]
Merge pull request !23008 from Isaac/FilterWikipediaXML
2022-01-29 08:35:29 +00:00
i-robot ea391f6eb1
!29505 Fix AllGather Cast when the parameters are shared
Merge pull request !29505 from huangxinjing/fx_allgather_cast
2022-01-29 07:46:10 +00:00
i-robot f5392f2460
!22443 [assistant][ops] Add new data operator MaskAlongAxis
Merge pull request !22443 from TR-nbu/MaskAlongAxis
2022-01-29 07:19:20 +00:00
zhoufeng 8e97ce5cd2 build core.so
Signed-off-by: zhoufeng <zhoufeng54@huawei.com>
2022-01-29 14:11:02 +08:00
tophand 797cd52c96 [feat][assistant][I40GXK]add new data operator Filter_Wikipedia_XML 2022-01-28 02:26:20 -08:00
QRICK 52c73ff1b3 [assistant][ops][I3T96S] Add new data operator MaskAlongAxis 2022-01-28 14:53:38 +08:00
i-robot 47a4860043
!27184 [assistant][ops]Add Cummax
Merge pull request !27184 from 陈钧/cummax
2022-01-28 06:40:25 +00:00
i-robot 92c59c3401
!25289 [assistant][ops] Add Coalesce
Merge pull request !25289 from 李定维/Coalesce
2022-01-27 12:31:30 +00:00
i-robot a24709b46a
!22446 [assistant][ops] Add new data operator MaskAlongAxisIID
Merge pull request !22446 from TR-nbu/MaskAlongAxisIID
2022-01-27 08:45:24 +00:00
王泰格 cc8d3e6279 [assistant][ops][I40FKE] add operator Coalesce 2022-01-27 16:08:31 +08:00
QRICK 9ed73eae94 [assistant][ops][I3T96P] Add new data operator MaskAlongAxisIID 2022-01-27 12:20:26 +08:00
chen-jun0211 5343a83f79 Add Cummax 2022-01-27 10:53:51 +08:00
i-robot 4861e9abab
!24098 [assistant][ops] Add UpperBound
Merge pull request !24098 from 王刚克/UpperBound
2022-01-26 10:46:26 +00:00
i-robot 2b82ba1e49
!26244 [assistant][ops] Add new data PhaseVocoder
Merge pull request !26244 from TR-nbu/PhaseVocoder
2022-01-26 06:46:36 +00:00
i-robot b0b3d7ede7
!29421 Implementation of BoundingBoxEncodeInfo, IOUInfo and RandomChoickWithMaskInfo
Merge pull request !29421 from liuluobin/parallel_ops_master
2022-01-26 06:40:32 +00:00
i-robot 6fac9c5d1d
!28505 Change SparseTensor to COOTensor
Merge pull request !28505 from wangrao124/COOTensor
2022-01-26 06:22:28 +00:00
i-robot 55ba926a04
!29087 optimize sharding propagation&add ut on pangu
Merge pull request !29087 from bichaoyang/master
2022-01-25 11:59:25 +00:00
huangxinjing 34101d748d Add cast ut and fix juge condition 2022-01-25 18:14:11 +08:00
yangzhenzhang 6df5e9aa5d support pad mode for conv2d 2022-01-25 17:26:47 +08:00
liuluobin fb79c508f2 Implementation of BoundingBoxEncodeInfo, IOUInfo, RandomChoiceWithMaskInfo. 2022-01-25 15:49:48 +08:00
wang-gangke e912bc8a24 UpperBound 2022-01-25 15:45:33 +08:00
wangrao124 9dfc703f7a change sparsetensor to cootensor 2022-01-25 15:39:37 +08:00
b00518648 2ff3425c76 1.consider comm cost inside a op when select a strategy;2.deal with the ops that share same param;3.add ut to fix the perfermence at pangu_alpha 2022-01-25 15:05:37 +08:00
i-robot bff5afe2bb
!25078 [assistant][ops] Add Cross
Merge pull request !25078 from 冯钰/cross
2022-01-25 02:16:41 +00:00
donrichnx eee07047a0 [feat] [assistant] [I40GYJ] add Captioning task of CocoDataset 2022-01-24 17:24:16 +08:00
minatofy 7832d77718 cross 2022-01-24 17:09:36 +08:00
QRICK 859f90cb85 [assistant][ops][I3T96R] Add new data operator PhaseVocoder 2022-01-24 16:00:24 +08:00
i-robot 8284a5590d
!29427 clean code
Merge pull request !29427 from huangbingjian/clean_code
2022-01-24 01:35:29 +00:00
i-robot 85ac17a8c6
!22736 [assistant][ops] Add new dataset operator LibriTTSDataset
Merge pull request !22736 from TR-nbu/LibriTTSDataset
2022-01-22 10:03:40 +00:00
i-robot ee8fba498a
!29376 support config allreduce group size for batch norm
Merge pull request !29376 from yangzhenzhang/support-config-group-for-batchnorm
2022-01-22 06:20:20 +00:00
huangbingjian c36a85e00d clean code 2022-01-22 11:39:12 +08:00
i-robot fe2a557901
!23919 [assistant][ops] Add HSVToRGB
Merge pull request !23919 from 苏家毅/HSVToRGB
2022-01-22 02:11:13 +00:00
yangzhenzhang 4d0b1a47ee support config group for batchnorm 2022-01-22 09:29:52 +08:00
davilsu af1333e1cc HSVToRGB 2022-01-21 20:57:30 +08:00
i-robot 690a660919
!17918 [assistant][MelScale]
Merge pull request !17918 from QingfengLi/MelScale
2022-01-21 09:52:56 +00:00
ZJM 8d339ccfea [assistant][ops][22736] Add new dataset operator LibriTTSDataset 2022-01-21 17:34:42 +08:00
i-robot b1213abe7d
!23937 [assistant][ops]Add GridSampler3D and GridSampler3DGrad
Merge pull request !23937 from 梁天凯/GridSampler3D
2022-01-21 07:09:17 +00:00
chenx2ovo bbba4fb940 [feat][assistant][I3CEG0] add op melscale 2022-01-21 11:30:34 +08:00
i-robot 4d81f9737f
!29324 support dilation for conv2d
Merge pull request !29324 from yangzhenzhang/support-dilation-for-conv2d
2022-01-21 02:57:15 +00:00
bubtltk f5ffb282ee Add GridSampler3D and GridSampler3DGrad 2022-01-20 22:23:30 +08:00
yangzhenzhang 000c5b5394 support dilation for conv2d 2022-01-20 19:04:52 +08:00
ZJM fff47f3886 [assistant][ops][I3T96T] Add new dataset operator CMUArcticDataset 2022-01-20 17:40:35 +08:00
i-robot 7bb5819889
!29265 fix resize_bilinear infer
Merge pull request !29265 from jiangzhenguang/resize_bilinear
2022-01-20 06:51:03 +00:00
zy 7543172bd9 [feat] [assistant] [I40GJE] Add new AICPU operator CholeskyInverse 2022-01-20 11:01:45 +08:00
i-robot 501614a61f
!22514 [assistant][ops] Add new data operator Multi30kDataset
Merge pull request !22514 from 杨旭华/Multi30kDataset
2022-01-20 02:13:17 +00:00
i-robot 46fdab3848
!23958 [assistant][ops] Add LuSolve
Merge pull request !23958 from 张渝/lu_solve
2022-01-20 01:42:24 +00:00
i-robot 6b6b001a78
!23443 [assistant][ops] Add LowerBound
Merge pull request !23443 from 梁天凯/LowerBound
2022-01-20 01:40:44 +00:00
i-robot f4bd87380b
!29195 Update API docs for Audio and Callback to the previous modifications
Merge pull request !29195 from xiaotianci/fix_chinese_api
2022-01-20 01:17:02 +00:00
bubtltk e514fc3feb Add LowerBound 2022-01-20 03:07:48 +08:00
unknown c4ee548905 [feat][assistant][I40GY6]add new data operator Multi30kDataset 2022-01-19 18:33:13 +08:00
i-robot 10c8b83068
!22421 [assistant][ops] Add new dataset operator GTZANDataset
Merge pull request !22421 from TR-nbu/GTZANDataset
2022-01-19 09:08:12 +00:00
zy 37bb297bc7 [feat] [assistant] [I40GI8] Add new operation LuSolve 2022-01-19 15:45:58 +08:00
Xiao Tianci 1d628bfb85 Fix API docs of Audio and Callback 2022-01-19 15:23:50 +08:00
ZJM 21973bbbec [assistant][ops][I3T96W] Add new dataset operator GTZANDataset 2022-01-19 13:48:50 +08:00
Weijun 85a7a994fe [feat] [assistant] [I40GJ3] Add new AICPU operator Lstsq 2022-01-19 13:19:46 +08:00
jiangzhenguang abddc993ea fix resize_bilinear 2022-01-18 20:33:44 +08:00
yangzhenzhang 5514189257 support group for conv2d 2022-01-18 17:28:13 +08:00
i-robot 19a9ad6f38
!22893 [assistant][ops] Add new data operator CreateFbMatrix
Merge pull request !22893 from TR-nbu/CreateFbMatrix
2022-01-17 08:33:48 +00:00
1462492739@qq.com bf5f24d62f [assistant][ops][I3T96S] Add new data operator CreateFbMatrix 2022-01-14 21:24:17 +08:00
yangzhenzhang 6dd7333c0b fix bug for conv2d transpose 2022-01-14 16:36:03 +08:00
i-robot fd5d5f1d21
!22754 [assistant][ops] Add math operators MatrixInverse, MatrixDeterminant and LogMatrixDeterminant
Merge pull request !22754 from 孟权令/MatrixInverse
2022-01-14 06:56:30 +00:00
i-robot 9dae085c46
!29054 Delete testcase for ops
Merge pull request !29054 from zhanlijun/addtest1
2022-01-14 06:08:51 +00:00
i-robot cfe5669de9
!28975 [MD] Chained Sampler Python UT fixes and updates
Merge pull request !28975 from cathwong/ckw_ut_sampler_chain_fixes2
2022-01-14 03:49:05 +00:00
Sawyer f335c44340 [feat][assistant][I3T92S][I3T92Q][I3T92R] add MatrixInverse, MatrixDeterminant and LogMatrixDeterminant 2022-01-14 09:20:55 +08:00
Cathy Wong 9d9013ba79 [MD] Chained Sampler UT fixes and updates 2022-01-13 17:39:33 -05:00
zhanlijun 5a5042e385 delete testcase for ops 2022-01-13 21:41:03 +08:00
i-robot a1b7b030c2
!29003 fix minddata api doc
Merge pull request !29003 from luoyang/fix_api_doc
2022-01-13 13:23:19 +00:00
i-robot e3ff16dac6
!28941 support single director exchange for conv2d
Merge pull request !28941 from yangzhenzhang/support-single-direction-exchange-for-conv2d
2022-01-13 11:43:28 +00:00
luoyang e5f2c75233 fix minddata api doc 2022-01-13 16:39:08 +08:00
yangzhenzhang a9d2e255f5 support single direction exchange for conv2d 2022-01-13 09:15:49 +08:00
markuskunej 54e584c300 Added input column validation for offloaded map operations.
Moved check_add_offload from dataset_helper.py to offload.py.
2022-01-12 16:16:49 +00:00
i-robot c5b650069f
!28662 Add input type check for instance Create in Cell.construct
Merge pull request !28662 from zhangzhaoju/master
2022-01-12 09:46:29 +00:00
i-robot fc4abc99f5
!28595 Fix some code review probelms
Merge pull request !28595 from xiaotianci/update_code_review
2022-01-12 09:30:28 +00:00
i-robot 5c961f9fb2
!28812 [MD UT] Misc AutoTune and Profiling UT updates
Merge pull request !28812 from cathwong/ckw_prof_misc
2022-01-12 03:03:13 +00:00
i-robot 017cb5f3ad
!27980 auto insert VirtualDataset node for master
Merge pull request !27980 from lilei/insert_virtualdataset_for_master
2022-01-12 01:14:44 +00:00
Cathy Wong 12b4e81f55 MD UT: Misc AutoTune and Profiling UT updates 2022-01-11 11:56:06 -05:00
i-robot 15bd3af3e3
!28821 Adapt to the old data format of aicpu
Merge pull request !28821 from zangqx/zangqx_5
2022-01-11 11:48:13 +00:00
i-robot d33f1c7e48
!28757 fix greater op infervalue issue & err massage
Merge pull request !28757 from Simson/op_infer
2022-01-11 11:03:42 +00:00
i-robot 8a6ca25539
!28641 unify ge and vm backend
Merge pull request !28641 from xiaoyao/master
2022-01-11 07:53:33 +00:00
Xiao Tianci 96a2e38d01 update review problems 2022-01-11 15:31:53 +08:00
i-robot 1ac5261dfe
!28598 fix softmaxcrossentropywithlogits infer
Merge pull request !28598 from jiangzhenguang/softmaxcrossentropywithlogits
2022-01-11 07:27:21 +00:00
臧庆香 10d275114e Adapt to the old data format of aicpu 2022-01-11 14:37:44 +08:00
xiao_yao1994 0843910260 unify ge and vm backend 2022-01-11 11:53:58 +08:00
zhangzhaoju a54d1f33e7 Add input type check for instance Create in Cell.construct
Adjust the order of resource release sequence

Save executor_info to cache after compile avoiding inconsistent between
python cache and c++ cache

Clean entire_costgraph at ClearResAtExit to avoid coredump.
2022-01-11 10:17:39 +08:00
simson 0191832a1d fix greater op infervalue 2022-01-11 09:21:48 +08:00
yangzhenzhang e5df74e9e4 compute top bottom overlap for conv2d 2022-01-10 16:15:04 +08:00
jiangzhenguang d895eadd7b fix softmaxcrossentropywithlogits infer 2022-01-06 20:44:11 +08:00
Margaret_wangrui 7d1f3d71ba Optimize error message 2022-01-06 20:32:08 +08:00
i-robot 0823ec3e9a
!28535 Modify Error Info
Merge pull request !28535 from liuyang/ms_error
2022-01-06 08:45:12 +00:00
i-robot d7c41f3f5f
!28553 [Fallback] Deal with constants such as np.pi
Merge pull request !28553 from huangbingjian/attr_const
2022-01-06 07:13:24 +00:00
liuyang_655 15a153a40d Modify error info 2022-01-06 01:28:53 -05:00
huangbingjian 45e8b52f6e [Fallback] Do not interpret constant from third-partiy library. 2022-01-06 09:42:54 +08:00
i-robot b073af4b5d
!28559 [MD][Offload] Added dimension check to required Offload ops.
Merge pull request !28559 from alashkari/offload-fix
2022-01-05 07:38:31 +00:00
i-robot a1720c210d
!22348 [assistant][ops] Add new loader EnWik9Dataset
Merge pull request !22348 from 杨旭华/EnWik9Dataset
2022-01-05 07:36:48 +00:00
i-robot ec485cb436
!28307 convert EnvSetItem/EnvGetItem to primitive EnvironSet/EnvironGet
Merge pull request !28307 from xychow/EnvGetItem-conversion
2022-01-05 07:10:30 +00:00
Rescue fac68dbb46 Added dimension check to required Offload ops. 2022-01-05 04:26:31 +00:00
i-robot de030c8d41
!28204 Improved HwcToChw support multi-channel input.
Merge pull request !28204 from unseenme/mindcon_coding_hwc2chw
2022-01-04 11:50:04 +00:00
unseenme 755cb0adfa Improved HwcToChw support multi-channel input. 2022-01-04 15:47:04 +09:00
Eden 63ab65e3a4 [feat] [assistant] [#I40GYF] add new loader EnWik9Dataset 2022-01-03 21:45:43 -08:00
zhousiyi 19e9571149 rename env_getitem/setitem to EnvironGet/EnvironGet 2022-01-04 02:17:30 +00:00
i-robot dba67422f9 !28384 [Fallback] Support the use of self
Merge pull request !28384 from huangbingjian/fallback_self
2021-12-31 09:26:03 +00:00
i-robot e8b4e34f02 !28352 Optimize the exception message while the return number of brop is wrong
Merge pull request !28352 from zhangzhaoju/master_I4NX1K
2021-12-31 08:17:18 +00:00
i-robot 03b4829b86 !28431 [Auto parallel] MoE Changes
Merge pull request !28431 from Xiaoda/122-moe-changes
2021-12-31 08:06:45 +00:00
zhangzhaoju 1950b6a823 Optimize the exception message while the return number of brop is wrong
Fix:I4OQM9 Bprop out check message optimize
2021-12-31 15:01:57 +08:00
vddong 7449b8364b [feat][assistant][I40GYQ] add new dataset operator WIDERFace 2021-12-31 13:49:47 +08:00
huangbingjian 58bafb539a [Fallback] Support self attr/method. 2021-12-31 10:57:41 +08:00
Xiaoda Zhang 6d8320fa66 1) fix the exact division in moe;
2) changing CumSum from composition to a single Operator;
3) add InferMirrorOps for CumSumInfo.
2021-12-30 19:20:05 +08:00
jingyangxiang a7f8b3af1c [feat][assistant][ops]New operator implementation, include IMDBDataset
Signed-off-by: zjuter0126 <977180923@qq.com>
2021-12-30 11:16:23 +08:00
zjuter0126 3c2ca41886 [feat][assistant][ops] add new data operator WikiTextDataset 2021-12-29 18:54:54 +08:00
i-robot 17b3e3cd13 !28267 Resuse ConvertData of GenerateArgumentsKey
Merge pull request !28267 from zhangzhaoju/master_res_base
2021-12-29 08:20:10 +00:00
i-robot ae1278fa42 !28074 convert execute mode to graph mode when loading mindir
Merge pull request !28074 from lianliguang/master
2021-12-29 06:23:57 +00:00
i-robot ac9e545dc8 !22546 add new dataset loading operator CaltechDataset
Merge pull request !22546 from 杨旭华/CaltechDataset
2021-12-29 03:40:06 +00:00
zhangzhaoju c5b2996b3f Resuse ConvertData of GenerateArgumentsKey 2021-12-29 10:30:17 +08:00
lianliguang d5efef0ae6 convert pynative mode to graph mode when loading mindir 2021-12-29 09:46:08 +08:00
donrichnx 9a0fc0ffc1 [feat] [assistant] [I40GYH] add new dataset loading operator CaltechDataset 2021-12-28 19:04:18 +08:00
Xiaoda Zhang 1bdb610b34 changing default value of single-loop flag 2021-12-28 17:56:30 +08:00
i-robot 67e843ec7f !27487 [assistant] [ops][SpectralCentroid]
Merge pull request !27487 from 杨旭华/SpectralCentroidOp
2021-12-28 09:10:32 +00:00
i-robot 7faf774063 !22310 [assistant][ops] Add new loader SemeionDataset
Merge pull request !22310 from 杨旭华/SemeionDataset
2021-12-28 03:13:02 +00:00
i-robot bff8d6ae01 !28247 Fix Python Vocab validation
Merge pull request !28247 from luoyang/vocab
2021-12-28 02:58:25 +00:00
luqilin 896f10d6d5 [feat][assistant][I40GYK] add new dataset loading operator SemeionDataset 2021-12-27 19:49:06 +08:00
tongyue 9a1dcd60fc [fix] [assistant] [I40GXI] fix bug in the infer function of the operator SpectralCentroid 2021-12-27 01:41:31 -08:00
i-robot 69a6c7db98 !23380 [assistant][ops] Add new loader AmazonReview
Merge pull request !23380 from 杨旭华/AmazonReview
2021-12-27 09:36:32 +00:00
luoyang ca3de66559 Fix Python Vocab validation 2021-12-27 17:12:07 +08:00
i-robot c13f31861c !26398 [feat] [assistant] [I48OBC] add new LogicalNot
Merge pull request !26398 from 桂胜楠/logicalnot
2021-12-27 02:54:53 +00:00
i-robot eb7a78a998 !28134 Python Vocab refactoring and add ids_to_tokens & tokens_to_ids
Merge pull request !28134 from luoyang/vocab
2021-12-25 06:39:41 +00:00
i-robot dd90a56d68 !28073 fix code warning && remove save_graphs use in st/ut
Merge pull request !28073 from huanghui/fix-warning
2021-12-24 09:29:25 +00:00
i-robot 170e6ad857 !28052 modify profiler ai_cpu ut and st
Merge pull request !28052 from zangqx/zangqx_04
2021-12-24 08:14:25 +00:00
i-robot 22c25ec10e !27862 [Auto parallel] [Sharding propagation] dealing with cast
Merge pull request !27862 from Xiaoda/119-adapting-sharding-propagation
2021-12-24 07:33:47 +00:00
luoyang 0288480722 Python Vocab refactoring and add ids_to_tokens & tokens_to_ids 2021-12-24 15:26:15 +08:00
huanghui 74ca50e652 fix code warning && remove save_graphs use in st/ut 2021-12-24 14:45:37 +08:00
zheng_pengfei 09b5e545dd [feat] [assistant] [I48OBC] add new LogicalNot 2021-12-23 20:39:17 +08:00
臧庆香 fcd3fface0 modify profiler ai_cpu ut and st 2021-12-23 20:10:28 +08:00
Xiaoda Zhang 66c7474e5a remove CastInfo from CNODE 2021-12-23 19:05:02 +08:00
i-robot f9270b8e09 !28069 change API:cell.shard
Merge pull request !28069 from zhuyuxiao/I4LE9J
2021-12-23 10:58:43 +00:00
chengchuanxing 07aa66ee43 [feat][assistant][I40GY2] add new loader AmazonReview 2021-12-23 16:32:31 +08:00
i-robot 1ea8693c83 !27356 dataset: check whether operator in nn or ops is mixed into operations
Merge pull request !27356 from ms_yan/operator_mixed
2021-12-23 07:04:52 +00:00
i-robot 3b6c40ff1e !28028 [ME] Optimize the error message of the operator module.
Merge pull request !28028 from Margaret_wangrui/operator
2021-12-23 06:48:00 +00:00
zhuyuxiao dd7bbf92dd change API 2021-12-23 14:26:06 +08:00
i-robot 91434597c3 !25861 [assistant][ops]New operator implementation, include BesselI0 and BesselI1
Merge pull request !25861 from 张璇/BesselI0
2021-12-23 02:41:41 +00:00
ms_yan 192983b5da check operator mixed or not 2021-12-23 10:22:03 +08:00
Margaret_wangrui 38ddd04c83 [ME] Optimize the error message of the operator module. 2021-12-23 10:10:48 +08:00
lilei 017aa359a6 insert VirtualDataset node for master 2021-12-23 09:25:05 +08:00
zx e0a7a87a9c [feat][assistant][I4CRK5] add new Ascend operator BesselI0 and BesselI1 2021-12-22 16:19:28 +08:00
gaocongli b9187f96da parsing the profiling data on ai cpu in binary 2021-12-22 11:59:47 +08:00
i-robot 14df5d3984 !23670 [assistant][ops] add new dataset loading operator CoNLL2000ChunkingDataset
Merge pull request !23670 from 杨旭华/CoNLL2000ChunkingDataset
2021-12-22 01:50:12 +00:00
i-robot b1de53fac2 !27728 [Fallback] Support numpy augassign method, subscript method and binary operations
Merge pull request !27728 from huangbingjian/fallback_parse
2021-12-22 01:01:46 +00:00