Commit Graph

2503 Commits

Author SHA1 Message Date
mindspore-ci-bot e19e14295a !17773 adapt offline debugger to new dump, sync support for iteration ranges, GPU default dump mode correction
From: @john_tzanakakis
Reviewed-by: @nsyca,@mikef
Signed-off-by: @mikef
2021-06-05 02:46:50 +08:00
John Tzanakakis 7d460de8b4 adapt offline debuger to new dump, sync support for iteration ranges, GPU default dump mode correction 2021-06-04 08:38:50 -04:00
mindspore-ci-bot 7d9bf65654 !16755 support list add list in net in GraphMode.
From: @fenglovebei
Reviewed-by: @zh_qh
Signed-off-by: @zh_qh
2021-06-04 16:37:40 +08:00
huanghui 29f632613f optimize log 2021-06-04 12:32:44 +08:00
mindspore-ci-bot 97989c3bbc !17669 Substitute _Loss for Loss
From: @c_34
Reviewed-by: @guoqi1024,@liangchenghui
Signed-off-by: @guoqi1024,@liangchenghui
2021-06-04 10:20:00 +08:00
mindspore-ci-bot a3f2b92839 !17638 modify vgg16 network for clould
From: @zhanghuiyao
Reviewed-by: @c_34,@wuxuejian
Signed-off-by: @c_34
2021-06-04 09:21:30 +08:00
fenglovebei 76b884dea9 list add list 2021-06-03 21:16:30 +08:00
chenhaozhe 9da8534396 change _Loss to Loss 2021-06-03 15:26:59 +08:00
zhanghuiyao 36ec89656a modify vgg16 network for clould 2021-06-03 11:39:27 +08:00
mindspore-ci-bot b52f9ddb08 !17060 Add HorizontalFlip and VerticalFlip for CV data processing
From: @tiancixiao
Reviewed-by: @liucunwei
Signed-off-by: @liucunwei
2021-06-02 17:48:28 +08:00
mindspore-ci-bot c4757f0692 !17461 MD Profiling Analyze: Handle cpu util with missing ops. Enhance UTs.
From: @cathwong
Reviewed-by: @robingrosman
Signed-off-by: @robingrosman
2021-06-02 11:12:59 +08:00
mindspore-ci-bot c77e7de1dc !17313 fix static check
From: @jiang-shuqiang
Reviewed-by: @yelihua,@ouwenchang
Signed-off-by: @ouwenchang
2021-06-02 08:50:37 +08:00
mindspore-ci-bot ae2e7c1288 !17541 Fix MD Cache code check warnings [master]
From: @lixiachen
Reviewed-by: @pandoublefeng,@robingrosman
Signed-off-by: @robingrosman
2021-06-02 04:53:54 +08:00
Lixia Chen 8c1b561149 Fix cache code check warnings 2021-06-01 10:02:38 -04:00
mindspore-ci-bot a92a450881 !17383 Optimize exception info print
From: @irmo
Reviewed-by: @zh_qh,@ginfung,@ginfung
Signed-off-by: @zh_qh
2021-06-01 17:43:24 +08:00
Cathy Wong f8beb933ad MD Profiling Analyze: Handle cpu util with missing ops. Enhance UTs. 2021-05-31 20:33:19 -04:00
mindspore-ci-bot 4338e4abac !17374 files_need_cleanup report fixes to master
From: @hfarahat
Reviewed-by: @pandoublefeng,@robingrosman
Signed-off-by: @robingrosman
2021-06-01 05:21:18 +08:00
jiangshuqiang 6154f957fb fix static check 2021-05-31 16:47:21 +08:00
Xiao Tianci 9c74d55a49 add HorizontalFlip and VerticalFlip APIs 2021-05-31 15:55:21 +08:00
huanghui 5acf3c8ade [dfx]optimize exception info print 2021-05-31 14:57:29 +08:00
mindspore-ci-bot 8c1e7ce231 !16345 [DFX]Optimize exception log
From: @irmo
Reviewed-by: 
Signed-off-by:
2021-05-31 09:22:48 +08:00
mindspore-ci-bot 04341b1dbc !17076 fix log error for pynative mode
From: @chujinjin
Reviewed-by: @zhoufeng54,@kisnwang
Signed-off-by: @kisnwang
2021-05-31 09:22:29 +08:00
hesham 3a9695313b files_need_cleanup fixes 2021-05-30 15:44:08 -04:00
huanghui 0610a8d99c 9fdc0035a0 2021-05-29 10:31:23 +08:00
mindspore-ci-bot 1c8fda25ef !16478 handle load op in step parallel
From: @gong_zi_yan
Reviewed-by: @yangzhenzhang,@stsuteng
Signed-off-by: @stsuteng
2021-05-29 09:32:35 +08:00
mindspore-ci-bot b45b63fc58 !17239 add parallel gathernd test case
From: @hanyang001
Reviewed-by: @stsuteng,@yangzhenzhang
Signed-off-by: @stsuteng
2021-05-29 09:11:25 +08:00
mindspore-ci-bot 5e87097d15 !17240 add parallel layernorm test case
From: @hanyang001
Reviewed-by: @stsuteng,@yangzhenzhang
Signed-off-by: @stsuteng
2021-05-29 09:11:08 +08:00
Wan Hanyang c51dff2634 add parallel gathernd test case 2021-05-28 17:21:11 +08:00
Wan Hanyang 3ce521d78f add parallel layernorm test case 2021-05-28 17:20:04 +08:00
YangLuo ac319b3a79 Fix mindrecord UTs: files existed causes write exception 2021-05-28 15:13:48 +08:00
Ziyan 4b17493e52 handle load in step parallel 2021-05-28 09:16:40 +08:00
chujinjin 7fb3b5ea97 fix log error 2021-05-27 17:19:10 +08:00
mindspore-ci-bot 1e8868cecc !16727 handle duplicate code
From: @yangzhenzhang
Reviewed-by: @stsuteng,@jjfeing
Signed-off-by: @stsuteng
2021-05-26 10:58:54 +08:00
mindspore-ci-bot 2433778dac !15963 Support getting dynamic_min_max_shape of data
From: @luoyang42
Reviewed-by: 
Signed-off-by:
2021-05-26 09:41:02 +08:00
mindspore-ci-bot 730e211e52 !16702 Add GaussianBlur API for CV data processing
From: @tiancixiao
Reviewed-by: @liucunwei,@jonyguo
Signed-off-by: @liucunwei
2021-05-26 09:09:41 +08:00
mindspore-ci-bot d0978f709a !16167 CocoDataset add a new column according a new arg extra_metadata
From: @xiefangqi
Reviewed-by: 
Signed-off-by:
2021-05-25 19:55:41 +08:00
yangzhenzhang d711d98f07 clean duplicate code 2021-05-25 17:25:34 +08:00
mindspore-ci-bot e513786709 !16565 Bug in Normalize when num_channels > 128
From: @hfarahat
Reviewed-by: @robingrosman,@pandoublefeng
Signed-off-by: @robingrosman
2021-05-25 10:24:25 +08:00
xiefangqi 53891dbdf7 coco add img_id column 2021-05-24 17:19:58 +08:00
luoyang 13ef5b61f3 support getting dynamic_min_max_shape of data 2021-05-24 17:04:50 +08:00
Xiao Tianci 3f495bbae1 add GaussianBlur API 2021-05-24 16:27:05 +08:00
liuxiao93 1160088c8f support avgpool3d 2021-05-22 14:44:11 +08:00
yao_yf 732d13ccff parallel dropout support repeated compute 2021-05-20 19:42:13 +08:00
mindspore-ci-bot 05bbd1d8fa !16426 Change the permission of cache shared memory to 600 for security
From: @lixiachen
Reviewed-by: @robingrosman,@pandoublefeng
Signed-off-by: @robingrosman
2021-05-20 02:36:26 +08:00
mindspore-ci-bot c47356d77b !16502 check Optimizer parameters must be list
From: @wangnan39
Reviewed-by: @zh_qh,@kingxian
Signed-off-by: @zh_qh,@kingxian
2021-05-19 09:23:56 +08:00
mindspore-ci-bot 1cb2d31642 !16509 GNN dataset new feature for invalid edge or node id case
From: @lizhenglong1992
Reviewed-by: @liucunwei,@heleiwang
Signed-off-by: @liucunwei
2021-05-19 09:20:24 +08:00
hesham bc92011b5f Bug in Normalize when num_channels > 128 2021-05-18 12:23:11 -04:00
mindspore-ci-bot ed72a0d9c3 !16497 Add more accurate abstract join error information
From: @huangbingjian
Reviewed-by: @zh_qh,@ginfung
Signed-off-by: @zh_qh
2021-05-18 19:36:11 +08:00
Zhenglong Li 9284b6d6af Add special case returned value for get feature function 2021-05-18 15:53:01 +08:00
mindspore-ci-bot ec9fc5fb3b !16384 Fix python multiprocessing with shared memory for edge case
From: @robingrosman
Reviewed-by: @mikef
Signed-off-by: @mikef
2021-05-18 10:30:19 +08:00
Lixia Chen 8812111f94 change shared mem perm to 600 & minor other fixes 2021-05-17 17:19:55 -04:00
wangnan39@huawei.com 2761776699 add check for optimizer parameters 2021-05-17 22:23:03 +08:00
huangbingjian 81624dd501 add more error report to abstract join 2021-05-17 21:14:17 +08:00
mindspore-ci-bot 9d423638b3 !16466 Fix code check of python pass
From: @irmo
Reviewed-by: @ginfung,@zh_qh
Signed-off-by: @zh_qh
2021-05-17 21:00:16 +08:00
mindspore-ci-bot 601322356b !15638 VOCDataset need to add img_id for evaluation
From: @xiefangqi
Reviewed-by: @liucunwei
Signed-off-by: @liucunwei
2021-05-17 19:33:24 +08:00
xiefangqi 9b846c7873 voc add image_id column 2021-05-17 16:09:04 +08:00
huanghui 53e32077c1 fix code check 2021-05-17 11:20:35 +08:00
mindspore-ci-bot 8cfb5b8aea !16332 Add environment variable to enable recursive evaluate.
From: @zh_qh
Reviewed-by: @ginfung,@kingxian
Signed-off-by: @ginfung
2021-05-17 11:00:40 +08:00
RobinGrosman ca88d7d76a fix edge case and docstring in minddata multiprocessing shared memory code 2021-05-16 06:46:44 -07:00
Zhang Qinghua bea16101c3 Add environment variable to enable recursive evaluate. 2021-05-14 20:01:19 +08:00
jonyguo 9452142b50 enhance: enable shuffle infile / files when mindrecord 2021-05-14 17:41:18 +08:00
mindspore-ci-bot b5cddcb8cc !16036 Improve performance of multi-processing option for dataset
From: @robingrosman
Reviewed-by: 
Signed-off-by:
2021-05-13 01:41:16 +08:00
RobinGrosman 9a7d1fc034 Optimize multi-process in minddata by passing data via shared memory instead of multiprocessing.queue 2021-05-11 15:16:29 -07:00
mindspore-ci-bot 8c5680360d !16200 [Debugger] Fix empty result when reading non-existent tensor
From: @tina_mengting_zhang
Reviewed-by: @john_tzanakakis,@robingrosman
Signed-off-by: @john_tzanakakis
2021-05-12 03:08:00 +08:00
mindspore-ci-bot ebed057e72 !16196 If Imagefolder input directory is not right, will occur out of memory error
From: @xiefangqi
Reviewed-by: @ms_yan,@heleiwang,@jonyguo
Signed-off-by: @jonyguo
2021-05-11 17:36:28 +08:00
mindspore-ci-bot 4c71d9d1df !15675 add more tensor methods
From: @yanglf1121
Reviewed-by: @liangchenghui
Signed-off-by:
2021-05-11 14:32:29 +08:00
mindspore-ci-bot 9b4ff99305 !16158 modify check strategy for scatter update
From: @yangzhenzhang
Reviewed-by: @kisnwang,@stsuteng
Signed-off-by: @stsuteng
2021-05-11 09:41:07 +08:00
mindspore-ci-bot d3a175e30f !16078 Modify Normalize to accept images with more than 3 channels
From: @hfarahat
Reviewed-by: @mikef,@robingrosman
Signed-off-by: @robingrosman
2021-05-11 02:03:34 +08:00
TinaMengtingZhang a44376f42a Fix empty tensor issue when reading an non-existing node in sync dump mode 2021-05-10 13:59:05 -04:00
xiefangqi e0148f7181 fix minddata directory path problem 2021-05-10 22:16:37 +08:00
yepei6 acbd7eb023 add more tensor methods 2021-05-10 14:40:53 +08:00
yangzhenzhang 6aa3859131 modify check strategy for scatter update 2021-05-10 11:22:00 +08:00
mindspore-ci-bot ee885b4e58 !15180 enable not fully use opt shard
From: @gong_zi_yan
Reviewed-by: @stsuteng
Signed-off-by: @stsuteng
2021-05-08 14:57:58 +08:00
mindspore-ci-bot b86747b9a6 !15813 Convert some pipeline Ops to be inlined
From: @hfarahat
Reviewed-by: 
Signed-off-by:
2021-05-08 05:30:39 +08:00
hesham fd943bbdc0 Modify normalize to support multiple of channels 2021-05-07 17:06:26 -04:00
Ziyan 2a752f24bf enable not fully use opt shard 2021-05-07 15:32:48 +08:00
mindspore-ci-bot d76574bbbf !15973 parallel env variables check
From: @yao_yf
Reviewed-by: @yangzhenzhang,@stsuteng
Signed-off-by: @stsuteng
2021-05-07 12:13:41 +08:00
mindspore-ci-bot e0422e972e !16019 If dataset_dir is not a dir or something else validation is not right somehow
From: @xiefangqi
Reviewed-by: @ms_yan,@liucunwei,@jonyguo
Signed-off-by: @liucunwei
2021-05-07 11:57:31 +08:00
mindspore-ci-bot b85bf60eb8 !15727 VOCDataset read xml will filter invalid bbox automationally
From: @xiefangqi
Reviewed-by: @heleiwang,@jonyguo
Signed-off-by: @jonyguo
2021-05-07 09:50:53 +08:00
yao_yf e967f1939b parallel envs variable check 2021-05-07 09:12:53 +08:00
mindspore-ci-bot eb08d2e952 !16024 dataset: modify some error msg in dataset module
From: @ms_yan
Reviewed-by: @jonyguo,@heleiwang
Signed-off-by: @jonyguo
2021-05-07 09:12:46 +08:00
ms_yan 34dcd2b463 modify log of dataset part 2021-05-06 23:33:17 +08:00
xiefangqi b6e1fceedc minddata modify dir validation 2021-05-06 21:35:31 +08:00
Lixia Chen 5967cb2cd7 Unblock mindrecord under cache 2021-05-05 13:28:13 -04:00
hesham e4c0bd51a5 Modify concat to be an inlined Op
Modify zip to be an inlined Op
Modify take to be an inlined Op
Modify skip to be an inlined Op
Modify rename to be an inlined Op
Modify filter to consume child's rows using one thread
2021-05-04 12:45:18 -04:00
mindspore-ci-bot c13ed20dad !15907 add type convert during load checkpoint
From: @changzherui
Reviewed-by: @kingxian,@zh_qh
Signed-off-by: @kingxian
2021-04-30 15:52:41 +08:00
mindspore-ci-bot ee3012fd3f !15303 add lsq quantization method
From: @erpim
Reviewed-by: 
Signed-off-by:
2021-04-30 11:26:42 +08:00
mindspore-ci-bot ac912672bf !15871 Support pass args or|and kwargs for OP CreateInstance.
From: @zh_qh
Reviewed-by: @ginfung,@hwhewei
Signed-off-by: @hwhewei
2021-04-30 10:39:04 +08:00
mindspore-ci-bot cefcf6c2ae !15931 Add eager tests for transform ops, and add a shape comparison in the EXPECT_MSTENSOR_EQ Macro
From: @alexyuyue
Reviewed-by: @nsyca,@robingrosman
Signed-off-by: @robingrosman
2021-04-30 06:15:12 +08:00
alex-yuyue 9ef4331f56 Add eager tests for transform ops
Signed-off-by: alex-yuyue <yue.yu1@huawei.com>
2021-04-29 13:19:24 -04:00
changzherui 85025804ef add type convert during load checkpoint 2021-04-29 17:38:16 +08:00
Zhang Qinghua cd7f7d40fb Support pass args or|and kwargs for OP CreateInstance. 2021-04-29 14:14:58 +08:00
mindspore-ci-bot 33966f0cd5 !15744 Add encryption for ckpt file
From: @liu_luobin
Reviewed-by: @baochong,@chongbao
Signed-off-by:
2021-04-29 11:18:50 +08:00
mindspore-ci-bot 577cf7cd1d !15758 [Offline Debug] Add UT tests to the offline debugger.
From: @islam_amin
Reviewed-by: @john_tzanakakis,@tom__chen,@robingrosman
Signed-off-by: @john_tzanakakis
2021-04-29 04:33:02 +08:00
Islam Amin 6bb740f137 added GPU UT tests 2021-04-28 13:03:51 -04:00
liuluobin a40229d171 Add encryption for ckpt file 2021-04-28 16:24:29 +08:00
mindspore-ci-bot 78fcdbc7c9 !15790 modify scatter update op
From: @yangzhenzhang
Reviewed-by: @kisnwang,@stsuteng
Signed-off-by: @stsuteng
2021-04-28 10:48:19 +08:00
mindspore-ci-bot b663bbd9ca !15597 [MD] save op support variable dimension 0
From: @liyong126
Reviewed-by: @liucunwei,@jonyguo
Signed-off-by: @liucunwei
2021-04-28 09:38:58 +08:00
mindspore-ci-bot cbca36d723 !15766 modify tile op
From: @yangzhenzhang
Reviewed-by: @kisnwang,@stsuteng
Signed-off-by: @stsuteng
2021-04-28 09:17:17 +08:00
Erpim 90d5d5dab3 add lsq quantization method 2021-04-27 20:48:04 +08:00