Nigel
/
mindspore
forked from mindspore-Ecosystem/mindspore
0
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity
56,551 Commits 24 Branches 0 Tags 3.9 GiB
Commit Graph

1312 Commits

Author SHA1 Message Date
jonyguo 4d1db74ff3 fix: read multi mindrecord files which contains one empty 2022-07-20 09:21:39 +08:00
i-robot 2d2ca4ddb7
!38122 [MD][UT] Trans Unified ops - Enhance interpolation mode test coverage 
Merge pull request !38122 from cathwong/ckw_xtranuni_ut_interp
 2022-07-15 18:46:17 +00:00
Cathy Wong 5637862059 [MD][UT] Trans Unified ops - Enhance interpolation mode test coverage 2022-07-15 10:47:45 -04:00
ms_yan f53b67ed78 add Graph and InMemoryGraphDataset 2022-07-13 22:33:33 +08:00
i-robot fd16733445
!37825 [MD] CutOut: image_utils.cc's Erase - replace SetItemAt to fix performance issue 
Merge pull request !37825 from cathwong/ckw_xtranuni_cutout_perf
 2022-07-12 07:39:58 +00:00
Cathy Wong 2e3ea52a22 [MD] CutOut: image_utils.cc's Erase - replace SetItemAt to fix performance issue 2022-07-11 20:59:48 -04:00
i-robot 3638e7fb88
!35858 add concat batch function 
Merge pull request !35858 from jiangshuqiang/add_concat_batch
 2022-07-09 09:20:09 +00:00
i-robot 9b50df63df
!35847 [FIX] [BUG] [MD] [FUNC] RandomEqualize 
Merge pull request !35847 from yangwm/randomequalize
 2022-07-08 09:46:31 +00:00
Xiao Tianci fb474f0f1e fix several bugs 2022-07-07 20:19:33 +08:00
shenyu 2c9cebab70 fix bug function 2022-07-06 14:53:47 +08:00
赵东婕 8c68998d19 update scripts/fl_restful_tool.py. 2022-07-06 12:21:04 +08:00
Cathy Wong 8e045ba83d [MD][UT] Transforms Unification - Verify selected op implementation 2022-07-05 20:53:07 -04:00
jiangshuqiang 90e58867ec add concat batch function 2022-07-05 15:18:02 +08:00
i-robot d7c72b6482
!37033 Support dict type data in noop mode 
Merge pull request !37033 from ZPaC/support-noop-dict-data
 2022-07-01 10:01:10 +00:00
ZPaC 75f11f8e6f Support dict type data in noop mode 2022-07-01 14:40:21 +08:00
liu-yongqi-63 0519f45bbe Fixed the error reported when expanding the ExpandDims operator when the scatter operator nested vmap errors 2022-07-01 09:30:26 +08:00
i-robot f9c4f425c1
!36853 Support non-numpy return in MP pyfunc 
Merge pull request !36853 from h.farahat/mp_to_numpy
 2022-06-30 12:41:24 +00:00
hesham 9ccd72b646 Make MP support non-numpy return 2022-06-29 10:07:44 -04:00
i-robot b6d4a45d1c
!36852 [MD] ResizeOp - Tests MD5 update 
Merge pull request !36852 from danishfarid/md5_resize_fix
 2022-06-29 13:07:11 +00:00
i-robot 2a9a8dde64
!36652 [FIX][BUG][MD][FUNC]RandomAutoContrast 
Merge pull request !36652 from yangwm/auto
 2022-06-29 06:22:53 +00:00
i-robot 61423da29c
!36734 [MD] Unified UniformAugment op: allow Pyfunc in transforms list 
Merge pull request !36734 from cathwong/ckw_xtranuni_code13_uniformaug
 2022-06-29 03:14:14 +00:00
danishfarid 83dc3faed3 md5_fix 
fix path
 2022-06-28 20:48:58 +00:00
Cathy Wong a61c709d00 [MD] Unified UniformAugment op: allow Pyfunc in transforms list 2022-06-28 14:07:32 -04:00
i-robot 69a39ecfb7
!36725 [MD][AT] File already exists Warn Message 
Merge pull request !36725 from harshvardhangupta/at_file_exist
 2022-06-28 17:24:55 +00:00
i-robot 1aea416e5f
!36446 [MD] Multi-Channel NormalizePad Op 
Merge pull request !36446 from harshvardhangupta/multi_ch_normalize_pad
 2022-06-28 17:10:12 +00:00
shenyu a5b3abab44 [FIX][BUG][FUNC] randomautocontrast 2022-06-28 16:35:01 +08:00
harshvardhangupta c96df0b9ad support multi-channel 2022-06-27 14:27:44 -04:00
harshvardhangupta 2b6a726c53 init 2022-06-27 10:27:20 -04:00
i-robot c9e2dda3d4
!35848 [BUG][MD][FUNC] RandomAdjustSharpness 
Merge pull request !35848 from yangwm/randomsharpness
 2022-06-27 11:30:33 +00:00
i-robot 7f8acb2f25
!36447 [MD] ResizeOp c++ aspect ratio calc fix 
Merge pull request !36447 from danishfarid/resize_c++_fix
 2022-06-24 17:29:46 +00:00
i-robot 9d13aec96b
!36445 [MD] fix: add input type check in eager mode 
Merge pull request !36445 from Mohammad Motallebi/fix_eager_type_check
 2022-06-24 13:20:58 +00:00
i-robot b1d99b0ed3
!36368 Decode: validation & support formats, Fix WithBbox op bugs 
Merge pull request !36368 from luoyang/fix-decode
 2022-06-24 02:06:59 +00:00
danishfarid e9d61667e1 commit 1 - Resize fix 
lint_fix

updated test file

lint fix

updated test file

fixed path

fix lint Majun
 2022-06-23 23:52:37 +00:00
mhmotallebi 8a3c817bb5 [MD] fix: check input type before creating tensor in vision eager 2022-06-23 17:23:59 -04:00
i-robot 1dd2f514e6
!36295 [MD] Support multi-channel images in CutOut op 
Merge pull request !36295 from harshvardhangupta/cutout_multi_channel
 2022-06-23 19:51:13 +00:00
i-robot 89d53bac85
!36285 Fix several bugs in old release 
Merge pull request !36285 from xiaotianci/fix_old_release
 2022-06-23 11:56:00 +00:00
YangLuo f275547ecd Decode: validation & support formats, Fix WithBbox op bugs 2022-06-23 17:07:39 +08:00
i-robot f90afe0f7d
!36370 [MD] Update check_tensor_op error message 
Merge pull request !36370 from cathwong/ckw_xtranuni_code12_randomapply
 2022-06-23 07:41:20 +00:00
i-robot e4ac46c48e
!36296 [MD] Eager Mode - invalid user input fix, UT enhancements 
Merge pull request !36296 from cathwong/ckw_xtranuni_code11_eager_list
 2022-06-23 06:21:40 +00:00
Cathy Wong 7171799bf1 [MD] Update check_tensor_op error message 2022-06-22 17:15:16 -04:00
hesham 70967b475d support multi-channel images in Cutout 2022-06-22 14:57:30 -04:00
Xiao Tianci 5993a3c52f fix several bugs in old release 2022-06-22 19:58:00 +08:00
i-robot fb52a6dbdb
!36172 Fix performance problem of Map's python multiprocessing 
Merge pull request !36172 from h.farahat/mp_problem2
 2022-06-22 02:43:26 +00:00
Cathy Wong 591640519b [MD] Eager Mode - invalid user input fix, UT enhancements 2022-06-21 17:04:43 -04:00
i-robot 3b035ebd9d
!36237 [MD] RandomResizedCrop - handle Interpolation and image input type mismatch 
Merge pull request !36237 from cathwong/ckw_xtranuni_code9_randomreizedcrop
 2022-06-21 15:32:55 +00:00
i-robot 976e13eff8
!36200 Fix PadToSize after Decode(to_pil=True) 
Merge pull request !36200 from xiaotianci/fix_pad_to_size
 2022-06-21 02:56:45 +00:00
Cathy Wong b77e8b9f08 [MD] RandomResizedCrop - handle Interpolation and image input type 
mismatch
 2022-06-20 20:41:42 -04:00
Cathy Wong 1230a962b9 [MD] MapDataset - correct count of old transforms 2022-06-20 16:12:53 -04:00
i-robot 0167368eb0
!36142 [MD] ToTensorOp: Add error msg for unsupported input image types: uint32, int64, uint64, string. 
Merge pull request !36142 from cathwong/ckw_xtranuni_code8_totensor_more
 2022-06-20 16:19:24 +00:00
Xiao Tianci bf7fc7064d fix PadToSize with Decode 2022-06-20 15:41:36 +08:00
hesham f66a4f7038 mp sharedQueue2 2022-06-19 22:24:25 -04:00
i-robot 4b4e53ec5a
!36025 [MD] decode string array in cpp when creating numpy from tensor - eager mode 
Merge pull request !36025 from Mohammad Motallebi/text_decode_cpp
 2022-06-18 11:38:02 +00:00
i-robot cf09e3582e
!35963 Fix HWC2CHW with 2 dim tensor 
Merge pull request !35963 from xiaotianci/fix_hwc
 2022-06-18 03:17:47 +00:00
Cathy Wong 427fb21057 [MD] ToTensorOp: Add error msg for unsupported input image types: uint32, int64, uint64, string. 2022-06-17 17:17:31 -04:00
i-robot 8af9b70f47
!36030 [MD] ToTensor, TypeCast and ToType fixes, plus UT enhancements 
Merge pull request !36030 from cathwong/ckw_xtranuni_code7_to_tensor_float16
 2022-06-17 16:28:47 +00:00
shenyu 98c98985cf fix bug function 2022-06-17 11:43:28 +08:00
mhmotallebi 96e21f6ed4 [MD] decode numpy strings in cpp for eager mode 2022-06-16 18:02:43 -04:00
Cathy Wong b718bc7cfa [MD] ToTensor, TypeCast and ToType fixes, plus UT enhancements 2022-06-16 16:30:17 -04:00
Xiao Tianci acf2e31f75 fix Hwc2Chw with 2 dim tensor 2022-06-16 11:51:13 +08:00
Cathy Wong b81ec04a50 [MD] Transform Unification - HWC2CHW op - remove Python implementation. 
Enhance HWC2CHW UTs
 2022-06-14 10:55:38 -04:00
i-robot 5b57829f11
!35595 per_batch_map input column optimization 
Merge pull request !35595 from 刘勇琪/master-decrypt-column
 2022-06-10 06:57:41 +00:00
i-robot 01642da438
!35426 Refactor Python Multiprocessing code of Map 
Merge pull request !35426 from h.farahat/new_work
 2022-06-09 19:27:27 +00:00
liu-yongqi-63 0dfd42153f per_batch_map input column optimization 2022-06-09 13:17:02 +08:00
i-robot 3e11eded32
!33594 [assistant][Resample] 
Merge pull request !33594 from chenchen/Resample
 2022-06-09 02:08:54 +00:00
hesham 13ce3b6fd4 python MP new 2022-06-08 10:52:44 -04:00
Xiao Tianci 2df7c5a193 fix bugs of Pad 2022-06-08 15:57:28 +08:00
chenx2ovo 6e62ed181e [feat][assistant[I3CEG6] add op resample 2022-06-08 11:46:12 +08:00
i-robot 8026412d84
!35305 Fix some remaining problems of Pad 
Merge pull request !35305 from xiaotianci/fix_pad
 2022-06-06 07:28:21 +00:00
i-robot e29229320e
!35284 [MD] Adding docstrings for minddata UT Python Part 4 
Merge pull request !35284 from davidanugraha/add_dataset_test_comment_part4
 2022-06-04 17:46:20 +00:00
David 229603b4b5 Added docs to ut part 4 2022-06-02 21:37:15 -04:00
Xiao Tianci 72fafb9251 fix remaining problems of Pad and PadToSize 2022-06-02 17:19:30 +08:00
i-robot 59e7689876
!34991 Remove SoftDVPP related APIs 
Merge pull request !34991 from xiaotianci/remove_soft_dvpp
 2022-06-02 09:00:58 +00:00
i-robot a3731db766
!35269 Fixed text.SlidingWindow operator has a problem when width>len(list) 
Merge pull request !35269 from 刘勇琪/master
 2022-06-02 07:26:59 +00:00
i-robot 4b0a2ea899
!35208 [MD] Transform Unification - Phase 3: Update Smoke tests and remaining UTs 
Merge pull request !35208 from cathwong/ckw_xtranuni_ut_nondataset
 2022-06-02 01:03:36 +00:00
Xiao Tianci 74a1f06362 remove soft dvpp 2022-06-01 17:45:58 +08:00
liu-yongqi-63 c826b1ec38 Fixed text.SlidingWindow operator has a problem when width>len(list) 2022-06-01 09:13:15 +08:00
Cathy Wong f4fd0300c6 [MD] Transform Unification - Phase 3: Update Smoke tests and remaining UTs 2022-05-31 16:14:32 -04:00
i-robot 875f06f060
!35206 [MD] Transform Unification Feature - ToTensor op: Remove Python implementation and regenerate md5 golden result files 
Merge pull request !35206 from cathwong/ckw_xtranuni_code4_to_tensor_new_golden
 2022-05-31 03:08:44 +00:00
i-robot d7ac70d50a
!35099 [assistant][Vad] 
Merge pull request !35099 from chenchen/vad
 2022-05-31 02:48:32 +00:00
i-robot fedccd0d63
!34529 Support specific data set decryption 
Merge pull request !34529 from 刘勇琪/master-decrypt-column
 2022-05-31 02:47:19 +00:00
i-robot a4785b9baa
!35134 [MD] Adding docstrings for minddata UT Python Part 3 
Merge pull request !35134 from davidanugraha/add_dataset_test_comment_part3
 2022-05-30 22:29:05 +00:00
David 34d3755e20 Added docs to ut part 3 2022-05-30 16:16:46 -04:00
i-robot 2886ecd6d1
!35147 [MD] Transform Unification Feature - OneHot op: Add smoothing_rate parm 
Merge pull request !35147 from cathwong/ckw_smoothing_rate
 2022-05-30 17:23:32 +00:00
Cathy Wong 1136ee3ffd [MD] Transform Unification Feature - ToTensor op: Remove Python implementation, regenerate golden files 2022-05-30 11:13:29 -04:00
i-robot f2df6b621c
!34834 Add PadToSize transform 
Merge pull request !34834 from xiaotianci/add_pad_to_size
 2022-05-30 11:42:45 +00:00
liu-yongqi-63 ea336c3109 Support specific data set decryption 2022-05-30 14:42:35 +08:00
chenx2ovo e13df22d96 [feat][assistant][I3CEGK] add op vad 2022-05-30 10:42:33 +08:00
hesham f9128c2f16 [MD] Transform Unification Feature - OneHot op: Add smoothing_rate parm 2022-05-29 22:19:04 -04:00
i-robot 57f66960ae
!35081 fix-output_shape_doc 
Merge pull request !35081 from luoyang/fix-output_shape_doc
 2022-05-30 01:22:29 +00:00
Xiao Tianci f34f053aea add PadToSize transform 2022-05-28 11:07:51 +08:00
YangLuo 4e31675447 fix-output_shape_doc 2022-05-28 10:03:46 +08:00
i-robot 8f57454372
!35028 [MD] Fix hanging problem in offload 
Merge pull request !35028 from h.farahat/offload_hang
 2022-05-27 17:25:06 +00:00
David 5f1849fd1c Added docs to ut part 2 2022-05-27 09:05:14 -04:00
i-robot 43b5ad6f12
!34901 Add flatten for flat_map 
Merge pull request !34901 from xiaotianci/fix_flat_map
 2022-05-27 03:37:01 +00:00
hesham 23e85ff083 avoid hangs when offload is true with pyfuncs and dict iterator 2022-05-26 23:16:16 -04:00
Xiao Tianci c564e840fb add flatten in flat_map 2022-05-26 11:17:32 +08:00
Cathy Wong b9396a32b7 [MD] Update import transforms statements in dataset UT 2022-05-25 19:51:26 -04:00
i-robot 797c21ec6c
!34539 [MD] Transform Unification: Phase 1C - Minor Code and UT Updates 
Merge pull request !34539 from cathwong/ckw_xtranuni_code3_typecast
 2022-05-25 18:05:40 +00:00
i-robot 1ecc6538d1
!34537 output_shape add param estimate 
Merge pull request !34537 from luoyang/dynamic_outputshape
 2022-05-25 02:37:32 +00:00
Cathy Wong b8e7c2acaf [MD] Transform Unification: Phase 1C - Minor Code Fixes and UT Updates 2022-05-24 22:24:38 -04:00
i-robot 7676d9958b
!34608 [MD] Adding docstrings for minddata UT Python 
Merge pull request !34608 from davidanugraha/add_dataset_test_comment
 2022-05-25 02:10:32 +00:00
David ee0a815990 Added docs for ut files 2022-05-24 17:43:53 -04:00
i-robot 1ade2e9f25
!34235 [assistant][MelScale] 
Merge pull request !34235 from QingfengLi/MelScale
 2022-05-24 12:08:00 +00:00
YangLuo e1df0d9688 output_shape add param estimate 2022-05-24 09:09:30 +08:00
chenx2ovo d62655e199 [fix] [assistant] [I4ZOCP] add validation for input param 2022-05-23 16:06:50 +08:00
jonyguo 0718a283c8 filter dumplicate log from python operation 2022-05-20 14:46:46 +08:00
i-robot 17fb1b2ea4
!34468 fix typecast on validate string 
Merge pull request !34468 from luoyang/fix_typecast
 2022-05-17 11:03:33 +00:00
luoyang bc4d57018c fix typecast on validate string 2022-05-16 23:21:06 +08:00
i-robot 4cde40189c
!34321 [MD] Normalize md5 tests - new golden files, unskip tests 
Merge pull request !34321 from cathwong/ckw_xtransuni_ut_new_golden_unskip_md5
 2022-05-13 21:00:38 +00:00
i-robot 2427626f10
!34250 Return -1 as unknown batch size for callable batch_size param 
Merge pull request !34250 from luoyang/get_batch_size
 2022-05-13 06:45:07 +00:00
Cathy Wong 1caa2012bb [MD] Normalize md5 tests - new golden files, unskip tests 2022-05-12 16:02:36 -04:00
Cathy Wong 942001ffc6 [MD] Tranform Unification - skip normalize md5 tests 2022-05-12 12:56:01 -04:00
i-robot 8d44248535
!34199 [MD] Transforms Unificiation Feature - Phase 1B - Minor Code Fixes and UT Updates 
Merge pull request !34199 from cathwong/ckw_xtranuni_code2_cutout_totensor
 2022-05-12 13:51:00 +00:00
luoyang 3cb854abc7 Return -1 as unknown batch size for callable batch_size param 2022-05-12 15:38:08 +08:00
fr199787 9e5eb88799 [faet] [assistant] [I4S2FH] add new operator GetImageSize 2022-05-12 11:17:38 +08:00
i-robot 5e31a54cbb
!32787 [assistant][ops]Add GetImageNumChannels 
Merge pull request !32787 from yangwm/channels
 2022-05-12 02:07:12 +00:00
Cathy Wong d08846176b [MD] Transforms Unificiation Feature - Phase 1B - Minor Code Fixes and UT updates 2022-05-11 16:21:35 -04:00
jonyguo 8ba8c930e7 fix: md AddressSanitizer 2022-05-10 18:45:42 +08:00
shenyu d69c2ce0d1 [feat] [assistant] [I4S2FG] add new operator GetImageNumChannels 2022-05-09 22:32:05 +08:00
i-robot 0828efe02e
!33811 [MD] Transform Unification Feature - Phase 2B - test_[r-z]*.py Python UT updates 
Merge pull request !33811 from cathwong/ckw_xtranuni_phase2b_uts_r_to_z
 2022-05-05 18:48:57 +00:00
Cathy Wong eb4fd20bac [MD] Transform Unification Feature - Phase 2A - test_[a-q]*.py Python UT updates 2022-05-04 15:24:18 -04:00
Cathy Wong 22adb200b2 [MD] Transform Unification Feature - Phase 2 - test_[r-q]*.py Python UT updates 2022-05-04 09:16:46 -04:00
i-robot fd22d69ca7
!33475 Remove test data containing sensitive word 
Merge pull request !33475 from xiaotianci/fix_examples
 2022-04-28 02:05:09 +00:00
i-robot 37a28be84a
!33574 [MD] fix test case 
Merge pull request !33574 from liyong126/fix_test_case1
 2022-04-27 20:59:48 +00:00
Xiao Tianci 0dc85a15b9 remove sensitive word in test data 2022-04-27 10:51:44 +08:00
liyong 29855385a5 fix random dataset 2022-04-26 19:48:17 +08:00
i-robot fcb0319747
!33270 [assistant][InverseMelScale] 
Merge pull request !33270 from chenchen/InverseMelScale
 2022-04-26 08:55:58 +00:00
chenx2ovo cd8395f352 [feat][assistant][I3CEG2] add op inversemelscale 2022-04-25 18:58:38 +08:00
Cathy Wong 907194dfb8 [MD] UT Updates - more eager tests, minor updates 2022-04-18 22:03:32 -04:00
liyong 830e76656a fix mindrecord rename issue 2022-04-12 19:36:24 +08:00
i-robot c1bacb39dc
!32644 [MD] fix doc issue2 in master 
Merge pull request !32644 from liyong126/fix_doc_issue2
 2022-04-08 06:32:34 +00:00
i-robot 0af661659c
!32534 [MD][Autotune] Add rank to filename 
Merge pull request !32534 from harshvardhangupta/rank_id_suffix_at_config
 2022-04-08 02:51:17 +00:00
liyong 2172934177 fix doc issue2 2022-04-08 10:46:08 +08:00
harshvardhangupta 6b8a76c59d add rank to filename 2022-04-07 16:12:06 -04:00
jonyguo b8205ee995 add testcase 2022-04-07 09:49:23 +08:00
liyong 3191160eca fix some issue 2022-04-06 17:48:55 +08:00
unknown a9e6a151da fix dataset two level pipeline hang problem 2022-04-06 12:39:48 +08:00
liu-yongqi-63 b125369f1e Fix the error is reported when the file name is directly transferred 2022-04-02 11:51:57 +08:00
i-robot d73ccd17f6
!32328 [MD][Offload] Reduce Offload UT Computation Time 
Merge pull request !32328 from alashkari/update-ut
 2022-03-31 14:55:28 +00:00
i-robot c3d7c425e9
!29856 [assistant][GriffinLim] 
Merge pull request !29856 from chenchen/g2
 2022-03-31 09:33:52 +00:00
yanghaoran c44de68591 takedown test_datasets_generator to ensure gate stability 2022-03-31 16:56:03 +08:00
chenx2ovo e19dc02b47 [feat][assistant][I3CECZ] add op griffinlim 2022-03-31 11:30:26 +08:00
Rescue 60d895fcd8 Decrease test_map_offload.py computation time. 2022-03-30 16:40:19 +00:00
jonyguo 4378014ffb fix: when return numpy in GeneratorDataset 2022-03-30 20:21:41 +08:00
liyong f703f66f6c fix gis issue 2022-03-29 11:17:01 +08:00
i-robot 3f18180606
!31742 [MD][Autotune] Change autotune config 
Merge pull request !31742 from harshvardhangupta/at_config_suggestion
 2022-03-25 21:00:12 +00:00
harshvardhangupta 0086f4ee05 Modify the Dataset AutoTune config #31742 2022-03-25 09:03:38 -04:00
i-robot 1af8b365de
!31880 [MD] add OBSMindDataset 
Merge pull request !31880 from liyong126/add_obs_mindrecord_dataset
 2022-03-25 06:50:58 +00:00
liyong 419109cf0a add OBSMindDataset 2022-03-25 11:03:53 +08:00
jonyguo 317c48a933 add log warning and op error check 2022-03-25 09:26:29 +08:00