Commit Graph

823 Commits

Author SHA1 Message Date
zetongzhao 10fdbd6c03 resize queue and test 2021-10-19 11:43:53 -04:00
yangyueren 6594b90a58 [feat][assistant][I3J6VE] add new data operator PhotoTour 2021-10-19 15:31:32 +00:00
chenx2ovo 77caf907c4 [feat][assistant][I3CEGR] add op random lighting 2021-10-19 20:38:01 +08:00
i-robot 690642d672 !18664 [assistant][ops]New operator implementation, include Places365Dataset
Merge pull request !18664 from Wangsong95/places365_dataset
2021-10-19 07:06:35 +00:00
i-robot 1606f19dfb !18265 [assistant][ops] Add new audio operator CreateDct
Merge pull request !18265 from 杨旭华/CreateDct
2021-10-19 06:30:18 +00:00
tophand 4f4777f0da [feat][assistant][I3J6U2] Add new audio operator CreateDct 2021-10-18 03:23:18 -07:00
zx d13d2413f5 [feat][assistant][I3J6UV] add new audio operator Phaser 2021-10-16 08:45:12 +08:00
yangyueren 951193c308 [feat][assistant][I3J6VF] add new data operator Places365 2021-10-15 14:56:09 +00:00
robert_luo_yibo 75e2c3041c [feat][assistant][I3CEG3] add new data OP MuLawEncoding 2021-10-14 20:24:18 +08:00
i-robot 1b9b85bbb2 !17851 [assistant][ops]New operator implementation, include FakeImageDataset
Merge pull request !17851 from 张璇/fakeimage_dataset
2021-10-13 09:22:55 +00:00
G-Dragon-Liu d2f22a8726 [feat][assistant][I3J6V3] add new data operator FakeImage 2021-10-12 14:00:43 +00:00
zx 9b7d7be65c [feat][assistant][I3J6UU] add new audio operator Overdrive 2021-10-12 18:10:31 +08:00
li-qiyao 2283442855 [feat][assistant][I3J6U9] add new data operator Flanger 2021-10-08 05:03:31 -07:00
i-robot 8598d08351 !17820 [assistant][ops]New operator implementation, include EMnistDataset
Merge pull request !17820 from 张璇/emnist_dataset
2021-10-08 07:18:31 +00:00
i-robot 02f9e3e077 !17805 [assistant][ops] Add new audio operator DetectPitchFrequency
Merge pull request !17805 from 杨旭华/DetectPitchFrequency
2021-10-08 03:34:46 +00:00
G-Dragon-Liu 32baa520bf [feat][assistant][I3J6V2] add new data operator EMnist 2021-10-08 02:53:00 +00:00
i-robot 55e165c50e !24517 MD Profiling: Move ProfilingManager from ExecutionTree to TreeConsumer
Merge pull request !24517 from cathwong/ckw_prof_mv_to_consumer6
2021-10-06 14:34:24 +00:00
hesham c3718327f2 Stage 1 2021-10-04 23:34:15 -04:00
Cathy Wong 0665a191dc MD Profiling: Move ProfilingManager from ExecutionTree to TreeConsumer 2021-10-04 11:12:02 -04:00
Zhang Lian b7e6520e62 [feat] [assistant] [I3J6U8] Add a new audio operator DetectPitchFrequency 2021-09-29 15:12:58 +08:00
i-robot dd544a9693 !21406 [MS][crowdfunding]New operator implementation, include TrebleBiquad
Merge pull request !21406 from 张璇/treble_biquad
2021-09-27 14:18:00 +00:00
i-robot d1d4797079 !24204 [MD] sync code for code review
Merge pull request !24204 from liyong126/sync_code
2021-09-27 11:16:03 +00:00
i-robot f16c003dfa !22610 [MS][crowdfunding]New operator implementation, RandomAdjustSharpness
Merge pull request !22610 from yangwm/sharpness
2021-09-27 09:20:05 +00:00
liyong 567f137a39 sync code 2021-09-27 14:31:49 +08:00
zx 3da0f0f487 [feat][assistant][I3J6V0] add new audio operator TrebleBiquad 2021-09-27 13:01:26 +08:00
despicablemme d072a4e4bf [feat] [assistant] [I40GZE] add new data operator RandomAdjustSharpness 2021-09-27 11:19:33 +08:00
i-robot ab567574c4 !21451 [MS][crowdfunding]New operator implementation, include RiaaBiquad
Merge pull request !21451 from 张璇/riaa_biquad
2021-09-27 03:14:16 +00:00
i-robot 2969688382 !20047 [assistant][ops]New operator implementation, include QmnistDataset
Merge pull request !20047 from Wangsong95/qmnist_dataset
2021-09-27 01:36:08 +00:00
despicablemme 7063907bdc [feat] [assidtant] [I40GZN] add new data ops RandomEqualize 2021-09-25 18:00:25 +08:00
zx 1d95e6f480 [feat][assistant][I3J6UY] add new audio operator RiaaBiquad 2021-09-23 21:36:59 +08:00
despicablemme 7a6b75c2e8 [feat] [assistant] [I40GZJ] add new data op RandomAutoContrast 2021-09-23 20:17:39 +08:00
despicablemme b977d8723b [feat] [assistant] [I40GZC] add new data ops RandomInvert 2021-09-22 16:55:14 +08:00
ckczzj 001b68c286 [feat][assistant][I3J6VG] add new data operator QMnist 2021-09-22 12:13:27 +08:00
zetongzhao a0c203df23 fix deser python operation error 2021-09-20 14:11:02 -04:00
葛崇林 a05a5de073 [feat][assistant][I3ALC5] add new method Magphase 2021-09-17 16:02:46 +08:00
chenx2ovo 9e62623b4a [feat][assistant][I3CEGI] add op vol 2021-09-17 09:34:20 +08:00
i-robot 4a553c8d57 !16607 [assistant][Fade]
Merge pull request !16607 from QingfengLi/Fade
2021-09-15 14:54:16 +00:00
zetongzhao 860810b9c4 check python interpreter 2021-09-14 16:25:04 -04:00
chenx2ovo 1a3196b052 [feat][assistant][I3CEGF] add op fade 2021-09-14 19:08:37 +08:00
i-robot 32c2b77595 !19093 [assistant][ops] Add new audio operator Biquad
Merge pull request !19093 from 杨旭华/Biquad
2021-09-14 01:52:01 +00:00
i-robot 3e1d1ff4d3 !18671 [assistant][ops]New operator implementation, include SBUDataset
Merge pull request !18671 from Wangsong95/sbu_dataset
2021-09-13 11:56:13 +00:00
liannai 631f49f044 [feat][assistant][I3T96A] Add new audio operator Biquad 2021-09-13 19:18:50 +08:00
Carry955 0ba84345b0 [feat][assistant][I3J6U3] add new audio operator DCShift 2021-09-11 23:58:18 -07:00
Isaac17555 53fc99b914 [feat][assistant][I3T96B]add new data operator LFilter 2021-09-11 02:05:43 -07:00
YJfuel123 16150d394f [feat][assistant][I3ALd1] add new operator EqualizerBiquad 2021-09-11 10:36:20 +08:00
ckczzj af97ba4a77 [feat][assistant][I3J6VH] add new data operator SBU 2021-09-11 09:58:17 +08:00
i-robot cdde54a146 !23202 codedex and pclint warning fix
Merge pull request !23202 from zetongzhao/warn_fix
2021-09-10 20:01:22 +00:00
i-robot 6d709c7254 !19624 [assistant][ops] Add new dataset operator DIV2K.
Merge pull request !19624 from Rainfor/div2k
2021-09-10 14:59:21 +00:00
wangkc123 3daf3a4d5f [feat] [assistant] [I3CKEK] add new dataset operator DIV2K 2021-09-10 17:55:34 +08:00
robert_luo_yibo ce71699734 [feat][assistant][I3CEG5] add new data OP MuLawDecoding 2021-09-10 15:49:46 +08:00
i-robot 32cc5d79b0 !18670 [assistant][ops]New operator implementation, include USPSDataset
Merge pull request !18670 from Wangsong95/usps_dataset
2021-09-10 03:06:19 +00:00
i-robot f91e1440c8 !17810 [assistant][ops][DeemphBiquadOp]
Merge pull request !17810 from Isaac/DeemphBiquad
2021-09-10 03:05:41 +00:00
zetongzhao e2a5dafaf3 warning fix 2021-09-09 15:18:51 -04:00
ckczzj d7c702e9d4 [feat][assistant][I3J6VL] add new data operator USPS 2021-09-09 21:46:57 +08:00
liyong 0959c54ae9 fix bug in random posterize 2021-09-09 19:50:57 +08:00
i-robot 936940da8e !22666 [assistant][ComplexNorm]
Merge pull request !22666 from QingfengLi/ComplexNorm
2021-09-09 06:41:53 +00:00
Isaac 5dc6c7fe96 [feat][assistant][I3J6U6]add new data operator DeemphBiquad 2021-09-09 14:34:24 +08:00
i-robot 9ad56e86ab !17846 [assistant][ops]Add new operator HighpassBiquad
Merge pull request !17846 from YJfuel123/HighpassBiquad
2021-09-09 02:06:41 +00:00
i-robot a918783f1a !22623 Add Random operation with Multi Field
Merge pull request !22623 from shenwei41/operation_dev
2021-09-09 01:49:04 +00:00
shenwei41 a8d02ab782 Add Random operation with MultiField 2021-09-08 19:29:23 +08:00
i-robot 938324b397 !19362 [assistant][ops] Add new dataset operator Cityscapes.
Merge pull request !19362 from Rainfor/cityscapes
2021-09-08 09:14:43 +00:00
yaoyuan 85a1b55b47 [feat] [assistant] [I3ALCL] add new operator HighpassBiquad 2021-09-08 15:12:40 +08:00
Guo Zhijian 4b413922ac fix: code review new 2021-09-08 09:10:54 +08:00
robert_luo_yibo 3d58da9023 [fix][assistant][I47V0O] fix bug about the wrong output in the operator ComplexNorm 2021-09-01 17:55:38 +08:00
wangkc123 70f741bcc1 [feat] [assistant] [I3CKEM] add new dataset operator Cityscapes 2021-08-31 20:03:37 +08:00
tongyue 329526dea8 [feat][assistant][I3J6U0] add new data operator Contrast 2021-08-30 19:34:37 -07:00
i-robot a47ab8d643 !18073 [assistant][ops]Add new operator LowpassBiquad
Merge pull request !18073 from YJfuel123/LowpassBiquad
2021-08-30 08:33:39 +00:00
shenwei41 1295744524 Add ConvertColor Operation 2021-08-26 21:14:28 +08:00
haizhouye d96d9e64b0 [feat] [assistant] [I3ALCA] add new LowpassBiquad operator
[feat] [assistant] [I3ALCA] add new LowpassBiquad operator

bugfix

fix

fix
2021-08-26 18:49:21 +08:00
i-robot ef195c9de6 !21631 Deserialize part4
Merge pull request !21631 from zetongzhao/deserialize_3
2021-08-23 20:09:23 +00:00
zetongzhao b17464e30c deserialize and tests 2021-08-23 14:11:05 -04:00
ms_yan 36a8886ca2 Revert "[feat] [assistant] [I3T96T] add new Dataset operator CMUARCTICDataset"
This reverts commit b077aa1cab.

Revert "[feat] [assistant] [I3T96X] add new Dataset operator LibriSpeechDataset"

This reverts commit 4e6f7dc97d.

delete pass_registry_test.cc

comment  hiai_nlu_model_multi.pb related  line
2021-08-23 01:46:38 +08:00
djc b077aa1cab [feat] [assistant] [I3T96T] add new Dataset operator CMUARCTICDataset 2021-08-22 16:26:45 +08:00
djc 4e6f7dc97d [feat] [assistant] [I3T96X] add new Dataset operator LibriSpeechDataset 2021-08-22 13:39:37 +08:00
i-robot 0f3de77e93 !21149 Sync MD code clean to master
Merge pull request !21149 from xiefangqi/md_sync_codeclean_master
2021-08-21 06:43:20 +00:00
robert_luo_yibo d837cbee0f [feat][assistant][I3CEG8] add new data operator ComplexNorm 2021-08-20 09:07:21 +08:00
xiefangqi 0e2efc2501 fix minddata code clean 2021-08-19 09:11:30 +08:00
i-robot 1e4dace193 !20683 deserializer 2nd part
Merge pull request !20683 from zetongzhao/deserialize_2
2021-08-17 19:50:43 +00:00
zetongzhao ea1f2d0d7c deserialize and tests 2021-08-16 10:23:36 -04:00
Xiao Tianci aae886bcd7 fix crowd contributing code 2021-08-16 19:33:10 +08:00
liyong c5fefa7523 fix ut bug 2021-08-16 09:15:48 +08:00
chenx2ovo 53615010a3 [feat][assistant][I3CEGG] add new data OP FrequencyMasking 2021-08-12 15:28:30 +08:00
chenx2ovo e72acf362c [feat][assistant][I3CEGH] add op timemasking 2021-08-10 12:06:07 +08:00
i-robot dc626617fc !19526 [assistant][TimeStretch]
Merge pull request !19526 from QingfengLi/TimeStretch
2021-08-09 09:40:38 +00:00
chenx2ovo 04705e5b0d [feat][assistant][I3CEGC] add op timestretch 2021-08-07 22:53:41 +08:00
i-robot c84e09cf45 !21072 Deserialize - python binding
Merge pull request !21072 from zetongzhao/deserialize_1
2021-08-06 21:47:09 +00:00
zetongzhao 4f199b548c deserialize and tests 2021-08-06 13:35:27 -04:00
i-robot 51a1ca61f2 !21357 [MD] fix SlicePatches when dealing with some special tensor.
Merge pull request !21357 from liyong126/fix_bug
2021-08-06 08:58:27 +00:00
liyong 5fea72473c fix HW1 issue 2021-08-06 14:07:26 +08:00
chenx2ovo aa45e2ebbc [feat] [assistant] [I3CEER] add op AmplitudeToDB 2021-08-06 10:07:29 +08:00
Carry955 98be266585 [feat][assistant][I3J6TQ] add a new operator Angle 2021-08-04 19:41:00 -07:00
i-robot 680e40a43a !16453 [assistant][BassBiquadOp]
Merge pull request !16453 from StyleHang/BassBiquadOp
2021-08-04 10:02:34 +00:00
i-robot f4e07c0dce !17486 [assistant][ops] Add new dataset operator Flickr.
Merge pull request !17486 from Rainfor/wangkc
2021-08-04 08:52:21 +00:00
i-robot d260f38936 !15722 [assistant][AdjustGamma]
Merge pull request !15722 from QingfengLi/adjust_gamma
2021-08-04 03:58:09 +00:00
li-qiyao ba326d76a8 [feat][assistant][I3J6TW] add new data operator BassBiquad 2021-08-03 19:40:12 -07:00
wangkc123 48f5fed1f8 [feat] [assistant] [I3CKEN] add new dataset operator Flickr 2021-08-04 10:20:43 +08:00
chenx2ovo 4191dde45f [feat][assistant][I3CEGM] add op adjust gamma 2021-08-04 02:32:13 +08:00
tongyue 08469e113a [feat][assistant][I3J6TV]add new audio operator BandrejectBiquad 2021-08-03 05:41:53 -07:00
administrator 29d57a89fb [feat][assistant][I3TDRD]add new audio operator BandpassBiquad 2021-08-02 07:08:47 -07:00
Isaac b49e1acc3e [feat] [assistant] [I3J6TM] add new audio operator AllpassBiquad 2021-07-31 16:15:17 +08:00
i-robot 6e84bf0f0f !16449 [assistant][ops] Add new audio operator BandBiquad
Merge pull request !16449 from StyleHang/BandBiquadOp
2021-07-30 03:35:07 +00:00
liannai 5a341e7e27 [feat] [assistant] [I3J6TS] add new audio operator BandBiquad 2021-07-29 17:57:27 +08:00
zetongzhao 8693fd730a deserialize and tests 2021-07-21 08:32:00 -04:00
i-robot 884f5ae1de !19764 [MD] Remove Builder class phase 3
Merge pull request !19764 from harshvardhangupta/rem-build-2
2021-07-12 13:25:34 +00:00
Xiao Tianci a77ed2a5c0 fix the param check of center in Rotate, and change the default value of center from {-1, -1} to {} 2021-07-09 15:24:27 +08:00
harshvardhangupta 52d1212239 remove builder 2021-07-08 16:55:52 -04:00
i-robot a5118ae5f2 !19507 Remove builder from CacheOp, CacheLookupOp, CacheMergeOp
Merge pull request !19507 from lixiachen/remove_cacheop_builders
2021-07-07 21:25:43 +00:00
Lixia Chen 946693e860 Remove builder from cache ops 2021-07-06 17:29:03 -04:00
i-robot 0a6732f6dc !19047 [MD] Remove Builder
Merge pull request !19047 from harshvardhangupta/remove_builder
2021-07-06 17:34:35 +00:00
Harshvardhan Gupta 5bd8fb57fc remove builder 2021-07-05 11:17:37 -04:00
liyong 9a1de0af65 add slice patches in python and c++ 2021-07-05 09:16:47 +08:00
YangLuo 4e9f6d54f7 update cpp api & doc 2021-06-23 11:14:52 +08:00
chenx2ovo 4b98099dcb [feat][assistant][I3CEGQ] add op rgb2bgr 2021-06-21 16:16:50 +08:00
Xiao Tianci a54e65a7ad add Crop Python API 2021-06-10 09:46:57 +08:00
Xiao Tianci 9c4c1c6a77 add Rotate Python API 2021-06-08 22:19:15 +08:00
Zhenglong Li 2c7d1bf603 Implement new gnn feature 2021-06-07 10:20:06 +08:00
mindspore-ci-bot cf27d3c8dc !12972 Removing Builder from dataset ops phase 1
From: @ezphlow
Reviewed-by: 
Signed-off-by:
2021-06-05 03:51:49 +08:00
EricZ 16837a4639 Removing Builder from dataset ops phase 1 2021-06-03 09:09:29 +08:00
Xiao Tianci 9c74d55a49 add HorizontalFlip and VerticalFlip APIs 2021-05-31 15:55:21 +08:00
zetongzhao 728b180329 delete_extra files 2021-05-26 16:41:27 -04:00
mindspore-ci-bot 686225ad2a !16382 Unity Input Order
From: @zetongzhao
Reviewed-by: 
Signed-off-by:
2021-05-26 21:28:36 +08:00
zetongzhao 4cb53823fd all the updated input order
Fix FillOperation to_json function

Fix FillOperation to_json function

serialize tensor without testing

fix unity input order bug, next step is to test the tensor serialization

serialize tensor, testing code in tensor_test.cc

change for tensor serialization

changes after review

changes after review

clang-format the files

rename
2021-05-25 18:26:43 -04:00
Xiao Tianci 3f495bbae1 add GaussianBlur API 2021-05-24 16:27:05 +08:00
mindspore-ci-bot 601322356b !15638 VOCDataset need to add img_id for evaluation
From: @xiefangqi
Reviewed-by: @liucunwei
Signed-off-by: @liucunwei
2021-05-17 19:33:24 +08:00
xiefangqi 9b846c7873 voc add image_id column 2021-05-17 16:09:04 +08:00
jonyguo 9452142b50 enhance: enable shuffle infile / files when mindrecord 2021-05-14 17:41:18 +08:00
mindspore-ci-bot b5cddcb8cc !16036 Improve performance of multi-processing option for dataset
From: @robingrosman
Reviewed-by: 
Signed-off-by:
2021-05-13 01:41:16 +08:00
RobinGrosman 9a7d1fc034 Optimize multi-process in minddata by passing data via shared memory instead of multiprocessing.queue 2021-05-11 15:16:29 -07:00
mindspore-ci-bot 6fcd6cab68 !16119 Fix minddata issues
From: @luoyang42
Reviewed-by: @jonyguo,@liucunwei
Signed-off-by: @jonyguo,@liucunwei
2021-05-11 14:33:29 +08:00
mindspore-ci-bot d3a175e30f !16078 Modify Normalize to accept images with more than 3 channels
From: @hfarahat
Reviewed-by: @mikef,@robingrosman
Signed-off-by: @robingrosman
2021-05-11 02:03:34 +08:00
YangLuo 0a4da64e13 Fix minddata issues 2021-05-10 11:12:00 +08:00
mindspore-ci-bot b86747b9a6 !15813 Convert some pipeline Ops to be inlined
From: @hfarahat
Reviewed-by: 
Signed-off-by:
2021-05-08 05:30:39 +08:00
hesham fd943bbdc0 Modify normalize to support multiple of channels 2021-05-07 17:06:26 -04:00
Lixia Chen 5967cb2cd7 Unblock mindrecord under cache 2021-05-05 13:28:13 -04:00
hesham e4c0bd51a5 Modify concat to be an inlined Op
Modify zip to be an inlined Op
Modify take to be an inlined Op
Modify skip to be an inlined Op
Modify rename to be an inlined Op
Modify filter to consume child's rows using one thread
2021-05-04 12:45:18 -04:00
mindspore-ci-bot 9c7cb65c3b !15801 Split sampler and vision IR files
From: @mhmotallebi
Reviewed-by: 
Signed-off-by:
2021-05-01 05:19:41 +08:00
mindspore-ci-bot cefcf6c2ae !15931 Add eager tests for transform ops, and add a shape comparison in the EXPECT_MSTENSOR_EQ Macro
From: @alexyuyue
Reviewed-by: @nsyca,@robingrosman
Signed-off-by: @robingrosman
2021-04-30 06:15:12 +08:00
mohammad 9ce28cd34c split sampler vision IR files 2021-04-29 16:30:18 -04:00
alex-yuyue 9ef4331f56 Add eager tests for transform ops
Signed-off-by: alex-yuyue <yue.yu1@huawei.com>
2021-04-29 13:19:24 -04:00
YangLuo bf48400c10 unify include file of minddata 2021-04-22 10:59:36 +08:00
Cathy Wong 636809c255 dataset: C++ API ToNumber & Lookup: change data_type parm from string to DataType 2021-04-19 12:35:37 -04:00
mindspore-ci-bot bd69a86931 !15124 Remove unused #include in cc files in MindData
From: @hfarahat
Reviewed-by: @nsyca,@mikef,@robingrosman
Signed-off-by: @robingrosman
2021-04-17 02:19:27 +08:00
mindspore-ci-bot 9c81c5d2a0 !15189 Add more ut for transform ops and subset sampler
From: @alexyuyue
Reviewed-by: 
Signed-off-by:
2021-04-17 01:46:13 +08:00
hesham 98f0f36a20 Removed unused include 2021-04-16 11:40:56 -04:00
mindspore-ci-bot 5b7ed56d57 !12521 [WIP] MindData C++ Plugin infrastructure with GDAL decode
From: @ziruiwu
Reviewed-by: 
Signed-off-by:
2021-04-16 20:37:18 +08:00
Zirui Wu 8ebc512d25 plugin infrastruture for mindddata
add single element string tensor support for plugin tensor
2021-04-15 21:11:16 -04:00
alex-yuyue 97e3232b6d Add more ut for transform ops and subset sampler
Signed-off-by: alex-yuyue <yue.yu1@huawei.com>
2021-04-15 16:51:41 -04:00
Cathy Wong a2ee0cb3e5 dataset UT: Update UTs to check for return Status 2021-04-14 17:52:31 -04:00
mindspore-ci-bot 1d89109d19 !14929 Cleanup stale comments about DataBuffers
From: @hfarahat
Reviewed-by: @pandoublefeng,@robingrosman
Signed-off-by: @pandoublefeng
2021-04-14 05:33:40 +08:00
mindspore-ci-bot 727203a498 !14911 dataset: Fill op: C++ API support, UTs and Pybind decoupling
From: @cathwong
Reviewed-by: 
Signed-off-by:
2021-04-14 05:32:11 +08:00
Cathy Wong 7e6a03487e dataset: Fill op: C++ API support, UTs and Pybind decoupling 2021-04-13 10:02:39 -04:00
mindspore-ci-bot e07a69d8f8 !14874 GNN dataset new feature
From: @lizhenglong1992
Reviewed-by: 
Signed-off-by:
2021-04-13 10:44:27 +08:00
hesham 15354569f2 Cleanup stale comments about DataBuffers 2021-04-12 14:26:59 -04:00
ms_yan ecb8d6bc08 add pil version cubic resize 2021-04-12 21:56:14 +08:00
Zhenglong Li f3895f2b09 Add get edges from nodes function for gnn network 2021-04-10 18:31:13 +08:00
mindspore-ci-bot 74f4c85315 !14750 Address some code issues in MD
From: @hfarahat
Reviewed-by: @robingrosman,@mikef
Signed-off-by: @robingrosman
2021-04-10 03:55:16 +08:00
hesham 97cf4e516e Address 3 code issues:
1- access out of bound memory in toFlatIndex
2- CreateFromNPArray function is 50>
3- duplicate variable init
2021-04-08 21:50:19 -04:00
Cathy Wong 82f4b3f757 dataset: C++ API TypeCast: change data_type parm from string to DataType 2021-04-08 11:47:58 -04:00
mindspore-ci-bot bdc5a9c88b !14579 TensorOp decoupling stage 3 (transform ops)
From: @alexyuyue
Reviewed-by: 
Signed-off-by:
2021-04-08 21:41:26 +08:00
mindspore-ci-bot b6cf16d8aa !14389 fix codex warning
From: @luoyang42
Reviewed-by: @heleiwang,@liucunwei
Signed-off-by: @liucunwei
2021-04-08 14:22:39 +08:00
alex-yuyue 83aaed09bb TensorOp decoupling stage 3 (transform ops)
Signed-off-by: alex-yuyue <yue.yu1@huawei.com>
2021-04-07 18:46:23 -04:00
mindspore-ci-bot a9ba9b90f6 !14680 MindData Codespell correction
From: @luoyang42
Reviewed-by: @heleiwang,@liucunwei
Signed-off-by: @liucunwei
2021-04-07 18:28:55 +08:00
luoyang 624511337c MindData Codespell correction 2021-04-07 10:20:29 +08:00
mindspore-ci-bot 3d85930cfe !14639 Remove DataBuffer class
From: @hfarahat
Reviewed-by: @pandoublefeng,@robingrosman
Signed-off-by: @pandoublefeng
2021-04-06 23:49:30 +08:00
hesham 8535e89e6c Remove DataBuffer 2021-04-05 17:55:59 -04:00
mindspore-ci-bot 0cfd8b7c9e !13364 Add WordpieceTokenizer and AddDict to Jieba
From: @alexyuyue
Reviewed-by: 
Signed-off-by:
2021-04-06 05:04:13 +08:00
hesham 0d86cb1423 Remove rows_per_buffer 2021-04-04 00:22:16 -04:00
hesham a9a80693be Replace Databuffers with TensorRows 2021-03-31 17:00:36 -04:00
liyong d5b7a67c71 fix asan issue 2021-03-31 14:18:50 +08:00
YangLuo b85a905eaf fix codex warning 2021-03-30 17:00:37 +08:00
Eric 38944eafc4 Added get dataset size and fix macro 2021-03-29 15:49:29 -04:00
mindspore-ci-bot a91e9c9294 !14106 reduce ut time of cutmix_batch and ConcatOp
From: @luoyang42
Reviewed-by: @pandoublefeng,@liucunwei
Signed-off-by: @pandoublefeng,@liucunwei
2021-03-25 22:58:35 +08:00
YangLuo f7ea18944d reduce ut time of cutmix_batch 2021-03-25 21:59:44 +08:00
mindspore-ci-bot 5926f7c386 !13906 【MD】【BUG】Conventional judgment on the parameter h w not less than zero
From: @xulei2020
Reviewed-by: 
Signed-off-by:
2021-03-25 14:10:42 +08:00
xulei2020 4122d1dcce fix code format bug 2021-03-25 09:18:36 +08:00
mindspore-ci-bot b1b99b97d7 !13750 Add check of ResizePreserveARWithFiller operation
From: @shenwei41
Reviewed-by: 
Signed-off-by:
2021-03-24 15:27:16 +08:00
shenwei41 d3265f0591 Add check of ResizePreserveARWithFiller operation 2021-03-23 23:30:53 +08:00
mindspore-ci-bot 06892b83ff !13575 Add check to RgbToGray
From: @shenwei41
Reviewed-by: @tiancixiao,@liucunwei
Signed-off-by: @liucunwei
2021-03-22 12:11:39 +08:00
Xiao Tianci b88a68c9e8 fix canny when size is 7 2021-03-19 17:26:59 +08:00
mindspore-ci-bot 0451a800bc !13571 Added seed to prevent random failures in ut
From: @ezphlow
Reviewed-by: 
Signed-off-by:
2021-03-19 17:17:42 +08:00
shenwei41 8a1f283efc Add check to RgbToGray 2021-03-19 10:35:35 +08:00
Eric a3c98d9d59 Added fix for random core dump 2021-03-18 21:15:50 -04:00
alex-yuyue 2d6d608fae Add WordpieceTokenizer to API and IR. Add AddDict to C++ API
Signed-off-by: alex-yuyue <yue.yu1@huawei.com>
2021-03-18 16:42:31 -04:00
mindspore-ci-bot b45f289801 !13504 Add check of GetPerspectiveTransform and GetAffineTransform
From: @shenwei41
Reviewed-by: @liucunwei,@pandoublefeng
Signed-off-by: @liucunwei
2021-03-18 09:57:52 +08:00
mindspore-ci-bot dd90f7f055 !13506 dataset UT: Reinstate INFO logging and data verification - part 3
From: @cathwong
Reviewed-by: @robingrosman,@liucunwei
Signed-off-by: @robingrosman
2021-03-18 09:35:22 +08:00
Cathy Wong 1775c4e83d dataset: Reinstate INFO logging and data verification - part 3
Updates: GetItemAt, CreateFromVector, CreateScalar.
Add TEST_MS_LOG_MSTENSOR.
Wrap ASSERT_OK around Tensor function calls.
2021-03-17 13:48:04 -04:00
shenwei41 89f109a3f3 add check of operation 2021-03-17 20:29:17 +08:00
Eric a7da38feeb Added example for ngram 2021-03-16 23:45:04 -04:00
mindspore-ci-bot c0aa5f8132 !13365 Fix random rotation input check
From: @ezphlow
Reviewed-by: 
Signed-off-by:
2021-03-17 09:42:37 +08:00
Eric c730df23e8 Added json fix 2 2021-03-16 18:32:42 -04:00
Eric 4be0ffb9ba removing type_id.h from header files 2021-03-16 16:51:10 -04:00
shenwei41 d1bb9d470e Add resizefiller 2021-03-16 20:35:27 +08:00
Eric 9b921953f9 misc issues 2021-03-15 22:30:04 -04:00
Lixia Chen 30017ea5a0 Fix codex issues 2021-03-15 11:15:15 -04:00
mindspore-ci-bot 0cf6fb5310 !13264 [MD] Fix input check bug in RandomSelectSubpolicyOperation and codex issues
From: @tina_mengting_zhang
Reviewed-by: @robingrosman,@nsyca
Signed-off-by: @robingrosman
2021-03-15 20:29:56 +08:00
mindspore-ci-bot 98a87ad969 !13220 Add data helper to C++ API
From: @ezphlow
Reviewed-by: 
Signed-off-by:
2021-03-13 09:52:54 +08:00
TinaMengtingZhang ac05546aeb fix input check bug in RandomSelectSubpolicy and codex issue 2021-03-12 16:04:38 -05:00
Eric 82f7c6f2c7 Moving data_helper
exposing json

WIP fix for removing templates

Removing templates part 1

replacing core/constants.h

removed template part 2

Changed fake data to filler data

Fixing CI

Fix CI part 2

CI fix part 2

Fix CI 3

Fixed lizard for data_util

Fixed clang

Fixing CI part 4
2021-03-12 15:30:34 -05:00
mindspore-ci-bot 483bb9de60 !13078 dataset UT: Reinstate INFO logging and data verification - part 2
From: @cathwong
Reviewed-by: 
Signed-off-by:
2021-03-13 03:55:34 +08:00
Cathy Wong 95bbaaf9f6 dataset: Reinstate INFO logging and data verification - part 2
for image/vision datasets
2021-03-12 10:07:45 -05:00