yystopf
/
mindspore
forked from mindspore-Ecosystem/mindspore
0
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity
45,876 Commits 23 Branches 0 Tags 3.9 GiB
Commit Graph

2085 Commits

Author SHA1 Message Date
dingpeifei 24ff7ab8b4 upgrade_ascend_0705 2021-07-06 11:31:45 +08:00
Zhang Qinghua b37a0cef63 Modify context searching and creating routine. 2021-07-06 11:01:52 +08:00
Harshvardhan Gupta 5bd8fb57fc remove builder 2021-07-05 11:17:37 -04:00
liyong 9a1de0af65 add slice patches in python and c++ 2021-07-05 09:16:47 +08:00
i-robot 0022d07d6e !19289 Lift FV before AD pass. 
Merge pull request !19289 from 张清华/cconv_opt0
 2021-07-03 01:27:35 +00:00
zhousiyi a5b4e7bbf8 lift fv before grad except weight, then convert 
switch(cond, partial(g1, xs), partial(g2, ys))(Zs)
 to
 switch(cond, g1, g2)(Xs, Ys, Zs)

 switch_layer(index, make_tuple(partial(g1, xs), partial(g2, ys)))(Zs)
 to
 switch_layer(index, make_tuple(g1, g2))(Xs, Ys, Zs)

put Zs at last when unifyparameter as it may have u-monad or io-monad

use joined args other than broadened one as some extra parameter which is not a parameter of while_header can be add to while_body

inline fprop_switch forcely

reorder the parameter if one of the parameter is Monad when incorporate call

incorporate switch tuple_getitem if item 0 of tuple is EnvInstance or
item 1 of tuple is bprop function

addn with shape() and shape(1)

remove context_ from FuncGraphEvaluator to make it re-entry able to resolve evaluator stuck issue because of re-entry of the same FuncGraphEvaluator
 2021-07-03 00:03:37 +08:00
ql_12345 5af0c890a1 cipher calling code 2021-07-01 00:23:06 +08:00
zhoufeng b9378e36c3 dynamic default memory size for vnpu 
Signed-off-by: zhoufeng <zhoufeng54@huawei.com>
 2021-06-29 19:23:04 +08:00
i-robot 3629df552c !18937 upgrade_ascend_0626_mindspore 
Merge pull request !18937 from mindspore_ding/upgrade_ascend_0626
 2021-06-28 15:50:14 +00:00
dingpeifei 63784e49f5 upgrade_ascend_0626_mindspore 2021-06-28 22:03:19 +08:00
zhoufeng 35797bd5da add serialization cpp ut 
Signed-off-by: zhoufeng <zhoufeng54@huawei.com>
 2021-06-26 13:22:10 +08:00
i-robot 708e56f659 !17899 Fix compile cache bug for resent50 
Merge pull request !17899 from LiangZhibo/cache
 2021-06-24 13:29:58 +00:00
l00591931 8ae5d7cc84 Fix compile cache for resnet50 2021-06-24 14:36:24 +08:00
i-robot 6bd66b3d70 !18729 Add register limit constraint 
Merge pull request !18729 from lishanni/for_regis_limit
 2021-06-24 03:48:17 +00:00
lishanni513 8c98146c76 Add register limit constraint 2021-06-23 17:39:22 +08:00
YangLuo 4e9f6d54f7 update cpp api & doc 2021-06-23 11:14:52 +08:00
i-robot dca5504fd4 !15672 [assistant][RGB2BGR] 
Merge pull request !15672 from QingfengLi/rgb2bgr
 2021-06-22 12:48:04 +00:00
lanzhineng fc8ec7fc49 infer_optv3:infer optimize to find exit of branches and resolve the 
endless issue.
 2021-06-22 11:55:56 +08:00
chenx2ovo 4b98099dcb [feat][assistant][I3CEGQ] add op rgb2bgr 2021-06-21 16:16:50 +08:00
i-robot 35b15bda0b !18465 clean code for memory reuse 
Merge pull request !18465 from laiyongqiang/mem_clean
 2021-06-19 10:12:09 +00:00
LaiYongqiang d4d6fb940d memory reuse code clean 2021-06-18 09:41:55 +08:00
chendongsheng bfaab72934 add node recovery 2021-06-17 23:52:09 +08:00
Xiao Tianci a54e65a7ad add Crop Python API 2021-06-10 09:46:57 +08:00
i-robot 1a626e3ff2 !17952 remove some code 
Merge pull request !17952 from caifubi/clean-codex
 2021-06-10 09:35:20 +08:00
Xiao Tianci 9c4c1c6a77 add Rotate Python API 2021-06-08 22:19:15 +08:00
caifubi 61efa2c23b remove some code 2021-06-08 10:55:41 +08:00
mindspore-ci-bot 8fa9e3e611 !17712 fix pclint & codex in profiler 
From: @yanghaitao1
Reviewed-by: @ouwenchang,@yelihua
Signed-off-by: @yelihua
 2021-06-08 10:31:49 +08:00
mindspore-ci-bot 823468d924 !17766 GNN new feature about two memory effeciency format 
From: @lizhenglong1992
Reviewed-by: @jonyguo,@heleiwang,@jonyguo
Signed-off-by: @jonyguo,@jonyguo
 2021-06-08 09:30:54 +08:00
mindspore-ci-bot 86cb781ce0 !17614 pass optimizer 
From: @zhupuxu
Reviewed-by: @zhoufeng54
Signed-off-by:
 2021-06-08 09:17:33 +08:00
mindspore-ci-bot 5f39e8bd59 !17818 upgrade_ascend_0604 
From: @ding_fei_fei
Reviewed-by: @xsmq,@zhoufeng54
Signed-off-by: @zhoufeng54
 2021-06-07 23:44:49 +08:00
dingpeifei 171d09457f upgrade_ascend_0604 2021-06-07 23:06:58 +08:00
mindspore-ci-bot f606951063 !17790 Fix code warning in master 
From: @liangzhibo
Reviewed-by: @ginfung,@zh_qh
Signed-off-by: @zh_qh
 2021-06-07 19:53:05 +08:00
yanghaitao1 127e4d4068 fix profiler pclint&codex 2021-06-07 02:17:23 -04:00
Zhenglong Li 2c7d1bf603 Implement new gnn feature 2021-06-07 10:20:06 +08:00
zhupuxu c0099abd03 pass optimizer 
Signed-off-by: zhupuxu <zhupuxu@huawei.com>
 2021-06-07 09:54:24 +08:00
mindspore-ci-bot cf27d3c8dc !12972 Removing Builder from dataset ops phase 1 
From: @ezphlow
Reviewed-by: 
Signed-off-by:
 2021-06-05 03:51:49 +08:00
mindspore-ci-bot 08847d2f64 !17635 [ME]Add parameter eliminate pass with func graph used search 
From: @chenfei52
Reviewed-by: @zh_qh,@ginfung
Signed-off-by: @zh_qh
 2021-06-04 17:46:01 +08:00
l00591931 0424b2d8d6 Fix warning in master 2021-06-04 17:40:01 +08:00
chenfei 4f50b3dfe1 add parameter eliminate pass 2021-06-04 11:53:34 +08:00
EricZ 16837a4639 Removing Builder from dataset ops phase 1 2021-06-03 09:09:29 +08:00
Xiao Tianci 9c74d55a49 add HorizontalFlip and VerticalFlip APIs 2021-05-31 15:55:21 +08:00
mindspore-ci-bot aa2093fb7d !17173 Add frontend compile cache 
From: @ginfung
Reviewed-by: @hwhewei,@zh_qh
Signed-off-by: @zh_qh
 2021-05-31 10:26:16 +08:00
yujianfeng 7cea682c7d Add func_graph caching after validate action 2021-05-30 13:00:35 +08:00
mindspore-ci-bot 497505631c !17061 Delete extra files, and change camel format 
From: @zetongzhao
Reviewed-by: @pandoublefeng,@robingrosman
Signed-off-by: @robingrosman
 2021-05-29 09:43:34 +08:00
mindspore-ci-bot 17099b048f !16936 Unify dump directory structure 
From: @tina_mengting_zhang
Reviewed-by: @john_tzanakakis,@pandoublefeng
Signed-off-by: @john_tzanakakis
 2021-05-29 04:12:48 +08:00
zetongzhao 728b180329 delete_extra files 2021-05-26 16:41:27 -04:00
TinaMengtingZhang 4926d74570 unify dir path 
dump hccl and config json files to dir

update filename for sync dump except cpu dump

update testcases
 2021-05-26 16:22:18 -04:00
mindspore-ci-bot 686225ad2a !16382 Unity Input Order 
From: @zetongzhao
Reviewed-by: 
Signed-off-by:
 2021-05-26 21:28:36 +08:00
mindspore-ci-bot b9a12ce3bf !16940 enable refactor thread pool 
From: @yangjie159
Reviewed-by: @zhang_xue_tong,@wangchengyuan
Signed-off-by: @zhang_xue_tong
 2021-05-26 11:07:12 +08:00
mindspore-ci-bot 1e8868cecc !16727 handle duplicate code 
From: @yangzhenzhang
Reviewed-by: @stsuteng,@jjfeing
Signed-off-by: @stsuteng
 2021-05-26 10:58:54 +08:00
yangjie159 a6622a70ba enable refactor thread pool 2021-05-26 07:58:58 +08:00
zetongzhao 4cb53823fd all the updated input order 
Fix FillOperation to_json function

Fix FillOperation to_json function

serialize tensor without testing

fix unity input order bug, next step is to test the tensor serialization

serialize tensor, testing code in tensor_test.cc

change for tensor serialization

changes after review

changes after review

clang-format the files

rename
 2021-05-25 18:26:43 -04:00
yangzhenzhang d711d98f07 clean duplicate code 2021-05-25 17:25:34 +08:00
Xiao Tianci 3f495bbae1 add GaussianBlur API 2021-05-24 16:27:05 +08:00
mindspore-ci-bot d906f15673 !16730 add ge runtime depends 
From: @zhoufeng54
Reviewed-by: @kisnwang,@jjfeing,@majorzhang
Signed-off-by: @jjfeing,@majorzhang
 2021-05-24 14:45:54 +08:00
mindspore-ci-bot 53b25b62c5 !16709 scheduler added client 
From: @anancds
Reviewed-by: 
Signed-off-by:
 2021-05-24 10:01:28 +08:00
chendongsheng 54a331d103 scheduler added client 2021-05-22 17:17:07 +08:00
zhoufeng 859b4e3ce3 add ge runtime depends 
Signed-off-by: zhoufeng <zhoufeng54@huawei.com>
 2021-05-21 19:00:14 +08:00
lianliguang e3d7304208 add function to remove monad abs when using cpp infer 2021-05-21 17:02:52 +08:00
mindspore-ci-bot 51fb25d525 !16540 fix codex in tensor_layout.cc 
From: @gong_zi_yan
Reviewed-by: @stsuteng,@yangzhenzhang
Signed-off-by: @stsuteng
 2021-05-21 09:47:52 +08:00
mindspore-ci-bot 4b4ad59f6a !16632 added cluster metadata 
From: @anancds
Reviewed-by: @cristoval
Signed-off-by:
 2021-05-20 21:57:07 +08:00
chendongsheng 35f6d39127 added cluster metadata 2021-05-20 15:15:06 +08:00
Ziyan 2a0a549ae8 fix codex in tensor_layout.cc 2021-05-20 10:10:20 +08:00
lvliang 116781041a fix ut compile failed 2021-05-19 19:59:10 +08:00
zhousiyi 4ff19b7082 add API to build bprop funcgraph which can be used for higher grad 2021-05-19 19:59:06 +08:00
lvliang c759effa94 fix_compile_ut_failed 2021-05-19 19:59:06 +08:00
zhousiyi 1843740e0d 1. set abstract for node. 2. Adjust the position of sens parameter from last to position just after Inputs. 3. Set debug name for Parameter 2021-05-19 19:58:51 +08:00
zhousiyi b40816e66a add support for stop_gradient primitive. And defer BackPropagation to End API 2021-05-19 19:58:49 +08:00
zhousiyi 0d6a5ccfe4 grad op for pynative 2021-05-19 19:58:46 +08:00
mindspore-ci-bot 84837b0bc9 !16467 modify defination of hccl origin function in hccl plugin 
From: @zhoufeng54
Reviewed-by: @jjfeing,@xu-yfei
Signed-off-by: @xu-yfei
 2021-05-18 17:44:45 +08:00
zhoufeng 4306145684 fix ut comiple task pb not found 
Signed-off-by: zhoufeng <zhoufeng54@huawei.com>
 2021-05-18 14:12:09 +08:00
mindspore-ci-bot 034193f2df !16496 concat allgather pass performance optimization 
From: @zhoufeng54
Reviewed-by: @kisnwang,@jjfeing
Signed-off-by: @jjfeing
 2021-05-17 21:09:44 +08:00
mindspore-ci-bot 601322356b !15638 VOCDataset need to add img_id for evaluation 
From: @xiefangqi
Reviewed-by: @liucunwei
Signed-off-by: @liucunwei
 2021-05-17 19:33:24 +08:00
zhoufeng ab5b36988e concat allgather pass performance optimization 
Signed-off-by: zhoufeng <zhoufeng54@huawei.com>
 2021-05-17 17:40:34 +08:00
xiefangqi 9b846c7873 voc add image_id column 2021-05-17 16:09:04 +08:00
zhoufeng fee35368d1 modify defination of hccl origin function in hccl plugin 
Signed-off-by: zhoufeng <zhoufeng54@huawei.com>
 2021-05-17 14:19:35 +08:00
jonyguo 9452142b50 enhance: enable shuffle infile / files when mindrecord 2021-05-14 17:41:18 +08:00
mindspore-ci-bot eb483a276b !16313 Evaluate the func graph in simulative stack, not to call recursively. 
From: @zh_qh
Reviewed-by: @hwhewei,@ginfung
Signed-off-by: @hwhewei
 2021-05-13 08:56:18 +08:00
mindspore-ci-bot b5cddcb8cc !16036 Improve performance of multi-processing option for dataset 
From: @robingrosman
Reviewed-by: 
Signed-off-by:
 2021-05-13 01:41:16 +08:00
RobinGrosman 9a7d1fc034 Optimize multi-process in minddata by passing data via shared memory instead of multiprocessing.queue 2021-05-11 15:16:29 -07:00
zhoufeng a76d58b52d hccl decouple 
Signed-off-by: zhoufeng <zhoufeng54@huawei.com>
 2021-05-11 22:25:00 +08:00
mindspore-ci-bot f356c51169 !16116 graph scheduler support multi graphs 
From: @limingqi107
Reviewed-by: @cristoval,@wilfchen
Signed-off-by: @wilfchen
 2021-05-11 19:33:57 +08:00
Zhang Qinghua 5ad27e7639 Evaluate the func graph in simulative stack, not to call recursively. 2021-05-11 15:36:22 +08:00
mindspore-ci-bot 6fcd6cab68 !16119 Fix minddata issues 
From: @luoyang42
Reviewed-by: @jonyguo,@liucunwei
Signed-off-by: @jonyguo,@liucunwei
 2021-05-11 14:33:29 +08:00
mindspore-ci-bot d3a175e30f !16078 Modify Normalize to accept images with more than 3 channels 
From: @hfarahat
Reviewed-by: @mikef,@robingrosman
Signed-off-by: @robingrosman
 2021-05-11 02:03:34 +08:00
limingqi107 d8399e61e7 graph scheduler support multi graphs 2021-05-10 20:57:04 +08:00
YangLuo 0a4da64e13 Fix minddata issues 2021-05-10 11:12:00 +08:00
mindspore-ci-bot c74e66e759 !15928 unify dump path and support npy format 
From: @zhangbuxue
Reviewed-by: @guoqi1024,@zhaizhiqiang
Signed-off-by: @zhaizhiqiang
 2021-05-08 11:32:24 +08:00
mindspore-ci-bot 7a271987d6 !15269 use avx to optimize cpu op 
From: @wangyanling10
Reviewed-by: 
Signed-off-by:
 2021-05-08 09:40:10 +08:00
buxue 0eab785cfa unify dump path and support npy format 2021-05-08 09:04:33 +08:00
mindspore-ci-bot b86747b9a6 !15813 Convert some pipeline Ops to be inlined 
From: @hfarahat
Reviewed-by: 
Signed-off-by:
 2021-05-08 05:30:39 +08:00
hesham fd943bbdc0 Modify normalize to support multiple of channels 2021-05-07 17:06:26 -04:00
wangyanling ee1b803416 optimize cpu adam op 2021-05-07 10:47:57 +08:00
Lixia Chen 5967cb2cd7 Unblock mindrecord under cache 2021-05-05 13:28:13 -04:00
hesham e4c0bd51a5 Modify concat to be an inlined Op 
Modify zip to be an inlined Op
Modify take to be an inlined Op
Modify skip to be an inlined Op
Modify rename to be an inlined Op
Modify filter to consume child's rows using one thread
 2021-05-04 12:45:18 -04:00
mindspore-ci-bot 9416502e90 !15916 Add federated learning server part1 
From: @zpac
Reviewed-by: 
Signed-off-by:
 2021-05-01 14:59:47 +08:00
mindspore-ci-bot 9c7cb65c3b !15801 Split sampler and vision IR files 
From: @mhmotallebi
Reviewed-by: 
Signed-off-by:
 2021-05-01 05:19:41 +08:00
ZPaC 232e525ff2 Add common.h, local_meta_storage.h, param_name_type.h 
Add class Message, AddrStorage

Add Kernel folder

Add consistent hash ring

Add updater for server.

Add iteration timer

Add executor for server
 2021-04-30 15:31:09 +08:00
mindspore-ci-bot cefcf6c2ae !15931 Add eager tests for transform ops, and add a shape comparison in the EXPECT_MSTENSOR_EQ Macro 
From: @alexyuyue
Reviewed-by: @nsyca,@robingrosman
Signed-off-by: @robingrosman
 2021-04-30 06:15:12 +08:00
mohammad 9ce28cd34c split sampler vision IR files 2021-04-29 16:30:18 -04:00
alex-yuyue 9ef4331f56 Add eager tests for transform ops 
Signed-off-by: alex-yuyue <yue.yu1@huawei.com>
 2021-04-29 13:19:24 -04:00
mindspore-ci-bot 966f89198a !15107 use internal ge runtime 
From: @zhoufeng54
Reviewed-by: 
Signed-off-by:
 2021-04-29 10:05:54 +08:00
mindspore-ci-bot a1aeb7b8f1 !15616 remove unnecessary ops && modify static check 
From: @lyvette
Reviewed-by: @jpc_chenjianping,@hangangqiang
Signed-off-by: @jpc_chenjianping
 2021-04-28 11:31:29 +08:00
mindspore-ci-bot 1e2e554db3 !15380 Unify minddata include files(except for datasets.h) in scenario of cloud and lite 
From: @luoyang42
Reviewed-by: @tiancixiao
Signed-off-by:
 2021-04-28 09:57:46 +08:00
mindspore-ci-bot 6dd37c5857 !15777 [Auto parallel] Fix some codestyle warnings 
From: @xiaoda_zh
Reviewed-by: @yangzhenzhang,@stsuteng
Signed-off-by: @stsuteng
 2021-04-28 09:09:25 +08:00
Xiaoda Zhang 5fecfe92a6 code style warnings fixing 2021-04-27 19:54:56 +08:00
chendongsheng 05d3ae2219 add http communicator 2021-04-27 19:44:57 +08:00
liuyu d259c159ba remove unnecessary ops 
fix static check
 2021-04-27 19:23:32 +08:00
zhoufeng 3d38bcf791 use internal ge runtime 
Signed-off-by: zhoufeng <zhoufeng54@huawei.com>
 2021-04-26 18:56:33 +08:00
YangLuo bf48400c10 unify include file of minddata 2021-04-22 10:59:36 +08:00
lianliguang d2c696667b change process of infer value 2021-04-20 19:02:22 +08:00
mindspore-ci-bot 490d2e1efb !15385 remove the useless prim type 
From: @lianliguang
Reviewed-by: @ginfung,@zh_qh,@ginfung
Signed-off-by: @zh_qh
 2021-04-20 16:07:45 +08:00
mindspore-ci-bot 512130d171 !15363 cpu unique support fp16 
From: @kisnwang
Reviewed-by: @jjfeing,@zhoufeng54
Signed-off-by: @zhoufeng54
 2021-04-20 14:25:56 +08:00
kswang 9c8ada03fe cpu unique kernel support fp16 2021-04-20 11:40:13 +08:00
lianliguang 4d7b7649e9 remove the useless PrimType 2021-04-20 10:16:36 +08:00
Cathy Wong 636809c255 dataset: C++ API ToNumber & Lookup: change data_type parm from string to DataType 2021-04-19 12:35:37 -04:00
mindspore-ci-bot 5f2603ed93 !15216 Pybind11::object && PrimitivePy recycle optimize 
From: @zhangzhaoju
Reviewed-by: @zh_qh,@hwhewei,@zh_qh
Signed-off-by: @zh_qh,@zh_qh
 2021-04-17 09:38:16 +08:00
mindspore-ci-bot bd69a86931 !15124 Remove unused #include in cc files in MindData 
From: @hfarahat
Reviewed-by: @nsyca,@mikef,@robingrosman
Signed-off-by: @robingrosman
 2021-04-17 02:19:27 +08:00
mindspore-ci-bot 9c81c5d2a0 !15189 Add more ut for transform ops and subset sampler 
From: @alexyuyue
Reviewed-by: 
Signed-off-by:
 2021-04-17 01:46:13 +08:00
hesham 98f0f36a20 Removed unused include 2021-04-16 11:40:56 -04:00
mindspore-ci-bot 5b7ed56d57 !12521 [WIP] MindData C++ Plugin infrastructure with GDAL decode 
From: @ziruiwu
Reviewed-by: 
Signed-off-by:
 2021-04-16 20:37:18 +08:00
zhangzhaoju a1db9c4959 Pybind11::object && PrimitivePy recycle optimize 2021-04-16 18:32:55 +08:00
mindspore-ci-bot 053b04ae6e !15204 replace memcpy_async with tensor move 
From: @laiyongqiang
Reviewed-by: @kisnwang,@jjfeing
Signed-off-by: @jjfeing
 2021-04-16 10:19:40 +08:00
Zirui Wu 8ebc512d25 plugin infrastruture for mindddata 
add single element string tensor support for plugin tensor
 2021-04-15 21:11:16 -04:00
alex-yuyue 97e3232b6d Add more ut for transform ops and subset sampler 
Signed-off-by: alex-yuyue <yue.yu1@huawei.com>
 2021-04-15 16:51:41 -04:00
laiyongqiang 1533435015 replace memcpy_async with tensor move 2021-04-15 16:12:01 +08:00
Cathy Wong a2ee0cb3e5 dataset UT: Update UTs to check for return Status 2021-04-14 17:52:31 -04:00
mindspore-ci-bot 1d89109d19 !14929 Cleanup stale comments about DataBuffers 
From: @hfarahat
Reviewed-by: @pandoublefeng,@robingrosman
Signed-off-by: @pandoublefeng
 2021-04-14 05:33:40 +08:00
mindspore-ci-bot 727203a498 !14911 dataset: Fill op: C++ API support, UTs and Pybind decoupling 
From: @cathwong
Reviewed-by: 
Signed-off-by:
 2021-04-14 05:32:11 +08:00
Cathy Wong 7e6a03487e dataset: Fill op: C++ API support, UTs and Pybind decoupling 2021-04-13 10:02:39 -04:00
mindspore-ci-bot 3068d66d63 !14745 refactor infer value 
From: @lianliguang
Reviewed-by: @ginfung
Signed-off-by:
 2021-04-13 11:27:08 +08:00
mindspore-ci-bot e07a69d8f8 !14874 GNN dataset new feature 
From: @lizhenglong1992
Reviewed-by: 
Signed-off-by:
 2021-04-13 10:44:27 +08:00
hesham 15354569f2 Cleanup stale comments about DataBuffers 2021-04-12 14:26:59 -04:00
ms_yan ecb8d6bc08 add pil version cubic resize 2021-04-12 21:56:14 +08:00
LianLiguang bdc21bcf3e change infer value 2021-04-12 20:38:10 +08:00
Zhenglong Li f3895f2b09 Add get edges from nodes function for gnn network 2021-04-10 18:31:13 +08:00
mindspore-ci-bot 74f4c85315 !14750 Address some code issues in MD 
From: @hfarahat
Reviewed-by: @robingrosman,@mikef
Signed-off-by: @robingrosman
 2021-04-10 03:55:16 +08:00
hesham 97cf4e516e Address 3 code issues: 
1- access out of bound memory in toFlatIndex
2- CreateFromNPArray function is 50>
3- duplicate variable init
 2021-04-08 21:50:19 -04:00
Cathy Wong 82f4b3f757 dataset: C++ API TypeCast: change data_type parm from string to DataType 2021-04-08 11:47:58 -04:00
mindspore-ci-bot bdc5a9c88b !14579 TensorOp decoupling stage 3 (transform ops) 
From: @alexyuyue
Reviewed-by: 
Signed-off-by:
 2021-04-08 21:41:26 +08:00
mindspore-ci-bot b6cf16d8aa !14389 fix codex warning 
From: @luoyang42
Reviewed-by: @heleiwang,@liucunwei
Signed-off-by: @liucunwei
 2021-04-08 14:22:39 +08:00
alex-yuyue 83aaed09bb TensorOp decoupling stage 3 (transform ops) 
Signed-off-by: alex-yuyue <yue.yu1@huawei.com>
 2021-04-07 18:46:23 -04:00
mindspore-ci-bot e2260a2f09 !14612 remove ControlDepend 
From: @huangbingjian
Reviewed-by: 
Signed-off-by:
 2021-04-07 18:36:29 +08:00
mindspore-ci-bot a9ba9b90f6 !14680 MindData Codespell correction 
From: @luoyang42
Reviewed-by: @heleiwang,@liucunwei
Signed-off-by: @liucunwei
 2021-04-07 18:28:55 +08:00
huangbingjian 63a89925ff remove ControlDepend and its use 2021-04-07 11:30:40 +08:00
luoyang 624511337c MindData Codespell correction 2021-04-07 10:20:29 +08:00
mindspore-ci-bot 1ff7f0f919 !14569 support https 
From: @anancds
Reviewed-by: @limingqi107
Signed-off-by:
 2021-04-07 09:09:15 +08:00
mindspore-ci-bot 3d85930cfe !14639 Remove DataBuffer class 
From: @hfarahat
Reviewed-by: @pandoublefeng,@robingrosman
Signed-off-by: @pandoublefeng
 2021-04-06 23:49:30 +08:00
chendongsheng ce4d86171f support https 2021-04-06 15:37:59 +08:00
mindspore-ci-bot ce248c37e0 !14407 separate irpass EnvGetItemEliminater and ItemTupleOrListEliminator 
From: @huangbingjian
Reviewed-by: @ginfung,@zh_qh
Signed-off-by: @zh_qh
 2021-04-06 08:52:23 +08:00
hesham 8535e89e6c Remove DataBuffer 2021-04-05 17:55:59 -04:00
mindspore-ci-bot 0cfd8b7c9e !13364 Add WordpieceTokenizer and AddDict to Jieba 
From: @alexyuyue
Reviewed-by: 
Signed-off-by:
 2021-04-06 05:04:13 +08:00
hesham 0d86cb1423 Remove rows_per_buffer 2021-04-04 00:22:16 -04:00
mindspore-ci-bot c99fe1e412 !12841 DataBuffer removal from DE 
From: @hfarahat
Reviewed-by: 
Signed-off-by:
 2021-04-04 03:28:13 +08:00
huangbingjian 134d5dfe4b separate irpass EnvGetItemEliminater and ItemTupleOrListEliminator 2021-04-02 17:38:40 +08:00
mindspore-ci-bot bb3a19363c !14548 change macro of infer function reg 
From: @lianliguang
Reviewed-by: @ginfung,@chujinjin
Signed-off-by: @chujinjin
 2021-04-02 15:49:16 +08:00
LianLiguang 8cabd090de change macro of reg infer 2021-04-02 09:07:24 +08:00
chendongsheng 43defcfaad add multithread http server 2021-04-01 15:59:13 +08:00
mindspore-ci-bot 5787e5dcb3 !14295 Clean codex 
From: @jojobugfree
Reviewed-by: @kisnwang,@jjfeing
Signed-off-by: @jjfeing
 2021-04-01 09:17:03 +08:00
hesham a9a80693be Replace Databuffers with TensorRows 2021-03-31 17:00:36 -04:00
caifubi 58f3ea002d clean codex 2021-03-31 15:41:37 +08:00
liyong d5b7a67c71 fix asan issue 2021-03-31 14:18:50 +08:00
YangLuo b85a905eaf fix codex warning 2021-03-30 17:00:37 +08:00
mindspore-ci-bot 75fdaaa6aa !14304 [GraphKernel] Dump GraphKernel split info as text; dump akg kernel launch fail message 
From: @dayschan
Reviewed-by: @gaoxiong1,@gaoxiong1,@anyrenwei
Signed-off-by: @anyrenwei
 2021-03-30 15:32:14 +08:00
Eric 38944eafc4 Added get dataset size and fix macro 2021-03-29 15:49:29 -04:00
dayschan 3c6c30024c dump graph_kernel_split info 2021-03-29 17:15:02 +08:00
mindspore-ci-bot 993d00c4ba !13774 The input and output of batchnorm reverse operator increase pass in ascend platform under the mode of pynitve 
From: @ding_fei_fei
Reviewed-by: @kingxian,@zhunaipan
Signed-off-by: @kingxian
 2021-03-29 16:00:53 +08:00
buxue d830a2b55a optimize exception mode when use undefined name in if for and while 2021-03-27 17:28:52 +08:00
dingpeifei 3c9d8cb073 The input and output of batchnorm reverse operator increase pass in ascend platform under the mode of pynitve 2021-03-27 16:49:38 +08:00
mindspore-ci-bot a91e9c9294 !14106 reduce ut time of cutmix_batch and ConcatOp 
From: @luoyang42
Reviewed-by: @pandoublefeng,@liucunwei
Signed-off-by: @pandoublefeng,@liucunwei
 2021-03-25 22:58:35 +08:00
YangLuo f7ea18944d reduce ut time of cutmix_batch 2021-03-25 21:59:44 +08:00
mindspore-ci-bot a48785cdcc !14052 add op atomic clean to clear input addr in launch allreduce 
From: @lvchangquan
Reviewed-by: @kisnwang,@chujinjin
Signed-off-by: @chujinjin
 2021-03-25 19:59:42 +08:00
mindspore-ci-bot 5926f7c386 !13906 【MD】【BUG】Conventional judgment on the parameter h w not less than zero 
From: @xulei2020
Reviewed-by: 
Signed-off-by:
 2021-03-25 14:10:42 +08:00
lvchangquan 0a7df321fe add op atomic clean to clear input addr in launch allreduce 2021-03-25 11:19:23 +08:00
xulei2020 4122d1dcce fix code format bug 2021-03-25 09:18:36 +08:00
mindspore-ci-bot b1b99b97d7 !13750 Add check of ResizePreserveARWithFiller operation 
From: @shenwei41
Reviewed-by: 
Signed-off-by:
 2021-03-24 15:27:16 +08:00
shenwei41 d3265f0591 Add check of ResizePreserveARWithFiller operation 2021-03-23 23:30:53 +08:00
mindspore-ci-bot 3329c07646 !13668 add glibcxx param 
From: @yepei6
Reviewed-by: 
Signed-off-by:
 2021-03-23 14:34:57 +08:00
yepei6 0cb6adce9e add ENABLE_GLIBCXX param 2021-03-22 20:40:01 +08:00
mindspore-ci-bot 06892b83ff !13575 Add check to RgbToGray 
From: @shenwei41
Reviewed-by: @tiancixiao,@liucunwei
Signed-off-by: @liucunwei
 2021-03-22 12:11:39 +08:00
mindspore-ci-bot 5ff9c08680 !13515 cpp api modify 
From: @zhoufeng54
Reviewed-by: 
Signed-off-by:
 2021-03-20 15:13:55 +08:00
mindspore-ci-bot 07ef04ad4b !13614 [MD] Fix Canny when the ksize is set to 7 
From: @tiancixiao
Reviewed-by: @heleiwang,@liucunwei
Signed-off-by: @liucunwei
 2021-03-20 15:05:08 +08:00
mindspore-ci-bot 8ab9d99714 !13507 dynamic shape adapter 
From: @liubuyu
Reviewed-by: @zhoufeng54,@jjfeing
Signed-off-by: @jjfeing
 2021-03-20 13:59:06 +08:00
liubuyu a03452ff2c dynamic shape bug fix 2021-03-19 17:57:31 +08:00
Xiao Tianci b88a68c9e8 fix canny when size is 7 2021-03-19 17:26:59 +08:00
mindspore-ci-bot 0451a800bc !13571 Added seed to prevent random failures in ut 
From: @ezphlow
Reviewed-by: 
Signed-off-by:
 2021-03-19 17:17:42 +08:00
lixian 693b4cfdc8 refactor cpp context, add string tensor, add get tensor by name 2021-03-19 16:08:55 +08:00
mindspore-ci-bot 8e8f3043f9 !12115 IR operators of GPU and CPU are unified as batchnorm 
From: @ding_fei_fei
Reviewed-by: 
Signed-off-by:
 2021-03-19 14:51:26 +08:00
shenwei41 8a1f283efc Add check to RgbToGray 2021-03-19 10:35:35 +08:00
Eric a3c98d9d59 Added fix for random core dump 2021-03-18 21:15:50 -04:00
alex-yuyue 2d6d608fae Add WordpieceTokenizer to API and IR. Add AddDict to C++ API 
Signed-off-by: alex-yuyue <yue.yu1@huawei.com>
 2021-03-18 16:42:31 -04:00
dingpeifei 87e41aaeee IR operators of GPU and CPU are unified as batchnorm 2021-03-18 19:02:28 +08:00
mindspore-ci-bot b45f289801 !13504 Add check of GetPerspectiveTransform and GetAffineTransform 
From: @shenwei41
Reviewed-by: @liucunwei,@pandoublefeng
Signed-off-by: @liucunwei
 2021-03-18 09:57:52 +08:00
mindspore-ci-bot dd90f7f055 !13506 dataset UT: Reinstate INFO logging and data verification - part 3 
From: @cathwong
Reviewed-by: @robingrosman,@liucunwei
Signed-off-by: @robingrosman
 2021-03-18 09:35:22 +08:00
Cathy Wong 1775c4e83d dataset: Reinstate INFO logging and data verification - part 3 
Updates: GetItemAt, CreateFromVector, CreateScalar.
Add TEST_MS_LOG_MSTENSOR.
Wrap ASSERT_OK around Tensor function calls.
 2021-03-17 13:48:04 -04:00
shenwei41 89f109a3f3 add check of operation 2021-03-17 20:29:17 +08:00
Eric a7da38feeb Added example for ngram 2021-03-16 23:45:04 -04:00
mindspore-ci-bot c0aa5f8132 !13365 Fix random rotation input check 
From: @ezphlow
Reviewed-by: 
Signed-off-by:
 2021-03-17 09:42:37 +08:00
mindspore-ci-bot f68732c15d !13279 Add resizefiller operation 
From: @shenwei41
Reviewed-by: 
Signed-off-by:
 2021-03-17 09:41:46 +08:00
Eric c730df23e8 Added json fix 2 2021-03-16 18:32:42 -04:00
Eric 4be0ffb9ba removing type_id.h from header files 2021-03-16 16:51:10 -04:00
mindspore-ci-bot 465ea885de !13399 workspace of comm op can be reused 
From: @zhoufeng54
Reviewed-by: @kisnwang,@chujinjin
Signed-off-by: @chujinjin
 2021-03-16 20:55:34 +08:00
shenwei41 d1bb9d470e Add resizefiller 2021-03-16 20:35:27 +08:00
mindspore-ci-bot bba879a703 !13379 fix bias add infer 
From: @lianliguang
Reviewed-by: @ginfung,@chujinjin
Signed-off-by: @chujinjin
 2021-03-16 15:10:16 +08:00
zhoufeng b7e5f956e5 workspace of comm op can be reused 
Signed-off-by: zhoufeng <zhoufeng54@huawei.com>
 2021-03-16 14:42:52 +08:00
mindspore-ci-bot 77cda67b3f !13012 add mul fusion based on allreduce fusion 
From: @lvchangquan
Reviewed-by: 
Signed-off-by:
 2021-03-16 14:04:51 +08:00
LianLiguang d34074d1ac fix bug of biasadd grad's infer shape 2021-03-16 10:45:20 +08:00
Eric 9b921953f9 misc issues 2021-03-15 22:30:04 -04:00
Lixia Chen 30017ea5a0 Fix codex issues 2021-03-15 11:15:15 -04:00
lvchangquan 31f9e6a42c add op_mul fusion based on allreduce fusion in pynative mode 2021-03-15 21:12:14 +08:00
mindspore-ci-bot 0cf6fb5310 !13264 [MD] Fix input check bug in RandomSelectSubpolicyOperation and codex issues 
From: @tina_mengting_zhang
Reviewed-by: @robingrosman,@nsyca
Signed-off-by: @robingrosman
 2021-03-15 20:29:56 +08:00
mindspore-ci-bot b8b96e15e7 !13155 [ME]Fix bug of embed J and side-by-side J 
From: @chenfei52
Reviewed-by: @zh_qh
Signed-off-by: @zh_qh
 2021-03-15 15:35:36 +08:00
ms_yan b4b4577b85 adapter profiling with new ascend lib 2021-03-14 11:38:20 +08:00
mindspore-ci-bot 98a87ad969 !13220 Add data helper to C++ API 
From: @ezphlow
Reviewed-by: 
Signed-off-by:
 2021-03-13 09:52:54 +08:00
TinaMengtingZhang ac05546aeb fix input check bug in RandomSelectSubpolicy and codex issue 2021-03-12 16:04:38 -05:00
Eric 82f7c6f2c7 Moving data_helper 
exposing json

WIP fix for removing templates

Removing templates part 1

replacing core/constants.h

removed template part 2

Changed fake data to filler data

Fixing CI

Fix CI part 2

CI fix part 2

Fix CI 3

Fixed lizard for data_util

Fixed clang

Fixing CI part 4
 2021-03-12 15:30:34 -05:00
mindspore-ci-bot 483bb9de60 !13078 dataset UT: Reinstate INFO logging and data verification - part 2 
From: @cathwong
Reviewed-by: 
Signed-off-by:
 2021-03-13 03:55:34 +08:00
Cathy Wong 95bbaaf9f6 dataset: Reinstate INFO logging and data verification - part 2 
for image/vision datasets
 2021-03-12 10:07:45 -05:00
chenfei fc335daa30 get all j node and then expand them 2021-03-12 20:06:43 +08:00
mindspore-ci-bot be17d5c54c !11678 Reactor pynative 
From: @zjun3021
Reviewed-by: 
Signed-off-by:
 2021-03-12 17:47:55 +08:00
mindspore-ci-bot 00f2ec66df !13173 Change switch to Switch 
From: @liangzhibo
Reviewed-by: @kingxian,@jpc_chenjianping
Signed-off-by: @kingxian
 2021-03-12 14:23:43 +08:00
mindspore-ci-bot f7040ca134 !13198 Fix get_dataset_size error with PKSampler 
From: @ziruiwu
Reviewed-by: 
Signed-off-by:
 2021-03-12 06:11:26 +08:00
Zirui Wu 2692e3cc3d fix pk sampler's get_dataset error due to num_class unavaiable at pre-runtime 2021-03-11 15:43:02 -05:00
mindspore-ci-bot 236c874e64 !12862 Increase default level of parallelism in minddata pipeline 
From: @robingrosman
Reviewed-by: 
Signed-off-by:
 2021-03-12 04:23:02 +08:00
zjun ae3a9d2883 Reactor pynative 
Signed-off-by: zjun <zhangjun0@huawei.com>

Modify real dynamic

Signed-off-by: zjun <zhangjun0@huawei.com>
 2021-03-11 19:37:43 +08:00
l00591931 bbdb050fc7 Change switch to Switch 2021-03-11 17:06:55 +08:00
RobinGrosman a0e75b845a Increase default level of parallelism in mindddata pipeline to 8. 
Adjust memory usage to not increase as parallelism increases.  it will
stay at same level it would be with 4 parallelism
 2021-03-10 18:38:11 -08:00
mindspore-ci-bot 0febd9c891 !13017 Add RGB2GRAY Operation 
From: @shenwei41
Reviewed-by: 
Signed-off-by:
 2021-03-11 10:08:25 +08:00
Cathy Wong 71ad3bb808 dataset: Specify Sampler and TensorTransform C++ API classes as final 2021-03-10 15:25:20 -05:00
shenwei41 7b56d1772e Add RGB2GRAY operation 2021-03-10 20:59:50 +08:00
mindspore-ci-bot c69142fdc1 !12968 update reshape type for 3d nodes 
From: @liubuyu
Reviewed-by: 
Signed-off-by:
 2021-03-10 20:33:04 +08:00
jonyguo c426cd5434 fix: increace deviation_ratio for gnn test case 2021-03-10 15:58:46 +08:00
liubuyu 518818fbef reshape type for 3d nodes 2021-03-10 08:56:10 +08:00
mindspore-ci-bot 62a6f783b3 !12809 Added pull-based prototype 
From: @mahdirahmanihanzaki
Reviewed-by: 
Signed-off-by:
 2021-03-09 22:31:38 +08:00
LianLiguang 4acab81599 using cpp infer firstly 2021-03-09 15:37:17 +08:00
Mahdi 9c3c36217e Added pull based prototype 2021-03-08 15:00:14 -05:00
YangLuo 74431ad117 Fix bugs of minddata transforms 2021-03-06 16:41:51 +08:00
Zirui Wu 8022001450 Remove datasets.h dependency on tensor.h and other related dependencies 
Replace std::function<TensorRow(TensorRow)> with std::function<MSTensorVec(MSTensor)> by using std::bind

Replace pad_info with MSTensor

Switch TensorShape/dataset::DataType to vector<int64> /mindspore::DataType

Change TypeId to mindspore::DataType in SchemaObj

address review cmts, add CreateFromMSTensor() in tensor.h/.cc
 2021-03-05 16:26:58 -05:00
mindspore-ci-bot ea8db7baf4 !12936 dataset UT: Reinstate INFO logging and data verification - part 1 
From: @cathwong
Reviewed-by: @robingrosman,@nsyca
Signed-off-by: @robingrosman
 2021-03-06 04:36:14 +08:00
Cathy Wong 521c313a3b dataset: Reinstate INFO logging and data verification - part1 2021-03-05 10:13:09 -05:00
mindspore-ci-bot d285692217 !12852 Change maketuple in coreops 
From: @liangzhibo
Reviewed-by: 
Signed-off-by:
 2021-03-05 19:13:42 +08:00
mindspore-ci-bot 9baec79337 !12881 add GaussianBlur and Canny for lite_cv 
From: @tiancixiao
Reviewed-by: 
Signed-off-by:
 2021-03-05 17:33:20 +08:00
l00591931 680324f225 Change make tuple in core.ops 2021-03-05 10:54:33 +08:00
TinaMengtingZhang 6280582315 fix testcases in c_api_random_test.cc 
fix testcases in c_api_transforms_test.cc
 2021-03-04 11:20:23 -05:00
Xiao Tianci 449452c7ef add gaussian blur and canny for lite cv process 2021-03-04 22:27:48 +08:00
mindspore-ci-bot 423dcfc917 !12836 Change return in core_ops 
From: @liangzhibo
Reviewed-by: @kingxian,@jpc_chenjianping
Signed-off-by: @kingxian
 2021-03-04 19:27:44 +08:00
mindspore-ci-bot 2f312dac66 !12091 Performance optimization for PyNative AllReduce 
From: @jojobugfree
Reviewed-by: 
Signed-off-by:
 2021-03-04 15:24:22 +08:00
l00591931 cf7c5840e3 Change return 2021-03-04 09:38:43 +08:00
Eric f9a2379a70 Fixing api leftover 
Fixed compile error

Fixed more testcases
 2021-03-03 09:37:06 -05:00
caifubi 171b468bb3 PyNative AllReduce Bucket 2021-03-03 15:47:18 +08:00
mindspore-ci-bot 4f504ad2a3 !12760 Fix include file(samplers.h) contains internal head file & Fix execute ut 
From: @luoyang42
Reviewed-by: @pandoublefeng,@heleiwang,@liucunwei
Signed-off-by: @pandoublefeng,@liucunwei
 2021-03-03 15:02:52 +08:00
mindspore-ci-bot 7f71525b0a !12810 dataset: Add IR vision error input tests, plus other UT updates 
From: @cathwong
Reviewed-by: 
Signed-off-by:
 2021-03-03 06:24:20 +08:00
Cathy Wong 253f389817 dataset: Add IR vision error input tests, plus other UT updates 2021-03-02 16:33:28 -05:00
mindspore-ci-bot 1f93d097cf !12762 [MD] C++ API class: make Parse() as protected 
From: @tina_mengting_zhang
Reviewed-by: 
Signed-off-by:
 2021-03-03 05:01:05 +08:00
YangLuo eecbc99b2e fix minddata include internal head file & fix execute ut 2021-03-02 14:12:33 +08:00
chendongsheng db0a6f1e19 replace ps-lite 2021-03-02 11:19:11 +08:00
mindspore-ci-bot 8800347e7a !12745 remove test address exception 
From: @anancds
Reviewed-by: @cristoval,@xsmq
Signed-off-by: @cristoval
 2021-03-02 10:22:39 +08:00
TinaMengtingZhang ccefa8b32e Make Parse() as protected 2021-03-01 15:07:36 -05:00
mindspore-ci-bot 85461bcdb3 !12532 add SyncBatchNorm 
From: @yuchaojie
Reviewed-by: 
Signed-off-by:
 2021-03-01 19:34:41 +08:00
chendongsheng dfcbf96424 remove test address exception 2021-03-01 16:20:26 +08:00
mindspore-ci-bot 614cf339ad !12703 Add a switch to control grad for scalar 
From: @ginfung
Reviewed-by: 
Signed-off-by:
 2021-02-27 20:53:42 +08:00
mindspore-ci-bot c34fda33df !12578 added server 
From: @anancds
Reviewed-by: @cristoval,@limingqi107
Signed-off-by: @cristoval
 2021-02-27 19:36:19 +08:00
yujianfeng f0b7c3dd4d Add a switch to control grad for scalar 2021-02-27 18:12:37 +08:00
mindspore-ci-bot dd22b5ea71 !12432 api support dual abi 
From: @zhoufeng54
Reviewed-by: 
Signed-off-by:
 2021-02-27 16:19:54 +08:00
chendongsheng febbbf45c8 added server 2021-02-27 15:49:18 +08:00
chendongsheng c2be37aee8 fixed ut 2021-02-27 14:36:03 +08:00
yuchaojie 6d195f340c add SyncBatchNorm 2021-02-27 14:32:54 +08:00
mindspore-ci-bot 7c32bbe972 !12516 added worker 
From: @anancds
Reviewed-by: 
Signed-off-by:
 2021-02-27 14:22:39 +08:00
mindspore-ci-bot 5589881ffe !12612 dataset: C++ API Vision support for Ops with TensorTransform op input 
From: @cathwong
Reviewed-by: 
Signed-off-by:
 2021-02-27 10:30:53 +08:00
zhoufeng 504f215800 api support dual abi 
Signed-off-by: zhoufeng <zhoufeng54@huawei.com>
 2021-02-27 10:09:40 +08:00
mindspore-ci-bot ac5b59c962 !12634 remove cpp infer reg 
From: @lianliguang
Reviewed-by: 
Signed-off-by:
 2021-02-27 09:29:44 +08:00
Cathy Wong 2f1f0c438d dataset: C++ API Vision support for Ops with TensorTransform op input 
plus UTs
 2021-02-26 16:25:30 -05:00
mohammad 6ffaf6bf00 fix sampler call in affine transform test 2021-02-26 15:35:06 -05:00
mindspore-ci-bot f1c7aeaa6b !12555 Create C++ API classes for Samplers 
From: @mhmotallebi
Reviewed-by: 
Signed-off-by:
 2021-02-27 02:54:19 +08:00
mindspore-ci-bot 5556b12da4 !12480 Adding Affine API 
From: @ezphlow
Reviewed-by: 
Signed-off-by:
 2021-02-26 21:35:05 +08:00
chendongsheng 6c22dc0d55 added worker 2021-02-26 21:10:16 +08:00
mindspore-ci-bot b13cabeb10 !12621 Eliminate all redundant nodes related to UpdateStates. 
From: @zh_qh
Reviewed-by: @ginfung,@hwhewei
Signed-off-by: @ginfung
 2021-02-26 16:14:47 +08:00
mindspore-ci-bot f269f3fc8d !12231 Support ad for scalar inputs 
From: @ginfung
Reviewed-by: 
Signed-off-by:
 2021-02-26 15:50:32 +08:00
LianLiguang 2fa4997674 remove all of cpp infer reg 2021-02-26 15:19:26 +08:00
Zhang Qinghua 9b26c210f4 Eliminate all useless nodes related to UpdateStates. 2021-02-26 14:32:50 +08:00
mohammad f73def5933 make C++ API classes for samplers 2021-02-25 17:29:43 -05:00
Eric d328d6eb54 Adding affine 
Need to move Affine to image_utilC

Refactored affine

Added affine API

Added API class impl

Removed default from vision ir

Compiling

removed extra file

Spell check
 2021-02-25 16:55:02 -05:00
yujianfeng 41189781f3 support scalar input for cell 2021-02-25 09:56:15 +08:00
TinaMengtingZhang 518828b639 Support 3 composed data ops and cleanup code 2021-02-24 11:05:44 -05:00
Zhenglong Li 581f1e82d4 API merge with new API class and new IR class 2021-02-24 10:55:18 +08:00
mindspore-ci-bot 182d928379 !12553 dataset: Split c_api_vision_test.cc 
From: @cathwong
Reviewed-by: @robingrosman,@nsyca
Signed-off-by: @robingrosman
 2021-02-24 02:01:30 +08:00
Cathy Wong 6112738fea dataset: Split c_api_vision_test.cc 2021-02-23 11:04:29 -05:00
mindspore-ci-bot 5c3fe0f9ac !12526 replace tab with spaces 
From: @xsmq
Reviewed-by: @ljl0711,@zhoufeng54
Signed-off-by: @zhoufeng54
 2021-02-23 10:12:32 +08:00
mindspore-ci-bot 086374f306 !12493 Added GetAffineTransform operator 
From: @shenwei41
Reviewed-by: 
Signed-off-by:
 2021-02-23 09:54:04 +08:00
xsmq 923b53d2e6 replace tab with spaces 2021-02-23 09:41:47 +08:00
chendongsheng 425e5cae49 added log callback 2021-02-22 20:05:35 +08:00
shenwei41 d8d56b5374 GetAffineTransform 2021-02-22 15:22:34 +08:00
Cathy Wong 3fc204e3bc dataset: Create C++ API classes for vision transform ops 
Review rework

Add overloaded API Map function

Overload Execute, update st test_de.cc and enhance ut execute_test.cc;
More TensorTransform support for BoundingBoxAugment and UniformAug

Execute support for auto case; UT updates

Rebase updates, including fix CI Ascend compilations

test_de.cc - comment out dvpp tests

Rebase updates

Added back testcase and constructor

Added back constructor

added missing macro

Fixed Macro

removed decode
 2021-02-20 15:29:33 -05:00
mindspore-ci-bot 98f6d78778 !12361 Provide C++ API class for text and data transforms ops 
From: @tina_mengting_zhang
Reviewed-by: 
Signed-off-by:
 2021-02-20 04:38:26 +08:00
TinaMengtingZhang c5aa3eeef2 Add API class for data transform ops except Compose, RandomApply, RandomChoice 
Add API class for text transforms ops
 2021-02-18 15:58:44 -05:00
mohammad 9e6fcc7f23 return status from sampler IR functions 2021-02-17 12:01:34 -05:00
mindspore-ci-bot c8332c16f7 !11957 getRotationMatrix2D and getPerspectiveTransform 
From: @shenwei41
Reviewed-by: 
Signed-off-by:
 2021-02-11 00:08:56 +08:00
shenwei41 737e828f3d getRotationMatrix2D and getPerspectiveTransform 2021-02-10 12:35:56 +08:00
Zhenglong Li e2d3495925 Dvpp refactor and 6 Dvpp operators 2021-02-10 10:34:38 +08:00
Eric 9953757ff4 Fixed tenor compile error for MD 2021-02-09 14:56:31 +08:00
He Wei 7d9a783993 [auto-monad] Support side-effects by auto-monad 
The basic idea is: exploits data dependency to control the execution order
of side-effect operations, and keep the semantics of ANF unchanged.

The ControlDepend primitive is removed and there are two primitives added:

1. UpdateState:
```
  a = Assign(para, value)
```
became:
```
  a = Assign(para, value, u)
  u = UpdateState(u, a)
```

2. Load:
```
  x = Add(para, value)
```
became:
```
  p = Load(para, u)
  x = Add(p, value)
  u = UpdateState(u, p)
```
 2021-02-08 09:01:15 +08:00
chendongsheng 40f2571f7e added http client 2021-02-06 21:50:44 +08:00
mindspore-ci-bot 500bbf5da7 !12164 update graphengine 
From: @shenwei41
Reviewed-by: 
Signed-off-by:
 2021-02-06 13:04:53 +08:00
jjfeing 502be04491 upgrade 0204 2021-02-06 10:51:43 +08:00
jinyaohui 30a27b2adb modify Gelu、FastGelu to GeLU and FastGeLU 2021-02-05 17:19:52 +08:00
mindspore-ci-bot aebe263dce !11895 unify mindir for different backend: the output num of optimizer ops, the backward of concat 
From: @wangnan39
Reviewed-by: 
Signed-off-by:
 2021-02-05 16:27:19 +08:00
mindspore-ci-bot c9aaa70b39 !12092 Change L2Norm, merge from r1.1 to master 
From: @liangzhibo
Reviewed-by: @ginfung
Signed-off-by:
 2021-02-05 14:13:12 +08:00
mindspore-ci-bot ad5b033cc5 Change L2Norm, r1.1 to master 2021-02-05 09:42:43 +08:00
Cathy Wong 3e6e35d749 dataset: delete minddata/dataset/include/status.h 2021-02-04 10:19:08 -05:00
jinyaohui 8022f9a6ed modify pack to stack 2021-02-04 18:54:31 +08:00
wangnan39@huawei.com cd9173fdfd unify the output num of optimizer ops 2021-02-04 17:46:00 +08:00
jinyaohui d9be0c102d add some ops 2021-02-03 19:47:53 +08:00
mindspore-ci-bot cbfba95ad0 !11921 do not broaden when arg is not tensor 
From: @zhangbuxue
Reviewed-by: 
Signed-off-by:
 2021-02-03 17:07:47 +08:00
buxue 6ccc4379b4 do not broaden scalar 2021-02-03 11:30:25 +08:00
hesham 715d70f397 Remove setting num_parallel_worker in UT, since it would fail for machines with less than 16 cores. 2021-02-02 11:38:27 -05:00
mindspore-ci-bot a616196586 !11982 Change tuple_getitem to TupleGetItem merge from r1.1 to master 
From: @liangzhibo
Reviewed-by: @ginfung,@zh_qh
Signed-off-by: @zh_qh
 2021-02-02 19:06:39 +08:00
shenghong96 49144fde37 fix UT of test_topk_no_split 2021-02-02 16:52:20 +08:00
mindspore-ci-bot b189f177bb Change tuple_getitem to TupleGetItem and some other ops, merge from r1.1 to master 2021-02-02 15:54:15 +08:00
xsmq a8259bae9b disable ut cpp case(test_topk_no_split) 2021-02-02 15:52:05 +08:00
mindspore-ci-bot e02058133e !11526 unifiled lite & cloud api 
From: @zhoufeng54
Reviewed-by: 
Signed-off-by:
 2021-02-02 10:39:57 +08:00
mindspore-ci-bot e897eb4c41 !11915 Change TensorAdd to Add, merge from r1.1 to master 
From: @liangzhibo
Reviewed-by: @ginfung,@zh_qh
Signed-off-by: @zh_qh
 2021-02-02 09:16:08 +08:00
lixian 7d2fd6e76c cxx api refactor: tensor/status/model 2021-02-02 09:15:56 +08:00
Nat Sutyanyong 5a7dc0accc Migrate repeat_pass.cc to IR optimizer and remove ExecTree optimizer 2021-02-01 12:30:16 -05:00
l00591931 9ec100d069 Change TensorAdd to Add, from r1.1 to master 2021-02-01 17:53:52 +08:00
mindspore-ci-bot 066ebe516e !11794 remove useless code of dot 
From: @yuan_shen_zhou
Reviewed-by: @liangchenghui
Signed-off-by: @liangchenghui
 2021-02-01 17:38:26 +08:00
mindspore-ci-bot a9dfb07cf1 !11660 get cube size by data type 
From: @liubuyu
Reviewed-by: 
Signed-off-by:
 2021-02-01 11:20:49 +08:00
zhouyuanshen d1e6878011 remove dot 2021-02-01 09:39:27 +08:00
mindspore-ci-bot ce89cc5e8b !11761 Change GatherV2 to Gather (merge from r1.1 to master) 
From: @liangzhibo
Reviewed-by: 
Signed-off-by:
 2021-01-30 17:53:20 +08:00
mindspore-ci-bot f980561fe1 !11598 [MD] add common validation functions for transform 
From: @luoyang42
Reviewed-by: 
Signed-off-by:
 2021-01-30 14:32:44 +08:00
mindspore-ci-bot 9fa0499fa0 Change GatherV2 to Gather r1.1 to master 2021-01-30 14:13:47 +08:00
liubuyu 4a3cbda322 get cube size by dtype 2021-01-29 20:08:56 +08:00
YangLuo 1893ae8662 [MD] add common validation functions for transform 2021-01-29 16:29:47 +08:00
mindspore-ci-bot 20c1727b2c !11631 Add SubsetSampler 
From: @hfarahat
Reviewed-by: 
Signed-off-by:
 2021-01-29 03:48:45 +08:00
mindspore-ci-bot 8e4a6842ff !11681 gnn support edge weight sampling 
From: @heleiwang
Reviewed-by: @liucunwei
Signed-off-by: @liucunwei
 2021-01-28 19:15:37 +08:00
heleiwang 2dc9ba761c gnn support weight sampling neighbors. 2021-01-28 15:28:44 +08:00
hesham 7120c66998 Add SubsetSampler 2021-01-27 15:00:18 -05:00
mindspore-ci-bot 4cd6588af0 !11393 Migrating cache transform pass from execution tree to IR tree 
From: @lixiachen
Reviewed-by: 
Signed-off-by:
 2021-01-27 22:10:13 +08:00
xulei2020 de630f1cb1 Add parameter verification for getROI function 
fix c++ ut compilation failed of Asan bug
 2021-01-27 09:06:59 +08:00
Lixia Chen 922e1f4f36 Migrate CacheTransformPass 2021-01-26 19:24:10 -05:00
chendongsheng c7fe82b43d Custom data transmission format 2021-01-25 10:37:05 +08:00
Cathy Wong a3b10213b7 dataset: Resolve protected parent_access from DeepCopyPass 2021-01-22 15:56:32 -05:00
xiefangqi 3f76abcdee minddata err msg add file path 2021-01-22 16:38:46 +08:00
mindspore-ci-bot 2beb916bf5 !11427 TextOp decoupling 
From: @alexyuyue
Reviewed-by: 
Signed-off-by:
 2021-01-22 09:09:41 +08:00
mindspore-ci-bot fcc4a4eaea !11397 add GetDatasetSize to concat_node 
From: @mhmotallebi
Reviewed-by: 
Signed-off-by:
 2021-01-22 08:08:41 +08:00
mohammad b5e91139e2 add GetDatasetSize to concat node 2021-01-21 16:25:48 -05:00
mindspore-ci-bot 0641940b87 !11472 support to checkpoint group info 
From: @yangzhenzhang
Reviewed-by: @stsuteng,@kisnwang
Signed-off-by: @stsuteng
 2021-01-21 17:11:23 +08:00
yangzhenzhang 7303c3d3b8 add group ckpt 2021-01-21 14:41:05 +08:00
xulei2020 de6356ebfa add warp affine 2021-01-21 14:20:14 +08:00
alex-yuyue a8675f8227 TextOp decoupling 
Signed-off-by: alex-yuyue <yue.yu1@huawei.com>
 2021-01-20 21:52:45 -05:00
mindspore-ci-bot d8323b5d51 !11342 Support device memeory profiling 
From: @yanghaitao1
Reviewed-by: @wangyue01,@lilongfei15
Signed-off-by: @lilongfei15
 2021-01-20 16:18:42 +08:00
yuchaojie 1932d87a26 update some op's attr name 2021-01-20 09:16:43 +08:00
mindspore-ci-bot 60feffad20 !11332 update Pooling's attr kernel_size, pad_mode 
From: @yuchaojie
Reviewed-by: 
Signed-off-by:
 2021-01-19 16:17:45 +08:00
yanghaitao1 8d147deb07 profiler memory 2021-01-19 10:21:43 +08:00
mindspore-ci-bot c155dfee7a !11329 [MD] Rename SamplerObj::Build() to SamplerBuild() 
From: @lixiachen
Reviewed-by: 
Signed-off-by:
 2021-01-19 06:33:49 +08:00
Lixia Chen 273865a72b Rename SamplerObj::Build() to SamplerBuild() 2021-01-18 15:42:34 -05:00
Lixia Chen 3c5917a03d Fix IR node Drop() 2021-01-18 13:06:07 -05:00
yuchaojie b51b3a6764 update Pool's attr kernel_size, pad_mode 2021-01-18 12:10:20 +08:00
mindspore-ci-bot cb58b5baa5 !11211 Provide functions for modifying IR tree (Insert/Drop/Append nodes) 
From: @lixiachen
Reviewed-by: 
Signed-off-by:
 2021-01-16 01:20:01 +08:00
Nat Sutyanyong e28fb6ce4d Tested with new test cases and all dataset UTs passed 2021-01-15 09:59:21 -05:00
yangzhenzhang 38ea8784c6 update infer mirror ops 2021-01-15 14:53:21 +08:00
mindspore-ci-bot dfa6daaa57 !11249 【MD】fix bug for extractchannel when type is UINT16 
From: @xulei2020
Reviewed-by: @heleiwang,@liucunwei
Signed-off-by: @liucunwei
 2021-01-15 14:33:16 +08:00
mindspore-ci-bot 581789c252 !11246 add num_epochs to CPP API Dataset::CreateIterator() 
From: @ziruiwu
Reviewed-by: 
Signed-off-by:
 2021-01-15 06:02:00 +08:00
mindspore-ci-bot 8ef663fa68 !11248 dataset: Use int32_t for text's vocab_size 
From: @cathwong
Reviewed-by: @mikef,@robingrosman
Signed-off-by:
 2021-01-15 00:03:09 +08:00
xulei2020 5dd651f1a5 fix bug for extractchannel when type is UINT16 2021-01-14 09:26:18 +08:00
Cathy Wong 4b1ccf9dd5 dataset: Use int32_t for text's vocab_size 2021-01-13 18:12:50 -05:00
Zirui Wu fa553956ef add num_epochs to CPP API Dataset::CreateIterator(columns,num_epochs) 2021-01-13 12:13:40 -05:00
lilei 9a45c4419c modify batch_normal 2021-01-13 20:59:46 +08:00
mohammad f48ab2b5c9 make MD public headers standalone 2021-01-11 06:35:53 -05:00
chendongsheng 048534b75e Switch bare pointer to shared_ptr 2021-01-09 10:21:07 +08:00
Zirui Wu 0e68575e77 port over getter pass 
port over tensor_op_fusion pass

add fusion support for prebuildOpeartion (TensorOpeartion)
 2021-01-07 10:52:49 -05:00
mindspore-ci-bot 1d1f6841b9 !10937 keep consistent in Graph mode and PyNative mode for 'isinstance' 
From: @zhangbuxue
Reviewed-by: 
Signed-off-by:
 2021-01-06 10:29:49 +08:00
mindspore-ci-bot 5fd77356c9 !9687 Adding AddChild functionality to SamplerObj 
From: @mahdirahmanihanzaki
Reviewed-by: 
Signed-off-by:
 2021-01-06 01:46:08 +08:00
buxue 6d395c1d3f keep consistent in Graph mode and PyNative mode for 'isinstance' 2021-01-05 21:36:10 +08:00
mindspore-ci-bot a5869f2984 !10142 dataset CPP UT: Updates for ExecTree to IRTree Support 
From: @cathwong
Reviewed-by: 
Signed-off-by:
 2021-01-05 06:53:14 +08:00
Nat Sutyanyong 7c5b2de1ec dataset CPP UT: Updates for ExecTree to IRTree Support 2021-01-04 16:00:30 -05:00
mindspore-ci-bot 4322d737f3 !10689 added retry and unit test 
From: @anancds
Reviewed-by: @limingqi107,@cristoval
Signed-off-by:
 2020-12-29 20:08:48 +08:00
mindspore-ci-bot 4bb54e5ffe !10710 add ut for primitve c of conv2d 
From: @lianliguang
Reviewed-by: @kisnwang,@chujinjin
Signed-off-by: @chujinjin
 2020-12-29 19:13:21 +08:00
LianLiguang 4504766bbd add ut for conv2d 2020-12-29 17:21:11 +08:00
chendongsheng b289c6184a added retry and unit test 2020-12-29 16:32:04 +08:00
mindspore-ci-bot ca3ac703c9 !10627 handler signal 
From: @zhupuxu
Reviewed-by: 
Signed-off-by:
 2020-12-29 15:47:30 +08:00
jonyguo f97d03f695 impl mindrecord lazy load 2020-12-29 11:47:32 +08:00
zhupuxu e36175b0dc handler signal 
Signed-off-by: zhupuxu <zhupuxu@huawei.com>
 2020-12-29 09:39:13 +08:00
Xiaoda Zhang 3c27c08b46 change memory cost calculation in auto-parallel 2020-12-28 14:38:30 +08:00
mindspore-ci-bot b96d4315dc !10393 added collective send and receive 
From: @anancds
Reviewed-by: 
Signed-off-by:
 2020-12-28 14:34:19 +08:00
LianLiguang cb909c7242 add ut for conv2d 2020-12-26 17:04:37 +08:00
chendongsheng 24345595f3 added collective send and receive 2020-12-26 11:19:26 +08:00
yanghaoran b8345d03b6 Synchronize latest Ascend software 18 Dec 2020, with profiler fixes 2020-12-24 15:11:01 +08:00
Zirui Wu 0a31f563a7 minor fixes 2020-12-23 10:06:01 -05:00
mindspore-ci-bot 20df37e428 !10287 Add ut for dataset to_number 
From: @luoyang42
Reviewed-by: @heleiwang
Signed-off-by:
 2020-12-23 21:14:42 +08:00
mindspore-ci-bot f1e1a2afd6 !10223 fixed codex and change bare pointer to shared pointer 
From: @anancds
Reviewed-by: @limingqi107
Signed-off-by:
 2020-12-22 11:06:07 +08:00
Zirui Wu 0e2f7a9e9e add validate param to base class DatasetNode 2020-12-21 11:14:37 -05:00
YangLuo da6de519ae Add ut for dataset to_number 2020-12-21 20:09:03 +08:00
chendongsheng ee23af58d8 fixed codex 2020-12-21 15:17:14 +08:00
Xiao Tianci 633f97bb48 fix get dataset size in distributed scenario 2020-12-15 20:41:45 +08:00
mindspore-ci-bot c4f284b928 !9910 [MD][Perf] MindData add NormalizePad for GPU performance 
From: @xiefangqi
Reviewed-by: 
Signed-off-by:
 2020-12-15 16:51:22 +08:00
mindspore-ci-bot dd134d7554 !9926 optimize list setitem in bprop 
From: @zhangbuxue
Reviewed-by: @zh_qh
Signed-off-by: @zh_qh
 2020-12-15 16:20:07 +08:00
buxue 2530943a7f optimize list getitem in bprop 2020-12-15 15:00:49 +08:00
xiefangqi 3c6cc3c58c Add NormalizePadOp 2020-12-15 11:45:41 +08:00
mindspore-ci-bot c403948961 !9931 add reset device, fix runtime delete ring buffer failed 
From: @zhoufeng54
Reviewed-by: @kisnwang,@chujinjin
Signed-off-by: @chujinjin
 2020-12-15 09:14:40 +08:00
mindspore-ci-bot fd311cf316 !9843 Added server node 
From: @anancds
Reviewed-by: 
Signed-off-by:
 2020-12-14 21:29:49 +08:00
zhoufeng ee7e412def add reset device, fix runtime delete ring buffer failed 
Signed-off-by: zhoufeng <zhoufeng54@huawei.com>
 2020-12-14 20:58:13 +08:00
mindspore-ci-bot 058fbd2d1f !9765 remove serving in mindspore repo 
From: @xu-yfei
Reviewed-by: @zhoufeng54,@kisnwang
Signed-off-by: @kisnwang
 2020-12-14 19:16:00 +08:00
chendongsheng 2d2bf2d0ee added server node 2020-12-14 17:04:44 +08:00
Mahdi 0f2b5d8cac Changed SamplerObj validate params to return status and added AddChild to it 2020-12-10 15:39:59 -05:00
xuyongfei c9b072b94d remove serving in mindspore repo 2020-12-10 18:50:30 +08:00
chendongsheng ee4132889e added worker node 2020-12-10 16:16:40 +08:00
Zirui Wu 91a6b2b0ca Add checks to RandomData and LookUp 2020-12-09 17:09:34 -05:00
mindspore-ci-bot e8a442eeb3 !9700 add ps cache manager 
From: @limingqi107
Reviewed-by: 
Signed-off-by:
 2020-12-09 23:47:12 +08:00
limingqi107 eca951c7bc add ps cache manager 2020-12-09 19:48:00 +08:00
yanghaoran 306bbe0e65 Synchronize latest Ascend software suite 09 Dec 2020, with refactored GraphEngine 2020-12-09 18:29:29 +08:00
mindspore-ci-bot 317a97e6b9 !9336 auto num_parallel_workers setup 
From: @ziruiwu
Reviewed-by: 
Signed-off-by:
 2020-12-09 08:32:55 +08:00
mindspore-ci-bot a2c80435ce !9685 Fix a core dump in TreeConsumer::Terminate() plus minor cache fixes 
From: @lixiachen
Reviewed-by: @mikef,@nsyca
Signed-off-by: @nsyca
 2020-12-09 08:21:22 +08:00
Zirui Wu d6df1b0832 Implemented AutoNumWorker Pass which sets num_workers of selected parallel ops automatically if enabled 2020-12-08 17:49:05 -05:00
Lixia Chen 32b82c2737 Fix a core dump in TreeConsumer::Terminate() 
and other minor cache changes
 2020-12-08 15:38:40 -05:00
limingqi107 a5f57ce8a0 add ps cache 2020-12-08 20:34:51 +08:00
Xiao Tianci 0659473535 add TruncateSequencePair, ToNumber C++ API and enable three test cases 2020-12-08 11:12:16 +08:00
mindspore-ci-bot 49fd5308a4 !9564 Tidy up code in dataset compilation phase 
From: @nsyca
Reviewed-by: 
Signed-off-by:
 2020-12-08 05:11:29 +08:00
mindspore-ci-bot 6b5626634c !9530 add BasicTokenizer and BertTokenizer C++ API 
From: @tiancixiao
Reviewed-by: @mikef,@nsyca
Signed-off-by: @nsyca
 2020-12-08 04:09:40 +08:00
Nat Sutyanyong 41ea1621c5 Tidy up TreeAdapter::Compile 2020-12-07 14:57:22 -05:00
shenwei41 206ae1a7c3 Add Three text C++ API 2020-12-05 10:32:15 +08:00
Nat Sutyanyong 4cb78f2e03 Remove Repeat(1),Take(-1), and Skip(0) in NodeRemovalPass 2020-12-04 17:37:07 -05:00
Jesse Lee 0c8dfc68ec Fix hang on RequestFreeTag 2020-12-04 14:54:30 -05:00
Xiao Tianci 290cc72887 add BasicTokenizer and BertTokenizer C++ API 2020-12-04 21:51:49 +08:00
Nat Sutyanyong d69a29a44e Migrate 3 pre passes to IR optimizer, namely, cache_error_pass, epoch_injection, and removal_pass 2020-12-03 14:40:03 -05:00
mindspore-ci-bot 73c91e05b1 !9398 Change Build to return status 
From: @alex-yuyue
Reviewed-by: @mikef
Signed-off-by:
 2020-12-03 23:43:08 +08:00
alex-yuyue ccff3a5316 Change build to return status 
Signed-off-by: alex-yuyue <yue.yu1@huawei.com>
 2020-12-02 22:46:49 -05:00
YangLuo a2ec7ad5bd Fix concat issue & fix schema doc 2020-12-02 21:06:22 +08:00
mindspore-ci-bot efc7e9f89e !9358 PyNative Bugfix 
From: @jojobugfree
Reviewed-by: @chujinjin,@jjfeing
Signed-off-by: @jjfeing
 2020-12-02 14:07:05 +08:00
caifubi c8e7c3edb0 PyNative Hccl Bugfix 2020-12-02 12:09:52 +08:00
mindspore-ci-bot 6c18c09121 !9266 PyNative Support Hccl(Data Parallel) 
From: @jojobugfree
Reviewed-by: @chujinjin,@jjfeing
Signed-off-by: @chujinjin
 2020-12-02 09:25:32 +08:00
mindspore-ci-bot df44e1339e !9249 Optimizing GetDatasetSize 
From: @mahdirahmanihanzaki
Reviewed-by: 
Signed-off-by:
 2020-12-02 08:26:34 +08:00
mindspore-ci-bot 9ed31ea2cb !9339 Disable getter optimization pass 
From: @ezphlow
Reviewed-by: @robingrosman,@nsyca
Signed-off-by: @nsyca
 2020-12-02 08:25:41 +08:00
Eric db2a8b5e1d Disable getter pass 2020-12-01 17:40:34 -05:00
Mahdi f80553bc6e Optimization for GetDatasetSize 2020-12-01 17:40:25 -05:00
caifubi 04d8cd5d8b Support PyNative Data Parallel 2020-12-01 20:55:39 +08:00
mindspore-ci-bot da8d7d661a !9283 Modify the dataset ut log message 
From: @liucunwei
Reviewed-by: @pandoublefeng,@heleiwang,@jonyguo
Signed-off-by: @heleiwang,@jonyguo
 2020-12-01 16:28:32 +08:00
mindspore-ci-bot c41a21d2c2 !9208 Add four new c++ api of text 
From: @shenwei41
Reviewed-by: @heleiwang
Signed-off-by:
 2020-12-01 14:46:14 +08:00
liucunwei 8ef4b2796e modify the dataset ut log message 2020-12-01 14:42:44 +08:00
mindspore-ci-bot 0e6d06c463 !9230 Set trace info of Primitive CNode only for equiv out node, not the whole fprop function. 
From: @zh_qh
Reviewed-by: @ginfung
Signed-off-by: @ginfung
 2020-12-01 14:27:53 +08:00
shenwei41 3bea84d0f7 add four new text API 2020-12-01 10:21:47 +08:00
Zhang Qinghua 8310236ff1 Set trace info of Primitive CNode only for equiv out node, not the whole fprop function. 2020-12-01 10:05:20 +08:00
mindspore-ci-bot ea191ed5da !8463 Run ops one by one in pynative bp graph 
From: @HulkTang
Reviewed-by: 
Signed-off-by:
 2020-12-01 09:06:44 +08:00
mindspore-ci-bot 7748ae823a !9141 Fix DatasetCache C++ API cache sharing behavior 
From: @lixiachen
Reviewed-by: @mikef,@nsyca
Signed-off-by: @nsyca
 2020-12-01 02:28:02 +08:00
mindspore-ci-bot a37e9710b5 !9038 port name functions from TensorOp to TensorOperation 
From: @mhmotallebi
Reviewed-by: 
Signed-off-by:
 2020-12-01 02:13:48 +08:00
HulkTang c36b477568 Run ops one by one in pynative bp graph 2020-11-29 15:48:21 +08:00
mindspore-ci-bot a6845395e6 !9181 Add some ut pipeline test case for C++ dataset 
From: @liucunwei
Reviewed-by: @pandoublefeng,@jonyguo
Signed-off-by: @jonyguo
 2020-11-28 21:34:03 +08:00
liucunwei e612e23a05 Add some ut pipeline test case for C++ dataset 2020-11-28 18:14:58 +08:00
liucunwei 7f13f2ad0e add some pipeline ut test cases for c++ dataset 2020-11-28 16:59:38 +08:00
mindspore-ci-bot cf383af36e !8624 updated cluster config and proto 
From: @anancds
Reviewed-by: 
Signed-off-by:
 2020-11-28 08:59:31 +08:00
mohammad 9cfeb08c4d port name() to TensorOperation and its derived classes 2020-11-27 19:00:56 -05:00
Lixia Chen e8247eadef Same runtime CacheClient if c++ cache is shared 2020-11-27 18:52:15 -05:00
Zirui Wu 36bfd0d8ad address left over cmts from 8802 2020-11-27 16:54:28 -05:00
liucunwei 3e57236fe3 add some new ut test for minddataset 2020-11-27 20:39:13 +08:00
anancds fc380b8071 updated cluster config and proto 2020-11-27 16:41:25 +08:00
shenwei41 4e56618d18 Add text C++ API 2020-11-27 11:41:35 +08:00
mindspore-ci-bot 3fa7233f68 !9072 fix topk fussion help tensor size = 4096*2 
From: @jjfeing
Reviewed-by: @chujinjin,@kisnwang
Signed-off-by:
 2020-11-27 10:03:48 +08:00
Nat Sutyanyong 5e1bb0b697 boilerplate code for future IR optimizer 
add 2 test cases to IRNode Deepcopy()

address review cmts

fix ut

samplerObj copy

ci

fix ci

fix ci round III

address further review cmts

add a missing macro

fix merge conflict

fix complie err

fix lite compile err

fix compile err

fix lite compile round III

address an issue

fix minor comments
 2020-11-26 09:42:01 -05:00
jjfeing 8fb7d11ecb fix topk help 4096 2020-11-26 19:16:37 +08:00
Eric Zhang 809e1d5086 added python api based on cpp api 
1st draft of python iterator

Added Cifar10 and Cifar100 pybind port

Change pybind to use IR for Skip and Manifest

Signed-off-by: alex-yuyue <yue.yu1@huawei.com>

DatasetNode as a base for all IR nodes

namespace change

Fix the namespace issue and make ut tests work

Signed-off-by: alex-yuyue <yue.yu1@huawei.com>

Add VOCDataset

!63 Added RandomDataset
* Added RandomDataset

add imagefolder ir

Pybind switch: CelebA and UT

!61 CLUE example with class definition
* Merge branch 'python-api' of gitee.com:ezphlow/mindspore into clue_class_pybind
* Passing testcases
* Added CLUE, not working

add ManifestDataset IR

Signed-off-by: alex-yuyue <yue.yu1@huawei.com>

Update Coco & VOC & TFReader, Update clang-format, Reorder
datasets_binding

!69 Add Generator and move c_dataset.Iterator to dataset.Iterator
* Add GeneratorDataset to c_dataset
* Add GeneratorDataset to c_dataset

!67 Moving c_datasets and adding sampler wrapper
* Need to add create() method in datasets.py
* migration from c_dataset to dataset part 1

!71 Fix indent error
* Fix indentation error

!72 Fix c_api tests cases
* Fix c_api tests cases

!73 Added CSV Dataset
* Added CSVDataset

pybind switch: Take and CelebA fixes

!75 move c_dataset functionality to datasets
* Fixed existing testcases
* Added working clue and imagefolder
* Added sampler conversion from pybind
* Added sampler creation

!77 Add Python API tree
* Python API tree

add minddataset

TextFileDataset pybind

Rename to skip test_concat.py and test_minddataset_exception.py

!80 Add batch IR to python-api branch, most test cases work
* staging III
* staging, add pybind

Enable more c_api take and CelebA tests; delete util_c_api

!84 Schema changes in datasets.py
* Schema changes

!85 Remove input_indexes from sub-classes
* remove input_index from each subclass

!83 Remove C datasets
* Removed c_dataset package
* Remove c_datasets

!82  pybind switch: shuffle
* pybind switch: shuffle

!86 Add build_vocab
* Add build_vocab

Rebase with upstream/master
_shuffle conflict
BatchNode error

!88 Fix rebase problem
* fix rebase problem

Enable more unit tests; code typo/nit fixes

!91 Fix python vocag hang
* Fix python vocab hang

!89 Added BucketBatchByLength Pybind switch
* Added BucketBatchByLength

Update and enable more tet_c_api_*.py tests

!95 Add BuildSentencePeiceVocab
* - Add BuildSentencePeiceVocab

!96 Fix more tests
* - Fix some tests

- Enable more test_c_api_*
- Add syncwait

!99 pybind switch for device op
* pybind switch for device op

!93 Add getters to python API
* Add getters to python API

!101 Validate tree, error if graph
* - Add sync wait

!103 TFrecord/Random Datasets schema problem
* - TfRecord/Random schem aproblem

!102 Added filter pybind switch
* Added Filter pybind switch

!104 Fix num_samples
* - TfRecord/Random schem aproblem

!105 Fix to_device hang
* Fix to_device hang

!94 Adds Cache support for CLUE dataset
* Added cache for all dataset ops
* format change
* Added CLUE cache support
* Added Cache conversion

Add save pybind

fix compile err

init modify concat_node

!107 Fix some tests cases
* Fix tests cases

Enable and fix more tests

!109 pybind switch for get dataset size
* pybind_get_dataset_size

some check-code fixes for pylint, cpplint and clang-format

!113 Add callback
* revert
* dataset_sz 1 line
* fix typo
* get callback to work

!114 Make Android compile clean
* Make Android Compile Clean

Fix build issues due to rebase

!115 Fix more tests
* Fix tests cases
* !93 Add getters to python API

fix test_profiling.py

!116 fix get dataset size
* fix get dataset size

!117 GetColumnNames pybind switch
* Added GetColumnNames pybind switch

code-check fixes: clangformat, cppcheck, cpplint, pylint

Delete duplicate test_c_api_*.py files; more lint fixes

!121 Fix cpp tests
* Remove extra call to getNext in cpp tests

!122 Fix Schema with Generator
* Fix Schema with Generator

fix some cases of csv & mindrecord

!124 fix tfrecord get_dataset_size and add some UTs
* fix tfrecord get dataset size and add some ut for get_dataset_size

!125 getter separation
* Getter separation

!126 Fix sampler.GetNumSamples
* Fix sampler.GetNumSampler

!127 Assign runtime getter to each get function
* Assign runtime getter to each get function

Fix compile issues

!128 Match master code
* Match master code

!129 Cleanup DeviceOp/save code
* Cleanup ToDevice/Save code

!130 Add cache fix
* Added cache fix for map and image folder

!132 Fix testing team issues
* Pass queue_name from python to C++
* Add Schema.from_json

!131 Fix Cache op issues and delete de_pipeline
* Roll back C++ change
* Removed de_pipeline and passing all cache tests.
* fixed cache tests

!134 Cleanup datasets.py part1
* Cleanup dataset.py part1

!133 Updated validation for SentencePieceVocab.from_dataset
* Added type_check for column names in SentencePieceVocab.from_dataset

Rebase on master 181120 10:20

fix profiling

temporary solution of catching stauts from Node.Build()

!141 ToDevice Termination
* ToDevice termination

pylint fixes

!137 Fix test team issues and add some corresponding tests
* Fix test team issues and add some corresponding tests

!138 TreeGetter changes to use OptPass
* Getter changes to use OptPass (Zirui)

Rebase fix

!143 Fix cpplint issue
* Fix cpplint issue

pylint fixes in updated testcases

!145 Reset exceptions testcase
* reset exception test to master

!146 Fix Check_Pylint Error
* Fix Check_Pylint Error

!147 fix android
* fix android

!148 ToDevice changes
* Add ToDevice to the iterator List for cleanup at exit

!149 Pylint issue
* Add ToDevice to the iterator List for cleanup at exit

!150 Pylint 2
* Add ToDevice to the iterator List for cleanup at exit

!152 ExecutionTree error
* ET destructor error

!153 in getter_pass, only remove callback, without deleting map op
* getter pass no longer removes map

!156 early __del__ of iterator/to_device
* early __del__ of iterator

!155 Address review comments Eric 1
* Added one liner fix to validators.py
* roll back signature fix
* lint fix
* Eric Address comments 2
* C++ lint fix
* Address comments Eric 1

!158 Review rework for dataset bindings - part 1
* Reorder nodes repeat and rename
* Review rework for dataset bindings - part 1

!154 Fixing minor problems in the comments (datasets.py, python_tree_consumer.cc, iterators_bindings.cc, and iterators.py)
* Fixing minor problems in the comments (datasets.py, python_tree_consum…

!157 add replace none
* Add replace_none to datasets.py, address comments in tests

Trying to resolve copy

Override the deepcopy method of deviceop

Create_ir_tree method

Create_ir_tree method 2

Create_ir_tree method 2

del to_device if already exists

del to_device if already exists

cache getters shapes and types

Added yolov3 relaxation, to be rolled back

Get shapes and types together

bypass yolo

NumWorkers for MapOp

revert Yolo

revert Thor

Print more info

Debug code: Update LOG INFO to LOG ERROR

do not remove epochctrl for getter pass

Remove repeat(1)

pritn batch size

add log to tree_consumer and device_queue op

Revert PR 8744

Signed-off-by: alex-yuyue <yue.yu1@huawei.com>

__del__ toDEvice

__del__ toDevice2

!165 add ifndef ENABLE_ANDROID to device queue print
* Add ifndef ENABLE_ANDROID to device queue print

revert some changes

!166 getter: get_data_info
* getter: get_data_info

!168 add back tree print
* revert info to warnning in one log
* add back the missed print tree log

Release GIL in GetDataInfo
 2020-11-25 15:46:59 -05:00
anancds 7c73bc50f4 fixed codex 2020-11-24 21:04:04 +08:00
zhoufeng d532af3a9a new hccl interface 
Signed-off-by: zhoufeng <zhoufeng54@huawei.com>
 2020-11-24 17:34:35 +08:00
mindspore-ci-bot 0d49650bd5 !8888 Add PyNative Device Profiling 
From: @jojobugfree
Reviewed-by: @kisnwang,@jjfeing
Signed-off-by: @jjfeing
 2020-11-24 10:50:43 +08:00
mindspore-ci-bot ab26dd71dc !8789 Enable value inference in Concat; Support const input in graph_ir convertor. 
From: @c_34
Reviewed-by: @liangchenghui,@guoqi1024
Signed-off-by: @liangchenghui
 2020-11-23 19:54:32 +08:00
caifubi 12643a33dd Add profiling reporter 2020-11-23 16:53:11 +08:00
shenwei41 4132228c10 Add c++ api of randomresizecropwithbbox 2020-11-21 17:06:56 +08:00
anancds 69338af0f5 fixed get post message return type 2020-11-20 09:27:56 +08:00
mindspore-ci-bot 812095de8f !8753 Adds execute testcase and fix get_dataset_size 
From: @ezphlow
Reviewed-by: 
Signed-off-by:
 2020-11-20 02:06:55 +08:00
chenhaozhe b3add83bf0 support const input in graph_ir convertor, add value inference in Concat 2020-11-19 20:34:49 +08:00
Eric 45292abdfd Added execute test case 
Added execute tests for server
 2020-11-18 23:44:46 -05:00
mindspore-ci-bot c8047a35ab !8632 [MD] Add Multiply op for Lite Dataset 
From: @jiangzhiwen8
Reviewed-by: @liucunwei,@heleiwang
Signed-off-by: @liucunwei
 2020-11-19 09:58:43 +08:00
mindspore-ci-bot 215fce192e !8737 [MD] Fix the Pad function return false if bottom value is 0 
From: @jiangzhiwen8
Reviewed-by: @xulei2020,@liucunwei,@heleiwang
Signed-off-by: @liucunwei
 2020-11-19 09:07:25 +08:00
mindspore-ci-bot f352f9a54a !8511 [MD] Decouple more Transform Ops 
From: @luoyang42
Reviewed-by: 
Signed-off-by:
 2020-11-18 23:19:36 +08:00
jiangzhiwen bc91f7b3ac fix pad zero 2020-11-18 16:14:37 +08:00
luoyang d43155b5b0 [MD] Decouple more Transform Ops 2020-11-18 14:50:59 +08:00
mindspore-ci-bot e3b1814401 !8713 Fix for GetDatasetSize issue in TextFile 
From: @mahdirahmanihanzaki
Reviewed-by: @nsyca,@robingrosman
Signed-off-by: @nsyca
 2020-11-18 05:39:51 +08:00
Mahdi 449e1526dc Fixed GetDatasetSize for TextFile 2020-11-17 15:22:58 -05:00
Zirui Wu ff5999fc2f add removal pass for getters 
fix CI round I

fix ci round II

address review cmts

fix ci round II
 2020-11-17 10:49:23 -05:00
Mahdi ecde0f03a2 Updated GetNumSamples logic 2020-11-16 16:03:14 -05:00
jiangzhiwen b30c1045e1 add lite_mat multiply 2020-11-16 15:05:31 +08:00
mindspore-ci-bot 9c957072e2 !8097 Remove redundant phi nodes 
From: @ginfung
Reviewed-by: 
Signed-off-by:
 2020-11-16 14:06:52 +08:00
yujianfeng 3176d377e6 Remove redundant phi nodes 2020-11-16 09:36:55 +08:00
yangzhenzhang 0c2c76d037 update get rank in parallel ops 2020-11-13 16:43:44 +08:00
mindspore-ci-bot edcb0cd86b !8370 [MD] Use neon instruction set to accelerate the Substract and Divide ops 
From: @jiangzhiwen8
Reviewed-by: @xulei2020
Signed-off-by:
 2020-11-13 16:14:00 +08:00
mindspore-ci-bot c54a4a4494 !8540 Added GetColumnNames to C++ 
From: @mahdirahmanihanzaki
Reviewed-by: @nsyca,@jonyguo
Signed-off-by: @nsyca
 2020-11-13 09:33:37 +08:00
Mahdi 5a7515e48f Added GetColumnNames to C++ 2020-11-12 16:36:59 -05:00
Zirui Wu 3555d19b07 fix get num_class in imagefolder 
address cmt

review cmts
 2020-11-12 14:26:54 -05:00
mindspore-ci-bot 7aa7ece5ce !8497 add stage id for pipeline parallel 
From: @yangzhenzhang
Reviewed-by: @stsuteng,@kisnwang
Signed-off-by: @stsuteng
 2020-11-12 17:23:39 +08:00
jiangzhiwen 03bbe5f2a8 neon acceleration for subtract 2020-11-12 16:30:40 +08:00
yangzhenzhang 41d925b68a set stage id 2020-11-12 11:14:31 +08:00
mindspore-ci-bot f60ae8925d !8480 Add RandomNode fix 
From: @ezphlow
Reviewed-by: @pandoublefeng,@heleiwang
Signed-off-by:
 2020-11-12 10:00:03 +08:00
mindspore-ci-bot 510a0399fc !8335 added cluster config and message handler unit test,change namespace from comm to core 
From: @anancds
Reviewed-by: 
Signed-off-by:
 2020-11-12 09:33:29 +08:00
Eric a21eb2d527 Added random node fix 2020-11-11 17:24:48 -05:00
mindspore-ci-bot 0de9d3e5b7 !6989 split sort for parallel run 
From: @kisnwang
Reviewed-by: 
Signed-off-by:
 2020-11-11 18:54:56 +08:00
mindspore-ci-bot 222a0bccf4 !8431 Add more check in unsorted_segment_sum_fission pass to avoid doing wrong optimize which cause runtime error 
From: @irmo
Reviewed-by: 
Signed-off-by:
 2020-11-11 15:44:51 +08:00
anancds 96d8c411e7 added message handler unit test 2020-11-11 14:50:18 +08:00
huanghui 1c6c280da7 fix unsorted_segment_sum_fission pass 2020-11-11 11:25:55 +08:00
mindspore-ci-bot 70f5775711 !8387 Add support for GetClassIndexing in C++ API 
From: @alex-yuyue
Reviewed-by: 
Signed-off-by:
 2020-11-11 03:02:22 +08:00
alex-yuyue d7c3a9ba00 Add GetClassIndexing 
Signed-off-by: alex-yuyue <yue.yu1@huawei.com>
 2020-11-10 11:02:02 -05:00
kswang b88e722e12 add graph partition 2020-11-10 22:01:10 +08:00
mindspore-ci-bot bc12c647a5 !8380 [MD]remove opencv api with lite_cv in lite train 
From: @xulei2020
Reviewed-by: @heleiwang
Signed-off-by:
 2020-11-10 14:41:36 +08:00
xulei2020 f8d1b20873 init litemat from memory 2020-11-10 10:12:39 +08:00
mindspore-ci-bot f8f2d127cd !7999 optimize cpu unique 
From: @kisnwang
Reviewed-by: 
Signed-off-by:
 2020-11-10 09:42:01 +08:00
kswang 618c05d454 optimize cpu unique 2020-11-09 17:05:42 +08:00
mindspore-ci-bot 908a494add !8299 [MD] Skip returning IR when meet nullptr in ConcatDataset 
From: @luoyang42
Reviewed-by: @nsyca,@pandoublefeng,@jonyguo
Signed-off-by: @jonyguo
 2020-11-09 11:49:48 +08:00
mindspore-ci-bot 36e69f6ef9 !8162 gpu support dynamic shape 
From: @wilfchen
Reviewed-by: @limingqi107,@cristoval
Signed-off-by: @cristoval
 2020-11-09 09:28:45 +08:00
mindspore-ci-bot 027916cb5f !8315 Cache server stability 
From: @jkl_lee
Reviewed-by: @mikef,@nsyca
Signed-off-by: @nsyca
 2020-11-08 20:03:23 +08:00
Lixia Chen f24a788eed Rebase up to 5792d50ca7 2020-11-06 18:39:05 -05:00
Zirui Wu 7538f82da9 add callback to map's c api 
fix test err

fix ci round 1

fix ci round 2

add col_order to batch_node

minor fix
 2020-11-06 16:36:47 -05:00
mindspore-ci-bot a9a9c45662 !8232 C++ api add Filter 
From: @tiancixiao
Reviewed-by: 
Signed-off-by:
 2020-11-07 00:32:15 +08:00
mindspore-ci-bot 97e1f6f438 !8248 Add macro and fix album testcase 
Merge pull request !8248 from EricZ/helper_dep_removal
 2020-11-06 20:33:20 +08:00
mindspore-ci-bot 073d991de4 !8106 add cpu adam delta 
Merge pull request !8106 from kisnwang/add-cpu-adam-delta
 2020-11-06 17:30:24 +08:00
Xiao Tianci 4a5e287b92 add filter c api and ut 2020-11-06 15:49:23 +08:00
luoyang dcb758bc77 [MD] Skip returning IR when meet nullptr in ConcatDataset 2020-11-06 15:08:11 +08:00
wilfChen e4e9362bd0 gpu suppor dynamic shape 2020-11-06 14:16:27 +08:00
kswang cbece74a23 add cpu adam delta 2020-11-06 13:52:54 +08:00
Eric 394dbf50e1 Added testing macro and fix album_op test 2020-11-05 13:22:51 -05:00
mindspore-ci-bot 523c3f07e3 !8194 Optimizer on IR #2 - Call ValidateParams in PrePass 
Merge pull request !8194 from nsyca/opt3
 2020-11-05 22:52:37 +08:00
mindspore-ci-bot 099765bbb7 !8135 [MD] Transform api decouple 
Merge pull request !8135 from luoyang/c-api-pyfunc
 2020-11-05 22:41:44 +08:00
Yi Huaijie d7faa77b5e support int64 shape 2020-11-05 21:15:49 +08:00
luoyang 9a57cbacd9 [MD] Transform api decouple 2020-11-05 19:56:15 +08:00
mindspore-ci-bot 8709005a57 !8200 Add two transforms API 
Merge pull request !8200 from shenwei41/new_master
 2020-11-05 11:19:00 +08:00
Nat Sutyanyong a2a2223373 Add optimizer to IR tree #2 2020-11-04 21:44:17 -05:00
mindspore-ci-bot 7123a2c1d1 !7911 added key value message 
Merge pull request !7911 from anancds/kv-patch
 2020-11-05 09:53:00 +08:00
mindspore-ci-bot ae7384ca2d !8208 Removed dependency on core module in util 
Merge pull request !8208 from EricZ/helper_dep_removal
 2020-11-05 02:44:30 +08:00
Eric 85f0a9bc8f Added dependency removal for data_helper 2020-11-04 12:15:32 -05:00
anancds 87edbdf720 added protobuf message 2020-11-04 22:19:04 +08:00
zhoufeng 183742009f mindspore cxx api for 310 inference 
Signed-off-by: zhoufeng <zhoufeng54@huawei.com>
 2020-11-04 13:59:33 +08:00
shenwei41 9ffdf0b6d4 Add transform API 2020-11-04 11:54:33 +08:00
Zirui Wu 5d1ada682c remove uncessary headers 
staging II

clean up

further clean up

fix merge conflict with 8139

fix complie err
 2020-11-03 11:59:55 -05:00
hesham 5169fb4c42 Remove api namespace 2020-11-02 20:09:37 -05:00
mindspore-ci-bot 6e2241d64f !8096 Add DatasetNode as a base Class for IR nodes 
Merge pull request !8096 from h.farahat/datasetNode
 2020-11-03 00:41:42 +08:00
jjfeing 56f1bb1b66 add dump data when task error 2020-11-02 09:42:53 +08:00
hesham 0ba0595caf DatasetNode as a base for all IR nodes 2020-11-01 16:57:10 -05:00
Mahdi 5005b434e6 Fixed repeat issue with GetDatasetSize 2020-10-30 19:36:56 -04:00
mindspore-ci-bot cacebd1211 !7933 Cache performance updates 
Merge pull request !7933 from Jamie/CacheOp_dev
 2020-10-31 04:25:47 +08:00
mindspore-ci-bot 7ef7bef1d8 !7796 [MD] Enable C++ cache api for non-mappable datasets 
Merge pull request !7796 from lixiachen/cache_c_api_nonmappable
 2020-10-30 22:11:13 +08:00
Lixia Chen 572c5e5f29 Rebase up to 542a52fbf8 2020-10-29 20:34:09 -04:00
Cathy Wong 18ecd65f93 C++ API: Delete ValidateParams in dataset APIs; update error input UTs 2020-10-29 16:29:16 -04:00
mindspore-ci-bot 8f44074796 !7866 [MD] C++ API support build_sentence_piece_vocab_node & sentence_piece_tokenizer 
Merge pull request !7866 from luoyang/c-api-pyfunc
 2020-10-29 14:26:13 +08:00
luoyang 52d278e858 [MD] C++ API support build_sentence_piece_vocab_node & sentence_piece_tokenizer 2020-10-29 11:05:28 +08:00
mindspore-ci-bot 2ba20dc8e7 !7848 Add five transforms API 
Merge pull request !7848 from shenwei41/new_master
 2020-10-29 10:18:57 +08:00
Zirui Wu 814044963b add validation check in tfrecord_node 
update ut
 2020-10-28 19:26:51 -04:00
Zirui Wu ea97197311 fix tfrecord_op core dump 
add test case

fix ci

fix ci round 2

address review cmts
 2020-10-28 17:38:43 -04:00
mindspore-ci-bot c39c3ccfbe !7734 Redesigned GetNumClasses 
Merge pull request !7734 from Alex Yuyue/IR_dataset_input
 2020-10-29 01:44:36 +08:00
shenwei41 779f0f387b Add transforms API 2020-10-28 22:06:58 +08:00
Xiao Tianci 823e52f1dc add params checking of offset in distributed_sampler 
add related comment to explain
add related UT
 2020-10-28 16:46:12 +08:00
Lixia Chen 02ddddfe7e Enable caching over non-mappable dataset c++ api 
and caching over Album dataset
 2020-10-27 23:02:13 -04:00
alex-yuyue 913a49f3af Add support for C++ API: GetNumClasses 
Signed-off-by: alex-yuyue <yue.yu1@huawei.com>
 2020-10-27 16:26:51 -04:00
anancds 4b995fe170 fixed Destructor and RegisterRoute 2020-10-26 15:53:31 +08:00
mindspore-ci-bot 3376e0d034 !7704 [MD] Add testcases for cache c api 
Merge pull request !7704 from lixiachen/cache_c_api
 2020-10-26 14:31:09 +08:00
mindspore-ci-bot d81bd7b17b !7710 Add c++ API for GetRepeatCount and GetBatchSize 
Merge pull request !7710 from h.farahat/getBatchRepeat
 2020-10-26 09:25:21 +08:00
mindspore-ci-bot 0124eef185 !7729 [MD] Fix log of vision and dataset in C++ api 
Merge pull request !7729 from luoyang/pylint
 2020-10-26 09:23:22 +08:00
mindspore-ci-bot 7982f05038 !7730 [MD] Fix bugs in WeightedRandomSampler & SubsetRandomSampler 
Merge pull request !7730 from luoyang/c-api-pyfunc
 2020-10-26 09:22:14 +08:00
luoyang f0e976dbda [MD] Fix bugs in WeightedRandomSampler & SubsetRandomSampler 2020-10-25 17:58:51 +08:00
hesham f3880f8355 Add GetBatchSizer and GetRepeatCount to c++ API 2020-10-24 22:45:19 -04:00
YangLuo f3fa5a441d [MD] Fix log of vision and dataset in C++ api 2020-10-24 19:24:30 +08:00
mindspore-ci-bot 3d194137da !7702 Redesigned GetOutputType and GetOutputShape 
Merge pull request !7702 from Alex Yuyue/IR_dataset_input
 2020-10-24 14:42:29 +08:00
mindspore-ci-bot 48edd9708c !7690 [MD] fix num_sample in randomsampler and sequential sampler 
Merge pull request !7690 from liyong126/fix_md_num_samples
 2020-10-24 11:13:19 +08:00
alex-yuyue 2906659673 Add support for GetOutputTypes and GetOutputShapes 
Signed-off-by: alex-yuyue <yue.yu1@huawei.com>
 2020-10-23 22:21:52 -04:00
mindspore-ci-bot fe2852df82 !7703 C++ API: Add a parameter validation and fix incorrect format of error message for TFrecord 
Merge pull request !7703 from TinaMengtingZhang/fix_tfrecord_issue_I1YKU6_I1YKXL
 2020-10-24 07:33:26 +08:00
Lixia Chen bc396febae Add testcases for cache c api 2020-10-23 17:10:18 -04:00
tinazhang e7346dd3a9 add a validation code to fix issue I1YKU6 and corresponded ut, fix format issue I1YKXL 2020-10-23 17:07:51 -04:00
Mahdi 0e03f5b0dd Added GetDatasetSize 2020-10-23 16:29:11 -04:00
Zirui Wu a85a256749 fix complie err in c_api_dataset_test_save 2020-10-23 10:09:35 -04:00
mindspore-ci-bot 3355bb5f2f !7657 Leaf node IR breakdown, port to engine/ir/datasetops/source/ 
Merge pull request !7657 from ZiruiWu/ir_breakdown_leaf_nodes
 2020-10-23 20:32:31 +08:00
liyong ee042b90f7 fix num_sample in sequentialSampler and randomSampler 2020-10-23 19:26:10 +08:00
mindspore-ci-bot 9d35fdc4e5 !7653 [MD] C++ api SaveOp 
Merge pull request !7653 from luoyang/c-api-pyfunc
 2020-10-23 14:14:11 +08:00
Zirui Wu cfc50e231a Added IR breakdown for remaining ops after PR7596 
fix android build failure

fix include in datasets.cc

attempting fix android build

fix complie err x86
 2020-10-22 23:07:41 -04:00
luoyang 8e60cfe800 [MD] C++ api save 2020-10-23 10:27:35 +08:00
mindspore-ci-bot 19b10d474f !7530 [MD] Add divide image proccessing 
Merge pull request !7530 from jiangzhiwen/lite/image_divide
 2020-10-23 09:22:36 +08:00
anancds 98bbb4ef7a added tcp base on libevent 2020-10-22 17:44:29 +08:00
mindspore-ci-bot 8bfe141680 !7571 fix bug in reshape strategy search when reshape as the first operator 
Merge pull request !7571 from yao_yf/reshape_strategy_search_fix_when_reshape_is_first_op
 2020-10-22 10:16:33 +08:00
yao_yf f7189adb91 fix bug in reshape strategy search when reshape is first operator 2020-10-22 09:15:18 +08:00
Zirui Wu d471552fc5 shuffle take repeat bucketbatch buildvocab repeat shuffle take project concat rename node IR added 
concat, bucketbatch project rename

fix ci round 1

fix ci round 2

fix up

fix ci
 2020-10-21 18:41:34 -04:00
mindspore-ci-bot de7ea8e68c !7540 Extract ImageFolderNode and BatchNode into its own files from datasets.cc/.h 
Merge pull request !7540 from ZiruiWu/refactor_ir_demo
 2020-10-21 19:23:20 +08:00
mindspore-ci-bot 566656ea72 !7558 fix the repeated calculation accuracy error 
Merge pull request !7558 from yangzhenzhang/update-repeated-calculation
 2020-10-21 19:14:13 +08:00
mindspore-ci-bot a8dc0f5c1b !7557 [MD] Add merge image proccessing 
Merge pull request !7557 from jiangzhiwen/lite/merge
 2020-10-21 17:28:55 +08:00
yangzhenzhang eb6f4e3ce8 update repeated calculation 2020-10-21 16:13:58 +08:00
jiangzhiwen 8e785648da add merge image processing 2020-10-21 14:44:22 +08:00
Zirui Wu 52eec881ec Refactor ImageFolderNode and BatchNode into its separate files 
fix ci

fix ci

address review cmmts, fix ci

remove dup code in datasets.cc
 2020-10-20 23:07:51 -04:00
mindspore-ci-bot 643366ea70 !7538 Add empty string tensor support for Album 
Merge pull request !7538 from EricZ/album_empty_string
 2020-10-21 08:41:29 +08:00
Eric 7e21648e16 Added Album empty string tensor support 2020-10-20 16:37:37 -04:00
Zirui Wu aa8442e0a3 add skeleton code for tree_adapter 
stage II tree_adapter

add test case for tree_adapter

use dfs to build tree

add more test case

add test case

fix ci

fix ci round 2

fix ci round 3

fix ci round 4

fix complie error

fix ci round 6

fix ci round 7

fix ci round 8

fix ci round 9
 2020-10-20 12:44:28 -04:00
jiangzhiwen fadc97a18f add divide op 2020-10-20 20:53:01 +08:00
mindspore-ci-bot 48601f51b0 !7295 [MD] C++ api add MindDataset 
Merge pull request !7295 from luoyang/c-api-pyfunc
 2020-10-20 19:45:45 +08:00
mindspore-ci-bot 8ff82843ea !7486 [MD] Add subtract image proccessing 
Merge pull request !7486 from jiangzhiwen/lite/image_subtract
 2020-10-20 19:39:11 +08:00
mindspore-ci-bot 947f6d96d1 !7414 C++ api add config_manager 
Merge pull request !7414 from xiaotianci/c_api_config
 2020-10-20 19:37:18 +08:00
mindspore-ci-bot 7707bb8540 !7439 [MD] Add YUV to BGR , Split, extractChannel ops 
Merge pull request !7439 from xulei/lite_test0905
 2020-10-20 19:36:19 +08:00
Xiao Tianci fb98fbcc9f add c api config 2020-10-20 16:32:57 +08:00
luoyang 2dc8e5f421 [MD] C++ api add MindDataset 2020-10-20 16:12:49 +08:00
xulei2020 442cee15ad add yuv420sp to bgr, split and extractchannel 2020-10-20 15:05:25 +08:00
jiangzhiwen 44f7978f7c add image subtract 2020-10-20 10:53:27 +08:00
yangzhenzhang fc4ed975c4 handle repeated calculation 2020-10-19 10:47:01 +08:00
mindspore-ci-bot 70bd405338 !7375 Change normalize input check 
Merge pull request !7375 from EricZ/change_normalize_check
 2020-10-19 09:14:28 +08:00
anancds 28c9e9d9a5 Added http server based on libevent 2020-10-16 14:26:42 +08:00
Eric d3e89086e5 Removed check for normalize mean 2020-10-15 17:03:41 -04:00
mindspore-ci-bot 0357088a73 !7319 C++ API: vision ops - reorder and add gap UTs; correct parm desc style 
Merge pull request !7319 from cathwong/ckw_c_api_transforms_tests2
 2020-10-16 00:31:24 +08:00
mindspore-ci-bot 15b1725198 !7321 Fix ordering for album dataset 
Merge pull request !7321 from EricZ/album_float_array
 2020-10-15 21:57:36 +08:00
Eric 359ad19884 Added float_array support 
Added fix, tests to come
 2020-10-15 00:18:30 -04:00
huanghui b7519b7418 unify save_graphs_path 2020-10-15 10:15:16 +08:00
Cathy Wong 9f7c1a32e7 C++ API: vision ops - reorder and add gap UTs; correct parm desc style 2020-10-14 19:10:13 -04:00
mindspore-ci-bot 7e7d47db50 !7266 RandomResizedCrop for C API 
Merge pull request !7266 from MahdiRahmaniHanzaki/c-api
 2020-10-14 23:00:15 +08:00
shenwei41 87b108f4e0 C++ api add Rescale 2020-10-14 10:32:48 +08:00
Mahdi c8333accfc RandomResizedCrop C API 2020-10-13 17:27:29 -04:00
mindspore-ci-bot 4a077f6341 !7035 Add validation for input image size of affine transformation 
Merge pull request !7035 from jiangzhiwen/fix/check_affine_size
 2020-10-13 15:03:46 +08:00
mindspore-ci-bot 2d645fdc7e !7184 [MD] C++ api add TypeCaseOp & RandomCropDecodeResizeOp 
Merge pull request !7184 from luoyang/pylint
 2020-10-13 09:08:23 +08:00
jiangzhiwen 197f0cabe0 check affine input size 2020-10-12 17:06:31 +08:00
YangLuo 9e033bbe19 C++ api add TypeCaseOp & RandomCropDecodeResizeOp 2020-10-12 15:13:17 +08:00
mindspore-ci-bot c951d42c2c !6728 [Ascend][DynamicShape] Dynamic shape feature 
Merge pull request !6728 from caifubi/dynamic_shape_share_2
 2020-10-12 09:07:39 +08:00
caifubi d3b978147f Ascend Dynamic Shape 2020-10-10 19:11:47 +08:00
jjfeing 755863ebae insert memcpy when hccl node 2020-10-10 17:03:50 +08:00
mindspore-ci-bot d3bd1cb5b8 !6668 [Data Dump] [master] Fix asynchronous data dump failure 
Merge pull request !6668 from caifubi/dump
 2020-10-09 10:48:24 +08:00
mindspore-ci-bot ccb2e8851c !6782 enable parallel optimizer in auto parallel 
Merge pull request !6782 from gziyan/enable_optimizer_shard_in_auto_parallel
 2020-10-09 09:03:40 +08:00
Cathy Wong 2aa26d245c C++ API: Move vision ops to new vision files; create transforms namespace 2020-10-08 11:59:58 -04:00
Ziyan ddc0113058 enable parallel optimizer in auto parallel 2020-10-08 14:51:15 +08:00
mindspore-ci-bot 03f0e64af9 !7049 Fixes to TensorRow conversion and Tensor 
Merge pull request !7049 from MahdiRahmaniHanzaki/c_func
 2020-10-01 04:19:19 +08:00
caifubi b17b4abf7d Fix async data dump failed 2020-09-30 11:39:19 +08:00
Mahdi e92d4edceb Updated TensorRow conversion 2020-09-29 14:50:44 -04:00
jjfeing 7dda95d247 set soc version 2020-09-28 21:04:07 +08:00
mindspore-ci-bot 9bd34a1b29 !6673 Add stage information for ops and strategy 
Merge pull request !6673 from huangxinjing/stage_strategy
 2020-09-27 11:31:07 +08:00
mindspore-ci-bot 0d8015b043 !5300 [MD] Add nD Slice Support 
Merge pull request !5300 from nhussain/slice_op
 2020-09-25 20:38:19 +08:00
huangxinjing 4ef439e27b Add stage information for ops and strategy 2020-09-25 10:42:43 +08:00
mindspore-ci-bot 70221f5261 !6307 Conversion to and from TensorRow in C++ 
Merge pull request !6307 from MahdiRahmaniHanzaki/c_func
 2020-09-25 04:14:05 +08:00
nhussain 77d507279c first working implementation 
add string implementation, can handle partial string slices

finish core implementation, added Index object

fix up the logic

add in new SliceOption object, cleaning up tests
 2020-09-24 15:39:28 -04:00
Lixia Chen 983827ec5c Rebase up to 88ded11f59 2020-09-24 14:09:08 -04:00
Mahdi 890598d112 Added TensorRow conversion 2020-09-24 11:12:22 -04:00
mindspore-ci-bot f847414117 !6471 Fix AdjustAllReduceMulAdd pass 
Merge pull request !6471 from thlinh/dev_Sep18_fix_AdjustAllReduceMulAdd_pass
 2020-09-19 21:48:46 +08:00
yanghaitao1 4ff4c17632 fix get_dataset_size in CelebADataset when usage is not all 2020-09-19 02:16:29 -04:00
Hoai Linh Tran 46f07efc31 Fix AdjustAllReduceMulAdd pass 
Revive tests in lib_test

Code cleaning
 2020-09-19 01:07:01 -04:00
mindspore-ci-bot c8e9d46391 !6450 Change PS directory. 
Merge pull request !6450 from ZPaC/master-change-ps-dir
 2020-09-18 11:42:41 +08:00
ZPaC 0b49f0fb57 change PS dir 2020-09-18 10:48:09 +08:00
mindspore-ci-bot 5d0b91cb22 !6436 [Ascend Profiling] FIx start profiling failed 
Merge pull request !6436 from caifubi/profiling
 2020-09-18 10:02:19 +08:00
caifubi 39039974b2 fix profiling start up fail 2020-09-18 08:56:13 +08:00
luoyang 9064a5f731 add num_row check & fix api distributed sampler, lookup, default value 
of usage
 2020-09-17 20:31:26 +08:00
mindspore-ci-bot af1c5aba5b !6354 [Data][BUG]add SubStractMeanNormalize input param check 
Merge pull request !6354 from xulei/lite_test0905
 2020-09-17 09:06:19 +08:00
xulei2020 85962adffd add code 2020-09-16 17:01:30 +08:00
wuweikang e24f534002 sync-from-trunk-to-blue-zone-0912-c75b220 2020-09-16 11:28:22 +08:00
mindspore-ci-bot bb4dbf4989 !5803 Combine Async Dump with E2e Dump 
Merge pull request !5803 from caifubi/dump
 2020-09-14 11:47:00 +08:00
caifubi 372c2e7951 Combine Async Dump and E2E Dump 2020-09-14 10:57:46 +08:00
luoyang 581335453e Change return val of GetNextRow 2020-09-12 14:19:46 +08:00
mindspore-ci-bot a084196300 !6094 Adding input checks to C API 
Merge pull request !6094 from EricZ/add_input_check_capi
 2020-09-12 09:47:50 +08:00
eric cd5af9472e Added input check 2020-09-11 20:36:12 -04:00
xulei2020 72861a68f3 add code 2020-09-11 19:28:35 +08:00
jiangzhiwen d4ce668b9b add Affine ut 2020-09-10 21:50:56 +08:00
mindspore-ci-bot c882e4863c !5961 [Data]Add lite_cv lib for mindspore lite data preprocessing 
Merge pull request !5961 from xulei/lite_test0905
 2020-09-10 18:51:07 +08:00
mindspore-ci-bot ea94756839 !5605 Introduce usage flag to MNIST and CIFAR dataset 
Merge pull request !5605 from ZiruiWu/add_usage_to_cifar_mnist_coco
 2020-09-10 15:47:48 +08:00
xulei2020 b356211413 add init code 2020-09-10 15:35:08 +08:00
mindspore-ci-bot 7152fe04be !5783 GraphKernel supports GPU 
Merge pull request !5783 from DeshiChen/graph_kernel_1.0
 2020-09-10 09:15:21 +08:00
Zirui Wu 1bb93580dd Add usage flag to MNIST/CIFAR 
ci fix 1

fix ci 2

fix ci 3

fix ci 4

fix ci 5

fix ci 6

fix ci 7

change var name in CelebA from dataset_type to usage

address review cmts

fix cpp ut fail

change mode to usage in VOCDataset

fix cmts

fix ci test case fail
 2020-09-09 09:59:55 -04:00
dayschan 37a48f6aac GraphKernel supports GPU 
1. Update akg submodule
2. Refactor akg_kernel_build, akg_ascend_kernel_build, akg_gpu_kernel_build
3. Add akg_kernel_json_decoder to support converting kernel_json to AnfNode.
4. Add GraphKernel Cost Model. (mindspore/_extends/graph_kernel)
5. Add some GraphKernel passes to GpuSession, move these passes to backend/optimizer/graph_kernel.
6. Add global id for ir files.
7. Fix bug in ConstInputToAttr.
 2020-09-09 17:46:50 +08:00
Zirui Wu bde9f18f5a update lookup api to take in a type 
ci

add test case

address some of the review cmts

address review cmts
 2020-09-08 11:59:28 -04:00
wuxuejian bd527a331d update aicpu proto and update module: graphengine 
Support Dynamic Shape Aicpu Run Package

adapt tensorengin modify, fix ub fusion
 2020-09-08 11:38:44 +08:00
mindspore-ci-bot 5c7fa52468 !5673 [performance]cell init opt performance 
Merge pull request !5673 from kingxian/master
 2020-09-05 17:10:08 +08:00
kingxian 92bebe65a9 cell init performance opt 2020-09-04 19:52:56 +08:00
mindspore-ci-bot d0c7b1ba24 !5731 Manifest Dataset check invalid filename string 
Merge pull request !5731 from jiangzhiwen/fix/invalid_filename_string
 2020-09-04 16:36:29 +08:00
mindspore-ci-bot bc4c5afc1a !5667 add kernel select after optimize pass 
Merge pull request !5667 from zyli2020/code_refactor
 2020-09-04 14:24:44 +08:00
mindspore-ci-bot 2f3684d4fa !4872 Add C++ API support for TFRecordDataset 
Merge pull request !4872 from TinaMengtingZhang/cpp-api-tfrecord-dataset
 2020-09-04 11:53:08 +08:00
lizhenyu c3d6918649 add kernel select after optimize pass 2020-09-04 10:52:51 +08:00
jiangzhiwen 294f7b620a check invalid filename string 2020-09-03 22:09:26 +08:00
mindspore-ci-bot 527d7f0afb !5580 [MD] remove finish in FileReader 
Merge pull request !5580 from liyong126/rmv_filereader_finish
 2020-09-03 21:18:35 +08:00
mindspore-ci-bot b961d47d40 !5642 UTest compiling performance optimization 
Merge pull request !5642 from zhoufeng/ut-compile-performance
 2020-09-03 20:21:13 +08:00
liyong ac39c20f41 del finish in FileReader 2020-09-03 20:05:38 +08:00
mindspore-ci-bot 01da7d2c1a !5713 Align num_samples of CSV with other dataset 
Merge pull request !5713 from jiangzhiwen/fix/csv_num_samples
 2020-09-03 19:52:43 +08:00
jiangzhiwen 1a1a8893a4 Align num_samples of CSV with other dataset 2020-09-03 17:30:39 +08:00
zhoufeng 6d500223b2 utest compiling performance optimization 
Signed-off-by: zhoufeng <zhoufeng54@huawei.com>
 2020-09-03 16:57:19 +08:00
zhousiyi 301aeaaf1d remove gvar source file from compile as ut_tests will use gvar so 2020-09-03 06:49:30 +00:00
tinazhang e430b4056a add tfrecord dataset to cpp api 
fix to support schema=nullptr
 2020-09-02 08:55:47 -04:00
luoyang 8aba39a71b Modify VOCDataset doc & Fix BuildVocab bug 2020-09-02 16:54:50 +08:00
mindspore-ci-bot 5e3e583b47 !5615 Add check for duplicate column to a-cpi & add voc descriptions for class_index 
Merge pull request !5615 from luoyang/c-api-pyfunc
 2020-09-02 11:59:34 +08:00
mindspore-ci-bot 3048240f16 !5508 Add AdamApplyOneWithDecayAssign fusion pass 
Merge pull request !5508 from YuJianfeng/adam_assign
 2020-09-02 11:23:34 +08:00
luoyang ce419404ce Add check for duplicate column for a-cpi 2020-09-02 10:41:52 +08:00
mindspore-ci-bot 0cb6d29f0c !4998 Shared memory pool clean up 
Merge pull request !4998 from JesseKLee/mem_pool
 2020-09-02 02:01:19 +08:00
mindspore-ci-bot dff02cc282 !5188 rename image files in imagefolder 
Merge pull request !5188 from tony_liu2/alt
 2020-09-01 11:16:30 +08:00
mindspore-ci-bot b99149dc37 !4901 Adding data_helper for minddata 
Merge pull request !4901 from EricZ/md_helper
 2020-09-01 11:14:11 +08:00
mindspore-ci-bot 3be136f6f0 !5354 Fix randomdata issue 
Merge pull request !5354 from xiefangqi/fix_randomdata_columnlist
 2020-09-01 11:03:54 +08:00
mahdi 84efa0b96b Added BucketBatchByLength for C API 2020-08-31 17:06:16 -04:00
Eric 65bf3ecc3e Adding data_helper class 2020-08-31 16:29:14 -04:00
Jesse Lee e89b4648f6 Memory pool clean up 2020-08-30 11:28:24 -04:00
fary86 fcbb3e0edc Refactor ms_context implementation 2020-08-30 22:51:40 +08:00
yujianfeng 4b77f6b53c Add AdamApplyOneWithDecayAssign fusion pass 2020-08-29 17:54:15 +08:00
mindspore-ci-bot d81b30e6a0 !5312 make backend/optimizer free of pybind 
Merge pull request !5312 from xychow/remove-backend-py-dependency-2
 2020-08-29 14:42:33 +08:00
kswang 756bb6d53f async run graph 2020-08-28 18:18:47 +08:00
zhousiyi c25e37e7bf make backend/optimizer pybind free 2020-08-28 08:56:58 +00:00
xiefangqi e2085a9505 fix randomdata issue 2020-08-28 11:55:49 +08:00
tony_liu2 5e4c314130 rename dataset dir and image files 
change references to use path original

revert folder name

revert graphengine
 2020-08-27 16:49:23 -04:00
mindspore-ci-bot 716329df6c !5278 Change default value of sampler in c-api 
Merge pull request !5278 from luoyang/c-api
 2020-08-27 21:33:18 +08:00
YangLuo 419478b410 Change default value of sampler to seperate behaviour of default sampler 
& null sampler, add check for duplicate column name
 2020-08-27 21:24:37 +08:00
mindspore-ci-bot a8317acee8 !5111 Support int64 for cpu sparse optimizers 
Merge pull request !5111 from YuJianfeng/int64
 2020-08-27 17:17:05 +08:00
mindspore-ci-bot e94416be0c !5283 Support setting operator io format in the frontend 
Merge pull request !5283 from liangchenghui/io_format
 2020-08-27 15:48:13 +08:00
yujianfeng e949540062 Support int64 for cpu sparse optimizers 2020-08-27 10:57:53 +08:00
mindspore-ci-bot d1b95ed9bd !5240 Fix remove internal output for unique device target 
Merge pull request !5240 from YuJianfeng/internal_output
 2020-08-27 10:35:37 +08:00
islam_amin 94a4a3f20b changing affine and posterize apis 2020-08-26 10:51:54 -04:00
梁成辉 34d433fd9a Set io format for old backend 2020-08-26 20:49:15 +08:00
yujianfeng e688e1df32 Fix remove internal output for unique device target 2020-08-26 17:31:35 +08:00
Cathy Wong 46e9d79a50 C++ API: Lexicographical order support for CLUE, CSV & TextFile Datasets 2020-08-25 14:11:19 -04:00
luoyang b26f6b3d5e fix bug in c-api: rename, concat, take 2020-08-25 16:25:00 +08:00
mindspore-ci-bot 0bbce9367e !4772 Adding AlbumDataset for device training 
Merge pull request !4772 from EricZ/ms-album
 2020-08-24 08:27:14 +08:00
Eric c79db93c48 Initial commit for album 
Added linter fix for album dataset

Added testDataset

Adding signature

Added JsonDataset example API

Example dataset

Resolving format

More fixing

Refactor

Small fix

Added compiling album dataset

Running tests

Added linter fix #1

Passing UT

Added dataset API

Addressing clang

Clang part 2

Fixing pass

Fixed tree check

lint fix

Added lint fix part 2
 2020-08-22 11:07:10 -04:00
mindspore-ci-bot 9b503e4f38 !4955 Fixes for Dynamic Augmentation Ops 
Merge pull request !4955 from MahdiRahmaniHanzaki/dynamic-ops-fix
 2020-08-22 10:27:50 +08:00
Mahdi a5f9b8f92e Added fix for MixUpBatch and CutMixBatch and for RandomAffine 
updated c color op descriptions
 2020-08-21 17:40:08 -04:00
mindspore-ci-bot 9d7250c483 !4776 Introduce 2 extra ctrl flags to DataBuffer in dataset, address remaining cmts to PR4632 
Merge pull request !4776 from ZiruiWu/map_callback_follow_up
 2020-08-22 03:31:31 +08:00
Zirui Wu 74c1e6da60 introducing pause and quit flags to DataBuffer 
fix review cmts

fix ci

fix Ci

fixci

address ci

ci

- add timeout
- add more test cases

fix CI

address review cmts
 2020-08-21 14:33:26 -04:00
mindspore-ci-bot b2a5440548 !4883 Fix compilation warnings 
Merge pull request !4883 from zhoufeng/xiu-ba-ge
 2020-08-21 12:33:20 +08:00
zhoufeng 22ec976ef7 fix compilation warnings 
Signed-off-by: zhoufeng <zhoufeng54@huawei.com>
 2020-08-21 09:44:04 +08:00
mindspore-ci-bot 9a3baf4f6c !4314 [MD] remove defaults used in binding code, and remove bilinear resize op 
Merge pull request !4314 from nhussain/clean_up_bindings
 2020-08-20 22:16:56 +08:00
mindspore-ci-bot bfa3cd900e !4843 remove ccsrc/common.h by explicit dependent header file 
Merge pull request !4843 from xychow/remove-ccsrc-common-h
 2020-08-20 22:02:13 +08:00
zhousiyi d0e58dd765 remove ccsrc/common.h 
replace frontend/operator/ops.h in backend with base/core_ops.h as
backend should not use any frontend-only primitive
 2020-08-20 09:34:07 +00:00
xiefangqi 7dc7990d75 add NumWorkers validate 2020-08-20 17:25:30 +08:00
mindspore-ci-bot 7b0a4e5929 !4796 Fix randomdata ut core dump problem 
Merge pull request !4796 from xiefangqi/fix_randomdata_ut
 2020-08-20 14:11:55 +08:00
xiefangqi c0d18d2517 fix randomdata ut problem 2020-08-20 11:13:02 +08:00
Jesse Lee 8a08d0c37b Phase 2 of CacheOp 2020-08-19 13:45:55 -04:00
Zirui Wu 78c1aa1d96 Implemented Callback for Dataset 
implment pause in MapOp, added more to callback

add ds_callback

- Initial drop of Python DSCallback

- Pybind DSCallback

- Pybind DSCallback

added callback to mapOp

- de_pipeline DSCallback

- de_pipeline DSCallback

add test case, segfault for now

fix seg fault

- de_pipeline DSCallback

remove 1 line

update callback test case, now works

use builder class for mapOp callback

- de_pipeline DSCallback

- de_pipeline DSCallback

- de_pipeline DSCallback

better test case

minor fix

add comments and minor clean ups

get rid of nullptr in MapOp, use other flag instead

fix a bug ParseMapOp only takes 1 callback

- Added WaitedDSCalabck

refactor callback param

fix text case incorrect number

- added testing

fix cpp test case

- added testing

- revert back lenet changes

- cleanup test_callbacks.py

- cleanup test_callbacks.py

fix CI stage I

fix CI stage II

fix CI and update epoch counter

- add validation
- add more testing  test_callbacks.py

use random data op to do tests

adjust when to call EpochBegin/End

- add repeat with callback

- addressing reviewers' comments

- docstring and CI fixes

- docstring and CI fixes

- docstring and CI fixes

- rebase with upstream/master

fix cpp test case

fix review comments

addr review cmts, add test case
 2020-08-19 12:31:04 -04:00
YangLuo b50ae27c72 !4597 [Dataset] C++ API Support for build_vocab 2020-08-19 22:47:47 +08:00
nhussain eb9a611041 remove old defaults 2020-08-19 10:11:19 -04:00
mindspore-ci-bot d541e261a0 !4696 C++ API Support for CSV Dataset 
Merge pull request !4696 from jiangzhiwen/jzw/c_api_csv
 2020-08-19 17:08:33 +08:00
mindspore-ci-bot 2db4bda219 !4622 serving restful init 
Merge pull request !4622 from hexia/http_init
 2020-08-19 16:15:58 +08:00
mindspore-ci-bot 256dccc651 !4498 Gnn data processing supports distributed scenarios 
Merge pull request !4498 from heleiwang/gnn_distributed
 2020-08-19 16:09:05 +08:00
jiangzhiwen 39df25f803 c++ api for CSV Dataset 2020-08-19 16:02:11 +08:00
heleiwang 8ee4d8e92d Gnn data processing supports distributed scenarios 2020-08-19 11:09:01 +08:00
mindspore-ci-bot 6868b9b6fa !4422 Minddata new dataset api ==> RandomData 
Merge pull request !4422 from xiefangqi/xfq_c++api_randomdata
 2020-08-19 10:48:53 +08:00
xiefangqi b91e56375e add randomdataset and schema 2020-08-19 09:13:43 +08:00
nhussain 738ae2c78d change api to use std_vector 
try to display image
 2020-08-18 15:41:21 -04:00
islam_amin 979111d0ef Added Posterize Op 2020-08-18 13:55:47 -04:00
hexia 75d116b5db serving http init 2020-08-18 20:15:47 +08:00
mindspore-ci-bot 4f46c4277a !4614 Code compile performance optimization 
Merge pull request !4614 from zhoufeng/code-compile-performance
 2020-08-18 16:45:15 +08:00
mindspore-ci-bot cc077de9e3 !4425 fix if statement with no return coredump 
Merge pull request !4425 from fary86/fix_if_no_return_coredump
 2020-08-18 16:09:31 +08:00
zhoufeng 663278112f optimize code compile performance 
Signed-off-by: zhoufeng <zhoufeng54@huawei.com>
 2020-08-18 14:33:24 +08:00
mindspore-ci-bot 26de4540ef !4639 rename param_value to param_info 
Merge pull request !4639 from vlne-v1/rename_param_value
 2020-08-18 14:16:23 +08:00
fary86 38083e055a Fix coredump missing return statement after while loop 2020-08-18 14:16:21 +08:00
Wei Luning c1c30a44f1 rename param_value -> param_info 2020-08-18 11:16:28 +08:00
jiangzhiwen a88273bd3f c++ api support for Manifest Dataset 2020-08-18 09:48:55 +08:00
mindspore-ci-bot 113ff6ca24 !4367 CutMixBatch Augmentation Op 
Merge pull request !4367 from MahdiRahmaniHanzaki/cutmix
 2020-08-18 03:27:33 +08:00
mahdi 3ecc53fb4e Added CutMix 2020-08-17 13:01:47 -04:00
mindspore-ci-bot dda4c26e7c !4423 add reshape type of tensor 
Merge pull request !4423 from lianliguang/add-reshape-type-to-tensor
 2020-08-17 11:34:11 +08:00
WilliamLian 6760d9976d add reshape type to tensor 2020-08-17 09:34:38 +08:00
mindspore-ci-bot f875bf21bc !2948 fix control flow 
Merge pull request !2948 from amongo/FixControlFlow
 2020-08-17 08:56:22 +08:00
huangdongrun 2a6d346d2f support if by if grad parameter 
add join for ref

adjust env eliminate to eliminate all env ops

add partial app cache

resolve while endless

fix env eliminate

support for "for while" cases

fix join shape error
 2020-08-15 09:47:10 +08:00
Cathy Wong 9c8af0d1cf C++ API: Provide validate param functions 2020-08-14 17:54:38 -04:00
mindspore-ci-bot 7d70fb4dc4 !3769 add RandomColorOp 
Merge pull request !3769 from Alexey_Shevlyakov/ColorOp
 2020-08-14 23:43:08 +08:00
mindspore-ci-bot ebdea65f70 !4402 [Dataset] Fix c_api bugs 
Merge pull request !4402 from luoyang/c-api-pyfunc
 2020-08-14 17:40:32 +08:00
mindspore-ci-bot e9629f95e1 !4418 Add UniqueWithPad cpu kernel 
Merge pull request !4418 from huanghui/unique-with-pad-cpu-kernel
 2020-08-14 09:46:22 +08:00
Alexey Shevlyakov 8526d5414d RandomColor 2020-08-13 16:10:21 -04:00
mindspore-ci-bot 2953720169 !4322 Adding RGBA color conversion 
Merge pull request !4322 from EricZ/bitmap_support
 2020-08-14 01:58:33 +08:00
mindspore-ci-bot 7a10e07fc6 !4383 Add distributedSampler to concatOP 
Merge pull request !4383 from genglishuai/genglshOK
 2020-08-14 00:06:06 +08:00
eric 6a41b44724 Added wrapper around color change function 2020-08-13 11:39:56 -04:00
genglishuai 4d4df68d1e add DistributedSampler for Concat op 2020-08-13 22:52:02 +08:00
luoyang 373adaa553 !4402 [Dataset] Fix c_api bugs 2020-08-13 22:38:55 +08:00
mindspore-ci-bot ed1244558c !4366 Fixing param type in c++ api and fixing python validator for Repeat Op 
Merge pull request !4366 from TinaMengtingZhang/cpp-api-repeat-count
 2020-08-13 22:03:39 +08:00
huanghui 8f70af7300 add unique_with_pad cpu kernel 2020-08-13 21:55:00 +08:00
mindspore-ci-bot 03210ef166 !2557 Add UnsortedSegmentSum fission pass 
Merge pull request !2557 from huanghui/unsorted-segment-sum-fission-pass
 2020-08-13 08:57:01 +08:00
mindspore-ci-bot 265064733b !4340 move cc file that end with py to pybind 
Merge pull request !4340 from liubuyu/master
 2020-08-13 08:56:01 +08:00
tinazhang 0868720eb7 fix parameter type for repeat op in c++ api and added c++/python ut. 2020-08-12 17:23:42 -04:00
tony_liu2 deb1146474 reorganize headers and tests 
remove headers

fix iterator file

fix pr comments

fix new file
 2020-08-12 14:36:01 -04:00
huanghui b8d7f6d77f add UnsortedSegmentSum fission pass 2020-08-12 19:17:19 +08:00
liubuyu 45b6600001 mv cc file that end with py to pybind 2020-08-12 14:50:18 +08:00
mindspore-ci-bot 5adba834d0 !3819 C++ API Support for CLUE Dataset 
Merge pull request !3819 from jiangzhiwen/jzw/c_api_clue
 2020-08-12 14:21:19 +08:00
jiangzhiwen 5842a0212f c++ api support for CLUE Dataset 2020-08-12 09:55:43 +08:00
mindspore-ci-bot e509f87f7f !4219 Add Unique CPU kernel 
Merge pull request !4219 from huanghui/add-op-unique
 2020-08-12 09:39:14 +08:00
mindspore-ci-bot 9276a9d2fb !3771 C++ API Support for TextFile Dataset 
Merge pull request !3771 from cathwong/ckw_c_api_textfile_nosampler
 2020-08-12 04:32:05 +08:00
avakh 477528de7f random sharpness cpp op support 2020-08-11 15:30:40 -04:00
nhussain 7766efd58e add files 2020-08-11 11:54:14 -04:00
Cathy Wong 7f39b5cfd7 C++ API Support for TextFile Dataset and Unit Tests 2020-08-11 11:53:53 -04:00
huanghui fa081092c8 add Unique cpu kernel 2020-08-11 21:36:55 +08:00
mindspore-ci-bot c6acafcbc6 !4268 fix glog level dont work 
Merge pull request !4268 from liubuyu/master
 2020-08-11 20:05:03 +08:00
liubuyu 25a67602b7 bug fix for glog level don't work 2020-08-11 16:44:50 +08:00
WilliamLian 1f5441d73a remove inferfunction to core 2020-08-11 14:22:29 +08:00
mindspore-ci-bot 0115876363 !4197 Add AdamApplyOneAssign fusion pass 
Merge pull request !4197 from YuJianfeng/adam_assign
 2020-08-11 11:26:00 +08:00
mindspore-ci-bot 266df56570 !4220 [MD] add num_samples support in minddataset pk sampler 
Merge pull request !4220 from liyong126/minddataset_pk_num_samples
 2020-08-11 11:22:28 +08:00
mindspore-ci-bot f21ee64fd0 !4247 Add ChannelSwap operation 
Merge pull request !4247 from 章一智/channel_swap_op
 2020-08-11 09:59:55 +08:00
mindspore-ci-bot 2cab58fdbc !3850 Implementing RandomAffine C Op and a CPP UT for AutoContrast & Equalize 
Merge pull request !3850 from islam_amin/randomaffine_op
 2020-08-11 09:29:09 +08:00
mindspore-ci-bot 0bb39830ae !4250 fix cpp api iterator build function 
Merge pull request !4250 from anthonyaje/fix_cpp_api_iterator
 2020-08-11 07:21:27 +08:00
anthonyaje 9774505ecd Fixed dataset cpp api iterator build function 2020-08-10 18:35:59 -04:00
islam_amin e52d85ba9d Added RandomAffine Op 2020-08-10 17:52:50 -04:00
mindspore-ci-bot 612f533d15 !3917 Add c++ ops CreateTupleIterator & Code changes in VOC DatasetOp 
Merge pull request !3917 from luoyang/c-api
 2020-08-11 05:50:21 +08:00
ervinzhang 3a1d8b5f39 added channel swap operation in transforms.h 2020-08-10 17:32:50 -04:00
mindspore-ci-bot 9ad82f79fd !3979 Adding MixUp operation 
Merge pull request !3979 from MahdiRahmaniHanzaki/mixup
 2020-08-10 23:01:26 +08:00
liyong 7341421d3b fix num samples in pk sampler 2020-08-10 19:05:47 +08:00
yujianfeng 8a77751988 Add AdamApplyOneAssign and AdamApplyOneWithDecayAssign fusion pass 2020-08-10 14:46:37 +08:00
YangLuo e6a12946c8 Updata voc build() & add C++ Op CreateTupleIterator 2020-08-10 11:53:58 +08:00
liubuyu 195f04e50e compile the core directory into a separate static library 2020-08-08 21:31:41 +08:00
Mahdi 1896950ae5 Added Mixup 2020-08-07 17:47:16 -04:00
tony_liu2 3a020fe877 split c_api_test 
split into datasets

review fix

fix merge issues

fixed minddata lite build issues
 2020-08-07 11:57:49 -04:00
mindspore-ci-bot a395907182 !3944 Minddata new dataset api ==> CelebA 
Merge pull request !3944 from xiefangqi/md_new_api_CelebA
 2020-08-07 14:26:54 +08:00
panyifeng 34e50e5d6e fix cell bprop 2020-08-07 10:57:01 +08:00
xiefangqi 6276c0f22f new c++ api ==> CelebA 2020-08-07 09:23:20 +08:00
He Wei d310891eca Fix a bug in Tensor::equals() 
1. Check if data_ is null before compare it;
2. Always allocate an array for data_ even if size is zero.
 2020-08-06 21:06:27 +08:00
mindspore-ci-bot fe7141e9e0 !4019 add tensor layout in strategy checkpoint 
Merge pull request !4019 from yao_yf/add_layout_in_stra_ckpt
 2020-08-06 19:00:11 +08:00
yao_yf 60a9fb0001 add_tensor_layout_in_stra_ckpt 2020-08-06 15:57:38 +08:00
kingfo 28dabf0332 fix grad flag update issue in pynative 2020-08-06 15:12:31 +08:00
mindspore-ci-bot 7ec0b5857a !3977 [MD] Refactor Concatenate Op 
Merge pull request !3977 from nhussain/multi_dim_concat_2
 2020-08-06 02:53:06 +08:00
ervinzhang c65670b1a7 changed sampler parameter to support std::move 2020-08-05 13:56:52 -04:00
mindspore-ci-bot 6aa65da5d2 !3860 Minddata new dataset api ==> Coco 
Merge pull request !3860 from xiefangqi/md_new_api_cocodataset
 2020-08-06 00:34:28 +08:00
nhussain 61769b2dd9 fix concat 2020-08-05 11:28:56 -04:00
mindspore-ci-bot b045f47428 !3983 Add ReduceMin fission pass 
Merge pull request !3983 from huanghui/reduce-min-fission-pass
 2020-08-05 19:46:00 +08:00
mindspore-ci-bot 52689a7dcf !3938 decoupling core and context 
Merge pull request !3938 from liubuyu/master
 2020-08-05 14:16:37 +08:00
mindspore-ci-bot 98c415a1d5 !3807 serving support acl dvpp process 
Merge pull request !3807 from 徐永飞/master
 2020-08-05 11:00:21 +08:00
xiefangqi bb7bda1d6b new dataset api -- Coco 2020-08-05 10:54:37 +08:00
huanghui 30000fdb52 add ReduceMin fission pass 2020-08-05 10:46:01 +08:00
liubuyu d81862a916 decoupling core and context 2020-08-05 08:15:14 +08:00
xuyongfei c0b8df4f7c serving add acl dvpp support, ut tests, return input status string message 2020-08-04 21:40:21 +08:00