He Wei
41dcac9c49
Replace std::unordered_map/set with robin-hood-hashing
...
Robin-hood-hashing (https://github.com/martinus/robin-hood-hashing )
is considered faster then std::unordered_map/set,
so we use it to improve mindspore performance.
1. robin_hood head file in `third_party/robin_hood/include`;
2. In `utils/hash_map.h` and `utils/hash_set.h`, we define:
- mindspore::HashMap as an alias of robin_hood::unordered_map;
- mindspore::HashSet as an alias of robin_hood::unordered_set;
3. Replace:
- `#include <unordered_map>` --> `#include "utils/hash_map.h"`;
- `#include <unordered_set>` --> `#include "utils/hash_set.h"`;
- `std::unordered_map` --> `mindspore::HashMap`;
- `std::unordered_set` --> `mindspore::HashSet`;
- `map.insert(std::pair(key, value))` --> `map.emplace(key, value)`;
- `[] (const std::pair<K, V> &p) {..} ` --> `[] (const auto &p) {..} `;
4. Fix issues found by switch to robin_hood:
- AnfNodeConfig hash and equal;
- Fix a bug in `Slice::operator==()`;
- Fix a bug in `CNode::HasPrimalAttr()`;
- Fix map.erase() usage bugs: `map.erase(iter++)` --> `iter = map.erase(iter)`;
- Fix some iterator invalidated problem;
5. Some std::unordered_map/set can not replace by robin_hood:
- As parameter of functions that exposed to python by pybind11;
- Use bad hash that cause robin_hood::map over_flow, such as AbstractBasePtrListHasher;
6. Update cpp unit tests;
7. Add build option '-F' to enable robin_hood, default on.
2021-11-24 10:47:40 +08:00
lizhenyu
a736171895
add file storage module
2021-11-24 10:43:05 +08:00
f016fe0775
unified runtime support the empty kernel graph in control flow
2021-11-24 10:37:22 +08:00
i-robot
354b7d44df
!26703 fix converter input shape
...
Merge pull request !26703 from yeyunpeng2020/converter
2021-11-24 02:33:11 +00:00
i-robot
4deba2a463
!26685 [Numpy-Native] fix searchsorted
...
Merge pull request !26685 from huangmengxi/fix_numpy
2021-11-24 02:23:04 +00:00
i-robot
a17849b669
!26567 fix allreduce notify bug
...
Merge pull request !26567 from hwjiaorui/fix-stream-label
2021-11-24 02:07:51 +00:00
i-robot
5b4dfee77c
!26704 Fix Embedding APi
...
Merge pull request !26704 from huangxinjing/code_docs_fix_embedding
2021-11-24 01:59:15 +00:00
i-robot
87cbffc456
!26641 Change the parallel strategy data to JSON format
...
Merge pull request !26641 from ougongchang/trans_json
2021-11-24 01:49:50 +00:00
i-robot
9a5fd32bd2
!26280 fix mindir export's error when using _encrypt
...
Merge pull request !26280 from lianliguang/master
2021-11-24 01:48:32 +00:00
86c5ad20c8
parallel_ut_refactoring1
2021-11-24 09:41:52 +08:00
i-robot
d996ad5e1e
!26642 [MS][LITE][develop] add c api
...
Merge pull request !26642 from sunsuodong/add_c_api_master
2021-11-24 01:33:22 +00:00
i-robot
3d0b785241
!26680 Decouple GraphKernelCluster from ME backend
...
Merge pull request !26680 from DeshiChen/1122_cluster
2021-11-24 01:32:50 +00:00
55efaabe2a
Fix api doc error
2021-11-24 09:24:40 +08:00
i-robot
9ce3acff4d
!26693 Calibrator support setting symmetry
...
Merge pull request !26693 from yeyunpeng2020/quant_bak
2021-11-24 01:18:17 +00:00
i-robot
08982f4888
!26648 add more log to locate op compile failed reason
...
Merge pull request !26648 from liubuyu/SB
2021-11-24 01:15:56 +00:00
i-robot
b1d878ca6b
!26648 add more log to locate op compile failed reason
...
Merge pull request !26648 from liubuyu/SB
2021-11-24 01:15:48 +00:00
yeyunpeng2020
dc44f76ac2
fix converter input shape
2021-11-24 09:04:19 +08:00
i-robot
c4a48ee77b
!26669 Fix MD Profiler connector size order
...
Merge pull request !26669 from Mohammad Motallebi/fix_profiler_save_sampling
2021-11-24 00:37:48 +00:00
mohammad
d6c50e0f26
fix connector size order + autotune thread + minor changes
2021-11-23 11:46:48 -05:00
i-robot
3fc995a6ae
!26664 Add support float64 as input type for ReduceProd GPU op.
...
Merge pull request !26664 from hezhenhao1/add_prod
2021-11-23 15:06:05 +00:00
i-robot
46e53a51c9
!26570 [Auto-par][d-rec] Change Onehot OP type to increase partitioning quality
...
Merge pull request !26570 from petitquentin/Gather_version_update
2021-11-23 14:31:56 +00:00
luon
5485ff0cb6
[feat] [assistant] [I40GGH] add new ascend operator Cummin
2021-11-23 21:57:23 +08:00
c493f19b6b
add warnings for set_dump API, see I4AUIR for details
2021-11-23 21:55:36 +08:00
ougongchang
c01dbf25c9
Change the parallel strategt data to JSON format
...
Because Protobuf parses the pb file data slowly, in a matter of seconds, it is changed to JSON data
2021-11-23 21:42:04 +08:00
i-robot
875f35d6d8
!26541 Fix file name and field type changes generated by HCCL in profiler.
...
Merge pull request !26541 from casgj/master_hccl
2021-11-23 13:16:28 +00:00
1615edc359
add c api
2021-11-23 05:11:45 -08:00
i-robot
df25ee8c68
!26675 Optimize oom log
...
Merge pull request !26675 from tanghuikang/oom_nullptr_log
2021-11-23 13:07:49 +00:00
i-robot
20be757f18
!26430 replace short-circuit eval with deferred evaluation of backward prop function.
...
Merge pull request !26430 from xychow/replace-shortcurit-eval-with-lazy-eval
2021-11-23 12:53:23 +00:00
47dbd2dd9c
add more log to locate compile failed reason
2021-11-23 20:44:02 +08:00
yeyunpeng2020
dd1f9eec30
Calibrator support setting symmetry
2021-11-23 20:05:58 +08:00
i-robot
77bf581af5
!26652 [MS][LITE]Fix glog
...
Merge pull request !26652 from gongdaguo/fix_glog
2021-11-23 12:01:25 +00:00
e0cfc0d833
modify api comments for parallel
2021-11-23 19:36:26 +08:00
i-robot
249fcbf812
!26614 Add api comments of chinese
...
Merge pull request !26614 from wangnan39/code_docs_api_chinese
2021-11-23 10:17:08 +00:00
huangmengxi
c27613e23d
fix
2021-11-23 18:08:57 +08:00
2038295a25
Decouple GraphKernelCluster from ME backend
...
Changed the callback function GetProcessorFromContext to GetTargetFromContext,
so that we can use it to filter the clusterable op list, added a GetProcessorByTarget into AkgKernelJsonGenerator.
Moved the function IsKeepBasicNode, GetValidOps, OpListFilter from graph_kernel_helper
to graph_kernel_utils. combined the GetValidOps and OpListFilter.
Decoupled the pass getitem_tuple from "optimizer/common/helper.h", by deleting the checking
of input size. cnode->input(i) also checks the input index.
2021-11-23 17:51:38 +08:00
bfa57ebd34
update dataset chinese comment
2021-11-23 17:11:19 +08:00
i-robot
6e886038bb
move GetUnsortedSegmentOpScalarArg to inner
2021-11-23 16:48:19 +08:00
buxue
89a688f3be
fix reduce ops axis multiple bug in GPU
2021-11-23 16:39:41 +08:00
977af36379
optim loss_scale datasethelper chinese api
2021-11-23 16:34:26 +08:00
i-robot
511441a27e
!26515 Refactor FuseNodesToSubGraph and decouple it from graph_kernel_helper
...
Merge pull request !26515 from DeshiChen/1111_fusenodes
2021-11-23 07:58:56 +00:00
i-robot
3489a615a5
!26650 cublas status to string
...
Merge pull request !26650 from VectorSL/cublas-log
2021-11-23 07:37:37 +00:00
tanghuikang
93e113e6e3
Optimize oom log
2021-11-23 15:22:10 +08:00
i-robot
15cc4673a0
!25557 Add ASGD and Rprop optimizer to Mindspore
...
Merge pull request !25557 from wanyiming/optimizer
2021-11-23 07:17:11 +00:00
i-robot
7f7655beef
!25791 Add new initializer
...
Merge pull request !25791 from wanyiming/initilizer
2021-11-23 07:14:19 +00:00
i-robot
e40f3ec063
!25791 Add new initializer
...
Merge pull request !25791 from wanyiming/initilizer
2021-11-23 07:14:03 +00:00
i-robot
69af0e93c0
!26651 add neighborlistupdate args
...
Merge pull request !26651 from jiahongQian/code_docs_master
2021-11-23 06:37:32 +00:00
i-robot
2e7c2ef782
!26653 [MSLITE][DEVELOP] add onnx hard sigmoid parser
...
Merge pull request !26653 from yangruoqi713/master
2021-11-23 06:30:33 +00:00
i-robot
4dcb007d4e
!26657 abstract QuantStrategy && optimize full quant prepare
...
Merge pull request !26657 from yeyunpeng2020/quant_bak
2021-11-23 06:24:32 +00:00
i-robot
8d38efb2c4
!26657 abstract QuantStrategy && optimize full quant prepare
...
Merge pull request !26657 from yeyunpeng2020/quant_bak
2021-11-23 06:24:27 +00:00
dingpeifei
407aa9a06f
md5 master 1123
2021-11-23 14:22:39 +08:00
He Wei
lizhenyu
limingqi107
i-robot
yeyunpeng2020
mohammad
luon
ougongchang
huangmengxi
buxue
tanghuikang
dingpeifei