p60975123
/
mindspore
forked from mindspore-Ecosystem/mindspore
1
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity

Use cmake to add robin-hood-hashing lib

This commit is contained in:
He Wei 2022-08-26 17:41:48 +08:00
parent f31a71cd9e
commit b3b148b6da
10 changed files with 22 additions and 2627 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

11
cmake/external_libs/robin.cmake Normal file
View File

@ -0,0 +1,11 @@
set(REQ_URL "https://github.com/martinus/robin-hood-hashing/archive/3.11.5.zip")
set(MD5 "35154dc71e47762d9b56b2114bc906ca")
set(INCLUDE "./src")
mindspore_add_pkg(robin_hood_hashing
VER 3.11.5
HEAD_ONLY ${INCLUDE}
URL ${REQ_URL}
MD5 ${MD5})
include_directories(${robin_hood_hashing_INC})

1
cmake/mind_expression.cmake
View File

@ -15,6 +15,7 @@ endif()
set(TOP_DIR ${CMAKE_SOURCE_DIR})
include(cmake/utils.cmake)
include(${CMAKE_SOURCE_DIR}/cmake/external_libs/robin.cmake)
include(${CMAKE_SOURCE_DIR}/cmake/external_libs/eigen.cmake)
include(${CMAKE_SOURCE_DIR}/cmake/external_libs/json.cmake)
include(${CMAKE_SOURCE_DIR}/cmake/dependency_securec.cmake)

4
mindspore/core/utils/hash_map.h
View File

@ -18,8 +18,8 @@
#define MINDSPORE_CORE_UTILS_HASH_MAP_H_
#include <functional>
#if (ENABLE_FAST_HASH_TABLE) && __has_include("robin_hood/robin_hood.h")
#include "robin_hood/robin_hood.h"
#if (ENABLE_FAST_HASH_TABLE) && __has_include("include/robin_hood.h")
#include "include/robin_hood.h"
namespace mindspore {
template <typename K, typename V, typename Hash = robin_hood::hash<K>, typename KeyEqual = std::equal_to<K>>

4
mindspore/core/utils/hash_set.h
View File

@ -18,8 +18,8 @@
#define MINDSPORE_CORE_UTILS_HASH_SET_H_
#include <functional>
#if (ENABLE_FAST_HASH_TABLE) && __has_include("robin_hood/robin_hood.h")
#include "robin_hood/robin_hood.h"
#if (ENABLE_FAST_HASH_TABLE) && __has_include("include/robin_hood.h")
#include "include/robin_hood.h"
namespace mindspore {
template <typename T, typename Hash = robin_hood::hash<T>, typename Equal = std::equal_to<T>>

2
mindspore/lite/CMakeLists.txt
View File

@ -217,7 +217,7 @@ endif()
if(DEFINED ENV{ENABLE_FAST_HASH_TABLE})
add_compile_definitions(ENABLE_FAST_HASH_TABLE)
include_directories(${TOP_DIR}/third_party/robin_hood/include)
set(MSLITE_DEPS_ROBIN_HOOD_HASHING on)
endif()
if(MSLITE_ENABLE_CLOUD_FUSION_INFERENCE)

1
mindspore/lite/cmake/ccsrc_module.cmake
View File

@ -16,6 +16,7 @@ message(${COMM_PROTO_IN})
ms_protobuf_generate(COMM_PROTO_SRCS COMM_PROTO_HDRS ${COMM_PROTO_IN})
list(APPEND MINDSPORE_PROTO_LIST ${COMM_PROTO_SRCS})
include(${TOP_DIR}/cmake/external_libs/robin.cmake)
include(${TOP_DIR}/cmake/external_libs/eigen.cmake)
include(${TOP_DIR}/cmake/external_libs/mkl_dnn.cmake)

4
mindspore/lite/cmake/lite_dependences.cmake
View File

@ -2,6 +2,10 @@ set(MINDSPORE_PROJECT_DIR ${TOP_DIR})
find_required_package(Patch)
if(MSLITE_DEPS_ROBIN_HOOD_HASHING)
include(${TOP_DIR}/cmake/external_libs/robin.cmake)
endif()
if(MSLITE_DEPS_FLATBUFFERS)
include(${TOP_DIR}/cmake/external_libs/flatbuffers.cmake)
endif()

21
third_party/robin_hood/LICENSE vendored
View File

@ -1,21 +0,0 @@
MIT License
Copyright (c) 2018-2021 Martin Ankerl
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.

72
third_party/robin_hood/README.md vendored
View File

@ -1,72 +0,0 @@
➵ robin_hood unordered map & set [![Release](https://img.shields.io/github/release/martinus/robin-hood-hashing.svg)](https://github.com/martinus/robin-hood-hashing/releases) [![GitHub license](https://img.shields.io/github/license/martinus/robin-hood-hashing.svg)](https://raw.githubusercontent.com/martinus/robin-hood-hashing/master/LICENSE)
============
[![Travis CI Build Status](https://travis-ci.com/martinus/robin-hood-hashing.svg?branch=master)](https://travis-ci.com/martinus/robin-hood-hashing)
[![Appveyor Build Status](https://ci.appveyor.com/api/projects/status/github/martinus/robin-hood-hashing?branch=master&svg=true)](https://ci.appveyor.com/project/martinus/robin-hood-hashing)
[![Codacy Badge](https://api.codacy.com/project/badge/Grade/9308495247b542c9802016caa6fd3461)](https://www.codacy.com/app/martinus/robin-hood-hashing?utm_source=github.com&amp;utm_medium=referral&amp;utm_content=martinus/robin-hood-hashing&amp;utm_campaign=Badge_Grade)
[![Total alerts](https://img.shields.io/lgtm/alerts/g/martinus/robin-hood-hashing.svg?logo=lgtm&logoWidth=18)](https://lgtm.com/projects/g/martinus/robin-hood-hashing/alerts/)
[![Language grade: C/C++](https://img.shields.io/lgtm/grade/cpp/g/martinus/robin-hood-hashing.svg?logo=lgtm&logoWidth=18)](https://lgtm.com/projects/g/martinus/robin-hood-hashing/context:cpp)
[![Coverage Status](https://coveralls.io/repos/github/martinus/robin-hood-hashing/badge.svg)](https://coveralls.io/github/martinus/robin-hood-hashing)
`robin_hood::unordered_map` and `robin_hood::unordered_set` is a platform independent replacement for `std::unordered_map` / `std::unordered_set` which is both faster and more memory efficient for real-world use cases.
## Installation & Usage
### Direct Inclusion
1. Add [`robin_hood.h`](https://github.com/martinus/robin-hood-hashing/releases) to your C++ project.
1. Use `robin_hood::unordered_map` instead of `std::unordered_map`
1. Use `robin_hood::unordered_set` instead of `std::unordered_set`
### [Conan](https://conan.io/), the C/C++ Package Manager
1. Setup your `CMakeLists.txt` (see [Conan documentation](https://docs.conan.io/en/latest/integrations/build_system.html) on how to use MSBuild, Meson and others) like this:
```CMake
project(myproject CXX)
add_executable(${PROJECT_NAME} main.cpp)
include(${CMAKE_BINARY_DIR}/conanbuildinfo.cmake) # Include Conan-generated file
conan_basic_setup(TARGETS) # Introduce Conan-generated targets
target_link_libraries(${PROJECT_NAME} CONAN_PKG::robin-hood-hashing)
```
1. Create `conanfile.txt` in your source dir (don't forget to update the version)
```ini
[requires]
robin-hood-hashing/3.10.0
[generators]
cmake
```
1. Install and run Conan, then build your project as always:
```Bash
pip install conan
mkdir build
cd build
conan install ../ --build=missing
cmake ../
cmake --build .
```
The `robin-hood-hashing` package in Conan is kept up to date by Conan contributors. If the version is out of date, please [create an issue or pull request](https://github.com/conan-io/conan-center-index) on the `conan-center-index` repository.
## Benchmarks
Please see extensive benchmarks at [Hashmaps Benchmarks](https://martin.ankerl.com/2019/04/01/hashmap-benchmarks-01-overview/). In short: `robin_hood` is always among the fastest maps and uses far less memory than `std::unordered_map`.
## Design Choices
- **Two memory layouts**. Data is either stored in a flat array, or with node indirection. Access for `unordered_flat_map` is extremely fast due to no indirection, but references to elements are not stable. It also causes allocation spikes when the map resizes, and will need plenty of memory for large objects. Node based map has stable references & pointers (NOT iterators! Similar to [std::unordered_map](https://en.cppreference.com/w/cpp/container/unordered_map)) and uses `const Key` in the pair. It is a bit slower due to indirection. The choice is yours; you can either use `robin_hood::unordered_flat_map` or `robin_hood::unordered_node_map` directly. If you use `robin_hood::unordered_map` It tries to choose the layout that seems appropriate for your data.
- **Custom allocator**. Node based representation has a custom bulk allocator that tries to make few memory allocations. All allocated memory is reused, so there won't be any allocation spikes. It's very fast as well.
- **Optimized hash**. `robin_hood::hash` has custom implementations for integer types and for `std::string` that are very fast and falls back to `std::hash` for everything else.
- **Depends on good Hashing**. For a really bad hash the performance will not only degrade like in `std::unordered_map`, the map will simply fail with an `std::overflow_error`. In practice, when using the standard `robin_hood::hash`, I have never seen this happening.
## License
Licensed under the MIT License. See the [LICENSE](https://github.com/martinus/robin-hood-hashing/blob/master/LICENSE) file for details.
by martinus

2529
third_party/robin_hood/include/robin_hood/robin_hood.h vendored
View File

File diff suppressed because it is too large Load Diff