From 208ede7c832a62c6fa2df6b356df40cb3dca0d72 Mon Sep 17 00:00:00 2001 From: lz Date: Wed, 23 Feb 2022 11:37:03 +0800 Subject: [PATCH] add micro examples & delete useless tests --- .jenkins/check/config/filter_cpplint.txt | 1 + .../quick_start_micro/mnist_x86/README.md | 268 ++++++++++++++++++ .../quick_start_micro/mnist_x86/micro.cfg | 27 ++ .../quick_start_micro/mnist_x86/mnist.sh | 101 +++++++ .../tools/converter/micro/test/CMakeLists.txt | 24 -- .../converter/micro/test/code_gen_test.cc | 33 --- 6 files changed, 397 insertions(+), 57 deletions(-) create mode 100644 mindspore/lite/examples/quick_start_micro/mnist_x86/README.md create mode 100644 mindspore/lite/examples/quick_start_micro/mnist_x86/micro.cfg create mode 100644 mindspore/lite/examples/quick_start_micro/mnist_x86/mnist.sh delete mode 100644 mindspore/lite/tools/converter/micro/test/CMakeLists.txt delete mode 100644 mindspore/lite/tools/converter/micro/test/code_gen_test.cc diff --git a/.jenkins/check/config/filter_cpplint.txt b/.jenkins/check/config/filter_cpplint.txt index bb223436ca7..7a3011b29ce 100644 --- a/.jenkins/check/config/filter_cpplint.txt +++ b/.jenkins/check/config/filter_cpplint.txt @@ -66,3 +66,4 @@ "mindspore/mindspore/lite/examples/runtime_gpu_extend/src/cl" "readability/fn_size" "mindspore/mindspore/lite/examples/quick_start_c/main.c" "readability/casting" "mindspore/mindspore/lite/examples/quick_start_c/main.c" "runtime/threadsafe_fn" +"mindspore/mindspore/lite/examples/quick_start_micro" "readability/casting" diff --git a/mindspore/lite/examples/quick_start_micro/mnist_x86/README.md b/mindspore/lite/examples/quick_start_micro/mnist_x86/README.md new file mode 100644 index 00000000000..fe3e9930964 --- /dev/null +++ b/mindspore/lite/examples/quick_start_micro/mnist_x86/README.md @@ -0,0 +1,268 @@ +# X86编译部署 + + `Linux` `IoT` `C++` `全流程` `模型编译` `模型代码生成` `模型部署` `推理应用` `初级` `中级` `高级` + + + +- [X86编译部署](#X86编译部署) + - [概述](#概述) + - [模型编译体验](#模型编译体验) + - [详细步骤](#详细步骤) + - [生成代码](#生成代码) + - [部署应用](#部署应用) + - [编译依赖](#编译依赖) + - [构建与运行](#构建与运行) + - [编写推理代码示例](#编写推理代码示例) + - [更多详情](#更多详情) + - [Android平台编译部署](#android平台编译部署) + - [Arm Cortex-M平台编译部署](#armcortex-m平台编译部署) + + + +## 概述 + +本教程以MNIST分类模型推理代码为例，帮助用户了解codegen生成代码、编译构建、部署等流程。 + +## 模型编译体验 + +用户可以使用脚本一键式编译生成MNIST分类模型的推理代码并执行推理，得到单次推理输出。下载[MindSpore源码](https://gitee.com/mindspore/mindspore)，进入[`mindspore/mindspore/lite/micro/examples/mnist_x86`](https://gitee.com/mindspore/mindspore/tree/master/mindspore/lite/micro/example/mnist_x86)目录，执行脚本`mnist.sh`自动生成模型推理代码并编译工程目录，即可得到单次推理输出。 + +```bash +bash mnist.sh +``` + +推理结果如下： + +```text +======run benchmark====== +input 0: mnist_input.bin + +outputs: +name: Softmax-7, DataType: 43, Size: 40, Shape: [1 10], Data: +0.000000, 0.000000, 0.000000, 1.000000, 0.000000, 0.000000, 0.000000, 0.000000, 0.000000, 0.000000, +======run success======= +``` + +也可以按照**详细步骤**从生成代码开始逐步完成使用codegen编译一个MNIST分类模型的全流程。 + +## 详细步骤 + +在编译此工程之前需要预先获取Ubuntu-x64 CPU平台的[Release包](https://www.mindspore.cn/lite/docs/zh-CN/master/use/downloads.html)，解压后得到`mindspore-lite-{version}-linux-x64`，将其拷贝到当前目录下。 + +> `{version}`为版本号字符串，如`1.2.0`。 + +以本教程为例，预置x86平台的Release包目录如下： + +```text + mindspore-lite-{version}-linux-x64 + └── tools + └── codegen # 代码生成工具 + ├── codegen # 可执行程序 + ├── include # 推理框架头文件 + │ ├── nnacl # nnacl 算子头文件 + │ └── wrapper + ├── lib + │ └── libwrapper.a # MindSpore Lite CodeGen生成代码依赖的部分算子静态库 + └── third_party + ├── include + │ └── CMSIS # ARM CMSIS NN 算子头文件 + └── lib + └── libcmsis_nn.a # ARM CMSIS NN 算子静态库 +``` + +### 生成代码 + +下载[MNIST分类网络](https://download.mindspore.cn/model_zoo/official/lite/mnist_lite/mnist.ms)。使用Release包中的codegen编译MNIST分类模型，生成对应的x86平台推理代码。生成代码的具体命令如下： + +```bash +./codegen --codePath=. --modelPath=mnist.ms --target=x86 +``` + +codegen在当前目录下将生成mnist目录，其中包含了可编译构建的mnist分类模型的代码。 +> 关于codegen的更多使用命令说明，可参见[codegen使用说明](https://www.mindspore.cn/lite/docs/zh-CN/master/use/micro.html#id4)。 + +### 部署应用 + +接下来介绍如何构建MindSpore Lite CodeGen生成的模型推理代码工程，并在x86平台完成部署。上文中codegen生成的代码与`mindspore/mindspore/lite/micro/example/mnist_x86`相同，本章节编译、构建步骤将对该目录展开，用户也可参照相同操作，编译上文codegen生成mnist目录代码。 + +#### 编译依赖 + +- [CMake](https://cmake.org/download/) >= 3.18.3 +- [GCC](https://gcc.gnu.org/releases.html) >= 7.3.0 + +#### 构建与运行 + +1. **生成代码工程说明** + + 进入`mindspore/mindspore/lite/micro/example/mnist_x86`目录中。 + + 生成代码工程目录说明： + + 当前目录下预置了MNIST分类网络生成的代码。 + + ```text + mnist_x86/ # 生成代码的根目录 + ├── benchmark # 生成代码的benchmark目录 + └── src # 模型推理代码目录 + ``` + +2. **代码编译** + + 组织模型生成的推理代码以及算子静态库，编译生成模型推理静态库并编译生成benchmark可执行文件, + + 进入代码工程目录下，新建并进入build目录： + + ```bash + mkdir build && cd build + ``` + + 开始编译： + + ```bash + cmake -DPKG_PATH={path to}/mindspore-lite-{version}-linux-x64 .. + make + ``` + + > `{path to}`和`{version}`需要用户根据实际情况填写。 + + 代码工程编译成功结果： + + ```text + Scanning dependencies of target net + [ 12%] Building C object src/CMakeFiles/net.dir/net.c.o + [ 25%] Building CXX object src/CMakeFiles/net.dir/session.cc.o + [ 37%] Building CXX object src/CMakeFiles/net.dir/tensor.cc.o + [ 50%] Building C object src/CMakeFiles/net.dir/weight.c.o + [ 62%] Linking CXX static library libnet.a + unzip raw static library libnet.a + raw static library libnet.a size: + -rw-r--r-- 1 user user 58K Mar 22 10:09 libnet.a + generate specified static library libnet.a + new static library libnet.a size: + -rw-r--r-- 1 user user 162K Mar 22 10:09 libnet.a + [ 62%] Built target net + Scanning dependencies of target benchmark + [ 75%] Building CXX object CMakeFiles/benchmark.dir/benchmark/benchmark.cc.o + [ 87%] Building C object CMakeFiles/benchmark.dir/benchmark/load_input.c.o + [100%] Linking CXX executable benchmark + [100%] Built target benchmark + ``` + + 此时在`mnist_x86/build/src/`目录下生成了`libnet.a`，推理执行库，在`mnist_x86/build`目录下生成了`benchmark`可执行文件。 + +3. **代码部署** + + 本示例部署于x86平台。由代码工程编译成功以后的产物为`benchmark`可执行文件，将其拷贝到用户的目标Linux服务器中即可执行。 + + 在目标Linux服务上执行编译成功的二进制文件： + + ```bash + ./benchmark mnist_input.bin net.bin + ``` + + > mnist_input.bin在`example/mnist_x86`目录下，`net.bin`为模型参数文件，在`example/mnist_x86/src`目录下。 + + 生成结果如下： + + ```text + start run benchmark + input 0: mnist_input.bin + output size: 1 + uint8: + Name: Softmax-7, DataType: 43, Size: 40, Shape: 1 10, Data: + 0.000000, 0.000000, 0.000000, 1.000000, 0.000000, 0.000000, 0.000000, 0.000000, 0.000000, 0.000000, + run benchmark success + ``` + +#### 编写推理代码示例 + +本教程中的`benchmark`内部实现主要用于指导用户如何编写以及调用codegen编译的模型推理代码接口。以下为接口调用的详细介绍，详情代码可以参见[examples/mnist_x86](https://gitee.com/mindspore/mindspore/tree/master/mindspore/lite/micro/example/mnist_x86)下的示例代码示例： + +1. **构建推理的上下文以及会话** + + 本教程生成的代码为非并行代码，无需上下文context，可直接设为空。 + + ```cpp + size_t model_size = 0; + Context *context = nullptr; + session::LiteSession *session = mindspore::session::LiteSession::CreateSession(model_buffer, model_size, context); + if (session == nullptr) { + std::cerr << "create lite session failed" << std::endl; + return RET_ERROR; + } + ``` + +2. **输入数据准备** + + 用户所需要准备的输入数据内存空间，若输入是持久化文件，可通过读文件方式获取。若输入数据已经存在内存中，则此处无需读取，可直接传入数据指针。 + + ```cpp + std::vector inputs = session->GetInputs(); + MSTensor *input = inputs.at(0); + if (input == nullptr) { + return RET_ERROR; + } + // Assume we have got input data in memory. + memcpy(input->MutableData(), input_buffer, input->Size()); + ``` + +3. **执行推理** + + ```cpp + session->RunGraph(); + ``` + +4. **推理结束获取输出** + + ```cpp + Vector outputs_name = session->GetOutputTensorNames(); + for (const auto &name : outputs_name) { + auto output = session->GetOutputByTensorName(name); + // deal with output + ...... + } + ``` + +5. **释放内存session** + + ```cpp + delete session; + ``` + +6. **推理代码整体调用流程** + + ```cpp + // Assume we have got model_buffer data in memory. + size_t model_size = 0; + Context *context = nullptr; + session::LiteSession *session = mindspore::session::LiteSession::CreateSession(model_buffer, model_size, context); + if (session == nullptr) { + std::cerr << "create lite session failed" << std::endl; + return RET_ERROR; + } + + std::vector inputs = session->GetInputs(); + MSTensor *input = inputs.at(0); + if (input == nullptr) { + return RET_ERROR; + } + // Assume we have got input data in memory. + memcpy(input->MutableData(), input_buffer, input->Size()); + + session->RunGraph(); + + Vector outputs_name = session->GetOutputTensorNames(); + for (const auto &name : outputs_name) { + auto output = session->GetOutputByTensorName(name); + // deal with output + ...... + } + + delete session; + ``` + +## 更多详情 + +### [Android平台编译部署](https://gitee.com/mindspore/mindspore/blob/master/mindspore/lite/micro/example/mobilenetv2/README.md#) + +### [Arm Cortex-M平台编译部署](https://www.mindspore.cn/lite/docs/zh-CN/master/use/micro.html) diff --git a/mindspore/lite/examples/quick_start_micro/mnist_x86/micro.cfg b/mindspore/lite/examples/quick_start_micro/mnist_x86/micro.cfg new file mode 100644 index 00000000000..da75e4447d4 --- /dev/null +++ b/mindspore/lite/examples/quick_start_micro/mnist_x86/micro.cfg @@ -0,0 +1,27 @@ +[common_quant_param] +# Supports WEIGHT_QUANT or FULL_QUANT +#quant_type=WEIGHT_QUANT +# Weight quantization support the number of bits [0,16], Set to 0 is mixed bit quantization, otherwise it is fixed bit quantization +# Full quantization support the number of bits [1,8] +#bit_num=8 +# Layers with size of weights exceeds threshold `min_quant_weight_size` will be quantized. +#min_quant_weight_size=0 +# Layers with channel size of weights exceeds threshold `min_quant_weight_channel` will be quantized. +#min_quant_weight_channel=16 + + +[micro_param] +# enable code-generation for MCU HW +enable_micro=true + +# specify HW target, support x86,ARM32M, AMR32A, ARM64 only. +target=x86 + +# code generation for Inference or Train +codegen_mode=Inference + +# enable parallel inference or not +support_parallel=false + +# enable debug +debug_mode=false diff --git a/mindspore/lite/examples/quick_start_micro/mnist_x86/mnist.sh b/mindspore/lite/examples/quick_start_micro/mnist_x86/mnist.sh new file mode 100644 index 00000000000..43bf9e53d9c --- /dev/null +++ b/mindspore/lite/examples/quick_start_micro/mnist_x86/mnist.sh @@ -0,0 +1,101 @@ +#!/bin/bash +# Copyright 2021 Huawei Technologies Co., Ltd +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +# ============================================================================ +set -e + +GEN=OFF +TARBALL="" +while getopts 'r:g:' OPT +do + case "${OPT}" in + g) + GEN=$OPTARG + ;; + r) + TARBALL=$OPTARG + ;; + ?) + echo "Usage: add -g on , -r specific release.tar.gz" + esac +done + +BASEPATH="$( cd "$( dirname "${BASH_SOURCE[0]}" )" &> /dev/null && pwd )" +ROOT_DIR=${BASEPATH%%/mindspore/lite/examples/quick_start_micro/mnist_x86} +DEMO_DIR=${ROOT_DIR}/mindspore/lite/examples/quick_start_micro/mnist_x86 +MODEL_DIR=${ROOT_DIR}/mindspore/lite/examples/quick_start_micro/models +PKG_DIR=${ROOT_DIR}/mindspore/lite/examples/quick_start_micro/pkgs +COFIG_FILE=${DEMO_DIR}/micro.cfg +echo "root dir is: ${ROOT_DIR}" +echo "current dir is: ${BASEPATH}" +echo "demo dir is: ${DEMO_DIR}" +echo "model dir is: ${MODEL_DIR}" + +MODEL_NAME=mnist +INPUT_BIN=${MODEL_DIR}/${MODEL_NAME}/mnist.tflite.ms.bin +VALICATION_DATA=${MODEL_DIR}/${MODEL_NAME}/mnist.tflite.ms.out +MODEL=${MODEL_DIR}/${MODEL_NAME}/mnist.tflite +MODEL_FILE=${MODEL_NAME}.tar.gz + +get_version() { + local VERSION_HEADER=${ROOT_DIR}/mindspore/lite/include/version.h + local VERSION_MAJOR=$(grep "const int ms_version_major =" ${VERSION_HEADER} | tr -dc "[0-9]") + local VERSION_MINOR=$(grep "const int ms_version_minor =" ${VERSION_HEADER} | tr -dc "[0-9]") + local VERSION_REVISION=$(grep "const int ms_version_revision =" ${VERSION_HEADER} | tr -dc "[0-9]") + VERSION_STR=${VERSION_MAJOR}.${VERSION_MINOR}.${VERSION_REVISION} +} + +download_mnist() { + rm -rf ${MODEL_DIR:?}/${MODEL_NAME} + mkdir -p ${MODEL_DIR}/${MODEL_NAME} + tar xzvf ${MODEL_DIR}/${MODEL_FILE} -C ${MODEL_DIR}/${MODEL_NAME} || exit 1 +} + +gen_mnist() { + tar xzvf ${PKG_DIR}/${MINDSPORE_FILE} -C ${PKG_DIR} || exit 1 + export LD_LIBRARY_PATH=${PKG_DIR}/${MINDSPORE_FILE_NAME}/tools/converter/lib:${LD_LIBRARY_PATH} + ${PKG_DIR}/${MINDSPORE_FILE_NAME}/tools/converter/converter/converter_lite --fmk=TFLITE --modelFile=${MODEL} --outputFile=${DEMO_DIR} --configFile=${COFIG_FILE} +} + +mkdir -p ${BASEPATH}/build + +get_version +MINDSPORE_FILE_NAME="mindspore-lite-${VERSION_STR}-linux-x64" +MINDSPORE_FILE="${MINDSPORE_FILE_NAME}.tar.gz" +PKG_PATH=${PKG_DIR}/${MINDSPORE_FILE_NAME} + +echo "tar ball is: ${TARBALL}" +if [ -n "$TARBALL" ]; then + echo "cp file" + rm -rf ${PKG_DIR} + mkdir -p ${PKG_DIR} + cp ${TARBALL} ${PKG_DIR} +fi + +# 1. code-generation +if [[ "${GEN}" == "ON" ]] || [[ "${GEN}" == "on" ]]; then + echo "downloading mnist.ms!" + download_mnist + echo "generating mnist" + gen_mnist +fi + +# 2. build benchmark +mkdir -p ${DEMO_DIR}/build && cd ${DEMO_DIR}/build || exit 1 +cmake -DPKG_PATH=${PKG_PATH} .. +make + +# 3. run benchmark +echo "net file: ${DEMO_DIR}/src/mnist.bin" +./benchmark ${INPUT_BIN} ../src/net.bin 1 ${VALICATION_DATA} \ No newline at end of file diff --git a/mindspore/lite/tools/converter/micro/test/CMakeLists.txt b/mindspore/lite/tools/converter/micro/test/CMakeLists.txt deleted file mode 100644 index 44694ccd07a..00000000000 --- a/mindspore/lite/tools/converter/micro/test/CMakeLists.txt +++ /dev/null @@ -1,24 +0,0 @@ -add_definitions(-DUSE_GLOG) -string(REPLACE "/test" "" MICRO_DIR ${CMAKE_CURRENT_SOURCE_DIR}) -string(REPLACE " -fvisibility=hidden " " -fvisibility=default " CMAKE_C_FLAGS "${CMAKE_C_FLAGS}") -string(REPLACE " -fvisibility=hidden " " -fvisibility=default " CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS}") - -set(3RD_DIR ${TOP_DIR}/third_party) -set(LITE_DIR ${TOP_DIR}/mindspore/lite) -set(BUILD_LITE "on") - -include(${TOP_DIR}/cmake/external_libs/gtest.cmake) -include(${MICRO_DIR}/cmake/file_list.cmake) -include(${MICRO_DIR}/cmake/package_wrapper.cmake) - -include_directories(${NNACL_DIR}/../) -include_directories(${TOP_DIR}) -include_directories(${TOP_DIR}/mindspore/core/) -include_directories(${LITE_DIR}) -include_directories(${MICRO_DIR}) -include_directories(${3RD_DIR}) - -add_executable(micro_test code_gen_test.cc ${FILE_SET}) -add_dependencies(micro_test fbs_src) -add_dependencies(micro_test fbs_inner_src) -target_link_libraries(micro_test dl mindspore::gtest ${SECUREC_LIBRARY} mindspore::glog) diff --git a/mindspore/lite/tools/converter/micro/test/code_gen_test.cc b/mindspore/lite/tools/converter/micro/test/code_gen_test.cc deleted file mode 100644 index 6d0859da086..00000000000 --- a/mindspore/lite/tools/converter/micro/test/code_gen_test.cc +++ /dev/null @@ -1,33 +0,0 @@ -/** - * Copyright 2019 Huawei Technologies Co., Ltd - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -#include "gtest/gtest.h" -#include "micro/coder/coder.h" - -namespace mindspore::lite::micro::test { -TEST(GenerateCodeTest, mnist_x86) { - const char *argv[] = {"./codegen", "--modelPath=../example/mnist/mnist.ms", "--moduleName=mnist", "--codePath=.", - "--isWeightFile"}; - STATUS status = RunCoder(5, argv); - ASSERT_EQ(status, RET_OK); -} -} // namespace mindspore::lite::micro::test - -GTEST_API_ int main(int argc, char **argv) { - testing::InitGoogleTest(&argc, argv); - int ret = RUN_ALL_TESTS(); - return ret; -}