forked from mindspore-Ecosystem/mindspore
!15390 add mnist_x86.md
From: @zoloft Reviewed-by: @wangchengyuan,@hangangqiang Signed-off-by: @wangchengyuan
This commit is contained in:
Gitee
commit
eece18fc38
|
|
@ -1,214 +0,0 @@
|
|||
|
||||
|
||||
# Arm Cortex-M编译部署
|
||||
|
||||
`Linux` `Cortex-M` `IOT` `C/C++` `全流程` `模型编译` `模型代码生成` `模型部署` `推理应用` `初级` `中级` `高级`
|
||||
|
||||
<!-- TOC -->
|
||||
|
||||
- Arm Cortex-M编译部署
|
||||
- [STM32F746编译依赖](#STM32F746编译依赖)
|
||||
- [STM32F746构建](#STM32F746构建)
|
||||
- [STM32F746工程部署](#STM32F746工程部署)
|
||||
- [更多详情](#更多详情)
|
||||
- [Linux_x86_64编译部署](#Linux_x86_64编译部署)
|
||||
- [Android平台编译部署](#Android平台编译部署)
|
||||
|
||||
<!-- /TOC -->
|
||||
|
||||
## Arm Cortex-M编译部署
|
||||
|
||||
本教程以在STM32F746单板上编译部署生成模型代码为例，演示了codegen编译模型在Cortex-M平台的使用。更多关于Arm Cortex-M的详情可参见其[官网](https://developer.arm.com/ip-products/processors/cortex-m)。
|
||||
|
||||
### STM32F746编译依赖
|
||||
|
||||
模型推理代码的编译部署需要在windows上安装[Jlink]((https://www.segger.com/))、[STM32CubeMX](https://www.st.com/content/st_com/en.html)、[gcc-arm-none-ebai](https://developer.arm.com/tools-and-software/open-source-software/developer-tools/gnu-toolchain/gnu-rm)等工具来进行交叉编译。
|
||||
|
||||
- [STM32CubeMX-Win](https://www.st.com/content/ccc/resource/technical/software/sw_development_suite/group0/0b/05/f0/25/c7/2b/42/9d/stm32cubemx_v6-1-1/files/stm32cubemx_v6-1-1.zip/jcr:content/translations/en.stm32cubemx_v6-1-1.zip) >= 6.0.1
|
||||
|
||||
- [gcc-arm-none-eabi](https://developer.arm.com/tools-and-software/open-source-software/developer-tools/gnu-toolchain/gnu-rm/downloads) >= 9-2019-q4-major-win32
|
||||
|
||||
- [JLink-windows](https://www.segger.com/downloads/jlink/) >= 6.56
|
||||
- [GCC](https://gcc.gnu.org/releases.html) >= 7.3.0
|
||||
- [CMake](https://cmake.org/download/) >= 3.18.3
|
||||
|
||||
### STM32F746构建
|
||||
|
||||
首先使用codegen编译MNIST手写数字识别模型，生成对应的STM32F46推理代码。具体命令如下：
|
||||
|
||||
```bash
|
||||
./codegen --codePath=. --modelPath=mnist.ms --target=ARM32M
|
||||
```
|
||||
|
||||
#### 代码工程说明
|
||||
|
||||
```bash
|
||||
├── MNIST
|
||||
└── include # 模型推理对外API头文件目录
|
||||
└── operator_library
|
||||
|
||||
```
|
||||
|
||||
##### 算子相关目录说明
|
||||
|
||||
在编译此工程之前需要预先获取对应平台所需要的算子文件，由于Cortex-M平台工程编译一般涉及到较复杂的交叉编译，此处不提供直接预编译的算子库静态库，而是用户根据模型自行组织文件，自主编译Cortex-M7 、Coretex-M4、Cortex-M3等工程(对应工程目录结构已在示例代码中给出，用户可自主将对应arm官方的CMSIS源码放置其中即可)。
|
||||
|
||||
预置算子静态库的目录如下:
|
||||
|
||||
```bash
|
||||
├── operator_library # 对应平台算子库目录
|
||||
├── include # 对应平台算子库头文件目录
|
||||
└── nnacl # 对应mindspore团队提供的平台算子库源文件
|
||||
└── wrapper # 对应mindspore团队提供的平台算子库源文件
|
||||
└── CMSIS # 对应Arm官方提供的CMSIS平台算子库源文件
|
||||
|
||||
```
|
||||
|
||||
> 在使用过程中，我们注意到引入Softmax相关的CMSIS算子文件时，头文件中需要加入`arm_nnfunctions.h`,使用者可以稍作注意。
|
||||
|
||||
生成代码工程目录如下：
|
||||
|
||||
模型推理对外API头文件可由mindspore团队发布的[Release包](https://www.mindspore.cn/tutorial/lite/zh-CN/master/use/downloads.html)中获取。
|
||||
|
||||
```bash
|
||||
├── MNIST # 生成代码的根目录
|
||||
├── benchmark # 生成代码的benchmark目录
|
||||
└── src # 模型推理代码目录
|
||||
```
|
||||
|
||||
#### 代码工程编译
|
||||
|
||||
##### 环境测试
|
||||
|
||||
安装好交叉编译所需环境后，需要在windows环境中依次将其加入到环境变量中
|
||||
|
||||
```bash
|
||||
gcc -v # 查看GCC版本
|
||||
arm-none-eabi-gdb -v # 查看交叉编译环境
|
||||
jlink -v # 查看jlink版本
|
||||
make -v # 查看make版本
|
||||
```
|
||||
|
||||
以上的命令均有成功返回值时，表明环境准备ok，可以继续进入下一步，否则先安装上述环境！！！
|
||||
|
||||
##### 生成STM32F746单板初始化代码([详情示例代码](https://gitee.com/mindspore/mindspore/tree/master/mindspore/lite/micro/example/mnist_stm32f746))
|
||||
|
||||
1. 启动 STM32CubeMX，新建project，选择单板STM32F746IG
|
||||
|
||||
2. 成功以后，选择`Makefile` ，`generator code`
|
||||
|
||||
3. 在生成的工程目录下打开`cmd`，执行`make`，测试初始代码是否成功编译。
|
||||
|
||||
```bash
|
||||
# make成功结果
|
||||
arm-none-eabi-size build/test_stm32f746.elf
|
||||
text data bss dec hex filename
|
||||
3660 20 1572 5252 1484 build/test_stm32f746.elf
|
||||
arm-none-eabi-objcopy -O ihex build/test_stm32f746.elf build/test_stm32f746.hex
|
||||
arm-none-eabi-objcopy -O binary -S build/test_stm32f746.elf build/test_stm32f746.bin
|
||||
```
|
||||
|
||||
##### 编译模型
|
||||
|
||||
1. 拷贝mindspore团队提供算子文件以及对应头文件到STM32CubeMX生成的工程目录中。
|
||||
|
||||
2. 拷贝codegen生成模型推理代码到 STM32CubeMX生成的代码工程目录中
|
||||
|
||||
```bash
|
||||
├── .mxproject
|
||||
└── build # 工程编译输出目录
|
||||
└── Core
|
||||
└── Drivers
|
||||
└── mnist # codegen生成的cortex-m7 模型推理代码
|
||||
└── Makefile # 组织工程makefile文件需要用户自己修改组织mnist && operator_library到工程目录中
|
||||
└── startup_stm32f746xx.s
|
||||
└── STM32F746IGKx_FLASH.ld
|
||||
└── test_stm32f746.ioc
|
||||
```
|
||||
|
||||
3. 修改makefile文件，组织算子静态库以及模型推理代码,具体makefile文件内容参见[示例](https://gitee.com/mindspore/mindspore/tree/master/mindspore/lite/micro/example/mnist_stm32f746)。
|
||||
|
||||
```bash
|
||||
# C includes
|
||||
C_INCLUDES = \
|
||||
-ICore/Inc \
|
||||
-IDrivers/STM32F7xx_HAL_Driver/Inc \
|
||||
-IDrivers/STM32F7xx_HAL_Driver/Inc/Legacy \
|
||||
-IDrivers/CMSIS/Device/ST/STM32F7xx/Include \
|
||||
-Imnist/operator_library/include \ # 新增，指定算子库头文件目录
|
||||
-Imnist/include \ # 新增，指定模型推理代码头文件
|
||||
-Imnist/src # 新增，指定模型推理代码头文件
|
||||
```
|
||||
|
||||
4. 在工程目录的Core/Src的main.c编写模型调用代码，具体代码新增如下：
|
||||
|
||||
```cpp
|
||||
while (1) {
|
||||
/* USER CODE END WHILE */
|
||||
SEGGER_RTT_printf(0, "***********mnist test start***********\n");
|
||||
const char *model_buffer = nullptr;
|
||||
int model_size = 0;
|
||||
session::LiteSession *session = mindspore::session::LiteSession::CreateSession(model_buffer, model_size, nullptr);
|
||||
Vector<tensor::MSTensor *> inputs = session->GetInputs();
|
||||
size_t inputs_num = inputs.size();
|
||||
void *inputs_binbuf[inputs_num];
|
||||
int inputs_size[inputs_num];
|
||||
for (size_t i = 0; i < inputs_num; ++i) {
|
||||
inputs_size[i] = inputs[i]->Size();
|
||||
}
|
||||
// here mnist only have one input data,just hard code to it's array;
|
||||
inputs_binbuf[0] = mnist_inputs_data;
|
||||
for (size_t i = 0; i < inputs_num; ++i) {
|
||||
void *input_data = inputs[i]->MutableData();
|
||||
memcpy(input_data, inputs_binbuf[i], inputs_size[i]);
|
||||
}
|
||||
int ret = session->RunGraph();
|
||||
if (ret != lite::RET_OK) {
|
||||
return lite::RET_ERROR;
|
||||
}
|
||||
Vector<String> outputs_name = session->GetOutputTensorNames();
|
||||
for (int i = 0; i < outputs_name.size(); ++i) {
|
||||
tensor::MSTensor *output_tensor = session->GetOutputByTensorName(outputs_name[i]);
|
||||
if (output_tensor == nullptr) {
|
||||
return -1;
|
||||
}
|
||||
float *casted_data = static_cast<float *>(output_tensor->MutableData());
|
||||
if (casted_data == nullptr) {
|
||||
return -1;
|
||||
}
|
||||
for (size_t j = 0; j < 10 && j < output_tensor->ElementsNum(); j++) {
|
||||
SEGGER_RTT_printf(0, "output: [%d] is : [%d]/100\n", i, casted_data[i] * 100);
|
||||
}
|
||||
}
|
||||
delete session;
|
||||
SEGGER_RTT_printf(0, "***********mnist test end***********\n");
|
||||
```
|
||||
|
||||
5. 在工程跟目中目录使用管理员权限打开`cmd` 执行 `make`进行编译
|
||||
|
||||
```bash
|
||||
make
|
||||
```
|
||||
|
||||
### STM32F746工程部署
|
||||
|
||||
使用jlink 将可执行文件拷贝到单板上并做推理
|
||||
|
||||
```bash
|
||||
jlinkgdbserver # 启动jlinkgdbserver 选定target device为STM32F746IG
|
||||
jlinkRTTViewer # 启动jlinkRTTViewer 选定target devices为STM32F746IG
|
||||
arm-none-eabi-gdb # 启动arm-gcc gdb服务
|
||||
file build/target.elf # 打开调测文件
|
||||
target remote 127.0.0.1 # 连接jlink服务器
|
||||
monitor reset # 重置单板
|
||||
monitor halt # 挂起单板
|
||||
load # 加载可执行文件到单板
|
||||
c # 执行模型推理
|
||||
```
|
||||
|
||||
## 更多详情
|
||||
|
||||
### [Linux_x86_64平台编译部署](https://www.mindspore.cn/tutorial/lite/zh-CN/master/quick_start/quick_start_codegen.html)
|
||||
|
||||
### [Android平台编译部署](https://gitee.com/mindspore/mindspore/tree/master/mindspore/lite/micro/example/mobilenetv2)
|
||||
|
||||
|
|
@ -0,0 +1,268 @@
|
|||
# X86编译部署
|
||||
|
||||
`Linux` `IoT` `C++` `全流程` `模型编译` `模型代码生成` `模型部署` `推理应用` `初级` `中级` `高级`
|
||||
|
||||
<!-- TOC -->
|
||||
|
||||
- [X86编译部署](#X86编译部署)
|
||||
- [概述](#概述)
|
||||
- [模型编译体验](#模型编译体验)
|
||||
- [详细步骤](#详细步骤)
|
||||
- [生成代码](#生成代码)
|
||||
- [部署应用](#部署应用)
|
||||
- [编译依赖](#编译依赖)
|
||||
- [构建与运行](#构建与运行)
|
||||
- [编写推理代码示例](#编写推理代码示例)
|
||||
- [更多详情](#更多详情)
|
||||
- [Android平台编译部署](#android平台编译部署)
|
||||
- [Arm Cortex-M平台编译部署](#armcortex-m平台编译部署)
|
||||
|
||||
<!-- /TOC -->
|
||||
|
||||
## 概述
|
||||
|
||||
本教程以MNIST分类模型推理代码为例，帮助用户了解codegen生成代码、编译构建、部署等流程。
|
||||
|
||||
## 模型编译体验
|
||||
|
||||
用户可以使用脚本一键式编译生成MNIST分类模型的推理代码并执行推理，得到单次推理输出。下载[MindSpore源码](https://gitee.com/mindspore/mindspore)，进入[`mindspore/mindspore/lite/micro/examples/mnist_x86`](https://gitee.com/mindspore/mindspore/tree/master/mindspore/lite/micro/example/mnist_x86)目录，执行脚本`mnist.sh`自动生成模型推理代码并编译工程目录，即可得到单次推理输出。
|
||||
|
||||
```bash
|
||||
bash mnist.sh
|
||||
```
|
||||
|
||||
推理结果如下：
|
||||
|
||||
```text
|
||||
======run benchmark======
|
||||
input 0: mnist_input.bin
|
||||
|
||||
outputs:
|
||||
name: Softmax-7, DataType: 43, Size: 40, Shape: [1 10], Data:
|
||||
0.000000, 0.000000, 0.000000, 1.000000, 0.000000, 0.000000, 0.000000, 0.000000, 0.000000, 0.000000,
|
||||
======run success=======
|
||||
```
|
||||
|
||||
也可以按照**详细步骤**从生成代码开始逐步完成使用codegen编译一个MNIST分类模型的全流程。
|
||||
|
||||
## 详细步骤
|
||||
|
||||
在编译此工程之前需要预先获取Ubuntu-x64 CPU平台的[Release包](https://www.mindspore.cn/tutorial/lite/zh-CN/master/use/downloads.html)，解压后得到`mindspore-lite-{version}-linux-x64`，将其拷贝到当前目录下。
|
||||
|
||||
> `{version}`为版本号字符串，如`1.2.0`。
|
||||
|
||||
以本教程为例，预置x86平台的Release包目录如下：
|
||||
|
||||
```text
|
||||
mindspore-lite-{version}-linux-x64
|
||||
└── tools
|
||||
└── codegen # 代码生成工具
|
||||
├── codegen # 可执行程序
|
||||
├── include # 推理框架头文件
|
||||
│ ├── nnacl # nnacl 算子头文件
|
||||
│ └── wrapper
|
||||
├── lib
|
||||
│ └── libwrapper.a # MindSpore Lite CodeGen生成代码依赖的部分算子静态库
|
||||
└── third_party
|
||||
├── include
|
||||
│ └── CMSIS # ARM CMSIS NN 算子头文件
|
||||
└── lib
|
||||
└── libcmsis_nn.a # ARM CMSIS NN 算子静态库
|
||||
```
|
||||
|
||||
### 生成代码
|
||||
|
||||
下载[MNIST分类网络](https://download.mindspore.cn/model_zoo/official/lite/mnist_lite/mnist.ms)。使用Release包中的codegen编译MNIST分类模型，生成对应的x86平台推理代码。生成代码的具体命令如下：
|
||||
|
||||
```bash
|
||||
./codegen --codePath=. --modelPath=mnist.ms --target=x86
|
||||
```
|
||||
|
||||
codegen在当前目录下将生成mnist目录，其中包含了可编译构建的mnist分类模型的代码。
|
||||
> 关于codegen的更多使用命令说明，可参见[codegen工具的详细介绍](https://www.mindspore.cn/tutorial/lite/zh-CN/master/use/code_generator.html)。
|
||||
|
||||
### 部署应用
|
||||
|
||||
接下来介绍如何构建MindSpore Lite CodeGen生成的模型推理代码工程，并在x86平台完成部署。上文中codegen生成的代码与`mindspore/mindspore/lite/micro/example/mnist_x86`相同，本章节编译、构建步骤将对该目录展开，用户也可参照相同操作，编译上文codegen生成mnist目录代码。
|
||||
|
||||
#### 编译依赖
|
||||
|
||||
- [CMake](https://cmake.org/download/) >= 3.18.3
|
||||
- [GCC](https://gcc.gnu.org/releases.html) >= 7.3.0
|
||||
|
||||
#### 构建与运行
|
||||
|
||||
1. **生成代码工程说明**
|
||||
|
||||
进入`mindspore/mindspore/lite/micro/example/mnist_x86`目录中。
|
||||
|
||||
生成代码工程目录说明：
|
||||
|
||||
当前目录下预置了MNIST分类网络生成的代码。
|
||||
|
||||
```text
|
||||
mnist_x86/ # 生成代码的根目录
|
||||
├── benchmark # 生成代码的benchmark目录
|
||||
└── src # 模型推理代码目录
|
||||
```
|
||||
|
||||
2. **代码编译**
|
||||
|
||||
组织模型生成的推理代码以及算子静态库，编译生成模型推理静态库并编译生成benchmark可执行文件,
|
||||
|
||||
进入代码工程目录下，新建并进入build目录：
|
||||
|
||||
```bash
|
||||
mkdir build && cd build
|
||||
```
|
||||
|
||||
开始编译：
|
||||
|
||||
```bash
|
||||
cmake -DPKG_PATH={path to}/mindspore-lite-{version}-linux-x64 ..
|
||||
make
|
||||
```
|
||||
|
||||
> `{path to}`和`{version}`需要用户根据实际情况填写。
|
||||
|
||||
代码工程编译成功结果：
|
||||
|
||||
```text
|
||||
Scanning dependencies of target net
|
||||
[ 12%] Building C object src/CMakeFiles/net.dir/net.c.o
|
||||
[ 25%] Building CXX object src/CMakeFiles/net.dir/session.cc.o
|
||||
[ 37%] Building CXX object src/CMakeFiles/net.dir/tensor.cc.o
|
||||
[ 50%] Building C object src/CMakeFiles/net.dir/weight.c.o
|
||||
[ 62%] Linking CXX static library libnet.a
|
||||
unzip raw static library libnet.a
|
||||
raw static library libnet.a size:
|
||||
-rw-r--r-- 1 user user 58K Mar 22 10:09 libnet.a
|
||||
generate specified static library libnet.a
|
||||
new static library libnet.a size:
|
||||
-rw-r--r-- 1 user user 162K Mar 22 10:09 libnet.a
|
||||
[ 62%] Built target net
|
||||
Scanning dependencies of target benchmark
|
||||
[ 75%] Building CXX object CMakeFiles/benchmark.dir/benchmark/benchmark.cc.o
|
||||
[ 87%] Building C object CMakeFiles/benchmark.dir/benchmark/load_input.c.o
|
||||
[100%] Linking CXX executable benchmark
|
||||
[100%] Built target benchmark
|
||||
```
|
||||
|
||||
此时在`mnist_x86/build/src/`目录下生成了`libnet.a`，推理执行库，在`mnist_x86/build`目录下生成了`benchmark`可执行文件。
|
||||
|
||||
3. **代码部署**
|
||||
|
||||
本示例部署于x86平台。由代码工程编译成功以后的产物为`benchmark`可执行文件，将其拷贝到用户的目标Linux服务器中即可执行。
|
||||
|
||||
在目标Linux服务上执行编译成功的二进制文件：
|
||||
|
||||
```bash
|
||||
./benchmark mnist_input.bin net.bin
|
||||
```
|
||||
|
||||
> mnist_input.bin在`example/mnist_x86`目录下，`net.bin`为模型参数文件，在`example/mnist_x86/src`目录下。
|
||||
|
||||
生成结果如下：
|
||||
|
||||
```text
|
||||
start run benchmark
|
||||
input 0: mnist_input.bin
|
||||
output size: 1
|
||||
uint8:
|
||||
Name: Softmax-7, DataType: 43, Size: 40, Shape: 1 10, Data:
|
||||
0.000000, 0.000000, 0.000000, 1.000000, 0.000000, 0.000000, 0.000000, 0.000000, 0.000000, 0.000000,
|
||||
run benchmark success
|
||||
```
|
||||
|
||||
#### 编写推理代码示例
|
||||
|
||||
本教程中的`benchmark`内部实现主要用于指导用户如何编写以及调用codegen编译的模型推理代码接口。以下为接口调用的详细介绍，详情代码可以参见[examples/mnist_x86](https://gitee.com/mindspore/mindspore/tree/master/mindspore/lite/micro/example/mnist_x86)下的示例代码示例：
|
||||
|
||||
1. **构建推理的上下文以及会话**
|
||||
|
||||
本教程生成的代码为非并行代码，无需上下文context，可直接设为空。
|
||||
|
||||
```cpp
|
||||
size_t model_size = 0;
|
||||
Context *context = nullptr;
|
||||
session::LiteSession *session = mindspore::session::LiteSession::CreateSession(model_buffer, model_size, context);
|
||||
if (session == nullptr) {
|
||||
std::cerr << "create lite session failed" << std::endl;
|
||||
return RET_ERROR;
|
||||
}
|
||||
```
|
||||
|
||||
2. **输入数据准备**
|
||||
|
||||
用户所需要准备的输入数据内存空间，若输入是持久化文件，可通过读文件方式获取。若输入数据已经存在内存中，则此处无需读取，可直接传入数据指针。
|
||||
|
||||
```cpp
|
||||
std::vector<MSTensor *> inputs = session->GetInputs();
|
||||
MSTensor *input = inputs.at(0);
|
||||
if (input == nullptr) {
|
||||
return RET_ERROR;
|
||||
}
|
||||
// Assume we have got input data in memory.
|
||||
memcpy(input->MutableData(), input_buffer, input->Size());
|
||||
```
|
||||
|
||||
3. **执行推理**
|
||||
|
||||
```cpp
|
||||
session->RunGraph();
|
||||
```
|
||||
|
||||
4. **推理结束获取输出**
|
||||
|
||||
```cpp
|
||||
Vector<String> outputs_name = session->GetOutputTensorNames();
|
||||
for (const auto &name : outputs_name) {
|
||||
auto output = session->GetOutputByTensorName(name);
|
||||
// deal with output
|
||||
......
|
||||
}
|
||||
```
|
||||
|
||||
5. **释放内存session**
|
||||
|
||||
```cpp
|
||||
delete session;
|
||||
```
|
||||
|
||||
6. **推理代码整体调用流程**
|
||||
|
||||
```cpp
|
||||
// Assume we have got model_buffer data in memory.
|
||||
size_t model_size = 0;
|
||||
Context *context = nullptr;
|
||||
session::LiteSession *session = mindspore::session::LiteSession::CreateSession(model_buffer, model_size, context);
|
||||
if (session == nullptr) {
|
||||
std::cerr << "create lite session failed" << std::endl;
|
||||
return RET_ERROR;
|
||||
}
|
||||
|
||||
std::vector<MSTensor *> inputs = session->GetInputs();
|
||||
MSTensor *input = inputs.at(0);
|
||||
if (input == nullptr) {
|
||||
return RET_ERROR;
|
||||
}
|
||||
// Assume we have got input data in memory.
|
||||
memcpy(input->MutableData(), input_buffer, input->Size());
|
||||
|
||||
session->RunGraph();
|
||||
|
||||
Vector<String> outputs_name = session->GetOutputTensorNames();
|
||||
for (const auto &name : outputs_name) {
|
||||
auto output = session->GetOutputByTensorName(name);
|
||||
// deal with output
|
||||
......
|
||||
}
|
||||
|
||||
delete session;
|
||||
```
|
||||
|
||||
## 更多详情
|
||||
|
||||
### [Android平台编译部署](https://gitee.com/mindspore/mindspore/blob/master/mindspore/lite/micro/example/mobilenetv2/README.md#)
|
||||
|
||||
### [Arm Cortex-M平台编译部署](https://www.mindspore.cn/tutorial/lite/zh-CN/master/use/micro.html)
|
||||
Loading…
Reference in New Issue