forked from mindspore-Ecosystem/mindspore
add network demo
This commit is contained in:
parent
d80a5d2f1e
commit
4ae6d2740a
|
@ -68,6 +68,21 @@ Dataset used:
|
|||
|
||||
在通过官方网站安装MindSpore之后,你可以通过如下步骤开始训练以及评估:
|
||||
|
||||
```backbone
|
||||
vgg16训练ImageNet数据集的ckpt文件做为FCN8s的backbone
|
||||
vgg16网络路径: model_zoo/official/cv/vgg16
|
||||
```
|
||||
|
||||
```default_config.yaml
|
||||
data_file: /home/DataSet/voc2012/vocaug_mindrecords/vocaug.mindrecord0
|
||||
ckpt_vgg16: /home/DataSet/predtrained/vgg16_predtrained.ckpt
|
||||
data_root: /home/DataSet/voc2012/VOCdevkit/VOC2012
|
||||
data_lst: /home/DataSet/voc2012/VOCdevkit/VOC2012/ImageSets/Segmentation/val.txt
|
||||
ckpt_file: /home/FCN8s/ckpt/FCN8s_1-133_300.ckpt
|
||||
|
||||
根据本地数据存放路径修改参数
|
||||
```
|
||||
|
||||
- running on Ascend with default parameters
|
||||
|
||||
```python
|
||||
|
@ -206,11 +221,15 @@ Dataset used:
|
|||
python train.py --device_id device_id
|
||||
or
|
||||
bash scripts/run_standalone_train.sh [DEVICE_ID]
|
||||
# example: bash scripts/run_standalone_train.sh 0
|
||||
|
||||
#Ascend八卡并行训练
|
||||
bash scripts/run_train.sh [DEVICE_NUM] rank_table.json
|
||||
# example: bash scripts/run_train.sh 8 /home/hccl_8p_01234567_10.155.170.71.json
|
||||
```
|
||||
|
||||
分布式训练需要提前创建JSON格式的HCCL配置文件,请遵循[链接说明](https://gitee.com/mindspore/mindspore/tree/master/model_zoo/utils/hccl_tools)
|
||||
|
||||
- running on GPU with gpu default parameters
|
||||
|
||||
```python
|
||||
|
@ -334,6 +353,8 @@ Dataset used:
|
|||
|
||||
```shell 评估
|
||||
bash scripts/run_eval.sh DATA_ROOT DATA_LST CKPT_PATH
|
||||
# example: bash scripts/run_eval.sh /home/DataSet/voc2012/VOCdevkit/VOC2012 \
|
||||
# /home/DataSet/voc2012/VOCdevkit/VOC2012/ImageSets/Segmentation/val.txt /home/FCN8s/ckpt/FCN8s_1-133_300.ckpt
|
||||
```
|
||||
|
||||
以上的python命令会在终端上运行,你可以在终端上查看此次评估的结果。测试集的精确度会以类似如下方式呈现:
|
||||
|
|
|
@ -69,8 +69,11 @@ After installing MindSpore via the official website, you can start training and
|
|||
```python
|
||||
# enter script dir, train AlexNet
|
||||
bash run_standalone_train_ascend.sh [DATA_PATH] [CKPT_SAVE_PATH]
|
||||
# example: bash run_standalone_train_ascend.sh /home/DataSet/Cifar10/cifar-10-batches-bin/ /home/model/alexnet/ckpt/
|
||||
|
||||
# enter script dir, evaluate AlexNet
|
||||
bash run_standalone_eval_ascend.sh [DATA_PATH] [CKPT_NAME]
|
||||
# example: bash run_standalone_eval_ascend.sh /home/DataSet/cifar10/cifar-10-verify-bin /home/model/cv/alxnet/ckpt/checkpoint_alexnet-1_1562.ckpt
|
||||
```
|
||||
|
||||
- Running on [ModelArts](https://support.huaweicloud.com/modelarts/)
|
||||
|
@ -228,7 +231,7 @@ Major parameters in train.py and config.py as follows:
|
|||
```bash
|
||||
python train.py --config_path default_config.yaml --data_path cifar-10-batches-bin --ckpt_path ckpt > log 2>&1 &
|
||||
# or enter script dir, and run the script
|
||||
bash run_standalone_train_ascend.sh cifar-10-batches-bin ckpt
|
||||
bash run_standalone_train_ascend.sh /home/DataSet/Cifar10/cifar-10-batches-bin/ /home/model/alexnet/ckpt/
|
||||
```
|
||||
|
||||
After training, the loss value will be achieved as follows:
|
||||
|
|
|
@ -71,8 +71,13 @@ AlexNet由5个卷积层和3个全连接层组成。多个卷积核用于提取
|
|||
```python
|
||||
# 进入脚本目录,训练AlexNet
|
||||
bash run_standalone_train_ascend.sh [DATA_PATH] [CKPT_SAVE_PATH]
|
||||
# example: bash run_standalone_train_ascend.sh /home/DataSet/Cifar10/cifar-10-batches-bin/ /home/model/alexnet/ckpt/
|
||||
|
||||
# 分布式训练AlexNet
|
||||
|
||||
# 进入脚本目录,评估AlexNet
|
||||
bash run_standalone_eval_ascend.sh [DATA_PATH] [CKPT_NAME]
|
||||
# example: bash run_standalone_eval_ascend.sh /home/DataSet/cifar10/cifar-10-verify-bin /home/model/cv/alxnet/ckpt/checkpoint_alexnet-1_1562.ckpt
|
||||
```
|
||||
|
||||
- 在 ModelArts 进行训练 (如果你想在modelarts上运行,可以参考以下文档 [modelarts](https://support.huaweicloud.com/modelarts/))
|
||||
|
@ -216,7 +221,7 @@ train.py和config.py中主要参数如下:
|
|||
```bash
|
||||
python train.py --config_path default_config.yaml --data_path cifar-10-batches-bin --ckpt_path ckpt > log 2>&1 &
|
||||
# 或进入脚本目录,执行脚本
|
||||
bash run_standalone_train_ascend.sh cifar-10-batches-bin ckpt
|
||||
bash run_standalone_train_ascend.sh /home/DataSet/Cifar10/cifar-10-batches-bin/ /home/model/alexnet/ckpt/
|
||||
```
|
||||
|
||||
经过训练后,损失值如下:
|
||||
|
@ -263,7 +268,7 @@ train.py和config.py中主要参数如下:
|
|||
```bash
|
||||
python eval.py --config_path default_config.yaml --data_path cifar-10-verify-bin --ckpt_path ckpt/checkpoint_alexnet-1_1562.ckpt > eval_log.txt 2>&1 &
|
||||
#或进入脚本目录,执行脚本
|
||||
bash run_standalone_eval_ascend.sh cifar-10-verify-bin ckpt/checkpoint_alexnet-1_1562.ckpt
|
||||
bash run_standalone_eval_ascend.sh /home/DataSet/cifar10/cifar-10-verify-bin /home/model/cv/alxnet/ckpt/checkpoint_alexnet-1_1562.ckpt
|
||||
```
|
||||
|
||||
可通过"eval_log”文件查看结果。测试数据集的准确率如下:
|
||||
|
|
|
@ -78,9 +78,8 @@ After installing MindSpore via the official website, you can start training and
|
|||
|
||||
```python
|
||||
# enter script dir, train CNNDirectionModel
|
||||
bash run_standalone_train.sh [DATASET_PATH] [PRETRAINED_CKPT_PATH](optional)
|
||||
# enter script dir, evaluate CNNDirectionModel
|
||||
bash run_standalone_train.sh [DATASET_PATH] [PRETRAINED_CKPT_PATH]
|
||||
bash scripts/run_standalone_train.sh [DEVICE_ID] [DATASET_PATH] [PRETRAINED_CKPT_PATH](optional)
|
||||
# example: bash run_standalone_train_ascend.sh 0 /home/DataSet/FSNS/train/
|
||||
```
|
||||
|
||||
# [Script Description](#contents)
|
||||
|
@ -96,8 +95,11 @@ bash run_standalone_train.sh [DATASET_PATH] [PRETRAINED_CKPT_PATH]
|
|||
├── scripts
|
||||
│ ├──run_distribute_train_ascend.sh // distributed training in ascend
|
||||
│ ├──run_standalone_eval_ascend.sh // evaluate in ascend
|
||||
│ ├──run_distribute_train_gpu.sh // distributed training in gpu
|
||||
│ ├──run_standalone_eval_gpush // evaluate in gpu
|
||||
│ ├──run_eval.sh // shell script for evaluation on Ascend
|
||||
│ ├──run_standalone_train_ascend.sh // train standalone in ascend
|
||||
│ ├──run_standalone_train_gpu.sh // train standalone in gpu
|
||||
├── src
|
||||
│ ├──dataset.py // creating dataset
|
||||
│ ├──cnn_direction_model.py // cnn_direction_model architecture
|
||||
|
@ -138,6 +140,7 @@ Major parameters in default_config.yaml as follows:
|
|||
|
||||
```python
|
||||
bash scripts/run_standalone_train_ascend.sh device_id path-to-train-mindrecords pre-trained-chkpt(optional)
|
||||
# example: bash run_standalone_train_ascend.sh 0 /home/DataSet/FSNS/train/
|
||||
```
|
||||
|
||||
The model checkpoint will be saved script/train.
|
||||
|
@ -152,6 +155,7 @@ Before running the command below, please check the checkpoint path used for eval
|
|||
|
||||
```python
|
||||
bash scripts/run_standalone_eval_ascend.sh device_id path-to-test-mindrecords trained-chkpt-path
|
||||
# example: bash run_standalone_eval_ascend.sh /home/DataSet/FSNS/test/ /home/model/cnn_direction_model/ckpt/cnn_direction_model-10_32644.ckpt
|
||||
```
|
||||
|
||||
Results of evaluation will be printed after evaluation process is completed.
|
||||
|
@ -175,7 +179,7 @@ bash scripts/run_distribute_train_ascend.sh [rank_table] [train_dataset_path] [P
|
|||
For example, you can run the shell command below to launch the training procedure.
|
||||
|
||||
```shell
|
||||
bash scripts/run_distribute_train_ascend.sh /home/rank_table.json /home/fsns/train/
|
||||
bash run_distribute_train_ascend.sh /home/hccl_8p_01234567_10.155.170.71.json /home/DataSet/FSNS/train/
|
||||
```
|
||||
|
||||
- running on ModelArts
|
||||
|
|
|
@ -121,34 +121,46 @@ pip install tqdm
|
|||
pip install six
|
||||
```
|
||||
|
||||
```default_config.yaml
|
||||
|
||||
TRAIN_DATASET_PATH: /home/DataSet/MJ-ST-IIIT/ST-MJ/
|
||||
TRAIN_DATASET_INDEX_PATH: /home/DataSet/MJ-ST-IIIT/st_mj_fixed_length_index_list.pkl
|
||||
TEST_DATASET_PATH: /home/DataSet/MJ-ST-IIIT/IIIT5K_3000
|
||||
|
||||
Modify the parameters according to the actual path
|
||||
```
|
||||
|
||||
- Standalone Ascend Training:
|
||||
|
||||
```bash
|
||||
bash scripts/run_standalone_train_ascend.sh $PRETRAINED_CKPT
|
||||
bash scripts/run_standalone_train_ascend.sh $DEVICE_ID $PRETRAINED_CKPT(options)
|
||||
# example: bash scripts/run_standalone_train_ascend.sh 0
|
||||
```
|
||||
|
||||
- Standalone GPU Training:
|
||||
|
||||
```bash
|
||||
bash scripts/run_standalone_train_gpu.sh $PRETRAINED_CKPT
|
||||
bash scripts/run_standalone_train_gpu.sh $PRETRAINED_CKPT(options)
|
||||
```
|
||||
|
||||
- Distributed Ascend Training:
|
||||
|
||||
```bash
|
||||
bash scripts/run_distribute_train_ascend.sh $RANK_TABLE_FILE $PRETRAINED_CKPT
|
||||
bash scripts/run_distribute_train_ascend.sh $RANK_TABLE_FILE $PRETRAINED_CKPT(options)
|
||||
# example: bash scripts/run_distribute_train_ascend.sh /home/hccl_8p_01234567_10.155.170.71.json
|
||||
```
|
||||
|
||||
- Distributed GPU Training:
|
||||
|
||||
```bash
|
||||
bash scripts/run_distribute_train_gpu.sh $PRETRAINED_CKPT
|
||||
bash scripts/run_distribute_train_gpu.sh $PRETRAINED_CKPT(options)
|
||||
```
|
||||
|
||||
- Ascend Evaluation:
|
||||
|
||||
```bash
|
||||
bash scripts/run_eval_ascend.sh $TRAINED_CKPT
|
||||
bash scripts/run_eval_ascend.sh $DEVICE_ID $TRAINED_CKPT
|
||||
# example: scripts/run_eval_ascend.sh 0 /home/model/cnnctc/ckpt/CNNCTC-1_8000.ckpt
|
||||
```
|
||||
|
||||
- GPU Evaluation:
|
||||
|
@ -231,6 +243,7 @@ Arguments:
|
|||
|
||||
```bash
|
||||
bash scripts/run_standalone_train_ascend.sh [DEVICE_ID] [PRETRAINED_CKPT(options)]
|
||||
# example: bash scripts/run_standalone_train_ascend.sh 0
|
||||
```
|
||||
|
||||
Results and checkpoints are written to `./train` folder. Log can be found in `./train/log` and loss values are recorded in `./train/loss.log`.
|
||||
|
@ -241,6 +254,7 @@ Results and checkpoints are written to `./train` folder. Log can be found in `./
|
|||
|
||||
```bash
|
||||
bash scripts/run_distribute_train_ascend.sh [RANK_TABLE_FILE] [PRETRAINED_CKPT(options)]
|
||||
# example: bash scripts/run_distribute_train_ascend.sh /home/hccl_8p_01234567_10.155.170.71.json
|
||||
```
|
||||
|
||||
For distributed training, a hccl configuration file with JSON format needs to be created in advance.
|
||||
|
@ -361,6 +375,7 @@ Log can be found in `./train_parallel/log` and loss values are recorded in `./tr
|
|||
|
||||
```bash
|
||||
bash scripts/run_eval_ascend.sh [DEVICE_ID] [TRAINED_CKPT]
|
||||
# example: scripts/run_eval_ascend.sh 0 /home/model/cnnctc/ckpt/CNNCTC-1_8000.ckpt
|
||||
```
|
||||
|
||||
The model will be evaluated on the IIIT dataset, sample results and overall accuracy will be printed.
|
||||
|
|
|
@ -124,22 +124,34 @@ pip install tqdm
|
|||
pip install six
|
||||
```
|
||||
|
||||
```default_config.yaml
|
||||
|
||||
TRAIN_DATASET_PATH: /home/DataSet/MJ-ST-IIIT/ST-MJ/
|
||||
TRAIN_DATASET_INDEX_PATH: /home/DataSet/MJ-ST-IIIT/st_mj_fixed_length_index_list.pkl
|
||||
TEST_DATASET_PATH: /home/DataSet/MJ-ST-IIIT/IIIT5K_3000
|
||||
|
||||
根据实际路径修改参数
|
||||
```
|
||||
|
||||
- 单机训练:
|
||||
|
||||
```shell
|
||||
bash scripts/run_standalone_train_ascend.sh [DEVICE_ID] [PRETRAINED_CKPT(options)]
|
||||
# example: bash scripts/run_standalone_train_ascend.sh 0
|
||||
```
|
||||
|
||||
- 分布式训练:
|
||||
|
||||
```shell
|
||||
bash scripts/run_distribute_train_ascend.sh [RANK_TABLE_FILE] [PRETRAINED_CKPT(options)]
|
||||
# example: bash scripts/run_distribute_train_ascend.sh /home/hccl_8p_01234567_10.155.170.71.json
|
||||
```
|
||||
|
||||
- 评估:
|
||||
|
||||
```shell
|
||||
bash scripts/run_eval_ascend.sh DEVICE_ID TRAINED_CKPT
|
||||
# example: scripts/run_eval_ascend.sh 0 /home/model/cnnctc/ckpt/CNNCTC-1_8000.ckpt
|
||||
```
|
||||
|
||||
# 脚本说明
|
||||
|
@ -216,6 +228,7 @@ bash scripts/run_eval_ascend.sh DEVICE_ID TRAINED_CKPT
|
|||
|
||||
```shell
|
||||
bash scripts/run_standalone_train_ascend.sh [DEVICE_ID] [PRETRAINED_CKPT(options)]
|
||||
# example: bash scripts/run_standalone_train_ascend.sh 0
|
||||
```
|
||||
|
||||
结果和检查点被写入`./train`文件夹。日志可以在`./train/log`中找到,损失值记录在`./train/loss.log`中。
|
||||
|
@ -226,6 +239,7 @@ bash scripts/run_standalone_train_ascend.sh [DEVICE_ID] [PRETRAINED_CKPT(options
|
|||
|
||||
```shell
|
||||
bash scripts/run_distribute_train_ascend.sh [RANK_TABLE_FILE] [PRETRAINED_CKPT(options)]
|
||||
# example: bash scripts/run_distribute_train_ascend.sh /home/hccl_8p_01234567_10.155.170.71.json
|
||||
```
|
||||
|
||||
结果和检查点分别写入设备`i`的`./train_parallel_{i}`文件夹。
|
||||
|
@ -271,6 +285,7 @@ epoch: 1 step: 8698 , loss is 9.708542263610315, average time per step is 0.3184
|
|||
|
||||
```shell
|
||||
bash scripts/run_eval_ascend.sh [DEVICE_ID] [TRAINED_CKPT]
|
||||
# example: scripts/run_eval_ascend.sh 0 /home/model/cnnctc/ckpt/CNNCTC-1_8000.ckpt
|
||||
```
|
||||
|
||||
在IIIT数据集上评估模型,并打印样本结果和总准确率。
|
||||
|
|
|
@ -69,7 +69,6 @@ Here we used 6 datasets for training, and 1 datasets for Evaluation.
|
|||
## [Script and sample code](#contents)
|
||||
|
||||
```shell
|
||||
.
|
||||
└─ctpn
|
||||
├── README.md # network readme
|
||||
├──ascend310_infer #application for 310 inference
|
||||
|
@ -161,13 +160,28 @@ python src/create_dataset.py
|
|||
|
||||
- Ascend:
|
||||
|
||||
```default_config.yaml
|
||||
if pretraining set pretraining_dataset_file: /home/DataSet/ctpn_dataset/pretrain/ctpn_pretrain.mindrecord0
|
||||
if finetune set pretraining_dataset_file: /home/DataSet/ctpn_dataset/finetune/ctpn_finetune.mindrecord0
|
||||
img_dir:/home/DataSet/ctpn_dataset/ICDAR2013/test
|
||||
|
||||
Modify the parameters according to the actual path
|
||||
```
|
||||
|
||||
```bash
|
||||
# distribute training example(8p)
|
||||
bash run_distribute_train_ascend.sh [RANK_TABLE_FILE] [TASK_TYPE] [PRETRAINED_PATH]
|
||||
# distribute training
|
||||
bash scripts/run_distribute_train_ascend.sh [RANK_TABLE_FILE] [TASK_TYPE] [PRETRAINED_PATH]
|
||||
# example: bash scripts/run_distribute_train_ascend.sh /home/hccl_8p_01234567_10.155.170.71.json Pretraining(or Finetune) \
|
||||
# /home/DataSet/ctpn_dataset/backbone/0-150_5004.ckpt
|
||||
|
||||
# standalone training
|
||||
bash run_standalone_train_ascend.sh [TASK_TYPE] [PRETRAINED_PATH]
|
||||
bash scrpits/run_standalone_train_ascend.sh [TASK_TYPE] [PRETRAINED_PATH] [DEVICE_ID]
|
||||
example: bash scrpits/run_standalone_train_ascend.sh Pretraining(or Finetune) /home/DataSet/ctpn_dataset/backbone/0-150_5004.ckpt 0
|
||||
|
||||
# evaluation:
|
||||
bash run_eval_ascend.sh [IMAGE_PATH] [DATASET_PATH] [CHECKPOINT_PATH]
|
||||
bash scripts/run_eval_ascend.sh [IMAGE_PATH] [DATASET_PATH] [CHECKPOINT_PATH]
|
||||
# example: bash script/run_eval_ascend.sh /home/DataSet/ctpn_dataset/ICDAR2013/test \
|
||||
# /home/DataSet/ctpn_dataset/ctpn_final_dataset/test/ctpn_test.mindrecord /home/model/cv/ctpn/train_parallel0/ckpt_0/
|
||||
```
|
||||
|
||||
The `pretrained_path` should be a checkpoint of vgg16 trained on Imagenet2012. The name of weight in dict should be totally the same, also the batch_norm should be enabled in the trainig of vgg16, otherwise fails in further steps.COCO_TEXT_PARSER_PATH coco_text.py can refer to [Link](https://github.com/andreasveit/coco-text).To get the vgg16 backbone, you can use the network structure defined in src/CTPN/vgg16.py.To train the backbone, copy the src/CTPN/vgg16.py under modelzoo/official/cv/vgg16/src/, and modify the vgg16/train.py to suit the new construction.You can fix it as below:
|
||||
|
@ -212,8 +226,11 @@ ICDAR2013, SCUT-FORU to improve precision and recall, and when doing Finetune, w
|
|||
Ascend:
|
||||
# distribute training example(8p)
|
||||
bash run_distribute_train_ascend.sh [RANK_TABLE_FILE] [TASK_TYPE] [PRETRAINED_PATH]
|
||||
# example: bash scripts/run_distribute_train_ascend.sh /home/hccl_8p_01234567_10.155.170.71.json Pretraining(or Finetune) /home/DataSet/ctpn_dataset/backbone/0-150_5004.ckpt
|
||||
|
||||
# standalone training
|
||||
bash run_standalone_train_ascend.sh [TASK_TYPE] [PRETRAINED_PATH]
|
||||
# example: bash scrpits/run_standalone_train_ascend.sh Pretraining(or Finetune) /home/DataSet/ctpn_dataset/backbone/0-150_5004.ckpt 0
|
||||
```
|
||||
|
||||
### Result
|
||||
|
@ -295,6 +312,7 @@ You can start training using python or shell scripts. The usage of shell scripts
|
|||
|
||||
```bash
|
||||
bash run_eval_ascend.sh [IMAGE_PATH] [DATASET_PATH] [CHECKPOINT_PATH]
|
||||
# example: bash script/run_eval_ascend.sh /home/DataSet/ctpn_dataset/ICDAR2013/test /home/DataSet/ctpn_dataset/ctpn_final_dataset/test/ctpn_test.mindrecord /home/model/cv/ctpn/train_parallel0/ckpt_0/
|
||||
```
|
||||
|
||||
After eval, you can get serval archive file named submit_ctpn-xx_xxxx.zip, which contains the name of your checkpoint file.To evalulate it, you can use the scripts provided by the ICDAR2013 network, you can download the Deteval scripts from the [link](https://rrc.cvc.uab.es/?com=downloads&action=download&ch=2&f=aHR0cHM6Ly9ycmMuY3ZjLnVhYi5lcy9zdGFuZGFsb25lcy9zY3JpcHRfdGVzdF9jaDJfdDFfZTItMTU3Nzk4MzA2Ny56aXA=)
|
||||
|
|
|
@ -122,6 +122,11 @@ Based on original DeepLabV3 paper, we reproduce two training experiments on voca
|
|||
For single device training, please config parameters, training script is:
|
||||
|
||||
```shell
|
||||
Enter the shell script to modify the data_file and ckpt_pre_trained parameters
|
||||
# example:
|
||||
data_file=/home/DataSet/VOC2012/vocaug_mindrecords/vocaug.mindrecord0
|
||||
ckpt_pre_trained=/home/model/deeplabv3/predtrained/resnet101_ascend_v120_imagenet2012_official_cv_bs32_acc78.ckpt
|
||||
|
||||
bash run_standalone_train.sh
|
||||
```
|
||||
|
||||
|
@ -129,47 +134,72 @@ bash run_standalone_train.sh
|
|||
|
||||
1. Train s16 with vocaug dataset, finetuning from resnet101 pretrained model, script is:
|
||||
|
||||
```shell
|
||||
bash run_distribute_train_s16_r1.sh
|
||||
```
|
||||
```shell
|
||||
Enter the shell script to modify the data_file and ckpt_pre_trained parameters
|
||||
# example:
|
||||
data_file=/home/DataSet/VOC2012/vocaug_mindrecords/vocaug.mindrecord0
|
||||
ckpt_pre_trained=/home/model/deeplabv3/predtrained/resnet101_ascend_v120_imagenet2012_official_cv_bs32_acc78.ckpt
|
||||
|
||||
bash run_distribute_train_s16_r1.sh
|
||||
```
|
||||
|
||||
2. Train s8 with vocaug dataset, finetuning from model in previous step, training script is:
|
||||
|
||||
```shell
|
||||
bash run_distribute_train_s8_r1.sh
|
||||
```
|
||||
```shell
|
||||
Enter the shell script to modify the data_file and ckpt_pre_trained parameters
|
||||
# example:
|
||||
data_file=/home/DataSet/VOC2012/vocaug_mindrecords/vocaug.mindrecord0
|
||||
ckpt_pre_trained=/home/model/deeplabv3/predtrained/resnet101_ascend_v120_imagenet2012_official_cv_bs32_acc78.ckpt
|
||||
|
||||
bash run_distribute_train_s8_r1.sh
|
||||
```
|
||||
|
||||
3. Train s8 with voctrain dataset, finetuning from model in previous step, training script is:
|
||||
|
||||
```shell
|
||||
bash run_distribute_train_s8_r2.sh
|
||||
```
|
||||
```shell
|
||||
Enter the shell script to modify the data_file and ckpt_pre_trained parameters
|
||||
Note: This training pre-training weight uses the weight file of the previous training, and the data set has also changed
|
||||
# example:
|
||||
data_file=/home/DataSet/VOC2012/voctrain_mindrecords/votrain.mindrecord0
|
||||
ckpt_pre_trained=/home/model/deeplabv3/ckpt/deeplabv3-800_330.ckpt
|
||||
|
||||
|
||||
bash run_distribute_train_s8_r2.sh
|
||||
```
|
||||
|
||||
- For evaluation, evaluating steps are as follows:
|
||||
|
||||
1. Eval s16 with voc val dataset, eval script is:
|
||||
1. Enter the shell script to modify the data_file and ckpt_pre_trained parameters
|
||||
|
||||
```shell
|
||||
bash run_eval_s16.sh
|
||||
```
|
||||
```default_config.yaml
|
||||
# example:
|
||||
data_root=/home/DataSet/VOC2012
|
||||
data_lst=/home/DataSet/VOC2012/voc_val_lst.txt
|
||||
```
|
||||
|
||||
2. Eval s8 with voc val dataset, eval script is:
|
||||
2. Eval s16 with voc val dataset, eval script is:
|
||||
|
||||
```shell
|
||||
bash run_eval_s8.sh
|
||||
```
|
||||
```shell
|
||||
bash run_eval_s16.sh
|
||||
```
|
||||
|
||||
3. Eval s8 multiscale with voc val dataset, eval script is:
|
||||
3. Eval s8 with voc val dataset, eval script is:
|
||||
|
||||
```shell
|
||||
bash run_eval_s8_multiscale.sh
|
||||
```
|
||||
```shell
|
||||
bash run_eval_s8.sh
|
||||
```
|
||||
|
||||
4. Eval s8 multiscale and flip with voc val dataset, eval script is:
|
||||
4. Eval s8 multiscale with voc val dataset, eval script is:
|
||||
|
||||
```shell
|
||||
bash run_eval_s8_multiscale_flip.sh
|
||||
```
|
||||
```shell
|
||||
bash run_eval_s8_multiscale.sh
|
||||
```
|
||||
|
||||
5. Eval s8 multiscale and flip with voc val dataset, eval script is:
|
||||
|
||||
```shell
|
||||
bash run_eval_s8_multiscale_flip.sh
|
||||
```
|
||||
|
||||
- Train on ModelArts (If you want to run in modelarts, please check the official documentation of [modelarts](https://support.huaweicloud.com/modelarts/), and you can start training as follows)
|
||||
|
||||
|
|
|
@ -126,54 +126,83 @@ Pascal VOC数据集和语义边界数据集(Semantic Boundaries Dataset,SBD
|
|||
运行以下训练脚本配置单卡训练参数:
|
||||
|
||||
```bash
|
||||
run_standalone_train.sh
|
||||
进入shell脚本修改 data_file和ckpt_pre_trained参数
|
||||
# example:
|
||||
data_file=/home/DataSet/VOC2012/vocaug_mindrecords/vocaug.mindrecord0
|
||||
ckpt_pre_trained=/home/model/deeplabv3/predtrained/resnet101_ascend_v120_imagenet2012_official_cv_bs32_acc78.ckpt
|
||||
|
||||
bash run_standalone_train.sh
|
||||
```
|
||||
|
||||
- 按照以下训练步骤进行8卡训练:
|
||||
|
||||
1. 使用VOCaug数据集训练s16,微调ResNet-101预训练模型。脚本如下:
|
||||
|
||||
```bash
|
||||
run_distribute_train_s16_r1.sh
|
||||
```
|
||||
```bash
|
||||
进入shell脚本修改 data_file和ckpt_pre_trained参数
|
||||
# example:
|
||||
data_file=/home/DataSet/VOC2012/vocaug_mindrecords/vocaug.mindrecord0
|
||||
ckpt_pre_trained=/home/model/deeplabv3/predtrained/resnet101_ascend_v120_imagenet2012_official_cv_bs32_acc78.ckpt
|
||||
|
||||
bash run_distribute_train_s16_r1.sh
|
||||
```
|
||||
|
||||
2. 使用VOCaug数据集训练s8,微调上一步的模型。脚本如下:
|
||||
|
||||
```bash
|
||||
run_distribute_train_s8_r1.sh
|
||||
```
|
||||
```bash
|
||||
进入shell脚本修改 data_file和ckpt_pre_trained参数
|
||||
# example:
|
||||
data_file=/home/DataSet/VOC2012/vocaug_mindrecords/vocaug.mindrecord0
|
||||
ckpt_pre_trained=/home/model/deeplabv3/predtrained/resnet101_ascend_v120_imagenet2012_official_cv_bs32_acc78.ckpt
|
||||
|
||||
bash run_distribute_train_s8_r1.sh
|
||||
```
|
||||
|
||||
3. 使用VOCtrain数据集训练s8,微调上一步的模型。脚本如下:
|
||||
|
||||
```bash
|
||||
run_distribute_train_s8_r2.sh
|
||||
```
|
||||
```bash
|
||||
进入shell脚本修改 data_file和ckpt_pre_trained参数
|
||||
注意:本次训练预训练权重采用上一步训练的权重文件,数据集也有所变化
|
||||
# example:
|
||||
data_file=/home/DataSet/VOC2012/voctrain_mindrecords/votrain.mindrecord0
|
||||
ckpt_pre_trained=/home/model/deeplabv3/ckpt/deeplabv3-800_330.ckpt
|
||||
|
||||
bash run_distribute_train_s8_r2.sh
|
||||
```
|
||||
|
||||
- 评估步骤如下:
|
||||
|
||||
1. 使用voc val数据集评估s16。评估脚本如下:
|
||||
1. 进入对应的shell脚本修改参数
|
||||
|
||||
```bash
|
||||
run_eval_s16.sh
|
||||
```
|
||||
```default_config.yaml
|
||||
# example:
|
||||
data_root=/home/DataSet/VOC2012
|
||||
data_lst=/home/DataSet/VOC2012/voc_val_lst.txt
|
||||
```
|
||||
|
||||
2. 使用voc val数据集评估s8。评估脚本如下:
|
||||
2. 使用voc val数据集评估s16。评估脚本如下:
|
||||
|
||||
```bash
|
||||
run_eval_s8.sh
|
||||
```
|
||||
```bash
|
||||
bash run_eval_s16.sh
|
||||
```
|
||||
|
||||
3. 使用voc val数据集评估多尺度s8。评估脚本如下:
|
||||
3. 使用voc val数据集评估s8。评估脚本如下:
|
||||
|
||||
```bash
|
||||
run_eval_s8_multiscale.sh
|
||||
```
|
||||
```bash
|
||||
bash run_eval_s8.sh
|
||||
```
|
||||
|
||||
4. 使用voc val数据集评估多尺度和翻转s8。评估脚本如下:
|
||||
4. 使用voc val数据集评估多尺度s8。评估脚本如下:
|
||||
|
||||
```bash
|
||||
run_eval_s8_multiscale_flip.sh
|
||||
```
|
||||
```bash
|
||||
bash run_eval_s8_multiscale.sh
|
||||
```
|
||||
|
||||
5. 使用voc val数据集评估多尺度和翻转s8。评估脚本如下:
|
||||
|
||||
```bash
|
||||
bash run_eval_s8_multiscale_flip.sh
|
||||
```
|
||||
|
||||
- 在 ModelArts 进行训练 (如果你想在modelarts上运行,可以参考以下文档 [modelarts](https://support.huaweicloud.com/modelarts/))
|
||||
|
||||
|
|
|
@ -53,6 +53,10 @@ Pascal VOC数据集和语义边界数据集(Semantic Boundaries Dataset,SBD
|
|||
|
||||
- 下载分段数据集。
|
||||
|
||||
- 准备Backbone模型
|
||||
|
||||
准备resnet101模型,点此下载(https://download.mindspore.cn/model_zoo/r1.2/resnet101_ascend_v120_imagenet2012_official_cv_bs32_acc78/resnet101_ascend_v120_imagenet2012_official_cv_bs32_acc78.ckpt).
|
||||
|
||||
- 准备训练数据清单文件。清单文件用于保存图片和标注对的相对路径。如下:
|
||||
|
||||
```text
|
||||
|
@ -112,7 +116,12 @@ Pascal VOC数据集和语义边界数据集(Semantic Boundaries Dataset,SBD
|
|||
运行以下训练脚本配置单卡训练参数:
|
||||
|
||||
```bash
|
||||
run_alone_train.sh
|
||||
进入shell脚本修改 data_file和ckpt_pre_trained参数
|
||||
# example:
|
||||
data_file=/home/DataSet/VOC2012/vocaug_mindrecords/vocaug.mindrecord0
|
||||
ckpt_pre_trained=/home/model/deeplabv3/predtrained/resnet101_ascend_v120_imagenet2012_official_cv_bs32_acc78.ckpt
|
||||
|
||||
bash run_alone_train.sh
|
||||
```
|
||||
|
||||
按照以下训练步骤进行8卡训练:
|
||||
|
@ -120,57 +129,81 @@ run_alone_train.sh
|
|||
1.使用VOCaug数据集训练s16,微调ResNet-101预训练模型。脚本如下:
|
||||
|
||||
```bash
|
||||
run_distribute_train_s16_r1.sh
|
||||
进入shell脚本修改 data_file和ckpt_pre_trained参数
|
||||
# example:
|
||||
data_file=/home/DataSet/VOC2012/vocaug_mindrecords/vocaug.mindrecord0
|
||||
ckpt_pre_trained=/home/model/deeplabv3/predtrained/resnet101_ascend_v120_imagenet2012_official_cv_bs32_acc78.ckpt
|
||||
|
||||
bash run_distribute_train_s16_r1.sh
|
||||
```
|
||||
|
||||
2.使用VOCaug数据集训练s8,微调上一步的模型。脚本如下:
|
||||
|
||||
```bash
|
||||
run_distribute_train_s8_r1.sh
|
||||
进入shell脚本修改 data_file和ckpt_pre_trained参数
|
||||
# example:
|
||||
data_file=/home/DataSet/VOC2012/vocaug_mindrecords/vocaug.mindrecord0
|
||||
ckpt_pre_trained=/home/model/deeplabv3/predtrained/resnet101_ascend_v120_imagenet2012_official_cv_bs32_acc78.ckpt
|
||||
|
||||
bash run_distribute_train_s8_r1.sh
|
||||
```
|
||||
|
||||
3.使用VOCtrain数据集训练s8,微调上一步的模型。脚本如下:
|
||||
|
||||
```bash
|
||||
进入shell脚本修改 data_file和ckpt_pre_trained参数
|
||||
注意:本次训练预训练权重采用上一步训练的权重文件,数据集也有所变化
|
||||
# example:
|
||||
data_file=/home/DataSet/VOC2012/voctrain_mindrecords/votrain.mindrecord0
|
||||
ckpt_pre_trained=/home/model/deeplabv3/ckpt/deeplabv3-800_330.ckpt
|
||||
|
||||
run_distribute_train_s8_r2.sh
|
||||
```
|
||||
|
||||
评估步骤如下:
|
||||
|
||||
1.使用voc val数据集评估s16。评估脚本如下:
|
||||
1. 进入对应的shell脚本修改参数
|
||||
|
||||
```bash
|
||||
run_eval_s16.sh
|
||||
```default_cofig.yaml
|
||||
# example:
|
||||
data_root=/home/DataSet/VOC2012
|
||||
data_lst=/home/DataSet/VOC2012/voc_val_lst.txt
|
||||
```
|
||||
|
||||
2.使用voc val数据集评估多尺度s16。评估脚本如下:
|
||||
2.使用voc val数据集评估s16。评估脚本如下:
|
||||
|
||||
```bash
|
||||
run_eval_s16_multiscale.sh
|
||||
bash run_eval_s16.sh
|
||||
```
|
||||
|
||||
3.使用voc val数据集评估多尺度和翻转s16。评估脚本如下:
|
||||
3.使用voc val数据集评估多尺度s16。评估脚本如下:
|
||||
|
||||
```bash
|
||||
run_eval_s16_multiscale_flip.sh
|
||||
bash run_eval_s16_multiscale.sh
|
||||
```
|
||||
|
||||
4.使用voc val数据集评估s8。评估脚本如下:
|
||||
4.使用voc val数据集评估多尺度和翻转s16。评估脚本如下:
|
||||
|
||||
```bash
|
||||
run_eval_s8.sh
|
||||
bash run_eval_s16_multiscale_flip.sh
|
||||
```
|
||||
|
||||
5.使用voc val数据集评估多尺度s8。评估脚本如下:
|
||||
5.使用voc val数据集评估s8。评估脚本如下:
|
||||
|
||||
```bash
|
||||
run_eval_s8_multiscale.sh
|
||||
bash run_eval_s8.sh
|
||||
```
|
||||
|
||||
6.使用voc val数据集评估多尺度和翻转s8。评估脚本如下:
|
||||
6.使用voc val数据集评估多尺度s8。评估脚本如下:
|
||||
|
||||
```bash
|
||||
run_eval_s8_multiscale_flip.sh
|
||||
bash run_eval_s8_multiscale.sh
|
||||
```
|
||||
|
||||
7.使用voc val数据集评估多尺度和翻转s8。评估脚本如下:
|
||||
|
||||
```bash
|
||||
bash run_eval_s8_multiscale_flip.sh
|
||||
```
|
||||
|
||||
# 脚本说明
|
||||
|
|
Loading…
Reference in New Issue