!22039 Add Spnas to Model_zoo

Merge pull request !22039 from 185******25/master

model_zoo/research/cv/Spnas/README.md

@@ -0,0 +1,139 @@
+# SP-NAS (Serial-to-Parallel Backbone Search for Object Detection)
+
+## Algorithm Introduction
+
+SP-NAS is an efficient architecture search algorithm for object detection and semantic segmentation based on the backbone network architecture. The existing object detectors usually use the feature extraction network designed and pre-trained on the image classification task as the backbone. We propose an efficient, flexible and task-oriented search scheme based on NAS. which is a two-phase search solution from serial to parallel to reduce repeated ImageNet pre-training or long-time training from scratch.
+
+## Algorithm Principles
+
+This method has two phases:
+
+1. In serial phase, the block sequence with optimal scaling ratio and output channel is found by using the "swap-expand-reignite" search policy. This search policy can guranteen a new searched architecture to completely inherit of weight from arichtectures before morphism.
+2. In parallel phase, parallized network structures are designed, sub-networks integrated by different feature layers are searched to better fuse the high-level and low-level semantic features. The following figure shows the search policy.
+
+![sp-nas](./image/sp_nas.png)
+
+## Search Space and Search Policy
+
+**Serial-level**
+
+- Swap-expand-reignite policy:  Growing starts from a small network to avoid repeated ImageNet pre-training.
+  - The new candidate network is obtained by "switching" or "expanding" the grown network for many times.
+  - Quickly train and evaluate candidate networks based on inherited parameters.
+  - When the growth reaches the bottleneck, the network is re-trained using ImageNet. The number of ignition times is no more than 2.
+
+- Constrained optimal network: A serial network with limited network resources (latency, video memory usage, or complexity) is selected to obtain the maximum performance.
+
+- Search space configuration:
+  - Block type: Basic Block, BottleNeck Block, and ResNext;
+  - Network depth: 8 to 60 blocks;
+  - Number of stages: 5 to 7;
+  - Width: Position where the channel size is doubled in the entire sequence.
+
+**Parallel-level**
+
+- Based on the result SerialNet from the serial search phase (or the existing handcraft serial network such as ResNet series), search for the parallel structure stacked on SerialNet to better utilize and fuse feature information with different resolutions from different feature layers.
+- Search policy: Random sampling to meet the resource constraints: The probability of adding additional subnets is inversely proportional to the FLOPS of the subnets to be added.
+- Search space: SerialNet is divided into L self-networks based on the number of feature layers and K sub-networks are searched for in each phase.
+
+## Dataset
+
+The benchmark datasets can be downloaded as follows:
+
+ COCO,
+[COCO2017](https://cocodataset.org/#download),
+
+
+## Requirements
+
+### Hardware (Ascend)
+
+> Prepare hardware environment with Ascend.
+
+### Framework
+
+> [MindSpore](https://www.mindspore.cn/install/en)
+
+### For more information, please check the resources below
+
+[MindSpore Tutorials](https://www.mindspore.cn/tutorials/en/r1.3/index.html)
+[MindSpore Python API](https://www.mindspore.cn/docs/api/en/r1.3/index.html)
+
+## Script Description
+
+### Scripts and Sample Code
+
+```bash
+Spnas
+├── eval.py # inference entry
+├── train.py # pre-training entry
+├── image
+│   └── spnas.png # the illustration of Spnas network
+├── readme.md # Readme
+├── scripts
+│   ├── run_distributed.sh # pre-training script for all tasks
+└── src
+    ├── spnas.yml # options/hyper-parameters of Spnas
+    └── spnas_distributed.yml # options/hyper-parameters of Spnas
+
+```
+
+### Script Parameter
+
+> For details about hyperparameters, see src/spnas.yml.
+
+## Training Process
+
+### For training
+
+```bash
+python3 train.py
+```
+
+> Or one can run following script for all tasks.
+
+```bash
+sh scripts/run_distributed.sh
+```
+
+## Evaluation
+
+### Evaluation Process
+
+> Inference example:
+
+Modify src/eval.yml:
+```bash
+models_folder: [CHECKPOINT_PATH]
+```
+
+```bash
+python3 eval.py
+```
+
+### Evaluation Result
+
+The result are evaluated by the value of AP (mean Average Precision), and the format is as following.
+
+```bash
+current valid perfs [mAP: 49.1, AP50: 67.1, AP_small: 31.0, AP_medium: 52.6, AP_large: 63.7]
+```
+
+## Performance
+
+### Inference Performance
+
+The Results on detection tasks are listed as below.
+
+COCO results:
+
+| Method | mAP | AP50 | AP_small | AP_medium | AP_large |
+| ----- | ----- | ----- | ----- | ----- | ----- |
+| **SPNet** | **49.1** | **67.1** | **31.0** | **52.6** | **63.7** |
+| AmoebaNet | 43.4 | - | - | - | - |
+| NAS-FPN | 48.0 | - | - | - | - |
+
+
+## ModeZoo Homepage
+
+Please check the official [homepage](https://gitee.com/mindspore/mindspore/tree/master/model_zoo).

model_zoo/research/cv/Spnas/eval.py

@@ -0,0 +1,20 @@
+# Copyright 2021 Huawei Technologies Co., Ltd
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+# ============================================================================
+"""eval script"""
+import vega
+
+if __name__ == '__main__':
+    vega.run('./src/eval.yml')
+    vega.set_backend('mindspore', 'NPU')

model_zoo/research/cv/Spnas/requirements.txt

@@ -0,0 +1 @@
+noah-vega

model_zoo/research/cv/Spnas/scripts/run_distributed.sh

@@ -0,0 +1,23 @@
+#!/bin/bash
+# Copyright 2021 Huawei Technologies Co., Ltd
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+# ============================================================================
+
+ulimit -u unlimited
+export DEVICE_NUM=8
+export RANK_SIZE=8
+RANK_TABLE_FILE=$(realpath $1)
+export RANK_TABLE_FILE
+
+python3 -m vega.tools.run_pipeline ../src/spnas_distributed.yml -b m -d NPU

model_zoo/research/cv/Spnas/src/eval.yml

@@ -0,0 +1,86 @@
+general:
+    backend: mindspore
+
+pipeline: [eval]
+
+eval:
+    pipe_step:
+        type: TrainPipeStep
+        models_folder: ~
+
+    model:
+        model_desc:
+            type: Faster_Rcnn_MD
+
+    trainer:
+        with_train: False
+        type: SpNasTrainerCallback
+        epochs: 6
+        get_train_metric_after_epoch: False
+        model_statistics: False
+        is_detection_trainer: True
+        perfs_cmp_key: AP50
+        optimizer:
+            type: SGD
+            params:
+                lr: 0.03
+                momentum: 0.9
+                weight_decay: !!float 1e-4
+        lr_scheduler:
+            type: WarmupScheduler
+            by_epoch: False
+            params:
+                warmup_type: linear
+                warmup_iters: 2000
+                warmup_ratio: 0.001
+                after_scheduler_config:
+                    type: MultiStepLR
+                    by_epoch: True
+                    params:
+                        milestones:  [10, 20]
+                        gamma: 0.1
+        loss:
+            type: SumLoss
+        metric:
+            type: coco
+            params:
+                anno_path: /cache/datasets/COCO2017/annotations/instances_val2017.json
+
+    dataset:
+        type: CocoDataset
+        common:
+            batch_size: 2
+            num_parallel_workers: 8
+            flip_ratio: 0.5
+            expand_ratio: 1.0
+            img_width: 1280
+            img_height: 768
+            keep_ratio: True
+            device_id: 0
+            device_num: 1
+            rank_id: 0
+            python_multiprocessing: True
+            coco_root: "/cache/datasets/COCO2017"
+            mindrecord_dir: "/cache/MindRecord_COCO_TRAIN"
+            instance_set: "annotations/instances_{}.json"
+            coco_classes: ['background', 'person', 'bicycle', 'car', 'motorcycle', 'airplane', 'bus',
+                           'train', 'truck', 'boat', 'traffic light', 'fire hydrant',
+                           'stop sign', 'parking meter', 'bench', 'bird', 'cat', 'dog',
+                           'horse', 'sheep', 'cow', 'elephant', 'bear', 'zebra',
+                           'giraffe', 'backpack', 'umbrella', 'handbag', 'tie',
+                           'suitcase', 'frisbee', 'skis', 'snowboard', 'sports ball',
+                           'kite', 'baseball bat', 'baseball glove', 'skateboard',
+                           'surfboard', 'tennis racket', 'bottle', 'wine glass', 'cup',
+                           'fork', 'knife', 'spoon', 'bowl', 'banana', 'apple',
+                           'sandwich', 'orange', 'broccoli', 'carrot', 'hot dog', 'pizza',
+                           'donut', 'cake', 'chair', 'couch', 'potted plant', 'bed',
+                           'dining table', 'toilet', 'tv', 'laptop', 'mouse', 'remote',
+                           'keyboard', 'cell phone', 'microwave', 'oven', 'toaster', 'sink',
+                           'refrigerator', 'book', 'clock', 'vase', 'scissors',
+                           'teddy bear', 'hair drier', 'toothbrush']
+            num_classes: 81
+        train:
+            train_data_type: "train2017"
+        val:
+            val_data_type: "val2017"
+            test_batch_size: 2

model_zoo/research/cv/Spnas/src/spnas.yml

@@ -0,0 +1,136 @@
+general:
+    backend: mindspore
+
+pipeline: [serial, parallel, fullytrain]
+
+serial:
+    pipe_step:
+        type: SearchPipeStep
+
+    search_algorithm:
+        type: SpNasS
+        max_sample: 20
+        objective_keys: AP50
+
+    search_space:
+        type: SearchSpace
+        hyperparameters:
+            -   key: network.backbone.code
+                type: CATEGORY
+                range: ['111-2111-211111-211']
+
+    model:
+        model_desc:
+            type: Faster_Rcnn_MD
+
+    trainer:
+        type: SpNasTrainerCallback
+        epochs: 6
+        get_train_metric_after_epoch: False
+        model_statistics: False
+        is_detection_trainer: True
+        perfs_cmp_key: AP50
+        optimizer:
+            type: SGD
+            params:
+                lr: 0.03
+                momentum: 0.9
+                weight_decay: !!float 1e-4
+        lr_scheduler:
+            type: WarmupScheduler
+            by_epoch: False
+            params:
+                warmup_type: linear
+                warmup_iters: 2000
+                warmup_ratio: 0.001
+                after_scheduler_config:
+                    type: MultiStepLR
+                    by_epoch: True
+                    params:
+                        milestones:  [10, 20]
+                        gamma: 0.1
+        loss:
+            type: SumLoss
+        metric:
+            type: coco
+            params:
+                anno_path: /cache/datasets/COCO2017/annotations/instances_val2017.json
+
+    dataset:
+        type: CocoDataset
+        common:
+            batch_size: 2
+            num_parallel_workers: 8
+            flip_ratio: 0.5
+            expand_ratio: 1.0
+            img_width: 1280
+            img_height: 768
+            keep_ratio: True
+            device_id: 0
+            device_num: 1
+            rank_id: 0
+            python_multiprocessing: True
+            coco_root: "/cache/datasets/COCO2017"
+            mindrecord_dir: "/cache/MindRecord_COCO_TRAIN"
+            instance_set: "annotations/instances_{}.json"
+            coco_classes: ['background', 'person', 'bicycle', 'car', 'motorcycle', 'airplane', 'bus',
+                           'train', 'truck', 'boat', 'traffic light', 'fire hydrant',
+                           'stop sign', 'parking meter', 'bench', 'bird', 'cat', 'dog',
+                           'horse', 'sheep', 'cow', 'elephant', 'bear', 'zebra',
+                           'giraffe', 'backpack', 'umbrella', 'handbag', 'tie',
+                           'suitcase', 'frisbee', 'skis', 'snowboard', 'sports ball',
+                           'kite', 'baseball bat', 'baseball glove', 'skateboard',
+                           'surfboard', 'tennis racket', 'bottle', 'wine glass', 'cup',
+                           'fork', 'knife', 'spoon', 'bowl', 'banana', 'apple',
+                           'sandwich', 'orange', 'broccoli', 'carrot', 'hot dog', 'pizza',
+                           'donut', 'cake', 'chair', 'couch', 'potted plant', 'bed',
+                           'dining table', 'toilet', 'tv', 'laptop', 'mouse', 'remote',
+                           'keyboard', 'cell phone', 'microwave', 'oven', 'toaster', 'sink',
+                           'refrigerator', 'book', 'clock', 'vase', 'scissors',
+                           'teddy bear', 'hair drier', 'toothbrush']
+            num_classes: 81
+        train:
+            train_data_type: "train2017"
+        val:
+            val_data_type: "val2017"
+            test_batch_size: 2
+
+
+parallel:
+    pipe_step:
+        type: SearchPipeStep
+        models_folder: "{local_base_path}/output/serial/"
+        pretrained_folder: "{local_base_path}/output/serial/"
+
+    search_algorithm:
+        type: SpNasP
+
+    search_space:
+        type: SearchSpace
+        hyperparameters:
+            -   key: network.neck.code
+                type: CATEGORY
+                range: [[0, 1, 2, 3]]
+
+    model:
+        model_desc:
+            type: Faster_Rcnn_MD
+
+    trainer:
+        ref: serial.trainer
+
+    dataset:
+        ref: serial.dataset
+
+fullytrain:
+    pipe_step:
+        type: TrainPipeStep
+        models_folder: "{local_base_path}/output/parallel/"
+        pretrained_folder: "{local_base_path}/output/parallel/"
+
+    trainer:
+        ref: serial.trainer
+        epochs: 24
+
+    dataset:
+        ref: serial.dataset

model_zoo/research/cv/Spnas/src/spnas_distributed.yml

@@ -0,0 +1,138 @@
+general:
+    backend: mindspore
+    parallel_search: True
+    parallel_fully_train: True
+
+pipeline: [serial, parallel, fullytrain]
+
+serial:
+    pipe_step:
+        type: SearchPipeStep
+
+    search_algorithm:
+        type: SpNasS
+        max_sample: 20
+        objective_keys: AP50
+
+    search_space:
+        type: SearchSpace
+        hyperparameters:
+            -   key: network.backbone.code
+                type: CATEGORY
+                range: ['111-2111-211111-211']
+
+    model:
+        model_desc:
+            type: Faster_Rcnn_MD
+
+    trainer:
+        type: SpNasTrainerCallback
+        epochs: 6
+        get_train_metric_after_epoch: False
+        model_statistics: False
+        is_detection_trainer: True
+        perfs_cmp_key: AP50
+        optimizer:
+            type: SGD
+            params:
+                lr: 0.03
+                momentum: 0.9
+                weight_decay: !!float 1e-4
+        lr_scheduler:
+            type: WarmupScheduler
+            by_epoch: False
+            params:
+                warmup_type: linear
+                warmup_iters: 2000
+                warmup_ratio: 0.001
+                after_scheduler_config:
+                    type: MultiStepLR
+                    by_epoch: True
+                    params:
+                        milestones:  [10, 20]
+                        gamma: 0.1
+        loss:
+            type: SumLoss
+        metric:
+            type: coco
+            params:
+                anno_path: /cache/datasets/COCO2017/annotations/instances_val2017.json
+
+    dataset:
+        type: CocoDataset
+        common:
+            batch_size: 2
+            num_parallel_workers: 8
+            flip_ratio: 0.5
+            expand_ratio: 1.0
+            img_width: 1280
+            img_height: 768
+            keep_ratio: True
+            device_id: 0
+            device_num: 1
+            rank_id: 0
+            python_multiprocessing: True
+            coco_root: "/cache/datasets/COCO2017"
+            mindrecord_dir: "/cache/MindRecord_COCO_TRAIN"
+            instance_set: "annotations/instances_{}.json"
+            coco_classes: ['background', 'person', 'bicycle', 'car', 'motorcycle', 'airplane', 'bus',
+                           'train', 'truck', 'boat', 'traffic light', 'fire hydrant',
+                           'stop sign', 'parking meter', 'bench', 'bird', 'cat', 'dog',
+                           'horse', 'sheep', 'cow', 'elephant', 'bear', 'zebra',
+                           'giraffe', 'backpack', 'umbrella', 'handbag', 'tie',
+                           'suitcase', 'frisbee', 'skis', 'snowboard', 'sports ball',
+                           'kite', 'baseball bat', 'baseball glove', 'skateboard',
+                           'surfboard', 'tennis racket', 'bottle', 'wine glass', 'cup',
+                           'fork', 'knife', 'spoon', 'bowl', 'banana', 'apple',
+                           'sandwich', 'orange', 'broccoli', 'carrot', 'hot dog', 'pizza',
+                           'donut', 'cake', 'chair', 'couch', 'potted plant', 'bed',
+                           'dining table', 'toilet', 'tv', 'laptop', 'mouse', 'remote',
+                           'keyboard', 'cell phone', 'microwave', 'oven', 'toaster', 'sink',
+                           'refrigerator', 'book', 'clock', 'vase', 'scissors',
+                           'teddy bear', 'hair drier', 'toothbrush']
+            num_classes: 81
+        train:
+            train_data_type: "train2017"
+        val:
+            val_data_type: "val2017"
+            test_batch_size: 2
+
+
+parallel:
+    pipe_step:
+        type: SearchPipeStep
+        models_folder: "{local_base_path}/output/serial/"
+        pretrained_folder: "{local_base_path}/output/serial/"
+
+    search_algorithm:
+        type: SpNasP
+
+    search_space:
+        type: SearchSpace
+        hyperparameters:
+            -   key: network.neck.code
+                type: CATEGORY
+                range: [[0, 1, 2, 3]]
+
+    model:
+        model_desc:
+            type: Faster_Rcnn_MD
+
+    trainer:
+        ref: serial.trainer
+
+    dataset:
+        ref: serial.dataset
+
+fullytrain:
+    pipe_step:
+        type: TrainPipeStep
+        models_folder: "{local_base_path}/output/parallel/"
+        pretrained_folder: "{local_base_path}/output/parallel/"
+
+    trainer:
+        ref: serial.trainer
+        epochs: 24
+
+    dataset:
+        ref: serial.dataset

model_zoo/research/cv/Spnas/train.py

@@ -0,0 +1,21 @@
+# Copyright 2021 Huawei Technologies Co., Ltd
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+# ============================================================================
+"""train"""
+import vega
+
+
+if __name__ == '__main__':
+    vega.run('./src/spnas.yml')
+    vega.set_backend('mindspore', 'NPU')