add ghostnet, ghostnet_quant, ssd_ghostnet and resnet50_adv_prune to model_zoo/research
This commit is contained in:
parent
2f14c40934
commit
d10be908b8
|
|
@ -0,0 +1,143 @@
|
|||
# Contents
|
||||
|
||||
- [GhostNet Description](#ghostnet-description)
|
||||
- [Model Architecture](#model-architecture)
|
||||
- [Dataset](#dataset)
|
||||
- [Environment Requirements](#environment-requirements)
|
||||
- [Script Description](#script-description)
|
||||
- [Script and Sample Code](#script-and-sample-code)
|
||||
- [Training Process](#training-process)
|
||||
- [Evaluation Process](#evaluation-process)
|
||||
- [Evaluation](#evaluation)
|
||||
- [Model Description](#model-description)
|
||||
- [Performance](#performance)
|
||||
- [Training Performance](#evaluation-performance)
|
||||
- [Inference Performance](#evaluation-performance)
|
||||
- [Description of Random Situation](#description-of-random-situation)
|
||||
- [ModelZoo Homepage](#modelzoo-homepage)
|
||||
|
||||
# [GhostNet Description](#contents)
|
||||
|
||||
The GhostNet architecture is based on an Ghost module structure which generate more features from cheap operations. Based on a set of intrinsic feature maps, a series of cheap operations are applied to generate many ghost feature maps that could fully reveal information underlying intrinsic features.
|
||||
|
||||
[Paper](https://openaccess.thecvf.com/content_CVPR_2020/papers/Han_GhostNet_More_Features_From_Cheap_Operations_CVPR_2020_paper.pdf): Kai Han, Yunhe Wang, Qi Tian, Jianyuan Guo, Chunjing Xu, Chang Xu. GhostNet: More Features from Cheap Operations. CVPR 2020.
|
||||
|
||||
# [Model architecture](#contents)
|
||||
|
||||
The overall network architecture of GhostNet is show below:
|
||||
|
||||
[Link](https://openaccess.thecvf.com/content_CVPR_2020/papers/Han_GhostNet_More_Features_From_Cheap_Operations_CVPR_2020_paper.pdf)
|
||||
|
||||
# [Dataset](#contents)
|
||||
|
||||
Dataset used: [Oxford-IIIT Pet](https://www.robots.ox.ac.uk/~vgg/data/pets/)
|
||||
|
||||
- Dataset size: 7049 colorful images in 1000 classes
|
||||
- Train: 3680 images
|
||||
- Test: 3369 images
|
||||
- Data format: RGB images.
|
||||
- Note: Data will be processed in src/dataset.py
|
||||
|
||||
# [Environment Requirements](#contents)
|
||||
|
||||
- Hardware（Ascend/GPU)
|
||||
- Prepare hardware environment with Ascend or GPU. If you want to try Ascend, please send the [application form](https://obs-9be7.obs.cn-east-2.myhuaweicloud.com/file/other/Ascend%20Model%20Zoo%E4%BD%93%E9%AA%8C%E8%B5%84%E6%BA%90%E7%94%B3%E8%AF%B7%E8%A1%A8.docx) to ascend@huawei.com. Once approved, you can get the resources.
|
||||
- Framework
|
||||
- [MindSpore](http://10.90.67.50/mindspore/archive/20200506/OpenSource/me_vm_x86/)
|
||||
- For more information, please check the resources below：
|
||||
- [MindSpore tutorials](https://www.mindspore.cn/tutorial/zh-CN/master/index.html)
|
||||
- [MindSpore API](https://www.mindspore.cn/api/zh-CN/master/index.html)
|
||||
|
||||
# [Script description](#contents)
|
||||
|
||||
## [Script and sample code](#contents)
|
||||
|
||||
```python
|
||||
├── GhostNet
|
||||
├── Readme.md # descriptions about ghostnet # shell script for evaluation with CPU, GPU or Ascend
|
||||
├── src
|
||||
│ ├──config.py # parameter configuration
|
||||
│ ├──dataset.py # creating dataset
|
||||
│ ├──launch.py # start python script
|
||||
│ ├──lr_generator.py # learning rate config
|
||||
│ ├──ghostnet.py # GhostNet architecture
|
||||
│ ├──ghostnet600.py # GhostNet-600M architecture
|
||||
├── eval.py # evaluation script
|
||||
├── mindspore_hub_conf.py # export model for hub
|
||||
```
|
||||
|
||||
## [Training process](#contents)
|
||||
To Be Done
|
||||
|
||||
## [Eval process](#contents)
|
||||
|
||||
### Usage
|
||||
|
||||
After installing MindSpore via the official website, you can start evaluation as follows:
|
||||
|
||||
|
||||
### Launch
|
||||
|
||||
```
|
||||
# infer example
|
||||
|
||||
Ascend: python eval.py --model [ghostnet/ghostnet-600] --dataset_path ~/Pets/test.mindrecord --platform Ascend --checkpoint_path [CHECKPOINT_PATH]
|
||||
GPU: python eval.py --model [ghostnet/ghostnet-600] --dataset_path ~/Pets/test.mindrecord --platform GPU --checkpoint_path [CHECKPOINT_PATH]
|
||||
```
|
||||
|
||||
> checkpoint can be produced in training process.
|
||||
|
||||
### Result
|
||||
|
||||
```
|
||||
result: {'acc': 0.8113927500681385} ckpt= ./ghostnet_nose_1x_pets.ckpt
|
||||
result: {'acc': 0.824475333878441} ckpt= ./ghostnet_1x_pets.ckpt
|
||||
result: {'acc': 0.8691741618969746} ckpt= ./ghostnet600M_pets.ckpt
|
||||
```
|
||||
|
||||
# [Model Description](#contents)
|
||||
|
||||
## [Performance](#contents)
|
||||
|
||||
#### Evaluation Performance
|
||||
|
||||
###### GhostNet on ImageNet2012
|
||||
| Parameters | | |
|
||||
| -------------------------- | -------------------------------------- |---------------------------------- |
|
||||
| Model Version | GhostNet |GhostNet-600|
|
||||
| uploaded Date | 09/08/2020 (month/day/year) ； | 09/08/2020 (month/day/year) |
|
||||
| MindSpore Version | 0.6.0-alpha |0.6.0-alpha |
|
||||
| Dataset | ImageNet2012 | ImageNet2012|
|
||||
| Parameters (M) | 5.2 | 11.9 |
|
||||
| FLOPs (M) | 142 | 591 |
|
||||
| Accuracy (Top1) | 73.9 |80.2 |
|
||||
|
||||
###### GhostNet on Oxford-IIIT Pet
|
||||
| Parameters | | |
|
||||
| -------------------------- | -------------------------------------- |---------------------------------- |
|
||||
| Model Version | GhostNet |GhostNet-600|
|
||||
| uploaded Date | 09/08/2020 (month/day/year) ； | 09/08/2020 (month/day/year) |
|
||||
| MindSpore Version | 0.6.0-alpha |0.6.0-alpha |
|
||||
| Dataset | Oxford-IIIT Pet | Oxford-IIIT Pet|
|
||||
| Parameters (M) | 3.9 | 10.6 |
|
||||
| FLOPs (M) | 140 | 590 |
|
||||
| Accuracy (Top1) | 82.4 |86.9 |
|
||||
|
||||
###### Comparison with other methods on Oxford-IIIT Pet
|
||||
|
||||
|Model|FLOPs (M)|Latency (ms)*|Accuracy (Top1)|
|
||||
|-|-|-|-|
|
||||
|MobileNetV2-1x|300|28.2|78.5|
|
||||
|Ghost-1x w\o SE|138|19.1|81.1|
|
||||
|Ghost-1x|140|25.3|82.4|
|
||||
|Ghost-600|590|-|86.9|
|
||||
|
||||
*The latency is measured on Huawei Kirin 990 chip under single-threaded mode with batch size 1.
|
||||
|
||||
# [Description of Random Situation](#contents)
|
||||
|
||||
In dataset.py, we set the seed inside “create_dataset" function. We also use random seed in train.py.
|
||||
|
||||
# [ModelZoo Homepage](#contents)
|
||||
|
||||
Please check the official [homepage](https://gitee.com/mindspore/mindspore/tree/master/model_zoo).
|
||||
|
|
@ -0,0 +1,84 @@
|
|||
# Copyright 2020 Huawei Technologies Co., Ltd
|
||||
#
|
||||
# Licensed under the Apache License, Version 2.0 (the "License");
|
||||
# you may not use this file except in compliance with the License.
|
||||
# You may obtain a copy of the License at
|
||||
#
|
||||
# http://www.apache.org/licenses/LICENSE-2.0
|
||||
#
|
||||
# Unless required by applicable law or agreed to in writing, software
|
||||
# distributed under the License is distributed on an "AS IS" BASIS,
|
||||
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
# See the License for the specific language governing permissions and
|
||||
# limitations under the License.
|
||||
# ============================================================================
|
||||
"""
|
||||
eval.
|
||||
"""
|
||||
import os
|
||||
import argparse
|
||||
from mindspore import context
|
||||
from mindspore import nn
|
||||
from mindspore.train.model import Model
|
||||
from mindspore.train.serialization import load_checkpoint, load_param_into_net
|
||||
from mindspore.common import dtype as mstype
|
||||
from src.dataset import create_dataset
|
||||
from src.config import config_ascend, config_gpu
|
||||
from src.ghostnet import ghostnet_1x, ghostnet_nose_1x
|
||||
from src.ghostnet600 import ghostnet_600m
|
||||
|
||||
|
||||
parser = argparse.ArgumentParser(description='Image classification')
|
||||
parser.add_argument('--checkpoint_path', type=str, default=None, help='Checkpoint file path')
|
||||
parser.add_argument('--dataset_path', type=str, default=None, help='Dataset path')
|
||||
parser.add_argument('--platform', type=str, default=None, help='run platform')
|
||||
parser.add_argument('--model', type=str, default=None, help='ghostnet')
|
||||
args_opt = parser.parse_args()
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
config_platform = None
|
||||
if args_opt.platform == "Ascend":
|
||||
config_platform = config_ascend
|
||||
device_id = int(os.getenv('DEVICE_ID'))
|
||||
context.set_context(mode=context.GRAPH_MODE, device_target="Ascend",
|
||||
device_id=device_id, save_graphs=False)
|
||||
elif args_opt.platform == "GPU":
|
||||
config_platform = config_gpu
|
||||
context.set_context(mode=context.GRAPH_MODE,
|
||||
device_target="GPU", save_graphs=False)
|
||||
else:
|
||||
raise ValueError("Unsupport platform.")
|
||||
|
||||
loss = nn.SoftmaxCrossEntropyWithLogits(
|
||||
is_grad=False, sparse=True, reduction='mean')
|
||||
|
||||
if args_opt.model == 'ghostnet':
|
||||
net = ghostnet_1x(num_classes=config_platform.num_classes)
|
||||
elif args_opt.model == 'ghostnet_nose':
|
||||
net = ghostnet_nose_1x(num_classes=config_platform.num_classes)
|
||||
elif args_opt.model == 'ghostnet-600':
|
||||
net = ghostnet_600m(num_classes=config_platform.num_classes)
|
||||
|
||||
if args_opt.platform == "Ascend":
|
||||
net.to_float(mstype.float16)
|
||||
for _, cell in net.cells_and_names():
|
||||
if isinstance(cell, nn.Dense):
|
||||
cell.to_float(mstype.float32)
|
||||
|
||||
dataset = create_dataset(dataset_path=args_opt.dataset_path,
|
||||
do_train=False,
|
||||
config=config_platform,
|
||||
platform=args_opt.platform,
|
||||
batch_size=config_platform.batch_size,
|
||||
model=args_opt.model)
|
||||
step_size = dataset.get_dataset_size()
|
||||
|
||||
if args_opt.checkpoint_path:
|
||||
param_dict = load_checkpoint(args_opt.checkpoint_path)
|
||||
load_param_into_net(net, param_dict)
|
||||
net.set_train(False)
|
||||
|
||||
model = Model(net, loss_fn=loss, metrics={'acc'})
|
||||
res = model.eval(dataset)
|
||||
print("result:", res, "ckpt=", args_opt.checkpoint_path)
|
||||
|
|
@ -0,0 +1,27 @@
|
|||
# Copyright 2020 Huawei Technologies Co., Ltd
|
||||
#
|
||||
# Licensed under the Apache License, Version 2.0 (the "License");
|
||||
# you may not use this file except in compliance with the License.
|
||||
# You may obtain a copy of the License at
|
||||
#
|
||||
# http://www.apache.org/licenses/LICENSE-2.0
|
||||
#
|
||||
# Unless required by applicable law or agreed to in writing, software
|
||||
# distributed under the License is distributed on an "AS IS" BASIS,
|
||||
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
# See the License for the specific language governing permissions and
|
||||
# limitations under the License.
|
||||
# ============================================================================
|
||||
"""hub config."""
|
||||
from src.ghostnet import ghostnet_1x, ghostnet_nose_1x
|
||||
from src.ghostnet600 import ghostnet_600m
|
||||
|
||||
|
||||
def create_network(name, *args, **kwargs):
|
||||
if name == 'ghostnet':
|
||||
return ghostnet_1x(*args, **kwargs)
|
||||
if name == 'ghostnet_nose':
|
||||
return ghostnet_nose_1x(*args, **kwargs)
|
||||
if name == 'ghostnet-600':
|
||||
return ghostnet_600m(*args, **kwargs)
|
||||
raise NotImplementedError(f"{name} is not implemented in the repo")
|
||||
|
|
@ -0,0 +1,54 @@
|
|||
# Copyright 2020 Huawei Technologies Co., Ltd
|
||||
#
|
||||
# Licensed under the Apache License, Version 2.0 (the "License");
|
||||
# you may not use this file except in compliance with the License.
|
||||
# You may obtain a copy of the License at
|
||||
#
|
||||
# http://www.apache.org/licenses/LICENSE-2.0
|
||||
#
|
||||
# Unless required by applicable law or agreed to in writing, software
|
||||
# distributed under the License is distributed on an "AS IS" BASIS,
|
||||
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
# See the License for the specific language governing permissions and
|
||||
# limitations under the License.
|
||||
# ============================================================================
|
||||
"""
|
||||
network config setting, will be used in train.py and eval.py
|
||||
"""
|
||||
from easydict import EasyDict as ed
|
||||
|
||||
config_ascend = ed({
|
||||
"num_classes": 37,
|
||||
"image_height": 224,
|
||||
"image_width": 224,
|
||||
"batch_size": 256,
|
||||
"epoch_size": 200,
|
||||
"warmup_epochs": 4,
|
||||
"lr": 0.4,
|
||||
"momentum": 0.9,
|
||||
"weight_decay": 4e-5,
|
||||
"label_smooth": 0.1,
|
||||
"loss_scale": 1024,
|
||||
"save_checkpoint": True,
|
||||
"save_checkpoint_epochs": 1,
|
||||
"keep_checkpoint_max": 200,
|
||||
"save_checkpoint_path": "./checkpoint",
|
||||
})
|
||||
|
||||
config_gpu = ed({
|
||||
"num_classes": 37,
|
||||
"image_height": 224,
|
||||
"image_width": 224,
|
||||
"batch_size": 3,
|
||||
"epoch_size": 370,
|
||||
"warmup_epochs": 4,
|
||||
"lr": 0.4,
|
||||
"momentum": 0.9,
|
||||
"weight_decay": 4e-5,
|
||||
"label_smooth": 0.1,
|
||||
"loss_scale": 1024,
|
||||
"save_checkpoint": True,
|
||||
"save_checkpoint_epochs": 1,
|
||||
"keep_checkpoint_max": 500,
|
||||
"save_checkpoint_path": "./checkpoint",
|
||||
})
|
||||
|
|
@ -0,0 +1,110 @@
|
|||
# Copyright 2020 Huawei Technologies Co., Ltd
|
||||
#
|
||||
# Licensed under the Apache License, Version 2.0 (the "License");
|
||||
# you may not use this file except in compliance with the License.
|
||||
# You may obtain a copy of the License at
|
||||
#
|
||||
# http://www.apache.org/licenses/LICENSE-2.0
|
||||
#
|
||||
# Unless required by applicable law or agreed to in writing, software
|
||||
# distributed under the License is distributed on an "AS IS" BASIS,
|
||||
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
# See the License for the specific language governing permissions and
|
||||
# limitations under the License.
|
||||
# ============================================================================
|
||||
"""
|
||||
create train or eval dataset.
|
||||
"""
|
||||
import os
|
||||
import mindspore.common.dtype as mstype
|
||||
import mindspore.dataset.engine as de
|
||||
import mindspore.dataset.transforms.vision.c_transforms as C
|
||||
import mindspore.dataset.transforms.vision.py_transforms as P
|
||||
import mindspore.dataset.transforms.c_transforms as C2
|
||||
from mindspore.dataset.transforms.vision import Inter
|
||||
|
||||
|
||||
def create_dataset(dataset_path, do_train, config, platform, repeat_num=1, batch_size=100, model='ghsotnet'):
|
||||
"""
|
||||
create a train or eval dataset
|
||||
|
||||
Args:
|
||||
dataset_path(string): the path of dataset.
|
||||
do_train(bool): whether dataset is used for train or eval.
|
||||
repeat_num(int): the repeat times of dataset. Default: 1
|
||||
batch_size(int): the batch size of dataset. Default: 32
|
||||
|
||||
Returns:
|
||||
dataset
|
||||
"""
|
||||
if platform == "Ascend":
|
||||
rank_size = int(os.getenv("RANK_SIZE"))
|
||||
rank_id = int(os.getenv("RANK_ID"))
|
||||
if rank_size == 1:
|
||||
ds = de.MindDataset(
|
||||
dataset_path, num_parallel_workers=8, shuffle=True)
|
||||
else:
|
||||
ds = de.MindDataset(dataset_path, num_parallel_workers=8, shuffle=True,
|
||||
num_shards=rank_size, shard_id=rank_id)
|
||||
elif platform == "GPU":
|
||||
if do_train:
|
||||
from mindspore.communication.management import get_rank, get_group_size
|
||||
ds = de.MindDataset(dataset_path, num_parallel_workers=8, shuffle=True,
|
||||
num_shards=get_group_size(), shard_id=get_rank())
|
||||
else:
|
||||
ds = de.MindDataset(
|
||||
dataset_path, num_parallel_workers=8, shuffle=True)
|
||||
else:
|
||||
raise ValueError("Unsupport platform.")
|
||||
|
||||
resize_height = config.image_height
|
||||
buffer_size = 1000
|
||||
|
||||
# define map operations
|
||||
resize_crop_op = C.RandomCropDecodeResize(
|
||||
resize_height, scale=(0.08, 1.0), ratio=(0.75, 1.333))
|
||||
horizontal_flip_op = C.RandomHorizontalFlip(prob=0.5)
|
||||
|
||||
color_op = C.RandomColorAdjust(
|
||||
brightness=0.4, contrast=0.4, saturation=0.4)
|
||||
rescale_op = C.Rescale(1/255.0, 0)
|
||||
normalize_op = C.Normalize(
|
||||
mean=[0.485, 0.456, 0.406], std=[0.229, 0.224, 0.225])
|
||||
change_swap_op = C.HWC2CHW()
|
||||
|
||||
# define python operations
|
||||
decode_p = P.Decode()
|
||||
if model == 'ghostnet-600':
|
||||
s = 274
|
||||
c = 240
|
||||
else:
|
||||
s = 256
|
||||
c = 224
|
||||
resize_p = P.Resize(s, interpolation=Inter.BICUBIC)
|
||||
center_crop_p = P.CenterCrop(c)
|
||||
totensor = P.ToTensor()
|
||||
normalize_p = P.Normalize((0.485, 0.456, 0.406), (0.229, 0.224, 0.225))
|
||||
composeop = P.ComposeOp(
|
||||
[decode_p, resize_p, center_crop_p, totensor, normalize_p])
|
||||
if do_train:
|
||||
trans = [resize_crop_op, horizontal_flip_op, color_op,
|
||||
rescale_op, normalize_op, change_swap_op]
|
||||
else:
|
||||
trans = composeop()
|
||||
type_cast_op = C2.TypeCast(mstype.int32)
|
||||
|
||||
ds = ds.map(input_columns="image", operations=trans,
|
||||
num_parallel_workers=8)
|
||||
ds = ds.map(input_columns="label_list",
|
||||
operations=type_cast_op, num_parallel_workers=8)
|
||||
|
||||
# apply shuffle operations
|
||||
ds = ds.shuffle(buffer_size=buffer_size)
|
||||
|
||||
# apply batch operations
|
||||
ds = ds.batch(batch_size, drop_remainder=True)
|
||||
|
||||
# apply dataset repeat operation
|
||||
ds = ds.repeat(repeat_num)
|
||||
|
||||
return ds
|
||||
|
|
@ -0,0 +1,490 @@
|
|||
# Copyright 2020 Huawei Technologies Co., Ltd
|
||||
#
|
||||
# Licensed under the Apache License, Version 2.0 (the "License");
|
||||
# you may not use this file except in compliance with the License.
|
||||
# You may obtain a copy of the License at
|
||||
#
|
||||
# http://www.apache.org/licenses/LICENSE-2.0
|
||||
#
|
||||
# Unless required by applicable law or agreed to in writing, software
|
||||
# distributed under the License is distributed on an "AS IS" BASIS,
|
||||
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
# See the License for the specific language governing permissions and
|
||||
# limitations under the License.
|
||||
# ============================================================================
|
||||
"""GhostNet model define"""
|
||||
from functools import partial
|
||||
import math
|
||||
import numpy as np
|
||||
import mindspore.nn as nn
|
||||
from mindspore.ops import operations as P
|
||||
from mindspore import Tensor
|
||||
|
||||
|
||||
__all__ = ['ghostnet']
|
||||
|
||||
|
||||
def _make_divisible(x, divisor=4):
|
||||
return int(np.ceil(x * 1. / divisor) * divisor)
|
||||
|
||||
|
||||
class MyHSigmoid(nn.Cell):
|
||||
"""
|
||||
Hard Sigmoid definition.
|
||||
|
||||
Args:
|
||||
|
||||
Returns:
|
||||
Tensor, output tensor.
|
||||
|
||||
Examples:
|
||||
>>> MyHSigmoid()
|
||||
"""
|
||||
|
||||
def __init__(self):
|
||||
super(MyHSigmoid, self).__init__()
|
||||
self.relu6 = nn.ReLU6()
|
||||
|
||||
def construct(self, x):
|
||||
return self.relu6(x + 3.) * 0.16666667
|
||||
|
||||
|
||||
class Activation(nn.Cell):
|
||||
"""
|
||||
Activation definition.
|
||||
|
||||
Args:
|
||||
act_func(string): activation name.
|
||||
|
||||
Returns:
|
||||
Tensor, output tensor.
|
||||
"""
|
||||
|
||||
def __init__(self, act_func):
|
||||
super(Activation, self).__init__()
|
||||
if act_func == 'relu':
|
||||
self.act = nn.ReLU()
|
||||
elif act_func == 'relu6':
|
||||
self.act = nn.ReLU6()
|
||||
elif act_func in ('hsigmoid', 'hard_sigmoid'):
|
||||
self.act = MyHSigmoid()
|
||||
elif act_func in ('hswish', 'hard_swish'):
|
||||
self.act = nn.HSwish()
|
||||
else:
|
||||
raise NotImplementedError
|
||||
|
||||
def construct(self, x):
|
||||
return self.act(x)
|
||||
|
||||
|
||||
class GlobalAvgPooling(nn.Cell):
|
||||
"""
|
||||
Global avg pooling definition.
|
||||
|
||||
Args:
|
||||
|
||||
Returns:
|
||||
Tensor, output tensor.
|
||||
|
||||
Examples:
|
||||
>>> GlobalAvgPooling()
|
||||
"""
|
||||
|
||||
def __init__(self, keep_dims=False):
|
||||
super(GlobalAvgPooling, self).__init__()
|
||||
self.mean = P.ReduceMean(keep_dims=keep_dims)
|
||||
|
||||
def construct(self, x):
|
||||
x = self.mean(x, (2, 3))
|
||||
return x
|
||||
|
||||
|
||||
class SE(nn.Cell):
|
||||
"""
|
||||
SE warpper definition.
|
||||
|
||||
Args:
|
||||
num_out (int): Output channel.
|
||||
ratio (int): middle output ratio.
|
||||
|
||||
Returns:
|
||||
Tensor, output tensor.
|
||||
|
||||
Examples:
|
||||
>>> SE(4)
|
||||
"""
|
||||
|
||||
def __init__(self, num_out, ratio=4):
|
||||
super(SE, self).__init__()
|
||||
num_mid = _make_divisible(num_out // ratio)
|
||||
self.pool = GlobalAvgPooling(keep_dims=True)
|
||||
self.conv_reduce = nn.Conv2d(in_channels=num_out, out_channels=num_mid,
|
||||
kernel_size=1, has_bias=True, pad_mode='pad')
|
||||
self.act1 = Activation('relu')
|
||||
self.conv_expand = nn.Conv2d(in_channels=num_mid, out_channels=num_out,
|
||||
kernel_size=1, has_bias=True, pad_mode='pad')
|
||||
self.act2 = Activation('hsigmoid')
|
||||
self.mul = P.Mul()
|
||||
|
||||
def construct(self, x):
|
||||
out = self.pool(x)
|
||||
out = self.conv_reduce(out)
|
||||
out = self.act1(out)
|
||||
out = self.conv_expand(out)
|
||||
out = self.act2(out)
|
||||
out = self.mul(x, out)
|
||||
return out
|
||||
|
||||
|
||||
class ConvUnit(nn.Cell):
|
||||
"""
|
||||
ConvUnit warpper definition.
|
||||
|
||||
Args:
|
||||
num_in (int): Input channel.
|
||||
num_out (int): Output channel.
|
||||
kernel_size (int): Input kernel size.
|
||||
stride (int): Stride size.
|
||||
padding (int): Padding number.
|
||||
num_groups (int): Output num group.
|
||||
use_act (bool): Used activation or not.
|
||||
act_type (string): Activation type.
|
||||
|
||||
Returns:
|
||||
Tensor, output tensor.
|
||||
|
||||
Examples:
|
||||
>>> ConvUnit(3, 3)
|
||||
"""
|
||||
|
||||
def __init__(self, num_in, num_out, kernel_size=1, stride=1, padding=0, num_groups=1,
|
||||
use_act=True, act_type='relu'):
|
||||
super(ConvUnit, self).__init__()
|
||||
self.conv = nn.Conv2d(in_channels=num_in,
|
||||
out_channels=num_out,
|
||||
kernel_size=kernel_size,
|
||||
stride=stride,
|
||||
padding=padding,
|
||||
group=num_groups,
|
||||
has_bias=False,
|
||||
pad_mode='pad')
|
||||
self.bn = nn.BatchNorm2d(num_out)
|
||||
self.use_act = use_act
|
||||
self.act = Activation(act_type) if use_act else None
|
||||
|
||||
def construct(self, x):
|
||||
out = self.conv(x)
|
||||
out = self.bn(out)
|
||||
if self.use_act:
|
||||
out = self.act(out)
|
||||
return out
|
||||
|
||||
|
||||
class GhostModule(nn.Cell):
|
||||
"""
|
||||
GhostModule warpper definition.
|
||||
|
||||
Args:
|
||||
num_in (int): Input channel.
|
||||
num_out (int): Output channel.
|
||||
kernel_size (int): Input kernel size.
|
||||
stride (int): Stride size.
|
||||
padding (int): Padding number.
|
||||
ratio (int): Reduction ratio.
|
||||
dw_size (int): kernel size of cheap operation.
|
||||
use_act (bool): Used activation or not.
|
||||
act_type (string): Activation type.
|
||||
|
||||
Returns:
|
||||
Tensor, output tensor.
|
||||
|
||||
Examples:
|
||||
>>> GhostModule(3, 3)
|
||||
"""
|
||||
|
||||
def __init__(self, num_in, num_out, kernel_size=1, stride=1, padding=0, ratio=2, dw_size=3,
|
||||
use_act=True, act_type='relu'):
|
||||
super(GhostModule, self).__init__()
|
||||
init_channels = math.ceil(num_out / ratio)
|
||||
new_channels = init_channels * (ratio - 1)
|
||||
|
||||
self.primary_conv = ConvUnit(num_in, init_channels, kernel_size=kernel_size, stride=stride, padding=padding,
|
||||
num_groups=1, use_act=use_act, act_type='relu')
|
||||
self.cheap_operation = ConvUnit(init_channels, new_channels, kernel_size=dw_size, stride=1, padding=dw_size//2,
|
||||
num_groups=init_channels, use_act=use_act, act_type='relu')
|
||||
self.concat = P.Concat(axis=1)
|
||||
|
||||
def construct(self, x):
|
||||
x1 = self.primary_conv(x)
|
||||
x2 = self.cheap_operation(x1)
|
||||
return self.concat((x1, x2))
|
||||
|
||||
|
||||
class GhostBottleneck(nn.Cell):
|
||||
"""
|
||||
GhostBottleneck warpper definition.
|
||||
|
||||
Args:
|
||||
num_in (int): Input channel.
|
||||
num_mid (int): Middle channel.
|
||||
num_out (int): Output channel.
|
||||
kernel_size (int): Input kernel size.
|
||||
stride (int): Stride size.
|
||||
act_type (str): Activation type.
|
||||
use_se (bool): Use SE warpper or not.
|
||||
|
||||
Returns:
|
||||
Tensor, output tensor.
|
||||
|
||||
Examples:
|
||||
>>> GhostBottleneck(16, 3, 1, 1)
|
||||
"""
|
||||
|
||||
def __init__(self, num_in, num_mid, num_out, kernel_size, stride=1, act_type='relu', use_se=False):
|
||||
super(GhostBottleneck, self).__init__()
|
||||
self.ghost1 = GhostModule(num_in, num_mid, kernel_size=1,
|
||||
stride=1, padding=0, act_type=act_type)
|
||||
|
||||
self.use_dw = stride > 1
|
||||
self.dw = None
|
||||
if self.use_dw:
|
||||
self.dw = ConvUnit(num_mid, num_mid, kernel_size=kernel_size, stride=stride,
|
||||
padding=self._get_pad(kernel_size), act_type=act_type, num_groups=num_mid, use_act=False)
|
||||
|
||||
self.use_se = use_se
|
||||
if use_se:
|
||||
self.se = SE(num_mid)
|
||||
|
||||
self.ghost2 = GhostModule(num_mid, num_out, kernel_size=1, stride=1,
|
||||
padding=0, act_type=act_type, use_act=False)
|
||||
|
||||
self.down_sample = False
|
||||
if num_in != num_out or stride != 1:
|
||||
self.down_sample = True
|
||||
self.shortcut = None
|
||||
if self.down_sample:
|
||||
self.shortcut = nn.SequentialCell([
|
||||
ConvUnit(num_in, num_in, kernel_size=kernel_size, stride=stride,
|
||||
padding=self._get_pad(kernel_size), num_groups=num_in, use_act=False),
|
||||
ConvUnit(num_in, num_out, kernel_size=1, stride=1,
|
||||
padding=0, num_groups=1, use_act=False),
|
||||
])
|
||||
self.add = P.TensorAdd()
|
||||
|
||||
def construct(self, x):
|
||||
r"""construct of ghostnet"""
|
||||
shortcut = x
|
||||
out = self.ghost1(x)
|
||||
if self.use_dw:
|
||||
out = self.dw(out)
|
||||
if self.use_se:
|
||||
out = self.se(out)
|
||||
out = self.ghost2(out)
|
||||
if self.down_sample:
|
||||
shortcut = self.shortcut(shortcut)
|
||||
out = self.add(shortcut, out)
|
||||
return out
|
||||
|
||||
def _get_pad(self, kernel_size):
|
||||
"""set the padding number"""
|
||||
pad = 0
|
||||
if kernel_size == 1:
|
||||
pad = 0
|
||||
elif kernel_size == 3:
|
||||
pad = 1
|
||||
elif kernel_size == 5:
|
||||
pad = 2
|
||||
elif kernel_size == 7:
|
||||
pad = 3
|
||||
else:
|
||||
raise NotImplementedError
|
||||
return pad
|
||||
|
||||
|
||||
class GhostNet(nn.Cell):
|
||||
"""
|
||||
GhostNet architecture.
|
||||
|
||||
Args:
|
||||
model_cfgs (Cell): number of classes.
|
||||
num_classes (int): Output number classes.
|
||||
multiplier (int): Channels multiplier for round to 8/16 and others. Default is 1.
|
||||
final_drop (float): Dropout number.
|
||||
round_nearest (list): Channel round to . Default is 8.
|
||||
Returns:
|
||||
Tensor, output tensor.
|
||||
|
||||
Examples:
|
||||
>>> GhostNet(num_classes=1000)
|
||||
"""
|
||||
|
||||
def __init__(self, model_cfgs, num_classes=1000, multiplier=1., final_drop=0., round_nearest=8):
|
||||
super(GhostNet, self).__init__()
|
||||
self.cfgs = model_cfgs['cfg']
|
||||
self.inplanes = 16
|
||||
first_conv_in_channel = 3
|
||||
first_conv_out_channel = _make_divisible(multiplier * self.inplanes)
|
||||
|
||||
self.conv_stem = nn.Conv2d(in_channels=first_conv_in_channel,
|
||||
out_channels=first_conv_out_channel,
|
||||
kernel_size=3, padding=1, stride=2,
|
||||
has_bias=False, pad_mode='pad')
|
||||
self.bn1 = nn.BatchNorm2d(first_conv_out_channel)
|
||||
self.act1 = Activation('relu')
|
||||
|
||||
self.blocks = []
|
||||
for layer_cfg in self.cfgs:
|
||||
self.blocks.append(self._make_layer(kernel_size=layer_cfg[0],
|
||||
exp_ch=_make_divisible(
|
||||
multiplier * layer_cfg[1]),
|
||||
out_channel=_make_divisible(
|
||||
multiplier * layer_cfg[2]),
|
||||
use_se=layer_cfg[3],
|
||||
act_func=layer_cfg[4],
|
||||
stride=layer_cfg[5]))
|
||||
output_channel = _make_divisible(
|
||||
multiplier * model_cfgs["cls_ch_squeeze"])
|
||||
self.blocks.append(ConvUnit(_make_divisible(multiplier * self.cfgs[-1][2]), output_channel,
|
||||
kernel_size=1, stride=1, padding=0, num_groups=1, use_act=True))
|
||||
self.blocks = nn.SequentialCell(self.blocks)
|
||||
|
||||
self.global_pool = GlobalAvgPooling(keep_dims=True)
|
||||
self.conv_head = nn.Conv2d(in_channels=output_channel,
|
||||
out_channels=model_cfgs['cls_ch_expand'],
|
||||
kernel_size=1, padding=0, stride=1,
|
||||
has_bias=True, pad_mode='pad')
|
||||
self.act2 = Activation('relu')
|
||||
self.squeeze = P.Flatten()
|
||||
self.final_drop = final_drop
|
||||
if self.final_drop > 0:
|
||||
self.dropout = nn.Dropout(self.final_drop)
|
||||
|
||||
self.classifier = nn.Dense(
|
||||
model_cfgs['cls_ch_expand'], num_classes, has_bias=True)
|
||||
|
||||
self._initialize_weights()
|
||||
|
||||
def construct(self, x):
|
||||
r"""construct of GhostNet"""
|
||||
x = self.conv_stem(x)
|
||||
x = self.bn1(x)
|
||||
x = self.act1(x)
|
||||
x = self.blocks(x)
|
||||
x = self.global_pool(x)
|
||||
x = self.conv_head(x)
|
||||
x = self.act2(x)
|
||||
x = self.squeeze(x)
|
||||
if self.final_drop > 0:
|
||||
x = self.dropout(x)
|
||||
x = self.classifier(x)
|
||||
return x
|
||||
|
||||
def _make_layer(self, kernel_size, exp_ch, out_channel, use_se, act_func, stride=1):
|
||||
mid_planes = exp_ch
|
||||
out_planes = out_channel
|
||||
layer = GhostBottleneck(self.inplanes, mid_planes, out_planes,
|
||||
kernel_size, stride=stride, act_type=act_func, use_se=use_se)
|
||||
self.inplanes = out_planes
|
||||
return layer
|
||||
|
||||
def _initialize_weights(self):
|
||||
"""
|
||||
Initialize weights.
|
||||
|
||||
Args:
|
||||
|
||||
Returns:
|
||||
None.
|
||||
|
||||
Examples:
|
||||
>>> _initialize_weights()
|
||||
"""
|
||||
self.init_parameters_data()
|
||||
for _, m in self.cells_and_names():
|
||||
if isinstance(m, (nn.Conv2d)):
|
||||
n = m.kernel_size[0] * m.kernel_size[1] * m.out_channels
|
||||
m.weight.set_parameter_data(Tensor(np.random.normal(0, np.sqrt(2. / n),
|
||||
m.weight.data.shape).astype("float32")))
|
||||
if m.bias is not None:
|
||||
m.bias.set_parameter_data(
|
||||
Tensor(np.zeros(m.bias.data.shape, dtype="float32")))
|
||||
elif isinstance(m, nn.BatchNorm2d):
|
||||
m.gamma.set_parameter_data(
|
||||
Tensor(np.ones(m.gamma.data.shape, dtype="float32")))
|
||||
m.beta.set_parameter_data(
|
||||
Tensor(np.zeros(m.beta.data.shape, dtype="float32")))
|
||||
elif isinstance(m, nn.Dense):
|
||||
m.weight.set_parameter_data(Tensor(np.random.normal(
|
||||
0, 0.01, m.weight.data.shape).astype("float32")))
|
||||
if m.bias is not None:
|
||||
m.bias.set_parameter_data(
|
||||
Tensor(np.zeros(m.bias.data.shape, dtype="float32")))
|
||||
|
||||
|
||||
def ghostnet(model_name, **kwargs):
|
||||
"""
|
||||
Constructs a GhostNet model
|
||||
"""
|
||||
model_cfgs = {
|
||||
"1x": {
|
||||
"cfg": [
|
||||
# k, exp, c, se, nl, s,
|
||||
# stage1
|
||||
[3, 16, 16, False, 'relu', 1],
|
||||
# stage2
|
||||
[3, 48, 24, False, 'relu', 2],
|
||||
[3, 72, 24, False, 'relu', 1],
|
||||
# stage3
|
||||
[5, 72, 40, True, 'relu', 2],
|
||||
[5, 120, 40, True, 'relu', 1],
|
||||
# stage4
|
||||
[3, 240, 80, False, 'relu', 2],
|
||||
[3, 200, 80, False, 'relu', 1],
|
||||
[3, 184, 80, False, 'relu', 1],
|
||||
[3, 184, 80, False, 'relu', 1],
|
||||
[3, 480, 112, True, 'relu', 1],
|
||||
[3, 672, 112, True, 'relu', 1],
|
||||
# stage5
|
||||
[5, 672, 160, True, 'relu', 2],
|
||||
[5, 960, 160, False, 'relu', 1],
|
||||
[5, 960, 160, True, 'relu', 1],
|
||||
[5, 960, 160, False, 'relu', 1],
|
||||
[5, 960, 160, True, 'relu', 1]],
|
||||
"cls_ch_squeeze": 960,
|
||||
"cls_ch_expand": 1280,
|
||||
},
|
||||
|
||||
"nose_1x": {
|
||||
"cfg": [
|
||||
# k, exp, c, se, nl, s,
|
||||
# stage1
|
||||
[3, 16, 16, False, 'relu', 1],
|
||||
# stage2
|
||||
[3, 48, 24, False, 'relu', 2],
|
||||
[3, 72, 24, False, 'relu', 1],
|
||||
# stage3
|
||||
[5, 72, 40, False, 'relu', 2],
|
||||
[5, 120, 40, False, 'relu', 1],
|
||||
# stage4
|
||||
[3, 240, 80, False, 'relu', 2],
|
||||
[3, 200, 80, False, 'relu', 1],
|
||||
[3, 184, 80, False, 'relu', 1],
|
||||
[3, 184, 80, False, 'relu', 1],
|
||||
[3, 480, 112, False, 'relu', 1],
|
||||
[3, 672, 112, False, 'relu', 1],
|
||||
# stage5
|
||||
[5, 672, 160, False, 'relu', 2],
|
||||
[5, 960, 160, False, 'relu', 1],
|
||||
[5, 960, 160, False, 'relu', 1],
|
||||
[5, 960, 160, False, 'relu', 1],
|
||||
[5, 960, 160, False, 'relu', 1]],
|
||||
"cls_ch_squeeze": 960,
|
||||
"cls_ch_expand": 1280,
|
||||
}
|
||||
}
|
||||
|
||||
return GhostNet(model_cfgs[model_name], **kwargs)
|
||||
|
||||
|
||||
ghostnet_1x = partial(ghostnet, model_name="1x", final_drop=0.8)
|
||||
ghostnet_nose_1x = partial(ghostnet, model_name="nose_1x", final_drop=0.8)
|
||||
|
|
@ -0,0 +1,466 @@
|
|||
# Copyright 2020 Huawei Technologies Co., Ltd
|
||||
#
|
||||
# Licensed under the Apache License, Version 2.0 (the "License");
|
||||
# you may not use this file except in compliance with the License.
|
||||
# You may obtain a copy of the License at
|
||||
#
|
||||
# http://www.apache.org/licenses/LICENSE-2.0
|
||||
#
|
||||
# Unless required by applicable law or agreed to in writing, software
|
||||
# distributed under the License is distributed on an "AS IS" BASIS,
|
||||
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
# See the License for the specific language governing permissions and
|
||||
# limitations under the License.
|
||||
# ============================================================================
|
||||
"""GhostNet model define"""
|
||||
from functools import partial
|
||||
import math
|
||||
import numpy as np
|
||||
import mindspore.nn as nn
|
||||
from mindspore.ops import operations as P
|
||||
from mindspore import Tensor
|
||||
|
||||
|
||||
__all__ = ['ghostnet_600m']
|
||||
|
||||
|
||||
def _make_divisible(v, divisor=8, min_value=None):
|
||||
min_value = min_value or divisor
|
||||
new_v = max(min_value, int(v + divisor / 2) // divisor * divisor)
|
||||
# Make sure that round down does not go down by more than 10%.
|
||||
if new_v < 0.9 * v:
|
||||
new_v += divisor
|
||||
return new_v
|
||||
|
||||
|
||||
def _round_channels(channels, multiplier=1.0, divisor=8, channel_min=None):
|
||||
"""Round number of filters based on depth multiplier."""
|
||||
if not multiplier:
|
||||
return channels
|
||||
channels *= multiplier
|
||||
return _make_divisible(channels, divisor, channel_min)
|
||||
|
||||
|
||||
class MyHSigmoid(nn.Cell):
|
||||
def __init__(self):
|
||||
super(MyHSigmoid, self).__init__()
|
||||
self.relu6 = nn.ReLU6()
|
||||
|
||||
def construct(self, x):
|
||||
return self.relu6(x + 3.) / 6.
|
||||
|
||||
|
||||
class Activation(nn.Cell):
|
||||
"""
|
||||
Activation definition.
|
||||
|
||||
Args:
|
||||
act_func(string): activation name.
|
||||
|
||||
Returns:
|
||||
Tensor, output tensor.
|
||||
"""
|
||||
|
||||
def __init__(self, act_func):
|
||||
super(Activation, self).__init__()
|
||||
if act_func == 'relu':
|
||||
self.act = nn.ReLU()
|
||||
elif act_func == 'relu6':
|
||||
self.act = nn.ReLU6()
|
||||
elif act_func in ('hsigmoid', 'hard_sigmoid'):
|
||||
self.act = MyHSigmoid() # nn.HSigmoid()
|
||||
elif act_func in ('hswish', 'hard_swish'):
|
||||
self.act = nn.HSwish()
|
||||
else:
|
||||
raise NotImplementedError
|
||||
|
||||
def construct(self, x):
|
||||
return self.act(x)
|
||||
|
||||
|
||||
class GlobalAvgPooling(nn.Cell):
|
||||
"""
|
||||
Global avg pooling definition.
|
||||
|
||||
Args:
|
||||
|
||||
Returns:
|
||||
Tensor, output tensor.
|
||||
|
||||
Examples:
|
||||
>>> GlobalAvgPooling()
|
||||
"""
|
||||
|
||||
def __init__(self, keep_dims=False):
|
||||
super(GlobalAvgPooling, self).__init__()
|
||||
self.mean = P.ReduceMean(keep_dims=keep_dims)
|
||||
|
||||
def construct(self, x):
|
||||
x = self.mean(x, (2, 3))
|
||||
return x
|
||||
|
||||
|
||||
class SE(nn.Cell):
|
||||
"""
|
||||
SE warpper definition.
|
||||
|
||||
Args:
|
||||
num_out (int): Output channel.
|
||||
ratio (int): middle output ratio.
|
||||
|
||||
Returns:
|
||||
Tensor, output tensor.
|
||||
|
||||
Examples:
|
||||
>>> SE(4)
|
||||
"""
|
||||
|
||||
def __init__(self, num_out, ratio=4):
|
||||
super(SE, self).__init__()
|
||||
num_mid = _make_divisible(num_out // ratio)
|
||||
self.pool = GlobalAvgPooling(keep_dims=True)
|
||||
self.conv_reduce = nn.Conv2d(in_channels=num_out, out_channels=num_mid,
|
||||
kernel_size=1, has_bias=True, pad_mode='pad')
|
||||
self.act1 = Activation('relu')
|
||||
self.conv_expand = nn.Conv2d(in_channels=num_mid, out_channels=num_out,
|
||||
kernel_size=1, has_bias=True, pad_mode='pad')
|
||||
self.act2 = Activation('hsigmoid')
|
||||
self.mul = P.Mul()
|
||||
|
||||
def construct(self, x):
|
||||
out = self.pool(x)
|
||||
out = self.conv_reduce(out)
|
||||
out = self.act1(out)
|
||||
out = self.conv_expand(out)
|
||||
out = self.act2(out)
|
||||
out = self.mul(x, out)
|
||||
return out
|
||||
|
||||
|
||||
class ConvUnit(nn.Cell):
|
||||
"""
|
||||
ConvUnit warpper definition.
|
||||
|
||||
Args:
|
||||
num_in (int): Input channel.
|
||||
num_out (int): Output channel.
|
||||
kernel_size (int): Input kernel size.
|
||||
stride (int): Stride size.
|
||||
padding (int): Padding number.
|
||||
num_groups (int): Output num group.
|
||||
use_act (bool): Used activation or not.
|
||||
act_type (string): Activation type.
|
||||
|
||||
Returns:
|
||||
Tensor, output tensor.
|
||||
|
||||
Examples:
|
||||
>>> ConvUnit(3, 3)
|
||||
"""
|
||||
|
||||
def __init__(self, num_in, num_out, kernel_size=1, stride=1, padding=0, num_groups=1,
|
||||
use_act=True, act_type='relu'):
|
||||
super(ConvUnit, self).__init__()
|
||||
self.conv = nn.Conv2d(in_channels=num_in,
|
||||
out_channels=num_out,
|
||||
kernel_size=kernel_size,
|
||||
stride=stride,
|
||||
padding=padding,
|
||||
group=num_groups,
|
||||
has_bias=False,
|
||||
pad_mode='pad')
|
||||
self.bn = nn.BatchNorm2d(num_out, momentum=0.9)
|
||||
self.use_act = use_act
|
||||
self.act = Activation(act_type) if use_act else None
|
||||
|
||||
def construct(self, x):
|
||||
out = self.conv(x)
|
||||
out = self.bn(out)
|
||||
if self.use_act:
|
||||
out = self.act(out)
|
||||
return out
|
||||
|
||||
|
||||
class GhostModule(nn.Cell):
|
||||
"""
|
||||
GhostModule warpper definition.
|
||||
|
||||
Args:
|
||||
num_in (int): Input channel.
|
||||
num_out (int): Output channel.
|
||||
kernel_size (int): Input kernel size.
|
||||
stride (int): Stride size.
|
||||
padding (int): Padding number.
|
||||
use_act (bool): Used activation or not.
|
||||
act_type (string): Activation type.
|
||||
|
||||
Returns:
|
||||
Tensor, output tensor.
|
||||
|
||||
Examples:
|
||||
>>> GhostModule(3, 3)
|
||||
"""
|
||||
|
||||
def __init__(self, num_in, num_out, kernel_size=1, stride=1, padding=0, ratio=2, dw_size=3,
|
||||
use_act=True, act_type='relu'):
|
||||
super(GhostModule, self).__init__()
|
||||
init_channels = math.ceil(num_out / ratio)
|
||||
new_channels = init_channels * (ratio - 1)
|
||||
|
||||
self.primary_conv = ConvUnit(num_in, init_channels, kernel_size=kernel_size, stride=stride, padding=padding,
|
||||
num_groups=1, use_act=use_act, act_type=act_type)
|
||||
self.cheap_operation = ConvUnit(init_channels, new_channels, kernel_size=dw_size, stride=1, padding=dw_size//2,
|
||||
num_groups=init_channels, use_act=use_act, act_type=act_type)
|
||||
self.concat = P.Concat(axis=1)
|
||||
|
||||
def construct(self, x):
|
||||
x1 = self.primary_conv(x)
|
||||
x2 = self.cheap_operation(x1)
|
||||
return self.concat((x1, x2))
|
||||
|
||||
|
||||
class GhostBottleneck(nn.Cell):
|
||||
"""
|
||||
GhostBottleneck warpper definition.
|
||||
|
||||
Args:
|
||||
num_in (int): Input channel.
|
||||
num_mid (int): Middle channel.
|
||||
num_out (int): Output channel.
|
||||
kernel_size (int): Input kernel size.
|
||||
stride (int): Stride size.
|
||||
act_type (str): Activation type.
|
||||
use_se (bool): Use SE warpper or not.
|
||||
|
||||
Returns:
|
||||
Tensor, output tensor.
|
||||
|
||||
Examples:
|
||||
>>> GhostBottleneck(16, 3, 1, 1)
|
||||
"""
|
||||
|
||||
def __init__(self, num_in, num_mid, num_out, kernel_size, stride=1, act_type='relu', use_se=False):
|
||||
super(GhostBottleneck, self).__init__()
|
||||
self.ghost1 = GhostModule(num_in, num_mid, kernel_size=1,
|
||||
stride=1, padding=0, act_type=act_type)
|
||||
|
||||
self.use_dw = stride > 1
|
||||
self.dw = None
|
||||
if self.use_dw:
|
||||
self.dw = ConvUnit(num_mid, num_mid, kernel_size=kernel_size, stride=stride,
|
||||
padding=self._get_pad(kernel_size), act_type=act_type, num_groups=num_mid, use_act=False)
|
||||
if not use_se:
|
||||
self.use_se = use_se
|
||||
else:
|
||||
self.use_se = True
|
||||
self.se = SE(num_mid, ratio=use_se)
|
||||
|
||||
self.ghost2 = GhostModule(num_mid, num_out, kernel_size=1, stride=1,
|
||||
padding=0, act_type=act_type, use_act=False)
|
||||
|
||||
self.down_sample = False
|
||||
if num_in != num_out or stride != 1:
|
||||
self.down_sample = True
|
||||
self.shortcut = None
|
||||
if self.down_sample:
|
||||
self.shortcut = nn.SequentialCell([
|
||||
ConvUnit(num_in, num_in, kernel_size=kernel_size, stride=stride,
|
||||
padding=self._get_pad(kernel_size), num_groups=num_in, use_act=False),
|
||||
ConvUnit(num_in, num_out, kernel_size=1, stride=1,
|
||||
padding=0, num_groups=1, use_act=False),
|
||||
])
|
||||
self.add = P.TensorAdd()
|
||||
|
||||
def construct(self, x):
|
||||
"""construct"""
|
||||
shortcut = x
|
||||
out = self.ghost1(x)
|
||||
if self.use_dw:
|
||||
out = self.dw(out)
|
||||
if self.use_se:
|
||||
out = self.se(out)
|
||||
out = self.ghost2(out)
|
||||
if self.down_sample:
|
||||
shortcut = self.shortcut(shortcut)
|
||||
out = self.add(shortcut, out)
|
||||
return out
|
||||
|
||||
def _get_pad(self, kernel_size):
|
||||
"""set the padding number"""
|
||||
pad = 0
|
||||
if kernel_size == 1:
|
||||
pad = 0
|
||||
elif kernel_size == 3:
|
||||
pad = 1
|
||||
elif kernel_size == 5:
|
||||
pad = 2
|
||||
elif kernel_size == 7:
|
||||
pad = 3
|
||||
else:
|
||||
raise NotImplementedError
|
||||
return pad
|
||||
|
||||
|
||||
class GhostNet(nn.Cell):
|
||||
"""
|
||||
GhostNet architecture.
|
||||
|
||||
Args:
|
||||
model_cfgs (Cell): number of classes.
|
||||
num_classes (int): Output number classes.
|
||||
multiplier (int): Channels multiplier for round to 8/16 and others. Default is 1.
|
||||
final_drop (float): Dropout number.
|
||||
round_nearest (list): Channel round to . Default is 8.
|
||||
Returns:
|
||||
Tensor, output tensor.
|
||||
|
||||
Examples:
|
||||
>>> GhostNet(num_classes=1000)
|
||||
"""
|
||||
|
||||
def __init__(self, model_cfgs, num_classes=1000, multiplier=1., final_drop=0., round_nearest=8):
|
||||
super(GhostNet, self).__init__()
|
||||
self.cfgs = model_cfgs['cfg']
|
||||
self.inplanes = 16
|
||||
first_conv_in_channel = 3
|
||||
first_conv_out_channel = _round_channels(
|
||||
self.inplanes, multiplier, divisor=4)
|
||||
self.inplanes = first_conv_out_channel
|
||||
self.conv_stem = nn.Conv2d(in_channels=first_conv_in_channel,
|
||||
out_channels=first_conv_out_channel,
|
||||
kernel_size=3, padding=1, stride=2,
|
||||
has_bias=False, pad_mode='pad')
|
||||
self.bn1 = nn.BatchNorm2d(first_conv_out_channel, momentum=0.9)
|
||||
self.act1 = Activation('hswish')
|
||||
|
||||
self.blocks = []
|
||||
for layer_cfg in self.cfgs:
|
||||
#print (layer_cfg)
|
||||
self.blocks.append(self._make_layer(kernel_size=layer_cfg[0],
|
||||
exp_ch=_make_divisible(
|
||||
self.inplanes * layer_cfg[3]),
|
||||
out_channel=_round_channels(
|
||||
layer_cfg[2], multiplier, 4),
|
||||
use_se=layer_cfg[4],
|
||||
act_func=layer_cfg[5],
|
||||
stride=layer_cfg[6]))
|
||||
output_channel = _make_divisible(
|
||||
multiplier * model_cfgs["cls_ch_squeeze"])
|
||||
self.blocks.append(ConvUnit(_make_divisible(multiplier * self.cfgs[-1][2]), output_channel,
|
||||
kernel_size=1, stride=1, padding=0, num_groups=1, use_act=True, act_type='hswish'))
|
||||
self.blocks = nn.SequentialCell(self.blocks)
|
||||
|
||||
self.global_pool = GlobalAvgPooling(keep_dims=True)
|
||||
self.conv_head = nn.Conv2d(in_channels=output_channel,
|
||||
out_channels=model_cfgs['cls_ch_expand'],
|
||||
kernel_size=1, padding=0, stride=1,
|
||||
has_bias=True, pad_mode='pad')
|
||||
self.act2 = Activation('hswish')
|
||||
self.squeeze = P.Squeeze(axis=(2, 3))
|
||||
self.final_drop = final_drop
|
||||
if self.final_drop > 0:
|
||||
self.dropout = nn.Dropout(self.final_drop)
|
||||
|
||||
self.classifier = nn.Dense(
|
||||
model_cfgs['cls_ch_expand'], num_classes, has_bias=True)
|
||||
|
||||
self._initialize_weights()
|
||||
|
||||
def construct(self, x):
|
||||
r"""construct of GhostNet"""
|
||||
x = self.conv_stem(x)
|
||||
x = self.bn1(x)
|
||||
x = self.act1(x)
|
||||
x = self.blocks(x)
|
||||
x = self.global_pool(x)
|
||||
x = self.conv_head(x)
|
||||
x = self.act2(x)
|
||||
x = self.squeeze(x)
|
||||
if self.final_drop > 0:
|
||||
x = self.dropout(x)
|
||||
x = self.classifier(x)
|
||||
return x
|
||||
|
||||
def _make_layer(self, kernel_size, exp_ch, out_channel, use_se, act_func, stride=1):
|
||||
mid_planes = exp_ch
|
||||
out_planes = out_channel
|
||||
layer = GhostBottleneck(self.inplanes, mid_planes, out_planes,
|
||||
kernel_size, stride=stride, act_type=act_func, use_se=use_se)
|
||||
self.inplanes = out_planes
|
||||
return layer
|
||||
|
||||
def _initialize_weights(self):
|
||||
"""
|
||||
Initialize weights.
|
||||
|
||||
Args:
|
||||
|
||||
Returns:
|
||||
None.
|
||||
|
||||
Examples:
|
||||
>>> _initialize_weights()
|
||||
"""
|
||||
self.init_parameters_data()
|
||||
for _, m in self.cells_and_names():
|
||||
if isinstance(m, (nn.Conv2d)):
|
||||
n = m.kernel_size[0] * m.kernel_size[1] * m.out_channels
|
||||
m.weight.set_parameter_data(Tensor(np.random.normal(0, np.sqrt(2. / n),
|
||||
m.weight.data.shape).astype("float32")))
|
||||
if m.bias is not None:
|
||||
m.bias.set_parameter_data(
|
||||
Tensor(np.zeros(m.bias.data.shape, dtype="float32")))
|
||||
elif isinstance(m, nn.BatchNorm2d):
|
||||
m.gamma.set_parameter_data(
|
||||
Tensor(np.ones(m.gamma.data.shape, dtype="float32")))
|
||||
m.beta.set_parameter_data(
|
||||
Tensor(np.zeros(m.beta.data.shape, dtype="float32")))
|
||||
elif isinstance(m, nn.Dense):
|
||||
m.weight.set_parameter_data(Tensor(np.random.normal(
|
||||
0, 0.01, m.weight.data.shape).astype("float32")))
|
||||
if m.bias is not None:
|
||||
m.bias.set_parameter_data(
|
||||
Tensor(np.zeros(m.bias.data.shape, dtype="float32")))
|
||||
|
||||
|
||||
def ghostnet(model_name, **kwargs):
|
||||
"""
|
||||
Constructs a GhostNet model
|
||||
"""
|
||||
model_cfgs = {
|
||||
"600M": {
|
||||
"cfg": [
|
||||
# k, exp, c, exp_ratio, se, nl, s,
|
||||
# stage1
|
||||
[3, 16, 16, 1, 10, 'hswish', 1],
|
||||
[3, 48, 24, 3, 10, 'hswish', 2],
|
||||
# stage2
|
||||
[3, 72, 24, 3, 10, 'hswish', 1],
|
||||
[5, 72, 40, 3, 10, 'hswish', 2],
|
||||
# stage3
|
||||
[3, 120, 40, 3, 10, 'hswish', 1],
|
||||
[3, 120, 40, 3, 10, 'hswish', 1],
|
||||
[3, 240, 80, 6, 10, 'hswish', 2],
|
||||
# stage4
|
||||
[3, 200, 80, 2.5, 10, 'hswish', 1],
|
||||
[3, 200, 80, 2.5, 10, 'hswish', 1],
|
||||
[3, 200, 80, 2.5, 10, 'hswish', 1],
|
||||
[3, 480, 112, 6, 10, 'hswish', 1],
|
||||
[3, 672, 112, 6, 10, 'hswish', 1],
|
||||
[3, 672, 112, 6, 10, 'hswish', 1],
|
||||
[5, 672, 160, 6, 10, 'hswish', 2],
|
||||
# stage5
|
||||
[3, 960, 160, 6, 10, 'hswish', 1],
|
||||
[3, 960, 160, 6, 10, 'hswish', 1],
|
||||
[3, 960, 160, 6, 10, 'hswish', 1],
|
||||
[3, 960, 160, 6, 10, 'hswish', 1],
|
||||
[3, 960, 160, 6, 10, 'hswish', 1]],
|
||||
"cls_ch_squeeze": 960,
|
||||
"cls_ch_expand": 1400,
|
||||
}
|
||||
}
|
||||
return GhostNet(model_cfgs[model_name], **kwargs)
|
||||
|
||||
|
||||
ghostnet_600m = partial(ghostnet, model_name="600M",
|
||||
multiplier=1.75, final_drop=0.8)
|
||||
|
|
@ -0,0 +1,165 @@
|
|||
# Copyright 2020 Huawei Technologies Co., Ltd
|
||||
#
|
||||
# Licensed under the Apache License, Version 2.0 (the "License");
|
||||
# you may not use this file except in compliance with the License.
|
||||
# You may obtain a copy of the License at
|
||||
#
|
||||
# http://www.apache.org/licenses/LICENSE-2.0
|
||||
#
|
||||
# Unless required by applicable law or agreed to in writing, software
|
||||
# distributed under the License is distributed on an "AS IS" BASIS,
|
||||
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
# See the License for the specific language governing permissions and
|
||||
# limitations under the License.
|
||||
# ============================================================================
|
||||
"""launch train script"""
|
||||
import os
|
||||
import sys
|
||||
import json
|
||||
import subprocess
|
||||
import shutil
|
||||
from argparse import ArgumentParser
|
||||
|
||||
|
||||
def parse_args():
|
||||
"""
|
||||
parse args .
|
||||
|
||||
Args:
|
||||
|
||||
Returns:
|
||||
args.
|
||||
|
||||
Examples:
|
||||
>>> parse_args()
|
||||
"""
|
||||
parser = ArgumentParser(description="mindspore distributed training launch "
|
||||
"helper utilty that will spawn up "
|
||||
"multiple distributed processes")
|
||||
parser.add_argument("--nproc_per_node", type=int, default=1,
|
||||
help="The number of processes to launch on each node, "
|
||||
"for D training, this is recommended to be set "
|
||||
"to the number of D in your system so that "
|
||||
"each process can be bound to a single D.")
|
||||
parser.add_argument("--visible_devices", type=str, default="0,1,2,3,4,5,6,7",
|
||||
help="will use the visible devices sequentially")
|
||||
parser.add_argument("--server_id", type=str, default="",
|
||||
help="server ip")
|
||||
parser.add_argument("--training_script", type=str,
|
||||
help="The full path to the single D training "
|
||||
"program/script to be launched in parallel, "
|
||||
"followed by all the arguments for the "
|
||||
"training script")
|
||||
# rest from the training program
|
||||
args, unknown = parser.parse_known_args()
|
||||
args.training_script_args = unknown
|
||||
return args
|
||||
|
||||
|
||||
def main():
|
||||
print("start", __file__)
|
||||
args = parse_args()
|
||||
print(args)
|
||||
visible_devices = args.visible_devices.split(',')
|
||||
assert os.path.isfile(args.training_script)
|
||||
assert len(visible_devices) >= args.nproc_per_node
|
||||
print('visible_devices:{}'.format(visible_devices))
|
||||
if not args.server_id:
|
||||
print('pleaser input server ip!!!')
|
||||
exit(0)
|
||||
print('server_id:{}'.format(args.server_id))
|
||||
|
||||
# construct hccn_table
|
||||
hccn_configs = open('/etc/hccn.conf', 'r').readlines()
|
||||
device_ips = {}
|
||||
for hccn_item in hccn_configs:
|
||||
hccn_item = hccn_item.strip()
|
||||
if hccn_item.startswith('address_'):
|
||||
device_id, device_ip = hccn_item.split('=')
|
||||
device_id = device_id.split('_')[1]
|
||||
device_ips[device_id] = device_ip
|
||||
print('device_id:{}, device_ip:{}'.format(device_id, device_ip))
|
||||
hccn_table = {}
|
||||
hccn_table['board_id'] = '0x0000'
|
||||
hccn_table['chip_info'] = '910'
|
||||
hccn_table['deploy_mode'] = 'lab'
|
||||
hccn_table['group_count'] = '1'
|
||||
hccn_table['group_list'] = []
|
||||
instance_list = []
|
||||
usable_dev = ''
|
||||
for instance_id in range(args.nproc_per_node):
|
||||
instance = {}
|
||||
instance['devices'] = []
|
||||
device_id = visible_devices[instance_id]
|
||||
device_ip = device_ips[device_id]
|
||||
usable_dev += str(device_id)
|
||||
instance['devices'].append({
|
||||
'device_id': device_id,
|
||||
'device_ip': device_ip,
|
||||
})
|
||||
instance['rank_id'] = str(instance_id)
|
||||
instance['server_id'] = args.server_id
|
||||
instance_list.append(instance)
|
||||
hccn_table['group_list'].append({
|
||||
'device_num': str(args.nproc_per_node),
|
||||
'server_num': '1',
|
||||
'group_name': '',
|
||||
'instance_count': str(args.nproc_per_node),
|
||||
'instance_list': instance_list,
|
||||
})
|
||||
hccn_table['para_plane_nic_location'] = 'device'
|
||||
hccn_table['para_plane_nic_name'] = []
|
||||
for instance_id in range(args.nproc_per_node):
|
||||
eth_id = visible_devices[instance_id]
|
||||
hccn_table['para_plane_nic_name'].append('eth{}'.format(eth_id))
|
||||
hccn_table['para_plane_nic_num'] = str(args.nproc_per_node)
|
||||
hccn_table['status'] = 'completed'
|
||||
|
||||
# save hccn_table to file
|
||||
table_path = os.getcwd()
|
||||
if not os.path.exists(table_path):
|
||||
os.mkdir(table_path)
|
||||
table_fn = os.path.join(table_path,
|
||||
'rank_table_{}p_{}_{}.json'.format(args.nproc_per_node, usable_dev, args.server_id))
|
||||
with open(table_fn, 'w') as table_fp:
|
||||
json.dump(hccn_table, table_fp, indent=4)
|
||||
sys.stdout.flush()
|
||||
|
||||
# spawn the processes
|
||||
processes = []
|
||||
cmds = []
|
||||
log_files = []
|
||||
env = os.environ.copy()
|
||||
env['RANK_SIZE'] = str(args.nproc_per_node)
|
||||
cur_path = os.getcwd()
|
||||
for rank_id in range(0, args.nproc_per_node):
|
||||
os.chdir(cur_path)
|
||||
device_id = visible_devices[rank_id]
|
||||
device_dir = os.path.join(cur_path, 'device{}'.format(rank_id))
|
||||
env['RANK_ID'] = str(rank_id)
|
||||
env['DEVICE_ID'] = str(device_id)
|
||||
if args.nproc_per_node > 1:
|
||||
env['RANK_TABLE_FILE'] = table_fn
|
||||
if os.path.exists(device_dir):
|
||||
shutil.rmtree(device_dir)
|
||||
os.mkdir(device_dir)
|
||||
os.chdir(device_dir)
|
||||
cmd = [sys.executable, '-u']
|
||||
cmd.append(args.training_script)
|
||||
cmd.extend(args.training_script_args)
|
||||
log_file = open(
|
||||
'{dir}/log{id}.log'.format(dir=device_dir, id=rank_id), 'w')
|
||||
process = subprocess.Popen(
|
||||
cmd, stdout=log_file, stderr=log_file, env=env)
|
||||
processes.append(process)
|
||||
cmds.append(cmd)
|
||||
log_files.append(log_file)
|
||||
for process, cmd, log_file in zip(processes, cmds, log_files):
|
||||
process.wait()
|
||||
if process.returncode != 0:
|
||||
raise subprocess.CalledProcessError(returncode=process, cmd=cmd)
|
||||
log_file.close()
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
|
|
@ -0,0 +1,55 @@
|
|||
# Copyright 2020 Huawei Technologies Co., Ltd
|
||||
#
|
||||
# Licensed under the Apache License, Version 2.0 (the "License");
|
||||
# you may not use this file except in compliance with the License.
|
||||
# You may obtain a copy of the License at
|
||||
#
|
||||
# http://www.apache.org/licenses/LICENSE-2.0
|
||||
#
|
||||
# Unless required by applicable law or agreed to in writing, software
|
||||
# distributed under the License is distributed on an "AS IS" BASIS,
|
||||
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
# See the License for the specific language governing permissions and
|
||||
# limitations under the License.
|
||||
# ============================================================================
|
||||
"""learning rate generator"""
|
||||
import math
|
||||
import numpy as np
|
||||
|
||||
|
||||
def get_lr(global_step, lr_init, lr_end, lr_max, warmup_epochs, total_epochs, steps_per_epoch):
|
||||
"""
|
||||
generate learning rate array
|
||||
|
||||
Args:
|
||||
global_step(int): total steps of the training
|
||||
lr_init(float): init learning rate
|
||||
lr_end(float): end learning rate
|
||||
lr_max(float): max learning rate
|
||||
warmup_epochs(int): number of warmup epochs
|
||||
total_epochs(int): total epoch of training
|
||||
steps_per_epoch(int): steps of one epoch
|
||||
|
||||
Returns:
|
||||
np.array, learning rate array
|
||||
"""
|
||||
lr_each_step = []
|
||||
total_steps = steps_per_epoch * total_epochs
|
||||
warmup_steps = steps_per_epoch * warmup_epochs
|
||||
for i in range(total_steps):
|
||||
if i < warmup_steps:
|
||||
lr = lr_init + (lr_max - lr_init) * i / warmup_steps
|
||||
else:
|
||||
lr = lr_end + \
|
||||
(lr_max - lr_end) * \
|
||||
(1. + math.cos(math.pi * (i - warmup_steps) /
|
||||
(total_steps - warmup_steps))) / 2.
|
||||
if lr < 0.0:
|
||||
lr = 0.0
|
||||
lr_each_step.append(lr)
|
||||
|
||||
current_step = global_step
|
||||
lr_each_step = np.array(lr_each_step).astype(np.float32)
|
||||
learning_rate = lr_each_step[current_step:]
|
||||
|
||||
return learning_rate
|
||||
|
|
@ -0,0 +1,136 @@
|
|||
# Contents
|
||||
|
||||
- [GhostNet Description](#ghostnet-description)
|
||||
- [Quantization Description](#ghostnet-quantization-description)
|
||||
- [Model Architecture](#model-architecture)
|
||||
- [Dataset](#dataset)
|
||||
- [Environment Requirements](#environment-requirements)
|
||||
- [Script Description](#script-description)
|
||||
- [Script and Sample Code](#script-and-sample-code)
|
||||
- [Training Process](#training-process)
|
||||
- [Evaluation Process](#evaluation-process)
|
||||
- [Evaluation](#evaluation)
|
||||
- [Model Description](#model-description)
|
||||
- [Performance](#performance)
|
||||
- [Training Performance](#evaluation-performance)
|
||||
- [Inference Performance](#evaluation-performance)
|
||||
- [Description of Random Situation](#description-of-random-situation)
|
||||
- [ModelZoo Homepage](#modelzoo-homepage)
|
||||
|
||||
# [GhostNet Description](#contents)
|
||||
|
||||
The GhostNet architecture is based on an Ghost module structure which generate more features from cheap operations. Based on a set of intrinsic feature maps, a series of cheap operations are applied to generate many ghost feature maps that could fully reveal information underlying intrinsic features.
|
||||
|
||||
[Paper](https://openaccess.thecvf.com/content_CVPR_2020/papers/Han_GhostNet_More_Features_From_Cheap_Operations_CVPR_2020_paper.pdf): Kai Han, Yunhe Wang, Qi Tian, Jianyuan Guo, Chunjing Xu, Chang Xu. GhostNet: More Features from Cheap Operations. CVPR 2020.
|
||||
|
||||
# [Quantization Description](#contents)
|
||||
|
||||
Quantization refers to techniques for performing computations and storing tensors at lower bitwidths than floating point precision. For 8bit quantization, we quantize the weights into [-128,127] and the activations into [0,255]. We finetune the model a few epochs after post-quantization to achieve better performance.
|
||||
|
||||
# [Model architecture](#contents)
|
||||
|
||||
The overall network architecture of GhostNet is show below:
|
||||
|
||||
[Link](https://openaccess.thecvf.com/content_CVPR_2020/papers/Han_GhostNet_More_Features_From_Cheap_Operations_CVPR_2020_paper.pdf)
|
||||
|
||||
# [Dataset](#contents)
|
||||
|
||||
Dataset used: [Oxford-IIIT Pet](https://www.robots.ox.ac.uk/~vgg/data/pets/)
|
||||
|
||||
- Dataset size: 7049 colorful images in 1000 classes
|
||||
- Train: 3680 images
|
||||
- Test: 3369 images
|
||||
- Data format: RGB images.
|
||||
- Note: Data will be processed in src/dataset.py
|
||||
|
||||
# [Environment Requirements](#contents)
|
||||
|
||||
- Hardware（Ascend/GPU）
|
||||
- Prepare hardware environment with Ascend or GPU processor. If you want to try Ascend, please send the [application form](https://obs-9be7.obs.cn-east-2.myhuaweicloud.com/file/other/Ascend%20Model%20Zoo%E4%BD%93%E9%AA%8C%E8%B5%84%E6%BA%90%E7%94%B3%E8%AF%B7%E8%A1%A8.docx) to ascend@huawei.com. Once approved, you can get the resources.
|
||||
- Framework
|
||||
- [MindSpore](http://10.90.67.50/mindspore/archive/20200506/OpenSource/me_vm_x86/)
|
||||
- For more information, please check the resources below：
|
||||
- [MindSpore tutorials](https://www.mindspore.cn/tutorial/zh-CN/master/index.html)
|
||||
- [MindSpore API](https://www.mindspore.cn/api/zh-CN/master/index.html)
|
||||
|
||||
# [Script description](#contents)
|
||||
|
||||
## [Script and sample code](#contents)
|
||||
|
||||
```python
|
||||
├── GhostNet
|
||||
├── Readme.md # descriptions about GhostNet # shell script for evaluation with CPU, GPU or Ascend
|
||||
├── src
|
||||
│ ├──config.py # parameter configuration
|
||||
│ ├──dataset.py # creating dataset
|
||||
│ ├──launch.py # start python script
|
||||
│ ├──lr_generator.py # learning rate config
|
||||
│ ├──ghostnet.py # GhostNet architecture
|
||||
│ ├──quant.py # GhostNet quantization
|
||||
├── eval.py # evaluation script
|
||||
├── mindspore_hub_conf.py # export model for hub
|
||||
```
|
||||
|
||||
## [Training process](#contents)
|
||||
To Be Done
|
||||
|
||||
## [Eval process](#contents)
|
||||
|
||||
### Usage
|
||||
|
||||
After installing MindSpore via the official website, you can start evaluation as follows:
|
||||
|
||||
|
||||
### Launch
|
||||
|
||||
```
|
||||
# infer example
|
||||
|
||||
Ascend: python eval.py --dataset_path ~/Pets/test.mindrecord --platform Ascend --checkpoint_path [CHECKPOINT_PATH]
|
||||
GPU: python eval.py --dataset_path ~/Pets/test.mindrecord --platform GPU --checkpoint_path [CHECKPOINT_PATH]
|
||||
```
|
||||
|
||||
> checkpoint can be produced in training process.
|
||||
|
||||
### Result
|
||||
|
||||
```
|
||||
result: {'acc': 0.825} ckpt= ./ghostnet_1x_pets_int8.ckpt
|
||||
```
|
||||
|
||||
# [Model Description](#contents)
|
||||
|
||||
## [Performance](#contents)
|
||||
|
||||
#### Evaluation Performance
|
||||
|
||||
###### GhostNet on ImageNet2012
|
||||
| Parameters | | |
|
||||
| -------------------------- | -------------------------------------- |---------------------------------- |
|
||||
| Model Version | GhostNet |GhostNet-int8|
|
||||
| uploaded Date | 09/08/2020 (month/day/year) ； | 09/08/2020 (month/day/year) |
|
||||
| MindSpore Version | 0.6.0-alpha |0.6.0-alpha |
|
||||
| Dataset | ImageNet2012 | ImageNet2012|
|
||||
| Parameters (M) | 5.2 | / |
|
||||
| FLOPs (M) | 142 | / |
|
||||
| Accuracy (Top1) | 73.9 | w/o finetune:72.2, w finetune:73.6 |
|
||||
|
||||
###### GhostNet on Oxford-IIIT Pet
|
||||
| Parameters | | |
|
||||
| -------------------------- | -------------------------------------- |---------------------------------- |
|
||||
| Model Version | GhostNet |GhostNet-int8|
|
||||
| uploaded Date | 09/08/2020 (month/day/year) ； | 09/08/2020 (month/day/year) |
|
||||
| MindSpore Version | 0.6.0-alpha |0.6.0-alpha |
|
||||
| Dataset | Oxford-IIIT Pet | Oxford-IIIT Pet|
|
||||
| Parameters (M) | 3.9 | / |
|
||||
| FLOPs (M) | 140 | / |
|
||||
| Accuracy (Top1) | 82.4 | w/o finetune:81.66, w finetune:82.45 |
|
||||
|
||||
|
||||
# [Description of Random Situation](#contents)
|
||||
|
||||
In dataset.py, we set the seed inside “create_dataset" function. We also use random seed in train.py.
|
||||
|
||||
# [ModelZoo Homepage](#contents)
|
||||
|
||||
Please check the official [homepage](https://gitee.com/mindspore/mindspore/tree/master/model_zoo).
|
||||
|
|
@ -0,0 +1,77 @@
|
|||
# Copyright 2020 Huawei Technologies Co., Ltd
|
||||
#
|
||||
# Licensed under the Apache License, Version 2.0 (the "License");
|
||||
# you may not use this file except in compliance with the License.
|
||||
# You may obtain a copy of the License at
|
||||
#
|
||||
# http://www.apache.org/licenses/LICENSE-2.0
|
||||
#
|
||||
# Unless required by applicable law or agreed to in writing, software
|
||||
# distributed under the License is distributed on an "AS IS" BASIS,
|
||||
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
# See the License for the specific language governing permissions and
|
||||
# limitations under the License.
|
||||
# ============================================================================
|
||||
"""
|
||||
eval.
|
||||
"""
|
||||
import os
|
||||
import argparse
|
||||
from mindspore import context
|
||||
from mindspore import nn
|
||||
from mindspore.train.model import Model
|
||||
from mindspore.train.serialization import load_checkpoint, load_param_into_net
|
||||
from mindspore.common import dtype as mstype
|
||||
from src.dataset import create_dataset
|
||||
from src.config import config_ascend, config_gpu
|
||||
from src.ghostnet import ghostnet_1x
|
||||
|
||||
parser = argparse.ArgumentParser(description='Image classification')
|
||||
parser.add_argument('--checkpoint_path', type=str,
|
||||
default=None, help='Checkpoint file path')
|
||||
parser.add_argument('--dataset_path', type=str,
|
||||
default=None, help='Dataset path')
|
||||
parser.add_argument('--platform', type=str, default=None, help='run platform')
|
||||
args_opt = parser.parse_args()
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
config_platform = None
|
||||
if args_opt.platform == "Ascend":
|
||||
config_platform = config_ascend
|
||||
device_id = int(os.getenv('DEVICE_ID'))
|
||||
context.set_context(mode=context.GRAPH_MODE, device_target="Ascend",
|
||||
device_id=device_id, save_graphs=False)
|
||||
elif args_opt.platform == "GPU":
|
||||
config_platform = config_gpu
|
||||
context.set_context(mode=context.GRAPH_MODE,
|
||||
device_target="GPU", save_graphs=False)
|
||||
else:
|
||||
raise ValueError("Unsupport platform.")
|
||||
|
||||
loss = nn.SoftmaxCrossEntropyWithLogits(
|
||||
is_grad=False, sparse=True, reduction='mean')
|
||||
|
||||
net = ghostnet_1x(num_classes=config_platform.num_classes)
|
||||
|
||||
if args_opt.platform == "Ascend":
|
||||
net.to_float(mstype.float16)
|
||||
for _, cell in net.cells_and_names():
|
||||
if isinstance(cell, nn.Dense):
|
||||
cell.to_float(mstype.float32)
|
||||
|
||||
dataset = create_dataset(dataset_path=args_opt.dataset_path,
|
||||
do_train=False,
|
||||
config=config_platform,
|
||||
platform=args_opt.platform,
|
||||
batch_size=config_platform.batch_size)
|
||||
step_size = dataset.get_dataset_size()
|
||||
|
||||
if args_opt.checkpoint_path:
|
||||
param_dict = load_checkpoint(args_opt.checkpoint_path)
|
||||
load_param_into_net(net, param_dict)
|
||||
net.set_train(False)
|
||||
|
||||
model = Model(net, loss_fn=loss, metrics={'acc'})
|
||||
res = model.eval(dataset)
|
||||
print("result:", res, "ckpt=", args_opt.checkpoint_path)
|
||||
|
|
@ -0,0 +1,22 @@
|
|||
# Copyright 2020 Huawei Technologies Co., Ltd
|
||||
#
|
||||
# Licensed under the Apache License, Version 2.0 (the "License");
|
||||
# you may not use this file except in compliance with the License.
|
||||
# You may obtain a copy of the License at
|
||||
#
|
||||
# http://www.apache.org/licenses/LICENSE-2.0
|
||||
#
|
||||
# Unless required by applicable law or agreed to in writing, software
|
||||
# distributed under the License is distributed on an "AS IS" BASIS,
|
||||
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
# See the License for the specific language governing permissions and
|
||||
# limitations under the License.
|
||||
# ============================================================================
|
||||
"""hub config."""
|
||||
from src.ghostnet import ghostnet_1x
|
||||
|
||||
|
||||
def create_network(name, *args, **kwargs):
|
||||
if name == 'ghostnet_int8':
|
||||
return ghostnet_1x(*args, **kwargs)
|
||||
raise NotImplementedError(f"{name} is not implemented in the repo")
|
||||
|
|
@ -0,0 +1,54 @@
|
|||
# Copyright 2020 Huawei Technologies Co., Ltd
|
||||
#
|
||||
# Licensed under the Apache License, Version 2.0 (the "License");
|
||||
# you may not use this file except in compliance with the License.
|
||||
# You may obtain a copy of the License at
|
||||
#
|
||||
# http://www.apache.org/licenses/LICENSE-2.0
|
||||
#
|
||||
# Unless required by applicable law or agreed to in writing, software
|
||||
# distributed under the License is distributed on an "AS IS" BASIS,
|
||||
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
# See the License for the specific language governing permissions and
|
||||
# limitations under the License.
|
||||
# ============================================================================
|
||||
"""
|
||||
network config setting, will be used in train.py and eval.py
|
||||
"""
|
||||
from easydict import EasyDict as ed
|
||||
|
||||
config_ascend = ed({
|
||||
"num_classes": 37,
|
||||
"image_height": 224,
|
||||
"image_width": 224,
|
||||
"batch_size": 256,
|
||||
"epoch_size": 200,
|
||||
"warmup_epochs": 4,
|
||||
"lr": 0.4,
|
||||
"momentum": 0.9,
|
||||
"weight_decay": 4e-5,
|
||||
"label_smooth": 0.1,
|
||||
"loss_scale": 1024,
|
||||
"save_checkpoint": True,
|
||||
"save_checkpoint_epochs": 1,
|
||||
"keep_checkpoint_max": 200,
|
||||
"save_checkpoint_path": "./checkpoint",
|
||||
})
|
||||
|
||||
config_gpu = ed({
|
||||
"num_classes": 37,
|
||||
"image_height": 224,
|
||||
"image_width": 224,
|
||||
"batch_size": 3,
|
||||
"epoch_size": 370,
|
||||
"warmup_epochs": 4,
|
||||
"lr": 0.4,
|
||||
"momentum": 0.9,
|
||||
"weight_decay": 4e-5,
|
||||
"label_smooth": 0.1,
|
||||
"loss_scale": 1024,
|
||||
"save_checkpoint": True,
|
||||
"save_checkpoint_epochs": 1,
|
||||
"keep_checkpoint_max": 500,
|
||||
"save_checkpoint_path": "./checkpoint",
|
||||
})
|
||||
|
|
@ -0,0 +1,110 @@
|
|||
# Copyright 2020 Huawei Technologies Co., Ltd
|
||||
#
|
||||
# Licensed under the Apache License, Version 2.0 (the "License");
|
||||
# you may not use this file except in compliance with the License.
|
||||
# You may obtain a copy of the License at
|
||||
#
|
||||
# http://www.apache.org/licenses/LICENSE-2.0
|
||||
#
|
||||
# Unless required by applicable law or agreed to in writing, software
|
||||
# distributed under the License is distributed on an "AS IS" BASIS,
|
||||
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
# See the License for the specific language governing permissions and
|
||||
# limitations under the License.
|
||||
# ============================================================================
|
||||
"""
|
||||
create train or eval dataset.
|
||||
"""
|
||||
import os
|
||||
import mindspore.common.dtype as mstype
|
||||
import mindspore.dataset.engine as de
|
||||
import mindspore.dataset.transforms.vision.c_transforms as C
|
||||
import mindspore.dataset.transforms.vision.py_transforms as P
|
||||
import mindspore.dataset.transforms.c_transforms as C2
|
||||
from mindspore.dataset.transforms.vision import Inter
|
||||
|
||||
|
||||
def create_dataset(dataset_path, do_train, config, platform, repeat_num=1, batch_size=100, model='ghsotnet'):
|
||||
"""
|
||||
create a train or eval dataset
|
||||
|
||||
Args:
|
||||
dataset_path(string): the path of dataset.
|
||||
do_train(bool): whether dataset is used for train or eval.
|
||||
repeat_num(int): the repeat times of dataset. Default: 1
|
||||
batch_size(int): the batch size of dataset. Default: 32
|
||||
|
||||
Returns:
|
||||
dataset
|
||||
"""
|
||||
if platform == "Ascend":
|
||||
rank_size = int(os.getenv("RANK_SIZE"))
|
||||
rank_id = int(os.getenv("RANK_ID"))
|
||||
if rank_size == 1:
|
||||
ds = de.MindDataset(
|
||||
dataset_path, num_parallel_workers=8, shuffle=True)
|
||||
else:
|
||||
ds = de.MindDataset(dataset_path, num_parallel_workers=8, shuffle=True,
|
||||
num_shards=rank_size, shard_id=rank_id)
|
||||
elif platform == "GPU":
|
||||
if do_train:
|
||||
from mindspore.communication.management import get_rank, get_group_size
|
||||
ds = de.MindDataset(dataset_path, num_parallel_workers=8, shuffle=True,
|
||||
num_shards=get_group_size(), shard_id=get_rank())
|
||||
else:
|
||||
ds = de.MindDataset(
|
||||
dataset_path, num_parallel_workers=8, shuffle=True)
|
||||
else:
|
||||
raise ValueError("Unsupport platform.")
|
||||
|
||||
resize_height = config.image_height
|
||||
buffer_size = 1000
|
||||
|
||||
# define map operations
|
||||
resize_crop_op = C.RandomCropDecodeResize(
|
||||
resize_height, scale=(0.08, 1.0), ratio=(0.75, 1.333))
|
||||
horizontal_flip_op = C.RandomHorizontalFlip(prob=0.5)
|
||||
|
||||
color_op = C.RandomColorAdjust(
|
||||
brightness=0.4, contrast=0.4, saturation=0.4)
|
||||
rescale_op = C.Rescale(1/255.0, 0)
|
||||
normalize_op = C.Normalize(
|
||||
mean=[0.485, 0.456, 0.406], std=[0.229, 0.224, 0.225])
|
||||
change_swap_op = C.HWC2CHW()
|
||||
|
||||
# define python operations
|
||||
decode_p = P.Decode()
|
||||
if model == 'ghostnet-600':
|
||||
s = 274
|
||||
c = 240
|
||||
else:
|
||||
s = 256
|
||||
c = 224
|
||||
resize_p = P.Resize(s, interpolation=Inter.BICUBIC)
|
||||
center_crop_p = P.CenterCrop(c)
|
||||
totensor = P.ToTensor()
|
||||
normalize_p = P.Normalize((0.485, 0.456, 0.406), (0.229, 0.224, 0.225))
|
||||
composeop = P.ComposeOp(
|
||||
[decode_p, resize_p, center_crop_p, totensor, normalize_p])
|
||||
if do_train:
|
||||
trans = [resize_crop_op, horizontal_flip_op, color_op,
|
||||
rescale_op, normalize_op, change_swap_op]
|
||||
else:
|
||||
trans = composeop()
|
||||
type_cast_op = C2.TypeCast(mstype.int32)
|
||||
|
||||
ds = ds.map(input_columns="image", operations=trans,
|
||||
num_parallel_workers=8)
|
||||
ds = ds.map(input_columns="label_list",
|
||||
operations=type_cast_op, num_parallel_workers=8)
|
||||
|
||||
# apply shuffle operations
|
||||
ds = ds.shuffle(buffer_size=buffer_size)
|
||||
|
||||
# apply batch operations
|
||||
ds = ds.batch(batch_size, drop_remainder=True)
|
||||
|
||||
# apply dataset repeat operation
|
||||
ds = ds.repeat(repeat_num)
|
||||
|
||||
return ds
|
||||
|
|
@ -0,0 +1,491 @@
|
|||
# Copyright 2020 Huawei Technologies Co., Ltd
|
||||
#
|
||||
# Licensed under the Apache License, Version 2.0 (the "License");
|
||||
# you may not use this file except in compliance with the License.
|
||||
# You may obtain a copy of the License at
|
||||
#
|
||||
# http://www.apache.org/licenses/LICENSE-2.0
|
||||
#
|
||||
# Unless required by applicable law or agreed to in writing, software
|
||||
# distributed under the License is distributed on an "AS IS" BASIS,
|
||||
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
# See the License for the specific language governing permissions and
|
||||
# limitations under the License.
|
||||
# ============================================================================
|
||||
"""GhostNet model define"""
|
||||
from functools import partial
|
||||
import math
|
||||
import numpy as np
|
||||
import mindspore.nn as nn
|
||||
from mindspore.ops import operations as P
|
||||
from mindspore import Tensor
|
||||
from .quant import QuanConv
|
||||
|
||||
|
||||
__all__ = ['ghostnet']
|
||||
|
||||
|
||||
def _make_divisible(x, divisor=4):
|
||||
return int(np.ceil(x * 1. / divisor) * divisor)
|
||||
|
||||
|
||||
class MyHSigmoid(nn.Cell):
|
||||
"""
|
||||
Hard Sigmoid definition.
|
||||
|
||||
Args:
|
||||
|
||||
Returns:
|
||||
Tensor, output tensor.
|
||||
|
||||
Examples:
|
||||
>>> MyHSigmoid()
|
||||
"""
|
||||
|
||||
def __init__(self):
|
||||
super(MyHSigmoid, self).__init__()
|
||||
self.relu6 = nn.ReLU6()
|
||||
|
||||
def construct(self, x):
|
||||
return self.relu6(x + 3.) * 0.16666667
|
||||
|
||||
|
||||
class Activation(nn.Cell):
|
||||
"""
|
||||
Activation definition.
|
||||
|
||||
Args:
|
||||
act_func(string): activation name.
|
||||
|
||||
Returns:
|
||||
Tensor, output tensor.
|
||||
"""
|
||||
|
||||
def __init__(self, act_func):
|
||||
super(Activation, self).__init__()
|
||||
if act_func == 'relu':
|
||||
self.act = nn.ReLU()
|
||||
elif act_func == 'relu6':
|
||||
self.act = nn.ReLU6()
|
||||
elif act_func in ('hsigmoid', 'hard_sigmoid'):
|
||||
self.act = MyHSigmoid()
|
||||
elif act_func in ('hswish', 'hard_swish'):
|
||||
self.act = nn.HSwish()
|
||||
else:
|
||||
raise NotImplementedError
|
||||
|
||||
def construct(self, x):
|
||||
return self.act(x)
|
||||
|
||||
|
||||
class GlobalAvgPooling(nn.Cell):
|
||||
"""
|
||||
Global avg pooling definition.
|
||||
|
||||
Args:
|
||||
|
||||
Returns:
|
||||
Tensor, output tensor.
|
||||
|
||||
Examples:
|
||||
>>> GlobalAvgPooling()
|
||||
"""
|
||||
|
||||
def __init__(self, keep_dims=False):
|
||||
super(GlobalAvgPooling, self).__init__()
|
||||
self.mean = P.ReduceMean(keep_dims=keep_dims)
|
||||
|
||||
def construct(self, x):
|
||||
x = self.mean(x, (2, 3))
|
||||
return x
|
||||
|
||||
|
||||
class SE(nn.Cell):
|
||||
"""
|
||||
SE warpper definition.
|
||||
|
||||
Args:
|
||||
num_out (int): Output channel.
|
||||
ratio (int): middle output ratio.
|
||||
|
||||
Returns:
|
||||
Tensor, output tensor.
|
||||
|
||||
Examples:
|
||||
>>> SE(4)
|
||||
"""
|
||||
|
||||
def __init__(self, num_out, ratio=4):
|
||||
super(SE, self).__init__()
|
||||
num_mid = _make_divisible(num_out // ratio)
|
||||
self.pool = GlobalAvgPooling(keep_dims=True)
|
||||
self.conv_reduce = QuanConv(in_channels=num_out, out_channels=num_mid,
|
||||
kernel_size=1, has_bias=True, pad_mode='pad')
|
||||
self.act1 = Activation('relu')
|
||||
self.conv_expand = QuanConv(in_channels=num_mid, out_channels=num_out,
|
||||
kernel_size=1, has_bias=True, pad_mode='pad')
|
||||
self.act2 = Activation('hsigmoid')
|
||||
self.mul = P.Mul()
|
||||
|
||||
def construct(self, x):
|
||||
out = self.pool(x)
|
||||
out = self.conv_reduce(out)
|
||||
out = self.act1(out)
|
||||
out = self.conv_expand(out)
|
||||
out = self.act2(out)
|
||||
out = self.mul(x, out)
|
||||
return out
|
||||
|
||||
|
||||
class ConvUnit(nn.Cell):
|
||||
"""
|
||||
ConvUnit warpper definition.
|
||||
|
||||
Args:
|
||||
num_in (int): Input channel.
|
||||
num_out (int): Output channel.
|
||||
kernel_size (int): Input kernel size.
|
||||
stride (int): Stride size.
|
||||
padding (int): Padding number.
|
||||
num_groups (int): Output num group.
|
||||
use_act (bool): Used activation or not.
|
||||
act_type (string): Activation type.
|
||||
|
||||
Returns:
|
||||
Tensor, output tensor.
|
||||
|
||||
Examples:
|
||||
>>> ConvUnit(3, 3)
|
||||
"""
|
||||
|
||||
def __init__(self, num_in, num_out, kernel_size=1, stride=1, padding=0, num_groups=1,
|
||||
use_act=True, act_type='relu'):
|
||||
super(ConvUnit, self).__init__()
|
||||
self.conv = QuanConv(in_channels=num_in,
|
||||
out_channels=num_out,
|
||||
kernel_size=kernel_size,
|
||||
stride=stride,
|
||||
padding=padding,
|
||||
group=num_groups,
|
||||
has_bias=False,
|
||||
pad_mode='pad')
|
||||
self.bn = nn.BatchNorm2d(num_out)
|
||||
self.use_act = use_act
|
||||
self.act = Activation(act_type) if use_act else None
|
||||
|
||||
def construct(self, x):
|
||||
out = self.conv(x)
|
||||
out = self.bn(out)
|
||||
if self.use_act:
|
||||
out = self.act(out)
|
||||
return out
|
||||
|
||||
|
||||
class GhostModule(nn.Cell):
|
||||
"""
|
||||
GhostModule warpper definition.
|
||||
|
||||
Args:
|
||||
num_in (int): Input channel.
|
||||
num_out (int): Output channel.
|
||||
kernel_size (int): Input kernel size.
|
||||
stride (int): Stride size.
|
||||
padding (int): Padding number.
|
||||
ratio (int): Reduction ratio.
|
||||
dw_size (int): kernel size of cheap operation.
|
||||
use_act (bool): Used activation or not.
|
||||
act_type (string): Activation type.
|
||||
|
||||
Returns:
|
||||
Tensor, output tensor.
|
||||
|
||||
Examples:
|
||||
>>> GhostModule(3, 3)
|
||||
"""
|
||||
|
||||
def __init__(self, num_in, num_out, kernel_size=1, stride=1, padding=0, ratio=2, dw_size=3,
|
||||
use_act=True, act_type='relu'):
|
||||
super(GhostModule, self).__init__()
|
||||
init_channels = math.ceil(num_out / ratio)
|
||||
new_channels = init_channels * (ratio - 1)
|
||||
|
||||
self.primary_conv = ConvUnit(num_in, init_channels, kernel_size=kernel_size, stride=stride, padding=padding,
|
||||
num_groups=1, use_act=use_act, act_type='relu')
|
||||
self.cheap_operation = ConvUnit(init_channels, new_channels, kernel_size=dw_size, stride=1, padding=dw_size//2,
|
||||
num_groups=init_channels, use_act=use_act, act_type='relu')
|
||||
self.concat = P.Concat(axis=1)
|
||||
|
||||
def construct(self, x):
|
||||
x1 = self.primary_conv(x)
|
||||
x2 = self.cheap_operation(x1)
|
||||
return self.concat((x1, x2))
|
||||
|
||||
|
||||
class GhostBottleneck(nn.Cell):
|
||||
"""
|
||||
GhostBottleneck warpper definition.
|
||||
|
||||
Args:
|
||||
num_in (int): Input channel.
|
||||
num_mid (int): Middle channel.
|
||||
num_out (int): Output channel.
|
||||
kernel_size (int): Input kernel size.
|
||||
stride (int): Stride size.
|
||||
act_type (str): Activation type.
|
||||
use_se (bool): Use SE warpper or not.
|
||||
|
||||
Returns:
|
||||
Tensor, output tensor.
|
||||
|
||||
Examples:
|
||||
>>> GhostBottleneck(16, 3, 1, 1)
|
||||
"""
|
||||
|
||||
def __init__(self, num_in, num_mid, num_out, kernel_size, stride=1, act_type='relu', use_se=False):
|
||||
super(GhostBottleneck, self).__init__()
|
||||
self.ghost1 = GhostModule(num_in, num_mid, kernel_size=1,
|
||||
stride=1, padding=0, act_type=act_type)
|
||||
|
||||
self.use_dw = stride > 1
|
||||
self.dw = None
|
||||
if self.use_dw:
|
||||
self.dw = ConvUnit(num_mid, num_mid, kernel_size=kernel_size, stride=stride,
|
||||
padding=self._get_pad(kernel_size), act_type=act_type, num_groups=num_mid, use_act=False)
|
||||
|
||||
self.use_se = use_se
|
||||
if use_se:
|
||||
self.se = SE(num_mid)
|
||||
|
||||
self.ghost2 = GhostModule(num_mid, num_out, kernel_size=1, stride=1,
|
||||
padding=0, act_type=act_type, use_act=False)
|
||||
|
||||
self.down_sample = False
|
||||
if num_in != num_out or stride != 1:
|
||||
self.down_sample = True
|
||||
self.shortcut = None
|
||||
if self.down_sample:
|
||||
self.shortcut = nn.SequentialCell([
|
||||
ConvUnit(num_in, num_in, kernel_size=kernel_size, stride=stride,
|
||||
padding=self._get_pad(kernel_size), num_groups=num_in, use_act=False),
|
||||
ConvUnit(num_in, num_out, kernel_size=1, stride=1,
|
||||
padding=0, num_groups=1, use_act=False),
|
||||
])
|
||||
self.add = P.TensorAdd()
|
||||
|
||||
def construct(self, x):
|
||||
r"""construct of GhostNet BottleNeck"""
|
||||
shortcut = x
|
||||
out = self.ghost1(x)
|
||||
if self.use_dw:
|
||||
out = self.dw(out)
|
||||
if self.use_se:
|
||||
out = self.se(out)
|
||||
out = self.ghost2(out)
|
||||
if self.down_sample:
|
||||
shortcut = self.shortcut(shortcut)
|
||||
out = self.add(shortcut, out)
|
||||
return out
|
||||
|
||||
def _get_pad(self, kernel_size):
|
||||
"""set the padding number"""
|
||||
pad = 0
|
||||
if kernel_size == 1:
|
||||
pad = 0
|
||||
elif kernel_size == 3:
|
||||
pad = 1
|
||||
elif kernel_size == 5:
|
||||
pad = 2
|
||||
elif kernel_size == 7:
|
||||
pad = 3
|
||||
else:
|
||||
raise NotImplementedError
|
||||
return pad
|
||||
|
||||
|
||||
class GhostNet(nn.Cell):
|
||||
"""
|
||||
GhostNet architecture.
|
||||
|
||||
Args:
|
||||
model_cfgs (Cell): number of classes.
|
||||
num_classes (int): Output number classes.
|
||||
multiplier (int): Channels multiplier for round to 8/16 and others. Default is 1.
|
||||
final_drop (float): Dropout number.
|
||||
round_nearest (list): Channel round to . Default is 8.
|
||||
Returns:
|
||||
Tensor, output tensor.
|
||||
|
||||
Examples:
|
||||
>>> GhostNet(num_classes=1000)
|
||||
"""
|
||||
|
||||
def __init__(self, model_cfgs, num_classes=1000, multiplier=1., final_drop=0., round_nearest=8):
|
||||
super(GhostNet, self).__init__()
|
||||
self.cfgs = model_cfgs['cfg']
|
||||
self.inplanes = 16
|
||||
first_conv_in_channel = 3
|
||||
first_conv_out_channel = _make_divisible(multiplier * self.inplanes)
|
||||
|
||||
self.conv_stem = QuanConv(in_channels=first_conv_in_channel,
|
||||
out_channels=first_conv_out_channel,
|
||||
kernel_size=3, padding=1, stride=2,
|
||||
has_bias=False, pad_mode='pad')
|
||||
self.bn1 = nn.BatchNorm2d(first_conv_out_channel)
|
||||
self.act1 = Activation('relu')
|
||||
|
||||
self.blocks = []
|
||||
for layer_cfg in self.cfgs:
|
||||
self.blocks.append(self._make_layer(kernel_size=layer_cfg[0],
|
||||
exp_ch=_make_divisible(
|
||||
multiplier * layer_cfg[1]),
|
||||
out_channel=_make_divisible(
|
||||
multiplier * layer_cfg[2]),
|
||||
use_se=layer_cfg[3],
|
||||
act_func=layer_cfg[4],
|
||||
stride=layer_cfg[5]))
|
||||
output_channel = _make_divisible(
|
||||
multiplier * model_cfgs["cls_ch_squeeze"])
|
||||
self.blocks.append(ConvUnit(_make_divisible(multiplier * self.cfgs[-1][2]), output_channel,
|
||||
kernel_size=1, stride=1, padding=0, num_groups=1, use_act=True))
|
||||
self.blocks = nn.SequentialCell(self.blocks)
|
||||
|
||||
self.global_pool = GlobalAvgPooling(keep_dims=True)
|
||||
self.conv_head = QuanConv(in_channels=output_channel,
|
||||
out_channels=model_cfgs['cls_ch_expand'],
|
||||
kernel_size=1, padding=0, stride=1,
|
||||
has_bias=True, pad_mode='pad')
|
||||
self.act2 = Activation('relu')
|
||||
self.squeeze = P.Flatten()
|
||||
self.final_drop = final_drop
|
||||
if self.final_drop > 0:
|
||||
self.dropout = nn.Dropout(self.final_drop)
|
||||
|
||||
self.classifier = nn.Dense(
|
||||
model_cfgs['cls_ch_expand'], num_classes, has_bias=True)
|
||||
|
||||
self._initialize_weights()
|
||||
|
||||
def construct(self, x):
|
||||
r"""construct of GhostNet"""
|
||||
x = self.conv_stem(x)
|
||||
x = self.bn1(x)
|
||||
x = self.act1(x)
|
||||
x = self.blocks(x)
|
||||
x = self.global_pool(x)
|
||||
x = self.conv_head(x)
|
||||
x = self.act2(x)
|
||||
x = self.squeeze(x)
|
||||
if self.final_drop > 0:
|
||||
x = self.dropout(x)
|
||||
x = self.classifier(x)
|
||||
return x
|
||||
|
||||
def _make_layer(self, kernel_size, exp_ch, out_channel, use_se, act_func, stride=1):
|
||||
mid_planes = exp_ch
|
||||
out_planes = out_channel
|
||||
layer = GhostBottleneck(self.inplanes, mid_planes, out_planes,
|
||||
kernel_size, stride=stride, act_type=act_func, use_se=use_se)
|
||||
self.inplanes = out_planes
|
||||
return layer
|
||||
|
||||
def _initialize_weights(self):
|
||||
"""
|
||||
Initialize weights.
|
||||
|
||||
Args:
|
||||
|
||||
Returns:
|
||||
None.
|
||||
|
||||
Examples:
|
||||
>>> _initialize_weights()
|
||||
"""
|
||||
self.init_parameters_data()
|
||||
for _, m in self.cells_and_names():
|
||||
if isinstance(m, (nn.Conv2d)):
|
||||
n = m.kernel_size[0] * m.kernel_size[1] * m.out_channels
|
||||
m.weight.set_parameter_data(Tensor(np.random.normal(0, np.sqrt(2. / n),
|
||||
m.weight.data.shape).astype("float32")))
|
||||
if m.bias is not None:
|
||||
m.bias.set_parameter_data(
|
||||
Tensor(np.zeros(m.bias.data.shape, dtype="float32")))
|
||||
elif isinstance(m, nn.BatchNorm2d):
|
||||
m.gamma.set_parameter_data(
|
||||
Tensor(np.ones(m.gamma.data.shape, dtype="float32")))
|
||||
m.beta.set_parameter_data(
|
||||
Tensor(np.zeros(m.beta.data.shape, dtype="float32")))
|
||||
elif isinstance(m, nn.Dense):
|
||||
m.weight.set_parameter_data(Tensor(np.random.normal(
|
||||
0, 0.01, m.weight.data.shape).astype("float32")))
|
||||
if m.bias is not None:
|
||||
m.bias.set_parameter_data(
|
||||
Tensor(np.zeros(m.bias.data.shape, dtype="float32")))
|
||||
|
||||
|
||||
def ghostnet(model_name, **kwargs):
|
||||
"""
|
||||
Constructs a GhostNet model
|
||||
"""
|
||||
model_cfgs = {
|
||||
"1x": {
|
||||
"cfg": [
|
||||
# k, exp, c, se, nl, s,
|
||||
# stage1
|
||||
[3, 16, 16, False, 'relu', 1],
|
||||
# stage2
|
||||
[3, 48, 24, False, 'relu', 2],
|
||||
[3, 72, 24, False, 'relu', 1],
|
||||
# stage3
|
||||
[5, 72, 40, True, 'relu', 2],
|
||||
[5, 120, 40, True, 'relu', 1],
|
||||
# stage4
|
||||
[3, 240, 80, False, 'relu', 2],
|
||||
[3, 200, 80, False, 'relu', 1],
|
||||
[3, 184, 80, False, 'relu', 1],
|
||||
[3, 184, 80, False, 'relu', 1],
|
||||
[3, 480, 112, True, 'relu', 1],
|
||||
[3, 672, 112, True, 'relu', 1],
|
||||
# stage5
|
||||
[5, 672, 160, True, 'relu', 2],
|
||||
[5, 960, 160, False, 'relu', 1],
|
||||
[5, 960, 160, True, 'relu', 1],
|
||||
[5, 960, 160, False, 'relu', 1],
|
||||
[5, 960, 160, True, 'relu', 1]],
|
||||
"cls_ch_squeeze": 960,
|
||||
"cls_ch_expand": 1280,
|
||||
},
|
||||
|
||||
"nose_1x": {
|
||||
"cfg": [
|
||||
# k, exp, c, se, nl, s,
|
||||
# stage1
|
||||
[3, 16, 16, False, 'relu', 1],
|
||||
# stage2
|
||||
[3, 48, 24, False, 'relu', 2],
|
||||
[3, 72, 24, False, 'relu', 1],
|
||||
# stage3
|
||||
[5, 72, 40, False, 'relu', 2],
|
||||
[5, 120, 40, False, 'relu', 1],
|
||||
# stage4
|
||||
[3, 240, 80, False, 'relu', 2],
|
||||
[3, 200, 80, False, 'relu', 1],
|
||||
[3, 184, 80, False, 'relu', 1],
|
||||
[3, 184, 80, False, 'relu', 1],
|
||||
[3, 480, 112, False, 'relu', 1],
|
||||
[3, 672, 112, False, 'relu', 1],
|
||||
# stage5
|
||||
[5, 672, 160, False, 'relu', 2],
|
||||
[5, 960, 160, False, 'relu', 1],
|
||||
[5, 960, 160, False, 'relu', 1],
|
||||
[5, 960, 160, False, 'relu', 1],
|
||||
[5, 960, 160, False, 'relu', 1]],
|
||||
"cls_ch_squeeze": 960,
|
||||
"cls_ch_expand": 1280,
|
||||
}
|
||||
}
|
||||
|
||||
return GhostNet(model_cfgs[model_name], **kwargs)
|
||||
|
||||
|
||||
ghostnet_1x = partial(ghostnet, model_name="1x", final_drop=0.8)
|
||||
ghostnet_nose_1x = partial(ghostnet, model_name="nose_1x", final_drop=0.8)
|
||||
|
|
@ -0,0 +1,165 @@
|
|||
# Copyright 2020 Huawei Technologies Co., Ltd
|
||||
#
|
||||
# Licensed under the Apache License, Version 2.0 (the "License");
|
||||
# you may not use this file except in compliance with the License.
|
||||
# You may obtain a copy of the License at
|
||||
#
|
||||
# http://www.apache.org/licenses/LICENSE-2.0
|
||||
#
|
||||
# Unless required by applicable law or agreed to in writing, software
|
||||
# distributed under the License is distributed on an "AS IS" BASIS,
|
||||
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
# See the License for the specific language governing permissions and
|
||||
# limitations under the License.
|
||||
# ============================================================================
|
||||
"""launch train script"""
|
||||
import os
|
||||
import sys
|
||||
import json
|
||||
import subprocess
|
||||
import shutil
|
||||
from argparse import ArgumentParser
|
||||
|
||||
|
||||
def parse_args():
|
||||
"""
|
||||
parse args .
|
||||
|
||||
Args:
|
||||
|
||||
Returns:
|
||||
args.
|
||||
|
||||
Examples:
|
||||
>>> parse_args()
|
||||
"""
|
||||
parser = ArgumentParser(description="mindspore distributed training launch "
|
||||
"helper utilty that will spawn up "
|
||||
"multiple distributed processes")
|
||||
parser.add_argument("--nproc_per_node", type=int, default=1,
|
||||
help="The number of processes to launch on each node, "
|
||||
"for D training, this is recommended to be set "
|
||||
"to the number of D in your system so that "
|
||||
"each process can be bound to a single D.")
|
||||
parser.add_argument("--visible_devices", type=str, default="0,1,2,3,4,5,6,7",
|
||||
help="will use the visible devices sequentially")
|
||||
parser.add_argument("--server_id", type=str, default="",
|
||||
help="server ip")
|
||||
parser.add_argument("--training_script", type=str,
|
||||
help="The full path to the single D training "
|
||||
"program/script to be launched in parallel, "
|
||||
"followed by all the arguments for the "
|
||||
"training script")
|
||||
# rest from the training program
|
||||
args, unknown = parser.parse_known_args()
|
||||
args.training_script_args = unknown
|
||||
return args
|
||||
|
||||
|
||||
def main():
|
||||
print("start", __file__)
|
||||
args = parse_args()
|
||||
print(args)
|
||||
visible_devices = args.visible_devices.split(',')
|
||||
assert os.path.isfile(args.training_script)
|
||||
assert len(visible_devices) >= args.nproc_per_node
|
||||
print('visible_devices:{}'.format(visible_devices))
|
||||
if not args.server_id:
|
||||
print('pleaser input server ip!!!')
|
||||
exit(0)
|
||||
print('server_id:{}'.format(args.server_id))
|
||||
|
||||
# construct hccn_table
|
||||
hccn_configs = open('/etc/hccn.conf', 'r').readlines()
|
||||
device_ips = {}
|
||||
for hccn_item in hccn_configs:
|
||||
hccn_item = hccn_item.strip()
|
||||
if hccn_item.startswith('address_'):
|
||||
device_id, device_ip = hccn_item.split('=')
|
||||
device_id = device_id.split('_')[1]
|
||||
device_ips[device_id] = device_ip
|
||||
print('device_id:{}, device_ip:{}'.format(device_id, device_ip))
|
||||
hccn_table = {}
|
||||
hccn_table['board_id'] = '0x0000'
|
||||
hccn_table['chip_info'] = '910'
|
||||
hccn_table['deploy_mode'] = 'lab'
|
||||
hccn_table['group_count'] = '1'
|
||||
hccn_table['group_list'] = []
|
||||
instance_list = []
|
||||
usable_dev = ''
|
||||
for instance_id in range(args.nproc_per_node):
|
||||
instance = {}
|
||||
instance['devices'] = []
|
||||
device_id = visible_devices[instance_id]
|
||||
device_ip = device_ips[device_id]
|
||||
usable_dev += str(device_id)
|
||||
instance['devices'].append({
|
||||
'device_id': device_id,
|
||||
'device_ip': device_ip,
|
||||
})
|
||||
instance['rank_id'] = str(instance_id)
|
||||
instance['server_id'] = args.server_id
|
||||
instance_list.append(instance)
|
||||
hccn_table['group_list'].append({
|
||||
'device_num': str(args.nproc_per_node),
|
||||
'server_num': '1',
|
||||
'group_name': '',
|
||||
'instance_count': str(args.nproc_per_node),
|
||||
'instance_list': instance_list,
|
||||
})
|
||||
hccn_table['para_plane_nic_location'] = 'device'
|
||||
hccn_table['para_plane_nic_name'] = []
|
||||
for instance_id in range(args.nproc_per_node):
|
||||
eth_id = visible_devices[instance_id]
|
||||
hccn_table['para_plane_nic_name'].append('eth{}'.format(eth_id))
|
||||
hccn_table['para_plane_nic_num'] = str(args.nproc_per_node)
|
||||
hccn_table['status'] = 'completed'
|
||||
|
||||
# save hccn_table to file
|
||||
table_path = os.getcwd()
|
||||
if not os.path.exists(table_path):
|
||||
os.mkdir(table_path)
|
||||
table_fn = os.path.join(table_path,
|
||||
'rank_table_{}p_{}_{}.json'.format(args.nproc_per_node, usable_dev, args.server_id))
|
||||
with open(table_fn, 'w') as table_fp:
|
||||
json.dump(hccn_table, table_fp, indent=4)
|
||||
sys.stdout.flush()
|
||||
|
||||
# spawn the processes
|
||||
processes = []
|
||||
cmds = []
|
||||
log_files = []
|
||||
env = os.environ.copy()
|
||||
env['RANK_SIZE'] = str(args.nproc_per_node)
|
||||
cur_path = os.getcwd()
|
||||
for rank_id in range(0, args.nproc_per_node):
|
||||
os.chdir(cur_path)
|
||||
device_id = visible_devices[rank_id]
|
||||
device_dir = os.path.join(cur_path, 'device{}'.format(rank_id))
|
||||
env['RANK_ID'] = str(rank_id)
|
||||
env['DEVICE_ID'] = str(device_id)
|
||||
if args.nproc_per_node > 1:
|
||||
env['RANK_TABLE_FILE'] = table_fn
|
||||
if os.path.exists(device_dir):
|
||||
shutil.rmtree(device_dir)
|
||||
os.mkdir(device_dir)
|
||||
os.chdir(device_dir)
|
||||
cmd = [sys.executable, '-u']
|
||||
cmd.append(args.training_script)
|
||||
cmd.extend(args.training_script_args)
|
||||
log_file = open(
|
||||
'{dir}/log{id}.log'.format(dir=device_dir, id=rank_id), 'w')
|
||||
process = subprocess.Popen(
|
||||
cmd, stdout=log_file, stderr=log_file, env=env)
|
||||
processes.append(process)
|
||||
cmds.append(cmd)
|
||||
log_files.append(log_file)
|
||||
for process, cmd, log_file in zip(processes, cmds, log_files):
|
||||
process.wait()
|
||||
if process.returncode != 0:
|
||||
raise subprocess.CalledProcessError(returncode=process, cmd=cmd)
|
||||
log_file.close()
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
|
|
@ -0,0 +1,55 @@
|
|||
# Copyright 2020 Huawei Technologies Co., Ltd
|
||||
#
|
||||
# Licensed under the Apache License, Version 2.0 (the "License");
|
||||
# you may not use this file except in compliance with the License.
|
||||
# You may obtain a copy of the License at
|
||||
#
|
||||
# http://www.apache.org/licenses/LICENSE-2.0
|
||||
#
|
||||
# Unless required by applicable law or agreed to in writing, software
|
||||
# distributed under the License is distributed on an "AS IS" BASIS,
|
||||
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
# See the License for the specific language governing permissions and
|
||||
# limitations under the License.
|
||||
# ============================================================================
|
||||
"""learning rate generator"""
|
||||
import math
|
||||
import numpy as np
|
||||
|
||||
|
||||
def get_lr(global_step, lr_init, lr_end, lr_max, warmup_epochs, total_epochs, steps_per_epoch):
|
||||
"""
|
||||
generate learning rate array
|
||||
|
||||
Args:
|
||||
global_step(int): total steps of the training
|
||||
lr_init(float): init learning rate
|
||||
lr_end(float): end learning rate
|
||||
lr_max(float): max learning rate
|
||||
warmup_epochs(int): number of warmup epochs
|
||||
total_epochs(int): total epoch of training
|
||||
steps_per_epoch(int): steps of one epoch
|
||||
|
||||
Returns:
|
||||
np.array, learning rate array
|
||||
"""
|
||||
lr_each_step = []
|
||||
total_steps = steps_per_epoch * total_epochs
|
||||
warmup_steps = steps_per_epoch * warmup_epochs
|
||||
for i in range(total_steps):
|
||||
if i < warmup_steps:
|
||||
lr = lr_init + (lr_max - lr_init) * i / warmup_steps
|
||||
else:
|
||||
lr = lr_end + \
|
||||
(lr_max - lr_end) * \
|
||||
(1. + math.cos(math.pi * (i - warmup_steps) /
|
||||
(total_steps - warmup_steps))) / 2.
|
||||
if lr < 0.0:
|
||||
lr = 0.0
|
||||
lr_each_step.append(lr)
|
||||
|
||||
current_step = global_step
|
||||
lr_each_step = np.array(lr_each_step).astype(np.float32)
|
||||
learning_rate = lr_each_step[current_step:]
|
||||
|
||||
return learning_rate
|
||||
|
|
@ -0,0 +1,61 @@
|
|||
# Copyright 2020 Huawei Technologies Co., Ltd
|
||||
#
|
||||
# Licensed under the Apache License, Version 2.0 (the "License");
|
||||
# you may not use this file except in compliance with the License.
|
||||
# You may obtain a copy of the License at
|
||||
#
|
||||
# http://www.apache.org/licenses/LICENSE-2.0
|
||||
#
|
||||
# Unless required by applicable law or agreed to in writing, software
|
||||
# distributed under the License is distributed on an "AS IS" BASIS,
|
||||
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
# See the License for the specific language governing permissions and
|
||||
# limitations under the License.
|
||||
# ============================================================================
|
||||
"""Quantization define"""
|
||||
import mindspore as ms
|
||||
import mindspore.nn as nn
|
||||
from mindspore import Parameter, Tensor
|
||||
from mindspore.ops import operations as P
|
||||
from mindspore.ops import composite as C
|
||||
from mindspore.common.initializer import initializer
|
||||
|
||||
#------weight symmetric, activation asymmetric------#
|
||||
|
||||
|
||||
class QuanConv(nn.Conv2d):
|
||||
r"""Conv for quantization"""
|
||||
def __init__(self, in_channels, out_channels, kernel_size, stride=1, pad_mode='same',
|
||||
padding=0, dilation=1, group=1, has_bias=True):
|
||||
super(QuanConv, self).__init__(in_channels, out_channels,
|
||||
kernel_size, stride, pad_mode, padding, dilation, group, has_bias)
|
||||
self.floor = P.Floor()
|
||||
self.expand_dims = P.ExpandDims()
|
||||
self.x_lower_bound = Tensor(0, ms.float32)
|
||||
self.x_upper_bound = Tensor(2 ** 8 - 1, ms.float32)
|
||||
self.w_lower_bound = Tensor(-2 ** 7 - 1, ms.float32)
|
||||
self.w_upper_bound = Tensor(2 ** 7, ms.float32)
|
||||
self.scale_a = Parameter(initializer('ones', [1]), name='scale_a')
|
||||
self.scale_w = Parameter(initializer(
|
||||
'ones', [out_channels]), name='scale_w')
|
||||
self.zp_a = Parameter(initializer('ones', [1]), name='zp_a')
|
||||
|
||||
def construct(self, in_data):
|
||||
r"""construct of QuantConv"""
|
||||
x = self.floor(in_data / self.scale_a - self.zp_a + 0.5)
|
||||
x = C.clip_by_value(x, self.x_lower_bound, self.x_upper_bound)
|
||||
x = (x + self.zp_a) * self.scale_a
|
||||
|
||||
exp_dim_scale_w = self.scale_w
|
||||
exp_dim_scale_w = self.expand_dims(exp_dim_scale_w, 1)
|
||||
exp_dim_scale_w = self.expand_dims(exp_dim_scale_w, 2)
|
||||
exp_dim_scale_w = self.expand_dims(exp_dim_scale_w, 3)
|
||||
w = self.floor(self.weight / exp_dim_scale_w + 0.5)
|
||||
w = C.clip_by_value(w, self.w_lower_bound, self.w_upper_bound)
|
||||
w = w * exp_dim_scale_w
|
||||
|
||||
# forward
|
||||
output = self.conv2d(x, w)
|
||||
if self.has_bias:
|
||||
output = self.bias_add(output, self.bias)
|
||||
return output
|
||||
|
|
@ -0,0 +1,122 @@
|
|||
# Contents
|
||||
|
||||
- [Adversarial Pruning Description](#adversarial-pruning-description)
|
||||
- [Dataset](#dataset)
|
||||
- [Environment Requirements](#environment-requirements)
|
||||
- [Script Description](#script-description)
|
||||
- [Script and Sample Code](#script-and-sample-code)
|
||||
- [Training Process](#training-process)
|
||||
- [Evaluation Process](#evaluation-process)
|
||||
- [Evaluation](#evaluation)
|
||||
- [Model Description](#model-description)
|
||||
- [Performance](#performance)
|
||||
- [Training Performance](#evaluation-performance)
|
||||
- [Inference Performance](#evaluation-performance)
|
||||
- [Description of Random Situation](#description-of-random-situation)
|
||||
- [ModelZoo Homepage](#modelzoo-homepage)
|
||||
|
||||
# [Adversarial Pruning Description](#contents)
|
||||
|
||||
The Adversarial Pruning method is a reliable neural network pruning algorithm by setting up a scientific control. We prefer to have a more rigorous research design by including a scientific control group as an essential part to minimize the effect of all factors except the association between the filter and expected network output. Acting as a control group, knockoff feature is generated to mimic the feature map produced by the network filter, but they are conditionally independent of the example label given the real feature map. Besides the real feature map on an intermediate layer, the corresponding knockoff feature is brought in as another auxiliary input signal for the subsequent layers.
|
||||
|
||||
[Paper](https://openaccess.thecvf.com/content_CVPR_2020/papers/Han_GhostNet_More_Features_From_Cheap_Operations_CVPR_2020_paper.pdf): Yehui Tang, Yunhe Wang, Yixing Xu, Dacheng Tao, Chunjing Xu, Chao Xu, Chang Xu. Scientific Control for Reliable Neural Network Pruning. Submitted to NeurIPS 2020.
|
||||
|
||||
# [Dataset](#contents)
|
||||
|
||||
Dataset used: [Oxford-IIIT Pet](https://www.robots.ox.ac.uk/~vgg/data/pets/)
|
||||
|
||||
- Dataset size: 7049 colorful images in 1000 classes
|
||||
- Train: 3680 images
|
||||
- Test: 3369 images
|
||||
- Data format: RGB images.
|
||||
- Note: Data will be processed in src/dataset.py
|
||||
|
||||
# [Environment Requirements](#contents)
|
||||
|
||||
- Hardware（Ascend/GPU）
|
||||
- Prepare hardware environment with Ascend or GPU processor. If you want to try Ascend, please send the [application form](https://obs-9be7.obs.cn-east-2.myhuaweicloud.com/file/other/Ascend%20Model%20Zoo%E4%BD%93%E9%AA%8C%E8%B5%84%E6%BA%90%E7%94%B3%E8%AF%B7%E8%A1%A8.docx) to ascend@huawei.com. Once approved, you can get the resources.
|
||||
- Framework
|
||||
- [MindSpore](http://10.90.67.50/mindspore/archive/20200506/OpenSource/me_vm_x86/)
|
||||
- For more information, please check the resources below：
|
||||
- [MindSpore tutorials](https://www.mindspore.cn/tutorial/zh-CN/master/index.html)
|
||||
- [MindSpore API](https://www.mindspore.cn/api/zh-CN/master/index.html)
|
||||
|
||||
# [Script description](#contents)
|
||||
|
||||
## [Script and sample code](#contents)
|
||||
|
||||
```python
|
||||
├── Adversarial Pruning
|
||||
├── Readme.md # descriptions about adversarial-pruning # shell script for evaluation with CPU, GPU or Ascend
|
||||
├── src
|
||||
│ ├──config.py # parameter configuration
|
||||
│ ├──dataset.py # creating dataset
|
||||
│ ├──resnet_imgnet.py # Pruned ResNet architecture
|
||||
├── eval.py # evaluation script
|
||||
├── index.txt # channel index of each layer after pruning
|
||||
├── mindspore_hub_conf.py # export model for hub
|
||||
```
|
||||
|
||||
## [Training process](#contents)
|
||||
To Be Done
|
||||
|
||||
## [Eval process](#contents)
|
||||
|
||||
### Usage
|
||||
|
||||
After installing MindSpore via the official website, you can start evaluation as follows:
|
||||
|
||||
|
||||
### Launch
|
||||
|
||||
```
|
||||
# infer example
|
||||
|
||||
Ascend: python eval.py --dataset_path ~/Pets/test.mindrecord --platform Ascend --checkpoint_path [CHECKPOINT_PATH]
|
||||
GPU: python eval.py --dataset_path ~/Pets/test.mindrecord --platform GPU --checkpoint_path [CHECKPOINT_PATH]
|
||||
```
|
||||
|
||||
> checkpoint can be produced in training process.
|
||||
|
||||
### Result
|
||||
|
||||
```
|
||||
result: {'acc': 0.8023984736985554} ckpt= ./resnet50-imgnet-0.65x-80.24.ckpt
|
||||
|
||||
```
|
||||
|
||||
# [Model Description](#contents)
|
||||
|
||||
## [Performance](#contents)
|
||||
|
||||
#### Evaluation Performance
|
||||
|
||||
###### ResNet50-0.65x on ImageNet2012
|
||||
| Parameters | |
|
||||
| -------------------------- | -------------------------------------- |
|
||||
| Model Version | ResNet50-0.65x |
|
||||
| uploaded Date | 09/10/2020 (month/day/year) ； |
|
||||
| MindSpore Version | 0.6.0-alpha |
|
||||
| Dataset | ImageNet2012 |
|
||||
| Parameters (M) | 14.6 |
|
||||
| FLOPs (G) | 2.1 |
|
||||
| Accuracy (Top1) | 75.80 |
|
||||
|
||||
###### ResNet50-0.65x on Oxford-IIIT Pet
|
||||
| Parameters | |
|
||||
| -------------------------- | -------------------------------------- |
|
||||
| Model Version | ResNet50-0.65x |
|
||||
| uploaded Date | 09/10/2020 (month/day/year) ； |
|
||||
| MindSpore Version | 0.6.0-alpha |
|
||||
| Dataset | Oxford-IIIT Pet |
|
||||
| Parameters (M) | 14.6 |
|
||||
| FLOPs (M) | 2.1 |
|
||||
| Accuracy (Top1) | 80.24 |
|
||||
|
||||
# [Description of Random Situation](#contents)
|
||||
|
||||
In dataset.py, we set the seed inside “create_dataset" function. We also use random seed in train.py.
|
||||
|
||||
# [ModelZoo Homepage](#contents)
|
||||
|
||||
Please check the official [homepage](https://gitee.com/mindspore/mindspore/tree/master/model_zoo).
|
||||
|
|
@ -0,0 +1,88 @@
|
|||
# Copyright 2020 Huawei Technologies Co., Ltd
|
||||
#
|
||||
# Licensed under the Apache License, Version 2.0 (the "License");
|
||||
# you may not use this file except in compliance with the License.
|
||||
# You may obtain a copy of the License at
|
||||
#
|
||||
# http://www.apache.org/licenses/LICENSE-2.0
|
||||
#
|
||||
# Unless required by applicable law or agreed to in writing, software
|
||||
# distributed under the License is distributed on an "AS IS" BASIS,
|
||||
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
# See the License for the specific language governing permissions and
|
||||
# limitations under the License.
|
||||
# ============================================================================
|
||||
"""
|
||||
eval.
|
||||
"""
|
||||
import os
|
||||
import argparse
|
||||
import numpy as np
|
||||
|
||||
from mindspore import context, Tensor
|
||||
from mindspore import nn
|
||||
from mindspore.train.model import Model
|
||||
from mindspore.train.serialization import load_checkpoint, load_param_into_net
|
||||
from mindspore.common import dtype as mstype
|
||||
|
||||
from src.pet_dataset import create_dataset
|
||||
from src.config import config_ascend, config_gpu
|
||||
from src.resnet_imgnet import resnet50
|
||||
|
||||
|
||||
parser = argparse.ArgumentParser(description='Image classification')
|
||||
parser.add_argument('--checkpoint_path', type=str,
|
||||
default='resnet50-imgnet-0.65x-80.24.ckpt', help='Checkpoint file path')
|
||||
parser.add_argument('--dataset_path', type=str,
|
||||
default='/home/hankai/xiaoan/data/test.mindrecord', help='Dataset path')
|
||||
parser.add_argument('--platform', type=str, default='GPU', help='run platform')
|
||||
args_opt = parser.parse_args()
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
config_platform = None
|
||||
if args_opt.platform == "Ascend":
|
||||
config_platform = config_ascend
|
||||
device_id = int(os.getenv('DEVICE_ID'))
|
||||
context.set_context(mode=context.GRAPH_MODE, device_target="Ascend",
|
||||
device_id=device_id, save_graphs=False)
|
||||
elif args_opt.platform == "GPU":
|
||||
config_platform = config_gpu
|
||||
context.set_context(mode=context.GRAPH_MODE,
|
||||
device_target="GPU", save_graphs=False)
|
||||
else:
|
||||
raise ValueError("Unsupport platform.")
|
||||
|
||||
loss = nn.SoftmaxCrossEntropyWithLogits(
|
||||
is_grad=False, sparse=True, reduction='mean')
|
||||
|
||||
if args_opt.platform == "Ascend":
|
||||
net.to_float(mstype.float16)
|
||||
for _, cell in net.cells_and_names():
|
||||
if isinstance(cell, nn.Dense):
|
||||
cell.to_float(mstype.float32)
|
||||
|
||||
dataset = create_dataset(dataset_path=args_opt.dataset_path,
|
||||
do_train=False,
|
||||
config=config_platform,
|
||||
platform=args_opt.platform,
|
||||
batch_size=config_platform.batch_size)
|
||||
step_size = dataset.get_dataset_size()
|
||||
|
||||
index = []
|
||||
with open('index.txt', 'r') as f:
|
||||
for line in f:
|
||||
ind = Tensor((np.array(line.strip('\n').split(' ')[:-1])).astype(np.int32).reshape(-1, 1))
|
||||
index.append(ind)
|
||||
|
||||
net = resnet50(
|
||||
rate=0.65, class_num=config_platform.num_classes, index=index)
|
||||
if args_opt.checkpoint_path:
|
||||
param_dict = load_checkpoint(args_opt.checkpoint_path)
|
||||
load_param_into_net(net, param_dict)
|
||||
|
||||
net.set_train(False)
|
||||
|
||||
model = Model(net, loss_fn=loss, metrics={'acc'})
|
||||
res = model.eval(dataset)
|
||||
print("result:", res, "ckpt=", args_opt.checkpoint_path)
|
||||
File diff suppressed because one or more lines are too long
|
|
@ -0,0 +1,22 @@
|
|||
# Copyright 2020 Huawei Technologies Co., Ltd
|
||||
#
|
||||
# Licensed under the Apache License, Version 2.0 (the "License");
|
||||
# you may not use this file except in compliance with the License.
|
||||
# You may obtain a copy of the License at
|
||||
#
|
||||
# http://www.apache.org/licenses/LICENSE-2.0
|
||||
#
|
||||
# Unless required by applicable law or agreed to in writing, software
|
||||
# distributed under the License is distributed on an "AS IS" BASIS,
|
||||
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
# See the License for the specific language governing permissions and
|
||||
# limitations under the License.
|
||||
# ============================================================================
|
||||
"""hub config."""
|
||||
from src.resnet_imgnet import resnet50
|
||||
|
||||
|
||||
def create_network(name, *args, **kwargs):
|
||||
if name == 'resnet-0.65x':
|
||||
return resnet50(*args, **kwargs)
|
||||
raise NotImplementedError(f"{name} is not implemented in the repo")
|
||||
|
|
@ -0,0 +1,54 @@
|
|||
# Copyright 2020 Huawei Technologies Co., Ltd
|
||||
#
|
||||
# Licensed under the Apache License, Version 2.0 (the "License");
|
||||
# you may not use this file except in compliance with the License.
|
||||
# You may obtain a copy of the License at
|
||||
#
|
||||
# http://www.apache.org/licenses/LICENSE-2.0
|
||||
#
|
||||
# Unless required by applicable law or agreed to in writing, software
|
||||
# distributed under the License is distributed on an "AS IS" BASIS,
|
||||
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
# See the License for the specific language governing permissions and
|
||||
# limitations under the License.
|
||||
# ============================================================================
|
||||
"""
|
||||
network config setting, will be used in train.py and eval.py
|
||||
"""
|
||||
from easydict import EasyDict as ed
|
||||
|
||||
config_ascend = ed({
|
||||
"num_classes": 438,
|
||||
"image_height": 224,
|
||||
"image_width": 224,
|
||||
"batch_size": 256,
|
||||
"epoch_size": 200,
|
||||
"warmup_epochs": 1,
|
||||
"lr": 0.02,
|
||||
"momentum": 0.9,
|
||||
"weight_decay": 4e-5,
|
||||
"label_smooth": 0.1,
|
||||
"loss_scale": 1024,
|
||||
"save_checkpoint": True,
|
||||
"save_checkpoint_epochs": 5,
|
||||
"keep_checkpoint_max": 200,
|
||||
"save_checkpoint_path": "./checkpoint",
|
||||
})
|
||||
|
||||
config_gpu = ed({
|
||||
"num_classes": 37,
|
||||
"image_height": 224,
|
||||
"image_width": 224,
|
||||
"batch_size": 1,
|
||||
"epoch_size": 200,
|
||||
"warmup_epochs": 0,
|
||||
"lr": 0.8,
|
||||
"momentum": 0.9,
|
||||
"weight_decay": 4e-5,
|
||||
# "label_smooth": 0.1,
|
||||
"loss_scale": 1024,
|
||||
"save_checkpoint": True,
|
||||
"save_checkpoint_epochs": 1,
|
||||
"keep_checkpoint_max": 200,
|
||||
"save_checkpoint_path": "./checkpoint",
|
||||
})
|
||||
|
|
@ -0,0 +1,106 @@
|
|||
# Copyright 2020 Huawei Technologies Co., Ltd
|
||||
#
|
||||
# Licensed under the Apache License, Version 2.0 (the "License");
|
||||
# you may not use this file except in compliance with the License.
|
||||
# You may obtain a copy of the License at
|
||||
#
|
||||
# http://www.apache.org/licenses/LICENSE-2.0
|
||||
#
|
||||
# Unless required by applicable law or agreed to in writing, software
|
||||
# distributed under the License is distributed on an "AS IS" BASIS,
|
||||
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
# See the License for the specific language governing permissions and
|
||||
# limitations under the License.
|
||||
# ============================================================================
|
||||
"""
|
||||
create train or eval dataset.
|
||||
"""
|
||||
import os
|
||||
import mindspore.common.dtype as mstype
|
||||
import mindspore.dataset.engine as de
|
||||
import mindspore.dataset.transforms.vision.c_transforms as C
|
||||
import mindspore.dataset.transforms.vision.py_transforms as P
|
||||
import mindspore.dataset.transforms.c_transforms as C2
|
||||
from mindspore.dataset.transforms.vision import Inter
|
||||
|
||||
|
||||
def create_dataset(dataset_path, do_train, config, platform, repeat_num=1, batch_size=100):
|
||||
"""
|
||||
create a train or eval dataset
|
||||
|
||||
Args:
|
||||
dataset_path(string): the path of dataset.
|
||||
do_train(bool): whether dataset is used for train or eval.
|
||||
repeat_num(int): the repeat times of dataset. Default: 1
|
||||
batch_size(int): the batch size of dataset. Default: 32
|
||||
|
||||
Returns:
|
||||
dataset
|
||||
"""
|
||||
if platform == "Ascend":
|
||||
rank_size = int(os.getenv("RANK_SIZE"))
|
||||
rank_id = int(os.getenv("RANK_ID"))
|
||||
if rank_size == 1:
|
||||
ds = de.MindDataset(
|
||||
dataset_path, num_parallel_workers=8, shuffle=True)
|
||||
else:
|
||||
ds = de.MindDataset(dataset_path, num_parallel_workers=8, shuffle=True,
|
||||
num_shards=rank_size, shard_id=rank_id)
|
||||
elif platform == "GPU":
|
||||
if do_train:
|
||||
from mindspore.communication.management import get_rank, get_group_size
|
||||
ds = de.MindDataset(dataset_path, num_parallel_workers=8, shuffle=True,
|
||||
num_shards=get_group_size(), shard_id=get_rank())
|
||||
else:
|
||||
ds = de.MindDataset(
|
||||
dataset_path, num_parallel_workers=8, shuffle=False)
|
||||
else:
|
||||
raise ValueError("Unsupport platform.")
|
||||
|
||||
resize_height = config.image_height
|
||||
buffer_size = 1000
|
||||
|
||||
# define map operations
|
||||
resize_crop_op = C.RandomCropDecodeResize(
|
||||
resize_height, scale=(0.08, 1.0), ratio=(0.75, 1.333))
|
||||
horizontal_flip_op = C.RandomHorizontalFlip(prob=0.5)
|
||||
|
||||
color_op = C.RandomColorAdjust(
|
||||
brightness=0.4, contrast=0.4, saturation=0.4)
|
||||
rescale_op = C.Rescale(1/255.0, 0)
|
||||
normalize_op = C.Normalize(
|
||||
mean=[0.485, 0.456, 0.406], std=[0.229, 0.224, 0.225])
|
||||
change_swap_op = C.HWC2CHW()
|
||||
|
||||
# define python operations
|
||||
decode_p = P.Decode()
|
||||
resize_p = P.Resize(256, interpolation=Inter.BILINEAR)
|
||||
center_crop_p = P.CenterCrop(224)
|
||||
totensor = P.ToTensor()
|
||||
normalize_p = P.Normalize((0.485, 0.456, 0.406), (0.229, 0.224, 0.225))
|
||||
#change_op_p = P.HWC2CHW()
|
||||
composeop = P.ComposeOp(
|
||||
[decode_p, resize_p, center_crop_p, totensor, normalize_p])
|
||||
if do_train:
|
||||
trans = [resize_crop_op, horizontal_flip_op, color_op,
|
||||
rescale_op, normalize_op, change_swap_op]
|
||||
else:
|
||||
#trans = [decode_op, resize_op, center_crop, rescale_op, normalize_op, change_swap_op]
|
||||
trans = composeop()
|
||||
type_cast_op = C2.TypeCast(mstype.int32)
|
||||
|
||||
ds = ds.map(input_columns="image", operations=trans,
|
||||
num_parallel_workers=8)
|
||||
ds = ds.map(input_columns="label_list",
|
||||
operations=type_cast_op, num_parallel_workers=8)
|
||||
|
||||
# apply shuffle operations
|
||||
ds = ds.shuffle(buffer_size=buffer_size)
|
||||
|
||||
# apply batch operations
|
||||
ds = ds.batch(batch_size, drop_remainder=True)
|
||||
|
||||
# apply dataset repeat operation
|
||||
ds = ds.repeat(repeat_num)
|
||||
|
||||
return ds
|
||||
|
|
@ -0,0 +1,391 @@
|
|||
# Copyright 2020 Huawei Technologies Co., Ltd
|
||||
#
|
||||
# Licensed under the Apache License, Version 2.0 (the "License");
|
||||
# you may not use this file except in compliance with the License.
|
||||
# You may obtain a copy of the License at
|
||||
#
|
||||
# http://www.apache.org/licenses/LICENSE-2.0
|
||||
#
|
||||
# Unless required by applicable law or agreed to in writing, software
|
||||
# distributed under the License is distributed on an "AS IS" BASIS,
|
||||
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
# See the License for the specific language governing permissions and
|
||||
# limitations under the License.
|
||||
# ============================================================================
|
||||
"""ResNet."""
|
||||
import math
|
||||
import numpy as np
|
||||
import mindspore.nn as nn
|
||||
from mindspore.ops import operations as P
|
||||
from mindspore.common.tensor import Tensor
|
||||
|
||||
|
||||
def _weight_variable(shape, factor=0.01):
|
||||
init_value = np.random.randn(*shape).astype(np.float32) * factor
|
||||
return Tensor(init_value)
|
||||
|
||||
|
||||
def _conv3x3(in_channel, out_channel, stride=1):
|
||||
weight_shape = (out_channel, in_channel, 3, 3)
|
||||
weight = _weight_variable(weight_shape)
|
||||
return nn.Conv2d(in_channel, out_channel,
|
||||
kernel_size=3, stride=stride, padding=1, pad_mode='pad', weight_init=weight)
|
||||
|
||||
|
||||
def _conv1x1(in_channel, out_channel, stride=1):
|
||||
weight_shape = (out_channel, in_channel, 1, 1)
|
||||
weight = _weight_variable(weight_shape)
|
||||
return nn.Conv2d(in_channel, out_channel,
|
||||
kernel_size=1, stride=stride, padding=0, pad_mode='same', weight_init=weight)
|
||||
|
||||
|
||||
def _conv7x7(in_channel, out_channel, stride=1):
|
||||
weight_shape = (out_channel, in_channel, 7, 7)
|
||||
weight = _weight_variable(weight_shape)
|
||||
return nn.Conv2d(in_channel, out_channel,
|
||||
kernel_size=7, stride=stride, padding=3, pad_mode='pad', weight_init=weight)
|
||||
|
||||
|
||||
def _bn(channel):
|
||||
return nn.BatchNorm2d(channel)
|
||||
|
||||
|
||||
def _bn_last(channel):
|
||||
return nn.BatchNorm2d(channel)
|
||||
|
||||
|
||||
def _fc(in_channel, out_channel):
|
||||
weight_shape = (out_channel, in_channel)
|
||||
weight = _weight_variable(weight_shape)
|
||||
return nn.Dense(in_channel, out_channel, has_bias=True, weight_init=weight, bias_init=0)
|
||||
|
||||
|
||||
class ResidualBlock(nn.Cell):
|
||||
"""
|
||||
ResNet V1 residual block definition.
|
||||
|
||||
Args:
|
||||
in_channel (int): Input channel.
|
||||
out_channel (int): Output channel.
|
||||
stride (int): Stride size for the first convolutional layer. Default: 1.
|
||||
|
||||
Returns:
|
||||
Tensor, output tensor.
|
||||
|
||||
Examples:
|
||||
>>> ResidualBlock(3, 256, stride=2)
|
||||
"""
|
||||
expansion = 4
|
||||
|
||||
def __init__(self,
|
||||
in_channel,
|
||||
out_channel,
|
||||
rate,
|
||||
stride=1,
|
||||
num=0,
|
||||
index=None):
|
||||
super(ResidualBlock, self).__init__()
|
||||
|
||||
channel = out_channel // self.expansion
|
||||
prune_channel = math.ceil(channel*rate)
|
||||
self.conv1 = _conv1x1(in_channel, prune_channel, stride=1)
|
||||
self.bn1 = _bn(prune_channel)
|
||||
|
||||
self.conv2 = _conv3x3(prune_channel, prune_channel, stride=stride)
|
||||
self.bn2 = _bn(prune_channel)
|
||||
|
||||
self.conv3 = _conv1x1(prune_channel, math.ceil(
|
||||
channel*rate*self.expansion), stride=1)
|
||||
self.bn3 = _bn_last(math.ceil(channel*rate*self.expansion))
|
||||
|
||||
self.relu = nn.ReLU()
|
||||
|
||||
self.down_sample = False
|
||||
|
||||
if stride != 1 or in_channel != out_channel:
|
||||
self.down_sample = True
|
||||
self.down_sample_layer = None
|
||||
|
||||
if self.down_sample:
|
||||
self.down_sample_layer = nn.SequentialCell([_conv1x1(in_channel, out_channel, stride),
|
||||
_bn(out_channel)])
|
||||
self.add = P.TensorAdd()
|
||||
|
||||
self.op = P.ScatterNd()
|
||||
self.transpose = P.Transpose()
|
||||
self.idx = None
|
||||
self.shape = None
|
||||
if out_channel == 256:
|
||||
self.shape = (out_channel, 1, 56, 56)
|
||||
if num == 0:
|
||||
self.idx = index[0]
|
||||
elif num == 1:
|
||||
self.idx = index[1]
|
||||
elif num == 2:
|
||||
self.idx = index[2]
|
||||
elif out_channel == 512:
|
||||
self.shape = (out_channel, 1, 28, 28)
|
||||
if num == 0:
|
||||
self.idx = index[3]
|
||||
elif num == 1:
|
||||
self.idx = index[4]
|
||||
elif num == 2:
|
||||
self.idx = index[5]
|
||||
elif num == 3:
|
||||
self.idx = index[6]
|
||||
elif out_channel == 1024:
|
||||
self.shape = (out_channel, 1, 14, 14)
|
||||
if num == 0:
|
||||
self.idx = index[7]
|
||||
elif num == 1:
|
||||
self.idx = index[8]
|
||||
elif num == 2:
|
||||
self.idx = index[9]
|
||||
elif num == 3:
|
||||
self.idx = index[10]
|
||||
elif num == 4:
|
||||
self.idx = index[11]
|
||||
elif num == 5:
|
||||
self.idx = index[12]
|
||||
elif out_channel == 2048:
|
||||
self.shape = (out_channel, 1, 7, 7)
|
||||
if num == 0:
|
||||
self.idx = index[13]
|
||||
elif num == 1:
|
||||
self.idx = index[14]
|
||||
elif num == 2:
|
||||
self.idx = index[15]
|
||||
|
||||
def construct(self, x):
|
||||
r"""construct of ResidualBlock"""
|
||||
identity = x
|
||||
|
||||
out = self.conv1(x)
|
||||
out = self.bn1(out)
|
||||
out = self.relu(out)
|
||||
|
||||
out = self.conv2(out)
|
||||
out = self.bn2(out)
|
||||
out = self.relu(out)
|
||||
|
||||
out = self.conv3(out)
|
||||
out = self.bn3(out)
|
||||
|
||||
if self.down_sample:
|
||||
identity = self.down_sample_layer(identity)
|
||||
|
||||
output = self.transpose(out, (1, 0, 2, 3))
|
||||
output = self.op(self.idx, output, self.shape)
|
||||
output = self.transpose(output, (1, 0, 2, 3))
|
||||
|
||||
output = self.add(identity, output)
|
||||
output = self.relu(output)
|
||||
|
||||
return output
|
||||
|
||||
|
||||
class ResNet(nn.Cell):
|
||||
"""
|
||||
ResNet architecture.
|
||||
|
||||
Args:
|
||||
block (Cell): Block for network.
|
||||
layer_nums (list): Numbers of block in different layers.
|
||||
in_channels (list): Input channel in each layer.
|
||||
out_channels (list): Output channel in each layer.
|
||||
strides (list): Stride size in each layer.
|
||||
num_classes (int): The number of classes that the training images are belonging to.
|
||||
Returns:
|
||||
Tensor, output tensor.
|
||||
|
||||
Examples:
|
||||
>>> ResNet(ResidualBlock,
|
||||
>>> [3, 4, 6, 3],
|
||||
>>> [64, 256, 512, 1024],
|
||||
>>> [256, 512, 1024, 2048],
|
||||
>>> [1, 2, 2, 2],
|
||||
>>> 10)
|
||||
"""
|
||||
|
||||
def __init__(self,
|
||||
rate,
|
||||
block,
|
||||
layer_nums,
|
||||
in_channels,
|
||||
out_channels,
|
||||
strides,
|
||||
num_classes,
|
||||
index):
|
||||
super(ResNet, self).__init__()
|
||||
|
||||
if not len(layer_nums) == len(in_channels) == len(out_channels) == 4:
|
||||
raise ValueError(
|
||||
"the length of layer_num, in_channels, out_channels list must be 4!")
|
||||
|
||||
self.conv1 = _conv7x7(3, 64, stride=2)
|
||||
self.bn1 = _bn(64)
|
||||
self.relu = P.ReLU()
|
||||
self.pad_op = P.Pad(((0, 0), (0, 0), (1, 1), (1, 1)))
|
||||
self.maxpool = nn.MaxPool2d(kernel_size=3, stride=2, pad_mode="valid")
|
||||
|
||||
self.layer1 = self._make_layer(block,
|
||||
rate,
|
||||
layer_nums[0],
|
||||
in_channel=in_channels[0],
|
||||
out_channel=out_channels[0],
|
||||
stride=strides[0],
|
||||
index=index)
|
||||
self.layer2 = self._make_layer(block,
|
||||
rate,
|
||||
layer_nums[1],
|
||||
in_channel=in_channels[1],
|
||||
out_channel=out_channels[1],
|
||||
stride=strides[1],
|
||||
index=index)
|
||||
self.layer3 = self._make_layer(block,
|
||||
rate,
|
||||
layer_nums[2],
|
||||
in_channel=in_channels[2],
|
||||
out_channel=out_channels[2],
|
||||
stride=strides[2],
|
||||
index=index)
|
||||
self.layer4 = self._make_layer(block,
|
||||
rate,
|
||||
layer_nums[3],
|
||||
in_channel=in_channels[3],
|
||||
out_channel=out_channels[3],
|
||||
stride=strides[3],
|
||||
index=index)
|
||||
|
||||
self.mean = P.ReduceMean(keep_dims=True)
|
||||
self.flatten = nn.Flatten()
|
||||
self.end_point = _fc(out_channels[3], num_classes)
|
||||
|
||||
self._initialize_weights()
|
||||
|
||||
def _make_layer(self, block, rate, layer_num, in_channel, out_channel, stride, index):
|
||||
"""
|
||||
Make stage network of ResNet.
|
||||
|
||||
Args:
|
||||
block (Cell): Resnet block.
|
||||
layer_num (int): Layer number.
|
||||
in_channel (int): Input channel.
|
||||
out_channel (int): Output channel.
|
||||
stride (int): Stride size for the first convolutional layer.
|
||||
|
||||
Returns:
|
||||
SequentialCell, the output layer.
|
||||
|
||||
Examples:
|
||||
>>> _make_layer(ResidualBlock, 3, 128, 256, 2)
|
||||
"""
|
||||
layers = []
|
||||
|
||||
resnet_block = block(in_channel, out_channel, rate,
|
||||
stride=stride, num=0, index=index)
|
||||
layers.append(resnet_block)
|
||||
|
||||
for _ in range(1, layer_num):
|
||||
resnet_block = block(out_channel, out_channel,
|
||||
rate, stride=1, num=_, index=index)
|
||||
layers.append(resnet_block)
|
||||
|
||||
return nn.SequentialCell(layers)
|
||||
|
||||
def construct(self, x):
|
||||
r"""construct of ResNet"""
|
||||
x1 = self.conv1(x)
|
||||
x2 = self.bn1(x1)
|
||||
x3 = self.relu(x2)
|
||||
x4 = self.pad_op(x3)
|
||||
c1 = self.maxpool(x4)
|
||||
|
||||
c2 = self.layer1(c1)
|
||||
c3 = self.layer2(c2)
|
||||
c4 = self.layer3(c3)
|
||||
c5 = self.layer4(c4)
|
||||
|
||||
out = self.mean(c5, (2, 3))
|
||||
out = self.flatten(out)
|
||||
out = self.end_point(out)
|
||||
return out
|
||||
|
||||
def _initialize_weights(self):
|
||||
"""
|
||||
Initialize weights.
|
||||
|
||||
Args:
|
||||
|
||||
Returns:
|
||||
None.
|
||||
|
||||
Examples:
|
||||
>>> _initialize_weights()
|
||||
"""
|
||||
self.init_parameters_data()
|
||||
for _, m in self.cells_and_names():
|
||||
if isinstance(m, (nn.Conv2d)):
|
||||
n = m.kernel_size[0] * m.kernel_size[1] * m.out_channels
|
||||
m.weight.set_parameter_data(Tensor(np.random.normal(0, np.sqrt(2. / n),
|
||||
m.weight.data.shape).astype("float32")))
|
||||
if m.bias is not None:
|
||||
m.bias.set_parameter_data(
|
||||
Tensor(np.zeros(m.bias.data.shape, dtype="float32")))
|
||||
elif isinstance(m, nn.BatchNorm2d):
|
||||
m.gamma.set_parameter_data(
|
||||
Tensor(np.ones(m.gamma.data.shape, dtype="float32")))
|
||||
m.beta.set_parameter_data(
|
||||
Tensor(np.zeros(m.beta.data.shape, dtype="float32")))
|
||||
elif isinstance(m, nn.Dense):
|
||||
m.weight.set_parameter_data(Tensor(np.random.normal(
|
||||
0, 0.01, m.weight.data.shape).astype("float32")))
|
||||
if m.bias is not None:
|
||||
m.bias.set_parameter_data(
|
||||
Tensor(np.zeros(m.bias.data.shape, dtype="float32")))
|
||||
|
||||
|
||||
def resnet50(rate=1, class_num=10, index=None):
|
||||
"""
|
||||
Get ResNet50 neural network.
|
||||
|
||||
Args:
|
||||
class_num (int): Class number.
|
||||
|
||||
Returns:
|
||||
Cell, cell instance of ResNet50 neural network.
|
||||
|
||||
Examples:
|
||||
>>> net = resnet50(10)
|
||||
"""
|
||||
return ResNet(rate,
|
||||
ResidualBlock,
|
||||
[3, 4, 6, 3],
|
||||
[64, 256, 512, 1024],
|
||||
[256, 512, 1024, 2048],
|
||||
[1, 2, 2, 2],
|
||||
class_num,
|
||||
index)
|
||||
|
||||
|
||||
def resnet101(rate=1, class_num=10, index=None):
|
||||
"""
|
||||
Get ResNet101 neural network.
|
||||
|
||||
Args:
|
||||
class_num (int): Class number.
|
||||
|
||||
Returns:
|
||||
Cell, cell instance of ResNet101 neural network.
|
||||
|
||||
Examples:
|
||||
>>> net = resnet101(1001)
|
||||
"""
|
||||
return ResNet(rate,
|
||||
ResidualBlock,
|
||||
[3, 4, 23, 3],
|
||||
[64, 256, 512, 1024],
|
||||
[256, 512, 1024, 2048],
|
||||
[1, 2, 2, 2],
|
||||
class_num,
|
||||
index)
|
||||
|
|
@ -0,0 +1,193 @@
|
|||
# [SSD Description](#contents)
|
||||
|
||||
SSD discretizes the output space of bounding boxes into a set of default boxes over different aspect ratios and scales per feature map location. At prediction time, the network generates scores for the presence of each object category in each default box and produces adjustments to the box to better match the object shape.Additionally, the network combines predictions from multiple feature maps with different resolutions to naturally handle objects of various sizes.
|
||||
|
||||
[Paper](https://arxiv.org/abs/1512.02325): Wei Liu, Dragomir Anguelov, Dumitru Erhan, Christian Szegedy, Scott Reed, Cheng-Yang Fu, Alexander C. Berg.European Conference on Computer Vision (ECCV), 2016 (In press).
|
||||
|
||||
# [Model Architecture](#contents)
|
||||
|
||||
The SSD approach is based on a feed-forward convolutional network that produces a fixed-size collection of bounding boxes and scores for the presence of object class instances in those boxes, followed by a non-maximum suppression step to produce the final detections. The early network layers are based on a standard architecture used for high quality image classification, which is called the base network. Then add auxiliary structure to the network to produce detections.
|
||||
|
||||
# [Dataset](#contents)
|
||||
Dataset used: [COCO2017](<http://images.cocodataset.org/>)
|
||||
|
||||
- Dataset size：19G
|
||||
- Train：18G，118000 images
|
||||
- Val：1G，5000 images
|
||||
- Annotations：241M，instances，captions，person_keypoints etc
|
||||
- Data format：image and json files
|
||||
- Note：Data will be processed in dataset.py
|
||||
|
||||
# [Environment Requirements](#contents)
|
||||
|
||||
- Hardware（Ascend/GPU）
|
||||
- Prepare hardware environment with Ascend or GPU processor. If you want to try Ascend, please send the [application form](https://obs-9be7.obs.cn-east-2.myhuaweicloud.com/file/other/Ascend%20Model%20Zoo%E4%BD%93%E9%AA%8C%E8%B5%84%E6%BA%90%E7%94%B3%E8%AF%B7%E8%A1%A8.docx) to ascend@huawei.com. Once approved, you can get the resources.
|
||||
- Framework
|
||||
- [MindSpore](http://10.90.67.50/mindspore/archive/20200506/OpenSource/me_vm_x86/)
|
||||
- For more information, please check the resources below：
|
||||
- [MindSpore tutorials](https://www.mindspore.cn/tutorial/zh-CN/master/index.html)
|
||||
- [MindSpore API](https://www.mindspore.cn/api/zh-CN/master/index.html)
|
||||
|
||||
- Install [MindSpore](https://www.mindspore.cn/install/en).
|
||||
|
||||
- Download the dataset COCO2017.
|
||||
|
||||
- We use COCO2017 as training dataset in this example by default, and you can also use your own datasets.
|
||||
|
||||
1. If coco dataset is used. **Select dataset to coco when run script.**
|
||||
Install Cython and pycocotool, and you can also install mmcv to process data.
|
||||
|
||||
```
|
||||
pip install Cython
|
||||
|
||||
pip install pycocotools
|
||||
|
||||
```
|
||||
And change the COCO_ROOT and other settings you need in `config.py`. The directory structure is as follows:
|
||||
|
||||
```
|
||||
.
|
||||
└─cocodataset
|
||||
├─annotations
|
||||
├─instance_train2017.json
|
||||
└─instance_val2017.json
|
||||
├─val2017
|
||||
└─train2017
|
||||
|
||||
```
|
||||
|
||||
2. If your own dataset is used. **Select dataset to other when run script.**
|
||||
Organize the dataset infomation into a TXT file, each row in the file is as follows:
|
||||
|
||||
```
|
||||
train2017/0000001.jpg 0,259,401,459,7 35,28,324,201,2 0,30,59,80,2
|
||||
|
||||
```
|
||||
|
||||
Each row is an image annotation which split by space, the first column is a relative path of image, the others are box and class infomations of the format [xmin,ymin,xmax,ymax,class]. We read image from an image path joined by the `IMAGE_DIR`(dataset directory) and the relative path in `ANNO_PATH`(the TXT file path), `IMAGE_DIR` and `ANNO_PATH` are setting in `config.py`.
|
||||
|
||||
# [Quick Start](#contents)
|
||||
|
||||
After installing MindSpore via the official website, you can start training and evaluation on Ascend as follows:
|
||||
|
||||
```
|
||||
# single npu training on Ascend
|
||||
python train.py
|
||||
|
||||
# distributed training on Ascend
|
||||
sh run_distribute_train_ghostnet.sh [DEVICE_NUM] [EPOCH_SIZE] [LR] [DATASET] [RANK_TABLE_FILE]
|
||||
|
||||
# run eval on Ascend
|
||||
python eval.py --device_id 0 --dataset coco --checkpoint_path LOG4/ssd-500_458.ckpt
|
||||
```
|
||||
|
||||
# [Script Description](#contents)
|
||||
|
||||
## [Script and Sample Code](#contents)
|
||||
|
||||
```shell
|
||||
|
||||
├── ssd_ghostnet
|
||||
├── README.md ## readme file of ssd_ghostnet
|
||||
├── scripts
|
||||
└─ run_distribute_train_ghostnet.sh ## shell script for distributed on ascend
|
||||
├── src
|
||||
├─ box_util.py ## bbox utils
|
||||
├─ coco_eval.py ## coco metrics utils
|
||||
├─ config_ghostnet_13x.py ## total config
|
||||
├─ dataset.py ## create dataset and process dataset
|
||||
├─ init_params.py ## parameters utils
|
||||
├─ lr_schedule.py ## learning ratio generator
|
||||
└─ ssd_ghostnet.py ## ssd architecture
|
||||
├── eval.py ## eval scripts
|
||||
├── train.py ## train scripts
|
||||
├── mindspore_hub_conf.py # export model for hub
|
||||
```
|
||||
|
||||
## [Script Parameters](#contents)
|
||||
|
||||
```
|
||||
Major parameters in train.py and config_ghostnet_13x.py as follows:
|
||||
|
||||
"device_num": 1 # Use device nums
|
||||
"lr": 0.05 # Learning rate init value
|
||||
"dataset": coco # Dataset name
|
||||
"epoch_size": 500 # Epoch size
|
||||
"batch_size": 32 # Batch size of input tensor
|
||||
"pre_trained": None # Pretrained checkpoint file path
|
||||
"pre_trained_epoch_size": 0 # Pretrained epoch size
|
||||
"save_checkpoint_epochs": 10 # The epoch interval between two checkpoints. By default, the checkpoint will be saved per 10 epochs
|
||||
"loss_scale": 1024 # Loss scale
|
||||
|
||||
"class_num": 81 # Dataset class number
|
||||
"image_shape": [300, 300] # Image height and width used as input to the model
|
||||
"mindrecord_dir": "/data/MindRecord_COCO" # MindRecord path
|
||||
"coco_root": "/data/coco2017" # COCO2017 dataset path
|
||||
"voc_root": "" # VOC original dataset path
|
||||
"image_dir": "" # Other dataset image path, if coco or voc used, it will be useless
|
||||
"anno_path": "" # Other dataset annotation path, if coco or voc used, it will be useless
|
||||
|
||||
```
|
||||
|
||||
|
||||
## [Training Process](#contents)
|
||||
|
||||
### Training on Ascend
|
||||
|
||||
To train the model, run `train.py`. If the `mindrecord_dir` is empty, it will generate [mindrecord](https://www.mindspore.cn/tutorial/en/master/use/data_preparation/converting_datasets.html) files by `coco_root`(coco dataset) or `iamge_dir` and `anno_path`(own dataset). **Note if mindrecord_dir isn't empty, it will use mindrecord_dir instead of raw images.**
|
||||
|
||||
|
||||
- Distribute mode
|
||||
|
||||
```
|
||||
sh run_distribute_train_ghostnet.sh [DEVICE_NUM] [EPOCH_SIZE] [LR] [DATASET] [RANK_TABLE_FILE] [PRE_TRAINED](optional) [PRE_TRAINED_EPOCH_SIZE](optional)
|
||||
```
|
||||
We need five or seven parameters for this scripts.
|
||||
- `DEVICE_NUM`: the device number for distributed train.
|
||||
- `EPOCH_NUM`: epoch num for distributed train.
|
||||
- `LR`: learning rate init value for distributed train.
|
||||
- `DATASET`：the dataset mode for distributed train.
|
||||
- `RANK_TABLE_FILE :` the path of [rank_table.json](https://gitee.com/mindspore/mindspore/tree/master/model_zoo/utils/hccl_tools), it is better to use absolute path.
|
||||
- `PRE_TRAINED :` the path of pretrained checkpoint file, it is better to use absolute path.
|
||||
- `PRE_TRAINED_EPOCH_SIZE :` the epoch num of pretrained.
|
||||
|
||||
Training result will be stored in the current path, whose folder name begins with "LOG". Under this, you can find checkpoint file together with result like the followings in LOG4/log.txt.
|
||||
|
||||
## [Evaluation Process](#contents)
|
||||
|
||||
### Evaluation on Ascend
|
||||
|
||||
```
|
||||
python eval.py --device_id 0 --dataset coco --checkpoint_path LOG4/ssd-500_458.ckpt
|
||||
```
|
||||
|
||||
# [Model Description](#contents)
|
||||
## [Performance](#contents)
|
||||
|
||||
### Evaluation Performance
|
||||
|
||||
| Parameters | Ascend |
|
||||
| -------------------------- | -------------------------------------------------------------|
|
||||
| Model Version | SSD ghostnet |
|
||||
| Resource | Ascend 910 ；CPU 2.60GHz，56cores；Memory，314G |
|
||||
| MindSpore Version | 0.7.0 |
|
||||
| Dataset | COCO2017 |
|
||||
| Training Parameters | epoch = 500, batch_size = 32 |
|
||||
| Optimizer | Momentum |
|
||||
| Loss Function | Sigmoid Cross Entropy,SmoothL1Loss |
|
||||
| Total time | 8pcs: 12hours |
|
||||
|
||||
|
||||
### Inference Performance
|
||||
|
||||
| Parameters | Ascend |
|
||||
| ------------------- | ----------------------------|
|
||||
| Model Version | SSD ghostnet |
|
||||
| Resource | Ascend 910 |
|
||||
| Uploaded Date | 09/08/2020 (month/day/year) |
|
||||
| MindSpore Version | 0.7.0 |
|
||||
| Dataset | COCO2017 |
|
||||
| batch_size | 1 |
|
||||
| outputs | mAP |
|
||||
| Accuracy | IoU=0.50: 24.1% |
|
||||
| Model for inference | 55M(.ckpt file) |
|
||||
|
|
@ -0,0 +1,116 @@
|
|||
# Copyright 2020 Huawei Technologies Co., Ltd
|
||||
#
|
||||
# Licensed under the Apache License, Version 2.0 (the "License");
|
||||
# you may not use this file except in compliance with the License.
|
||||
# You may obtain a copy of the License at
|
||||
#
|
||||
# http://www.apache.org/licenses/LICENSE-2.0
|
||||
#
|
||||
# less required by applicable law or agreed to in writing, software
|
||||
# distributed under the License is distributed on an "AS IS" BASIS,
|
||||
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
# See the License for the specific language governing permissions and
|
||||
# limitations under the License.
|
||||
# ============================================================================
|
||||
|
||||
"""Evaluation for SSD"""
|
||||
|
||||
import os
|
||||
import argparse
|
||||
import time
|
||||
import numpy as np
|
||||
from mindspore import context, Tensor
|
||||
from mindspore.train.serialization import load_checkpoint, load_param_into_net
|
||||
from src.ssd_ghostnet import SSD300, ssd_ghostnet
|
||||
from src.dataset import create_ssd_dataset, data_to_mindrecord_byte_image, voc_data_to_mindrecord
|
||||
# from src.config_ghostnet import config
|
||||
from src.config_ghostnet_13x import config
|
||||
from src.coco_eval import metrics
|
||||
|
||||
|
||||
def ssd_eval(dataset_path, ckpt_path):
|
||||
"""SSD evaluation."""
|
||||
batch_size = 1
|
||||
ds = create_ssd_dataset(
|
||||
dataset_path, batch_size=batch_size, repeat_num=1, is_training=False)
|
||||
net = SSD300(ssd_ghostnet(), config, is_training=False)
|
||||
print("Load Checkpoint!")
|
||||
param_dict = load_checkpoint(ckpt_path)
|
||||
net.init_parameters_data()
|
||||
load_param_into_net(net, param_dict)
|
||||
|
||||
net.set_train(False)
|
||||
i = batch_size
|
||||
total = ds.get_dataset_size() * batch_size
|
||||
start = time.time()
|
||||
pred_data = []
|
||||
print("\n========================================\n")
|
||||
print("total images num: ", total)
|
||||
print("Processing, please wait a moment.")
|
||||
for data in ds.create_dict_iterator():
|
||||
img_id = data['img_id']
|
||||
img_np = data['image']
|
||||
image_shape = data['image_shape']
|
||||
|
||||
output = net(Tensor(img_np))
|
||||
for batch_idx in range(img_np.shape[0]):
|
||||
pred_data.append({"boxes": output[0].asnumpy()[batch_idx],
|
||||
"box_scores": output[1].asnumpy()[batch_idx],
|
||||
"img_id": int(np.squeeze(img_id[batch_idx])),
|
||||
"image_shape": image_shape[batch_idx]})
|
||||
percent = round(i / total * 100., 2)
|
||||
|
||||
print(f' {str(percent)} [{i}/{total}]', end='\r')
|
||||
i += batch_size
|
||||
cost_time = int((time.time() - start) * 1000)
|
||||
print(f' 100% [{total}/{total}] cost {cost_time} ms')
|
||||
mAP = metrics(pred_data)
|
||||
print("\n========================================\n")
|
||||
print(f"mAP: {mAP}")
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
parser = argparse.ArgumentParser(description='SSD evaluation')
|
||||
parser.add_argument("--device_id", type=int, default=0,
|
||||
help="Device id, default is 0.")
|
||||
parser.add_argument("--dataset", type=str, default="coco",
|
||||
help="Dataset, default is coco.")
|
||||
parser.add_argument("--checkpoint_path", type=str,
|
||||
required=True, help="Checkpoint file path.")
|
||||
args_opt = parser.parse_args()
|
||||
|
||||
context.set_context(mode=context.GRAPH_MODE,
|
||||
device_target="Ascend", device_id=args_opt.device_id)
|
||||
|
||||
prefix = "ssd_eval.mindrecord"
|
||||
mindrecord_dir = config.mindrecord_dir
|
||||
mindrecord_file = os.path.join(mindrecord_dir, prefix + "0")
|
||||
if args_opt.dataset == "voc":
|
||||
config.coco_root = config.voc_root
|
||||
if not os.path.exists(mindrecord_file):
|
||||
if not os.path.isdir(mindrecord_dir):
|
||||
os.makedirs(mindrecord_dir)
|
||||
if args_opt.dataset == "coco":
|
||||
if os.path.isdir(config.coco_root):
|
||||
print("Create Mindrecord.")
|
||||
data_to_mindrecord_byte_image("coco", False, prefix)
|
||||
print("Create Mindrecord Done, at {}".format(mindrecord_dir))
|
||||
else:
|
||||
print("coco_root not exits.")
|
||||
elif args_opt.dataset == "voc":
|
||||
if os.path.isdir(config.voc_dir) and os.path.isdir(config.voc_root):
|
||||
print("Create Mindrecord.")
|
||||
voc_data_to_mindrecord(mindrecord_dir, False, prefix)
|
||||
print("Create Mindrecord Done, at {}".format(mindrecord_dir))
|
||||
else:
|
||||
print("voc_root or voc_dir not exits.")
|
||||
else:
|
||||
if os.path.isdir(config.image_dir) and os.path.exists(config.anno_path):
|
||||
print("Create Mindrecord.")
|
||||
data_to_mindrecord_byte_image("other", False, prefix)
|
||||
print("Create Mindrecord Done, at {}".format(mindrecord_dir))
|
||||
else:
|
||||
print("IMAGE_DIR or ANNO_PATH not exits.")
|
||||
|
||||
print("Start Eval!")
|
||||
ssd_eval(mindrecord_file, args_opt.checkpoint_path)
|
||||
|
|
@ -0,0 +1,22 @@
|
|||
# Copyright 2020 Huawei Technologies Co., Ltd
|
||||
#
|
||||
# Licensed under the Apache License, Version 2.0 (the "License");
|
||||
# you may not use this file except in compliance with the License.
|
||||
# You may obtain a copy of the License at
|
||||
#
|
||||
# http://www.apache.org/licenses/LICENSE-2.0
|
||||
#
|
||||
# Unless required by applicable law or agreed to in writing, software
|
||||
# distributed under the License is distributed on an "AS IS" BASIS,
|
||||
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
# See the License for the specific language governing permissions and
|
||||
# limitations under the License.
|
||||
# ============================================================================
|
||||
"""hub config."""
|
||||
from src.ssd_ghostnet import SSD300, ssd_ghostnet
|
||||
from src.config_ghostnet_13x import config
|
||||
|
||||
def create_network(name, *args, **kwargs):
|
||||
if name == 'ghostnet_ssd':
|
||||
return SSD300(ssd_ghostnet(), config, **kwargs)
|
||||
raise NotImplementedError(f"{name} is not implemented in the repo")
|
||||
|
|
@ -0,0 +1,82 @@
|
|||
#!/bin/bash
|
||||
# Copyright 2020 Huawei Technologies Co., Ltd
|
||||
#
|
||||
# Licensed under the Apache License, Version 2.0 (the "License");
|
||||
# you may not use this file except in compliance with the License.
|
||||
# You may obtain a copy of the License at
|
||||
#
|
||||
# http://www.apache.org/licenses/LICENSE-2.0
|
||||
#
|
||||
# Unless required by applicable law or agreed to in writing, software
|
||||
# distributed under the License is distributed on an "AS IS" BASIS,
|
||||
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
# See the License for the specific language governing permissions and
|
||||
# limitations under the License.
|
||||
# ============================================================================
|
||||
|
||||
echo "=============================================================================================================="
|
||||
echo "Please run the scipt as: "
|
||||
echo "sh run_distribute_train_ghostnet.sh DEVICE_NUM EPOCH_SIZE LR DATASET RANK_TABLE_FILE PRE_TRAINED PRE_TRAINED_EPOCH_SIZE"
|
||||
echo "for example: sh run_distribute_train_ghostnet.sh 8 500 0.2 coco /data/hccl.json /opt/ssd-300.ckpt(optional) 200(optional)"
|
||||
echo "It is better to use absolute path."
|
||||
echo "================================================================================================================="
|
||||
|
||||
if [ $# != 5 ] && [ $# != 7 ]
|
||||
then
|
||||
echo "Usage: sh run_distribute_train_ghostnet.sh [DEVICE_NUM] [EPOCH_SIZE] [LR] [DATASET] \
|
||||
[RANK_TABLE_FILE] [PRE_TRAINED](optional) [PRE_TRAINED_EPOCH_SIZE](optional)"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
# Before start distribute train, first create mindrecord files.
|
||||
BASE_PATH=$(cd "`dirname $0`" || exit; pwd)
|
||||
cd $BASE_PATH/../ || exit
|
||||
python train.py --only_create_dataset=True
|
||||
|
||||
echo "After running the scipt, the network runs in the background. The log will be generated in LOGx/log.txt"
|
||||
|
||||
export RANK_SIZE=$1
|
||||
EPOCH_SIZE=$2
|
||||
LR=$3
|
||||
DATASET=$4
|
||||
PRE_TRAINED=$6
|
||||
PRE_TRAINED_EPOCH_SIZE=$7
|
||||
export RANK_TABLE_FILE=$5
|
||||
|
||||
for((i=0;i<RANK_SIZE;i++))
|
||||
do
|
||||
export DEVICE_ID=$i
|
||||
rm -rf LOG$i
|
||||
mkdir ./LOG$i
|
||||
cp ./*.py ./LOG$i
|
||||
cp -r ./src ./LOG$i
|
||||
cd ./LOG$i || exit
|
||||
export RANK_ID=$i
|
||||
echo "start training for rank $i, device $DEVICE_ID"
|
||||
env > env.log
|
||||
if [ $# == 5 ]
|
||||
then
|
||||
python train.py \
|
||||
--distribute=True \
|
||||
--lr=$LR \
|
||||
--dataset=$DATASET \
|
||||
--device_num=$RANK_SIZE \
|
||||
--device_id=$DEVICE_ID \
|
||||
--epoch_size=$EPOCH_SIZE > log.txt 2>&1 &
|
||||
fi
|
||||
|
||||
if [ $# == 7 ]
|
||||
then
|
||||
python train.py \
|
||||
--distribute=True \
|
||||
--lr=$LR \
|
||||
--dataset=$DATASET \
|
||||
--device_num=$RANK_SIZE \
|
||||
--device_id=$DEVICE_ID \
|
||||
--pre_trained=$PRE_TRAINED \
|
||||
--pre_trained_epoch_size=$PRE_TRAINED_EPOCH_SIZE \
|
||||
--epoch_size=$EPOCH_SIZE > log.txt 2>&1 &
|
||||
fi
|
||||
|
||||
cd ../
|
||||
done
|
||||
|
|
@ -0,0 +1,175 @@
|
|||
# Copyright 2020 Huawei Technologies Co., Ltd
|
||||
#
|
||||
# Licensed under the Apache License, Version 2.0 (the "License");
|
||||
# you may not use this file except in compliance with the License.
|
||||
# You may obtain a copy of the License at
|
||||
#
|
||||
# http://www.apache.org/licenses/LICENSE-2.0
|
||||
#
|
||||
# Unless required by applicable law or agreed to in writing, software
|
||||
# distributed under the License is distributed on an "AS IS" BASIS,
|
||||
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
# See the License for the specific language governing permissions and
|
||||
# limitations under the License.
|
||||
# ============================================================================
|
||||
|
||||
"""Bbox utils"""
|
||||
|
||||
import math
|
||||
import itertools as it
|
||||
import numpy as np
|
||||
from .config_ghostnet_13x import config
|
||||
|
||||
|
||||
class GeneratDefaultBoxes():
|
||||
"""
|
||||
Generate Default boxes for SSD, follows the order of (W, H, archor_sizes).
|
||||
`self.default_boxes` has a shape of [archor_sizes, H, W, 4], the last dimension is [y, x, h, w].
|
||||
`self.default_boxes_ltrb` has a shape as `self.default_boxes`, the last dimension is [y1, x1, y2, x2].
|
||||
"""
|
||||
|
||||
def __init__(self):
|
||||
fk = config.img_shape[0] / np.array(config.steps)
|
||||
scale_rate = (config.max_scale - config.min_scale) / \
|
||||
(len(config.num_default) - 1)
|
||||
scales = [config.min_scale + scale_rate *
|
||||
i for i in range(len(config.num_default))] + [1.0]
|
||||
self.default_boxes = []
|
||||
for idex, feature_size in enumerate(config.feature_size):
|
||||
sk1 = scales[idex]
|
||||
sk2 = scales[idex + 1]
|
||||
sk3 = math.sqrt(sk1 * sk2)
|
||||
if idex == 0:
|
||||
w, h = sk1 * math.sqrt(2), sk1 / math.sqrt(2)
|
||||
all_sizes = [(0.1, 0.1), (w, h), (h, w)]
|
||||
else:
|
||||
all_sizes = [(sk1, sk1)]
|
||||
for aspect_ratio in config.aspect_ratios[idex]:
|
||||
w, h = sk1 * math.sqrt(aspect_ratio), sk1 / \
|
||||
math.sqrt(aspect_ratio)
|
||||
all_sizes.append((w, h))
|
||||
all_sizes.append((h, w))
|
||||
all_sizes.append((sk3, sk3))
|
||||
|
||||
assert len(all_sizes) == config.num_default[idex]
|
||||
|
||||
for i, j in it.product(range(feature_size), repeat=2):
|
||||
for w, h in all_sizes:
|
||||
cx, cy = (j + 0.5) / fk[idex], (i + 0.5) / fk[idex]
|
||||
self.default_boxes.append([cy, cx, h, w])
|
||||
|
||||
def to_ltrb(cy, cx, h, w):
|
||||
return cy - h / 2, cx - w / 2, cy + h / 2, cx + w / 2
|
||||
|
||||
# For IoU calculation
|
||||
self.default_boxes_ltrb = np.array(
|
||||
tuple(to_ltrb(*i) for i in self.default_boxes), dtype='float32')
|
||||
self.default_boxes = np.array(self.default_boxes, dtype='float32')
|
||||
|
||||
|
||||
default_boxes_ltrb = GeneratDefaultBoxes().default_boxes_ltrb
|
||||
default_boxes = GeneratDefaultBoxes().default_boxes
|
||||
y1, x1, y2, x2 = np.split(default_boxes_ltrb[:, :4], 4, axis=-1)
|
||||
vol_anchors = (x2 - x1) * (y2 - y1)
|
||||
matching_threshold = config.match_thershold
|
||||
|
||||
|
||||
def ssd_bboxes_encode(boxes):
|
||||
"""
|
||||
Labels anchors with ground truth inputs.
|
||||
|
||||
Args:
|
||||
boxex: ground truth with shape [N, 5], for each row, it stores [y, x, h, w, cls].
|
||||
|
||||
Returns:
|
||||
gt_loc: location ground truth with shape [num_anchors, 4].
|
||||
gt_label: class ground truth with shape [num_anchors, 1].
|
||||
num_matched_boxes: number of positives in an image.
|
||||
"""
|
||||
|
||||
def jaccard_with_anchors(bbox):
|
||||
"""Compute jaccard score a box and the anchors."""
|
||||
# Intersection bbox and volume.
|
||||
ymin = np.maximum(y1, bbox[0])
|
||||
xmin = np.maximum(x1, bbox[1])
|
||||
ymax = np.minimum(y2, bbox[2])
|
||||
xmax = np.minimum(x2, bbox[3])
|
||||
w = np.maximum(xmax - xmin, 0.)
|
||||
h = np.maximum(ymax - ymin, 0.)
|
||||
|
||||
# Volumes.
|
||||
inter_vol = h * w
|
||||
union_vol = vol_anchors + \
|
||||
(bbox[2] - bbox[0]) * (bbox[3] - bbox[1]) - inter_vol
|
||||
jaccard = inter_vol / union_vol
|
||||
return np.squeeze(jaccard)
|
||||
|
||||
pre_scores = np.zeros((config.num_ssd_boxes), dtype=np.float32)
|
||||
t_boxes = np.zeros((config.num_ssd_boxes, 4), dtype=np.float32)
|
||||
t_label = np.zeros((config.num_ssd_boxes), dtype=np.int64)
|
||||
for bbox in boxes:
|
||||
label = int(bbox[4])
|
||||
scores = jaccard_with_anchors(bbox)
|
||||
idx = np.argmax(scores)
|
||||
scores[idx] = 2.0
|
||||
mask = (scores > matching_threshold)
|
||||
mask = mask & (scores > pre_scores)
|
||||
pre_scores = np.maximum(pre_scores, scores * mask)
|
||||
t_label = mask * label + (1 - mask) * t_label
|
||||
for i in range(4):
|
||||
t_boxes[:, i] = mask * bbox[i] + (1 - mask) * t_boxes[:, i]
|
||||
|
||||
index = np.nonzero(t_label)
|
||||
|
||||
# Transform to ltrb.
|
||||
bboxes = np.zeros((config.num_ssd_boxes, 4), dtype=np.float32)
|
||||
bboxes[:, [0, 1]] = (t_boxes[:, [0, 1]] + t_boxes[:, [2, 3]]) / 2
|
||||
bboxes[:, [2, 3]] = t_boxes[:, [2, 3]] - t_boxes[:, [0, 1]]
|
||||
|
||||
# Encode features.
|
||||
bboxes_t = bboxes[index]
|
||||
default_boxes_t = default_boxes[index]
|
||||
bboxes_t[:, :2] = (bboxes_t[:, :2] - default_boxes_t[:, :2]) / \
|
||||
(default_boxes_t[:, 2:] * config.prior_scaling[0])
|
||||
bboxes_t[:, 2:4] = np.log(
|
||||
bboxes_t[:, 2:4] / default_boxes_t[:, 2:4]) / config.prior_scaling[1]
|
||||
bboxes[index] = bboxes_t
|
||||
|
||||
num_match = np.array([len(np.nonzero(t_label)[0])], dtype=np.int32)
|
||||
return bboxes, t_label.astype(np.int32), num_match
|
||||
|
||||
|
||||
def ssd_bboxes_decode(boxes):
|
||||
"""Decode predict boxes to [y, x, h, w]"""
|
||||
boxes_t = boxes.copy()
|
||||
default_boxes_t = default_boxes.copy()
|
||||
boxes_t[:, :2] = boxes_t[:, :2] * config.prior_scaling[0] * \
|
||||
default_boxes_t[:, 2:] + default_boxes_t[:, :2]
|
||||
boxes_t[:, 2:4] = np.exp(
|
||||
boxes_t[:, 2:4] * config.prior_scaling[1]) * default_boxes_t[:, 2:4]
|
||||
|
||||
bboxes = np.zeros((len(boxes_t), 4), dtype=np.float32)
|
||||
|
||||
bboxes[:, [0, 1]] = boxes_t[:, [0, 1]] - boxes_t[:, [2, 3]] / 2
|
||||
bboxes[:, [2, 3]] = boxes_t[:, [0, 1]] + boxes_t[:, [2, 3]] / 2
|
||||
|
||||
return np.clip(bboxes, 0, 1)
|
||||
|
||||
|
||||
def intersect(box_a, box_b):
|
||||
"""Compute the intersect of two sets of boxes."""
|
||||
max_yx = np.minimum(box_a[:, 2:4], box_b[2:4])
|
||||
min_yx = np.maximum(box_a[:, :2], box_b[:2])
|
||||
inter = np.clip((max_yx - min_yx), a_min=0, a_max=np.inf)
|
||||
return inter[:, 0] * inter[:, 1]
|
||||
|
||||
|
||||
def jaccard_numpy(box_a, box_b):
|
||||
"""Compute the jaccard overlap of two sets of boxes."""
|
||||
inter = intersect(box_a, box_b)
|
||||
area_a = ((box_a[:, 2] - box_a[:, 0]) *
|
||||
(box_a[:, 3] - box_a[:, 1]))
|
||||
area_b = ((box_b[2] - box_b[0]) *
|
||||
(box_b[3] - box_b[1]))
|
||||
union = area_a + area_b - inter
|
||||
return inter / union
|
||||
|
|
@ -0,0 +1,129 @@
|
|||
# Copyright 2020 Huawei Technologies Co., Ltd
|
||||
#
|
||||
# Licensed under the Apache License, Version 2.0 (the "License");
|
||||
# you may not use this file except in compliance with the License.
|
||||
# You may obtain a copy of the License at
|
||||
#
|
||||
# http://www.apache.org/licenses/LICENSE-2.0
|
||||
#
|
||||
# Unless required by applicable law or agreed to in writing, software
|
||||
# distributed under the License is distributed on an "AS IS" BASIS,
|
||||
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
# See the License for the specific language governing permissions and
|
||||
# limitations under the License.
|
||||
# ============================================================================
|
||||
"""Coco metrics utils"""
|
||||
|
||||
import os
|
||||
import json
|
||||
import numpy as np
|
||||
from .config_ghostnet_13x import config
|
||||
from .box_utils import ssd_bboxes_decode
|
||||
|
||||
|
||||
def apply_nms(all_boxes, all_scores, thres, max_boxes):
|
||||
"""Apply NMS to bboxes."""
|
||||
y1 = all_boxes[:, 0]
|
||||
x1 = all_boxes[:, 1]
|
||||
y2 = all_boxes[:, 2]
|
||||
x2 = all_boxes[:, 3]
|
||||
areas = (x2 - x1 + 1) * (y2 - y1 + 1)
|
||||
|
||||
order = all_scores.argsort()[::-1]
|
||||
keep = []
|
||||
|
||||
while order.size > 0:
|
||||
i = order[0]
|
||||
keep.append(i)
|
||||
|
||||
if len(keep) >= max_boxes:
|
||||
break
|
||||
|
||||
xx1 = np.maximum(x1[i], x1[order[1:]])
|
||||
yy1 = np.maximum(y1[i], y1[order[1:]])
|
||||
xx2 = np.minimum(x2[i], x2[order[1:]])
|
||||
yy2 = np.minimum(y2[i], y2[order[1:]])
|
||||
|
||||
w = np.maximum(0.0, xx2 - xx1 + 1)
|
||||
h = np.maximum(0.0, yy2 - yy1 + 1)
|
||||
inter = w * h
|
||||
|
||||
ovr = inter / (areas[i] + areas[order[1:]] - inter)
|
||||
|
||||
inds = np.where(ovr <= thres)[0]
|
||||
|
||||
order = order[inds + 1]
|
||||
return keep
|
||||
|
||||
|
||||
def metrics(pred_data):
|
||||
"""Calculate mAP of predicted bboxes."""
|
||||
from pycocotools.coco import COCO
|
||||
from pycocotools.cocoeval import COCOeval
|
||||
num_classes = config.num_classes
|
||||
|
||||
coco_root = config.coco_root
|
||||
data_type = config.val_data_type
|
||||
|
||||
# Classes need to train or test.
|
||||
val_cls = config.coco_classes
|
||||
val_cls_dict = {}
|
||||
for i, cls in enumerate(val_cls):
|
||||
val_cls_dict[i] = cls
|
||||
|
||||
anno_json = os.path.join(coco_root, config.instances_set.format(data_type))
|
||||
coco_gt = COCO(anno_json)
|
||||
classs_dict = {}
|
||||
cat_ids = coco_gt.loadCats(coco_gt.getCatIds())
|
||||
for cat in cat_ids:
|
||||
classs_dict[cat["name"]] = cat["id"]
|
||||
|
||||
predictions = []
|
||||
img_ids = []
|
||||
|
||||
for sample in pred_data:
|
||||
pred_boxes = sample['boxes']
|
||||
box_scores = sample['box_scores']
|
||||
img_id = sample['img_id']
|
||||
h, w = sample['image_shape']
|
||||
|
||||
pred_boxes = ssd_bboxes_decode(pred_boxes)
|
||||
final_boxes = []
|
||||
final_label = []
|
||||
final_score = []
|
||||
img_ids.append(img_id)
|
||||
|
||||
for c in range(1, num_classes):
|
||||
class_box_scores = box_scores[:, c]
|
||||
score_mask = class_box_scores > config.min_score
|
||||
class_box_scores = class_box_scores[score_mask]
|
||||
class_boxes = pred_boxes[score_mask] * [h, w, h, w]
|
||||
|
||||
if score_mask.any():
|
||||
nms_index = apply_nms(
|
||||
class_boxes, class_box_scores, config.nms_thershold, config.max_boxes)
|
||||
class_boxes = class_boxes[nms_index]
|
||||
class_box_scores = class_box_scores[nms_index]
|
||||
|
||||
final_boxes += class_boxes.tolist()
|
||||
final_score += class_box_scores.tolist()
|
||||
final_label += [classs_dict[val_cls_dict[c]]] * \
|
||||
len(class_box_scores)
|
||||
|
||||
for loc, label, score in zip(final_boxes, final_label, final_score):
|
||||
res = {}
|
||||
res['image_id'] = img_id
|
||||
res['bbox'] = [loc[1], loc[0], loc[3] - loc[1], loc[2] - loc[0]]
|
||||
res['score'] = score
|
||||
res['category_id'] = label
|
||||
predictions.append(res)
|
||||
with open('predictions.json', 'w') as f:
|
||||
json.dump(predictions, f)
|
||||
|
||||
coco_dt = coco_gt.loadRes('predictions.json')
|
||||
E = COCOeval(coco_gt, coco_dt, iouType='bbox')
|
||||
E.params.imgIds = img_ids
|
||||
E.evaluate()
|
||||
E.accumulate()
|
||||
E.summarize()
|
||||
return E.stats[0]
|
||||
|
|
@ -0,0 +1,81 @@
|
|||
# Copyright 2020 Huawei Technologies Co., Ltd
|
||||
#
|
||||
# Licensed under the Apache License, Version 2.0 (the "License");
|
||||
# you may not use this file except in compliance with the License.
|
||||
# You may obtain a copy of the License at
|
||||
#
|
||||
# http://www.apache.org/licenses/LICENSE-2.0
|
||||
#
|
||||
# Unless required by applicable law or agreed to in writing, software
|
||||
# distributed under the License is distributed on an "AS IS" BASIS,
|
||||
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
# See the License for the specific language governing permissions and
|
||||
# limitations under the License.
|
||||
# " ============================================================================
|
||||
|
||||
"""Config parameters for SSD models."""
|
||||
|
||||
from easydict import EasyDict as ed
|
||||
|
||||
config = ed({
|
||||
"img_shape": [300, 300],
|
||||
"num_ssd_boxes": 1917,
|
||||
"neg_pre_positive": 3,
|
||||
"match_thershold": 0.5,
|
||||
"nms_thershold": 0.6,
|
||||
"min_score": 0.1,
|
||||
"max_boxes": 100,
|
||||
|
||||
# learing rate settings
|
||||
"global_step": 0,
|
||||
"lr_init": 0.001,
|
||||
"lr_end_rate": 0.001,
|
||||
"warmup_epochs": 2,
|
||||
"momentum": 0.9,
|
||||
"weight_decay": 1.5e-4,
|
||||
|
||||
# network
|
||||
"num_default": [3, 6, 6, 6, 6, 6],
|
||||
"extras_in_channels": [256, 864, 1248, 512, 256, 256],
|
||||
"extras_out_channels": [864, 1248, 512, 256, 256, 128],
|
||||
"extras_srides": [1, 1, 2, 2, 2, 2],
|
||||
"extras_ratio": [0.2, 0.2, 0.2, 0.25, 0.5, 0.25],
|
||||
"feature_size": [19, 10, 5, 3, 2, 1],
|
||||
"min_scale": 0.2,
|
||||
"max_scale": 0.95,
|
||||
"aspect_ratios": [(2,), (2, 3), (2, 3), (2, 3), (2, 3), (2, 3)],
|
||||
"steps": (16, 32, 64, 100, 150, 300),
|
||||
"prior_scaling": (0.1, 0.2),
|
||||
"gamma": 2.0,
|
||||
"alpha": 0.75,
|
||||
|
||||
# `mindrecord_dir` and `coco_root` are better to use absolute path.
|
||||
"mindrecord_dir": "/ssd0/liuchuanjian/mscoco2017/MindRecord_COCO",
|
||||
"coco_root": "/ssd0/liuchuanjian/mscoco2017",
|
||||
"train_data_type": "train2017",
|
||||
"val_data_type": "val2017",
|
||||
"instances_set": "annotations/instances_{}.json",
|
||||
"coco_classes": ('background', 'person', 'bicycle', 'car', 'motorcycle', 'airplane', 'bus',
|
||||
'train', 'truck', 'boat', 'traffic light', 'fire hydrant',
|
||||
'stop sign', 'parking meter', 'bench', 'bird', 'cat', 'dog',
|
||||
'horse', 'sheep', 'cow', 'elephant', 'bear', 'zebra',
|
||||
'giraffe', 'backpack', 'umbrella', 'handbag', 'tie',
|
||||
'suitcase', 'frisbee', 'skis', 'snowboard', 'sports ball',
|
||||
'kite', 'baseball bat', 'baseball glove', 'skateboard',
|
||||
'surfboard', 'tennis racket', 'bottle', 'wine glass', 'cup',
|
||||
'fork', 'knife', 'spoon', 'bowl', 'banana', 'apple',
|
||||
'sandwich', 'orange', 'broccoli', 'carrot', 'hot dog', 'pizza',
|
||||
'donut', 'cake', 'chair', 'couch', 'potted plant', 'bed',
|
||||
'dining table', 'toilet', 'tv', 'laptop', 'mouse', 'remote',
|
||||
'keyboard', 'cell phone', 'microwave', 'oven', 'toaster', 'sink',
|
||||
'refrigerator', 'book', 'clock', 'vase', 'scissors',
|
||||
'teddy bear', 'hair drier', 'toothbrush'),
|
||||
"num_classes": 81,
|
||||
# The annotation.json position of voc validation dataset.
|
||||
"voc_root": "",
|
||||
# voc original dataset.
|
||||
"voc_dir": "",
|
||||
# if coco or voc used, `image_dir` and `anno_path` are useless.
|
||||
"image_dir": "",
|
||||
"anno_path": "",
|
||||
})
|
||||
|
|
@ -0,0 +1,424 @@
|
|||
# Copyright 2020 Huawei Technologies Co., Ltd
|
||||
#
|
||||
# Licensed under the Apache License, Version 2.0 (the "License");
|
||||
# you may not use this file except in compliance with the License.
|
||||
# You may obtain a copy of the License at
|
||||
#
|
||||
# http://www.apache.org/licenses/LICENSE-2.0
|
||||
#
|
||||
# Unless required by applicable law or agreed to in writing, software
|
||||
# distributed under the License is distributed on an "AS IS" BASIS,
|
||||
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
# See the License for the specific language governing permissions and
|
||||
# limitations under the License.
|
||||
# ============================================================================
|
||||
|
||||
"""SSD dataset"""
|
||||
|
||||
from __future__ import division
|
||||
|
||||
import os
|
||||
import json
|
||||
import xml.etree.ElementTree as et
|
||||
import numpy as np
|
||||
import cv2
|
||||
|
||||
import mindspore.dataset as de
|
||||
import mindspore.dataset.transforms.vision.c_transforms as C
|
||||
from mindspore.mindrecord import FileWriter
|
||||
from .config_ghostnet_13x import config
|
||||
from .box_utils import jaccard_numpy, ssd_bboxes_encode
|
||||
|
||||
|
||||
def _rand(a=0., b=1.):
|
||||
"""Generate random."""
|
||||
return np.random.rand() * (b - a) + a
|
||||
|
||||
|
||||
def get_imageId_from_fileName(filename):
|
||||
"""Get imageID from fileName"""
|
||||
try:
|
||||
filename = os.path.splitext(filename)[0]
|
||||
return int(filename)
|
||||
except:
|
||||
raise NotImplementedError(
|
||||
'Filename %s is supposed to be an integer.' % (filename))
|
||||
|
||||
|
||||
def random_sample_crop(image, boxes):
|
||||
"""Random Crop the image and boxes"""
|
||||
height, width, _ = image.shape
|
||||
min_iou = np.random.choice([None, 0.1, 0.3, 0.5, 0.7, 0.9])
|
||||
|
||||
if min_iou is None:
|
||||
return image, boxes
|
||||
|
||||
# max trails (50)
|
||||
for _ in range(50):
|
||||
image_t = image
|
||||
|
||||
w = _rand(0.3, 1.0) * width
|
||||
h = _rand(0.3, 1.0) * height
|
||||
|
||||
# aspect ratio constraint b/t .5 & 2
|
||||
if h / w < 0.5 or h / w > 2:
|
||||
continue
|
||||
|
||||
left = _rand() * (width - w)
|
||||
top = _rand() * (height - h)
|
||||
|
||||
rect = np.array([int(top), int(left), int(top+h), int(left+w)])
|
||||
overlap = jaccard_numpy(boxes, rect)
|
||||
|
||||
# dropout some boxes
|
||||
drop_mask = overlap > 0
|
||||
if not drop_mask.any():
|
||||
continue
|
||||
|
||||
if overlap[drop_mask].min() < min_iou and overlap[drop_mask].max() > (min_iou + 0.2):
|
||||
continue
|
||||
|
||||
image_t = image_t[rect[0]:rect[2], rect[1]:rect[3], :]
|
||||
|
||||
centers = (boxes[:, :2] + boxes[:, 2:4]) / 2.0
|
||||
|
||||
m1 = (rect[0] < centers[:, 0]) * (rect[1] < centers[:, 1])
|
||||
m2 = (rect[2] > centers[:, 0]) * (rect[3] > centers[:, 1])
|
||||
|
||||
# mask in that both m1 and m2 are true
|
||||
mask = m1 * m2 * drop_mask
|
||||
|
||||
# have any valid boxes? try again if not
|
||||
if not mask.any():
|
||||
continue
|
||||
|
||||
# take only matching gt boxes
|
||||
boxes_t = boxes[mask, :].copy()
|
||||
|
||||
boxes_t[:, :2] = np.maximum(boxes_t[:, :2], rect[:2])
|
||||
boxes_t[:, :2] -= rect[:2]
|
||||
boxes_t[:, 2:4] = np.minimum(boxes_t[:, 2:4], rect[2:4])
|
||||
boxes_t[:, 2:4] -= rect[:2]
|
||||
|
||||
return image_t, boxes_t
|
||||
return image, boxes
|
||||
|
||||
|
||||
def preprocess_fn(img_id, image, box, is_training):
|
||||
"""Preprocess function for dataset."""
|
||||
def _infer_data(image, input_shape):
|
||||
img_h, img_w, _ = image.shape
|
||||
input_h, input_w = input_shape
|
||||
|
||||
image = cv2.resize(image, (input_w, input_h))
|
||||
|
||||
# When the channels of image is 1
|
||||
if len(image.shape) == 2:
|
||||
image = np.expand_dims(image, axis=-1)
|
||||
image = np.concatenate([image, image, image], axis=-1)
|
||||
|
||||
return img_id, image, np.array((img_h, img_w), np.float32)
|
||||
|
||||
def _data_aug(image, box, is_training, image_size=(300, 300)):
|
||||
"""Data augmentation function."""
|
||||
ih, iw, _ = image.shape
|
||||
w, h = image_size
|
||||
|
||||
if not is_training:
|
||||
return _infer_data(image, image_size)
|
||||
|
||||
# Random crop
|
||||
box = box.astype(np.float32)
|
||||
image, box = random_sample_crop(image, box)
|
||||
ih, iw, _ = image.shape
|
||||
|
||||
# Resize image
|
||||
image = cv2.resize(image, (w, h))
|
||||
|
||||
# Flip image or not
|
||||
flip = _rand() < .5
|
||||
if flip:
|
||||
image = cv2.flip(image, 1, dst=None)
|
||||
|
||||
# When the channels of image is 1
|
||||
if len(image.shape) == 2:
|
||||
image = np.expand_dims(image, axis=-1)
|
||||
image = np.concatenate([image, image, image], axis=-1)
|
||||
|
||||
box[:, [0, 2]] = box[:, [0, 2]] / ih
|
||||
box[:, [1, 3]] = box[:, [1, 3]] / iw
|
||||
|
||||
if flip:
|
||||
box[:, [1, 3]] = 1 - box[:, [3, 1]]
|
||||
|
||||
box, label, num_match = ssd_bboxes_encode(box)
|
||||
return image, box, label, num_match
|
||||
return _data_aug(image, box, is_training, image_size=config.img_shape)
|
||||
|
||||
|
||||
def create_voc_label(is_training):
|
||||
"""Get image path and annotation from VOC."""
|
||||
voc_dir = config.voc_dir
|
||||
cls_map = {name: i for i, name in enumerate(config.coco_classes)}
|
||||
sub_dir = 'train' if is_training else 'eval'
|
||||
#sub_dir = 'train'
|
||||
voc_dir = os.path.join(voc_dir, sub_dir)
|
||||
if not os.path.isdir(voc_dir):
|
||||
raise ValueError(f'Cannot find {sub_dir} dataset path.')
|
||||
|
||||
image_dir = anno_dir = voc_dir
|
||||
if os.path.isdir(os.path.join(voc_dir, 'Images')):
|
||||
image_dir = os.path.join(voc_dir, 'Images')
|
||||
if os.path.isdir(os.path.join(voc_dir, 'Annotations')):
|
||||
anno_dir = os.path.join(voc_dir, 'Annotations')
|
||||
|
||||
if not is_training:
|
||||
data_dir = config.voc_root
|
||||
json_file = os.path.join(
|
||||
data_dir, config.instances_set.format(sub_dir))
|
||||
file_dir = os.path.split(json_file)[0]
|
||||
if not os.path.isdir(file_dir):
|
||||
os.makedirs(file_dir)
|
||||
json_dict = {"images": [], "type": "instances", "annotations": [],
|
||||
"categories": []}
|
||||
bnd_id = 1
|
||||
|
||||
image_files_dict = {}
|
||||
image_anno_dict = {}
|
||||
images = []
|
||||
for anno_file in os.listdir(anno_dir):
|
||||
print(anno_file)
|
||||
if not anno_file.endswith('xml'):
|
||||
continue
|
||||
tree = et.parse(os.path.join(anno_dir, anno_file))
|
||||
root_node = tree.getroot()
|
||||
file_name = root_node.find('filename').text
|
||||
img_id = get_imageId_from_fileName(file_name)
|
||||
image_path = os.path.join(image_dir, file_name)
|
||||
print(image_path)
|
||||
if not os.path.isfile(image_path):
|
||||
print(f'Cannot find image {file_name} according to annotations.')
|
||||
continue
|
||||
|
||||
labels = []
|
||||
for obj in root_node.iter('object'):
|
||||
cls_name = obj.find('name').text
|
||||
if cls_name not in cls_map:
|
||||
print(f'Label "{cls_name}" not in "{config.coco_classes}"')
|
||||
continue
|
||||
bnd_box = obj.find('bndbox')
|
||||
x_min = int(bnd_box.find('xmin').text) - 1
|
||||
y_min = int(bnd_box.find('ymin').text) - 1
|
||||
x_max = int(bnd_box.find('xmax').text) - 1
|
||||
y_max = int(bnd_box.find('ymax').text) - 1
|
||||
labels.append([y_min, x_min, y_max, x_max, cls_map[cls_name]])
|
||||
|
||||
if not is_training:
|
||||
o_width = abs(x_max - x_min)
|
||||
o_height = abs(y_max - y_min)
|
||||
ann = {'area': o_width * o_height, 'iscrowd': 0, 'image_id':
|
||||
img_id, 'bbox': [x_min, y_min, o_width, o_height],
|
||||
'category_id': cls_map[cls_name], 'id': bnd_id,
|
||||
'ignore': 0,
|
||||
'segmentation': []}
|
||||
json_dict['annotations'].append(ann)
|
||||
bnd_id = bnd_id + 1
|
||||
|
||||
if labels:
|
||||
images.append(img_id)
|
||||
image_files_dict[img_id] = image_path
|
||||
image_anno_dict[img_id] = np.array(labels)
|
||||
|
||||
if not is_training:
|
||||
size = root_node.find("size")
|
||||
width = int(size.find('width').text)
|
||||
height = int(size.find('height').text)
|
||||
image = {'file_name': file_name, 'height': height, 'width': width,
|
||||
'id': img_id}
|
||||
json_dict['images'].append(image)
|
||||
|
||||
if not is_training:
|
||||
for cls_name, cid in cls_map.items():
|
||||
cat = {'supercategory': 'none', 'id': cid, 'name': cls_name}
|
||||
json_dict['categories'].append(cat)
|
||||
json_fp = open(json_file, 'w')
|
||||
json_str = json.dumps(json_dict)
|
||||
json_fp.write(json_str)
|
||||
json_fp.close()
|
||||
|
||||
return images, image_files_dict, image_anno_dict
|
||||
|
||||
|
||||
def create_coco_label(is_training):
|
||||
"""Get image path and annotation from COCO."""
|
||||
from pycocotools.coco import COCO
|
||||
|
||||
coco_root = config.coco_root
|
||||
data_type = config.val_data_type
|
||||
if is_training:
|
||||
data_type = config.train_data_type
|
||||
|
||||
# Classes need to train or test.
|
||||
train_cls = config.coco_classes
|
||||
train_cls_dict = {}
|
||||
for i, cls in enumerate(train_cls):
|
||||
train_cls_dict[cls] = i
|
||||
|
||||
anno_json = os.path.join(coco_root, config.instances_set.format(data_type))
|
||||
|
||||
coco = COCO(anno_json)
|
||||
classs_dict = {}
|
||||
cat_ids = coco.loadCats(coco.getCatIds())
|
||||
for cat in cat_ids:
|
||||
classs_dict[cat["id"]] = cat["name"]
|
||||
|
||||
image_ids = coco.getImgIds()
|
||||
images = []
|
||||
image_path_dict = {}
|
||||
image_anno_dict = {}
|
||||
|
||||
for img_id in image_ids:
|
||||
image_info = coco.loadImgs(img_id)
|
||||
file_name = image_info[0]["file_name"]
|
||||
anno_ids = coco.getAnnIds(imgIds=img_id, iscrowd=None)
|
||||
anno = coco.loadAnns(anno_ids)
|
||||
image_path = os.path.join(coco_root, data_type, file_name)
|
||||
annos = []
|
||||
iscrowd = False
|
||||
for label in anno:
|
||||
bbox = label["bbox"]
|
||||
class_name = classs_dict[label["category_id"]]
|
||||
iscrowd = iscrowd or label["iscrowd"]
|
||||
if class_name in train_cls:
|
||||
x_min, x_max = bbox[0], bbox[0] + bbox[2]
|
||||
y_min, y_max = bbox[1], bbox[1] + bbox[3]
|
||||
annos.append(
|
||||
list(map(round, [y_min, x_min, y_max, x_max])) + [train_cls_dict[class_name]])
|
||||
|
||||
if not is_training and iscrowd:
|
||||
continue
|
||||
if len(annos) >= 1:
|
||||
images.append(img_id)
|
||||
image_path_dict[img_id] = image_path
|
||||
image_anno_dict[img_id] = np.array(annos)
|
||||
|
||||
return images, image_path_dict, image_anno_dict
|
||||
|
||||
|
||||
def anno_parser(annos_str):
|
||||
"""Parse annotation from string to list."""
|
||||
annos = []
|
||||
for anno_str in annos_str:
|
||||
anno = list(map(int, anno_str.strip().split(',')))
|
||||
annos.append(anno)
|
||||
return annos
|
||||
|
||||
|
||||
def filter_valid_data(image_dir, anno_path):
|
||||
"""Filter valid image file, which both in image_dir and anno_path."""
|
||||
images = []
|
||||
image_path_dict = {}
|
||||
image_anno_dict = {}
|
||||
if not os.path.isdir(image_dir):
|
||||
raise RuntimeError("Path given is not valid.")
|
||||
if not os.path.isfile(anno_path):
|
||||
raise RuntimeError("Annotation file is not valid.")
|
||||
|
||||
with open(anno_path, "rb") as f:
|
||||
lines = f.readlines()
|
||||
for img_id, line in enumerate(lines):
|
||||
line_str = line.decode("utf-8").strip()
|
||||
line_split = str(line_str).split(' ')
|
||||
file_name = line_split[0]
|
||||
image_path = os.path.join(image_dir, file_name)
|
||||
if os.path.isfile(image_path):
|
||||
images.append(img_id)
|
||||
image_path_dict[img_id] = image_path
|
||||
image_anno_dict[img_id] = anno_parser(line_split[1:])
|
||||
|
||||
return images, image_path_dict, image_anno_dict
|
||||
|
||||
|
||||
def voc_data_to_mindrecord(mindrecord_dir, is_training, prefix="ssd.mindrecord", file_num=8):
|
||||
"""Create MindRecord file by image_dir and anno_path."""
|
||||
mindrecord_path = os.path.join(mindrecord_dir, prefix)
|
||||
writer = FileWriter(mindrecord_path, file_num)
|
||||
images, image_path_dict, image_anno_dict = create_voc_label(is_training)
|
||||
|
||||
ssd_json = {
|
||||
"img_id": {"type": "int32", "shape": [1]},
|
||||
"image": {"type": "bytes"},
|
||||
"annotation": {"type": "int32", "shape": [-1, 5]},
|
||||
}
|
||||
writer.add_schema(ssd_json, "ssd_json")
|
||||
|
||||
for img_id in images:
|
||||
image_path = image_path_dict[img_id]
|
||||
with open(image_path, 'rb') as f:
|
||||
img = f.read()
|
||||
annos = np.array(image_anno_dict[img_id], dtype=np.int32)
|
||||
img_id = np.array([img_id], dtype=np.int32)
|
||||
row = {"img_id": img_id, "image": img, "annotation": annos}
|
||||
writer.write_raw_data([row])
|
||||
writer.commit()
|
||||
|
||||
|
||||
def data_to_mindrecord_byte_image(dataset="coco", is_training=True, prefix="ssd.mindrecord", file_num=8):
|
||||
"""Create MindRecord file."""
|
||||
mindrecord_dir = config.mindrecord_dir
|
||||
mindrecord_path = os.path.join(mindrecord_dir, prefix)
|
||||
writer = FileWriter(mindrecord_path, file_num)
|
||||
if dataset == "coco":
|
||||
images, image_path_dict, image_anno_dict = create_coco_label(
|
||||
is_training)
|
||||
else:
|
||||
images, image_path_dict, image_anno_dict = filter_valid_data(
|
||||
config.image_dir, config.anno_path)
|
||||
|
||||
ssd_json = {
|
||||
"img_id": {"type": "int32", "shape": [1]},
|
||||
"image": {"type": "bytes"},
|
||||
"annotation": {"type": "int32", "shape": [-1, 5]},
|
||||
}
|
||||
writer.add_schema(ssd_json, "ssd_json")
|
||||
|
||||
for img_id in images:
|
||||
image_path = image_path_dict[img_id]
|
||||
with open(image_path, 'rb') as f:
|
||||
img = f.read()
|
||||
annos = np.array(image_anno_dict[img_id], dtype=np.int32)
|
||||
img_id = np.array([img_id], dtype=np.int32)
|
||||
row = {"img_id": img_id, "image": img, "annotation": annos}
|
||||
writer.write_raw_data([row])
|
||||
writer.commit()
|
||||
|
||||
|
||||
def create_ssd_dataset(mindrecord_file, batch_size=32, repeat_num=10, device_num=1, rank=0,
|
||||
is_training=True, num_parallel_workers=4):
|
||||
"""Creatr SSD dataset with MindDataset."""
|
||||
ds = de.MindDataset(mindrecord_file, columns_list=["img_id", "image", "annotation"], num_shards=device_num,
|
||||
shard_id=rank, num_parallel_workers=num_parallel_workers, shuffle=is_training)
|
||||
decode = C.Decode()
|
||||
ds = ds.map(input_columns=["image"], operations=decode)
|
||||
change_swap_op = C.HWC2CHW()
|
||||
normalize_op = C.Normalize(
|
||||
mean=[0.485*255, 0.456*255, 0.406*255], std=[0.229*255, 0.224*255, 0.225*255])
|
||||
color_adjust_op = C.RandomColorAdjust(
|
||||
brightness=0.4, contrast=0.4, saturation=0.4)
|
||||
compose_map_func = (lambda img_id, image, annotation: preprocess_fn(
|
||||
img_id, image, annotation, is_training))
|
||||
if is_training:
|
||||
output_columns = ["image", "box", "label", "num_match"]
|
||||
trans = [color_adjust_op, normalize_op, change_swap_op]
|
||||
else:
|
||||
output_columns = ["img_id", "image", "image_shape"]
|
||||
trans = [normalize_op, change_swap_op]
|
||||
ds = ds.map(input_columns=["img_id", "image", "annotation"],
|
||||
output_columns=output_columns, columns_order=output_columns,
|
||||
operations=compose_map_func, python_multiprocessing=is_training,
|
||||
num_parallel_workers=num_parallel_workers)
|
||||
ds = ds.map(input_columns=["image"], operations=trans, python_multiprocessing=is_training,
|
||||
num_parallel_workers=num_parallel_workers)
|
||||
ds = ds.batch(batch_size, drop_remainder=True)
|
||||
ds = ds.repeat(repeat_num)
|
||||
return ds
|
||||
|
|
@ -0,0 +1,53 @@
|
|||
# Copyright 2020 Huawei Technologies Co., Ltd
|
||||
#
|
||||
# Licensed under the Apache License, Version 2.0 (the "License");
|
||||
# you may not use this file except in compliance with the License.
|
||||
# You may obtain a copy of the License at
|
||||
#
|
||||
# http://www.apache.org/licenses/LICENSE-2.0
|
||||
#
|
||||
# Unless required by applicable law or agreed to in writing, software
|
||||
# distributed under the License is distributed on an "AS IS" BASIS,
|
||||
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
# See the License for the specific language governing permissions and
|
||||
# limitations under the License.
|
||||
# ============================================================================
|
||||
"""Parameters utils"""
|
||||
|
||||
import numpy as np
|
||||
from mindspore.common.initializer import initializer, TruncatedNormal
|
||||
|
||||
|
||||
def init_net_param(network, initialize_mode='TruncatedNormal'):
|
||||
"""Init the parameters in net."""
|
||||
params = network.trainable_params()
|
||||
for p in params:
|
||||
if 'beta' not in p.name and 'gamma' not in p.name and 'bias' not in p.name:
|
||||
np.random.seed(seed=1)
|
||||
if initialize_mode == 'TruncatedNormal':
|
||||
p.set_parameter_data(initializer(
|
||||
TruncatedNormal(), p.data.shape, p.data.dtype))
|
||||
else:
|
||||
p.set_parameter_data(
|
||||
initialize_mode, p.data.shape, p.data.dtype)
|
||||
|
||||
|
||||
def load_backbone_params(network, param_dict):
|
||||
"""Init the parameters from pre-train model, default is mobilenetv2."""
|
||||
for _, param in net.parameters_and_names():
|
||||
param_name = param.name.replace('network.backbone.', '')
|
||||
name_split = param_name.split('.')
|
||||
if 'features_1' in param_name:
|
||||
param_name = param_name.replace('features_1', 'features')
|
||||
if 'features_2' in param_name:
|
||||
param_name = '.'.join(
|
||||
['features', str(int(name_split[1]) + 14)] + name_split[2:])
|
||||
if param_name in param_dict:
|
||||
param.set_parameter_data(param_dict[param_name].data)
|
||||
|
||||
|
||||
def filter_checkpoint_parameter(param_dict):
|
||||
"""remove useless parameters"""
|
||||
for key in list(param_dict.keys()):
|
||||
if 'multi_loc_layers' in key or 'multi_cls_layers' in key:
|
||||
del param_dict[key]
|
||||
|
|
@ -0,0 +1,57 @@
|
|||
# Copyright 2020 Huawei Technologies Co., Ltd
|
||||
#
|
||||
# Licensed under the Apache License, Version 2.0 (the "License");
|
||||
# you may not use this file except in compliance with the License.
|
||||
# You may obtain a copy of the License at
|
||||
#
|
||||
# http://www.apache.org/licenses/LICENSE-2.0
|
||||
#
|
||||
# Unless required by applicable law or agreed to in writing, software
|
||||
# distributed under the License is distributed on an "AS IS" BASIS,
|
||||
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
# See the License for the specific language governing permissions and
|
||||
# limitations under the License.
|
||||
# ============================================================================
|
||||
|
||||
"""Learning rate schedule"""
|
||||
|
||||
import math
|
||||
import numpy as np
|
||||
|
||||
|
||||
def get_lr(global_step, lr_init, lr_end, lr_max, warmup_epochs, total_epochs, steps_per_epoch):
|
||||
"""
|
||||
generate learning rate array
|
||||
|
||||
Args:
|
||||
global_step(int): total steps of the training
|
||||
lr_init(float): init learning rate
|
||||
lr_end(float): end learning rate
|
||||
lr_max(float): max learning rate
|
||||
warmup_epochs(float): number of warmup epochs
|
||||
total_epochs(int): total epoch of training
|
||||
steps_per_epoch(int): steps of one epoch
|
||||
|
||||
Returns:
|
||||
np.array, learning rate array
|
||||
"""
|
||||
lr_each_step = []
|
||||
total_steps = steps_per_epoch * total_epochs
|
||||
warmup_steps = steps_per_epoch * warmup_epochs
|
||||
for i in range(total_steps):
|
||||
if i < warmup_steps:
|
||||
lr = lr_init + (lr_max - lr_init) * i / warmup_steps
|
||||
else:
|
||||
lr = lr_end + \
|
||||
(lr_max - lr_end) * \
|
||||
(1. + math.cos(math.pi * (i - warmup_steps) /
|
||||
(total_steps - warmup_steps))) / 2.
|
||||
if lr < 0.0:
|
||||
lr = 0.0
|
||||
lr_each_step.append(lr)
|
||||
|
||||
current_step = global_step
|
||||
lr_each_step = np.array(lr_each_step).astype(np.float32)
|
||||
learning_rate = lr_each_step[current_step:]
|
||||
|
||||
return learning_rate
|
||||
|
|
@ -0,0 +1,785 @@
|
|||
# Copyright 2020 Huawei Technologies Co., Ltd
|
||||
#
|
||||
# Licensed under the Apache License, Version 2.0 (the "License");
|
||||
# you may not use this file except in compliance with the License.
|
||||
# You may obtain a copy of the License at
|
||||
#
|
||||
# http://www.apache.org/licenses/LICENSE-2.0
|
||||
#
|
||||
# Unless required by applicable law or agreed to in writing, software
|
||||
# distributed under the License is distributed on an "AS IS" BASIS,
|
||||
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
# See the License for the specific language governing permissions and
|
||||
# limitations under the License.
|
||||
# ============================================================================
|
||||
|
||||
"""SSD net based MobilenetV2."""
|
||||
import math
|
||||
import numpy as np
|
||||
|
||||
import mindspore.common.dtype as mstype
|
||||
import mindspore as ms
|
||||
import mindspore.nn as nn
|
||||
from mindspore import Parameter, context, Tensor
|
||||
from mindspore.parallel._auto_parallel_context import auto_parallel_context
|
||||
from mindspore.communication.management import get_group_size
|
||||
from mindspore.ops import operations as P
|
||||
from mindspore.ops import functional as F
|
||||
from mindspore.ops import composite as C
|
||||
from mindspore.common.initializer import initializer
|
||||
|
||||
|
||||
def _make_divisible(x, divisor=4):
|
||||
return int(np.ceil(x * 1. / divisor) * divisor)
|
||||
|
||||
|
||||
def _conv2d(in_channel, out_channel, kernel_size=3, stride=1, pad_mod='same'):
|
||||
return nn.Conv2d(in_channel, out_channel, kernel_size=kernel_size, stride=stride,
|
||||
padding=0, pad_mode=pad_mod, has_bias=True)
|
||||
|
||||
|
||||
def _bn(channel):
|
||||
return nn.BatchNorm2d(channel, eps=1e-3, momentum=0.97,
|
||||
gamma_init=1, beta_init=0, moving_mean_init=0, moving_var_init=1)
|
||||
|
||||
|
||||
def _last_conv2d(in_channel, out_channel, kernel_size=3, stride=1, pad_mod='same', pad=0):
|
||||
depthwise_conv = DepthwiseConv(
|
||||
in_channel, kernel_size, stride, pad_mode='same', pad=pad)
|
||||
conv = _conv2d(in_channel, out_channel, kernel_size=1)
|
||||
return nn.SequentialCell([depthwise_conv, _bn(in_channel), nn.ReLU6(), conv])
|
||||
|
||||
|
||||
class ConvBNReLU(nn.Cell):
|
||||
"""
|
||||
Convolution/Depthwise fused with Batchnorm and ReLU block definition.
|
||||
|
||||
Args:
|
||||
in_planes (int): Input channel.
|
||||
out_planes (int): Output channel.
|
||||
kernel_size (int): Input kernel size.
|
||||
stride (int): Stride size for the first convolutional layer. Default: 1.
|
||||
groups (int): channel group. Convolution is 1 while Depthiwse is input channel. Default: 1.
|
||||
|
||||
Returns:
|
||||
Tensor, output tensor.
|
||||
|
||||
Examples:
|
||||
>>> ConvBNReLU(16, 256, kernel_size=1, stride=1, groups=1)
|
||||
"""
|
||||
|
||||
def __init__(self, in_planes, out_planes, kernel_size=3, stride=1, groups=1, use_act=True, act_type='relu'):
|
||||
super(ConvBNReLU, self).__init__()
|
||||
padding = 0
|
||||
if groups == 1:
|
||||
conv = nn.Conv2d(in_planes, out_planes, kernel_size, stride, pad_mode='same',
|
||||
padding=padding)
|
||||
else:
|
||||
conv = DepthwiseConv(in_planes, kernel_size,
|
||||
stride, pad_mode='same', pad=padding)
|
||||
layers = [conv, _bn(out_planes)]
|
||||
if use_act:
|
||||
layers.append(Activation(act_type))
|
||||
self.features = nn.SequentialCell(layers)
|
||||
|
||||
def construct(self, x):
|
||||
output = self.features(x)
|
||||
# print(output.shape)
|
||||
return output
|
||||
|
||||
|
||||
class DepthwiseConv(nn.Cell):
|
||||
"""
|
||||
Depthwise Convolution warpper definition.
|
||||
|
||||
Args:
|
||||
in_planes (int): Input channel.
|
||||
kernel_size (int): Input kernel size.
|
||||
stride (int): Stride size.
|
||||
pad_mode (str): pad mode in (pad, same, valid)
|
||||
channel_multiplier (int): Output channel multiplier
|
||||
has_bias (bool): has bias or not
|
||||
|
||||
Returns:
|
||||
Tensor, output tensor.
|
||||
|
||||
Examples:
|
||||
>>> DepthwiseConv(16, 3, 1, 'pad', 1, channel_multiplier=1)
|
||||
"""
|
||||
|
||||
def __init__(self, in_planes, kernel_size, stride, pad_mode, pad, channel_multiplier=1, has_bias=False):
|
||||
super(DepthwiseConv, self).__init__()
|
||||
self.has_bias = has_bias
|
||||
self.in_channels = in_planes
|
||||
self.channel_multiplier = channel_multiplier
|
||||
self.out_channels = in_planes * channel_multiplier
|
||||
self.kernel_size = (kernel_size, kernel_size)
|
||||
self.depthwise_conv = P.DepthwiseConv2dNative(channel_multiplier=channel_multiplier,
|
||||
kernel_size=self.kernel_size,
|
||||
stride=stride, pad_mode=pad_mode, pad=pad)
|
||||
self.bias_add = P.BiasAdd()
|
||||
weight_shape = [channel_multiplier, in_planes, *self.kernel_size]
|
||||
self.weight = Parameter(initializer(
|
||||
'ones', weight_shape), name='weight')
|
||||
|
||||
if has_bias:
|
||||
bias_shape = [channel_multiplier * in_planes]
|
||||
self.bias = Parameter(initializer(
|
||||
'zeros', bias_shape), name='bias')
|
||||
else:
|
||||
self.bias = None
|
||||
|
||||
def construct(self, x):
|
||||
output = self.depthwise_conv(x, self.weight)
|
||||
if self.has_bias:
|
||||
output = self.bias_add(output, self.bias)
|
||||
return output
|
||||
|
||||
|
||||
class MyHSigmoid(nn.Cell):
|
||||
def __init__(self):
|
||||
super(MyHSigmoid, self).__init__()
|
||||
self.relu6 = nn.ReLU6()
|
||||
|
||||
def construct(self, x):
|
||||
return self.relu6(x + 3.) / 6.
|
||||
|
||||
|
||||
class Activation(nn.Cell):
|
||||
"""
|
||||
Activation definition.
|
||||
|
||||
Args:
|
||||
act_func(string): activation name.
|
||||
|
||||
Returns:
|
||||
Tensor, output tensor.
|
||||
"""
|
||||
|
||||
def __init__(self, act_func):
|
||||
super(Activation, self).__init__()
|
||||
if act_func == 'relu':
|
||||
self.act = nn.ReLU()
|
||||
elif act_func == 'relu6':
|
||||
self.act = nn.ReLU6()
|
||||
elif act_func in ('hsigmoid', 'hard_sigmoid'):
|
||||
self.act = MyHSigmoid() # nn.HSigmoid()
|
||||
elif act_func in ('hswish', 'hard_swish'):
|
||||
self.act = nn.HSwish()
|
||||
else:
|
||||
raise NotImplementedError
|
||||
|
||||
def construct(self, x):
|
||||
return self.act(x)
|
||||
|
||||
|
||||
class GlobalAvgPooling(nn.Cell):
|
||||
"""
|
||||
Global avg pooling definition.
|
||||
|
||||
Args:
|
||||
|
||||
Returns:
|
||||
Tensor, output tensor.
|
||||
|
||||
Examples:
|
||||
>>> GlobalAvgPooling()
|
||||
"""
|
||||
|
||||
def __init__(self, keep_dims=False):
|
||||
super(GlobalAvgPooling, self).__init__()
|
||||
self.mean = P.ReduceMean(keep_dims=keep_dims)
|
||||
|
||||
def construct(self, x):
|
||||
x = self.mean(x, (2, 3))
|
||||
return x
|
||||
|
||||
|
||||
class SE(nn.Cell):
|
||||
"""
|
||||
SE warpper definition.
|
||||
|
||||
Args:
|
||||
num_out (int): Output channel.
|
||||
ratio (int): middle output ratio.
|
||||
|
||||
Returns:
|
||||
Tensor, output tensor.
|
||||
|
||||
Examples:
|
||||
>>> SE(4)
|
||||
"""
|
||||
|
||||
def __init__(self, num_out, ratio=4):
|
||||
super(SE, self).__init__()
|
||||
num_mid = _make_divisible(num_out // ratio)
|
||||
self.pool = GlobalAvgPooling(keep_dims=True)
|
||||
self.conv_reduce = nn.Conv2d(in_channels=num_out, out_channels=num_mid,
|
||||
kernel_size=1, has_bias=True, pad_mode='pad')
|
||||
self.act1 = Activation('relu')
|
||||
self.conv_expand = nn.Conv2d(in_channels=num_mid, out_channels=num_out,
|
||||
kernel_size=1, has_bias=True, pad_mode='pad')
|
||||
self.act2 = Activation('hsigmoid')
|
||||
self.mul = P.Mul()
|
||||
|
||||
def construct(self, x):
|
||||
out = self.pool(x)
|
||||
out = self.conv_reduce(out)
|
||||
out = self.act1(out)
|
||||
out = self.conv_expand(out)
|
||||
out = self.act2(out)
|
||||
out = self.mul(x, out)
|
||||
return out
|
||||
|
||||
|
||||
class GhostModule(nn.Cell):
|
||||
"""
|
||||
GhostModule warpper definition.
|
||||
|
||||
Args:
|
||||
num_in (int): Input channel.
|
||||
num_out (int): Output channel.
|
||||
kernel_size (int): Input kernel size.
|
||||
stride (int): Stride size.
|
||||
padding (int): Padding number.
|
||||
use_act (bool): Used activation or not.
|
||||
act_type (string): Activation type.
|
||||
|
||||
Returns:
|
||||
Tensor, output tensor.
|
||||
|
||||
Examples:
|
||||
>>> GhostModule(3, 3)
|
||||
"""
|
||||
|
||||
def __init__(self, num_in, num_out, kernel_size=1, stride=1, padding=0, ratio=2, dw_size=3,
|
||||
use_act=True, act_type='relu'):
|
||||
super(GhostModule, self).__init__()
|
||||
init_channels = math.ceil(num_out / ratio)
|
||||
new_channels = init_channels * (ratio - 1)
|
||||
|
||||
self.primary_conv = ConvBNReLU(num_in, init_channels, kernel_size=kernel_size, stride=stride,
|
||||
groups=1, use_act=use_act, act_type='relu')
|
||||
self.cheap_operation = ConvBNReLU(init_channels, new_channels, kernel_size=dw_size, stride=1,
|
||||
groups=init_channels, use_act=use_act, act_type='relu')
|
||||
self.concat = P.Concat(axis=1)
|
||||
|
||||
def construct(self, x):
|
||||
x1 = self.primary_conv(x)
|
||||
x2 = self.cheap_operation(x1)
|
||||
# print(x1.shape)
|
||||
# print(x2.shape)
|
||||
return self.concat((x1, x2))
|
||||
|
||||
|
||||
class GhostBottleneck(nn.Cell):
|
||||
"""
|
||||
GhostBottleneck warpper definition.
|
||||
|
||||
Args:
|
||||
num_in (int): Input channel.
|
||||
num_mid (int): Middle channel.
|
||||
num_out (int): Output channel.
|
||||
kernel_size (int): Input kernel size.
|
||||
stride (int): Stride size.
|
||||
act_type (str): Activation type.
|
||||
use_se (bool): Use SE warpper or not.
|
||||
|
||||
Returns:
|
||||
Tensor, output tensor.
|
||||
|
||||
Examples:
|
||||
>>> GhostBottleneck(16, 3, 1, 1)
|
||||
"""
|
||||
|
||||
def __init__(self, num_in, num_mid, num_out, kernel_size, stride=1, act_type='relu', use_se=False,
|
||||
use_res_connect=True, last_relu=False):
|
||||
super(GhostBottleneck, self).__init__()
|
||||
self.ghost1 = GhostModule(num_in, num_mid, kernel_size=1,
|
||||
stride=1, padding=0, act_type=act_type)
|
||||
|
||||
self.use_res_connect = use_res_connect
|
||||
self.last_relu = last_relu
|
||||
self.use_dw = stride > 1
|
||||
self.dw = None
|
||||
if self.use_dw:
|
||||
self.dw = ConvBNReLU(num_mid, num_mid, kernel_size=kernel_size, stride=stride,
|
||||
act_type=act_type, groups=num_mid, use_act=False)
|
||||
|
||||
self.use_se = use_se
|
||||
if use_se:
|
||||
self.se = SE(num_mid)
|
||||
|
||||
self.ghost2 = GhostModule(num_mid, num_out, kernel_size=1, stride=1,
|
||||
padding=0, act_type=act_type, use_act=False)
|
||||
self.relu = nn.ReLU()
|
||||
if self.use_res_connect:
|
||||
self.down_sample = False
|
||||
if num_in != num_out or stride != 1:
|
||||
self.down_sample = True
|
||||
self.shortcut = None
|
||||
if self.down_sample:
|
||||
self.shortcut = nn.SequentialCell([
|
||||
ConvBNReLU(num_in, num_in, kernel_size=kernel_size, stride=stride,
|
||||
groups=num_in, use_act=False),
|
||||
ConvBNReLU(num_in, num_out, kernel_size=1, stride=1,
|
||||
groups=1, use_act=False),
|
||||
])
|
||||
self.add = P.TensorAdd()
|
||||
|
||||
def construct(self, x):
|
||||
"""construct"""
|
||||
shortcut = x
|
||||
out = self.ghost1(x)
|
||||
if self.use_dw:
|
||||
out = self.dw(out)
|
||||
if self.use_se:
|
||||
out = self.se(out)
|
||||
out = self.ghost2(out)
|
||||
if self.use_res_connect:
|
||||
if self.down_sample:
|
||||
shortcut = self.shortcut(shortcut)
|
||||
out = self.add(shortcut, out)
|
||||
if self.last_relu:
|
||||
out = self.relu(out)
|
||||
# print(out.shape)
|
||||
return out
|
||||
|
||||
def _get_pad(self, kernel_size):
|
||||
"""set the padding number"""
|
||||
pad = 0
|
||||
if kernel_size == 1:
|
||||
pad = 0
|
||||
elif kernel_size == 3:
|
||||
pad = 1
|
||||
elif kernel_size == 5:
|
||||
pad = 2
|
||||
elif kernel_size == 7:
|
||||
pad = 3
|
||||
else:
|
||||
raise NotImplementedError
|
||||
return pad
|
||||
|
||||
|
||||
class InvertedResidual(nn.Cell):
|
||||
"""
|
||||
Residual block definition.
|
||||
|
||||
Args:
|
||||
inp (int): Input channel.
|
||||
oup (int): Output channel.
|
||||
stride (int): Stride size for the first convolutional layer. Default: 1.
|
||||
expand_ratio (int): expand ration of input channel
|
||||
|
||||
Returns:
|
||||
Tensor, output tensor.
|
||||
|
||||
Examples:
|
||||
>>> ResidualBlock(3, 256, 1, 1)
|
||||
"""
|
||||
|
||||
def __init__(self, inp, oup, stride, expand_ratio, last_relu=False):
|
||||
super(InvertedResidual, self).__init__()
|
||||
assert stride in [1, 2]
|
||||
|
||||
hidden_dim = int(round(inp * expand_ratio))
|
||||
self.use_res_connect = stride == 1 and inp == oup
|
||||
|
||||
layers = []
|
||||
if expand_ratio != 1:
|
||||
layers.append(ConvBNReLU(inp, hidden_dim, kernel_size=1))
|
||||
layers.extend([
|
||||
# dw
|
||||
ConvBNReLU(hidden_dim, hidden_dim,
|
||||
stride=stride, groups=hidden_dim),
|
||||
# pw-linear
|
||||
nn.Conv2d(hidden_dim, oup, kernel_size=1,
|
||||
stride=1, has_bias=False),
|
||||
_bn(oup),
|
||||
])
|
||||
self.conv = nn.SequentialCell(layers)
|
||||
self.add = P.TensorAdd()
|
||||
self.cast = P.Cast()
|
||||
self.last_relu = last_relu
|
||||
self.relu = nn.ReLU6()
|
||||
|
||||
def construct(self, x):
|
||||
identity = x
|
||||
x = self.conv(x)
|
||||
if self.use_res_connect:
|
||||
x = self.add(identity, x)
|
||||
if self.last_relu:
|
||||
x = self.relu(x)
|
||||
# print(x.shape)
|
||||
return x
|
||||
|
||||
|
||||
class FlattenConcat(nn.Cell):
|
||||
"""
|
||||
Concatenate predictions into a single tensor.
|
||||
|
||||
Args:
|
||||
config (dict): The default config of SSD.
|
||||
|
||||
Returns:
|
||||
Tensor, flatten predictions.
|
||||
"""
|
||||
|
||||
def __init__(self, config):
|
||||
super(FlattenConcat, self).__init__()
|
||||
self.num_ssd_boxes = config.num_ssd_boxes
|
||||
self.concat = P.Concat(axis=1)
|
||||
self.transpose = P.Transpose()
|
||||
|
||||
def construct(self, inputs):
|
||||
output = ()
|
||||
batch_size = F.shape(inputs[0])[0]
|
||||
for x in inputs:
|
||||
x = self.transpose(x, (0, 2, 3, 1))
|
||||
output += (F.reshape(x, (batch_size, -1)),)
|
||||
res = self.concat(output)
|
||||
return F.reshape(res, (batch_size, self.num_ssd_boxes, -1))
|
||||
|
||||
|
||||
class MultiBox(nn.Cell):
|
||||
"""
|
||||
Multibox conv layers. Each multibox layer contains class conf scores and localization predictions.
|
||||
|
||||
Args:
|
||||
config (dict): The default config of SSD.
|
||||
|
||||
Returns:
|
||||
Tensor, localization predictions.
|
||||
Tensor, class conf scores.
|
||||
"""
|
||||
|
||||
def __init__(self, config):
|
||||
super(MultiBox, self).__init__()
|
||||
num_classes = config.num_classes
|
||||
out_channels = config.extras_out_channels
|
||||
num_default = config.num_default
|
||||
|
||||
loc_layers = []
|
||||
cls_layers = []
|
||||
for k, out_channel in enumerate(out_channels):
|
||||
loc_layers += [_last_conv2d(out_channel, 4 * num_default[k],
|
||||
kernel_size=3, stride=1, pad_mod='same', pad=0)]
|
||||
cls_layers += [_last_conv2d(out_channel, num_classes * num_default[k],
|
||||
kernel_size=3, stride=1, pad_mod='same', pad=0)]
|
||||
|
||||
self.multi_loc_layers = nn.layer.CellList(loc_layers)
|
||||
self.multi_cls_layers = nn.layer.CellList(cls_layers)
|
||||
self.flatten_concat = FlattenConcat(config)
|
||||
|
||||
def construct(self, inputs):
|
||||
loc_outputs = ()
|
||||
cls_outputs = ()
|
||||
for i in range(len(self.multi_loc_layers)):
|
||||
loc_outputs += (self.multi_loc_layers[i](inputs[i]),)
|
||||
cls_outputs += (self.multi_cls_layers[i](inputs[i]),)
|
||||
return self.flatten_concat(loc_outputs), self.flatten_concat(cls_outputs)
|
||||
|
||||
|
||||
class SSD300(nn.Cell):
|
||||
"""
|
||||
SSD300 Network. Default backbone is resnet34.
|
||||
|
||||
Args:
|
||||
backbone (Cell): Backbone Network.
|
||||
config (dict): The default config of SSD.
|
||||
|
||||
Returns:
|
||||
Tensor, localization predictions.
|
||||
Tensor, class conf scores.
|
||||
|
||||
Examples:backbone
|
||||
SSD300(backbone=resnet34(num_classes=None),
|
||||
config=config).
|
||||
"""
|
||||
|
||||
def __init__(self, backbone, config, is_training=True, **kwargs):
|
||||
super(SSD300, self).__init__()
|
||||
|
||||
self.backbone = backbone
|
||||
in_channels = config.extras_in_channels
|
||||
out_channels = config.extras_out_channels
|
||||
ratios = config.extras_ratio
|
||||
strides = config.extras_srides
|
||||
residual_list = []
|
||||
for i in range(2, len(in_channels)):
|
||||
residual = InvertedResidual(in_channels[i], out_channels[i], stride=strides[i],
|
||||
expand_ratio=ratios[i], last_relu=True)
|
||||
residual_list.append(residual)
|
||||
self.multi_residual = nn.layer.CellList(residual_list)
|
||||
self.multi_box = MultiBox(config)
|
||||
self.is_training = is_training
|
||||
if not is_training:
|
||||
self.activation = P.Sigmoid()
|
||||
|
||||
def construct(self, x):
|
||||
r"""construct of SSD300"""
|
||||
layer_out_11, output = self.backbone(x)
|
||||
multi_feature = (layer_out_11, output)
|
||||
feature = output
|
||||
for residual in self.multi_residual:
|
||||
feature = residual(feature)
|
||||
multi_feature += (feature,)
|
||||
pred_loc, pred_label = self.multi_box(multi_feature)
|
||||
if not self.is_training:
|
||||
pred_label = self.activation(pred_label)
|
||||
return pred_loc, pred_label
|
||||
|
||||
|
||||
class SigmoidFocalClassificationLoss(nn.Cell):
|
||||
""""
|
||||
Sigmoid focal-loss for classification.
|
||||
|
||||
Args:
|
||||
gamma (float): Hyper-parameter to balance the easy and hard examples. Default: 2.0
|
||||
alpha (float): Hyper-parameter to balance the positive and negative example. Default: 0.25
|
||||
|
||||
Returns:
|
||||
Tensor, the focal loss.
|
||||
"""
|
||||
|
||||
def __init__(self, gamma=2.0, alpha=0.25):
|
||||
super(SigmoidFocalClassificationLoss, self).__init__()
|
||||
self.sigmiod_cross_entropy = P.SigmoidCrossEntropyWithLogits()
|
||||
self.sigmoid = P.Sigmoid()
|
||||
self.pow = P.Pow()
|
||||
self.onehot = P.OneHot()
|
||||
self.on_value = Tensor(1.0, mstype.float32)
|
||||
self.off_value = Tensor(0.0, mstype.float32)
|
||||
self.gamma = gamma
|
||||
self.alpha = alpha
|
||||
|
||||
def construct(self, logits, label):
|
||||
r"""construct of SigmoidFocalClassificationLoss"""
|
||||
label = self.onehot(label, F.shape(
|
||||
logits)[-1], self.on_value, self.off_value)
|
||||
sigmiod_cross_entropy = self.sigmiod_cross_entropy(logits, label)
|
||||
sigmoid = self.sigmoid(logits)
|
||||
label = F.cast(label, mstype.float32)
|
||||
p_t = label * sigmoid + (1 - label) * (1 - sigmoid)
|
||||
modulating_factor = self.pow(1 - p_t, self.gamma)
|
||||
alpha_weight_factor = label * self.alpha + \
|
||||
(1 - label) * (1 - self.alpha)
|
||||
focal_loss = modulating_factor * alpha_weight_factor * sigmiod_cross_entropy
|
||||
return focal_loss
|
||||
|
||||
|
||||
class SSDWithLossCell(nn.Cell):
|
||||
""""
|
||||
Provide SSD training loss through network.
|
||||
|
||||
Args:
|
||||
network (Cell): The training network.
|
||||
config (dict): SSD config.
|
||||
|
||||
Returns:
|
||||
Tensor, the loss of the network.
|
||||
"""
|
||||
|
||||
def __init__(self, network, config):
|
||||
super(SSDWithLossCell, self).__init__()
|
||||
self.network = network
|
||||
self.less = P.Less()
|
||||
self.tile = P.Tile()
|
||||
self.reduce_sum = P.ReduceSum()
|
||||
self.reduce_mean = P.ReduceMean()
|
||||
self.expand_dims = P.ExpandDims()
|
||||
self.class_loss = SigmoidFocalClassificationLoss(
|
||||
config.gamma, config.alpha)
|
||||
self.loc_loss = nn.SmoothL1Loss()
|
||||
|
||||
def construct(self, x, gt_loc, gt_label, num_matched_boxes):
|
||||
r"""construct of SSDWithLossCell"""
|
||||
pred_loc, pred_label = self.network(x)
|
||||
mask = F.cast(self.less(0, gt_label), mstype.float32)
|
||||
num_matched_boxes = self.reduce_sum(
|
||||
F.cast(num_matched_boxes, mstype.float32))
|
||||
|
||||
# Localization Loss
|
||||
mask_loc = self.tile(self.expand_dims(mask, -1), (1, 1, 4))
|
||||
smooth_l1 = self.loc_loss(pred_loc, gt_loc) * mask_loc
|
||||
loss_loc = self.reduce_sum(self.reduce_mean(smooth_l1, -1), -1)
|
||||
|
||||
# Classification Loss
|
||||
loss_cls = self.class_loss(pred_label, gt_label)
|
||||
loss_cls = self.reduce_sum(loss_cls, (1, 2))
|
||||
|
||||
return self.reduce_sum((loss_cls + loss_loc) / num_matched_boxes)
|
||||
|
||||
|
||||
class TrainingWrapper(nn.Cell):
|
||||
"""
|
||||
Encapsulation class of SSD network training.
|
||||
|
||||
Append an optimizer to the training network after that the construct
|
||||
function can be called to create the backward graph.
|
||||
|
||||
Args:
|
||||
network (Cell): The training network. Note that loss function should have been added.
|
||||
optimizer (Optimizer): Optimizer for updating the weights.
|
||||
sens (Number): The adjust parameter. Default: 1.0.
|
||||
"""
|
||||
|
||||
def __init__(self, network, optimizer, sens=1.0):
|
||||
super(TrainingWrapper, self).__init__(auto_prefix=False)
|
||||
self.network = network
|
||||
self.weights = ms.ParameterTuple(network.trainable_params())
|
||||
self.optimizer = optimizer
|
||||
self.grad = C.GradOperation('grad', get_by_list=True, sens_param=True)
|
||||
self.sens = sens
|
||||
self.reducer_flag = False
|
||||
self.grad_reducer = None
|
||||
self.parallel_mode = context.get_auto_parallel_context("parallel_mode")
|
||||
if self.parallel_mode in [ms.ParallelMode.DATA_PARALLEL, ms.ParallelMode.HYBRID_PARALLEL]:
|
||||
self.reducer_flag = True
|
||||
if self.reducer_flag:
|
||||
mean = context.get_auto_parallel_context("mirror_mean")
|
||||
if auto_parallel_context().get_device_num_is_set():
|
||||
degree = context.get_auto_parallel_context("device_num")
|
||||
else:
|
||||
degree = get_group_size()
|
||||
self.grad_reducer = nn.DistributedGradReducer(
|
||||
optimizer.parameters, mean, degree)
|
||||
|
||||
def construct(self, *args):
|
||||
weights = self.weights
|
||||
loss = self.network(*args)
|
||||
sens = P.Fill()(P.DType()(loss), P.Shape()(loss), self.sens)
|
||||
grads = self.grad(self.network, weights)(*args, sens)
|
||||
if self.reducer_flag:
|
||||
# apply grad reducer on grads
|
||||
grads = self.grad_reducer(grads)
|
||||
return F.depend(loss, self.optimizer(grads))
|
||||
|
||||
|
||||
class SSDWithGhostNet(nn.Cell):
|
||||
"""
|
||||
Ghostnett architecture for SSD backbone.
|
||||
|
||||
Args:
|
||||
model_cfgs (list): model config
|
||||
multiplier (int): Channels multiplier for round to 8/16 and others. Default is 1.
|
||||
round_nearest (list): Channel round to. Default is 8
|
||||
Returns:
|
||||
Tensor, the 11th feature after ConvBNReLU in MobileNetV2.
|
||||
Tensor, the last feature in MobileNetV2.
|
||||
|
||||
Examples:
|
||||
>>> SSDWithGhostNet()
|
||||
"""
|
||||
|
||||
def __init__(self, model_cfgs, multiplier=1., round_nearest=8):
|
||||
super(SSDWithGhostNet, self).__init__()
|
||||
self.cfgs = model_cfgs['cfg']
|
||||
# self.inplanes = 16 ## for "1x"
|
||||
self.inplanes = 20 # for "1.3x"
|
||||
first_conv_in_channel = 3
|
||||
first_conv_out_channel = _make_divisible(multiplier * self.inplanes)
|
||||
|
||||
blocks = []
|
||||
blocks.append(ConvBNReLU(first_conv_in_channel, first_conv_out_channel,
|
||||
kernel_size=3, stride=2, groups=1, use_act=True, act_type='relu'))
|
||||
|
||||
layer_index = 0
|
||||
for layer_cfg in self.cfgs:
|
||||
# print(layer_cfg)
|
||||
if layer_index == 11:
|
||||
hidden_dim = int(round(self.inplanes * 6))
|
||||
self.expand_layer_conv_11 = ConvBNReLU(
|
||||
self.inplanes, hidden_dim, kernel_size=1)
|
||||
blocks.append(self._make_layer(kernel_size=layer_cfg[0],
|
||||
exp_ch=_make_divisible(
|
||||
multiplier * layer_cfg[1]),
|
||||
out_channel=_make_divisible(
|
||||
multiplier * layer_cfg[2]),
|
||||
use_se=layer_cfg[3],
|
||||
act_func=layer_cfg[4],
|
||||
stride=layer_cfg[5]))
|
||||
layer_index += 1
|
||||
output_channel = _make_divisible(
|
||||
multiplier * model_cfgs["cls_ch_squeeze"])
|
||||
blocks.append(ConvBNReLU(_make_divisible(multiplier * self.cfgs[-1][2]), output_channel,
|
||||
kernel_size=1, stride=1, groups=1, use_act=True))
|
||||
|
||||
self.features_1 = nn.SequentialCell(blocks[:12])
|
||||
self.features_2 = nn.SequentialCell(blocks[12:])
|
||||
|
||||
def _make_layer(self, kernel_size, exp_ch, out_channel, use_se, act_func, stride=1):
|
||||
mid_planes = exp_ch
|
||||
out_planes = out_channel
|
||||
# num_in, num_mid, num_out, kernel_size, stride=1, act_type='relu', use_se=False):
|
||||
layer = GhostBottleneck(self.inplanes, mid_planes, out_planes,
|
||||
kernel_size, stride=stride, act_type=act_func, use_se=use_se)
|
||||
self.inplanes = out_planes
|
||||
return layer
|
||||
|
||||
def construct(self, x):
|
||||
out = self.features_1(x)
|
||||
expand_layer_conv_11 = self.expand_layer_conv_11(out)
|
||||
out = self.features_2(out)
|
||||
return expand_layer_conv_11, out
|
||||
|
||||
|
||||
def ssd_ghostnet(**kwargs):
|
||||
"""
|
||||
Constructs a SSD GhostNet model
|
||||
"""
|
||||
model_cfgs = {
|
||||
"1x": {
|
||||
"cfg": [
|
||||
# k, exp, c, se, nl, s,
|
||||
# stage1
|
||||
[3, 16, 16, False, 'relu', 1],
|
||||
# stage2
|
||||
[3, 48, 24, False, 'relu', 2],
|
||||
[3, 72, 24, False, 'relu', 1],
|
||||
# stage3
|
||||
[5, 72, 40, True, 'relu', 2],
|
||||
[5, 120, 40, True, 'relu', 1],
|
||||
# stage4
|
||||
[3, 240, 80, False, 'relu', 2],
|
||||
[3, 200, 80, False, 'relu', 1],
|
||||
[3, 184, 80, False, 'relu', 1],
|
||||
[3, 184, 80, False, 'relu', 1],
|
||||
[3, 480, 112, True, 'relu', 1],
|
||||
[3, 672, 112, True, 'relu', 1],
|
||||
# stage5
|
||||
[5, 672, 160, True, 'relu', 2],
|
||||
[5, 960, 160, False, 'relu', 1],
|
||||
[5, 960, 160, True, 'relu', 1],
|
||||
[5, 960, 160, False, 'relu', 1],
|
||||
[5, 960, 160, True, 'relu', 1]],
|
||||
"cls_ch_squeeze": 960,
|
||||
},
|
||||
"1.3x": {
|
||||
"cfg": [
|
||||
# k, exp, c, se, nl, s
|
||||
# stage1
|
||||
[3, 24, 20, False, 'relu', 1],
|
||||
# stage2
|
||||
[3, 64, 32, False, 'relu', 2],
|
||||
[3, 96, 32, False, 'relu', 1],
|
||||
# stage3
|
||||
[5, 96, 52, True, 'relu', 2],
|
||||
[5, 160, 52, True, 'relu', 1],
|
||||
# stage4
|
||||
[3, 312, 104, False, 'relu', 2],
|
||||
[3, 264, 104, False, 'relu', 1],
|
||||
[3, 240, 104, False, 'relu', 1],
|
||||
[3, 240, 104, False, 'relu', 1],
|
||||
[3, 624, 144, True, 'relu', 1],
|
||||
[3, 864, 144, True, 'relu', 1],
|
||||
# stage5
|
||||
[5, 864, 208, True, 'relu', 2],
|
||||
[5, 1248, 208, False, 'relu', 1],
|
||||
[5, 1248, 208, True, 'relu', 1],
|
||||
[5, 1248, 208, False, 'relu', 1],
|
||||
[5, 1248, 208, True, 'relu', 1]],
|
||||
"cls_ch_squeeze": 1248,
|
||||
}
|
||||
}
|
||||
return SSDWithGhostNet(model_cfgs["1.3x"], **kwargs)
|
||||
|
|
@ -0,0 +1,169 @@
|
|||
# Copyright 2020 Huawei Technologies Co., Ltd
|
||||
#
|
||||
# Licensed under the Apache License, Version 2.0 (the "License");
|
||||
# you may not use this file except in compliance with the License.
|
||||
# You may obtain a copy of the License at
|
||||
#
|
||||
# http://www.apache.org/licenses/LICENSE-2.0
|
||||
#
|
||||
# less required by applicable law or agreed to in writing, software
|
||||
# distributed under the License is distributed on an "AS IS" BASIS,
|
||||
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
# See the License for the specific language governing permissions and
|
||||
# limitations under the License.
|
||||
# ============================================================================
|
||||
|
||||
"""Train SSD and get checkpoint files."""
|
||||
|
||||
import os
|
||||
import argparse
|
||||
import ast
|
||||
import mindspore.nn as nn
|
||||
from mindspore import context, Tensor
|
||||
from mindspore.communication.management import init
|
||||
from mindspore.train.callback import CheckpointConfig, ModelCheckpoint, LossMonitor, TimeMonitor
|
||||
from mindspore.train import Model, ParallelMode
|
||||
# from mindspore.context import ParallelMode
|
||||
from mindspore.train.serialization import load_checkpoint, load_param_into_net
|
||||
from src.ssd_ghostnet import SSD300, SSDWithLossCell, TrainingWrapper, ssd_ghostnet
|
||||
# from src.config_ghostnet_1x import config
|
||||
from src.config_ghostnet_13x import config
|
||||
from src.dataset import create_ssd_dataset, data_to_mindrecord_byte_image, voc_data_to_mindrecord
|
||||
from src.lr_schedule import get_lr
|
||||
from src.init_params import init_net_param, filter_checkpoint_parameter
|
||||
|
||||
|
||||
def main():
|
||||
parser = argparse.ArgumentParser(description="SSD training")
|
||||
parser.add_argument("--only_create_dataset", type=ast.literal_eval, default=False,
|
||||
help="If set it true, only create Mindrecord, default is False.")
|
||||
parser.add_argument("--distribute", type=ast.literal_eval, default=False,
|
||||
help="Run distribute, default is False.")
|
||||
parser.add_argument("--device_id", type=int, default=4,
|
||||
help="Device id, default is 0.")
|
||||
parser.add_argument("--device_num", type=int, default=1,
|
||||
help="Use device nums, default is 1.")
|
||||
parser.add_argument("--lr", type=float, default=0.05,
|
||||
help="Learning rate, default is 0.05.")
|
||||
parser.add_argument("--mode", type=str, default="sink",
|
||||
help="Run sink mode or not, default is sink.")
|
||||
parser.add_argument("--dataset", type=str, default="coco",
|
||||
help="Dataset, defalut is coco.")
|
||||
parser.add_argument("--epoch_size", type=int, default=500,
|
||||
help="Epoch size, default is 500.")
|
||||
parser.add_argument("--batch_size", type=int, default=32,
|
||||
help="Batch size, default is 32.")
|
||||
parser.add_argument("--pre_trained", type=str, default=None,
|
||||
help="Pretrained Checkpoint file path.")
|
||||
parser.add_argument("--pre_trained_epoch_size", type=int,
|
||||
default=0, help="Pretrained epoch size.")
|
||||
parser.add_argument("--save_checkpoint_epochs", type=int,
|
||||
default=10, help="Save checkpoint epochs, default is 10.")
|
||||
parser.add_argument("--loss_scale", type=int, default=1024,
|
||||
help="Loss scale, default is 1024.")
|
||||
parser.add_argument("--filter_weight", type=ast.literal_eval, default=False,
|
||||
help="Filter weight parameters, default is False.")
|
||||
args_opt = parser.parse_args()
|
||||
|
||||
context.set_context(mode=context.GRAPH_MODE,
|
||||
device_target="Ascend", device_id=args_opt.device_id)
|
||||
|
||||
if args_opt.distribute:
|
||||
device_num = args_opt.device_num
|
||||
context.reset_auto_parallel_context()
|
||||
context.set_auto_parallel_context(parallel_mode=ParallelMode.DATA_PARALLEL, mirror_mean=True,
|
||||
device_num=device_num)
|
||||
init()
|
||||
rank = args_opt.device_id % device_num
|
||||
else:
|
||||
rank = 0
|
||||
device_num = 1
|
||||
|
||||
print("Start create dataset!")
|
||||
|
||||
# It will generate mindrecord file in args_opt.mindrecord_dir,
|
||||
# and the file name is ssd.mindrecord0, 1, ... file_num.
|
||||
|
||||
prefix = "ssd.mindrecord"
|
||||
mindrecord_dir = config.mindrecord_dir
|
||||
mindrecord_file = os.path.join(mindrecord_dir, prefix + "0")
|
||||
if not os.path.exists(mindrecord_file):
|
||||
if not os.path.isdir(mindrecord_dir):
|
||||
os.makedirs(mindrecord_dir)
|
||||
if args_opt.dataset == "coco":
|
||||
if os.path.isdir(config.coco_root):
|
||||
print("Create Mindrecord.")
|
||||
data_to_mindrecord_byte_image("coco", True, prefix)
|
||||
print("Create Mindrecord Done, at {}".format(mindrecord_dir))
|
||||
else:
|
||||
print("coco_root not exits.")
|
||||
elif args_opt.dataset == "voc":
|
||||
if os.path.isdir(config.voc_dir):
|
||||
print("Create Mindrecord.")
|
||||
voc_data_to_mindrecord(mindrecord_dir, True, prefix)
|
||||
print("Create Mindrecord Done, at {}".format(mindrecord_dir))
|
||||
else:
|
||||
print("voc_dir not exits.")
|
||||
else:
|
||||
if os.path.isdir(config.image_dir) and os.path.exists(config.anno_path):
|
||||
print("Create Mindrecord.")
|
||||
data_to_mindrecord_byte_image("other", True, prefix)
|
||||
print("Create Mindrecord Done, at {}".format(mindrecord_dir))
|
||||
else:
|
||||
print("image_dir or anno_path not exits.")
|
||||
|
||||
if not args_opt.only_create_dataset:
|
||||
loss_scale = float(args_opt.loss_scale)
|
||||
|
||||
# When create MindDataset, using the fitst mindrecord file, such as ssd.mindrecord0.
|
||||
dataset = create_ssd_dataset(mindrecord_file, repeat_num=1,
|
||||
batch_size=args_opt.batch_size, device_num=device_num, rank=rank)
|
||||
|
||||
dataset_size = dataset.get_dataset_size()
|
||||
print("Create dataset done!")
|
||||
|
||||
backbone = ssd_ghostnet()
|
||||
ssd = SSD300(backbone=backbone, config=config)
|
||||
# print(ssd)
|
||||
net = SSDWithLossCell(ssd, config)
|
||||
init_net_param(net)
|
||||
|
||||
# checkpoint
|
||||
ckpt_config = CheckpointConfig(
|
||||
save_checkpoint_steps=dataset_size * args_opt.save_checkpoint_epochs, keep_checkpoint_max=60)
|
||||
ckpoint_cb = ModelCheckpoint(
|
||||
prefix="ssd", directory=None, config=ckpt_config)
|
||||
|
||||
if args_opt.pre_trained:
|
||||
if args_opt.pre_trained_epoch_size <= 0:
|
||||
raise KeyError(
|
||||
"pre_trained_epoch_size must be greater than 0.")
|
||||
param_dict = load_checkpoint(args_opt.pre_trained)
|
||||
if args_opt.filter_weight:
|
||||
filter_checkpoint_parameter(param_dict)
|
||||
load_param_into_net(net, param_dict)
|
||||
|
||||
lr = Tensor(get_lr(global_step=config.global_step,
|
||||
lr_init=config.lr_init, lr_end=config.lr_end_rate * args_opt.lr, lr_max=args_opt.lr,
|
||||
warmup_epochs=config.warmup_epochs,
|
||||
total_epochs=args_opt.epoch_size,
|
||||
steps_per_epoch=dataset_size))
|
||||
opt = nn.Momentum(filter(lambda x: x.requires_grad, net.get_parameters()), lr,
|
||||
config.momentum, config.weight_decay, loss_scale)
|
||||
net = TrainingWrapper(net, opt, loss_scale)
|
||||
|
||||
callback = [TimeMonitor(data_size=dataset_size),
|
||||
LossMonitor(), ckpoint_cb]
|
||||
|
||||
model = Model(net)
|
||||
dataset_sink_mode = False
|
||||
if args_opt.mode == "sink":
|
||||
print("In sink mode, one epoch return a loss.")
|
||||
dataset_sink_mode = True
|
||||
print("Start train SSD, the first epoch will be slower because of the graph compilation.")
|
||||
model.train(args_opt.epoch_size, dataset,
|
||||
callbacks=callback, dataset_sink_mode=dataset_sink_mode)
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
main()
|
||||
Loading…
Reference in New Issue