forked from mindspore-Ecosystem/mindspore
!6759 Add music auto tagging network in modelzoo
Merge pull request !6759 from jiangzhenguang/add_music_auto_tagging
This commit is contained in:
Gitee
commit
2b6a88e09e
|
|
@ -0,0 +1,203 @@
|
||||||
|
# Contents
|
||||||
|
|
||||||
|
- [Music Auto Tagging Description](#fcn-4-description)
|
||||||
|
- [Model Architecture](#model-architecture)
|
||||||
|
- [Features](#features)
|
||||||
|
- [Mixed Precision](#mixed-precision)
|
||||||
|
- [Environment Requirements](#environment-requirements)
|
||||||
|
- [Quick Start](#quick-start)
|
||||||
|
- [Script Description](#script-description)
|
||||||
|
- [Script and Sample Code](#script-and-sample-code)
|
||||||
|
- [Script Parameters](#script-parameters)
|
||||||
|
- [Training Process](#training-process)
|
||||||
|
- [Training](#training)
|
||||||
|
- [Evaluation Process](#evaluation-process)
|
||||||
|
- [Evaluation](#evaluation)
|
||||||
|
- [Model Description](#model-description)
|
||||||
|
- [Performance](#performance)
|
||||||
|
- [Evaluation Performance](#evaluation-performance)
|
||||||
|
- [ModelZoo Homepage](#modelzoo-homepage)
|
||||||
|
|
||||||
|
|
||||||
|
# [Music Auto Tagging Description](#contents)
|
||||||
|
|
||||||
|
This repository provides a script and recipe to train the Music Auto Tagging model to achieve state-of-the-art accuracy.
|
||||||
|
|
||||||
|
[Paper](https://arxiv.org/abs/1606.00298): `"Keunwoo Choi, George Fazekas, and Mark Sandler, “Automatic tagging using deep convolutional neural networks,” in International Society of Music Information Retrieval Conference. ISMIR, 2016."
|
||||||
|
|
||||||
|
|
||||||
|
# [Model Architecture](#contents)
|
||||||
|
|
||||||
|
Music Auto Tagging is a convolutional neural network architecture, its name Music Auto Tagging comes from the fact that it has 4 layers. Its layers consists of Convolutional layers, Max Pooling layers, Activation layers, Fully connected layers.
|
||||||
|
|
||||||
|
# [Features](#contents)
|
||||||
|
|
||||||
|
## Mixed Precision
|
||||||
|
|
||||||
|
The [mixed precision](https://www.mindspore.cn/tutorial/zh-CN/master/advanced_use/mixed_precision.html) training method accelerates the deep learning neural network training process by using both the single-precision and half-precision data formats, and maintains the network precision achieved by the single-precision training at the same time. Mixed precision training can accelerate the computation process, reduce memory usage, and enable a larger model or batch size to be trained on specific hardware.
|
||||||
|
For FP16 operators, if the input data type is FP32, the backend of MindSpore will automatically handle it with reduced precision. Users could check the reduced-precision operators by enabling INFO log and then searching ‘reduce precision’.
|
||||||
|
|
||||||
|
|
||||||
|
# [Environment Requirements](#contents)
|
||||||
|
|
||||||
|
- Hardware（Ascend
|
||||||
|
- If you want to try Ascend , please send the [application form](https://obs-9be7.obs.cn-east-2.myhuaweicloud.com/file/other/Ascend%20Model%20Zoo%E4%BD%93%E9%AA%8C%E8%B5%84%E6%BA%90%E7%94%B3%E8%AF%B7%E8%A1%A8.docx) to ascend@huawei.com. Once approved, you can get the resources.
|
||||||
|
- Framework
|
||||||
|
- [MindSpore](https://www.mindspore.cn/install/en)
|
||||||
|
- For more information, please check the resources below：
|
||||||
|
- [MindSpore tutorials](https://www.mindspore.cn/tutorial/zh-CN/master/index.html)
|
||||||
|
- [MindSpore API](https://www.mindspore.cn/api/zh-CN/master/index.html)
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
# [Quick Start](#contents)
|
||||||
|
|
||||||
|
After installing MindSpore via the official website, you can start training and evaluation as follows:
|
||||||
|
|
||||||
|
### 1. Download and preprocess the dataset
|
||||||
|
|
||||||
|
1. down load the classification dataset (for instance, MagnaTagATune Dataset, Million Song Dataset, etc)
|
||||||
|
2. Extract the dataset
|
||||||
|
3. The information file of each clip should contain the label and path. Please refer to the annotations_final.csv in MagnaTagATune Dataset.
|
||||||
|
4. The provided pre-processing script use MagnaTagATune Dataset as an example. Please modify the code accprding to your own need.
|
||||||
|
|
||||||
|
### 2. setup parameters (src/config.py)
|
||||||
|
|
||||||
|
### 3. Train
|
||||||
|
|
||||||
|
after having your dataset, first convert the audio clip into mindrecord dataset by using the following codes
|
||||||
|
```shell
|
||||||
|
python pre_process_data.py --device_id 0
|
||||||
|
```
|
||||||
|
|
||||||
|
Then, you can start training the model by using the following codes
|
||||||
|
```shell
|
||||||
|
SLOG_PRINT_TO_STDOUT=1 python train.py --device_id 0
|
||||||
|
```
|
||||||
|
|
||||||
|
### 4. Test
|
||||||
|
|
||||||
|
Then you can test your model
|
||||||
|
```shell
|
||||||
|
SLOG_PRINT_TO_STDOUT=1 python eval.py --device_id 0
|
||||||
|
```
|
||||||
|
|
||||||
|
# [Script Description](#contents)
|
||||||
|
|
||||||
|
## [Script and Sample Code](#contents)
|
||||||
|
|
||||||
|
```
|
||||||
|
├── model_zoo
|
||||||
|
├── README.md // descriptions about all the models
|
||||||
|
├── music_auto_tagging
|
||||||
|
├── README.md // descriptions about googlenet
|
||||||
|
├── scripts
|
||||||
|
│ ├──run_train.sh // shell script for distributed on Ascend
|
||||||
|
│ ├──run_eval.sh // shell script for evaluation on Ascend
|
||||||
|
│ ├──run_process_data.sh // shell script for convert audio clips to mindrecord
|
||||||
|
├── src
|
||||||
|
│ ├──dataset.py // creating dataset
|
||||||
|
│ ├──pre_process_data.py // pre-process dataset
|
||||||
|
│ ├──musictagger.py // googlenet architecture
|
||||||
|
│ ├──config.py // parameter configuration
|
||||||
|
│ ├──loss.py // loss function
|
||||||
|
│ ├──tag.txt // tag for each number
|
||||||
|
├── train.py // training script
|
||||||
|
├── eval.py // evaluation script
|
||||||
|
├── export.py // export model in air format
|
||||||
|
```
|
||||||
|
|
||||||
|
## [Script Parameters](#contents)
|
||||||
|
|
||||||
|
Parameters for both training and evaluation can be set in config.py
|
||||||
|
|
||||||
|
- config for Music Auto tagging
|
||||||
|
|
||||||
|
```python
|
||||||
|
|
||||||
|
'num_classes': 50 # number of tagging classes
|
||||||
|
'num_consumer': 4 # file number for mindrecord
|
||||||
|
'get_npy': 1 # mode for converting to npy, default 1 in this case
|
||||||
|
'get_mindrecord': 1 # mode for converting npy file into mindrecord file，default 1 in this case
|
||||||
|
'audio_path': "/dev/data/Music_Tagger_Data/fea/" # path to audio clips
|
||||||
|
'npy_path': "/dev/data/Music_Tagger_Data/fea/" # path to numpy
|
||||||
|
'info_path': "/dev/data/Music_Tagger_Data/fea/" # path to info_name, which provide the label of each audio clips
|
||||||
|
'info_name': 'annotations_final.csv' # info_name
|
||||||
|
'device_target': 'Ascend' # device running the program
|
||||||
|
'device_id': 0 # device ID used to train or evaluate the dataset. Ignore it when you use run_train.sh for distributed training
|
||||||
|
'mr_path': '/dev/data/Music_Tagger_Data/fea/' # path to mindrecord
|
||||||
|
'mr_name': ['train', 'val'] # mindrecord name
|
||||||
|
|
||||||
|
'pre_trained': False # whether training based on the pre-trained model
|
||||||
|
'lr': 0.0005 # learning rate
|
||||||
|
'batch_size': 32 # training batch size
|
||||||
|
'epoch_size': 10 # total training epochs
|
||||||
|
'loss_scale': 1024.0 # loss scale
|
||||||
|
'num_consumer': 4 # file number for mindrecord
|
||||||
|
'mixed_precision': False # if use mix precision calculation
|
||||||
|
'train_filename': 'train.mindrecord0' # file name of the train mindrecord data
|
||||||
|
'val_filename': 'val.mindrecord0' # file name of the evaluation mindrecord data
|
||||||
|
'data_dir': '/dev/data/Music_Tagger_Data/fea/' # directory of mindrecord data
|
||||||
|
'device_target': 'Ascend' # device running the program
|
||||||
|
'device_id': 0, # device ID used to train or evaluate the dataset. Ignore it when you use run_train.sh for distributed training
|
||||||
|
'keep_checkpoint_max': 10, # only keep the last keep_checkpoint_max checkpoint
|
||||||
|
'save_step': 2000, # steps for saving checkpoint
|
||||||
|
'checkpoint_path': '/dev/data/Music_Tagger_Data/model/', # the absolute full path to save the checkpoint file
|
||||||
|
'prefix': 'MusicTagger', # prefix of checkpoint
|
||||||
|
'model_name': 'MusicTagger_3-50_543.ckpt', # checkpoint name
|
||||||
|
```
|
||||||
|
|
||||||
|
|
||||||
|
## [Training Process](#contents)
|
||||||
|
|
||||||
|
### Training
|
||||||
|
|
||||||
|
- running on Ascend
|
||||||
|
|
||||||
|
```
|
||||||
|
python train.py > train.log 2>&1 &
|
||||||
|
```
|
||||||
|
|
||||||
|
The python command above will run in the background, you can view the results through the file `train.log`.
|
||||||
|
|
||||||
|
After training, you'll get some checkpoint files under the script folder by default. The loss value will be achieved as follows:
|
||||||
|
|
||||||
|
```
|
||||||
|
# grep "loss is " train.log
|
||||||
|
epoch: 1 step: 100, loss is 0.23264095
|
||||||
|
epoch: 1 step: 200, loss is 0.2013525
|
||||||
|
...
|
||||||
|
```
|
||||||
|
|
||||||
|
The model checkpoint will be saved in the set directory.
|
||||||
|
|
||||||
|
## [Evaluation Process](#contents)
|
||||||
|
|
||||||
|
### Evaluation
|
||||||
|
|
||||||
|
|
||||||
|
# [Model Description](#contents)
|
||||||
|
## [Performance](#contents)
|
||||||
|
|
||||||
|
### Evaluation Performance
|
||||||
|
|
||||||
|
| Parameters | Ascend |
|
||||||
|
| -------------------------- | ----------------------------------------------------------- |
|
||||||
|
| Model Version | FCN-4 |
|
||||||
|
| Resource | Ascend 910 ；CPU 2.60GHz，56cores；Memory，314G |
|
||||||
|
| uploaded Date | 09/11/2020 (month/day/year) |
|
||||||
|
| MindSpore Version | r0.7.0 |
|
||||||
|
| Training Parameters | epoch=10, steps=534, batch_size = 32, lr=0.005 |
|
||||||
|
| Optimizer | Adam |
|
||||||
|
| Loss Function | Binary cross entropy |
|
||||||
|
| outputs | probability |
|
||||||
|
| Loss | AUC 0.909 |
|
||||||
|
| Speed | 1pc: 160 samples/sec; |
|
||||||
|
| Total time | 1pc: 20 mins; |
|
||||||
|
| Checkpoint for Fine tuning | 198.73M(.ckpt file) |
|
||||||
|
| Scripts | [music_auto_tagging script](https://gitee.com/mindspore/mindspore/tree/master/model_zoo/official/audio/music_auto_tagging) |
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
# [ModelZoo Homepage](#contents)
|
||||||
|
Please check the official [homepage](https://gitee.com/mindspore/mindspore/tree/master/model_zoo).
|
||||||
|
|
@ -0,0 +1,137 @@
|
||||||
|
# Copyright 2020 Huawei Technologies Co., Ltd
|
||||||
|
#
|
||||||
|
# Licensed under the Apache License, Version 2.0 (the "License");
|
||||||
|
# you may not use this file except in compliance with the License.
|
||||||
|
# You may obtain a copy of the License at
|
||||||
|
#
|
||||||
|
# http://www.apache.org/licenses/LICENSE-2.0
|
||||||
|
#
|
||||||
|
# Unless required by applicable law or agreed to in writing, software
|
||||||
|
# distributed under the License is distributed on an "AS IS" BASIS,
|
||||||
|
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||||
|
# See the License for the specific language governing permissions and
|
||||||
|
# limitations under the License.
|
||||||
|
# ============================================================================
|
||||||
|
'''
|
||||||
|
##############evaluate trained models#################
|
||||||
|
python eval.py
|
||||||
|
'''
|
||||||
|
|
||||||
|
import argparse
|
||||||
|
import numpy as np
|
||||||
|
import mindspore.common.dtype as mstype
|
||||||
|
from mindspore import context
|
||||||
|
from mindspore import Tensor
|
||||||
|
from mindspore.train.serialization import load_checkpoint, load_param_into_net
|
||||||
|
from src.musictagger import MusicTaggerCNN
|
||||||
|
from src.config import music_cfg as cfg
|
||||||
|
from src.dataset import create_dataset
|
||||||
|
|
||||||
|
|
||||||
|
def calculate_auc(labels_list, preds_list):
|
||||||
|
"""
|
||||||
|
The AUC calculation function
|
||||||
|
Input:
|
||||||
|
labels_list: list of true label
|
||||||
|
preds_list: list of predicted label
|
||||||
|
Outputs
|
||||||
|
Float, means of AUC
|
||||||
|
"""
|
||||||
|
auc = []
|
||||||
|
n_bins = labels_list.shape[0] // 2
|
||||||
|
if labels_list.ndim == 1:
|
||||||
|
labels_list = labels_list.reshape(-1, 1)
|
||||||
|
preds_list = preds_list.reshape(-1, 1)
|
||||||
|
for i in range(labels_list.shape[1]):
|
||||||
|
labels = labels_list[:, i]
|
||||||
|
preds = preds_list[:, i]
|
||||||
|
postive_len = labels.sum()
|
||||||
|
negative_len = labels.shape[0] - postive_len
|
||||||
|
total_case = postive_len * negative_len
|
||||||
|
positive_histogram = np.zeros((n_bins))
|
||||||
|
negative_histogram = np.zeros((n_bins))
|
||||||
|
bin_width = 1.0 / n_bins
|
||||||
|
|
||||||
|
for j, _ in enumerate(labels):
|
||||||
|
nth_bin = int(preds[j] // bin_width)
|
||||||
|
if labels[j]:
|
||||||
|
positive_histogram[nth_bin] = positive_histogram[nth_bin] + 1
|
||||||
|
else:
|
||||||
|
negative_histogram[nth_bin] = negative_histogram[nth_bin] + 1
|
||||||
|
|
||||||
|
accumulated_negative = 0
|
||||||
|
satisfied_pair = 0
|
||||||
|
for k in range(n_bins):
|
||||||
|
satisfied_pair += (
|
||||||
|
positive_histogram[k] * accumulated_negative +
|
||||||
|
positive_histogram[k] * negative_histogram[k] * 0.5)
|
||||||
|
accumulated_negative += negative_histogram[k]
|
||||||
|
auc.append(satisfied_pair / total_case)
|
||||||
|
|
||||||
|
return np.mean(auc)
|
||||||
|
|
||||||
|
|
||||||
|
def val(net, data_dir, filename, num_consumer=4, batch=32):
|
||||||
|
"""
|
||||||
|
Validation function, estimate the performance of trained model
|
||||||
|
|
||||||
|
Input:
|
||||||
|
net: the trained neural network
|
||||||
|
data_dir: path to the validation dataset
|
||||||
|
filename: name of the validation dataset
|
||||||
|
num_consumer: split number of validation dataset
|
||||||
|
batch: validation batch size
|
||||||
|
Outputs
|
||||||
|
Float, AUC
|
||||||
|
"""
|
||||||
|
data_train = create_dataset(data_dir, filename, 32, ['feature', 'label'],
|
||||||
|
num_consumer)
|
||||||
|
data_train = data_train.create_tuple_iterator()
|
||||||
|
res_pred = []
|
||||||
|
res_true = []
|
||||||
|
for data, label in data_train:
|
||||||
|
x = net(Tensor(data, dtype=mstype.float32))
|
||||||
|
res_pred.append(x.asnumpy())
|
||||||
|
res_true.append(label.asnumpy())
|
||||||
|
res_pred = np.concatenate(res_pred, axis=0)
|
||||||
|
res_true = np.concatenate(res_true, axis=0)
|
||||||
|
auc = calculate_auc(res_true, res_pred)
|
||||||
|
return auc
|
||||||
|
|
||||||
|
|
||||||
|
def validation(net, model_path, data_dir, filename, num_consumer, batch):
|
||||||
|
param_dict = load_checkpoint(model_path)
|
||||||
|
load_param_into_net(net, param_dict)
|
||||||
|
|
||||||
|
auc = val(net, data_dir, filename, num_consumer, batch)
|
||||||
|
return auc
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
parser = argparse.ArgumentParser(description='Evaluate model')
|
||||||
|
parser.add_argument('--device_id',
|
||||||
|
type=int,
|
||||||
|
help='device ID',
|
||||||
|
default=None)
|
||||||
|
args = parser.parse_args()
|
||||||
|
|
||||||
|
if args.device_id is not None:
|
||||||
|
context.set_context(device_target=cfg.device_target,
|
||||||
|
mode=context.GRAPH_MODE,
|
||||||
|
device_id=args.device_id)
|
||||||
|
else:
|
||||||
|
context.set_context(device_target=cfg.device_target,
|
||||||
|
mode=context.GRAPH_MODE,
|
||||||
|
device_id=cfg.device_id)
|
||||||
|
|
||||||
|
network = MusicTaggerCNN(in_classes=[1, 128, 384, 768, 2048],
|
||||||
|
kernel_size=[3, 3, 3, 3, 3],
|
||||||
|
padding=[0] * 5,
|
||||||
|
maxpool=[(2, 4), (4, 5), (3, 8), (4, 8)],
|
||||||
|
has_bias=True)
|
||||||
|
network.set_train(False)
|
||||||
|
auc_val = validation(network, cfg.checkpoint_path + "/" + cfg.model_name, cfg.data_dir,
|
||||||
|
cfg.val_filename, cfg.num_consumer, cfg.batch_size)
|
||||||
|
|
||||||
|
print("=" * 10 + "Validation Peformance" + "=" * 10)
|
||||||
|
print("AUC: {:.5f}".format(auc_val))
|
||||||
|
|
@ -0,0 +1,40 @@
|
||||||
|
# Copyright 2020 Huawei Technologies Co., Ltd
|
||||||
|
#
|
||||||
|
# Licensed under the Apache License, Version 2.0 (the "License");
|
||||||
|
# you may not use this file except in compliance with the License.
|
||||||
|
# You may obtain a copy of the License at
|
||||||
|
#
|
||||||
|
# http://www.apache.org/licenses/LICENSE-2.0
|
||||||
|
#
|
||||||
|
# Unless required by applicable law or agreed to in writing, software
|
||||||
|
# distributed under the License is distributed on an "AS IS" BASIS,
|
||||||
|
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||||
|
# See the License for the specific language governing permissions and
|
||||||
|
# limitations under the License.
|
||||||
|
# ============================================================================
|
||||||
|
'''
|
||||||
|
##############evaluate trained models#################
|
||||||
|
python export.py
|
||||||
|
'''
|
||||||
|
|
||||||
|
import numpy as np
|
||||||
|
from mindspore.train.serialization import export
|
||||||
|
from mindspore import Tensor
|
||||||
|
from mindspore.train.serialization import load_checkpoint, load_param_into_net
|
||||||
|
from src.musictagger import MusicTaggerCNN
|
||||||
|
from src.config import music_cfg as cfg
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
network = MusicTaggerCNN(in_classes=[1, 128, 384, 768, 2048],
|
||||||
|
kernel_size=[3, 3, 3, 3, 3],
|
||||||
|
padding=[0] * 5,
|
||||||
|
maxpool=[(2, 4), (4, 5), (3, 8), (4, 8)],
|
||||||
|
has_bias=True)
|
||||||
|
param_dict = load_checkpoint(cfg.checkpoint_path + "/" + cfg.model_name)
|
||||||
|
load_param_into_net(network, param_dict)
|
||||||
|
input_data = np.random.uniform(0.0, 1.0, size=[1, 1, 96, 1366]).astype(np.float32)
|
||||||
|
export(network,
|
||||||
|
Tensor(input_data),
|
||||||
|
filename="{}/{}.air".format(cfg.checkpoint_path,
|
||||||
|
cfg.model_name[:-5]),
|
||||||
|
file_format="AIR")
|
||||||
|
|
@ -0,0 +1,18 @@
|
||||||
|
#!/bin/bash
|
||||||
|
# Copyright 2020 Huawei Technologies Co., Ltd
|
||||||
|
#
|
||||||
|
# Licensed under the Apache License, Version 2.0 (the "License");
|
||||||
|
# you may not use this file except in compliance with the License.
|
||||||
|
# You may obtain a copy of the License at
|
||||||
|
#
|
||||||
|
# http://www.apache.org/licenses/LICENSE-2.0
|
||||||
|
#
|
||||||
|
# Unless required by applicable law or agreed to in writing, software
|
||||||
|
# distributed under the License is distributed on an "AS IS" BASIS,
|
||||||
|
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||||
|
# See the License for the specific language governing permissions and
|
||||||
|
# limitations under the License.
|
||||||
|
# ============================================================================
|
||||||
|
|
||||||
|
export SLOG_PRINT_TO_STDOUT=1
|
||||||
|
python ../eval.py --device_id 0
|
||||||
|
|
@ -0,0 +1,18 @@
|
||||||
|
#!/bin/bash
|
||||||
|
# Copyright 2020 Huawei Technologies Co., Ltd
|
||||||
|
#
|
||||||
|
# Licensed under the Apache License, Version 2.0 (the "License");
|
||||||
|
# you may not use this file except in compliance with the License.
|
||||||
|
# You may obtain a copy of the License at
|
||||||
|
#
|
||||||
|
# http://www.apache.org/licenses/LICENSE-2.0
|
||||||
|
#
|
||||||
|
# Unless required by applicable law or agreed to in writing, software
|
||||||
|
# distributed under the License is distributed on an "AS IS" BASIS,
|
||||||
|
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||||
|
# See the License for the specific language governing permissions and
|
||||||
|
# limitations under the License.
|
||||||
|
# ============================================================================
|
||||||
|
|
||||||
|
export SLOG_PRINT_TO_STDOUT=1
|
||||||
|
python ../src/pre_process_data.py --device_id 0
|
||||||
|
|
@ -0,0 +1,18 @@
|
||||||
|
#!/bin/bash
|
||||||
|
# Copyright 2020 Huawei Technologies Co., Ltd
|
||||||
|
#
|
||||||
|
# Licensed under the Apache License, Version 2.0 (the "License");
|
||||||
|
# you may not use this file except in compliance with the License.
|
||||||
|
# You may obtain a copy of the License at
|
||||||
|
#
|
||||||
|
# http://www.apache.org/licenses/LICENSE-2.0
|
||||||
|
#
|
||||||
|
# Unless required by applicable law or agreed to in writing, software
|
||||||
|
# distributed under the License is distributed on an "AS IS" BASIS,
|
||||||
|
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||||
|
# See the License for the specific language governing permissions and
|
||||||
|
# limitations under the License.
|
||||||
|
# ============================================================================
|
||||||
|
|
||||||
|
export SLOG_PRINT_TO_STDOUT=1
|
||||||
|
python ../train.py --device_id 0
|
||||||
|
|
@ -0,0 +1,23 @@
|
||||||
|
# Copyright 2020 Huawei Technologies Co., Ltd
|
||||||
|
#
|
||||||
|
# Licensed under the Apache License, Version 2.0 (the "License");
|
||||||
|
# you may not use this file except in compliance with the License.
|
||||||
|
# You may obtain a copy of the License at
|
||||||
|
#
|
||||||
|
# http://www.apache.org/licenses/LICENSE-2.0
|
||||||
|
#
|
||||||
|
# Unless required by applicable law or agreed to in writing, software
|
||||||
|
# distributed under the License is distributed on an "AS IS" BASIS,
|
||||||
|
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||||
|
# See the License for the specific language governing permissions and
|
||||||
|
# limitations under the License.
|
||||||
|
# ============================================================================
|
||||||
|
"""
|
||||||
|
__init__.py
|
||||||
|
"""
|
||||||
|
|
||||||
|
from . import musictagger
|
||||||
|
from . import loss
|
||||||
|
from . import dataset
|
||||||
|
from . import config
|
||||||
|
from . import pre_process_data
|
||||||
|
|
@ -0,0 +1,53 @@
|
||||||
|
# Copyright 2020 Huawei Technologies Co., Ltd
|
||||||
|
#
|
||||||
|
# Licensed under the Apache License, Version 2.0 (the "License");
|
||||||
|
# you may not use this file except in compliance with the License.
|
||||||
|
# You may obtain a copy of the License at
|
||||||
|
#
|
||||||
|
# http://www.apache.org/licenses/LICENSE-2.0
|
||||||
|
#
|
||||||
|
# Unless required by applicable law or agreed to in writing, software
|
||||||
|
# distributed under the License is distributed on an "AS IS" BASIS,
|
||||||
|
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||||
|
# See the License for the specific language governing permissions and
|
||||||
|
# limitations under the License.
|
||||||
|
# ============================================================================
|
||||||
|
"""
|
||||||
|
network config setting, will be used in train.py, eval.py
|
||||||
|
"""
|
||||||
|
from easydict import EasyDict as edict
|
||||||
|
|
||||||
|
data_cfg = edict({
|
||||||
|
'num_classes': 50,
|
||||||
|
'num_consumer': 4,
|
||||||
|
'get_npy': 1,
|
||||||
|
'get_mindrecord': 1,
|
||||||
|
'audio_path': "/dev/data/Music_Tagger_Data/fea/",
|
||||||
|
'npy_path': "/dev/data/Music_Tagger_Data/fea/",
|
||||||
|
'info_path': "/dev/data/Music_Tagger_Data/fea/",
|
||||||
|
'info_name': 'annotations_final.csv',
|
||||||
|
'device_target': 'Ascend',
|
||||||
|
'device_id': 0,
|
||||||
|
'mr_path': '/dev/data/Music_Tagger_Data/fea/',
|
||||||
|
'mr_name': ['train', 'val'],
|
||||||
|
})
|
||||||
|
|
||||||
|
music_cfg = edict({
|
||||||
|
'pre_trained': False,
|
||||||
|
'lr': 0.0005,
|
||||||
|
'batch_size': 32,
|
||||||
|
'epoch_size': 10,
|
||||||
|
'loss_scale': 1024.0,
|
||||||
|
'num_consumer': 4,
|
||||||
|
'mixed_precision': False,
|
||||||
|
'train_filename': 'train.mindrecord0',
|
||||||
|
'val_filename': 'val.mindrecord0',
|
||||||
|
'data_dir': '/dev/data/Music_Tagger_Data/fea/',
|
||||||
|
'device_target': 'Ascend',
|
||||||
|
'device_id': 0,
|
||||||
|
'keep_checkpoint_max': 10,
|
||||||
|
'save_step': 2000,
|
||||||
|
'checkpoint_path': '/dev/data/Music_Tagger_Data/model',
|
||||||
|
'prefix': 'MusicTagger',
|
||||||
|
'model_name': 'MusicTagger_3-50_543.ckpt',
|
||||||
|
})
|
||||||
|
|
@ -0,0 +1,30 @@
|
||||||
|
# Copyright 2020 Huawei Technologies Co., Ltd
|
||||||
|
#
|
||||||
|
# Licensed under the Apache License, Version 2.0 (the "License");
|
||||||
|
# you may not use this file except in compliance with the License.
|
||||||
|
# You may obtain a copy of the License at
|
||||||
|
#
|
||||||
|
# http://www.apache.org/licenses/LICENSE-2.0
|
||||||
|
#
|
||||||
|
# Unless required by applicable law or agreed to in writing, software
|
||||||
|
# distributed under the License is distributed on an "AS IS" BASIS,
|
||||||
|
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||||
|
# See the License for the specific language governing permissions and
|
||||||
|
# limitations under the License.
|
||||||
|
# ============================================================================
|
||||||
|
'''python dataset.py'''
|
||||||
|
|
||||||
|
import os
|
||||||
|
import mindspore.dataset as ds
|
||||||
|
|
||||||
|
|
||||||
|
def create_dataset(base_path, filename, batch_size, columns_list,
|
||||||
|
num_consumer):
|
||||||
|
"""Create dataset"""
|
||||||
|
|
||||||
|
path = os.path.join(base_path, filename)
|
||||||
|
dtrain = ds.MindDataset(path, columns_list, num_consumer)
|
||||||
|
dtrain = dtrain.shuffle(buffer_size=dtrain.get_dataset_size())
|
||||||
|
dtrain = dtrain.batch(batch_size, drop_remainder=True)
|
||||||
|
|
||||||
|
return dtrain
|
||||||
|
|
@ -0,0 +1,41 @@
|
||||||
|
# Copyright 2020 Huawei Technologies Co., Ltd
|
||||||
|
#
|
||||||
|
# Licensed under the Apache License, Version 2.0 (the "License");
|
||||||
|
# you may not use this file except in compliance with the License.
|
||||||
|
# You may obtain a copy of the License at
|
||||||
|
#
|
||||||
|
# http://www.apache.org/licenses/LICENSE-2.0
|
||||||
|
#
|
||||||
|
# Unless required by applicable law or agreed to in writing, software
|
||||||
|
# distributed under the License is distributed on an "AS IS" BASIS,
|
||||||
|
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||||
|
# See the License for the specific language governing permissions and
|
||||||
|
# limitations under the License.
|
||||||
|
# ============================================================================
|
||||||
|
"""
|
||||||
|
define loss
|
||||||
|
"""
|
||||||
|
from mindspore import nn
|
||||||
|
import mindspore.common.dtype as mstype
|
||||||
|
from mindspore.ops import operations as P
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
class BCELoss(nn.Cell):
|
||||||
|
"""
|
||||||
|
BCELoss
|
||||||
|
"""
|
||||||
|
def __init__(self, record=None):
|
||||||
|
super(BCELoss, self).__init__(record)
|
||||||
|
self.sm_scalar = P.ScalarSummary()
|
||||||
|
self.cast = P.Cast()
|
||||||
|
self.record = record
|
||||||
|
self.weight = None
|
||||||
|
self.bce = P.BinaryCrossEntropy()
|
||||||
|
|
||||||
|
def construct(self, input_data, target):
|
||||||
|
target = self.cast(target, mstype.float32)
|
||||||
|
loss = self.bce(input_data, target, self.weight)
|
||||||
|
if self.record:
|
||||||
|
self.sm_scalar("loss", loss)
|
||||||
|
return loss
|
||||||
|
|
@ -0,0 +1,83 @@
|
||||||
|
# Copyright 2020 Huawei Technologies Co., Ltd
|
||||||
|
#
|
||||||
|
# Licensed under the Apache License, Version 2.0 (the "License");
|
||||||
|
# you may not use this file except in compliance with the License.
|
||||||
|
# You may obtain a copy of the License at
|
||||||
|
#
|
||||||
|
# http://www.apache.org/licenses/LICENSE-2.0
|
||||||
|
#
|
||||||
|
# Unless required by applicable law or agreed to in writing, software
|
||||||
|
# distributed under the License is distributed on an "AS IS" BASIS,
|
||||||
|
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||||
|
# See the License for the specific language governing permissions and
|
||||||
|
# limitations under the License.
|
||||||
|
# ============================================================================
|
||||||
|
'''model'''
|
||||||
|
|
||||||
|
from mindspore import nn
|
||||||
|
from mindspore.ops import operations as P
|
||||||
|
|
||||||
|
|
||||||
|
class MusicTaggerCNN(nn.Cell):
|
||||||
|
"""
|
||||||
|
Music Tagger CNN
|
||||||
|
"""
|
||||||
|
def __init__(self, in_classes, kernel_size, padding, maxpool, has_bias):
|
||||||
|
super(MusicTaggerCNN, self).__init__()
|
||||||
|
self.in_classes = in_classes
|
||||||
|
self.kernel_size = kernel_size
|
||||||
|
self.maxpool = maxpool
|
||||||
|
self.padding = padding
|
||||||
|
self.has_bias = has_bias
|
||||||
|
# build model
|
||||||
|
self.conv1 = nn.Conv2d(self.in_classes[0], self.in_classes[1],
|
||||||
|
self.kernel_size[0])
|
||||||
|
self.conv2 = nn.Conv2d(self.in_classes[1], self.in_classes[2],
|
||||||
|
self.kernel_size[1])
|
||||||
|
self.conv3 = nn.Conv2d(self.in_classes[2], self.in_classes[3],
|
||||||
|
self.kernel_size[2])
|
||||||
|
self.conv4 = nn.Conv2d(self.in_classes[3], self.in_classes[4],
|
||||||
|
self.kernel_size[3])
|
||||||
|
|
||||||
|
self.bn1 = nn.BatchNorm2d(self.in_classes[1])
|
||||||
|
self.bn2 = nn.BatchNorm2d(self.in_classes[2])
|
||||||
|
self.bn3 = nn.BatchNorm2d(self.in_classes[3])
|
||||||
|
self.bn4 = nn.BatchNorm2d(self.in_classes[4])
|
||||||
|
|
||||||
|
self.pool1 = nn.MaxPool2d(maxpool[0], maxpool[0])
|
||||||
|
self.pool2 = nn.MaxPool2d(maxpool[1], maxpool[1])
|
||||||
|
self.pool3 = nn.MaxPool2d(maxpool[2], maxpool[2])
|
||||||
|
self.pool4 = nn.MaxPool2d(maxpool[3], maxpool[3])
|
||||||
|
self.poolreduce = P.ReduceMax(keep_dims=False)
|
||||||
|
self.Act = nn.ReLU()
|
||||||
|
self.flatten = nn.Flatten()
|
||||||
|
self.dense = nn.Dense(2048, 50, activation='sigmoid')
|
||||||
|
self.sigmoid = nn.Sigmoid()
|
||||||
|
|
||||||
|
def construct(self, input_data):
|
||||||
|
"""
|
||||||
|
Music Tagger CNN
|
||||||
|
"""
|
||||||
|
x = self.conv1(input_data)
|
||||||
|
x = self.bn1(x)
|
||||||
|
x = self.Act(x)
|
||||||
|
x = self.pool1(x)
|
||||||
|
|
||||||
|
x = self.conv2(x)
|
||||||
|
x = self.bn2(x)
|
||||||
|
x = self.Act(x)
|
||||||
|
x = self.pool2(x)
|
||||||
|
|
||||||
|
x = self.conv3(x)
|
||||||
|
x = self.bn3(x)
|
||||||
|
x = self.Act(x)
|
||||||
|
x = self.pool3(x)
|
||||||
|
|
||||||
|
x = self.conv4(x)
|
||||||
|
x = self.bn4(x)
|
||||||
|
x = self.Act(x)
|
||||||
|
x = self.poolreduce(x, (2, 3))
|
||||||
|
x = self.flatten(x)
|
||||||
|
x = self.dense(x)
|
||||||
|
|
||||||
|
return x
|
||||||
|
|
@ -0,0 +1,226 @@
|
||||||
|
# Copyright 2020 Huawei Technologies Co., Ltd
|
||||||
|
#
|
||||||
|
# Licensed under the Apache License, Version 2.0 (the "License");
|
||||||
|
# you may not use this file except in compliance with the License.
|
||||||
|
# You may obtain a copy of the License at
|
||||||
|
#
|
||||||
|
# http://www.apache.org/licenses/LICENSE-2.0
|
||||||
|
#
|
||||||
|
# Unless required by applicable law or agreed to in writing, software
|
||||||
|
# distributed under the License is distributed on an "AS IS" BASIS,
|
||||||
|
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||||
|
# See the License for the specific language governing permissions and
|
||||||
|
# limitations under the License.
|
||||||
|
# ============================================================================
|
||||||
|
'''python dataset.py'''
|
||||||
|
|
||||||
|
import os
|
||||||
|
import argparse
|
||||||
|
import pandas as pd
|
||||||
|
import numpy as np
|
||||||
|
import librosa
|
||||||
|
from mindspore.mindrecord import FileWriter
|
||||||
|
from mindspore import context
|
||||||
|
from src.config import data_cfg as cfg
|
||||||
|
|
||||||
|
|
||||||
|
def compute_melgram(audio_path, save_path='', filename='', save_npy=True):
|
||||||
|
"""
|
||||||
|
extract melgram feature from the audio and save as numpy array
|
||||||
|
|
||||||
|
Args:
|
||||||
|
audio_path (str): path to the audio clip.
|
||||||
|
save_path (str): path to save the numpy array.
|
||||||
|
filename (str): filename of the audio clip.
|
||||||
|
|
||||||
|
Returns:
|
||||||
|
numpy array.
|
||||||
|
|
||||||
|
"""
|
||||||
|
SR = 12000
|
||||||
|
N_FFT = 512
|
||||||
|
N_MELS = 96
|
||||||
|
HOP_LEN = 256
|
||||||
|
DURA = 29.12 # to make it 1366 frame..
|
||||||
|
|
||||||
|
src, _ = librosa.load(audio_path, sr=SR) # whole signal
|
||||||
|
n_sample = src.shape[0]
|
||||||
|
n_sample_fit = int(DURA * SR)
|
||||||
|
|
||||||
|
if n_sample < n_sample_fit: # if too short
|
||||||
|
src = np.hstack((src, np.zeros((int(DURA * SR) - n_sample,))))
|
||||||
|
elif n_sample > n_sample_fit: # if too long
|
||||||
|
src = src[(n_sample - n_sample_fit) // 2:(n_sample + n_sample_fit) //
|
||||||
|
2]
|
||||||
|
logam = librosa.core.amplitude_to_db
|
||||||
|
melgram = librosa.feature.melspectrogram
|
||||||
|
ret = logam(
|
||||||
|
melgram(y=src, sr=SR, hop_length=HOP_LEN, n_fft=N_FFT, n_mels=N_MELS))
|
||||||
|
ret = ret[np.newaxis, np.newaxis, :]
|
||||||
|
if save_npy:
|
||||||
|
|
||||||
|
save_path = save_path + filename[:-4] + '.npy'
|
||||||
|
np.save(save_path, ret)
|
||||||
|
return ret
|
||||||
|
|
||||||
|
|
||||||
|
def get_data(features_data, labels_data):
|
||||||
|
data_list = []
|
||||||
|
for i, (label, feature) in enumerate(zip(labels_data, features_data)):
|
||||||
|
data_json = {"id": i, "feature": feature, "label": label}
|
||||||
|
data_list.append(data_json)
|
||||||
|
return data_list
|
||||||
|
|
||||||
|
|
||||||
|
def convert(s):
|
||||||
|
if s.isdigit():
|
||||||
|
return int(s)
|
||||||
|
return s
|
||||||
|
|
||||||
|
|
||||||
|
def GetLabel(info_path, info_name):
|
||||||
|
"""
|
||||||
|
separate dataset into training set and validation set
|
||||||
|
|
||||||
|
Args:
|
||||||
|
info_path (str): path to the information file.
|
||||||
|
info_name (str): name of the information file.
|
||||||
|
|
||||||
|
"""
|
||||||
|
T = []
|
||||||
|
with open(info_path + '/' + info_name, 'rb') as info:
|
||||||
|
data = info.readline()
|
||||||
|
while data:
|
||||||
|
T.append([
|
||||||
|
convert(i[1:-1])
|
||||||
|
for i in data.strip().decode('utf-8').split("\t")
|
||||||
|
])
|
||||||
|
data = info.readline()
|
||||||
|
|
||||||
|
annotation = pd.DataFrame(T[1:], columns=T[0])
|
||||||
|
count = []
|
||||||
|
for i in annotation.columns[1:-2]:
|
||||||
|
count.append([annotation[i].sum() / len(annotation), i])
|
||||||
|
count = sorted(count)
|
||||||
|
full_label = []
|
||||||
|
for i in count[-50:]:
|
||||||
|
full_label.append(i[1])
|
||||||
|
out = []
|
||||||
|
for i in T[1:]:
|
||||||
|
index = [k for k, x in enumerate(i) if x == 1]
|
||||||
|
label = [T[0][k] for k in index]
|
||||||
|
L = [str(0) for k in range(50)]
|
||||||
|
L.append(i[-1])
|
||||||
|
for j in label:
|
||||||
|
if j in full_label:
|
||||||
|
ind = full_label.index(j)
|
||||||
|
L[ind] = '1'
|
||||||
|
out.append(L)
|
||||||
|
out = np.array(out)
|
||||||
|
|
||||||
|
Train = []
|
||||||
|
Val = []
|
||||||
|
|
||||||
|
for i in out:
|
||||||
|
if np.random.rand() > 0.2:
|
||||||
|
Train.append(i)
|
||||||
|
else:
|
||||||
|
Val.append(i)
|
||||||
|
np.savetxt("{}/music_tagging_train_tmp.csv".format(info_path),
|
||||||
|
np.array(Train),
|
||||||
|
fmt='%s',
|
||||||
|
delimiter=',')
|
||||||
|
np.savetxt("{}/music_tagging_val_tmp.csv".format(info_path),
|
||||||
|
np.array(Val),
|
||||||
|
fmt='%s',
|
||||||
|
delimiter=',')
|
||||||
|
|
||||||
|
|
||||||
|
def generator_md(info_name, file_path, num_classes):
|
||||||
|
"""
|
||||||
|
generate numpy array from features of all audio clips
|
||||||
|
|
||||||
|
Args:
|
||||||
|
info_path (str): path to the information file.
|
||||||
|
file_path (str): path to the npy files.
|
||||||
|
|
||||||
|
Returns:
|
||||||
|
2 numpy array.
|
||||||
|
|
||||||
|
"""
|
||||||
|
df = pd.read_csv(info_name, header=None)
|
||||||
|
df.columns = [str(i) for i in range(num_classes)] + ["mp3_path"]
|
||||||
|
data = []
|
||||||
|
label = []
|
||||||
|
for i in range(len(df)):
|
||||||
|
try:
|
||||||
|
data.append(
|
||||||
|
np.load(file_path + df.mp3_path.values[i][:-4] +
|
||||||
|
'.npy').reshape(1, 96, 1366))
|
||||||
|
label.append(np.array(df[df.columns[:-1]][i:i + 1])[0])
|
||||||
|
except FileNotFoundError:
|
||||||
|
print("Exception occurred in generator_md.")
|
||||||
|
return np.array(data), np.array(label, dtype=np.int32)
|
||||||
|
|
||||||
|
|
||||||
|
def convert_to_mindrecord(info_name, file_path, store_path, mr_name,
|
||||||
|
num_classes):
|
||||||
|
""" convert dataset to mindrecord """
|
||||||
|
num_shard = 4
|
||||||
|
data, label = generator_md(info_name, file_path, num_classes)
|
||||||
|
schema_json = {
|
||||||
|
"id": {
|
||||||
|
"type": "int32"
|
||||||
|
},
|
||||||
|
"feature": {
|
||||||
|
"type": "float32",
|
||||||
|
"shape": [1, 96, 1366]
|
||||||
|
},
|
||||||
|
"label": {
|
||||||
|
"type": "int32",
|
||||||
|
"shape": [num_classes]
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
writer = FileWriter(
|
||||||
|
os.path.join(store_path, '{}.mindrecord'.format(mr_name)), num_shard)
|
||||||
|
datax = get_data(data, label)
|
||||||
|
writer.add_schema(schema_json, "music_tagger_schema")
|
||||||
|
writer.add_index(["id"])
|
||||||
|
writer.write_raw_data(datax)
|
||||||
|
writer.commit()
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
parser = argparse.ArgumentParser(description='get feature')
|
||||||
|
parser.add_argument('--device_id',
|
||||||
|
type=int,
|
||||||
|
help='device ID',
|
||||||
|
default=None)
|
||||||
|
args = parser.parse_args()
|
||||||
|
|
||||||
|
if cfg.get_npy:
|
||||||
|
GetLabel(cfg.info_path, cfg.info_name)
|
||||||
|
dirname = os.listdir(cfg.audio_path)
|
||||||
|
for d in dirname:
|
||||||
|
file_name = os.listdir("{}/{}".format(cfg.audio_path, d))
|
||||||
|
if not os.path.isdir("{}/{}".format(cfg.npy_path, d)):
|
||||||
|
os.mkdir("{}/{}".format(cfg.npy_path, d))
|
||||||
|
for f in file_name:
|
||||||
|
compute_melgram("{}/{}/{}".format(cfg.audio_path, d, f),
|
||||||
|
"{}/{}/".format(cfg.npy_path, d), f)
|
||||||
|
|
||||||
|
if cfg.get_mindrecord:
|
||||||
|
if args.device_id is not None:
|
||||||
|
context.set_context(device_target='Ascend',
|
||||||
|
mode=context.GRAPH_MODE,
|
||||||
|
device_id=args.device_id)
|
||||||
|
else:
|
||||||
|
context.set_context(device_target='Ascend',
|
||||||
|
mode=context.GRAPH_MODE,
|
||||||
|
device_id=cfg.device_id)
|
||||||
|
for cmn in cfg.mr_nam:
|
||||||
|
if cmn in ['train', 'val']:
|
||||||
|
convert_to_mindrecord('music_tagging_{}_tmp.csv'.format(cmn),
|
||||||
|
cfg.npy_path, cfg.mr_path, cmn,
|
||||||
|
cfg.num_classes)
|
||||||
|
|
@ -0,0 +1,50 @@
|
||||||
|
choral
|
||||||
|
female voice
|
||||||
|
metal
|
||||||
|
country
|
||||||
|
weird
|
||||||
|
no voice
|
||||||
|
cello
|
||||||
|
harp
|
||||||
|
beats
|
||||||
|
female vocal
|
||||||
|
male voice
|
||||||
|
dance
|
||||||
|
new age
|
||||||
|
voice
|
||||||
|
choir
|
||||||
|
classic
|
||||||
|
man
|
||||||
|
solo
|
||||||
|
sitar
|
||||||
|
soft
|
||||||
|
no vocal
|
||||||
|
pop
|
||||||
|
male vocal
|
||||||
|
woman
|
||||||
|
flute
|
||||||
|
quiet
|
||||||
|
loud
|
||||||
|
harpsichord
|
||||||
|
no vocals
|
||||||
|
vocals
|
||||||
|
singing
|
||||||
|
male
|
||||||
|
opera
|
||||||
|
indian
|
||||||
|
female
|
||||||
|
synth
|
||||||
|
vocal
|
||||||
|
violin
|
||||||
|
beat
|
||||||
|
ambient
|
||||||
|
piano
|
||||||
|
fast
|
||||||
|
rock
|
||||||
|
electronic
|
||||||
|
drums
|
||||||
|
strings
|
||||||
|
techno
|
||||||
|
slow
|
||||||
|
classical
|
||||||
|
guitar
|
||||||
|
|
@ -0,0 +1,109 @@
|
||||||
|
# Copyright 2020 Huawei Technologies Co., Ltd
|
||||||
|
#
|
||||||
|
# Licensed under the Apache License, Version 2.0 (the "License");
|
||||||
|
# you may not use this file except in compliance with the License.
|
||||||
|
# You may obtain a copy of the License at
|
||||||
|
#
|
||||||
|
# http://www.apache.org/licenses/LICENSE-2.0
|
||||||
|
#
|
||||||
|
# Unless required by applicable law or agreed to in writing, software
|
||||||
|
# distributed under the License is distributed on an "AS IS" BASIS,
|
||||||
|
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||||
|
# See the License for the specific language governing permissions and
|
||||||
|
# limitations under the License.
|
||||||
|
# ============================================================================
|
||||||
|
'''
|
||||||
|
##############train models#################
|
||||||
|
python train.py
|
||||||
|
'''
|
||||||
|
import argparse
|
||||||
|
from mindspore import context, nn
|
||||||
|
from mindspore.train import Model
|
||||||
|
from mindspore.common import set_seed
|
||||||
|
from mindspore.train.loss_scale_manager import FixedLossScaleManager
|
||||||
|
from mindspore.train.serialization import load_checkpoint, load_param_into_net
|
||||||
|
from mindspore.train.callback import ModelCheckpoint, CheckpointConfig, LossMonitor, TimeMonitor
|
||||||
|
from src.dataset import create_dataset
|
||||||
|
from src.musictagger import MusicTaggerCNN
|
||||||
|
from src.loss import BCELoss
|
||||||
|
from src.config import music_cfg as cfg
|
||||||
|
|
||||||
|
def train(model, dataset_direct, filename, columns_list, num_consumer=4,
|
||||||
|
batch=16, epoch=50, save_checkpoint_steps=2172, keep_checkpoint_max=50,
|
||||||
|
prefix="model", directory='./'):
|
||||||
|
"""
|
||||||
|
train network
|
||||||
|
"""
|
||||||
|
config_ck = CheckpointConfig(save_checkpoint_steps=save_checkpoint_steps,
|
||||||
|
keep_checkpoint_max=keep_checkpoint_max)
|
||||||
|
ckpoint_cb = ModelCheckpoint(prefix=prefix,
|
||||||
|
directory=directory,
|
||||||
|
config=config_ck)
|
||||||
|
data_train = create_dataset(dataset_direct, filename, batch, columns_list,
|
||||||
|
num_consumer)
|
||||||
|
|
||||||
|
|
||||||
|
model.train(epoch,
|
||||||
|
data_train,
|
||||||
|
callbacks=[
|
||||||
|
ckpoint_cb,
|
||||||
|
LossMonitor(per_print_times=181),
|
||||||
|
TimeMonitor()
|
||||||
|
],
|
||||||
|
dataset_sink_mode=True)
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
set_seed(1)
|
||||||
|
parser = argparse.ArgumentParser(description='Train model')
|
||||||
|
parser.add_argument('--device_id',
|
||||||
|
type=int,
|
||||||
|
help='device ID',
|
||||||
|
default=None)
|
||||||
|
|
||||||
|
args = parser.parse_args()
|
||||||
|
|
||||||
|
if args.device_id is not None:
|
||||||
|
context.set_context(device_target='Ascend',
|
||||||
|
mode=context.GRAPH_MODE,
|
||||||
|
device_id=args.device_id)
|
||||||
|
else:
|
||||||
|
context.set_context(device_target='Ascend',
|
||||||
|
mode=context.GRAPH_MODE,
|
||||||
|
device_id=cfg.device_id)
|
||||||
|
|
||||||
|
context.set_context(enable_auto_mixed_precision=cfg.mixed_precision)
|
||||||
|
network = MusicTaggerCNN(in_classes=[1, 128, 384, 768, 2048],
|
||||||
|
kernel_size=[3, 3, 3, 3, 3],
|
||||||
|
padding=[0] * 5,
|
||||||
|
maxpool=[(2, 4), (4, 5), (3, 8), (4, 8)],
|
||||||
|
has_bias=True)
|
||||||
|
|
||||||
|
if cfg.pre_trained:
|
||||||
|
param_dict = load_checkpoint(cfg.checkpoint_path + '/' +
|
||||||
|
cfg.model_name)
|
||||||
|
load_param_into_net(network, param_dict)
|
||||||
|
|
||||||
|
net_loss = BCELoss()
|
||||||
|
|
||||||
|
network.set_train(True)
|
||||||
|
net_opt = nn.Adam(params=network.trainable_params(),
|
||||||
|
learning_rate=cfg.lr,
|
||||||
|
loss_scale=cfg.loss_scale)
|
||||||
|
|
||||||
|
loss_scale_manager = FixedLossScaleManager(loss_scale=cfg.loss_scale,
|
||||||
|
drop_overflow_update=False)
|
||||||
|
net_model = Model(network, net_loss, net_opt, loss_scale_manager=loss_scale_manager)
|
||||||
|
|
||||||
|
train(model=net_model,
|
||||||
|
dataset_direct=cfg.data_dir,
|
||||||
|
filename=cfg.train_filename,
|
||||||
|
columns_list=['feature', 'label'],
|
||||||
|
num_consumer=cfg.num_consumer,
|
||||||
|
batch=cfg.batch_size,
|
||||||
|
epoch=cfg.epoch_size,
|
||||||
|
save_checkpoint_steps=cfg.save_step,
|
||||||
|
keep_checkpoint_max=cfg.keep_checkpoint_max,
|
||||||
|
prefix=cfg.prefix,
|
||||||
|
directory=cfg.checkpoint_path + "_{}".format(cfg.device_id))
|
||||||
|
print("train success")
|
||||||
Loading…
Reference in New Issue