mindspore/model_zoo/official/cv/alexnet

Contents

• AlexNet Description
• Model Architecture
• Dataset
• Environment Requirements
• Quick Start
• Script Description
  • Script and Sample Code
  • Script Parameters
  • Training Process
    • Training
  • Evaluation Process
    • Evaluation
• Model Description
  • Performance
    • Evaluation Performance
• ModelZoo Homepage

AlexNet Description

AlexNet was proposed in 2012, one of the most influential neural networks. It got big success in ImageNet Dataset recognition than other models.

Paper: Krizhevsky A, Sutskever I, Hinton G E. ImageNet Classification with Deep ConvolutionalNeural Networks. Advances In Neural Information Processing Systems. 2012.

Model Architecture

AlexNet composition consists of 5 convolutional layers and 3 fully connected layers. Multiple convolutional kernels can extract interesting features in images and get more accurate classification.

Dataset

Note that you can run the scripts based on the dataset mentioned in original paper or widely used in relevant domain/network architecture. In the following sections, we will introduce how to run the scripts using the related dataset below.

Dataset used: CIFAR-10

• Dataset size：175M，60,000 32*32 colorful images in 10 classes
  • Train：146M，50,000 images
  • Test：29.3M，10,000 images
• Data format：binary files
  • Note：Data will be processed in dataset.py
• Download the dataset, the directory structure is as follows:

├─cifar-10-batches-bin
│
└─cifar-10-verify-bin

Environment Requirements

• Hardware（Ascend/GPU）
  • Prepare hardware environment with Ascend or GPU processor.
• Framework
  • MindSpore
• For more information, please check the resources below：
  • MindSpore Tutorials
  • MindSpore Python API

Quick Start

After installing MindSpore via the official website, you can start training and evaluation as follows:

# enter script dir, train AlexNet
sh run_standalone_train_ascend.sh [DATA_PATH] [CKPT_SAVE_PATH]
# enter script dir, evaluate AlexNet
sh run_standalone_eval_ascend.sh [DATA_PATH] [CKPT_NAME]

Script Description

Script and Sample Code

├── cv
    ├── alexnet
        ├── README.md                    // descriptions about alexnet
        ├── requirements.txt             // package needed
        ├── scripts
        │   ├──run_standalone_train_gpu.sh             // train in gpu
        │   ├──run_standalone_train_ascend.sh          // train in ascend
        │   ├──run_standalone_eval_gpu.sh             //  evaluate in gpu
        │   ├──run_standalone_eval_ascend.sh          //  evaluate in ascend
        ├── src
        │   ├──dataset.py             // creating dataset
        │   ├──alexnet.py              // alexnet architecture
        │   └──model_utils
        │       ├──config.py               // Processing configuration parameters
        │       ├──device_adapter.py       // Get cloud ID
        │       ├──local_adapter.py        // Get local ID
        │       └──moxing_adapter.py       // Parameter processing
        ├── default_config.yaml            // Training parameter profile(cifar10)
        ├── config_imagenet.yaml           // Training parameter profile(imagenet)
        ├── train.py               // training script
        ├── eval.py                // evaluation script

Script Parameters

Major parameters in train.py and config.py as follows:

--data_path: The absolute full path to the train and evaluation datasets.
--epoch_size: Total training epochs.
--batch_size: Training batch size.
--image_height: Image height used as input to the model.
--image_width: Image width used as input the model.
--device_target: Device where the code will be implemented. Optional values are "Ascend", "GPU".
--checkpoint_path: The absolute full path to the checkpoint file saved after training.
--data_path: Path where the dataset is saved

Training Process

Training

• Running on Ascend

  python train.py --config_path default_config.yaml --data_path cifar-10-batches-bin --ckpt_path ckpt > log 2>&1 &
  # or enter script dir, and run the script
  sh run_standalone_train_ascend.sh cifar-10-batches-bin ckpt
  

  After training, the loss value will be achieved as follows:

  # grep "loss is " log
  epoch: 1 step: 1, loss is 2.2791853
  ...
  epoch: 1 step: 1536, loss is 1.9366643
  epoch: 1 step: 1537, loss is 1.6983616
  epoch: 1 step: 1538, loss is 1.0221305
  ...
  

  The model checkpoint will be saved in the current directory.

• running on GPU

  python train.py --config_path default_config.yaml --device_target "GPU" --data_path cifar-10-batches-bin --ckpt_path ckpt > log 2>&1 &
  # or enter script dir, and run the script
  sh run_standalone_train_for_gpu.sh cifar-10-batches-bin ckpt
  

  After training, the loss value will be achieved as follows:

  # grep "loss is " log
  epoch: 1 step: 1, loss is 2.3125906
  ...
  epoch: 30 step: 1560, loss is 0.6687547
  epoch: 30 step: 1561, loss is 0.20055409
  epoch: 30 step: 1561, loss is 0.103845775
  

Evaluation Process

Evaluation

Before running the command below, please check the checkpoint path used for evaluation.

• running on Ascend

  python eval.py --config_path default_config.yaml --data_path cifar-10-verify-bin --ckpt_path ckpt/checkpoint_alexnet-1_1562.ckpt > eval_log.txt 2>&1 &
  # or enter script dir, and run the script
  sh run_standalone_eval_ascend.sh cifar-10-verify-bin ckpt/checkpoint_alexnet-1_1562.ckpt
  

  You can view the results through the file "eval_log". The accuracy of the test dataset will be as follows:

  # grep "Accuracy: " eval_log
  'Accuracy': 0.8832
  

• running on GPU

  python eval.py --config_path default_config.yaml --device_target "GPU" --data_path cifar-10-verify-bin --ckpt_path ckpt/checkpoint_alexnet-30_1562.ckpt > eval_log 2>&1 &
  # or enter script dir, and run the script
  sh run_standalone_eval_for_gpu.sh cifar-10-verify-bin ckpt/checkpoint_alexnet-30_1562.ckpt
  

  You can view the results through the file "eval_log". The accuracy of the test dataset will be as follows:

  # grep "Accuracy: " eval_log
  'Accuracy': 0.88512
  

Inference Process

Export MindIR

python export.py --config_path [CONFIG_PATH] --ckpt_file [CKPT_PATH] --file_name [FILE_NAME] --file_format [FILE_FORMAT]

The ckpt_file parameter is required, EXPORT_FORMAT should be in ["AIR", "MINDIR"]

Infer on Ascend310

Before performing inference, the mindir file must be exported by export.py script. We only provide an example of inference using MINDIR model. Current batch_Size for imagenet2012 dataset can only be set to 1.

# Ascend310 inference
bash run_infer_310.sh [MINDIR_PATH] [DATASET_PATH] [NEED_PREPROCESS] [DEVICE_ID]

• MINDIR_PATH specifies path of used "MINDIR" OR "AIR" model.
• DATASET_PATH specifies path of cifar10 datasets
• NEED_PREPROCESS means weather need preprocess or not, it's value is 'y' or 'n', if you choose y, the cifar10 dataset will be processed in bin format, the imagenet2012 dataset will generate label json file.
• DEVICE_ID is optional, default value is 0.

result

Inference result is saved in current path, you can find result like this in acc.log file.

'acc': 0.88772

• Running on ModelArts

  # Train 8p with Ascend
  # (1) Perform a or b.
  #       a. Set "enable_modelarts=True" on default_config.yaml file.
  #          Set "distribute=True" on default_config.yaml file.
  #          Set "data_path='/cache/data'" on default_config.yaml file.
  #          Set "ckpt_path='/cache/train'" on default_config.yaml file.
  #          (optional)Set "checkpoint_url='s3://dir_to_your_pretrained/'" on default_config.yaml file.
  #          Set other parameters on default_config.yaml file you need.
  #       b. Add "enable_modelarts=True" on the website UI interface.
  #          Add "distribute=True" on the website UI interface.
  #          Add "data_path=/cache/data" on the website UI interface.
  #          Add "ckpt_path=/cache/train" on the website UI interface.
  #          (optional)Add "checkpoint_url='s3://dir_to_your_pretrained/'" on the website UI interface.
  #          Add other parameters on the website UI interface.
  # (2) Prepare model code
  # (3) Upload or copy your pretrained model to S3 bucket if you want to finetune.
  # (4) Upload the original cifar10 dataset to S3 bucket.
  # (5) Set the code directory to "/path/alexnet" on the website UI interface.
  # (6) Set the startup file to "train.py" on the website UI interface.
  # (7) Set the "Dataset path" and "Output file path" and "Job log path" to your path on the website UI interface.
  # (8) Create your job.
  #
  # Train 1p with Ascend
  # (1) Perform a or b.
  #       a. Set "enable_modelarts=True" on default_config.yaml file.
  #          Set "data_path='/cache/data'" on default_config.yaml file.
  #          Set "ckpt_path='/cache/train'" on default_config.yaml file.
  #          (optional)Set "checkpoint_url='s3://dir_to_your_pretrained/'" on default_config.yaml file.
  #          Set other parameters on default_config.yaml file you need.
  #       b. Add "enable_modelarts=True" on the website UI interface.
  #          Add "data_path=/cache/data" on the website UI interface.
  #          Add "ckpt_path=/cache/train" on the website UI interface.
  #          (optional)Add "checkpoint_url='s3://dir_to_your_pretrained/'" on the website UI interface.
  #          Add other parameters on the website UI interface.
  # (2) Prepare model code
  # (3) Upload or copy your pretrained model to S3 bucket if you want to finetune.
  # (4) Upload the original cifar10 dataset to S3 bucket.
  # (5) Set the code directory to "/path/alexnet" on the website UI interface.
  # (6) Set the startup file to "train.py" on the website UI interface.
  # (7) Set the "Dataset path" and "Output file path" and "Job log path" to your path on the website UI interface.
  # (8) Create your job.
  #
  # Eval 1p with Ascend
  # (1) Perform a or b.
  #       a. Set "enable_modelarts=True" on default_config.yaml file.
  #          Set "data_path='/cache/data'" on default_config.yaml file.
  #          Set "ckpt_file='/cache/train/checkpoint_alexnet-30_1562.ckpt'" on default_config.yaml file.
  #          Set other parameters on default_config.yaml file you need.
  #       b. Add "enable_modelarts=True" on the website UI interface.
  #          Add "data_path=/cache/data" on the website UI interface.
  #          Add "ckpt_file=/cache/train/checkpoint_alexnet-30_1562.ckpt" on the website UI interface.
  #          Add other parameters on the website UI interface.
  # (2) Prepare model code
  # (3) Upload or copy your trained model to S3 bucket.
  # (4) Upload the original cifar10 dataset to S3 bucket.
  # (5) Set the code directory to "/path/alexnet" on the website UI interface.
  # (6) Set the startup file to "eval.py" on the website UI interface.
  # (7) Set the "Dataset path" and "Output file path" and "Job log path" to your path on the website UI interface.
  # (8) Create your job.
  

Model Description

Performance

Evaluation Performance

Parameters	Ascend	GPU
Resource	Ascend 910; CPU 2.60GHz, 192cores; Memory 755G; OS Euler2.8	NV SMX2 V100-32G
uploaded Date	06/09/2020 (month/day/year)	17/09/2020 (month/day/year)
MindSpore Version	1.0.0	0.7.0-beta
Dataset	CIFAR-10	CIFAR-10
Training Parameters	epoch=30, steps=1562, batch_size = 32, lr=0.002	epoch=30, steps=1562, batch_size = 32, lr=0.002
Optimizer	Momentum	Momentum
Loss Function	Softmax Cross Entropy	Softmax Cross Entropy
outputs	probability	probability
Loss	0.08	0.01
Speed	7.3 ms/step	16.8 ms/step
Total time	6 mins	14 mins
Checkpoint for Fine tuning	445M (.ckpt file)	445M (.ckpt file)
Scripts	AlexNet Script	AlexNet Script

Description of Random Situation

In dataset.py, we set the seed inside create_dataset function.

ModelZoo Homepage

Please check the official homepage.