forked from mindspore-Ecosystem/mindspore
modify README
This commit is contained in:
parent
d429ea3f7d
commit
1c35880d55
|
@ -171,7 +171,7 @@ optional arguments:
|
|||
--device_target device where the code will be implemented: "Ascend", default is "Ascend"
|
||||
--data_dir DATA_DIR Train dataset directory.
|
||||
--per_batch_size PER_BATCH_SIZE
|
||||
Batch size for Training. Default: 32.
|
||||
Batch size for Training. Default: 8.
|
||||
--pretrained_backbone PRETRAINED_BACKBONE
|
||||
The ckpt file of CspDarkNet53. Default: "".
|
||||
--resume_yolov4 RESUME_YOLOV4
|
||||
|
@ -437,7 +437,7 @@ YOLOv4 on 20K images(The annotation and data format must be the same as coco tes
|
|||
# [Description of Random Situation](#contents)
|
||||
|
||||
In dataset.py, we set the seed inside ```create_dataset``` function.
|
||||
In var_init.py, we set seed for weight initilization
|
||||
In var_init.py, we set seed for weight initialization
|
||||
|
||||
# [ModelZoo Homepage](#contents)
|
||||
|
||||
|
|
|
@ -52,7 +52,7 @@ parser.add_argument('--device_target', type=str, default='Ascend',
|
|||
|
||||
# dataset related
|
||||
parser.add_argument('--data_dir', type=str, help='Train dataset directory.')
|
||||
parser.add_argument('--per_batch_size', default=8, type=int, help='Batch size for Training. Default: 32.')
|
||||
parser.add_argument('--per_batch_size', default=8, type=int, help='Batch size for Training. Default: 8.')
|
||||
|
||||
# network related
|
||||
parser.add_argument('--pretrained_backbone', default='', type=str,
|
||||
|
@ -125,7 +125,7 @@ if args.is_distributed:
|
|||
args.rank = get_rank()
|
||||
args.group_size = get_group_size()
|
||||
|
||||
# select for master rank save ckpt or all rank save, compatiable for model parallel
|
||||
# select for master rank save ckpt or all rank save, compatible for model parallel
|
||||
args.rank_save_ckpt_flag = 0
|
||||
if args.is_save_on_master:
|
||||
if args.rank == 0:
|
||||
|
|
|
@ -8,16 +8,15 @@
|
|||
- [Generate MindRecord](#generate-mindrecord)
|
||||
- [Create MindDataset By MindRecord](#create-minddataset-by-mindrecord)
|
||||
|
||||
|
||||
<!-- /TOC -->
|
||||
|
||||
## What does the example do
|
||||
|
||||
This example is used to read data from aclImdb dataset and generate mindrecord. It just transfers the aclImdb dataset to mindrecord without any data preprocessing. You can modify the example or follow the example to implement your own example.
|
||||
This example is used to read data from aclImdb dataset and generate mindrecord. It just transfers the aclImdb dataset to mindrecord without any data preprocessing. You can modify the example or refer to the example to implement your own example.
|
||||
|
||||
1. run.sh: generate MindRecord entry script.
|
||||
1. run.sh: generate MindRecord entry script.
|
||||
- gen_mindrecord.py : read the aclImdb data and transfer it to mindrecord.
|
||||
2. run_read.py: create MindDataset by MindRecord entry script.
|
||||
2. run_read.py: create MindDataset by MindRecord entry script.
|
||||
- create_dataset.py: use MindDataset to read MindRecord to generate dataset.
|
||||
|
||||
## How to use the example to generate MindRecord
|
||||
|
@ -30,6 +29,7 @@ Download aclImdb dataset, transfer it to mindrecord, use MindDataset to read min
|
|||
> [aclImdb dataset download address](http://ai.stanford.edu/~amaas/data/sentiment/) **-> Large Movie Review Dataset v1.0**
|
||||
|
||||
2. Unzip the training data to dir example/nlp_to_mindrecord/aclImdb/data.
|
||||
|
||||
```
|
||||
tar -zxvf aclImdb_v1.tar.gz -C {your-mindspore}/example/nlp_to_mindrecord/aclImdb/data/
|
||||
```
|
||||
|
@ -37,11 +37,13 @@ Download aclImdb dataset, transfer it to mindrecord, use MindDataset to read min
|
|||
### Generate MindRecord
|
||||
|
||||
1. Run the run.sh script.
|
||||
|
||||
```bash
|
||||
bash run.sh
|
||||
```
|
||||
|
||||
2. Output like this:
|
||||
|
||||
```
|
||||
...
|
||||
>> begin generate mindrecord by train data
|
||||
|
@ -77,6 +79,7 @@ Download aclImdb dataset, transfer it to mindrecord, use MindDataset to read min
|
|||
```
|
||||
|
||||
3. Generate mindrecord files
|
||||
|
||||
```
|
||||
$ ls output/
|
||||
aclImdb_test.mindrecord aclImdb_test.mindrecord.db aclImdb_train.mindrecord aclImdb_train.mindrecord.db README.md
|
||||
|
@ -85,12 +88,14 @@ Download aclImdb dataset, transfer it to mindrecord, use MindDataset to read min
|
|||
### Create MindDataset By MindRecord
|
||||
|
||||
1. Run the run_read.sh script.
|
||||
|
||||
```bash
|
||||
bash run_read.sh
|
||||
```
|
||||
|
||||
2. Output like this:
|
||||
> Caution: field "review" which is string type output is displayed in type uint8.
|
||||
|
||||
```
|
||||
...
|
||||
example 2056: {'label': array(1, dtype=int32), 'score': array(4, dtype=int32), 'id': array(5871, dtype=int32), 'review': array([ 70, 111, 114, ..., 111, 110, 46], dtype=uint8)}
|
||||
|
@ -98,7 +103,8 @@ Download aclImdb dataset, transfer it to mindrecord, use MindDataset to read min
|
|||
example 2058: {'label': array(1, dtype=int32), 'score': array(4, dtype=int32), 'id': array(1357, dtype=int32), 'review': array([ 42, 109, 97, ..., 58, 32, 67], dtype=uint8)}
|
||||
...
|
||||
```
|
||||
- id : the id "3219" is from review docs like **3219**_10.txt.
|
||||
- label : indicates whether the review is positive or negative, positive: 0, negative: 1.
|
||||
- score : the score "10" is from review docs like 3219_**10**.txt.
|
||||
- review : the review is from the review dos's content.
|
||||
|
||||
- id : the id "3219" is from review docs like **3219**_10.txt.
|
||||
- label : indicates whether the review is positive or negative, positive: 0, negative: 1.
|
||||
- score : the score "10" is from review docs like 3219_**10**.txt.
|
||||
- review : the review is from the review dos's content.
|
||||
|
|
Loading…
Reference in New Issue