README.md 1.76 KB
Newer Older
qianyj's avatar
qianyj committed
1
# 简介
huchen's avatar
huchen committed
2

qianyj's avatar
qianyj committed
3
  该测试用例用于TensorFlow分类模型性能测试,使用的数据集是imagenet。
huchen's avatar
huchen committed
4

qianyj's avatar
qianyj committed
5
# 运行
6

qianyj's avatar
qianyj committed
7
## 单卡
qianyj's avatar
qianyj committed
8
9
10
11
    
    export PYTHONPATH=/path/to/tensorflow/model:$PYTHONPATH
    export HIP_VISIBLE_DEVICES=0
    python3 models-master/official/vision/image_classification/resnet/resnet_ctl_imagenet_main.py --data_dir=/path/to/{ImageNet-tensorflow_data_dir} --model_dir=/path/to/{model_save_dir} --batch_size=128 --num_gpus=1  --use_synthetic_data=false
huchen's avatar
huchen committed
12

qianyj's avatar
qianyj committed
13
## 单机多卡
qianyj's avatar
qianyj committed
14
15
16
17
    
    export PYTHONPATH=/path/to/tensorflow/model:$PYTHONPATH
    export HIP_VISIBLE_DEVICES=0,1,2,3
    python3 models-master/official/vision/image_classification/resnet/resnet_ctl_imagenet_main.py --data_dir=/path/to/{ImageNet-tensorflow_data_dir} --model_dir=/path/to/{model_save_dir} --batch_size=512 --num_gpus=4  --use_synthetic_data=false
huchen's avatar
huchen committed
18

qianyj's avatar
qianyj committed
19
## 分布式多卡
qianyj's avatar
qianyj committed
20
21
    
    # sed指令只需要执行一次,添加支持多卡运行的代码
qianyj's avatar
qianyj committed
22
    sed -i '101 r configfile' models-master/official/vision/image_classification/resnet/resnet_ctl_imagenet_main.py
qianyj's avatar
qianyj committed
23
24
25
26
27
28
29
30
31
    
    export PYTHONPATH=/path/to/tensorflow/model:$PYTHONPATH
    mpirun -np ${num_gpu} --hostfile hostfile  -mca btl self,tcp  --bind-to none scripts-run/single_process.sh

### 测试说明

   多卡测试时需要修改部分代码,具体可参考https://tensorflow.google.cn/guide/migrate/multi_worker_cpu_gpu_training?hl=en
   
   hostfile格式参考:
qianyj's avatar
qianyj committed
32
     
qianyj's avatar
qianyj committed
33
34
     node1 slots=4
     node2 slots=4
huchen's avatar
huchen committed
35

qianyj's avatar
qianyj committed
36
   nodefile格式参考(与hostfile文件内保持一致,用于构造TF_CONFIG):
qianyj's avatar
qianyj committed
37
     
qianyj's avatar
qianyj committed
38
39
     node1
     node2
40

qianyj's avatar
qianyj committed
41
   上面分布式多卡测试方法默认每节点4卡
42

huchen's avatar
huchen committed
43

44
# 参考资料
qianyj's avatar
qianyj committed
45
[https://github.com/tensorflow/benchmarks/tree/master/scripts/tf_cnn_benchmarks](https://github.com/tensorflow/benchmarks/tree/master/scripts/tf_cnn_benchmarks)