# MobileNetV3 An implementation of MobileNetV3 with pyTorch # Theory  You can find the paper of MobileNetV3 at [Searching for MobileNetV3](https://arxiv.org/abs/1905.02244). # Prepare data * CIFAR-10 * CIFAR-100 * SVHN * Tiny-ImageNet * ImageNet: Please move validation images to labeled subfolders, you can use the script [here](https://raw.githubusercontent.com/soumith/imagenetloader.torch/master/valprep.sh). # Train * Train from scratch: ``` CUDA_VISIBLE_DEVICES=3 python train.py --batch-size=128 --mode=small \ --print-freq=100 --dataset=CIFAR100 --ema-decay=0 --label-smoothing=0.1 \ --lr=0.3 --save-epoch-freq=1000 --lr-decay=cos --lr-min=0 \ --warmup-epochs=5 --weight-decay=6e-5 --num-epochs=200 --width-multiplier=1 \ -nbd -zero-gamma -mixup ``` where the meaning of the parameters are as followed: ``` batch-size mode: using MobileNetV3-Small(if set to small) or MobileNetV3-Large(if set to large). dataset: which dataset to use(CIFAR10, CIFAR100, SVHN, TinyImageNet or ImageNet). ema-decay: decay of EMA, if set to 0, do not use EMA. label-smoothing: $epsilon$ using in label smoothing, if set to 0, do not use label smoothing. lr-decay: learning rate decay schedule, step or cos. lr-min: min lr in cos lr decay. warmup-epochs: warmup epochs using in cos lr deacy. num-epochs: total training epochs. nbd: no bias decay. zero-gamma: zero $gamma$ of last BN in each block. mixup: using Mixup. ``` # Pretrained models  We have provided the pretrained MobileNetV3-Small model in `pretrained`. # Experiments ## Training setting ### on ImageNet ``` CUDA_VISIBLE_DEVICES=5 python train.py --batch-size=128 --mode=small --print-freq=2000 --dataset=imagenet \ --ema-decay=0.99 --label-smoothing=0.1 --lr=0.1 --save-epoch-freq=50 --lr-decay=cos --lr-min=0 --warmup-epochs=5 \ --weight-decay=1e-5 --num-epochs=250 --num-workers=2 --width-multiplier=1 -dali -nbd -mixup -zero-gamma -save ``` ### on CIFAR-10 ``` CUDA_VISIBLE_DEVICES=1 python train.py --batch-size=128 --mode=small --print-freq=100 --dataset=CIFAR10\ --ema-decay=0 --label-smoothing=0 --lr=0.35 --save-epoch-freq=1000 --lr-decay=cos --lr-min=0\ --warmup-epochs=5 --weight-decay=6e-5 --num-epochs=400 --num-workers=2 --width-multiplier=1 ``` ### on CIFAR-100 ``` CUDA_VISIBLE_DEVICES=1 python train.py --batch-size=128 --mode=small --print-freq=100 --dataset=CIFAR100\ --ema-decay=0 --label-smoothing=0 --lr=0.35 --save-epoch-freq=1000 --lr-decay=cos --lr-min=0\ --warmup-epochs=5 --weight-decay=6e-5 --num-epochs=400 --num-workers=2 --width-multiplier=1 ```  Using more tricks： ``` CUDA_VISIBLE_DEVICES=1 python train.py --batch-size=128 --mode=small --print-freq=100 --dataset=CIFAR100\ --ema-decay=0.999 --label-smoothing=0.1 --lr=0.35 --save-epoch-freq=1000 --lr-decay=cos --lr-min=0\ --warmup-epochs=5 --weight-decay=6e-5 --num-epochs=400 --num-workers=2 --width-multiplier=1\ -zero-gamma -nbd -mixup ``` ### on SVHN ``` CUDA_VISIBLE_DEVICES=3 python train.py --batch-size=128 --mode=small --print-freq=1000 --dataset=SVHN\ --ema-decay=0 --label-smoothing=0 --lr=0.35 --save-epoch-freq=1000 --lr-decay=cos --lr-min=0\ --warmup-epochs=5 --weight-decay=6e-5 --num-epochs=20 --num-workers=2 --width-multiplier=1 ``` ### on Tiny-ImageNet ``` CUDA_VISIBLE_DEVICES=7 python train.py --batch-size=128 --mode=small --print-freq=100 --dataset=tinyimagenet\ --data-dir=/media/data2/chenjiarong/ImageData/tiny-imagenet --ema-decay=0 --label-smoothing=0 --lr=0.15\ --save-epoch-freq=1000 --lr-decay=cos --lr-min=0 --warmup-epochs=5 --weight-decay=6e-5 --num-epochs=200\ --num-workers=2 --width-multiplier=1 -dali ```  Using more tricks： ``` CUDA_VISIBLE_DEVICES=7 python train.py --batch-size=128 --mode=small --print-freq=100 --dataset=tinyimagenet\ --data-dir=/media/data2/chenjiarong/ImageData/tiny-imagenet --ema-decay=0.999 --label-smoothing=0.1 --lr=0.15\ --save-epoch-freq=1000 --lr-decay=cos --lr-min=0 --warmup-epochs=5 --weight-decay=6e-5 --num-epochs=200\ --num-workers=2 --width-multiplier=1 -dali -nbd -mixup ``` ## MobileNetV3-Large ### on ImageNet | | Madds | Parameters | Top1-acc | Top5-acc | | ----------- | --------- | ---------- | --------- | --------- | | Offical 1.0 | 219 M | 5.4 M | 75.2% | - | | Ours 1.0 | 216.6 M | 5.47 M | - | - | ### on CIFAR-10 | | Madds | Parameters | Top1-acc | Top5-acc | | ----------- | --------- | ---------- | --------- | --------- | | Ours 1.0 | 66.47 M | 4.21 M | - | - | ### on CIFAR-100 | | Madds | Parameters | Top1-acc | Top5-acc | | ----------- | --------- | ---------- | --------- | --------- | | Ours 1.0 | 66.58 M | 4.32 M | - | - | ## MobileNetV3-Small ### on ImageNet | | Madds | Parameters | Top1-acc | Top5-acc | | ----------- | --------- | ---------- | --------- | --------- | | Offical 1.0 | 56.5 M | 2.53 M | 67.4% | - | | Ours 1.0 | 56.51 M | 2.53 M | 67.52% | 87.58% |  The pretrained model with top-1 accuracy 67.52% is provided in the folder [pretrained](https://github.com/ShowLo/MobileNetV3/tree/master/pretrained). ### on CIFAR-10 (Average accuracy of 5 runs) | | Madds | Parameters | Top1-acc | Top5-acc | | ----------- | --------- | ---------- | --------- | --------- | | Ours 1.0 | 17.51 M | 1.52 M | 92.97% | - | ### on CIFAR-100 (Average accuracy of 5 runs) | | Madds | Parameters | Top1-acc | Top5-acc | | ----------- | --------- | ---------- | --------- | --------- | | Ours 1.0 | 17.60 M | 1.61 M | 73.69% | 92.31% | | More Tricks | same | same | 76.24% | 92.58% | ### on SVHN (Average accuracy of 5 runs) | | Madds | Parameters | Top1-acc | Top5-acc | | ----------- | --------- | ---------- | --------- | --------- | | Ours 1.0 | 17.51 M | 1.52 M | 97.92% | - | ### on Tiny-ImageNet (Average accuracy of 5 runs) | | Madds | Parameters | Top1-acc | Top5-acc | | ----------- | --------- | ---------- | --------- | --------- | | Ours 1.0 | 51.63 M | 1.71 M | 59.32% | 81.38% | | More Tricks | same | same | 62.62% | 84.04% | ## Dependency  This project uses Python 3.7 and PyTorch 1.1.0. The FLOPs and Parameters and measured using [torchsummaryX](https://github.com/nmhkahn/torchsummaryX).