Access code for `baidu` is `swin`.

## ImageNet-1K and ImageNet-22K Pretrained Swin-V1 Models

| name | pretrain | resolution |acc@1 | acc@5 | #params | FLOPs | FPS| 22K model | 1K model |
| :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: |:---: |:---: |
| Swin-T | ImageNet-1K | 224x224 | 81.2 | 95.5 | 28M | 4.5G | 755 | - | [github](https://github.com/SwinTransformer/storage/releases/download/v1.0.0/swin_tiny_patch4_window7_224.pth)/[baidu](https://pan.baidu.com/s/156nWJy4Q28rDlrX-rRbI3w)/[config](configs/swin/swin_tiny_patch4_window7_224.yaml)/[log](https://github.com/SwinTransformer/storage/files/7745562/log_swin_tiny_patch4_window7_224.txt) |
| Swin-S | ImageNet-1K | 224x224 | 83.2 | 96.2 | 50M | 8.7G | 437 | - | [github](https://github.com/SwinTransformer/storage/releases/download/v1.0.0/swin_small_patch4_window7_224.pth)/[baidu](https://pan.baidu.com/s/1KFjpj3Efey3LmtE1QqPeQg)/[config](configs/swin/swin_small_patch4_window7_224.yaml)/[log](https://github.com/SwinTransformer/storage/files/7745563/log_swin_small_patch4_window7_224.txt) |
| Swin-B | ImageNet-1K | 224x224 | 83.5 | 96.5 | 88M | 15.4G | 278  | - | [github](https://github.com/SwinTransformer/storage/releases/download/v1.0.0/swin_base_patch4_window7_224.pth)/[baidu](https://pan.baidu.com/s/16bqCTEc70nC_isSsgBSaqQ)/[config](configs/swin/swin_base_patch4_window7_224.yaml)/[log](https://github.com/SwinTransformer/storage/files/7745564/log_swin_base_patch4_window7_224.txt) |
| Swin-B | ImageNet-1K | 384x384 | 84.5 | 97.0 | 88M | 47.1G | 85 | - | [github](https://github.com/SwinTransformer/storage/releases/download/v1.0.0/swin_base_patch4_window12_384.pth)/[baidu](https://pan.baidu.com/s/1xT1cu740-ejW7htUdVLnmw)/[config](configs/swin/swin_base_patch4_window12_384_finetune.yaml) |
| Swin-T | ImageNet-22K | 224x224 | 80.9 | 96.0 | 28M | 4.5G | 755 | [github](https://github.com/SwinTransformer/storage/releases/download/v1.0.8/swin_tiny_patch4_window7_224_22k.pth)/[baidu](https://pan.baidu.com/s/1vct0VYwwQQ8PYkBjwSSBZQ?pwd=swin)/[config](configs/swin/swin_tiny_patch4_window7_224_22k.yaml) | [github](https://github.com/SwinTransformer/storage/releases/download/v1.0.8/swin_tiny_patch4_window7_224_22kto1k_finetune.pth)/[baidu](https://pan.baidu.com/s/1K0OO-nGZDPkR8fm_r83e8Q?pwd=swin)/[config](configs/swin/swin_tiny_patch4_window7_224_22kto1k_finetune.yaml) |
| Swin-S | ImageNet-22K | 224x224 | 83.2 | 97.0 | 50M | 8.7G | 437 | [github](https://github.com/SwinTransformer/storage/releases/download/v1.0.8/swin_small_patch4_window7_224_22k.pth)/[baidu](https://pan.baidu.com/s/11NC1xdT5BAGBgazdTme5Sg?pwd=swin)/[config](configs/swin/swin_small_patch4_window7_224_22k.yaml) | [github](https://github.com/SwinTransformer/storage/releases/download/v1.0.8/swin_small_patch4_window7_224_22kto1k_finetune.pth)/[baidu](https://pan.baidu.com/s/10RFVfjQJhwPfeHrmxQUaLw?pwd=swin)/[config](configs/swin/swin_small_patch4_window7_224_22kto1k_finetune.yaml) |
| Swin-B | ImageNet-22K | 224x224 | 85.2 | 97.5 | 88M | 15.4G | 278 | [github](https://github.com/SwinTransformer/storage/releases/download/v1.0.0/swin_base_patch4_window7_224_22k.pth)/[baidu](https://pan.baidu.com/s/1y1Ec3UlrKSI8IMtEs-oBXA)/[config](configs/swin/swin_base_patch4_window7_224_22k.yaml) | [github](https://github.com/SwinTransformer/storage/releases/download/v1.0.0/swin_base_patch4_window7_224_22kto1k.pth)/[baidu](https://pan.baidu.com/s/1n_wNkcbRxVXit8r_KrfAVg)/[config](configs/swin/swin_base_patch4_window7_224_22kto1k_finetune.yaml) |
| Swin-B | ImageNet-22K | 384x384 | 86.4 | 98.0 | 88M | 47.1G | 85 | [github](https://github.com/SwinTransformer/storage/releases/download/v1.0.0/swin_base_patch4_window12_384_22k.pth)/[baidu](https://pan.baidu.com/s/1vwJxnJcVqcLZAw9HaqiR6g) | [github](https://github.com/SwinTransformer/storage/releases/download/v1.0.0/swin_base_patch4_window12_384_22kto1k.pth)/[baidu](https://pan.baidu.com/s/1caKTSdoLJYoi4WBcnmWuWg)/[config](configs/swin/swin_base_patch4_window12_384_22kto1k_finetune.yaml) |
| Swin-L | ImageNet-22K | 224x224 | 86.3 | 97.9 | 197M | 34.5G | 141 | [github](https://github.com/SwinTransformer/storage/releases/download/v1.0.0/swin_large_patch4_window7_224_22k.pth)/[baidu](https://pan.baidu.com/s/1pws3rOTFuOebBYP3h6Kx8w)/[config](configs/swin/swin_large_patch4_window7_224_22k.yaml) | [github](https://github.com/SwinTransformer/storage/releases/download/v1.0.0/swin_large_patch4_window7_224_22kto1k.pth)/[baidu](https://pan.baidu.com/s/1NkQApMWUhxBGjk1ne6VqBQ)/[config](configs/swin/swin_large_patch4_window7_224_22kto1k_finetune.yaml) |
| Swin-L | ImageNet-22K | 384x384 | 87.3 | 98.2 | 197M | 103.9G | 42 | [github](https://github.com/SwinTransformer/storage/releases/download/v1.0.0/swin_large_patch4_window12_384_22k.pth)/[baidu](https://pan.baidu.com/s/1sl7o_bJA143OD7UqSLAMoA) | [github](https://github.com/SwinTransformer/storage/releases/download/v1.0.0/swin_large_patch4_window12_384_22kto1k.pth)/[baidu](https://pan.baidu.com/s/1X0FLHQyPOC6Kmv2CmgxJvA)/[config](configs/swin/swin_large_patch4_window12_384_22kto1k_finetune.yaml) |

## ImageNet-1K and ImageNet-22K Pretrained Swin-V2 Models

| name | pretrain | resolution | window |acc@1 | acc@5 | #params | FLOPs | FPS |22K model | 1K model |
|:---------------------:| :---: | :---: | :---: | :---: | :---: | :---: | :---: |:---:|:---: |:---: |
| SwinV2-T | ImageNet-1K | 256x256 | 8x8 | 81.8 | 95.9 | 28M | 5.9G | 572 | - | [github](https://github.com/SwinTransformer/storage/releases/download/v2.0.0/swinv2_tiny_patch4_window8_256.pth)/[baidu](https://pan.baidu.com/s/1RzLkAH_5OtfRCJe6Vlg6rg?pwd=swin)/[config](configs/swinv2/swinv2_tiny_patch4_window8_256.yaml) |
| SwinV2-S | ImageNet-1K | 256x256 | 8x8 | 83.7 | 96.6 | 50M | 11.5G | 327 | - | [github](https://github.com/SwinTransformer/storage/releases/download/v2.0.0/swinv2_small_patch4_window8_256.pth)/[baidu](https://pan.baidu.com/s/195PdA41szEduW3jEtRSa4Q?pwd=swin)/[config](configs/swinv2/swinv2_small_patch4_window8_256.yaml) |
| SwinV2-B | ImageNet-1K | 256x256 | 8x8 | 84.2 | 96.9 | 88M | 20.3G | 217 | - | [github](https://github.com/SwinTransformer/storage/releases/download/v2.0.0/swinv2_base_patch4_window8_256.pth)/[baidu](https://pan.baidu.com/s/18AfMSz3dPyzIvP1dKuERvQ?pwd=swin)/[config](configs/swinv2/swinv2_base_patch4_window8_256.yaml) |
| SwinV2-T | ImageNet-1K | 256x256 | 16x16 | 82.8 | 96.2 | 28M | 6.6G | 437 | - | [github](https://github.com/SwinTransformer/storage/releases/download/v2.0.0/swinv2_tiny_patch4_window16_256.pth)/[baidu](https://pan.baidu.com/s/1dyK3cK9Xipmv6RnTtrPocw?pwd=swin)/[config](configs/swinv2/swinv2_tiny_patch4_window16_256.yaml) |
| SwinV2-S | ImageNet-1K | 256x256 | 16x16 | 84.1 | 96.8 | 50M | 12.6G  | 257 | - | [github](https://github.com/SwinTransformer/storage/releases/download/v2.0.0/swinv2_small_patch4_window16_256.pth)/[baidu](https://pan.baidu.com/s/1ZIPiSfWNKTPp821Ka-Mifw?pwd=swin)/[config](configs/swinv2/swinv2_small_patch4_window16_256.yaml) |
| SwinV2-B | ImageNet-1K | 256x256 | 16x16 | 84.6 | 97.0 | 88M | 21.8G | 174 | - | [github](https://github.com/SwinTransformer/storage/releases/download/v2.0.0/swinv2_base_patch4_window16_256.pth)/[baidu](https://pan.baidu.com/s/1dlDQGn8BXCmnh7wQSM5Nhw?pwd=swin)/[config](configs/swinv2/swinv2_base_patch4_window16_256.yaml) |
| SwinV2-B<sup>\*</sup> | ImageNet-22K | 256x256 | 16x16 | 86.2 | 97.9 |  88M | 21.8G | 174 | [github](https://github.com/SwinTransformer/storage/releases/download/v2.0.0/swinv2_base_patch4_window12_192_22k.pth)/[baidu](https://pan.baidu.com/s/1Xc2rsSsRQz_sy5mjgfxrMQ?pwd=swin)/[config](configs/swinv2/swinv2_base_patch4_window12_192_22k.yaml) | [github](https://github.com/SwinTransformer/storage/releases/download/v2.0.0/swinv2_base_patch4_window12to16_192to256_22kto1k_ft.pth)/[baidu](https://pan.baidu.com/s/1sgstld4MgGsZxhUAW7MlmQ?pwd=swin)/[config](configs/swinv2/swinv2_base_patch4_window12to16_192to256_22kto1k_ft.yaml) |
| SwinV2-B<sup>\*</sup> | ImageNet-22K | 384x384 | 24x24 | 87.1 | 98.2 | 88M | 54.7G | 57  | [github](https://github.com/SwinTransformer/storage/releases/download/v2.0.0/swinv2_base_patch4_window12_192_22k.pth)/[baidu](https://pan.baidu.com/s/1Xc2rsSsRQz_sy5mjgfxrMQ?pwd=swin)/[config](configs/swinv2/swinv2_base_patch4_window12_192_22k.yaml) | [github](https://github.com/SwinTransformer/storage/releases/download/v2.0.0/swinv2_base_patch4_window12to24_192to384_22kto1k_ft.pth)/[baidu](https://pan.baidu.com/s/17u3sEQaUYlvfL195rrORzQ?pwd=swin)/[config](configs/swinv2/swinv2_base_patch4_window12to24_192to384_22kto1k_ft.yaml) |
| SwinV2-L<sup>\*</sup> | ImageNet-22K | 256x256 | 16x16 | 86.9 | 98.0 | 197M | 47.5G | 95  | [github](https://github.com/SwinTransformer/storage/releases/download/v2.0.0/swinv2_large_patch4_window12_192_22k.pth)/[baidu](https://pan.baidu.com/s/11PhCV7qAGXtZ8dXNgyiGOw?pwd=swin)/[config](configs/swinv2/swinv2_large_patch4_window12_192_22k.yaml) | [github](https://github.com/SwinTransformer/storage/releases/download/v2.0.0/swinv2_large_patch4_window12to16_192to256_22kto1k_ft.pth)/[baidu](https://pan.baidu.com/s/1pqp31N80qIWjFPbudzB6Bw?pwd=swin)/[config](configs/swinv2/swinv2_large_patch4_window12to16_192to256_22kto1k_ft.yaml) |
| SwinV2-L<sup>\*</sup> | ImageNet-22K | 384x384 | 24x24 | 87.6 | 98.3 | 197M | 115.4G | 33  | [github](https://github.com/SwinTransformer/storage/releases/download/v2.0.0/swinv2_large_patch4_window12_192_22k.pth)/[baidu](https://pan.baidu.com/s/11PhCV7qAGXtZ8dXNgyiGOw?pwd=swin)/[config](configs/swinv2/swinv2_large_patch4_window12_192_22k.yaml) | [github](https://github.com/SwinTransformer/storage/releases/download/v2.0.0/swinv2_large_patch4_window12to24_192to384_22kto1k_ft.pth)/[baidu](https://pan.baidu.com/s/13URdNkygr3Xn0N3e6IwjgA?pwd=swin)/[config](configs/swinv2/swinv2_large_patch4_window12to24_192to384_22kto1k_ft.yaml) |

Note:

- SwinV2-B<sup>\*</sup>  (SwinV2-L<sup>\*</sup>) with input resolution of 256x256 and 384x384 both fine-tuned from the
  same pre-training model using a smaller input resolution of 192x192.
- SwinV2-B<sup>\*</sup> (384x384) achieves 78.08 acc@1 on ImageNet-1K-V2 while SwinV2-L<sup>\*</sup> (384x384) achieves
  78.31.

## ImageNet-1K Pretrained Swin MLP Models

| name | pretrain | resolution |acc@1 | acc@5 | #params | FLOPs | FPS |  1K model |
| :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: |
| [Mixer-B/16](https://arxiv.org/pdf/2105.01601.pdf) | ImageNet-1K | 224x224 | 76.4 | - | 59M | 12.7G | - | [official repo](https://github.com/google-research/vision_transformer) |
| [ResMLP-S24](https://arxiv.org/abs/2105.03404) | ImageNet-1K | 224x224 | 79.4 | - | 30M | 6.0G | 715 | [timm](https://github.com/rwightman/pytorch-image-models) |
| [ResMLP-B24](https://arxiv.org/abs/2105.03404) | ImageNet-1K | 224x224 | 81.0 | - | 116M | 23.0G |  231 | [timm](https://github.com/rwightman/pytorch-image-models) |
| Swin-T/C24 | ImageNet-1K | 256x256 | 81.6 | 95.7 | 28M | 5.9G | 563 | [github](https://github.com/SwinTransformer/storage/releases/download/v1.0.5/swin_tiny_c24_patch4_window8_256.pth)/[baidu](https://pan.baidu.com/s/17k-7l6Sxt7uZ7IV0f26GNQ)/[config](configs/swin/swin_tiny_c24_patch4_window8_256.yaml) |
| SwinMLP-T/C24 | ImageNet-1K | 256x256 | 79.4 | 94.6 | 20M | 4.0G | 807 | [github](https://github.com/SwinTransformer/storage/releases/download/v1.0.5/swin_mlp_tiny_c24_patch4_window8_256.pth)/[baidu](https://pan.baidu.com/s/1Sa4vP5R0M2RjfIe9HIga-Q)/[config](configs/swin/swin_mlp_tiny_c24_patch4_window8_256.yaml) |
| SwinMLP-T/C12 | ImageNet-1K | 256x256 | 79.6 | 94.7 | 21M | 4.0G | 792 | [github](https://github.com/SwinTransformer/storage/releases/download/v1.0.5/swin_mlp_tiny_c12_patch4_window8_256.pth)/[baidu](https://pan.baidu.com/s/1mM9J2_DEVZHUB5ASIpFl0w)/[config](configs/swin/swin_mlp_tiny_c12_patch4_window8_256.yaml) |
| SwinMLP-T/C6 | ImageNet-1K | 256x256 | 79.7 | 94.9 | 23M | 4.0G | 766 | [github](https://github.com/SwinTransformer/storage/releases/download/v1.0.5/swin_mlp_tiny_c6_patch4_window8_256.pth)/[baidu](https://pan.baidu.com/s/1hUTYVT2W1CsjICw-3W-Vjg)/[config](configs/swin/swin_mlp_tiny_c6_patch4_window8_256.yaml) |
| SwinMLP-B | ImageNet-1K | 224x224 | 81.3 | 95.3 | 61M | 10.4G | 409 | [github](https://github.com/SwinTransformer/storage/releases/download/v1.0.5/swin_mlp_base_patch4_window7_224.pth)/[baidu](https://pan.baidu.com/s/1zww3dnbX3GxNiGfb-GwyUg)/[config](configs/swin/swin_mlp_base_patch4_window7_224.yaml) |

Note: C24 means each head has 24 channels.

## ImageNet-22K Pretrained Swin-MoE Models

| name | #experts | k | router | resolution | window | IN-22K acc@1 | IN-1K/ft acc@1 | IN-1K/5-shot acc@1 | 22K model |
| :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: |
| Swin-MoE-S | 1 (dense) | - | - | 192x192 | 8x8 | 35.5| 83.5 | 70.3 | [github](https://github.com/SwinTransformer/storage/releases/download/v2.0.2/swin_moe_small_patch4_window12_192_densebaseline_22k.zip)/[baidu](https://pan.baidu.com/s/1O1m9jT2pGoago_RiRX914w?pwd=swin)/[config](configs/swinmoe/swin_moe_small_patch4_window12_192_densebaseline_22k.yaml) |
| Swin-MoE-S | 8 | 1 | Linear | 192x192 | 8x8 | 36.8 | 84.5 | 75.2 | [github](https://github.com/SwinTransformer/storage/releases/download/v2.0.2/swin_moe_small_patch4_window12_192_8expert_32gpu_22k.zip)/[baidu](https://pan.baidu.com/s/198IlYUrWOxEUp7wNdoJT5Q?pwd=swin)/[config](configs/swinmoe/swin_moe_small_patch4_window12_192_8expert_32gpu_22k.yaml) |
| Swin-MoE-S | 16 | 1 | Linear |192x192 | 8x8 | 37.6 | 84.9 | 76.5 | [github](https://github.com/SwinTransformer/storage/releases/download/v2.0.2/swin_moe_small_patch4_window12_192_16expert_32gpu_22k.zip)/[baidu](https://pan.baidu.com/s/1vRQweedtT42VwMTqe9-r2A?pwd=swin)/[config](configs/swinmoe/swin_moe_small_patch4_window12_192_16expert_32gpu_22k.yaml) |
| Swin-MoE-S | 32 | 1 | Linear | 192x192 | 8x8 | 37.4 | 84.7 | 75.9 | [github](https://github.com/SwinTransformer/storage/releases/download/v2.0.2/swin_moe_small_patch4_window12_192_32expert_32gpu_22k.zip)/[baidu](https://pan.baidu.com/s/1i7rImt5pwO8gJC-PRRuZwQ?pwd=swin)/[config](configs/swinmoe/swin_moe_small_patch4_window12_192_32expert_32gpu_22k.yaml) |
| Swin-MoE-S | 32 | 1 | Cosine | 192x192 | 8x8 | 37.2 | 84.3 | 75.2 | [github](https://github.com/SwinTransformer/storage/releases/download/v2.0.2/swin_moe_small_patch4_window12_192_cosine_router_32expert_32gpu_22k.zip)/[baidu](https://pan.baidu.com/s/1Yghr_12ntSrv01I9yatPDQ?pwd=swin)/[config](configs/swinmoe/swin_moe_small_patch4_window12_192_cosine_router_32expert_32gpu_22k.yaml) |
| Swin-MoE-S | 64 | 1 | Linear | 192x192 | 8x8 | 37.8 | 84.7 | 75.7 | - |
| Swin-MoE-S | 128 | 1 | Linear | 192x192 | 8x8 | 37.4 | 84.5 | 75.4 | - |
| Swin-MoE-B | 1 (dense) | - | - | 192x192 | 8x8 | 37.3 | 85.1 | 75.9 | [config](configs/swinmoe/swin_moe_base_patch4_window12_192_densebaseline_22k.yaml) |
| Swin-MoE-B | 8 | 1 | Linear | 192x192 | 8x8 | 38.1 | 85.3 | 77.2 | [config](configs/swinmoe/swin_moe_base_patch4_window12_192_8expert_32gpu_22k.yaml) |
| Swin-MoE-B | 16 | 1 | Linear | 192x192 | 8x8 | 38.7 | 85.5 | 78.2 | [config](configs/swinmoe/swin_moe_base_patch4_window12_192_16expert_32gpu_22k.yaml) |
| Swin-MoE-B | 32 | 1 | Linear | 192x192 | 8x8 | 38.6 | 85.5 | 77.9 | [config](configs/swinmoe/swin_moe_base_patch4_window12_192_32expert_32gpu_22k.yaml) |
| Swin-MoE-B | 32 | 1 | Cosine | 192x192 | 8x8 | 38.5 | 85.3 | 77.3 | [config](configs/swinmoe/swin_moe_base_patch4_window12_192_cosine_router_32expert_32gpu_22k.yaml) |
| Swin-MoE-B | 32 | 2 | Linear | 192x192 | 8x8 | 38.6 | 85.5 | 78.7 | - |

## Simmim Pretrained Swin-V2 Models

- **model size** counts only the backbone weights and does not include weights in the decoders / classification heads
- **batch size** of all models is set 2048.
- **validation loss** is counted on the ImageNet-1K validation set.
- **fine-tuned acc@1** means the top-1 accuracy on the ImageNet-1K validation set by fine-tuning.

| name | model size | pre-train dataset | pre-train iterations | validation loss | fine-tuned acc@1 | pre-trained model | fine-tuned model |
| :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: |
| SwinV2-Small | 49M | ImageNet-1K 10% | 125k | 0.4820 | 82.69 | [azure](https://msravcghub.blob.core.windows.net/simmim-release/swinv2/pretrain/swinv2_small_1kper10_125k.pth) | [azure](https://msravcghub.blob.core.windows.net/simmim-release/swinv2/finetune/finetune_swinv2_small_1kper10_125k.pth) |
| SwinV2-Small | 49M | ImageNet-1K 10% | 250k | 0.4961 | 83.11 | [azure](https://msravcghub.blob.core.windows.net/simmim-release/swinv2/pretrain/swinv2_small_1kper10_250k.pth) | [azure](https://msravcghub.blob.core.windows.net/simmim-release/swinv2/finetune/finetune_swinv2_small_1kper10_250k.pth) |
| SwinV2-Small | 49M | ImageNet-1K 10% | 500k | 0.5115 | 83.17 | [azure](https://msravcghub.blob.core.windows.net/simmim-release/swinv2/pretrain/swinv2_small_1kper10_500k.pth) | [azure](https://msravcghub.blob.core.windows.net/simmim-release/swinv2/finetune/finetune_swinv2_small_1kper10_500k.pth) |
| SwinV2-Small | 49M | ImageNet-1K 20% | 125k | 0.4751 | 83.05 | [azure](https://msravcghub.blob.core.windows.net/simmim-release/swinv2/pretrain/swinv2_small_1kper20_125k.pth) | [azure](https://msravcghub.blob.core.windows.net/simmim-release/swinv2/finetune/finetune_swinv2_small_1kper20_125k.pth) |
| SwinV2-Small | 49M | ImageNet-1K 20% | 250k | 0.4722 | 83.56 | [azure](https://msravcghub.blob.core.windows.net/simmim-release/swinv2/pretrain/swinv2_small_1kper20_250k.pth) | [azure](https://msravcghub.blob.core.windows.net/simmim-release/swinv2/finetune/finetune_swinv2_small_1kper20_250k.pth) |
| SwinV2-Small | 49M | ImageNet-1K 20% | 500k | 0.4734 | 83.75 | [azure](https://msravcghub.blob.core.windows.net/simmim-release/swinv2/pretrain/swinv2_small_1kper20_500k.pth) | [azure](https://msravcghub.blob.core.windows.net/simmim-release/swinv2/finetune/finetune_swinv2_small_1kper20_500k.pth) |
| SwinV2-Small | 49M | ImageNet-1K 50% | 125k | 0.4732 | 83.04 | [azure](https://msravcghub.blob.core.windows.net/simmim-release/swinv2/pretrain/swinv2_small_1kper50_125k.pth) | [azure](https://msravcghub.blob.core.windows.net/simmim-release/swinv2/finetune/finetune_swinv2_small_1kper50_125k.pth) |
| SwinV2-Small | 49M | ImageNet-1K 50% | 250k | 0.4681 | 83.67 | [azure](https://msravcghub.blob.core.windows.net/simmim-release/swinv2/pretrain/swinv2_small_1kper50_250k.pth) | [azure](https://msravcghub.blob.core.windows.net/simmim-release/swinv2/finetune/finetune_swinv2_small_1kper50_250k.pth) |
| SwinV2-Small | 49M | ImageNet-1K 50% | 500k | 0.4646 | 83.96 | [azure](https://msravcghub.blob.core.windows.net/simmim-release/swinv2/pretrain/swinv2_small_1kper50_500k.pth) | [azure](https://msravcghub.blob.core.windows.net/simmim-release/swinv2/finetune/finetune_swinv2_small_1kper50_500k.pth) |
| SwinV2-Small | 49M | ImageNet-1K | 125k | 0.4728 | 82.92 | [azure](https://msravcghub.blob.core.windows.net/simmim-release/swinv2/pretrain/swinv2_small_1k_125k.pth) | [azure](https://msravcghub.blob.core.windows.net/simmim-release/swinv2/finetune/finetune_swinv2_small_1k_125k.pth) |
| SwinV2-Small | 49M | ImageNet-1K | 250k | 0.4674 | 83.66 | [azure](https://msravcghub.blob.core.windows.net/simmim-release/swinv2/pretrain/swinv2_small_1k_250k.pth) | [azure](https://msravcghub.blob.core.windows.net/simmim-release/swinv2/finetune/finetune_swinv2_small_1k_250k.pth) |
| SwinV2-Small | 49M | ImageNet-1K | 500k | 0.4641 | 84.08 | [azure](https://msravcghub.blob.core.windows.net/simmim-release/swinv2/pretrain/swinv2_small_1k_500k.pth) | [azure](https://msravcghub.blob.core.windows.net/simmim-release/swinv2/finetune/finetune_swinv2_small_1k_500k.pth) |
| SwinV2-Base | 87M | ImageNet-1K 10% | 125k | 0.4822 | 83.33 | [azure](https://msravcghub.blob.core.windows.net/simmim-release/swinv2/pretrain/swinv2_base_1kper10_125k.pth) | [azure](https://msravcghub.blob.core.windows.net/simmim-release/swinv2/finetune/finetune_swinv2_base_1kper10_125k.pth) |
| SwinV2-Base | 87M | ImageNet-1K 10% | 250k | 0.4997 | 83.60 | [azure](https://msravcghub.blob.core.windows.net/simmim-release/swinv2/pretrain/swinv2_base_1kper10_250k.pth) | [azure](https://msravcghub.blob.core.windows.net/simmim-release/swinv2/finetune/finetune_swinv2_base_1kper10_250k.pth) |
| SwinV2-Base | 87M | ImageNet-1K 10% | 500k | 0.5112 | 83.41 | [azure](https://msravcghub.blob.core.windows.net/simmim-release/swinv2/pretrain/swinv2_base_1kper10_500k.pth) | [azure](https://msravcghub.blob.core.windows.net/simmim-release/swinv2/finetune/finetune_swinv2_base_1kper10_500k.pth) |
| SwinV2-Base | 87M | ImageNet-1K 20% | 125k | 0.4703 | 83.86 | [azure](https://msravcghub.blob.core.windows.net/simmim-release/swinv2/pretrain/swinv2_base_1kper20_125k.pth) | [azure](https://msravcghub.blob.core.windows.net/simmim-release/swinv2/finetune/finetune_swinv2_base_1kper20_125k.pth) |
| SwinV2-Base | 87M | ImageNet-1K 20% | 250k | 0.4679 | 84.37 | [azure](https://msravcghub.blob.core.windows.net/simmim-release/swinv2/pretrain/swinv2_base_1kper20_250k.pth) | [azure](https://msravcghub.blob.core.windows.net/simmim-release/swinv2/finetune/finetune_swinv2_base_1kper20_250k.pth) |
| SwinV2-Base | 87M | ImageNet-1K 20% | 500k | 0.4711 | 84.61 | [azure](https://msravcghub.blob.core.windows.net/simmim-release/swinv2/pretrain/swinv2_base_1kper20_500k.pth) | [azure](https://msravcghub.blob.core.windows.net/simmim-release/swinv2/finetune/finetune_swinv2_base_1kper20_500k.pth) |
| SwinV2-Base | 87M | ImageNet-1K 50% | 125k | 0.4683 | 84.04 | [azure](https://msravcghub.blob.core.windows.net/simmim-release/swinv2/pretrain/swinv2_base_1kper50_125k.pth) | [azure](https://msravcghub.blob.core.windows.net/simmim-release/swinv2/finetune/finetune_swinv2_base_1kper50_125k.pth) |
| SwinV2-Base | 87M | ImageNet-1K 50% | 250k | 0.4633 | 84.57 | [azure](https://msravcghub.blob.core.windows.net/simmim-release/swinv2/pretrain/swinv2_base_1kper50_250k.pth) | [azure](https://msravcghub.blob.core.windows.net/simmim-release/swinv2/finetune/finetune_swinv2_base_1kper50_250k.pth) |
| SwinV2-Base | 87M | ImageNet-1K 50% | 500k | 0.4598 | 84.95 | [azure](https://msravcghub.blob.core.windows.net/simmim-release/swinv2/pretrain/swinv2_base_1kper50_500k.pth) | [azure](https://msravcghub.blob.core.windows.net/simmim-release/swinv2/finetune/finetune_swinv2_base_1kper50_500k.pth) |
| SwinV2-Base | 87M | ImageNet-1K | 125k | 0.4680 | 84.13 | [azure](https://msravcghub.blob.core.windows.net/simmim-release/swinv2/pretrain/swinv2_base_1k_125k.pth) | [azure](https://msravcghub.blob.core.windows.net/simmim-release/swinv2/finetune/finetune_swinv2_base_1k_125k.pth) |
| SwinV2-Base | 87M | ImageNet-1K | 250k | 0.4626 | 84.65 | [azure](https://msravcghub.blob.core.windows.net/simmim-release/swinv2/pretrain/swinv2_base_1k_250k.pth) | [azure](https://msravcghub.blob.core.windows.net/simmim-release/swinv2/finetune/finetune_swinv2_base_1k_250k.pth) |
| SwinV2-Base | 87M | ImageNet-1K | 500k | 0.4588 | 85.04 | [azure](https://msravcghub.blob.core.windows.net/simmim-release/swinv2/pretrain/swinv2_base_1k_500k.pth) | [azure](https://msravcghub.blob.core.windows.net/simmim-release/swinv2/finetune/finetune_swinv2_base_1k_500k.pth) |
| SwinV2-Base | 87M | ImageNet-22K | 125k | 0.4695 | 84.11 | [azure](https://msravcghub.blob.core.windows.net/simmim-release/swinv2/pretrain/swinv2_base_22k_125k.pth) | [azure](https://msravcghub.blob.core.windows.net/simmim-release/swinv2/finetune/finetune_swinv2_base_22k_125k.pth) |
| SwinV2-Base | 87M | ImageNet-22K | 250k | 0.4649 | 84.57 | [azure](https://msravcghub.blob.core.windows.net/simmim-release/swinv2/pretrain/swinv2_base_22k_250k.pth) | [azure](https://msravcghub.blob.core.windows.net/simmim-release/swinv2/finetune/finetune_swinv2_base_22k_250k.pth) |
| SwinV2-Base | 87M | ImageNet-22K | 500k | 0.4614 | 85.11 | [azure](https://msravcghub.blob.core.windows.net/simmim-release/swinv2/pretrain/swinv2_base_22k_500k.pth) | [azure](https://msravcghub.blob.core.windows.net/simmim-release/swinv2/finetune/finetune_swinv2_base_22k_500k.pth) |
| SwinV2-Large | 195M | ImageNet-1K 10% | 125k | 0.4995 | 83.69 | [azure](https://msravcghub.blob.core.windows.net/simmim-release/swinv2/pretrain/swinv2_large_1kper10_125k.pth) | [azure](https://msravcghub.blob.core.windows.net/simmim-release/swinv2/finetune/finetune_swinv2_large_1kper10_125k.pth) |
| SwinV2-Large | 195M | ImageNet-1K 10% | 250k | 0.5140 | 83.66 | [azure](https://msravcghub.blob.core.windows.net/simmim-release/swinv2/pretrain/swinv2_large_1kper10_250k.pth) | [azure](https://msravcghub.blob.core.windows.net/simmim-release/swinv2/finetune/finetune_swinv2_large_1kper10_250k.pth) |
| SwinV2-Large | 195M | ImageNet-1K 10% | 500k | 0.5150 | 83.50 | [azure](https://msravcghub.blob.core.windows.net/simmim-release/swinv2/pretrain/swinv2_large_1kper10_500k.pth) | [azure](https://msravcghub.blob.core.windows.net/simmim-release/swinv2/finetune/finetune_swinv2_large_1kper10_500k.pth) |
| SwinV2-Large | 195M | ImageNet-1K 20% | 125k | 0.4675 | 84.38 | [azure](https://msravcghub.blob.core.windows.net/simmim-release/swinv2/pretrain/swinv2_large_1kper20_125k.pth) | [azure](https://msravcghub.blob.core.windows.net/simmim-release/swinv2/finetune/finetune_swinv2_large_1kper20_125k.pth) |
| SwinV2-Large | 195M | ImageNet-1K 20% | 250k | 0.4746 | 84.71 | [azure](https://msravcghub.blob.core.windows.net/simmim-release/swinv2/pretrain/swinv2_large_1kper20_250k.pth) | [azure](https://msravcghub.blob.core.windows.net/simmim-release/swinv2/finetune/finetune_swinv2_large_1kper20_250k.pth) |
| SwinV2-Large | 195M | ImageNet-1K 20% | 500k | 0.4960 | 84.59 | [azure](https://msravcghub.blob.core.windows.net/simmim-release/swinv2/pretrain/swinv2_large_1kper20_500k.pth) | [azure](https://msravcghub.blob.core.windows.net/simmim-release/swinv2/finetune/finetune_swinv2_large_1kper20_500k.pth) |
| SwinV2-Large | 195M | ImageNet-1K 50% | 125k | 0.4622 | 84.78 | [azure](https://msravcghub.blob.core.windows.net/simmim-release/swinv2/pretrain/swinv2_large_1kper50_125k.pth) | [azure](https://msravcghub.blob.core.windows.net/simmim-release/swinv2/finetune/finetune_swinv2_large_1kper50_125k.pth) |
| SwinV2-Large | 195M | ImageNet-1K 50% | 250k | 0.4566 | 85.38 | [azure](https://msravcghub.blob.core.windows.net/simmim-release/swinv2/pretrain/swinv2_large_1kper50_250k.pth) | [azure](https://msravcghub.blob.core.windows.net/simmim-release/swinv2/finetune/finetune_swinv2_large_1kper50_250k.pth) |
| SwinV2-Large | 195M | ImageNet-1K 50% | 500k | 0.4530 | 85.80 | [azure](https://msravcghub.blob.core.windows.net/simmim-release/swinv2/pretrain/swinv2_large_1kper50_500k.pth) | [azure](https://msravcghub.blob.core.windows.net/simmim-release/swinv2/finetune/finetune_swinv2_large_1kper50_500k.pth) |
| SwinV2-Large | 195M | ImageNet-1K | 125k | 0.4611 | 84.98 | [azure](https://msravcghub.blob.core.windows.net/simmim-release/swinv2/pretrain/swinv2_large_1k_125k.pth) | [azure](https://msravcghub.blob.core.windows.net/simmim-release/swinv2/finetune/finetune_swinv2_large_1k_125k.pth) |
| SwinV2-Large | 195M | ImageNet-1K | 250k | 0.4552 | 85.45 | [azure](https://msravcghub.blob.core.windows.net/simmim-release/swinv2/pretrain/swinv2_large_1k_250k.pth) | [azure](https://msravcghub.blob.core.windows.net/simmim-release/swinv2/finetune/finetune_swinv2_large_1k_250k.pth) |
| SwinV2-Large | 195M | ImageNet-1K | 500k | 0.4507 | 85.91 | [azure](https://msravcghub.blob.core.windows.net/simmim-release/swinv2/pretrain/swinv2_large_1k_500k.pth) | [azure](https://msravcghub.blob.core.windows.net/simmim-release/swinv2/finetune/finetune_swinv2_large_1k_500k.pth) |
| SwinV2-Large | 195M | ImageNet-22K | 125k | 0.4649 | 84.61 | [azure](https://msravcghub.blob.core.windows.net/simmim-release/swinv2/pretrain/swinv2_large_22k_125k.pth) | [azure](https://msravcghub.blob.core.windows.net/simmim-release/swinv2/finetune/finetune_swinv2_large_22k_125k.pth) |
| SwinV2-Large | 195M | ImageNet-22K | 250k | 0.4586 | 85.39 | [azure](https://msravcghub.blob.core.windows.net/simmim-release/swinv2/pretrain/swinv2_large_22k_250k.pth) | [azure](https://msravcghub.blob.core.windows.net/simmim-release/swinv2/finetune/finetune_swinv2_large_22k_250k.pth) |
| SwinV2-Large | 195M | ImageNet-22K | 500k | 0.4536 | 85.81 | [azure](https://msravcghub.blob.core.windows.net/simmim-release/swinv2/pretrain/swinv2_large_22k_500k.pth) | [azure](https://msravcghub.blob.core.windows.net/simmim-release/swinv2/finetune/finetune_swinv2_large_22k_500k.pth) |
| SwinV2-Huge | 655M | ImageNet-1K 20% | 125k | 0.4789 | 84.35 | [azure](https://msravcghub.blob.core.windows.net/simmim-release/swinv2/pretrain/swinv2_huge_1kper20_125k.pth) | [azure](https://msravcghub.blob.core.windows.net/simmim-release/swinv2/finetune/finetune_swinv2_huge_1kper20_125k.pth) |
| SwinV2-Huge | 655M | ImageNet-1K 20% | 250k | 0.5038 | 84.16 | [azure](https://msravcghub.blob.core.windows.net/simmim-release/swinv2/pretrain/swinv2_huge_1kper20_250k.pth) | [azure](https://msravcghub.blob.core.windows.net/simmim-release/swinv2/finetune/finetune_swinv2_huge_1kper20_250k.pth) |
| SwinV2-Huge | 655M | ImageNet-1K 20% | 500k | 0.5071 | 83.44 | [azure](https://msravcghub.blob.core.windows.net/simmim-release/swinv2/pretrain/swinv2_huge_1kper20_500k.pth) | [azure](https://msravcghub.blob.core.windows.net/simmim-release/swinv2/finetune/finetune_swinv2_huge_1kper20_500k.pth) |
| SwinV2-Huge | 655M | ImageNet-1K 50% | 125k | 0.4549 | 85.09 | [azure](https://msravcghub.blob.core.windows.net/simmim-release/swinv2/pretrain/swinv2_huge_1kper50_125k.pth) | [azure](https://msravcghub.blob.core.windows.net/simmim-release/swinv2/finetune/finetune_swinv2_huge_1kper50_125k.pth) |
| SwinV2-Huge | 655M | ImageNet-1K 50% | 250k | 0.4511 | 85.64 | [azure](https://msravcghub.blob.core.windows.net/simmim-release/swinv2/pretrain/swinv2_huge_1kper50_250k.pth) | [azure](https://msravcghub.blob.core.windows.net/simmim-release/swinv2/finetune/finetune_swinv2_huge_1kper50_250k.pth) |
| SwinV2-Huge | 655M | ImageNet-1K 50% | 500k | 0.4559 | 85.69 | [azure](https://msravcghub.blob.core.windows.net/simmim-release/swinv2/pretrain/swinv2_huge_1kper50_500k.pth) | [azure](https://msravcghub.blob.core.windows.net/simmim-release/swinv2/finetune/finetune_swinv2_huge_1kper50_500k.pth) |
| SwinV2-Huge | 655M | ImageNet-1K | 125k | 0.4531 | 85.23 | [azure](https://msravcghub.blob.core.windows.net/simmim-release/swinv2/pretrain/swinv2_huge_1k_125k.pth) | [azure](https://msravcghub.blob.core.windows.net/simmim-release/swinv2/finetune/finetune_swinv2_huge_1k_125k.pth) |
| SwinV2-Huge | 655M | ImageNet-1K | 250k | 0.4464 | 85.90 | [azure](https://msravcghub.blob.core.windows.net/simmim-release/swinv2/pretrain/swinv2_huge_1k_250k.pth) | [azure](https://msravcghub.blob.core.windows.net/simmim-release/swinv2/finetune/finetune_swinv2_huge_1k_250k.pth) |
| SwinV2-Huge | 655M | ImageNet-1K | 500k | 0.4416 | 86.34 | [azure](https://msravcghub.blob.core.windows.net/simmim-release/swinv2/pretrain/swinv2_huge_1k_500k.pth) | [azure](https://msravcghub.blob.core.windows.net/simmim-release/swinv2/finetune/finetune_swinv2_huge_1k_500k.pth) |
| SwinV2-Huge | 655M | ImageNet-22K | 125k | 0.4564 | 85.14 | [azure](https://msravcghub.blob.core.windows.net/simmim-release/swinv2/pretrain/swinv2_huge_22k_125k.pth) | [azure](https://msravcghub.blob.core.windows.net/simmim-release/swinv2/finetune/finetune_swinv2_huge_22k_125k.pth) |
| SwinV2-Huge | 655M | ImageNet-22K | 250k | 0.4499 | 85.86 | [azure](https://msravcghub.blob.core.windows.net/simmim-release/swinv2/pretrain/swinv2_huge_22k_250k.pth) | [azure](https://msravcghub.blob.core.windows.net/simmim-release/swinv2/finetune/finetune_swinv2_huge_22k_250k.pth) |
| SwinV2-Huge | 655M | ImageNet-22K | 500k | 0.4444 | 86.27 | [azure](https://msravcghub.blob.core.windows.net/simmim-release/swinv2/pretrain/swinv2_huge_22k_500k.pth) | [azure](https://msravcghub.blob.core.windows.net/simmim-release/swinv2/finetune/finetune_swinv2_huge_22k_500k.pth) |
| SwinV2-giant | 1.06B | ImageNet-1K 50% | 125k | 0.4534 | 85.44 | [azure](https://msravcghub.blob.core.windows.net/simmim-release/swinv2/pretrain/swinv2_giant_1kper50_125k.pth) | [azure](https://msravcghub.blob.core.windows.net/simmim-release/swinv2/finetune/finetune_swinv2_giant_1kper50_125k.pth) |
| SwinV2-giant | 1.06B | ImageNet-1K 50% | 250k | 0.4515 | 85.76 | [azure](https://msravcghub.blob.core.windows.net/simmim-release/swinv2/pretrain/swinv2_giant_1kper50_250k.pth) | [azure](https://msravcghub.blob.core.windows.net/simmim-release/swinv2/finetune/finetune_swinv2_giant_1kper50_250k.pth) |
| SwinV2-giant | 1.06B | ImageNet-1K 50% | 500k | 0.4719 | 85.51 | [azure](https://msravcghub.blob.core.windows.net/simmim-release/swinv2/pretrain/swinv2_giant_1kper50_500k.pth) | [azure](https://msravcghub.blob.core.windows.net/simmim-release/swinv2/finetune/finetune_swinv2_giant_1kper50_500k.pth) |
| SwinV2-giant | 1.06B | ImageNet-1K | 125k | 0.4513 | 85.57 | [azure](https://msravcghub.blob.core.windows.net/simmim-release/swinv2/pretrain/swinv2_giant_1k_125k.pth) | [azure](https://msravcghub.blob.core.windows.net/simmim-release/swinv2/finetune/finetune_swinv2_giant_1k_125k.pth) |
| SwinV2-giant | 1.06B | ImageNet-1K | 250k | 0.4442 | 86.12 | [azure](https://msravcghub.blob.core.windows.net/simmim-release/swinv2/pretrain/swinv2_giant_1k_250k.pth) | [azure](https://msravcghub.blob.core.windows.net/simmim-release/swinv2/finetune/finetune_swinv2_giant_1k_250k.pth) |
| SwinV2-giant | 1.06B | ImageNet-1K | 500k | 0.4395 | 86.46 | [azure](https://msravcghub.blob.core.windows.net/simmim-release/swinv2/pretrain/swinv2_giant_1k_500k.pth) | [azure](https://msravcghub.blob.core.windows.net/simmim-release/swinv2/finetune/finetune_swinv2_giant_1k_500k.pth) |
| SwinV2-giant | 1.06B | ImageNet-22K | 125k | 0.4544 | 85.39 | [azure](https://msravcghub.blob.core.windows.net/simmim-release/swinv2/pretrain/swinv2_giant_22k_125k.pth) | [azure](https://msravcghub.blob.core.windows.net/simmim-release/swinv2/finetune/finetune_swinv2_giant_22k_125k.pth) |
| SwinV2-giant | 1.06B | ImageNet-22K | 250k | 0.4475 | 85.96 | [azure](https://msravcghub.blob.core.windows.net/simmim-release/swinv2/pretrain/swinv2_giant_22k_250k.pth) | [azure](https://msravcghub.blob.core.windows.net/simmim-release/swinv2/finetune/finetune_swinv2_giant_22k_250k.pth) |
| SwinV2-giant | 1.06B | ImageNet-22K | 500k | 0.4416 | 86.53 | [azure](https://msravcghub.blob.core.windows.net/simmim-release/swinv2/pretrain/swinv2_giant_22k_500k.pth) | [azure](https://msravcghub.blob.core.windows.net/simmim-release/swinv2/finetune/finetune_swinv2_giant_22k_500k.pth) |

## Simmim Pretrained Swin-V1 Models

**ImageNet-1K Pre-trained and Fine-tuned Models**

| name | pre-train epochs | pre-train resolution | fine-tune resolution | acc@1 | pre-trained model | fine-tuned model |
| :---: | :---: | :---: | :---: | :---: | :---: | :---: |
| Swin-Base | 100 | 192x192 | 192x192 | 82.8 | [google](https://drive.google.com/file/d/1Wcbr66JL26FF30Kip9fZa_0lXrDAKP-d/view?usp=sharing)/[config](configs/swin_base__100ep/simmim_pretrain__swin_base__img192_window6__100ep.yaml) | [google](https://drive.google.com/file/d/1RsgHfjB4B1ZYblXEQVT-FPX3WSvBrxcs/view?usp=sharing)/[config](configs/swin_base__100ep/simmim_finetune__swin_base__img192_window6__100ep.yaml) |
| Swin-Base | 100 | 192x192 | 224x224 | 83.5 | [google](https://drive.google.com/file/d/1Wcbr66JL26FF30Kip9fZa_0lXrDAKP-d/view?usp=sharing)/[config](configs/swin_base__100ep/simmim_pretrain__swin_base__img192_window6__100ep.yaml) | [google](https://drive.google.com/file/d/1mb43BkW56F5smwiX-g7QUUD7f1Rftq8u/view?usp=sharing)/[config](configs/swin_base__100ep/simmim_finetune__swin_base__img224_window7__100ep.yaml) |
| Swin-Base | 800 | 192x192 | 224x224 | 84.0 | [google](https://drive.google.com/file/d/15zENvGjHlM71uKQ3d2FbljWPubtrPtjl/view?usp=sharing)/[config](configs/swin_base__800ep/simmim_pretrain__swin_base__img192_window6__800ep.yaml) | [google](https://drive.google.com/file/d/1xEKyfMTsdh6TfnYhk5vbw0Yz7a-viZ0w/view?usp=sharing)/[config](configs/swin_base__800ep/simmim_finetune__swin_base__img224_window7__800ep.yaml) |
| Swin-Large | 800 | 192x192 | 224x224 | 85.4 | [google](https://drive.google.com/file/d/1qDxrTl2YUDB0505_4QrU5LU2R1kKmcBP/view?usp=sharing)/[config](configs/swin_large__800ep/simmim_pretrain__swin_large__img192_window12__800ep.yaml) | [google](https://drive.google.com/file/d/1mf0ZpXttEvFsH87Www4oQ-t8Kwr0x485/view?usp=sharing)/[config](configs/swin_large__800ep/simmim_finetune__swin_large__img224_window14__800ep.yaml) |
| SwinV2-Huge | 800 | 192x192 | 224x224 | 85.7 | / | / |
| SwinV2-Huge | 800 | 192x192 | 512x512 | 87.1 | / | / |