Internal change

PiperOrigin-RevId: 387177259

Internal change
PiperOrigin-RevId: 387177259
8dd291f7 · Xianzhi Du · A. Unique TensorFlower · b3d7c348 · 8dd291f7
Commit 8dd291f7 authored Jul 27, 2021 by Xianzhi Du Committed by A. Unique TensorFlower Jul 27, 2021
Hide whitespace changes
Inline Side-by-side

Showing with 6 additions and 0 deletions

official/vision/beta/projects/vit/modeling/vit.py official/vision/beta/projects/vit/modeling/vit.py +6 -0

No files found.
--- a/official/vision/beta/projects/vit/modeling/vit.py
+++ b/official/vision/beta/projects/vit/modeling/vit.py
@@ -59,6 +59,12 @@ VIT_SPECS = {
            patch_size=14,
            transformer=dict(mlp_dim=5120, num_heads=16, num_layers=32),
        ),
+    'vit-g14':
+        dict(
+            hidden_size=1664,
+            patch_size=14,
+            transformer=dict(mlp_dim=8192, num_heads=16, num_layers=48),
+        ),
 }