Add bfloat16 training option

29be56e5 · Gustaf Ahdritz · 78e79962 · 29be56e5
Commit 29be56e5 authored Nov 29, 2021 by Gustaf Ahdritz
Show whitespace changes
Inline Side-by-side

Showing with 3 additions and 0 deletions

deepspeed_config.json deepspeed_config.json +3 -0

No files found.
--- a/deepspeed_config.json
+++ b/deepspeed_config.json
@@ -22,6 +22,9 @@
    "enabled": false,
    "opt_level": "O2"
  },
+  "bfloat16": {
+    "enabled": false
+  },
  "zero_optimization": {
    "stage": 2,
    "cpu_offload": true,