Skip to content

GitLab

Commit 29be56e5 authored by Gustaf Ahdritz's avatar Gustaf Ahdritz
Browse files

Add bfloat16 training option

parent 78e79962
Hide whitespace changes
Inline Side-by-side
Showing with 3 additions and 0 deletions
deepspeed_config.json
View file @ 29be56e5
...@@ -22,6 +22,9 @@ ...@@ -22,6 +22,9 @@
"enabled": false, "enabled": false,
"opt_level": "O2" "opt_level": "O2"
}, },
"bfloat16": {
"enabled": false
},
"zero_optimization": { "zero_optimization": {
"stage": 2, "stage": 2,
"cpu_offload": true, "cpu_offload": true,
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment