Skip to content

GitLab

Switch branch/tag
Find file Normal viewHistoryPermalink
dist_8gpus_zero3_offload.yaml 322 Bytes
Newer Older
Zeqiang Lai's avatar
[Classification] support deepspeed, fix optimizer bugs #(83)  
Zeqiang Lai committed
1
2 
compute_environment: LOCAL_MACHINE
deepspeed_config:
Zeqiang Lai's avatar
Minor update on Deepspeed (#112)  
Zeqiang Lai committed
3 
  deepspeed_config_file: configs/accelerate/deepspeed/ds_config_zero3_offload.json
Zeqiang Lai's avatar
[Classification] support deepspeed, fix optimizer bugs #(83)  
Zeqiang Lai committed
4
5
6
7
8
9
10
11
12 
  zero3_init_flag: false
distributed_type: DEEPSPEED
downcast_bf16: 'no'
machine_rank: 0
main_training_function: main
num_machines: 1
num_processes: 8
rdzv_backend: static
use_cpu: false