Skip to content

GitLab

Unverified Commit a2b75527 authored by Jeff Rasley's avatar Jeff Rasley Committed by GitHub
Browse files

if using lamb force legacy_fusion=True (#236)

parent bc36b91d
Hide whitespace changes
Inline Side-by-side
Showing with 1 addition and 2 deletions
deepspeed/pt/deepspeed_light.py
View file @ a2b75527
......@@ -551,8 +551,7 @@ class DeepSpeedLight(Module):
dynamic_loss_args=dynamic_loss_args,
mpu=self.mpu,
clip_grad=clip_grad,
fused_lamb_legacy=self.optimizer_legacy_fusion()
if self.optimizer_name() == LAMB_OPTIMIZER else False)
fused_lamb_legacy=self.optimizer_name() == LAMB_OPTIMIZER)
return optimizer
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment