Unverified Commit 0410a29a authored by Phuc Van Phan's avatar Phuc Van Phan Committed by GitHub
Browse files

fix: fix gradient accumulate step for learning rate (#27667)

parent f84d85ba
...@@ -640,7 +640,7 @@ def main(): ...@@ -640,7 +640,7 @@ def main():
# Create learning rate schedule # Create learning rate schedule
linear_decay_lr_schedule_fn = create_learning_rate_fn( linear_decay_lr_schedule_fn = create_learning_rate_fn(
len(vectorized_datasets["train"]), total_train_steps,
training_args.warmup_steps, training_args.warmup_steps,
training_args.learning_rate, training_args.learning_rate,
) )
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment