Skip to content

GitLab

Commit dfd77717 authored by alexeib's avatar alexeib Committed by Myle Ott
Browse files

fix cosine lr sched for t_mult=1 with warmup

parent 0a7f9e64
Show whitespace changes
Inline Side-by-side
Showing with 1 addition and 1 deletion
fairseq/optim/lr_scheduler/cosine_lr_scheduler.py
View file @ dfd77717
...@@ -93,7 +93,7 @@ class CosineSchedule(FairseqLRScheduler): ...@@ -93,7 +93,7 @@ class CosineSchedule(FairseqLRScheduler):
else: else:
i = math.floor(curr_updates / self.period) i = math.floor(curr_updates / self.period)
t_i = self.period t_i = self.period
t_curr = num_updates - (self.period * i) t_curr = curr_updates - (self.period * i)
lr_shrink = self.lr_shrink ** i lr_shrink = self.lr_shrink ** i
min_lr = self.min_lr * lr_shrink min_lr = self.min_lr * lr_shrink
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment