Skip to content

GitLab

Switch branch/tag
Find file BlameHistoryPermalink
  • Adrian Sager La Ganga's avatar
    Add inverse sqrt learning rate scheduler (#21495) · a3034c70
    Adrian Sager La Ganga authored 
    * added inverse sqrt lr scheduler

* Updated get_scheduler in src/transformers/optimization.py

* Updated src/transformers/__init__.py

* Added inverse sqrt lr scheduler test

* Updated docs/source/en/main_classes/optimizer_schedules.mdx

* Ran style and quality scripts

* Fix get_inverse_sqrt_schedule docstring

* Comment implementation URL
    a3034c70
test_optimization.py 6.14 KB