megatron/optimizer/__init__.py · 5d2e13a66b2e23de78c3341634d0ef5433e253b7 · OpenDAS / Megatron-LM · GitLab

Find file Blame History Permalink

reverting pre allocation of optimizer states; it does not seem to help with fragmentation · 5d2e13a6
Vijay Korthikanti authored Apr 28, 2022

5d2e13a6

__init__.py 5.54 KB