Commits · 345f5d0d741db26efe7183e5eb53c385a554c85f · OpenDAS / Megatron-LM

30 Dec, 2020 3 commits
- moved grad scaling to multi-tensor-apply · 345f5d0d
  mohammad authored Dec 29, 2020
  
  345f5d0d
- further refactor, matching old results · e68ea218
  mohammad authored Dec 29, 2020
  
  e68ea218
- slight refactor of clip grads · b0a3fdfe
  mohammad authored Dec 29, 2020
  
  b0a3fdfe
28 Dec, 2020 1 commit
- found a bug in l2 norm calculation · 6191ff59
  Mohammad authored Dec 27, 2020
  
  6191ff59
27 Dec, 2020 1 commit
- load and save state dicts added · 97ba5c0e
  mohammad authored Dec 26, 2020
  
  97ba5c0e
25 Dec, 2020 7 commits
- further refactoring · 0888a3e1
  mohammad authored Dec 25, 2020
  
  0888a3e1
- grads is removed from mpu · dfd8ed47
  mohammad authored Dec 25, 2020
  
  dfd8ed47
- clip grad fixed and moved to optimizer · 46879674
  mohammad authored Dec 25, 2020
  
  46879674
- moved entire optimizer build and tested · 28062e14
  mohammad authored Dec 24, 2020
  
  28062e14
- megatron optimizer tested, before working on clip grad · fb218c9d
  mohammad authored Dec 24, 2020
  
  fb218c9d
- fp32 is also working · 2eaa3ccc
  mohammad authored Dec 24, 2020
  
  2eaa3ccc
- working on the new optimizer · c6a58e41
  mohammad authored Dec 24, 2020
  
  c6a58e41