Commits · 24e07399fc3ee95417d7a582c3e8952e1ac7d567 · OpenDAS / deepspeed

21 Nov, 2020 1 commit
- Support non-tensor state in checkpoint (#548) · 6021b702
  Olatunji Ruwase authored Nov 21, 2020
  
  6021b702
20 Nov, 2020 1 commit

Fix unbalanced gradients bug in ZeRO-2 gradient accumulation (#545) · 0178e6cc

Olatunji Ruwase authored Nov 20, 2020

* Use zero-tensors for missing gradients to avoid size mismatch

* Unit test for unbalanced gradients in ZeRO

* Formatting fixes

0178e6cc

10 Nov, 2020 1 commit

PLD release (#513) · be1147c0

Olatunji Ruwase authored Nov 10, 2020



* Progressive layer dropping docs (#499)

* test

* Adding tutorial and news page for pld

* updating the tutorial and posts of PLD

* update the finetune tutorial

* Update PLD tutorial (#512)

* Update installation instructions

* Format fix

* ZeRO tutorial

* Format fixes

* ZeRO-Offload

* ZeRO and ZeRO-Offload tutorials

* Update navigation page

* Format fixes

* Add yuxhe feedback

* Fix blog post link

* Fix OneBit-Adam link
Tweak scheduler example

* Fix date link

* Add DeepSpeed_Adam

* Add PLD tutorial to navigation
Co-authored-by: Shaden Smith <Shaden.Smith@microsoft.com>
Co-authored-by: Jeff Rasley <jerasley@microsoft.com>

* updating the pld docs

* DeepSpeed implementation of PLD (#508)

* DeepSpeed implementation of PLD

* Format fixes

* Formatting fixes

* Fix broken url

* Address PR feedback

* Bump DSE
Co-authored-by: Minjia Zhang <33713995+minjiaz@users.noreply.github.com>
Co-authored-by: Shaden Smith <Shaden.Smith@microsoft.com>
Co-authored-by: Jeff Rasley <jerasley@microsoft.com>
Co-authored-by: Minjia Zhang <minjiaz@microsoft.com>

be1147c0

10 Sep, 2020 1 commit
- Pipeline parallel training engine. (#392) · 65c2f974
  Shaden Smith authored Sep 09, 2020
```
Co-authored-by: Jeff Rasley <jerasley@microsoft.com>
```
  65c2f974
15 Jul, 2020 2 commits
- Empty grad fix (#291) · 376818ef
  Jeff Rasley authored Jul 15, 2020
```
* empty grad fix
* add unit tests for empty grad
```
  376818ef
- Fix bug in fp32 optimizer state loading (#289) · 607814fe
  Olatunji Ruwase authored Jul 15, 2020
  
  607814fe
27 Mar, 2020 1 commit

Add "zero_allow_untested_optimizer" option in conf file (#173) · 43f27332

Calogero Zarbo authored Mar 27, 2020

* added zero_allow_untested_optimizer flag helpers

* add zero_allow_untested_optimizer config constants

* zero_allow_untested_optimizer logic with assertion

* Added unit test and CustomOptimizer helper class

43f27332

20 Feb, 2020 1 commit
- Refactor simple model test, fix pythonpath issue (#96) · 001abe23
  Jeff Rasley authored Feb 20, 2020
```
Also a fix for #94 
```
  001abe23