Commits · 7b07e123067bbc49cadd77270e8d898c86f71aaa · OpenDAS / deepspeed

15 Jan, 2021 2 commits
- doc fix (#651) · 7b07e123
  Stas Bekman authored Jan 15, 2021
```
Co-authored-by: Jeff Rasley <jerasley@microsoft.com>
```
  7b07e123
- Add AdamW to the supported optimizers (#672) · c5e42641
  Stas Bekman authored Jan 15, 2021
```
Co-authored-by: Jeff Rasley <jerasley@microsoft.com>
```
  c5e42641
30 Oct, 2020 1 commit

Add CPUAdam optimizer for zero-offload in deepspeed engine (#484) · f5aa2547

Reza Yazdani authored Oct 30, 2020



* add adamW to CPU-ADAM implementation

* supporting cpu-adam optimizer for zero-offload on deepspeed side

* bump DSE to match cpu-adam updates
Co-authored-by: Jeff Rasley <jerasley@microsoft.com>

f5aa2547

10 Oct, 2020 1 commit

Add DeepSpeed_Adam optimizer (#468) · 23fc48f3

Olatunji Ruwase authored Oct 10, 2020



* Update installation instructions

* Format fix

* ZeRO tutorial

* Format fixes

* ZeRO-Offload

* ZeRO and ZeRO-Offload tutorials

* Update navigation page

* Format fixes

* Add yuxhe feedback

* Fix blog post link

* Fix OneBit-Adam link
Tweak scheduler example

* Fix date link

* Add DeepSpeed_Adam
Co-authored-by: Shaden Smith <Shaden.Smith@microsoft.com>
Co-authored-by: Jeff Rasley <jerasley@microsoft.com>

23fc48f3

16 Sep, 2020 1 commit

Minor doc fixes (#417) · 7d91be97

Olatunji Ruwase authored Sep 16, 2020



* Update installation instructions

* Format fix

* ZeRO tutorial

* Format fixes

* ZeRO-Offload

* ZeRO and ZeRO-Offload tutorials

* Update navigation page

* Format fixes

* Add yuxhe feedback

* Fix blog post link

* Fix OneBit-Adam link
Tweak scheduler example

* Fix date link
Co-authored-by: Shaden Smith <Shaden.Smith@microsoft.com>
Co-authored-by: Jeff Rasley <jerasley@microsoft.com>

7d91be97

10 Sep, 2020 2 commits

ZeRO tutorials (#384) · 2dea61f2

Olatunji Ruwase authored Sep 10, 2020


Co-authored-by: Shaden Smith <Shaden.Smith@microsoft.com>
Co-authored-by: Jeff Rasley <jerasley@microsoft.com>

2dea61f2

Update documentation for 1-bit Adam (#388) · 093f09ff
Ammar Ahmad Awan authored Sep 10, 2020
```
Co-authored-by: Jeff Rasley <jerasley@microsoft.com>
```
093f09ff

02 Sep, 2020 1 commit

Sparse attn + ops/runtime refactor + v0.3.0 (#343) · e5bbc2e5

Jeff Rasley authored Sep 01, 2020



* Sparse attn + ops/runtime refactor + v0.3.0
Co-authored-by: Arash Ashari <arashari@microsoft.com>
Co-authored-by: Arash Ashari <arashari@microsoft.com>

e5bbc2e5

13 Jul, 2020 1 commit
- Update amp docs (#287) · 366d8816
  Jeff Rasley authored Jul 13, 2020
```
* add amp docs
```
  366d8816
29 May, 2020 1 commit
- adding overlap_comm documentation (#247) · e04e4016
  Samyam Rajbhandari authored May 29, 2020
  
  e04e4016
27 May, 2020 1 commit

add support for predivide as a config option (#235) · bc36b91d

Jeff Rasley authored May 27, 2020

* add support for predivide as a flag
* add predivide json config, remove allgather_disable (as it's not currently used anymore)

bc36b91d

19 May, 2020 1 commit

ZeRO-2 (#217) · f2ac7eaf

Jeff Rasley authored May 19, 2020



Updates for ZeRO stage 2 + ZeRO stage 1 w. RS
Co-authored-by: Tunji Ruwase <olruwase@microsoft.com>
Co-authored-by: Samyam Rajbhandari <samyamr@microsoft.com>
Co-authored-by: Shaden Smith <ShadenTSmith@gmail.com>
Co-authored-by: Elton Zheng <eltonz@microsoft.com>
Co-authored-by: Shaden Smith <Shaden.Smith@microsoft.com>
Co-authored-by: yuxionghe <yuxhe@microsoft.com>
Co-authored-by: Arash Ashari <arashari@microsoft.com>

f2ac7eaf

18 Mar, 2020 3 commits
- JSON configuration cleanup. (#151) · 1496247a
  Shaden Smith authored Mar 18, 2020
```
* Better config filename

* Clean up configuration ToC
```
  1496247a
- Fix permalinks (#149) · 29855c27
  Shaden Smith authored Mar 18, 2020
```
* fix docs permalink

* fix docs permalink
```
  29855c27
- Web edits (#147) · b84a1fa4
  Shaden Smith authored Mar 18, 2020
  
  b84a1fa4