Commits · 9f8e8f38290ecea6b5054df49ec8635b248c18a8 · OpenDAS / deepspeed

30 Oct, 2020 1 commit

Add CPUAdam optimizer for zero-offload in deepspeed engine (#484) · f5aa2547

Reza Yazdani authored Oct 30, 2020



* add adamW to CPU-ADAM implementation

* supporting cpu-adam optimizer for zero-offload on deepspeed side

* bump DSE to match cpu-adam updates
Co-authored-by: Jeff Rasley <jerasley@microsoft.com>

f5aa2547

10 Oct, 2020 1 commit

Add DeepSpeed_Adam optimizer (#468) · 23fc48f3

Olatunji Ruwase authored Oct 10, 2020



* Update installation instructions

* Format fix

* ZeRO tutorial

* Format fixes

* ZeRO-Offload

* ZeRO and ZeRO-Offload tutorials

* Update navigation page

* Format fixes

* Add yuxhe feedback

* Fix blog post link

* Fix OneBit-Adam link
Tweak scheduler example

* Fix date link

* Add DeepSpeed_Adam
Co-authored-by: Shaden Smith <Shaden.Smith@microsoft.com>
Co-authored-by: Jeff Rasley <jerasley@microsoft.com>

23fc48f3

16 Sep, 2020 1 commit

Minor doc fixes (#417) · 7d91be97

Olatunji Ruwase authored Sep 16, 2020



* Update installation instructions

* Format fix

* ZeRO tutorial

* Format fixes

* ZeRO-Offload

* ZeRO and ZeRO-Offload tutorials

* Update navigation page

* Format fixes

* Add yuxhe feedback

* Fix blog post link

* Fix OneBit-Adam link
Tweak scheduler example

* Fix date link
Co-authored-by: Shaden Smith <Shaden.Smith@microsoft.com>
Co-authored-by: Jeff Rasley <jerasley@microsoft.com>

7d91be97

10 Sep, 2020 6 commits

Website edits (#398) · 6bb5c69f
Shaden Smith authored Sep 10, 2020
```
Co-authored-by: Jeff Rasley <jerasley@microsoft.com>
```
6bb5c69f
Minjiaz/zero offload (#382) · 59ce90d0
Minjia Zhang authored Sep 10, 2020
```
Co-authored-by: Jeff Rasley <jerasley@microsoft.com>
```
59ce90d0
Sparse attention: updating code tag in documentation (#394) · be4b94be
Arash Ashari authored Sep 10, 2020
```
Co-authored-by: Jeff Rasley <jerasley@microsoft.com>
```
be4b94be

ZeRO tutorials (#384) · 2dea61f2

Olatunji Ruwase authored Sep 10, 2020


Co-authored-by: Shaden Smith <Shaden.Smith@microsoft.com>
Co-authored-by: Jeff Rasley <jerasley@microsoft.com>

2dea61f2

Update documentation for 1-bit Adam (#388) · 093f09ff
Ammar Ahmad Awan authored Sep 10, 2020
```
Co-authored-by: Jeff Rasley <jerasley@microsoft.com>
```
093f09ff

ZeRO-Offload release (#391) · 41db1c2f

Jeff Rasley authored Sep 09, 2020



* ZeRO-Offload (squash) (#381)
Co-authored-by: Olatunji Ruwase <olruwase@microsoft.com>
Co-authored-by: Reza Yazdani <reyazda@microsoft.com>
Co-authored-by: Jeff Rasley <jerasley@microsoft.com>
Co-authored-by: Jie <37380896+jren73@users.noreply.github.com>
Co-authored-by: Arash Ashari <arashari@microsoft.com>
Co-authored-by: Reza Yazdani <reyazda@microsoft.com>
Co-authored-by: Samyam Rajbhandari <samyamr@microsoft.com>
Co-authored-by: Shaden Smith <Shaden.Smith@microsoft.com>
Co-authored-by: arashashari <arashashari@ArashMSLaptop.redmond.corp.microsoft.com>
Co-authored-by: RezaYazdaniAminabadi <44502768+RezaYazdaniAminabadi@users.noreply.github.com>
Co-authored-by: Reza Yazdani <reyazda@microsoft.com>
Co-authored-by: Samyam Rajbhandari <samyamr@microsoft.com>
Co-authored-by: Shaden Smith <Shaden.Smith@microsoft.com>

41db1c2f

09 Sep, 2020 1 commit
- adding sparse attention to feature index page (#377) · b73894de
  Arash Ashari authored Sep 08, 2020
  
  b73894de
02 Sep, 2020 1 commit

Sparse attn + ops/runtime refactor + v0.3.0 (#343) · e5bbc2e5

Jeff Rasley authored Sep 01, 2020



* Sparse attn + ops/runtime refactor + v0.3.0
Co-authored-by: Arash Ashari <arashari@microsoft.com>
Co-authored-by: Arash Ashari <arashari@microsoft.com>

e5bbc2e5

13 Jul, 2020 1 commit
- Update amp docs (#287) · 366d8816
  Jeff Rasley authored Jul 13, 2020
```
* add amp docs
```
  366d8816
24 Jun, 2020 1 commit

Improving deepspeed.ai website (#269) · 6379292c

Conglong Li authored Jun 23, 2020

* syntax/typo fix

* add README for documentation

* fix links

* update navigation

* typo fix

* docs readme fix

6379292c

29 May, 2020 1 commit
- adding overlap_comm documentation (#247) · e04e4016
  Samyam Rajbhandari authored May 29, 2020
  
  e04e4016
27 May, 2020 1 commit

add support for predivide as a config option (#235) · bc36b91d

Jeff Rasley authored May 27, 2020

* add support for predivide as a flag
* add predivide json config, remove allgather_disable (as it's not currently used anymore)

bc36b91d

25 May, 2020 1 commit
- fix redundant (#228) · b466e84e
  Chunyang Wen authored May 25, 2020
  
  b466e84e
19 May, 2020 1 commit

ZeRO-2 (#217) · f2ac7eaf

Jeff Rasley authored May 19, 2020



Updates for ZeRO stage 2 + ZeRO stage 1 w. RS
Co-authored-by: Tunji Ruwase <olruwase@microsoft.com>
Co-authored-by: Samyam Rajbhandari <samyamr@microsoft.com>
Co-authored-by: Shaden Smith <ShadenTSmith@gmail.com>
Co-authored-by: Elton Zheng <eltonz@microsoft.com>
Co-authored-by: Shaden Smith <Shaden.Smith@microsoft.com>
Co-authored-by: yuxionghe <yuxhe@microsoft.com>
Co-authored-by: Arash Ashari <arashari@microsoft.com>

f2ac7eaf

18 Mar, 2020 3 commits
- JSON configuration cleanup. (#151) · 1496247a
  Shaden Smith authored Mar 18, 2020
```
* Better config filename

* Clean up configuration ToC
```
  1496247a
- Fix permalinks (#149) · 29855c27
  Shaden Smith authored Mar 18, 2020
```
* fix docs permalink

* fix docs permalink
```
  29855c27
- Web edits (#147) · b84a1fa4
  Shaden Smith authored Mar 18, 2020
  
  b84a1fa4
17 Mar, 2020 1 commit
- drafting Jekyll webpage (#143) · 5042dc00
  Shaden Smith authored Mar 17, 2020
  
  5042dc00