Commits · f5aa2547d88f145c4b5223d5bb8f79db69ee5288 · OpenDAS / deepspeed

30 Oct, 2020 1 commit

Add CPUAdam optimizer for zero-offload in deepspeed engine (#484) · f5aa2547

Reza Yazdani authored Oct 30, 2020



* add adamW to CPU-ADAM implementation

* supporting cpu-adam optimizer for zero-offload on deepspeed side

* bump DSE to match cpu-adam updates
Co-authored-by: Jeff Rasley <jerasley@microsoft.com>

f5aa2547

19 Oct, 2020 1 commit
- updating website dependencies (#475) · d720fdb6
  Shaden Smith authored Oct 19, 2020
  
  d720fdb6
12 Oct, 2020 1 commit
- fixing typo (#460) · e25f2a23
  Reza Yazdani authored Oct 12, 2020
  
  e25f2a23
10 Oct, 2020 1 commit

Add DeepSpeed_Adam optimizer (#468) · 23fc48f3

Olatunji Ruwase authored Oct 10, 2020



* Update installation instructions

* Format fix

* ZeRO tutorial

* Format fixes

* ZeRO-Offload

* ZeRO and ZeRO-Offload tutorials

* Update navigation page

* Format fixes

* Add yuxhe feedback

* Fix blog post link

* Fix OneBit-Adam link
Tweak scheduler example

* Fix date link

* Add DeepSpeed_Adam
Co-authored-by: Shaden Smith <Shaden.Smith@microsoft.com>
Co-authored-by: Jeff Rasley <jerasley@microsoft.com>

23fc48f3

07 Oct, 2020 1 commit

gan tutorial (#462) · 2efea694

niumanar authored Oct 06, 2020



* gan tutorial

* formatting fix

* adding pointer to repo; adding navigation link
Co-authored-by: Jeff Rasley <jerasley@microsoft.com>

2efea694

25 Sep, 2020 2 commits
- link fix part two :-) (#441) · 6d176c45
  Shaden Smith authored Sep 25, 2020
  
  6d176c45
- Update pipeline.md (#439) · 0ca82156
  Haibin Lin authored Sep 24, 2020
  
  0ca82156
24 Sep, 2020 2 commits
- Update azure.md (#437) · 192cf7c8
  Conglong Li authored Sep 24, 2020
  
  192cf7c8
- Fix urls in tutorial (#436) · 5d40f006
  Conglong Li authored Sep 24, 2020
```
* url fix

* revert absolute path but keep some actual fix

* add real readme
```
  5d40f006
17 Sep, 2020 2 commits
- Fix few typos in the docs (#418) · c66f3881
  Gowtham Prudhvi authored Sep 17, 2020
  
  c66f3881
- readthedocs yaml configuration (#410) · 5812e845
  Shaden Smith authored Sep 16, 2020
```
Co-authored-by: Jeff Rasley <jerasley@microsoft.com>
```
  5812e845
16 Sep, 2020 1 commit

Minor doc fixes (#417) · 7d91be97

Olatunji Ruwase authored Sep 16, 2020



* Update installation instructions

* Format fix

* ZeRO tutorial

* Format fixes

* ZeRO-Offload

* ZeRO and ZeRO-Offload tutorials

* Update navigation page

* Format fixes

* Add yuxhe feedback

* Fix blog post link

* Fix OneBit-Adam link
Tweak scheduler example

* Fix date link
Co-authored-by: Shaden Smith <Shaden.Smith@microsoft.com>
Co-authored-by: Jeff Rasley <jerasley@microsoft.com>

7d91be97

10 Sep, 2020 14 commits

readthedocs upgrade (#402) · c82756cd
Shaden Smith authored Sep 10, 2020

c82756cd

Update ZeRO-Offload blog post link (#401) · d15015e9

Olatunji Ruwase authored Sep 10, 2020


Co-authored-by: Shaden Smith <Shaden.Smith@microsoft.com>
Co-authored-by: Jeff Rasley <jerasley@microsoft.com>

d15015e9

Update news site with press release link · 5dc4d6c7
Jeff Rasley authored Sep 10, 2020

5dc4d6c7
Update _config.yml · ea92ed29
Jeff Rasley authored Sep 10, 2020

ea92ed29
bump DSE and doc tweak · 4b1df25a
Jeff Rasley authored Sep 10, 2020

4b1df25a
Website edits (#398) · 6bb5c69f
Shaden Smith authored Sep 10, 2020
```
Co-authored-by: Jeff Rasley <jerasley@microsoft.com>
```
6bb5c69f

Landing page updates (#395) · a8a8b3d2

Jeff Rasley authored Sep 10, 2020


Co-authored-by: Shaden Smith <ShadenTSmith@gmail.com>
Co-authored-by: Shaden Smith <Shaden.Smith@microsoft.com>

a8a8b3d2

Adding sparse attention news index item (#376) · c76769c4
Arash Ashari authored Sep 10, 2020
```
Co-authored-by: Jeff Rasley <jerasley@microsoft.com>
```
c76769c4
Minjiaz/zero offload (#382) · 59ce90d0
Minjia Zhang authored Sep 10, 2020
```
Co-authored-by: Jeff Rasley <jerasley@microsoft.com>
```
59ce90d0
Sparse attention: updating code tag in documentation (#394) · be4b94be
Arash Ashari authored Sep 10, 2020
```
Co-authored-by: Jeff Rasley <jerasley@microsoft.com>
```
be4b94be

ZeRO tutorials (#384) · 2dea61f2

Olatunji Ruwase authored Sep 10, 2020


Co-authored-by: Shaden Smith <Shaden.Smith@microsoft.com>
Co-authored-by: Jeff Rasley <jerasley@microsoft.com>

2dea61f2

Update documentation for 1-bit Adam (#388) · 093f09ff
Ammar Ahmad Awan authored Sep 10, 2020
```
Co-authored-by: Jeff Rasley <jerasley@microsoft.com>
```
093f09ff
Pipeline parallel training engine. (#392) · 65c2f974
Shaden Smith authored Sep 09, 2020
```
Co-authored-by: Jeff Rasley <jerasley@microsoft.com>
```
65c2f974

ZeRO-Offload release (#391) · 41db1c2f

Jeff Rasley authored Sep 09, 2020



* ZeRO-Offload (squash) (#381)
Co-authored-by: Olatunji Ruwase <olruwase@microsoft.com>
Co-authored-by: Reza Yazdani <reyazda@microsoft.com>
Co-authored-by: Jeff Rasley <jerasley@microsoft.com>
Co-authored-by: Jie <37380896+jren73@users.noreply.github.com>
Co-authored-by: Arash Ashari <arashari@microsoft.com>
Co-authored-by: Reza Yazdani <reyazda@microsoft.com>
Co-authored-by: Samyam Rajbhandari <samyamr@microsoft.com>
Co-authored-by: Shaden Smith <Shaden.Smith@microsoft.com>
Co-authored-by: arashashari <arashashari@ArashMSLaptop.redmond.corp.microsoft.com>
Co-authored-by: RezaYazdaniAminabadi <44502768+RezaYazdaniAminabadi@users.noreply.github.com>
Co-authored-by: Reza Yazdani <reyazda@microsoft.com>
Co-authored-by: Samyam Rajbhandari <samyamr@microsoft.com>
Co-authored-by: Shaden Smith <Shaden.Smith@microsoft.com>

41db1c2f

09 Sep, 2020 3 commits

fixing a link issue with SA tutorial (#387) · 161e8e60
Arash Ashari authored Sep 09, 2020
```
Co-authored-by: Jeff Rasley <jerasley@microsoft.com>
```
161e8e60

Add 1-bit Adam support to DeepSpeed (#380) · 01726ce2

Ammar Ahmad Awan authored Sep 09, 2020



* 1-bit adam (#353)
Co-authored-by: Jeff Rasley <jerasley@microsoft.com>
Co-authored-by: Your Name <you@example.com>
Co-authored-by: tanghl1994 <htang14@ur.rochester.edu>
Co-authored-by: Hank <tanghl1994@gmail.com>
Co-authored-by: root <root@node2x12b.cs.rochester.edu>
Co-authored-by: Ammar Ahmad Awan <awan.ammar@microsoft.com>

01726ce2

adding sparse attention to feature index page (#377) · b73894de
Arash Ashari authored Sep 08, 2020

b73894de

06 Sep, 2020 2 commits

Update Sparse Attention Tutorial (#357) · 9dadf38d

Arash Ashari authored Sep 06, 2020



* adding BingSqaud e2e test

* updating the draft test; bring final step under try section

* finalizinf test for base deepspeed and deepspeed with ZeRO

* applying the comment (thanks Jeff); fixed formatting

* update Sparse Attention Tutorial

* fixed few issues and applied comments for better organization and readability

* updated sparse attention tutorial with making how to use section incremental; applying more comments
Co-authored-by: arashashari <arashashari@ArashMSLaptop.redmond.corp.microsoft.com>

9dadf38d

Update installation instructions (#362) · 9e83ef21
Olatunji Ruwase authored Sep 06, 2020

9e83ef21

04 Sep, 2020 1 commit
- Jekyll installation instructions (#351) · ac12833e
  Shaden Smith authored Sep 03, 2020
  
  ac12833e
03 Sep, 2020 1 commit
- Adding link to Sparse Attention in Navigation page (#355) · 6deac82c
  Arash Ashari authored Sep 03, 2020
```
* adding link to Sparse Attention in Navigation page
```
  6deac82c
02 Sep, 2020 1 commit

Sparse attn + ops/runtime refactor + v0.3.0 (#343) · e5bbc2e5

Jeff Rasley authored Sep 01, 2020



* Sparse attn + ops/runtime refactor + v0.3.0
Co-authored-by: Arash Ashari <arashari@microsoft.com>
Co-authored-by: Arash Ashari <arashari@microsoft.com>

e5bbc2e5

08 Aug, 2020 1 commit
- updates website gems after kramdown alert (#311) · 903a41a5
  Shaden Smith authored Aug 08, 2020
  
  903a41a5
07 Aug, 2020 1 commit
- Add webinar link (#309) · 29c5fe26
  Jeff Rasley authored Aug 07, 2020
```
Add webinar on-demand links and update readme
```
  29c5fe26
28 Jul, 2020 1 commit
- Fixing a typo (#303) · 97c54273
  Emmanuel Kahembwe authored Jul 28, 2020
  
  97c54273
25 Jul, 2020 1 commit
- DeepSpeed webinar announcement (#301) · 7ae8f8bc
  Shaden Smith authored Jul 24, 2020
  
  7ae8f8bc
13 Jul, 2020 1 commit
- Update amp docs (#287) · 366d8816
  Jeff Rasley authored Jul 13, 2020
```
* add amp docs
```
  366d8816
24 Jun, 2020 1 commit

Improving deepspeed.ai website (#269) · 6379292c

Conglong Li authored Jun 23, 2020

* syntax/typo fix

* add README for documentation

* fix links

* update navigation

* typo fix

* docs readme fix

6379292c