Commits · 0ad4fd880b90cae452695d9b95de41feb1cfbba0 · OpenDAS / deepspeed

11 Nov, 2020 1 commit

Update zero.md tutorial (#495) · 0ad4fd88

Samyam Rajbhandari authored Nov 11, 2020



* Update zero.md

Update to ZeRO tutorial to specify the use of activation checkpointing

* Update zero-offload.md

Use activation checkpointing with ZeRO-Offload
Co-authored-by: Jeff Rasley <jerasley@microsoft.com>

0ad4fd88

10 Nov, 2020 2 commits

PLD release (#513) · be1147c0

Olatunji Ruwase authored Nov 10, 2020



* Progressive layer dropping docs (#499)

* test

* Adding tutorial and news page for pld

* updating the tutorial and posts of PLD

* update the finetune tutorial

* Update PLD tutorial (#512)

* Update installation instructions

* Format fix

* ZeRO tutorial

* Format fixes

* ZeRO-Offload

* ZeRO and ZeRO-Offload tutorials

* Update navigation page

* Format fixes

* Add yuxhe feedback

* Fix blog post link

* Fix OneBit-Adam link
Tweak scheduler example

* Fix date link

* Add DeepSpeed_Adam

* Add PLD tutorial to navigation
Co-authored-by: Shaden Smith <Shaden.Smith@microsoft.com>
Co-authored-by: Jeff Rasley <jerasley@microsoft.com>

* updating the pld docs

* DeepSpeed implementation of PLD (#508)

* DeepSpeed implementation of PLD

* Format fixes

* Formatting fixes

* Fix broken url

* Address PR feedback

* Bump DSE
Co-authored-by: Minjia Zhang <33713995+minjiaz@users.noreply.github.com>
Co-authored-by: Shaden Smith <Shaden.Smith@microsoft.com>
Co-authored-by: Jeff Rasley <jerasley@microsoft.com>
Co-authored-by: Minjia Zhang <minjiaz@microsoft.com>

be1147c0

updating pld docs (#517) · e082d475
Minjia Zhang authored Nov 10, 2020

e082d475

09 Nov, 2020 2 commits
- Fix PLD news url (#515) · 41fb24b3
  Olatunji Ruwase authored Nov 09, 2020
```
* PLD documentation

* Formatting fixes

* Fix url bug
```
  41fb24b3
- PLD documentation (#514) · e351090c
  Olatunji Ruwase authored Nov 09, 2020
```
* PLD documentation

* Formatting fixes
```
  e351090c
30 Oct, 2020 1 commit

Add CPUAdam optimizer for zero-offload in deepspeed engine (#484) · f5aa2547

Reza Yazdani authored Oct 30, 2020



* add adamW to CPU-ADAM implementation

* supporting cpu-adam optimizer for zero-offload on deepspeed side

* bump DSE to match cpu-adam updates
Co-authored-by: Jeff Rasley <jerasley@microsoft.com>

f5aa2547

19 Oct, 2020 1 commit
- updating website dependencies (#475) · d720fdb6
  Shaden Smith authored Oct 19, 2020
  
  d720fdb6
12 Oct, 2020 1 commit
- fixing typo (#460) · e25f2a23
  Reza Yazdani authored Oct 12, 2020
  
  e25f2a23
10 Oct, 2020 1 commit

Add DeepSpeed_Adam optimizer (#468) · 23fc48f3

Olatunji Ruwase authored Oct 10, 2020



* Update installation instructions

* Format fix

* ZeRO tutorial

* Format fixes

* ZeRO-Offload

* ZeRO and ZeRO-Offload tutorials

* Update navigation page

* Format fixes

* Add yuxhe feedback

* Fix blog post link

* Fix OneBit-Adam link
Tweak scheduler example

* Fix date link

* Add DeepSpeed_Adam
Co-authored-by: Shaden Smith <Shaden.Smith@microsoft.com>
Co-authored-by: Jeff Rasley <jerasley@microsoft.com>

23fc48f3

07 Oct, 2020 1 commit

gan tutorial (#462) · 2efea694

niumanar authored Oct 06, 2020



* gan tutorial

* formatting fix

* adding pointer to repo; adding navigation link
Co-authored-by: Jeff Rasley <jerasley@microsoft.com>

2efea694

25 Sep, 2020 2 commits
- link fix part two :-) (#441) · 6d176c45
  Shaden Smith authored Sep 25, 2020
  
  6d176c45
- Update pipeline.md (#439) · 0ca82156
  Haibin Lin authored Sep 24, 2020
  
  0ca82156
24 Sep, 2020 2 commits
- Update azure.md (#437) · 192cf7c8
  Conglong Li authored Sep 24, 2020
  
  192cf7c8
- Fix urls in tutorial (#436) · 5d40f006
  Conglong Li authored Sep 24, 2020
```
* url fix

* revert absolute path but keep some actual fix

* add real readme
```
  5d40f006
17 Sep, 2020 2 commits
- Fix few typos in the docs (#418) · c66f3881
  Gowtham Prudhvi authored Sep 17, 2020
  
  c66f3881
- readthedocs yaml configuration (#410) · 5812e845
  Shaden Smith authored Sep 16, 2020
```
Co-authored-by: Jeff Rasley <jerasley@microsoft.com>
```
  5812e845
16 Sep, 2020 1 commit

Minor doc fixes (#417) · 7d91be97

Olatunji Ruwase authored Sep 16, 2020



* Update installation instructions

* Format fix

* ZeRO tutorial

* Format fixes

* ZeRO-Offload

* ZeRO and ZeRO-Offload tutorials

* Update navigation page

* Format fixes

* Add yuxhe feedback

* Fix blog post link

* Fix OneBit-Adam link
Tweak scheduler example

* Fix date link
Co-authored-by: Shaden Smith <Shaden.Smith@microsoft.com>
Co-authored-by: Jeff Rasley <jerasley@microsoft.com>

7d91be97

10 Sep, 2020 14 commits

readthedocs upgrade (#402) · c82756cd
Shaden Smith authored Sep 10, 2020

c82756cd

Update ZeRO-Offload blog post link (#401) · d15015e9

Olatunji Ruwase authored Sep 10, 2020


Co-authored-by: Shaden Smith <Shaden.Smith@microsoft.com>
Co-authored-by: Jeff Rasley <jerasley@microsoft.com>

d15015e9

Update news site with press release link · 5dc4d6c7
Jeff Rasley authored Sep 10, 2020

5dc4d6c7
Update _config.yml · ea92ed29
Jeff Rasley authored Sep 10, 2020

ea92ed29
bump DSE and doc tweak · 4b1df25a
Jeff Rasley authored Sep 10, 2020

4b1df25a
Website edits (#398) · 6bb5c69f
Shaden Smith authored Sep 10, 2020
```
Co-authored-by: Jeff Rasley <jerasley@microsoft.com>
```
6bb5c69f

Landing page updates (#395) · a8a8b3d2

Jeff Rasley authored Sep 10, 2020


Co-authored-by: Shaden Smith <ShadenTSmith@gmail.com>
Co-authored-by: Shaden Smith <Shaden.Smith@microsoft.com>

a8a8b3d2

Adding sparse attention news index item (#376) · c76769c4
Arash Ashari authored Sep 10, 2020
```
Co-authored-by: Jeff Rasley <jerasley@microsoft.com>
```
c76769c4
Minjiaz/zero offload (#382) · 59ce90d0
Minjia Zhang authored Sep 10, 2020
```
Co-authored-by: Jeff Rasley <jerasley@microsoft.com>
```
59ce90d0
Sparse attention: updating code tag in documentation (#394) · be4b94be
Arash Ashari authored Sep 10, 2020
```
Co-authored-by: Jeff Rasley <jerasley@microsoft.com>
```
be4b94be

ZeRO tutorials (#384) · 2dea61f2

Olatunji Ruwase authored Sep 10, 2020


Co-authored-by: Shaden Smith <Shaden.Smith@microsoft.com>
Co-authored-by: Jeff Rasley <jerasley@microsoft.com>

2dea61f2

Update documentation for 1-bit Adam (#388) · 093f09ff
Ammar Ahmad Awan authored Sep 10, 2020
```
Co-authored-by: Jeff Rasley <jerasley@microsoft.com>
```
093f09ff
Pipeline parallel training engine. (#392) · 65c2f974
Shaden Smith authored Sep 09, 2020
```
Co-authored-by: Jeff Rasley <jerasley@microsoft.com>
```
65c2f974

ZeRO-Offload release (#391) · 41db1c2f

Jeff Rasley authored Sep 09, 2020



* ZeRO-Offload (squash) (#381)
Co-authored-by: Olatunji Ruwase <olruwase@microsoft.com>
Co-authored-by: Reza Yazdani <reyazda@microsoft.com>
Co-authored-by: Jeff Rasley <jerasley@microsoft.com>
Co-authored-by: Jie <37380896+jren73@users.noreply.github.com>
Co-authored-by: Arash Ashari <arashari@microsoft.com>
Co-authored-by: Reza Yazdani <reyazda@microsoft.com>
Co-authored-by: Samyam Rajbhandari <samyamr@microsoft.com>
Co-authored-by: Shaden Smith <Shaden.Smith@microsoft.com>
Co-authored-by: arashashari <arashashari@ArashMSLaptop.redmond.corp.microsoft.com>
Co-authored-by: RezaYazdaniAminabadi <44502768+RezaYazdaniAminabadi@users.noreply.github.com>
Co-authored-by: Reza Yazdani <reyazda@microsoft.com>
Co-authored-by: Samyam Rajbhandari <samyamr@microsoft.com>
Co-authored-by: Shaden Smith <Shaden.Smith@microsoft.com>

41db1c2f

09 Sep, 2020 3 commits

fixing a link issue with SA tutorial (#387) · 161e8e60
Arash Ashari authored Sep 09, 2020
```
Co-authored-by: Jeff Rasley <jerasley@microsoft.com>
```
161e8e60

Add 1-bit Adam support to DeepSpeed (#380) · 01726ce2

Ammar Ahmad Awan authored Sep 09, 2020



* 1-bit adam (#353)
Co-authored-by: Jeff Rasley <jerasley@microsoft.com>
Co-authored-by: Your Name <you@example.com>
Co-authored-by: tanghl1994 <htang14@ur.rochester.edu>
Co-authored-by: Hank <tanghl1994@gmail.com>
Co-authored-by: root <root@node2x12b.cs.rochester.edu>
Co-authored-by: Ammar Ahmad Awan <awan.ammar@microsoft.com>

01726ce2

adding sparse attention to feature index page (#377) · b73894de
Arash Ashari authored Sep 08, 2020

b73894de

06 Sep, 2020 2 commits

Update Sparse Attention Tutorial (#357) · 9dadf38d

Arash Ashari authored Sep 06, 2020



* adding BingSqaud e2e test

* updating the draft test; bring final step under try section

* finalizinf test for base deepspeed and deepspeed with ZeRO

* applying the comment (thanks Jeff); fixed formatting

* update Sparse Attention Tutorial

* fixed few issues and applied comments for better organization and readability

* updated sparse attention tutorial with making how to use section incremental; applying more comments
Co-authored-by: arashashari <arashashari@ArashMSLaptop.redmond.corp.microsoft.com>

9dadf38d

Update installation instructions (#362) · 9e83ef21
Olatunji Ruwase authored Sep 06, 2020

9e83ef21

04 Sep, 2020 1 commit
- Jekyll installation instructions (#351) · ac12833e
  Shaden Smith authored Sep 03, 2020
  
  ac12833e
03 Sep, 2020 1 commit
- Adding link to Sparse Attention in Navigation page (#355) · 6deac82c
  Arash Ashari authored Sep 03, 2020
```
* adding link to Sparse Attention in Navigation page
```
  6deac82c
02 Sep, 2020 1 commit

Sparse attn + ops/runtime refactor + v0.3.0 (#343) · e5bbc2e5

Jeff Rasley authored Sep 01, 2020



* Sparse attn + ops/runtime refactor + v0.3.0
Co-authored-by: Arash Ashari <arashari@microsoft.com>
Co-authored-by: Arash Ashari <arashari@microsoft.com>

e5bbc2e5

08 Aug, 2020 1 commit
- updates website gems after kramdown alert (#311) · 903a41a5
  Shaden Smith authored Aug 08, 2020
  
  903a41a5