Commits · e2dfe0d17be21ce08e759dab4712bc37debd44c4 · OpenDAS / deepspeed

"vscode:/vscode.git/clone" did not exist on "5b485a628b0dccde0bbf37ee78a2b283669e9219"

11 Feb, 2021 1 commit

Add flops profiler tutorial (#682) · e2dfe0d1

Cheng Li authored Feb 10, 2021

* work on flops profiler tutorial

* update flops profiler tutorial

* add flops profiler tutorial and fix names

* work on flops profiler tutorial

* update flops profiler tutorial

* add flops profiler tutorial and fix names

* fix tailing ws

* fix names

* remove multistep profiling and update docs

* fix cases where functionals and submodules coexist in a parent module, update readme

* fix typo

* always invoke post hook function

* fix module flops sum and update tests

* update tutorial

e2dfe0d1

20 Jan, 2021 1 commit
- [tutorials] typos (#676) · 7b0bee0b
  Stas Bekman authored Jan 20, 2021
```
Co-authored-by: Jeff Rasley <jerasley@microsoft.com>
```
  7b0bee0b
15 Jan, 2021 2 commits
- doc fix (#651) · 7b07e123
  Stas Bekman authored Jan 15, 2021
```
Co-authored-by: Jeff Rasley <jerasley@microsoft.com>
```
  7b07e123
- Add AdamW to the supported optimizers (#672) · c5e42641
  Stas Bekman authored Jan 15, 2021
```
Co-authored-by: Jeff Rasley <jerasley@microsoft.com>
```
  c5e42641
05 Jan, 2021 1 commit
- docs: minor spelling tweaks (#623) · 46d2e287
  brett koonce authored Jan 05, 2021
```
Co-authored-by: Jeff Rasley <jerasley@microsoft.com>
```
  46d2e287
15 Dec, 2020 1 commit

[doc] xref to hostfile discussion (#604) · 007466e5

Stas Bekman authored Dec 15, 2020

* [doc] xref to hostfile discussion

wasn't clear where to find what was meant by `hostfile` - so adding a link to where it's discussed.

* remove whitespace

007466e5

30 Oct, 2020 1 commit

Add CPUAdam optimizer for zero-offload in deepspeed engine (#484) · f5aa2547

Reza Yazdani authored Oct 30, 2020



* add adamW to CPU-ADAM implementation

* supporting cpu-adam optimizer for zero-offload on deepspeed side

* bump DSE to match cpu-adam updates
Co-authored-by: Jeff Rasley <jerasley@microsoft.com>

f5aa2547

10 Oct, 2020 1 commit

Add DeepSpeed_Adam optimizer (#468) · 23fc48f3

Olatunji Ruwase authored Oct 10, 2020



* Update installation instructions

* Format fix

* ZeRO tutorial

* Format fixes

* ZeRO-Offload

* ZeRO and ZeRO-Offload tutorials

* Update navigation page

* Format fixes

* Add yuxhe feedback

* Fix blog post link

* Fix OneBit-Adam link
Tweak scheduler example

* Fix date link

* Add DeepSpeed_Adam
Co-authored-by: Shaden Smith <Shaden.Smith@microsoft.com>
Co-authored-by: Jeff Rasley <jerasley@microsoft.com>

23fc48f3

16 Sep, 2020 1 commit

Minor doc fixes (#417) · 7d91be97

Olatunji Ruwase authored Sep 16, 2020



* Update installation instructions

* Format fix

* ZeRO tutorial

* Format fixes

* ZeRO-Offload

* ZeRO and ZeRO-Offload tutorials

* Update navigation page

* Format fixes

* Add yuxhe feedback

* Fix blog post link

* Fix OneBit-Adam link
Tweak scheduler example

* Fix date link
Co-authored-by: Shaden Smith <Shaden.Smith@microsoft.com>
Co-authored-by: Jeff Rasley <jerasley@microsoft.com>

7d91be97

10 Sep, 2020 6 commits

Website edits (#398) · 6bb5c69f
Shaden Smith authored Sep 10, 2020
```
Co-authored-by: Jeff Rasley <jerasley@microsoft.com>
```
6bb5c69f
Minjiaz/zero offload (#382) · 59ce90d0
Minjia Zhang authored Sep 10, 2020
```
Co-authored-by: Jeff Rasley <jerasley@microsoft.com>
```
59ce90d0
Sparse attention: updating code tag in documentation (#394) · be4b94be
Arash Ashari authored Sep 10, 2020
```
Co-authored-by: Jeff Rasley <jerasley@microsoft.com>
```
be4b94be

ZeRO tutorials (#384) · 2dea61f2

Olatunji Ruwase authored Sep 10, 2020


Co-authored-by: Shaden Smith <Shaden.Smith@microsoft.com>
Co-authored-by: Jeff Rasley <jerasley@microsoft.com>

2dea61f2

Update documentation for 1-bit Adam (#388) · 093f09ff
Ammar Ahmad Awan authored Sep 10, 2020
```
Co-authored-by: Jeff Rasley <jerasley@microsoft.com>
```
093f09ff

ZeRO-Offload release (#391) · 41db1c2f

Jeff Rasley authored Sep 09, 2020



* ZeRO-Offload (squash) (#381)
Co-authored-by: Olatunji Ruwase <olruwase@microsoft.com>
Co-authored-by: Reza Yazdani <reyazda@microsoft.com>
Co-authored-by: Jeff Rasley <jerasley@microsoft.com>
Co-authored-by: Jie <37380896+jren73@users.noreply.github.com>
Co-authored-by: Arash Ashari <arashari@microsoft.com>
Co-authored-by: Reza Yazdani <reyazda@microsoft.com>
Co-authored-by: Samyam Rajbhandari <samyamr@microsoft.com>
Co-authored-by: Shaden Smith <Shaden.Smith@microsoft.com>
Co-authored-by: arashashari <arashashari@ArashMSLaptop.redmond.corp.microsoft.com>
Co-authored-by: RezaYazdaniAminabadi <44502768+RezaYazdaniAminabadi@users.noreply.github.com>
Co-authored-by: Reza Yazdani <reyazda@microsoft.com>
Co-authored-by: Samyam Rajbhandari <samyamr@microsoft.com>
Co-authored-by: Shaden Smith <Shaden.Smith@microsoft.com>

41db1c2f

09 Sep, 2020 1 commit
- adding sparse attention to feature index page (#377) · b73894de
  Arash Ashari authored Sep 08, 2020
  
  b73894de
02 Sep, 2020 1 commit

Sparse attn + ops/runtime refactor + v0.3.0 (#343) · e5bbc2e5

Jeff Rasley authored Sep 01, 2020



* Sparse attn + ops/runtime refactor + v0.3.0
Co-authored-by: Arash Ashari <arashari@microsoft.com>
Co-authored-by: Arash Ashari <arashari@microsoft.com>

e5bbc2e5

13 Jul, 2020 1 commit
- Update amp docs (#287) · 366d8816
  Jeff Rasley authored Jul 13, 2020
```
* add amp docs
```
  366d8816
24 Jun, 2020 1 commit

Improving deepspeed.ai website (#269) · 6379292c

Conglong Li authored Jun 23, 2020

* syntax/typo fix

* add README for documentation

* fix links

* update navigation

* typo fix

* docs readme fix

6379292c

29 May, 2020 1 commit
- adding overlap_comm documentation (#247) · e04e4016
  Samyam Rajbhandari authored May 29, 2020
  
  e04e4016
27 May, 2020 1 commit

add support for predivide as a config option (#235) · bc36b91d

Jeff Rasley authored May 27, 2020

* add support for predivide as a flag
* add predivide json config, remove allgather_disable (as it's not currently used anymore)

bc36b91d

25 May, 2020 1 commit
- fix redundant (#228) · b466e84e
  Chunyang Wen authored May 25, 2020
  
  b466e84e
19 May, 2020 1 commit

ZeRO-2 (#217) · f2ac7eaf

Jeff Rasley authored May 19, 2020



Updates for ZeRO stage 2 + ZeRO stage 1 w. RS
Co-authored-by: Tunji Ruwase <olruwase@microsoft.com>
Co-authored-by: Samyam Rajbhandari <samyamr@microsoft.com>
Co-authored-by: Shaden Smith <ShadenTSmith@gmail.com>
Co-authored-by: Elton Zheng <eltonz@microsoft.com>
Co-authored-by: Shaden Smith <Shaden.Smith@microsoft.com>
Co-authored-by: yuxionghe <yuxhe@microsoft.com>
Co-authored-by: Arash Ashari <arashari@microsoft.com>

f2ac7eaf

18 Mar, 2020 3 commits
- JSON configuration cleanup. (#151) · 1496247a
  Shaden Smith authored Mar 18, 2020
```
* Better config filename

* Clean up configuration ToC
```
  1496247a
- Fix permalinks (#149) · 29855c27
  Shaden Smith authored Mar 18, 2020
```
* fix docs permalink

* fix docs permalink
```
  29855c27
- Web edits (#147) · b84a1fa4
  Shaden Smith authored Mar 18, 2020
  
  b84a1fa4
17 Mar, 2020 1 commit
- drafting Jekyll webpage (#143) · 5042dc00
  Shaden Smith authored Mar 17, 2020
  
  5042dc00