Commits · d7de9165db2b9be7389d4dde931bd4595138b044 · OpenDAS / deepspeed

"tests/mxnet/test_basics.py" did not exist on "b2e4bdc03f019b886350aff5b0ffdff6f4372843"

08 Mar, 2021 3 commits

update tutorial/doc links for zero3 (#835) · d7de9165
Jeff Rasley authored Mar 08, 2021

d7de9165

Samyam Rajbhandari authored Mar 08, 2021



* Squash stage3 v1 (#146)
Co-authored-by: Samyam <samyamr@microsoft.com>
Co-authored-by: Jeff Rasley <jerasley@microsoft.com>
Co-authored-by: Samyam Rajbhandari <samyamr@microsoft.com>
Co-authored-by: Olatunji Ruwase <olruwase@microsoft.com>
Co-authored-by: Shaden Smith <Shaden.Smith@microsoft.com>
Co-authored-by: Shaden Smith <ShadenTSmith@gmail.com>
Co-authored-by: eltonzheng <eltonz@microsoft.com>

* Fix correctness bug (#147)

* formatting fix (#150)

* stage3 bugfix (API) update and simplified FP16 Z3 tests (#151)

* fp16 Z3 API update and bugfix

* revert debug change

* ZeRO-3 detach and race condition bugfixes (#149)

* trying out ZeRO-3 race condition fix

* CUDA sync instead of stream

* reduction stream sync

* remove commented code

* Fix optimizer state_dict KeyError (#148)
Co-authored-by: Jeff Rasley <jerasley@microsoft.com>

* fix for smaller SGS sizes, ensures each grad is backed by unique tensors (#152)

* Simplifying the logic for getting averaged gradients (#153)

* skip for now

* Z3 Docs redux (#154)

* removing some TODOs and commented code (#155)

* New Z3 defaults (#156)
Co-authored-by: Jeff Rasley <jerasley@microsoft.com>

* formatting

* megatron external params
Co-authored-by: Jeff Rasley <jerasley@microsoft.com>
Co-authored-by: Olatunji Ruwase <olruwase@microsoft.com>
Co-authored-by: Shaden Smith <Shaden.Smith@microsoft.com>
Co-authored-by: Shaden Smith <ShadenTSmith@gmail.com>
Co-authored-by: eltonzheng <eltonz@microsoft.com>

599258f9

Update ZeRO-Offload tutorials (#824) · ba33e86e
Olatunji Ruwase authored Mar 08, 2021

ba33e86e

26 Feb, 2021 1 commit
- document the requirement to call for all ranks (#801) · 7eb083c2
  Stas Bekman authored Feb 26, 2021
  
  7eb083c2
20 Feb, 2021 1 commit
- [doc] fix incorrect param name (#773) · e60e92eb
  Stas Bekman authored Feb 20, 2021
```
Invalid param name

Thanks.
```
  e60e92eb
16 Feb, 2021 1 commit

Minor doc tweaks (#761) · c28a71f9

Olatunji Ruwase authored Feb 16, 2021

* Fix docstring

* Make screenshots clickable for easier viewing

* Navigation menu in alphabetical order; More clicable screenshots

* Rename 1Cycle doc

* Tweak naming

c28a71f9

12 Feb, 2021 1 commit
- [install] fixes/improvements/docs (#752) · 78e776a9
  Stas Bekman authored Feb 12, 2021
```
Co-authored-by: Jeff Rasley <jerasley@microsoft.com>
```
  78e776a9
11 Feb, 2021 2 commits

1-bit Adam documentation fix (#747) · 248f6383

Conglong Li authored Feb 11, 2021



* 1-bit adam doc fix

* 1-bit adam doc fix

* 1-bit adam doc fix
Co-authored-by: Jeff Rasley <jerasley@microsoft.com>

248f6383

Add flops profiler tutorial (#682) · e2dfe0d1

Cheng Li authored Feb 10, 2021

* work on flops profiler tutorial

* update flops profiler tutorial

* add flops profiler tutorial and fix names

* work on flops profiler tutorial

* update flops profiler tutorial

* add flops profiler tutorial and fix names

* fix tailing ws

* fix names

* remove multistep profiling and update docs

* fix cases where functionals and submodules coexist in a parent module, update readme

* fix typo

* always invoke post hook function

* fix module flops sum and update tests

* update tutorial

e2dfe0d1

10 Feb, 2021 2 commits
- Clickable screenshots (#746) · 6ee3b296
  Olatunji Ruwase authored Feb 10, 2021
```
* Fix docstring

* Make screenshots clickable for easier viewing
```
  6ee3b296
- 1bit_adam dependencies (#742) · 6beca3ce
  Stas Bekman authored Feb 10, 2021
  
  6beca3ce
20 Jan, 2021 1 commit
- [tutorials] typos (#676) · 7b0bee0b
  Stas Bekman authored Jan 20, 2021
```
Co-authored-by: Jeff Rasley <jerasley@microsoft.com>
```
  7b0bee0b
19 Jan, 2021 1 commit
- add zero-offload paper (#680) · 82cecf69
  Jeff Rasley authored Jan 19, 2021
```
* Update README.md

* Update index.md
```
  82cecf69
15 Jan, 2021 3 commits
- doc fix (#651) · 7b07e123
  Stas Bekman authored Jan 15, 2021
```
Co-authored-by: Jeff Rasley <jerasley@microsoft.com>
```
  7b07e123
- add missing config menu entries (#652) · e729a3fd
  Stas Bekman authored Jan 15, 2021
```
Co-authored-by: Jeff Rasley <jerasley@microsoft.com>
```
  e729a3fd
- Add AdamW to the supported optimizers (#672) · c5e42641
  Stas Bekman authored Jan 15, 2021
```
Co-authored-by: Jeff Rasley <jerasley@microsoft.com>
```
  c5e42641
08 Jan, 2021 1 commit
- document deepspeed.initialize() (#644) · 828d75ba
  Stas Bekman authored Jan 08, 2021
```
Co-authored-by: Jeff Rasley <jerasley@microsoft.com>
```
  828d75ba
07 Jan, 2021 2 commits

Add deepspeed.init_distributed to RTD page (#645) · 4e2dc4e4
Jeff Rasley authored Jan 07, 2021
```
Co-authored-by: Olatunji Ruwase <olruwase@microsoft.com>
```
4e2dc4e4

Bump nokogiri from 1.10.10 to 1.11.0 in /docs (#630) · 8cea96dd

dependabot[bot] authored Jan 07, 2021

Bumps [nokogiri](https://github.com/sparklemotion/nokogiri) from 1.10.10 to 1.11.0.
- [Release notes](https://github.com/sparklemotion/nokogiri/releases)
- [Changelog](https://github.com/sparklemotion/nokogiri/blob/master/CHANGELOG.md)
- [Commits](https://github.com/sparklemotion/nokogiri/compare/v1.10.10...v1.11.0

)
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Jeff Rasley <jerasley@microsoft.com>

8cea96dd

06 Jan, 2021 1 commit

Module replacement support (#586) · 44bd538b

Jeff Rasley authored Jan 06, 2021


Co-authored-by: Reza Yazdani <reyazda@microsoft.com>
Co-authored-by: Olatunji Ruwase <olruwase@microsoft.com>

44bd538b

05 Jan, 2021 1 commit
- docs: minor spelling tweaks (#623) · 46d2e287
  brett koonce authored Jan 05, 2021
```
Co-authored-by: Jeff Rasley <jerasley@microsoft.com>
```
  46d2e287
18 Dec, 2020 1 commit
- Ability to initialize distributed backend outside deepspeed runtime (#608) · 7435b2f1
  Jeff Rasley authored Dec 17, 2020
  
  7435b2f1
15 Dec, 2020 2 commits

Fixes for RTD build errors (#606) · 6380ee35
Jeff Rasley authored Dec 15, 2020
```
Co-authored-by: Shaden Smith <Shaden.Smith@microsoft.com>
```
6380ee35

[doc] xref to hostfile discussion (#604) · 007466e5

Stas Bekman authored Dec 15, 2020

* [doc] xref to hostfile discussion

wasn't clear where to find what was meant by `hostfile` - so adding a link to where it's discussed.

* remove whitespace

007466e5

09 Dec, 2020 1 commit
- Add papers/videos to readme/website (#592) · 19acd6cf
  Jeff Rasley authored Dec 09, 2020
  
  19acd6cf
07 Dec, 2020 1 commit
- [build] make builder smarter and configurable wrt compute capabilities + docs (#578) · ce363d0e
  Stas Bekman authored Dec 07, 2020
  
  ce363d0e
02 Dec, 2020 1 commit
- [cifar tutorial] improve readability (#567) · 7a75f8b3
  Stas Bekman authored Dec 02, 2020
```
* [cifar tutorial] improve readability 
```
  7a75f8b3
28 Nov, 2020 1 commit

[doc] typo fix and clarification (#563) · 17f36f1b

Stas Bekman authored Nov 27, 2020

This PR:
* fixes a misspelled method name
* also `( () )` doesn't read too well, until one reads the code and understands that it's not a formatting bug. I proposed to simply say that it's a callable object.

17f36f1b

12 Nov, 2020 2 commits

Installation documentation updates. (#525) · d779bd53
Shaden Smith authored Nov 12, 2020
```
* Adds torch install requirement to documentation.

* build ops documentation
```
d779bd53

DeepSpeed JIT op + PyPI support (#496) · 31f46fee

Jeff Rasley authored Nov 12, 2020


Co-authored-by: Shaden Smith <Shaden.Smith@microsoft.com>
Co-authored-by: Reza Yazdani <reyazda@microsoft.com>

31f46fee

11 Nov, 2020 1 commit

Update zero.md tutorial (#495) · 0ad4fd88

Samyam Rajbhandari authored Nov 11, 2020



* Update zero.md

Update to ZeRO tutorial to specify the use of activation checkpointing

* Update zero-offload.md

Use activation checkpointing with ZeRO-Offload
Co-authored-by: Jeff Rasley <jerasley@microsoft.com>

0ad4fd88

10 Nov, 2020 2 commits

PLD release (#513) · be1147c0

Olatunji Ruwase authored Nov 10, 2020



* Progressive layer dropping docs (#499)

* test

* Adding tutorial and news page for pld

* updating the tutorial and posts of PLD

* update the finetune tutorial

* Update PLD tutorial (#512)

* Update installation instructions

* Format fix

* ZeRO tutorial

* Format fixes

* ZeRO-Offload

* ZeRO and ZeRO-Offload tutorials

* Update navigation page

* Format fixes

* Add yuxhe feedback

* Fix blog post link

* Fix OneBit-Adam link
Tweak scheduler example

* Fix date link

* Add DeepSpeed_Adam

* Add PLD tutorial to navigation
Co-authored-by: Shaden Smith <Shaden.Smith@microsoft.com>
Co-authored-by: Jeff Rasley <jerasley@microsoft.com>

* updating the pld docs

* DeepSpeed implementation of PLD (#508)

* DeepSpeed implementation of PLD

* Format fixes

* Formatting fixes

* Fix broken url

* Address PR feedback

* Bump DSE
Co-authored-by: Minjia Zhang <33713995+minjiaz@users.noreply.github.com>
Co-authored-by: Shaden Smith <Shaden.Smith@microsoft.com>
Co-authored-by: Jeff Rasley <jerasley@microsoft.com>
Co-authored-by: Minjia Zhang <minjiaz@microsoft.com>

be1147c0

updating pld docs (#517) · e082d475
Minjia Zhang authored Nov 10, 2020

e082d475

09 Nov, 2020 2 commits
- Fix PLD news url (#515) · 41fb24b3
  Olatunji Ruwase authored Nov 09, 2020
```
* PLD documentation

* Formatting fixes

* Fix url bug
```
  41fb24b3
- PLD documentation (#514) · e351090c
  Olatunji Ruwase authored Nov 09, 2020
```
* PLD documentation

* Formatting fixes
```
  e351090c
30 Oct, 2020 1 commit

Add CPUAdam optimizer for zero-offload in deepspeed engine (#484) · f5aa2547

Reza Yazdani authored Oct 30, 2020



* add adamW to CPU-ADAM implementation

* supporting cpu-adam optimizer for zero-offload on deepspeed side

* bump DSE to match cpu-adam updates
Co-authored-by: Jeff Rasley <jerasley@microsoft.com>

f5aa2547

19 Oct, 2020 1 commit
- updating website dependencies (#475) · d720fdb6
  Shaden Smith authored Oct 19, 2020
  
  d720fdb6
12 Oct, 2020 1 commit
- fixing typo (#460) · e25f2a23
  Reza Yazdani authored Oct 12, 2020
  
  e25f2a23
10 Oct, 2020 1 commit

Add DeepSpeed_Adam optimizer (#468) · 23fc48f3

Olatunji Ruwase authored Oct 10, 2020



* Update installation instructions

* Format fix

* ZeRO tutorial

* Format fixes

* ZeRO-Offload

* ZeRO and ZeRO-Offload tutorials

* Update navigation page

* Format fixes

* Add yuxhe feedback

* Fix blog post link

* Fix OneBit-Adam link
Tweak scheduler example

* Fix date link

* Add DeepSpeed_Adam
Co-authored-by: Shaden Smith <Shaden.Smith@microsoft.com>
Co-authored-by: Jeff Rasley <jerasley@microsoft.com>

23fc48f3

07 Oct, 2020 1 commit

gan tutorial (#462) · 2efea694

niumanar authored Oct 06, 2020



* gan tutorial

* formatting fix

* adding pointer to repo; adding navigation link
Co-authored-by: Jeff Rasley <jerasley@microsoft.com>

2efea694