Commits · 5bcc463d05955ba7a11238450d039978a2d67387 · OpenDAS / deepspeed

"git@developer.sourcefind.cn:OpenDAS/torch-scatter.git" did not exist on "6cf70eadbd74f0ec97488861090246ab9ef0c71f"

29 May, 2023 1 commit
- update v0.9.2 · 5bcc463d
  aiss authored May 29, 2023
  
  5bcc463d
30 Mar, 2023 1 commit
- push dsv0.8.2 version · 67ea635f
  aiss authored Mar 30, 2023
  
  67ea635f
25 May, 2022 1 commit
- push Deepspeed 0.6.3 rocm version · 7d1a83a9
  aiss authored May 25, 2022
  
  7d1a83a9
11 Mar, 2021 1 commit
- small tweaks (#839) · 7925d0c3
  Stas Bekman authored Mar 11, 2021
  
  7925d0c3
08 Mar, 2021 1 commit

Samyam Rajbhandari authored Mar 08, 2021



* Squash stage3 v1 (#146)
Co-authored-by: Samyam <samyamr@microsoft.com>
Co-authored-by: Jeff Rasley <jerasley@microsoft.com>
Co-authored-by: Samyam Rajbhandari <samyamr@microsoft.com>
Co-authored-by: Olatunji Ruwase <olruwase@microsoft.com>
Co-authored-by: Shaden Smith <Shaden.Smith@microsoft.com>
Co-authored-by: Shaden Smith <ShadenTSmith@gmail.com>
Co-authored-by: eltonzheng <eltonz@microsoft.com>

* Fix correctness bug (#147)

* formatting fix (#150)

* stage3 bugfix (API) update and simplified FP16 Z3 tests (#151)

* fp16 Z3 API update and bugfix

* revert debug change

* ZeRO-3 detach and race condition bugfixes (#149)

* trying out ZeRO-3 race condition fix

* CUDA sync instead of stream

* reduction stream sync

* remove commented code

* Fix optimizer state_dict KeyError (#148)
Co-authored-by: Jeff Rasley <jerasley@microsoft.com>

* fix for smaller SGS sizes, ensures each grad is backed by unique tensors (#152)

* Simplifying the logic for getting averaged gradients (#153)

* skip for now

* Z3 Docs redux (#154)

* removing some TODOs and commented code (#155)

* New Z3 defaults (#156)
Co-authored-by: Jeff Rasley <jerasley@microsoft.com>

* formatting

* megatron external params
Co-authored-by: Jeff Rasley <jerasley@microsoft.com>
Co-authored-by: Olatunji Ruwase <olruwase@microsoft.com>
Co-authored-by: Shaden Smith <Shaden.Smith@microsoft.com>
Co-authored-by: Shaden Smith <ShadenTSmith@gmail.com>
Co-authored-by: eltonzheng <eltonz@microsoft.com>

599258f9