- 27 Apr, 2023 1 commit
-
-
aiss authored
-
- 26 Apr, 2023 1 commit
-
-
aiss authored
-
- 30 Mar, 2023 1 commit
-
-
aiss authored
-
- 10 Aug, 2022 1 commit
-
-
aiss authored
-
- 26 May, 2022 1 commit
-
-
aiss authored
-
- 25 May, 2022 1 commit
-
-
aiss authored
-
- 08 Mar, 2021 1 commit
-
-
Samyam Rajbhandari authored
* Squash stage3 v1 (#146) Co-authored-by:
Samyam <samyamr@microsoft.com> Co-authored-by:
Jeff Rasley <jerasley@microsoft.com> Co-authored-by:
Samyam Rajbhandari <samyamr@microsoft.com> Co-authored-by:
Olatunji Ruwase <olruwase@microsoft.com> Co-authored-by:
Shaden Smith <Shaden.Smith@microsoft.com> Co-authored-by:
Shaden Smith <ShadenTSmith@gmail.com> Co-authored-by:
eltonzheng <eltonz@microsoft.com> * Fix correctness bug (#147) * formatting fix (#150) * stage3 bugfix (API) update and simplified FP16 Z3 tests (#151) * fp16 Z3 API update and bugfix * revert debug change * ZeRO-3 detach and race condition bugfixes (#149) * trying out ZeRO-3 race condition fix * CUDA sync instead of stream * reduction stream sync * remove commented code * Fix optimizer state_dict KeyError (#148) Co-authored-by:
Jeff Rasley <jerasley@microsoft.com> * fix for smaller SGS sizes, ensures each grad is backed by unique tensors (#152) * Simplifying the logic for getting averaged gradients (#153) * skip for now * Z3 Docs redux (#154) * removing some TODOs and commented code (#155) * New Z3 defaults (#156) Co-authored-by:
Jeff Rasley <jerasley@microsoft.com> * formatting * megatron external params Co-authored-by:
Jeff Rasley <jerasley@microsoft.com> Co-authored-by:
Olatunji Ruwase <olruwase@microsoft.com> Co-authored-by:
Shaden Smith <Shaden.Smith@microsoft.com> Co-authored-by:
Shaden Smith <ShadenTSmith@gmail.com> Co-authored-by:
eltonzheng <eltonz@microsoft.com>
-
- 26 Feb, 2021 1 commit
-
-
Reza Yazdani authored
Co-authored-by:Jeff Rasley <jerasley@microsoft.com>
-
- 07 Jan, 2021 1 commit
-
-
Xingjian Shi authored
-
- 04 Jan, 2021 1 commit
-
-
Jeff Rasley authored
-
- 23 Dec, 2020 1 commit
-
-
Jeff Rasley authored
Co-authored-by:Samyam Rajbhandari <samyamr@microsoft.com>
-
- 11 Dec, 2020 1 commit
-
-
Stas Bekman authored
* fix arch flags, add PTX * bug fix Co-authored-by:Jeff Rasley <jerasley@microsoft.com>
-
- 07 Dec, 2020 2 commits
-
-
Stas Bekman authored
RTX-30 series are compute_86 ``` python -c "import torch; print(torch.cuda.get_device_capability())" ``` This PR adds support for this compute capability. Reference: https://developer.nvidia.com/cuda-gpus Co-authored-by:
Jeff Rasley <jerasley@microsoft.com>
-
Stas Bekman authored
-
- 03 Dec, 2020 2 commits
-
-
Stas Bekman authored
-
Jeff Rasley authored
-
- 14 Nov, 2020 1 commit
-
-
Jeff Rasley authored
* remove cpu-feature * remove psutils requirement
-
- 12 Nov, 2020 1 commit
-
-
Jeff Rasley authored
Co-authored-by:
Shaden Smith <Shaden.Smith@microsoft.com> Co-authored-by:
Reza Yazdani <reyazda@microsoft.com>
-