Commits · 4c972a6eb96c250cf40166d2f1845deb7d15fe81 · OpenDAS / deepspeed

12 Oct, 2023 1 commit
- add llvm compile arg for dtk2310 · 4c972a6e
  aiss authored Oct 12, 2023
  
  4c972a6e
14 Jun, 2023 1 commit
- add block_thread arg · 25d5540b
  aiss authored Jun 14, 2023
  
  25d5540b
29 May, 2023 1 commit
- update v0.9.2 · 5bcc463d
  aiss authored May 29, 2023
  
  5bcc463d
27 Apr, 2023 1 commit
- modify error · 0f3656b9
  aiss authored Apr 27, 2023
  
  0f3656b9
26 Apr, 2023 1 commit
- support torch1.13 and torch1.10 · 7dd68788
  aiss authored Apr 26, 2023
  
  7dd68788
30 Mar, 2023 1 commit
- push dsv0.8.2 version · 67ea635f
  aiss authored Mar 30, 2023
  
  67ea635f
10 Aug, 2022 1 commit
- modify version code · 1b2721ad
  aiss authored Aug 10, 2022
  
  1b2721ad
26 May, 2022 1 commit
- modify dtk path · d8669b08
  aiss authored May 26, 2022
  
  d8669b08
25 May, 2022 1 commit
- push Deepspeed 0.6.3 rocm version · 7d1a83a9
  aiss authored May 25, 2022
  
  7d1a83a9
08 Mar, 2021 1 commit

Samyam Rajbhandari authored Mar 08, 2021



* Squash stage3 v1 (#146)
Co-authored-by: Samyam <samyamr@microsoft.com>
Co-authored-by: Jeff Rasley <jerasley@microsoft.com>
Co-authored-by: Samyam Rajbhandari <samyamr@microsoft.com>
Co-authored-by: Olatunji Ruwase <olruwase@microsoft.com>
Co-authored-by: Shaden Smith <Shaden.Smith@microsoft.com>
Co-authored-by: Shaden Smith <ShadenTSmith@gmail.com>
Co-authored-by: eltonzheng <eltonz@microsoft.com>

* Fix correctness bug (#147)

* formatting fix (#150)

* stage3 bugfix (API) update and simplified FP16 Z3 tests (#151)

* fp16 Z3 API update and bugfix

* revert debug change

* ZeRO-3 detach and race condition bugfixes (#149)

* trying out ZeRO-3 race condition fix

* CUDA sync instead of stream

* reduction stream sync

* remove commented code

* Fix optimizer state_dict KeyError (#148)
Co-authored-by: Jeff Rasley <jerasley@microsoft.com>

* fix for smaller SGS sizes, ensures each grad is backed by unique tensors (#152)

* Simplifying the logic for getting averaged gradients (#153)

* skip for now

* Z3 Docs redux (#154)

* removing some TODOs and commented code (#155)

* New Z3 defaults (#156)
Co-authored-by: Jeff Rasley <jerasley@microsoft.com>

* formatting

* megatron external params
Co-authored-by: Jeff Rasley <jerasley@microsoft.com>
Co-authored-by: Olatunji Ruwase <olruwase@microsoft.com>
Co-authored-by: Shaden Smith <Shaden.Smith@microsoft.com>
Co-authored-by: Shaden Smith <ShadenTSmith@gmail.com>
Co-authored-by: eltonzheng <eltonz@microsoft.com>

599258f9

26 Feb, 2021 1 commit
- fixing the compiling issue for the AMD architecture (#796) · 490e6f7c
  Reza Yazdani authored Feb 26, 2021
```
Co-authored-by: Jeff Rasley <jerasley@microsoft.com>
```
  490e6f7c
07 Jan, 2021 1 commit
- Update builder.py (#642) · 64461da4
  Xingjian Shi authored Jan 06, 2021
  
  64461da4
04 Jan, 2021 1 commit
- update SA comp check to fix torch-cpu issue (#631) · 24e07399
  Jeff Rasley authored Jan 04, 2021
  
  24e07399
23 Dec, 2020 1 commit
- Elastic training support (#602) · 81aeea36
  Jeff Rasley authored Dec 22, 2020
```
Co-authored-by: Samyam Rajbhandari <samyamr@microsoft.com>
```
  81aeea36
11 Dec, 2020 1 commit
- [build] fix computer capability arch flags, add PTX, handle PTX (#591) · 8a184b6b
  Stas Bekman authored Dec 11, 2020
```
* fix arch flags, add PTX

* bug fix
Co-authored-by: Jeff Rasley <jerasley@microsoft.com>
```
  8a184b6b
07 Dec, 2020 2 commits

[build] add compute_86 (#577) · e8b126d9

Stas Bekman authored Dec 07, 2020

RTX-30 series are compute_86
```
python -c "import torch; print(torch.cuda.get_device_capability())"
```
This PR adds support for this compute capability.

Reference: https://developer.nvidia.com/cuda-gpus

Co-authored-by: Jeff Rasley <jerasley@microsoft.com>

e8b126d9

[build] make builder smarter and configurable wrt compute capabilities + docs (#578) · ce363d0e
Stas Bekman authored Dec 07, 2020

ce363d0e

03 Dec, 2020 2 commits
- [build] build against installed cuda-11.1 while torch built w/ cuda-11.0 (#570) · ff58fa7e
  Stas Bekman authored Dec 02, 2020
  
  ff58fa7e
- Add compute capability 8.0 if on cuda 11+ (#572) · be33bea4
  Jeff Rasley authored Dec 02, 2020
  
  be33bea4
14 Nov, 2020 1 commit
- Dependency pruning (#528) · 0dc84200
  Jeff Rasley authored Nov 13, 2020
```
* remove cpu-feature

* remove psutils requirement
```
  0dc84200
12 Nov, 2020 1 commit

DeepSpeed JIT op + PyPI support (#496) · 31f46fee

Jeff Rasley authored Nov 12, 2020


Co-authored-by: Shaden Smith <Shaden.Smith@microsoft.com>
Co-authored-by: Reza Yazdani <reyazda@microsoft.com>

31f46fee