Commits · 7dd6878820df93ffbf17d37d942bd940dde5827a · OpenDAS / deepspeed

26 Apr, 2023 1 commit
- support torch1.13 and torch1.10 · 7dd68788
  aiss authored Apr 26, 2023
  
  7dd68788
30 Mar, 2023 1 commit
- push dsv0.8.2 version · 67ea635f
  aiss authored Mar 30, 2023
  
  67ea635f
10 Aug, 2022 1 commit
- modify version code · 1b2721ad
  aiss authored Aug 10, 2022
  
  1b2721ad
14 Jun, 2022 1 commit
- Update setup.py · c3e434ae
  aiss authored Jun 14, 2022
  
  c3e434ae
11 Jun, 2022 2 commits
- modify whl name · 5da48343
  aiss authored Jun 11, 2022
  
  5da48343
- add dtk version · 9b6449e6
  aiss authored Jun 11, 2022
  
  9b6449e6
25 May, 2022 1 commit
- push Deepspeed 0.6.3 rocm version · 7d1a83a9
  aiss authored May 25, 2022
  
  7d1a83a9
08 Mar, 2021 1 commit

Samyam Rajbhandari authored Mar 08, 2021



* Squash stage3 v1 (#146)
Co-authored-by: Samyam <samyamr@microsoft.com>
Co-authored-by: Jeff Rasley <jerasley@microsoft.com>
Co-authored-by: Samyam Rajbhandari <samyamr@microsoft.com>
Co-authored-by: Olatunji Ruwase <olruwase@microsoft.com>
Co-authored-by: Shaden Smith <Shaden.Smith@microsoft.com>
Co-authored-by: Shaden Smith <ShadenTSmith@gmail.com>
Co-authored-by: eltonzheng <eltonz@microsoft.com>

* Fix correctness bug (#147)

* formatting fix (#150)

* stage3 bugfix (API) update and simplified FP16 Z3 tests (#151)

* fp16 Z3 API update and bugfix

* revert debug change

* ZeRO-3 detach and race condition bugfixes (#149)

* trying out ZeRO-3 race condition fix

* CUDA sync instead of stream

* reduction stream sync

* remove commented code

* Fix optimizer state_dict KeyError (#148)
Co-authored-by: Jeff Rasley <jerasley@microsoft.com>

* fix for smaller SGS sizes, ensures each grad is backed by unique tensors (#152)

* Simplifying the logic for getting averaged gradients (#153)

* skip for now

* Z3 Docs redux (#154)

* removing some TODOs and commented code (#155)

* New Z3 defaults (#156)
Co-authored-by: Jeff Rasley <jerasley@microsoft.com>

* formatting

* megatron external params
Co-authored-by: Jeff Rasley <jerasley@microsoft.com>
Co-authored-by: Olatunji Ruwase <olruwase@microsoft.com>
Co-authored-by: Shaden Smith <Shaden.Smith@microsoft.com>
Co-authored-by: Shaden Smith <ShadenTSmith@gmail.com>
Co-authored-by: eltonzheng <eltonz@microsoft.com>

599258f9

23 Dec, 2020 1 commit
- Elastic training support (#602) · 81aeea36
  Jeff Rasley authored Dec 22, 2020
```
Co-authored-by: Samyam Rajbhandari <samyamr@microsoft.com>
```
  81aeea36
03 Dec, 2020 1 commit
- Add compute capability 8.0 if on cuda 11+ (#572) · be33bea4
  Jeff Rasley authored Dec 02, 2020
  
  be33bea4
25 Nov, 2020 1 commit
- Adds long_description to setup.py (#560) · 60097136
  Shaden Smith authored Nov 25, 2020
  
  60097136
19 Nov, 2020 1 commit

Fix setup.py for cpu-only environment installation (#538) · d81cb26d

Seunghwan Hong authored Nov 20, 2020



* Add guard to not using `torch.version.cuda` above no-CUDA environment.
* Fix several typos on setup.py.
Signed-off-by: Seunghwan Hong <seunghwan@scatterlab.co.kr>
Co-authored-by: Jeff Rasley <jerasley@microsoft.com>

d81cb26d

12 Nov, 2020 2 commits

ds_report bug fix on cpu and guard torch import in setup.py (#524) · ca9ab120

Jeff Rasley authored Nov 12, 2020

* on cpu box error gracefully if cuda home doesn't exist

* gaurd against torch import issue

* fix sytax error

* fix import

ca9ab120

DeepSpeed JIT op + PyPI support (#496) · 31f46fee

Jeff Rasley authored Nov 12, 2020


Co-authored-by: Shaden Smith <Shaden.Smith@microsoft.com>
Co-authored-by: Reza Yazdani <reyazda@microsoft.com>

31f46fee

14 Oct, 2020 1 commit
- Add support for p100 in transformer kernels (#470) · 7ddfda85
  Jeff Rasley authored Oct 14, 2020
```
add compute cap of 6.0, support p100
```
  7ddfda85
12 Oct, 2020 2 commits
- revert previous (accidental) change · 1afca8f7
  Jeff Rasley authored Oct 12, 2020
  
  1afca8f7
- add compute cap of 6.0 to transformer kernels · b8eb40eb
  Jeff Rasley authored Oct 12, 2020
```
add compute cap of 6.0 to transformer kernels
```
  b8eb40eb
29 Sep, 2020 1 commit
- Disable default installation of CPU Adam (#450) · 7b8be2a7
  Olatunji Ruwase authored Sep 29, 2020
```
* Disable default installation of CPU Adam

* Handle cpufeature import/use errors separately
```
  7b8be2a7
17 Sep, 2020 1 commit
- readthedocs yaml configuration (#410) · 5812e845
  Shaden Smith authored Sep 16, 2020
```
Co-authored-by: Jeff Rasley <jerasley@microsoft.com>
```
  5812e845
10 Sep, 2020 2 commits

only add 1bit adam reqs if mpi is installed, update cond build for cpu-adam (#400) · 240ea97b
Jeff Rasley authored Sep 10, 2020

240ea97b

ZeRO-Offload release (#391) · 41db1c2f

Jeff Rasley authored Sep 09, 2020



* ZeRO-Offload (squash) (#381)
Co-authored-by: Olatunji Ruwase <olruwase@microsoft.com>
Co-authored-by: Reza Yazdani <reyazda@microsoft.com>
Co-authored-by: Jeff Rasley <jerasley@microsoft.com>
Co-authored-by: Jie <37380896+jren73@users.noreply.github.com>
Co-authored-by: Arash Ashari <arashari@microsoft.com>
Co-authored-by: Reza Yazdani <reyazda@microsoft.com>
Co-authored-by: Samyam Rajbhandari <samyamr@microsoft.com>
Co-authored-by: Shaden Smith <Shaden.Smith@microsoft.com>
Co-authored-by: arashashari <arashashari@ArashMSLaptop.redmond.corp.microsoft.com>
Co-authored-by: RezaYazdaniAminabadi <44502768+RezaYazdaniAminabadi@users.noreply.github.com>
Co-authored-by: Reza Yazdani <reyazda@microsoft.com>
Co-authored-by: Samyam Rajbhandari <samyamr@microsoft.com>
Co-authored-by: Shaden Smith <Shaden.Smith@microsoft.com>

41db1c2f

09 Sep, 2020 1 commit

Add 1-bit Adam support to DeepSpeed (#380) · 01726ce2

Ammar Ahmad Awan authored Sep 09, 2020



* 1-bit adam (#353)
Co-authored-by: Jeff Rasley <jerasley@microsoft.com>
Co-authored-by: Your Name <you@example.com>
Co-authored-by: tanghl1994 <htang14@ur.rochester.edu>
Co-authored-by: Hank <tanghl1994@gmail.com>
Co-authored-by: root <root@node2x12b.cs.rochester.edu>
Co-authored-by: Ammar Ahmad Awan <awan.ammar@microsoft.com>

01726ce2

02 Sep, 2020 1 commit

Sparse attn + ops/runtime refactor + v0.3.0 (#343) · e5bbc2e5

Jeff Rasley authored Sep 01, 2020



* Sparse attn + ops/runtime refactor + v0.3.0
Co-authored-by: Arash Ashari <arashari@microsoft.com>
Co-authored-by: Arash Ashari <arashari@microsoft.com>

e5bbc2e5

22 Jul, 2020 1 commit
- Update setup.py (#298) · 871f7e63
  Jeff Rasley authored Jul 22, 2020
  
  871f7e63
29 May, 2020 1 commit

Transformer kernel release (#242) · 734d8991

Jeff Rasley authored May 29, 2020



* Transformer kernels release
Co-authored-by: Shaden Smith <ShadenTSmith@gmail.com>
Co-authored-by: Elton Zheng <eltonz@microsoft.com>
Co-authored-by: Reza Yazdani <reyazda@microsoft.com>
Co-authored-by: RezaYazdaniAminabadi <44502768+RezaYazdaniAminabadi@users.noreply.github.com>
Co-authored-by: Tunji Ruwase <olruwase@microsoft.com>
Co-authored-by: Shaden Smith <ShadenTSmith@gmail.com>
Co-authored-by: Shaden Smith <Shaden.Smith@microsoft.com>
Co-authored-by: Samyam Rajbhandari <samyamr@microsoft.com>
Co-authored-by: Shaden Smith <ShadenTSmith@gmail.com>
Co-authored-by: Jeff Rasley <jerasley@microsoft.com>
Co-authored-by: Samyam Rajbhandari <samyamr@microsoft.com>
Co-authored-by: Shaden Smith <ShadenTSmith@gmail.com>
Co-authored-by: Elton Zheng <eltonz@microsoft.com>
Co-authored-by: Reza Yazdani <reyazda@microsoft.com>
Co-authored-by: RezaYazdaniAminabadi <44502768+RezaYazdaniAminabadi@users.noreply.github.com>
Co-authored-by: Tunji Ruwase <olruwase@microsoft.com>
Co-authored-by: Shaden Smith <Shaden.Smith@microsoft.com>
Co-authored-by: Samyam Rajbhandari <samyamr@microsoft.com>

734d8991

12 Mar, 2020 1 commit

PyTorch 1.3+ build support (#135) · 3d3f8d36

Jeff Rasley authored Mar 12, 2020

* add support for torch 1.3+ builds inside a docker build environment
* remove apex imports

3d3f8d36

10 Feb, 2020 1 commit
- Moving to major/minor/patch versioning. (#51) · 50ae149f
  Shaden Smith authored Feb 09, 2020
  
  50ae149f
09 Feb, 2020 1 commit
- Azure tutorial updates and cleanup (#43) · 20ff66a0
  Jeff Rasley authored Feb 08, 2020
  
  20ff66a0
03 Feb, 2020 1 commit
- updating formatting to pass yapf · a5acb5b2
  Jeff Rasley authored Feb 03, 2020
  
  a5acb5b2
31 Jan, 2020 1 commit
- Initial commit of setup · 26e5ecab
  Jeff Rasley authored Jan 31, 2020
  
  26e5ecab