Commits · f5aa2547d88f145c4b5223d5bb8f79db69ee5288 · OpenDAS / deepspeed

30 Oct, 2020 1 commit

Add CPUAdam optimizer for zero-offload in deepspeed engine (#484) · f5aa2547

Reza Yazdani authored Oct 30, 2020



* add adamW to CPU-ADAM implementation

* supporting cpu-adam optimizer for zero-offload on deepspeed side

* bump DSE to match cpu-adam updates
Co-authored-by: Jeff Rasley <jerasley@microsoft.com>

f5aa2547

19 Oct, 2020 1 commit
- updating website dependencies (#475) · d720fdb6
  Shaden Smith authored Oct 19, 2020
  
  d720fdb6
14 Oct, 2020 1 commit
- Add support for p100 in transformer kernels (#470) · 7ddfda85
  Jeff Rasley authored Oct 14, 2020
```
add compute cap of 6.0, support p100
```
  7ddfda85
12 Oct, 2020 3 commits
- revert previous (accidental) change · 1afca8f7
  Jeff Rasley authored Oct 12, 2020
  
  1afca8f7
- add compute cap of 6.0 to transformer kernels · b8eb40eb
  Jeff Rasley authored Oct 12, 2020
```
add compute cap of 6.0 to transformer kernels
```
  b8eb40eb
- fixing typo (#460) · e25f2a23
  Reza Yazdani authored Oct 12, 2020
  
  e25f2a23
10 Oct, 2020 1 commit

Add DeepSpeed_Adam optimizer (#468) · 23fc48f3

Olatunji Ruwase authored Oct 10, 2020



* Update installation instructions

* Format fix

* ZeRO tutorial

* Format fixes

* ZeRO-Offload

* ZeRO and ZeRO-Offload tutorials

* Update navigation page

* Format fixes

* Add yuxhe feedback

* Fix blog post link

* Fix OneBit-Adam link
Tweak scheduler example

* Fix date link

* Add DeepSpeed_Adam
Co-authored-by: Shaden Smith <Shaden.Smith@microsoft.com>
Co-authored-by: Jeff Rasley <jerasley@microsoft.com>

23fc48f3

07 Oct, 2020 4 commits
- Fix printing momentum for non-deepspeed optimizer (#464) · c39a76fe
  Sylwester Klocek authored Oct 08, 2020
```
* Fix printing momentum for non-deepspeed optimizer

Fix printing momentum for non-deepspeed optimizer

* fix momentum access for Adam

fix momentum access for Adam
```
  c39a76fe
- gan tutorial (#462) · 2efea694
  niumanar authored Oct 06, 2020
```
* gan tutorial

* formatting fix

* adding pointer to repo; adding navigation link
Co-authored-by: Jeff Rasley <jerasley@microsoft.com>
```
  2efea694
- turning off different tests (temp) · 679fc135
  Jeff Rasley authored Oct 06, 2020
  
  679fc135
- temporarily disable lr unit tests · 11cf47ef
  Jeff Rasley authored Oct 06, 2020
  
  11cf47ef
05 Oct, 2020 1 commit
- Update engine.py (#458) · 67176383
  Sylwester Klocek authored Oct 05, 2020
```
Do not load optimizer states for fp32 if flag set to False.
```
  67176383
01 Oct, 2020 1 commit

Use parentesis around min and max to enable Windows build (#449) · 95575579

Bruno authored Oct 01, 2020



* Towards Windows build

* formatting
Co-authored-by: Bruno Cabral <bruno@potelo.com.br>
Co-authored-by: Jeff Rasley <jerasley@microsoft.com>
Co-authored-by: Olatunji Ruwase <olruwase@microsoft.com>

95575579

29 Sep, 2020 1 commit
- Disable default installation of CPU Adam (#450) · 7b8be2a7
  Olatunji Ruwase authored Sep 29, 2020
```
* Disable default installation of CPU Adam

* Handle cpufeature import/use errors separately
```
  7b8be2a7
28 Sep, 2020 1 commit
- fix typos (#446) · 6f28ea30
  Haibin Lin authored Sep 28, 2020
  
  6f28ea30
25 Sep, 2020 3 commits
- unit test rename (#442) · 5412a334
  Shaden Smith authored Sep 25, 2020
  
  5412a334
- link fix part two :-) (#441) · 6d176c45
  Shaden Smith authored Sep 25, 2020
  
  6d176c45
- Update pipeline.md (#439) · 0ca82156
  Haibin Lin authored Sep 24, 2020
  
  0ca82156
24 Sep, 2020 2 commits
- Update azure.md (#437) · 192cf7c8
  Conglong Li authored Sep 24, 2020
  
  192cf7c8
- Fix urls in tutorial (#436) · 5d40f006
  Conglong Li authored Sep 24, 2020
```
* url fix

* revert absolute path but keep some actual fix

* add real readme
```
  5d40f006
22 Sep, 2020 1 commit
- support dynamic sequence length in transformer kernels (#424) · f0f2a702
  RezaYazdaniAminabadi authored Sep 21, 2020
```
Co-authored-by: Conglong Li <conglong.li@gmail.com>
```
  f0f2a702
21 Sep, 2020 2 commits
- DSE bump (#427) · 71f7df39
  Shaden Smith authored Sep 21, 2020
  
  71f7df39
- Add configurable intermediate size to transformer kernels (#423) · a148bd33
  RezaYazdaniAminabadi authored Sep 21, 2020
  
  a148bd33
18 Sep, 2020 3 commits
- Fix activation checkpoint unit tests for GPU systems (#421) · a825f996
  Shaden Smith authored Sep 18, 2020
  
  a825f996
- Revert "Activation checkpointing bugfix and unit tests (#420)" (#422) · a74a604a
  Jeff Rasley authored Sep 18, 2020
```
This reverts commit 01b6e27e

.
Co-authored-by: Shaden Smith <ShadenTSmith@gmail.com>
```
  a74a604a
- Activation checkpointing bugfix and unit tests (#420) · 01b6e27e
  Shaden Smith authored Sep 18, 2020
```
* Activation checkpointing bugfix and unit tests.

* Activation checkpointing bugfix and unit tests.
Co-authored-by: Jeff Rasley <jerasley@microsoft.com>
```
  01b6e27e
17 Sep, 2020 3 commits
- Remove pip --use-feature (#419) · 5bc7d4e1
  Jeff Rasley authored Sep 17, 2020
  
  5bc7d4e1
- Fix few typos in the docs (#418) · c66f3881
  Gowtham Prudhvi authored Sep 17, 2020
  
  c66f3881
- readthedocs yaml configuration (#410) · 5812e845
  Shaden Smith authored Sep 16, 2020
```
Co-authored-by: Jeff Rasley <jerasley@microsoft.com>
```
  5812e845
16 Sep, 2020 4 commits

Fix a typo in comments (#415) · 4fef478f
Haibin Lin authored Sep 16, 2020
```
* Update stage2.py
Co-authored-by: Jeff Rasley <jerasley@microsoft.com>
```
4fef478f
Overflow fix (#416) · f5cce75e
Shaden Smith authored Sep 16, 2020
```
* Switches fused_optimizer overflow calculation
```
f5cce75e

Minor doc fixes (#417) · 7d91be97

Olatunji Ruwase authored Sep 16, 2020



* Update installation instructions

* Format fix

* ZeRO tutorial

* Format fixes

* ZeRO-Offload

* ZeRO and ZeRO-Offload tutorials

* Update navigation page

* Format fixes

* Add yuxhe feedback

* Fix blog post link

* Fix OneBit-Adam link
Tweak scheduler example

* Fix date link
Co-authored-by: Shaden Smith <Shaden.Smith@microsoft.com>
Co-authored-by: Jeff Rasley <jerasley@microsoft.com>

7d91be97

Add Linear warmup+decay lr schedule (#414) · 0e942df0
Olatunji Ruwase authored Sep 16, 2020
```
Update lr schedule unit tests
```
0e942df0

15 Sep, 2020 3 commits
- add cpu-adam, reformat, add colors (#413) · a9e8325f
  Jeff Rasley authored Sep 15, 2020
  
  a9e8325f
- fix bug related to stitching reduced grads across communication partitions (#318) · 55ed1057
  Jeff Rasley authored Sep 15, 2020
  
  55ed1057
- pytest skips for tests requiring certain ops (#411) · 91b4a93d
  Jeff Rasley authored Sep 15, 2020
```
* add pytest skips around tests that require certain ops to be installed
```
  91b4a93d
13 Sep, 2020 1 commit
- scales throughput by logging freq (#408) · 473ff985
  Shaden Smith authored Sep 13, 2020
  
  473ff985
11 Sep, 2020 2 commits
- Revert "supporting different intermediate sizes other than 4 * hidden_dim (#389)" (#404) · 4ac9bf60
  Jeff Rasley authored Sep 11, 2020
```
This reverts commit e549be60.
```
  4ac9bf60
- supporting different intermediate sizes other than 4 * hidden_dim (#389) · e549be60
  RezaYazdaniAminabadi authored Sep 11, 2020
```
* supporting different intermediate sizes other than 4*hidden_dim

* run precommit

* uncommnet the unit tests
Co-authored-by: Jeff Rasley <jerasley@microsoft.com>
```
  e549be60
10 Sep, 2020 1 commit
- readthedocs upgrade (#402) · c82756cd
  Shaden Smith authored Sep 10, 2020
  
  c82756cd