Commits · be1147c08acde34d7a13b73f8d33e55c3f719de4 · OpenDAS / deepspeed

10 Nov, 2020 2 commits

Olatunji Ruwase authored Nov 10, 2020



* Progressive layer dropping docs (#499)

* test

* Adding tutorial and news page for pld

* updating the tutorial and posts of PLD

* update the finetune tutorial

* Update PLD tutorial (#512)

* Update installation instructions

* Format fix

* ZeRO tutorial

* Format fixes

* ZeRO-Offload

* ZeRO and ZeRO-Offload tutorials

* Update navigation page

* Format fixes

* Add yuxhe feedback

* Fix blog post link

* Fix OneBit-Adam link
Tweak scheduler example

* Fix date link

* Add DeepSpeed_Adam

* Add PLD tutorial to navigation
Co-authored-by: Shaden Smith <Shaden.Smith@microsoft.com>
Co-authored-by: Jeff Rasley <jerasley@microsoft.com>

* updating the pld docs

* DeepSpeed implementation of PLD (#508)

* DeepSpeed implementation of PLD

* Format fixes

* Formatting fixes

* Fix broken url

* Address PR feedback

* Bump DSE
Co-authored-by: Minjia Zhang <33713995+minjiaz@users.noreply.github.com>
Co-authored-by: Shaden Smith <Shaden.Smith@microsoft.com>
Co-authored-by: Jeff Rasley <jerasley@microsoft.com>
Co-authored-by: Minjia Zhang <minjiaz@microsoft.com>

be1147c0

updating pld docs (#517) · e082d475
Minjia Zhang authored Nov 10, 2020

e082d475

09 Nov, 2020 2 commits
- Fix PLD news url (#515) · 41fb24b3
  Olatunji Ruwase authored Nov 09, 2020
```
* PLD documentation

* Formatting fixes

* Fix url bug
```
  41fb24b3
- PLD documentation (#514) · e351090c
  Olatunji Ruwase authored Nov 09, 2020
```
* PLD documentation

* Formatting fixes
```
  e351090c
05 Nov, 2020 1 commit

Fixing CPU-Adam convergence issue (#503) · 7d4d742b

Reza Yazdani authored Nov 05, 2020

* fixing cpu-adam

* fixing copy with optimizer for data and model parallelism

* fixing cpu-adam

* fix cpu-adam

* fix cpu-adam

7d4d742b

30 Oct, 2020 2 commits
- fixing the AVX_256 compatibility (#497) · 4c37d705
  Reza Yazdani authored Oct 30, 2020
  
  4c37d705
- Add CPUAdam optimizer for zero-offload in deepspeed engine (#484) · f5aa2547
  Reza Yazdani authored Oct 30, 2020
```
* add adamW to CPU-ADAM implementation

* supporting cpu-adam optimizer for zero-offload on deepspeed side

* bump DSE to match cpu-adam updates
Co-authored-by: Jeff Rasley <jerasley@microsoft.com>
```
  f5aa2547
19 Oct, 2020 1 commit
- updating website dependencies (#475) · d720fdb6
  Shaden Smith authored Oct 19, 2020
  
  d720fdb6
14 Oct, 2020 1 commit
- Add support for p100 in transformer kernels (#470) · 7ddfda85
  Jeff Rasley authored Oct 14, 2020
```
add compute cap of 6.0, support p100
```
  7ddfda85
12 Oct, 2020 3 commits
- revert previous (accidental) change · 1afca8f7
  Jeff Rasley authored Oct 12, 2020
  
  1afca8f7
- add compute cap of 6.0 to transformer kernels · b8eb40eb
  Jeff Rasley authored Oct 12, 2020
```
add compute cap of 6.0 to transformer kernels
```
  b8eb40eb
- fixing typo (#460) · e25f2a23
  Reza Yazdani authored Oct 12, 2020
  
  e25f2a23
10 Oct, 2020 1 commit

Add DeepSpeed_Adam optimizer (#468) · 23fc48f3

Olatunji Ruwase authored Oct 10, 2020



* Update installation instructions

* Format fix

* ZeRO tutorial

* Format fixes

* ZeRO-Offload

* ZeRO and ZeRO-Offload tutorials

* Update navigation page

* Format fixes

* Add yuxhe feedback

* Fix blog post link

* Fix OneBit-Adam link
Tweak scheduler example

* Fix date link

* Add DeepSpeed_Adam
Co-authored-by: Shaden Smith <Shaden.Smith@microsoft.com>
Co-authored-by: Jeff Rasley <jerasley@microsoft.com>

23fc48f3

07 Oct, 2020 4 commits
- Fix printing momentum for non-deepspeed optimizer (#464) · c39a76fe
  Sylwester Klocek authored Oct 08, 2020
```
* Fix printing momentum for non-deepspeed optimizer

Fix printing momentum for non-deepspeed optimizer

* fix momentum access for Adam

fix momentum access for Adam
```
  c39a76fe
- gan tutorial (#462) · 2efea694
  niumanar authored Oct 06, 2020
```
* gan tutorial

* formatting fix

* adding pointer to repo; adding navigation link
Co-authored-by: Jeff Rasley <jerasley@microsoft.com>
```
  2efea694
- turning off different tests (temp) · 679fc135
  Jeff Rasley authored Oct 06, 2020
  
  679fc135
- temporarily disable lr unit tests · 11cf47ef
  Jeff Rasley authored Oct 06, 2020
  
  11cf47ef
05 Oct, 2020 1 commit
- Update engine.py (#458) · 67176383
  Sylwester Klocek authored Oct 05, 2020
```
Do not load optimizer states for fp32 if flag set to False.
```
  67176383
01 Oct, 2020 1 commit

Use parentesis around min and max to enable Windows build (#449) · 95575579

Bruno authored Oct 01, 2020



* Towards Windows build

* formatting
Co-authored-by: Bruno Cabral <bruno@potelo.com.br>
Co-authored-by: Jeff Rasley <jerasley@microsoft.com>
Co-authored-by: Olatunji Ruwase <olruwase@microsoft.com>

95575579

29 Sep, 2020 1 commit
- Disable default installation of CPU Adam (#450) · 7b8be2a7
  Olatunji Ruwase authored Sep 29, 2020
```
* Disable default installation of CPU Adam

* Handle cpufeature import/use errors separately
```
  7b8be2a7
28 Sep, 2020 1 commit
- fix typos (#446) · 6f28ea30
  Haibin Lin authored Sep 28, 2020
  
  6f28ea30
25 Sep, 2020 3 commits
- unit test rename (#442) · 5412a334
  Shaden Smith authored Sep 25, 2020
  
  5412a334
- link fix part two :-) (#441) · 6d176c45
  Shaden Smith authored Sep 25, 2020
  
  6d176c45
- Update pipeline.md (#439) · 0ca82156
  Haibin Lin authored Sep 24, 2020
  
  0ca82156
24 Sep, 2020 2 commits
- Update azure.md (#437) · 192cf7c8
  Conglong Li authored Sep 24, 2020
  
  192cf7c8
- Fix urls in tutorial (#436) · 5d40f006
  Conglong Li authored Sep 24, 2020
```
* url fix

* revert absolute path but keep some actual fix

* add real readme
```
  5d40f006
22 Sep, 2020 1 commit
- support dynamic sequence length in transformer kernels (#424) · f0f2a702
  RezaYazdaniAminabadi authored Sep 21, 2020
```
Co-authored-by: Conglong Li <conglong.li@gmail.com>
```
  f0f2a702
21 Sep, 2020 2 commits
- DSE bump (#427) · 71f7df39
  Shaden Smith authored Sep 21, 2020
  
  71f7df39
- Add configurable intermediate size to transformer kernels (#423) · a148bd33
  RezaYazdaniAminabadi authored Sep 21, 2020
  
  a148bd33
18 Sep, 2020 3 commits
- Fix activation checkpoint unit tests for GPU systems (#421) · a825f996
  Shaden Smith authored Sep 18, 2020
  
  a825f996
- Revert "Activation checkpointing bugfix and unit tests (#420)" (#422) · a74a604a
  Jeff Rasley authored Sep 18, 2020
```
This reverts commit 01b6e27e

.
Co-authored-by: Shaden Smith <ShadenTSmith@gmail.com>
```
  a74a604a
- Activation checkpointing bugfix and unit tests (#420) · 01b6e27e
  Shaden Smith authored Sep 18, 2020
```
* Activation checkpointing bugfix and unit tests.

* Activation checkpointing bugfix and unit tests.
Co-authored-by: Jeff Rasley <jerasley@microsoft.com>
```
  01b6e27e
17 Sep, 2020 3 commits
- Remove pip --use-feature (#419) · 5bc7d4e1
  Jeff Rasley authored Sep 17, 2020
  
  5bc7d4e1
- Fix few typos in the docs (#418) · c66f3881
  Gowtham Prudhvi authored Sep 17, 2020
  
  c66f3881
- readthedocs yaml configuration (#410) · 5812e845
  Shaden Smith authored Sep 16, 2020
```
Co-authored-by: Jeff Rasley <jerasley@microsoft.com>
```
  5812e845
16 Sep, 2020 4 commits

Fix a typo in comments (#415) · 4fef478f
Haibin Lin authored Sep 16, 2020
```
* Update stage2.py
Co-authored-by: Jeff Rasley <jerasley@microsoft.com>
```
4fef478f
Overflow fix (#416) · f5cce75e
Shaden Smith authored Sep 16, 2020
```
* Switches fused_optimizer overflow calculation
```
f5cce75e

Minor doc fixes (#417) · 7d91be97

Olatunji Ruwase authored Sep 16, 2020



* Update installation instructions

* Format fix

* ZeRO tutorial

* Format fixes

* ZeRO-Offload

* ZeRO and ZeRO-Offload tutorials

* Update navigation page

* Format fixes

* Add yuxhe feedback

* Fix blog post link

* Fix OneBit-Adam link
Tweak scheduler example

* Fix date link
Co-authored-by: Shaden Smith <Shaden.Smith@microsoft.com>
Co-authored-by: Jeff Rasley <jerasley@microsoft.com>

7d91be97

Add Linear warmup+decay lr schedule (#414) · 0e942df0
Olatunji Ruwase authored Sep 16, 2020
```
Update lr schedule unit tests
```
0e942df0

15 Sep, 2020 1 commit
- add cpu-adam, reformat, add colors (#413) · a9e8325f
  Jeff Rasley authored Sep 15, 2020
  
  a9e8325f