Commits · 0ca82156e1799672a9522e539c65f379bdb91a2a · OpenDAS / deepspeed

25 Sep, 2020 1 commit
- Update pipeline.md (#439) · 0ca82156
  Haibin Lin authored Sep 24, 2020
  
  0ca82156
24 Sep, 2020 2 commits
- Update azure.md (#437) · 192cf7c8
  Conglong Li authored Sep 24, 2020
  
  192cf7c8
- Fix urls in tutorial (#436) · 5d40f006
  Conglong Li authored Sep 24, 2020
```
* url fix

* revert absolute path but keep some actual fix

* add real readme
```
  5d40f006
22 Sep, 2020 1 commit
- support dynamic sequence length in transformer kernels (#424) · f0f2a702
  RezaYazdaniAminabadi authored Sep 21, 2020
```
Co-authored-by: Conglong Li <conglong.li@gmail.com>
```
  f0f2a702
21 Sep, 2020 2 commits
- DSE bump (#427) · 71f7df39
  Shaden Smith authored Sep 21, 2020
  
  71f7df39
- Add configurable intermediate size to transformer kernels (#423) · a148bd33
  RezaYazdaniAminabadi authored Sep 21, 2020
  
  a148bd33
18 Sep, 2020 3 commits
- Fix activation checkpoint unit tests for GPU systems (#421) · a825f996
  Shaden Smith authored Sep 18, 2020
  
  a825f996
- Revert "Activation checkpointing bugfix and unit tests (#420)" (#422) · a74a604a
  Jeff Rasley authored Sep 18, 2020
```
This reverts commit 01b6e27e

.
Co-authored-by: Shaden Smith <ShadenTSmith@gmail.com>
```
  a74a604a
- Activation checkpointing bugfix and unit tests (#420) · 01b6e27e
  Shaden Smith authored Sep 18, 2020
```
* Activation checkpointing bugfix and unit tests.

* Activation checkpointing bugfix and unit tests.
Co-authored-by: Jeff Rasley <jerasley@microsoft.com>
```
  01b6e27e
17 Sep, 2020 3 commits
- Remove pip --use-feature (#419) · 5bc7d4e1
  Jeff Rasley authored Sep 17, 2020
  
  5bc7d4e1
- Fix few typos in the docs (#418) · c66f3881
  Gowtham Prudhvi authored Sep 17, 2020
  
  c66f3881
- readthedocs yaml configuration (#410) · 5812e845
  Shaden Smith authored Sep 16, 2020
```
Co-authored-by: Jeff Rasley <jerasley@microsoft.com>
```
  5812e845
16 Sep, 2020 4 commits

Fix a typo in comments (#415) · 4fef478f
Haibin Lin authored Sep 16, 2020
```
* Update stage2.py
Co-authored-by: Jeff Rasley <jerasley@microsoft.com>
```
4fef478f
Overflow fix (#416) · f5cce75e
Shaden Smith authored Sep 16, 2020
```
* Switches fused_optimizer overflow calculation
```
f5cce75e

Olatunji Ruwase authored Sep 16, 2020



* Update installation instructions

* Format fix

* ZeRO tutorial

* Format fixes

* ZeRO-Offload

* ZeRO and ZeRO-Offload tutorials

* Update navigation page

* Format fixes

* Add yuxhe feedback

* Fix blog post link

* Fix OneBit-Adam link
Tweak scheduler example

* Fix date link
Co-authored-by: Shaden Smith <Shaden.Smith@microsoft.com>
Co-authored-by: Jeff Rasley <jerasley@microsoft.com>

7d91be97

Add Linear warmup+decay lr schedule (#414) · 0e942df0
Olatunji Ruwase authored Sep 16, 2020
```
Update lr schedule unit tests
```
0e942df0

15 Sep, 2020 3 commits
- add cpu-adam, reformat, add colors (#413) · a9e8325f
  Jeff Rasley authored Sep 15, 2020
  
  a9e8325f
- fix bug related to stitching reduced grads across communication partitions (#318) · 55ed1057
  Jeff Rasley authored Sep 15, 2020
  
  55ed1057
- pytest skips for tests requiring certain ops (#411) · 91b4a93d
  Jeff Rasley authored Sep 15, 2020
```
* add pytest skips around tests that require certain ops to be installed
```
  91b4a93d
13 Sep, 2020 1 commit
- scales throughput by logging freq (#408) · 473ff985
  Shaden Smith authored Sep 13, 2020
  
  473ff985
11 Sep, 2020 2 commits
- Revert "supporting different intermediate sizes other than 4 * hidden_dim (#389)" (#404) · 4ac9bf60
  Jeff Rasley authored Sep 11, 2020
```
This reverts commit e549be60.
```
  4ac9bf60
- supporting different intermediate sizes other than 4 * hidden_dim (#389) · e549be60
  RezaYazdaniAminabadi authored Sep 11, 2020
```
* supporting different intermediate sizes other than 4*hidden_dim

* run precommit

* uncommnet the unit tests
Co-authored-by: Jeff Rasley <jerasley@microsoft.com>
```
  e549be60
10 Sep, 2020 18 commits
- readthedocs upgrade (#402) · c82756cd
  Shaden Smith authored Sep 10, 2020
  
  c82756cd
- remove old pt file · 15ca99c6
  Jeff Rasley authored Sep 10, 2020
  
  15ca99c6
- Update ZeRO-Offload blog post link (#401) · d15015e9
  Olatunji Ruwase authored Sep 10, 2020
```
Co-authored-by: Shaden Smith <Shaden.Smith@microsoft.com>
Co-authored-by: Jeff Rasley <jerasley@microsoft.com>
```
  d15015e9
- Update news site with press release link · 5dc4d6c7
  Jeff Rasley authored Sep 10, 2020
  
  5dc4d6c7
- Update _config.yml · ea92ed29
  Jeff Rasley authored Sep 10, 2020
  
  ea92ed29
- Update README.md · 96935958
  Jeff Rasley authored Sep 10, 2020
  
  96935958
- bump DSE and doc tweak · 4b1df25a
  Jeff Rasley authored Sep 10, 2020
  
  4b1df25a
- only add 1bit adam reqs if mpi is installed, update cond build for cpu-adam (#400) · 240ea97b
  Jeff Rasley authored Sep 10, 2020
  
  240ea97b
- update docker image and bump DSE · b29229bf
  Jeff Rasley authored Sep 10, 2020
  
  b29229bf
- Website edits (#398) · 6bb5c69f
  Shaden Smith authored Sep 10, 2020
```
Co-authored-by: Jeff Rasley <jerasley@microsoft.com>
```
  6bb5c69f
- Update README.md · 7baf3c3a
  Jeff Rasley authored Sep 10, 2020
  
  7baf3c3a
- Landing page updates (#395) · a8a8b3d2
  Jeff Rasley authored Sep 10, 2020
```
Co-authored-by: Shaden Smith <ShadenTSmith@gmail.com>
Co-authored-by: Shaden Smith <Shaden.Smith@microsoft.com>
```
  a8a8b3d2
- Adding sparse attention news index item (#376) · c76769c4
  Arash Ashari authored Sep 10, 2020
```
Co-authored-by: Jeff Rasley <jerasley@microsoft.com>
```
  c76769c4
- Minjiaz/zero offload (#382) · 59ce90d0
  Minjia Zhang authored Sep 10, 2020
```
Co-authored-by: Jeff Rasley <jerasley@microsoft.com>
```
  59ce90d0
- Sparse attention: updating code tag in documentation (#394) · be4b94be
  Arash Ashari authored Sep 10, 2020
```
Co-authored-by: Jeff Rasley <jerasley@microsoft.com>
```
  be4b94be
- fix for 16GB v100 nodes (#393) · b1d4bd73
  Jeff Rasley authored Sep 10, 2020
  
  b1d4bd73
- ZeRO tutorials (#384) · 2dea61f2
  Olatunji Ruwase authored Sep 10, 2020
```
Co-authored-by: Shaden Smith <Shaden.Smith@microsoft.com>
Co-authored-by: Jeff Rasley <jerasley@microsoft.com>
```
  2dea61f2
- Add openmpi to dockerfile · c0d5424f
  Jeff Rasley authored Sep 10, 2020
  
  c0d5424f