Commits · 7e3536dd9acf0e5ec529bdaedb515cd3df6bafc6 · OpenDAS / apex

06 May, 2020 1 commit
- Ultra-simple global all-reduce version of distributed optimizer · 7e3536dd
  Thor Johnsen authored May 05, 2020
  
  7e3536dd
05 May, 2020 1 commit
- Try out different partition scheme · a60bbe63
  Thor Johnsen authored May 04, 2020
  
  a60bbe63
04 May, 2020 1 commit
- Bug fix · 7da28fc3
  Thor Johnsen authored May 04, 2020
  
  7da28fc3
30 Apr, 2020 2 commits
- Remove implicit memcpy of grad tensor in do_overlapped function · 9c82241d
  Thor Johnsen authored Apr 30, 2020
  
  9c82241d
- Don't pad between consecutive parameters · 5d1993cf
  Thor Johnsen authored Apr 29, 2020
  
  5d1993cf
29 Apr, 2020 5 commits
- Bug fix · e1a4deba
  Thor Johnsen authored Apr 29, 2020
  
  e1a4deba
- Perf improvement (less CPU work) · 415e2646
  Thor Johnsen authored Apr 29, 2020
  
  415e2646
- Make L2 grad norm a CPU variable · 9d6d2e01
  Thor Johnsen authored Apr 29, 2020
  
  9d6d2e01
- Bug fix · bc81b1c1
  Thor Johnsen authored Apr 29, 2020
  
  bc81b1c1
- Reduce CPU overhead, bigger step, all-gather · 44f54712
  Thor Johnsen authored Apr 28, 2020
  
  44f54712
20 Apr, 2020 1 commit
- Add alternate distributed optimizer implementation · f0448054
  Thor Johnsen authored Apr 20, 2020
  
  f0448054
16 Apr, 2020 5 commits
- Partial move towards syncfree optimizer · 4a01ff26
  Thor Johnsen authored Apr 16, 2020
  
  4a01ff26
- Use glob_chunk to index streams and process groups · 2622d7f1
  Thor Johnsen authored Apr 16, 2020
  
  2622d7f1
- Bug fix · 85497632
  Thor Johnsen authored Apr 16, 2020
  
  85497632
- Bug fix · cef660ba
  Thor Johnsen authored Apr 16, 2020
  
  cef660ba
- Pragmatic change, seems like WAR for NCCL crash · 6eca2389
  Thor Johnsen authored Apr 16, 2020
  
  6eca2389
15 Apr, 2020 2 commits
- Bug(?) fix · 2c744ee5
  Thor Johnsen authored Apr 15, 2020
  
  2c744ee5
- internal pipelining more similar to micro-benchmarks · 208c91e0
  Thor Johnsen authored Apr 14, 2020
  
  208c91e0
10 Apr, 2020 2 commits
- Add option to skip overflow check in step() method · 7ba6a038
  Thor Johnsen authored Apr 10, 2020
  
  7ba6a038
- Add no-flattening e5m2-allgather option · c7b34549
  Thor Johnsen authored Apr 09, 2020
  
  c7b34549
09 Apr, 2020 1 commit
- Add e5m2 allgather option · cd206434
  Thor Johnsen authored Apr 09, 2020
  
  cd206434
08 Apr, 2020 1 commit
- Add internal pipelining option · aa90d31f
  Thor Johnsen authored Apr 08, 2020
  
  aa90d31f
07 Apr, 2020 1 commit
- Bug fix · be4c41c2
  Thor Johnsen authored Apr 07, 2020
  
  be4c41c2
02 Apr, 2020 2 commits
- Add back support for multi tensor scale flattening · db8fb976
  Thor Johnsen authored Apr 01, 2020
  
  db8fb976
- Bug fix in internal pipelining · 3f717d95
  Thor Johnsen authored Apr 01, 2020
  
  3f717d95
01 Apr, 2020 4 commits
- Bug fix · 17160f34
  Thor Johnsen authored Apr 01, 2020
  
  17160f34
- Add separate dwu_num_chunks argument · e5c8e3c9
  Thor Johnsen authored Apr 01, 2020
  
  e5c8e3c9
- Add support for 4 all-reduce IB communicators · f2c9aa33
  Thor Johnsen authored Apr 01, 2020
  
  f2c9aa33
- Move partial_step out of complete reductions: · 5c1cf020
  Thor Johnsen authored Mar 31, 2020
  
  5c1cf020
23 Mar, 2020 1 commit
- Revert "Avoid 1 copy for double buffering scheme" · 3f4fb81f
  Thor Johnsen authored Mar 23, 2020
```
This reverts commit 029cd5e1.

.
```
  3f4fb81f
20 Mar, 2020 2 commits
- Broadcast parameters from rank 0 just to be safe · 604d423b
  Thor Johnsen authored Mar 20, 2020
  
  604d423b
- Bug fix · bc98827b
  Thor Johnsen authored Mar 20, 2020
  
  bc98827b
19 Mar, 2020 5 commits
- Bug fixes · 90795025
  Thor Johnsen authored Mar 19, 2020
  
  90795025
- Bug fixes · 174abea7
  Thor Johnsen authored Mar 19, 2020
  
  174abea7
- Bug fix · dde13741
  Thor Johnsen authored Mar 19, 2020
  
  dde13741
- Avoid 1 copy for double buffering scheme · 029cd5e1
  Thor Johnsen authored Mar 19, 2020
  
  029cd5e1
- Add option to revert step through double buffering · b85ff391
  Thor Johnsen authored Mar 18, 2020
  
  b85ff391
18 Mar, 2020 2 commits
- Merge branch 'revertable_fused_adam_with_mt_support' of... · ffed6e80
  Thor Johnsen authored Mar 18, 2020
```
Merge branch 'revertable_fused_adam_with_mt_support' of https://github.com/NVIDIA/apex into revertable_fused_adam_with_mt_support

.
```
  ffed6e80
- Change inplace to no_copy · eca3a2c4
  Thor Johnsen authored Mar 18, 2020
  
  eca3a2c4
17 Mar, 2020 1 commit
- Remove dead code · 6b2ef787
  Thor Johnsen authored Mar 16, 2020
  
  6b2ef787