Commits · 690b1f718d33a4bf2f27e2be95e414793d7186d6 · OpenDAS / apex

08 Aug, 2019 1 commit
- initial commit to make fused optimizers compatible with AMP · 690b1f71
  Deyu Fu authored Aug 08, 2019
  
  690b1f71
01 Aug, 2019 2 commits
- Merge pull request #412 from ngimel/fix_layer_norm · 37795aac
  ngimel authored Aug 01, 2019
```
fix fused layer norm for >65535 batch
```
  37795aac
- fix fused layer norm for >65535 batch · 4a8e1a87
  Natalia Gimelshein authored Aug 01, 2019
  
  4a8e1a87
31 Jul, 2019 2 commits
- Merge pull request #400 from myleott/new_function_api · 3d01e4a0
  ngimel authored Jul 31, 2019
```
Update FusedLayerNorm for new function API
```
  3d01e4a0
- use ctx.eps instead of eps · 0dbf6c2a
  ngimel authored Jul 31, 2019
  
  0dbf6c2a
20 Jul, 2019 1 commit
- Update FusedLayerNorm for new function API · d1626ccc
  Myle Ott authored Jul 20, 2019
  
  d1626ccc
12 Jul, 2019 2 commits

jjsjann123 authored Jul 12, 2019

fixing empty return from python implementation
  adding proper test to verify functional correctness for python implementation

574fe244

Add missing semicolon. (#390) · 80e0143e
Edward Z. Yang authored Jul 12, 2019
```
Signed-off-by: Edward Z. Yang <ezyang@fb.com>
```
80e0143e

03 Jul, 2019 4 commits
- Pulling in deprecation warning changes · 665b2dd7
  Michael Carilli authored Jul 03, 2019
  
  665b2dd7
- Bumping container version for test · d352d440
  Michael Carilli authored Jul 03, 2019
  
  d352d440
- Remove deprecated Type.h · 816813f9
  Michael Carilli authored Jul 03, 2019
  
  816813f9
- Fix for #181 (add dot to CAST list) · d6271729
  mcarilli authored Jul 03, 2019
```
https://github.com/NVIDIA/apex/issues/181#issuecomment-507958143
```
  d6271729
02 Jul, 2019 1 commit
- Update scaler.py · 14fe2b67
  mcarilli authored Jul 01, 2019
  
  14fe2b67
28 Jun, 2019 2 commits
- Merge pull request #383 from NVIDIA/lamb_add_fp16_support_update_term · ff6b8bb0
  Thor Johnsen authored Jun 28, 2019
```
Add support for fp16 update term (new UPD_T typename in template)
```
  ff6b8bb0
- Add support for fp16 update term (new UPD_T typename in template) · 3aeea0d8
  Thor Johnsen authored Jun 28, 2019
  
  3aeea0d8
27 Jun, 2019 1 commit
- Clarifying documentation on gradient accumulation · 18f2eaee
  Michael Carilli authored Jun 26, 2019
  
  18f2eaee
25 Jun, 2019 1 commit
- grid_sample should be fp32 for now · 656d14b0
  Michael Carilli authored Jun 25, 2019
  
  656d14b0
24 Jun, 2019 4 commits
- Merge branch 'master' of https://github.com/NVIDIA/apex · 9ce80178
  Michael Carilli authored Jun 24, 2019
  
  9ce80178
- Docstring for multiple losses · f8557569
  Michael Carilli authored Jun 24, 2019
  
  f8557569
- Update README.md · f17cd953
  mcarilli authored Jun 24, 2019
  
  f17cd953
- Updating gradient accumulation guidance · ca35aa79
  Michael Carilli authored Jun 24, 2019
  
  ca35aa79
21 Jun, 2019 2 commits
- Make main_amp.py more profiling-friendly · f29b3f8d
  Michael Carilli authored Jun 21, 2019
  
  f29b3f8d
- Don't need to blacklist mean for pytorch >= 1.1 · 4b9858ec
  Michael Carilli authored Jun 21, 2019
  
  4b9858ec
20 Jun, 2019 1 commit
- Fix end-of-epoch with record_stream · 90e5b05a
  Michael Carilli authored Jun 20, 2019
  
  90e5b05a
19 Jun, 2019 2 commits
- Merge branch 'master' of https://github.com/NVIDIA/apex · 1ccaaf4a
  Michael Carilli authored Jun 18, 2019
  
  1ccaaf4a
- Fix for https://github.com/NVIDIA/apex/issues/361 · 68c850d3
  Michael Carilli authored Jun 18, 2019
  
  68c850d3
18 Jun, 2019 2 commits
- Give custom to method a higher priority (#364) · cd6e46c2
  mcarilli authored Jun 18, 2019
  
  cd6e46c2
- Fix rare caching allocator race condition in imagenet prefetcher · d5e2bb4b
  Michael Carilli authored Jun 17, 2019
  
  d5e2bb4b
17 Jun, 2019 1 commit
- More helpful message for unexpected opt_level · c3bcf18e
  Michael Carilli authored Jun 17, 2019
  
  c3bcf18e
14 Jun, 2019 4 commits
- Merge branch 'master' of https://github.com/NVIDIA/apex · a9f5f711
  Michael Carilli authored Jun 14, 2019
  
  a9f5f711
- Removing gradient_average_split_factor · 41c98511
  Michael Carilli authored Jun 14, 2019
  
  41c98511
- Separate LDG/STG from compute loop (#359) · 121a2500
  Thor Johnsen authored Jun 13, 2019
  
  121a2500
- Adding delay_overflow_check=False ninja control point · 4a9c2a53
  Michael Carilli authored Jun 13, 2019
  
  4a9c2a53
13 Jun, 2019 2 commits
- disable_allreduce -> _disable_allreduce · ae7f0def
  Michael Carilli authored Jun 13, 2019
  
  ae7f0def
- Add option to turn on/off allreduce in DDP (useful for gradient accumulation) (#356) · 1c2ba890
  Thor Johnsen authored Jun 13, 2019
  
  1c2ba890
11 Jun, 2019 1 commit
- Allow multi_tensor_lamb to update fp16 params · 47e3367f
  Michael Carilli authored Jun 11, 2019
  
  47e3367f
07 Jun, 2019 1 commit
- Fix for https://github.com/NVIDIA/apex/issues/344 · 04667139
  Michael Carilli authored Jun 07, 2019
  
  04667139
06 Jun, 2019 1 commit
- Making O1 the default opt level · 1dca16cc
  Michael Carilli authored Jun 06, 2019
  
  1dca16cc
04 Jun, 2019 1 commit
- Adding min_loss_scale and max_loss_scale arguments to amp.initialize · b82c6bd7
  Michael Carilli authored Jun 03, 2019
  
  b82c6bd7
31 May, 2019 1 commit

Multi tensor lamb optimizer (#334) · 8be5b6be

Thor Johnsen authored May 31, 2019

* First draft, for discussion

* Fix mistakes in LAMB equations

* Add loop over chunk

* Bug fix

* Bug fix

* Bug fix

* Undo bug fix

* Bug fix

* Add multi tensor LAMB optimizer to setup.py

* Rename step_size to learning_rate

* Fix compilation errors

8be5b6be