- 17 Aug, 2019 1 commit
-
-
Deyu Fu authored
-
- 16 Aug, 2019 3 commits
-
-
Deyu Fu authored
correctly not apply bias correction to epsilon(same as recent upstream change) correctly not apply bias correction to weight decay(consistent with upstream AdamW) Make adam_w_mode for FusedAdam/LAMB, to do L2 or Weight Decay (Adam vs AdamW) Correct document reg_inside_moment differently from adam_w_mode in FusedNovoGrad Removed legacy eps_mode from FusedAdam Make internal math type float across fused optimizers
-
Deyu Fu authored
-
Deyu Fu authored
-
- 13 Aug, 2019 4 commits
-
-
Deyu Fu authored
-
Deyu Fu authored
FusedSGD now work as before FusedAdam now work with o1/o2, no longer fuse scaling and casting Removed special backend handling for FusedAdam Moved and updated test for FusedAdam into run_optimizers Removed legacy tests for optimizers.FP16_optimizer and FusedAdam in run_mixed_adam
-
Marek Kolodziej authored
Co-authored-by:
Aditya Agrawal <aditya.iitb@gmail.com> Co-authored-by:
Marek Kolodziej <mkolod@gmail.com>
-
Deyu Fu authored
-
- 12 Aug, 2019 3 commits
- 08 Aug, 2019 1 commit
-
-
Deyu Fu authored
-
- 07 Aug, 2019 1 commit
-
-
ptrblck authored
* initial commit * add default O1 mode, enable other modes, add README * add carilli's review suggestions to README
-
- 06 Aug, 2019 1 commit
-
-
ngimel authored
* Bug fix for non-affine layer-norm + add backward unit test * clean up tests and add tests for a large batch
-
- 01 Aug, 2019 2 commits
-
-
ngimel authored
fix fused layer norm for >65535 batch
-
Natalia Gimelshein authored
-
- 31 Jul, 2019 2 commits
- 28 Jul, 2019 1 commit
-
-
Syed Tousif Ahmed authored
-
- 27 Jul, 2019 1 commit
-
-
Syed Tousif Ahmed authored
-
- 26 Jul, 2019 3 commits
-
-
Syed Tousif Ahmed authored
-
jjsjann123 authored
fixing empty return from python implementation adding proper test to verify functional correctness for python implementation
-
Edward Z. Yang authored
Signed-off-by:Edward Z. Yang <ezyang@fb.com>
-
- 20 Jul, 2019 1 commit
-
-
Myle Ott authored
-
- 13 Jul, 2019 2 commits
-
-
jjsjann123 authored
Fixing rank mapping for bn_group size == 8
-
Jie authored
-
- 12 Jul, 2019 2 commits
-
-
jjsjann123 authored
fixing empty return from python implementation adding proper test to verify functional correctness for python implementation
-
Edward Z. Yang authored
Signed-off-by:Edward Z. Yang <ezyang@fb.com>
-
- 03 Jul, 2019 8 commits
-
-
Michael Carilli authored
-
Michael Carilli authored
-
Michael Carilli authored
-
Michael Carilli authored
-
Michael Carilli authored
-
Michael Carilli authored
-
Michael Carilli authored
-
-
- 02 Jul, 2019 3 commits
-
-
Michael Carilli authored
-
Michael Carilli authored
-
mcarilli authored
-
- 28 Jun, 2019 1 commit
-
-
Thor Johnsen authored
Add support for fp16 update term (new UPD_T typename in template)
-