pytorch_pretrained_bert/optimization_openai.py · e04bab59e1cf7e683e0365fd78e9e59e83c16d32 · chenpangpang / transformers

fix for negative learning rate with warmup_linear in BertAdam (happens when... · e04bab59

lukovnikov authored Feb 26, 2019

fix for negative learning rate with warmup_linear in BertAdam (happens when t_total is specified incorrectly)
+ copied BERT optimization warmup functions to OpenAI optimization file + added comments

e04bab59

optimization_openai.py 5.86 KB

Replace optimization_openai.py