pytorch_pretrained_bert/optimization_openai.py · da2d8ca2652d9256fdd3edbdfb7253a3f21fb833 · chenpangpang / transformers

fix for negative learning rate with warmup_linear in BertAdam (happens when... · da2d8ca2

lukovnikov authored Feb 26, 2019

fix for negative learning rate with warmup_linear in BertAdam (happens when t_total is specified incorrectly)
+ copied BERT optimization warmup functions to OpenAI optimization file + added comments

da2d8ca2

optimization_openai.py 5.87 KB

Replace optimization_openai.py