GitLab

Updating learning rate with special warm up in examples

Make the epsilon of LayerNorm configurable.

move pytroch_pretrained_bert cache folder under same path as torch

Create optimizer only when args.do_train is True

Fix documentation typo

Small fix to remove shifting of lm labels during pre process of RocStories.

Fix #537

small fix to remove shifting of lm labels during pre process of roc stories, as this shifting happens interanlly in the model

Update example files so that tr_loss is not affected by args.gradient…

fixed BertForMultipleChoice model init and forward pass

Docs for new learning rate code

Fix training schedules in examples to match new API

- added some images for illustration
- updated comments in optimization

Hubconf

Fix --reduce_memory in finetune_on_pregenerated

Fix indentation weirdness in GPT-2 example.

Learning rate schedules improvement + extension

Pulling commits from main repo