GitLab

apply Whole Word Masking technique

apply Whole Word Masking technique.
referred to [create_pretraining_data.py](https://github.com/google-research/bert/blob/master/create_pretraining_data.py)

Add GPT* compatibility to torchhub

Updating learning rate with special warm up in examples

Make the epsilon of LayerNorm configurable.

move pytroch_pretrained_bert cache folder under same path as torch

Create optimizer only when args.do_train is True

Fix documentation typo

Small fix to remove shifting of lm labels during pre process of RocStories.

Fix #537

small fix to remove shifting of lm labels during pre process of roc stories, as this shifting happens interanlly in the model

Update example files so that tr_loss is not affected by args.gradient…

fixed BertForMultipleChoice model init and forward pass