GitLab

Fixing issue "Training beyond specified 't_total' steps with schedule 'warmup_linear'" reported in #556

GPT-2 (medium size model, special_tokens, fine-tuning, attention) + repo code coverage metric 

[hotfix] Fix frozen pooler parameters in SWAG example.

apply Whole Word Masking technique

apply Whole Word Masking technique.
referred to [create_pretraining_data.py](https://github.com/google-research/bert/blob/master/create_pretraining_data.py)

This reverts commit de5e5682.

Add GPT* compatibility to torchhub

Reason for issue was that optimzation steps where computed from example size, which is different from actual size of dataloader when an example is chunked into multiple instances.

Solution in this pull request is to compute num_optimization_steps directly from len(data_loader).

Updating learning rate with special warm up in examples

Make the epsilon of LayerNorm configurable.

move pytroch_pretrained_bert cache folder under same path as torch