GitLab

Currently the L2 regularization is hard-coded to "0.01", even though there is a --weight_decay flag implemented (that is unused). I'm making this flag control the weight decay used for fine-tuning in this script.

Small fixes

fix multi-gpu training bug when using fp16

Fixed import to use torchscript flag.

Update docs for parameter rename

fix squad v1 error (na_prob_file should be None)

make save_pretrained do the right thing with added tokens

Update torchscript.rst

Import fixed to pytorch_transformers else torchscript flag can't be used.

right now it's dumping the *decoder* when it should be dumping the *encoder*. this fixes that.

OpenAIGPTLMHeadModel now accepts `labels` instead of `lm_labels`

Add nn.Identity replacement for old PyTorch

Rework how PreTrainedModel.from_pretrained handles its arguments

Updating schedules for state_dict saving/loading

fixed version issues in run_openai_gpt

typos

read().splitlines() -> readlines()

Remove '\n' before adding token into vocab

splitlines() does not work as what we expect here for bert-base-chinese because there is a '\u2028' (unicode line seperator) token in vocab file. Value of '\u2028'.splitlines() is ['', ''].
Perhaps we should use readlines() instead.