GitLab

This is the result of:

    $ black --line-length 119 examples templates transformers utils hubconf.py setup.py

There's a lot of fairly long lines in the project. As a consequence, I'm
picking the longest widely accepted line length, 119 characters.

This is also Thomas' preference, because it allows for explicit variable
names, to make the code easier to understand.

deay -> decay

TPU runner is currently implemented in:
https://github.com/pytorch-tpu/transformers/blob/tpu/examples/run_glue_tpu.py.

We plan to upstream this directly into `huggingface/transformers`
(either `master` or `tpu`) branch once it's been more thoroughly tested.

gradient norm clipping should be done right before calling the optimiser - fixing run_glue and run_ner as well

add DistilBert model shortcut into ALL_MODELS

Add RoBERTa to the comment as it was not explicit that RoBERTa don't use token_type_ids.