GitLab

[WIP] copy paste bert token classification for roberta

see also https://github.com/salesforce/ctrl/pull/50

Gradient norm clipping should be done right before calling the optimiser

gradient norm clipping should be done right before calling the optimiser - fixing run_glue and run_ner as well

Fix hanging when loading pretrained models

update

- Fix hanging when loading pretrained models from the cache without having internet access. This is a widespread issue on supercomputers whose internal compute nodes are firewalled.

Co-authored-by: Thomas Wolf @thomwolf

Force einsum to run in fp16

remove leftover usage of DUMMY_INPUTS

Added performance enhancements (XLA, AMP) to examples

Fixed the sample code in the title 'Quick tour'.

Remove unnecessary use of FusedLayerNorm in XLNet

Implement fine-tuning BERT on CoNLL-2003 named entity recognition task