GitLab

distillation: fix ModuleNotFoundError error in token counts script

Pruning changes so that deleted heads are kept on save/load

Now raises a warning when a head to be deleted already has been deleted. An integration test verifying the total pipeline (-> from config -> save model -> load model -> additional head pruning) has been added.

See #1089

cc @thomwolf @lysandrejik

Also @dhpollack

Instead we correctly store it on the config

(regenerating the hosted config files)

cc @lysandrejik

Update apex fp16 implementation

fix: hard coding for max number

fix adding special tokens

Shortcut to special tokens' ids - fix GPT2 & RoBERTa tokenizers - improved testing for GPT/GPT-2

Added cleaned configuration properties for tokenizer with serialization - improve tokenization of XLM

Torch.hub now based on AutoModels - Updating AutoModels with AutoModelWithLMHead, Sequence Classification and Question Answering