GitLab

Necessary to run xlnet (at least in squad) with `--fp16 --fp16_opt_level="O2"`, otherwise loss is immediately `NaN` and fine-tuning cannot proceed.

Now raises a warning when a head to be deleted already has been deleted. An integration test verifying the total pipeline (-> from config -> save model -> load model -> additional head pruning) has been added.

See #1089

cc @thomwolf @lysandrejik

Also @dhpollack

Instead we correctly store it on the config

(regenerating the hosted config files)

cc @lysandrejik