GitLab

Tokenization behave the same as original XLM proprocessing for most languages except zh, ja and th; Change API to allow specifying language in `tokenize`

Fixed file extensions for config/vocab/merges of XLM models.

Fixed all links. Removed TPU. Changed CLI to Converting TF models. Many minor formatting adjustments. Added "TODO Lysandre filled" where necessary.

For many applications requiring randomized data access, it's easier to cache the tokenized representations than the words. So why not turn this into a warning?