src/transformers/models/auto/tokenization_auto.py · 8e908c8c74f556a82534f4cf1e7a1b4f7b55d24c · chenpangpang / transformers · GitLab

Find file Blame History Permalink

[AutoTokenizer] Allow creation of tokenizers by tokenizer type (#13668) · 8e908c8c
Patrick von Platen authored Sep 22, 2021
```
* up

* up
```
8e908c8c

tokenization_auto.py 25 KB