docs/source/main_classes/tokenizer.rst · 9f4e0c23d68366985f9f584388874477ad6472d8 · chenpangpang / transformers

Documentation about loading a fast tokenizer within Transformers (#11029) · 9f4e0c23

Lysandre Debut authored Apr 05, 2021



* Documentation about loading a fast tokenizer within Transformers

* Apply suggestions from code review
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>

* style
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>

9f4e0c23

tokenizer.rst 5 KB

Replace tokenizer.rst