docs/source/fast_tokenizers.rst · b0c7d2ec58a8080780fb21327cc7cd7e1a177780 · chenpangpang / transformers

Documentation about loading a fast tokenizer within Transformers (#11029) · 9f4e0c23

Lysandre Debut authored Apr 05, 2021



* Documentation about loading a fast tokenizer within Transformers

* Apply suggestions from code review
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>

* style
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>

9f4e0c23

fast_tokenizers.rst 2.58 KB

Replace fast_tokenizers.rst