"examples/vscode:/vscode.git/clone" did not exist on "dddd6b99272652f58b3d9f3a29d5c17cc4e119c9"
-
Sylvain Gugger authored
* Start doc tokenizers * Tokenizer documentation * Start doc tokenizers * Tokenizer documentation * Formatting after rebase * Formatting after merge * Update docs/source/main_classes/tokenizer.rst Co-authored-by:
Lysandre Debut <lysandre@huggingface.co> * Address comment * Update src/transformers/tokenization_utils_base.py Co-authored-by:
Thomas Wolf <thomwolf@users.noreply.github.com> * Address Thom's comments Co-authored-by:
f3065abd
