[`PretrainedTokenizer`] add some of the most important functions to the doc (#27313)

147f7746 · Arthur · GitHub · 1ffc4dee · 147f7746
Unverified Commit 147f7746 authored Nov 06, 2023 by Arthur Committed by GitHub Nov 06, 2023
Hide whitespace changes
Inline Side-by-side

Showing with 4 additions and 0 deletions

docs/source/en/main_classes/tokenizer.md docs/source/en/main_classes/tokenizer.md +4 -0

No files found.
--- a/docs/source/en/main_classes/tokenizer.md
+++ b/docs/source/en/main_classes/tokenizer.md
@@ -55,6 +55,8 @@ to a given token).

 [[autodoc]] PreTrainedTokenizer
    - __call__
+    - add_tokens
+    - add_special_tokens
    - apply_chat_template
    - batch_decode
    - decode
@@ -69,6 +71,8 @@ loaded very simply into 🤗 transformers. Take a look at the [Using tokenizers

 [[autodoc]] PreTrainedTokenizerFast
    - __call__
+    - add_tokens
+    - add_special_tokens
    - apply_chat_template
    - batch_decode
    - decode