Skip to content

GitLab

Unverified Commit b0f1c0ee authored by Sylvain Gugger's avatar Sylvain Gugger Committed by GitHub
Browse files

Document tokenizer_class in configurations (#8152)

parent 969859d5
Hide whitespace changes
Inline Side-by-side
Showing with 2 additions and 0 deletions
src/transformers/configuration_utils.py
View file @ b0f1c0ee
...@@ -134,6 +134,8 @@ class PretrainedConfig(object): ...@@ -134,6 +134,8 @@ class PretrainedConfig(object):
Parameters linked to the tokenizer Parameters linked to the tokenizer
- **tokenizer_class** (:obj:`str`, `optional`) -- The name of the associated tokenizer class to use (if none is
set, will use the tokenizer associated to the model by default).
- **prefix** (:obj:`str`, `optional`) -- A specific prompt that should be added at the beginning of each text - **prefix** (:obj:`str`, `optional`) -- A specific prompt that should be added at the beginning of each text
before calling the model. before calling the model.
- **bos_token_id** (:obj:`int`, `optional`)) -- The id of the `beginning-of-stream` token. - **bos_token_id** (:obj:`int`, `optional`)) -- The id of the `beginning-of-stream` token.
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment