Skip to content

GitLab

Unverified Commit f1a1eb4a authored by Yih-Dar's avatar Yih-Dar Committed by GitHub
Browse files

Fix `llama` tokenization doctest (#24990) 



fix
Co-authored-by: default avatarydshieh <ydshieh@users.noreply.github.com>
parent a7d21318
Show whitespace changes
Inline Side-by-side
Showing with 2 additions and 2 deletions
src/transformers/models/llama/tokenization_llama.py
View file @ f1a1eb4a
......@@ -332,7 +332,7 @@ class LlamaTokenizer(PreTrainedTokenizer):
return output
def _build_conversation_input_ids(self, conversation: "Conversation") -> List[int]:
"""Builds the input ids for a conversation.
r"""Builds the input ids for a conversation.
This is the format used in the provided examples. System prompts should be manually added at the beginning of
the conversation. If no system prompt is given, the `DEFAULT_SYSTEM_PROMPT` will be used.
```
......@@ -347,7 +347,7 @@ class LlamaTokenizer(PreTrainedTokenizer):
>>> Conversation(
... "<<SYS>>\n Only answer with emojis, and charades\n<</SYS>>\n\nHow can I build a house in 10 septs?"
... )
... ) # doctest: +IGNORE_RESULT
```
Args:
conversation (`Conversation`):
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment