Skip to content

GitLab

Unverified Commit a7501f6f authored by Arthur's avatar Arthur Committed by GitHub
Browse files

[BlenderBotSmall] Update doc example (#24092) 

* small tokenizer uses `__start__` and `__end__`

* fix PR doctest
parent 5af3a1aa
Hide whitespace changes
Inline Side-by-side
Showing with 4 additions and 4 deletions
src/transformers/models/blenderbot_small/modeling_blenderbot_small.py
View file @ a7501f6f
...@@ -536,14 +536,14 @@ BLENDERBOT_SMALL_GENERATION_EXAMPLE = r""" ...@@ -536,14 +536,14 @@ BLENDERBOT_SMALL_GENERATION_EXAMPLE = r"""
Human: I'm not sure Human: I'm not sure
>>> NEXT_UTTERANCE = ( >>> NEXT_UTTERANCE = (
... "My friends are cool but they eat too many carbs.</s> <s>what kind of carbs do they eat? " ... "My friends are cool but they eat too many carbs.__end__ __start__what kind of carbs do they eat? "
... "i don't know much about carbs</s> " ... "i don't know much about carbs__end__ "
... "<s> I'm not sure." ... "__start__ I'm not sure."
... ) ... )
>>> inputs = tokenizer([NEXT_UTTERANCE], return_tensors="pt") >>> inputs = tokenizer([NEXT_UTTERANCE], return_tensors="pt")
>>> next_reply_ids = model.generate(**inputs) >>> next_reply_ids = model.generate(**inputs)
>>> print("Bot: ", tokenizer.batch_decode(next_reply_ids, skip_special_tokens=True)[0]) >>> print("Bot: ", tokenizer.batch_decode(next_reply_ids, skip_special_tokens=True)[0])
Bot: they eat a lot of carbs. carbs are high in fat, protein, and carbohydrates. Bot: they eat a lot of carbs. carbs are high in fat, protein, and fats.
``` ```
""" """
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment