Unverified Commit 58e7f9bb authored by Joao Gante's avatar Joao Gante Committed by GitHub
Browse files

Generate: All logits processors are documented and have examples (#27796)


Co-authored-by: default avatarArthur <48595927+ArthurZucker@users.noreply.github.com>
parent 47500b1d
...@@ -250,7 +250,7 @@ While the autoregressive generation process is relatively straightforward, makin ...@@ -250,7 +250,7 @@ While the autoregressive generation process is relatively straightforward, makin
1. [Guide](generation_strategies) on how to control different generation methods, how to set up the generation configuration file, and how to stream the output; 1. [Guide](generation_strategies) on how to control different generation methods, how to set up the generation configuration file, and how to stream the output;
2. [Guide](chat_templating) on the prompt template for chat LLMs; 2. [Guide](chat_templating) on the prompt template for chat LLMs;
3. [Guide](tasks/prompting) on to get the most of prompt design; 3. [Guide](tasks/prompting) on to get the most of prompt design;
4. API reference on [`~generation.GenerationConfig`], [`~generation.GenerationMixin.generate`], and [generate-related classes](internal/generation_utils). 4. API reference on [`~generation.GenerationConfig`], [`~generation.GenerationMixin.generate`], and [generate-related classes](internal/generation_utils). Most of the classes, including the logits processors, have usage examples!
### LLM leaderboards ### LLM leaderboards
......
...@@ -63,6 +63,14 @@ class GenerationConfig(PushToHubMixin): ...@@ -63,6 +63,14 @@ class GenerationConfig(PushToHubMixin):
You do not need to call any of the above methods directly. Pass custom parameter values to '.generate()'. To learn You do not need to call any of the above methods directly. Pass custom parameter values to '.generate()'. To learn
more about decoding strategies refer to the [text generation strategies guide](../generation_strategies). more about decoding strategies refer to the [text generation strategies guide](../generation_strategies).
<Tip>
A large number of these flags control the logits or the stopping criteria of the generation. Make sure you check
the [generate-related classes](https://huggingface.co/docs/transformers/internal/generation_utils) for a full
description of the possible manipulations, as well as examples of their usage.
</Tip>
Arg: Arg:
> Parameters that control the length of the output > Parameters that control the length of the output
......
...@@ -1031,16 +1031,9 @@ class GenerationMixin: ...@@ -1031,16 +1031,9 @@ class GenerationMixin:
generation_config.encoder_no_repeat_ngram_size is not None generation_config.encoder_no_repeat_ngram_size is not None
and generation_config.encoder_no_repeat_ngram_size > 0 and generation_config.encoder_no_repeat_ngram_size > 0
): ):
if self.config.is_encoder_decoder: processors.append(
processors.append( EncoderNoRepeatNGramLogitsProcessor(generation_config.encoder_no_repeat_ngram_size, encoder_input_ids)
EncoderNoRepeatNGramLogitsProcessor( )
generation_config.encoder_no_repeat_ngram_size, encoder_input_ids
)
)
else:
raise ValueError(
"It's impossible to use `encoder_no_repeat_ngram_size` with decoder-only architecture"
)
if generation_config.bad_words_ids is not None: if generation_config.bad_words_ids is not None:
processors.append( processors.append(
NoBadWordsLogitsProcessor(generation_config.bad_words_ids, generation_config.eos_token_id) NoBadWordsLogitsProcessor(generation_config.bad_words_ids, generation_config.eos_token_id)
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment