Generate: fix `tokenizer` being popped twice (#31427)

377e9039 · Joao Gante · GitHub · 02c525d2 · 377e9039
Unverified Commit 377e9039 authored Jun 17, 2024 by Joao Gante Committed by GitHub Jun 17, 2024
Hide whitespace changes
Inline Side-by-side

Showing with 0 additions and 2 deletions

src/transformers/generation/utils.py src/transformers/generation/utils.py +0 -2

No files found.
--- a/src/transformers/generation/utils.py
+++ b/src/transformers/generation/utils.py
@@ -1640,8 +1640,6 @@ class GenerationMixin:
            else:
                synced_gpus = False

-        tokenizer = kwargs.pop("tokenizer", None)
-
        logits_processor = logits_processor if logits_processor is not None else LogitsProcessorList()
        stopping_criteria = stopping_criteria if stopping_criteria is not None else StoppingCriteriaList()