"vscode:/vscode.git/clone" did not exist on "86dd23bb8bef09f1869fd9691bf43ece75c5675a"
Unverified Commit 7b5e943c authored by Joao Gante's avatar Joao Gante Committed by GitHub
Browse files

Generate: TF contrastive search must pop `use_cache` from `model_kwargs` (#21149)

parent 7f3dab39
......@@ -2437,6 +2437,8 @@ class TFGenerationMixin:
else self.generation_config.return_dict_in_generate
)
use_cache = True # In contrastive search, we always use cache
model_kwargs.pop("use_cache", None)
use_xla = not tf.executing_eagerly()
# TODO (Joao): fix cache format or find programatic way to detect cache index
# GPT2 and other models has a slightly different cache structure, with a different batch axis
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment