Skip to content

GitLab

Unverified Commit dc76e9fa authored by Raushan Turganbay's avatar Raushan Turganbay Committed by GitHub
Browse files

Generation: past kv can be None (#31051) 

* fix

* better
parent 1de7dc74
Show whitespace changes
Inline Side-by-side
Showing with 1 addition and 1 deletion
src/transformers/generation/utils.py
View file @ dc76e9fa
...@@ -1394,7 +1394,7 @@ class GenerationMixin: ...@@ -1394,7 +1394,7 @@ class GenerationMixin:
return model_kwargs return model_kwargs
past_length = 0 past_length = 0
if "past_key_values" in model_kwargs: if model_kwargs.get("past_key_values") is not None:
if isinstance(model_kwargs["past_key_values"], Cache): if isinstance(model_kwargs["past_key_values"], Cache):
past_length = model_kwargs["past_key_values"].get_seq_length() past_length = model_kwargs["past_key_values"].get_seq_length()
else: else:
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment