Skip to content

GitLab

Unverified Commit 3e83c12b authored by Li, Jiang's avatar Li, Jiang Committed by GitHub
Browse files

[Bugfix][CPU] fix missing input intermediate_tensors in the cpu_model_runner (#8733)

parent e551ca15
Hide whitespace changes
Inline Side-by-side
Showing with 2 additions and 0 deletions
vllm/worker/cpu_model_runner.py
View file @ 3e83c12b
...@@ -456,6 +456,8 @@ class CPUModelRunner(ModelRunnerBase[ModelInputForCPU]): ...@@ -456,6 +456,8 @@ class CPUModelRunner(ModelRunnerBase[ModelInputForCPU]):
model_input.attn_metadata, model_input.attn_metadata,
**MultiModalInputs.as_kwargs(model_input.multi_modal_kwargs or {}, **MultiModalInputs.as_kwargs(model_input.multi_modal_kwargs or {},
device=self.device), device=self.device),
"intermediate_tensors":
intermediate_tensors,
} }
hidden_states = model_executable(**execute_model_kwargs) hidden_states = model_executable(**execute_model_kwargs)
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment