Skip to content

GitLab

Unverified Commit f63ed7b5 authored by Benjamin Chislett's avatar Benjamin Chislett Committed by GitHub
Browse files

[Bugfix] Fix DP MTP Dummy Run (#35243) 


Signed-off-by: default avatarBenjamin Chislett <bchislett@nvidia.com>
parent c9e50962
Show whitespace changes
Inline Side-by-side
Showing with 2 additions and 1 deletion
vllm/v1/worker/gpu_worker.py
View file @ f63ed7b5
......@@ -903,7 +903,8 @@ class Worker(WorkerBase):
self.profiler.stop()
def execute_dummy_batch(self) -> None:
self.model_runner._dummy_run(1, uniform_decode=True)
num_tokens = getattr(self.model_runner, "uniform_decode_query_len", 1)
self.model_runner._dummy_run(num_tokens, uniform_decode=True)
def add_lora(self, lora_request: LoRARequest) -> bool:
return self.model_runner.add_lora(lora_request)
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment