Skip to content

GitLab

Unverified Commit 2cbd4d29 authored by Bryan Lu's avatar Bryan Lu Committed by GitHub
Browse files

[V1][Spec Dec Bug Fix] Respect Spec Dec Method Specification (#16636) 


Signed-off-by: default avatarBryan Lu <yuzhelu@amazon.com>
parent 3092375e
Hide whitespace changes
Inline Side-by-side
Showing with 3 additions and 1 deletion
vllm/config.py
View file @ 2cbd4d29
...@@ -2341,7 +2341,9 @@ class SpeculativeConfig: ...@@ -2341,7 +2341,9 @@ class SpeculativeConfig:
) )
# Automatically detect the method # Automatically detect the method
if "eagle-" in self.draft_model_config.model.lower(): if self.method == 'eagle':
pass
elif "eagle-" in self.draft_model_config.model.lower():
self.method = "eagle" self.method = "eagle"
elif self.draft_model_config.hf_config.model_type == "medusa": elif self.draft_model_config.hf_config.model_type == "medusa":
self.method = "medusa" self.method = "medusa"
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment