Commit 8d6b0b0a authored by zhuwenwen's avatar zhuwenwen
Browse files

Merge branch 'v0.9.2-dev-wm-1110' into 'v0.9.2-dev'

[fix]修复mtp中的笔误

See merge request dcutoolkit/deeplearing/vllm!249
parents 475128c1 d21e2db7
......@@ -402,7 +402,7 @@ class MLACommonMetadataBuilder(AttentionMetadataBuilder[M]):
self.decode_token_num_threshold = 1
vllm_config = get_current_vllm_config()
speculative_config = vllm_config.speculative_config
if speculative_config and speculative_config.num_speculative_tokens > 1:
if speculative_config and speculative_config.num_speculative_tokens > 0:
self.use_spec_decode = True
self.decode_token_num_threshold = 1 + speculative_config.num_speculative_tokens
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment