Commit a26e8dc7 authored by Matthew Bonanni's avatar Matthew Bonanni Committed by khluu
Browse files

[Bugfix][MLA] Change default SM100 MLA prefill backend back to TRT-LLM (#38562)


Signed-off-by: default avatarMatthew Bonanni <mbonanni@redhat.com>
(cherry picked from commit 2c734ed0e06a48808522fe8f59f6b4ffe0cf0397)
parent 599e7359
...@@ -30,7 +30,7 @@ class AttentionConfig: ...@@ -30,7 +30,7 @@ class AttentionConfig:
use_cudnn_prefill: bool = False use_cudnn_prefill: bool = False
"""Whether to use cudnn prefill.""" """Whether to use cudnn prefill."""
use_trtllm_ragged_deepseek_prefill: bool = False use_trtllm_ragged_deepseek_prefill: bool = True
"""Whether to use TRTLLM ragged deepseek prefill.""" """Whether to use TRTLLM ragged deepseek prefill."""
use_trtllm_attention: bool | None = None use_trtllm_attention: bool | None = None
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment