Skip to content

GitLab

Commit 3a8708f6 authored by Lucas Wilkinson's avatar Lucas Wilkinson Committed by simon-mo
Browse files

[BugFix] fix CUTLASS MLA full cudagraph (#23200) 


Signed-off-by: default avatarLucas Wilkinson <lwilkins@redhat.com>
Signed-off-by: default avatarsimon-mo <simon.mo@hey.com>
parent aab54987
Hide whitespace changes
Inline Side-by-side
Showing with 1 addition and 1 deletion
vllm/v1/attention/backends/mla/cutlass_mla.py
View file @ 3a8708f6
...@@ -21,7 +21,7 @@ logger = init_logger(__name__) ...@@ -21,7 +21,7 @@ logger = init_logger(__name__)
class CutlassMLAMetadataBuilder(MLACommonMetadataBuilder[MLACommonMetadata]): class CutlassMLAMetadataBuilder(MLACommonMetadataBuilder[MLACommonMetadata]):
# enable full CUDA Graph support for decode-only capture # enable full CUDA Graph support for decode-only capture
attn_cudagraph_support: ClassVar[ cudagraph_support: ClassVar[
AttentionCGSupport] = AttentionCGSupport.UNIFORM_SINGLE_TOKEN_DECODE AttentionCGSupport] = AttentionCGSupport.UNIFORM_SINGLE_TOKEN_DECODE
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment