Skip to content

GitLab

Unverified Commit 6afc28a9 authored by Wentao Ye's avatar Wentao Ye Committed by GitHub
Browse files

[Test] Batch Invariant: Unit test using parameterized backend (#27478) 


Signed-off-by: default avataryewentao256 <zhyanwentao@126.com>
parent 141e6a05
Expand all Hide whitespace changes
Inline Side-by-side
Showing with 208 additions and 204 deletions
vllm/model_executor/layers/batch_invariant.py
View file @ 6afc28a9
...@@ -753,13 +753,13 @@ def override_envs_for_invariance(): ...@@ -753,13 +753,13 @@ def override_envs_for_invariance():
curr_attn_backend = envs.VLLM_ATTENTION_BACKEND curr_attn_backend = envs.VLLM_ATTENTION_BACKEND
supported_backends = [ supported_backends = [
"FLASH_ATTN", # best supported backend "FLASH_ATTN", # best supported backend
"FLEX_ATTENTION",
"FLASHINFER", "FLASHINFER",
"FLASH_ATTN_MLA", "FLASH_ATTN_MLA",
"FLASHINFER_MLA", "FLASHINFER_MLA",
"TRITON_MLA", "TRITON_MLA",
# Not yet supported MLA backends # Not yet supported MLA backends
# "FLASHMLA", # "FLASHMLA",
# "FLEX_ATTENTION", # IMA issue even if we disable batch invariance
] ]
if curr_attn_backend not in supported_backends: if curr_attn_backend not in supported_backends:
warning = ( warning = (
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment