Commit e470f6a1 authored by zhuwenwen's avatar zhuwenwen
Browse files

when v1 is set to block_size 16, switch to triton implementation

parent 3903e15e
......@@ -279,6 +279,7 @@ class RocmPlatform(Platform):
logger.info_once("Using Flash Attention backend on V1 engine. (only supports block size 64)")
return FLASH_ATTN_V1
else:
os.environ['VLLM_USE_FLASH_ATTN_PA'] = '0'
logger.info_once("Using Triton backend on V1 engine.")
return TRITON_ATTN_VLLM_V1
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment