Skip to content

GitLab

Unverified Commit cfba4def authored by Woosuk Kwon's avatar Woosuk Kwon Committed by GitHub
Browse files

[Bugfix] Fix logit soft cap in flash-attn backend (#7425)

parent d2bc4510
Show whitespace changes
Inline Side-by-side
Showing with 1 addition and 0 deletions
vllm/attention/backends/flash_attn.py
View file @ cfba4def
...@@ -563,6 +563,7 @@ class FlashAttentionImpl(AttentionImpl): ...@@ -563,6 +563,7 @@ class FlashAttentionImpl(AttentionImpl):
softmax_scale=self.scale, softmax_scale=self.scale,
causal=True, causal=True,
alibi_slopes=self.alibi_slopes, alibi_slopes=self.alibi_slopes,
softcap=self.logits_soft_cap,
).squeeze(1) ).squeeze(1)
# Reshape the output tensor. # Reshape the output tensor.
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment