Skip to content

GitLab

Commit 72e27c63 authored by Tri Dao's avatar Tri Dao
Browse files

Fix typo with softcapping

parent 3d41db3e
Hide whitespace changes
Inline Side-by-side
Showing with 2 additions and 2 deletions
flash_attn/flash_attn_interface.py
View file @ 72e27c63
...@@ -721,7 +721,7 @@ def flash_attn_qkvpacked_func( ...@@ -721,7 +721,7 @@ def flash_attn_qkvpacked_func(
softmax_scale, softmax_scale,
causal, causal,
window_size, window_size,
softcapping, softcap,
alibi_slopes, alibi_slopes,
deterministic, deterministic,
return_attn_probs, return_attn_probs,
...@@ -1270,4 +1270,4 @@ def flash_attn_with_kvcache( ...@@ -1270,4 +1270,4 @@ def flash_attn_with_kvcache(
rotary_interleaved, rotary_interleaved,
num_splits, num_splits,
) )
return (out, softmax_lse) if return_softmax_lse else out return (out, softmax_lse) if return_softmax_lse else out
\ No newline at end of file
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment