Skip to content

GitLab

Unverified Commit 1cebccc7 authored by drbh's avatar drbh Committed by GitHub
Browse files

fix: adds causal to attention params (#2408) 

fix: adds causal to attention params to check when using flash attn v1
parent 59922f9b
Hide whitespace changes
Inline Side-by-side
Showing with 1 addition and 0 deletions
server/text_generation_server/layers/attention/cuda.py
View file @ 1cebccc7
...@@ -293,6 +293,7 @@ else: ...@@ -293,6 +293,7 @@ else:
max_s, max_s,
softmax_scale, softmax_scale,
window_size_left=-1, window_size_left=-1,
causal=None,
softcap=None, softcap=None,
): ):
if window_size_left != -1: if window_size_left != -1:
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment