Skip to content

GitLab

"vllm/vscode:/vscode.git/clone" did not exist on "fc3eeea966d0b1fc0915709ef0de7a5d378619fa"
Commit dc5f1906 authored by zhuwenwen's avatar zhuwenwen
Browse files

update alibi

parent 3f946cfb
Show whitespace changes
Inline Side-by-side
Showing with 1 addition and 1 deletion
csrc/attention/attention_kernels_opt_tc.cu
View file @ dc5f1906
...@@ -281,7 +281,7 @@ __device__ void paged_attention_kernel_TC( ...@@ -281,7 +281,7 @@ __device__ void paged_attention_kernel_TC(
const int token_idx = block_idx * BLOCK_SIZE+rowid; const int token_idx = block_idx * BLOCK_SIZE+rowid;
if(alibi_slope[i] != 0){ if(alibi_slope[i] != 0){
float alibi=alibi_slope[i]* (token_idx - seq_len + 1); float alibi=alibi_slope[i]* (token_idx - seq_len + 1);
qk_vec[i] = alibi; qk_vec[i] += alibi;
} }
const bool mask = (token_idx >= seq_len); const bool mask = (token_idx >= seq_len);
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment