Skip to content
GitLab
Menu
Projects
Groups
Snippets
Loading...
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in
Toggle navigation
Menu
Open sidebar
OpenDAS
vllm_cscc
Commits
dc5f1906
"components/vscode:/vscode.git/clone" did not exist on "6642e23e0f731319536cc514ca75c7ba7e66ba68"
Commit
dc5f1906
authored
Jan 13, 2025
by
zhuwenwen
Browse files
update alibi
parent
3f946cfb
Changes
1
Show whitespace changes
Inline
Side-by-side
Showing
1 changed file
with
1 addition
and
1 deletion
+1
-1
csrc/attention/attention_kernels_opt_tc.cu
csrc/attention/attention_kernels_opt_tc.cu
+1
-1
No files found.
csrc/attention/attention_kernels_opt_tc.cu
View file @
dc5f1906
...
...
@@ -281,7 +281,7 @@ __device__ void paged_attention_kernel_TC(
const
int
token_idx
=
block_idx
*
BLOCK_SIZE
+
rowid
;
if
(
alibi_slope
[
i
]
!=
0
){
float
alibi
=
alibi_slope
[
i
]
*
(
token_idx
-
seq_len
+
1
);
qk_vec
[
i
]
=
alibi
;
qk_vec
[
i
]
+
=
alibi
;
}
const
bool
mask
=
(
token_idx
>=
seq_len
);
...
...
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
.
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment