Skip to content
GitLab
Menu
Projects
Groups
Snippets
Loading...
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in / Register
Toggle navigation
Menu
Open sidebar
gaoqiong
flash-attention
Commits
4df62e14
"deploy/hubserving/structure_table/module.py" did not exist on "bfcc5f737edc132e3221f8dbf3d2362c4bf9300f"
Unverified
Commit
4df62e14
authored
Jul 22, 2024
by
Jorge António
Committed by
GitHub
Jul 21, 2024
Browse files
catch typo (#1058)
parent
74b0761f
Changes
1
Hide whitespace changes
Inline
Side-by-side
Showing
1 changed file
with
2 additions
and
2 deletions
+2
-2
csrc/flash_attn/flash_api.cpp
csrc/flash_attn/flash_api.cpp
+2
-2
No files found.
csrc/flash_attn/flash_api.cpp
View file @
4df62e14
...
...
@@ -383,7 +383,7 @@ mha_fwd(at::Tensor &q, // batch_size x seqlen_q x num_heads x head_size
const
int
head_size_og
=
sizes
[
3
];
const
int
seqlen_k
=
k
.
size
(
1
);
const
int
num_heads_k
=
k
.
size
(
2
);
TORCH_CHECK
(
batch_size
>
0
,
"batch size must be postive"
);
TORCH_CHECK
(
batch_size
>
0
,
"batch size must be pos
i
tive"
);
TORCH_CHECK
(
head_size_og
<=
256
,
"FlashAttention forward only supports head dimension at most 256"
);
TORCH_CHECK
(
num_heads
%
num_heads_k
==
0
,
"Number of heads in key/value must divide number of heads in query"
);
...
...
@@ -1350,7 +1350,7 @@ mha_fwd_kvcache(at::Tensor &q, // batch_size x seqlen_q x num_he
const
int
seqlen_k
=
!
paged_KV
?
kcache
.
size
(
1
)
:
max_num_blocks_per_seq
*
page_block_size
;
const
int
num_heads_k
=
kcache
.
size
(
2
);
const
int
batch_size_c
=
!
paged_KV
?
kcache
.
size
(
0
)
:
batch_size
;
TORCH_CHECK
(
batch_size
>
0
,
"batch size must be postive"
);
TORCH_CHECK
(
batch_size
>
0
,
"batch size must be pos
i
tive"
);
TORCH_CHECK
(
head_size_og
<=
256
,
"FlashAttention forward only supports head dimension at most 256"
);
TORCH_CHECK
(
num_heads
%
num_heads_k
==
0
,
"Number of heads in key/value must divide number of heads in query"
);
...
...
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
.
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment