Skip to content
GitLab
Menu
Projects
Groups
Snippets
Loading...
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in / Register
Toggle navigation
Menu
Open sidebar
gaoqiong
composable_kernel_ROCM
Commits
8f360206
"...composable_kernel_rocm.git" did not exist on "2a0e5439e176fd5063c1c39fc1e14bd68e0f6796"
Commit
8f360206
authored
Dec 03, 2024
by
Po Yen Chen
Browse files
Only use vllm kcache layout in group mode
parent
ead9c3cb
Changes
2
Hide whitespace changes
Inline
Side-by-side
Showing
2 changed files
with
4 additions
and
3 deletions
+4
-3
example/ck_tile/01_fmha/fmha_fwd.cpp
example/ck_tile/01_fmha/fmha_fwd.cpp
+3
-2
include/ck_tile/ops/fmha/kernel/fmha_fwd_splitkv_kernel.hpp
include/ck_tile/ops/fmha/kernel/fmha_fwd_splitkv_kernel.hpp
+1
-1
No files found.
example/ck_tile/01_fmha/fmha_fwd.cpp
View file @
8f360206
...
@@ -770,9 +770,10 @@ bool run(const ck_tile::ArgParser& arg_parser)
...
@@ -770,9 +770,10 @@ bool run(const ck_tile::ArgParser& arg_parser)
q_buf
.
ToDevice
(
q_host
.
data
());
q_buf
.
ToDevice
(
q_host
.
data
());
if
(
0
<
page_block_size
)
if
(
mode
==
mode_enum
::
group
&&
0
<
page_block_size
)
{
{
if
(
!
(
i_perm
&&
!
is_v_rowmajor
))
{
if
(
!
(
i_perm
&&
!
is_v_rowmajor
))
{
std
::
cerr
<<
"make sure input layout is correct"
<<
std
::
endl
;
std
::
cerr
<<
"make sure input layout is correct"
<<
std
::
endl
;
return
false
;
return
false
;
}
}
...
...
include/ck_tile/ops/fmha/kernel/fmha_fwd_splitkv_kernel.hpp
View file @
8f360206
...
@@ -636,7 +636,7 @@ struct FmhaFwdSplitKVKernel
...
@@ -636,7 +636,7 @@ struct FmhaFwdSplitKVKernel
const
auto
make_k_dram
=
[
&
](
const
KDataType
*
data
,
index_t
height
)
{
const
auto
make_k_dram
=
[
&
](
const
KDataType
*
data
,
index_t
height
)
{
auto
k_dram_naive
=
[
&
]
{
auto
k_dram_naive
=
[
&
]
{
if
constexpr
(
kIsPagedKV
)
if
constexpr
(
kIsGroupMode
&&
kIsPagedKV
)
{
{
constexpr
index_t
vector_size
=
16
/
sizeof
(
KDataType
);
constexpr
index_t
vector_size
=
16
/
sizeof
(
KDataType
);
// (hdim_q/vector_size, page_block_size, vector_size)
// (hdim_q/vector_size, page_block_size, vector_size)
...
...
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
.
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment