Skip to content
GitLab
Menu
Projects
Groups
Snippets
Loading...
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in / Register
Toggle navigation
Menu
Open sidebar
gaoqiong
flash-attention
Commits
8166063a
"docs/source/en/using-diffusers/other-modalities.mdx" did not exist on "c3d78cd3067612175ac9f0f8b234abf5a2e1f510"
Commit
8166063a
authored
Sep 12, 2022
by
Tri Dao
Browse files
Use block_size=128 for d=128 on SM86 to avoid exceeding smem limit
parent
13403e81
Changes
1
Hide whitespace changes
Inline
Side-by-side
Showing
1 changed file
with
1 addition
and
1 deletion
+1
-1
csrc/flash_attn/src/fmha_fprop_fp16_kernel.sm80.cu
csrc/flash_attn/src/fmha_fprop_fp16_kernel.sm80.cu
+1
-1
No files found.
csrc/flash_attn/src/fmha_fprop_fp16_kernel.sm80.cu
View file @
8166063a
...
@@ -133,7 +133,7 @@ void run_fmha_fp16_sm80(Launch_params<FMHA_fprop_params> &launch_params,
...
@@ -133,7 +133,7 @@ void run_fmha_fp16_sm80(Launch_params<FMHA_fprop_params> &launch_params,
using
Kernel_traits
=
FMHA_kernel_traits
<
128
,
128
,
16
,
1
,
4
,
0x08u
,
elem_type
>
;
using
Kernel_traits
=
FMHA_kernel_traits
<
128
,
128
,
16
,
1
,
4
,
0x08u
,
elem_type
>
;
run_fmha_fp16_sm80_loop_
<
Kernel_traits
>
(
launch_params
,
configure
);
run_fmha_fp16_sm80_loop_
<
Kernel_traits
>
(
launch_params
,
configure
);
}
else
{
}
else
{
if
(
dprops
->
major
==
8
&&
dprops
->
minor
>
=
0
&&
!
launch_params
.
is_dropout
)
{
if
(
dprops
->
major
==
8
&&
dprops
->
minor
=
=
0
&&
!
launch_params
.
is_dropout
)
{
// TD [2022-06-05] Keep K in registers to reduce register spilling
// TD [2022-06-05] Keep K in registers to reduce register spilling
// Gives about 6% speedup compared to using block size 128.
// Gives about 6% speedup compared to using block size 128.
using
Kernel_traits
=
FMHA_kernel_traits
<
256
,
128
,
16
,
1
,
4
,
0x18u
,
elem_type
>
;
using
Kernel_traits
=
FMHA_kernel_traits
<
256
,
128
,
16
,
1
,
4
,
0x18u
,
elem_type
>
;
...
...
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
.
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment