Skip to content

GitLab

"git@developer.sourcefind.cn:OpenDAS/TransformerEngine.git" did not exist on "0653951488f2ff63bccbe3604da064b242803a4f"
Unverified Commit f4f35c2f authored by Charlene Yang's avatar Charlene Yang Committed by GitHub
Browse files

[common] Add max_t support for KV in THD (#1370) 



add max_t for KV
Signed-off-by: default avatarCharlene Yang <8636796+cyanguwa@users.noreply.github.com>
parent 0196ed44
Hide whitespace changes
Inline Side-by-side
Showing with 1 addition and 0 deletions
transformer_engine/common/fused_attn/fused_attn_f16_arbitrary_seqlen.cu
View file @ f4f35c2f
...@@ -661,6 +661,7 @@ void fused_attn_arbitrary_seqlen_bwd_impl( ...@@ -661,6 +661,7 @@ void fused_attn_arbitrary_seqlen_bwd_impl(
if (is_ragged && cudnn_runtime_version >= 90600) { if (is_ragged && cudnn_runtime_version >= 90600) {
sdpa_backward_options.set_max_total_seq_len_q(s_q); sdpa_backward_options.set_max_total_seq_len_q(s_q);
sdpa_backward_options.set_max_total_seq_len_kv(s_kv);
} }
if (cudnn_runtime_version >= 90200 && window_size_left != -1) { if (cudnn_runtime_version >= 90200 && window_size_left != -1) {
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment