Unverified Commit db986c19 authored by Michael Goin's avatar Michael Goin Committed by GitHub
Browse files

Fix precommit fail in fused_moe intermediate_cache2 chunking (#13772)


Signed-off-by: default avatarmgoin <mgoin64@gmail.com>
parent 22757848
...@@ -1271,7 +1271,8 @@ def fused_experts_impl(hidden_states: torch.Tensor, ...@@ -1271,7 +1271,8 @@ def fused_experts_impl(hidden_states: torch.Tensor,
# so the cache size and config are already set correctly and # so the cache size and config are already set correctly and
# do not need to be adjusted. # do not need to be adjusted.
intermediate_cache1 = intermediate_cache1[:tokens_in_chunk] intermediate_cache1 = intermediate_cache1[:tokens_in_chunk]
intermediate_cache2 = intermediate_cache2[:tokens_in_chunk * topk_ids.shape[1]] intermediate_cache2 = intermediate_cache2[:tokens_in_chunk *
topk_ids.shape[1]]
intermediate_cache3 = intermediate_cache3[:tokens_in_chunk] intermediate_cache3 = intermediate_cache3[:tokens_in_chunk]
config = get_config_func(tokens_in_chunk) config = get_config_func(tokens_in_chunk)
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment