Unverified Commit 1ee35382 authored by Wentao Ye's avatar Wentao Ye Committed by GitHub
Browse files

[Bug] Fix modular_kernel: ZeroDivisionError: integer division or modulo by zero (#26528)


Signed-off-by: default avataryewentao256 <zhyanwentao@126.com>
parent 6e783bc5
...@@ -717,10 +717,13 @@ class FusedMoEModularKernel(torch.nn.Module): ...@@ -717,10 +717,13 @@ class FusedMoEModularKernel(torch.nn.Module):
get num_chunks == 1. Take max(M, 1) to avoid divide by zero. get num_chunks == 1. Take max(M, 1) to avoid divide by zero.
If there are no tokens to process, the number of chunks will be zero. If there are no tokens to process, the number of chunks will be zero.
""" """
CHUNK_SIZE = ( CHUNK_SIZE = max(
max(M, 1) 1,
(
M
if not self.fused_experts.supports_chunking() if not self.fused_experts.supports_chunking()
else min(M, envs.VLLM_FUSED_MOE_CHUNK_SIZE) else min(M, envs.VLLM_FUSED_MOE_CHUNK_SIZE)
),
) )
num_chunks = cdiv(M, CHUNK_SIZE) num_chunks = cdiv(M, CHUNK_SIZE)
# If there are no tokens, then there should be no loop iterations. # If there are no tokens, then there should be no loop iterations.
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment