Skip to content

GitLab

Unverified Commit f9170209 authored by Xin Yang's avatar Xin Yang Committed by GitHub
Browse files

[Perf] Optimize FusedMoEModularKernel output tensor using torch.empty (#35794) 


Signed-off-by: default avatarXin Yang <xyangx@amazon.com>
parent 86483ca7
Hide whitespace changes
Inline Side-by-side
Showing with 1 addition and 1 deletion
vllm/model_executor/layers/fused_moe/modular_kernel.py
View file @ f9170209
...@@ -1519,7 +1519,7 @@ class FusedMoEKernelModularImpl: ...@@ -1519,7 +1519,7 @@ class FusedMoEKernelModularImpl:
assert not disable_inplace() assert not disable_inplace()
output = hidden_states output = hidden_states
else: else:
output = torch.zeros_like(hidden_states) output = torch.empty_like(hidden_states)
local_num_experts = w1.size(0) local_num_experts = w1.size(0)
if global_num_experts == -1: if global_num_experts == -1:
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment