Skip to content

GitLab

Commit 3946292a authored by 王敏's avatar 王敏
Browse files

[fix]添加VLLM_USE_LIGHTOP控制lightop topp_topk融合算子开关

parent a82a91db
Show whitespace changes
Inline Side-by-side
Showing with 1 addition and 1 deletion
vllm/v1/sample/ops/topk_topp_sampler.py
View file @ 3946292a
...@@ -94,7 +94,7 @@ class TopKTopPSampler(nn.Module): ...@@ -94,7 +94,7 @@ class TopKTopPSampler(nn.Module):
self.forward = self.forward_native self.forward = self.forward_native
else: else:
self.forward = self.forward_native self.forward = self.forward_native
if HAS_LIGHTOP_OPT_KERNEL: if HAS_LIGHTOP_OPT_KERNEL and envs.VLLM_USE_LIGHTOP:
self.forward = self.forward_lightop_opt self.forward = self.forward_lightop_opt
self.apply_top_k_top_p = apply_top_k_top_p self.apply_top_k_top_p = apply_top_k_top_p
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment