Skip to content

GitLab

Unverified Commit 60e419c1 authored by bnellnm's avatar bnellnm Committed by GitHub
Browse files

[Misc] cache result of disable_inplace (#26666) 


Signed-off-by: default avatarBill Nell <bnell@redhat.com>
parent 7ef60528
Show whitespace changes
Inline Side-by-side
Showing with 2 additions and 0 deletions
vllm/model_executor/layers/fused_moe/utils.py
View file @ 60e419c1
# SPDX-License-Identifier: Apache-2.0 # SPDX-License-Identifier: Apache-2.0
# SPDX-FileCopyrightText: Copyright contributors to the vLLM project # SPDX-FileCopyrightText: Copyright contributors to the vLLM project
import functools
from math import prod from math import prod
import torch import torch
...@@ -325,5 +326,6 @@ def activation_without_mul(activation: str) -> str: ...@@ -325,5 +326,6 @@ def activation_without_mul(activation: str) -> str:
# Torch custom ops can't deal with outputs aliasing inputs so we need to # Torch custom ops can't deal with outputs aliasing inputs so we need to
# disable inplace for torch >= 2.9. # disable inplace for torch >= 2.9.
# See https://github.com/vllm-project/vllm/issues/26378 # See https://github.com/vllm-project/vllm/issues/26378
@functools.cache
def disable_inplace() -> bool: def disable_inplace() -> bool:
return is_torch_equal_or_newer("2.9") return is_torch_equal_or_newer("2.9")
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment