Skip to content

GitLab

Unverified Commit 843af7f7 authored by Li, Jiang's avatar Li, Jiang Committed by GitHub
Browse files

[Bugfix][CPU] Disable dual stream execution for experts on CPU (#27320) 


Signed-off-by: default avatarjiang1.li <jiang1.li@intel.com>
parent 1f633b86
Hide whitespace changes
Inline Side-by-side
Showing with 3 additions and 0 deletions
vllm/platforms/cpu.py
View file @ 843af7f7
...@@ -297,6 +297,9 @@ class CpuPlatform(Platform): ...@@ -297,6 +297,9 @@ class CpuPlatform(Platform):
# Disable torch async compiling which won't work with daemonic processes # Disable torch async compiling which won't work with daemonic processes
os.environ["TORCHINDUCTOR_COMPILE_THREADS"] = "1" os.environ["TORCHINDUCTOR_COMPILE_THREADS"] = "1"
# Disable multi-stream for shared experts as no Stream on CPU
os.environ["VLLM_DISABLE_SHARED_EXPERTS_STREAM"] = "0"
# Intel OpenMP setting # Intel OpenMP setting
ld_prealod_str = os.getenv("LD_PRELOAD", "") ld_prealod_str = os.getenv("LD_PRELOAD", "")
if "libiomp5.so" in ld_prealod_str: if "libiomp5.so" in ld_prealod_str:
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment