Skip to content
GitLab
Menu
Projects
Groups
Snippets
Loading...
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in
Toggle navigation
Menu
Open sidebar
OpenDAS
vllm_cscc
Commits
8bab4959
Unverified
Commit
8bab4959
authored
Jun 11, 2024
by
Woosuk Kwon
Committed by
GitHub
Jun 11, 2024
Browse files
[Misc] Remove VLLM_BUILD_WITH_NEURON env variable (#5389)
parent
3c4cebf7
Changes
3
Show whitespace changes
Inline
Side-by-side
Showing
3 changed files
with
2 additions
and
7 deletions
+2
-7
Dockerfile.neuron
Dockerfile.neuron
+1
-1
setup.py
setup.py
+1
-1
vllm/envs.py
vllm/envs.py
+0
-5
No files found.
Dockerfile.neuron
View file @
8bab4959
...
...
@@ -28,7 +28,7 @@ COPY ./requirements-neuron.txt /app/vllm/requirements-neuron.txt
RUN cd /app/vllm \
&& python3 -m pip install -U -r requirements-neuron.txt
ENV VLLM_
BUILD_WITH_NEURON 1
ENV VLLM_
TARGET_DEVICE neuron
RUN cd /app/vllm \
&& pip install -e . \
&& cd ..
...
...
setup.py
View file @
8bab4959
...
...
@@ -222,7 +222,7 @@ def _is_neuron() -> bool:
subprocess
.
run
([
"neuron-ls"
],
capture_output
=
True
,
check
=
True
)
except
(
FileNotFoundError
,
PermissionError
,
subprocess
.
CalledProcessError
):
torch_neuronx_installed
=
False
return
torch_neuronx_installed
or
envs
.
VLLM_BUILD_WITH_NEURON
return
torch_neuronx_installed
or
VLLM_TARGET_DEVICE
==
"neuron"
def
_is_cpu
()
->
bool
:
...
...
vllm/envs.py
View file @
8bab4959
...
...
@@ -33,7 +33,6 @@ if TYPE_CHECKING:
VLLM_TARGET_DEVICE
:
str
=
"cuda"
MAX_JOBS
:
Optional
[
str
]
=
None
NVCC_THREADS
:
Optional
[
str
]
=
None
VLLM_BUILD_WITH_NEURON
:
bool
=
False
VLLM_USE_PRECOMPILED
:
bool
=
False
VLLM_INSTALL_PUNICA_KERNELS
:
bool
=
False
CMAKE_BUILD_TYPE
:
Optional
[
str
]
=
None
...
...
@@ -63,10 +62,6 @@ environment_variables: Dict[str, Callable[[], Any]] = {
"NVCC_THREADS"
:
lambda
:
os
.
getenv
(
"NVCC_THREADS"
,
None
),
# If set, vllm will build with Neuron support
"VLLM_BUILD_WITH_NEURON"
:
lambda
:
bool
(
os
.
environ
.
get
(
"VLLM_BUILD_WITH_NEURON"
,
False
)),
# If set, vllm will use precompiled binaries (*.so)
"VLLM_USE_PRECOMPILED"
:
lambda
:
bool
(
os
.
environ
.
get
(
"VLLM_USE_PRECOMPILED"
)),
...
...
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
.
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment