Skip to content
GitLab
Menu
Projects
Groups
Snippets
Loading...
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in
Toggle navigation
Menu
Open sidebar
OpenDAS
vllm_cscc
Commits
fabe89bb
Unverified
Commit
fabe89bb
authored
May 16, 2025
by
Woosuk Kwon
Committed by
GitHub
May 16, 2025
Browse files
[Spec Decode] Don't fall back to V0 when spec decoding is enabled (#18265)
parent
e73b7dfd
Changes
1
Hide whitespace changes
Inline
Side-by-side
Showing
1 changed file
with
1 addition
and
9 deletions
+1
-9
vllm/engine/arg_utils.py
vllm/engine/arg_utils.py
+1
-9
No files found.
vllm/engine/arg_utils.py
View file @
fabe89bb
...
...
@@ -1325,7 +1325,7 @@ class EngineArgs:
recommend_to_remove
=
False
)
return
False
#
Only
Ngram speculative decoding
so far
.
#
V1 supports
N
-
gram
, Medusa, and Eagle
speculative decoding.
is_ngram_enabled
=
False
is_eagle_enabled
=
False
is_medusa_enabled
=
False
...
...
@@ -1390,14 +1390,6 @@ class EngineArgs:
_raise_or_fallback
(
feature_name
=
name
,
recommend_to_remove
=
False
)
return
False
# ngram is supported on V1, but off by default for now.
if
is_ngram_enabled
and
_warn_or_fallback
(
"ngram"
):
return
False
# Eagle is under development, so we don't support it yet.
if
is_eagle_enabled
and
_warn_or_fallback
(
"Eagle"
):
return
False
# Non-[CUDA, TPU] may be supported on V1, but off by default for now.
v0_hardware
=
not
any
(
(
current_platform
.
is_cuda
(),
current_platform
.
is_tpu
()))
...
...
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
.
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment