Skip to content
GitLab
Menu
Projects
Groups
Snippets
Loading...
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in
Toggle navigation
Menu
Open sidebar
OpenDAS
vllm_cscc
Commits
324a3119
Unverified
Commit
324a3119
authored
May 07, 2025
by
Yong Hoon Shin
Committed by
GitHub
May 07, 2025
Browse files
Fix test_memory_usage_no_spec (#17754)
Signed-off-by:
Yong Hoon Shin
<
yhshin@meta.com
>
parent
8a15c260
Changes
1
Hide whitespace changes
Inline
Side-by-side
Showing
1 changed file
with
6 additions
and
6 deletions
+6
-6
tests/spec_decode/test_memory_usage.py
tests/spec_decode/test_memory_usage.py
+6
-6
No files found.
tests/spec_decode/test_memory_usage.py
View file @
324a3119
...
...
@@ -42,12 +42,12 @@ we can ensure we go through the _no_spec codepath for most of our engine steps.
def
test_memory_usage_no_spec
():
previous_memory_allocated
=
None
llm
=
vllm
.
LLM
(
model
=
MAIN_MODEL
,
speculative_
model
=
SPEC_MODEL
,
num_speculative_tokens
=
3
,
speculative_
disable_by_batch_size
=
SPEC_DISABLE_BATCH_SIZE
,
)
llm
=
vllm
.
LLM
(
model
=
MAIN_MODEL
,
speculative_config
=
{
"
model
"
:
SPEC_MODEL
,
"
num_speculative_tokens
"
:
3
,
"
disable_by_batch_size
"
:
SPEC_DISABLE_BATCH_SIZE
,
}
)
batch_sequences
=
set
()
engine
=
llm
.
llm_engine
...
...
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
.
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment