Skip to content
GitLab
Menu
Projects
Groups
Snippets
Loading...
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in
Toggle navigation
Menu
Open sidebar
OpenDAS
vllm_cscc
Commits
0967110e
Unverified
Commit
0967110e
authored
Mar 10, 2025
by
Woosuk Kwon
Committed by
GitHub
Mar 10, 2025
Browse files
[Minor] Update the tqdm bar for parallel sampling (#14571)
Signed-off-by:
Woosuk Kwon
<
woosuk.kwon@berkeley.edu
>
parent
fb0acb6c
Changes
1
Hide whitespace changes
Inline
Side-by-side
Showing
1 changed file
with
3 additions
and
2 deletions
+3
-2
vllm/entrypoints/llm.py
vllm/entrypoints/llm.py
+3
-2
No files found.
vllm/entrypoints/llm.py
View file @
0967110e
...
...
@@ -1384,8 +1384,9 @@ class LLM:
if
use_tqdm
:
if
isinstance
(
output
,
RequestOutput
):
# Calculate tokens only for RequestOutput
n
=
len
(
output
.
outputs
)
assert
output
.
prompt_token_ids
is
not
None
total_in_toks
+=
len
(
output
.
prompt_token_ids
)
total_in_toks
+=
len
(
output
.
prompt_token_ids
)
*
n
in_spd
=
total_in_toks
/
pbar
.
format_dict
[
"elapsed"
]
total_out_toks
+=
sum
(
len
(
stp
.
token_ids
)
for
stp
in
output
.
outputs
)
...
...
@@ -1394,7 +1395,7 @@ class LLM:
pbar
.
postfix
=
(
f
"est. speed input:
{
in_spd
:.
2
f
}
toks/s, "
f
"output:
{
out_spd
:.
2
f
}
toks/s"
)
pbar
.
update
(
len
(
output
.
outputs
)
)
pbar
.
update
(
n
)
else
:
pbar
.
update
(
1
)
...
...
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
.
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment