Skip to content
GitLab
Menu
Projects
Groups
Snippets
Loading...
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in / Register
Toggle navigation
Menu
Open sidebar
change
sglang
Commits
b808a383
"src/vscode:/vscode.git/clone" did not exist on "fe7948941d0ef556319900ebf965cda995bc14bb"
Unverified
Commit
b808a383
authored
Nov 12, 2024
by
Ke Bao
Committed by
GitHub
Nov 12, 2024
Browse files
Filter empty prompt in random bench serving (#2011)
parent
602ebc66
Changes
1
Hide whitespace changes
Inline
Side-by-side
Showing
1 changed file
with
10 additions
and
2 deletions
+10
-2
python/sglang/bench_serving.py
python/sglang/bench_serving.py
+10
-2
No files found.
python/sglang/bench_serving.py
View file @
b808a383
...
@@ -596,12 +596,20 @@ def sample_random_requests(
...
@@ -596,12 +596,20 @@ def sample_random_requests(
# Filter out sequences that are too long or too short
# Filter out sequences that are too long or too short
input_requests
:
List
[
Tuple
[
str
,
int
,
int
]]
=
[]
input_requests
:
List
[
Tuple
[
str
,
int
,
int
]]
=
[]
for
i
in
range
(
num_prompts
):
for
data
in
dataset
:
i
=
len
(
input_requests
)
if
i
==
num_prompts
:
break
# Tokenize the prompts and completions.
# Tokenize the prompts and completions.
prompt
=
data
set
[
i
]
[
0
]
prompt
=
data
[
0
]
prompt_token_ids
=
tokenizer
.
encode
(
prompt
)
prompt_token_ids
=
tokenizer
.
encode
(
prompt
)
prompt_len
=
len
(
prompt_token_ids
)
prompt_len
=
len
(
prompt_token_ids
)
# Skip empty prompt
if
prompt_len
==
0
:
continue
if
prompt_len
>
input_lens
[
i
]:
if
prompt_len
>
input_lens
[
i
]:
input_ids
=
prompt_token_ids
[:
input_lens
[
i
]]
input_ids
=
prompt_token_ids
[:
input_lens
[
i
]]
else
:
else
:
...
...
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
.
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment