Skip to content
GitLab
Menu
Projects
Groups
Snippets
Loading...
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in / Register
Toggle navigation
Menu
Open sidebar
zhaoyu6
sglang
Commits
39c57317
"vscode:/vscode.git/clone" did not exist on "1d76a7c0b69f725e86cb91647dcc4b66f87abd9a"
Unverified
Commit
39c57317
authored
Jul 20, 2024
by
Liangsheng Yin
Committed by
GitHub
Jul 20, 2024
Browse files
Revert "Temporary fix invalid sample results" (#673)
parent
9592a1f3
Changes
1
Hide whitespace changes
Inline
Side-by-side
Showing
1 changed file
with
0 additions
and
10 deletions
+0
-10
python/sglang/srt/managers/controller/infer_batch.py
python/sglang/srt/managers/controller/infer_batch.py
+0
-10
No files found.
python/sglang/srt/managers/controller/infer_batch.py
View file @
39c57317
...
@@ -673,16 +673,6 @@ class Batch:
...
@@ -673,16 +673,6 @@ class Batch:
batch_next_token_ids
,
_
=
top_k_top_p_sampling_from_probs
(
batch_next_token_ids
,
_
=
top_k_top_p_sampling_from_probs
(
probs
,
uniform_samples
,
self
.
top_ks
,
self
.
top_ps
probs
,
uniform_samples
,
self
.
top_ks
,
self
.
top_ps
)
)
# FIXME: This is a temporary fix for the illegal token ids in sampling.
illegal_mask
=
(
batch_next_token_ids
<
0
or
batch_next_token_ids
>=
probs
.
shape
[
-
1
]
)
if
torch
.
any
(
illegal_mask
):
warnings
.
warn
(
"Illegal token ids in sampling."
)
batch_next_token_ids
=
torch
.
where
(
illegal_mask
,
torch
.
argmax
(
probs
,
dim
=-
1
),
batch_next_token_ids
)
except
RuntimeError
as
e
:
except
RuntimeError
as
e
:
warnings
.
warn
(
f
"Ignore errors in sampling:
{
e
}
"
)
warnings
.
warn
(
f
"Ignore errors in sampling:
{
e
}
"
)
batch_next_token_ids
=
torch
.
argmax
(
probs
,
dim
=-
1
)
batch_next_token_ids
=
torch
.
argmax
(
probs
,
dim
=-
1
)
...
...
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
.
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment