Skip to content
GitLab
Menu
Projects
Groups
Snippets
Loading...
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in / Register
Toggle navigation
Menu
Open sidebar
change
sglang
Commits
538fa0ae
Unverified
Commit
538fa0ae
authored
Nov 25, 2024
by
Yixin Dong
Committed by
GitHub
Nov 25, 2024
Browse files
[Fix] Avoid calling fill_vocab_mask for terminated requests (#2175)
parent
55842eb8
Changes
1
Show whitespace changes
Inline
Side-by-side
Showing
1 changed file
with
4 additions
and
1 deletion
+4
-1
python/sglang/srt/sampling/sampling_batch_info.py
python/sglang/srt/sampling/sampling_batch_info.py
+4
-1
No files found.
python/sglang/srt/sampling/sampling_batch_info.py
View file @
538fa0ae
...
@@ -170,7 +170,10 @@ class SamplingBatchInfo:
...
@@ -170,7 +170,10 @@ class SamplingBatchInfo:
for
i
,
grammar
in
enumerate
(
self
.
grammars
):
for
i
,
grammar
in
enumerate
(
self
.
grammars
):
if
grammar
is
not
None
:
if
grammar
is
not
None
:
try
:
grammar
.
fill_vocab_mask
(
self
.
vocab_mask
,
i
)
grammar
.
fill_vocab_mask
(
self
.
vocab_mask
,
i
)
except
RuntimeError
:
continue
def
filter_batch
(
self
,
unfinished_indices
:
List
[
int
],
new_indices
:
torch
.
Tensor
):
def
filter_batch
(
self
,
unfinished_indices
:
List
[
int
],
new_indices
:
torch
.
Tensor
):
self
.
penalizer_orchestrator
.
filter
(
unfinished_indices
,
new_indices
)
self
.
penalizer_orchestrator
.
filter
(
unfinished_indices
,
new_indices
)
...
...
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
.
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment