Skip to content
GitLab
Menu
Projects
Groups
Snippets
Loading...
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in / Register
Toggle navigation
Menu
Open sidebar
change
sglang
Commits
fcc0f5ed
"tests/vscode:/vscode.git/clone" did not exist on "7eb4de4bed6115e9d87be77586593dd4364b9be2"
Unverified
Commit
fcc0f5ed
authored
Aug 11, 2024
by
Liangsheng Yin
Committed by
GitHub
Aug 11, 2024
Browse files
Fix wrong assert (#1028)
parent
a97df791
Changes
1
Hide whitespace changes
Inline
Side-by-side
Showing
1 changed file
with
3 additions
and
1 deletion
+3
-1
python/sglang/srt/mem_cache/chunk_cache.py
python/sglang/srt/mem_cache/chunk_cache.py
+3
-1
No files found.
python/sglang/srt/mem_cache/chunk_cache.py
View file @
fcc0f5ed
...
...
@@ -39,10 +39,12 @@ class ChunkCache(BasePrefixCache):
kv_indices
=
self
.
req_to_token_pool
.
req_to_token
[
req
.
req_pool_idx
,
:
len
(
token_ids
)
]
assert
req
.
rid
in
self
.
entries
self
.
req_to_token_pool
.
free
(
req
.
req_pool_idx
)
self
.
token_to_kv_pool
.
free
(
kv_indices
)
if
req
.
rid
in
self
.
entries
:
del
self
.
entries
[
req
.
rid
]
def
cache_unfinished_req
(
self
,
req
:
"Req"
,
token_ids
=
None
):
if
token_ids
is
None
:
token_ids
=
req
.
fill_ids
...
...
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
.
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment