Skip to content
GitLab
Menu
Projects
Groups
Snippets
Loading...
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in
Toggle navigation
Menu
Open sidebar
OpenDAS
dynamo
Commits
db8e52f2
Unverified
Commit
db8e52f2
authored
Oct 22, 2025
by
Yan Ru Pei
Committed by
GitHub
Oct 22, 2025
Browse files
fix: correct prefill/decode block defaults when no overlaps (#3811)
Signed-off-by:
PeaBrane
<
yanrpei@gmail.com
>
parent
87ea346d
Changes
1
Show whitespace changes
Inline
Side-by-side
Showing
1 changed file
with
23 additions
and
19 deletions
+23
-19
lib/llm/src/kv_router/sequence.rs
lib/llm/src/kv_router/sequence.rs
+23
-19
No files found.
lib/llm/src/kv_router/sequence.rs
View file @
db8e52f2
...
...
@@ -826,17 +826,22 @@ impl ActiveSequencesMultiWorker {
let
token_sequence_shared
=
token_sequence
.map
(
Arc
::
new
);
let
mut
receivers
=
Vec
::
new
();
// Iterate through overlaps to process each WorkerWithDpRank
for
(
worker
,
overlap
)
in
overlaps
.scores
.iter
()
{
// Check if the worker has a sender
if
let
Some
(
sender
)
=
self
.senders
.get
(
worker
)
{
// Iterate through all workers, not just those with overlap
// This ensures we properly account for active tokens/blocks on all workers
for
sender_entry
in
self
.senders
.iter
()
{
let
worker
=
*
sender_entry
.key
();
let
sender
=
sender_entry
.value
();
// Get overlap for this worker (defaults to 0 if not in overlaps)
let
overlap
=
*
overlaps
.scores
.get
(
&
worker
)
.unwrap_or
(
&
0
);
let
(
resp_tx
,
resp_rx
)
=
tokio
::
sync
::
oneshot
::
channel
();
receivers
.push
((
*
worker
,
resp_rx
));
receivers
.push
((
worker
,
resp_rx
));
if
let
Err
(
e
)
=
sender
.send
(
UpdateSequences
::
PotentialBlocksAndTokens
{
token_sequence
:
token_sequence_shared
.clone
(),
isl
,
overlap
:
*
overlap
,
overlap
,
resp_tx
,
})
{
tracing
::
error!
(
...
...
@@ -846,7 +851,6 @@ impl ActiveSequencesMultiWorker {
);
}
}
}
// Collect results from all workers
for
(
worker
,
receiver
)
in
receivers
{
...
...
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
.
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment