Skip to content

GitLab

Unverified Commit 19849db5 authored by Zhuofan's avatar Zhuofan Committed by GitHub
Browse files

[Fix] Fix bugs in scheduler (#1727)

parent 3d4ceb29
Hide whitespace changes
Inline Side-by-side
Showing with 2 additions and 1 deletion
vllm/core/scheduler.py
View file @ 19849db5
...@@ -196,7 +196,8 @@ class Scheduler: ...@@ -196,7 +196,8 @@ class Scheduler:
scheduler_outputs = SchedulerOutputs( scheduler_outputs = SchedulerOutputs(
scheduled_seq_groups=scheduled, scheduled_seq_groups=scheduled,
prompt_run=True, prompt_run=True,
num_batched_tokens=len(seq_lens) * max(seq_lens), num_batched_tokens=len(seq_lens) *
max(seq_lens) if seq_lens else 0,
blocks_to_swap_in=blocks_to_swap_in, blocks_to_swap_in=blocks_to_swap_in,
blocks_to_swap_out=blocks_to_swap_out, blocks_to_swap_out=blocks_to_swap_out,
blocks_to_copy=blocks_to_copy, blocks_to_copy=blocks_to_copy,
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment