Skip to content

GitLab

Unverified Commit f11eb90f authored by Enrique Shockwave's avatar Enrique Shockwave Committed by GitHub
Browse files

Initialize model_worker_batch variable (#1973)

parent 95a4ed12
Hide whitespace changes
Inline Side-by-side
Showing with 1 addition and 1 deletion
python/sglang/srt/managers/scheduler.py
View file @ f11eb90f
...@@ -787,8 +787,8 @@ class Scheduler: ...@@ -787,8 +787,8 @@ class Scheduler:
self.forward_ct += 1 self.forward_ct += 1
if self.is_generation: if self.is_generation:
model_worker_batch = batch.get_model_worker_batch()
if batch.forward_mode.is_decode() or batch.extend_num_tokens != 0: if batch.forward_mode.is_decode() or batch.extend_num_tokens != 0:
model_worker_batch = batch.get_model_worker_batch()
batch.mark_reqs_started() batch.mark_reqs_started()
logits_output, next_token_ids = self.tp_worker.forward_batch_generation( logits_output, next_token_ids = self.tp_worker.forward_batch_generation(
model_worker_batch model_worker_batch
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment