Unverified Commit ad65177a authored by ChenqianCao's avatar ChenqianCao Committed by GitHub
Browse files

[Bugfix] Fix 'remove_instance_endpoint' method logic in disagg_proxy_demo (#32922)


Signed-off-by: default avatarChenqianCao <39755070+ChenqianCao@users.noreply.github.com>
Co-authored-by: default avatarHarry Mellor <19981378+hmellor@users.noreply.github.com>
parent d44a5b6c
......@@ -328,9 +328,9 @@ class Proxy:
if instance_type == "decode" and instance in self.decode_instances:
self.decode_instances.remove(instance)
self.decode_cycler = itertools.cycle(self.decode_instances)
if instance_type == "prefill" and instance in self.decode_instances:
if instance_type == "prefill" and instance in self.prefill_instances:
self.prefill_instances.remove(instance)
self.prefill_cycler = itertools.cycle(self.decode_instances)
self.prefill_cycler = itertools.cycle(self.prefill_instances)
class RoundRobinSchedulingPolicy(SchedulingPolicy):
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment