Skip to content

GitLab

Unverified Commit 8ac3ccc0 authored by Byron Hsu's avatar Byron Hsu Committed by GitHub
Browse files

Backend method not found when SRT Runtime is used (#1576)

parent 9b0926ce
Hide whitespace changes
Inline Side-by-side
Showing with 8 additions and 0 deletions
python/sglang/api.py
View file @ 8ac3ccc0
...@@ -48,6 +48,10 @@ def flush_cache(backend: Optional[BaseBackend] = None): ...@@ -48,6 +48,10 @@ def flush_cache(backend: Optional[BaseBackend] = None):
backend = backend or global_config.default_backend backend = backend or global_config.default_backend
if backend is None: if backend is None:
return False return False
# If backend is Runtime
if hasattr(backend, "endpoint"):
backend = backend.endpoint
return backend.flush_cache() return backend.flush_cache()
...@@ -55,6 +59,10 @@ def get_server_args(backend: Optional[BaseBackend] = None): ...@@ -55,6 +59,10 @@ def get_server_args(backend: Optional[BaseBackend] = None):
backend = backend or global_config.default_backend backend = backend or global_config.default_backend
if backend is None: if backend is None:
return None return None
# If backend is Runtime
if hasattr(backend, "endpoint"):
backend = backend.endpoint
return backend.get_server_args() return backend.get_server_args()
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment