Skip to content

GitLab

Unverified Commit 3f41b184 authored by fzyzcjy's avatar fzyzcjy Committed by GitHub
Browse files

Improve: Extract generation_manager from tokenizer_manager (#3115)

parent 45205d88
Expand all Hide whitespace changes
Inline Side-by-side
Showing with 748 additions and 571 deletions
python/sglang/srt/entrypoints/engine.py
View file @ 3f41b184
...@@ -463,5 +463,5 @@ def _launch_subprocesses(server_args: ServerArgs) -> Tuple[TokenizerManager, Dic ...@@ -463,5 +463,5 @@ def _launch_subprocesses(server_args: ServerArgs) -> Tuple[TokenizerManager, Dic
# Assume all schedulers have the same scheduler_info # Assume all schedulers have the same scheduler_info
scheduler_info = scheduler_infos[0] scheduler_info = scheduler_infos[0]
tokenizer_manager.max_req_input_len = scheduler_info["max_req_input_len"] tokenizer_manager.configure_max_req_input_len(scheduler_info["max_req_input_len"])
return tokenizer_manager, scheduler_info return tokenizer_manager, scheduler_info
python/sglang/srt/managers/generation_manager.py 0 → 100644
View file @ 3f41b184
This diff is collapsed.
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment