Skip to content

GitLab

"...text-generation-inference.git" did not exist on "14a0df3a387d321cef5868db07e8499b6838dc1c"
Unverified Commit ac971ff6 authored by zhyncs's avatar zhyncs Committed by GitHub
Browse files

perf: reduce ttft and itl with stream_interval 1 (#658)

parent e1792cca
Hide whitespace changes
Inline Side-by-side
Showing with 1 addition and 1 deletion
python/sglang/srt/server_args.py
View file @ ac971ff6
...@@ -33,7 +33,7 @@ class ServerArgs: ...@@ -33,7 +33,7 @@ class ServerArgs:
# Other runtime options # Other runtime options
tp_size: int = 1 tp_size: int = 1
stream_interval: int = 8 stream_interval: int = 1
random_seed: Optional[int] = None random_seed: Optional[int] = None
# Logging # Logging
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment