Skip to content

GitLab

Unverified Commit 2b079f89 authored by Lianmin Zheng's avatar Lianmin Zheng Committed by GitHub
Browse files

Increase interpreter parallelism (#46)

parent 05b4c398
Hide whitespace changes
Inline Side-by-side
Showing with 1 addition and 1 deletion
python/sglang/lang/interpreter.py
View file @ 2b079f89
...@@ -80,7 +80,7 @@ def run_program_batch( ...@@ -80,7 +80,7 @@ def run_program_batch(
# Run all programs # Run all programs
if num_threads == "auto": if num_threads == "auto":
num_threads = multiprocessing.cpu_count() num_threads = min(64, multiprocessing.cpu_count() * 8)
num_threads = min(num_threads, len(batch_arguments)) num_threads = min(num_threads, len(batch_arguments))
if num_threads == 1: if num_threads == 1:
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment