Skip to content

GitLab

Unverified Commit dc67d976 authored by Yineng Zhang's avatar Yineng Zhang Committed by GitHub
Browse files

misc: speedup load safetensors (#1319) 


Co-authored-by: default avatarispobock <ISPObaoke@163.com>
parent 1e495e08
Hide whitespace changes
Inline Side-by-side
Showing with 1 addition and 0 deletions
python/sglang/srt/model_executor/model_runner.py
View file @ dc67d976
......@@ -162,6 +162,7 @@ class ModelRunner:
return min_per_gpu_memory
def load_model(self):
torch.set_num_threads(1)
logger.info(
f"Load weight begin. avail mem={get_available_gpu_memory(self.gpu_id):.2f} GB"
)
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment