"test/git@developer.sourcefind.cn:change/sglang.git" did not exist on "7907f9eb20a249aa4d931d8e8a9ae2514c292d13"
Unverified Commit bd35e9d0 authored by Xiaomeng Zhao's avatar Xiaomeng Zhao Committed by GitHub
Browse files

Merge pull request #1636 from myhloli/dev

perf(model): adjust batch ratio for GPU memory range
parents bbc8a9a0 29e7a948
......@@ -182,9 +182,9 @@ def doc_analyze(
batch_ratio = 2
elif 10 <= gpu_memory <= 12:
batch_ratio = 4
elif 12 < gpu_memory <= 16:
elif 12 < gpu_memory <= 20:
batch_ratio = 8
elif 16 < gpu_memory <= 24:
elif 20 < gpu_memory <= 32:
batch_ratio = 16
else:
batch_ratio = 32
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment