Skip to content

GitLab

Unverified Commit bfb95df7 authored by Hongkuan Zhou's avatar Hongkuan Zhou Committed by GitHub
Browse files

fix: add mem frac for sglang dsr1 8gpu (#5260) 


Signed-off-by: default avatarhongkuanz <hongkuanz@nvidia.com>
parent 6458ef80
Hide whitespace changes
Inline Side-by-side
Showing with 4 additions and 0 deletions
recipes/deepseek-r1/sglang/disagg-8gpu/deploy.yaml
View file @ bfb95df7
...@@ -60,6 +60,8 @@ spec: ...@@ -60,6 +60,8 @@ spec:
- decode - decode
- --disaggregation-bootstrap-port - --disaggregation-bootstrap-port
- "30001" - "30001"
- --mem-fraction-static
- "0.75"
- --host - --host
- 0.0.0.0 - 0.0.0.0
- --prefill-round-robin-balance - --prefill-round-robin-balance
...@@ -97,6 +99,8 @@ spec: ...@@ -97,6 +99,8 @@ spec:
- prefill - prefill
- --disaggregation-bootstrap-port - --disaggregation-bootstrap-port
- "30001" - "30001"
- --mem-fraction-static
- "0.75"
- --host - --host
- 0.0.0.0 - 0.0.0.0
- --load-balance-method - --load-balance-method
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment