Skip to content

GitLab

Unverified Commit e4865b69 authored by Hongkuan Zhou's avatar Hongkuan Zhou Committed by GitHub
Browse files

fix: add kv transfer config to planner vllm examples (#7163) 


Signed-off-by: default avatarhongkuanz <hongkuanz@nvidia.com>
parent 6634f33f
Show whitespace changes
Inline Side-by-side
Showing with 6 additions and 0 deletions
tests/planner/perf_test_configs/disagg_8b_planner.yaml
View file @ e4865b69
...@@ -190,6 +190,8 @@ spec: ...@@ -190,6 +190,8 @@ spec:
- nvidia/Llama-3.1-8B-Instruct-FP8 - nvidia/Llama-3.1-8B-Instruct-FP8
- --disaggregation-mode - --disaggregation-mode
- prefill - prefill
- --kv-transfer-config
- '{"kv_connector":"NixlConnector","kv_role":"kv_both"}'
- --no-enable-prefix-caching - --no-enable-prefix-caching
- --block-size - --block-size
- "128" - "128"
tests/planner/scaling/disagg_planner_load.yaml
View file @ e4865b69
...@@ -74,3 +74,5 @@ spec: ...@@ -74,3 +74,5 @@ spec:
- nvidia/Llama-3.1-8B-Instruct-FP8 - nvidia/Llama-3.1-8B-Instruct-FP8
- --disaggregation-mode - --disaggregation-mode
- prefill - prefill
- --kv-transfer-config
- '{"kv_connector":"NixlConnector","kv_role":"kv_both"}'
tests/planner/scaling/disagg_planner_throughput.yaml
View file @ e4865b69
...@@ -68,3 +68,5 @@ spec: ...@@ -68,3 +68,5 @@ spec:
- nvidia/Llama-3.1-8B-Instruct-FP8 - nvidia/Llama-3.1-8B-Instruct-FP8
- --disaggregation-mode - --disaggregation-mode
- prefill - prefill
- --kv-transfer-config
- '{"kv_connector":"NixlConnector","kv_role":"kv_both"}'
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment