Skip to content

GitLab

Unverified Commit 2c910b2e authored by Ziqi Fan's avatar Ziqi Fan Committed by GitHub
Browse files

docs: update KVBM runbooks on metrics due to dynamo metrics change (#4266) 


Signed-off-by: default avatarZiqi Fan <ziqif@nvidia.com>
parent 744cda65
Show whitespace changes
Inline Side-by-side
Showing with 6 additions and 4 deletions
docs/kvbm/trtllm-setup.md
View file @ 2c910b2e
...@@ -135,11 +135,12 @@ export DYN_KVBM_DISK_ZEROFILL_FALLBACK=true ...@@ -135,11 +135,12 @@ export DYN_KVBM_DISK_ZEROFILL_FALLBACK=true
Follow below steps to enable metrics collection and view via Grafana dashboard: Follow below steps to enable metrics collection and view via Grafana dashboard:
```bash ```bash
# Start the basic services (etcd & natsd), along with Prometheus and Grafana # Start the basic services (etcd & natsd), along with Prometheus and Grafana
docker compose -f deploy/docker-compose.yml --profile metrics up -d docker compose -f deploy/docker-observability.yml up -d
# Set env var DYN_KVBM_METRICS to true, when launch via dynamo # Set env var DYN_KVBM_METRICS to true, when launch via dynamo
# Optionally set DYN_KVBM_METRICS_PORT to choose the /metrics port (default: 6880). # Optionally set DYN_KVBM_METRICS_PORT to choose the /metrics port (default: 6880).
DYN_KVBM_METRICS=true \ DYN_KVBM_METRICS=true \
DYN_KVBM_CPU_CACHE_GB=20 \
python3 -m dynamo.trtllm \ python3 -m dynamo.trtllm \
--model-path Qwen/Qwen3-0.6B \ --model-path Qwen/Qwen3-0.6B \
--served-model-name Qwen/Qwen3-0.6B \ --served-model-name Qwen/Qwen3-0.6B \
...@@ -149,7 +150,7 @@ python3 -m dynamo.trtllm \ ...@@ -149,7 +150,7 @@ python3 -m dynamo.trtllm \
sudo ufw allow 6880/tcp sudo ufw allow 6880/tcp
``` ```
View grafana metrics via http://localhost:3001 (default login: dynamo/dynamo) and look for KVBM Dashboard View grafana metrics via http://localhost:3000 (default login: dynamo/dynamo) and look for KVBM Dashboard
## Benchmark KVBM ## Benchmark KVBM
......
docs/kvbm/vllm-setup.md
View file @ 2c910b2e
...@@ -127,12 +127,13 @@ export DYN_KVBM_DISK_ZEROFILL_FALLBACK=true ...@@ -127,12 +127,13 @@ export DYN_KVBM_DISK_ZEROFILL_FALLBACK=true
Follow below steps to enable metrics collection and view via Grafana dashboard: Follow below steps to enable metrics collection and view via Grafana dashboard:
```bash ```bash
# Start the basic services (etcd & natsd), along with Prometheus and Grafana # Start the basic services (etcd & natsd), along with Prometheus and Grafana
docker compose -f deploy/docker-compose.yml --profile metrics up -d docker compose -f deploy/docker-observability.yml up -d
# Set env var DYN_KVBM_METRICS to true, when launch via dynamo # Set env var DYN_KVBM_METRICS to true, when launch via dynamo
# Optionally set DYN_KVBM_METRICS_PORT to choose the /metrics port (default: 6880). # Optionally set DYN_KVBM_METRICS_PORT to choose the /metrics port (default: 6880).
# NOTE: update launch/disagg_kvbm.sh or launch/disagg_kvbm_2p2d.sh as needed # NOTE: update launch/disagg_kvbm.sh or launch/disagg_kvbm_2p2d.sh as needed
DYN_KVBM_METRICS=true \ DYN_KVBM_METRICS=true \
DYN_KVBM_CPU_CACHE_GB=20 \
python -m dynamo.vllm \ python -m dynamo.vllm \
--model Qwen/Qwen3-0.6B \ --model Qwen/Qwen3-0.6B \
--enforce-eager \ --enforce-eager \
...@@ -142,7 +143,7 @@ python -m dynamo.vllm \ ...@@ -142,7 +143,7 @@ python -m dynamo.vllm \
sudo ufw allow 6880/tcp sudo ufw allow 6880/tcp
``` ```
View grafana metrics via http://localhost:3001 (default login: dynamo/dynamo) and look for KVBM Dashboard View grafana metrics via http://localhost:3000 (default login: dynamo/dynamo) and look for KVBM Dashboard
## Benchmark KVBM ## Benchmark KVBM
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment