docs(observability): explain Prometheus label warmup behavior (#8545)

Signed-off-by: Jie Hao <jihao@nvidia.com>

docs/observability/metrics.md

@@ -100,6 +100,8 @@ This hierarchical structure allows you to create metrics at the appropriate leve
 
 ## Available Metrics
 
+**Note:** Labeled metrics (`HistogramVec`, `CounterVec`, `GaugeVec`) register a metric *family*, not individual time series. A series for a given label combination only appears at `/metrics` after the first `with_label_values(...)` call for that combination — i.e., after the first matching request is served. For example, `dynamo_frontend_request_duration_seconds{model="Qwen/Qwen3-0.6B"}` will not appear on a freshly-started frontend until a request for that model is handled. This is expected Prometheus client behavior, not a missing metric.
+
 ### Backend Component Metrics
 
 **Backend workers** (`python -m dynamo.vllm`, `python -m dynamo.sglang`, etc.) expose `dynamo_component_*` metrics on the system status port (configurable via `DYN_SYSTEM_PORT`, disabled by default). In Kubernetes the operator typically sets `DYN_SYSTEM_PORT=9090`; for local development you must set it explicitly (e.g. `DYN_SYSTEM_PORT=8081`).

docs/observability/prometheus-grafana.md

@@ -58,6 +58,8 @@ After sending a few requests, the Prometheus Exposition Format text metrics are
 - Frontend: `http://localhost:8000/metrics`
 - Backend worker: `http://localhost:8081/metrics`
 
+**Note:** Labeled series (e.g., `...{model="..."}`) only appear after the first matching request is served. See [Available Metrics](metrics.md#available-metrics) for details.
+
 ### Access Web Interfaces
 
 Once Dynamo components are running: