Unverified Commit ffe2d333 authored by Anant Sharma's avatar Anant Sharma Committed by GitHub
Browse files

docs: sync fern with observability doc changes (#5795)


Signed-off-by: default avatarAnant Sharma <anants@nvidia.com>
parent a2fbda3e
...@@ -128,141 +128,9 @@ let counter = endpoint.metrics().create_counter( ...@@ -128,141 +128,9 @@ let counter = endpoint.metrics().create_counter(
--- ---
## Metrics API in Python
Python components can create and manage Prometheus metrics using the same metrics API through Python bindings.
### Available Methods
- `endpoint.metrics.create_counter()` / `create_intcounter()`: Create a counter metric
- `endpoint.metrics.create_gauge()` / `create_intgauge()`: Create a gauge metric
- `endpoint.metrics.create_histogram()`: Create a histogram metric
- `endpoint.metrics.create_countervec()` / `create_intcountervec()`: Create a counter with labels
- `endpoint.metrics.create_gaugevec()` / `create_intgaugevec()`: Create a gauge with labels
- `endpoint.metrics.create_histogramvec()`: Create a histogram with labels
All metrics are imported from `dynamo.prometheus_metrics`.
### Creating Metrics
```python
from dynamo.runtime import DistributedRuntime
drt = DistributedRuntime()
endpoint = drt.namespace("my_namespace").component("my_component").endpoint("my_endpoint")
# Simple metrics
requests_total = endpoint.metrics.create_intcounter(
"requests_total",
"Total requests"
)
active_connections = endpoint.metrics.create_intgauge(
"active_connections",
"Active connections"
)
latency = endpoint.metrics.create_histogram(
"latency_seconds",
"Request latency",
buckets=[0.001, 0.01, 0.1, 1.0, 10.0]
)
```
### Using Metrics
```python
# Counters
requests_total.inc()
requests_total.inc_by(5)
# Gauges
active_connections.set(42)
active_connections.inc()
active_connections.dec()
# Histograms
latency.observe(0.023) # 23ms
```
### Vector Metrics with Labels
```python
# Create vector metrics with label names
requests_by_model = endpoint.metrics.create_intcountervec(
"requests_by_model",
"Requests by model",
["model_type", "model_size"]
)
memory_by_gpu = endpoint.metrics.create_intgaugevec(
"gpu_memory_bytes",
"GPU memory by device",
["gpu_id", "memory_type"]
)
# Use with specific label values
requests_by_model.inc({"model_type": "llama", "model_size": "7b"})
memory_by_gpu.set(8192, {"gpu_id": "0", "memory_type": "allocated"})
```
### Advanced Features
**Constant labels:**
```python
counter = endpoint.metrics.create_intcounter(
"requests_total",
"Total requests",
[("region", "us-west"), ("env", "prod")]
)
```
**Metric introspection:**
```python
print(counter.name()) # "my_namespace_my_component_my_endpoint_requests_total"
print(counter.const_labels()) # {"dynamo_namespace": "my_namespace", ...}
print(gauge_vec.variable_labels()) # ["model_type", "model_size"]
```
**Update patterns:**
Background thread updates:
```python
import threading
import time
def update_loop():
while True:
active_connections.set(compute_current_connections())
time.sleep(2)
threading.Thread(target=update_loop, daemon=True).start()
```
Callback-based updates (called before each `/metrics` scrape):
```python
def update_metrics():
active_connections.set(compute_current_connections())
endpoint.metrics.register_callback(update_metrics)
```
### Examples
Example scripts: [lib/bindings/python/examples/metrics/](https://github.com/ai-dynamo/dynamo/tree/main/lib/bindings/python/examples/metrics/)
```bash
cd ~/dynamo/lib/bindings/python/examples/metrics
DYN_SYSTEM_PORT=8081 ./server_with_loop.py
DYN_SYSTEM_PORT=8081 ./server_with_callback.py
```
---
## Related Documentation ## Related Documentation
- [Metrics Overview](metrics.md) - [Metrics Overview](metrics.md)
- [Prometheus and Grafana Setup](prometheus-grafana.md) - [Prometheus and Grafana Setup](prometheus-grafana.md)
- [Distributed Runtime Architecture](../design-docs/distributed-runtime.md) - [Distributed Runtime Architecture](../design-docs/distributed-runtime.md)
- [Python Metrics Examples](https://github.com/ai-dynamo/dynamo/tree/main/lib/bindings/python/examples/metrics/)
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment