Unverified Commit cb39311b authored by Tanmay Verma's avatar Tanmay Verma Committed by GitHub
Browse files

docs: Add docs of backend logging behavior (#6608)

parent a35df0f5
......@@ -20,6 +20,9 @@ enabled via the `DYN_LOGGING_SPAN_EVENTS` environment variable.
| `DYN_LOG` | Log levels per target `<default_level>,<module_path>=<level>,<module_path>=<level>` | `info` | `DYN_LOG=info,dynamo_runtime::system_status_server:trace` |
| `DYN_LOG_USE_LOCAL_TZ` | Use local timezone for timestamps (default is UTC) | `false` | `true` |
| `DYN_LOGGING_CONFIG_PATH` | Path to custom TOML logging configuration | none | `/path/to/config.toml` |
| `VLLM_LOGGING_LEVEL` | vLLM backend log level (independent of `DYN_LOG`) | `INFO` | `DEBUG` |
| `TLLM_LOG_LEVEL` | TensorRT-LLM backend log level (independent of `DYN_LOG`) | `INFO` | `DEBUG` |
| `DYN_SKIP_SGLANG_LOG_FORMATTING` | Disable Dynamo's SGLang log configuration | `false` | `true` |
| `OTEL_SERVICE_NAME` | Service name for trace and span information | `dynamo` | `dynamo-frontend` |
| `OTEL_EXPORT_ENABLED` | Enable OTLP trace exporting | `false` | `true` |
| `OTEL_EXPORTER_OTLP_TRACES_ENDPOINT` | OTLP exporter endpoint | `http://localhost:4317` | `http://tempo:4317` |
......@@ -259,6 +262,55 @@ Notice how the `x_request_id` field appears in all log entries, alongside the `t
## Backend Engine Log Levels
Dynamo's `DYN_LOG` environment variable controls Dynamo's own logging. Each
inference backend has its own log level control that is **independent** of
`DYN_LOG`.
### vLLM
vLLM log level is controlled by the `VLLM_LOGGING_LEVEL` environment variable.
It defaults to `INFO` and is completely independent of `DYN_LOG`.
```bash
# Set vLLM to debug while keeping Dynamo at info
export DYN_LOG=info
export VLLM_LOGGING_LEVEL=DEBUG
```
Valid values: `DEBUG`, `INFO`, `WARNING`, `ERROR`, `CRITICAL`.
### TensorRT-LLM
TensorRT-LLM log level is controlled by the `TLLM_LOG_LEVEL` environment
variable. It defaults to `INFO` and is completely independent of `DYN_LOG`.
```bash
# Set TRT-LLM to info while keeping Dynamo at warn
export DYN_LOG=warn
export TLLM_LOG_LEVEL=INFO
```
Valid values: `TRACE`, `DEBUG`, `INFO`, `WARNING`, `ERROR`, `INTERNAL_ERROR`.
**Note:** `TLLM_LOG_LEVEL` is read once at TensorRT-LLM import time. It must
be set before the process starts.
### SGLang
SGLang logging is currently configured through Dynamo and follows the
`DYN_LOG` level by default. To disable Dynamo's SGLang log configuration
and manage it independently, set:
```bash
export DYN_SKIP_SGLANG_LOG_FORMATTING=true
```
Alternatively, pass the `--log-level` argument to the SGLang worker
command to set the SGLang engine's log level directly (e.g.
`--log-level DEBUG`). This is independent of `DYN_LOG`.
## Related Documentation
- [Distributed Runtime Architecture](../design-docs/distributed-runtime.md)
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment