refactor: move core logics of DPP -> AIC and support static profiling (#6285)

Signed-off-by: hongkuanz <hongkuanz@nvidia.com>
Signed-off-by: Hannah Zhang <hannahz@nvidia.com>
Co-authored-by: hhzhang16 <54051230+hhzhang16@users.noreply.github.com>

docs/pages/components/planner/planner-guide.md

@@ -6,475 +6,118 @@ title: Planner Guide
 
 # Planner Guide
 
-Deployment, configuration, and integration guide for the Dynamo SLA Planner. For a quick overview, see the [Planner README](README.md). For architecture internals, see [Planner Design](../../design-docs/planner-design.md).
+The Dynamo SLA Planner is an autoscaling controller that adjusts prefill and decode engine replica counts at runtime to meet latency SLAs. It reads traffic signals (Prometheus metrics or load predictor output) and engine performance profiles to decide when to scale up or down.
 
-## Scaling Modes
-
-The SLA Planner supports two scaling modes:
-
-- **Throughput-based scaling**: Uses pre-deployment profiling data and traffic prediction. Best for stable, predictable workloads where profiling data is available.
-- **Load-based scaling (Experimental)**: Uses real-time per-worker engine metrics and online regression. Best for bursty or unpredictable traffic. Does not require profiling data.
-
-**When to use which mode:**
-- Enable **throughput-based scaling** whenever engine profiling data is available. It provides stable, prediction-based capacity planning.
-- Enable **load-based scaling** when traffic is bursty or hard to predict. It reacts quickly to real-time load changes.
-- Enable **both modes together** for the best of both worlds: throughput-based scaling provides a lower bound (long-term capacity), while load-based scaling handles bursts above that floor. When both are enabled, use a longer `--adjustment-interval` for throughput-based scaling.
-
-**DGDR and scaling modes:** Deploying via DGDR automatically triggers profiling and enables throughput-based scaling. To additionally enable load-based scaling, pass the planner arguments through the DGDR's planner config section:
-
-```yaml
-profilingConfig:
-  config:
-    planner:
-      plannerEnableLoadbasedScaling: true
-      plannerLoadbasedAdjustmentInterval: 5
-```
-
-## Deployment
-
-### Prerequisites
-
-Before deploying the planner, ensure:
-
-- **Dynamo platform installed** with the operator running (see [Installation Guide](../../kubernetes/installation-guide.md))
-- **[kube-prometheus-stack](../../kubernetes/observability/metrics.md) installed and running** (required for SLA planner metric collection)
-- **Image pull secrets configured** if using private registries (typically `nvcr-imagepullsecret` for NVIDIA images)
-- **Sufficient GPU resources** available in your cluster for profiling
-- **Runtime images available** that contain both profiler and runtime components
-
-### Container Images
-
-Each DGDR requires container images for the profiling and deployment process:
-
-**profilingConfig.profilerImage** (Required):
-The container image used for the profiling job. Must contain the profiler code and dependencies for SLA-based profiling.
-
-**deploymentOverrides.workersImage** (Optional):
-The container image used for DGD worker components (frontend, workers, planner). Used for:
-- Temporary DGDs created during online profiling (for performance measurements)
-- The final DGD deployed after profiling completes
-
-If `workersImage` is omitted, the image from the base config file (e.g., `disagg.yaml`) is used. Public images are available from 0.6.1 onward.
-
-```yaml
-spec:
-  profilingConfig:
-    profilerImage: "nvcr.io/nvidia/ai-dynamo/vllm-runtime:0.6.1"
-  deploymentOverrides:
-    workersImage: "nvcr.io/nvidia/ai-dynamo/vllm-runtime:0.6.1"  # Optional
-```
-
-### What is a DynamoGraphDeploymentRequest (DGDR)?
-
-A **DGDR** is a Kubernetes Custom Resource that serves as the primary interface for deploying models with specific performance and resource constraints. It specifies:
-
-- **What** model to deploy (`model`)
-- **How** it should perform (SLA targets: `ttft`, `itl`)
-- **Where** it should run (optional GPU preferences)
-- **Which** backend to use (`backend`: sglang, trtllm, or vllm)
-- **Which** images to use (`profilingConfig.profilerImage`, `deploymentOverrides.workersImage`)
-
-The Dynamo Operator watches for DGDRs and automatically:
-1. Discovers available GPU resources in your cluster
-2. Runs profiling (online or offline) to find optimal configurations
-3. Generates an optimized DynamoGraphDeployment (DGD) configuration
-4. Deploys the DGD to your cluster
-
-**Key Benefits:**
-- **Declarative**: Specify what you want, not how to achieve it
-- **Automated**: No manual profiling job setup or result processing
-- **SLA-Driven**: Ensures deployments meet your performance requirements
-- **Integrated**: Works seamlessly with the Dynamo Operator
-
-### DGDR Workflow
-
-The DGDR workflow automates the entire process from SLA specification to deployment:
-
-1. **Define SLAs**: Specify performance requirements (TTFT, ITL) and model information
-2. **Automatic Profiling**: The operator profiles your model to find optimal configurations
-3. **Auto-Deploy**: The system deploys the optimal configuration that meets your SLAs
-
-```mermaid
-flowchart TD
-    A[Create DGDR] --> B[DGDR Controller]
-    B --> C{Profiling Method}
-    C -->|Online| D[Run Profiling Job<br/>2-4 hours]
-    C -->|Offline/AIC| E[AI Configurator<br/>20-30 seconds]
-    D --> F[Generate DGD Config]
-    E --> F
-    F --> G[Auto-Deploy DGD]
-    G --> H[Monitor & Scale]
-
-    style A fill:#e1f5fe
-    style D fill:#fff3e0
-    style E fill:#e8f5e8
-    style G fill:#f3e5f5
-    style H fill:#fff8e1
-```
-
-### Monitoring Progress
-
-Watch DGDR status:
-
-```bash
-# View status
-kubectl get dgdr -n $NAMESPACE
-
-# Detailed status
-kubectl describe dgdr sla-aic -n $NAMESPACE
-
-# Watch profiling job logs
-kubectl logs -f job/profile-sla-aic -n $NAMESPACE
-```
-
-**DGDR Status States:**
-- `Pending`: Initial state, preparing to profile
-- `Profiling`: Running profiling job (20-30 seconds for AIC, 2-4 hours for online)
-- `Deploying`: Generating and applying DGD configuration
-- `Ready`: DGD successfully deployed and running
-- `Failed`: Error occurred (check events for details)
-
-### Relationship to DGD
-
-- **DGDR**: High-level "intent" -- what you want deployed
-- **DGD**: Low-level "implementation" -- how it's deployed
-
-The DGDR controller generates a DGD that:
-- Uses optimal TP configurations from profiling
-- Includes the SLA planner for autoscaling
-- Has deployment and engine settings tuned for your SLAs
-
-The generated DGD is tracked via labels:
-```yaml
-metadata:
-  labels:
-    dgdr.nvidia.com/name: sla-aic
-    dgdr.nvidia.com/namespace: your-namespace
-```
-
-## Configuration
-
-### DGDR Configuration
-
-#### Required Fields
-
-| Field | Type | Description |
-|-------|------|-------------|
-| `spec.model` | string | Model identifier (e.g., `meta-llama/Llama-3-70b`) |
-| `spec.backend` | enum | Inference backend: `sglang`, `trtllm`, or `vllm` |
-| `spec.profilingConfig.profilerImage` | string | Container image for profiling job |
-| `spec.profilingConfig.config.sla` | object | SLA targets (isl, osl, ttft, itl) |
-
-#### Optional Fields
-
-| Field | Type | Description |
-|-------|------|-------------|
-| `spec.deploymentOverrides.workersImage` | string | Container image for DGD workers. If omitted, uses image from base config. |
-| `spec.autoApply` | boolean | Automatically deploy DGD after profiling (default: false) |
-| `spec.useMocker` | boolean | Deploy mocker instead of real backend (default: false) |
-| `spec.deploymentOverrides` | object | Customize metadata and image for auto-created DGD |
-
-#### SLA Configuration
-
-```yaml
-sla:
-  isl: 3000      # Average input sequence length (tokens)
-  osl: 150       # Average output sequence length (tokens)
-  ttft: 200      # Target Time To First Token (milliseconds, float)
-  itl: 20        # Target Inter-Token Latency (milliseconds, float)
-```
-
-**Choosing SLA Values:**
-- **ISL/OSL**: Based on your expected traffic patterns
-- **TTFT**: First token latency target (lower = more GPUs needed)
-- **ITL**: Token generation latency target (lower = more GPUs needed)
-- **Trade-offs**: Tighter SLAs require more GPU resources
-
-For comprehensive documentation of all configuration options, see the [DGDR Configuration Reference](../profiler/profiler-guide.md#dgdr-configuration-structure).
-
-### Profiling Methods
-
-Choose between **online profiling** (real measurements, 2-4 hours) or **offline profiling** with AI Configurator (estimated, 20-30 seconds):
-
-```yaml
-# Online Profiling (Default)
-sweep:
-  useAiConfigurator: false
-
-# Offline Profiling (AI Configurator)
-sweep:
-  useAiConfigurator: true
-  aicSystem: h200_sxm
-  aicHfId: Qwen/Qwen3-32B
-  aicBackendVersion: "0.20.0"
-```
-
-For detailed comparison, supported configurations, and limitations, see [SLA-Driven Profiling Documentation](../profiler/profiler-guide.md#profiling-method).
-
-### Load Predictors
+For a quick overview, see the [Planner README](README.md). For architecture internals, see [Planner Design](../../design-docs/planner-design.md).
 
-The throughput-based scaling mode forecasts the number of requests, ISL, and OSL in the next adjustment interval. Four prediction models are supported:
-
-#### Constant Predictor
-- **Use case**: Stable workloads with long prediction intervals
-- **Behavior**: Assumes next load equals current load
-- **Configuration**: `load-predictor: "constant"`
-
-#### ARIMA Predictor
-- **Use case**: Time-series data with trends and seasonality
-- **Behavior**: Uses auto-ARIMA to fit optimal model parameters
-- **Configuration**: `load-predictor: "arima"`
-- **Tunable parameters**:
-  - `--load-predictor-log1p`: model `log1p(y)` instead of `y`. If not set, ARIMA starts in raw space, and if it collapses to `(0,d,0)`, it falls back to `log1p` automatically.
-
-#### Kalman Predictor
-- **Use case**: Low-latency online forecasting (observe 1 -> predict 1) with smooth adaptation
-- **Behavior**: Local linear trend Kalman filter (fast online updates; good default when ARIMA collapses to mean-only)
-- **Configuration**: `load-predictor: "kalman"`
-- **Tunable parameters**:
-  - `--kalman-q-level`: process noise for level (higher = more responsive)
-  - `--kalman-q-trend`: process noise for trend (higher = trend changes faster)
-  - `--kalman-r`: measurement noise (lower = trusts new measurements more)
-  - `--kalman-min-points`: minimum points before forecasting
-  - `--load-predictor-log1p`: model `log1p(y)` instead of `y` (often helps request-rate/count series)
-
-#### Prophet Predictor
-- **Use case**: Complex seasonal patterns and trend changes
-- **Behavior**: Facebook's [Prophet](https://facebook.github.io/prophet/) model for time-series forecasting
-- **Configuration**: `load-predictor: "prophet"`
-- **Tunable parameters**:
-  - `--prophet-window-size`: bounds internal history to control refit cost
-  - `--load-predictor-log1p`: model `log1p(y)` instead of `y`
-
-#### Warm-starting Load Predictors (Optional)
-
-You can warm-start load predictors with a mooncake-style JSONL trace file:
+## Scaling Modes
 
-- **CLI argument**: `--load-predictor-warmup-trace <path/to/trace.jsonl>`
-- **Effect**: preloads predictors with historical request-count / ISL / OSL samples extracted from the trace
+The planner supports two scaling modes that can be used independently or together:
 
-### Throughput-Based Scaling Parameters
+- **Throughput-based scaling** (`enable_throughput_scaling: true`): Uses pre-deployment engine interpolation data and traffic prediction to plan capacity. Best for stable, predictable workloads. Requires profiling data generated by the [Profiler](../profiler/profiler-guide.md).
+- **Load-based scaling** (`enable_load_scaling: true`): Uses real-time per-worker engine metrics and online regression. Best for bursty or unpredictable traffic. Does not require profiling data.
 
-| Argument | Default | Description |
-|----------|---------|-------------|
-| `--adjustment-interval` | `180` | Seconds between scaling decisions |
-| `--ttft` | `500.0` | Target Time To First Token (ms) |
-| `--itl` | `50.0` | Target Inter-Token Latency (ms) |
-| `--max-gpu-budget` | `8` | Maximum GPUs across all workers |
-| `--min-endpoint` | `1` | Minimum replicas per worker type |
-| `--decode-engine-num-gpu` | `1` | GPUs per decode engine |
-| `--prefill-engine-num-gpu` | `1` | GPUs per prefill engine |
-| `--no-operation` | `false` | Observation mode (no actual scaling) |
-| `--no-correction` | `false` | Disable correction factors |
+**When to use which:**
 
-For the full list of arguments including load-based scaling options, see the [Planner README](README.md#key-arguments).
+- Enable **throughput-based scaling** whenever profiling data is available. It provides stable, prediction-based capacity planning.
+- Enable **load-based scaling** when traffic is bursty. It reacts quickly to real-time load changes.
+- Enable **both** for the best of both worlds: throughput-based provides a capacity floor, load-based handles bursts above it. When both are enabled, use a longer `throughput_adjustment_interval`.
 
-#### Planner Configuration Passthrough
+## PlannerConfig Reference
 
-Add planner-specific settings in the DGDR:
+The planner is configured via a `PlannerConfig` JSON/YAML object. When using the profiler, this is placed under the `features.planner` section of the DGDR spec:
 
 ```yaml
-profilingConfig:
-  config:
+features:
   planner:
-      plannerMinEndpoint: 2
+    enable_throughput_scaling: true
+    enable_load_scaling: false
+    pre_deployment_sweeping_mode: rapid
+    mode: disagg
+    backend: vllm
 ```
 
-## Integration
-
-### Prometheus Setup
-
-The planner queries Prometheus to collect frontend request metrics. The architecture:
-
-```mermaid
-flowchart LR
-  Frontend --"/metrics"--> Prometheus
-  Planner --"query API"--> Prometheus
-  Planner --"scaling decisions"--> Workers
-  Frontend -.->|"requests"| Workers
-```
-
-**Components:**
-- **Frontend**: Serves requests and exposes `/metrics`
-- **Prometheus**: Scrapes frontend metrics every 5s (configurable in podmonitor manifest)
-- **Planner**: Queries Prometheus and adjusts worker scaling every adjustment interval
-- **Workers**: Prefill and backend workers handle inference
-
-The planner requires a frontend that reports metrics at the `/metrics` HTTP endpoint with request count, ISL, OSL, TTFT, and ITL in the correct format. The Dynamo frontend provides these metrics automatically.
-
-**Prometheus endpoint configuration:**
+### Scaling Mode Fields
 
-| Variable | Default |
-|----------|---------|
-| `PROMETHEUS_ENDPOINT` | `http://prometheus-kube-prometheus-prometheus.monitoring.svc.cluster.local:9090` |
+| Field | Type | Default | Description |
+|-------|------|---------|-------------|
+| `enable_throughput_scaling` | bool | `true` | Enable throughput-based scaling (requires pre-deployment profiling data). |
+| `enable_load_scaling` | bool | `true` | Enable load-based scaling (no pre-deployment profiling data required). |
 
-If you see errors like "Failed to resolve prometheus service", ensure `PROMETHEUS_ENDPOINT` points to your Prometheus service.
+At least one scaling mode must be enabled.
 
-### Virtual Deployment
+### Pre-Deployment Sweeping
 
-The SLA planner supports virtual deployment mode for customized environments (e.g., custom orchestrators) through the `VirtualConnector`. This connector enables the planner to communicate scaling decisions without directly managing Kubernetes resources.
+| Field | Type | Default | Description |
+|-------|------|---------|-------------|
+| `pre_deployment_sweeping_mode` | string | `rapid` | How to generate engine interpolation data: `rapid` (AIC simulation, ~30s), `thorough` (real GPUs, 2-4h), or `none` (skip). |
 
-The `VirtualConnector` acts as a bridge between the SLA planner and external deployment environments. Instead of PATCHing DGD resources, it writes scaling decisions and waits for the external environment to acknowledge completion.
+When throughput-based scaling is enabled, the planner needs interpolation curves that map ISL to TTFT (prefill) and KV-cache utilization to ITL (decode). The profiler generates this data based on the `pre_deployment_sweeping_mode` setting. See the [Profiler Guide](../profiler/profiler-guide.md) for details on how this data is produced.
 
-#### Scaling Decision Flow
+### Throughput-Based Scaling Settings
 
-1. **Decision Generation**: The planner calculates optimal worker counts
-2. **Change Detection**: Skips scaling if target counts match current counts, logging: `"No scaling needed (prefill=X, decode=Y)"`
-3. **Readiness Check**: Verifies previous scaling operations completed by checking `scaled_decision_id >= decision_id`
-4. **Timeout Handling**: If not acknowledged within 30 minutes (1800 seconds), proceeds with new decisions
-5. **Completion Tracking**: Optionally waits for scaling completion confirmation (blocking mode)
+| Field | Type | Default | Description |
+|-------|------|---------|-------------|
+| `throughput_adjustment_interval` | int | `60` | Seconds between throughput-based scaling decisions. |
+| `min_endpoint` | int | `1` | Minimum number of engine endpoints to maintain. |
+| `max_gpu_budget` | int | `128` | Maximum total GPUs the planner may allocate. |
+| `ttft` | float | `2000.0` | TTFT SLA target (ms) for scaling decisions. |
+| `itl` | float | `30.0` | ITL SLA target (ms) for scaling decisions. |
+| `no_correction` | bool | `false` | Disable latency correction factor. Auto-disabled when load-based scaling is on. |
 
-#### Configuration
+### Load-Based Scaling Settings
 
-To use virtual deployment mode:
+| Field | Type | Default | Description |
+|-------|------|---------|-------------|
+| `load_adjustment_interval` | int | `10` | Seconds between load-based scaling decisions. Must be shorter than `throughput_adjustment_interval`. |
+| `load_learning_window` | int | `120` | Seconds of history used for online regression. |
+| `load_scaling_down_sensitivity` | int | `3` | Number of consecutive underutilized intervals before scaling down. |
+| `load_metric_samples` | int | `10` | Number of metric samples to collect per decision. |
+| `load_min_observations` | int | `5` | Minimum observations before making scaling decisions. |
+| `load_router_metrics_url` | string | `null` | Router metrics endpoint. Required outside Kubernetes mode. |
 
-```yaml
-environment: "virtual"
-backend: "vllm"  # or "sglang"
-```
-
-#### Deployment Environment Requirements
-
-The external deployment environment must use `VirtualConnectorClient`:
-
-```python
-from dynamo._core import DistributedRuntime, VirtualConnectorClient
-
-client = VirtualConnectorClient(distributed_runtime, namespace)
-```
-
-1. **Monitor Planner**: Continuously watch for scaling decisions: `await client.wait()` (blocks until change)
-2. **Parse Decisions**: Read values: `decision = await client.get()`
-3. **Execute Scaling**: Apply the scaling decisions to your infrastructure
-4. **Acknowledge Completion**: Mark done: `await client.complete(decision)`
-
-A scaling decision (returned by `client.get()`) contains:
-- `num_prefill_workers`: Target number of prefill workers (-1 if not set)
-- `num_decode_workers`: Target number of decode workers (-1 if not set)
-- `decision_id`: Incremental ID for each scaling decision
-
-See `components/planner/test/test_virtual_connector.py` for a full example.
-
-### Grafana Dashboard
-
-Deploy the planner Grafana dashboard:
-
-```bash
-kubectl apply -n monitoring -f deploy/observability/k8s/grafana-planner-dashboard-configmap.yaml
-```
+### General Settings
 
-Follow [Dynamo Metrics Collection on Kubernetes](../../kubernetes/observability/metrics.md) to access the Grafana UI and select the **Dynamo Planner Dashboard**.
+| Field | Type | Default | Description |
+|-------|------|---------|-------------|
+| `mode` | string | `disagg` | Planner mode: `disagg`, `prefill`, `decode`, or `agg`. |
+| `backend` | string | `vllm` | Backend: `vllm`, `sglang`, `trtllm`, or `mocker`. |
+| `environment` | string | `kubernetes` | Runtime environment: `kubernetes`, `virtual`, or `global-planner`. |
+| `namespace` | string | env `DYN_NAMESPACE` | Kubernetes namespace for the deployment. |
 
-The dashboard displays:
-- **Worker Counts & GPU Usage**: Current prefill/decode worker counts and cumulative GPU hours
-- **Observed Metrics**: Real-time TTFT, ITL, request rate, and sequence lengths from Prometheus
-- **Predicted Metrics**: Planner's load predictions and recommended replica counts
-- **Correction Factors**: How the planner adjusts predictions based on observed vs expected performance
+### Traffic Prediction Settings
 
-> Use the **Namespace** dropdown at the top of the dashboard to filter metrics for your deployment namespace.
+| Field | Type | Default | Description |
+|-------|------|---------|-------------|
+| `load_predictor` | string | `linear` | Prediction method: `linear`, `kalman`, or `prophet`. |
+| `load_predictor_log1p` | bool | `true` | Apply log1p transform to load data before prediction. |
+| `prophet_window_size` | int | `300` | Window size (seconds) for Prophet predictor. |
+| `load_predictor_warmup_trace` | string | `null` | Path to a warmup trace file for bootstrapping predictions. |
 
-## DGDR Immutability
+### Kalman Filter Settings
 
-DGDRs are **immutable**. To update SLAs or configuration:
+| Field | Type | Default | Description |
+|-------|------|---------|-------------|
+| `kalman_q_level` | float | `0.1` | Process noise for level component. |
+| `kalman_q_trend` | float | `0.01` | Process noise for trend component. |
+| `kalman_r` | float | `1.0` | Measurement noise. |
+| `kalman_min_points` | int | `10` | Minimum data points before Kalman predictions activate. |
 
-1. Delete the existing DGDR: `kubectl delete dgdr sla-aic`
-2. Create a new DGDR with updated specifications
+## Integration with Profiler
 
-## Manual Deployment Control
-
-### Option 1: Use DGDR-Generated Configuration (Recommended)
-
-Disable auto-deployment to review the generated DGD before applying:
-
-```yaml
-spec:
-  autoApply: false
-```
-
-Then manually extract and apply:
-
-```bash
-# Extract generated DGD from DGDR status
-kubectl get dgdr sla-aic -n $NAMESPACE -o jsonpath='{.status.generatedDeployment}' | kubectl apply -f -
-
-# Or save to file first for review/modification
-kubectl get dgdr sla-aic -n $NAMESPACE -o jsonpath='{.status.generatedDeployment}' > my-dgd.yaml
-vi my-dgd.yaml
-kubectl apply -f my-dgd.yaml -n $NAMESPACE
-```
-
-### Option 2: Use Standalone Planner Templates (Advanced)
-
-For advanced use cases, use the standalone planner templates in `examples/backends/*/deploy/disagg_planner.yaml`:
-
-```bash
-# After profiling completes, profiling data is stored in ConfigMaps
-kubectl get configmap dgdr-output-<dgdr-name> -n $NAMESPACE -o yaml
-kubectl get configmap planner-profile-data -n $NAMESPACE -o yaml
-
-# Update PROMETHEUS_ENDPOINT in the template, then deploy
-kubectl apply -f examples/backends/<backend>/deploy/disagg_planner.yaml -n $NAMESPACE
-```
-
-## Accessing Profiling Artifacts
-
-By default, profiling jobs save essential data to ConfigMaps. For detailed artifacts, configure the DGDR to use `dynamo-pvc`:
-
-**ConfigMaps (always created):**
-- Generated DGD configuration
-- Profiling data for Planner (`.json` files)
-
-**PVC (optional):**
-- Performance plots (PNGs)
-- DGD configuration and logs for each profiled deployment
-- AIPerf profiling artifacts
-- Raw profiling data (`.npz` files)
-- Profiler log
-
-```bash
-# Setup PVC
-deploy/utils/setup_benchmarking_resources.sh
-
-# Access results after profiling
-kubectl apply -f deploy/utils/manifests/pvc-access-pod.yaml -n $NAMESPACE
-kubectl wait --for=condition=Ready pod/pvc-access-pod -n $NAMESPACE --timeout=60s
-kubectl cp $NAMESPACE/pvc-access-pod:/data ./profiling-results
-kubectl delete pod pvc-access-pod -n $NAMESPACE
-```
-
-## Troubleshooting
-
-### Quick Diagnostics
-
-```bash
-# Check DGDR status and events
-kubectl describe dgdr sla-aic -n $NAMESPACE
-
-# Check operator logs
-kubectl logs -n $NAMESPACE -l app.kubernetes.io/name=dynamo-operator --tail=100
-
-# Check profiling job logs
-kubectl logs -l job-name=profile-sla-aic -n $NAMESPACE
-```
+When the profiler runs with planner enabled, it:
 
-### Common Issues
+1. Selects the best prefill and decode engine configurations
+2. Generates interpolation curves (TTFT vs ISL, ITL vs KV-cache utilization)
+3. Saves the `PlannerConfig` and profiling data into separate Kubernetes ConfigMaps
+4. Adds the planner service to the generated DGD, configured to read from those ConfigMaps
 
-| Issue | Quick Fix |
-|-------|-----------|
-| **DGDR stuck in Pending** | Check GPU availability: `kubectl get nodes -o jsonpath='{.items[*].status.allocatable.nvidia\.com/gpu}'` |
-| **Image pull errors** | Verify secret exists: `kubectl get secret nvcr-imagepullsecret -n $NAMESPACE` |
-| **Profiling fails** | Check job logs: `kubectl logs -l job-name=profile-sla-aic -n $NAMESPACE` |
-| **SLA cannot be met** | Relax TTFT/ITL targets or add more GPUs |
-| **DGD not deployed** | Verify `autoApply: true` in DGDR spec |
-| **Prometheus errors** | Ensure `PROMETHEUS_ENDPOINT` env var points to your Prometheus service |
+The planner receives its config via `--config /path/to/planner_config.json` which is mounted from the `planner-config-XXXX` ConfigMap. Profiling data is mounted from the `planner-profile-data-XXXX` ConfigMap.
 
-For comprehensive troubleshooting including AI Configurator constraints, performance debugging, and backend-specific issues, see [SLA-Driven Profiling Troubleshooting](../profiler/profiler-guide.md#troubleshooting).
+See the [Profiler Guide](../profiler/profiler-guide.md) for the full profiling workflow and how to configure pre-deployment sweeping.
 
-## Related Documentation
+## See Also
 
-- [Planner README](README.md) -- Overview and quick start
-- [Planner Examples](planner-examples.md) -- DGDR YAML examples and sample configurations
-- [Planner Design](../../design-docs/planner-design.md) -- Architecture deep-dive for contributors
-- [DGDR API Reference](../../kubernetes/api-reference.md)
-- [Pre-Deployment Profiling](../profiler/profiler-guide.md)
-- [Dynamo Operator Guide](../../kubernetes/dynamo-operator.md)
+- [Planner README](README.md) — Quick overview
+- [Planner Design](../../design-docs/planner-design.md) — Architecture internals
+- [Profiler Guide](../profiler/profiler-guide.md) — How profiling data is generated

docs/pages/components/profiler/profiler-guide.md

@@ -6,664 +6,213 @@ title: Profiler Guide
 
 # Profiler Guide
 
-This guide covers deployment, configuration, integration, and troubleshooting for the Dynamo Profiler.
+## Overview
 
-## What is a DynamoGraphDeploymentRequest (DGDR)?
+The Dynamo Profiler analyzes model inference performance and generates optimized deployment configurations (DynamoGraphDeployments). Given a model, hardware, and SLA targets, it determines the best parallelization strategy, selects optimal prefill and decode engine configurations, and produces a ready-to-deploy DGD YAML.
 
-A **DynamoGraphDeploymentRequest (DGDR)** is a Kubernetes Custom Resource that serves as the primary interface for users to request model deployments with specific performance and resource constraints. You specify:
+The profiler accepts a `DynamoGraphDeploymentRequestSpec` (DGDR) as input and uses [AI Configurator (AIC)](https://github.com/ai-dynamo/aiconfigurator) for performance simulation, candidate enumeration, and configuration picking. When the planner is enabled, the profiler additionally generates engine interpolation curves used for runtime autoscaling.
 
-- **What** model you want to deploy (`model`)
-- **How** it should perform (SLA targets: `ttft`, `itl`)
-- **Where** it should run (optional GPU preferences)
-- **Which** backend to use (`backend`: sglang, trtllm, or vllm)
-- **Which** images to use (`profilingConfig.profilerImage`, `deploymentOverrides.workersImage`)
+## Workflow
 
-The Dynamo Operator watches for DGDRs and automatically:
-1. Discovers available GPU resources in your cluster
-2. Runs profiling (online or offline) to find optimal configurations
-3. Generates an optimized DynamoGraphDeployment (DGD) configuration
-4. Deploys the DGD to your cluster
+The profiler follows this pipeline:
 
-**Relationship to DGD:**
-- **DGDR**: High-level "intent" - what you want deployed
-- **DGD**: Low-level "implementation" - how it's deployed
+```mermaid
+flowchart TD
+    Input["DGDR Spec"] --> Validate["Validate + Gate Checks"]
+    Validate --> Strategy{searchStrategy?}
 
-## Support Matrix
-
-| Backend | Dense Models | MoE Models |
-|---------|-------------|------------|
-| vLLM | ✅ | 🚧 |
-| SGLang | ✅ | ✅ |
-| TensorRT-LLM | ✅ | 🚧 |
-
-The profiler sweeps over the following parallelization mappings for prefill and decode:
-
-| Model Architecture | Prefill Parallelization Mapping | Decode Parallelization Mapping |
-|---------|-------------|------------|
-| MLA+MoE (DeepseekV3ForCausalLM, DeepseekV32ForCausalLM) | TEP, DEP | TEP, DEP |
-| GQA+MoE (Qwen3MoeForCausalLM) | TP, TEP, DEP | TP, TEP, DEP |
-| Other Models | TP | TP |
-
-> [!NOTE]
-> Exact model x parallelization mapping support is dependent on the backend. The profiler does not guarantee that the recommended P/D engine configuration is supported and bug-free by the backend.
+    Strategy -->|rapid| AICCheck{"AIC supports\nmodel/hw/backend?"}
+    Strategy -->|thorough| Enumerate["Enumerate candidates\nvia AIC"]
 
-## Deployment
+    AICCheck -->|yes| Simulate["AIC Simulation\n+ Picking"]
+    AICCheck -->|no| Naive["Naive Config\nGeneration"]
 
-### Kubernetes Deployment (DGDR)
+    Enumerate --> Deploy["Deploy + Benchmark\neach candidate"]
+    Deploy --> Pick["AIC Picking"]
 
-The recommended deployment method is through DGDRs. Sample configurations are provided in `components/src/dynamo/profiler/deploy/`:
+    Simulate --> DGDGen["DGD Generation"]
+    Pick --> DGDGen
+    Naive --> DGDGen
 
-| Sample | Description |
-|--------|-------------|
-| `profile_sla_dgdr.yaml` | Standard online profiling with AIPerf |
-| `profile_sla_aic_dgdr.yaml` | Fast offline profiling with AI Configurator |
-| `profile_sla_moe_dgdr.yaml` | MoE model profiling (SGLang) |
+    DGDGen --> PlannerCheck{"Planner\nenabled?"}
+    PlannerCheck -->|yes| Interpolation["Interpolation\nCurves"]
+    PlannerCheck -->|no| MockerCheck
 
-#### Container Images
+    Interpolation --> AddPlanner["Add Planner\nService + ConfigMaps"]
+    AddPlanner --> MockerCheck{"Mocker\nenabled?"}
 
-Each DGDR requires container images for profiling and deployment:
+    MockerCheck -->|yes| Mocker["Output Mocker DGD"]
+    MockerCheck -->|no| RealDGD["Output Real DGD"]
 
-- **`profilingConfig.profilerImage`** (Required): Container image for the profiling job. Must contain the profiler code and dependencies.
-- **`deploymentOverrides.workersImage`** (Optional): Container image for DGD worker components (frontend, workers, planner). If omitted, uses image from the base config file.
-
-```yaml
-spec:
-  profilingConfig:
-    profilerImage: "nvcr.io/nvidia/ai-dynamo/vllm-runtime:0.9.0"
-  deploymentOverrides:
-    workersImage: "nvcr.io/nvidia/ai-dynamo/vllm-runtime:0.9.0"
+    Mocker --> Final["final_config.yaml"]
+    RealDGD --> Final
 ```
 
-#### Quick Start: Deploy with DGDR
-
-**Step 1: Create Your DGDR**
+### Stage-by-stage walkthrough
 
-Use a sample configuration or create your own:
-
-```yaml
-apiVersion: nvidia.com/v1alpha1
-kind: DynamoGraphDeploymentRequest
-metadata:
-  name: my-model-profiling
-spec:
-  model: "Qwen/Qwen3-0.6B"
-  backend: vllm
-  profilingConfig:
-    profilerImage: "nvcr.io/nvidia/ai-dynamo/vllm-runtime:0.9.0"
-    config:
-      sla:
-        isl: 3000
-        osl: 150
-        ttft: 200.0
-        itl: 20.0
-  deploymentOverrides:
-    workersImage: "nvcr.io/nvidia/ai-dynamo/vllm-runtime:0.9.0"
-  autoApply: true
-```
+1. **Validation**: The DGDR spec is validated — required fields checked (`image`, `hardware.gpuSku`, `hardware.numGpusPerNode`), SLA targets verified, and gate checks applied (see [Gate Checks](#gate-checks-and-constraints)).
 
-**Step 2: Apply the DGDR**
+2. **Search Strategy**: The profiler branches based on `searchStrategy`:
+   - **Rapid**: Uses AIC simulation to estimate performance across parallelization configs. No GPUs needed, completes in ~30 seconds.
+   - **Thorough**: Enumerates candidate parallelization configs via AIC, deploys each on real GPUs, benchmarks with AIPerf, then picks the best. Takes 2-4 hours, disagg mode only.
 
-```bash
-export NAMESPACE=your-namespace
-kubectl apply -f my-profiling-dgdr.yaml -n $NAMESPACE
-```
+3. **Picking**: The profiler selects the best configuration using one of three modes, determined automatically from the DGDR spec (see [Picking Modes](#picking-modes)).
 
-**Step 3: Monitor Progress**
+4. **DGD Generation**: The picked configuration is rendered into a complete DGD YAML via AIC's generator pipeline, including correct parallelization, replica counts, container image, and PVC mounts.
 
-```bash
-# View status
-kubectl get dgdr -n $NAMESPACE
+5. **Interpolation** (planner only): When the planner is enabled, the profiler generates detailed performance interpolation curves — TTFT vs ISL for prefill, ITL vs KV-cache utilization for decode. These are saved into ConfigMaps for the planner to use at runtime.
 
-# Detailed status
-kubectl describe dgdr my-model-profiling -n $NAMESPACE
+6. **Final Assembly**: The planner service is added to the DGD if enabled. If mocker is enabled, the mocker DGD is used instead of real workers. The result is written to `final_config.yaml`.
 
-# Watch profiling job logs
-kubectl logs -f job/profile-my-model-profiling -n $NAMESPACE
-```
+## Search Strategies
 
-**DGDR Status States:**
-- `Pending`: Initial state, preparing to profile
-- `Profiling`: Running profiling job (20-30 seconds for AIC, 2-4 hours for online)
-- `Deploying`: Generating and applying DGD configuration
-- `Ready`: DGD successfully deployed and running
-- `Failed`: Error occurred (check events for details)
+### Rapid
 
-**Step 4: Access Your Deployment**
-
-```bash
-# Find the frontend service
-kubectl get svc -n $NAMESPACE | grep frontend
-
-# Port-forward to access locally
-kubectl port-forward svc/<deployment>-frontend 8000:8000 -n $NAMESPACE
-
-# Test the endpoint
-curl http://localhost:8000/v1/models
-```
-
-> [!NOTE]
-> DGDRs are **immutable**. To update SLAs or configuration, delete the existing DGDR and create a new one.
-
-### Direct Script Execution
-
-For advanced use cases or local development:
-
-```bash
-python -m dynamo.profiler.profile_sla \
-  --backend vllm \
-  --config path/to/disagg.yaml \
-  --model meta-llama/Llama-3-8B \
-  --ttft 200 --itl 15 \
-  --isl 3000 --osl 150 \
-  --min-num-gpus 1 \
-  --max-num-gpus 8
-```
-
-## Profiling Method
-
-The profiler follows a 5-step process:
-
-1. **Hardware Setup**: Uses defaults or user-specified hardware configuration. Optionally, cluster-scoped operators can enable automatic GPU discovery to detect specifications from cluster nodes.
-2. **Identify Sweep Ranges**: Automatically determine minimum and maximum number of GPUs per engine. Minimum is determined by the model size and GPU VRAM. Maximum is set to one node for dense models and 4 nodes for MoE models.
-3. **Parallelization Mapping Sweep**: Test performance of engines with different parallelization mappings using the input ISL and OSL.
-   - For dense models, test different TP sizes for both prefill and decode.
-   - For MoE models (SGLang), evaluate both TEP and DEP as candidates for prefill and decode.
-   - **Prefill**:
-     - TP/TEP: Measure TTFT with batch size = 1 (assuming ISL is long enough to saturate compute) without KV reuse.
-     - DEP: Attention uses data parallelism. Send a single burst with total concurrency `attention_dp_size × attn_dp_num_req_ratio` (defaults to 4) and compute the reported TTFT as `time_to_first_token.max / attn_dp_num_req_ratio` from the AIPerf summary of that burst.
-   ![Prefill Performance](../../../assets/img/h100-prefill-performance.png)
-   - **Decode**: Measure the ITL under different numbers of in-flight requests, from 1 to the maximum the KV cache can hold. To measure ITL without being affected by piggy-backed prefill requests, the script enables KV-reuse and warms up the engine by issuing the same prompts before measuring.
-   ![Decode Performance](../../../assets/img/h100-decode-performance.png)
-4. **Recommendation**: Select optimal parallelization mapping for prefill and decode that achieves the highest per-GPU throughput while adhering to the SLA on TTFT and ITL.
-5. **In-Depth Profiling on the Recommended P/D Engine**: Interpolate TTFT with ISL and ITL with active KV cache and decode context length for more accurate performance estimation.
-![ITL Interpolation](../../../assets/img/pd-interpolation.png)
-   - **Prefill**: Measures TTFT and throughput per GPU across different input lengths with batch size=1.
-   - **Decode**: Measures ITL and throughput per GPU under various KV cache loads and decode context lengths.
-
-### AIPerf on Real Engines
-
-Profiles your model by creating real test deployments in Kubernetes and measuring their performance.
-
-- **Duration**: 2-4 hours
-- **Accuracy**: Highest (real measurements)
-- **GPU Requirements**: Full access to test different parallelization mappings
-- **Backends**: SGLang, TensorRT-LLM, vLLM
+Uses AIC's performance simulation to estimate optimal configurations without deploying real engines. Completes in ~30 seconds.
 
 ```yaml
-profilingConfig:
-  config:
-    sweep:
-      useAiConfigurator: false  # Default
+searchStrategy: rapid
 ```
 
-### AI Configurator Simulation
+- Supports all backends: vLLM, SGLang, TensorRT-LLM
+- If the model/hardware/backend combination is not supported by AIC, falls back to a naive config (memory-fit TP calculation)
+- No GPU resources consumed during profiling
 
-Uses performance simulation to rapidly estimate optimal configurations without running real deployments.
+### Thorough
 
-- **Duration**: 20-30 seconds
-- **Accuracy**: Estimated (may have errors for unusual configurations)
-- **GPU Requirements**: None
-- **Backends**: TensorRT-LLM only (SGLang/vLLM coming soon)
+Enumerates candidate parallelization configs, deploys each as a real K8s workload, and benchmarks with AIPerf.
 
 ```yaml
-profilingConfig:
-  config:
-    sweep:
-      useAiConfigurator: true
-      aicSystem: h200_sxm
-      aicHfId: Qwen/Qwen3-32B
-      aicBackendVersion: "0.20.0"      # TRT-LLM version simulated by AIC
+searchStrategy: thorough
 ```
 
-> [!NOTE]
-> `aicBackendVersion` specifies the TensorRT-LLM version that AI Configurator simulates. See the [AI Configurator supported features](https://github.com/ai-dynamo/aiconfigurator#supported-features) for available versions.
+- Only disaggregated mode is supported
+- Does not support `auto` backend — specify `vllm`, `sglang`, or `trtllm`
+- Takes 2-4 hours depending on the number of candidates
+- Provides highest accuracy since measurements come from real hardware
 
-**Currently supports:**
-- **Backends**: TensorRT-LLM (versions 0.20.0, 1.0.0rc3, 1.0.0rc6)
-- **Systems**: H100 SXM, H200 SXM, B200 SXM, GB200 SXM, A100 SXM
-- **Models**: Wide range including GPT, Llama, Mixtral, DeepSeek, Qwen, and more
+## Picking Modes
 
-See [AI Configurator documentation](https://github.com/ai-dynamo/aiconfigurator#supported-features) for the full list.
+The profiler automatically selects a picking mode based on the DGDR spec:
 
-### Automatic GPU Discovery
+### Autoscale
 
-The operator automatically discovers GPU resources from cluster nodes, providing hardware info (GPU model, VRAM, GPUs per node) and automatic profiling search space calculation.
+Triggered when the **planner is enabled** (scaling enabled in `features.planner`). Picks prefill and decode engines independently, each with 1 replica. The planner handles scaling at runtime.
 
-**Requirements:**
-- **Cluster-scoped operators**: Have node read permissions by default
-- **Namespace-scoped operators**: GPU discovery is enabled by default when installing via Helm — the chart provisions the required ClusterRole/ClusterRoleBinding automatically
+### Load Match
 
-**For namespace-scoped operators**, GPU discovery is controlled by a Helm value:
-
-```bash
-# GPU discovery enabled (default) — Helm provisions read-only node access automatically
-helm install dynamo-platform ... --set dynamo-operator.gpuDiscovery.enabled=true
-
-# GPU discovery disabled — you must provide hardware config manually in each DGDR
-helm install dynamo-platform ... --set dynamo-operator.gpuDiscovery.enabled=false
-```
-
-If GPU discovery is disabled, provide hardware config manually in the DGDR:
-
-```yaml
-spec:
-  profilingConfig:
-    config:
-      hardware:
-        numGpusPerNode: 8
-        gpuModel: "H100-SXM5-80GB"
-        gpuVramMib: 81920
-```
-
-If GPU discovery is disabled and no manual hardware config is provided, the DGDR will be rejected at admission time.
-
-## Configuration
-
-### DGDR Configuration Structure
-
-All profiler configuration goes under `spec.profilingConfig.config`:
-
-```yaml
-apiVersion: nvidia.com/v1alpha1
-kind: DynamoGraphDeploymentRequest
-metadata:
-  name: my-deployment
-spec:
-  model: "Qwen/Qwen3-0.6B"
-  backend: vllm
-
-  profilingConfig:
-    profilerImage: "nvcr.io/nvidia/ai-dynamo/vllm-runtime:0.9.0"
-    configMapRef:                  # Optional: base DGD config
-      name: my-config
-      key: disagg.yaml
-
-    config:
-      sla: { ... }
-      hardware: { ... }
-      sweep: { ... }
-      planner: { ... }
-
-  deploymentOverrides:
-    workersImage: "nvcr.io/nvidia/ai-dynamo/vllm-runtime:0.9.0"
-```
-
-### SLA Configuration (Required)
-
-```yaml
-sla:
-  isl: 3000      # Average input sequence length (tokens)
-  osl: 150       # Average output sequence length (tokens)
-  ttft: 200.0    # Target Time To First Token (milliseconds)
-  itl: 20.0      # Target Inter-Token Latency (milliseconds)
-```
-
-- **ISL/OSL**: Based on your expected traffic patterns
-- **TTFT**: First token latency target (lower = more GPUs needed, affects prefill engine)
-- **ITL**: Token generation latency target (lower = more GPUs needed, affects decode engine)
-- **Trade-offs**: Tighter SLAs require more GPU resources
-
-### Hardware Configuration (Optional)
-
-```yaml
-hardware:
-  minNumGpusPerEngine: 2      # Auto-determined from model size and VRAM if not provided
-  maxNumGpusPerEngine: 8      # Maximum GPUs to test
-  numGpusPerNode: 8           # GPUs per node (for multi-node MoE)
-  gpuType: h200_sxm           # GPU type hint (informational, auto-detected)
-```
-
-- **minNumGpusPerEngine**: Skip small TP sizes if your model is large
-- **maxNumGpusPerEngine**: Limit search space or work around constraints (e.g., [AIC attention heads](#ai-configurator-attention-head-constraint-error))
-- **numGpusPerNode**: Determine the upper bound of GPUs per node for dense models and configure Grove for multi-node MoE engines
-- **gpuType**: Informational only, auto-detected by the controller. For AI Configurator, use `aicSystem` in the [sweep configuration](#ai-configurator-configuration) instead
-
-> [!TIP]
-> If you don't specify hardware constraints, the controller auto-detects based on your model size and available cluster resources.
-
-### Sweep Configuration (Optional)
+Triggered when a **target load** is specified (`workload.requestRate` or `workload.concurrency`). Finds the configuration that serves the target load with the minimum number of GPUs under SLA.
 
 ```yaml
-sweep:
-  useAiConfigurator: false              # Use real profiling (default)
-  prefillInterpolationGranularity: 16   # Samples for prefill TTFT curve
-  decodeInterpolationGranularity: 6     # Samples for decode ITL curve
+workload:
+  requestRate: 5.0   # target 5 req/s
 ```
 
-- **useAiConfigurator**: Set to `true` for 20-30 second profiling (TensorRT-LLM only)
-- **prefillInterpolationGranularity**: Samples for prefill TTFT curve (lower = faster but less accurate)
-- **decodeInterpolationGranularity**: Samples for decode ITL curve. Since ITL interpolation is 3D and takes longer, we default to fewer samples. Increasing this value may quadratically increase profiling time.
-
-### AI Configurator Configuration
+### Default
 
-Required if `useAiConfigurator: true`:
+Triggered when there is **no planner and no target load**. Maximizes throughput for the available GPU budget under SLA.
 
-```yaml
-sweep:
-  useAiConfigurator: true
-  aicSystem: h200_sxm              # h100_sxm, h200_sxm, b200_sxm, gb200_sxm, a100_sxm
-  aicHfId: Qwen/Qwen3-32B         # HuggingFace model ID
-  aicBackendVersion: "0.20.0"      # TensorRT-LLM version
-```
-
-### Planner Configuration (Optional)
+## Planner Integration
 
-Pass arguments to the SLA planner:
+When the planner is enabled, the profiler generates engine interpolation data needed for throughput-based autoscaling. The `pre_deployment_sweeping_mode` field controls how this data is produced:
 
 ```yaml
-planner:
-  planner_min_endpoint: 2                    # Minimum endpoints to maintain
-  planner_adjustment_interval: 60            # Adjustment interval (seconds)
-  planner_load_predictor: linear             # Load prediction method
+features:
+  planner:
+    pre_deployment_sweeping_mode: rapid   # rapid | thorough | none
+    enable_throughput_scaling: true
 ```
 
-> [!NOTE]
-> Planner arguments use `planner_` prefix. See [SLA Planner documentation](../planner/planner-guide.md) for full list.
+- **rapid**: Uses AIC simulation to generate interpolation curves (~30s, no GPUs)
+- **thorough**: Deploys the selected engine config on real GPUs and sweeps across ISL/concurrency ranges (2-4h)
+- **none**: Skips interpolation. Only valid when using load-based scaling without throughput-based scaling.
 
-### Model Cache PVC (Advanced)
+The profiler saves two ConfigMaps into the generated DGD:
+- **planner-config-XXXX**: Serialized `PlannerConfig` JSON (with `profile_results_dir` pointing to the profiling data mount)
+- **planner-profile-data-XXXX**: Prefill and decode interpolation data (JSON)
 
-For large models, use a pre-populated PVC containing model weights instead of downloading from HuggingFace:
+See the [Planner Guide](../planner/planner-guide.md) for the full `PlannerConfig` reference.
 
-```yaml
-deployment:
-  modelCache:
-    pvcName: "model-cache"
-    pvcPath: "hub/models--deepseek-ai--DeepSeek-R1"
-    mountPath: "/opt/model-cache"
-```
+## Mocker
 
-Requirements:
-- The PVC must exist in the same namespace as the DGDR
-- The model weights must be accessible at `{mountPath}/{pvcPath}`
+When `features.mocker.enabled: true`, the profiler outputs a mocker DGD that simulates engine behavior without real GPUs. This is useful for testing planner behavior and validating configurations at scale.
 
-### Engine Configuration (Auto-configured)
+Mocker requires pre-deployment sweeping to generate simulated performance profiles — `pre_deployment_sweeping_mode` cannot be `none` when mocker is enabled.
 
-The controller automatically injects these from high-level fields:
+## Gate Checks and Constraints
 
-```yaml
-# You specify:
-spec:
-  model: "Qwen/Qwen3-0.6B"
-  backend: vllm
-
-# Controller auto-injects:
-profilingConfig:
-  config:
-    deployment:
-      model: "Qwen/Qwen3-0.6B"
-    engine:
-      backend: vllm
-      config: /path/to/configmap
-```
+The profiler enforces these rules at startup:
 
-You should **not** manually set `deployment.model` or `engine.backend` in `profilingConfig.config`.
+| Condition | Behavior |
+|-----------|----------|
+| `searchStrategy: thorough` + `backend: auto` | Rejected. Specify a concrete backend. |
+| AIC unsupported + `enable_throughput_scaling: true` | Rejected. Throughput planner requires AIC support. |
+| AIC unsupported + `pre_deployment_sweeping_mode: rapid` | Falls back to `none` with a warning. |
+| `e2eLatency` provided without `ttft: null, itl: null` | Rejected by SLA validator. When using `e2eLatency`, explicitly null out `ttft` and `itl`. |
+| SLA unachievable | Warning logged, SLA updated to best achievable value. |
+| Load-match needs more GPUs than available | Warning logged. |
 
-### Using Existing DGD Configs (ConfigMap)
+## CLI Usage
 
-Reference an existing DGD config via ConfigMap:
+The profiler can be run directly for local development and testing:
 
 ```bash
-kubectl create configmap my-config \
-  --from-file=disagg.yaml=/path/to/your/disagg.yaml \
-  --namespace $NAMESPACE \
-  --dry-run=client -o yaml | kubectl apply -f -
-```
-
-```yaml
-profilingConfig:
-  configMapRef:
-    name: my-config
-    key: disagg.yaml
+python -m dynamo.profiler --config <spec.yaml>
 ```
 
-The profiler uses the DGD config as a **base template**, then optimizes it based on your SLA targets.
-
-### CLI Arguments
-
-| Argument | Type | Default | Description |
-|----------|------|---------|-------------|
-| `--backend` | string | - | Inference backend: sglang, trtllm, vllm |
-| `--config` | string | - | Path to DGD YAML config file |
-| `--model` | string | - | HuggingFace model ID |
-| `--ttft` | float | - | Target TTFT in milliseconds |
-| `--itl` | float | - | Target ITL in milliseconds |
-| `--isl` | int | - | Average input sequence length |
-| `--osl` | int | - | Average output sequence length |
-| `--min-num-gpus` | int | auto | Minimum GPUs per engine |
-| `--max-num-gpus` | int | 8 | Maximum GPUs per engine |
-| `--use-ai-configurator` | flag | false | Use offline AI Configurator |
-| `--pick-with-webui` | flag | false | Launch interactive WebUI |
-| `--webui-port` | int | 8000 | Port for WebUI |
-
-> [!NOTE]
-> CLI arguments map to DGDR config fields: `--min-num-gpus` = `hardware.minNumGpusPerEngine`, `--max-num-gpus` = `hardware.maxNumGpusPerEngine`, `--use-ai-configurator` = `sweep.useAiConfigurator`. See [DGDR Configuration Structure](#dgdr-configuration-structure) for all field mappings.
-
-## Integration
-
-### With SLA Planner
+Where `<spec.yaml>` is a DGDR spec (JSON or YAML file, or inline JSON string).
 
-The Profiler generates interpolation data that the SLA Planner uses for autoscaling decisions.
+### Operational flags
 
-**Prefill Interpolation** (`selected_prefill_interpolation/raw_data.npz`):
-- `prefill_isl`: 1D array of input sequence lengths tested
-- `prefill_ttft`: 1D array of TTFTs (ms) at each ISL
-- `prefill_thpt_per_gpu`: 1D array of throughput (tokens/s/GPU) at each ISL
+| Flag | Default | Description |
+|------|---------|-------------|
+| `--output-dir` | `profiling_results` | Directory for output files |
+| `--deployment-timeout` | `3600` | Max seconds to wait for K8s deployment readiness |
+| `--prefill-interpolation-granularity` | `16` | Number of ISL samples for prefill interpolation |
+| `--decode-interpolation-granularity` | `6` | Number of samples for decode interpolation |
+| `--dry-run` | `false` | Skip all deployments and benchmarking (dev mode) |
 
-**Decode Interpolation** (`selected_decode_interpolation/raw_data.npz`):
-- `max_kv_tokens`: Total KV tokens capacity in decode engine
-- `x_kv_usage`: 1D array of active KV usage percentages [0, 1]
-- `y_context_length`: 1D array of average context lengths tested
-- `z_itl`: 1D array of ITLs (ms) at each (KV usage, context length) point
-- `z_thpt_per_gpu`: 1D array of throughput (tokens/s/GPU) at each point
+### Output
 
-### With Dynamo Operator
+The profiler writes `final_config.yaml` to the output directory. When the planner is enabled, this is a multi-document YAML containing ConfigMaps + DGD. The `profiler_status.yaml` file tracks job status (`success` / `failed`).
 
-When using DGDR, the Dynamo Operator:
-
-1. Creates profiling jobs automatically
-2. Stores profiling data in ConfigMaps (`planner-profile-data`)
-3. Generates optimized DGD configurations
-4. Deploys the DGD with SLA Planner integration
-
-The generated DGD is tracked via labels:
-```yaml
-metadata:
-  labels:
-    dgdr.nvidia.com/name: my-deployment
-    dgdr.nvidia.com/namespace: your-namespace
-```
-
-### With Observability
-
-Monitor profiling jobs:
-
-```bash
-kubectl logs -f job/profile-<dgdr-name> -n $NAMESPACE
-kubectl describe dgdr <name> -n $NAMESPACE
-```
-
-## Advanced Topics
-
-### Manual Deployment Control
-
-Disable auto-deployment to review the generated DGD before applying:
-
-```yaml
-spec:
-  autoApply: false
-```
-
-Then manually extract and apply:
-
-```bash
-# Extract generated DGD from DGDR status
-kubectl get dgdr my-deployment -n $NAMESPACE -o jsonpath='{.status.generatedDeployment}' | kubectl apply -f -
-
-# Or save to file for review
-kubectl get dgdr my-deployment -n $NAMESPACE -o jsonpath='{.status.generatedDeployment}' > my-dgd.yaml
-```
-
-### Mocker Deployment
-
-Deploy a mocker deployment that simulates engines without GPUs:
-
-```yaml
-spec:
-  model: <model-name>
-  backend: trtllm
-  useMocker: true    # Deploy mocker instead of real backend
-  autoApply: true
-```
-
-Profiling still runs against the real backend to collect performance data. The mocker uses this data to simulate realistic timing behavior. Useful for large-scale experiments, testing Planner behavior, and validating configurations.
-
-### Accessing Profiling Artifacts
-
-By default, profiling data is stored in ConfigMaps. For detailed artifacts (plots, logs, raw data), attach a PVC:
-
-```yaml
-profilingConfig:
-  outputPVC: "dynamo-pvc"
-```
-
-**ConfigMaps (always created):**
-- `dgdr-output-<name>`: Generated DGD configuration
-- `planner-profile-data`: Profiling data for Planner (JSON)
-
-**PVC artifacts (optional):**
-- Performance plots (PNGs)
-- DGD configurations for each profiled deployment
-- AIPerf profiling artifacts
-- Raw profiling data (`.npz` files)
-- Profiler logs
-
-Access PVC results:
-```bash
-kubectl apply -f deploy/utils/manifests/pvc-access-pod.yaml -n $NAMESPACE
-kubectl wait --for=condition=Ready pod/pvc-access-pod -n $NAMESPACE --timeout=60s
-kubectl cp $NAMESPACE/pvc-access-pod:/data ./profiling-results
-kubectl delete pod pvc-access-pod -n $NAMESPACE
-```
-
-### Output Performance Plots
-
-The profiler generates plots to visualize performance data:
-
-**Parallelization Mapping Sweep Plots:**
-- `prefill_performance.png`: TTFT vs Parallelization Mapping size
-- `decode_performance.png`: ITL vs Parallelization Mapping size and in-flight requests
-
-**In-Depth Profiling Plots:**
-- `selected_prefill_interpolation/prefill_ttft_interpolation.png`: TTFT vs ISL
-- `selected_prefill_interpolation/prefill_throughput_interpolation.png`: Throughput vs ISL
-- `selected_decode_interpolation/decode_itl_interplation.png`: ITL vs KV usage and context length
-- `selected_decode_interpolation/decode_throughput_interpolation.png`: Throughput vs KV usage and context length
-
-## Runtime Profiling (SGLang)
-
-SGLang workers expose profiling endpoints for runtime performance analysis:
-
-```bash
-# Start profiling
-curl -X POST http://localhost:9090/engine/start_profile \
-  -H "Content-Type: application/json" \
-  -d '{"output_dir": "/tmp/profiler_output"}'
-
-# Run inference requests...
-
-# Stop profiling
-curl -X POST http://localhost:9090/engine/stop_profile
-```
+## Support Matrix
 
-View traces using Chrome's `chrome://tracing`, [Perfetto UI](https://ui.perfetto.dev/), or TensorBoard.
+| Backend | Dense Models | MoE Models |
+|---------|-------------|------------|
+| vLLM | ✅ | 🚧 |
+| SGLang | ✅ | ✅ |
+| TensorRT-LLM | ✅ | 🚧 |
 
 ## Troubleshooting
 
-### Profiling Takes Too Long
-
-**Solution 1**: Use AI Configurator for rapid profiling (TensorRT-LLM only):
-```yaml
-sweep:
-  useAiConfigurator: true
-```
-
-**Solution 2**: Reduce search space:
-```yaml
-hardware:
-  minNumGpusPerEngine: 4  # Skip TP1, TP2
-  maxNumGpusPerEngine: 8  # Don't test beyond TP8
-```
-
 ### SLA Cannot Be Met
 
-**Symptoms**: Profiler reports no configuration meets targets
-
-**Solutions:**
-1. Relax SLA targets (increase TTFT/ITL)
-2. Add more GPU resources
-3. Try a different backend
-4. Use a smaller model
-
-### AI Configurator: Attention Head Constraint Error
-
-**Symptoms**: Profiling fails with error:
-```text
-AssertionError: num_heads <N> should be divisible by tp_size <M> and the division result should be >= 4
-```
-
-**Cause**: AI Configurator requires **≥4 attention heads per GPU**. Small models with few heads cannot use high TP sizes.
-
-**Affected Models:**
-- **Qwen3-0.6B** (16 heads): Max TP = 4
-- **GPT-2** (12 heads): Max TP = 3
-- Most models **\<1B parameters**: May hit this constraint
-
-**Solution**: Limit `maxNumGpusPerEngine`:
-```yaml
-hardware:
-  maxNumGpusPerEngine: 4  # For Qwen3-0.6B (16 heads / 4 = max TP of 4)
-```
-
-**Calculate Max TP**: `max_tp = num_attention_heads / 4`
-
-> [!NOTE]
-> This is an AI Configurator limitation. Online profiling doesn't have this constraint.
+The profiler logs a warning and updates the SLA to the best achievable value. To improve results:
+- Relax SLA targets (increase TTFT/ITL)
+- Add more GPU resources
+- Try a different backend
+- Use a smaller or quantized model
 
-### Image Pull Errors
-
-**Symptoms**: `ErrImagePull` or `ImagePullBackOff`
+### Profiling Takes Too Long
 
-**Solution**: Ensure image pull secrets are configured:
-```bash
-kubectl create secret docker-registry nvcr-imagepullsecret \
-  --docker-server=nvcr.io \
-  --docker-username='$oauthtoken' \
-  --docker-password=<NGC_API_KEY> \
-  --namespace <your-namespace>
-```
+- Use `searchStrategy: rapid` for ~30s profiling
+- Reduce interpolation granularity
+- Reduce the GPU search space via hardware constraints
 
 ### Out of Memory During Profiling
 
-**Symptoms**: OOM errors in profiling jobs
-
-**Solutions:**
-1. Reduce `gpu_memory_utilization` in engine config
-2. Reduce `--max-context-length`
-3. Skip larger TP configurations
-4. Use fewer GPUs per test
+- Reduce `max_batch_size` in engine config
+- Skip larger TP configurations by constraining hardware
+- Use a quantized model variant
 
-### Unsupported Parallelization Mapping in Backend
-
-**Symptoms**: Startup/runtime error in the backend (e.g., prime number of attention heads constraining TP to 1, or backend not supporting different TP sizes for prefill and decode).
+### Image Pull Errors
 
-**Solutions:**
-1. Contact the backend to add support and bump backend version in Dynamo
-2. Constrain the max and min number of GPUs per engine to the supported range
+Ensure image pull secrets are configured in your namespace for the container registry.
 
 ## See Also
 
-- [Profiler Examples](profiler-examples.md) - Complete DGDR YAML examples
-- [SLA Planner Guide](../planner/planner-guide.md) - End-to-end deployment workflow
-- [SLA Planner Architecture](../planner/planner-guide.md) - How the Planner uses profiling data
-- [DGDR API Reference](../../kubernetes/api-reference.md) - DGDR specification
-- [Profiler Arguments Reference](https://github.com/ai-dynamo/dynamo/blob/main/components/src/dynamo/profiler/utils/profiler_argparse.py) - Full CLI reference
+- [Profiler README](README.md) — Quick overview and feature matrix
+- [Profiler Examples](profiler-examples.md) — Complete DGDR YAML examples
+- [Planner Guide](../planner/planner-guide.md) — PlannerConfig reference and scaling modes
+- [DGDR API Reference](../../kubernetes/api-reference.md) — Full DGDR specification

docs/pages/kubernetes/api-reference.md

@@ -1327,7 +1327,7 @@ _Appears in:_
 | --- | --- | --- | --- |
 | `model` _string_ | Model specifies the model to deploy (e.g., "Qwen/Qwen3-0.6B", "meta-llama/Llama-3-70b").<br />Can be a HuggingFace ID or a private model name. |  | MinLength: 1 <br />Required: \{\} <br /> |
 | `backend` _[BackendType](#backendtype)_ | Backend specifies the inference backend to use for profiling and deployment. | auto | Enum: [auto sglang trtllm vllm] <br />Optional: \{\} <br /> |
-| `image` _string_ | Image is the container image reference for the profiling job (frontend image).<br />Example: "nvcr.io/nvidia/dynamo-runtime:latest"<br />backend type automatically; backend images can be overridden via overrides.dgd. |  | Optional: \{\} <br /> |
+| `image` _string_ | Image is the container image reference for the profiling job (frontend image).<br />Example: "nvcr.io/nvidia/ai-dynamo/dynamo-frontend:1.0.0". |  | Optional: \{\} <br /> |
 | `modelCache` _[ModelCacheSpec](#modelcachespec)_ | ModelCache provides optional PVC configuration for pre-downloaded model weights.<br />When provided, weights are loaded from the PVC instead of downloading from HuggingFace. |  | Optional: \{\} <br /> |
 | `hardware` _[HardwareSpec](#hardwarespec)_ | Hardware describes the hardware resources available for profiling and deployment.<br />Typically auto-filled by the operator from cluster discovery. |  | Optional: \{\} <br /> |
 | `workload` _[WorkloadSpec](#workloadspec)_ | Workload defines the expected workload characteristics for SLA-based profiling. |  | Optional: \{\} <br /> |

pyproject.toml

@@ -139,14 +139,14 @@ line_length = 88
 balanced_wrapping = true
 indent = "    "
 skip = ["build"]
-known_first_party = ["dynamo"]
+known_first_party = ["dynamo", "deploy"]
 # isort may confuse what is 1st or 3rd library. e.g.
 # when dynamo/vllm/omni/xx.py import vllm, local isort may treat this `vllm` as first
 # party heuristically. This causes local sort differs from GitHub sort and pre-commit
 # failure. To mitigate 1) one can install 3rd party lib so that isort is aware of it,
 # 2) hardcode 3rd party lib here, 3) add "# isort: skip_file" to problematic files
 # as the last resort.
-known_third_party = ["vllm", "tensorrt_llm", "sglang"]
+known_third_party = ["vllm", "tensorrt_llm", "sglang", "aiconfigurator"]
 
 [tool.pytest.ini_options]
 minversion = "8.0"
@@ -187,6 +187,7 @@ filterwarnings = [
     "ignore:.*unclosed event loop.*:ResourceWarning", # Ignore unclosed event loop warnings
     "ignore:.*Exception ignored in.*:pytest.PytestUnraisableExceptionWarning", # Ignore unraisable exception warnings
     "ignore:The pynvml package is deprecated.*:FutureWarning", # Ignore pynvml deprecation warning, temporary until upstream library updates to nvidia-ml-py
+    "ignore:The behavior of DataFrame concatenation with empty or all-NA entries is deprecated.*:FutureWarning", # pandas 2.x concat deprecation in AIC SDK TODO: fix in AIC
     # Pydantic V2 deprecation warnings from TRTLLM dependencies (raised at import time during collection)
     "ignore:Support for class-based `config`.*:pydantic.warnings.PydanticDeprecatedSince20",
     "ignore:Using extra keyword arguments on `Field`.*:pydantic.warnings.PydanticDeprecatedSince20",

tests/profiler/configs/10_thorough_override_security_context.yaml

+# SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
+# SPDX-License-Identifier: Apache-2.0
+
+# Case 10: Thorough sweep with DGD overrides for imagePullSecrets.
+# Verifies that overrides can inject new spec-level fields (imagePullSecrets)
+# that do not exist in the base DGD template.
+model: "Qwen/Qwen3-32B"
+backend: trtllm
+image: "nvcr.io/nvidia/dynamo:latest"
+hardware:
+  gpuSku: h200_sxm
+  totalGpus: 8
+  numGpusPerNode: 8
+searchStrategy: thorough
+overrides:
+  dgd:
+    spec:
+      imagePullSecrets:
+        - name: my-registry-secret
+        - name: nvcr-pull-secret

tests/profiler/configs/11_auto_rapid_no_planner_no_load.yaml

+# SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
+# SPDX-License-Identifier: Apache-2.0
+
+# Case 11: Auto backend, rapid, without planner, no input load
+model: "Qwen/Qwen3-32B"
+image: "hongkuanz196/trtllm-runtime:hzhou-0224"
+hardware:
+  gpuSku: h200_sxm
+  totalGpus: 8
+  numGpusPerNode: 8

tests/profiler/configs/1_rapid_supported_no_planner_no_load.yaml

+# SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
+# SPDX-License-Identifier: Apache-2.0
+
+# Case 1: AIC supported model, rapid, without planner, no input load
+model: "Qwen/Qwen3-32B"
+backend: trtllm
+image: "nvcr.io/nvidia/ai-dynamo/dynamo-frontend:latest"
+hardware:
+  gpuSku: h200_sxm
+  totalGpus: 8
+  numGpusPerNode: 8
+sla:
+  itl: 50.0

tests/profiler/configs/2_rapid_supported_no_planner_with_load.yaml

+# SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
+# SPDX-License-Identifier: Apache-2.0
+
+# Case 2: AIC supported model, rapid, without planner, input load (request rate)
+model: "Qwen/Qwen3-32B"
+backend: trtllm
+image: "nvcr.io/nvidia/ai-dynamo/dynamo-frontend:latest"
+hardware:
+  gpuSku: h200_sxm
+  totalGpus: 64
+  numGpusPerNode: 8
+workload:
+  requestRate: 5.0
+sla:
+  itl: 50.0

tests/profiler/configs/2b_rapid_supported_pvc_no_planner_with_load.yaml

+# SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
+# SPDX-License-Identifier: Apache-2.0
+
+# Case 2b: AIC supported model, rapid, without planner, input load, with PVC model cache
+model: "Qwen/Qwen3-32B"
+backend: trtllm
+image: "nvcr.io/nvidia/ai-dynamo/dynamo-frontend:latest"
+hardware:
+  gpuSku: h200_sxm
+  totalGpus: 64
+  numGpusPerNode: 8
+modelCache:
+  pvcName: model-cache
+  pvcModelPath: /model/Qwen3-32B
+workload:
+  requestRate: 5.0
+sla:
+  itl: 50.0

tests/profiler/configs/2c_rapid_supported_e2e_latency.yaml

+# SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
+# SPDX-License-Identifier: Apache-2.0
+
+# Case 2c: AIC supported model, rapid, without planner, e2eLatency instead of ttft/itl
+model: "Qwen/Qwen3-32B"
+backend: trtllm
+image: "nvcr.io/nvidia/ai-dynamo/dynamo-frontend:latest"
+hardware:
+  gpuSku: h200_sxm
+  totalGpus: 64
+  numGpusPerNode: 8
+workload:
+  requestRate: 5.0
+sla:
+  ttft: null
+  itl: null
+  e2eLatency: 35000.0

tests/profiler/configs/2d_rapid_both_concurrency_and_rate_error.yaml

+# SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
+# SPDX-License-Identifier: Apache-2.0
+
+# Case 2d: Both concurrency and requestRate specified — should fail validation
+model: "Qwen/Qwen3-32B"
+backend: trtllm
+image: "nvcr.io/nvidia/ai-dynamo/dynamo-frontend:latest"
+hardware:
+  gpuSku: h200_sxm
+  totalGpus: 64
+  numGpusPerNode: 8
+workload:
+  concurrency: 50
+  requestRate: 5.0
+sla:
+  itl: 50.0

tests/profiler/configs/3_rapid_supported_planner_rapid_sweep.yaml

+# SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
+# SPDX-License-Identifier: Apache-2.0
+
+# Case 3: AIC supported model, rapid, with planner, rapid pre-deployment sweeping
+model: "Qwen/Qwen3-32B"
+backend: trtllm
+image: "nvcr.io/nvidia/ai-dynamo/dynamo-frontend:latest"
+hardware:
+  gpuSku: h200_sxm
+  totalGpus: 8
+  numGpusPerNode: 8
+sla:
+  itl: 50.0
+features:
+  planner:
+    pre_deployment_sweeping_mode: rapid
+    enable_throughput_scaling: true
+    enable_load_scaling: false
+    mode: disagg
+    backend: trtllm

tests/profiler/configs/3b_rapid_supported_planner_rapid_sweep_mocker.yaml

+# SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
+# SPDX-License-Identifier: Apache-2.0
+
+# Case 3b: AIC supported model, rapid, with planner, rapid pre-deployment sweeping, enable mocker
+model: "Qwen/Qwen3-32B"
+backend: trtllm
+image: "nvcr.io/nvidia/ai-dynamo/dynamo-frontend:latest"
+hardware:
+  gpuSku: h200_sxm
+  totalGpus: 8
+  numGpusPerNode: 8
+sla:
+  itl: 50.0
+features:
+  planner:
+    pre_deployment_sweeping_mode: rapid
+    enable_throughput_scaling: true
+    enable_load_scaling: false
+    mode: disagg
+    backend: trtllm
+  mocker:
+    enabled: true

tests/profiler/configs/4_rapid_unsupported_no_planner.yaml

+# SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
+# SPDX-License-Identifier: Apache-2.0
+
+# Case 4: AIC unsupported model, rapid, without planner
+# l40s + vllm has no disagg support in AIC
+model: "Qwen/Qwen3-32B"
+backend: vllm
+image: "nvcr.io/nvidia/ai-dynamo/dynamo-frontend:latest"
+hardware:
+  gpuSku: l40s
+  totalGpus: 4
+  numGpusPerNode: 4
+  vramMb: 48000
+sla:
+  itl: 50.0

tests/profiler/configs/5_rapid_unsupported_planner.yaml

+# SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
+# SPDX-License-Identifier: Apache-2.0
+
+# Case 5: AIC unsupported model, rapid, with planner
+# Should raise error if throughput scaling is enabled (AIC unsupported).
+# This config uses load-based scaling only to test the non-error path.
+model: "Qwen/Qwen3-32B"
+backend: vllm
+image: "nvcr.io/nvidia/ai-dynamo/dynamo-frontend:latest"
+hardware:
+  gpuSku: l40s
+  totalGpus: 4
+  numGpusPerNode: 4
+sla:
+  itl: 50.0
+features:
+  planner:
+    pre_deployment_sweeping_mode: rapid
+    enable_throughput_scaling: false
+    enable_load_scaling: true
+    mode: disagg
+    backend: vllm

tests/profiler/configs/5b_rapid_unsupported_planner_throughput_error.yaml

+# SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
+# SPDX-License-Identifier: Apache-2.0
+
+# Case 5b: AIC unsupported model, rapid, with planner + throughput scaling
+# This should FAIL with a ValueError because throughput-based planner
+# requires AIC support.
+model: "Qwen/Qwen3-32B"
+backend: vllm
+image: "nvcr.io/nvidia/ai-dynamo/dynamo-frontend:latest"
+hardware:
+  gpuSku: l40s
+  totalGpus: 4
+  numGpusPerNode: 4
+sla:
+  itl: 50.0
+features:
+  planner:
+    pre_deployment_sweeping_mode: rapid
+    enable_throughput_scaling: true
+    enable_load_scaling: false
+    mode: disagg
+    backend: vllm

tests/profiler/configs/6_thorough_no_planner_with_load.yaml

+# SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
+# SPDX-License-Identifier: Apache-2.0
+
+# Case 6: Thorough, without planner, input load (test with --dry-run)
+model: "Qwen/Qwen3-32B"
+backend: trtllm
+image: "nvcr.io/nvidia/ai-dynamo/dynamo-frontend:latest"
+hardware:
+  gpuSku: h200_sxm
+  totalGpus: 64
+  numGpusPerNode: 8
+workload:
+  requestRate: 5.0
+sla:
+  itl: 50.0
+searchStrategy: thorough

tests/profiler/configs/7_thorough_planner_rapid_sweep.yaml

+# SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
+# SPDX-License-Identifier: Apache-2.0
+
+# Case 7: Thorough, with planner, rapid mode for planner pre-deployment sweeping
+# (test with --dry-run)
+# Gate check: if AIC unsupported, rapid pre-deployment sweeping falls back to none.
+# This uses a supported combo so rapid sweeping should work.
+model: "Qwen/Qwen3-32B"
+backend: trtllm
+image: "nvcr.io/nvidia/ai-dynamo/dynamo-frontend:latest"
+hardware:
+  gpuSku: h200_sxm
+  totalGpus: 8
+  numGpusPerNode: 8
+sla:
+  itl: 50.0
+searchStrategy: thorough
+features:
+  planner:
+    pre_deployment_sweeping_mode: rapid
+    enable_throughput_scaling: true
+    enable_load_scaling: false
+    mode: disagg
+    backend: trtllm

tests/profiler/configs/7b_thorough_planner_thorough_sweep.yaml

+# SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
+# SPDX-License-Identifier: Apache-2.0
+
+# Case 7b: Thorough search + planner with thorough pre-deployment sweeping
+# Both the config picking and the interpolation use real GPUs (mocked in tests).
+model: "Qwen/Qwen3-32B"
+backend: trtllm
+image: "nvcr.io/nvidia/ai-dynamo/dynamo-frontend:latest"
+hardware:
+  gpuSku: h200_sxm
+  totalGpus: 8
+  numGpusPerNode: 8
+sla:
+  itl: 50.0
+searchStrategy: thorough
+features:
+  planner:
+    pre_deployment_sweeping_mode: thorough
+    enable_throughput_scaling: true
+    enable_load_scaling: false
+    mode: disagg
+    backend: trtllm

tests/profiler/configs/8_thorough_empty_candidates.yaml

+# SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
+# SPDX-License-Identifier: Apache-2.0
+
+# Case 8: Thorough mode with empty candidate list
+# DeepSeek-R1 (671B MoE) on 1 L40S GPU — far too small to fit,
+# enumerate_profiling_configs should return empty lists.
+model: "deepseek-ai/DeepSeek-R1"
+backend: sglang
+image: "nvcr.io/nvidia/ai-dynamo/dynamo-frontend:latest"
+hardware:
+  gpuSku: l40s
+  totalGpus: 1
+  numGpusPerNode: 1
+workload:
+  requestRate: 1.0
+sla:
+  ttft: 5000.0
+  itl: 100.0
+searchStrategy: thorough