Unverified Commit 06c35f2d authored by hhzhang16's avatar hhzhang16 Committed by GitHub
Browse files

feat: add explanation for circles in Profiler UI (#4998)


Signed-off-by: default avatarHannah Zhang <hannahz@nvidia.com>
parent e36b7e7a
...@@ -640,6 +640,8 @@ def create_gradio_interface( ...@@ -640,6 +640,8 @@ def create_gradio_interface(
2. **Individual**: Select one row in the Prefill table AND one row in the Decode table 2. **Individual**: Select one row in the Prefill table AND one row in the Decode table
The selection will be processed automatically once complete. The selection will be processed automatically once complete.
**Chart Reference Points:** 🔴 Max Throughput Under SLA · 🟡 Max Throughput Overall · 🟢 Latency-Optimized (lowest latency under SLA)
> 📝 **Note:** The dotted red line in the prefill and decode charts are default TTFT and ITL SLAs if not specified. > 📝 **Note:** The dotted red line in the prefill and decode charts are default TTFT and ITL SLAs if not specified.
> ⚠️ **Warning:** The TTFT values here represent the ideal case when requests arrive uniformly, minimizing queueing. Real-world TTFT may be higher than profiling results. To mitigate the issue, planner uses [correction factors](https://github.com/ai-dynamo/dynamo/blob/main/docs/planner/sla_planner.md#2-correction-factor-calculation) to adjust dynamically at runtime. > ⚠️ **Warning:** The TTFT values here represent the ideal case when requests arrive uniformly, minimizing queueing. Real-world TTFT may be higher than profiling results. To mitigate the issue, planner uses [correction factors](https://github.com/ai-dynamo/dynamo/blob/main/docs/planner/sla_planner.md#2-correction-factor-calculation) to adjust dynamically at runtime.
......
...@@ -12,7 +12,7 @@ ...@@ -12,7 +12,7 @@
# For Multimodal EPD (required for device_map="auto" in vision model loading) # For Multimodal EPD (required for device_map="auto" in vision model loading)
accelerate accelerate
aiconfigurator[webapp] @ git+https://github.com/ai-dynamo/aiconfigurator.git@0c8f38d354e9138f2cc00efcde66245b3801df1d aiconfigurator[webapp] @ git+https://github.com/ai-dynamo/aiconfigurator.git@9db6a1a5011c0ff80b5349f377f17846c5274b02
aiofiles aiofiles
aiperf @ git+https://github.com/ai-dynamo/aiperf.git@4d3fa29403c8f75da22a14f1f7b3aeb27db9288f aiperf @ git+https://github.com/ai-dynamo/aiperf.git@4d3fa29403c8f75da22a14f1f7b3aeb27db9288f
av==15.0.0 av==15.0.0
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment