Commits · aaf283bbb898bb7fae2909fe11a2762b6350d1cc · OpenDAS / dynamo

01 Jul, 2025 1 commit
- feat: Approximate KV Routing (#1636) · aaf283bb
  jthomson04 authored Jun 30, 2025
  
  aaf283bb
30 Jun, 2025 2 commits
- fix: bump vLLM commit and revert side channel host change for DS R1 DEP deployment (#1696) · 9cd9993d
  GuanLuo authored Jun 30, 2025
```
Signed-off-by: GuanLuo <41310872+GuanLuo@users.noreply.github.com>
```
  9cd9993d
- feat: support sla planner in vllm_v1 example (#1680) · 2bed47eb
  Hongkuan Zhou authored Jun 30, 2025
  
  2bed47eb
28 Jun, 2025 1 commit
- feat: add kv router to sglang (#1605) · 0b47f897
  Faradawn Yang authored Jun 28, 2025
  
  0b47f897
27 Jun, 2025 6 commits
- fix: docker compose command for sglang (#1482) · ae47638b
  Faradawn Yang authored Jun 27, 2025
  
  ae47638b
- docs: instructions to run DSR1 with SGLang wideep on 104+ GPUs (#1583) · 9d7624f1
  ishandhanani authored Jun 27, 2025
```
Co-authored-by: kkranen <kyle.kranen@gmail.com>
```
  9d7624f1
- fix: fix the if statement for checking tllm_disagg_params.opaque_state is none (#1679) · 96cc0698
  richardhuo-nv authored Jun 27, 2025
  
  96cc0698
- fix: Little kv routing fix (#1677) · e3f15ee1
  jthomson04 authored Jun 27, 2025
  
  e3f15ee1
- feat: Unnormalize waiting requests + predictive load updates for Python router... · 8392e7a1
  Yan Ru Pei authored Jun 27, 2025
```
feat: Unnormalize waiting requests + predictive load updates for Python router (mirroring Rust) + softmax sampling to reduce thrashing (#1638)
```
  8392e7a1
- fix: remove undefined headless config (#1660) · e53a759c
  GuanLuo authored Jun 26, 2025
  
  e53a759c
26 Jun, 2025 2 commits
- refactor: Refactor the TRTLLM example components and improve UI (#1654) · 03d976c7
  Tanmay Verma authored Jun 26, 2025
```
Signed-off-by: Tanmay Verma <tanmayv@nvidia.com>
```
  03d976c7
- feat: Add experimental WideEP + EPLB aggregated example for TRTLLM (#1652) · 5fe5a950
  Ryan McCormick authored Jun 27, 2025
  
  5fe5a950
25 Jun, 2025 3 commits
- feat: support batch `/completions` (#1626) · fc16a79b
  ishandhanani authored Jun 25, 2025
```
Co-authored-by: Ryan McCormick <rmccormick@nvidia.com>
```
  fc16a79b
- fix: Disable NIXL backend for TRTLLM on ARM (#1639) · e84b1e77
  Tanmay Verma authored Jun 25, 2025
  
  e84b1e77
- chore: Add SERVED_MODEL_NAME for consistent model name regardless of MODEL_PATH (#1632) · 2becce56
  Ryan McCormick authored Jun 25, 2025
  
  2becce56
24 Jun, 2025 1 commit
- feat: Using NIXL for KV cache transfer when using disaggregated serving in TRTLLM (#1591) · 0b7cdf55
  Tanmay Verma authored Jun 24, 2025
```
Signed-off-by: Tanmay Verma <tanmay2592@gmail.com>
Co-authored-by: Ryan McCormick <rmccormick@nvidia.com>
```
  0b7cdf55
17 Jun, 2025 5 commits
- fix: Fix message truncation in disagg flow (#1572) · 84454ab4
  Tanmay Verma authored Jun 17, 2025
  
  84454ab4
- feat: kubernetes overrides for the entrypoint and cmd (#1396) · fbf1ffd7
  atchernych authored Jun 17, 2025
  
  fbf1ffd7
- feat: add graceful shutdown in vllm_1 (#1562) · 49b7e930
  Hongkuan Zhou authored Jun 17, 2025
  
  49b7e930
- feat(sglang): deepep sglang support in dynamo (#1120) · 64445c0a
  ishandhanani authored Jun 17, 2025
```
Co-authored-by: kkranen <kyle.kranen@gmail.com>
```
  64445c0a
- fix: Handle model not found error for multimodal example (#1545) · e2716073
  Kris Hung authored Jun 17, 2025
  
  e2716073
16 Jun, 2025 2 commits
- build: DIS-148 use the tensorrt_llm public wheel from pypi by default in container build (#1525) · 47d05d7a
  richardhuo-nv authored Jun 16, 2025
  
  47d05d7a
- docs: update the multinode.md link to the correct path (#1535) · ce48a863
  runzhen authored Jun 15, 2025
```
Signed-off-by: Runzhen Wang <wangrunzhen@gmail.com>
```
  ce48a863
14 Jun, 2025 3 commits

feat: Standalone Router (#1409) · 13a99b7f

Yan Ru Pei authored Jun 14, 2025


Signed-off-by: PeaBrane <yanrpei@gmail.com>
Signed-off-by: Yan Ru Pei <yanrpei@gmail.com>
Signed-off-by: jain-ria <riajain@NVIDIA.com>
Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>
Co-authored-by: jain-ria <riajain@NVIDIA.com>

13a99b7f

feat: add VllmDpWorker for DP setup (#1513) · 1906b702

GuanLuo authored Jun 13, 2025


Signed-off-by: GuanLuo <41310872+GuanLuo@users.noreply.github.com>
Co-authored-by: ptarasiewiczNV <104908264+ptarasiewiczNV@users.noreply.github.com>

1906b702

feat: SLA-based Planner (#1420) · 3f53a78e

Hongkuan Zhou authored Jun 13, 2025

Signed-off-by: Hongkuan Zhou <tedzhouhk@gmail.com>
Co-authored-by: hhzhang16 <54051230+hhzhang16@users.noreply.github.com>
Co-authored-by: Alec <35311602+alec-flowers@users.noreply.github.com>
Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>

3f53a78e

13 Jun, 2025 6 commits
- fix: Improve dynamo.connect Error Reporting (#1524) · 281a69e5
  J Wyman authored Jun 13, 2025
  
  281a69e5
- docs: add trouble shooting section in benchmarking guide. Add known pitfall (#1503) · e621f249
  GuanLuo authored Jun 13, 2025
```
Signed-off-by: GuanLuo <41310872+GuanLuo@users.noreply.github.com>
Co-authored-by: Ryan McCormick <rmccormick@nvidia.com>
```
  e621f249
- fix: Fix NATS_SERVER value, add details on customizing MOUNTS (#1520) · ae7e08a3
  Ryan McCormick authored Jun 14, 2025
  
  ae7e08a3
- feat: Video support with Dynamo (#1443) · 75503dae
  Indrajit Bhosale authored Jun 13, 2025
```
Added support in examples for multimodal video with aggregated and disaggregated architecture using LLaVA-NeXT-Video-7B
```
  75503dae
- docs: Add multi-node TRTLLM worker example (Deepseek R1) (#1511) · 40ca062f
  Ryan McCormick authored Jun 14, 2025
  
  40ca062f
- chore: add utilities for benchmarking (#1371) · fcfc21f2
  GuanLuo authored Jun 12, 2025
```
Signed-off-by: GuanLuo <41310872+GuanLuo@users.noreply.github.com>
Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>
Co-authored-by: Ryan McCormick <rmccormick@nvidia.com>
```
  fcfc21f2
12 Jun, 2025 5 commits
- docs: fix the README link to the perf.sh file (#1501) · 0bba09a4
  richardhuo-nv authored Jun 12, 2025
  
  0bba09a4
- docs: add concurrency choice to the perf.sh (#1497) · 8585c300
  richardhuo-nv authored Jun 12, 2025
  
  8585c300
- docs: Fix Markdown Render Error (#1502) · 73e0f8ca
  J Wyman authored Jun 12, 2025
  
  73e0f8ca
- docs: Benchmarking guide interpreting results (#701) · fc5ddd2f
  Jacky authored Jun 12, 2025
```
Co-authored-by: Ziqi Fan <ziqif@nvidia.com>
```
  fc5ddd2f
- chore: Move Benchmarking to Top Level (#1461) · 2ae9ab9f
  J Wyman authored Jun 11, 2025
```
Signed-off-by: Tanmay Verma <tanmay2592@gmail.com>
Co-authored-by: Tanmay Verma <tanmay2592@gmail.com>
Co-authored-by: Jacky <18255193+kthui@users.noreply.github.com>
```
  2ae9ab9f
11 Jun, 2025 3 commits
- feat: generalize VLM embedding extraction (#1388) · e924a7c7
  hhzhang16 authored Jun 11, 2025
```
Signed-off-by: hhzhang16 <54051230+hhzhang16@users.noreply.github.com>
Co-authored-by: Kris Hung <krish@nvidia.com>
```
  e924a7c7
- docs: Add note about ignore_eos for MTP (#1475) · 150e983a
  Ryan McCormick authored Jun 12, 2025
  
  150e983a
- docs: MTP + TensorRT LLM + DS R1 disaggregated example (#1473) · 3363d8b6
  richardhuo-nv authored Jun 11, 2025
  
  3363d8b6