Commits · fc16a79bfc5a4c4f58503d3c36f2013340244cac · OpenDAS / dynamo

25 Jun, 2025 3 commits
- feat: support batch `/completions` (#1626) · fc16a79b
  ishandhanani authored Jun 25, 2025
```
Co-authored-by: Ryan McCormick <rmccormick@nvidia.com>
```
  fc16a79b
- fix: Disable NIXL backend for TRTLLM on ARM (#1639) · e84b1e77
  Tanmay Verma authored Jun 25, 2025
  
  e84b1e77
- chore: Add SERVED_MODEL_NAME for consistent model name regardless of MODEL_PATH (#1632) · 2becce56
  Ryan McCormick authored Jun 25, 2025
  
  2becce56
24 Jun, 2025 1 commit
- feat: Using NIXL for KV cache transfer when using disaggregated serving in TRTLLM (#1591) · 0b7cdf55
  Tanmay Verma authored Jun 24, 2025
```
Signed-off-by: Tanmay Verma <tanmay2592@gmail.com>
Co-authored-by: Ryan McCormick <rmccormick@nvidia.com>
```
  0b7cdf55
17 Jun, 2025 5 commits
- fix: Fix message truncation in disagg flow (#1572) · 84454ab4
  Tanmay Verma authored Jun 17, 2025
  
  84454ab4
- feat: kubernetes overrides for the entrypoint and cmd (#1396) · fbf1ffd7
  atchernych authored Jun 17, 2025
  
  fbf1ffd7
- feat: add graceful shutdown in vllm_1 (#1562) · 49b7e930
  Hongkuan Zhou authored Jun 17, 2025
  
  49b7e930
- feat(sglang): deepep sglang support in dynamo (#1120) · 64445c0a
  ishandhanani authored Jun 17, 2025
```
Co-authored-by: kkranen <kyle.kranen@gmail.com>
```
  64445c0a
- fix: Handle model not found error for multimodal example (#1545) · e2716073
  Kris Hung authored Jun 17, 2025
  
  e2716073
16 Jun, 2025 2 commits
- build: DIS-148 use the tensorrt_llm public wheel from pypi by default in container build (#1525) · 47d05d7a
  richardhuo-nv authored Jun 16, 2025
  
  47d05d7a
- docs: update the multinode.md link to the correct path (#1535) · ce48a863
  runzhen authored Jun 15, 2025
```
Signed-off-by: Runzhen Wang <wangrunzhen@gmail.com>
```
  ce48a863
14 Jun, 2025 3 commits

feat: Standalone Router (#1409) · 13a99b7f

Yan Ru Pei authored Jun 14, 2025


Signed-off-by: PeaBrane <yanrpei@gmail.com>
Signed-off-by: Yan Ru Pei <yanrpei@gmail.com>
Signed-off-by: jain-ria <riajain@NVIDIA.com>
Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>
Co-authored-by: jain-ria <riajain@NVIDIA.com>

13a99b7f

feat: add VllmDpWorker for DP setup (#1513) · 1906b702

GuanLuo authored Jun 13, 2025


Signed-off-by: GuanLuo <41310872+GuanLuo@users.noreply.github.com>
Co-authored-by: ptarasiewiczNV <104908264+ptarasiewiczNV@users.noreply.github.com>

1906b702

feat: SLA-based Planner (#1420) · 3f53a78e

Hongkuan Zhou authored Jun 13, 2025

Signed-off-by: Hongkuan Zhou <tedzhouhk@gmail.com>
Co-authored-by: hhzhang16 <54051230+hhzhang16@users.noreply.github.com>
Co-authored-by: Alec <35311602+alec-flowers@users.noreply.github.com>
Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>

3f53a78e

13 Jun, 2025 6 commits
- fix: Improve dynamo.connect Error Reporting (#1524) · 281a69e5
  J Wyman authored Jun 13, 2025
  
  281a69e5
- docs: add trouble shooting section in benchmarking guide. Add known pitfall (#1503) · e621f249
  GuanLuo authored Jun 13, 2025
```
Signed-off-by: GuanLuo <41310872+GuanLuo@users.noreply.github.com>
Co-authored-by: Ryan McCormick <rmccormick@nvidia.com>
```
  e621f249
- fix: Fix NATS_SERVER value, add details on customizing MOUNTS (#1520) · ae7e08a3
  Ryan McCormick authored Jun 14, 2025
  
  ae7e08a3
- feat: Video support with Dynamo (#1443) · 75503dae
  Indrajit Bhosale authored Jun 13, 2025
```
Added support in examples for multimodal video with aggregated and disaggregated architecture using LLaVA-NeXT-Video-7B
```
  75503dae
- docs: Add multi-node TRTLLM worker example (Deepseek R1) (#1511) · 40ca062f
  Ryan McCormick authored Jun 14, 2025
  
  40ca062f
- chore: add utilities for benchmarking (#1371) · fcfc21f2
  GuanLuo authored Jun 12, 2025
```
Signed-off-by: GuanLuo <41310872+GuanLuo@users.noreply.github.com>
Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>
Co-authored-by: Ryan McCormick <rmccormick@nvidia.com>
```
  fcfc21f2
12 Jun, 2025 5 commits
- docs: fix the README link to the perf.sh file (#1501) · 0bba09a4
  richardhuo-nv authored Jun 12, 2025
  
  0bba09a4
- docs: add concurrency choice to the perf.sh (#1497) · 8585c300
  richardhuo-nv authored Jun 12, 2025
  
  8585c300
- docs: Fix Markdown Render Error (#1502) · 73e0f8ca
  J Wyman authored Jun 12, 2025
  
  73e0f8ca
- docs: Benchmarking guide interpreting results (#701) · fc5ddd2f
  Jacky authored Jun 12, 2025
```
Co-authored-by: Ziqi Fan <ziqif@nvidia.com>
```
  fc5ddd2f
- chore: Move Benchmarking to Top Level (#1461) · 2ae9ab9f
  J Wyman authored Jun 11, 2025
```
Signed-off-by: Tanmay Verma <tanmay2592@gmail.com>
Co-authored-by: Tanmay Verma <tanmay2592@gmail.com>
Co-authored-by: Jacky <18255193+kthui@users.noreply.github.com>
```
  2ae9ab9f
11 Jun, 2025 4 commits
- feat: generalize VLM embedding extraction (#1388) · e924a7c7
  hhzhang16 authored Jun 11, 2025
```
Signed-off-by: hhzhang16 <54051230+hhzhang16@users.noreply.github.com>
Co-authored-by: Kris Hung <krish@nvidia.com>
```
  e924a7c7
- docs: Add note about ignore_eos for MTP (#1475) · 150e983a
  Ryan McCormick authored Jun 12, 2025
  
  150e983a
- docs: MTP + TensorRT LLM + DS R1 disaggregated example (#1473) · 3363d8b6
  richardhuo-nv authored Jun 11, 2025
  
  3363d8b6
- docs: add message to guide users to the stable version (#1457) · e32fe675
  richardhuo-nv authored Jun 11, 2025
  
  e32fe675
10 Jun, 2025 2 commits
- chore(sglang): support dp attention routing (#1415) · 47c3dad7
  ishandhanani authored Jun 09, 2025
  
  47c3dad7
- fix: remove unused bentoml references (#1412) · 75d7c3b9
  Biswa Panda authored Jun 09, 2025
  
  75d7c3b9
07 Jun, 2025 2 commits
- docs: Reference Deepseek R1 configs in TRTLLM README (#1414) · 9281c95f
  Ryan McCormick authored Jun 08, 2025
  
  9281c95f
- docs: add aggregated example turning on MTP with DeepSeek R1 (#1421) · 4de7f44c
  richardhuo-nv authored Jun 06, 2025
  
  4de7f44c
05 Jun, 2025 4 commits
- chore: Remove nats-py dependency (#1387) · e61f1c8a
  Kris Hung authored Jun 05, 2025
  
  e61f1c8a
- feat: introduce abstract classes to dynamo services (#924) · b293b45b
  mohammedabdulwahhab authored Jun 04, 2025
```
Signed-off-by: mohammedabdulwahhab <furkhan324@berkeley.edu>
Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>
```
  b293b45b
- fix: Use Rust Ingress (dynamo-run) for the Frontend (#1391) · 568eb100
  Tanmay Verma authored Jun 04, 2025
  
  568eb100
- fix: remove unused arg in planner (#1390) · a6887fa0
  Hongkuan Zhou authored Jun 04, 2025
  
  a6887fa0
04 Jun, 2025 3 commits
- feat: decouple bento dependency (#1266) · afb8495e
  Biswa Panda authored Jun 04, 2025
  
  afb8495e
- fix: Fix link path for dynamo_run doc (#1382) · 9e8731e5
  Kris Hung authored Jun 04, 2025
  
  9e8731e5
- feat: set model specific prompt templates in the multimodal config files, add... · b80482a4
  hhzhang16 authored Jun 04, 2025
```
feat: set model specific prompt templates in the multimodal config files, add documentation for multimodal example deployment (#1366)
```
  b80482a4