Commits · 5bf23d54f3e46a15ff5000773a32d8829befa919 · OpenDAS / dynamo

14 Jul, 2025 1 commit

docs: TRTLLM Example of Llama4+Eagle3 (Speculative Decoding) (#1828) · ef59ac8d

KrishnanPrash authored Jul 14, 2025


Signed-off-by: KrishnanPrash <140860868+KrishnanPrash@users.noreply.github.com>
Co-authored-by: Iman Tabrizian <10105175+Tabrizian@users.noreply.github.com>

ef59ac8d

07 Jul, 2025 1 commit
- docs: Remove the outdated limitation (#1802) · b1509ea7
  Tanmay Verma authored Jul 07, 2025
  
  b1509ea7
03 Jul, 2025 1 commit
- feat: Add experimental WideEP + EPLB dis-aggregated example for TRTLLM (#1690) · 7a353e61
  Ryan McCormick authored Jul 04, 2025
```
Co-authored-by: tanmayv25 <tanmay2592@gmail.com>
```
  7a353e61
26 Jun, 2025 2 commits
- refactor: Refactor the TRTLLM example components and improve UI (#1654) · 03d976c7
  Tanmay Verma authored Jun 26, 2025
```
Signed-off-by: Tanmay Verma <tanmayv@nvidia.com>
```
  03d976c7
- feat: Add experimental WideEP + EPLB aggregated example for TRTLLM (#1652) · 5fe5a950
  Ryan McCormick authored Jun 27, 2025
  
  5fe5a950
25 Jun, 2025 1 commit
- chore: Add SERVED_MODEL_NAME for consistent model name regardless of MODEL_PATH (#1632) · 2becce56
  Ryan McCormick authored Jun 25, 2025
  
  2becce56
17 Jun, 2025 1 commit
- fix: Fix message truncation in disagg flow (#1572) · 84454ab4
  Tanmay Verma authored Jun 17, 2025
  
  84454ab4
13 Jun, 2025 2 commits
- fix: Fix NATS_SERVER value, add details on customizing MOUNTS (#1520) · ae7e08a3
  Ryan McCormick authored Jun 14, 2025
  
  ae7e08a3
- docs: Add multi-node TRTLLM worker example (Deepseek R1) (#1511) · 40ca062f
  Ryan McCormick authored Jun 14, 2025
  
  40ca062f
11 Jun, 2025 1 commit
- docs: MTP + TensorRT LLM + DS R1 disaggregated example (#1473) · 3363d8b6
  richardhuo-nv authored Jun 11, 2025
  
  3363d8b6
07 Jun, 2025 1 commit
- docs: add aggregated example turning on MTP with DeepSeek R1 (#1421) · 4de7f44c
  richardhuo-nv authored Jun 06, 2025
  
  4de7f44c
05 Jun, 2025 1 commit
- fix: Use Rust Ingress (dynamo-run) for the Frontend (#1391) · 568eb100
  Tanmay Verma authored Jun 04, 2025
  
  568eb100
02 Jun, 2025 1 commit
- fix: Flatten pytorch_backend_config section to address breaking change to trtllm config (#1326) · d9f6d7a5
  Ryan McCormick authored Jun 03, 2025
  
  d9f6d7a5
31 May, 2025 1 commit
- fix: Update breaking change to enable_overlap_scheduler field from TRTLLM commit b4e5df0e (#1310) · 859944f4
  Ryan McCormick authored May 31, 2025
  
  859944f4
16 May, 2025 1 commit
- chore: Add example TRTLLM configs for Deepseek R1 (GB200) (#1099) · b6774b88
  Ryan McCormick authored May 15, 2025
  
  b6774b88
07 May, 2025 1 commit

build: Cleans the TensorRTLLM + Dynamo container build (#968) · 7dd79013

Tanmay Verma authored May 07, 2025


Signed-off-by: Tanmay Verma <tanmay2592@gmail.com>
Co-authored-by: Ryan McCormick <rmccormick@nvidia.com>

7dd79013

02 May, 2025 1 commit
- feat: Update to support completion endpoint in TRTLLM (#837) · 960ee927
  Tanmay Verma authored May 02, 2025
  
  960ee927
01 May, 2025 1 commit
- fix: add dedicated llmapi config for trtllm disagg kv routing example (#916) · 0086ebc6
  Ziqi Fan authored Apr 30, 2025
  
  0086ebc6
28 Apr, 2025 1 commit

fix: change the processor number to 5 to reduce the tokenization bottleneck (#865) · 6630fa5c

richardhuo-nv authored Apr 28, 2025

We were observing a 40% performance drop compared with trtllm serve when benchmarking with isl=1000 and osl=200 at a concurrency level > 128.

The number of the tokenization worker is the bottleneck. After bumping the tokenization processors number to 5, dynamo's benchmarking perf could match the trtllm serve's perf.

6630fa5c

11 Apr, 2025 1 commit

feat: TRT-LLM disaggregated serving using UCX (#562) · da38e96a

Tanmay Verma authored Apr 10, 2025


Signed-off-by: Tanmay Verma <tanmay2592@gmail.com>
Signed-off-by: Tanmay Verma <tanmayv@nvidia.com>
Co-authored-by: Neelay Shah <neelays@nvidia.com>

da38e96a

10 Apr, 2025 1 commit
- docs: update dynamo serve trtllm agg example yaml files (#600) · 34be4418
  Ziqi Fan authored Apr 10, 2025
  
  34be4418
08 Apr, 2025 1 commit

chore: Update TRTLLM version. Fix router. (#527) · 7dca64df

Tanmay Verma authored Apr 07, 2025


Signed-off-by: Tanmay Verma <tanmay2592@gmail.com>
Co-authored-by: Dmitry Tokarev <dtokarev@nvidia.com>

7dca64df

03 Apr, 2025 1 commit
- feat: Add TensorRT-LLM example for dynamo serve/run (#456) · 6eb10540
  Tanmay Verma authored Apr 03, 2025
```
Co-authored-by: Neelay Shah <neelays@nvidia.com>
```
  6eb10540