Commits · efe82b86f63a8a6315beef0a882356deda2f3200 · OpenDAS / dynamo

09 Mar, 2025 3 commits

feat: make vllm baseline support both chat and completions (#70) · efe82b86
Alec authored Mar 09, 2025

efe82b86

chore: left over renaming (#67) · 678cffb4

Neelay Shah authored Mar 09, 2025


Co-authored-by: Harrison Saturley-Hall <454891+saturley-hall@users.noreply.github.com>
Co-authored-by: Harrison King Saturley-Hall <hsaturleyhal@nvidia.com>

678cffb4

feat: kv aware router + disagg router + prefill queue (#11) · 19844fc0

Hongkuan Zhou authored Mar 08, 2025


Signed-off-by: Hongkuan Zhou <tedzhouhk@gmail.com>
Co-authored-by: hongkuan <hongkuanz@nvidia.com>
Co-authored-by: Piotr Tarasiewicz <ptarasiewicz@nvidia.com>
Co-authored-by: Piotr Tarasiewicz Nvidia <ptarasiewicznv@Piotrs-MacBook-Pro.local>
Co-authored-by: alec-flowers <aflowers@nvidia.com>
Co-authored-by: Neelay Shah <neelays@nvidia.com>

19844fc0

08 Mar, 2025 5 commits
- ci: rename project to dynamo (#60) · 61abae51
  Harrison Saturley-Hall authored Mar 08, 2025
  
  61abae51
- chore: Renamed Triton Distributed to Dynamo (#56) · b4d56a57
  Dmitry Tokarev authored Mar 08, 2025
  
  b4d56a57
- chore: remove debug statements (#57) · dd7646ef
  Neelay Shah authored Mar 08, 2025
  
  dd7646ef
- chore: rename dynamo (#44) · 602352ce
  Neelay Shah authored Mar 08, 2025
```
Co-authored-by: Biswa Panda <biswa.panda@gmail.com>
```
  602352ce
- test: add tests for kv bindings (#35) · dcecc47d
  GuanLuo authored Mar 07, 2025
  
  dcecc47d
07 Mar, 2025 1 commit
- feat: onboard dynamo-sdk basic and kv-router examples (#20) · aacc5d76
  Biswa Panda authored Mar 07, 2025
```
Co-authored-by: Neelay Shah <neelays@nvidia.com>
```
  aacc5d76
06 Mar, 2025 3 commits
- feat: Use round robin for disagg routing (#40) · 3c60fe2a
  ptarasiewiczNV authored Mar 07, 2025
```
Co-authored-by: ptarasiewicz@nvidia.com <Piotr Tarasiewicz>
```
  3c60fe2a
- feat: Add estimated kv cache hit metric events (#30) · 09656f6c
  Ryan McCormick authored Mar 06, 2025
  
  09656f6c
- feat: expose KV routing components for easier router customization (#15) · e159e53f
  GuanLuo authored Mar 05, 2025
  
  e159e53f
05 Mar, 2025 2 commits

refactor: rename triton_distributed to dynemo (#22) · 1af7433b
Neelay Shah authored Mar 05, 2025
```
Co-authored-by: Graham King <grahamk@nvidia.com>
```
1af7433b

feat: OAI compatible endpoints for TRTLLM (#14) · 2791b9ea

NVShreyas authored Mar 04, 2025


Co-authored-by: Tanmay Verma <tanmayv@nvidia.com>
Co-authored-by: Tanmay Verma <tanmayv@login-eos01.eos.clusters.nvidia.com>
Co-authored-by: Tanmay Verma <tanmay2592@gmail.com>
Co-authored-by: Ryan McCormick <rmccormick@nvidia.com>

2791b9ea

04 Mar, 2025 3 commits

feat: add python binding for rust llm modules (#13) · a32cdad6
Biswa Panda authored Mar 04, 2025

a32cdad6

feat: nixl metadata store and retrieved from etcd (#6) · 3a5fe17d

Neelay Shah authored Mar 04, 2025

Co-authored-by: hongkuanz <hongkuanz@nvidia.com>
Co-authored-by: Piotr Tarasiewicz <ptarasiewicz@nvidia.com>
Co-authored-by: Piotr Tarasiewicz Nvidia <ptarasiewicznv@Piotrs-MacBook-Pro.local>
Co-authored-by: Neelay Shah <neelays@ipp2-0493.ipp2u1.colossus.nvidia.com>
Co-authored-by: Neelay Shah <neelays@ipp1-1941.ipp1a1.colossus.nvidia.com>
Co-authored-by: ishandhanani <ishandhanani@gmail.com>
Co-authored-by: Neelay Shah <neelays@4u8g-gen-0078.ipp3a2.colossus.nvidia.com>
Co-authored-by: ptarasiewiczNV <104908264+ptarasiewiczNV@users.noreply.github.com>

3a5fe17d

Update NIXL Dockerfile + vLLM patch with variable TP (#9) · fe83f8aa
ptarasiewiczNV authored Mar 04, 2025
```
Co-authored-by: Piotr Tarasiewicz Nvidia <ptarasiewicznv@Piotrs-MacBook-Pro.local>
```
fe83f8aa

02 Mar, 2025 1 commit
- chore: Update README.md · 3f12b570
  Neelay Shah authored Mar 01, 2025
```
Signed-off-by: Neelay Shah <neelays@nvidia.com>
```
  3f12b570
01 Mar, 2025 1 commit
- Add lazy import to nixl.py · 5f14e467
  Piotr Marcinkiewicz authored Mar 01, 2025
  
  5f14e467
28 Feb, 2025 3 commits
- [fix] KV Router Example fixes (#314) · 11a36651
  Alec authored Feb 28, 2025
```
Co-authored-by: Ryan McCormick <rmccormick@nvidia.com>
```
  11a36651
- Add missing inits (#307) · f229b41b
  Piotr Marcinkiewicz authored Feb 28, 2025
  
  f229b41b
- Updates to support DS R1 in TRTLLM example (#301) · 12d73a82
  NVShreyas authored Feb 28, 2025
  
  12d73a82
27 Feb, 2025 3 commits

feat: vLLM + NIXL example · f7a60cba

ptarasiewiczNV authored Feb 27, 2025


Co-authored-by: Piotr Tarasiewicz Nvidia <ptarasiewicznv@Piotrs-MacBook-Pro.local>
Co-authored-by: nnshah1 <neelays@nvidia.com>
Co-authored-by: alec-flowers <aflowers@nvidia.com>

f7a60cba

feat: LLM API example integration (#182) · 4b42b232
Tanmay Verma authored Feb 27, 2025
```
Co-authored-by: NVShreyas <158103197+NVShreyas@users.noreply.github.com>
```
4b42b232
feat: Add HTTP completions endpoint to kv router example (#258) · 03d0f6a2
Sean SH Choi authored Feb 26, 2025
```
Co-authored-by: Alec <35311602+alec-flowers@users.noreply.github.com>
```
03d0f6a2

25 Feb, 2025 2 commits

feat: enable metrics polling · 861c5098

GuanLuo authored Feb 25, 2025

Signed-off-by: Meenakshi Sharma <163925564+nvda-mesharma@users.noreply.github.com>
Signed-off-by: Neelay Shah <neelays@nvidia.com>
Co-authored-by: Neelay Shah <neelays@nvidia.com>
Co-authored-by: Ryan Olson <ryanolson@users.noreply.github.com>
Co-authored-by: Meenakshi Sharma <163925564+nvda-mesharma@users.noreply.github.com>
Co-authored-by: Biswa Panda <biswapanda@users.noreply.github.com>
Co-authored-by: Ryan McCormick <rmccormick@nvidia.com>

861c5098

refactor: move libs to lib dir · 08fcd7e9

Neelay Shah authored Feb 24, 2025


Signed-off-by: Neelay Shah <neelays@nvidia.com>
Co-authored-by: Ryan McCormick <rmccormick@nvidia.com>

08fcd7e9

22 Feb, 2025 1 commit
- [fix] initialize vLLM engine before runtime (#238) · ccd153af
  Alec authored Feb 21, 2025
```
Co-authored-by: hongkuanz <hongkuanz@nvidia.com>
```
  ccd153af
21 Feb, 2025 2 commits
- build: Multi-stage VLLM Build (#225) · 6e0ccccb
  Ryan McCormick authored Feb 21, 2025
  
  6e0ccccb
- feat: Add http support to vllm kv router example (#217) · 15de1807
  Alec authored Feb 21, 2025
```
Co-authored-by: Sean Choi <choishsean@gmail.com>
Co-authored-by: aflowers <aflowers@nvidia.com>
```
  15de1807
20 Feb, 2025 2 commits

feat: add cli args for example http service (#221) · 73c10ae9
Biswa Panda authored Feb 20, 2025
```
Co-authored-by: Biswa Ranjan Panda <biswaranjanp@nvidia.com>
```
73c10ae9

feat: use vllm out of process engine · 60a73634

ptarasiewiczNV authored Feb 20, 2025


Signed-off-by: Piotr Marcinkiewicz <piotrm@nvidia.com>
Co-authored-by: Piotr Marcinkiewicz <piotrm@nvidia.com>
Co-authored-by: Ryan McCormick <rmccormick@nvidia.com>

60a73634

18 Feb, 2025 3 commits

feat: add openai endpoint to the vllm example (#183) · 9e4a548d

ptarasiewiczNV authored Feb 18, 2025


Co-authored-by: Ryan Olson <rolson@nvidia.com>
Co-authored-by: Ryan McCormick <rmccormick@nvidia.com>

9e4a548d

feat: Add KV publisher and receiver. Add KV aware routing example. · 8588e33a

GuanLuo authored Feb 18, 2025


Signed-off-by: Neelay Shah <neelays@nvidia.com>
Co-authored-by: aflowers <aflowers@nvidia.com>
Co-authored-by: Ryan McCormick <rmccormick@nvidia.com>
Co-authored-by: hongkuanz <hongkuanz@nvidia.com>
Co-authored-by: Neelay Shah <neelays@nvidia.com>

8588e33a

feat: Add Dockerfile.vllm (#184) · d8aada0b
ptarasiewiczNV authored Feb 18, 2025

d8aada0b

13 Feb, 2025 1 commit
- feat: vLLM v0.7.2 patch supporting XpYd and heterogeneous P/D parallel configs (#167) · f38aa469
  ptarasiewiczNV authored Feb 13, 2025
```
Co-authored-by: Ryan McCormick <rmccormick@nvidia.com>
Co-authored-by: Neelay Shah <neelays@nvidia.com>
```
  f38aa469
08 Feb, 2025 1 commit

refactor: Modify rust vllm example to use container · d2e7ae02

Ryan McCormick authored Feb 07, 2025


Co-authored-by: Piotr Tarasiewicz <ptarasiewicz@nvidia.com>
Co-authored-by: Neelay Shah <neelays@nvidia.com>

d2e7ae02

07 Feb, 2025 1 commit
- feat: vLLM DistributedRuntime Monolith and Disagg Workers Example · 45bc426c
  ptarasiewiczNV authored Feb 08, 2025
```
Co-authored-by: Neelay Shah <neelays@nvidia.com>
Co-authored-by: Ryan McCormick <rmccormick@nvidia.com>
```
  45bc426c