Commits · 7eee891aafd3bd82d069f3594a5b96674b6cc074 · OpenDAS / dynamo · GitLab

20 Feb, 2025 1 commit

feat: use vllm out of process engine · 60a73634

ptarasiewiczNV authored Feb 20, 2025


Signed-off-by: Piotr Marcinkiewicz <piotrm@nvidia.com>
Co-authored-by: Piotr Marcinkiewicz <piotrm@nvidia.com>
Co-authored-by: Ryan McCormick <rmccormick@nvidia.com>

60a73634

18 Feb, 2025 5 commits
- feat: add openai endpoint to the vllm example (#183) · 9e4a548d
  ptarasiewiczNV authored Feb 18, 2025
```
Co-authored-by: Ryan Olson <rolson@nvidia.com>
Co-authored-by: Ryan McCormick <rmccormick@nvidia.com>
```
  9e4a548d
- [fix] add kv routing to vllm dockerfile · 9c9f0086
  aflowers authored Feb 18, 2025
  
  9c9f0086
- fix: Remove duplicate venv setup in Dockerfile (#205) · 94601e77
  Ryan McCormick authored Feb 18, 2025
  
  94601e77
- feat: Add KV publisher and receiver. Add KV aware routing example. · 8588e33a
  GuanLuo authored Feb 18, 2025
```
Signed-off-by: Neelay Shah <neelays@nvidia.com>
Co-authored-by: aflowers <aflowers@nvidia.com>
Co-authored-by: Ryan McCormick <rmccormick@nvidia.com>
Co-authored-by: hongkuanz <hongkuanz@nvidia.com>
Co-authored-by: Neelay Shah <neelays@nvidia.com>
```
  8588e33a
- feat: Add Dockerfile.vllm (#184) · d8aada0b
  ptarasiewiczNV authored Feb 18, 2025
  
  d8aada0b
17 Feb, 2025 1 commit
- feat: disaggregation support for deepseek arch models (#196) · 9a7b506c
  ptarasiewiczNV authored Feb 17, 2025
  
  9a7b506c
14 Feb, 2025 1 commit

feat: Add event plane · 6d2abdba

Blazej authored Feb 14, 2025


Signed-off-by: Piotr Marcinkiewicz <piotrm@nvidia.com>
Co-authored-by: Piotr Marcinkiewicz <piotrm@nvidia.com>
Co-authored-by: Neelay Shah <neelays@nvidia.com>

6d2abdba

13 Feb, 2025 1 commit
- feat: vLLM v0.7.2 patch supporting XpYd and heterogeneous P/D parallel configs (#167) · f38aa469
  ptarasiewiczNV authored Feb 13, 2025
```
Co-authored-by: Ryan McCormick <rmccormick@nvidia.com>
Co-authored-by: Neelay Shah <neelays@nvidia.com>
```
  f38aa469
12 Feb, 2025 1 commit
- fix: Fix TRTLLM Backend rebuild · c2a9636c
  Tanmay Verma authored Feb 12, 2025
  
  c2a9636c
11 Feb, 2025 2 commits
- chore: Add triton_distributed_rs wheel install to container build (#135) · 139a9a83
  Ryan McCormick authored Feb 11, 2025
  
  139a9a83
- chore: Update base from from 24.12 to 25.01 Triton version (#100) · 20b36843
  Tanmay Verma authored Feb 10, 2025
  
  20b36843
08 Feb, 2025 2 commits
- refactor: Modify rust vllm example to use container · d2e7ae02
  Ryan McCormick authored Feb 07, 2025
```
Co-authored-by: Piotr Tarasiewicz <ptarasiewicz@nvidia.com>
Co-authored-by: Neelay Shah <neelays@nvidia.com>
```
  d2e7ae02
- build: Support rebuilding and replacing trtllm backend (#99) · 2cfb1b6d
  Tanmay Verma authored Feb 07, 2025
  
  2cfb1b6d
06 Feb, 2025 1 commit
- fix: Set default 'latest' tag if git tags fail to get parsed (#118) · 7894967e
  Ryan McCormick authored Feb 05, 2025
  
  7894967e
05 Feb, 2025 1 commit
- ci: Add Copyright Verification Scripts w/ Automation (#110) · c9130f8f
  J Wyman authored Feb 05, 2025
  
  c9130f8f
04 Feb, 2025 3 commits
- build: pip based installation of icp and runtime. Also make tritonserver optional · ccfe101e
  Neelay Shah authored Feb 04, 2025
```
Signed-off-by: Neelay Shah <neelays@nvidia.com>
Co-authored-by: Ryan McCormick <rmccormick@nvidia.com>
```
  ccfe101e
- ci: add rust build and pre-merge checks (#91) · 5701753a
  Anant Sharma authored Feb 04, 2025
  
  5701753a
- chore: updating to mv src directory · b34c6f77
  Neelay Shah authored Feb 03, 2025
  
  b34c6f77
03 Feb, 2025 1 commit
- refactor: worker rename to runtime · 4017bd18
  Neelay Shah authored Feb 03, 2025
  
  4017bd18
01 Feb, 2025 2 commits

docs: Add Llama 70B benchmark reproduction and results · 04e50aba

ptarasiewiczNV authored Feb 01, 2025


Signed-off-by: Neelay Shah <neelays@nvidia.com>
Co-authored-by: Neelay Shah <neelays@nvidia.com>
Co-authored-by: Piotr Marcinkiewicz <piotrm@nvidia.com>
Co-authored-by: Ryan McCormick <rmccormick@nvidia.com>

04e50aba

feat(llm): adding initial TRTLLM disaggregation support · deb6c7e8

ishandhanani authored Feb 01, 2025


Co-authored-by: nnshah1 <neelays@nvidia.com>
Co-authored-by: Ryan McCormick <rmccormick@nvidia.com>

deb6c7e8

25 Jan, 2025 1 commit
- chore: Update build version to v0.1.1 and add Teardown section to LLM example README (#66) · 1c1bd7da
  Ryan McCormick authored Jan 24, 2025
  
  1c1bd7da
24 Jan, 2025 1 commit
- feat: Add vLLM workers to LLM example (#41) · f6d723f6
  Blazej authored Jan 24, 2025
```
Add example of LLM disaggregated serving
```
  f6d723f6
23 Jan, 2025 1 commit

refactor: openai server skeleton · 607fac29

Neelay Shah authored Jan 22, 2025


Co-authored-by: rmccorm4 <21284872+rmccorm4@users.noreply.github.com>
Co-authored-by: piotrm-nvidia <62554872+piotrm-nvidia@users.noreply.github.com>

607fac29

22 Jan, 2025 1 commit
- build: update isort known first party config (#50) · c3b84790
  Anant Sharma authored Jan 22, 2025
  
  c3b84790
17 Jan, 2025 1 commit

feat: hello world · 4698c0f4

Neelay Shah authored Jan 17, 2025


Co-authored-by: Piotr Marcinkiewicz <piotrm@nvidia.com>
Co-authored-by: Tanmay Verma <tanmay2592@gmail.com>

4698c0f4

16 Jan, 2025 1 commit
- test: Add sanity test for vllm.distributed patched modules · 7af0d2ef
  Ryan McCormick authored Jan 15, 2025
  
  7af0d2ef
15 Jan, 2025 3 commits
- build: Add vllm wheel patch scripts for disaggregated serving · a0e1da03
  Ryan McCormick authored Jan 15, 2025
```
---------
Co-authored-by: Piotr Marcinkiewicz <piotrm@nvidia.com>
```
  a0e1da03
- build: adding framework as environment variable. Can use to select tests. · 587addb9
  Neelay Shah authored Jan 14, 2025
  
  587addb9
- feat: initial worker · f1f29171
  Neelay Shah authored Jan 14, 2025
  
  f1f29171
14 Jan, 2025 1 commit
- ci: updating run to enable leaving container image · 15529df7
  Neelay Shah authored Jan 13, 2025
  
  15529df7
13 Jan, 2025 1 commit
- feat: adding initial icp · fd42de29
  Neelay Shah authored Jan 13, 2025
  
  fd42de29
11 Jan, 2025 1 commit
- feat: container build and run enhancements · f753fffc
  Neelay Shah authored Jan 10, 2025
  
  f753fffc
10 Jan, 2025 1 commit
- feat: Add ICP Protos and Python Stub Generation · 1d69e2cb
  Neelay Shah authored Jan 09, 2025
  
  1d69e2cb
09 Jan, 2025 3 commits
- fix: update copyright year · 70699987
  Neelay Shah authored Jan 09, 2025
```
updating to reflect correct years
```
  70699987
- chore: add pre-commit and additional boilerplate files · df90e29e
  Neelay Shah authored Jan 09, 2025
  
  df90e29e
- updating copyright and license header to match guidance · 61af664b
  nnshah1 authored Jan 08, 2025
  
  61af664b
08 Jan, 2025 2 commits
- fix: remove pinned starlette version · 65d784e8
  Neelay Shah authored Jan 08, 2025
  
  65d784e8
- build: adding in container build scripts · d39695b7
  Neelay Shah authored Jan 08, 2025
  
  d39695b7