- 21 May, 2025 1 commit
-
-
Tanmay Verma authored
-
- 20 May, 2025 3 commits
-
-
julienmancuso authored
-
Tanmay Verma authored
-
Hongkuan Zhou authored
-
- 19 May, 2025 2 commits
-
-
Graham King authored
We can now do this: - Node 1: ``` dynamo-run in=http out=dyn ``` - Node 2 and 3, two instances of component 'backend' in the nemotron_ultra pipeline: ``` dynamo-run in=dyn://nemotron_ultra.backend.generate out=vllm /data/models/NemotronUltra ``` - Node 4 and 5, two instances of the 'backend' component in nemotron_super pipeline: ``` dynamo-run in=dyn://nemotron_super.backend.generate out=vllm /data/models/NemotronSuper ``` The ingress node will discover all four instances and route correctly. We have been planning for this for a long time now. As part of this auto-discovery is now always `out=dyn`, with no extra URL parts. Previously it could only route to a single pipeline. Also: - Refactor endpoint / instance naming now that I understand them - Fix removing models when their instance stops.
-
ishandhanani authored
-
- 16 May, 2025 3 commits
-
-
ptarasiewiczNV authored
-
Ryan McCormick authored
-
Biswa Panda authored
-
- 15 May, 2025 3 commits
-
-
mohammedabdulwahhab authored
-
Biswa Panda authored
-
Ryan McCormick authored
-
- 14 May, 2025 1 commit
-
-
ishandhanani authored
Co-authored-by:ishandhanani <ishandhananai@gmail.com>
-
- 12 May, 2025 2 commits
-
-
Hongkuan Zhou authored
-
Hongkuan Zhou authored
Co-authored-by:Biswa Panda <biswa.panda@gmail.com>
-
- 09 May, 2025 3 commits
-
-
ishandhanani authored
Signed-off-by:
ishandhanani <82981111+ishandhanani@users.noreply.github.com> Co-authored-by:
Ryan McCormick <rmccormick@nvidia.com>
-
Biswa Panda authored
-
ishandhanani authored
Co-authored-by:ishandhanani <ishandhananai@gmail.com>
-
- 08 May, 2025 3 commits
-
-
Hongkuan Zhou authored
-
julienmancuso authored
Co-authored-by:mohammedabdulwahhab <furkhan324@berkeley.edu>
-
Ryan McCormick authored
-
- 07 May, 2025 5 commits
-
-
Hongkuan Zhou authored
-
Kris Hung authored
-
Ryan McCormick authored
-
Tanmay Verma authored
Signed-off-by:
Tanmay Verma <tanmay2592@gmail.com> Co-authored-by:
Ryan McCormick <rmccormick@nvidia.com>
-
ptarasiewiczNV authored
-
- 06 May, 2025 2 commits
-
-
jthomson04 authored
-
hhzhang16 authored
-
- 05 May, 2025 2 commits
-
-
Hongkuan Zhou authored
-
Hongkuan Zhou authored
-
- 02 May, 2025 3 commits
-
-
Tanmay Verma authored
-
Ryan McCormick authored
-
Kris Hung authored
-
- 01 May, 2025 2 commits
-
-
Yan Ru Pei authored
-
Ziqi Fan authored
-
- 30 Apr, 2025 3 commits
-
-
Biswa Panda authored
-
ishandhanani authored
-
Yan Ru Pei authored
-
- 29 Apr, 2025 2 commits
-
-
mohammedabdulwahhab authored
Signed-off-by:
mohammedabdulwahhab <furkhan324@berkeley.edu> Co-authored-by:
hhzhang16 <54051230+hhzhang16@users.noreply.github.com>
-
Ziqi Fan authored
refactor: change trtllm example kv routing use python bindings | deal with trtllm partial blocks | trtllm event change (#866)
-