Commits · fca0a801dc8f0b91343dd702125de31290643f4f · OpenDAS / dynamo

"cacheflow/parallel_utils/tensor_parallel/utils.py" did not exist on "cfae35b861c5fc0c9f3689f99c7aba2e4501beb8"

11 Mar, 2026 2 commits

feat: follow up of optinal, batching-of-client-side-events (#6741) · fca0a801

Michael Feil authored Mar 11, 2026

Signed-off-by: michaelfeil <63565275+michaelfeil@users.noreply.github.com>
Signed-off-by: Michael Feil <michaelfeil@users.noreply.github.com>
Signed-off-by: Michael Feil <63565275+michaelfeil@users.noreply.github.com>

fca0a801

feat: GlobalPlanner --max-total-gpus for cluster-wide GPU budget (#7103) · 5d5fd243

daiyaanarfeen authored Mar 11, 2026


Signed-off-by: Daiyaan <darfeen@nvidia.com>
Signed-off-by: athreesh <anish.maddipoti@utexas.edu>
Signed-off-by: Anish <80174047+athreesh@users.noreply.github.com>
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-authored-by: athreesh <anish.maddipoti@utexas.edu>
Co-authored-by: Anish <80174047+athreesh@users.noreply.github.com>

5d5fd243

09 Mar, 2026 2 commits
- feat(kv-router): P2P recovery for standalone KV indexer [DYN-2367] (#6934) · 48eb52e7
  Yan Ru Pei authored Mar 09, 2026
```
Signed-off-by: PeaBrane <yanrpei@gmail.com>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
```
  48eb52e7
- fix(prefill-router): pass preselected_worker in sync prefill fallback path (#7055) · e96f71ba
  Yan Ru Pei authored Mar 09, 2026
```
Signed-off-by: PeaBrane <yanrpei@gmail.com>
```
  e96f71ba
06 Mar, 2026 1 commit
- fix: restore --enforce-disagg to reject requests before prefill router activates (#6957) · f934744f
  Yan Ru Pei authored Mar 05, 2026
```
Signed-off-by: PeaBrane <yanrpei@gmail.com>
```
  f934744f
04 Mar, 2026 2 commits
- chore(kv-router): split monolithic indexer.rs into indexer/ directory (#6870) · b94cb543
  Yan Ru Pei authored Mar 04, 2026
```
Signed-off-by: PeaBrane <yanrpei@gmail.com>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
```
  b94cb543
- feat(kv-indexer): multi-model and multi-tenant isolation (#6830) · 35f99f93
  Yan Ru Pei authored Mar 04, 2026
```
Signed-off-by: PeaBrane <yanrpei@gmail.com>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
```
  35f99f93
03 Mar, 2026 3 commits
- fix: properly setup and register vLLM worker for external / hybrid load... · 90d74637
  GuanLuo authored Mar 03, 2026
```
fix: properly setup and register vLLM worker for external / hybrid load balancing. Update launch script (#6695)
Signed-off-by: Guan Luo <41310872+GuanLuo@users.noreply.github.com>
```
  90d74637
- fix(kv-router): plumb expected_osl through scheduler queue path (#6812) · 8055b7dd
  Yan Ru Pei authored Mar 03, 2026
```
Signed-off-by: PeaBrane <yanrpei@gmail.com>
```
  8055b7dd
- feat(kv-router): add router queue depth Prometheus metric and nvext field (#6786) · 073fb437
  Yan Ru Pei authored Mar 02, 2026
```
Signed-off-by: PeaBrane <yanrpei@gmail.com>
```
  073fb437
02 Mar, 2026 2 commits
- feat: move standalone KV indexer into kv-router crate with HTTP integration tests (#6569) · 07742cc2
  Yan Ru Pei authored Mar 02, 2026
```
Signed-off-by: PeaBrane <yanrpei@gmail.com>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
```
  07742cc2
- docs: restructure docs directory and move fern config to fern/ (#6700) · ece08dc9
  Neal Vaidya authored Mar 01, 2026
```
Signed-off-by: Neal Vaidya <nealv@nvidia.com>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
```
  ece08dc9
28 Feb, 2026 2 commits
- feat: dynamic batching of client side events (#6733) · 713c96d2
  Michael Feil authored Feb 28, 2026
```
Signed-off-by: michaelfeil <63565275+michaelfeil@users.noreply.github.com>
```
  713c96d2
- feat(kv-router): add ActiveSequences benchmark and extract common bench utils (#6633) · e5850e23
  Yan Ru Pei authored Feb 27, 2026
```
Signed-off-by: PeaBrane <yanrpei@gmail.com>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
```
  e5850e23
27 Feb, 2026 2 commits
- feat: default router_event_threads to 4 (#6672) · 55c0a769
  Yan Ru Pei authored Feb 27, 2026
```
Signed-off-by: PeaBrane <yanrpei@gmail.com>
```
  55c0a769
- feat: wire nvext.cache_control TTL-based pinning through Dynamo router (#6213) · f1d5c95a
  ishandhanani authored Feb 26, 2026
```
Co-authored-by: Claude <noreply@anthropic.com>
```
  f1d5c95a
26 Feb, 2026 1 commit
- chore: move ActiveSequences to kv-router and add unit tests (#6600) · 3f99cf21
  Yan Ru Pei authored Feb 25, 2026
```
Signed-off-by: PeaBrane <yanrpei@gmail.com>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
```
  3f99cf21
25 Feb, 2026 3 commits
- chore: plumb allowed_worker_ids through RoutingHints (#6580) · ebaf048d
  Yan Ru Pei authored Feb 25, 2026
```
Signed-off-by: PeaBrane <yanrpei@gmail.com>
```
  ebaf048d
- fix: guarantee RouterRequestMetrics availability & documentation updates (#6558) · ff06b17e
  Keiven C authored Feb 25, 2026
```
Signed-off-by: Keiven Chang <keivenchang@users.noreply.github.com>
Co-authored-by: Keiven Chang <keivenchang@users.noreply.github.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
```
  ff06b17e
- feat: Support epp's "pods" interface in Dynamo fixes [DEP-424] (#6302) · c916cd42
  atchernych authored Feb 24, 2026
```
Signed-off-by: Anna Tchernych <atchernych@nvidia.com>
```
  c916cd42
24 Feb, 2026 9 commits
- feat: sglang to 0.5.9 + updated docs (#6518) · 6642e23e
  ishandhanani authored Feb 24, 2026
```
Co-authored-by: baihuitian <baihuitian.bht@gmail.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
```
  6642e23e
- feat: router metrics with dynamo_router_* {worker_id=...}. Update docs (#6227) · 1df620b4
  Keiven C authored Feb 24, 2026
```
Signed-off-by: Keiven Chang <keivenchang@users.noreply.github.com>
Co-authored-by: Keiven Chang <keivenchang@users.noreply.github.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
```
  1df620b4
- chore: de-async scheduler read paths and unblock decode output tracking (#6510) · efa89448
  Yan Ru Pei authored Feb 24, 2026
```
Signed-off-by: PeaBrane <yanrpei@gmail.com>
```
  efa89448
- chore: split JetStream subscriber into dedicated module and deprecate... · bc00ef38
  Yan Ru Pei authored Feb 24, 2026
```
chore: split JetStream subscriber into dedicated module and deprecate durable_kv_events [DYN-2203] (#6477)
Signed-off-by: PeaBrane <yanrpei@gmail.com>
```
  bc00ef38
- feat: make KV cache events and routing LoRA-aware (#6517) · 75fea787
  Biswa Panda authored Feb 24, 2026
  
  75fea787
- chore: replace --enforce-disagg with --decode-fallback, default to enforcing disagg (#6515) · c9ff6235
  Yan Ru Pei authored Feb 24, 2026
```
Signed-off-by: PeaBrane <yanrpei@gmail.com>
```
  c9ff6235
- feat: Standardized Dynamo Error Type (#6303) · c8276cd2
  Jacky authored Feb 23, 2026
```
Signed-off-by: Jacky <18255193+kthui@users.noreply.github.com>
```
  c8276cd2
- fix: fold scheduling into queue so backpressure actually works (#6470) · 0cb1d733
  Yan Ru Pei authored Feb 23, 2026
```
Signed-off-by: PeaBrane <yanrpei@gmail.com>
Signed-off-by: Yan Ru Pei <yanrpei@gmail.com>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
```
  0cb1d733
- feat: expose predicted KV hit rate as Prometheus histogram (#6507) · ddeee88b
  Yan Ru Pei authored Feb 23, 2026
```
Signed-off-by: Pea Brane <peabrane@peabrane.com>
Signed-off-by: PeaBrane <yanrpei@gmail.com>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
```
  ddeee88b
23 Feb, 2026 3 commits
- feat(runtime): add hierarchical Model/WorkerSet architecture for multi-namespace support (#6054) · cb55766c
  Thomas Montfort authored Feb 23, 2026
```
Signed-off-by: tmontfort <tmontfort@nvidia.com>
```
  cb55766c
- feat: add standalone KV indexer with query endpoint [DYN-2164] (#6446) · 038b50d2
  Yan Ru Pei authored Feb 23, 2026
```
Signed-off-by: PeaBrane <yanrpei@gmail.com>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
```
  038b50d2
- fix: derive kv-router discovery name from target worker component (#6475) · e66f3267
  Yan Ru Pei authored Feb 23, 2026
```
Signed-off-by: PeaBrane <yanrpei@gmail.com>
```
  e66f3267
19 Feb, 2026 3 commits
- chore: derive router_id and cancellation_token from component instead of plumbing (#6419) · 1549c338
  Yan Ru Pei authored Feb 19, 2026
```
Signed-off-by: PeaBrane <yanrpei@gmail.com>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
```
  1549c338
- chore: Remove ZmqKvEventListener binding and rework standalone TRT-LLM example... · fc229004
  Yan Ru Pei authored Feb 18, 2026
```
chore: Remove ZmqKvEventListener binding and rework standalone TRT-LLM example to use native Python ZMQ (#6164)
Signed-off-by: PeaBrane <yanrpei@gmail.com>
```
  fc229004
- refactor: move per-item stream tracking into RequestGuard (#6355) · c7986b35
  Yan Ru Pei authored Feb 18, 2026
```
Signed-off-by: PeaBrane <yanrpei@gmail.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
```
  c7986b35
14 Feb, 2026 1 commit

feat: mm aware routing for vllm (#6235) · 815b1291

zhongdaor-nv authored Feb 13, 2026


Signed-off-by: zhongdaor <zhongdaor@nvidia.com>
Signed-off-by: zhongdaor-nv <zhongdaor@nvidia.com>

815b1291

13 Feb, 2026 4 commits
- chore: otel tracings for routing overheads and metrics (#6194) · f6c34e07
  Yan Ru Pei authored Feb 13, 2026
```
Signed-off-by: PeaBrane <yanrpei@gmail.com>
```
  f6c34e07
- feat: Decomposed pipeline for EPP integration [DEP-730] (#5446) · 52271760
  atchernych authored Feb 12, 2026
```
Signed-off-by: Anna Tchernych <atchernych@nvidia.com>
```
  52271760
- feat: router priority queue (#6010) · bc514fbe
  Yan Ru Pei authored Feb 12, 2026
```
Signed-off-by: PeaBrane <yanrpei@gmail.com>
Co-authored-by: ishandhanani <82981111+ishandhanani@users.noreply.github.com>
```
  bc514fbe
- fix: to make sure free is always run on stream drop (#6246) · 4673e47f
  Yan Ru Pei authored Feb 12, 2026
```
Signed-off-by: PeaBrane <yanrpei@gmail.com>
```
  4673e47f