Commits · 70e1ec1de81eb7db181730090586f9a36b397ee9 · OpenDAS / dynamo

25 Mar, 2026 5 commits
- perf: Miscellaneous router perf improvements (#7477) · 1368ccd6
  jthomson04 authored Mar 24, 2026
```
Signed-off-by: jthomson04 <jwillthomson19@gmail.com>
```
  1368ccd6
- fix(frontend): carry forward real stream metadata in JailedStream finalization (#7608) · 0adfd98d
  vvagias authored Mar 24, 2026
  
  0adfd98d
- feat: add power-of-two-choices router mode (#7614) · db49dfd2
  Aaron Batilo authored Mar 24, 2026
```
Signed-off-by: Aaron Batilo <abatilo@coreweave.com>
```
  db49dfd2
- fix: sglang eagle bigram tokens kv event report. (#6872) · 2fe37a51
  huitian bai authored Mar 25, 2026
```
Signed-off-by: baihuitian <baihuitian.bht@gmail.com>
Signed-off-by: PeaBrane <yanrpei@gmail.com>
Co-authored-by: PeaBrane <yanrpei@gmail.com>
```
  2fe37a51
- fix: Request Rejection busy signal passage (#7615) · db79f324
  Jacky authored Mar 24, 2026
```
Signed-off-by: Jacky <18255193+kthui@users.noreply.github.com>
```
  db79f324
24 Mar, 2026 3 commits
- chore(kv-router): share recovery cursor state (#7596) · 7389a369
  Yan Ru Pei authored Mar 24, 2026
```
Signed-off-by: PeaBrane <yanrpei@gmail.com>
```
  7389a369
- feat(router): add best_worker indexer update flag (#7603) · 149bf5a2
  Yan Ru Pei authored Mar 24, 2026
```
Signed-off-by: PeaBrane <yanrpei@gmail.com>
```
  149bf5a2
- feat(replay): add shared loadgen workload paths [DYN-2510] (#7593) · b2c59aa4
  Yan Ru Pei authored Mar 24, 2026
```
Signed-off-by: PeaBrane <yanrpei@gmail.com>
```
  b2c59aa4
23 Mar, 2026 3 commits
- feat: Frontend and Runtime Request Cancellation Metrics (#7493) · e252f82d
  Jacky authored Mar 23, 2026
```
Signed-off-by: Jacky <18255193+kthui@users.noreply.github.com>
```
  e252f82d
- feat(mocker): add multi-worker replay and router startup fixes (#7553) · b7fe46b1
  Yan Ru Pei authored Mar 23, 2026
```
Signed-off-by: PeaBrane <yanrpei@gmail.com>
```
  b7fe46b1
- feat(perf): add pipeline instrumentation and feature gated NVTX markers (#6746) · 82794761
  Biswa Panda authored Mar 23, 2026
  
  82794761
20 Mar, 2026 2 commits
- feat(mocker): add offline trace replay mode [DYN-2502] (#7543) · c3908a36
  Yan Ru Pei authored Mar 20, 2026
```
Signed-off-by: PeaBrane <yanrpei@gmail.com>
```
  c3908a36
- feat(llm): unify user-facing priority hints (#7492) · ebd06a1f
  Yan Ru Pei authored Mar 20, 2026
```
Signed-off-by: PeaBrane <yanrpei@gmail.com>
```
  ebd06a1f
19 Mar, 2026 1 commit
- feat: make the kvbm v1's MAX_CONCURRENT_TRANSFERS and MAX_TRANSFER_BATCH_SIZE... · e7bcf651
  Richard Huo authored Mar 19, 2026
```
feat: make the kvbm v1's MAX_CONCURRENT_TRANSFERS and MAX_TRANSFER_BATCH_SIZE configurable from env var (#7527)
```
  e7bcf651
17 Mar, 2026 8 commits
- feat(mocker): add SGLang engine simulation (#6977) · 930721c8
  Yongming Ding authored Mar 17, 2026
```
Signed-off-by: Yongming Ding <yongmingd@nvidia.com>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
```
  930721c8
- chore(kv-router): drop easy llm facade reexports (#7474) · 45be2fdc
  Yan Ru Pei authored Mar 17, 2026
```
Signed-off-by: PeaBrane <yanrpei@gmail.com>
```
  45be2fdc
- feat: enable kvbm v1 to use the v2 ::new_with_device (#7151) · 5b6624bd
  Ryan Olson authored Mar 17, 2026
```
Signed-off-by: Ryan Olson <rolson@nvidia.com>
```
  5b6624bd
- feat(kv-router): plumb medium into worker-side placements (#7462) · e811db50
  Yan Ru Pei authored Mar 17, 2026
```
Signed-off-by: PeaBrane <yanrpei@gmail.com>
```
  e811db50
- feat: Concurrent KV event consumer (#7293) · 7ff5e0be
  jthomson04 authored Mar 17, 2026
```
Signed-off-by: jthomson04 <jwillthomson19@gmail.com>
```
  7ff5e0be
- fix(kv-router): recover initial standalone-indexer ZMQ gaps (#7463) · 0f8e1a9e
  Yan Ru Pei authored Mar 17, 2026
```
Signed-off-by: PeaBrane <yanrpei@gmail.com>
```
  0f8e1a9e
- fix: Improve active sequences request expiration (#7340) · 1e261dbd
  blarson-b10 authored Mar 16, 2026
```
Signed-off-by: Brian Larson <brian.larson@baseten.co>
```
  1e261dbd
- feat: Enable EPP worker discovery mode (#6592) · a441aaf8
  atchernych authored Mar 16, 2026
```
Signed-off-by: Anna Tchernych <atchernych@nvidia.com>
```
  a441aaf8
15 Mar, 2026 2 commits
- feat: integrate fastokens BPE tokenizer backend (#7387) · da810a26
  Biswa Panda authored Mar 15, 2026
  
  da810a26
- chore(kv-router): remove native kv-indexer binary, use maturin-built one (#7338) · b6596c52
  Yan Ru Pei authored Mar 15, 2026
```
Signed-off-by: PeaBrane <yanrpei@gmail.com>
```
  b6596c52
14 Mar, 2026 1 commit
- feat: standalone KV indexer runtime integration (#7295) · bba70a41
  Thomas Montfort authored Mar 14, 2026
  
  bba70a41
13 Mar, 2026 5 commits
- fix: populate logprobs bytes and token fields in OpenAI-compatible responses (#6953) · 947939c7
  Biswa Panda authored Mar 13, 2026
  
  947939c7
- fix: Upgrade futures-util to 0.3.32 (#7346) · e6cb6847
  Graham King authored Mar 13, 2026
```
Signed-off-by: Graham King <grahamk@nvidia.com>
```
  e6cb6847
- feat(kv-router): pluggable scheduling policy for router queue [DYN-2454] (#7260) · bddaaa26
  Yan Ru Pei authored Mar 12, 2026
```
Signed-off-by: PeaBrane <yanrpei@gmail.com>
```
  bddaaa26
- fix: harden numa cpuset lookup (#7060) · 17db1b6a
  Ryan Olson authored Mar 12, 2026
```
Signed-off-by: Ryan Olson <rolson@nvidia.com>
```
  17db1b6a
- fix: cancellation kv commit router (#7178) · d170d56a
  Michael Feil authored Mar 12, 2026
```
Signed-off-by: michaelfeil <63565275+michaelfeil@users.noreply.github.com>
```
  d170d56a
12 Mar, 2026 6 commits
- feat(kv-router): add worker_type to router selection log (#7258) · dcbccbcd
  Yan Ru Pei authored Mar 12, 2026
```
Signed-off-by: PeaBrane <yanrpei@gmail.com>
```
  dcbccbcd
- feat: ForwardPassMetrics dynamo event plane integration (#7250) · cd4773fb
  Hongkuan Zhou authored Mar 12, 2026
```
Signed-off-by: hongkuanz <hongkuanz@nvidia.com>
```
  cd4773fb
- feat(kv-router): ZMQ gap detection + replay for standalone indexer [LLM-126] (#7209) · bb07b2f4
  Yan Ru Pei authored Mar 12, 2026
```
Signed-off-by: PeaBrane <yanrpei@gmail.com>
```
  bb07b2f4
- feat: support multimodal (vision) inputs in Anthropic Messages API (#7256) · ba52c8ff
  MatejKosec authored Mar 12, 2026
```
Signed-off-by: Matej Kosec <mkosec@nvidia.com>
```
  ba52c8ff
- feat: vllm omni image to video support (#6530) · 1182e207
  Ayush Agarwal authored Mar 12, 2026
```
Signed-off-by: ayushag <ayushag@nvidia.com>
```
  1182e207
- fix: respect per-request chat_template_kwargs for DeepSeek V3.2 thinking mode (#7286) · 745d7e4c
  brluo authored Mar 12, 2026
```
Co-authored-by: kangclzjc <kangz@nvidia.com>
```
  745d7e4c
11 Mar, 2026 4 commits

feat: follow up of optinal, batching-of-client-side-events (#6741) · fca0a801

Michael Feil authored Mar 11, 2026

Signed-off-by: michaelfeil <63565275+michaelfeil@users.noreply.github.com>
Signed-off-by: Michael Feil <michaelfeil@users.noreply.github.com>
Signed-off-by: Michael Feil <63565275+michaelfeil@users.noreply.github.com>

fca0a801

feat: streaming tool call and reasoning dispatch SSE events (#7114) · 5178a4a4
MatejKosec authored Mar 11, 2026
```
Signed-off-by: Matej Kosec <mkosec@nvidia.com>
```
5178a4a4

feat: GlobalPlanner --max-total-gpus for cluster-wide GPU budget (#7103) · 5d5fd243

daiyaanarfeen authored Mar 11, 2026


Signed-off-by: Daiyaan <darfeen@nvidia.com>
Signed-off-by: athreesh <anish.maddipoti@utexas.edu>
Signed-off-by: Anish <80174047+athreesh@users.noreply.github.com>
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-authored-by: athreesh <anish.maddipoti@utexas.edu>
Co-authored-by: Anish <80174047+athreesh@users.noreply.github.com>

5d5fd243

fix: add force_nonempty_content for nemotron (#7225) · c95bfc2e
Neal Vaidya authored Mar 11, 2026
```
Signed-off-by: Neal Vaidya <nealv@nvidia.com>
```
c95bfc2e