Commits · e3d00b899f0c8a417cdfc45106c94bba5e3836a3 · OpenDAS / dynamo

23 Apr, 2026 1 commit

feat: Tier-based KV Routing (#8380) · e3d00b89

jthomson04 authored Apr 23, 2026


Signed-off-by: jthomson04 <jwillthomson19@gmail.com>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

e3d00b89

22 Apr, 2026 4 commits
- fix: Remove double tokenization in EPP Integration fixes [DYN-2076] (#8093) · a2a2753d
  atchernych authored Apr 22, 2026
```
Signed-off-by: Anna Tchernych <atchernych@nvidia.com>
```
  a2a2753d
- feat: Interface for shared kv cache handling in kv routing (#7536) · b441e26a
  jthomson04 authored Apr 22, 2026
```
Signed-off-by: jthomson04 <jwillthomson19@gmail.com>
Co-authored-by: Ishan Dhanani <ishandhanani@gmail.com>
Co-authored-by: ishandhanani <82981111+ishandhanani@users.noreply.github.com>
```
  b441e26a
- fix(responses): align wire shape with OpenResponses spec + add compliance CI (#8283) · 2a95ef63
  ishandhanani authored Apr 22, 2026
```
Signed-off-by: Anant Sharma <anants@nvidia.com>
Co-authored-by: Anant Sharma <anants@nvidia.com>
```
  2a95ef63
- fix: DIS-1768 emit structured SSE error + [DONE] on mid-stream fault (#8430) · f53fa64c
  Keiven C authored Apr 22, 2026
```
Signed-off-by: Keiven Chang <keivenchang@users.noreply.github.com>
```
  f53fa64c
21 Apr, 2026 4 commits
- fix(discovery): send ModelUpdate::Added after add_worker_set completes (#8242) · 7ffc699f
  MatejKosec authored Apr 21, 2026
```
Signed-off-by: Matej Kosec <mkosec@nvidia.com>
```
  7ffc699f
- feat(lpu): add opaque engine_data field to nvext response with per-request opt-in via extra_fields · 1c9412d2
  Biswa Panda authored Apr 21, 2026
  
  1c9412d2
- chore: upgrade modelexpress version (#8336) · 9514236c
  Nate Mailhot authored Apr 21, 2026
```
Signed-off-by: Nate Mailhot <nmailhot@nvidia.com>
Co-authored-by: Dmitry Tokarev <dtokarev@nvidia.com>
```
  9514236c
- feat(kv-router): add start positions to flat KV events (#8426) · 4b3cd459
  Yan Ru Pei authored Apr 21, 2026
```
Signed-off-by: PeaBrane <yanrpei@gmail.com>
```
  4b3cd459
20 Apr, 2026 6 commits
- fix(parsers): recover Kimi K2 tool calls when section_end is missing (#8208) · 47cfbd46
  Keiven C authored Apr 20, 2026
```
Signed-off-by: Keiven Chang <keivenchang@users.noreply.github.com>
```
  47cfbd46
- feat: (Router) Guard threshold-based rejection logic when CLI thresholds are unset (#8333) · adfc02d5
  Karen Chung authored Apr 20, 2026
  
  adfc02d5
- fix(protocols): always serialize chat message content (null when absent) (#8372) · 846ce71a
  Ryan McCormick authored Apr 20, 2026
```
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
```
  846ce71a
- fix(responses): own input chain in dynamo-protocols to accept Codex/Agents-SDK shapes (#8275) · 4ef28940
  ishandhanani authored Apr 20, 2026
  
  4ef28940
- feat(frontend): gate nvext response metadata behind extra_fields (#8252) · f437c8cf
  Biswa Panda authored Apr 20, 2026
```
Co-authored-by: AmeenP <ameenp360@gmail.com>
```
  f437c8cf
- feat: add staged frontend gauges (#8162) · 0c98d9a1
  Jie Hao authored Apr 20, 2026
  
  0c98d9a1
17 Apr, 2026 4 commits
- chore(kv-router): make sequences stop doing token math (#8260) · d94b350d
  Yan Ru Pei authored Apr 17, 2026
```
Signed-off-by: PeaBrane <yanrpei@gmail.com>
```
  d94b350d
- fix(discovery): address concurrency edge cases in handle_put lifecycle (#8237) · 7f40761b
  MatejKosec authored Apr 17, 2026
```
Signed-off-by: Matej Kosec <mkosec@nvidia.com>
```
  7f40761b
- perf(discovery): replace polling with Notify in concurrent registration wait (#8291) · f05cabac
  MatejKosec authored Apr 17, 2026
```
Signed-off-by: Matej Kosec <mkosec@nvidia.com>
```
  f05cabac
- fix(jail): preserve logprobs through tool-call jailing (#8072) · 59d18d8e
  jthomson04 authored Apr 16, 2026
```
Signed-off-by: jthomson04 <jothomson@nvidia.com>
Signed-off-by: jthomson04 <jwillthomson19@gmail.com>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
```
  59d18d8e
16 Apr, 2026 2 commits
- fix(discovery): add missing Audios model type to is_model_type_list_empty (#8241) · 67bc6a1f
  MatejKosec authored Apr 16, 2026
```
Signed-off-by: Matej Kosec <mkosec@nvidia.com>
```
  67bc6a1f
- fix(runtime): lock-free TCP connection pool for OOM prevention and... · 6633be54
  Biswa Panda authored Apr 15, 2026
```
fix(runtime): lock-free TCP connection pool for OOM prevention and inflight/resource-leak fixes (#7806)
```
  6633be54
15 Apr, 2026 7 commits
- fix(discovery): spawn handle_put tasks concurrently to prevent replay wedging (#7931) · 470bb48f
  MatejKosec authored Apr 15, 2026
```
Signed-off-by: Matej Kosec <mkosec@nvidia.com>
```
  470bb48f
- feat(multimodal): forward mm_processor_kwargs through pipeline for use_audio_in_video (#8150) · e232bec0
  Neal Vaidya authored Apr 15, 2026
```
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
```
  e232bec0
- fix(metrics): widen histogram buckets for request duration and router overhead (#8087) · bba26d55
  Thomas Montfort authored Apr 15, 2026
```
Signed-off-by: tmontfort <tmontfort@nvidia.com>
Signed-off-by: Thomas Montfort <tmontfort@nvidia.com>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
```
  bba26d55
- fix(metrics): skip output token histogram for zero-output requests (#8085) · 60acaa7b
  Thomas Montfort authored Apr 15, 2026
```
Signed-off-by: tmontfort <tmontfort@nvidia.com>
Signed-off-by: Thomas Montfort <tmontfort@nvidia.com>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
```
  60acaa7b
- feat(kv-router): add prompt membership index for scheduler reads (#8175) · 134d484d
  Yan Ru Pei authored Apr 15, 2026
```
Signed-off-by: PeaBrane <yanrpei@gmail.com>
```
  134d484d
- fix(kv-router): expose kv_cache_events_applied in multithreaded indexer (#8088) · d0d9c030
  Thomas Montfort authored Apr 15, 2026
```
Signed-off-by: tmontfort <tmontfort@nvidia.com>
Signed-off-by: Thomas Montfort <tmontfort@nvidia.com>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
```
  d0d9c030
- fix(llm): close push-router cleanup gap (#8181) · 3efc733f
  Yan Ru Pei authored Apr 14, 2026
```
Signed-off-by: PeaBrane <yanrpei@gmail.com>
```
  3efc733f
14 Apr, 2026 6 commits
- feat: add unified backend architecture with DynamoBackend (#8003) · da40db40
  Tanmay Verma authored Apr 14, 2026
```
Signed-off-by: Tanmay Verma <tanmayv@nvidia.com>
```
  da40db40
- fix(runtime): graceful removal of disagg model from /v1/models when prefill engine dies (#7131) · d2aad651
  Biswa Panda authored Apr 14, 2026
  
  d2aad651
- feat(mocker): emit forward pass metrics to event plane for planner consumption (#8032) · 4e6c3964
  jthomson04 authored Apr 13, 2026
```
Signed-off-by: jthomson04 <jwillthomson19@gmail.com>
Signed-off-by: jthomson04 <jothomson@nvidia.com>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
```
  4e6c3964
- fix(kv-router): route pinned requests through scheduler (#8142) · 1142478a
  Yan Ru Pei authored Apr 13, 2026
```
Signed-off-by: PeaBrane <yanrpei@gmail.com>
```
  1142478a
- feat: router, metric only add queue isl tokens (#8136) · eac6a5fc
  Michael Feil authored Apr 13, 2026
```
Signed-off-by: michaelfeil <63565275+michaelfeil@users.noreply.github.com>
```
  eac6a5fc
- feat: Decoder clean-up for handling incomplete multi-byte sequence (#8022) · 2cabf441
  Biswa Panda authored Apr 13, 2026
  
  2cabf441
13 Apr, 2026 6 commits
- feat(sglang): add ephemeral KV session routing (#7665) · 9498f016
  ishandhanani authored Apr 13, 2026
```
Signed-off-by: Ishan Dhanani <ishandhanani@gmail.com>
```
  9498f016
- fix(kvbm): query NCCL version at runtime instead of hardcoding (#8130) · 47c4bd46
  Alec authored Apr 13, 2026
```
Signed-off-by: Alec Flowers <aflowers@nvidia.com>
```
  47c4bd46
- feat: add KV cache transfer estimated latency metric for disaggregated serving (#7590) · 1ebe7060
  Jonathan Tong authored Apr 13, 2026
```
Signed-off-by: Jont828 <jt572@cornell.edu>
Co-authored-by: Hongkuan Zhou <tedzhouhk@gmail.com>
```
  1ebe7060
- fix: populate input_tokens in Anthropic streaming + add model retrieval endpoint (#7234) · f7fe8005
  MatejKosec authored Apr 13, 2026
```
Signed-off-by: Matej Kosec <mkosec@nvidia.com>
```
  f7fe8005
- feat: route requests by device type and load for sglang epd (#7215) · a7d51b39
  Wang, Yi authored Apr 14, 2026
```
Signed-off-by: Wang, Yi <yi.a.wang@intel.com>
```
  a7d51b39
- fix(kv-router): wake queue on remote lifecycle (#8097) · 36e8b8c8
  Yan Ru Pei authored Apr 13, 2026
```
Signed-off-by: PeaBrane <yanrpei@gmail.com>
```
  36e8b8c8