Commits · 01a4b6d6ddd5b0173f0d110040d1f2cccf759bf9 · OpenDAS / dynamo · GitLab

15 Apr, 2026 15 commits
- chore: bump dynamo version numbers (#8191) · 01a4b6d6
  Harrison Saturley-Hall authored Apr 15, 2026
  
  01a4b6d6
- fix(gms): rewrite GMS checkpoint/restore operator support (#8194) · b2f7f220
  Schwinn Saereesitthipitak authored Apr 15, 2026
```
Co-authored-by: Dmitry Tokarev <dtokarev@nvidia.com>
```
  b2f7f220
- fix(discovery): spawn handle_put tasks concurrently to prevent replay wedging (#7931) · 470bb48f
  MatejKosec authored Apr 15, 2026
```
Signed-off-by: Matej Kosec <mkosec@nvidia.com>
```
  470bb48f
- feat(multimodal): forward mm_processor_kwargs through pipeline for use_audio_in_video (#8150) · e232bec0
  Neal Vaidya authored Apr 15, 2026
```
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
```
  e232bec0
- fix(metrics): widen histogram buckets for request duration and router overhead (#8087) · bba26d55
  Thomas Montfort authored Apr 15, 2026
```
Signed-off-by: tmontfort <tmontfort@nvidia.com>
Signed-off-by: Thomas Montfort <tmontfort@nvidia.com>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
```
  bba26d55
- fix(metrics): skip output token histogram for zero-output requests (#8085) · 60acaa7b
  Thomas Montfort authored Apr 15, 2026
```
Signed-off-by: tmontfort <tmontfort@nvidia.com>
Signed-off-by: Thomas Montfort <tmontfort@nvidia.com>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
```
  60acaa7b
- fix(replay): drain traffic metrics only on throughput ticks (#8232) · b92c9593
  Hongkuan Zhou authored Apr 15, 2026
```
Signed-off-by: hongkuanz <hongkuanz@nvidia.com>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
```
  b92c9593
- feat: add TRT-LLM sleep/wake integration with GMS (#7575) · d96a2cf1
  hhzhang16 authored Apr 15, 2026
```
Co-authored-by: Schwinn Saereesitthipitak <schwinns@nvidia.com>
Co-authored-by: Dmitry Tokarev <dtokarev@nvidia.com>
```
  d96a2cf1
- feat(kv-router): add prompt membership index for scheduler reads (#8175) · 134d484d
  Yan Ru Pei authored Apr 15, 2026
```
Signed-off-by: PeaBrane <yanrpei@gmail.com>
```
  134d484d
- fix(kv-router): expose kv_cache_events_applied in multithreaded indexer (#8088) · d0d9c030
  Thomas Montfort authored Apr 15, 2026
```
Signed-off-by: tmontfort <tmontfort@nvidia.com>
Signed-off-by: Thomas Montfort <tmontfort@nvidia.com>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
```
  d0d9c030
- feat(async-openai): accept dict format for tool-call arguments (#7772) · 59f474a2
  davilu-nvidia authored Apr 15, 2026
```
Signed-off-by: davilu-nvidia <davilu@nvidia.com>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
```
  59f474a2
- fix(llm): close push-router cleanup gap (#8181) · 3efc733f
  Yan Ru Pei authored Apr 14, 2026
```
Signed-off-by: PeaBrane <yanrpei@gmail.com>
```
  3efc733f
- feat(replay): planner-in-the-loop offline replay (#8187) · 89d67172
  jthomson04 authored Apr 14, 2026
```
Signed-off-by: jthomson04 <jwillthomson19@gmail.com>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
```
  89d67172
- feat(metrics): auto-inject worker_id label into metrics (#8089) · cabdaebd
  Thomas Montfort authored Apr 14, 2026
```
Signed-off-by: tmontfort <tmontfort@nvidia.com>
Signed-off-by: Thomas Montfort <tmontfort@nvidia.com>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
```
  cabdaebd
- fix(trtllm): prevent decode worker segfault on prefill scale-down (#7933) · b13a440e
  MatejKosec authored Apr 14, 2026
```
Signed-off-by: Matej Kosec <mkosec@nvidia.com>
```
  b13a440e
14 Apr, 2026 16 commits
- feat: add unified backend architecture with DynamoBackend (#8003) · da40db40
  Tanmay Verma authored Apr 14, 2026
```
Signed-off-by: Tanmay Verma <tanmayv@nvidia.com>
```
  da40db40
- feat(gms): operator-managed GMS checkpoint/restore support (#8153) · f3b181a9
  Schwinn Saereesitthipitak authored Apr 14, 2026
  
  f3b181a9
- feat: remove enable_nats usage (#7265) · af0ff07c
  Biswa Panda authored Apr 14, 2026
  
  af0ff07c
- fix(kvbm): set CUDA device per rank before ncclCommInitRank (#8147) · 2263defc
  Alec authored Apr 14, 2026
```
Signed-off-by: Alec Flowers <aflowers@nvidia.com>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
```
  2263defc
- feat: per-container kube discovery for multi-engine pods (#8067) · 2f658327
  mohammedabdulwahhab authored Apr 14, 2026
```
Signed-off-by: mohammedabdulwahhab <furkhan324@berkeley.edu>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
```
  2f658327
- test: reorganize and prune GPU Memory Service tests (#7828) · b78ec99a
  Schwinn Saereesitthipitak authored Apr 14, 2026
```
Signed-off-by: Schwinn Saereesitthipitak <schwinns@nvidia.com>
```
  b78ec99a
- fix(runtime): graceful removal of disagg model from /v1/models when prefill engine dies (#7131) · d2aad651
  Biswa Panda authored Apr 14, 2026
  
  d2aad651
- feat(gms): exported GMS handles on allocation (#8108) · 49feb284
  Schwinn Saereesitthipitak authored Apr 13, 2026
  
  49feb284
- feat: add fasttokens benchmarks and tokenizer backend docs (#7388) · 8f4bd3b4
  Biswa Panda authored Apr 13, 2026
  
  8f4bd3b4
- fix(gms): restore default allocator cleanup before checkpoint (#8049) · 6f5b460c
  Schwinn Saereesitthipitak authored Apr 13, 2026
```
Signed-off-by: Schwinn Saereesitthipitak <schwinns@nvidia.com>
```
  6f5b460c
- fix(replay): short-circuit .npz planner imports (#8050) · 31909ca3
  Yan Ru Pei authored Apr 13, 2026
```
Signed-off-by: PeaBrane <yanrpei@gmail.com>
```
  31909ca3
- feat(mocker): emit forward pass metrics to event plane for planner consumption (#8032) · 4e6c3964
  jthomson04 authored Apr 13, 2026
```
Signed-off-by: jthomson04 <jwillthomson19@gmail.com>
Signed-off-by: jthomson04 <jothomson@nvidia.com>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
```
  4e6c3964
- fix(kv-router): route pinned requests through scheduler (#8142) · 1142478a
  Yan Ru Pei authored Apr 13, 2026
```
Signed-off-by: PeaBrane <yanrpei@gmail.com>
```
  1142478a
- feat: router, metric only add queue isl tokens (#8136) · eac6a5fc
  Michael Feil authored Apr 13, 2026
```
Signed-off-by: michaelfeil <63565275+michaelfeil@users.noreply.github.com>
```
  eac6a5fc
- feat: Decoder clean-up for handling incomplete multi-byte sequence (#8022) · 2cabf441
  Biswa Panda authored Apr 13, 2026
  
  2cabf441
- fix(deps): pin mio after lockfile drift (#8080) · 20ce329b
  Yan Ru Pei authored Apr 13, 2026
```
Signed-off-by: PeaBrane <yanrpei@gmail.com>
```
  20ce329b
13 Apr, 2026 9 commits
- feat(sglang): add ephemeral KV session routing (#7665) · 9498f016
  ishandhanani authored Apr 13, 2026
```
Signed-off-by: Ishan Dhanani <ishandhanani@gmail.com>
```
  9498f016
- fix(kvbm): query NCCL version at runtime instead of hardcoding (#8130) · 47c4bd46
  Alec authored Apr 13, 2026
```
Signed-off-by: Alec Flowers <aflowers@nvidia.com>
```
  47c4bd46
- feat: add KV cache transfer estimated latency metric for disaggregated serving (#7590) · 1ebe7060
  Jonathan Tong authored Apr 13, 2026
```
Signed-off-by: Jont828 <jt572@cornell.edu>
Co-authored-by: Hongkuan Zhou <tedzhouhk@gmail.com>
```
  1ebe7060
- feat(runtime): add DYN_TCP_RESPONSE_STREAM_PORT/HOST env vars for configurable... · 3acdc0f8
  Biswa Panda authored Apr 13, 2026
```
feat(runtime): add DYN_TCP_RESPONSE_STREAM_PORT/HOST env vars for configurable response stream server port
```
  3acdc0f8
- fix: populate input_tokens in Anthropic streaming + add model retrieval endpoint (#7234) · f7fe8005
  MatejKosec authored Apr 13, 2026
```
Signed-off-by: Matej Kosec <mkosec@nvidia.com>
```
  f7fe8005
- fix(kvbm): handle typed KvCacheConnectorConfig in consolidator check (#8117) · edce7d3d
  Alec authored Apr 13, 2026
```
Signed-off-by: Alec Flowers <aflowers@nvidia.com>
```
  edce7d3d
- feat: route requests by device type and load for sglang epd (#7215) · a7d51b39
  Wang, Yi authored Apr 14, 2026
```
Signed-off-by: Wang, Yi <yi.a.wang@intel.com>
```
  a7d51b39
- fix(kv-router): wake queue on remote lifecycle (#8097) · 36e8b8c8
  Yan Ru Pei authored Apr 13, 2026
```
Signed-off-by: PeaBrane <yanrpei@gmail.com>
```
  36e8b8c8
- fix(frontend): classify cancelled requests as 499 instead of 500 (#8090) · 1284237e
  Thomas Montfort authored Apr 13, 2026
```
Signed-off-by: tmontfort <tmontfort@nvidia.com>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
```
  1284237e