Commits · 49feb28404094d986738d41c63d6f6b64908a858 · OpenDAS / dynamo

"docs/design-docs/router-design.md" did not exist on "937398cf61d5cc329e80cb5c7b70e574c02c2251"

14 Apr, 2026 5 commits
- feat: add fasttokens benchmarks and tokenizer backend docs (#7388) · 8f4bd3b4
  Biswa Panda authored Apr 13, 2026
  
  8f4bd3b4
- feat(mocker): emit forward pass metrics to event plane for planner consumption (#8032) · 4e6c3964
  jthomson04 authored Apr 13, 2026
```
Signed-off-by: jthomson04 <jwillthomson19@gmail.com>
Signed-off-by: jthomson04 <jothomson@nvidia.com>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
```
  4e6c3964
- fix(kv-router): route pinned requests through scheduler (#8142) · 1142478a
  Yan Ru Pei authored Apr 13, 2026
```
Signed-off-by: PeaBrane <yanrpei@gmail.com>
```
  1142478a
- feat: router, metric only add queue isl tokens (#8136) · eac6a5fc
  Michael Feil authored Apr 13, 2026
```
Signed-off-by: michaelfeil <63565275+michaelfeil@users.noreply.github.com>
```
  eac6a5fc
- feat: Decoder clean-up for handling incomplete multi-byte sequence (#8022) · 2cabf441
  Biswa Panda authored Apr 13, 2026
  
  2cabf441
13 Apr, 2026 7 commits
- feat(sglang): add ephemeral KV session routing (#7665) · 9498f016
  ishandhanani authored Apr 13, 2026
```
Signed-off-by: Ishan Dhanani <ishandhanani@gmail.com>
```
  9498f016
- fix(kvbm): query NCCL version at runtime instead of hardcoding (#8130) · 47c4bd46
  Alec authored Apr 13, 2026
```
Signed-off-by: Alec Flowers <aflowers@nvidia.com>
```
  47c4bd46
- feat: add KV cache transfer estimated latency metric for disaggregated serving (#7590) · 1ebe7060
  Jonathan Tong authored Apr 13, 2026
```
Signed-off-by: Jont828 <jt572@cornell.edu>
Co-authored-by: Hongkuan Zhou <tedzhouhk@gmail.com>
```
  1ebe7060
- fix: populate input_tokens in Anthropic streaming + add model retrieval endpoint (#7234) · f7fe8005
  MatejKosec authored Apr 13, 2026
```
Signed-off-by: Matej Kosec <mkosec@nvidia.com>
```
  f7fe8005
- feat: route requests by device type and load for sglang epd (#7215) · a7d51b39
  Wang, Yi authored Apr 14, 2026
```
Signed-off-by: Wang, Yi <yi.a.wang@intel.com>
```
  a7d51b39
- fix(kv-router): wake queue on remote lifecycle (#8097) · 36e8b8c8
  Yan Ru Pei authored Apr 13, 2026
```
Signed-off-by: PeaBrane <yanrpei@gmail.com>
```
  36e8b8c8
- fix(frontend): classify cancelled requests as 499 instead of 500 (#8090) · 1284237e
  Thomas Montfort authored Apr 13, 2026
```
Signed-off-by: tmontfort <tmontfort@nvidia.com>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
```
  1284237e
11 Apr, 2026 2 commits
- fix(llm): support reading eos_token_ids from tokenizer_config.json for models... · b425b65c
  Ryan McCormick authored Apr 10, 2026
```
fix(llm): support reading eos_token_ids from tokenizer_config.json for models like Qwen3.5 with <|im_end|> token (#8091)
```
  b425b65c
- feat: Request Migration disable on max seq len exceeded (#8020) · f6976e7f
  Jacky authored Apr 10, 2026
```
Signed-off-by: Jacky <18255193+kthui@users.noreply.github.com>
```
  f6976e7f
10 Apr, 2026 5 commits
- feat(frontend): ResponseTimeout error type with request-plane worker quarantine (#8011) · 7f8e88fb
  MatejKosec authored Apr 10, 2026
```
Signed-off-by: Matej Kosec <mkosec@nvidia.com>
```
  7f8e88fb
- fix(frontend): backend stream inactivity timeout to fix inflight gauge leak (#7929) · adae833e
  MatejKosec authored Apr 10, 2026
```
Signed-off-by: Matej Kosec <mkosec@nvidia.com>
```
  adae833e
- fix: recovery etc order, full recovery, thundering herd on same worker ids in same order. (#8016) · 6b75d6b0
  Michael Feil authored Apr 10, 2026
```
Signed-off-by: michaelfeil <63565275+michaelfeil@users.noreply.github.com>
```
  6b75d6b0
- fix: prevent KV block leak from cancel during disagg KV transfer (#7489) · 9055b2d3
  Yuewei Na authored Apr 09, 2026
```
Signed-off-by: Yuewei Na <nv-yna@users.noreply.github.com>
Co-authored-by: Yuewei Na <nv-yna@users.noreply.github.com>
```
  9055b2d3
- fix: add worker dump throtteling (#8013) · 859bd6ea
  Michael Feil authored Apr 09, 2026
```
Signed-off-by: michaelfeil <63565275+michaelfeil@users.noreply.github.com>
```
  859bd6ea
09 Apr, 2026 5 commits
- fix(llm): preserve unresolved dp rank routing (#8000) · b579a772
  Yan Ru Pei authored Apr 09, 2026
```
Signed-off-by: PeaBrane <yanrpei@gmail.com>
```
  b579a772
- fix(kv-router): coalesce worker recovery dumps (#8021) · f3d3a8b3
  Yan Ru Pei authored Apr 09, 2026
```
Signed-off-by: PeaBrane <yanrpei@gmail.com>
```
  f3d3a8b3
- perf: avoid cloning content parts in gather_multi_modal_data (#8009) · f49df565
  KrishnanPrash authored Apr 08, 2026
  
  f49df565
- feat(kv-router): add dedup filter for duplicate vLLM KV block events (#8012) · 9b3e9249
  jthomson04 authored Apr 08, 2026
```
Signed-off-by: jthomson04 <jwillthomson19@gmail.com>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
```
  9b3e9249
- fix(llm): log routing input hashes for replay debug (#8017) · e45bb0fe
  Yan Ru Pei authored Apr 08, 2026
```
Signed-off-by: PeaBrane <yanrpei@gmail.com>
```
  e45bb0fe
08 Apr, 2026 8 commits
- feat: adding kvbm-engine (#6773) · 008683d6
  Ryan Olson authored Apr 08, 2026
```
Signed-off-by: Ryan Olson <rolson@nvidia.com>
```
  008683d6
- fix: tool call loss under speculative decoding (#7768) · f69580b0
  Oleg Zhelezniak authored Apr 08, 2026
```
Signed-off-by: jellysnack <oleg.jellysnack@gmail.com>
```
  f69580b0
- feat(kvbm): add MLA support for DeepSeek-V2 and extend determinism tests (#7786) · ba34f131
  Olga Andreeva authored Apr 08, 2026
```
Signed-off-by: Olga Andreeva <oandreeva@nvidia.com>
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-authored-by: wz1qqx <ziqi.wang@novita.ai>
```
  ba34f131
- refactor(3/3): switch dynamo-protocols to upstream async-openai types (#7625) · fd5cc288
  ishandhanani authored Apr 08, 2026
```
Co-authored-by: Dmitry Tokarev <dtokarev@nvidia.com>
```
  fd5cc288
- test(router): cover round-robin unset dp-rank flow (#7991) · c09abf7d
  Yan Ru Pei authored Apr 08, 2026
```
Signed-off-by: PeaBrane <yanrpei@gmail.com>
Co-authored-by: Ishan Dhanani <ishandhanani@gmail.com>
Co-authored-by: Dmitry Tokarev <dtokarev@nvidia.com>
```
  c09abf7d
- fix: dp_rank always 0 in non-KV router mode (#7984) · a210efa6
  ishandhanani authored Apr 08, 2026
  
  a210efa6
- feat(kv-router): split Dynamo-native remote indexer [DYN-2593] (#7973) · 49eb397a
  Yan Ru Pei authored Apr 07, 2026
```
Signed-off-by: PeaBrane <yanrpei@gmail.com>
```
  49eb397a
- fix(migration): disable migration for guided-decoding requests (#7930) · c53ef736
  MatejKosec authored Apr 07, 2026
```
Signed-off-by: Matej Kosec <mkosec@nvidia.com>
```
  c53ef736
07 Apr, 2026 6 commits
- feat: add I2I (image-to-image) support for SGLang diffusion backend (#7870) · 225dad4d
  KrishnanPrash authored Apr 07, 2026
```
Signed-off-by: Krishnan Prashanth <kprashanth@nvidia.com>
```
  225dad4d
- test: e2e JSONL log verification for request tracing (#7817) · 73f8557a
  Neelay Shah authored Apr 07, 2026
```
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
```
  73f8557a
- fix: call inflight.mark_error() in media handler error paths (#7917) · a27475de
  MatejKosec authored Apr 07, 2026
```
Signed-off-by: Matej Kosec <mkosec@nvidia.com>
```
  a27475de
- chore: bump vLLM to 0.19.0 (#7894) · d5803cbe
  Alec authored Apr 07, 2026
  
  d5803cbe
- feat: Request Rejection Frontend metrics (#7644) · 3205e7db
  Jacky authored Apr 06, 2026
```
Signed-off-by: Jacky <18255193+kthui@users.noreply.github.com>
```
  3205e7db
- fix(parsers): parallel XML/JSON tool calls no longer drop N-1 of N calls (#7914) · 91700375
  MatejKosec authored Apr 06, 2026
```
Signed-off-by: Matej Kosec <mkosec@nvidia.com>
```
  91700375
06 Apr, 2026 2 commits
- perf(multimodal): strip inline base64 from TCP transport in frontend decoding (#7895) · 6b8545fc
  Qi Wang authored Apr 06, 2026
```
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
```
  6b8545fc
- fix: latch busy load until both cool off (#7912) · ffe20629
  Yan Ru Pei authored Apr 06, 2026
```
Signed-off-by: AmeenP <ameenp360@gmail.com>
Signed-off-by: PeaBrane <yanrpei@gmail.com>
Co-authored-by: AmeenP <ameenp360@gmail.com>
```
  ffe20629