Commits · 1f8b7c536be40975573eeebf36204286cfb4e4e9 · OpenDAS / vllm_cscc

09 Jan, 2026 2 commits

[Feature] Add iteration level logging and enhance nvtx marker (#31193) · 6ebe34d6

Max Hu authored Jan 08, 2026


Signed-off-by: Max Hu <maxhu@nvidia.com>
Signed-off-by: Max Hu <hyoung2991@gmail.com>
Co-authored-by: Max Hu <maxhu@nvidia.com>

6ebe34d6

[Quantization] Deprecate Long Tail of Schemes (#31688) · 5825bbc1

Robert Shaw authored Jan 08, 2026

Signed-off-by: Robert Shaw <robshaw@redhat.com>
Signed-off-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com>
Co-authored-by: Robert Shaw <robshaw@redhat.com>
Co-authored-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com>

5825bbc1

08 Jan, 2026 2 commits

RayLLM Bugfix - Preserve obj store URL for multi engine_config creation (#30803) · 04a49669

omer-dayan authored Jan 08, 2026


Signed-off-by: Omer Dayan <omdayan@nvidia.com>
Signed-off-by: Isotr0py <2037008807@qq.com>
Co-authored-by: Isotr0py <2037008807@qq.com>
Co-authored-by: Isotr0py <mozf@mail2.sysu.edu.cn>

04a49669

[Fix] Enable mm_processor_cache with vision LoRA (#31927) · d3235cb5
prashanth058 authored Jan 08, 2026
```
Signed-off-by: prashanth058 <prashanth.dannamaneni@uipath.com>
```
d3235cb5

06 Jan, 2026 1 commit
- [UX] Add `-ep` shorthand for `--enable-expert-parallel` (#30890) · a68e703c
  Michael Goin authored Jan 05, 2026
```
Signed-off-by: mgoin <mgoin64@gmail.com>
```
  a68e703c
02 Jan, 2026 2 commits
- [BugFix] Support online dense model DP without overhead (#30739) · bd877162
  Nick Hill authored Jan 02, 2026
```
Signed-off-by: Nick Hill <nhill@redhat.com>
Signed-off-by: njhill <nickhill123@gmail.com>
```
  bd877162
- Add Multimodal Processor Benchmark (#29105) · 1f5b7c41
  Reagan Lee authored Jan 01, 2026
```
Signed-off-by: Reagan Lee <reaganjlee@gmail.com>
Signed-off-by: Reagan <reaganjlee@gmail.com>
```
  1f5b7c41
26 Dec, 2025 1 commit

[Core] Initialize LoRA support for tower and connector in multi-modal models (#26674) · ce1eafd1

Jee Jee Li authored Dec 26, 2025


Signed-off-by: bk-201 <joy25810@foxmail.com>
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
Signed-off-by: prashanth058 <prashanth.dannamaneni@uipath.com>
Co-authored-by: bk-201 <joy25810@foxmail.com>
Co-authored-by: prashanth058 <prashanth.dannamaneni@uipath.com>
Co-authored-by: Anexdeus <5142168@mail.ru>

ce1eafd1

24 Dec, 2025 2 commits
- [Bugfix] Fix `max_model_len="auto"` handling (#31260) · 7adeb4bf
  Cyrus Leung authored Dec 24, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  7adeb4bf
- Add `--max-model-len auto` to auto-fit context to available memory (#29431) · 8ee90c83
  Michael Goin authored Dec 24, 2025
```
Signed-off-by: mgoin <mgoin64@gmail.com>
```
  8ee90c83
18 Dec, 2025 2 commits

Remove all2all backend envvar (#30363) · 41b6f920

Elizabeth Thomas authored Dec 18, 2025

Signed-off-by: Elizabeth Thomas <email2eliza@gmail.com>
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>

41b6f920

[Metrics] Model FLOPs Utilization estimation (#30738) · a0b782f9

SungMinCho authored Dec 17, 2025


Signed-off-by: SungMinCho <tjdals4565@gmail.com>
Signed-off-by: Mark McLoughlin <markmc@redhat.com>
Co-authored-by: Mark McLoughlin <markmc@redhat.com>

a0b782f9

17 Dec, 2025 1 commit
- [Fix]Load kv-cache dtype from hf_quant_config.json automatically (fix for reverted PR) (#30785) · 7b966ae2
  danielafrimi authored Dec 17, 2025
```
Signed-off-by: <>
Co-authored-by: root <root@gpu-937.slurm-workers-slurm.slurm.svc.cluster.local>
```
  7b966ae2
16 Dec, 2025 1 commit

[feature] extend DBO to XBO (#30120) · b9ff4f2a

jiangkuaixue123 authored Dec 16, 2025


Signed-off-by: jiangkuaixue123 <jiangxiaozhou111@163.com>
Co-authored-by: root <root@hk01dgx028.cm.cluster>

b9ff4f2a

15 Dec, 2025 1 commit

[Misc][Hybrid allocator + kv connector] Optionally enable hybrid allocator +... · 185c22bf

Nicolò Lucchesi authored Dec 15, 2025


[Misc][Hybrid allocator + kv connector] Optionally enable hybrid allocator + KV cache connector (#29805)
Signed-off-by: NickLucche <nlucches@redhat.com>

185c22bf

12 Dec, 2025 1 commit
- [Bug] Fix attention_backend arg string parsing (#30534) · 09ad3b76
  Michael Goin authored Dec 12, 2025
```
Signed-off-by: mgoin <mgoin64@gmail.com>
```
  09ad3b76
11 Dec, 2025 4 commits
- [Misc][PCP&DCP] relocate PCP feature check (#30050) · a11f4a81
  Qiu authored Dec 11, 2025
```
Signed-off-by: QiuChunshuo <qiuchunshuo@huawei.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
```
  a11f4a81
- fix: enhance human_readable_int function (#30337) · d02d1043
  Ning Xie authored Dec 11, 2025
```
Signed-off-by: Andy Xie <andy.xning@gmail.com>
```
  d02d1043
- [Deprecation] Remove deprecated task, seed and MM settings (#30397) · 7e24e5d4
  Cyrus Leung authored Dec 11, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  7e24e5d4
- [Deprecation] Remove deprecated plugin and compilation fields for v0.13 release (#30396) · 5a87d8b9
  Cyrus Leung authored Dec 11, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  5a87d8b9
09 Dec, 2025 1 commit

[Cleanup] Refactor profiling env vars into a CLI config (#29912) · e858bfe0

Benjamin Chislett authored Dec 09, 2025

Signed-off-by: Benjamin Chislett <bchislett@nvidia.com>
Signed-off-by: Benjamin Chislett <chislett.ben@gmail.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>

e858bfe0

07 Dec, 2025 2 commits
- Revert "[Renderer] Separate out `RendererConfig` from `ModelConfig` (#30145)" (#30199) · e83b7e37
  Cyrus Leung authored Dec 07, 2025
  
  e83b7e37
- [Renderer] Separate out `RendererConfig` from `ModelConfig` (#30145) · 27f4c2fd
  Cyrus Leung authored Dec 07, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  27f4c2fd
06 Dec, 2025 1 commit
- [Misc] Move `disable_nccl_for_dp_synchronization` init logic into `VllmConfig` (#30161) · 4026ae31
  Nick Hill authored Dec 05, 2025
```
Signed-off-by: Nick Hill <nhill@redhat.com>
```
  4026ae31
05 Dec, 2025 2 commits

[Attention][UX][1/N] Add AttentionConfig and change attention env vars to CLI arguments (#26315) · 66e674cd

Matthew Bonanni authored Dec 05, 2025

Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
Signed-off-by: Matthew Bonanni <mbonanni001@gmail.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com>

66e674cd

[Feature] Add Layer-wise NVTX Support (#29990) · c2894d38

Max Hu authored Dec 05, 2025


Signed-off-by: Max Hu <hyoung2991@gmail.com>
Signed-off-by: Max Hu <maxhu@nvidia.com>
Co-authored-by: Max Hu <maxhu@nvidia.com>

c2894d38

03 Dec, 2025 1 commit
- Add logging for cudagraph related info (#29825) · 69520bc6
  Yong Hoon Shin authored Dec 02, 2025
```
Signed-off-by: Yong Hoon Shin <yhshin@meta.com>
```
  69520bc6
02 Dec, 2025 2 commits
- [Bugfix][EPLB] Prevent user-provided EPLB config from being overwritten with defaults (#29911) · e6f114ac
  Sage Moore authored Dec 02, 2025
```
Signed-off-by: Sage Moore <sage@neuralmagic.com>
```
  e6f114ac
- [Chore]: Reorganize gguf utils funtions under `transformers_utils` (#29891) · 63b1da76
  Isotr0py authored Dec 03, 2025
```
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
```
  63b1da76
01 Dec, 2025 2 commits

[Core][Observability] Add KV cache residency metrics (#27793) · cabc77cc

shivampr authored Dec 01, 2025



Introduces three new Prometheus histograms for fine-grained observability of KV cache residency behavior:

vllm:kv_block_lifetime_seconds — total lifetime from allocation to free
vllm:kv_block_idle_before_evict_seconds — idle duration before eviction
vllm:kv_block_reuse_gap_seconds — time between consecutive reuses of the same block

These metrics help operators analyze KV cache efficiency, reuse patterns, and eviction timing beyond simple utilization rates.

Implementation uses monotonic timestamps for accuracy, 1% sampling for minimal overhead (~48 bytes/block), and is fully thread-safe with zero runtime cost when disabled.

Two new runtime flags are introduced:

--kv-cache-metrics – enable KV cache residency metrics
--kv-cache-metrics-sample – control sampling ratio (default: 0.01)
Signed-off-by: Shivam <shivamprasad91@gmail.com>

cabc77cc

[Misc] Unify tokenizer registration (#29767) · f0a28bf6
Cyrus Leung authored Dec 01, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
f0a28bf6

29 Nov, 2025 1 commit
- [Frontend] Perform offline path replacement to `tokenizer` (#29706) · 762a4a6c
  Tsukasa OI authored Nov 29, 2025
```
Signed-off-by: Tsukasa OI <floss_llm@irq.a4lg.com>
```
  762a4a6c
28 Nov, 2025 4 commits
- [Frontend] Remap -O to -cc commandline flag (#29557) · 3461e7ef
  Yanan Cao authored Nov 28, 2025
```
Signed-off-by: Yanan Cao <gmagogsfm@gmail.com>
Co-authored-by: Claude <noreply@anthropic.com>
```
  3461e7ef
- [Chore]: Reorganize model repo operating functions in `transformers_utils` (#29680) · f946a8d7
  Isotr0py authored Nov 29, 2025
```
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
```
  f946a8d7
- [mypy] Enable type checking for more directories (#29674) · 9e6bcda3
  Cyrus Leung authored Nov 29, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  9e6bcda3
- Improve enable chunked_prefill & prefix_caching logic. (#26623) · f4b76056
  wang.yuqi authored Nov 28, 2025
```
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>
Signed-off-by: wang.yuqi <noooop@126.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
```
  f4b76056
27 Nov, 2025 1 commit

[Frontend][torch.compile] CompilationConfig Overhaul (#20283): Set up -O infrastructure (#26847) · 0838b52e

Morrison Turnansky authored Nov 27, 2025


Signed-off-by: morrison-turnansky <mturnans@redhat.com>
Signed-off-by: adabeyta <aabeyta@redhat.com>
Signed-off-by: Morrison Turnansky <mturnans@redhat.com>
Co-authored-by: adabeyta <aabeyta@redhat.com>
Co-authored-by: Luka Govedič <ProExpertProg@users.noreply.github.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>

0838b52e

25 Nov, 2025 3 commits

Scheduled removal of `override_pooler_config` and `disable_log_requests` (#29402) · a1f26768

Harry Mellor authored Nov 25, 2025


Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>

a1f26768

[Hybrid Allocator] Support KV cache groups with different block_size (#29143) · 48ddb02b
Yifan Qiao authored Nov 25, 2025
```
Signed-off-by: Yifan Qiao <yifanqiao@berkeley.edu>
Co-authored-by: Chen Zhang <zhangch99@outlook.com>
```
48ddb02b

[Feature]: Improve GGUF loading from HuggingFace user experience like repo_id:quant_type (#29137) · 794029f0

Injae Ryou authored Nov 25, 2025


Signed-off-by: Injae Ryou <injaeryou@gmail.com>
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
Co-authored-by: Isotr0py <mozf@mail2.sysu.edu.cn>

794029f0