Commits · 6879cd80ae4dba88121db226d5bfbc6a75b072ba · OpenDAS / vllm_cscc

"vscode:/vscode.git/clone" did not exist on "179285893c939c3841e211a7fd86719f54c60acb"

25 Aug, 2025 7 commits

[Refactor] Pass `tokenizer` explicitly instead of binding to prompt update (#23542) · 6879cd80
Cyrus Leung authored Aug 25, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
6879cd80
[Attention] Unify mamba and attention backend selection (#23171) · 5c4b6e66
Ayush Satyam authored Aug 25, 2025
```
Signed-off-by: Ayush Satyam <ayushsatyam146@gmail.com>
```
5c4b6e66
Feature/benchmark/random mm data/images (#23119) · 0cb7b065
Breno Baldas Skuk authored Aug 25, 2025
```
Signed-off-by: breno.skuk <breno.skuk@hcompany.ai>
```
0cb7b065

[Core][Multimodal] Track encode cache entries by mm_hash and enable embedding... · d765cf01

Chenguang Zheng authored Aug 25, 2025


[Core][Multimodal] Track encode cache entries by mm_hash and enable embedding sharing between requests (#22711)
Signed-off-by: knlnguyen1802 <knlnguyen1802@gmail.com>
Signed-off-by: Roger Wang <hey@rogerw.io>
Co-authored-by: knlnguyen1802 <knlnguyen1802@gmail.com>
Co-authored-by: Roger Wang <hey@rogerw.io>

d765cf01

[Refactor] Dynamic `target` and `content` for prompt updates (#23411) · 712d0f88
Cyrus Leung authored Aug 25, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
712d0f88

[Bugfix] Fix Dense module loading for sentence-transformers embedding models... · c9abb104

LIYIFAN_liyifan authored Aug 24, 2025


[Bugfix] Fix Dense module loading for sentence-transformers embedding models (simplified V2) (#23408)
Signed-off-by: FFFfff1FFFfff <yifanli0919@gmail.com>

c9abb104

Frontend: Adding LM Format Enforcer support to V1 engine (#22564) · 39971db3

Noam Gat authored Aug 25, 2025


Signed-off-by: Noam Gat <noamgat@gmail.com>
Co-authored-by: Russell Bryant <rbryant@redhat.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>

39971db3

24 Aug, 2025 3 commits
- [New Model]Donut model (#23229) · 416f0592
  汪志鹏 authored Aug 24, 2025
```
Signed-off-by: 汪志鹏 <wangzhipeng628@gmail.com>
```
  416f0592
- (Misc): add missing test for zero truncation size. (#23457) · 5e021b49
  TeeKen Lau authored Aug 24, 2025
```
Signed-off-by: teekenl <teekenlau@gmail.com>
```
  5e021b49
- [kernel] Support W4A8 on Hopper (#23198) · e76e2335
  czhu-cohere authored Aug 24, 2025
```
Signed-off-by: czhu-cohere <conway.zhu@cohere.com>
```
  e76e2335
23 Aug, 2025 1 commit
- fix(tests): Correct unreachable assertion in truncation test (#23425) · d9a55204
  Aziz authored Aug 23, 2025
```
Signed-off-by: AzizCode92 <azizbenothman76@gmail.com>
```
  d9a55204
22 Aug, 2025 14 commits
- [NVIDIA][torch.compile] Support Flashinfer TRTLLM FP8-q/kv NVFP4-out Attention Kernel (#22703) · 24d0c9e6
  elvischenv authored Aug 23, 2025
```
Signed-off-by: elvischenv <219235043+elvischenv@users.noreply.github.com>
Co-authored-by: Luka Govedič <ProExpertProg@users.noreply.github.com>
```
  24d0c9e6
- [PERF] PyTorch Symmetric Memory All-Reduce (#20759) · 0313cf85
  Ilya Markov authored Aug 22, 2025
```
Signed-off-by: ilmarkov <imarkov@redhat.com>
Signed-off-by: ilmarkov <markovilya197@gmail.com>
Signed-off-by: Michael Goin <mgoin64@gmail.com>
Co-authored-by: ilmarkov <imarkov@redhat.com>
Co-authored-by: Michael Goin <mgoin64@gmail.com>
```
  0313cf85
- [Model] Add Ovis2.5 PP support (#23405) · 32d2b406
  Isotr0py authored Aug 23, 2025
```
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
```
  32d2b406
- Add unit tests for batched guided and non-guided requests (#23389) · b6d7d34f
  Yong Hoon Shin authored Aug 22, 2025
```
Signed-off-by: Yong Hoon Shin <yhshin@meta.com>
```
  b6d7d34f
- fix(tests): Ensure reliable CUDA cache clearing in MoE test (#23416) · 341923b9
  Aziz authored Aug 22, 2025
```
Signed-off-by: AzizCode92 <azizbenothman76@gmail.com>
Signed-off-by: Michael Goin <mgoin64@gmail.com>
Co-authored-by: Michael Goin <mgoin64@gmail.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
```
  341923b9
- [V0 Deprecation] Remove V0 LoRA test (#23418) · 285178b3
  Jee Jee Li authored Aug 22, 2025
```
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
```
  285178b3
- [P/D][Nixl] Make kv cache register compatible with hybrid memory allocator (#23079) · 53415653
  Flora Feng authored Aug 21, 2025
```
Signed-off-by: sfeng33 <4florafeng@gmail.com>
```
  53415653
- [Attention] Refactor AttentionMetadata Preparation for Encoder-only Models (#23154) · 17373dcd
  Chen Zhang authored Aug 21, 2025
```
Signed-off-by: Chen Zhang <zhangch99@outlook.com>
```
  17373dcd
- [New Model] Add Seed-Oss model (#23241) · 59640693
  Bin Jia authored Aug 22, 2025
```
Signed-off-by: jiabin.00 <jiabin.00@bytedance.com>
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
Co-authored-by: Jee Jee Li <pandaleefree@gmail.com>
```
  59640693
- [CI] Add end-to-end V1 min_tokens test coverage (#22495) · 111692bb
  Arjun Reddy authored Aug 21, 2025
```
Signed-off-by: Arjun Reddy <189282188+arjunbreddy22@users.noreply.github.com>
Co-authored-by: Arjun Reddy <189282188+arjunbreddy22@users.noreply.github.com>
```
  111692bb
- [CI/Build] Skip Idefics3 and SmolVLM generation test again (#23356) · 3ac84966
  Isotr0py authored Aug 22, 2025
```
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
```
  3ac84966
- [Deprecation] Remove `prompt_token_ids` arg fallback in `LLM.generate` and `LLM.embed` (#18800) · 8896eb72
  Cyrus Leung authored Aug 22, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  8896eb72
- [Kernel] Add FP8 support with FlashMLA backend (#22668) · 19fe1a05
  Matthew Bonanni authored Aug 21, 2025
```
Signed-off-by: Matthew Bonanni <mbonanni001@gmail.com>
```
  19fe1a05
- [Core] Support custom executor qualname (#23314) · 480bdf5a
  22quinn authored Aug 21, 2025
```
Signed-off-by: 22quinn <33176974+22quinn@users.noreply.github.com>
```
  480bdf5a
21 Aug, 2025 10 commits

[Feature][Responses API] Support logprobs(non-stream) (#23319) · 5368f768
Kebe authored Aug 22, 2025
```
Signed-off-by: Kebe <mail@kebe7jun.com>
```
5368f768
[Perf] Small optimizations for silu_mul_fp8_quant_deep_gemm (#23265) · 3bbe11cc
Michael Goin authored Aug 21, 2025
```
Signed-off-by: mgoin <mgoin64@gmail.com>
```
3bbe11cc
[Core] Always use tensor cores for Flashinfer Decode Wrapper (#23214) · 1d353b63
Pavani Majety authored Aug 21, 2025
```
Signed-off-by: Pavani Majety <pmajety@nvidia.com>
```
1d353b63

add tg-mxfp4-moe-test (#22540) · f8ce0229

Lain authored Aug 21, 2025


Signed-off-by: siyuanf <siyuanf@nvidia.com>
Signed-off-by: Siyuan Fu <siyuanf@nvidia.com>
Co-authored-by: Michael Goin <mgoin64@gmail.com>

f8ce0229

[Model] Add LFM2 architecture (#22845) · 2e2000f3
Paul Pak authored Aug 21, 2025
```
Signed-off-by: Paul Pak <paulpak58@gmail.com>
```
2e2000f3

[Sampler] Support returning final logprobs (#22387) · f571ff8e

22quinn authored Aug 20, 2025


Signed-off-by: 22quinn <33176974+22quinn@users.noreply.github.com>
Co-authored-by: Nick Hill <nhill@redhat.com>
Co-authored-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>

f571ff8e

[Model][VLM] Support R-4B Model (#23246) · 655a09f6

杨奇(yann qi) authored Aug 21, 2025

Signed-off-by: yannqi <yannqi@qq.com>
Signed-off-by: 杨奇(yann qi) <51905299+yannqi@users.noreply.github.com>
Signed-off-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
Co-authored-by: yannqiyang <yannqiyang@tencent.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>

655a09f6

[V1][Mamba1] - Full CUDA and Piecewise CUDA Graphs Support (#23035) · 3663870c

Asaf Joseph Gardin authored Aug 21, 2025


Signed-off-by: asafg <asafg@ai21.com>
Signed-off-by: asafg <39553475+Josephasafg@users.noreply.github.com>
Co-authored-by: asafg <asafg@ai21.com>

3663870c

[CI/Build] Split out mm processor tests (#23260) · 2461d9e5
Cyrus Leung authored Aug 21, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
2461d9e5
[CPU] Refactor CPU W8A8 scaled_mm (#23071) · 7be5d113
Li, Jiang authored Aug 21, 2025
```
Signed-off-by: jiang1.li <jiang1.li@intel.com>
```
7be5d113

20 Aug, 2025 5 commits
- Feature/mla tests (#23195) · 10cc12ba
  Matthew Bonanni authored Aug 20, 2025
```
Signed-off-by: Matthew Bonanni <mbonanni001@gmail.com>
Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
```
  10cc12ba
- [Fix] correct tool_id for kimi-k2 when use tool_choice=required (#21259) · 582bbe6b
  bigmoyan authored Aug 21, 2025
```
Co-authored-by: wangzhengtao <wangzhengtao@msh.team>
```
  582bbe6b
- [Kernel/Quant] Remove the original marlin format and qqq (#23204) · 0cdbf5e6
  Michael Goin authored Aug 20, 2025
```
Signed-off-by: mgoin <mgoin64@gmail.com>
```
  0cdbf5e6
- [torch.compile] Support conditional torch.compile per module (#22269) · dfd23820
  Yong Hoon Shin authored Aug 20, 2025
```
Signed-off-by: Yong Hoon Shin <yhshin@meta.com>
```
  dfd23820
- [Misc] Add max_seq_len to CommonAttentionMetadata (#23216) · d6d13bd4
  Woosuk Kwon authored Aug 20, 2025
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  d6d13bd4