Commits · 82dfb12e52ded608da2a458b2da5f48ba1acecdf · OpenDAS / vllm_cscc

"vllm/entrypoints/pooling/embed/serving.py" did not exist on "1c2bc7ead019cdf5b04b2f1d07b00982352f85ef"

09 Sep, 2025 7 commits
- [Core] Use sha256 bytes instead of BlockHash to reduce GC overhead (#23673) · 82dfb12e
  Zebing Lin authored Sep 09, 2025
```
Signed-off-by: linzebing <linzebing1995@gmail.com>
```
  82dfb12e
- [Flashinfer] Support Flashinfer TRTLLM FP8-qkv BF16/FP16-out Attention Kernel (#23647) · bba1042c
  elvischenv authored Sep 09, 2025
```
Signed-off-by: elvischenv <219235043+elvischenv@users.noreply.github.com>
```
  bba1042c
- [BugFix][Model] Fix Ernie4.5-VL hanging on long inputs (#24074) · b6fbc156
  CSWYF3634076 authored Sep 09, 2025
```
Signed-off-by: wangyafeng <wangyafeng@baidu.com>
```
  b6fbc156
- Move `KVTransferConfig` from `config/__init__.py` to `config/kv_transfer.py` (#24434) · 3e0d4a34
  Harry Mellor authored Sep 09, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  3e0d4a34
- [doc] update `vllm serve` cli args documentation (#24329) · 13b89bd8
  cjackal authored Sep 09, 2025
```
Signed-off-by: cjackal <44624812+cjackal@users.noreply.github.com>
```
  13b89bd8
- [gpt-oss] Harmony changes with container tool support (#23386) · 170129eb
  zhiweiz authored Sep 08, 2025
```
Signed-off-by: zhiweiz <zhiweiz@fb.com>
Signed-off-by: Aaron Pham <contact@aarnphm.xyz>
Signed-off-by: Lu Fang <30275821+houseroad@users.noreply.github.com>
Co-authored-by: zhiweiz <zhiweiz@fb.com>
Co-authored-by: Aaron Pham <contact@aarnphm.xyz>
Co-authored-by: Simon Mo <simon.mo@hey.com>
Co-authored-by: Lu Fang <30275821+houseroad@users.noreply.github.com>
```
  170129eb
- [Bugfix][Wide EP] Fix redundant work when using DeepEP, TP Attn, and EP MoE (#24134) · 955c6249
  Tyler Michael Smith authored Sep 08, 2025
```
Signed-off-by: Tyler Michael Smith <tlrmchlsmth@gmail.com>
```
  955c6249
08 Sep, 2025 19 commits

[Bugfix] Disable the statslogger if the api_server_count is greater than 1 (#22227) · e680723e
Chauncey authored Sep 09, 2025
```
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
Co-authored-by: Nick Hill <nhill@redhat.com>
```
e680723e

[Attention] FlashAttention MLA cudagraph support (#23958) · 620db1fc

Matthew Bonanni authored Sep 08, 2025


Signed-off-by: Matthew Bonanni <mbonanni001@gmail.com>
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com>

620db1fc

[Spec Decode] Fix offline spec_decode.py (#24257) · 41183c1f

Ekagra Ranjan authored Sep 08, 2025


Signed-off-by: Ekagra Ranjan <3116519+ekagra-ranjan@users.noreply.github.com>
Co-authored-by: Roger Wang <hey@rogerw.io>

41183c1f

[Model loader]: support multi-thread model weight loading (#23928) · 43d9ad03

Yang Kaiyong authored Sep 09, 2025


Signed-off-by: Yang Kaiyong <yangkaiyong.yky@antgroup.com>
Signed-off-by: Simon Mo <simon.mo@hey.com>
Co-authored-by: Simon Mo <simon.mo@hey.com>

43d9ad03

[Model] Remove quantized mixtral (#24437) · 8d7f39b4
Jee Jee Li authored Sep 09, 2025
```
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
```
8d7f39b4

[Spec Decode][Benchmark] Add Blitzedit dataset (#23605) · cd086369

Ekagra Ranjan authored Sep 08, 2025


Signed-off-by: Ekagra Ranjan <3116519+ekagra-ranjan@users.noreply.github.com>
Co-authored-by: Roger Wang <hey@rogerw.io>

cd086369

[Spec Decode][Benchmark] Add Spec Bench Dataset for benchmarking (#23563) · 3feeeb9f
Ekagra Ranjan authored Sep 08, 2025
```
Signed-off-by: Ekagra Ranjan <3116519+ekagra-ranjan@users.noreply.github.com>
```
3feeeb9f
[Model] Enable BNB support for qwen2_5_omni_thinker (#24420) · 6f4a82f8
Jee Jee Li authored Sep 09, 2025
```
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
```
6f4a82f8

[Frontend] User-provided uuids for medias in chat. (RFC #22044) (#23449) · 01dfb5e9

Chenheli Hua authored Sep 08, 2025


Signed-off-by: Roger Wang <hey@rogerw.io>
Signed-off-by: Chenheli Hua <huachenheli@outlook.com>
Signed-off-by: Roger Wang <hey@rogerw.me>
Signed-off-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
Co-authored-by: Roger Wang <hey@rogerw.io>
Co-authored-by: Roger Wang <hey@rogerw.me>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>

01dfb5e9

Move `KVEventsConfig` from `config/__init__.py` to `config/kv_events.py` (#24433) · 03dd652c
Harry Mellor authored Sep 08, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
03dd652c

[Bugfix] Fix mamba2 prefill chunking (#23279) · e0413141

tomeras91 authored Sep 08, 2025

Signed-off-by: Tomer Asida <57313761+tomeras91@users.noreply.github.com>
Signed-off-by: tomeras91 <57313761+tomeras91@users.noreply.github.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>

e0413141

[Bugfix] Fix get_quant_config when using modelscope (#24421) · 5e537f45
Li Wang authored Sep 08, 2025
```
Signed-off-by: wangli <wangli858794774@gmail.com>
```
5e537f45
[Doc]: fix typos in Python comments (#24417) · f4962a6d
Didier Durand authored Sep 08, 2025
```
Signed-off-by: Didier Durand <durand.didier@gmail.com>
```
f4962a6d
[gpt-oss][Responses API] Fix the function call id format (#24409) · 425b04b8
Chauncey authored Sep 08, 2025
```
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
```
425b04b8
[Model] Remove unnecessary CUDA sync of Qwen2VL image and video preprocess (#24334) · 60f0843e
Chatcharin Sangbutsarakum authored Sep 08, 2025
```
Signed-off-by: Win <chatcharinsang@gmail.com>
Co-authored-by: Roger Wang <hey@rogerw.io>
```
60f0843e
[Model] Remove unnecessary CUDA sync of GLM-4.1V image and video preprocess (#24332) · 8a466026
Chatcharin Sangbutsarakum authored Sep 08, 2025
```
Signed-off-by: Win <chatcharinsang@gmail.com>
Co-authored-by: Roger Wang <hey@rogerw.io>
```
8a466026
[P/D] Add a shutdown method to the Connector API (#22699) · 61aa4b29
Chauncey authored Sep 08, 2025
```
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
```
61aa4b29

[Sampler] Support returning all prompt logprobs (#23868) · b3d7e3c8

Xingyu Liu authored Sep 07, 2025


Signed-off-by: Xingyu Liu <charlotteliu12x@gmail.com>
Co-authored-by: 22quinn <33176974+22quinn@users.noreply.github.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>

b3d7e3c8

[Kernel] Support decode context parallelism on Blackwell with CUTLASS MLA (#24385) · 86173ad5

Ming Yang authored Sep 07, 2025


Signed-off-by: Ming Yang <minos.future@gmail.com>
Signed-off-by: youkaichao <youkaichao@gmail.com>
Co-authored-by: youkaichao <youkaichao@gmail.com>

86173ad5

07 Sep, 2025 8 commits
- Skip MM Encoder for non-first PP ranks (#24387) · 2e5d2137
  Woosuk Kwon authored Sep 07, 2025
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  2e5d2137
- Add renderer-based prompt processing for embedding and classification endpoints (#24356) · 0661cb9d
  Flora Feng authored Sep 07, 2025
```
Signed-off-by: sfeng33 <4florafeng@gmail.com>
```
  0661cb9d
- [TPU] Remove TopKTopPSampler dependency for TPU sampler (#24391) · 105d3d62
  Woosuk Kwon authored Sep 07, 2025
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  105d3d62
- [Bugfix] Fix Qwen3-coder moe tuned config (#24072) · 62f66be1
  Jee Jee Li authored Sep 07, 2025
```
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
```
  62f66be1
- [Misc] collect flashinfer version in collect_env.py (#24378) · 81c53ef5
  Ye (Charlotte) Qi authored Sep 06, 2025
```
Signed-off-by: Ye (Charlotte) Qi <yeq@meta.com>
```
  81c53ef5
- QWEN3 Thinking Fused MoE kernels Optimization configs (#24330) · 75334956
  Saman A. Pour authored Sep 06, 2025
```
Signed-off-by: Saman Keon <samanamp@outlook.com>
```
  75334956
- Migrate Qwen2 inputs to TensorSchema (#23475) · 37a6fa95
  Benji Beck authored Sep 06, 2025
```
Signed-off-by: Benji Beck <benjibeck@meta.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
```
  37a6fa95
- [attention][DCP] use AttentionImpl.need_to_return_lse_for_decode (#24372) · 558f0907
  youkaichao authored Sep 07, 2025
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  558f0907
06 Sep, 2025 6 commits
- [V0 deprecation] Deprecate V0 Neuron backend (#21159) · 4172235a
  Woosuk Kwon authored Sep 06, 2025
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  4172235a
- break execute_model in gpu_model_runner into sub-functions for custom scopes (#24265) · 848562bd
  Bangsheng Tang authored Sep 06, 2025
```
Co-authored-by: Bangsheng Tang <bangsheng@meta.com>
```
  848562bd
- [Frontend][Responses API] Support reporting tool output tokens and fix... · a3645ed9
  Ye (Charlotte) Qi authored Sep 06, 2025
```
[Frontend][Responses API] Support reporting tool output tokens and fix reasoning token count (#24285)
Signed-off-by: Ye (Charlotte) Qi <yeq@meta.com>
```
  a3645ed9
- [Fix] [gpt-oss] fix non-tool calling path for chat completion (#24324) · fb691ee4
  Aaron Pham authored Sep 06, 2025
  
  fb691ee4
- Lora bias(enable_lora_bias) deprecate warning (#24339) · 6024d115
  Ashwin Phadke authored Sep 06, 2025
```
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
Co-authored-by: Jee Jee Li <pandaleefree@gmail.com>
```
  6024d115
- [Bugfix] Fix broken deepseek fp8 TP weights loading (#24367) · 00a4e56d
  Isotr0py authored Sep 07, 2025
```
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
```
  00a4e56d