Commits · 3d9a1d2de5091455bb2fbf6b21fc9188fd4612a4 · OpenDAS / vllm_cscc

"vscode:/vscode.git/clone" did not exist on "4863e5fba51b8e1a5012e2a7582aece0ca575b89"

20 Sep, 2025 14 commits
- [V1] Support `LLM.apply_model` (#18465) · 3d9a1d2d
  Cyrus Leung authored Sep 20, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  3d9a1d2d
- [Bugfix] Fix Qwen3-VL-MoE weight loading for EP (#25300) · be874c02
  Roger Wang authored Sep 20, 2025
```
Signed-off-by: Roger Wang <hey@rogerw.io>
```
  be874c02
- [Hybrid Allocator] Support full attention with different hidden size (#25101) · 9607d5eb
  Chen Zhang authored Sep 19, 2025
```
Signed-off-by: Chen Zhang <zhangch99@outlook.com>
```
  9607d5eb
- [Optimization] Avoid repeated model architecture conversion for pooling models (#25261) · c60e6137
  Cyrus Leung authored Sep 20, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  c60e6137
- [Bugfix] fix tool call arguments is empty (#25223) · f91480b2
  Chauncey authored Sep 20, 2025
```
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
Co-authored-by: xin.li <xin.li@daocloud.io>
```
  f91480b2
- [BUG FIX][NON-CUDA]quick fix to avoid call cudagraph_unsafe in attention (#25298) · 6c5f82e5
  Chendi.Xue authored Sep 19, 2025
```
Signed-off-by: Chendi Xue <Chendi.Xue@intel.com>
```
  6c5f82e5
- [BugFix] Exclude self when checking for port collision (#25286) · b7f186bb
  Nick Hill authored Sep 19, 2025
```
Signed-off-by: Nick Hill <nhill@redhat.com>
```
  b7f186bb
- [BUGFIX] GPTQ quantization compatibility for Qwen3 Next MOE models (AutoGPTQ... · 36429096
  JartX authored Sep 20, 2025
```
[BUGFIX] GPTQ quantization compatibility for Qwen3 Next MOE models (AutoGPTQ and AutoRound-GPTQ) (#25268)
Signed-off-by: JartX <sagformas@epdcenter.es>
```
  36429096
- Improve weight loading for encoder models in Transformers backend (#25289) · c308501c
  Harry Mellor authored Sep 20, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  c308501c
- [Misc] Support more collective_rpc return types (#25294) · 535d8005
  Nick Hill authored Sep 19, 2025
```
Signed-off-by: Nick Hill <nhill@redhat.com>
```
  535d8005
- [BugFix] Ensure appropriate guards in destructors (#25284) · a25ade5d
  Nick Hill authored Sep 19, 2025
```
Signed-off-by: Nick Hill <nhill@redhat.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
```
  a25ade5d
- [torch.compile] CUDAGraph Inductor partition integration (#24281) · 8945b001
  Boyuan Feng authored Sep 19, 2025
```
Signed-off-by: Boyuan Feng <boyuan@meta.com>
Signed-off-by: Boyuan Feng <fby.1994@gmail.com>
Signed-off-by: boyuanfeng <boyuan@meta.com>
Co-authored-by: Luka Govedič <ProExpertProg@users.noreply.github.com>
```
  8945b001
- Don't skip special tokens with hermes-style tool calling (#25281) · a36c6758
  Maximilien de Bayser authored Sep 19, 2025
```
Signed-off-by: Max de Bayser <mbayser@br.ibm.com>
```
  a36c6758
- [Bugfix] Remove VLLM_TEST_DYNAMO_FULLGRAPH_CAPTURE #2969 (#25090) · 3da17c2c
  Lucas Kabela authored Sep 19, 2025
```
Signed-off-by: Lucas Kabela <lucaskabela@meta.com>
```
  3da17c2c
19 Sep, 2025 26 commits
- [BugFix] Fix async scheduling CPU tensor race take 2 (#25279) · 14c14327
  Nick Hill authored Sep 19, 2025
```
Signed-off-by: Nick Hill <nhill@redhat.com>
```
  14c14327
- allow disable flashinfer prefill (#25276) · ee7a66dd
  Lucia Fang authored Sep 19, 2025
```
Signed-off-by: Lu Fang <fanglu@fb.com>
```
  ee7a66dd
- Enable modelopt gemma3 nvfp4/fp8, make workflow more robust (#22771) · 431535b5
  Zhiyu authored Sep 19, 2025
```
Signed-off-by: Zhiyu Cheng <zhiyuc@nvidia.com>
Signed-off-by: Michael Goin <mgoin64@gmail.com>
Co-authored-by: Michael Goin <mgoin64@gmail.com>
```
  431535b5
- [Frontend] Responses API messages out, just harmony for now (#24985) · e69e0b8b
  Alec S authored Sep 19, 2025
```
Signed-off-by: Alec Solder <alecs@fb.com>
Co-authored-by: Alec Solder <alecs@fb.com>
Co-authored-by: Ye (Charlotte) Qi <yeq@meta.com>
```
  e69e0b8b
- Fix: Correct FusedMoE layer reference in auto_round quantization (#24818) · ddc90483
  David-Wen authored Sep 20, 2025
```
Signed-off-by: David-Wen <18927700430@163.com>
Signed-off-by: Michael Goin <mgoin64@gmail.com>
Co-authored-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com>
Co-authored-by: Michael Goin <mgoin64@gmail.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
```
  ddc90483
- [BugFix] Make FlashInferMetadataBuilder non-blocking (#25040) · b1a63d1b
  nvjullin authored Sep 20, 2025
```
Signed-off-by: Julien Lin <jullin@nvidia.com>
Co-authored-by: Michael Goin <mgoin64@gmail.com>
```
  b1a63d1b
- [Perf] Use FlashInfer RoPE for RotaryEmbedding.forward_cuda when available (#21126) · 48ecb443
  Michael Goin authored Sep 19, 2025
```
Signed-off-by: mgoin <mgoin64@gmail.com>
Signed-off-by: Michael Goin <mgoin64@gmail.com>
Co-authored-by: Luka Govedič <ProExpertProg@users.noreply.github.com>
```
  48ecb443
- [Bugfix] Fix chunked a2_scales in modular kernels (#25264) · 4bdf4002
  bnellnm authored Sep 19, 2025
```
Signed-off-by: Bill Nell <bnell@redhat.com>
```
  4bdf4002
- [Bugfix] GPT OSS Attritbute error on H100 (#25228) · 7852b82b
  Varun Sundar Rabindranath authored Sep 19, 2025
```
Signed-off-by: Varun Sundar Rabindranath <vsundarr@redhat.com>
Co-authored-by: Varun Sundar Rabindranath <vsundarr@redhat.com>
```
  7852b82b
- Optimize triton unified attention performance for sliding window attention (#24390) · a2a5f79e
  qizixi authored Sep 19, 2025
```
Signed-off-by: zixi-qi <qizixi@meta.com>
```
  a2a5f79e
- [KV offload][4/N] Offloading KV connector (#22595) · c59a0eca
  Or Ozeri authored Sep 19, 2025
```
Signed-off-by: Or Ozeri <oro@il.ibm.com>
```
  c59a0eca
- [bugfix] fix structured outputs key missing issue from #24929 (#25195) · b716ab93
  Lucia Fang authored Sep 19, 2025
```
Signed-off-by: Lu Fang <fanglu@fb.com>
```
  b716ab93
- [Docs] add __init__.py to... · 138f0d1e
  samzong authored Sep 20, 2025
```
[Docs] add __init__.py to vllm/model_executor/layers/quantization/compressed_tensors/transform (#24974)
Signed-off-by: samzong <samzong.lu@gmail.com>
```
  138f0d1e
- [Core][Prefix Hash] Fix prefix hash metrics sliding window maintainance (#24990) · 2506ce51
  Jialin Ouyang authored Sep 19, 2025
```
Signed-off-by: Jialin Ouyang <Jialin.Ouyang@gmail.com>
```
  2506ce51
- Encoder model support for the Transformers backend (#25174) · 12aed7e4
  Harry Mellor authored Sep 19, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  12aed7e4
- Remove Redundant Assignment in Qwen3_VisionPatchMerger (#25224) · d90e212a
  LJH-LBJ authored Sep 20, 2025
```
Signed-off-by: Junhong <liujunhong11@huawei.com>
Co-authored-by: Junhong <liujunhong11@huawei.com>
Co-authored-by: Roger Wang <hey@rogerw.io>
```
  d90e212a
- [Core] Modify the initialization parameters of the lora manager (#25249) · 28219864
  Jee Jee Li authored Sep 20, 2025
```
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
```
  28219864
- [Frontend] Pass API server count to each process (#23717) · 6c117cff
  Cyrus Leung authored Sep 20, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  6c117cff
- [KV offload][3/N] Add worker-side CPU support (#21448) · 7ac67ea5
  Or Ozeri authored Sep 19, 2025
```
Signed-off-by: Or Ozeri <oro@il.ibm.com>
```
  7ac67ea5
- refactor(benchmarks): add type annotations to wait_for_endpoint parameters (#25218) · ce75e153
  samzong authored Sep 20, 2025
```
Signed-off-by: samzong <samzong.lu@gmail.com>
```
  ce75e153
- Move `ModelConfig` from `config/__init__.py` to `config/model.py` (#25252) · aed16879
  Harry Mellor authored Sep 19, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  aed16879
- [Qwen] Remove cuda hard-code in qwen3 next (#25243) · 838d7116
  Icey authored Sep 19, 2025
```
Signed-off-by: Icey <1790571317@qq.com>
```
  838d7116
- [V0 Deprecation] Remove V0 logic from `get_input_embeddings` interface (#25242) · 5089fd74
  Cyrus Leung authored Sep 19, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  5089fd74
- [P/D][Nixl] Introduce `KVTransferMetrics` and aggregation strategy (#22188) · a3d087ad
  Nicolò Lucchesi authored Sep 19, 2025
```
Signed-off-by: NickLucche <nlucches@redhat.com>
```
  a3d087ad
- Move `PoolerConfig` from `config/__init__.py` to `config/pooler.py` (#25181) · 058525b9
  Harry Mellor authored Sep 19, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  058525b9
- [Bugfix][Perf] Misc fixes for Qwen3 VL (#25238) · 1dfea5f4
  Roger Wang authored Sep 19, 2025
```
Signed-off-by: Roger Wang <hey@rogerw.io>
```
  1dfea5f4