Commits · 239ef0c1ac0dfe68d8d2e28c54ecf9aa9bcd945b · OpenDAS / vllm_cscc

22 Sep, 2025 5 commits
- [CLI env var] Add VLLM_FLASH_ATTN_MAX_NUM_SPLITS_FOR_CUDA_GRAPH in env variables (#25274) · cfbee3d0
  Daisy-Ma-coder authored Sep 22, 2025
```
Signed-off-by: qqma <qqma@amazon.com>
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Co-authored-by: qqma <qqma@amazon.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
```
  cfbee3d0
- [V1][Attention] Split triton_attn in triton-only and rocm specific backends (#24648) · 175811e3
  Burkhard Ringlein authored Sep 22, 2025
```
Signed-off-by: Burkhard Ringlein <ngl@zurich.ibm.com>
```
  175811e3
- [TPU] Deprecate `xm.mark_step` in favor of ``torch_xla.sync` (#25254) · 4cf71cc8
  Nicolò Lucchesi authored Sep 22, 2025
```
Signed-off-by: NickLucche <nlucches@redhat.com>
Co-authored-by: Ye (Charlotte) Qi <yeq@meta.com>
```
  4cf71cc8
- Enable Eagle3 speculative decoding for GPT-OSS model (#25246) · 21467f9a
  Eldar Kurtić authored Sep 22, 2025
```
Signed-off-by: Eldar Kurtic <8884008+eldarkurtic@users.noreply.github.com>
```
  21467f9a
- [V0 Deprecation] Remove `MultiModalPlaceholderMap` (#25366) · f92d9526
  Cyrus Leung authored Sep 22, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  f92d9526
21 Sep, 2025 5 commits
- [V0 Deprecation] Remove V0 sampling metadata (#25345) · 1c3ffdbe
  Woosuk Kwon authored Sep 21, 2025
```
Signed-off-by: Woosuk Kwon <woosuk@thinkingmachines.ai>
```
  1c3ffdbe
- [V1] Add sliding window support to Flex Attention backend (#24089) · cf56cf78
  Isotr0py authored Sep 21, 2025
```
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
```
  cf56cf78
- [V0 Deprecation] Remove V0 MP executor (#25329) · 7ed82d19
  Woosuk Kwon authored Sep 20, 2025
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  7ed82d19
- [V0 Deprecation] Remove V0 core (#25321) · c99db8c8
  Woosuk Kwon authored Sep 20, 2025
```
Signed-off-by: Woosuk Kwon <woosuk@thinkingmachines.ai>
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  c99db8c8
- [V0 Deprecation] Remove V0 Output Processor (#25320) · 86647d1c
  Woosuk Kwon authored Sep 20, 2025
```
Signed-off-by: Woosuk Kwon <woosuk@thinkingmachines.ai>
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  86647d1c
20 Sep, 2025 7 commits
- [Docs] Fix warnings in mkdocs build (continued) (#25042) · 032d661d
  Wenlong Wang authored Sep 20, 2025
```
Signed-off-by: wwl2755 <wangwenlong2755@gmail.com>
```
  032d661d
- [V1] Support `LLM.apply_model` (#18465) · 3d9a1d2d
  Cyrus Leung authored Sep 20, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  3d9a1d2d
- [Hybrid Allocator] Support full attention with different hidden size (#25101) · 9607d5eb
  Chen Zhang authored Sep 19, 2025
```
Signed-off-by: Chen Zhang <zhangch99@outlook.com>
```
  9607d5eb
- [Misc] Support more collective_rpc return types (#25294) · 535d8005
  Nick Hill authored Sep 19, 2025
```
Signed-off-by: Nick Hill <nhill@redhat.com>
```
  535d8005
- [BugFix] Ensure appropriate guards in destructors (#25284) · a25ade5d
  Nick Hill authored Sep 19, 2025
```
Signed-off-by: Nick Hill <nhill@redhat.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
```
  a25ade5d
- [torch.compile] CUDAGraph Inductor partition integration (#24281) · 8945b001
  Boyuan Feng authored Sep 19, 2025
```
Signed-off-by: Boyuan Feng <boyuan@meta.com>
Signed-off-by: Boyuan Feng <fby.1994@gmail.com>
Signed-off-by: boyuanfeng <boyuan@meta.com>
Co-authored-by: Luka Govedič <ProExpertProg@users.noreply.github.com>
```
  8945b001
- [Bugfix] Remove VLLM_TEST_DYNAMO_FULLGRAPH_CAPTURE #2969 (#25090) · 3da17c2c
  Lucas Kabela authored Sep 19, 2025
```
Signed-off-by: Lucas Kabela <lucaskabela@meta.com>
```
  3da17c2c
19 Sep, 2025 13 commits
- [BugFix] Fix async scheduling CPU tensor race take 2 (#25279) · 14c14327
  Nick Hill authored Sep 19, 2025
```
Signed-off-by: Nick Hill <nhill@redhat.com>
```
  14c14327
- allow disable flashinfer prefill (#25276) · ee7a66dd
  Lucia Fang authored Sep 19, 2025
```
Signed-off-by: Lu Fang <fanglu@fb.com>
```
  ee7a66dd
- [BugFix] Make FlashInferMetadataBuilder non-blocking (#25040) · b1a63d1b
  nvjullin authored Sep 20, 2025
```
Signed-off-by: Julien Lin <jullin@nvidia.com>
Co-authored-by: Michael Goin <mgoin64@gmail.com>
```
  b1a63d1b
- [KV offload][4/N] Offloading KV connector (#22595) · c59a0eca
  Or Ozeri authored Sep 19, 2025
```
Signed-off-by: Or Ozeri <oro@il.ibm.com>
```
  c59a0eca
- [bugfix] fix structured outputs key missing issue from #24929 (#25195) · b716ab93
  Lucia Fang authored Sep 19, 2025
```
Signed-off-by: Lu Fang <fanglu@fb.com>
```
  b716ab93
- [Core][Prefix Hash] Fix prefix hash metrics sliding window maintainance (#24990) · 2506ce51
  Jialin Ouyang authored Sep 19, 2025
```
Signed-off-by: Jialin Ouyang <Jialin.Ouyang@gmail.com>
```
  2506ce51
- [Core] Modify the initialization parameters of the lora manager (#25249) · 28219864
  Jee Jee Li authored Sep 20, 2025
```
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
```
  28219864
- [Frontend] Pass API server count to each process (#23717) · 6c117cff
  Cyrus Leung authored Sep 20, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  6c117cff
- [KV offload][3/N] Add worker-side CPU support (#21448) · 7ac67ea5
  Or Ozeri authored Sep 19, 2025
```
Signed-off-by: Or Ozeri <oro@il.ibm.com>
```
  7ac67ea5
- Move `ModelConfig` from `config/__init__.py` to `config/model.py` (#25252) · aed16879
  Harry Mellor authored Sep 19, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  aed16879
- [P/D][Nixl] Introduce `KVTransferMetrics` and aggregation strategy (#22188) · a3d087ad
  Nicolò Lucchesi authored Sep 19, 2025
```
Signed-off-by: NickLucche <nlucches@redhat.com>
```
  a3d087ad
- [KV offload][2/N] Introduce LRU-based CPU offloading management (#20075) · 9d1c50a5
  Or Ozeri authored Sep 19, 2025
```
Signed-off-by: Or Ozeri <oro@il.ibm.com>
```
  9d1c50a5
- [CORE] Prompt Embeddings Support for v1 Engine (#24278) · 9a4600e4
  Andrew Sansom authored Sep 18, 2025
```
Signed-off-by: Andrew Sansom <andrew@protopia.ai>
Signed-off-by: Andrew Sansom <qthequartermasterman@gmail.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
```
  9a4600e4
18 Sep, 2025 10 commits

[KV offload][1b/N] rename offloading to kv_offload (#25191) · a53ad626
Or Ozeri authored Sep 18, 2025
```
Signed-off-by: Or Ozeri <oro@il.ibm.com>
```
a53ad626

[Bug] Fix `returned_lse` not Defined issue (#25106) · 75fb112d

Wentao Ye authored Sep 18, 2025


Signed-off-by: yewentao256 <zhyanwentao@126.com>
Co-authored-by: Tyler Michael Smith <tyler@neuralmagic.com>

75fb112d

[feat]: Create interface for model-specific M-RoPE (#24194) · 38db529f

Aziz authored Sep 18, 2025

Signed-off-by: AzizCode92 <azizbenothman76@gmail.com>
Signed-off-by: Aziz <azizbenothman76@gmail.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>

38db529f

[KV offload][1/N] Introduce an offloading component (#19848) · 505805b6
Or Ozeri authored Sep 18, 2025
```
Signed-off-by: Or Ozeri <oro@il.ibm.com>
```
505805b6
[ROCm][AITER][Bugfix] Switch AITER to use PIECEWISE_AND_FULL compilation (#25104) · bbdc0f23
Rohan Potdar authored Sep 18, 2025
```
Signed-off-by: Rohan138 <rohanpotdar138@gmail.com>
```
bbdc0f23

[New Model] Support BertForTokenClassification / Named Entity Recognition (NER) task (#24872) · 5f696c33

wang.yuqi authored Sep 18, 2025


Signed-off-by: wang.yuqi <noooop@126.com>
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
Co-authored-by: Isotr0py <mozf@mail2.sysu.edu.cn>

5f696c33

[PERF] Add `conv1d` metadata to GDN attn (#25105) · 072d7e53
Vadim Gimpelson authored Sep 18, 2025
```
Signed-off-by: Vadim Gimpelson <vadim.gimpelson@gmail.com>
```
072d7e53

[Structured Output][Refactor] Move `apply_grammar_bitmask()` method from... · 470484a4

Shanshan Shen authored Sep 18, 2025


[Structured Output][Refactor] Move `apply_grammar_bitmask()` method from `ModelRunner` to structured output utils (#21999)
Signed-off-by: shen-shanshan <467638484@qq.com>

470484a4

[Model] Improve Pooling Model (#25149) · 37970105
Jee Jee Li authored Sep 18, 2025
```
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
```
37970105

[Chore] Cleanup guided namespace, move to structured outputs config (#22772) · 29283e89

Aaron Pham authored Sep 18, 2025

Signed-off-by: Aaron Pham <contact@aarnphm.xyz>
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>

29283e89