Commits · 8945b001db3202f882108e50d16b6f9c5e6f01ed · OpenDAS / vllm_cscc

20 Sep, 2025 1 commit

[torch.compile] CUDAGraph Inductor partition integration (#24281) · 8945b001

Boyuan Feng authored Sep 19, 2025


Signed-off-by: Boyuan Feng <boyuan@meta.com>
Signed-off-by: Boyuan Feng <fby.1994@gmail.com>
Signed-off-by: boyuanfeng <boyuan@meta.com>
Co-authored-by: Luka Govedič <ProExpertProg@users.noreply.github.com>

8945b001

19 Sep, 2025 4 commits
- Enable modelopt gemma3 nvfp4/fp8, make workflow more robust (#22771) · 431535b5
  Zhiyu authored Sep 19, 2025
```
Signed-off-by: Zhiyu Cheng <zhiyuc@nvidia.com>
Signed-off-by: Michael Goin <mgoin64@gmail.com>
Co-authored-by: Michael Goin <mgoin64@gmail.com>
```
  431535b5
- [Frontend] Pass API server count to each process (#23717) · 6c117cff
  Cyrus Leung authored Sep 20, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  6c117cff
- Move `ModelConfig` from `config/__init__.py` to `config/model.py` (#25252) · aed16879
  Harry Mellor authored Sep 19, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  aed16879
- Move `PoolerConfig` from `config/__init__.py` to `config/pooler.py` (#25181) · 058525b9
  Harry Mellor authored Sep 19, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  058525b9
18 Sep, 2025 7 commits
- [Bug] Fix torch Compilation Cache Hit Error (#25093) · d2a30a2d
  Wentao Ye authored Sep 18, 2025
```
Signed-off-by: yewentao256 <zhyanwentao@126.com>
```
  d2a30a2d
- [spec decode] Fix MTP inference path for MiMo-7B model (#25136) · c4cb0af9
  qizixi authored Sep 18, 2025
```
Signed-off-by: zixi-qi <qizixi@meta.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
```
  c4cb0af9
- Fix `validate-config` pre-commit check (#25157) · 3ed1ec4a
  Harry Mellor authored Sep 18, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  3ed1ec4a
- Move `StructuredOutputsConfig` from `config/__init__.py` to `config/structured_outputs.py` (#25153) · 8ed039d5
  Harry Mellor authored Sep 18, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  8ed039d5
- [Chore] Cleanup guided namespace, move to structured outputs config (#22772) · 29283e89
  Aaron Pham authored Sep 18, 2025
```
Signed-off-by: Aaron Pham <contact@aarnphm.xyz>
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  29283e89
- [Bugfix] when use s3 model cannot use default load_format (#24435) · 350c94de
  rongfu.leng authored Sep 18, 2025
```
Signed-off-by: rongfu.leng <rongfu.leng@daocloud.io>
Co-authored-by: 22quinn <33176974+22quinn@users.noreply.github.com>
```
  350c94de
- [Spec Decode] Efficient padded speculation (#24539) · b7433ca1
  Benjamin Chislett authored Sep 18, 2025
```
Signed-off-by: Benjamin Chislett <bchislett@nvidia.com>
```
  b7433ca1
17 Sep, 2025 2 commits
- [BUG] Exclude .pth files when pulling remote files (#25092) · f20c3b09
  ahao-anyscale authored Sep 17, 2025
```
Signed-off-by: ahao-anyscale <ahao@anyscale.com>
```
  f20c3b09
- [UX] Remove "quantization is not fully optimized yet" log (#25012) · 67532a1a
  Michael Goin authored Sep 16, 2025
```
Signed-off-by: mgoin <mgoin64@gmail.com>
```
  67532a1a
16 Sep, 2025 4 commits

[Core/DBO][1/N] Add Dual-Batch Overlap mechanism to VLLM (#23693) · 56793995

Sage Moore authored Sep 16, 2025


Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Signed-off-by: Sage Moore <sage@neuralmagic.com>
Signed-off-by: Lucas Wilkinson <lwilkinson@neuralmagic.com>
Signed-off-by: yewentao256 <zhyanwentao@126.com>
Co-authored-by: Lucas Wilkinson <lwilkins@redhat.com>
Co-authored-by: Lucas Wilkinson <lwilkinson@neuralmagic.com>
Co-authored-by: yewentao256 <zhyanwentao@126.com>
Co-authored-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com>
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com>

56793995

Move `SpeculativeConfig` from `config/__init__.py` to `config/speculative.py` (#24904) · 0faf3cc3
Harry Mellor authored Sep 16, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
0faf3cc3

[Feat][EPLB] A novel static EPLB placement strategy for MoE models. (#23745) · 7ea5c73a

Chen Bruce authored Sep 16, 2025


Signed-off-by: bruceszchen <bruceszchen@tencent.com>
Signed-off-by: Chen Bruce <bruceszchen@tencent.com>
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Signed-off-by: Chen Bruce <cszwwdz@vip.qq.com>
Co-authored-by: lemon412 <lemon412@foxmail.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>

7ea5c73a

Remove V0 Encoder-Decoder Support (#24907) · 759ef49b
Woosuk Kwon authored Sep 15, 2025
```
Signed-off-by: Woosuk Kwon <woosuk@thinkingmachines.ai>
```
759ef49b

15 Sep, 2025 1 commit

Move `MultiModalConfig` from `config/__init__.py` to `config/multimodal.py` (#24659) · c4afdb69

Harry Mellor authored Sep 15, 2025


Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>

c4afdb69

12 Sep, 2025 2 commits
- [Core] Shared memory based object store for Multimodal data caching and IPC (#20452) · a5b84f1c
  dongluw authored Sep 12, 2025
```
Signed-off-by: donglu <donglu@cohere.com>
```
  a5b84f1c
- [CI] Add ci_envs for convenient local testing (#24630) · d21a36f5
  wang.yuqi authored Sep 12, 2025
```
Signed-off-by: wang.yuqi <noooop@126.com>
```
  d21a36f5
11 Sep, 2025 10 commits

[Doc] Clarify cudagraph capture size logic and default behavior in scheduler (#18698) · 7a30fa87

Zazzle516 authored Sep 12, 2025

Signed-off-by: Zazzle516 <2405677060@qq.com>
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>

7a30fa87

[HybridKVCache][Platform] Add support_hybrid_kv_cache for platform (#24646) · 4f6593b0
Mengqing Cao authored Sep 11, 2025
```
Signed-off-by: MengqingCao <cmq0113@163.com>
```
4f6593b0

Allow users to specify kv cache memory size (#21489) · 94e6b2d5

Boyuan Feng authored Sep 11, 2025

Signed-off-by: Boyuan Feng <boyuan@meta.com>
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>

94e6b2d5

Move `LoRAConfig` from `config/__init__.py` to `config/lora.py` (#24644) · 5f5271f1
Harry Mellor authored Sep 11, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
5f5271f1
Fix typing for `safetensors_load_strategy` (#24641) · d6249d06
Harry Mellor authored Sep 11, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
d6249d06
[CI Failure] fix models/language/pooling/test_auto_prefix_cache_support.py (#24636) · 25bb9e8c
wang.yuqi authored Sep 11, 2025
```
Signed-off-by: wang.yuqi <noooop@126.com>
```
25bb9e8c
Add the support for the qwen3 next model (a hybrid attention model). (#24526) · e93f4cc9
Tao He authored Sep 11, 2025
```
Signed-off-by: Tao He <linzhu.ht@alibaba-inc.com>
Co-authored-by: Jee Jee Li <pandaleefree@gmail.com>
```
e93f4cc9
[Doc]: fixing doc typos (#24635) · e2b1f863
Didier Durand authored Sep 11, 2025
```
Signed-off-by: Didier Durand <durand.didier@gmail.com>
```
e2b1f863

[Core] feat: Add --safetensors-load-strategy flag for faster safetensors... · 41329a0f

shengshiqi-google authored Sep 11, 2025


[Core] feat: Add --safetensors-load-strategy flag for faster safetensors loading from Lustre (#24469)
Signed-off-by: Shiqi Sheng <shengshiqi@google.com>
Signed-off-by: shengshiqi-google <160179165+shengshiqi-google@users.noreply.github.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>

41329a0f

[Ultravox] Fix Gemma instantiation, support quantization via --hf-overrides (#24131) · f17a6aa4
Peter Salas authored Sep 10, 2025
```
Signed-off-by: Peter Salas <peter@fixie.ai>
```
f17a6aa4

10 Sep, 2025 8 commits

[v1] Add Whisper model support (encoder-decoder) (#21088) · 37e8182b
Russell Bryant authored Sep 10, 2025
```
Signed-off-by: Russell Bryant <rbryant@redhat.com>
Co-authored-by: NickLucche <nlucches@redhat.com>
```
37e8182b

[Core] Support configuration parsing plugin (#24277) · 9fb74c27

Xingyu Liu authored Sep 10, 2025

Signed-off-by: Xingyu Liu <charlotteliu12x@gmail.com>
Signed-off-by: Xingyu Liu <38244988+charlotte12l@users.noreply.github.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>

9fb74c27

Move `LoadConfig` from `config/__init__.py` to `config/load.py` (#24566) · f36355ab
Harry Mellor authored Sep 10, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
f36355ab
Support for NemotronH Nano VLM (#23644) · 72d30108
danielafrimi authored Sep 10, 2025
```
Signed-off-by: Daniel Afrimi <danielafrimi8@gmail.com>
```
72d30108
[Bugfix] Guard `_may_reorder_batch` for encoder-only models on CPU (#24319) (#24348) · feaf202e
Remy authored Sep 10, 2025
```
Signed-off-by: Remy <eunhwan.shin@dtonic.io>
Co-authored-by: Li, Jiang <jiang1.li@intel.com>
```
feaf202e

[Bugfix] Update Run:AI Model Streamer Loading Integration (#23845) · 4377b1ae

pwschuurman authored Sep 09, 2025


Signed-off-by: Omer Dayan (SW-GPU) <omer@run.ai>
Signed-off-by: Peter Schuurman <psch@google.com>
Co-authored-by: Omer Dayan (SW-GPU) <omer@run.ai>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>

4377b1ae

[BugFix][Spec Decode] Fix out-of-range index triggered by eagle3; re-enable... · 53b42f41

Wenlong Wang authored Sep 09, 2025


[BugFix][Spec Decode] Fix out-of-range index triggered by eagle3; re-enable test for LlamaForCausalLMEagle3 (#24392)
Signed-off-by: wwl2755 <wangwenlong2755@gmail.com>

53b42f41

[Bugfix] Improve EPLB config validation error message (#24524) · 561f38dc
Tyler Michael Smith authored Sep 09, 2025
```
Signed-off-by: Tyler Michael Smith <tyler@neuralmagic.com>
```
561f38dc

09 Sep, 2025 1 commit
- [Model] Systematic support for fp32 head, pooling models part (#23810) · 19332c04
  wang.yuqi authored Sep 09, 2025
```
Signed-off-by: wang.yuqi <noooop@126.com>
```
  19332c04