Commits · 136a17fe6edacba292d899af4cfd424e0ca98d9f · OpenDAS / vllm_cscc

15 Oct, 2025 5 commits

[Chore] Separate out `vllm.utils.func` (#26904) · 136a17fe
Cyrus Leung authored Oct 15, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
136a17fe

[BugFix] Patch inductor memory plan logic (#26878) · f5743833

Boyuan Feng authored Oct 15, 2025

Signed-off-by: Boyuan Feng <boyuan@meta.com>
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>

f5743833

[Misc] rename torch_dtype to dtype (#26695) · 8f4b313c
wangxiyuan authored Oct 15, 2025
```
Signed-off-by: wangxiyuan <wangxiyuan1007@gmail.com>
```
8f4b313c

[Frontend][torch.compile] CompilationConfig Overhaul (#20283): name change ... · 96b9aa5a

Morrison Turnansky authored Oct 14, 2025


[Frontend][torch.compile] CompilationConfig Overhaul (#20283): name change  compilation level to compilation mode, deprecation compilation level (#26355)
Signed-off-by: morrison-turnansky <mturnans@redhat.com>
Signed-off-by: Morrison Turnansky <mturnans@redhat.com>
Co-authored-by: Luka Govedič <ProExpertProg@users.noreply.github.com>

96b9aa5a

[Core][Easy] Use envs.__getattr__ for all Unify to environment variable access (#26810) · 07ca70af
Jialin Ouyang authored Oct 14, 2025
```
Signed-off-by: Jialin Ouyang <Jialin.Ouyang@gmail.com>
```
07ca70af

13 Oct, 2025 1 commit
- [Log] Optimize Startup Log (#26601) · e251e457
  Wentao Ye authored Oct 13, 2025
```
Signed-off-by: yewentao256 <zhyanwentao@126.com>
```
  e251e457
12 Oct, 2025 1 commit
- Update `Optional[x]` -> `x | None` and `Union[x, y]` to `x | y` (#26633) · 8fcaaf6a
  Harry Mellor authored Oct 12, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  8fcaaf6a
09 Oct, 2025 4 commits
- [UX] Add FlashInfer as default CUDA dependency (#26443) · c9d33c60
  Michael Goin authored Oct 09, 2025
```
Signed-off-by: mgoin <mgoin64@gmail.com>
Co-authored-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com>
```
  c9d33c60
- [Flashinfer][gpt-oss] Support FP8-qkv Flashinfer TRTLLM Sinks Attention (#25674) · 44f633db
  elvischenv authored Oct 10, 2025
```
Signed-off-by: elvischenv <219235043+elvischenv@users.noreply.github.com>
```
  44f633db
- [Bugfix] Enable padded FP4 quantization (#25947) · 4069db3f
  roikoren755 authored Oct 09, 2025
```
Signed-off-by: Roi Koren <roik@nvidia.com>
```
  4069db3f
- [Misc] Upgrade more code to Python 3.10 (#26463) · dc7976dd
  Cyrus Leung authored Oct 09, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  dc7976dd
08 Oct, 2025 1 commit

[Feature] Enable E8M0 by Default on Hopper for DeepGEMM, 5% E2E throughput improvement (#26197) · f8607863

Wentao Ye authored Oct 08, 2025

Signed-off-by: yewentao256 <zhyanwentao@126.com>
Signed-off-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>

f8607863

07 Oct, 2025 3 commits
- [Bugfix] Fix MTP+FlashInfer crash when trtllm kernels are available but disabled (#26361) · caf8b1c0
  Benjamin Chislett authored Oct 07, 2025
```
Signed-off-by: Benjamin Chislett <bchislett@nvidia.com>
Signed-off-by: Benjamin Chislett <chislett.ben@gmail.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
```
  caf8b1c0
- [Misc] Move `LRUCache` into its own file (#26342) · c0a7b89d
  Cyrus Leung authored Oct 07, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  c0a7b89d
- [Feature][OCP MX] Support mxfp6 and mixed mxfp6-mxfp4 (#21166) · 41f1cf38
  fxmarty-amd authored Oct 07, 2025
  
  41f1cf38
05 Oct, 2025 5 commits
- Fix per file ruff ignores related to simplification (#26259) · b893d661
  Harry Mellor authored Oct 05, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  b893d661
- [NVIDIA] flashinfer TRTLLM attention prefill token limit (#25998) · 6b6e9877
  Jason Li authored Oct 05, 2025
```
Signed-off-by: jasonlizhengjian <jason.li@centml.ai>
Signed-off-by: jasonlizhengjian <jasonlizhengjian@gmail.com>
```
  6b6e9877
- Fix per file ruff ignores related to typing (#26254) · 1c0c6820
  Harry Mellor authored Oct 05, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  1c0c6820
- Convert formatting to use `ruff` instead of `yapf` + `isort` (#26247) · d6953beb
  Harry Mellor authored Oct 05, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  d6953beb
- [Model] Use `merge_by_field_config` for MM models (H-L) (#26230) · 59a85c36
  Cyrus Leung authored Oct 05, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  59a85c36
04 Oct, 2025 1 commit
- [Model] Support nested structures for TensorSchema (#26212) · 44ea8513
  Cyrus Leung authored Oct 04, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  44ea8513
03 Oct, 2025 2 commits
- Fix undefined symbol: cutlass_moe_mm_sm100 (#26098) · 4f8f47e8
  Jun Jiang authored Oct 03, 2025
```
Signed-off-by: Jun Jiang <jasl9187@hotmail.com>
Co-authored-by: Luka Govedič <ProExpertProg@users.noreply.github.com>
```
  4f8f47e8
- [Log] Optimize DeepGEMM Missing Log (#26106) · 36ce76c6
  Wentao Ye authored Oct 02, 2025
```
Signed-off-by: yewentao256 <zhyanwentao@126.com>
```
  36ce76c6
02 Oct, 2025 2 commits

[Perf] Fix and reapply move apply w8a8 block fp8 linear to class (#25696) · 502640c3

ElizaWszola authored Oct 02, 2025


Signed-off-by: ElizaWszola <ewszola@redhat.com>
Signed-off-by: ElizaWszola <elizaw.9289@gmail.com>
Signed-off-by: Luka Govedič <lgovedic@redhat.com>
Signed-off-by: Luka Govedič <ProExpertProg@users.noreply.github.com>
Co-authored-by: Luka Govedič <ProExpertProg@users.noreply.github.com>
Co-authored-by: Michael Goin <mgoin64@gmail.com>
Co-authored-by: Luka Govedič <lgovedic@redhat.com>

502640c3

[CI/Build] Replace `vllm.entrypoints.openai.api_server` entrypoint with `vllm... · d00d6529

Cyrus Leung authored Oct 03, 2025


[CI/Build] Replace `vllm.entrypoints.openai.api_server` entrypoint with `vllm serve` command (#25967)
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>

d00d6529

30 Sep, 2025 2 commits

[Bugfix] Fix accuracy issue of TRTLLM FP8 MOE and improve logging (#25895) · ef283548
Pavani Majety authored Sep 30, 2025
```
Signed-off-by: Pavani Majety <pmajety@nvidia.com>
```
ef283548

[New Model] DeepSeek-V3.2 (Rebased to Main) (#25896) · fa7e254a

Yongye Zhu authored Sep 30, 2025


Signed-off-by: Chen Zhang <zhangch99@outlook.com>
Signed-off-by: youkaichao <youkaichao@gmail.com>
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Signed-off-by: mgoin <mgoin64@gmail.com>
Signed-off-by: NickLucche <nlucches@redhat.com>
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com>
Signed-off-by: Barry Kang <43644113+Barry-Delaney@users.noreply.github.com>
Signed-off-by: Lucia Fang <fanglu@meta.com>
Co-authored-by: Chen Zhang <zhangch99@outlook.com>
Co-authored-by: youkaichao <youkaichao@gmail.com>
Co-authored-by: Lucas Wilkinson <lwilkins@redhat.com>
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com>
Co-authored-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com>
Co-authored-by: yewentao256 <zhyanwentao@126.com>
Co-authored-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com>
Co-authored-by: mgoin <mgoin64@gmail.com>
Co-authored-by: Lucia Fang <116399278+luccafong@users.noreply.github.com>
Co-authored-by: Lucia Fang <fanglu@meta.com>
Co-authored-by: NickLucche <nlucches@redhat.com>
Co-authored-by: Siyuan Fu <siyuanf@nvidia.com>
Co-authored-by: Matthew Bonanni <mbonanni@redhat.com>
Co-authored-by: Xiaozhu Meng <mxz297@gmail.com>
Co-authored-by: Barry Kang <43644113+Barry-Delaney@users.noreply.github.com>

fa7e254a

27 Sep, 2025 1 commit

[Core] GC Debug callback (#24829) · c216119d

Jialin Ouyang authored Sep 27, 2025


Signed-off-by: Jialin Ouyang <jialino@meta.com>
Signed-off-by: Jialin Ouyang <Jialin.Ouyang@gmail.com>
Co-authored-by: Jialin Ouyang <jialino@meta.com>

c216119d

26 Sep, 2025 1 commit

[Misc] fix unique_filepath (#25732) · 56aafa8c

Jiangyun Zhu authored Sep 27, 2025


Signed-off-by: zjy0516 <riverclouds.zhu@qq.com>
Co-authored-by: Luka Govedič <ProExpertProg@users.noreply.github.com>

56aafa8c

25 Sep, 2025 2 commits
- [mypy] Further improve MM type annotations (#25654) · 12c1287d
  Cyrus Leung authored Sep 25, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  12c1287d
- Revert "[Performance] Move apply_w8a8_block_fp8_linear to an op class… (#25607) · 1260180c
  Tyler Michael Smith authored Sep 25, 2025
```
Signed-off-by: Tyler Michael Smith <tlrmchlsmth@gmail.com>
```
  1260180c
24 Sep, 2025 5 commits

Improve `--help` for enhanced user experience (#24903) · e7f27ea6
Harry Mellor authored Sep 25, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
e7f27ea6
[Refactor] Use DeepGEMM Col Major TMA Aligned Tensor (#25517) · 1f291412
Wentao Ye authored Sep 24, 2025
```
Signed-off-by: yewentao256 <zhyanwentao@126.com>
```
1f291412

Support mnnvl all2allv from Flashinfer (#21003) · 54e42b72

Shu Wang authored Sep 24, 2025


Signed-off-by: Shu Wang <shuw@nvidia.com>
Signed-off-by: Shu Wang. <shuw@nvidia.com>
Signed-off-by: Tyler Michael Smith <tyler@neuralmagic.com>
Signed-off-by: Tyler Michael Smith <tlrmchlsmth@gmail.com>
Co-authored-by: Tyler Michael Smith <tyler@neuralmagic.com>
Co-authored-by: Tyler Michael Smith <tlrmchlsmth@gmail.com>

54e42b72

[Misc] Improve type annotations for jsontree (#25577) · 9313be50
Cyrus Leung authored Sep 24, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
9313be50

[Spec Decode] Enable FlashInfer Spec Decoding (#25196) · c30b405b

Benjamin Chislett authored Sep 23, 2025


Signed-off-by: Benjamin Chislett <benjamin.chislett@centml.ai>
Signed-off-by: Benjamin Chislett <bchislett@nvidia.com>
Co-authored-by: lhsjohn <huashuoli@tencent.com>

c30b405b

23 Sep, 2025 4 commits

Remove redundant mutates_args and dispatch_key for direct_register_custom_op (#25512) · 7361ab37
Michael Goin authored Sep 23, 2025
```
Signed-off-by: mgoin <mgoin64@gmail.com>
```
7361ab37

[Performance] Move apply_w8a8_block_fp8_linear to an op class (#24666) · 63400259

ElizaWszola authored Sep 23, 2025


Signed-off-by: ElizaWszola <ewszola@redhat.com>
Signed-off-by: ElizaWszola <elizaw.9289@gmail.com>
Signed-off-by: Luka Govedič <lgovedic@redhat.com>
Signed-off-by: Luka Govedič <ProExpertProg@users.noreply.github.com>
Co-authored-by: Luka Govedič <ProExpertProg@users.noreply.github.com>
Co-authored-by: Michael Goin <mgoin64@gmail.com>
Co-authored-by: Luka Govedič <lgovedic@redhat.com>

63400259

[core] add nccl symmetric memory for all reduce (#24532) · 8c1c81a3

Amir Samani authored Sep 23, 2025


Signed-off-by: Amir Samani <asamani@nvidia.com>
Signed-off-by: Michael Goin <mgoin64@gmail.com>
Co-authored-by: Michael Goin <mgoin64@gmail.com>

8c1c81a3

[Perf] Apply torch.compile for `per_block_cast_to_fp8` (#24611) · 9949aa2e
Wentao Ye authored Sep 22, 2025
```
Signed-off-by: yewentao256 <zhyanwentao@126.com>
```
9949aa2e