Commits · d52358c1e07768266e3db92e847cd28af87ca4b9 · OpenDAS / vllm_cscc

26 Aug, 2025 4 commits

[Perf] Remove duplicated NVFP4 blockscales to save memory (#23379) · d52358c1
Michael Goin authored Aug 26, 2025
```
Signed-off-by: mgoin <mgoin64@gmail.com>
```
d52358c1

[Bugfix] Fix Qwen25VL packed_modules_mapping (#23604) · 9b5f6423

Jee Jee Li authored Aug 26, 2025


Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
Co-authored-by: Michael Goin <mgoin64@gmail.com>

9b5f6423

[mypy] Fix incorrect type hint for EAGLE3 support (#23617) · 7d67a9d9
Cyrus Leung authored Aug 26, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
7d67a9d9

Update Flashinfer to 0.2.14.post1 (#23537) · ae067888

weiliang authored Aug 26, 2025


Signed-off-by: Siyuan Fu <siyuanf@nvidia.com>
Signed-off-by: siyuanf <siyuanf@nvidia.com>
Signed-off-by: Weiliang Liu <weiliangl@nvidia.com>
Signed-off-by: Michael Goin <mgoin64@gmail.com>
Co-authored-by: Siyuan Fu <siyuanf@nvidia.com>
Co-authored-by: Michael Goin <mgoin64@gmail.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>

ae067888

25 Aug, 2025 13 commits
- [Kernel] Add fused grouped_topk kernel for MoE (#23274) · 8a3cd90a
  Xin Yang authored Aug 25, 2025
```
Signed-off-by: Xin Yang <xyangx@amazon.com>
Co-authored-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com>
```
  8a3cd90a
- [Bugfix] Fix Qwen3 MoE GPTQ inference (#23490) · a9082a4d
  Isotr0py authored Aug 25, 2025
```
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
```
  a9082a4d
- [Refactor] Pass `tokenizer` explicitly instead of binding to prompt update (#23542) · 6879cd80
  Cyrus Leung authored Aug 25, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  6879cd80
- [Attention] Unify mamba and attention backend selection (#23171) · 5c4b6e66
  Ayush Satyam authored Aug 25, 2025
```
Signed-off-by: Ayush Satyam <ayushsatyam146@gmail.com>
```
  5c4b6e66
- [Bugfix] Allow dynamic number of patches for llava_onevision (#23525) · ebafb093
  Cyrus Leung authored Aug 25, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  ebafb093
- [Refactor] Dynamic `target` and `content` for prompt updates (#23411) · 712d0f88
  Cyrus Leung authored Aug 25, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  712d0f88
- [Bugfix] Fix Dense module loading for sentence-transformers embedding models... · c9abb104
  LIYIFAN_liyifan authored Aug 24, 2025
```
[Bugfix] Fix Dense module loading for sentence-transformers embedding models (simplified V2) (#23408)
Signed-off-by: FFFfff1FFFfff <yifanli0919@gmail.com>
```
  c9abb104
- Migrate DonutImagePixelInputs to TensorSchema (#23509) · 787cdb38
  Benji Beck authored Aug 24, 2025
```
Signed-off-by: Benji Beck <benjibeck@meta.com>
```
  787cdb38
- Migrate skyworkr1v inputs to TensorSchema (#23499) · a5203d04
  Benji Beck authored Aug 24, 2025
```
Signed-off-by: Benji Beck <benjibeck@meta.com>
```
  a5203d04
- Migrate tarsier inputs to TensorSchema (#23500) · 99f80944
  Benji Beck authored Aug 24, 2025
```
Signed-off-by: Benji Beck <benjibeck@meta.com>
```
  99f80944
- [Misc] Unified linear print info (#23516) · 170e8ea9
  Jee Jee Li authored Aug 25, 2025
```
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
```
  170e8ea9
- [Bugfix] Fix Qwen2.5-VL quantized model weights loading (#23512) · a71e4765
  zifeitong authored Aug 24, 2025
```
Signed-off-by: Zifei Tong <zifeitong@gmail.com>
```
  a71e4765
- [Perf] Add Triton config for DeepSeek V3 FP8 EP32 H200 (#23504) · 504d9143
  Ming Yang authored Aug 24, 2025
```
Signed-off-by: Ming Yang <minos.future@gmail.com>
```
  504d9143
24 Aug, 2025 5 commits
- [Model] Enable BLOOM on V1 (#23488) · e2db1164
  Cyrus Leung authored Aug 24, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  e2db1164
- [New Model]Donut model (#23229) · 416f0592
  汪志鹏 authored Aug 24, 2025
```
Signed-off-by: 汪志鹏 <wangzhipeng628@gmail.com>
```
  416f0592
- [kernel] Support W4A8 on Hopper (#23198) · e76e2335
  czhu-cohere authored Aug 24, 2025
```
Signed-off-by: czhu-cohere <conway.zhu@cohere.com>
```
  e76e2335
- Migrate Paligemma inputs to TensorSchema (#23470) · a7527728
  Benji Beck authored Aug 23, 2025
```
Signed-off-by: Benji Beck <benjibeck@meta.com>
```
  a7527728
- Migrate Pixtral inputs to TensorSchema (#23472) · 053278a5
  Benji Beck authored Aug 23, 2025
```
Signed-off-by: Benji Beck <benjibeck@meta.com>
```
  053278a5
23 Aug, 2025 3 commits
- Add glm4.5v tp2,4 fp8 config on H100_80GB (#23443) · 308fa287
  Chenxi Yang authored Aug 22, 2025
```
Co-authored-by: Chenxi Yang <cxyang@meta.com>
```
  308fa287
- Quantization: support FP4 quantized models on AMD CDNA2/CDNA3 GPUs (#22527) · fa78de9d
  Daifeng Li authored Aug 23, 2025
```
Signed-off-by: feng <fengli1702@gmail.com>
Signed-off-by: Michael Goin <mgoin64@gmail.com>
Co-authored-by: Michael Goin <mgoin64@gmail.com>
```
  fa78de9d
- [Model] Support DP for ViT on MiniCPM-V-4 (#23327) · 23c939fd
  WeiQing Chen authored Aug 23, 2025
```
Signed-off-by: ycyaw66 <497410282@qq.com>
Co-authored-by: ycyaw66 <497410282@qq.com>
```
  23c939fd
22 Aug, 2025 13 commits
- [NVIDIA][torch.compile] Support Flashinfer TRTLLM FP8-q/kv NVFP4-out Attention Kernel (#22703) · 24d0c9e6
  elvischenv authored Aug 23, 2025
```
Signed-off-by: elvischenv <219235043+elvischenv@users.noreply.github.com>
Co-authored-by: Luka Govedič <ProExpertProg@users.noreply.github.com>
```
  24d0c9e6
- [BugFix][AMD][Quantization] Fix torch.compile issue where wvSplitKQ not being... · cc7ae5e7
  rasmith authored Aug 22, 2025
```
[BugFix][AMD][Quantization] Fix torch.compile issue where wvSplitKQ not being called when it should when using quantized FP8 model (#22281)
Signed-off-by: Randall Smith <Randall.Smith@amd.com>
```
  cc7ae5e7
- [Quantization] Allow GGUF quantization to skip unquantized layer (#23188) · 4645024d
  Isotr0py authored Aug 23, 2025
```
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
```
  4645024d
- [Bugfix] Fix broken Florence-2 model (#23426) · cd7a3df2
  Isotr0py authored Aug 23, 2025
```
Signed-off-by: 汪志鹏 <wangzhipeng628@gmail.com>
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
Co-authored-by: 汪志鹏 <wangzhipeng628@gmail.com>
```
  cd7a3df2
- [Model] Add Ovis2.5 PP support (#23405) · 32d2b406
  Isotr0py authored Aug 23, 2025
```
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
```
  32d2b406
- [BugFix] Fix the issue where image embeddings were incorrectly split.… (#23366) · 424fb7a5
  bppps authored Aug 23, 2025
```
Signed-off-by: bppps <bpppsaka@gmail.com>
Co-authored-by: zouyu.zzx <zouyu.zzx@alibaba-inc.com>
Co-authored-by: bppps <bpppsaka@gmail.com>
```
  424fb7a5
- [Speculators][Speculative Decoding] Fix Qwen 2 Eagle3 Support (#23337) · 88491c1b
  PapaGoose authored Aug 22, 2025
  
  88491c1b
- [misc] Remove outdate comment about runai_model_streamer (#23421) · 695e7adc
  杨朱 · Kiki authored Aug 22, 2025
```
Signed-off-by: carlory <baofa.fan@daocloud.io>
```
  695e7adc
- Migrate MiniCPMOAudioInputs to TensorSchema (#21847) · 99872085
  Benji Beck authored Aug 22, 2025
```
Signed-off-by: Benji Beck <benjibeck@meta.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
```
  99872085
- [Attention] Refactor AttentionMetadata Preparation for Encoder-only Models (#23154) · 17373dcd
  Chen Zhang authored Aug 21, 2025
```
Signed-off-by: Chen Zhang <zhangch99@outlook.com>
```
  17373dcd
- [New Model] Add Seed-Oss model (#23241) · 59640693
  Bin Jia authored Aug 22, 2025
```
Signed-off-by: jiabin.00 <jiabin.00@bytedance.com>
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
Co-authored-by: Jee Jee Li <pandaleefree@gmail.com>
```
  59640693
- [Feature] Enable DeepGEMM Linear on B200; 1.5% E2E throughput improvement (#23351) · 394591e3
  Wentao Ye authored Aug 22, 2025
```
Signed-off-by: yewentao256 <zhyanwentao@126.com>
```
  394591e3
- Migrate MllamaImagePixelInputs to TensorSchema (#22020) · 0b9cc56f
  Benji Beck authored Aug 21, 2025
```
Signed-off-by: Benji Beck <benjibeck@meta.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
```
  0b9cc56f
21 Aug, 2025 2 commits

[Perf] Small optimizations for silu_mul_fp8_quant_deep_gemm (#23265) · 3bbe11cc
Michael Goin authored Aug 21, 2025
```
Signed-off-by: mgoin <mgoin64@gmail.com>
```
3bbe11cc

[Bug] Fix R1 Accuracy 0 Bug (#23294) · 48bfb0c9

Wentao Ye authored Aug 21, 2025


Signed-off-by: yewentao256 <zhyanwentao@126.com>
Signed-off-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com>
Co-authored-by: Michael Goin <mgoin64@gmail.com>

48bfb0c9