Commits · 43ed4143c4ec00f4b587c5bcefdb3b6520fbe966 · OpenDAS / vllm_cscc

"vllm/vscode:/vscode.git/clone" did not exist on "9d1e611f0e98934092684cd4c7ef0e960f33813c"

27 Mar, 2025 5 commits

[Quantization] Fp8 Channelwise Dynamic Per Token GroupedGEMM (#15587) · 43ed4143

Robert Shaw authored Mar 27, 2025


Signed-off-by: ElizaWszola <eliza@neuralmagic.com>
Signed-off-by: ElizaWszola <ewszola@redhat.com>
Signed-off-by: rshaw@neuralmagic.com <robertgshaw2@gmail.com>
Co-authored-by: ElizaWszola <eliza@neuralmagic.com>
Co-authored-by: Lucas Wilkinson <wilkinson.lucas@gmail.com>
Co-authored-by: ElizaWszola <ewszola@redhat.com>

43ed4143

[TPU] Avoid Triton Import (#15589) · e1e0fd75
Robert Shaw authored Mar 27, 2025
```
Signed-off-by: rshaw@neuralmagic.com <robertgshaw2@gmail.com>
```
e1e0fd75
Allow torchao quantization in SiglipMLP (#15575) · dcf2a590
Jerry Zhang authored Mar 26, 2025

dcf2a590
[moe][quant] add weight name case for offset (#15515) · fb22be58
Mengqing Cao authored Mar 27, 2025
```
Signed-off-by: Mengqing Cao <cmq0113@163.com>
```
fb22be58

[Kernel] CUTLASS grouped gemm fp8 MoE kernel (#13972) · 9239bf71

ElizaWszola authored Mar 27, 2025


Signed-off-by: ElizaWszola <eliza@neuralmagic.com>
Signed-off-by: ElizaWszola <ewszola@redhat.com>
Co-authored-by: Lucas Wilkinson <wilkinson.lucas@gmail.com>

9239bf71

26 Mar, 2025 10 commits
- Use Cache Hinting for fused_moe kernel (#15511) · 7a888271
  Wes authored Mar 26, 2025
  
  7a888271
- Fix weight loading for some models in Transformers backend (#15544) · 0af4d764
  Harry Mellor authored Mar 26, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  0af4d764
- [Model] Add Reasoning Parser for Granite Models (#14202) · 1711b929
  Alex Brooks authored Mar 26, 2025
```
Signed-off-by: Alex-Brooks <Alex.brooks@ibm.com>
Co-authored-by: Joe Runde <joe@joerun.de>
```
  1711b929
- Improve validation of TP in Transformers backend (#15540) · c091c0a5
  Harry Mellor authored Mar 26, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  c091c0a5
- Apply torchfix (#15532) · 1aa162e0
  cyyever authored Mar 26, 2025
```
Signed-off-by: cyy <cyyever@outlook.com>
```
  1aa162e0
- Separate base model from `TransformersModel` (#15467) · cf5c8f16
  Harry Mellor authored Mar 26, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Signed-off-by: Isotr0py <2037008807@qq.com>
Co-authored-by: Isotr0py <2037008807@qq.com>
Co-authored-by: Isotr0py <mozf@mail2.sysu.edu.cn>
```
  cf5c8f16
- [FEAT][ROCm] Integrate Fused MoE Kernels from AITER (#14967) · 5ebf6674
  vllmellm authored Mar 26, 2025
```
Signed-off-by: vllmellm <vllm.ellm@embeddedllm.com>
Signed-off-by: tjtanaa <tunjian.tan@embeddedllm.com>
Co-authored-by: tjtanaa <tunjian.tan@embeddedllm.com>
```
  5ebf6674
- [Feature] Enhance EAGLE Architecture with Proper RMS Norms (#14990) · 781d0562
  Bryan Lu authored Mar 26, 2025
```
Signed-off-by: Bryan Lu <yuzhelu@amazon.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
```
  781d0562
- [Model] Support multi-image for Molmo (#15438) · 997c8811
  Cyrus Leung authored Mar 26, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  997c8811
- Transformers backend already supports V1 (#15463) · e42389f9
  Harry Mellor authored Mar 26, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  e42389f9
25 Mar, 2025 4 commits
- [Misc] Remove redundant `num_embeds` (#15443) · 5994430b
  Cyrus Leung authored Mar 25, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  5994430b
- [Misc] Clean up MiniCPM-V/O code (#15337) · a9e879b3
  Cyrus Leung authored Mar 25, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  a9e879b3
- Revert "Fix non-contiguous input passed to Marlin kernel (#15319)" (#15398) · b5269db9
  Tyler Michael Smith authored Mar 24, 2025
  
  b5269db9
- Add pipeline parallel support to `TransformersModel` (#12832) · 97cfa65d
  Harry Mellor authored Mar 25, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Signed-off-by: Isotr0py <2037008807@qq.com>
Co-authored-by: Isotr0py <2037008807@qq.com>
```
  97cfa65d
24 Mar, 2025 7 commits
- [ROCm][Kernel] MoE weights padding (#14454) · f533b583
  Gregory Shtrasberg authored Mar 24, 2025
```
Signed-off-by: Gregory Shtrasberg <Gregory.Shtrasberg@amd.com>
Signed-off-by: charlifu <charlifu@amd.com>
Co-authored-by: charlifu <charlifu@amd.com>
```
  f533b583
- [Build] Cython compilation support fix (#14296) · 8279201c
  Gregory Shtrasberg authored Mar 24, 2025
```
Signed-off-by: Gregory Shtrasberg <Gregory.Shtrasberg@amd.com>
```
  8279201c
- [Hardware][Gaudi][Feature] Enable Dynamic MoE for Mixtral (#12303) · 5eeadc26
  liuzhenwei authored Mar 25, 2025
```
Signed-off-by: zhenwei <zhenweiliu@habana.ai>
```
  5eeadc26
- [Core] Integrate `fastsafetensors` loader for loading model weights (#10647) · 761702fd
  Manish Sethi authored Mar 24, 2025
```
Signed-off-by: Manish Sethi <Manish.sethi1@ibm.com>
```
  761702fd
- [Bugfix][V1] Avoid importing PreTrainedModel (#15366) · 948ab03e
  ℍ𝕠𝕝𝕝𝕠𝕨 𝕄𝕒𝕟 authored Mar 24, 2025
```
Signed-off-by: Hollow Man <hollowman@opensuse.org>
```
  948ab03e
- [Misc] Upgrade BNB version (#15183) · 3892e58a
  Jee Jee Li authored Mar 24, 2025
  
  3892e58a
- Fix non-contiguous input passed to Marlin kernel (#15319) · d20e2611
  Qubitium-ModelCloud authored Mar 24, 2025
  
  d20e2611
23 Mar, 2025 1 commit
- [Misc] Add tuned R1 w8a8 and MoE configs for NVIDIA L20 (#15322) · f90d34b4
  DefTruth authored Mar 23, 2025
```
Signed-off-by: DefTruth <qiustudent_r@163.com>
```
  f90d34b4
22 Mar, 2025 3 commits

Remove openvino support in favor of external plugin (#15339) · b877031d
Russell Bryant authored Mar 22, 2025
```
Signed-off-by: Russell Bryant <rbryant@redhat.com>
```
b877031d

[Model] Support Tele-FLM Model (#15023) · 2f4bd358

Naitong Yu authored Mar 22, 2025


Signed-off-by: Naitong Yu <ntyu@baai.ac.cn>
Signed-off-by: jiangxin <horizon94@outlook.com>
Co-authored-by: Jason Fang <jasonfang3900@gmail.com>
Co-authored-by: jiangxin <horizon94@outlook.com>

2f4bd358

[FEAT] [ROCm]: Add AITER RMS Norm (Layer Norm) Feature (#14959) · ec870fba
TJian authored Mar 22, 2025
```
Signed-off-by: tjtanaa <tunjian.tan@embeddedllm.com>
```
ec870fba

21 Mar, 2025 6 commits
- [Bugfix][VLM] fix llava processor (#15285) · c21b99b9
  Mengqing Cao authored Mar 21, 2025
```
Signed-off-by: Mengqing Cao <cmq0113@163.com>
```
  c21b99b9
- [Bugfix] Fix incorrect resolving order for transformers fallback (#15279) · 84e00adc
  Isotr0py authored Mar 21, 2025
```
Signed-off-by: Isotr0py <2037008807@qq.com>
```
  84e00adc
- [Misc] Add attention mask pre-computation optimization back to Qwen2.5-VL (#15273) · 47c71262
  Isotr0py authored Mar 21, 2025
```
Signed-off-by: Isotr0py <2037008807@qq.com>
```
  47c71262
- [Model] RE: Mamba2 Prefill Performance Tweaks: Fixing Flurry of Unnecessary Memory Copies (#14857) · 296f927f
  Chih-Chieh Yang authored Mar 20, 2025
```
Signed-off-by: Chih-Chieh-Yang <7364402+cyang49@users.noreply.github.com>
```
  296f927f
- [Bugfix] Fix incorrect qwen2.5-vl attention mask pre-computation (#15200) · 1e508343
  Isotr0py authored Mar 21, 2025
```
Signed-off-by: Isotr0py <2037008807@qq.com>
```
  1e508343
- Enforce that TP > 1 is not supported for Mamba2 if Quantization is Enabled. (#14617) · 06dd0825
  Yu Chin Fabian Lim authored Mar 21, 2025
```
Signed-off-by: Yu Chin Fabian Lim <flim@sg.ibm.com>
```
  06dd0825
20 Mar, 2025 4 commits
- [Bugfix] Multi-video inference on LLaVA-Onevision (#15082) · 27261e40
  Cyrus Leung authored Mar 20, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Co-authored-by: Isotr0py <2037008807@qq.com>
```
  27261e40
- [Frontend][Bugfix] support prefill decode disaggregation on deepseek (#14824) · 742369d3
  billishyahao authored Mar 20, 2025
```
Signed-off-by: billishyahao <bill.he@amd.com>
Co-authored-by: Zhai Feiyue <80079571+ZhaiFeiyue@users.noreply.github.com>
```
  742369d3
- Enable CUDA graph support for llama 3.2 vision (#14917) · a8652f4f
  Matt Ritter authored Mar 19, 2025
```
Signed-off-by: Matt Ritter <100659061+mritterfigma@users.noreply.github.com>
```
  a8652f4f
- [Attention] Flash Attention 3 - fp8 (#14570) · a597a575
  Mickaël Seznec authored Mar 20, 2025
```
Signed-off-by: Mickael Seznec <mickael@mistral.ai>
```
  a597a575