Commits · 1a504aff6c79878238e535d78f322e6cae40b71a · OpenDAS / vllm_cscc

18 Mar, 2025 2 commits

[Bugfix] Fix broken CPU quantization due to triton import (#15038) · 1a504aff
Isotr0py authored Mar 18, 2025
```
Signed-off-by: Isotr0py <2037008807@qq.com>
```
1a504aff

[MODEL] Add support for Zamba2 models (#13185) · 01ca85bb

yury-tokpanov authored Mar 18, 2025


Signed-off-by: Yury Tokpanov <yury@zyphra.com>
Signed-off-by: Quentin Anthony <qganthony@yahoo.com>
Co-authored-by: Quentin Anthony <qganthony@yahoo.com>
Co-authored-by: Tyler Michael Smith <tysmith@redhat.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>

01ca85bb

16 Mar, 2025 1 commit
- [Kernel] Add more tuned configs (#14877) · 71c1e071
  Simon Mo authored Mar 15, 2025
```
Signed-off-by: simon-mo <simon.mo@hey.com>
```
  71c1e071
15 Mar, 2025 1 commit
- Revert "[Model] Mamba2 Prefill Performance Tweaks: Fixing Flurry of U… (#14848) · ccf02fcb
  Tyler Michael Smith authored Mar 14, 2025
  
  ccf02fcb
14 Mar, 2025 3 commits
- [Model] Mamba2 Prefill Performance Tweaks: Fixing Flurry of Unnecessary Memory Copies (#14778) · fe66b347
  Chih-Chieh Yang authored Mar 14, 2025
```
Signed-off-by: Chih-Chieh-Yang <7364402+cyang49@users.noreply.github.com>
```
  fe66b347
- [Attention] MLA get rid of materialization (#14770) · 9532c498
  Lucas Wilkinson authored Mar 14, 2025
```
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
```
  9532c498
- [Bugfix][IPEX] Add `VLLM_CPU_MOE_PREPACK` to allow disabling MoE prepack when... · 95d680b8
  Thien Tran authored Mar 14, 2025
```
[Bugfix][IPEX] Add `VLLM_CPU_MOE_PREPACK` to allow disabling MoE prepack when CPU does not support it (#14681)
Signed-off-by: Thien Tran <gau.nernst@yahoo.com.sg>
```
  95d680b8
13 Mar, 2025 3 commits
- [Attention] Remove slow setattr in MLA (#14769) · d47807ba
  Lucas Wilkinson authored Mar 13, 2025
```
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
```
  d47807ba
- [Bugfix] Fix chunked prefill for GGUF (#14666) · 55211b01
  Szymon Ożóg authored Mar 13, 2025
```
Signed-off-by: SzymonOzog <szymon.ozog@aleph-alpha.com>
```
  55211b01
- [ROCm][FP8] Fix for adjustments needed only for fnuz (#14689) · a94a699c
  Gregory Shtrasberg authored Mar 12, 2025
```
Signed-off-by: Gregory Shtrasberg <Gregory.Shtrasberg@amd.com>
```
  a94a699c
12 Mar, 2025 4 commits
- [CPU] Upgrade CPU backend to torch-2.6 (#13381) · ff47aab0
  Li, Jiang authored Mar 12, 2025
```
Signed-off-by: jiang1.li <jiang1.li@intel.com>
Co-authored-by: Isotr0py <2037008807@qq.com>
```
  ff47aab0
- [Kernel] Add ModelOpt FP4 Checkpoint Support (#12520) · debd6bbf
  Pavani Majety authored Mar 11, 2025
```
Signed-off-by: Pavani Majety <pmajety@nvidia.com>
```
  debd6bbf
- [Kernel] GGUF MoE kernel (#14613) · e22ee1e7
  Szymon Ożóg authored Mar 12, 2025
```
Signed-off-by: SzymonOzog <szymon.ozog@aleph-alpha.com>
```
  e22ee1e7
- [Core] Refactor `QKVCrossParallelLinear` implementation to support BNB 4-bit quantization (#14545) · e392d858
  Isotr0py authored Mar 12, 2025
```
Signed-off-by: Isotr0py <2037008807@qq.com>
```
  e392d858
11 Mar, 2025 2 commits
- dynamic distpatch of fp8 kernels (#14245) · a1c8f379
  Jeff Daily authored Mar 11, 2025
```
Signed-off-by: Jeff Daily <jeff.daily@amd.com>
```
  a1c8f379
- [Kernel] moe wna16 cuda kernel (#13321) · 90e88ab7
  Jinzhen Lin authored Mar 11, 2025
```
Signed-off-by: Jinzhen Lin <linjinzhen@hotmail.com>
Co-authored-by: mgoin <mgoin64@gmail.com>
```
  90e88ab7
10 Mar, 2025 2 commits
- [V1][Bugfix] Fix handing of `second_per_grid_ts` for Qwen2-VL & Qwen2.5-VL (#14548) · dea985ae
  Roger Wang authored Mar 10, 2025
```
Signed-off-by: Roger Wang <ywang@roblox.com>
```
  dea985ae
- [Kernel] Add more dtype support for GGUF kernels (#14043) · 89cdaa83
  Szymon Ożóg authored Mar 10, 2025
```
Signed-off-by: SzymonOzog <szymon.ozog@aleph-alpha.com>
Signed-off-by: SzymonOzog <szymon.ozog@gmail.com>
```
  89cdaa83
08 Mar, 2025 1 commit
- Revert "[Perf] Reduce MLA CPU overheads in V1 (#14384)" (#14471) · ca7a2d5f
  Tyler Michael Smith authored Mar 08, 2025
  
  ca7a2d5f
07 Mar, 2025 3 commits
- [Kernel] optimize performance of gptq marlin kernel when n is small (#14138) · d0feea31
  Jinzhen Lin authored Mar 08, 2025
```
Signed-off-by: Jinzhen Lin <linjinzhen@hotmail.com>
```
  d0feea31
- [FP8] Refactor apply_fp8_linear and apply_fp8_linear_generic into an object (#14390) · e1744502
  Luka Govedič authored Mar 07, 2025
```
Signed-off-by: luka <luka@neuralmagic.com>
```
  e1744502
- [Perf] Reduce MLA CPU overheads in V1 (#14384) · dae68969
  Lucas Wilkinson authored Mar 06, 2025
```
Signed-off-by: Lucas Wilkinson <lwilkinson@neuralmagic.com>
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
```
  dae68969
06 Mar, 2025 3 commits
- [Bugfix] Fix use_direct_call condition in FusedMoE layer for (#14382) · 958adce4
  Tyler Michael Smith authored Mar 06, 2025
```
Signed-off-by: Tyler Michael Smith <tyler@neuralmagic.com>
```
  958adce4
- [Distributed] Add enable_expert_parallel arg (#14305) · cc2f9b32
  Tyler Michael Smith authored Mar 06, 2025
```
Signed-off-by: Tyler Michael Smith <tyler@neuralmagic.com>
```
  cc2f9b32
- [Core] Optimizing cross-attention `QKVParallelLinear` computation (#12325) · 69ff99fd
  Nicolò Lucchesi authored Mar 06, 2025
```
Signed-off-by: NickLucche <nlucches@redhat.com>
Signed-off-by: NickLucche <nick@nlucches-4xa100.c.openshift-330514.internal>
Co-authored-by: NickLucche <nick@nlucches-4xa100.c.openshift-330514.internal>
```
  69ff99fd
05 Mar, 2025 3 commits
- [LoRA] Remove linear hack outside transformers backend (#14177) · e17e4488
  Isotr0py authored Mar 05, 2025
```
Signed-off-by: Isotr0py <2037008807@qq.com>
```
  e17e4488
- [V1] EP/TP MoE + DP Attention (#13931) · 72c62eae
  Tyler Michael Smith authored Mar 05, 2025
  
  72c62eae
- [Bugfix] Fix gptq_marlin for deepseek-v3 (#13750) · 0df25101
  rainkert authored Mar 05, 2025
```
Signed-off-by: dangshunya <dangshunya@baichuan-inc.com>
Co-authored-by: dangshunya <dangshunya@baichuan-inc.com>
```
  0df25101
04 Mar, 2025 1 commit
- [core] moe fp8 block quant tuning support (#14068) · bb5b6403
  Divakar Verma authored Mar 03, 2025
```
Signed-off-by: Divakar Verma <divakar.verma@amd.com>
```
  bb5b6403
03 Mar, 2025 2 commits
- [Kernel] Optimize moe intermediate_cache usage (#13625) · 19d98e0c
  Michael Goin authored Mar 03, 2025
```
Signed-off-by: mgoin <mgoin64@gmail.com>
```
  19d98e0c
- [Misc][Platform] Move use allgather to platform (#14010) · b87c21fc
  Mengqing Cao authored Mar 03, 2025
```
Signed-off-by: Mengqing Cao <cmq0113@163.com>
```
  b87c21fc
01 Mar, 2025 1 commit
- [Misc][Kernel]: Add GPTQAllSpark Quantization (#12931) · 6a92ff93
  YajieWang authored Mar 01, 2025
  
  6a92ff93
28 Feb, 2025 2 commits
- [torch.compile] Fix RMSNorm + quant fusion in the non-cutlass-fp8 case, rename... · bd56c983
  Luka Govedič authored Feb 28, 2025
```
[torch.compile] Fix RMSNorm + quant fusion in the non-cutlass-fp8 case, rename RedundantReshapesPass to NoopEliminationPass (#10902)
Signed-off-by: luka <luka@neuralmagic.com>
```
  bd56c983
- [Bugfix] Fix MoeWNA16Method activation (#14024) · e0734387
  Jee Jee Li authored Feb 28, 2025
  
  e0734387
27 Feb, 2025 3 commits
- [Misc] Print FusedMoE detail info (#13974) · 67fc4268
  Jee Jee Li authored Feb 28, 2025
  
  67fc4268
- [core] Perf improvement for DSv3 on AMD GPUs (#13718) · 8294773e
  qli88 authored Feb 27, 2025
```
Signed-off-by: qli88 <qiang.li2@amd.com>
```
  8294773e
- [Model] Deepseek GGUF support (#13167) · 7f0be2aa
  Szymon Ożóg authored Feb 27, 2025
  
  7f0be2aa
26 Feb, 2025 2 commits
- [BugFix] Make FP8 Linear compatible with torch.compile (#13918) · b382a7f2
  Woosuk Kwon authored Feb 26, 2025
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  b382a7f2
- [Model] Support Grok1 (#13795) · 07c43530
  Michael Goin authored Feb 25, 2025
```
Signed-off-by: mgoin <mgoin64@gmail.com>
```
  07c43530
25 Feb, 2025 1 commit

[Neuron] Add custom_ops for neuron backend (#13246) · f75aa727

Liangfu Chen authored Feb 25, 2025


Signed-off-by: Liangfu Chen <liangfc@amazon.com>
Co-authored-by: George Novack <gnovack@amazon.com>
Co-authored-by: Aoyu Zhang <aoyuzhan@amazon.com>

f75aa727