Commits · dae68969774e41b93b01cd31171ca033a92b574a · OpenDAS / vllm_cscc

07 Mar, 2025 1 commit

[Perf] Reduce MLA CPU overheads in V1 (#14384) · dae68969

Lucas Wilkinson authored Mar 06, 2025


Signed-off-by: Lucas Wilkinson <lwilkinson@neuralmagic.com>
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>

dae68969

06 Mar, 2025 3 commits
- [Bugfix] Fix use_direct_call condition in FusedMoE layer for (#14382) · 958adce4
  Tyler Michael Smith authored Mar 06, 2025
```
Signed-off-by: Tyler Michael Smith <tyler@neuralmagic.com>
```
  958adce4
- [Distributed] Add enable_expert_parallel arg (#14305) · cc2f9b32
  Tyler Michael Smith authored Mar 06, 2025
```
Signed-off-by: Tyler Michael Smith <tyler@neuralmagic.com>
```
  cc2f9b32
- [Core] Optimizing cross-attention `QKVParallelLinear` computation (#12325) · 69ff99fd
  Nicolò Lucchesi authored Mar 06, 2025
```
Signed-off-by: NickLucche <nlucches@redhat.com>
Signed-off-by: NickLucche <nick@nlucches-4xa100.c.openshift-330514.internal>
Co-authored-by: NickLucche <nick@nlucches-4xa100.c.openshift-330514.internal>
```
  69ff99fd
05 Mar, 2025 3 commits
- [LoRA] Remove linear hack outside transformers backend (#14177) · e17e4488
  Isotr0py authored Mar 05, 2025
```
Signed-off-by: Isotr0py <2037008807@qq.com>
```
  e17e4488
- [V1] EP/TP MoE + DP Attention (#13931) · 72c62eae
  Tyler Michael Smith authored Mar 05, 2025
  
  72c62eae
- [Bugfix] Fix gptq_marlin for deepseek-v3 (#13750) · 0df25101
  rainkert authored Mar 05, 2025
```
Signed-off-by: dangshunya <dangshunya@baichuan-inc.com>
Co-authored-by: dangshunya <dangshunya@baichuan-inc.com>
```
  0df25101
04 Mar, 2025 1 commit
- [core] moe fp8 block quant tuning support (#14068) · bb5b6403
  Divakar Verma authored Mar 03, 2025
```
Signed-off-by: Divakar Verma <divakar.verma@amd.com>
```
  bb5b6403
03 Mar, 2025 2 commits
- [Kernel] Optimize moe intermediate_cache usage (#13625) · 19d98e0c
  Michael Goin authored Mar 03, 2025
```
Signed-off-by: mgoin <mgoin64@gmail.com>
```
  19d98e0c
- [Misc][Platform] Move use allgather to platform (#14010) · b87c21fc
  Mengqing Cao authored Mar 03, 2025
```
Signed-off-by: Mengqing Cao <cmq0113@163.com>
```
  b87c21fc
01 Mar, 2025 1 commit
- [Misc][Kernel]: Add GPTQAllSpark Quantization (#12931) · 6a92ff93
  YajieWang authored Mar 01, 2025
  
  6a92ff93
28 Feb, 2025 2 commits
- [torch.compile] Fix RMSNorm + quant fusion in the non-cutlass-fp8 case, rename... · bd56c983
  Luka Govedič authored Feb 28, 2025
```
[torch.compile] Fix RMSNorm + quant fusion in the non-cutlass-fp8 case, rename RedundantReshapesPass to NoopEliminationPass (#10902)
Signed-off-by: luka <luka@neuralmagic.com>
```
  bd56c983
- [Bugfix] Fix MoeWNA16Method activation (#14024) · e0734387
  Jee Jee Li authored Feb 28, 2025
  
  e0734387
27 Feb, 2025 3 commits
- [Misc] Print FusedMoE detail info (#13974) · 67fc4268
  Jee Jee Li authored Feb 28, 2025
  
  67fc4268
- [core] Perf improvement for DSv3 on AMD GPUs (#13718) · 8294773e
  qli88 authored Feb 27, 2025
```
Signed-off-by: qli88 <qiang.li2@amd.com>
```
  8294773e
- [Model] Deepseek GGUF support (#13167) · 7f0be2aa
  Szymon Ożóg authored Feb 27, 2025
  
  7f0be2aa
26 Feb, 2025 2 commits
- [BugFix] Make FP8 Linear compatible with torch.compile (#13918) · b382a7f2
  Woosuk Kwon authored Feb 26, 2025
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  b382a7f2
- [Model] Support Grok1 (#13795) · 07c43530
  Michael Goin authored Feb 25, 2025
```
Signed-off-by: mgoin <mgoin64@gmail.com>
```
  07c43530
25 Feb, 2025 5 commits
- [Neuron] Add custom_ops for neuron backend (#13246) · f75aa727
  Liangfu Chen authored Feb 25, 2025
```
Signed-off-by: Liangfu Chen <liangfc@amazon.com>
Co-authored-by: George Novack <gnovack@amazon.com>
Co-authored-by: Aoyu Zhang <aoyuzhan@amazon.com>
```
  f75aa727
- Fix CompressedTensorsWNA16MoE with grouped scales (#13769) · 4d251ad0
  Michael Goin authored Feb 25, 2025
  
  4d251ad0
- [Bugfix][Quantization] Fix FP8 + EP (#13784) · 1e15aaef
  Tyler Michael Smith authored Feb 24, 2025
```
Signed-off-by: Tyler Michael Smith <tyler@neuralmagic.com>
```
  1e15aaef
- [Misc] set single whitespace between log sentences (#13771) · 51010a18
  cjackal authored Feb 25, 2025
```
Signed-off-by: cjackal <44624812+cjackal@users.noreply.github.com>
```
  51010a18
- Remove unused kwargs from model definitions (#13555) · cdc1fa12
  Harry Mellor authored Feb 25, 2025
  
  cdc1fa12
24 Feb, 2025 4 commits
- Fix precommit fail in fused_moe intermediate_cache2 chunking (#13772) · db986c19
  Michael Goin authored Feb 24, 2025
```
Signed-off-by: mgoin <mgoin64@gmail.com>
```
  db986c19
- [BugFix] Illegal memory access for MoE On H20 (#13693) · ccc00515
  Zhonghua Deng authored Feb 24, 2025
  
  ccc00515
- Expert Parallelism (EP) Support for DeepSeek V2 (#12583) · 781096e3
  Jongseok Park authored Feb 24, 2025
  
  781096e3
- [model][refactor] remove cuda hard code in models and layers (#13658) · 23eca9cf
  Mengqing Cao authored Feb 24, 2025
  
  23eca9cf
22 Feb, 2025 3 commits
- [ROCm] Apply FP8 weights padding to values not divisible by 512 bytes on ROCm (#13231) · c904fddd
  Gregory Shtrasberg authored Feb 22, 2025
  
  c904fddd
- [CI/Build] Fix pre-commit errors (#13696) · 7f6bae56
  Cyrus Leung authored Feb 22, 2025
  
  7f6bae56
- Correction to TP logic for Mamba Mixer 2 when Num Groups not divisible by TP Size (#13660) · fca20841
  Yu Chin Fabian Lim authored Feb 22, 2025
  
  fca20841
21 Feb, 2025 1 commit

[Attention] MLA with chunked prefill (#12639) · 288cc6c2

Lucas Wilkinson authored Feb 21, 2025


Signed-off-by: Lucas Wilkinson <lwilkinson@neuralmagic.com>
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Co-authored-by: Patrick Horn <patrick.horn@gmail.com>
Co-authored-by: simon-mo <xmo@berkeley.edu>
Co-authored-by: Tyler Michael Smith <tyler@neuralmagic.com>

288cc6c2

20 Feb, 2025 2 commits
- [Bugfix] Fix deepseekv3 grouped topk error (#13474) · ed6e9075
  chenxiaobing authored Feb 20, 2025
```
Signed-off-by: Chen-XiaoBing <chenxb002@whu.edu.cn>
```
  ed6e9075
- [ROCm][MoE] mi300 mixtral8x7B perf for specific BS (#13577) · 0d243f2a
  Divakar Verma authored Feb 19, 2025
```
Signed-off-by: Divakar Verma <divakar.verma@amd.com>
```
  0d243f2a
19 Feb, 2025 1 commit
- [ROCm][MoE configs] mi325 mixtral & mi300 qwen_moe (#13503) · 8aada19d
  Divakar Verma authored Feb 19, 2025
  
  8aada19d
18 Feb, 2025 3 commits
- [V1] Optimize handling of sampling metadata and req_ids list (#13244) · 30172b49
  Nick Hill authored Feb 18, 2025
```
Signed-off-by: Nick Hill <nhill@redhat.com>
```
  30172b49
- [Misc] Remove dangling references to `SamplingType.BEAM` (#13402) · efbe8544
  Harry Mellor authored Feb 18, 2025
  
  efbe8544
- [Bugfix][CI][V1] Work around V1 + CUDA Graph + torch._scaled_mm fallback issue (#13425) · b3942e15
  Tyler Michael Smith authored Feb 17, 2025
```
Signed-off-by: Tyler Michael Smith <tyler@neuralmagic.com>
```
  b3942e15
17 Feb, 2025 1 commit

[Model] Support Mamba2 (Codestral Mamba) (#9292) · 1f69c4a8

Tyler Michael Smith authored Feb 17, 2025


Signed-off-by: Tyler Michael Smith <tyler@neuralmagic.com>
Co-authored-by: Yu Chin Fabian Lim <flim@sg.ibm.com>

1f69c4a8

16 Feb, 2025 1 commit
- [Quant] Add `SupportsQuant` to phi3 and clip (#13104) · 12913d17
  Kyle Sayers authored Feb 15, 2025
  
  12913d17
15 Feb, 2025 1 commit
- [AMD] [Model] DeepSeek tunings (#13199) · ed0de3e4
  rasmith authored Feb 15, 2025
  
  ed0de3e4