Commits · 01ca85bbd8d773fc5c234763d969ed46e0efd004 · OpenDAS / vllm_cscc

16 Mar, 2025 1 commit
- [Kernel] Add more tuned configs (#14877) · 71c1e071
  Simon Mo authored Mar 15, 2025
```
Signed-off-by: simon-mo <simon.mo@hey.com>
```
  71c1e071
14 Mar, 2025 1 commit
- [Attention] MLA get rid of materialization (#14770) · 9532c498
  Lucas Wilkinson authored Mar 14, 2025
```
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
```
  9532c498
13 Mar, 2025 2 commits
- [Bugfix] Fix chunked prefill for GGUF (#14666) · 55211b01
  Szymon Ożóg authored Mar 13, 2025
```
Signed-off-by: SzymonOzog <szymon.ozog@aleph-alpha.com>
```
  55211b01
- [ROCm][FP8] Fix for adjustments needed only for fnuz (#14689) · a94a699c
  Gregory Shtrasberg authored Mar 12, 2025
```
Signed-off-by: Gregory Shtrasberg <Gregory.Shtrasberg@amd.com>
```
  a94a699c
12 Mar, 2025 2 commits
- [Kernel] Add ModelOpt FP4 Checkpoint Support (#12520) · debd6bbf
  Pavani Majety authored Mar 11, 2025
```
Signed-off-by: Pavani Majety <pmajety@nvidia.com>
```
  debd6bbf
- [Kernel] GGUF MoE kernel (#14613) · e22ee1e7
  Szymon Ożóg authored Mar 12, 2025
```
Signed-off-by: SzymonOzog <szymon.ozog@aleph-alpha.com>
```
  e22ee1e7
11 Mar, 2025 1 commit
- dynamic distpatch of fp8 kernels (#14245) · a1c8f379
  Jeff Daily authored Mar 11, 2025
```
Signed-off-by: Jeff Daily <jeff.daily@amd.com>
```
  a1c8f379
10 Mar, 2025 1 commit

[Kernel] Add more dtype support for GGUF kernels (#14043) · 89cdaa83

Szymon Ożóg authored Mar 10, 2025


Signed-off-by: SzymonOzog <szymon.ozog@aleph-alpha.com>
Signed-off-by: SzymonOzog <szymon.ozog@gmail.com>

89cdaa83

07 Mar, 2025 2 commits
- [Kernel] optimize performance of gptq marlin kernel when n is small (#14138) · d0feea31
  Jinzhen Lin authored Mar 08, 2025
```
Signed-off-by: Jinzhen Lin <linjinzhen@hotmail.com>
```
  d0feea31
- [FP8] Refactor apply_fp8_linear and apply_fp8_linear_generic into an object (#14390) · e1744502
  Luka Govedič authored Mar 07, 2025
```
Signed-off-by: luka <luka@neuralmagic.com>
```
  e1744502
05 Mar, 2025 1 commit

[Bugfix] Fix gptq_marlin for deepseek-v3 (#13750) · 0df25101

rainkert authored Mar 05, 2025


Signed-off-by: dangshunya <dangshunya@baichuan-inc.com>
Co-authored-by: dangshunya <dangshunya@baichuan-inc.com>

0df25101

01 Mar, 2025 1 commit
- [Misc][Kernel]: Add GPTQAllSpark Quantization (#12931) · 6a92ff93
  YajieWang authored Mar 01, 2025
  
  6a92ff93
28 Feb, 2025 2 commits
- [torch.compile] Fix RMSNorm + quant fusion in the non-cutlass-fp8 case, rename... · bd56c983
  Luka Govedič authored Feb 28, 2025
```
[torch.compile] Fix RMSNorm + quant fusion in the non-cutlass-fp8 case, rename RedundantReshapesPass to NoopEliminationPass (#10902)
Signed-off-by: luka <luka@neuralmagic.com>
```
  bd56c983
- [Bugfix] Fix MoeWNA16Method activation (#14024) · e0734387
  Jee Jee Li authored Feb 28, 2025
  
  e0734387
27 Feb, 2025 1 commit
- [Model] Deepseek GGUF support (#13167) · 7f0be2aa
  Szymon Ożóg authored Feb 27, 2025
  
  7f0be2aa
26 Feb, 2025 2 commits
- [BugFix] Make FP8 Linear compatible with torch.compile (#13918) · b382a7f2
  Woosuk Kwon authored Feb 26, 2025
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  b382a7f2
- [Model] Support Grok1 (#13795) · 07c43530
  Michael Goin authored Feb 25, 2025
```
Signed-off-by: mgoin <mgoin64@gmail.com>
```
  07c43530
25 Feb, 2025 3 commits
- Fix CompressedTensorsWNA16MoE with grouped scales (#13769) · 4d251ad0
  Michael Goin authored Feb 25, 2025
  
  4d251ad0
- [Bugfix][Quantization] Fix FP8 + EP (#13784) · 1e15aaef
  Tyler Michael Smith authored Feb 24, 2025
```
Signed-off-by: Tyler Michael Smith <tyler@neuralmagic.com>
```
  1e15aaef
- [Misc] set single whitespace between log sentences (#13771) · 51010a18
  cjackal authored Feb 25, 2025
```
Signed-off-by: cjackal <44624812+cjackal@users.noreply.github.com>
```
  51010a18
24 Feb, 2025 1 commit
- Expert Parallelism (EP) Support for DeepSeek V2 (#12583) · 781096e3
  Jongseok Park authored Feb 24, 2025
  
  781096e3
22 Feb, 2025 1 commit
- [ROCm] Apply FP8 weights padding to values not divisible by 512 bytes on ROCm (#13231) · c904fddd
  Gregory Shtrasberg authored Feb 22, 2025
  
  c904fddd
21 Feb, 2025 1 commit

[Attention] MLA with chunked prefill (#12639) · 288cc6c2

Lucas Wilkinson authored Feb 21, 2025


Signed-off-by: Lucas Wilkinson <lwilkinson@neuralmagic.com>
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Co-authored-by: Patrick Horn <patrick.horn@gmail.com>
Co-authored-by: simon-mo <xmo@berkeley.edu>
Co-authored-by: Tyler Michael Smith <tyler@neuralmagic.com>

288cc6c2

18 Feb, 2025 1 commit
- [Bugfix][CI][V1] Work around V1 + CUDA Graph + torch._scaled_mm fallback issue (#13425) · b3942e15
  Tyler Michael Smith authored Feb 17, 2025
```
Signed-off-by: Tyler Michael Smith <tyler@neuralmagic.com>
```
  b3942e15
16 Feb, 2025 1 commit
- [Quant] Add `SupportsQuant` to phi3 and clip (#13104) · 12913d17
  Kyle Sayers authored Feb 15, 2025
  
  12913d17
15 Feb, 2025 1 commit
- [AMD] [Model] DeepSeek tunings (#13199) · ed0de3e4
  rasmith authored Feb 15, 2025
  
  ed0de3e4
14 Feb, 2025 2 commits
- [Quant][Perf] Use moe_wna16 kernel by default for MoEs with many experts (#13236) · 5e5c8e09
  Michael Goin authored Feb 14, 2025
```
Signed-off-by: mgoin <mgoin64@gmail.com>
```
  5e5c8e09
- [Kernel][Bugfix] Refactor and Fix CUTLASS 2:4 Sparse Kernels (#13198) · c1e37bf7
  Tyler Michael Smith authored Feb 13, 2025
```
Signed-off-by: Tyler Michael Smith <tyler@neuralmagic.com>
```
  c1e37bf7
13 Feb, 2025 1 commit
- Allow Unsloth Dynamic 4bit BnB quants to work (#12974) · cb944d58
  Daniel Han authored Feb 12, 2025
  
  cb944d58
12 Feb, 2025 2 commits
- [Bugfix] Allow fallback to AWQ from AWQMarlin at per-layer granularity (#13119) · 09972e71
  Michael Goin authored Feb 12, 2025
  
  09972e71
- [CORE] [QUANT] Support for GPTQModel's `dynamic` quantization per module override/control (#7086) · 36a08630
  Qubitium-ModelCloud authored Feb 13, 2025
  
  36a08630
07 Feb, 2025 1 commit

[ROCm] [Feature] [Doc] [Dockerfile] [BugFix] Support Per-Token-Activation... · eaa92d44

TJian authored Feb 08, 2025

[ROCm] [Feature] [Doc] [Dockerfile] [BugFix] Support Per-Token-Activation Per-Channel-Weight FP8 Quantization Inferencing (#12501)

eaa92d44

06 Feb, 2025 4 commits
- [MISC] Check space in the file names in the pre commit checks (#12804) · 741429a4
  Lu Fang authored Feb 06, 2025
```
Signed-off-by: Lu Fang <lufang@fb.com>
```
  741429a4
- [Misc] Update w2 scale loading for GPTQMarlinMoE (#12757) · 7ca9934f
  Dipika Sikka authored Feb 06, 2025
  
  7ca9934f
- [Misc][Easy] Remove the space from the file name · 9cdea30b
  Lu Fang authored Feb 05, 2025
  
  9cdea30b
- [Bugfix] Better FP8 supported defaults · 76abd0c8
  Lucas Wilkinson authored Feb 05, 2025
  
  76abd0c8
05 Feb, 2025 2 commits
- Add: Support for Sparse24Bitmask Compressed Models · 3b2005e1
  Rahul Tuli authored Feb 05, 2025
  
  3b2005e1
- [Model][Quant] Fix GLM, Fix fused module mappings for quantization (#12634) · 7ff7a638
  Kyle Sayers authored Feb 05, 2025
```
Signed-off-by: mgoin <michael@neuralmagic.com>
Signed-off-by: Kyle Sayers <kylesayrs@gmail.com>
Co-authored-by: mgoin <michael@neuralmagic.com>
```
  7ff7a638
04 Feb, 2025 2 commits
- [AMD][ROCm] Enable DeepSeek model on ROCm (#12662) · c36ac98d
  Hongxia Yang authored Feb 04, 2025
```
Signed-off-by: Hongxia Yang <hongxia.yang@amd.com>
Co-authored-by: Matthew Wong <Matthew.Wong2@amd.com>
```
  c36ac98d
- [Quant] Fix use_mla TypeError and support loading pure-sparsity Compressed Tensors configs (#12711) · 4896d0c2
  Kyle Sayers authored Feb 04, 2025
  
  4896d0c2