Commits · eb5cb5e5280c83226408c3d1e30928364c6258db · OpenDAS / vllm_cscc

23 Jan, 2025 3 commits

[BugFix] Fix parameter names and `process_after_weight_loading` for W4A16 MoE... · eb5cb5e5

Dipika Sikka authored Jan 23, 2025


[BugFix] Fix parameter names and `process_after_weight_loading` for W4A16 MoE Group Act Order  (#11528)
Signed-off-by: ElizaWszola <eliza@neuralmagic.com>
Co-authored-by: ElizaWszola <eliza@neuralmagic.com>
Co-authored-by: Michael Goin <michael@neuralmagic.com>

eb5cb5e5

[FP8][Kernel] Dynamic kv cache scaling factors computation (#11906) · e97f802b

Gregory Shtrasberg authored Jan 23, 2025


Signed-off-by: Gregory Shtrasberg <Gregory.Shtrasberg@amd.com>
Co-authored-by: Micah Williamson <micah.williamson@amd.com>

e97f802b

[AMD][Quantization] Add TritonScaledMMLinearKernel since int8 is broken for AMD (#12282) · 68c4421b
rasmith authored Jan 22, 2025
```
Signed-off-by: Randall Smith <Randall.Smith@amd.com>
```
68c4421b

19 Jan, 2025 1 commit
- [Misc] Support register quantization method out-of-tree (#11969) · 32eb0da8
  yancong authored Jan 19, 2025
  
  32eb0da8
17 Jan, 2025 1 commit
- [AMD][FP8] Using MI300 FP8 format on ROCm for block_quant (#12134) · b5b57e30
  Gregory Shtrasberg authored Jan 17, 2025
```
Signed-off-by: Gregory Shtrasberg <Gregory.Shtrasberg@amd.com>
```
  b5b57e30
16 Jan, 2025 2 commits
- Various cosmetic/comment fixes (#12089) · 9aa1519f
  Michael Goin authored Jan 16, 2025
```
Signed-off-by: mgoin <michael@neuralmagic.com>
```
  9aa1519f
- [Core] Default to using per_token quantization for fp8 when cutlass is supported. (#8651) · fa0050db
  Elfie Guo authored Jan 15, 2025
```
Signed-off-by: mgoin <michael@neuralmagic.com>
Co-authored-by: Michael Goin <mgoin@redhat.com>
Co-authored-by: mgoin <michael@neuralmagic.com>
```
  fa0050db
15 Jan, 2025 2 commits

[Misc][Quark] Upstream Quark format to VLLM (#10765) · de0526f6

kewang-xlnx authored Jan 16, 2025


Signed-off-by: kewang-xlnx <kewang@xilinx.com>
Signed-off-by: kewang2 <kewang2@amd.com>
Co-authored-by: kewang2 <kewang2@amd.com>
Co-authored-by: Michael Goin <michael@neuralmagic.com>

de0526f6

Fix: cases with empty sparsity config (#12057) · cbe94391
Rahul Tuli authored Jan 15, 2025
```
Signed-off-by: Rahul Tuli <rahul@neuralmagic.com>
```
cbe94391

09 Jan, 2025 1 commit

[Misc] Move `print_*_once` from utils to logger (#11298) · d848800e

Cyrus Leung authored Jan 09, 2025

Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Signed-off-by: Maxime Fournioux <55544262+mfournioux@users.noreply.github.com>
Co-authored-by: Maxime Fournioux <55544262+mfournioux@users.noreply.github.com>

d848800e

08 Jan, 2025 2 commits
- [Kernel][Triton][AMD] Use block size heuristic for avg 2.8x speedup for int8 models (#11698) · 526de822
  rasmith authored Jan 08, 2025
```
Signed-off-by: Randall Smith <Randall.Smith@amd.com>
```
  526de822
- [TPU][Quantization] TPU `W8A8` (#11785) · 56fe4c29
  Robert Shaw authored Jan 08, 2025
```
Co-authored-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  56fe4c29
30 Dec, 2024 1 commit
- [CI/Build][CPU] Fix CPU CI by lazy importing triton FP8 kernels (#11618) · 5dbf8545
  Li, Jiang authored Dec 30, 2024
```
Signed-off-by: jiang1.li <jiang1.li@intel.com>
```
  5dbf8545
27 Dec, 2024 3 commits
- [Bugfix] Fix for ROCM compressed tensor support (#11561) · ac797994
  Selali authored Dec 27, 2024
  
  ac797994
- [BugFix] Fix quantization for all other methods (#11547) · 2339d59f
  Robert Shaw authored Dec 27, 2024
  
  2339d59f
- Deepseek v3 (#11502) · f49777ba
  Simon Mo authored Dec 26, 2024
```
Signed-off-by: mgoin <michael@neuralmagic.com>
Co-authored-by: mgoin <michael@neuralmagic.com>
Co-authored-by: robertgshaw2-neuralmagic <rshaw@neuralmagic.com>
```
  f49777ba
26 Dec, 2024 1 commit

[Model] [Quantization] Support deepseek_v3 w8a8 fp8 block-wise quantization (#11523) · 2072924d

Michael Goin authored Dec 26, 2024


Signed-off-by: mgoin <michael@neuralmagic.com>
Signed-off-by: simon-mo <simon.mo@hey.com>
Signed-off-by: simon-mo <xmo@berkeley.edu>
Co-authored-by: simon-mo <simon.mo@hey.com>
Co-authored-by: simon-mo <xmo@berkeley.edu>
Co-authored-by: HandH1998 <1335248067@qq.com>

2072924d

23 Dec, 2024 1 commit
- [Misc] Add assertion and helpful message for marlin24 compressed models (#11388) · b866cdbd
  Dipika Sikka authored Dec 23, 2024
  
  b866cdbd
21 Dec, 2024 1 commit
- [Bugfix] update should_ignore_layer (#11354) · 51ff216d
  George authored Dec 21, 2024
```
Signed-off-by: George Ohashi <george@neuralmagic.com>
```
  51ff216d
19 Dec, 2024 2 commits
- [Bugfix] Fix broken CPU compressed-tensors test (#11338) · 276738ce
  Isotr0py authored Dec 20, 2024
```
Signed-off-by: Isotr0py <2037008807@qq.com>
```
  276738ce
- [Bugfix][Build/CI] Fix sparse CUTLASS compilation on CUDA [12.0, 12.2) (#11311) · 5a9da2e6
  Tyler Michael Smith authored Dec 18, 2024
```
Signed-off-by: Tyler Michael Smith <tyler@neuralmagic.com>
```
  5a9da2e6
18 Dec, 2024 1 commit

[Kernel]: Cutlass 2:4 Sparsity + FP8/Int8 Quant Support (#10995) · 60508ffd

Dipika Sikka authored Dec 18, 2024


Co-authored-by: Faraz Shahsavan <faraz.shahsavan@gmail.com>
Co-authored-by: ilmarkov <markovilya197@gmail.com>
Co-authored-by: Rahul Tuli <rahul@neuralmagic.com>
Co-authored-by: rshaw@neuralmagic.com <rshaw@neuralmagic.com>

60508ffd

15 Dec, 2024 1 commit
- [[Misc]Upgrade bitsandbytes to the latest version 0.45.0 (#11201) · 15859f23
  Jee Jee Li authored Dec 15, 2024
  
  15859f23
27 Nov, 2024 2 commits
- [Bugfix] Fix GGUF inference with FP16 unquantized checkpoint (#10675) · b98c62ba
  Isotr0py authored Nov 28, 2024
```
Signed-off-by: Isotr0py <2037008807@qq.com>
```
  b98c62ba
- [bugfix] fix the default value of llm_int8_threshold in BitsAndBytesConfig (#10657) · cfb3bf25
  yansh97 authored Nov 27, 2024
  
  cfb3bf25
26 Nov, 2024 1 commit
- [Bugfix] Check bnb_4bit_quant_storage for bitsandbytes (#10642) · 7576cd38
  Michael Goin authored Nov 26, 2024
  
  7576cd38
21 Nov, 2024 1 commit
- [torch.compile] limit inductor threads and lazy import quant (#10482) · 388ee3de
  youkaichao authored Nov 20, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  388ee3de
19 Nov, 2024 1 commit
- [Model][Quantization] HQQ support through Marlin kernel expansion (#9766) · b00b33d7
  ElizaWszola authored Nov 19, 2024
```
Signed-off-by: ElizaWszola <eliza@neuralmagic.com>
```
  b00b33d7
18 Nov, 2024 3 commits
- [Kernel] Initial Machete W4A8 support + Refactors (#9855) · 96d999fb
  Lucas Wilkinson authored Nov 18, 2024
```
Signed-off-by: Lucas Wilkinson <lwilkinson@neuralmagic.com>
```
  96d999fb
- [Kernel] Explicitly specify other value in tl.load calls (#9014) · c2170a5b
  Angus Wang authored Nov 18, 2024
```
Signed-off-by: Angus Wang <wangjadehao@gmail.com>
```
  c2170a5b
- [Hardware][XPU] AWQ/GPTQ support for xpu backend (#10107) · 6b2d25ef
  Yan Ma authored Nov 19, 2024
```
Signed-off-by: yan ma <yan.ma@intel.com>
```
  6b2d25ef
15 Nov, 2024 1 commit

[bugfix] Fix static asymmetric quantization case (#10334) · bf2ddc66

Luka Govedič authored Nov 14, 2024


Signed-off-by: Daniël de Kok <me@danieldk.eu>
Signed-off-by: luka <luka@neuralmagic.com>
Co-authored-by: Daniël de Kok <me@danieldk.eu>

bf2ddc66

09 Nov, 2024 1 commit

[Kernel][Triton] Add Triton implementation for scaled_mm_triton to support fp8... · 127c0748

rasmith authored Nov 08, 2024


[Kernel][Triton] Add Triton implementation for scaled_mm_triton to support fp8 and int8 SmoothQuant, symmetric case (#9857)
Signed-off-by: Randall Smith <Randall.Smith@amd.com>

127c0748

07 Nov, 2024 2 commits
- [Hardware][CPU] Update torch 2.5 (#9911) · a4b3e0c1
  Li, Jiang authored Nov 07, 2024
```
Signed-off-by: jiang1.li <jiang1.li@intel.com>
```
  a4b3e0c1
- [Bugfix] Fix FP8 torch._scaled_mm fallback for torch>2.5 with CUDA<12.4 (#10095) · 4ab32566
  Michael Goin authored Nov 06, 2024
```
Signed-off-by: mgoin <michael@neuralmagic.com>
```
  4ab32566
06 Nov, 2024 1 commit
- Remove ScaledActivation for AWQ (#10057) · 399c7986
  Michael Goin authored Nov 06, 2024
```
Signed-off-by: mgoin <michael@neuralmagic.com>
```
  399c7986
05 Nov, 2024 1 commit
- [Misc] Modify BNB parameter name (#9997) · b9c64c0c
  Jee Jee Li authored Nov 06, 2024
```
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
```
  b9c64c0c
04 Nov, 2024 1 commit
- [Bugfix] Respect modules_to_not_convert within awq_marlin (#9895) · 8f0a9ca8
  Michael Goin authored Nov 04, 2024
```
Signed-off-by: mgoin <michael@neuralmagic.com>
```
  8f0a9ca8
01 Nov, 2024 2 commits
- [Bugfix/Core] Flashinfer k_scale and v_scale (#9861) · 598b6d7b
  Pavani Majety authored Nov 01, 2024
  
  598b6d7b
- [Bugfix] Fix layer skip logic with bitsandbytes (#9887) · 37a4947d
  Michael Goin authored Nov 01, 2024
```
Signed-off-by: mgoin <michael@neuralmagic.com>
```
  37a4947d