Commits · 60508ffda91c22e4cde3b18f149d222211db8886 · OpenDAS / vllm_cscc

18 Dec, 2024 1 commit

[Kernel]: Cutlass 2:4 Sparsity + FP8/Int8 Quant Support (#10995) · 60508ffd

Dipika Sikka authored Dec 18, 2024


Co-authored-by: Faraz Shahsavan <faraz.shahsavan@gmail.com>
Co-authored-by: ilmarkov <markovilya197@gmail.com>
Co-authored-by: Rahul Tuli <rahul@neuralmagic.com>
Co-authored-by: rshaw@neuralmagic.com <rshaw@neuralmagic.com>

60508ffd

15 Nov, 2024 1 commit

[bugfix] Fix static asymmetric quantization case (#10334) · bf2ddc66

Luka Govedič authored Nov 14, 2024


Signed-off-by: Daniël de Kok <me@danieldk.eu>
Signed-off-by: luka <luka@neuralmagic.com>
Co-authored-by: Daniël de Kok <me@danieldk.eu>

bf2ddc66

15 Oct, 2024 1 commit
- [Misc] Directly use compressed-tensors for checkpoint definitions (#8909) · 22f8a695
  Michael Goin authored Oct 15, 2024
```
Co-authored-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  22f8a695
27 Sep, 2024 1 commit
- [Kernel] AQ AZP 4/4: Integrate asymmetric quantization to linear method (#7271) · 172d1cd2
  Luka Govedič authored Sep 27, 2024
  
  172d1cd2
11 Sep, 2024 1 commit
- [Hardware][Intel] Support compressed-tensor W8A8 for CPU backend (#7257) · 0b952af4
  Li, Jiang authored Sep 12, 2024
  
  0b952af4
27 Aug, 2024 1 commit
- [Kernel] Expand MoE weight loading + Add Fused Marlin MoE Kernel (#7766) · fc911880
  Dipika Sikka authored Aug 27, 2024
```
Co-authored-by: ElizaWszola <eliza@neuralmagic.com>
```
  fc911880
14 Aug, 2024 1 commit
- [Misc] Revert `compressed-tensors` code reuse (#7521) · f55a9aea
  Kyle Sayers authored Aug 14, 2024
  
  f55a9aea
13 Aug, 2024 1 commit
- [Misc] `compressed-tensors` code reuse (#7277) · 373538f9
  Kyle Sayers authored Aug 13, 2024
  
  373538f9
07 Aug, 2024 1 commit
- [Misc] Refactor linear layer weight loading; introduce `BasevLLMParameter` and... · 0f7052bc
  Dipika Sikka authored Aug 07, 2024
```
[Misc] Refactor linear layer weight loading; introduce `BasevLLMParameter` and `weight_loader_v2` (#5874)
```
  0f7052bc
01 Aug, 2024 1 commit
- [CI/Build] Remove sparseml requirement from testing (#7037) · fb3db616
  Michael Goin authored Aug 01, 2024
  
  fb3db616
23 Jul, 2024 1 commit
- [Misc] Support FP8 kv cache scales from compressed-tensors (#6528) · 9e0b558a
  Michael Goin authored Jul 23, 2024
  
  9e0b558a
11 Jul, 2024 1 commit
- [ Misc ] Refactor Marlin Python Utilities (#6082) · b675069d
  Robert Shaw authored Jul 11, 2024
```
Co-authored-by: Robert Shaw <rshaw@neuralmagic.com>
```
  b675069d
07 Jul, 2024 1 commit
- [ Misc ] Support Fp8 via `llm-compressor` (#6110) · abfe705a
  Robert Shaw authored Jul 07, 2024
```
Co-authored-by: Robert Shaw <rshaw@neuralmagic>
```
  abfe705a
03 Jul, 2024 1 commit
- [ Misc ] Clean Up `CompressedTensorsW8A8` (#6113) · 62963d12
  Robert Shaw authored Jul 03, 2024
  
  62963d12
30 Jun, 2024 1 commit
- [ Misc ] Refactor w8a8 to use `process_weights_after_load` (Simplify Weight Loading) (#5940) · af9ad46f
  Robert Shaw authored Jun 30, 2024
```
Co-authored-by: Robert Shaw <rshaw@neuralmagic>
```
  af9ad46f
25 Jun, 2024 1 commit
- [Misc] Update `w4a16` `compressed-tensors` support to include `w8a16` (#5794) · dd248f76
  Dipika Sikka authored Jun 25, 2024
  
  dd248f76
19 Jun, 2024 1 commit
- [Misc] Add per channel support for static activation quantization; update w8a8... · 4a30d7e3
  Dipika Sikka authored Jun 19, 2024
```
[Misc] Add per channel support for static activation quantization; update w8a8 schemes to share base classes (#5650)
```
  4a30d7e3
18 Jun, 2024 1 commit
- [Misc] Add channel-wise quantization support for w8a8 dynamic per token... · 95db455e
  Dipika Sikka authored Jun 18, 2024
```
[Misc] Add channel-wise quantization support for w8a8 dynamic per token activation quantization (#5542)
```
  95db455e
17 Jun, 2024 1 commit
- [Kernel] `compressed-tensors` marlin 24 support (#5435) · 890d8d96
  Dipika Sikka authored Jun 17, 2024
  
  890d8d96
13 Jun, 2024 1 commit
- [Kernel] `w4a16` support for `compressed-tensors` (#5385) · c2637a61
  Dipika Sikka authored Jun 13, 2024
```
Co-authored-by: Robert Shaw <114415538+robertgshaw2-neuralmagic@users.noreply.github.com>
```
  c2637a61
10 Jun, 2024 1 commit
- [Misc] Update to comply with the new `compressed-tensors` config (#5350) · 5884c2b4
  Dipika Sikka authored Jun 09, 2024
```
Co-authored-by: Michael Goin <michael@neuralmagic.com>
```
  5884c2b4
08 Jun, 2024 1 commit
- [CI/Test] improve robustness of test (vllm_runner) (#5357) · 8ea5e44a
  youkaichao authored Jun 08, 2024
```
[CI/Test] improve robustness of test by replacing del with context manager (vllm_runner) (#5357)
```
  8ea5e44a
07 Jun, 2024 1 commit

[Kernel] Dynamic Per-Token Activation Quantization (#5037) · ca3ea51b

Dipika Sikka authored Jun 07, 2024


Co-authored-by: Varun Sundar Rabindranath <varunsundar08@gmail.com>
Co-authored-by: Varun Sundar Rabindranath <varun@neuralmagic.com>

ca3ea51b

23 May, 2024 1 commit

[Kernel] Initial Activation Quantization Support (#4525) · a1242324

Dipika Sikka authored May 23, 2024


Co-authored-by: Varun Sundar Rabindranath <varunsundar08@gmail.com>
Co-authored-by: Varun Sundar Rabindranath <varun@neuralmagic.com>

a1242324