Commits · 2188a60c7e0e5a414a87a4f0fd798333b2e0f625 · OpenDAS / vllm_cscc

03 Sep, 2024 1 commit
- [Misc] Update `GPTQ` to use `vLLMParameters` (#7976) · 2188a60c
  Dipika Sikka authored Sep 03, 2024
  
  2188a60c
30 Aug, 2024 1 commit

[Model] Adding support for MSFT Phi-3.5-MoE (#7729) · 1248e850

Wenxiang authored Aug 31, 2024


Co-authored-by: Your Name <you@example.com>
Co-authored-by: Zeqi Lin <zelin@microsoft.com>
Co-authored-by: Zeqi Lin <Zeqi.Lin@microsoft.com>

1248e850

29 Aug, 2024 2 commits
- support bitsandbytes 8-bit and FP4 quantized models (#7445) · 4664ceaa
  chenqianfzh authored Aug 29, 2024
  
  4664ceaa
- [misc] update tpu int8 to use new vLLM Parameters (#7973) · 86a677de
  Dipika Sikka authored Aug 29, 2024
  
  86a677de
28 Aug, 2024 1 commit
- [Kernel] [Triton] [AMD] Adding Triton implementations awq_dequantize and... · e5697d16
  rasmith authored Aug 28, 2024
```
[Kernel] [Triton] [AMD] Adding Triton implementations awq_dequantize and awq_gemm to support AWQ (#7386)
```
  e5697d16
27 Aug, 2024 2 commits
- [Kernel] Expand MoE weight loading + Add Fused Marlin MoE Kernel (#7766) · fc911880
  Dipika Sikka authored Aug 27, 2024
```
Co-authored-by: ElizaWszola <eliza@neuralmagic.com>
```
  fc911880
- [Misc] Update compressed tensors lifecycle to remove `prefix` from `create_weights` (#7825) · 015e6cc2
  Dipika Sikka authored Aug 26, 2024
  
  015e6cc2
26 Aug, 2024 2 commits
- [Misc] Update `gptq_marlin_24` to use vLLMParameters (#7762) · dd9857f5
  Dipika Sikka authored Aug 26, 2024
```
Co-authored-by: Michael Goin <michael@neuralmagic.com>
```
  dd9857f5
- [Misc] Update `qqq` to use vLLMParameters (#7805) · 66530409
  Dipika Sikka authored Aug 26, 2024
  
  66530409
23 Aug, 2024 1 commit
- [Misc] Update `marlin` to use vLLMParameters (#7803) · f1df5dbf
  Dipika Sikka authored Aug 23, 2024
  
  f1df5dbf
22 Aug, 2024 2 commits
- [Misc] update fp8 to use `vLLMParameter` (#7437) · 955b5191
  Dipika Sikka authored Aug 22, 2024
  
  955b5191
- Revert "[Kernel] Expand MoE weight loading + Add Fused Marlin MoE Kernel (#7527)" (#7764) · aae74ef9
  Michael Goin authored Aug 21, 2024
  
  aae74ef9
21 Aug, 2024 1 commit
- [Kernel] Expand MoE weight loading + Add Fused Marlin MoE Kernel (#7527) · 8678a69a
  Dipika Sikka authored Aug 21, 2024
```
Co-authored-by: ElizaWszola <eliza@neuralmagic.com>
```
  8678a69a
20 Aug, 2024 1 commit
- [Kernel] (1/N) Machete - Hopper Optimized Mixed Precision Linear Kernel (#7174) · 5288c06a
  Lucas Wilkinson authored Aug 20, 2024
  
  5288c06a
19 Aug, 2024 1 commit
- [Core] Support tensor parallelism for GGUF quantization (#7520) · 7601cb04
  Isotr0py authored Aug 20, 2024
  
  7601cb04
16 Aug, 2024 3 commits
- [Kernel] fix types used in aqlm and ggml kernels to support dynamo (#7596) · 37fd47e7
  bnellnm authored Aug 16, 2024
  
  37fd47e7
- [Kernel] W8A16 Int8 inside FusedMoE (#7415) · 7fc23be8
  Mor Zusman authored Aug 16, 2024
  
  7fc23be8
- [Feature][Hardware][Amd] Add fp8 Linear Layer for Rocm (#7210) · e837b624
  Charlie Fu authored Aug 16, 2024
  
  e837b624
14 Aug, 2024 1 commit
- [Misc] Revert `compressed-tensors` code reuse (#7521) · f55a9aea
  Kyle Sayers authored Aug 14, 2024
  
  f55a9aea
13 Aug, 2024 4 commits
- [Misc] `compressed-tensors` code reuse (#7277) · 373538f9
  Kyle Sayers authored Aug 13, 2024
  
  373538f9
- [Misc] Update `awq` and `awq_marlin` to use `vLLMParameters` (#7422) · b1e5afc3
  Dipika Sikka authored Aug 13, 2024
  
  b1e5afc3
- [Misc] Update Fused MoE weight loading (#7334) · d3bdfd3a
  Dipika Sikka authored Aug 13, 2024
  
  d3bdfd3a
- [Misc] Update `gptq_marlin` to use new vLLMParameters (#7281) · fb377d7e
  Dipika Sikka authored Aug 13, 2024
  
  fb377d7e
09 Aug, 2024 1 commit
- [TPU] Add Load-time W8A16 quantization for TPU Backend (#7005) · 0fa14907
  Siyuan Liu authored Aug 08, 2024
  
  0fa14907
07 Aug, 2024 4 commits
- [Bugfix] Fix gptq failure on T4s (#7264) · 311f7438
  Lucas Wilkinson authored Aug 07, 2024
  
  311f7438
- [Bugfix][FP8] Fix dynamic FP8 Marlin quantization (#7219) · 5223199e
  Michael Goin authored Aug 07, 2024
  
  5223199e
- [Misc] Refactor linear layer weight loading; introduce `BasevLLMParameter` and... · 0f7052bc
  Dipika Sikka authored Aug 07, 2024
```
[Misc] Refactor linear layer weight loading; introduce `BasevLLMParameter` and `weight_loader_v2` (#5874)
```
  0f7052bc
- [Bugfix] Fix GPTQ and GPTQ Marlin CPU Offloading (#7225) · f9a56006
  Michael Goin authored Aug 06, 2024
  
  f9a56006
05 Aug, 2024 1 commit
- [Core] Support loading GGUF model (#5191) · 360bd67c
  Isotr0py authored Aug 06, 2024
```
Co-authored-by: Michael Goin <michael@neuralmagic.com>
```
  360bd67c
02 Aug, 2024 1 commit
- [Misc] Disambiguate quantized types via a new ScalarType (#6396) · a8d604ca
  Lucas Wilkinson authored Aug 02, 2024
  
  a8d604ca
31 Jul, 2024 1 commit
- Support W4A8 quantization for vllm (#5218) · 6512937d
  HandH1998 authored Jul 31, 2024
  
  6512937d
30 Jul, 2024 1 commit
- [Kernel] Remove scaled_fp8_quant kernel padding footgun (#6842) · d7a299ed
  Tyler Michael Smith authored Jul 30, 2024
  
  d7a299ed
29 Jul, 2024 1 commit
- [Bugfix] Allow vllm to still work if triton is not installed. (#6786) · 9a7e2d05
  Thomas Parnell authored Jul 29, 2024
```
Signed-off-by: Thomas Parnell <tpa@zurich.ibm.com>
```
  9a7e2d05
28 Jul, 2024 1 commit
- [Misc] Pass cutlass_fp8_supported correctly in fbgemm_fp8 (#6871) · 3eeb148f
  Elsa Granger authored Jul 28, 2024
  
  3eeb148f
27 Jul, 2024 1 commit
- [Kernel] Increase precision of GPTQ/AWQ Marlin kernel (#6795) · 75acdaa4
  Alexander Matveev authored Jul 27, 2024
  
  75acdaa4
25 Jul, 2024 4 commits
- [Bugfix] Fix empty (nullptr) channelwise scales when loading wNa16 using... · cd7edc4e
  Lucas Wilkinson authored Jul 25, 2024
```
[Bugfix] Fix empty (nullptr) channelwise  scales when loading wNa16 using compressed tensors (#6798)
```
  cd7edc4e
- [Bugfix] Fix `kv_cache_dtype=fp8` without scales for FP8 checkpoints (#6761) · 65b1f121
  Michael Goin authored Jul 25, 2024
  
  65b1f121
- [ Misc ] `fp8-marlin` channelwise via `compressed-tensors` (#6524) · 889da130
  Robert Shaw authored Jul 25, 2024
```
Co-authored-by: mgoin <michael@neuralmagic.com>
```
  889da130
- [Bugfix] Fix awq_marlin and gptq_marlin flags (#6745) · 0310029a
  Alexander Matveev authored Jul 25, 2024
  
  0310029a
23 Jul, 2024 1 commit
- [bitsandbytes]: support read bnb pre-quantized model (#5753) · 87525fab
  dongmao zhang authored Jul 23, 2024
```
Co-authored-by: Michael Goin <michael@neuralmagic.com>
```
  87525fab