Commits · 51679bbda84df6e68d4cef8f59abf8f962bf59e0 · norm / vllm

29 Jan, 2024 1 commit

Support FP8-E5M2 KV Cache (#2279) · 9090bf02

zhaoyang-star authored Jan 29, 2024


Co-authored-by: zhaoyang <zhao.yang16@zte.com.cn>
Co-authored-by: Zhuohan Li <zhuohan123@gmail.com>

9090bf02

27 Jan, 2024 1 commit
- AWQ: Up to 2.66x higher throughput (#2566) · beb89f68
  Casper authored Jan 27, 2024
  
  beb89f68
03 Jan, 2024 2 commits
- Enable CUDA graph for GPTQ & SqueezeLLM (#2318) · 6ef00b03
  Woosuk Kwon authored Jan 03, 2024
  
  6ef00b03
- [FIX] Support non-zero CUDA devices in custom kernels (#1959) · 77af974b
  Jee Li authored Jan 03, 2024
  
  77af974b
18 Dec, 2023 1 commit
- [ROCm] Fixes for GPTQ on ROCm (#2180) · 1b7c791d
  kliuae authored Dec 19, 2023
  
  1b7c791d
15 Dec, 2023 1 commit
- Add GPTQ support (#916) · 0fbfc4b8
  CHU Tianxiang authored Dec 15, 2023
  
  0fbfc4b8
08 Dec, 2023 1 commit

Merge EmbeddedLLM/vllm-rocm into vLLM main (#1836) · 6ccc0bff

TJian authored Dec 08, 2023


Co-authored-by: Philipp Moritz <pcmoritz@gmail.com>
Co-authored-by: Amir Balwel <amoooori04@gmail.com>
Co-authored-by: root <kuanfu.liu@akirakan.com>
Co-authored-by: tjtanaa <tunjian.tan@embeddedllm.com>
Co-authored-by: kuanfu <kuanfu.liu@embeddedllm.com>
Co-authored-by: miloice <17350011+kliuae@users.noreply.github.com>

6ccc0bff

22 Oct, 2023 1 commit

Support SqueezeLLM (#1326) · 1f24755b

chooper1 authored Oct 22, 2023


Co-authored-by: squeeze-ai-lab <squeezeailab.bair@gmail.com>
Co-authored-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>

1f24755b

16 Oct, 2023 1 commit
- Minor fix on AWQ kernel launch (#1356) · 29678cd2
  Woosuk Kwon authored Oct 15, 2023
  
  29678cd2
11 Oct, 2023 2 commits
- Fix overflow in awq kernel (#1295) · 980dd4a2
  CHU Tianxiang authored Oct 11, 2023
```
Co-authored-by: 楚天翔 <tianxiang.ctx@alibaba-inc.com>
```
  980dd4a2
- workaround of AWQ for Turing GPUs (#1252) · 82857368
  twaka authored Oct 11, 2023
  
  82857368
18 Sep, 2023 1 commit
- Add minimum capability requirement for AWQ (#1064) · 2b1c116b
  Woosuk Kwon authored Sep 18, 2023
  
  2b1c116b
16 Sep, 2023 1 commit

Implement AWQ quantization support for LLaMA (#1032) · e3e79e9e

Woosuk Kwon authored Sep 16, 2023


Co-authored-by: Robert Irvine <robert@seamlessml.com>
Co-authored-by: root <rirv938@gmail.com>
Co-authored-by: Casper <casperbh.96@gmail.com>
Co-authored-by: julian-q <julianhquevedo@gmail.com>

e3e79e9e