Commits · 9090bf02e74334a8020b454814e0d00fa780fd79 · norm / vllm · GitLab

29 Jan, 2024 1 commit

Support FP8-E5M2 KV Cache (#2279) · 9090bf02

zhaoyang-star authored Jan 29, 2024


Co-authored-by: zhaoyang <zhao.yang16@zte.com.cn>
Co-authored-by: Zhuohan Li <zhuohan123@gmail.com>

9090bf02

26 Jan, 2024 1 commit
- [ROCm] add support to ROCm 6.0 and MI300 (#2274) · 6b7de1a0
  Hongxia Yang authored Jan 26, 2024
  
  6b7de1a0
25 Jan, 2024 1 commit
- fix names and license for Qwen2 (#2589) · 2832e7b9
  Junyang Lin authored Jan 25, 2024
  
  2832e7b9
24 Jan, 2024 1 commit
- Fix the syntax error in the doc of supported_models (#2584) · 223c1922
  LastWhisper authored Jan 25, 2024
  
  223c1922
23 Jan, 2024 1 commit
- [Feature] Simple API token authentication and pluggable middlewares (#1106) · 9c1352eb
  Erfan Al-Hossami authored Jan 23, 2024
  
  9c1352eb
22 Jan, 2024 1 commit
- Add qwen2 (#2495) · 94b5edeb
  Junyang Lin authored Jan 23, 2024
  
  94b5edeb
17 Jan, 2024 1 commit
- Add StableLM3B model (#2372) · e1957c6e
  Hyunsung Lee authored Jan 17, 2024
  
  e1957c6e
12 Jan, 2024 3 commits
- Update quickstart.rst (#2369) · 827cbcd3
  Simon authored Jan 12, 2024
  
  827cbcd3
- [Minor] Fix the format in quick start guide related to Model Scope (#2425) · f745847e
  Zhuohan Li authored Jan 11, 2024
  
  f745847e
- [DOC] Add additional comments for LLMEngine and AsyncLLMEngine (#1011) · 6549aef2
  Jiaxiang authored Jan 12, 2024
  
  6549aef2
03 Jan, 2024 1 commit
- Use NCCL instead of ray for control-plane communication to remove serialization overhead (#2221) · fd4ea8ef
  Zhuohan Li authored Jan 04, 2024
  
  fd4ea8ef
23 Dec, 2023 1 commit
- [Docs] Update installation instructions to include CUDA 11.8 xFormers (#2246) · 1db83e31
  Shivam Thakkar authored Dec 23, 2023
  
  1db83e31
20 Dec, 2023 1 commit
- [Docs] Fix broken links (#2222) · c17daa9f
  Ronen Schaffer authored Dec 20, 2023
  
  c17daa9f
19 Dec, 2023 1 commit
- Added DeciLM-7b and DeciLM-7b-instruct (#2062) · de60a3fb
  avideci authored Dec 19, 2023
  
  de60a3fb
18 Dec, 2023 1 commit
- [ROCm] Fixes for GPTQ on ROCm (#2180) · 1b7c791d
  kliuae authored Dec 19, 2023
  
  1b7c791d
17 Dec, 2023 3 commits
- [Docs] Update documentation for gpu-memory-utilization option (#2162) · 3ec8c25c
  Suhong Moon authored Dec 17, 2023
  
  3ec8c25c
- [Minor] Add Phi 2 to supported models (#2159) · f8c688d7
  Woosuk Kwon authored Dec 17, 2023
  
  f8c688d7
- [Docs] Add CUDA graph support to docs (#2148) · 26c52a5e
  Woosuk Kwon authored Dec 17, 2023
  
  26c52a5e
15 Dec, 2023 1 commit
- [Docs] Add supported quantization methods to docs (#2135) · b81a6a6b
  Woosuk Kwon authored Dec 15, 2023
  
  b81a6a6b
14 Dec, 2023 1 commit
- Optimize Mixtral with expert parallelism (#2090) · 21d93c14
  Antoni Baum authored Dec 13, 2023
  
  21d93c14
13 Dec, 2023 3 commits
- [Docs] Add notes on ROCm-supported models (#2087) · 096827c2
  Woosuk Kwon authored Dec 13, 2023
  
  096827c2
- Update installation instruction for vLLM + CUDA 11.8 (#2086) · 6565d9e3
  Woosuk Kwon authored Dec 13, 2023
  
  6565d9e3
- [ROCm] Upgrade xformers version for ROCm & update doc (#2079) · f375ec84
  TJian authored Dec 13, 2023
```
Co-authored-by: miloice <jeffaw99@hotmail.com>
```
  f375ec84
11 Dec, 2023 2 commits
- Update run_on_sky.rst (#2025) · c0ce15df
  Ikko Eltociear Ashimine authored Dec 12, 2023
```
sharable -> shareable
```
  c0ce15df
- Minor fixes for Mixtral (#2015) · 4ff02039
  Woosuk Kwon authored Dec 11, 2023
  
  4ff02039
08 Dec, 2023 2 commits

[Docker] Add cuda arch list as build option (#1950) · c85b80c2
Simon Mo authored Dec 08, 2023

c85b80c2

Merge EmbeddedLLM/vllm-rocm into vLLM main (#1836) · 6ccc0bff

TJian authored Dec 08, 2023


Co-authored-by: Philipp Moritz <pcmoritz@gmail.com>
Co-authored-by: Amir Balwel <amoooori04@gmail.com>
Co-authored-by: root <kuanfu.liu@akirakan.com>
Co-authored-by: tjtanaa <tunjian.tan@embeddedllm.com>
Co-authored-by: kuanfu <kuanfu.liu@embeddedllm.com>
Co-authored-by: miloice <17350011+kliuae@users.noreply.github.com>

6ccc0bff

07 Dec, 2023 2 commits
- [Docker] Adding number of nvcc_threads during build as envar (#1893) · 24f60a54
  AguirreNicolas authored Dec 07, 2023
  
  24f60a54
- Fix quickstart.rst typo jinja (#1964) · 42c02f58
  gottlike authored Dec 07, 2023
  
  42c02f58
06 Dec, 2023 1 commit
- Fix typo in adding_model.rst (#1947) · d940ce49
  Peter Götz authored Dec 06, 2023
```
adpated -> adapted
```
  d940ce49
03 Dec, 2023 2 commits
- chore(examples-docs): upgrade to OpenAI V1 (#1785) · c07a4428
  Massimiliano Pronesti authored Dec 03, 2023
  
  c07a4428
- Add Production Metrics in Prometheus format (#1890) · 5313c2cb
  Simon Mo authored Dec 02, 2023
  
  5313c2cb
02 Dec, 2023 1 commit
- [Docs] Update the AWQ documentation to highlight performance issue (#1883) · 4cefa9b4
  Simon Mo authored Dec 02, 2023
  
  4cefa9b4
01 Dec, 2023 2 commits
- Normalize head weights for Baichuan 2 (#1876) · e5452ddf
  Woosuk Kwon authored Nov 30, 2023
  
  e5452ddf
- Support chat template and `echo` for chat API (#1756) · 66785cc0
  Adam Brusselback authored Nov 30, 2023
  
  66785cc0
30 Nov, 2023 2 commits
- docs: add instruction for langchain (#1162) · 05a38612
  Massimiliano Pronesti authored Nov 30, 2023
  
  05a38612
- [Docs] Add information about using shared memory in docker (#1845) · 0f621c2c
  Simon Mo authored Nov 29, 2023
  
  0f621c2c
22 Nov, 2023 1 commit
- [DOCS] Add engine args documentation (#1741) · a921d8be
  Casper authored Nov 22, 2023
  
  a921d8be
20 Nov, 2023 2 commits
- [Docs] Fix the code block's format in deploying_with_docker page (#1722) · 112627e8
  Wen Sun authored Nov 20, 2023
  
  112627e8
- Documentation about official docker image (#1709) · 37c1e3c2
  Simon Mo authored Nov 19, 2023
  
  37c1e3c2