Commits · ead94d9383644234fa9f153a950ab8cdc334c588 · SIYIXNI / vllm

15 Jan, 2024 1 commit
- fix weigit loading for GQA with TP (#2379) · f780504d
  Chenhui Zhang authored Jan 16, 2024
  
  f780504d
12 Jan, 2024 3 commits
- Aligning `top_p` and `top_k` Sampling (#1885) · 218dc2cc
  陈序 authored Jan 13, 2024
```
* Align top_p and top_k with huggingface

* remove _get_prompt_and_output_tokens

* rename _apply_top_p_top_k

* compare top_p top_k with hf

* fix test errors
```
  218dc2cc
- Address Phi modeling update 2 (#2428) · 7878958c
  Gary Hui authored Jan 13, 2024
  
  7878958c
- Rename phi_1_5 -> phi (#2385) · 50376faa
  Woosuk Kwon authored Jan 11, 2024
  
  50376faa
09 Jan, 2024 1 commit
- [Speculative decoding 1/9] Optimized rejection sampler (#2336) · 79d64c49
  Cade Daniel authored Jan 09, 2024
  
  79d64c49
08 Jan, 2024 1 commit
- [Minor] Remove unused code in attention (#2384) · 28c3f121
  Woosuk Kwon authored Jan 08, 2024
  
  28c3f121
03 Jan, 2024 2 commits
- Use NCCL instead of ray for control-plane communication to remove serialization overhead (#2221) · fd4ea8ef
  Zhuohan Li authored Jan 04, 2024
  
  fd4ea8ef
- [Minor] Fix typo and remove unused code (#2305) · 91405610
  Roy authored Jan 03, 2024
  
  91405610
30 Dec, 2023 1 commit
- Support GPT-NeoX Models without attention biases (#2301) · 4934d492
  Jong-hun Shin authored Dec 31, 2023
  
  4934d492
20 Dec, 2023 2 commits
- Remove Sampler copy stream (#2209) · bd29cf3d
  Antoni Baum authored Dec 20, 2023
  
  bd29cf3d
- [BugFix] Fix weight loading for Mixtral with TP (#2208) · ba4f8267
  Woosuk Kwon authored Dec 19, 2023
  
  ba4f8267
19 Dec, 2023 1 commit
- Added DeciLM-7b and DeciLM-7b-instruct (#2062) · de60a3fb
  avideci authored Dec 19, 2023
  
  de60a3fb
17 Dec, 2023 4 commits
- [Minor] Fix a typo in .pt weight support (#2160) · 2c9b6380
  Woosuk Kwon authored Dec 17, 2023
  
  2c9b6380
- Make sampler less blocking (#1889) · a7347d9a
  Antoni Baum authored Dec 17, 2023
  
  a7347d9a
- Remove dependency on CuPy (#2152) · c3372e87
  Woosuk Kwon authored Dec 17, 2023
  
  c3372e87
- Optimize model execution with CUDA graph (#1926) · 37ca5581
  Woosuk Kwon authored Dec 16, 2023
```
Co-authored-by: Chen Shen <scv119@gmail.com>
Co-authored-by: Antoni Baum <antoni.baum@protonmail.com>
```
  37ca5581
16 Dec, 2023 1 commit
- Simplify weight loading logic (#2133) · eed74a55
  Roy authored Dec 17, 2023
  
  eed74a55
15 Dec, 2023 1 commit
- Add GPTQ support (#916) · 0fbfc4b8
  CHU Tianxiang authored Dec 15, 2023
  
  0fbfc4b8
14 Dec, 2023 1 commit
- Optimize Mixtral with expert parallelism (#2090) · 21d93c14
  Antoni Baum authored Dec 13, 2023
  
  21d93c14
13 Dec, 2023 1 commit
- Implement lazy model loader (#2044) · 518369d7
  Woosuk Kwon authored Dec 12, 2023
  
  518369d7
12 Dec, 2023 2 commits
- Support MPT with GQA (#1938) · 6428f1d0
  Megha Agarwal authored Dec 12, 2023
```
Co-authored-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  6428f1d0
- Upgrade transformers version to 4.36.0 (#2046) · cb3f30c6
  Woosuk Kwon authored Dec 11, 2023
  
  cb3f30c6
11 Dec, 2023 5 commits
- Remove python 3.10 requirement (#2040) · 31d2ab4a
  Woosuk Kwon authored Dec 11, 2023
  
  31d2ab4a
- Fix import error msg for megablocks (#2038) · 6120e5aa
  Woosuk Kwon authored Dec 11, 2023
  
  6120e5aa
- [Minor] Fix type annotation in Mixtral (#2036) · 81ce2a4b
  Woosuk Kwon authored Dec 11, 2023
  
  81ce2a4b
- Minor fixes for Mixtral (#2015) · 4ff02039
  Woosuk Kwon authored Dec 11, 2023
  
  4ff02039
- Mixtral 8x7B support (#2011) · b5f882cc
  Pierre Stock authored Dec 11, 2023
```
Co-authored-by: Pierre Stock <p@mistral.ai>
Co-authored-by: Zhuohan Li <zhuohan123@gmail.com>
```
  b5f882cc
10 Dec, 2023 3 commits
- Replace head_mapping params with num_kv_heads to attention kernel. (#1997) · dacaf5a4
  wbn authored Dec 11, 2023
```
Co-authored-by: wangguoya <wangguoya@baidu.com>
Co-authored-by: Yang Zhao <zhaoyangstar@foxmail.com>
```
  dacaf5a4
- [Minor] Add comment on skipping rope caches (#2004) · 24cde76a
  Woosuk Kwon authored Dec 10, 2023
  
  24cde76a
- [Minor] Fix code style for baichuan (#2003) · fe470ae5
  Woosuk Kwon authored Dec 09, 2023
  
  fe470ae5
09 Dec, 2023 1 commit
- Fix for KeyError on Loading LLaMA (#1978) · 3a8c2381
  Jun Gao authored Dec 10, 2023
  
  3a8c2381
08 Dec, 2023 2 commits

Fix Baichuan2-7B-Chat (#1987) · 2b981012
firebook authored Dec 09, 2023

2b981012

Merge EmbeddedLLM/vllm-rocm into vLLM main (#1836) · 6ccc0bff

TJian authored Dec 08, 2023


Co-authored-by: Philipp Moritz <pcmoritz@gmail.com>
Co-authored-by: Amir Balwel <amoooori04@gmail.com>
Co-authored-by: root <kuanfu.liu@akirakan.com>
Co-authored-by: tjtanaa <tunjian.tan@embeddedllm.com>
Co-authored-by: kuanfu <kuanfu.liu@embeddedllm.com>
Co-authored-by: miloice <17350011+kliuae@users.noreply.github.com>

6ccc0bff

07 Dec, 2023 1 commit
- Make InternLM follow `rope_scaling` in `config.json` (#1956) · ebede26e
  Jie Li authored Dec 08, 2023
```
Co-authored-by: lijie8 <lijie8@sensetime.com>
```
  ebede26e
05 Dec, 2023 1 commit
- Support ChatGLMForConditionalGeneration (#1932) · 1d9b737e
  dancingpipi authored Dec 06, 2023
```
Co-authored-by: shujunhua1 <shujunhua1@jd.com>
```
  1d9b737e
03 Dec, 2023 1 commit
- Add PyTorch-native implementation of custom layers (#1898) · 9b294976
  Woosuk Kwon authored Dec 02, 2023
  
  9b294976
01 Dec, 2023 1 commit
- Normalize head weights for Baichuan 2 (#1876) · e5452ddf
  Woosuk Kwon authored Nov 30, 2023
  
  e5452ddf
30 Nov, 2023 3 commits
- Fix rope cache key error (#1867) · d27f4bae
  Roy authored Dec 01, 2023
  
  d27f4bae
- Avoid multiple instantiations of the RoPE class (#1828) · 63b2206a
  Jee Li authored Nov 30, 2023
  
  63b2206a
- Refactor Worker & InputMetadata (#1843) · 27feead2
  Woosuk Kwon authored Nov 29, 2023
  
  27feead2