Commits · c1376e0f825e88e32b5aca85c676fe547bcb03c9 · kecinstone / 2024pra-vllm

17 Oct, 2023 1 commit
- Change scheduler & input tensor shape (#1381) · c1376e0f
  Woosuk Kwon authored Oct 16, 2023
  
  c1376e0f
16 Oct, 2023 2 commits
- Implement prompt logprobs & Batched topk for computing logprobs (#1328) · 9d9072a0
  Zhuohan Li authored Oct 16, 2023
```
Co-authored-by: Yunmo Chen <16273544+wanmok@users.noreply.github.com>
```
  9d9072a0
- Implement PagedAttention V2 (#1348) · 928de468
  Woosuk Kwon authored Oct 16, 2023
  
  928de468
13 Oct, 2023 2 commits
- Fix the issue for AquilaChat2-* models (#1339) · de894728
  Lu Wang authored Oct 13, 2023
  
  de894728
- Bump up transformers version & Remove MistralConfig (#1254) · e7c8555d
  Woosuk Kwon authored Oct 13, 2023
  
  e7c8555d
12 Oct, 2023 1 commit
- Add blacklist in model checkpoint (#1325) · 875afe38
  Woosuk Kwon authored Oct 12, 2023
  
  875afe38
11 Oct, 2023 3 commits
- Add Mistral to quantization model list (#1278) · ee8217e5
  amaleshvemula authored Oct 11, 2023
  
  ee8217e5
- workaround of AWQ for Turing GPUs (#1252) · 82857368
  twaka authored Oct 11, 2023
  
  82857368
- change the timing of sorting logits (#1309) · 91fce82c
  yhlskt23 authored Oct 11, 2023
  
  91fce82c
10 Oct, 2023 1 commit
- [Minor] Fix comment in mistral.py (#1303) · b95ee898
  Zhuohan Li authored Oct 09, 2023
  
  b95ee898
02 Oct, 2023 3 commits
- TP/quantization/weight loading refactor part 1 - Simplify parallel linear logic (#1181) · ba0bfd40
  Zhuohan Li authored Oct 02, 2023
  
  ba0bfd40
- [Minor] Fix type annotations (#1238) · 84e4e37d
  Woosuk Kwon authored Oct 02, 2023
  
  84e4e37d
- support sharding llama2-70b on more than 8 GPUs (#1209) · a60b3530
  Zhuohan Li authored Oct 02, 2023
```
Co-authored-by: JiCheng <247153481@qq.com>
```
  a60b3530
28 Sep, 2023 3 commits
- Fix Mistral model (#1220) · a8e98aee
  Woosuk Kwon authored Sep 28, 2023
  
  a8e98aee
- [Mistral] Mistral-7B-v0.1 support (#1196) · bb1ba58f
  Chris Bamford authored Sep 28, 2023
```
Co-authored-by: timlacroix <t@mistral.ai>
```
  bb1ba58f
- Add rope_scaling to Qwen (#1210) · 7bedab57
  Qing authored Sep 28, 2023
  
  7bedab57
27 Sep, 2023 2 commits
- fix qwen-14b model (#1173) · 28e616c4
  Qing authored Sep 28, 2023
  
  28e616c4
- Support Longchat and RoPE scaling (#555) · 21877b0d
  Lily Liu authored Sep 27, 2023
```
Co-authored-by: Wing Lian <wing.lian@gmail.com>
Co-authored-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  21877b0d
26 Sep, 2023 1 commit
- Add comments on RoPE initialization (#1176) · 03ffd0a0
  Woosuk Kwon authored Sep 26, 2023
  
  03ffd0a0
24 Sep, 2023 1 commit
- [FIX] Simplify sampler logic (#1156) · f1878779
  Zhuohan Li authored Sep 23, 2023
  
  f1878779
23 Sep, 2023 1 commit
- [Sampler] Vectorized sampling (simplified) (#1048) · 947b7941
  Zhuohan Li authored Sep 22, 2023
```
Co-authored-by: Antoni Baum <antoni.baum@protonmail.com>
```
  947b7941
20 Sep, 2023 1 commit
- rope_theta and max_position_embeddings from config (#1096) · 3302f0ae
  Antoni Baum authored Sep 20, 2023
```
Co-authored-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
Co-authored-by: wnma3mz <wnma3mz@gmail.com>
```
  3302f0ae
18 Sep, 2023 3 commits
- Add minimum capability requirement for AWQ (#1064) · 2b1c116b
  Woosuk Kwon authored Sep 18, 2023
  
  2b1c116b
- Convert before transpose (#1073) · cc796b13
  Woosuk Kwon authored Sep 18, 2023
  
  cc796b13
- [FIX] Don't initialize parameter by default (#1067) · 90979c38
  Zhuohan Li authored Sep 17, 2023
  
  90979c38
16 Sep, 2023 1 commit

Implement AWQ quantization support for LLaMA (#1032) · e3e79e9e

Woosuk Kwon authored Sep 16, 2023


Co-authored-by: Robert Irvine <robert@seamlessml.com>
Co-authored-by: root <rirv938@gmail.com>
Co-authored-by: Casper <casperbh.96@gmail.com>
Co-authored-by: julian-q <julianhquevedo@gmail.com>

e3e79e9e

13 Sep, 2023 2 commits

[FIX] Minor bug fixes (#1035) · f04908ca
Zhuohan Li authored Sep 13, 2023
```
* [FIX] Minor bug fixes

* Address review comments
```
f04908ca

Add Model Revision Support (#1014) · ab019eea

Jasmond L authored Sep 14, 2023


Co-authored-by: Jasmond Loh <Jasmond.Loh@hotmail.com>
Co-authored-by: Zhuohan Li <zhuohan123@gmail.com>

ab019eea

11 Sep, 2023 1 commit
- Use FP32 in RoPE initialization (#1004) · e67b4f2c
  Woosuk Kwon authored Sep 11, 2023
```
Co-authored-by: One <imone@tuta.io>
```
  e67b4f2c
09 Sep, 2023 1 commit
- Fix wrong dtype in PagedAttentionWithALiBi bias (#996) · a62de9ec
  Antoni Baum authored Sep 09, 2023
```
---------
Signed-off-by: Antoni Baum <antoni.baum@protonmail.com>
```
  a62de9ec
08 Sep, 2023 1 commit
- faster startup of vLLM (#982) · 4b5bcf89
  Robert Irvine authored Sep 08, 2023
```
* update

---------
Co-authored-by: Robert Irvine <robert@seamlessml.com>
```
  4b5bcf89
07 Sep, 2023 2 commits
- Enable safetensors loading for all models (#974) · c957c741
  Zhuohan Li authored Sep 07, 2023
  
  c957c741
- Set torch default dtype in a context manager (#971) · 005ba458
  Antoni Baum authored Sep 06, 2023
```
Signed-off-by: Antoni Baum <antoni.baum@protonmail.com>
```
  005ba458
06 Sep, 2023 1 commit
- [BugFix] Implement RoPE for GPT-J (#941) · 320a622e
  Woosuk Kwon authored Sep 06, 2023
  
  320a622e
05 Sep, 2023 1 commit
- Align vLLM's beam search implementation with HF generate (#857) · 002800f0
  Zhuohan Li authored Sep 04, 2023
  
  002800f0
31 Aug, 2023 2 commits
- fix: bug fix when penalties are negative (#913) · e1122233
  Dong-Yong Lee authored Sep 01, 2023
```
Co-authored-by: dongyong-lee <dongyong.lee@navercorp.com>
```
  e1122233
- Improve _prune_hidden_states micro-benchmark (#707) · 28873a27
  Aman Gupta Karmani authored Aug 31, 2023
  
  28873a27
30 Aug, 2023 2 commits
- Accelerate LLaMA model loading (#234) · 0d93f156
  JFDuan authored Aug 30, 2023
  
  0d93f156
- use flash-attn via xformers (#877) · 75471386
  Aman Gupta Karmani authored Aug 30, 2023
  
  75471386
25 Aug, 2023 1 commit
- Add support for CodeLlama (#854) · 4b6f069b
  Antoni Baum authored Aug 25, 2023
  
  4b6f069b