Commits · e3e79e9e8a2224e03a711c3d1ef7a35daa447083 · kecinstone / 2024pra-vllm

16 Sep, 2023 1 commit

Implement AWQ quantization support for LLaMA (#1032) · e3e79e9e

Woosuk Kwon authored Sep 16, 2023


Co-authored-by: Robert Irvine <robert@seamlessml.com>
Co-authored-by: root <rirv938@gmail.com>
Co-authored-by: Casper <casperbh.96@gmail.com>
Co-authored-by: julian-q <julianhquevedo@gmail.com>

e3e79e9e

13 Sep, 2023 2 commits
- Add Model Revision Support (#1014) · ab019eea
  Jasmond L authored Sep 14, 2023
```
Co-authored-by: Jasmond Loh <Jasmond.Loh@hotmail.com>
Co-authored-by: Zhuohan Li <zhuohan123@gmail.com>
```
  ab019eea
- Use TGI-like incremental detokenization (#984) · 9841d48a
  Antoni Baum authored Sep 13, 2023
  
  9841d48a
09 Sep, 2023 1 commit
- fix "tansformers_module" ModuleNotFoundError when load model with `trust_remote_code=True` (#871) · 4042d192
  Jingru authored Sep 09, 2023
  
  4042d192
07 Sep, 2023 1 commit
- Enable safetensors loading for all models (#974) · c957c741
  Zhuohan Li authored Sep 07, 2023
  
  c957c741
05 Sep, 2023 1 commit
- Align vLLM's beam search implementation with HF generate (#857) · 002800f0
  Zhuohan Li authored Sep 04, 2023
  
  002800f0
04 Sep, 2023 1 commit
- Refactor AsyncLLMEngine (#880) · ce741ba3
  Antoni Baum authored Sep 03, 2023
  
  ce741ba3
02 Aug, 2023 1 commit
- Refactor scheduler (#658) · 55fe8a81
  Woosuk Kwon authored Aug 02, 2023
  
  55fe8a81
31 Jul, 2023 1 commit
- fix doc (#622) · aa39e42c
  Chaofan Lin authored Aug 01, 2023
  
  aa39e42c
30 Jul, 2023 1 commit
- fix ModuleNotFoundError (#599) · 953f28cf
  Fang li authored Jul 30, 2023
```
Co-authored-by: fangli <fangli@tencent.com>
```
  953f28cf
20 Jul, 2023 1 commit
- Ray placement group support (#397) · 9925c179
  Antoni Baum authored Jul 19, 2023
  
  9925c179
18 Jul, 2023 1 commit
- fix max seq len (#489) · b4b195b3
  Lily Liu authored Jul 17, 2023
  
  b4b195b3
17 Jul, 2023 1 commit
- [Fix] Fix the condition of max_seq_len (#477) · 2bdea7ac
  Zhuohan Li authored Jul 17, 2023
  
  2bdea7ac
12 Jul, 2023 1 commit
- Fix handling of special tokens in decoding. (#418) · c6dfc3cd
  xcnick authored Jul 12, 2023
  
  c6dfc3cd
07 Jul, 2023 1 commit
- Add trust-remote-code flag to handle remote tokenizers (#364) · a945fcc2
  codethazine authored Jul 07, 2023
  
  a945fcc2
03 Jul, 2023 2 commits
- [Quality] Add CI for formatting (#343) · 42e0c1df
  Zhuohan Li authored Jul 03, 2023
  
  42e0c1df
- [Quality] Add code formatter and linter (#326) · d6fa1be3
  Zhuohan Li authored Jul 03, 2023
  
  d6fa1be3
01 Jul, 2023 1 commit
- Raise error for long prompt (#273) · dafd924c
  Lily Liu authored Jun 30, 2023
  
  dafd924c
28 Jun, 2023 2 commits
- [Tokenizer] Add tokenizer mode (#298) · 998d9d15
  Woosuk Kwon authored Jun 28, 2023
  
  998d9d15
- [Tokenizer] Add an option to specify tokenizer (#284) · 4338cc47
  Woosuk Kwon authored Jun 28, 2023
  
  4338cc47
26 Jun, 2023 1 commit
- [Bug] Fix the OOM condition for CPU cache (#260) · 0b7db411
  Zhuohan Li authored Jun 26, 2023
  
  0b7db411
22 Jun, 2023 1 commit
- [Fix] Better error message when there is OOM during cache initialization (#203) · 1d24ccb9
  Zhuohan Li authored Jun 22, 2023
  
  1d24ccb9
21 Jun, 2023 1 commit
- fix-ray (#193) · 2e0d3143
  Zhuohan Li authored Jun 22, 2023
  
  2e0d3143
17 Jun, 2023 2 commits
- Change the name to vLLM (#150) · 0b98ba15
  Woosuk Kwon authored Jun 17, 2023
  
  0b98ba15
- Rename servers to engines (#152) · e5464ee4
  Zhuohan Li authored Jun 17, 2023
  
  e5464ee4
16 Jun, 2023 1 commit
- Rename servers and change port numbers to reduce confusion (#149) · eedb46bf
  Zhuohan Li authored Jun 17, 2023
  
  eedb46bf
07 Jun, 2023 1 commit
- Add docstrings for LLMServer and related classes and examples (#142) · 42983742
  Zhuohan Li authored Jun 07, 2023
  
  42983742
05 Jun, 2023 1 commit
- Fix various issues of async servers (#135) · 1a956e13
  Zhuohan Li authored Jun 05, 2023
  
  1a956e13
04 Jun, 2023 1 commit
- Add docstrings for LLM (#137) · 8274ca23
  Woosuk Kwon authored Jun 04, 2023
  
  8274ca23
28 May, 2023 1 commit
- Add throughput benchmarking script (#133) · 211318d4
  Woosuk Kwon authored May 28, 2023
  
  211318d4
24 May, 2023 3 commits
- OpenAI Compatible Frontend (#116) · 057daef7
  Zhuohan Li authored May 23, 2023
  
  057daef7
- Incrementally decode output tokens (#121) · e8671783
  Woosuk Kwon authored May 23, 2023
  
  e8671783
- Print warnings/errors for large swap space (#123) · aedba6d5
  Woosuk Kwon authored May 23, 2023
  
  aedba6d5
22 May, 2023 1 commit
- Introduce LLM class for offline inference (#115) · 655a5e48
  Woosuk Kwon authored May 21, 2023
  
  655a5e48
21 May, 2023 1 commit
- Implement stop strings and best_of (#114) · f746ced0
  Woosuk Kwon authored May 21, 2023
  
  f746ced0
20 May, 2023 1 commit
- Refactor system architecture (#109) · c3442c1f
  Woosuk Kwon authored May 20, 2023
  
  c3442c1f