Commits · 6f88f762bf907bccdadc52948001b21ccb616a01 · norm / vllm

28 Sep, 2023 8 commits
- Fix OOM in attention kernel test (#1223) · 6f88f762
  Woosuk Kwon authored Sep 28, 2023
  
  6f88f762
- Add Mistral to supported model list (#1221) · 202351d5
  Woosuk Kwon authored Sep 28, 2023
  
  202351d5
- [Fix] Remove false assertion (#1222) · 2e8e49fc
  Woosuk Kwon authored Sep 28, 2023
  
  2e8e49fc
- Fix Mistral model (#1220) · a8e98aee
  Woosuk Kwon authored Sep 28, 2023
  
  a8e98aee
- [Mistral] Mistral-7B-v0.1 support (#1196) · bb1ba58f
  Chris Bamford authored Sep 28, 2023
```
Co-authored-by: timlacroix <t@mistral.ai>
```
  bb1ba58f
- Add rope_scaling to Qwen (#1210) · 7bedab57
  Qing authored Sep 28, 2023
  
  7bedab57
- Add `skip_special_tokens` sampling params (#1186) · 20f7cc4c
  Dan Lord authored Sep 27, 2023
  
  20f7cc4c
- Use standard extras for uvicorn (#1166) · 649aa730
  Danilo Peixoto authored Sep 27, 2023
  
  649aa730
27 Sep, 2023 5 commits
- Automatically configure `max_num_batched_tokens` (#1198) · a19bc5c6
  Woosuk Kwon authored Sep 27, 2023
  
  a19bc5c6
- fix qwen-14b model (#1173) · 28e616c4
  Qing authored Sep 28, 2023
  
  28e616c4
- fix typo (#1184) · 30e77528
  Wang Ran (汪然) authored Sep 28, 2023
```
Co-authored-by: Zhuohan Li <zhuohan123@gmail.com>
```
  30e77528
- Support Longchat and RoPE scaling (#555) · 21877b0d
  Lily Liu authored Sep 27, 2023
```
Co-authored-by: Wing Lian <wing.lian@gmail.com>
Co-authored-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  21877b0d
- Allocate more shared memory to attention kernel (#1154) · cf5cb1e3
  Antoni Baum authored Sep 26, 2023
  
  cf5cb1e3
26 Sep, 2023 2 commits
- Add comments on RoPE initialization (#1176) · 03ffd0a0
  Woosuk Kwon authored Sep 26, 2023
  
  03ffd0a0
- [Setup] Enable `TORCH_CUDA_ARCH_LIST` for selecting target GPUs (#1074) · a425bd9a
  Woosuk Kwon authored Sep 26, 2023
  
  a425bd9a
24 Sep, 2023 3 commits
- Align `max_tokens` behavior with openai (#852) · bbbf8656
  Wen Sun authored Sep 24, 2023
  
  bbbf8656
- Fix config for Falcon (#1164) · 9f6be869
  Woosuk Kwon authored Sep 23, 2023
  
  9f6be869
- [FIX] Simplify sampler logic (#1156) · f1878779
  Zhuohan Li authored Sep 23, 2023
  
  f1878779
23 Sep, 2023 1 commit
- [Sampler] Vectorized sampling (simplified) (#1048) · 947b7941
  Zhuohan Li authored Sep 22, 2023
```
Co-authored-by: Antoni Baum <antoni.baum@protonmail.com>
```
  947b7941
22 Sep, 2023 3 commits
- Announce the First vLLM Meetup (#1148) · 8d926e91
  Woosuk Kwon authored Sep 22, 2023
  
  8d926e91
- Docs: Fix broken link to openai example (#1145) · 4ee52bb1
  Nick Perez authored Sep 22, 2023
```
Link to `openai_client.py` is no longer valid - updated to `openai_completion_client.py`
```
  4ee52bb1
- Use `--ipc=host` in docker run for distributed inference (#1125) · 7d7e3b78
  Woosuk Kwon authored Sep 21, 2023
  
  7d7e3b78
21 Sep, 2023 4 commits
- feat: support stop_token_ids parameter. (#1097) · f98b745a
  Ricardo Lu authored Sep 22, 2023
  
  f98b745a
- clean api code, remove redundant background task. (#1102) · 2d1e86f1
  Roy authored Sep 22, 2023
  
  2d1e86f1
- Add float16 and float32 (#1115) · 1ac4ccf7
  Woosuk Kwon authored Sep 21, 2023
  
  1ac4ccf7
- Replace DtypeTensor (#1123) · 2ac4d5e2
  Woosuk Kwon authored Sep 21, 2023
  
  2ac4d5e2
20 Sep, 2023 3 commits
- rope_theta and max_position_embeddings from config (#1096) · 3302f0ae
  Antoni Baum authored Sep 20, 2023
```
Co-authored-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
Co-authored-by: wnma3mz <wnma3mz@gmail.com>
```
  3302f0ae
- Add documentation to Triton server tutorial (#983) · 6f2dd6c3
  Tanmay Verma authored Sep 20, 2023
  
  6f2dd6c3
- Add gpu_memory_utilization and swap_space to LLM (#1090) · bc064457
  Woosuk Kwon authored Sep 19, 2023
  
  bc064457
19 Sep, 2023 1 commit
- Add pyarrow to dependencies & Print warning on Ray import error (#1094) · 400b8289
  Woosuk Kwon authored Sep 18, 2023
  
  400b8289
18 Sep, 2023 7 commits
- [Community] Add vLLM Discord server (#1086) · c1026311
  Zhuohan Li authored Sep 18, 2023
  
  c1026311
- Add minimum capability requirement for AWQ (#1064) · 2b1c116b
  Woosuk Kwon authored Sep 18, 2023
  
  2b1c116b
- Convert before transpose (#1073) · cc796b13
  Woosuk Kwon authored Sep 18, 2023
  
  cc796b13
- Fix get_max_num_running_seqs for waiting and swapped seq groups (#1068) · f029ef94
  Zhuohan Li authored Sep 18, 2023
  
  f029ef94
- align llm_engine and async_engine. (#1081) · 95592fa0
  Roy authored Sep 19, 2023
  
  95592fa0
- added support for quantize on LLM module (#1080) · fbe66e1d
  orellavie1212 authored Sep 18, 2023
  
  fbe66e1d
- [FIX] Don't initialize parameter by default (#1067) · 90979c38
  Zhuohan Li authored Sep 17, 2023
  
  90979c38
17 Sep, 2023 2 commits
- Fix hanging when prompt exceeds limit (#1029) · e21d7687
  陈序 authored Sep 17, 2023
  
  e21d7687
- Remove AsyncLLMEngine busy loop, shield background task (#1059) · ff36139f
  Antoni Baum authored Sep 17, 2023
  
  ff36139f
16 Sep, 2023 1 commit

Implement AWQ quantization support for LLaMA (#1032) · e3e79e9e

Woosuk Kwon authored Sep 16, 2023


Co-authored-by: Robert Irvine <robert@seamlessml.com>
Co-authored-by: root <rirv938@gmail.com>
Co-authored-by: Casper <casperbh.96@gmail.com>
Co-authored-by: julian-q <julianhquevedo@gmail.com>

e3e79e9e