Commits · 33e5d7e6b6d672eb7ecef038bbffc8b366f31220 · OpenDAS / vllm_cscc

13 Aug, 2024 4 commits
- [frontend] spawn engine process from api server process (#7484) · 33e5d7e6
  youkaichao authored Aug 13, 2024
  
  33e5d7e6
- [Frontend][Core] Add plumbing to support audio language models (#7446) · 00c3d68e
  Peter Salas authored Aug 13, 2024
  
  00c3d68e
- [Bugfix] Fix weight loading for Chameleon when TP>1 (#7410) · 7025b11d
  Cyrus Leung authored Aug 13, 2024
  
  7025b11d
- [Misc] improve logits processors logging message (#7435) · 97a6be95
  Andrew Wang authored Aug 12, 2024
  
  97a6be95
09 Aug, 2024 2 commits
- [Frontend] Support embeddings in the run_batch API (#7132) · 249b8822
  Pooya Davoodi authored Aug 09, 2024
```
Co-authored-by: Simon Mo <simon.mo@hey.com>
```
  249b8822
- [Core] Support serving encoder/decoder models (#7258) · 7eb4a51c
  Cyrus Leung authored Aug 09, 2024
  
  7eb4a51c
08 Aug, 2024 1 commit
- [Frontend] Kill the server on engine death (#6594) · 21b9c49a
  Joe Runde authored Aug 08, 2024
```
Signed-off-by: Joe Runde <joe@joerun.de>
Signed-off-by: Joe Runde <Joseph.Runde@ibm.com>
```
  21b9c49a
07 Aug, 2024 2 commits
- [BugFix] Fix frontend multiprocessing hang (#7217) · fde47d3b
  Maximilien de Bayser authored Aug 07, 2024
```
Signed-off-by: Max de Bayser <mbayser@br.ibm.com>
Co-authored-by: Robert Shaw <114415538+robertgshaw2-neuralmagic@users.noreply.github.com>
```
  fde47d3b
- [Frontend] Gracefully handle missing chat template and fix CI failure (#7238) · 66d617e3
  Cyrus Leung authored Aug 07, 2024
```
Co-authored-by: Roger Wang <ywang@roblox.com>
```
  66d617e3
05 Aug, 2024 1 commit
- [ci][frontend] deduplicate tests (#7101) · dfb1a15d
  youkaichao authored Aug 05, 2024
  
  dfb1a15d
04 Aug, 2024 1 commit
- Support for guided decoding for offline LLM (#6878) · 654bc5ca
  Yihuan Bu authored Aug 03, 2024
```
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
```
  654bc5ca
03 Aug, 2024 1 commit

[ Frontend ] Multiprocessing for OpenAI Server with `zeromq` (#6883) · ed812a73

Robert Shaw authored Aug 02, 2024


Signed-off-by: Joe Runde <Joseph.Runde@ibm.com>
Co-authored-by: Joe Runde <Joseph.Runde@ibm.com>
Co-authored-by: Joe Runde <joe@joerun.de>
Co-authored-by: Nick Hill <nickhill@us.ibm.com>
Co-authored-by: Simon Mo <simon.mo@hey.com>

ed812a73

02 Aug, 2024 1 commit
- [ci] set timeout for test_oot_registration.py (#7082) · 80694951
  youkaichao authored Aug 02, 2024
  
  80694951
01 Aug, 2024 1 commit
- [Bugfix] Set SamplingParams.max_tokens for OpenAI requests if not provided by user (#6954) · 3c10591e
  zifeitong authored Jul 31, 2024
  
  3c10591e
29 Jul, 2024 1 commit
- [Frontend] New `allowed_token_ids` decoding request parameter (#6753) · 9f69d824
  Nick Hill authored Jul 29, 2024
  
  9f69d824
25 Jul, 2024 2 commits
- [Bugfix] Fix encoding_format in examples/openai_embedding_client.py (#6755) · 316a41ac
  Chang Su authored Jul 24, 2024
  
  316a41ac
- [Frontend] Represent tokens with identifiable strings (#6626) · 5689e256
  Evan Z. Liu authored Jul 24, 2024
  
  5689e256
23 Jul, 2024 2 commits
- [Frontend] Add Usage data in each chunk for chat_serving. #6540 (#6652) · 58f53034
  Yehoshua Cohen authored Jul 23, 2024
  
  58f53034
- [Misc] Manage HTTP connections in one place (#6600) · 97234be0
  Cyrus Leung authored Jul 23, 2024
  
  97234be0
22 Jul, 2024 1 commit
- [Frontend] Refactor prompt processing (#4028) · 739b61a3
  Cyrus Leung authored Jul 23, 2024
```
Co-authored-by: Roger Wang <ywang@roblox.com>
```
  739b61a3
19 Jul, 2024 1 commit
- [Bugfix][Frontend] Fix missing `/metrics` endpoint (#6463) · 6366efc6
  Cyrus Leung authored Jul 19, 2024
  
  6366efc6
18 Jul, 2024 1 commit
- [BugFix][Frontend] Use LoRA tokenizer in OpenAI APIs (#6227) · e2fbaee7
  Nick Hill authored Jul 18, 2024
```
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
```
  e2fbaee7
17 Jul, 2024 1 commit
- [Doc][CI/Build] Update docs and tests to use `vllm serve` (#6431) · 5bf35a91
  Cyrus Leung authored Jul 17, 2024
  
  5bf35a91
16 Jul, 2024 2 commits
- [Frontend] Support for chat completions input in the tokenize endpoint (#5923) · 7a3d2a5b
  sasha0552 authored Jul 16, 2024
  
  7a3d2a5b
- [Bugfix][CI/Build] Test prompt adapters in openai entrypoint tests (#6419) · d92b3c5c
  Joe authored Jul 15, 2024
  
  d92b3c5c
15 Jul, 2024 1 commit
- [BugFix] BatchResponseData body should be optional (#6345) · b47008b4
  zifeitong authored Jul 14, 2024
```
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
```
  b47008b4
13 Jul, 2024 1 commit

[ci] try to add multi-node tests (#6280) · 41708e50

youkaichao authored Jul 12, 2024


Signed-off-by: Muralidhar Andoorveedu <muralidhar.andoorveedu@centml.ai>
Co-authored-by: Muralidhar Andoorveedu <muralidhar.andoorveedu@centml.ai>

41708e50

12 Jul, 2024 1 commit
- [Misc] add fixture to guided processor tests (#6341) · b039cbbc
  Yihuan Bu authored Jul 12, 2024
  
  b039cbbc
05 Jul, 2024 1 commit
- [Frontend] Continuous usage stats in OpenAI completion API (#5742) · f1e15da6
  jvlunteren authored Jul 05, 2024
  
  f1e15da6
03 Jul, 2024 2 commits
- [vlm] Remove vision language config. (#6089) · d9e98f42
  xwjiang2010 authored Jul 03, 2024
```
Signed-off-by: Xiaowei Jiang <xwjiang2010@gmail.com>
Co-authored-by: Roger Wang <ywang@roblox.com>
```
  d9e98f42
- [CI] Fix base url doesn't strip "/" (#6087) · d18bab35
  SangBin Cho authored Jul 03, 2024
  
  d18bab35
02 Jul, 2024 2 commits

[Core] Pipeline Parallel Support (#4412) · c5832d2a
Murali Andoorveedu authored Jul 02, 2024
```
Signed-off-by: Muralidhar Andoorveedu <muralidhar.andoorveedu@centml.ai>
```
c5832d2a

[VLM] Remove `image_input_type` from VLM config (#5852) · 98d6682c

xwjiang2010 authored Jul 02, 2024


Signed-off-by: Xiaowei Jiang <xwjiang2010@gmail.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
Co-authored-by: Roger Wang <ywang@roblox.com>

98d6682c

30 Jun, 2024 2 commits
- [Frontend]: Support base64 embedding (#5935) · c6c240aa
  llmpros authored Jun 30, 2024
```
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
```
  c6c240aa
- [CI/Build] [3/3] Reorganize entrypoints tests (#5966) · 9d47f64e
  Cyrus Leung authored Jun 30, 2024
  
  9d47f64e
28 May, 2024 1 commit
- [Core] Consolidate prompt arguments to LLM engines (#4328) · 5ae5ed1e
  Cyrus Leung authored May 29, 2024
```
Co-authored-by: Roger Wang <ywang@roblox.com>
```
  5ae5ed1e
11 May, 2024 1 commit
- [Model][Misc] Add e5-mistral-7b-instruct and Embedding API (#3734) · e254497b
  Chang Su authored May 11, 2024
  
  e254497b
09 May, 2024 1 commit
- [Frontend] Move async logic outside of constructor (#4674) · f12b20de
  Cyrus Leung authored May 09, 2024
  
  f12b20de
03 May, 2024 1 commit
- Fix/async chat serving (#2727) · f8e7adda
  Sebastian Schoennenbeck authored May 03, 2024
  
  f8e7adda