Commits · 33e5d7e6b6d672eb7ecef038bbffc8b366f31220 · OpenDAS / vllm_cscc

13 Aug, 2024 3 commits
- [frontend] spawn engine process from api server process (#7484) · 33e5d7e6
  youkaichao authored Aug 13, 2024
  
  33e5d7e6
- [Misc] improve logits processors logging message (#7435) · 97a6be95
  Andrew Wang authored Aug 12, 2024
  
  97a6be95
- [Core] Shut down aDAG workers with clean async llm engine exit (#7224) · 198d6a28
  Rui Qiao authored Aug 12, 2024
```
Signed-off-by: Rui Qiao <ruisearch42@gmail.com>
```
  198d6a28
09 Aug, 2024 2 commits
- [Frontend] Support embeddings in the run_batch API (#7132) · 249b8822
  Pooya Davoodi authored Aug 09, 2024
```
Co-authored-by: Simon Mo <simon.mo@hey.com>
```
  249b8822
- [Core] Support serving encoder/decoder models (#7258) · 7eb4a51c
  Cyrus Leung authored Aug 09, 2024
  
  7eb4a51c
08 Aug, 2024 1 commit
- [Frontend] Kill the server on engine death (#6594) · 21b9c49a
  Joe Runde authored Aug 08, 2024
```
Signed-off-by: Joe Runde <joe@joerun.de>
Signed-off-by: Joe Runde <Joseph.Runde@ibm.com>
```
  21b9c49a
07 Aug, 2024 4 commits
- [BugFix] Fix frontend multiprocessing hang (#7217) · fde47d3b
  Maximilien de Bayser authored Aug 07, 2024
```
Signed-off-by: Max de Bayser <mbayser@br.ibm.com>
Co-authored-by: Robert Shaw <114415538+robertgshaw2-neuralmagic@users.noreply.github.com>
```
  fde47d3b
- [ BugFix ] Move `zmq` frontend to IPC instead of TCP (#7222) · 56498572
  Robert Shaw authored Aug 07, 2024
  
  56498572
- [Frontend] Gracefully handle missing chat template and fix CI failure (#7238) · 66d617e3
  Cyrus Leung authored Aug 07, 2024
```
Co-authored-by: Roger Wang <ywang@roblox.com>
```
  66d617e3
- [BugFix] Overhaul async request cancellation (#7111) · 9a3f49ae
  Nick Hill authored Aug 06, 2024
  
  9a3f49ae
05 Aug, 2024 3 commits
- [BugFix] Use args.trust_remote_code (#7121) · 57f560aa
  Aditya Paliwal authored Aug 05, 2024
  
  57f560aa
- [BugFix] Use IP4 localhost form for zmq bind (#7163) · 003f8ee1
  Nick Hill authored Aug 05, 2024
  
  003f8ee1
- [Frontend] Reapply "Factor out code for running uvicorn" (#7095) · cc08fc72
  Cyrus Leung authored Aug 05, 2024
  
  cc08fc72
04 Aug, 2024 1 commit
- Support for guided decoding for offline LLM (#6878) · 654bc5ca
  Yihuan Bu authored Aug 03, 2024
```
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
```
  654bc5ca
03 Aug, 2024 2 commits

[Frontend] Factor out chat message parsing (#7055) · 8c025fa7
Cyrus Leung authored Aug 03, 2024

8c025fa7

[ Frontend ] Multiprocessing for OpenAI Server with `zeromq` (#6883) · ed812a73

Robert Shaw authored Aug 02, 2024


Signed-off-by: Joe Runde <Joseph.Runde@ibm.com>
Co-authored-by: Joe Runde <Joseph.Runde@ibm.com>
Co-authored-by: Joe Runde <joe@joerun.de>
Co-authored-by: Nick Hill <nickhill@us.ibm.com>
Co-authored-by: Simon Mo <simon.mo@hey.com>

ed812a73

01 Aug, 2024 1 commit
- [Bugfix] Set SamplingParams.max_tokens for OpenAI requests if not provided by user (#6954) · 3c10591e
  zifeitong authored Jul 31, 2024
  
  3c10591e
31 Jul, 2024 3 commits
- Revert "[Frontend] Factor out code for running uvicorn" (#7012) · 7eb0cb4a
  Simon Mo authored Jul 31, 2024
```
Co-authored-by: Robert Shaw <114415538+robertgshaw2-neuralmagic@users.noreply.github.com>
```
  7eb0cb4a
- [Bugfix] fix logit processor excceed vocab size issue (#6927) · c0644cf9
  Fei authored Jul 31, 2024
  
  c0644cf9
- [mypy] Enable following imports for some directories (#6681) · da1f7cc1
  Cyrus Leung authored Jul 31, 2024
  
  da1f7cc1
29 Jul, 2024 1 commit
- [Frontend] New `allowed_token_ids` decoding request parameter (#6753) · 9f69d824
  Nick Hill authored Jul 29, 2024
  
  9f69d824
27 Jul, 2024 1 commit
- [Frontend] Factor out code for running uvicorn (#6828) · 981b0d56
  Cyrus Leung authored Jul 27, 2024
  
  981b0d56
25 Jul, 2024 1 commit
- [Frontend] Represent tokens with identifiable strings (#6626) · 5689e256
  Evan Z. Liu authored Jul 24, 2024
  
  5689e256
24 Jul, 2024 1 commit
- [Frontend] split run_server into build_server and run_server (#6740) · ee812580
  Daniele authored Jul 24, 2024
  
  ee812580
23 Jul, 2024 1 commit
- [Frontend] Add Usage data in each chunk for chat_serving. #6540 (#6652) · 58f53034
  Yehoshua Cohen authored Jul 23, 2024
  
  58f53034
22 Jul, 2024 2 commits
- [Core] Support dynamically loading Lora adapter from HuggingFace (#6234) · 42c7f66a
  Jiaxin Shan authored Jul 22, 2024
```
Co-authored-by: Antoni Baum <antoni.baum@protonmail.com>
```
  42c7f66a
- [Frontend] Refactor prompt processing (#4028) · 739b61a3
  Cyrus Leung authored Jul 23, 2024
```
Co-authored-by: Roger Wang <ywang@roblox.com>
```
  739b61a3
21 Jul, 2024 1 commit
- [Frontend] Move chat utils (#6602) · d7f4178d
  Cyrus Leung authored Jul 21, 2024
```
Co-authored-by: Roger Wang <ywang@roblox.com>
```
  d7f4178d
19 Jul, 2024 2 commits
- [Bugfix][Frontend] remove duplicate init logger (#6581) · 51f8aa90
  Daniele authored Jul 19, 2024
  
  51f8aa90
- [Bugfix][Frontend] Fix missing `/metrics` endpoint (#6463) · 6366efc6
  Cyrus Leung authored Jul 19, 2024
  
  6366efc6
18 Jul, 2024 1 commit
- [BugFix][Frontend] Use LoRA tokenizer in OpenAI APIs (#6227) · e2fbaee7
  Nick Hill authored Jul 18, 2024
```
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
```
  e2fbaee7
16 Jul, 2024 2 commits
- [Frontend] Support for chat completions input in the tokenize endpoint (#5923) · 7a3d2a5b
  sasha0552 authored Jul 16, 2024
  
  7a3d2a5b
- [Bugfix][CI/Build] Test prompt adapters in openai entrypoint tests (#6419) · d92b3c5c
  Joe authored Jul 15, 2024
  
  d92b3c5c
15 Jul, 2024 1 commit
- [BugFix] BatchResponseData body should be optional (#6345) · b47008b4
  zifeitong authored Jul 14, 2024
```
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
```
  b47008b4
14 Jul, 2024 1 commit
- [Feature] vLLM CLI (#5090) · dbfe254e
  Ethan Xu authored Jul 14, 2024
```
Co-authored-by: simon-mo <simon.mo@hey.com>
```
  dbfe254e
09 Jul, 2024 1 commit

[CORE] Adding support for insertion of soft-tuned prompts (#4645) · 4d6ada94

Swapnil Parekh authored Jul 09, 2024


Co-authored-by: Swapnil Parekh <swapnilp@ibm.com>
Co-authored-by: Joe G <joseph.granados@h2o.ai>
Co-authored-by: Antoni Baum <antoni.baum@protonmail.com>

4d6ada94

08 Jul, 2024 1 commit
- do not exclude `object` field in CompletionStreamResponse (#6196) · 16620f43
  kczimm authored Jul 07, 2024
  
  16620f43
07 Jul, 2024 1 commit
- [misc][frontend] log all available endpoints (#6195) · 3b08fe2b
  youkaichao authored Jul 07, 2024
```
Co-authored-by: Cody Yu <hao.yu.cody@gmail.com>
```
  3b08fe2b
05 Jul, 2024 1 commit
- [Frontend] Continuous usage stats in OpenAI completion API (#5742) · f1e15da6
  jvlunteren authored Jul 05, 2024
  
  f1e15da6
03 Jul, 2024 1 commit

[vlm] Remove vision language config. (#6089) · d9e98f42

xwjiang2010 authored Jul 03, 2024


Signed-off-by: Xiaowei Jiang <xwjiang2010@gmail.com>
Co-authored-by: Roger Wang <ywang@roblox.com>

d9e98f42