Commits · 33e5d7e6b6d672eb7ecef038bbffc8b366f31220 · OpenDAS / vllm_cscc

13 Aug, 2024 4 commits
- [frontend] spawn engine process from api server process (#7484) · 33e5d7e6
  youkaichao authored Aug 13, 2024
  
  33e5d7e6
- [Frontend][Core] Add plumbing to support audio language models (#7446) · 00c3d68e
  Peter Salas authored Aug 13, 2024
  
  00c3d68e
- [Misc] improve logits processors logging message (#7435) · 97a6be95
  Andrew Wang authored Aug 12, 2024
  
  97a6be95
- [Core] Shut down aDAG workers with clean async llm engine exit (#7224) · 198d6a28
  Rui Qiao authored Aug 12, 2024
```
Signed-off-by: Rui Qiao <ruisearch42@gmail.com>
```
  198d6a28
09 Aug, 2024 2 commits
- [Frontend] Support embeddings in the run_batch API (#7132) · 249b8822
  Pooya Davoodi authored Aug 09, 2024
```
Co-authored-by: Simon Mo <simon.mo@hey.com>
```
  249b8822
- [Core] Support serving encoder/decoder models (#7258) · 7eb4a51c
  Cyrus Leung authored Aug 09, 2024
  
  7eb4a51c
08 Aug, 2024 1 commit
- [Frontend] Kill the server on engine death (#6594) · 21b9c49a
  Joe Runde authored Aug 08, 2024
```
Signed-off-by: Joe Runde <joe@joerun.de>
Signed-off-by: Joe Runde <Joseph.Runde@ibm.com>
```
  21b9c49a
07 Aug, 2024 4 commits
- [BugFix] Fix frontend multiprocessing hang (#7217) · fde47d3b
  Maximilien de Bayser authored Aug 07, 2024
```
Signed-off-by: Max de Bayser <mbayser@br.ibm.com>
Co-authored-by: Robert Shaw <114415538+robertgshaw2-neuralmagic@users.noreply.github.com>
```
  fde47d3b
- [ BugFix ] Move `zmq` frontend to IPC instead of TCP (#7222) · 56498572
  Robert Shaw authored Aug 07, 2024
  
  56498572
- [Frontend] Gracefully handle missing chat template and fix CI failure (#7238) · 66d617e3
  Cyrus Leung authored Aug 07, 2024
```
Co-authored-by: Roger Wang <ywang@roblox.com>
```
  66d617e3
- [BugFix] Overhaul async request cancellation (#7111) · 9a3f49ae
  Nick Hill authored Aug 06, 2024
  
  9a3f49ae
06 Aug, 2024 1 commit

[Core] Subclass ModelRunner to support cross-attention & encoder sequences... · fd95e026

afeldman-nm authored Aug 06, 2024


[Core] Subclass ModelRunner to support cross-attention & encoder sequences (towards eventual encoder/decoder model support) (#4942)
Co-authored-by: Andrew Feldman <afeld2012@gmail.com>
Co-authored-by: Nick Hill <nickhill@us.ibm.com>

fd95e026

05 Aug, 2024 3 commits
- [BugFix] Use args.trust_remote_code (#7121) · 57f560aa
  Aditya Paliwal authored Aug 05, 2024
  
  57f560aa
- [BugFix] Use IP4 localhost form for zmq bind (#7163) · 003f8ee1
  Nick Hill authored Aug 05, 2024
  
  003f8ee1
- [Frontend] Reapply "Factor out code for running uvicorn" (#7095) · cc08fc72
  Cyrus Leung authored Aug 05, 2024
  
  cc08fc72
04 Aug, 2024 1 commit
- Support for guided decoding for offline LLM (#6878) · 654bc5ca
  Yihuan Bu authored Aug 03, 2024
```
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
```
  654bc5ca
03 Aug, 2024 2 commits

[Frontend] Factor out chat message parsing (#7055) · 8c025fa7
Cyrus Leung authored Aug 03, 2024

8c025fa7

[ Frontend ] Multiprocessing for OpenAI Server with `zeromq` (#6883) · ed812a73

Robert Shaw authored Aug 02, 2024


Signed-off-by: Joe Runde <Joseph.Runde@ibm.com>
Co-authored-by: Joe Runde <Joseph.Runde@ibm.com>
Co-authored-by: Joe Runde <joe@joerun.de>
Co-authored-by: Nick Hill <nickhill@us.ibm.com>
Co-authored-by: Simon Mo <simon.mo@hey.com>

ed812a73

01 Aug, 2024 1 commit
- [Bugfix] Set SamplingParams.max_tokens for OpenAI requests if not provided by user (#6954) · 3c10591e
  zifeitong authored Jul 31, 2024
  
  3c10591e
31 Jul, 2024 3 commits
- Revert "[Frontend] Factor out code for running uvicorn" (#7012) · 7eb0cb4a
  Simon Mo authored Jul 31, 2024
```
Co-authored-by: Robert Shaw <114415538+robertgshaw2-neuralmagic@users.noreply.github.com>
```
  7eb0cb4a
- [Bugfix] fix logit processor excceed vocab size issue (#6927) · c0644cf9
  Fei authored Jul 31, 2024
  
  c0644cf9
- [mypy] Enable following imports for some directories (#6681) · da1f7cc1
  Cyrus Leung authored Jul 31, 2024
  
  da1f7cc1
29 Jul, 2024 2 commits
- [Frontend] New `allowed_token_ids` decoding request parameter (#6753) · 9f69d824
  Nick Hill authored Jul 29, 2024
  
  9f69d824
- [Model] Initialize support for InternVL2 series models (#6514) · 7cbd9ec7
  Isotr0py authored Jul 29, 2024
```
Co-authored-by: Roger Wang <ywang@roblox.com>
```
  7cbd9ec7
27 Jul, 2024 1 commit
- [Frontend] Factor out code for running uvicorn (#6828) · 981b0d56
  Cyrus Leung authored Jul 27, 2024
  
  981b0d56
25 Jul, 2024 2 commits
- [Bugfix] Add image placeholder for OpenAI Compatible Server of MiniCPM-V (#6787) · b75e314f
  Alphi authored Jul 26, 2024
```
Co-authored-by: hezhihui <hzh7269@modelbest.cn>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
```
  b75e314f
- [Frontend] Represent tokens with identifiable strings (#6626) · 5689e256
  Evan Z. Liu authored Jul 24, 2024
  
  5689e256
24 Jul, 2024 2 commits
- [Frontend] split run_server into build_server and run_server (#6740) · ee812580
  Daniele authored Jul 24, 2024
  
  ee812580
- Adding f-string to validation error which is missing (#6748) · 54514634
  LF Marques authored Jul 24, 2024
  
  54514634
23 Jul, 2024 2 commits
- [Frontend] Add Usage data in each chunk for chat_serving. #6540 (#6652) · 58f53034
  Yehoshua Cohen authored Jul 23, 2024
  
  58f53034
- [VLM][Model] Support image input for Chameleon (#6633) · 22fa2e35
  Roger Wang authored Jul 22, 2024
  
  22fa2e35
22 Jul, 2024 2 commits
- [Core] Support dynamically loading Lora adapter from HuggingFace (#6234) · 42c7f66a
  Jiaxin Shan authored Jul 22, 2024
```
Co-authored-by: Antoni Baum <antoni.baum@protonmail.com>
```
  42c7f66a
- [Frontend] Refactor prompt processing (#4028) · 739b61a3
  Cyrus Leung authored Jul 23, 2024
```
Co-authored-by: Roger Wang <ywang@roblox.com>
```
  739b61a3
21 Jul, 2024 1 commit
- [Frontend] Move chat utils (#6602) · d7f4178d
  Cyrus Leung authored Jul 21, 2024
```
Co-authored-by: Roger Wang <ywang@roblox.com>
```
  d7f4178d
19 Jul, 2024 2 commits
- [Bugfix][Frontend] remove duplicate init logger (#6581) · 51f8aa90
  Daniele authored Jul 19, 2024
  
  51f8aa90
- [Bugfix][Frontend] Fix missing `/metrics` endpoint (#6463) · 6366efc6
  Cyrus Leung authored Jul 19, 2024
  
  6366efc6
18 Jul, 2024 2 commits
- [BugFix][Frontend] Use LoRA tokenizer in OpenAI APIs (#6227) · e2fbaee7
  Nick Hill authored Jul 18, 2024
```
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
```
  e2fbaee7
- [core][model] yet another cpu offload implementation (#6496) · 1c27d25f
  youkaichao authored Jul 17, 2024
```
Co-authored-by: Michael Goin <michael@neuralmagic.com>
```
  1c27d25f
16 Jul, 2024 2 commits
- [Frontend] Support for chat completions input in the tokenize endpoint (#5923) · 7a3d2a5b
  sasha0552 authored Jul 16, 2024
  
  7a3d2a5b
- [Bugfix][CI/Build] Test prompt adapters in openai entrypoint tests (#6419) · d92b3c5c
  Joe authored Jul 15, 2024
  
  d92b3c5c