Commits · d4db9f53c8a50a2b0788cf1e03b5b91f20de4313 · OpenDAS / vllm_cscc

04 Sep, 2024 1 commit
- [Benchmark] Add `--async-engine` option to benchmark_throughput.py (#7964) · d4db9f53
  Nick Hill authored Sep 03, 2024
  
  d4db9f53
31 Aug, 2024 1 commit
- [Frontend][VLM] Add support for multiple multi-modal items (#8049) · 5231f089
  Roger Wang authored Aug 31, 2024
  
  5231f089
29 Aug, 2024 1 commit
- [Frontend] Minor optimizations to zmq decoupled front-end (#7957) · 4289cad3
  Nick Hill authored Aug 28, 2024
```
Co-authored-by: Robert Shaw <rshaw@neuralmagic>
```
  4289cad3
28 Aug, 2024 1 commit
- [mypy][CI/Build] Fix mypy errors (#7929) · 51f86bf4
  Cyrus Leung authored Aug 28, 2024
  
  51f86bf4
27 Aug, 2024 2 commits
- [Model] Add Mistral Tokenization to improve robustness and chat encoding (#7739) · 6fc4e6e0
  Patrick von Platen authored Aug 27, 2024
  
  6fc4e6e0
- [Core] Asynchronous Output Processor (#7049) · 2eedede8
  Megha Agarwal authored Aug 26, 2024
```
Co-authored-by: Alexander Matveev <alexm@neuralmagic.com>
```
  2eedede8
26 Aug, 2024 1 commit
- [Bugfix]: Use float32 for base64 embedding (#7855) · 0b769992
  ℍ𝕠𝕝𝕝𝕠𝕨 𝕄𝕒𝕟 authored Aug 26, 2024
```
Signed-off-by: Hollow Man <hollowman@opensuse.org>
```
  0b769992
24 Aug, 2024 2 commits
- [Frontend] add json_schema support from OpenAI protocol (#7654) · d81abefd
  Tyler Rockwood authored Aug 24, 2024
  
  d81abefd
- [Frontend] Publish Prometheus metrics in run_batch API (#7641) · 8da48e4d
  Pooya Davoodi authored Aug 23, 2024
  
  8da48e4d
23 Aug, 2024 1 commit
- [Bugfix] Fix run_batch logger (#7640) · 6885fde3
  Pooya Davoodi authored Aug 23, 2024
  
  6885fde3
22 Aug, 2024 2 commits
- [Frontend] error suppression cleanup (#7786) · b903e1ba
  Joe Runde authored Aug 22, 2024
```
Signed-off-by: Joe Runde <Joseph.Runde@ibm.com>
```
  b903e1ba
- [Bug][Frontend] Improve ZMQ client robustness (#7443) · cde9183b
  Joe Runde authored Aug 21, 2024
```
Signed-off-by: Joe Runde <Joseph.Runde@ibm.com>
```
  cde9183b
21 Aug, 2024 7 commits
- [Model] Add UltravoxModel and UltravoxConfig (#7615) · 1ca0d4f8
  Peter Salas authored Aug 21, 2024
  
  1ca0d4f8
- [misc] Add Torch profiler support (#7451) · dd53c4b0
  William Lin authored Aug 21, 2024
```
Co-authored-by: Cody Yu <hao.yu.cody@gmail.com>
```
  dd53c4b0
- [Frontend] Improve Startup Failure UX (#7716) · 970dfdc0
  Robert Shaw authored Aug 21, 2024
  
  970dfdc0
- [Bugfix][Frontend] Fix Issues Under High Load With `zeromq` Frontend (#7394) · f7e3b0c5
  Robert Shaw authored Aug 21, 2024
```
Co-authored-by: Nick Hill <nickhill@us.ibm.com>
```
  f7e3b0c5
- [Bugfix] chat method add_generation_prompt param (#7734) · d3c002ea
  Brian Li authored Aug 22, 2024
  
  d3c002ea
- [mypy] Enable following imports for entrypoints (#7248) · baaedfdb
  Cyrus Leung authored Aug 21, 2024
```
Co-authored-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
Co-authored-by: Fei <dfdfcai4@gmail.com>
```
  baaedfdb
- [ci] try to log process using the port to debug the port usage (#7711) · b74a1258
  youkaichao authored Aug 20, 2024
  
  b74a1258
18 Aug, 2024 1 commit
- [ Bugfix ] Fix Prometheus Metrics With `zeromq` Frontend (#7279) · e3b31821
  Robert Shaw authored Aug 18, 2024
```
Co-authored-by: Nick Hill <nickhill@us.ibm.com>
```
  e3b31821
17 Aug, 2024 1 commit
- [Bugfix] Clear engine reference in AsyncEngineRPCServer (#7618) · bae888cb
  Rui Qiao authored Aug 16, 2024
```
Signed-off-by: Rui Qiao <ruisearch42@gmail.com>
```
  bae888cb
16 Aug, 2024 6 commits
- support tqdm in notebooks (#7510) · ec724a72
  fzyzcjy authored Aug 17, 2024
  
  ec724a72
- [Bugfix][Hardware][AMD][Frontend] add quantization param to embedding checking method (#7513) · 0e39a33c
  Gordon Wong authored Aug 17, 2024
  
  0e39a33c
- [Core] Use uvloop with zmq-decoupled front-end (#7570) · 9587b050
  Nick Hill authored Aug 15, 2024
  
  9587b050
- Chat method for offline llm (#5049) · 3b19e39d
  nunjunj authored Aug 16, 2024
```
Co-authored-by: nunjunj <ray@g-3ff9f30f2ed650001.c.vllm-405802.internal>
Co-authored-by: nunjunj <ray@g-1df6075697c3f0001.c.vllm-405802.internal>
Co-authored-by: nunjunj <ray@g-c5a2c23abc49e0001.c.vllm-405802.internal>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
Co-authored-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  3b19e39d
- [Feature]: Add OpenAI server prompt_logprobs support #6508 (#7453) · f878c8fe
  Grant Pinkert authored Aug 16, 2024
  
  f878c8fe
- [Bugfix][Harmless] Fix float16 dtype for model_is_embedding (#7566) · 9c8e2d11
  Michael Goin authored Aug 15, 2024
  
  9c8e2d11
14 Aug, 2024 1 commit
- [Bugfix][Frontend] Disable embedding API for chat models (#7504) · 67d115db
  jack authored Aug 15, 2024
```
Co-authored-by: jack <jack@alex>
```
  67d115db
13 Aug, 2024 4 commits
- [frontend] spawn engine process from api server process (#7484) · 33e5d7e6
  youkaichao authored Aug 13, 2024
  
  33e5d7e6
- [Frontend][Core] Add plumbing to support audio language models (#7446) · 00c3d68e
  Peter Salas authored Aug 13, 2024
  
  00c3d68e
- [Misc] improve logits processors logging message (#7435) · 97a6be95
  Andrew Wang authored Aug 12, 2024
  
  97a6be95
- [Core] Shut down aDAG workers with clean async llm engine exit (#7224) · 198d6a28
  Rui Qiao authored Aug 12, 2024
```
Signed-off-by: Rui Qiao <ruisearch42@gmail.com>
```
  198d6a28
09 Aug, 2024 2 commits
- [Frontend] Support embeddings in the run_batch API (#7132) · 249b8822
  Pooya Davoodi authored Aug 09, 2024
```
Co-authored-by: Simon Mo <simon.mo@hey.com>
```
  249b8822
- [Core] Support serving encoder/decoder models (#7258) · 7eb4a51c
  Cyrus Leung authored Aug 09, 2024
  
  7eb4a51c
08 Aug, 2024 1 commit
- [Frontend] Kill the server on engine death (#6594) · 21b9c49a
  Joe Runde authored Aug 08, 2024
```
Signed-off-by: Joe Runde <joe@joerun.de>
Signed-off-by: Joe Runde <Joseph.Runde@ibm.com>
```
  21b9c49a
07 Aug, 2024 4 commits
- [BugFix] Fix frontend multiprocessing hang (#7217) · fde47d3b
  Maximilien de Bayser authored Aug 07, 2024
```
Signed-off-by: Max de Bayser <mbayser@br.ibm.com>
Co-authored-by: Robert Shaw <114415538+robertgshaw2-neuralmagic@users.noreply.github.com>
```
  fde47d3b
- [ BugFix ] Move `zmq` frontend to IPC instead of TCP (#7222) · 56498572
  Robert Shaw authored Aug 07, 2024
  
  56498572
- [Frontend] Gracefully handle missing chat template and fix CI failure (#7238) · 66d617e3
  Cyrus Leung authored Aug 07, 2024
```
Co-authored-by: Roger Wang <ywang@roblox.com>
```
  66d617e3
- [BugFix] Overhaul async request cancellation (#7111) · 9a3f49ae
  Nick Hill authored Aug 06, 2024
  
  9a3f49ae
06 Aug, 2024 1 commit

[Core] Subclass ModelRunner to support cross-attention & encoder sequences... · fd95e026

afeldman-nm authored Aug 06, 2024


[Core] Subclass ModelRunner to support cross-attention & encoder sequences (towards eventual encoder/decoder model support) (#4942)
Co-authored-by: Andrew Feldman <afeld2012@gmail.com>
Co-authored-by: Nick Hill <nickhill@us.ibm.com>

fd95e026