Commits · 3c9817d297432c39c89fda666291de1f440838f0 · OpenDAS / vllm_cscc · GitLab

27 Nov, 2024 1 commit
- add VLLM_OPTEST_MODELS_PATH/OPTEST_MODELS_PATH to load models from local path... · 3c9817d2
  zhuwenwen authored Nov 27, 2024
```
add VLLM_OPTEST_MODELS_PATH/OPTEST_MODELS_PATH  to load models from local path instead of Hugging Face Hub
```
  3c9817d2
19 Nov, 2024 2 commits
- update fa interface tests · fc7db442
  zhuwenwen authored Nov 19, 2024
  
  fc7db442
- update torch version · aa389394
  zhuwenwen authored Nov 19, 2024
  
  aa389394
25 Sep, 2024 2 commits
- Revert "rename PromptInputs and inputs with backward compatibility (#8760) (#8810) · 4f1ba084
  Simon Mo authored Sep 25, 2024
  
  4f1ba084
- rename PromptInputs and inputs with backward compatibility (#8760) · 28e1299e
  Cyrus Leung authored Sep 26, 2024
  
  28e1299e
24 Sep, 2024 1 commit

[Frontend] Batch inference for llm.chat() API (#8648) · 2529d09b

Andy authored Sep 24, 2024


Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
Co-authored-by: Roger Wang <ywang@roblox.com>
Co-authored-by: Roger Wang <136131678+ywang96@users.noreply.github.com>

2529d09b

23 Sep, 2024 1 commit
- Add output streaming support to multi-step + async while ensuring RequestOutput obj reuse (#8335) · 1a2aef3e
  Alexander Matveev authored Sep 23, 2024
  
  1a2aef3e
20 Sep, 2024 1 commit
- [Core] Support Lora lineage and base model metadata management (#6315) · 260d40b5
  Jiaxin Shan authored Sep 19, 2024
  
  260d40b5
18 Sep, 2024 1 commit

[Core][Bugfix][Perf] Introduce `MQLLMEngine` to avoid `asyncio` OH (#8157) · 7c7714d8

Alexander Matveev authored Sep 18, 2024


Co-authored-by: Nick Hill <nickhill@us.ibm.com>
Co-authored-by: rshaw@neuralmagic.com <rshaw@neuralmagic.com>
Co-authored-by: Robert Shaw <114415538+robertgshaw2-neuralmagic@users.noreply.github.com>
Co-authored-by: Simon Mo <simon.mo@hey.com>

7c7714d8

12 Sep, 2024 1 commit
- [Bugfix] Offline mode fix (#8376) · f2e263b8
  Joe Runde authored Sep 12, 2024
```
Signed-off-by: Joe Runde <Joseph.Runde@ibm.com>
```
  f2e263b8
11 Sep, 2024 1 commit
- [Frontend] Create ErrorResponse instead of raising exceptions in run_batch (#8347) · cea95dfb
  Pooya Davoodi authored Sep 10, 2024
  
  cea95dfb
06 Sep, 2024 1 commit
- [Core] Support load and unload LoRA in api server (#6566) · db3bf7c9
  Jiaxin Shan authored Sep 05, 2024
```
Co-authored-by: Jee Jee Li <pandaleefree@gmail.com>
```
  db3bf7c9
04 Sep, 2024 1 commit
- [Frontend] Multimodal support in offline chat (#8098) · 855c262a
  Cyrus Leung authored Sep 04, 2024
  
  855c262a
31 Aug, 2024 1 commit
- [Frontend][VLM] Add support for multiple multi-modal items (#8049) · 5231f089
  Roger Wang authored Aug 31, 2024
  
  5231f089
27 Aug, 2024 1 commit
- [Tests] Disable retries and use context manager for openai client (#7565) · 39178c7f
  Nick Hill authored Aug 26, 2024
  
  39178c7f
26 Aug, 2024 1 commit
- [Bugfix]: Use float32 for base64 embedding (#7855) · 0b769992
  ℍ𝕠𝕝𝕝𝕠𝕨 𝕄𝕒𝕟 authored Aug 26, 2024
```
Signed-off-by: Hollow Man <hollowman@opensuse.org>
```
  0b769992
24 Aug, 2024 3 commits
- [misc][core] lazy import outlines (#7831) · 7d9ffa2a
  youkaichao authored Aug 24, 2024
  
  7d9ffa2a
- [Frontend] add json_schema support from OpenAI protocol (#7654) · d81abefd
  Tyler Rockwood authored Aug 24, 2024
  
  d81abefd
- [Frontend] Publish Prometheus metrics in run_batch API (#7641) · 8da48e4d
  Pooya Davoodi authored Aug 23, 2024
  
  8da48e4d
23 Aug, 2024 1 commit
- [BugFix] Fix server crash on empty prompt (#7746) · e25fee57
  Maximilien de Bayser authored Aug 23, 2024
```
Signed-off-by: Max de Bayser <mbayser@br.ibm.com>
```
  e25fee57
22 Aug, 2024 2 commits
- [Frontend] error suppression cleanup (#7786) · b903e1ba
  Joe Runde authored Aug 22, 2024
```
Signed-off-by: Joe Runde <Joseph.Runde@ibm.com>
```
  b903e1ba
- [Bug][Frontend] Improve ZMQ client robustness (#7443) · cde9183b
  Joe Runde authored Aug 21, 2024
```
Signed-off-by: Joe Runde <Joseph.Runde@ibm.com>
```
  cde9183b
21 Aug, 2024 4 commits
- [Model] Add UltravoxModel and UltravoxConfig (#7615) · 1ca0d4f8
  Peter Salas authored Aug 21, 2024
  
  1ca0d4f8
- [Frontend] Improve Startup Failure UX (#7716) · 970dfdc0
  Robert Shaw authored Aug 21, 2024
  
  970dfdc0
- [Bugfix][Frontend] Fix Issues Under High Load With `zeromq` Frontend (#7394) · f7e3b0c5
  Robert Shaw authored Aug 21, 2024
```
Co-authored-by: Nick Hill <nickhill@us.ibm.com>
```
  f7e3b0c5
- [mypy] Enable following imports for entrypoints (#7248) · baaedfdb
  Cyrus Leung authored Aug 21, 2024
```
Co-authored-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
Co-authored-by: Fei <dfdfcai4@gmail.com>
```
  baaedfdb
18 Aug, 2024 1 commit
- [ Bugfix ] Fix Prometheus Metrics With `zeromq` Frontend (#7279) · e3b31821
  Robert Shaw authored Aug 18, 2024
```
Co-authored-by: Nick Hill <nickhill@us.ibm.com>
```
  e3b31821
17 Aug, 2024 1 commit
- [VLM] Refactor `MultiModalConfig` initialization and profiling (#7530) · bbf55c48
  Roger Wang authored Aug 17, 2024
  
  bbf55c48
16 Aug, 2024 2 commits

Chat method for offline llm (#5049) · 3b19e39d

nunjunj authored Aug 16, 2024

Co-authored-by: nunjunj <ray@g-3ff9f30f2ed650001.c.vllm-405802.internal>
Co-authored-by: nunjunj <ray@g-1df6075697c3f0001.c.vllm-405802.internal>
Co-authored-by: nunjunj <ray@g-c5a2c23abc49e0001.c.vllm-405802.internal>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
Co-authored-by: DarkLight1337 <tlleungac@connect.ust.hk>

3b19e39d

[Feature]: Add OpenAI server prompt_logprobs support #6508 (#7453) · f878c8fe
Grant Pinkert authored Aug 16, 2024

f878c8fe

13 Aug, 2024 5 commits
- [misc][plugin] add plugin system implementation (#7426) · 16422ea7
  youkaichao authored Aug 13, 2024
  
  16422ea7
- [frontend] spawn engine process from api server process (#7484) · 33e5d7e6
  youkaichao authored Aug 13, 2024
  
  33e5d7e6
- [Frontend][Core] Add plumbing to support audio language models (#7446) · 00c3d68e
  Peter Salas authored Aug 13, 2024
  
  00c3d68e
- [Bugfix] Fix weight loading for Chameleon when TP>1 (#7410) · 7025b11d
  Cyrus Leung authored Aug 13, 2024
  
  7025b11d
- [Misc] improve logits processors logging message (#7435) · 97a6be95
  Andrew Wang authored Aug 12, 2024
  
  97a6be95
09 Aug, 2024 2 commits
- [Frontend] Support embeddings in the run_batch API (#7132) · 249b8822
  Pooya Davoodi authored Aug 09, 2024
```
Co-authored-by: Simon Mo <simon.mo@hey.com>
```
  249b8822
- [Core] Support serving encoder/decoder models (#7258) · 7eb4a51c
  Cyrus Leung authored Aug 09, 2024
  
  7eb4a51c
08 Aug, 2024 1 commit
- [Frontend] Kill the server on engine death (#6594) · 21b9c49a
  Joe Runde authored Aug 08, 2024
```
Signed-off-by: Joe Runde <joe@joerun.de>
Signed-off-by: Joe Runde <Joseph.Runde@ibm.com>
```
  21b9c49a
07 Aug, 2024 2 commits
- [BugFix] Fix frontend multiprocessing hang (#7217) · fde47d3b
  Maximilien de Bayser authored Aug 07, 2024
```
Signed-off-by: Max de Bayser <mbayser@br.ibm.com>
Co-authored-by: Robert Shaw <114415538+robertgshaw2-neuralmagic@users.noreply.github.com>
```
  fde47d3b
- [Frontend] Gracefully handle missing chat template and fix CI failure (#7238) · 66d617e3
  Cyrus Leung authored Aug 07, 2024
```
Co-authored-by: Roger Wang <ywang@roblox.com>
```
  66d617e3