Commits · 473e7b3606e9b95b39c7da46cce00a33c069dc00 · OpenDAS / vllm_cscc

11 Oct, 2024 2 commits
- [Bugfix] Fix priority in multiprocessing engine (#9277) · df3dcdf4
  Sebastian Schoennenbeck authored Oct 11, 2024
  
  df3dcdf4
- [Misc] Collect model support info in a single process per model (#9233) · e808156f
  Cyrus Leung authored Oct 11, 2024
  
  e808156f
10 Oct, 2024 1 commit
- [Core] Fix invalid args to _process_request (#9201) · cf25b93b
  Russell Bryant authored Oct 10, 2024
```
Signed-off-by: Russell Bryant <rbryant@redhat.com>
```
  cf25b93b
08 Oct, 2024 1 commit
- [Frontend] API support for beam search for MQLLMEngine (#9117) · 8c746226
  Brendan Wong authored Oct 07, 2024
  
  8c746226
01 Oct, 2024 2 commits
- [Core] [Frontend] Priority scheduling for embeddings and in the OpenAI-API (#8965) · 35bd2151
  Sebastian Schoennenbeck authored Oct 01, 2024
  
  35bd2151
- [Frontend][Core] Move guided decoding params into sampling params (#8252) · 062c89e7
  Joe Runde authored Sep 30, 2024
```
Signed-off-by: Joe Runde <Joseph.Runde@ibm.com>
Co-authored-by: Nick Hill <nickhill@us.ibm.com>
```
  062c89e7
27 Sep, 2024 1 commit
- [Core] rename`PromptInputs` and `inputs` (#8876) · 3b00b9c2
  Cyrus Leung authored Sep 27, 2024
  
  3b00b9c2
25 Sep, 2024 4 commits
- Revert "rename PromptInputs and inputs with backward compatibility (#8760) (#8810) · 4f1ba084
  Simon Mo authored Sep 25, 2024
  
  4f1ba084
- [Frontend] MQLLMEngine supports profiling. (#8761) · 64840dfa
  科英 authored Sep 26, 2024
  
  64840dfa
- rename PromptInputs and inputs with backward compatibility (#8760) · 28e1299e
  Cyrus Leung authored Sep 26, 2024
  
  28e1299e
- [Bugfix] Use heartbeats instead of health checks (#8583) · 6e0c9d6b
  Joe Runde authored Sep 24, 2024
  
  6e0c9d6b
24 Sep, 2024 1 commit
- Revert "[Core] Rename `PromptInputs` to `PromptType`, and `inputs` to `prompt`" (#8750) · 3185fb0c
  Simon Mo authored Sep 23, 2024
  
  3185fb0c
23 Sep, 2024 1 commit
- Add output streaming support to multi-step + async while ensuring RequestOutput obj reuse (#8335) · 1a2aef3e
  Alexander Matveev authored Sep 23, 2024
  
  1a2aef3e
21 Sep, 2024 1 commit
- [Core] Rename `PromptInputs` and `inputs`(#8673) · 0057894e
  Cyrus Leung authored Sep 21, 2024
  
  0057894e
19 Sep, 2024 1 commit
- [Frontend] Use MQLLMEngine for embeddings models too (#8584) · 76515f30
  Nick Hill authored Sep 19, 2024
  
  76515f30
18 Sep, 2024 2 commits

[Bugfix] add `dead_error` property to engine client (#8574) · 0d47bf3b
Joe Runde authored Sep 18, 2024
```
Signed-off-by: Joe Runde <Joseph.Runde@ibm.com>
```
0d47bf3b

[Core][Bugfix][Perf] Introduce `MQLLMEngine` to avoid `asyncio` OH (#8157) · 7c7714d8

Alexander Matveev authored Sep 18, 2024


Co-authored-by: Nick Hill <nickhill@us.ibm.com>
Co-authored-by: rshaw@neuralmagic.com <rshaw@neuralmagic.com>
Co-authored-by: Robert Shaw <114415538+robertgshaw2-neuralmagic@users.noreply.github.com>
Co-authored-by: Simon Mo <simon.mo@hey.com>

7c7714d8