Commits · 18b296fdb2248e8a65bf005e7193ebd523b875b6 · OpenDAS / vllm_cscc · GitLab

07 Oct, 2024 1 commit
- [core] remove beam search from the core (#9105) · 18b296fd
  youkaichao authored Oct 06, 2024
  
  18b296fd
06 Oct, 2024 1 commit
- [Frontend] API support for beam search (#9087) · 168cab6b
  Brendan Wong authored Oct 05, 2024
```
Co-authored-by: youkaichao <youkaichao@126.com>
```
  168cab6b
01 Oct, 2024 2 commits
- [Core] [Frontend] Priority scheduling for embeddings and in the OpenAI-API (#8965) · 35bd2151
  Sebastian Schoennenbeck authored Oct 01, 2024
  
  35bd2151
- [Frontend][Core] Move guided decoding params into sampling params (#8252) · 062c89e7
  Joe Runde authored Sep 30, 2024
```
Signed-off-by: Joe Runde <Joseph.Runde@ibm.com>
Co-authored-by: Nick Hill <nickhill@us.ibm.com>
```
  062c89e7
27 Sep, 2024 3 commits
- [Core] Priority-based scheduling in async engine (#8850) · bd429f2b
  Sebastian Schoennenbeck authored Sep 28, 2024
  
  bd429f2b
- [Core] Multi-Step + Single Step Prefills via Chunked Prefill code path (#8378) · c2ec430a
  Varun Sundar Rabindranath authored Sep 27, 2024
```
Co-authored-by: Varun Sundar Rabindranath <varun@neuralmagic.com>
```
  c2ec430a
- [Core] rename`PromptInputs` and `inputs` (#8876) · 3b00b9c2
  Cyrus Leung authored Sep 27, 2024
  
  3b00b9c2
25 Sep, 2024 2 commits
- Revert "rename PromptInputs and inputs with backward compatibility (#8760) (#8810) · 4f1ba084
  Simon Mo authored Sep 25, 2024
  
  4f1ba084
- rename PromptInputs and inputs with backward compatibility (#8760) · 28e1299e
  Cyrus Leung authored Sep 26, 2024
  
  28e1299e
24 Sep, 2024 1 commit
- Revert "[Core] Rename `PromptInputs` to `PromptType`, and `inputs` to `prompt`" (#8750) · 3185fb0c
  Simon Mo authored Sep 23, 2024
  
  3185fb0c
21 Sep, 2024 1 commit
- [Core] Rename `PromptInputs` and `inputs`(#8673) · 0057894e
  Cyrus Leung authored Sep 21, 2024
  
  0057894e
18 Sep, 2024 2 commits

[Core][Bugfix][Perf] Introduce `MQLLMEngine` to avoid `asyncio` OH (#8157) · 7c7714d8

Alexander Matveev authored Sep 18, 2024


Co-authored-by: Nick Hill <nickhill@us.ibm.com>
Co-authored-by: rshaw@neuralmagic.com <rshaw@neuralmagic.com>
Co-authored-by: Robert Shaw <114415538+robertgshaw2-neuralmagic@users.noreply.github.com>
Co-authored-by: Simon Mo <simon.mo@hey.com>

7c7714d8

[CI/Build] Update Ruff version (#8469) · 9d104b5b

Aaron Pham authored Sep 18, 2024


Signed-off-by: Aaron Pham <contact@aarnphm.xyz>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>

9d104b5b

16 Sep, 2024 1 commit
- [BugFix] Fix clean shutdown issues (#8492) · acd5511b
  Nick Hill authored Sep 16, 2024
  
  acd5511b
13 Sep, 2024 2 commits
- [bugfix] torch profiler bug for single gpu with GPUExecutor (#8354) · ba775279
  William Lin authored Sep 12, 2024
  
  ba775279
- [Core] Factor out input preprocessing to a separate class (#7329) · 5ec9c0fb
  Cyrus Leung authored Sep 13, 2024
  
  5ec9c0fb
12 Sep, 2024 1 commit
- [misc] remove engine_use_ray (#8126) · f842a7af
  youkaichao authored Sep 11, 2024
  
  f842a7af
08 Sep, 2024 1 commit
- [Bugfix] Fix async postprocessor in case of preemption (#8267) · 4ef41b84
  Alexander Matveev authored Sep 08, 2024
  
  4ef41b84
03 Sep, 2024 1 commit
- [Core] Optimize Async + Multi-step (#8050) · 6d646d08
  Alexander Matveev authored Sep 03, 2024
  
  6d646d08
30 Aug, 2024 1 commit
- [Core] Logprobs support in Multi-step (#7652) · 428dd144
  afeldman-nm authored Aug 29, 2024
  
  428dd144
29 Aug, 2024 1 commit
- [Core] Combine async postprocessor and multi-step (#7921) · 3f60f224
  Alexander Matveev authored Aug 29, 2024
  
  3f60f224
28 Aug, 2024 1 commit
- [Core] Async_output_proc: Add virtual engine support (towards pipeline parallel) (#7911) · f508e03e
  Alexander Matveev authored Aug 28, 2024
  
  f508e03e
27 Aug, 2024 2 commits
- [Hardware][Intel GPU] Add intel GPU pipeline parallel support. (#7810) · 076169f6
  Kunshang Ji authored Aug 28, 2024
  
  076169f6
- [Core] Asynchronous Output Processor (#7049) · 2eedede8
  Megha Agarwal authored Aug 26, 2024
```
Co-authored-by: Alexander Matveev <alexm@neuralmagic.com>
```
  2eedede8
23 Aug, 2024 1 commit
- [Core] Add multi-step support to LLMEngine (#7789) · 9db93de2
  Alexander Matveev authored Aug 23, 2024
  
  9db93de2
21 Aug, 2024 4 commits
- [misc] Add Torch profiler support (#7451) · dd53c4b0
  William Lin authored Aug 21, 2024
```
Co-authored-by: Cody Yu <hao.yu.cody@gmail.com>
```
  dd53c4b0
- [Bugfix][Frontend] Fix Issues Under High Load With `zeromq` Frontend (#7394) · f7e3b0c5
  Robert Shaw authored Aug 21, 2024
```
Co-authored-by: Nick Hill <nickhill@us.ibm.com>
```
  f7e3b0c5
- [BugFix] Avoid premature async generator exit and raise all exception variations (#7698) · c75363fb
  Nick Hill authored Aug 21, 2024
  
  c75363fb
- [mypy] Enable following imports for entrypoints (#7248) · baaedfdb
  Cyrus Leung authored Aug 21, 2024
```
Co-authored-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
Co-authored-by: Fei <dfdfcai4@gmail.com>
```
  baaedfdb
19 Aug, 2024 1 commit
- [core] Multi Step Scheduling (#7000) · 47b65a55
  William Lin authored Aug 19, 2024
```
Co-authored-by: afeldman-nm <156691304+afeldman-nm@users.noreply.github.com>
```
  47b65a55
18 Aug, 2024 1 commit
- [ Bugfix ] Fix Prometheus Metrics With `zeromq` Frontend (#7279) · e3b31821
  Robert Shaw authored Aug 18, 2024
```
Co-authored-by: Nick Hill <nickhill@us.ibm.com>
```
  e3b31821
14 Aug, 2024 1 commit

[Misc] Deprecation Warning when setting --engine-use-ray (#7424) · 70b746ef

Wallas Henrique authored Aug 14, 2024


Signed-off-by: Wallas Santos <wallashss@ibm.com>
Co-authored-by: youkaichao <youkaichao@gmail.com>
Co-authored-by: Nick Hill <nickhill@us.ibm.com>
Co-authored-by: youkaichao <youkaichao@126.com>

70b746ef

13 Aug, 2024 1 commit
- [Core] Shut down aDAG workers with clean async llm engine exit (#7224) · 198d6a28
  Rui Qiao authored Aug 12, 2024
```
Signed-off-by: Rui Qiao <ruisearch42@gmail.com>
```
  198d6a28
09 Aug, 2024 2 commits
- [Core] Streamline stream termination in `AsyncLLMEngine` (#7336) · b4e9528f
  Nick Hill authored Aug 09, 2024
  
  b4e9528f
- [Core] Support serving encoder/decoder models (#7258) · 7eb4a51c
  Cyrus Leung authored Aug 09, 2024
  
  7eb4a51c
08 Aug, 2024 1 commit
- [Frontend] Kill the server on engine death (#6594) · 21b9c49a
  Joe Runde authored Aug 08, 2024
```
Signed-off-by: Joe Runde <joe@joerun.de>
Signed-off-by: Joe Runde <Joseph.Runde@ibm.com>
```
  21b9c49a
07 Aug, 2024 1 commit
- [BugFix] Overhaul async request cancellation (#7111) · 9a3f49ae
  Nick Hill authored Aug 06, 2024
  
  9a3f49ae
03 Aug, 2024 1 commit

[ Frontend ] Multiprocessing for OpenAI Server with `zeromq` (#6883) · ed812a73

Robert Shaw authored Aug 02, 2024


Signed-off-by: Joe Runde <Joseph.Runde@ibm.com>
Co-authored-by: Joe Runde <Joseph.Runde@ibm.com>
Co-authored-by: Joe Runde <joe@joerun.de>
Co-authored-by: Nick Hill <nickhill@us.ibm.com>
Co-authored-by: Simon Mo <simon.mo@hey.com>

ed812a73

29 Jul, 2024 1 commit
- [TPU] Support tensor parallelism in async llm engine (#6891) · 7f8d612d
  Earthwalker authored Jul 30, 2024
  
  7f8d612d
26 Jul, 2024 1 commit
- [Hardware] [Intel] Enable Multiprocessing and tensor parallel in CPU backend... · 3bbb4936
  Li, Jiang authored Jul 27, 2024
```
[Hardware] [Intel] Enable Multiprocessing and tensor parallel in CPU backend and update documentation  (#6125)
```
  3bbb4936