Commits · 9b8c8ba1198cbcd311d28b7647f0f8d5dcdc9212 · OpenDAS / vllm_cscc

23 Sep, 2024 1 commit
- [Core][Frontend] Support Passing Multimodal Processor Kwargs (#8657) · 9b8c8ba1
  Alex Brooks authored Sep 23, 2024
```
Signed-off-by: Alex-Brooks <Alex.Brooks@ibm.com>
```
  9b8c8ba1
21 Sep, 2024 1 commit
- [Core] Rename `PromptInputs` and `inputs`(#8673) · 0057894e
  Cyrus Leung authored Sep 21, 2024
  
  0057894e
18 Sep, 2024 2 commits

[Core][Bugfix][Perf] Introduce `MQLLMEngine` to avoid `asyncio` OH (#8157) · 7c7714d8

Alexander Matveev authored Sep 18, 2024


Co-authored-by: Nick Hill <nickhill@us.ibm.com>
Co-authored-by: rshaw@neuralmagic.com <rshaw@neuralmagic.com>
Co-authored-by: Robert Shaw <114415538+robertgshaw2-neuralmagic@users.noreply.github.com>
Co-authored-by: Simon Mo <simon.mo@hey.com>

7c7714d8

[CI/Build] Update Ruff version (#8469) · 9d104b5b

Aaron Pham authored Sep 18, 2024


Signed-off-by: Aaron Pham <contact@aarnphm.xyz>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>

9d104b5b

16 Sep, 2024 1 commit
- [BugFix] Fix clean shutdown issues (#8492) · acd5511b
  Nick Hill authored Sep 16, 2024
  
  acd5511b
13 Sep, 2024 3 commits
- [bugfix] torch profiler bug for single gpu with GPUExecutor (#8354) · ba775279
  William Lin authored Sep 12, 2024
  
  ba775279
- [Bugfix] Fix async log stats (#8417) · 68210201
  Alexander Matveev authored Sep 12, 2024
  
  68210201
- [Core] Factor out input preprocessing to a separate class (#7329) · 5ec9c0fb
  Cyrus Leung authored Sep 13, 2024
  
  5ec9c0fb
12 Sep, 2024 2 commits
- [Core] Add engine option to return only deltas or final output (#7381) · 551ce010
  Nick Hill authored Sep 12, 2024
  
  551ce010
- [misc] remove engine_use_ray (#8126) · f842a7af
  youkaichao authored Sep 11, 2024
  
  f842a7af
08 Sep, 2024 1 commit
- [Bugfix] Fix async postprocessor in case of preemption (#8267) · 4ef41b84
  Alexander Matveev authored Sep 08, 2024
  
  4ef41b84
07 Sep, 2024 1 commit
- [misc] [doc] [frontend] LLM torch profiler support (#7943) · 12dd7158
  William Lin authored Sep 06, 2024
  
  12dd7158
04 Sep, 2024 1 commit
- [Neuron] Adding support for adding/ overriding neuron configuration a… (#8062) · 008cf886
  Harsha vardhan manoj Bikki authored Sep 04, 2024
```
Co-authored-by: Harsha Bikki <harbikh@amazon.com>
```
  008cf886
03 Sep, 2024 2 commits
- [Misc] Raise a more informative exception in add/remove_logger (#7750) · 652c83b6
  Antoni Baum authored Sep 03, 2024
  
  652c83b6
- [Core] Optimize Async + Multi-step (#8050) · 6d646d08
  Alexander Matveev authored Sep 03, 2024
  
  6d646d08
30 Aug, 2024 3 commits
- [Core] Increase default `max_num_batched_tokens` for multimodal models (#8028) · 98cef6a2
  Cyrus Leung authored Aug 30, 2024
  
  98cef6a2
- [Core] Logprobs support in Multi-step (#7652) · 428dd144
  afeldman-nm authored Aug 29, 2024
  
  428dd144
- [VLM] Disallow overflowing `max_model_len` for multimodal models (#7998) · 4abed65c
  Cyrus Leung authored Aug 30, 2024
  
  4abed65c
29 Aug, 2024 1 commit
- [Core] Combine async postprocessor and multi-step (#7921) · 3f60f224
  Alexander Matveev authored Aug 29, 2024
  
  3f60f224
28 Aug, 2024 1 commit
- [Core] Async_output_proc: Add virtual engine support (towards pipeline parallel) (#7911) · f508e03e
  Alexander Matveev authored Aug 28, 2024
  
  f508e03e
27 Aug, 2024 2 commits
- [Hardware][Intel GPU] Add intel GPU pipeline parallel support. (#7810) · 076169f6
  Kunshang Ji authored Aug 28, 2024
  
  076169f6
- [Core] Asynchronous Output Processor (#7049) · 2eedede8
  Megha Agarwal authored Aug 26, 2024
```
Co-authored-by: Alexander Matveev <alexm@neuralmagic.com>
```
  2eedede8
23 Aug, 2024 2 commits
- [Core] Add multi-step support to LLMEngine (#7789) · 9db93de2
  Alexander Matveev authored Aug 23, 2024
  
  9db93de2
- [BugFix] Fix server crash on empty prompt (#7746) · e25fee57
  Maximilien de Bayser authored Aug 23, 2024
```
Signed-off-by: Max de Bayser <mbayser@br.ibm.com>
```
  e25fee57
21 Aug, 2024 2 commits
- [multi-step] Raise error if not using async engine (#7703) · 91f4522c
  William Lin authored Aug 21, 2024
  
  91f4522c
- [mypy] Enable following imports for entrypoints (#7248) · baaedfdb
  Cyrus Leung authored Aug 21, 2024
```
Co-authored-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
Co-authored-by: Fei <dfdfcai4@gmail.com>
```
  baaedfdb
19 Aug, 2024 2 commits
- [MISC] Add prefix cache hit rate to metrics (#7606) · 3ac50b47
  Cody Yu authored Aug 19, 2024
  
  3ac50b47
- [Core] Optimize SPMD architecture with delta + serialization optimization (#7109) · ff7ec82c
  SangBin Cho authored Aug 18, 2024
  
  ff7ec82c
18 Aug, 2024 1 commit
- [ Bugfix ] Fix Prometheus Metrics With `zeromq` Frontend (#7279) · e3b31821
  Robert Shaw authored Aug 18, 2024
```
Co-authored-by: Nick Hill <nickhill@us.ibm.com>
```
  e3b31821
17 Aug, 2024 1 commit
- [VLM] Refactor `MultiModalConfig` initialization and profiling (#7530) · bbf55c48
  Roger Wang authored Aug 17, 2024
  
  bbf55c48
14 Aug, 2024 1 commit
- [VLM][Core] Support profiling with multiple multi-modal inputs per prompt (#7126) · 3f674a49
  Cyrus Leung authored Aug 15, 2024
  
  3f674a49
13 Aug, 2024 2 commits
- [misc][plugin] add plugin system implementation (#7426) · 16422ea7
  youkaichao authored Aug 13, 2024
  
  16422ea7
- [Core] Shut down aDAG workers with clean async llm engine exit (#7224) · 198d6a28
  Rui Qiao authored Aug 12, 2024
```
Signed-off-by: Rui Qiao <ruisearch42@gmail.com>
```
  198d6a28
09 Aug, 2024 2 commits
- [Core] Add span metrics for model_forward, scheduler and sampler time (#7089) · 933790c2
  Mahesh Keralapura authored Aug 09, 2024
  
  933790c2
- [Core] Support serving encoder/decoder models (#7258) · 7eb4a51c
  Cyrus Leung authored Aug 09, 2024
  
  7eb4a51c
06 Aug, 2024 1 commit

[Core] Subclass ModelRunner to support cross-attention & encoder sequences... · fd95e026

afeldman-nm authored Aug 06, 2024


[Core] Subclass ModelRunner to support cross-attention & encoder sequences (towards eventual encoder/decoder model support) (#4942)
Co-authored-by: Andrew Feldman <afeld2012@gmail.com>
Co-authored-by: Nick Hill <nickhill@us.ibm.com>

fd95e026

03 Aug, 2024 1 commit

[ Frontend ] Multiprocessing for OpenAI Server with `zeromq` (#6883) · ed812a73

Robert Shaw authored Aug 02, 2024


Signed-off-by: Joe Runde <Joseph.Runde@ibm.com>
Co-authored-by: Joe Runde <Joseph.Runde@ibm.com>
Co-authored-by: Joe Runde <joe@joerun.de>
Co-authored-by: Nick Hill <nickhill@us.ibm.com>
Co-authored-by: Simon Mo <simon.mo@hey.com>

ed812a73

31 Jul, 2024 1 commit
- [mypy] Enable following imports for some directories (#6681) · da1f7cc1
  Cyrus Leung authored Jul 31, 2024
  
  da1f7cc1
27 Jul, 2024 1 commit
- [Hardware][TPU] Implement tensor parallelism with Ray (#5871) · 52f07e3d
  Woosuk Kwon authored Jul 26, 2024
  
  52f07e3d
24 Jul, 2024 1 commit
- [Bugfix] Miscalculated latency lead to time_to_first_token_seconds inaccurate. (#6686) · 40468b13
  Allen.Dou authored Jul 24, 2024
  
  40468b13