Commits · 4ef41b84766670c1bd8079f58d35bf32b5bcb3ab · OpenDAS / vllm_cscc

08 Sep, 2024 1 commit
- [Bugfix] Fix async postprocessor in case of preemption (#8267) · 4ef41b84
  Alexander Matveev authored Sep 08, 2024
  
  4ef41b84
07 Sep, 2024 2 commits
- [Bugfix] Fix broken OpenAI tensorizer test (#8258) · 9f68e00d
  Cyrus Leung authored Sep 07, 2024
  
  9f68e00d
- [misc] [doc] [frontend] LLM torch profiler support (#7943) · 12dd7158
  William Lin authored Sep 06, 2024
  
  12dd7158
06 Sep, 2024 1 commit
- [Model] Allow loading from original Mistral format (#8168) · 29f49cd6
  Patrick von Platen authored Sep 07, 2024
```
Co-authored-by: Michael Goin <michael@neuralmagic.com>
```
  29f49cd6
04 Sep, 2024 1 commit
- [Neuron] Adding support for adding/ overriding neuron configuration a… (#8062) · 008cf886
  Harsha vardhan manoj Bikki authored Sep 04, 2024
```
Co-authored-by: Harsha Bikki <harbikh@amazon.com>
```
  008cf886
03 Sep, 2024 3 commits
- [Misc] Raise a more informative exception in add/remove_logger (#7750) · 652c83b6
  Antoni Baum authored Sep 03, 2024
  
  652c83b6
- [Core] Optimize Async + Multi-step (#8050) · 6d646d08
  Alexander Matveev authored Sep 03, 2024
  
  6d646d08
- [Bugfix] Fix single output condition in output processor (#7881) · 0fbc6696
  Woosuk Kwon authored Sep 02, 2024
  
  0fbc6696
02 Sep, 2024 1 commit
- [Core][Bugfix] Accept GGUF model without .gguf extension (#8056) · 4ca65a97
  Isotr0py authored Sep 02, 2024
  
  4ca65a97
31 Aug, 2024 1 commit
- [BugFix][Core] Multistep Fix Crash on Request Cancellation (#8059) · 8423aef4
  Robert Shaw authored Aug 31, 2024
  
  8423aef4
30 Aug, 2024 3 commits
- [Core] Increase default `max_num_batched_tokens` for multimodal models (#8028) · 98cef6a2
  Cyrus Leung authored Aug 30, 2024
  
  98cef6a2
- [Core] Logprobs support in Multi-step (#7652) · 428dd144
  afeldman-nm authored Aug 29, 2024
  
  428dd144
- [VLM] Disallow overflowing `max_model_len` for multimodal models (#7998) · 4abed65c
  Cyrus Leung authored Aug 30, 2024
  
  4abed65c
29 Aug, 2024 1 commit
- [Core] Combine async postprocessor and multi-step (#7921) · 3f60f224
  Alexander Matveev authored Aug 29, 2024
  
  3f60f224
28 Aug, 2024 1 commit
- [Core] Async_output_proc: Add virtual engine support (towards pipeline parallel) (#7911) · f508e03e
  Alexander Matveev authored Aug 28, 2024
  
  f508e03e
27 Aug, 2024 3 commits
- [Hardware][Intel GPU] Add intel GPU pipeline parallel support. (#7810) · 076169f6
  Kunshang Ji authored Aug 28, 2024
  
  076169f6
- [Model] Add Mistral Tokenization to improve robustness and chat encoding (#7739) · 6fc4e6e0
  Patrick von Platen authored Aug 27, 2024
  
  6fc4e6e0
- [Core] Asynchronous Output Processor (#7049) · 2eedede8
  Megha Agarwal authored Aug 26, 2024
```
Co-authored-by: Alexander Matveev <alexm@neuralmagic.com>
```
  2eedede8
26 Aug, 2024 2 commits
- [Bugfix] neuron: enable tensor parallelism (#7562) · 760e9f71
  omrishiv authored Aug 26, 2024
```
Signed-off-by: omrishiv <327609+omrishiv@users.noreply.github.com>
```
  760e9f71
- [CI/Build] Avoid downloading all HF files in `RemoteOpenAIServer` (#7836) · 029c71de
  Cyrus Leung authored Aug 26, 2024
  
  029c71de
23 Aug, 2024 2 commits
- [Core] Add multi-step support to LLMEngine (#7789) · 9db93de2
  Alexander Matveev authored Aug 23, 2024
  
  9db93de2
- [BugFix] Fix server crash on empty prompt (#7746) · e25fee57
  Maximilien de Bayser authored Aug 23, 2024
```
Signed-off-by: Max de Bayser <mbayser@br.ibm.com>
```
  e25fee57
22 Aug, 2024 1 commit
- [Bugfix] Use LoadFormat values for `vllm serve --load-format` (#7784) · 15310b51
  Michael Goin authored Aug 22, 2024
  
  15310b51
21 Aug, 2024 5 commits
- [misc] Add Torch profiler support (#7451) · dd53c4b0
  William Lin authored Aug 21, 2024
```
Co-authored-by: Cody Yu <hao.yu.cody@gmail.com>
```
  dd53c4b0
- [multi-step] Raise error if not using async engine (#7703) · 91f4522c
  William Lin authored Aug 21, 2024
  
  91f4522c
- [Bugfix][Frontend] Fix Issues Under High Load With `zeromq` Frontend (#7394) · f7e3b0c5
  Robert Shaw authored Aug 21, 2024
```
Co-authored-by: Nick Hill <nickhill@us.ibm.com>
```
  f7e3b0c5
- [BugFix] Avoid premature async generator exit and raise all exception variations (#7698) · c75363fb
  Nick Hill authored Aug 21, 2024
  
  c75363fb
- [mypy] Enable following imports for entrypoints (#7248) · baaedfdb
  Cyrus Leung authored Aug 21, 2024
```
Co-authored-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
Co-authored-by: Fei <dfdfcai4@gmail.com>
```
  baaedfdb
20 Aug, 2024 1 commit
- [Bugfix] use StoreBoolean instead of type=bool for --disable-logprobs-during-spec-decoding (#7665) · 67e02fa8
  Travis Johnson authored Aug 19, 2024
```
Signed-off-by: Travis Johnson <tsjohnso@us.ibm.com>
```
  67e02fa8
19 Aug, 2024 4 commits
- [core] Multi Step Scheduling (#7000) · 47b65a55
  William Lin authored Aug 19, 2024
```
Co-authored-by: afeldman-nm <156691304+afeldman-nm@users.noreply.github.com>
```
  47b65a55
- [Bugfix] fix lora_dtype value type in arg_utils.py - part 2 (#5428) · dad961ef
  Ali Panahi authored Aug 19, 2024
  
  dad961ef
- [MISC] Add prefix cache hit rate to metrics (#7606) · 3ac50b47
  Cody Yu authored Aug 19, 2024
  
  3ac50b47
- [Core] Optimize SPMD architecture with delta + serialization optimization (#7109) · ff7ec82c
  SangBin Cho authored Aug 18, 2024
  
  ff7ec82c
18 Aug, 2024 1 commit
- [ Bugfix ] Fix Prometheus Metrics With `zeromq` Frontend (#7279) · e3b31821
  Robert Shaw authored Aug 18, 2024
```
Co-authored-by: Nick Hill <nickhill@us.ibm.com>
```
  e3b31821
17 Aug, 2024 1 commit
- [VLM] Refactor `MultiModalConfig` initialization and profiling (#7530) · bbf55c48
  Roger Wang authored Aug 17, 2024
  
  bbf55c48
16 Aug, 2024 2 commits
- [Core] Fix tracking of model forward time in case of PP>1 (#7440) · 93478b63
  Mahesh Keralapura authored Aug 16, 2024
```
[Core] Fix tracking of model forward time to the span traces in case of PP>1 (#7440)
```
  93478b63
- [Misc] Add quantization config support for speculative model. (#7343) · b67ae00c
  shangmingc authored Aug 16, 2024
  
  b67ae00c
15 Aug, 2024 1 commit

[Bugfix] update neuron for version > 0.5.0 (#7175) · 9c1f78d5

omrishiv authored Aug 15, 2024


Signed-off-by: omrishiv <327609+omrishiv@users.noreply.github.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>

9c1f78d5

14 Aug, 2024 2 commits
- [core] [3/N] multi-step args and sequence.py (#7452) · 2ecf7b17
  William Lin authored Aug 14, 2024
  
  2ecf7b17
- [VLM][Core] Support profiling with multiple multi-modal inputs per prompt (#7126) · 3f674a49
  Cyrus Leung authored Aug 15, 2024
  
  3f674a49