Commits · ed812a73fae77bb520b739cfeaad36dbd61e2b03 · OpenDAS / vllm_cscc

03 Aug, 2024 1 commit

[ Frontend ] Multiprocessing for OpenAI Server with `zeromq` (#6883) · ed812a73

Robert Shaw authored Aug 02, 2024


Signed-off-by: Joe Runde <Joseph.Runde@ibm.com>
Co-authored-by: Joe Runde <Joseph.Runde@ibm.com>
Co-authored-by: Joe Runde <joe@joerun.de>
Co-authored-by: Nick Hill <nickhill@us.ibm.com>
Co-authored-by: Simon Mo <simon.mo@hey.com>

ed812a73

31 Jul, 2024 1 commit
- [mypy] Enable following imports for some directories (#6681) · da1f7cc1
  Cyrus Leung authored Jul 31, 2024
  
  da1f7cc1
30 Jul, 2024 1 commit
- [Doc] Super tiny fix doc typo (#6949) · f0584036
  fzyzcjy authored Jul 31, 2024
  
  f0584036
29 Jul, 2024 1 commit
- [TPU] Support tensor parallelism in async llm engine (#6891) · 7f8d612d
  Earthwalker authored Jul 30, 2024
  
  7f8d612d
27 Jul, 2024 2 commits
- [Hardware][TPU] Implement tensor parallelism with Ray (#5871) · 52f07e3d
  Woosuk Kwon authored Jul 26, 2024
  
  52f07e3d
- [Bugfix][Model] Jamba assertions and no chunked prefill by default for Jamba (#6784) · ed94e4f4
  tomeras91 authored Jul 27, 2024
  
  ed94e4f4
26 Jul, 2024 1 commit
- [Hardware] [Intel] Enable Multiprocessing and tensor parallel in CPU backend... · 3bbb4936
  Li, Jiang authored Jul 27, 2024
```
[Hardware] [Intel] Enable Multiprocessing and tensor parallel in CPU backend and update documentation  (#6125)
```
  3bbb4936
24 Jul, 2024 1 commit
- [Bugfix] Miscalculated latency lead to time_to_first_token_seconds inaccurate. (#6686) · 40468b13
  Allen.Dou authored Jul 24, 2024
  
  40468b13
23 Jul, 2024 4 commits
- [bitsandbytes]: support read bnb pre-quantized model (#5753) · 87525fab
  dongmao zhang authored Jul 23, 2024
```
Co-authored-by: Michael Goin <michael@neuralmagic.com>
```
  87525fab
- [Bugfix] StatLoggers: cache spec decode metrics when they get collected. (#6645) · 2f808e69
  Thomas Parnell authored Jul 24, 2024
```
Signed-off-by: Thomas Parnell <tpa@zurich.ibm.com>
```
  2f808e69
- support ignore patterns in model loader (#6673) · 3eda4ec7
  Simon Mo authored Jul 22, 2024
  
  3eda4ec7
- [Misc] Enable chunked prefill by default for long context models (#6666) · 729171ae
  Woosuk Kwon authored Jul 22, 2024
  
  729171ae
22 Jul, 2024 1 commit
- [Frontend] Refactor prompt processing (#4028) · 739b61a3
  Cyrus Leung authored Jul 23, 2024
```
Co-authored-by: Roger Wang <ywang@roblox.com>
```
  739b61a3
21 Jul, 2024 1 commit
- [Spec Decode] Disable Log Prob serialization to CPU for spec decoding for both... · 14f91fe6
  sroy745 authored Jul 20, 2024
```
[Spec Decode] Disable Log Prob serialization to CPU for spec decoding for both draft and target models. (#6485)
```
  14f91fe6
20 Jul, 2024 2 commits
- Pipeline Parallel: Guard for KeyErrors at request abort (#6587) · 3f8d42c8
  Travis Johnson authored Jul 19, 2024
```
Signed-off-by: Travis Johnson <tsjohnso@us.ibm.com>
```
  3f8d42c8
- [Core] Allow specifying custom Executor (#6557) · 7bd82002
  Antoni Baum authored Jul 19, 2024
  
  7bd82002
18 Jul, 2024 3 commits
- [BugFix][Frontend] Use LoRA tokenizer in OpenAI APIs (#6227) · e2fbaee7
  Nick Hill authored Jul 18, 2024
```
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
```
  e2fbaee7
- [Core] Introduce SPMD worker execution using Ray accelerated DAG (#6032) · 61e59274
  Rui Qiao authored Jul 17, 2024
```
Signed-off-by: Rui Qiao <ruisearch42@gmail.com>
Co-authored-by: Stephanie Wang <swang@cs.berkeley.edu>
```
  61e59274
- [core][model] yet another cpu offload implementation (#6496) · 1c27d25f
  youkaichao authored Jul 17, 2024
```
Co-authored-by: Michael Goin <michael@neuralmagic.com>
```
  1c27d25f
17 Jul, 2024 1 commit
- [Bugfix] Fix Ray Metrics API usage (#6354) · 5f0b9933
  Antoni Baum authored Jul 17, 2024
  
  5f0b9933
16 Jul, 2024 2 commits
- [Misc] Log spec decode metrics (#6454) · 160e1d8c
  Cody Yu authored Jul 16, 2024
  
  160e1d8c
- [Misc] Fix typos in spec. decode metrics logging. (#6470) · 7508a3dc
  Thomas Parnell authored Jul 16, 2024
```
Signed-off-by: Thomas Parnell <tpa@zurich.ibm.com>
```
  7508a3dc
12 Jul, 2024 1 commit
- [Bugfix] Fix usage stats logging exception warning with OpenVINO (#6349) · adf32e0a
  Helena Kloosterman authored Jul 12, 2024
  
  adf32e0a
11 Jul, 2024 3 commits
- [ BugFix ] Prompt Logprobs Detokenization (#6223) · 7ed6a4f0
  Robert Shaw authored Jul 11, 2024
```
Co-authored-by: Zifei Tong <zifeitong@gmail.com>
```
  7ed6a4f0
- [BugFix] get_and_reset only when scheduler outputs are not empty (#6266) · 55f692b4
  Mor Zusman authored Jul 11, 2024
  
  55f692b4
- [BugFix]: fix engine timeout due to request abort (#6255) · 546b101f
  pushan authored Jul 11, 2024
```
Signed-off-by: yatta zhang <ytzhang01@foxmail.com>
Signed-off-by: zhangyuntao.dev <zhangyuntao.dev@bytedance.com>
Co-authored-by: zhangyuntao.dev <zhangyuntao.dev@bytedance.com>
```
  546b101f
09 Jul, 2024 1 commit

[CORE] Adding support for insertion of soft-tuned prompts (#4645) · 4d6ada94

Swapnil Parekh authored Jul 09, 2024


Co-authored-by: Swapnil Parekh <swapnilp@ibm.com>
Co-authored-by: Joe G <joseph.granados@h2o.ai>
Co-authored-by: Antoni Baum <antoni.baum@protonmail.com>

4d6ada94

03 Jul, 2024 2 commits
- [vlm] Remove vision language config. (#6089) · d9e98f42
  xwjiang2010 authored Jul 03, 2024
```
Signed-off-by: Xiaowei Jiang <xwjiang2010@gmail.com>
Co-authored-by: Roger Wang <ywang@roblox.com>
```
  d9e98f42
- [BugFix] Avoid unnecessary Ray import warnings (#6079) · d830656a
  Nick Hill authored Jul 02, 2024
  
  d830656a
02 Jul, 2024 4 commits

[Model] Jamba support (#4115) · 9d6a8daa

Mor Zusman authored Jul 03, 2024


Signed-off-by: Muralidhar Andoorveedu <muralidhar.andoorveedu@centml.ai>
Co-authored-by: Erez Schwartz <erezs@ai21.com>
Co-authored-by: Mor Zusman <morz@ai21.com>
Co-authored-by: tomeras91 <57313761+tomeras91@users.noreply.github.com>
Co-authored-by: Tomer Asida <tomera@ai21.com>
Co-authored-by: Zhuohan Li <zhuohan123@gmail.com>
Co-authored-by: Muralidhar Andoorveedu <muralidhar.andoorveedu@centml.ai>

9d6a8daa

[Core] Pipeline Parallel Support (#4412) · c5832d2a
Murali Andoorveedu authored Jul 02, 2024
```
Signed-off-by: Muralidhar Andoorveedu <muralidhar.andoorveedu@centml.ai>
```
c5832d2a

[VLM] Remove `image_input_type` from VLM config (#5852) · 98d6682c

xwjiang2010 authored Jul 02, 2024


Signed-off-by: Xiaowei Jiang <xwjiang2010@gmail.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
Co-authored-by: Roger Wang <ywang@roblox.com>

98d6682c

[Core] Optimize block_manager_v2 vs block_manager_v1 (to make V2 default) (#5602) · 3476ed08
Alexander Matveev authored Jul 01, 2024

3476ed08

01 Jul, 2024 2 commits
- [BugFix] Ensure worker model loop is always stopped at the right time (#5987) · c87ebc3e
  Nick Hill authored Jul 01, 2024
  
  c87ebc3e
- [Speculative Decoding 2/2 ] Integrate typical acceptance sampler into Spec Decode Worker (#5348) · 80ca1e6a
  sroy745 authored Jul 01, 2024
  
  80ca1e6a
29 Jun, 2024 2 commits
- [Bugfix] Support `eos_token_id` from `config.json` (#5954) · 51e971d3
  Cyrus Leung authored Jun 29, 2024
  
  51e971d3
- [Misc] Extend vLLM Metrics logging API (#5925) · 906a19cd
  William Lin authored Jun 28, 2024
```
Co-authored-by: Antoni Baum <antoni.baum@protonmail.com>
```
  906a19cd
28 Jun, 2024 2 commits
- [Hardware][Intel] OpenVINO vLLM backend (#5379) · 57f09a41
  Ilya Lavrenov authored Jun 28, 2024
  
  57f09a41
- [Core] Registry for processing model inputs (#5214) · 5cbe8d15
  Cyrus Leung authored Jun 28, 2024
```
Co-authored-by: ywang96 <ywang@roblox.com>
```
  5cbe8d15
27 Jun, 2024 1 commit
- [BugFix] Fix `min_tokens` behaviour for multiple eos tokens (#5849) · 365791ff
  Nick Hill authored Jun 27, 2024
  
  365791ff