Commits · b4e9528f9569d6eb8c29624771a4058fe794cb5a · OpenDAS / vllm_cscc

09 Aug, 2024 2 commits
- [Core] Streamline stream termination in `AsyncLLMEngine` (#7336) · b4e9528f
  Nick Hill authored Aug 09, 2024
  
  b4e9528f
- [Core] Support serving encoder/decoder models (#7258) · 7eb4a51c
  Cyrus Leung authored Aug 09, 2024
  
  7eb4a51c
08 Aug, 2024 1 commit
- [Frontend] Kill the server on engine death (#6594) · 21b9c49a
  Joe Runde authored Aug 08, 2024
```
Signed-off-by: Joe Runde <joe@joerun.de>
Signed-off-by: Joe Runde <Joseph.Runde@ibm.com>
```
  21b9c49a
07 Aug, 2024 1 commit
- [BugFix] Overhaul async request cancellation (#7111) · 9a3f49ae
  Nick Hill authored Aug 06, 2024
  
  9a3f49ae
06 Aug, 2024 1 commit

[Core] Subclass ModelRunner to support cross-attention & encoder sequences... · fd95e026

afeldman-nm authored Aug 06, 2024


[Core] Subclass ModelRunner to support cross-attention & encoder sequences (towards eventual encoder/decoder model support) (#4942)
Co-authored-by: Andrew Feldman <afeld2012@gmail.com>
Co-authored-by: Nick Hill <nickhill@us.ibm.com>

fd95e026

05 Aug, 2024 2 commits
- [Core] Support loading GGUF model (#5191) · 360bd67c
  Isotr0py authored Aug 06, 2024
```
Co-authored-by: Michael Goin <michael@neuralmagic.com>
```
  360bd67c
- [Speculative decoding] Add periodic log with time spent in proposal/scoring/verification (#6963) · 82a1b1a8
  Cade Daniel authored Aug 05, 2024
  
  82a1b1a8
04 Aug, 2024 1 commit
- [core][misc] simply output processing with shortcut code path (#7117) · 83c644fe
  youkaichao authored Aug 04, 2024
  
  83c644fe
03 Aug, 2024 1 commit

[ Frontend ] Multiprocessing for OpenAI Server with `zeromq` (#6883) · ed812a73

Robert Shaw authored Aug 02, 2024


Signed-off-by: Joe Runde <Joseph.Runde@ibm.com>
Co-authored-by: Joe Runde <Joseph.Runde@ibm.com>
Co-authored-by: Joe Runde <joe@joerun.de>
Co-authored-by: Nick Hill <nickhill@us.ibm.com>
Co-authored-by: Simon Mo <simon.mo@hey.com>

ed812a73

31 Jul, 2024 1 commit
- [mypy] Enable following imports for some directories (#6681) · da1f7cc1
  Cyrus Leung authored Jul 31, 2024
  
  da1f7cc1
30 Jul, 2024 1 commit
- [Doc] Super tiny fix doc typo (#6949) · f0584036
  fzyzcjy authored Jul 31, 2024
  
  f0584036
29 Jul, 2024 1 commit
- [TPU] Support tensor parallelism in async llm engine (#6891) · 7f8d612d
  Earthwalker authored Jul 30, 2024
  
  7f8d612d
27 Jul, 2024 2 commits
- [Hardware][TPU] Implement tensor parallelism with Ray (#5871) · 52f07e3d
  Woosuk Kwon authored Jul 26, 2024
  
  52f07e3d
- [Bugfix][Model] Jamba assertions and no chunked prefill by default for Jamba (#6784) · ed94e4f4
  tomeras91 authored Jul 27, 2024
  
  ed94e4f4
26 Jul, 2024 1 commit
- [Hardware] [Intel] Enable Multiprocessing and tensor parallel in CPU backend... · 3bbb4936
  Li, Jiang authored Jul 27, 2024
```
[Hardware] [Intel] Enable Multiprocessing and tensor parallel in CPU backend and update documentation  (#6125)
```
  3bbb4936
24 Jul, 2024 1 commit
- [Bugfix] Miscalculated latency lead to time_to_first_token_seconds inaccurate. (#6686) · 40468b13
  Allen.Dou authored Jul 24, 2024
  
  40468b13
23 Jul, 2024 4 commits
- [bitsandbytes]: support read bnb pre-quantized model (#5753) · 87525fab
  dongmao zhang authored Jul 23, 2024
```
Co-authored-by: Michael Goin <michael@neuralmagic.com>
```
  87525fab
- [Bugfix] StatLoggers: cache spec decode metrics when they get collected. (#6645) · 2f808e69
  Thomas Parnell authored Jul 24, 2024
```
Signed-off-by: Thomas Parnell <tpa@zurich.ibm.com>
```
  2f808e69
- support ignore patterns in model loader (#6673) · 3eda4ec7
  Simon Mo authored Jul 22, 2024
  
  3eda4ec7
- [Misc] Enable chunked prefill by default for long context models (#6666) · 729171ae
  Woosuk Kwon authored Jul 22, 2024
  
  729171ae
22 Jul, 2024 1 commit
- [Frontend] Refactor prompt processing (#4028) · 739b61a3
  Cyrus Leung authored Jul 23, 2024
```
Co-authored-by: Roger Wang <ywang@roblox.com>
```
  739b61a3
21 Jul, 2024 1 commit
- [Spec Decode] Disable Log Prob serialization to CPU for spec decoding for both... · 14f91fe6
  sroy745 authored Jul 20, 2024
```
[Spec Decode] Disable Log Prob serialization to CPU for spec decoding for both draft and target models. (#6485)
```
  14f91fe6
20 Jul, 2024 2 commits
- Pipeline Parallel: Guard for KeyErrors at request abort (#6587) · 3f8d42c8
  Travis Johnson authored Jul 19, 2024
```
Signed-off-by: Travis Johnson <tsjohnso@us.ibm.com>
```
  3f8d42c8
- [Core] Allow specifying custom Executor (#6557) · 7bd82002
  Antoni Baum authored Jul 19, 2024
  
  7bd82002
18 Jul, 2024 3 commits
- [BugFix][Frontend] Use LoRA tokenizer in OpenAI APIs (#6227) · e2fbaee7
  Nick Hill authored Jul 18, 2024
```
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
```
  e2fbaee7
- [Core] Introduce SPMD worker execution using Ray accelerated DAG (#6032) · 61e59274
  Rui Qiao authored Jul 17, 2024
```
Signed-off-by: Rui Qiao <ruisearch42@gmail.com>
Co-authored-by: Stephanie Wang <swang@cs.berkeley.edu>
```
  61e59274
- [core][model] yet another cpu offload implementation (#6496) · 1c27d25f
  youkaichao authored Jul 17, 2024
```
Co-authored-by: Michael Goin <michael@neuralmagic.com>
```
  1c27d25f
17 Jul, 2024 1 commit
- [Bugfix] Fix Ray Metrics API usage (#6354) · 5f0b9933
  Antoni Baum authored Jul 17, 2024
  
  5f0b9933
16 Jul, 2024 2 commits
- [Misc] Log spec decode metrics (#6454) · 160e1d8c
  Cody Yu authored Jul 16, 2024
  
  160e1d8c
- [Misc] Fix typos in spec. decode metrics logging. (#6470) · 7508a3dc
  Thomas Parnell authored Jul 16, 2024
```
Signed-off-by: Thomas Parnell <tpa@zurich.ibm.com>
```
  7508a3dc
12 Jul, 2024 1 commit
- [Bugfix] Fix usage stats logging exception warning with OpenVINO (#6349) · adf32e0a
  Helena Kloosterman authored Jul 12, 2024
  
  adf32e0a
11 Jul, 2024 3 commits
- [ BugFix ] Prompt Logprobs Detokenization (#6223) · 7ed6a4f0
  Robert Shaw authored Jul 11, 2024
```
Co-authored-by: Zifei Tong <zifeitong@gmail.com>
```
  7ed6a4f0
- [BugFix] get_and_reset only when scheduler outputs are not empty (#6266) · 55f692b4
  Mor Zusman authored Jul 11, 2024
  
  55f692b4
- [BugFix]: fix engine timeout due to request abort (#6255) · 546b101f
  pushan authored Jul 11, 2024
```
Signed-off-by: yatta zhang <ytzhang01@foxmail.com>
Signed-off-by: zhangyuntao.dev <zhangyuntao.dev@bytedance.com>
Co-authored-by: zhangyuntao.dev <zhangyuntao.dev@bytedance.com>
```
  546b101f
09 Jul, 2024 1 commit

[CORE] Adding support for insertion of soft-tuned prompts (#4645) · 4d6ada94

Swapnil Parekh authored Jul 09, 2024


Co-authored-by: Swapnil Parekh <swapnilp@ibm.com>
Co-authored-by: Joe G <joseph.granados@h2o.ai>
Co-authored-by: Antoni Baum <antoni.baum@protonmail.com>

4d6ada94

03 Jul, 2024 2 commits
- [vlm] Remove vision language config. (#6089) · d9e98f42
  xwjiang2010 authored Jul 03, 2024
```
Signed-off-by: Xiaowei Jiang <xwjiang2010@gmail.com>
Co-authored-by: Roger Wang <ywang@roblox.com>
```
  d9e98f42
- [BugFix] Avoid unnecessary Ray import warnings (#6079) · d830656a
  Nick Hill authored Jul 02, 2024
  
  d830656a
02 Jul, 2024 3 commits

[Model] Jamba support (#4115) · 9d6a8daa

Mor Zusman authored Jul 03, 2024


Signed-off-by: Muralidhar Andoorveedu <muralidhar.andoorveedu@centml.ai>
Co-authored-by: Erez Schwartz <erezs@ai21.com>
Co-authored-by: Mor Zusman <morz@ai21.com>
Co-authored-by: tomeras91 <57313761+tomeras91@users.noreply.github.com>
Co-authored-by: Tomer Asida <tomera@ai21.com>
Co-authored-by: Zhuohan Li <zhuohan123@gmail.com>
Co-authored-by: Muralidhar Andoorveedu <muralidhar.andoorveedu@centml.ai>

9d6a8daa

[Core] Pipeline Parallel Support (#4412) · c5832d2a
Murali Andoorveedu authored Jul 02, 2024
```
Signed-off-by: Muralidhar Andoorveedu <muralidhar.andoorveedu@centml.ai>
```
c5832d2a

[VLM] Remove `image_input_type` from VLM config (#5852) · 98d6682c

xwjiang2010 authored Jul 02, 2024


Signed-off-by: Xiaowei Jiang <xwjiang2010@gmail.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
Co-authored-by: Roger Wang <ywang@roblox.com>

98d6682c