Commits · 705f6a35aa92800b9c69c8b346168c3055fdebbd · OpenDAS / vllm_cscc

15 Jul, 2024 1 commit
- [Bugfix] Add custom Triton cache manager to resolve MoE MP issue (#6140) · eaec4b91
  Thomas Parnell authored Jul 15, 2024
```
Signed-off-by: Thomas Parnell <tpa@zurich.ibm.com>
Co-authored-by: Chih-Chieh-Yang <chih.chieh.yang@ibm.com>
```
  eaec4b91
13 Jul, 2024 1 commit

[ci] try to add multi-node tests (#6280) · 41708e50

youkaichao authored Jul 12, 2024


Signed-off-by: Muralidhar Andoorveedu <muralidhar.andoorveedu@centml.ai>
Co-authored-by: Muralidhar Andoorveedu <muralidhar.andoorveedu@centml.ai>

41708e50

10 Jul, 2024 3 commits
- [Bugfix][Neuron] Fix soft prompt method error in NeuronExecutor (#6313) · 997df46a
  Woosuk Kwon authored Jul 10, 2024
  
  997df46a
- [Bugfix] Fix OpenVINOExecutor abstractmethod error (#6296) · 44cc7661
  sangjune.park authored Jul 11, 2024
```
Signed-off-by: sangjune.park <sangjune.park@navercorp.com>
```
  44cc7661
- [Bugfix][TPU] Add prompt adapter methods to TPUExecutor (#6279) · 5ed3505d
  Woosuk Kwon authored Jul 09, 2024
  
  5ed3505d
09 Jul, 2024 2 commits
- [CORE] Adding support for insertion of soft-tuned prompts (#4645) · 4d6ada94
  Swapnil Parekh authored Jul 09, 2024
```
Co-authored-by: Swapnil Parekh <swapnilp@ibm.com>
Co-authored-by: Joe G <joseph.granados@h2o.ai>
Co-authored-by: Antoni Baum <antoni.baum@protonmail.com>
```
  4d6ada94
- [core][distributed] fix ray worker rank assignment (#6235) · 70c232f8
  youkaichao authored Jul 08, 2024
  
  70c232f8
04 Jul, 2024 1 commit
- [Distributed][Core] Support Py39 and Py38 for PP (#6120) · 0ed646b7
  Murali Andoorveedu authored Jul 03, 2024
```
Signed-off-by: Muralidhar Andoorveedu <muralidhar.andoorveedu@centml.ai>
```
  0ed646b7
03 Jul, 2024 4 commits
- [Bugfix] set OMP_NUM_THREADS to 1 by default for multiprocessing (#6109) · 1dab9bc8
  Travis Johnson authored Jul 03, 2024
```
Signed-off-by: Travis Johnson <tsjohnso@us.ibm.com>
Co-authored-by: Nick Hill <nickhill@us.ibm.com>
```
  1dab9bc8
- [vlm] Remove vision language config. (#6089) · d9e98f42
  xwjiang2010 authored Jul 03, 2024
```
Signed-off-by: Xiaowei Jiang <xwjiang2010@gmail.com>
Co-authored-by: Roger Wang <ywang@roblox.com>
```
  d9e98f42
- [misc][distributed] error on invalid state (#6092) · f6662071
  youkaichao authored Jul 02, 2024
  
  f6662071
- [BugFix] Avoid unnecessary Ray import warnings (#6079) · d830656a
  Nick Hill authored Jul 02, 2024
  
  d830656a
02 Jul, 2024 1 commit
- [Core] Pipeline Parallel Support (#4412) · c5832d2a
  Murali Andoorveedu authored Jul 02, 2024
```
Signed-off-by: Muralidhar Andoorveedu <muralidhar.andoorveedu@centml.ai>
```
  c5832d2a
28 Jun, 2024 1 commit
- [Hardware][Intel] OpenVINO vLLM backend (#5379) · 57f09a41
  Ilya Lavrenov authored Jun 28, 2024
  
  57f09a41
26 Jun, 2024 2 commits
- [Core] Refactor Worker and ModelRunner to consolidate control plane communication (#5408) · dda48115
  Stephanie Wang authored Jun 25, 2024
```
Signed-off-by: Stephanie Wang <swang@cs.berkeley.edu>
Signed-off-by: Stephanie <swang@anyscale.com>
Co-authored-by: Stephanie <swang@anyscale.com>
```
  dda48115
- [Bugfix] Fix assertion in NeuronExecutor (#5841) · 82079729
  aws-patlange authored Jun 25, 2024
  
  82079729
25 Jun, 2024 2 commits
- [Hardware][AMD][CI/Build][Doc] Upgrade to ROCm 6.1, Dockerfile improvements, test fixes (#5422) · dd793d1d
  Matt Wong authored Jun 25, 2024
  
  dd793d1d
- [Hardware][TPU] Refactor TPU backend (#5831) · bc34937d
  Woosuk Kwon authored Jun 25, 2024
  
  bc34937d
22 Jun, 2024 1 commit
- [Bugfix] Fix pin_lora error in TPU executor (#5760) · 0cbc1d2b
  Woosuk Kwon authored Jun 21, 2024
  
  0cbc1d2b
21 Jun, 2024 1 commit
- [LoRA] Add support for pinning lora adapters in the LRU cache (#5603) · f5dda63e
  rohithkrn authored Jun 21, 2024
  
  f5dda63e
19 Jun, 2024 1 commit
- [misc][distributed] use 127.0.0.1 for single-node (#5619) · 3eea7488
  youkaichao authored Jun 19, 2024
  
  3eea7488
17 Jun, 2024 2 commits
- [bugfix][distributed] fix 16 gpus local rank arrangement (#5604) · 1b44aaf4
  youkaichao authored Jun 17, 2024
  
  1b44aaf4
- [Hardware][Intel GPU] Add Intel GPU(XPU) inference backend (#3814) · 728c4c8a
  Kunshang Ji authored Jun 18, 2024
```
Co-authored-by: Jiang Li <jiang1.li@intel.com>
Co-authored-by: Abhilash Majumder <abhilash.majumder@intel.com>
Co-authored-by: Abhilash Majumder <30946547+abhilash1910@users.noreply.github.com>
```
  728c4c8a
13 Jun, 2024 1 commit
- Add `cuda_device_count_stateless` (#5473) · 50eed24d
  Antoni Baum authored Jun 13, 2024
  
  50eed24d
12 Jun, 2024 1 commit
- [Hardware] Initial TPU integration (#5292) · 1a8bfd92
  Woosuk Kwon authored Jun 12, 2024
  
  1a8bfd92
11 Jun, 2024 2 commits
- [Core][Doc] Default to multiprocessing for single-node distributed case (#5230) · 99dac099
  Nick Hill authored Jun 11, 2024
```
Co-authored-by: Antoni Baum <antoni.baum@protonmail.com>
```
  99dac099
- [Bugfix] Fix `MultiprocessingGPUExecutor.check_health` when world_size == 1 (#5254) · 2e02311a
  Junichi Sato authored Jun 12, 2024
  
  2e02311a
07 Jun, 2024 1 commit
- Remove Ray health check (#4693) · 18a277b5
  Antoni Baum authored Jun 07, 2024
  
  18a277b5
04 Jun, 2024 1 commit
- [Bugfix] Fix torch.compile() error when using MultiprocessingGPUExecutor (#5229) · a58f24e5
  zifeitong authored Jun 03, 2024
  
  a58f24e5
22 May, 2024 1 commit
- [Core] Eliminate parallel worker per-step task scheduling overhead (#4894) · eb6d3c26
  Nick Hill authored May 22, 2024
  
  eb6d3c26
16 May, 2024 2 commits
- [Speculative decoding][Re-take] Enable TP>1 speculative decoding (#4840) · 973617ae
  Cody Yu authored May 16, 2024
```
Co-authored-by: Cade Daniel <edacih@gmail.com>
Co-authored-by: Cade Daniel <cade@anyscale.com>
```
  973617ae
- [Core] Implement sharded state loader (#4690) · 30e75439
  Aurick Qiao authored May 16, 2024
```
Co-authored-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  30e75439
14 May, 2024 1 commit
- [Core] Add MultiprocessingGPUExecutor (#4539) · 676a9998
  Nick Hill authored May 14, 2024
```
Co-authored-by: SAHIL SUNEJA <suneja@us.ibm.com>
```
  676a9998
11 May, 2024 1 commit
- [Model][Misc] Add e5-mistral-7b-instruct and Embedding API (#3734) · e254497b
  Chang Su authored May 11, 2024
  
  e254497b
08 May, 2024 1 commit
- [Dynamic Spec Decoding] Auto-disable by the running queue size (#4592) · f942efb5
  Cody Yu authored May 08, 2024
```
Co-authored-by: Cade Daniel <edacih@gmail.com>
```
  f942efb5
07 May, 2024 1 commit

[Bug fix][Core] fixup ngram not setup correctly (#4551) · 8344f774

leiwen83 authored May 08, 2024


Co-authored-by: Lei Wen <wenlei03@qiyi.com>
Co-authored-by: Cade Daniel <edacih@gmail.com>
Co-authored-by: Cody Yu <hao.yu.cody@gmail.com>

8344f774

04 May, 2024 1 commit
- [Misc][Refactor] Introduce ExecuteModelData (#4540) · bc8ad684
  Cody Yu authored May 03, 2024
  
  bc8ad684
02 May, 2024 1 commit
- [Misc] centralize all usage of environment variables (#4548) · 5b8a7c1c
  youkaichao authored May 02, 2024
  
  5b8a7c1c
01 May, 2024 2 commits
- [Core] Add `multiproc_worker_utils` for multiprocessing-based workers (#4357) · a657bfc4
  Nick Hill authored May 01, 2024
  
  a657bfc4
- [Speculative decoding] Add ngram prompt lookup decoding (#4237) · b38e42fb
  leiwen83 authored May 02, 2024
```
Co-authored-by: Lei Wen <wenlei03@qiyi.com>
```
  b38e42fb