Commits · 4d2dc5072b8cfaff0c02e504c79a72ee95d839a1 · OpenDAS / vllm_cscc · GitLab

13 Aug, 2024 2 commits
- [hardware] unify usage of is_tpu to current_platform.is_tpu() (#7102) · 4d2dc507
  youkaichao authored Aug 13, 2024
  
  4d2dc507
- [Core] Shut down aDAG workers with clean async llm engine exit (#7224) · 198d6a28
  Rui Qiao authored Aug 12, 2024
```
Signed-off-by: Rui Qiao <ruisearch42@gmail.com>
```
  198d6a28
12 Aug, 2024 1 commit
- [Core] Consolidate `GB` constant and enable float GB arguments (#7416) · 4ddc4743
  Cyrus Leung authored Aug 13, 2024
  
  4ddc4743
09 Aug, 2024 1 commit
- [Core] Add span metrics for model_forward, scheduler and sampler time (#7089) · 933790c2
  Mahesh Keralapura authored Aug 09, 2024
  
  933790c2
02 Aug, 2024 3 commits
- [Misc] Revive to use loopback address for driver IP (#7091) · 22e718ff
  Rui Qiao authored Aug 02, 2024
```
Signed-off-by: Rui Qiao <ruisearch42@gmail.com>
```
  22e718ff
- [Core] Pipeline parallel with Ray ADAG (#6837) · 05308891
  Rui Qiao authored Aug 02, 2024
```
Support pipeline-parallelism with Ray accelerated DAG.
Signed-off-by: Rui Qiao <ruisearch42@gmail.com>
```
  05308891
- [cuda][misc] remove error_on_invalid_device_count_status (#7069) · 660dea12
  youkaichao authored Aug 02, 2024
  
  660dea12
27 Jul, 2024 2 commits
- [Bugfix] torch.set_num_threads() in multiproc_gpu_executor (#6802) · 593e79e7
  Travis Johnson authored Jul 26, 2024
```
[Bugfix] Use torch.set_num_threads() to configure parallelism in multiproc_gpu_executor (#6802)
Signed-off-by: Travis Johnson <tsjohnso@us.ibm.com>
```
  593e79e7
- [Hardware][TPU] Implement tensor parallelism with Ray (#5871) · 52f07e3d
  Woosuk Kwon authored Jul 26, 2024
  
  52f07e3d
26 Jul, 2024 3 commits
- [Hardware] [Intel] Enable Multiprocessing and tensor parallel in CPU backend... · 3bbb4936
  Li, Jiang authored Jul 27, 2024
```
[Hardware] [Intel] Enable Multiprocessing and tensor parallel in CPU backend and update documentation  (#6125)
```
  3bbb4936
- [Misc][TPU] Support TPU in initialize_ray_cluster (#6812) · aa486779
  Woosuk Kwon authored Jul 26, 2024
  
  aa486779
- [Bugfix] [Easy] Fixed a bug in the multiprocessing GPU executor. (#6770) · 084a01fd
  Anthony Platanios authored Jul 26, 2024
  
  084a01fd
25 Jul, 2024 1 commit
- [Core] Fix ray forward_dag error mssg (#6792) · 1adddb14
  SangBin Cho authored Jul 25, 2024
  
  1adddb14
20 Jul, 2024 1 commit
- [Core] Allow specifying custom Executor (#6557) · 7bd82002
  Antoni Baum authored Jul 19, 2024
  
  7bd82002
19 Jul, 2024 1 commit
- [Core] Multiprocessing Pipeline Parallel support (#6130) · b5672a11
  Nick Hill authored Jul 18, 2024
```
Co-authored-by: Murali Andoorveedu <muralidhar.andoorveedu@centml.ai>
```
  b5672a11
18 Jul, 2024 1 commit
- [Core] Introduce SPMD worker execution using Ray accelerated DAG (#6032) · 61e59274
  Rui Qiao authored Jul 17, 2024
```
Signed-off-by: Rui Qiao <ruisearch42@gmail.com>
Co-authored-by: Stephanie Wang <swang@cs.berkeley.edu>
```
  61e59274
17 Jul, 2024 1 commit
- [Bugfix] Fix for multinode crash on 4 PP (#6495) · 5fa6e987
  Murali Andoorveedu authored Jul 17, 2024
```
Signed-off-by: Muralidhar Andoorveedu <muralidhar.andoorveedu@centml.ai>
```
  5fa6e987
16 Jul, 2024 1 commit
- [ci][distributed] add pipeline parallel correctness test (#6410) · 09c2eb85
  youkaichao authored Jul 16, 2024
  
  09c2eb85
15 Jul, 2024 1 commit
- [Bugfix] Add custom Triton cache manager to resolve MoE MP issue (#6140) · eaec4b91
  Thomas Parnell authored Jul 15, 2024
```
Signed-off-by: Thomas Parnell <tpa@zurich.ibm.com>
Co-authored-by: Chih-Chieh-Yang <chih.chieh.yang@ibm.com>
```
  eaec4b91
13 Jul, 2024 1 commit

[ci] try to add multi-node tests (#6280) · 41708e50

youkaichao authored Jul 12, 2024


Signed-off-by: Muralidhar Andoorveedu <muralidhar.andoorveedu@centml.ai>
Co-authored-by: Muralidhar Andoorveedu <muralidhar.andoorveedu@centml.ai>

41708e50

10 Jul, 2024 3 commits
- [Bugfix][Neuron] Fix soft prompt method error in NeuronExecutor (#6313) · 997df46a
  Woosuk Kwon authored Jul 10, 2024
  
  997df46a
- [Bugfix] Fix OpenVINOExecutor abstractmethod error (#6296) · 44cc7661
  sangjune.park authored Jul 11, 2024
```
Signed-off-by: sangjune.park <sangjune.park@navercorp.com>
```
  44cc7661
- [Bugfix][TPU] Add prompt adapter methods to TPUExecutor (#6279) · 5ed3505d
  Woosuk Kwon authored Jul 09, 2024
  
  5ed3505d
09 Jul, 2024 2 commits
- [CORE] Adding support for insertion of soft-tuned prompts (#4645) · 4d6ada94
  Swapnil Parekh authored Jul 09, 2024
```
Co-authored-by: Swapnil Parekh <swapnilp@ibm.com>
Co-authored-by: Joe G <joseph.granados@h2o.ai>
Co-authored-by: Antoni Baum <antoni.baum@protonmail.com>
```
  4d6ada94
- [core][distributed] fix ray worker rank assignment (#6235) · 70c232f8
  youkaichao authored Jul 08, 2024
  
  70c232f8
04 Jul, 2024 1 commit
- [Distributed][Core] Support Py39 and Py38 for PP (#6120) · 0ed646b7
  Murali Andoorveedu authored Jul 03, 2024
```
Signed-off-by: Muralidhar Andoorveedu <muralidhar.andoorveedu@centml.ai>
```
  0ed646b7
03 Jul, 2024 4 commits
- [Bugfix] set OMP_NUM_THREADS to 1 by default for multiprocessing (#6109) · 1dab9bc8
  Travis Johnson authored Jul 03, 2024
```
Signed-off-by: Travis Johnson <tsjohnso@us.ibm.com>
Co-authored-by: Nick Hill <nickhill@us.ibm.com>
```
  1dab9bc8
- [vlm] Remove vision language config. (#6089) · d9e98f42
  xwjiang2010 authored Jul 03, 2024
```
Signed-off-by: Xiaowei Jiang <xwjiang2010@gmail.com>
Co-authored-by: Roger Wang <ywang@roblox.com>
```
  d9e98f42
- [misc][distributed] error on invalid state (#6092) · f6662071
  youkaichao authored Jul 02, 2024
  
  f6662071
- [BugFix] Avoid unnecessary Ray import warnings (#6079) · d830656a
  Nick Hill authored Jul 02, 2024
  
  d830656a
02 Jul, 2024 1 commit
- [Core] Pipeline Parallel Support (#4412) · c5832d2a
  Murali Andoorveedu authored Jul 02, 2024
```
Signed-off-by: Muralidhar Andoorveedu <muralidhar.andoorveedu@centml.ai>
```
  c5832d2a
28 Jun, 2024 1 commit
- [Hardware][Intel] OpenVINO vLLM backend (#5379) · 57f09a41
  Ilya Lavrenov authored Jun 28, 2024
  
  57f09a41
26 Jun, 2024 2 commits
- [Core] Refactor Worker and ModelRunner to consolidate control plane communication (#5408) · dda48115
  Stephanie Wang authored Jun 25, 2024
```
Signed-off-by: Stephanie Wang <swang@cs.berkeley.edu>
Signed-off-by: Stephanie <swang@anyscale.com>
Co-authored-by: Stephanie <swang@anyscale.com>
```
  dda48115
- [Bugfix] Fix assertion in NeuronExecutor (#5841) · 82079729
  aws-patlange authored Jun 25, 2024
  
  82079729
25 Jun, 2024 2 commits
- [Hardware][AMD][CI/Build][Doc] Upgrade to ROCm 6.1, Dockerfile improvements, test fixes (#5422) · dd793d1d
  Matt Wong authored Jun 25, 2024
  
  dd793d1d
- [Hardware][TPU] Refactor TPU backend (#5831) · bc34937d
  Woosuk Kwon authored Jun 25, 2024
  
  bc34937d
22 Jun, 2024 1 commit
- [Bugfix] Fix pin_lora error in TPU executor (#5760) · 0cbc1d2b
  Woosuk Kwon authored Jun 21, 2024
  
  0cbc1d2b
21 Jun, 2024 1 commit
- [LoRA] Add support for pinning lora adapters in the LRU cache (#5603) · f5dda63e
  rohithkrn authored Jun 21, 2024
  
  f5dda63e
19 Jun, 2024 1 commit
- [misc][distributed] use 127.0.0.1 for single-node (#5619) · 3eea7488
  youkaichao authored Jun 19, 2024
  
  3eea7488
17 Jun, 2024 1 commit
- [bugfix][distributed] fix 16 gpus local rank arrangement (#5604) · 1b44aaf4
  youkaichao authored Jun 17, 2024
  
  1b44aaf4