Commits · 198d6a289849bc64a17555e0cbb15a197a1229cb · OpenDAS / vllm_cscc · GitLab

13 Aug, 2024 1 commit
- [Core] Shut down aDAG workers with clean async llm engine exit (#7224) · 198d6a28
  Rui Qiao authored Aug 12, 2024
```
Signed-off-by: Rui Qiao <ruisearch42@gmail.com>
```
  198d6a28
02 Aug, 2024 3 commits
- [Misc] Revive to use loopback address for driver IP (#7091) · 22e718ff
  Rui Qiao authored Aug 02, 2024
```
Signed-off-by: Rui Qiao <ruisearch42@gmail.com>
```
  22e718ff
- [Core] Pipeline parallel with Ray ADAG (#6837) · 05308891
  Rui Qiao authored Aug 02, 2024
```
Support pipeline-parallelism with Ray accelerated DAG.
Signed-off-by: Rui Qiao <ruisearch42@gmail.com>
```
  05308891
- [cuda][misc] remove error_on_invalid_device_count_status (#7069) · 660dea12
  youkaichao authored Aug 02, 2024
  
  660dea12
25 Jul, 2024 1 commit
- [Core] Fix ray forward_dag error mssg (#6792) · 1adddb14
  SangBin Cho authored Jul 25, 2024
  
  1adddb14
20 Jul, 2024 1 commit
- [Core] Allow specifying custom Executor (#6557) · 7bd82002
  Antoni Baum authored Jul 19, 2024
  
  7bd82002
19 Jul, 2024 1 commit
- [Core] Multiprocessing Pipeline Parallel support (#6130) · b5672a11
  Nick Hill authored Jul 18, 2024
```
Co-authored-by: Murali Andoorveedu <muralidhar.andoorveedu@centml.ai>
```
  b5672a11
18 Jul, 2024 1 commit
- [Core] Introduce SPMD worker execution using Ray accelerated DAG (#6032) · 61e59274
  Rui Qiao authored Jul 17, 2024
```
Signed-off-by: Rui Qiao <ruisearch42@gmail.com>
Co-authored-by: Stephanie Wang <swang@cs.berkeley.edu>
```
  61e59274
17 Jul, 2024 1 commit
- [Bugfix] Fix for multinode crash on 4 PP (#6495) · 5fa6e987
  Murali Andoorveedu authored Jul 17, 2024
```
Signed-off-by: Muralidhar Andoorveedu <muralidhar.andoorveedu@centml.ai>
```
  5fa6e987
13 Jul, 2024 1 commit

[ci] try to add multi-node tests (#6280) · 41708e50

youkaichao authored Jul 12, 2024


Signed-off-by: Muralidhar Andoorveedu <muralidhar.andoorveedu@centml.ai>
Co-authored-by: Muralidhar Andoorveedu <muralidhar.andoorveedu@centml.ai>

41708e50

09 Jul, 2024 1 commit
- [core][distributed] fix ray worker rank assignment (#6235) · 70c232f8
  youkaichao authored Jul 08, 2024
  
  70c232f8
04 Jul, 2024 1 commit
- [Distributed][Core] Support Py39 and Py38 for PP (#6120) · 0ed646b7
  Murali Andoorveedu authored Jul 03, 2024
```
Signed-off-by: Muralidhar Andoorveedu <muralidhar.andoorveedu@centml.ai>
```
  0ed646b7
03 Jul, 2024 1 commit
- [misc][distributed] error on invalid state (#6092) · f6662071
  youkaichao authored Jul 02, 2024
  
  f6662071
02 Jul, 2024 1 commit
- [Core] Pipeline Parallel Support (#4412) · c5832d2a
  Murali Andoorveedu authored Jul 02, 2024
```
Signed-off-by: Muralidhar Andoorveedu <muralidhar.andoorveedu@centml.ai>
```
  c5832d2a
26 Jun, 2024 1 commit

[Core] Refactor Worker and ModelRunner to consolidate control plane communication (#5408) · dda48115

Stephanie Wang authored Jun 25, 2024


Signed-off-by: Stephanie Wang <swang@cs.berkeley.edu>
Signed-off-by: Stephanie <swang@anyscale.com>
Co-authored-by: Stephanie <swang@anyscale.com>

dda48115

19 Jun, 2024 1 commit
- [misc][distributed] use 127.0.0.1 for single-node (#5619) · 3eea7488
  youkaichao authored Jun 19, 2024
  
  3eea7488
17 Jun, 2024 1 commit
- [bugfix][distributed] fix 16 gpus local rank arrangement (#5604) · 1b44aaf4
  youkaichao authored Jun 17, 2024
  
  1b44aaf4
07 Jun, 2024 1 commit
- Remove Ray health check (#4693) · 18a277b5
  Antoni Baum authored Jun 07, 2024
  
  18a277b5
22 May, 2024 1 commit
- [Core] Eliminate parallel worker per-step task scheduling overhead (#4894) · eb6d3c26
  Nick Hill authored May 22, 2024
  
  eb6d3c26
16 May, 2024 1 commit
- [Speculative decoding][Re-take] Enable TP>1 speculative decoding (#4840) · 973617ae
  Cody Yu authored May 16, 2024
```
Co-authored-by: Cade Daniel <edacih@gmail.com>
Co-authored-by: Cade Daniel <cade@anyscale.com>
```
  973617ae
14 May, 2024 1 commit
- [Core] Add MultiprocessingGPUExecutor (#4539) · 676a9998
  Nick Hill authored May 14, 2024
```
Co-authored-by: SAHIL SUNEJA <suneja@us.ibm.com>
```
  676a9998
04 May, 2024 1 commit
- [Misc][Refactor] Introduce ExecuteModelData (#4540) · bc8ad684
  Cody Yu authored May 03, 2024
  
  bc8ad684
02 May, 2024 1 commit
- [Misc] centralize all usage of environment variables (#4548) · 5b8a7c1c
  youkaichao authored May 02, 2024
  
  5b8a7c1c
01 May, 2024 1 commit
- [Core] Centralize GPU Worker construction (#4419) · 2e240c69
  Nick Hill authored Apr 30, 2024
  
  2e240c69
30 Apr, 2024 1 commit
- [BugFix] fix num_lookahead_slots missing in async executor (#4165) · 4bb53e2d
  leiwen83 authored May 01, 2024
```
Co-authored-by: Lei Wen <wenlei03@qiyi.com>
```
  4bb53e2d
27 Apr, 2024 2 commits
- [BugFix] Fix return type of executor execute_model methods (#4402) · ba4be44c
  Nick Hill authored Apr 27, 2024
  
  ba4be44c
- [Core] Introduce `DistributedGPUExecutor` abstract class (#4348) · 258a2c58
  Nick Hill authored Apr 26, 2024
  
  258a2c58
26 Apr, 2024 1 commit
- [CI] Disable non-lazy string operation on logging (#4326) · a88081bf
  SangBin Cho authored Apr 26, 2024
```
Co-authored-by: Danny Guinther <dguinther@neuralmagic.com>
```
  a88081bf
25 Apr, 2024 1 commit
- [Core] Move ray_utils.py from `engine` to `executor` package (#4347) · 479d69fa
  Nick Hill authored Apr 24, 2024
  
  479d69fa
23 Apr, 2024 2 commits
- [Bugfix] Add init_cached_hf_modules to RayWorkerWrapper (#4286) · d87f39e9
  DefTruth authored Apr 24, 2024
  
  d87f39e9
- [Core] Some simplification of WorkerWrapper changes (#4183) · 8f2ea22b
  Nick Hill authored Apr 23, 2024
  
  8f2ea22b
18 Apr, 2024 1 commit

[Core] add an option to log every function call to for debugging hang/crash in... · 8a7a3e44

youkaichao authored Apr 18, 2024


[Core] add an option to log every function call to for debugging hang/crash in distributed inference (#4079)
Co-authored-by: Simon Mo <simon.mo@hey.com>

8a7a3e44

17 Apr, 2024 1 commit
- [Core] RayWorkerVllm --> WorkerWrapper to reduce duplication (#4024) · 8438e056
  youkaichao authored Apr 17, 2024
```
[Core] replace narrow-usage RayWorkerVllm to general WorkerWrapper to reduce code duplication (#4024)
```
  8438e056
16 Apr, 2024 2 commits
- [Speculative decoding 6/9] Integrate speculative decoding with LLMEngine (#3894) · e95cd879
  Cade Daniel authored Apr 16, 2024
  
  e95cd879
- [Core] Refactor model loading code (#4097) · 69e1d2fb
  Antoni Baum authored Apr 16, 2024
  
  69e1d2fb
15 Apr, 2024 2 commits
- [Bugfix] Fix ray workers profiling with nsight (#4095) · 4695397d
  Ricky Xu authored Apr 15, 2024
  
  4695397d
- [Core] Simplifications to executor classes (#4071) · eb46fbfd
  Nick Hill authored Apr 15, 2024
  
  eb46fbfd
14 Apr, 2024 1 commit
- [Frontend] [Core] feat: Add model loading using `tensorizer` (#3476) · 711a0002
  Sanger Steel authored Apr 13, 2024
  
  711a0002
12 Apr, 2024 1 commit
- [mypy] Add mypy type annotation part 1 (#4006) · 09473ee4
  SangBin Cho authored Apr 13, 2024
  
  09473ee4
09 Apr, 2024 1 commit
- [Misc] [Core] Implement RFC "Augment BaseExecutor interfaces to enable... · e7c7067b
  Cade Daniel authored Apr 09, 2024
```
[Misc] [Core] Implement RFC "Augment BaseExecutor interfaces to enable hardware-agnostic speculative decoding" (#3837)
```
  e7c7067b