Commits · d25877dd9b7a09d5f9552d9d185a86dc6497bc2f · OpenDAS / vllm_cscc · GitLab

17 Jul, 2024 1 commit
- [Bugfix] Fix for multinode crash on 4 PP (#6495) · 5fa6e987
  Murali Andoorveedu authored Jul 17, 2024
```
Signed-off-by: Muralidhar Andoorveedu <muralidhar.andoorveedu@centml.ai>
```
  5fa6e987
13 Jul, 2024 1 commit

[ci] try to add multi-node tests (#6280) · 41708e50

youkaichao authored Jul 12, 2024


Signed-off-by: Muralidhar Andoorveedu <muralidhar.andoorveedu@centml.ai>
Co-authored-by: Muralidhar Andoorveedu <muralidhar.andoorveedu@centml.ai>

41708e50

09 Jul, 2024 1 commit
- [core][distributed] fix ray worker rank assignment (#6235) · 70c232f8
  youkaichao authored Jul 08, 2024
  
  70c232f8
04 Jul, 2024 1 commit
- [Distributed][Core] Support Py39 and Py38 for PP (#6120) · 0ed646b7
  Murali Andoorveedu authored Jul 03, 2024
```
Signed-off-by: Muralidhar Andoorveedu <muralidhar.andoorveedu@centml.ai>
```
  0ed646b7
03 Jul, 2024 1 commit
- [misc][distributed] error on invalid state (#6092) · f6662071
  youkaichao authored Jul 02, 2024
  
  f6662071
02 Jul, 2024 1 commit
- [Core] Pipeline Parallel Support (#4412) · c5832d2a
  Murali Andoorveedu authored Jul 02, 2024
```
Signed-off-by: Muralidhar Andoorveedu <muralidhar.andoorveedu@centml.ai>
```
  c5832d2a
26 Jun, 2024 1 commit

[Core] Refactor Worker and ModelRunner to consolidate control plane communication (#5408) · dda48115

Stephanie Wang authored Jun 25, 2024


Signed-off-by: Stephanie Wang <swang@cs.berkeley.edu>
Signed-off-by: Stephanie <swang@anyscale.com>
Co-authored-by: Stephanie <swang@anyscale.com>

dda48115

19 Jun, 2024 1 commit
- [misc][distributed] use 127.0.0.1 for single-node (#5619) · 3eea7488
  youkaichao authored Jun 19, 2024
  
  3eea7488
17 Jun, 2024 1 commit
- [bugfix][distributed] fix 16 gpus local rank arrangement (#5604) · 1b44aaf4
  youkaichao authored Jun 17, 2024
  
  1b44aaf4
07 Jun, 2024 1 commit
- Remove Ray health check (#4693) · 18a277b5
  Antoni Baum authored Jun 07, 2024
  
  18a277b5
22 May, 2024 1 commit
- [Core] Eliminate parallel worker per-step task scheduling overhead (#4894) · eb6d3c26
  Nick Hill authored May 22, 2024
  
  eb6d3c26
16 May, 2024 1 commit
- [Speculative decoding][Re-take] Enable TP>1 speculative decoding (#4840) · 973617ae
  Cody Yu authored May 16, 2024
```
Co-authored-by: Cade Daniel <edacih@gmail.com>
Co-authored-by: Cade Daniel <cade@anyscale.com>
```
  973617ae
14 May, 2024 1 commit
- [Core] Add MultiprocessingGPUExecutor (#4539) · 676a9998
  Nick Hill authored May 14, 2024
```
Co-authored-by: SAHIL SUNEJA <suneja@us.ibm.com>
```
  676a9998
04 May, 2024 1 commit
- [Misc][Refactor] Introduce ExecuteModelData (#4540) · bc8ad684
  Cody Yu authored May 03, 2024
  
  bc8ad684
02 May, 2024 1 commit
- [Misc] centralize all usage of environment variables (#4548) · 5b8a7c1c
  youkaichao authored May 02, 2024
  
  5b8a7c1c
01 May, 2024 1 commit
- [Core] Centralize GPU Worker construction (#4419) · 2e240c69
  Nick Hill authored Apr 30, 2024
  
  2e240c69
30 Apr, 2024 1 commit
- [BugFix] fix num_lookahead_slots missing in async executor (#4165) · 4bb53e2d
  leiwen83 authored May 01, 2024
```
Co-authored-by: Lei Wen <wenlei03@qiyi.com>
```
  4bb53e2d
27 Apr, 2024 2 commits
- [BugFix] Fix return type of executor execute_model methods (#4402) · ba4be44c
  Nick Hill authored Apr 27, 2024
  
  ba4be44c
- [Core] Introduce `DistributedGPUExecutor` abstract class (#4348) · 258a2c58
  Nick Hill authored Apr 26, 2024
  
  258a2c58
26 Apr, 2024 1 commit
- [CI] Disable non-lazy string operation on logging (#4326) · a88081bf
  SangBin Cho authored Apr 26, 2024
```
Co-authored-by: Danny Guinther <dguinther@neuralmagic.com>
```
  a88081bf
25 Apr, 2024 1 commit
- [Core] Move ray_utils.py from `engine` to `executor` package (#4347) · 479d69fa
  Nick Hill authored Apr 24, 2024
  
  479d69fa
23 Apr, 2024 2 commits
- [Bugfix] Add init_cached_hf_modules to RayWorkerWrapper (#4286) · d87f39e9
  DefTruth authored Apr 24, 2024
  
  d87f39e9
- [Core] Some simplification of WorkerWrapper changes (#4183) · 8f2ea22b
  Nick Hill authored Apr 23, 2024
  
  8f2ea22b
18 Apr, 2024 1 commit

[Core] add an option to log every function call to for debugging hang/crash in... · 8a7a3e44

youkaichao authored Apr 18, 2024


[Core] add an option to log every function call to for debugging hang/crash in distributed inference (#4079)
Co-authored-by: Simon Mo <simon.mo@hey.com>

8a7a3e44

17 Apr, 2024 1 commit
- [Core] RayWorkerVllm --> WorkerWrapper to reduce duplication (#4024) · 8438e056
  youkaichao authored Apr 17, 2024
```
[Core] replace narrow-usage RayWorkerVllm to general WorkerWrapper to reduce code duplication (#4024)
```
  8438e056
16 Apr, 2024 2 commits
- [Speculative decoding 6/9] Integrate speculative decoding with LLMEngine (#3894) · e95cd879
  Cade Daniel authored Apr 16, 2024
  
  e95cd879
- [Core] Refactor model loading code (#4097) · 69e1d2fb
  Antoni Baum authored Apr 16, 2024
  
  69e1d2fb
15 Apr, 2024 2 commits
- [Bugfix] Fix ray workers profiling with nsight (#4095) · 4695397d
  Ricky Xu authored Apr 15, 2024
  
  4695397d
- [Core] Simplifications to executor classes (#4071) · eb46fbfd
  Nick Hill authored Apr 15, 2024
  
  eb46fbfd
14 Apr, 2024 1 commit
- [Frontend] [Core] feat: Add model loading using `tensorizer` (#3476) · 711a0002
  Sanger Steel authored Apr 13, 2024
  
  711a0002
12 Apr, 2024 1 commit
- [mypy] Add mypy type annotation part 1 (#4006) · 09473ee4
  SangBin Cho authored Apr 13, 2024
  
  09473ee4
09 Apr, 2024 1 commit
- [Misc] [Core] Implement RFC "Augment BaseExecutor interfaces to enable... · e7c7067b
  Cade Daniel authored Apr 09, 2024
```
[Misc] [Core] Implement RFC "Augment BaseExecutor interfaces to enable hardware-agnostic speculative decoding" (#3837)
```
  e7c7067b
07 Apr, 2024 1 commit
- [Bugfix] Fix Llava inference with Tensor Parallelism. (#3883) · 0ce0539d
  Isotr0py authored Apr 07, 2024
  
  0ce0539d
03 Apr, 2024 1 commit
- [Speculative decoding] Adding configuration object for speculative decoding (#3706) · 5757d90e
  Cade Daniel authored Apr 02, 2024
```
Co-authored-by: Lily Liu <lilyliupku@gmail.com>
```
  5757d90e
28 Mar, 2024 2 commits
- [Core] Support multi-node inference(eager and cuda graph) (#3686) · 515386ef
  Roy authored Mar 29, 2024
  
  515386ef
- [Core][Bugfix]Refactor block manager for better testability (#3492) · 14ccd94c
  Cade Daniel authored Mar 27, 2024
  
  14ccd94c
27 Mar, 2024 1 commit
- [Core] remove cupy dependency (#3625) · 8f44facd
  youkaichao authored Mar 27, 2024
  
  8f44facd
25 Mar, 2024 2 commits
- [Feature] Add vision language model support. (#3042) · 64172a97
  xwjiang2010 authored Mar 25, 2024
  
  64172a97
- [CI] Try introducing isort. (#3495) · 01bfb22b
  SangBin Cho authored Mar 25, 2024
  
  01bfb22b
22 Mar, 2024 1 commit
- [Hardware][Neuron] Refactor neuron support (#3471) · e90fc21f
  Zhuohan Li authored Mar 21, 2024
  
  e90fc21f