Commits · afd0da2186c1d58fb48e138df0a2f548612b5d7d · OpenDAS / vllm_cscc · GitLab

27 Jan, 2025 1 commit

[FlashInfer] Upgrade to 0.2.0 (#11194) · 2bc3fbba

Bowen Wang authored Jan 28, 2025


Signed-off-by: Bowen Wang <abmfy@icloud.com>
Signed-off-by: youkaichao <youkaichao@gmail.com>
Co-authored-by: youkaichao <youkaichao@gmail.com>

2bc3fbba

20 Jan, 2025 2 commits
- [core][bugfix] configure env var during import vllm (#12209) · c222f479
  youkaichao authored Jan 20, 2025
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  c222f479
- [Core] Interface for accessing model from `VllmRunner` (#10353) · 59a0192f
  Cyrus Leung authored Jan 20, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  59a0192f
17 Jan, 2025 1 commit
- [core] allow callable in collective_rpc (#12151) · 87a0c076
  youkaichao authored Jan 17, 2025
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  87a0c076
16 Jan, 2025 2 commits
- [core] LLM.collective_rpc interface and RLHF example (#12084) · 92e793d9
  youkaichao authored Jan 16, 2025
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  92e793d9
- Support torchrun and SPMD-style offline inference (#12071) · bf53e0c7
  youkaichao authored Jan 16, 2025
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  bf53e0c7
15 Jan, 2025 1 commit
- [core] platform agnostic executor via collective_rpc (#11256) · ad34c0df
  youkaichao authored Jan 15, 2025
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  ad34c0df
10 Jan, 2025 1 commit
- [torch.compile] Hide KV cache behind torch.compile boundary (#11677) · cf5f000d
  Chen Zhang authored Jan 10, 2025
```
Signed-off-by: Chen Zhang <zhangch99@outlook.com>
```
  cf5f000d
03 Jan, 2025 1 commit
- skip ggml_dequantize · 28a8a733
  zhuwenwen authored Jan 03, 2025
  
  28a8a733
30 Dec, 2024 1 commit
- [platforms] enable platform plugins (#11602) · b12e87f9
  youkaichao authored Dec 30, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  b12e87f9
23 Dec, 2024 1 commit

mypy type checking for vllm/worker (#11418) · e51719ae

Lucas Tucker authored Dec 23, 2024


Signed-off-by: lucast2021 <lucast2021@headroyce.org>
Co-authored-by: lucast2021 <lucast2021@headroyce.org>

e51719ae

11 Dec, 2024 1 commit
- [feat]优化medusa代码，通过VLLM_TREE_DECODING环境变量控制是否采用tree-style解码，计算逻辑主干隔离 · 44e3ca68
  王敏 authored Dec 11, 2024
  
  44e3ca68
07 Dec, 2024 1 commit
- [core][executor] simplify instance id (#10976) · 1b62745b
  youkaichao authored Dec 07, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  1b62745b
02 Dec, 2024 1 commit

[Core] Implement disagg prefill by StatelessProcessGroup (#10502) · 0590ec3f

Kuntai Du authored Dec 01, 2024



This PR provides initial support for single-node disaggregated prefill in 1P1D scenario.
Signed-off-by: KuntaiDu <kuntai@uchicago.edu>
Co-authored-by: ApostaC <yihua98@uchicago.edu>
Co-authored-by: YaoJiayi <120040070@link.cuhk.edu.cn>

0590ec3f

27 Nov, 2024 2 commits
- 1.优化medusa推理，节省cpu耗时 · 28375803
  王敏 authored Nov 27, 2024
```
2.更新medusa readme
3.解决benchmark_moe报错问题
```
  28375803
- Remove hard-dependencies of Speculative decode to CUDA workers (#10587) · 0a71900b
  Chendi.Xue authored Nov 26, 2024
```
Signed-off-by: Chendi Xue <chendi.xue@intel.com>
```
  0a71900b
22 Nov, 2024 1 commit
- [platforms] absorb worker cls difference into platforms folder (#10555) · a111d015
  youkaichao authored Nov 21, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
Co-authored-by: Nick Hill <nhill@redhat.com>
```
  a111d015
06 Nov, 2024 1 commit

1.解决medusa模型多卡推理结果异常问题 · f9060e6b

王敏 authored Nov 06, 2024

2.examples中添加medusa readme
3.修复model_runner中input_positions配置错误的笔误，解决多个模型运行失败问题

f9060e6b

02 Nov, 2024 1 commit
- [2/N] executor pass the complete config to worker/modelrunner (#9938) · e8937954
  youkaichao authored Nov 02, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
Co-authored-by: Nick Hill <nhill@redhat.com>
```
  e8937954
24 Oct, 2024 1 commit
- 增加medusa并行解码功能,后续增加使用说明和测试文档 · 19bc93d9
  王敏 authored Oct 24, 2024
  
  19bc93d9
30 Aug, 2024 1 commit
- [Core] Logprobs support in Multi-step (#7652) · 428dd144
  afeldman-nm authored Aug 29, 2024
  
  428dd144
28 Aug, 2024 1 commit
- [Core] Async_output_proc: Add virtual engine support (towards pipeline parallel) (#7911) · f508e03e
  Alexander Matveev authored Aug 28, 2024
  
  f508e03e
27 Aug, 2024 1 commit
- [Core] Asynchronous Output Processor (#7049) · 2eedede8
  Megha Agarwal authored Aug 26, 2024
```
Co-authored-by: Alexander Matveev <alexm@neuralmagic.com>
```
  2eedede8
22 Aug, 2024 1 commit
- [Speculative Decoding] EAGLE Implementation with Top-1 proposer (#6830) · a3fce56b
  Abhinav Goyal authored Aug 22, 2024
  
  a3fce56b
19 Aug, 2024 1 commit
- [core] Multi Step Scheduling (#7000) · 47b65a55
  William Lin authored Aug 19, 2024
```
Co-authored-by: afeldman-nm <156691304+afeldman-nm@users.noreply.github.com>
```
  47b65a55
13 Aug, 2024 1 commit
- [misc][plugin] add plugin system implementation (#7426) · 16422ea7
  youkaichao authored Aug 13, 2024
  
  16422ea7
11 Aug, 2024 1 commit
- [core] [2/N] refactor worker_base input preparation for multi-step (#7387) · c08e2b30
  William Lin authored Aug 11, 2024
  
  c08e2b30
09 Aug, 2024 1 commit
- [Core] Add span metrics for model_forward, scheduler and sampler time (#7089) · 933790c2
  Mahesh Keralapura authored Aug 09, 2024
  
  933790c2
02 Aug, 2024 1 commit

[Core] Pipeline parallel with Ray ADAG (#6837) · 05308891

Rui Qiao authored Aug 02, 2024



Support pipeline-parallelism with Ray accelerated DAG.
Signed-off-by: Rui Qiao <ruisearch42@gmail.com>

05308891

01 Aug, 2024 1 commit
- PP comm optimization: replace send with partial send + allgather (#6695) · 0437492e
  Aurick Qiao authored Jul 31, 2024
```
Co-authored-by: Aurick Qiao <aurick.qiao@snowflake.com>
```
  0437492e
22 Jul, 2024 1 commit
- [Misc] Add a wrapper for torch.inference_mode (#6618) · 42de2cef
  Woosuk Kwon authored Jul 21, 2024
  
  42de2cef
20 Jul, 2024 1 commit
- [Core] Allow specifying custom Executor (#6557) · 7bd82002
  Antoni Baum authored Jul 19, 2024
  
  7bd82002
19 Jul, 2024 1 commit
- [Core] Multiprocessing Pipeline Parallel support (#6130) · b5672a11
  Nick Hill authored Jul 18, 2024
```
Co-authored-by: Murali Andoorveedu <muralidhar.andoorveedu@centml.ai>
```
  b5672a11
18 Jul, 2024 1 commit
- [Core] Introduce SPMD worker execution using Ray accelerated DAG (#6032) · 61e59274
  Rui Qiao authored Jul 17, 2024
```
Signed-off-by: Rui Qiao <ruisearch42@gmail.com>
Co-authored-by: Stephanie Wang <swang@cs.berkeley.edu>
```
  61e59274
12 Jul, 2024 1 commit
- [ROCm][AMD] unify CUDA_VISIBLE_DEVICES usage in cuda/rocm (#6352) · b6c16cf8
  Hongxia Yang authored Jul 12, 2024
  
  b6c16cf8
02 Jul, 2024 2 commits

[Model] Jamba support (#4115) · 9d6a8daa

Mor Zusman authored Jul 03, 2024


Signed-off-by: Muralidhar Andoorveedu <muralidhar.andoorveedu@centml.ai>
Co-authored-by: Erez Schwartz <erezs@ai21.com>
Co-authored-by: Mor Zusman <morz@ai21.com>
Co-authored-by: tomeras91 <57313761+tomeras91@users.noreply.github.com>
Co-authored-by: Tomer Asida <tomera@ai21.com>
Co-authored-by: Zhuohan Li <zhuohan123@gmail.com>
Co-authored-by: Muralidhar Andoorveedu <muralidhar.andoorveedu@centml.ai>

9d6a8daa

[Core] Pipeline Parallel Support (#4412) · c5832d2a
Murali Andoorveedu authored Jul 02, 2024
```
Signed-off-by: Muralidhar Andoorveedu <muralidhar.andoorveedu@centml.ai>
```
c5832d2a

28 Jun, 2024 1 commit
- [Spec Decode] Introduce DraftModelRunner (#5799) · b2c62023
  Cody Yu authored Jun 28, 2024
  
  b2c62023
26 Jun, 2024 1 commit

[Core] Refactor Worker and ModelRunner to consolidate control plane communication (#5408) · dda48115

Stephanie Wang authored Jun 25, 2024


Signed-off-by: Stephanie Wang <swang@cs.berkeley.edu>
Signed-off-by: Stephanie <swang@anyscale.com>
Co-authored-by: Stephanie <swang@anyscale.com>

dda48115

25 Jun, 2024 1 commit
- [Hardware][AMD][CI/Build][Doc] Upgrade to ROCm 6.1, Dockerfile improvements, test fixes (#5422) · dd793d1d
  Matt Wong authored Jun 25, 2024
  
  dd793d1d