Commits · 96ae75ad7f96c70c436f9297b06d059ad85140f9 · OpenDAS / vllm_cscc

03 Jan, 2025 1 commit
- fix run error · 184b50f7
  zhuwenwen authored Jan 03, 2025
  
  184b50f7
20 Dec, 2024 1 commit
- [Misc] Add tqdm progress bar during graph capture (#11349) · b880ffb8
  Michael Goin authored Dec 19, 2024
```
Signed-off-by: mgoin <michael@neuralmagic.com>
```
  b880ffb8
13 Dec, 2024 1 commit
- [core] clean up cudagraph batchsize padding logic (#10996) · be39e3cd
  youkaichao authored Dec 12, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  be39e3cd
11 Dec, 2024 3 commits
- [torch.compile] use depyf to dump torch.compile internals (#10972) · 91642db9
  youkaichao authored Dec 11, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  91642db9
- [feat]优化medusa代码，通过VLLM_TREE_DECODING环境变量控制是否采用tree-style解码，计算逻辑主干隔离 · 44e3ca68
  王敏 authored Dec 11, 2024
  
  44e3ca68
- [Misc] LoRA + Chunked Prefill (#9057) · d5c5154f
  Aurick Qiao authored Dec 10, 2024
  
  d5c5154f
06 Dec, 2024 1 commit
- [ci] fix broken tests (#10956) · dcdc3faf
  youkaichao authored Dec 06, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  dcdc3faf
03 Dec, 2024 1 commit
- [torch.compile] remove compilation_context and simplify code (#10838) · dc5ce861
  youkaichao authored Dec 02, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  dc5ce861
02 Dec, 2024 1 commit

[Core] Implement disagg prefill by StatelessProcessGroup (#10502) · 0590ec3f

Kuntai Du authored Dec 01, 2024



This PR provides initial support for single-node disaggregated prefill in 1P1D scenario.
Signed-off-by: KuntaiDu <kuntai@uchicago.edu>
Co-authored-by: ApostaC <yihua98@uchicago.edu>
Co-authored-by: YaoJiayi <120040070@link.cuhk.edu.cn>

0590ec3f

27 Nov, 2024 1 commit
- 1.优化medusa推理，节省cpu耗时 · 28375803
  王敏 authored Nov 27, 2024
```
2.更新medusa readme
3.解决benchmark_moe报错问题
```
  28375803
25 Nov, 2024 1 commit

Support Cross encoder models (#10400) · 214efc2c

Maximilien de Bayser authored Nov 24, 2024


Signed-off-by: Max de Bayser <maxdebayser@gmail.com>
Signed-off-by: Max de Bayser <mbayser@br.ibm.com>
Signed-off-by: Flavia Beo <flavia.beo@ibm.com>
Co-authored-by: Flavia Beo <flavia.beo@ibm.com>

214efc2c

22 Nov, 2024 1 commit
- [torch.compile] support all attention backends (#10558) · eebad39f
  youkaichao authored Nov 22, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  eebad39f
20 Nov, 2024 2 commits
- [fix]修复单测test_medusa_correctness.py中的错误 · 215f33b0
  王敏 authored Nov 20, 2024
  
  215f33b0
- [fix]修复单测test_encoder_decoder_model_runner报错问题 · 93089fb2
  王敏 authored Nov 20, 2024
  
  93089fb2
18 Nov, 2024 2 commits
- [5/N][torch.compile] torch.jit.script --> torch.compile (#10406) · 7851b451
  youkaichao authored Nov 18, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  7851b451
- [4/N][torch.compile] clean up set_torch_compile_backend (#10401) · 51bb12d1
  youkaichao authored Nov 17, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  51bb12d1
17 Nov, 2024 1 commit
- [2/N][torch.compile] make compilation cfg part of vllm cfg (#10383) · 4fd93750
  youkaichao authored Nov 16, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  4fd93750
16 Nov, 2024 1 commit
- [Bugfix] Fix M-RoPE position calculation when chunked prefill is enabled (#10388) · 361c29e1
  电脑星人 authored Nov 17, 2024
```
Signed-off-by: imkero <kerorek@outlook.com>
```
  361c29e1
13 Nov, 2024 1 commit
- [1/N] Initial prototype for multi-modal processor (#10044) · 0b8bb86b
  Cyrus Leung authored Nov 13, 2024
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  0b8bb86b
12 Nov, 2024 1 commit
- [Frontend] Add per-request number of cached token stats (#10174) · 47db6ec8
  zifeitong authored Nov 12, 2024
  
  47db6ec8
09 Nov, 2024 1 commit
- [0/N] Rename `MultiModalInputs` to `MultiModalKwargs` (#10040) · e0191a95
  Cyrus Leung authored Nov 09, 2024
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  e0191a95
07 Nov, 2024 1 commit
- [Misc] Consolidate ModelConfig code related to HF config (#10104) · db7db4aa
  Cyrus Leung authored Nov 07, 2024
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  db7db4aa
06 Nov, 2024 2 commits
- 1.解决medusa模型多卡推理结果异常问题 · f9060e6b
  王敏 authored Nov 06, 2024
```
2.examples中添加medusa readme
3.修复model_runner中input_positions配置错误的笔误，解决多个模型运行失败问题
```
  f9060e6b
- [Bugfix] Fix pickle of input when async output processing is on (#9931) · 966e3169
  Wallas Henrique authored Nov 05, 2024
```
Signed-off-by: Wallas Santos <wallashss@ibm.com>
```
  966e3169
05 Nov, 2024 1 commit

[Misc] Add logging for CUDA memory (#10027) · 09d35503

Chenghao (Alan) Yang authored Nov 05, 2024


Signed-off-by: Chenghao Yang <yangalan1996@gmail.com>
Signed-off-by: youkaichao <youkaichao@gmail.com>
Co-authored-by: Chenghao Yang <yangalan1996@gmail.com>
Co-authored-by: youkaichao <youkaichao@gmail.com>

09d35503

02 Nov, 2024 2 commits
- [3/N] model runner pass the whole config to model (#9958) · cea808f3
  youkaichao authored Nov 02, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  cea808f3
- [2/N] executor pass the complete config to worker/modelrunner (#9938) · e8937954
  youkaichao authored Nov 02, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
Co-authored-by: Nick Hill <nhill@redhat.com>
```
  e8937954
01 Nov, 2024 1 commit
- [Core][VLM] Add precise multi-modal placeholder tracking (#8346) · 6c0b7f54
  Peter Salas authored Nov 01, 2024
```
Signed-off-by: Peter Salas <peter@fixie.ai>
```
  6c0b7f54
31 Oct, 2024 1 commit
- [Misc] Remove deprecated arg for cuda graph capture (#9864) · 3ea2dc2e
  Roger Wang authored Oct 31, 2024
```
Signed-off-by: Roger Wang <ywang@roblox.com>
```
  3ea2dc2e
28 Oct, 2024 1 commit
- [Hardware][ROCM] using current_platform.is_rocm (#9642) · 4e2d95e3
  wangshuai09 authored Oct 28, 2024
```
Signed-off-by: wangshuai09 <391746016@qq.com>
```
  4e2d95e3
27 Oct, 2024 1 commit
- [core] cudagraph output with tensor weak reference (#9724) · 8549c826
  youkaichao authored Oct 27, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  8549c826
24 Oct, 2024 1 commit
- 增加medusa并行解码功能,后续增加使用说明和测试文档 · 19bc93d9
  王敏 authored Oct 24, 2024
  
  19bc93d9
21 Oct, 2024 2 commits
- [Bugfix][Misc]: fix graph capture for decoder (#9549) · 8ca89548
  yudian0504 authored Oct 22, 2024
  
  8ca89548
- [Doc] Consistent naming of attention backends (#9498) · 496e991d
  Thomas Parnell authored Oct 21, 2024
```
Signed-off-by: Thomas Parnell <tpa@zurich.ibm.com>
```
  496e991d
20 Oct, 2024 1 commit
- [Kernel] Support sliding window in flash attention backend (#9403) · 4fa3e333
  Chen Zhang authored Oct 20, 2024
  
  4fa3e333
17 Oct, 2024 2 commits

[Core] Deprecating block manager v1 and make block manager v2 default (#8704) · 81ede99c

Kuntai Du authored Oct 17, 2024

Removing the block manager v1. This is the initial piece of prefix-caching-centric design. In order to achieve prefix-caching-centric design, we need to simplify the code path so that we only use v2 block manager (which has much higher performance on prefix caching).

81ede99c

[misc] CUDA Time Layerwise Profiler (#8337) · 9d30a056

Lucas Wilkinson authored Oct 17, 2024


Co-authored-by: Varun Sundar Rabindranath <varun@neuralmagic.com>
Co-authored-by: Michael Goin <michael@neuralmagic.com>

9d30a056

16 Oct, 2024 1 commit
- [Misc] Standardize RoPE handling for Qwen2-VL (#9250) · 7e7eae33
  Cyrus Leung authored Oct 16, 2024
  
  7e7eae33
13 Oct, 2024 1 commit
- [Bugfix] Bandaid fix for speculative decoding tests (#9327) · 16b24e7d
  Tyler Michael Smith authored Oct 13, 2024
  
  16b24e7d
11 Oct, 2024 1 commit
- [Model] Support Mamba (#6484) · 7342a7d7
  Tyler Michael Smith authored Oct 11, 2024
  
  7342a7d7