Commits · fa63e710c7fbaae3a445f669d3b5ba6b9a4ef412 · OpenDAS / vllm_cscc

26 Jan, 2025 1 commit
- [V1][Perf] Reduce scheduling overhead in model runner after cuda sync (#12094) · fa63e710
  Keyun Tong authored Jan 26, 2025
```
Signed-off-by: Keyun Tong <tongkeyun@gmail.com>
```
  fa63e710
23 Jan, 2025 3 commits
- [torch.compile] decouple compile sizes and cudagraph sizes (#12243) · 6e650f56
  youkaichao authored Jan 24, 2025
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  6e650f56
- [V1] Simplify M-RoPE (#12352) · 99d01a5e
  Roger Wang authored Jan 23, 2025
```
Signed-off-by: Roger Wang <ywang@roblox.com>
Co-authored-by: imkero <kerorek@outlook.com>
```
  99d01a5e
- [Kernel] Flash Attention 3 Support (#12093) · 978b45f3
  Lucas Wilkinson authored Jan 23, 2025
```
Signed-off-by: Lucas Wilkinson <lwilkinson@neuralmagic.com>
```
  978b45f3
22 Jan, 2025 1 commit
- [Core] Support fully transparent sleep mode (#11743) · 68ad4e3a
  youkaichao authored Jan 22, 2025
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  68ad4e3a
21 Jan, 2025 1 commit
- [V1][Bugfix] Fix data item ordering in mixed-modality inference (#12259) · b197a5cc
  Roger Wang authored Jan 21, 2025
```
Signed-off-by: Roger Wang <ywang@roblox.com>
```
  b197a5cc
20 Jan, 2025 2 commits
- [V1] Remove `_get_cache_block_size` (#12214) · 5f0ec393
  Chen Zhang authored Jan 20, 2025
```
Signed-off-by: Chen Zhang <zhangch99@outlook.com>
```
  5f0ec393
- [Core] Interface for accessing model from `VllmRunner` (#10353) · 59a0192f
  Cyrus Leung authored Jan 20, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  59a0192f
19 Jan, 2025 1 commit

[V1] Add V1 support of Qwen2-VL (#12128) · 81763c58

Roger Wang authored Jan 19, 2025


Signed-off-by: Roger Wang <ywang@roblox.com>
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Co-authored-by: imkero <kerorek@outlook.com>
Co-authored-by: DarkLight1337 <tlleungac@connect.ust.hk>

81763c58

17 Jan, 2025 1 commit
- [V1] Move more control of kv cache initialization from model_executor to EngineCore (#11960) · 69d765f5
  Chen Zhang authored Jan 17, 2025
```
Signed-off-by: Chen Zhang <zhangch99@outlook.com>
Co-authored-by: Cody Yu <hao.yu.cody@gmail.com>
```
  69d765f5
15 Jan, 2025 3 commits
- [V1][Core] Autotune encoder cache budget (#11895) · 70755e81
  Roger Wang authored Jan 15, 2025
```
Signed-off-by: Roger Wang <ywang@roblox.com>
```
  70755e81
- Type-fix: make execute_model output type optional (#12020) · ad388d25
  Keyun Tong authored Jan 15, 2025
  
  ad388d25
- [core] platform agnostic executor via collective_rpc (#11256) · ad34c0df
  youkaichao authored Jan 15, 2025
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  ad34c0df
13 Jan, 2025 1 commit
- [Misc]Minor Changes about Worker (#11555) · c3f05b09
  Chenguang Li authored Jan 13, 2025
```
Signed-off-by: Chenguang Li <757486878@qq.com>
```
  c3f05b09
10 Jan, 2025 1 commit
- [torch.compile] Hide KV cache behind torch.compile boundary (#11677) · cf5f000d
  Chen Zhang authored Jan 10, 2025
```
Signed-off-by: Chen Zhang <zhangch99@outlook.com>
```
  cf5f000d
06 Jan, 2025 2 commits

[V1] Extend beyond image modality and support mixed-modality inference with... · 91b361ae

Roger Wang authored Jan 06, 2025


[V1] Extend beyond image modality and support mixed-modality inference with Llava-OneVision (#11685)
Signed-off-by: Roger Wang <ywang@roblox.com>
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Co-authored-by: DarkLight1337 <tlleungac@connect.ust.hk>

91b361ae

[V1] Add BlockTable class (#11693) · 06bfb519
Woosuk Kwon authored Jan 06, 2025
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
06bfb519

04 Jan, 2025 1 commit
- [Core][Bugfix] Use correct device to initialize GPU data during CUDA-graph-capture (#11233) · 300acb83
  Yan Burman authored Jan 04, 2025
```
Signed-off-by: Yan Burman <yanburman@users.noreply.github.com>
Signed-off-by: Ido Asraff <idoa@atero.ai>
```
  300acb83
02 Jan, 2025 1 commit
- [V1][Minor] Optimize token_ids_cpu copy (#11692) · b55ed6ef
  Woosuk Kwon authored Jan 03, 2025
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  b55ed6ef
01 Jan, 2025 1 commit
- [V1] Implement Cascade Attention (#11635) · 73001445
  Woosuk Kwon authored Jan 01, 2025
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  73001445
31 Dec, 2024 1 commit

[V1][VLM] V1 support for selected single-image models. (#11632) · e7c7c5e8

Roger Wang authored Dec 31, 2024


Signed-off-by: Roger Wang <ywang@roblox.com>
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Signed-off-by: Isotr0py <2037008807@qq.com>
Co-authored-by: DarkLight1337 <tlleungac@connect.ust.hk>
Co-authored-by: Isotr0py <2037008807@qq.com>

e7c7c5e8

30 Dec, 2024 1 commit
- [v1] fix compilation cache (#11598) · 3682e33f
  youkaichao authored Dec 30, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  3682e33f
28 Dec, 2024 1 commit
- [V1][Minor] Set pin_memory=False for token_ids_cpu tensor (#11581) · 42bb201f
  Woosuk Kwon authored Dec 28, 2024
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  42bb201f
26 Dec, 2024 1 commit

[V1] Adding min tokens/repetition/presence/frequence penalties to V1 sampler (#10681) · dcb1a944

sroy745 authored Dec 26, 2024


Signed-off-by: Sourashis Roy <sroy@roblox.com>
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
Co-authored-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>

dcb1a944

23 Dec, 2024 1 commit
- [V1] TP Ray executor (#11107) · a491d6f5
  Rui Qiao authored Dec 23, 2024
```
Signed-off-by: Rui Qiao <ruisearch42@gmail.com>
```
  a491d6f5
20 Dec, 2024 1 commit
- [V1] Fix profiling for models with merged input processor (#11370) · 04139ade
  Roger Wang authored Dec 20, 2024
```
Signed-off-by: ywang96 <ywang@roblox.com>
```
  04139ade
19 Dec, 2024 1 commit
- [V1] Fix multimodal profiling for `Molmo` (#11325) · 7379b3d4
  Roger Wang authored Dec 19, 2024
```
Signed-off-by: ywang96 <ywang@example.com>
Co-authored-by: ywang96 <ywang@example.com>
```
  7379b3d4
18 Dec, 2024 1 commit
- [V1] VLM - enable processor cache by default (#11305) · fdea8ec1
  Alexander Matveev authored Dec 18, 2024
```
Signed-off-by: Alexander Matveev <alexm@neuralmagic.com>
```
  fdea8ec1
17 Dec, 2024 1 commit
- [V1][VLM] Proper memory profiling for image language models (#11210) · 59c9b6eb
  Roger Wang authored Dec 16, 2024
```
Signed-off-by: Roger Wang <ywang@roblox.com>
Co-authored-by: ywang96 <ywang@example.com>
```
  59c9b6eb
15 Dec, 2024 1 commit
- [V1][Minor] Cache np arange to reduce input preparation overhead (#11214) · 25ebed2f
  Woosuk Kwon authored Dec 15, 2024
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  25ebed2f
14 Dec, 2024 1 commit
- Enable mypy checking on V1 code (#11105) · 6d917d0e
  Mark McLoughlin authored Dec 14, 2024
```
Signed-off-by: Mark McLoughlin <markmc@redhat.com>
```
  6d917d0e
13 Dec, 2024 1 commit
- [core] clean up cudagraph batchsize padding logic (#10996) · be39e3cd
  youkaichao authored Dec 12, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  be39e3cd
12 Dec, 2024 1 commit
- [V1] Use more persistent buffers to optimize input preparation overheads (#11111) · f092153f
  Woosuk Kwon authored Dec 11, 2024
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  f092153f
11 Dec, 2024 2 commits
- [V1] Use input_ids as input for text-only models (#11032) · d643c2ab
  Woosuk Kwon authored Dec 11, 2024
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  d643c2ab
- [Model] PP support for Mamba-like models (#10992) · ffa48c91
  Mor Zusman authored Dec 11, 2024
```
Signed-off-by: mzusman <mor.zusmann@gmail.com>
```
  ffa48c91
10 Dec, 2024 2 commits
- [torch.compile] add a flag to track batchsize statistics (#11059) · 75f89dc4
  youkaichao authored Dec 10, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  75f89dc4
- [V1] Multiprocessing Tensor Parallel Support for v1 (#9856) · 28b3a1c7
  Tyler Michael Smith authored Dec 10, 2024
```
Signed-off-by: Tyler Michael Smith <tyler@neuralmagic.com>
```
  28b3a1c7
09 Dec, 2024 3 commits
- [V1] Do not store `None` in self.generators (#11038) · 6faec545
  Woosuk Kwon authored Dec 09, 2024
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  6faec545
- [v1] fix use compile sizes (#11000) · 1a2f8fb8
  youkaichao authored Dec 09, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  1a2f8fb8
- [V1] Input Batch Relocation (#10962) · 25b79d9f
  Varun Sundar Rabindranath authored Dec 09, 2024
```
Signed-off-by: Varun Sundar Rabindranath <varun@neuralmagic.com>
Co-authored-by: Varun Sundar Rabindranath <varun@neuralmagic.com>
```
  25b79d9f