Commits · 5f0ec3935a0118fee8cf2764728f765c8cc53d2a · OpenDAS / vllm_cscc

20 Jan, 2025 2 commits
- [V1] Remove `_get_cache_block_size` (#12214) · 5f0ec393
  Chen Zhang authored Jan 20, 2025
```
Signed-off-by: Chen Zhang <zhangch99@outlook.com>
```
  5f0ec393
- [Core] Interface for accessing model from `VllmRunner` (#10353) · 59a0192f
  Cyrus Leung authored Jan 20, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  59a0192f
19 Jan, 2025 1 commit

[V1] Add V1 support of Qwen2-VL (#12128) · 81763c58

Roger Wang authored Jan 19, 2025


Signed-off-by: Roger Wang <ywang@roblox.com>
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Co-authored-by: imkero <kerorek@outlook.com>
Co-authored-by: DarkLight1337 <tlleungac@connect.ust.hk>

81763c58

18 Jan, 2025 1 commit
- [core] clean up executor class hierarchy between v1 and v0 (#12171) · 6d0e3d37
  youkaichao authored Jan 18, 2025
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  6d0e3d37
17 Jan, 2025 2 commits
- [core] allow callable in collective_rpc (#12151) · 87a0c076
  youkaichao authored Jan 17, 2025
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  87a0c076
- [V1] Move more control of kv cache initialization from model_executor to EngineCore (#11960) · 69d765f5
  Chen Zhang authored Jan 17, 2025
```
Signed-off-by: Chen Zhang <zhangch99@outlook.com>
Co-authored-by: Cody Yu <hao.yu.cody@gmail.com>
```
  69d765f5
16 Jan, 2025 1 commit
- Support torchrun and SPMD-style offline inference (#12071) · bf53e0c7
  youkaichao authored Jan 16, 2025
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  bf53e0c7
15 Jan, 2025 5 commits
- [V1][Core] Autotune encoder cache budget (#11895) · 70755e81
  Roger Wang authored Jan 15, 2025
```
Signed-off-by: Roger Wang <ywang@roblox.com>
```
  70755e81
- Type-fix: make execute_model output type optional (#12020) · ad388d25
  Keyun Tong authored Jan 15, 2025
  
  ad388d25
- [V1][Prefix Cache] Move the logic of num_computed_tokens into KVCacheManager (#12003) · 994fc655
  Chen Zhang authored Jan 15, 2025
  
  994fc655
- [core] platform agnostic executor via collective_rpc (#11256) · ad34c0df
  youkaichao authored Jan 15, 2025
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  ad34c0df
- [V1][BugFix] Fix edge case in VLM scheduling (#12065) · b7ee940a
  Woosuk Kwon authored Jan 14, 2025
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  b7ee940a
14 Jan, 2025 1 commit
- [Platform] Add output for Attention Backend (#11981) · 2e0e0176
  wangxiyuan authored Jan 14, 2025
```
Signed-off-by: wangxiyuan <wangxiyuan1007@gmail.com>
```
  2e0e0176
13 Jan, 2025 3 commits
- [platform] add ray_device_key (#11948) · 89ce62a3
  youkaichao authored Jan 13, 2025
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  89ce62a3
- [Misc]Minor Changes about Worker (#11555) · c3f05b09
  Chenguang Li authored Jan 13, 2025
```
Signed-off-by: Chenguang Li <757486878@qq.com>
```
  c3f05b09
- [V1] [2/n] Logging and Metrics - `OutputProcessor` Abstraction (#11973) · 619ae268
  Robert Shaw authored Jan 12, 2025
```
Signed-off-by: rshaw@neuralmagic.com <rshaw@neuralmagic.com>
```
  619ae268
12 Jan, 2025 2 commits
- [V1][Core][1/n] Logging and Metrics (#11962) · 9597a095
  Robert Shaw authored Jan 12, 2025
```
Signed-off-by: rshaw@neuralmagic.com <rshaw@neuralmagic.com>
```
  9597a095
- [V1] Avoid sending text prompt to core engine (#11963) · b25cfab9
  Roger Wang authored Jan 11, 2025
```
Signed-off-by: Roger Wang <ywang@roblox.com>
```
  b25cfab9
10 Jan, 2025 2 commits
- [Bugfix] Validate lora adapters to avoid crashing server (#11727) · ac2f3f7f
  Joe Runde authored Jan 10, 2025
```
Signed-off-by: Joe Runde <Joseph.Runde@ibm.com>
Co-authored-by: Jee Jee Li <pandaleefree@gmail.com>
```
  ac2f3f7f
- [torch.compile] Hide KV cache behind torch.compile boundary (#11677) · cf5f000d
  Chen Zhang authored Jan 10, 2025
```
Signed-off-by: Chen Zhang <zhangch99@outlook.com>
```
  cf5f000d
08 Jan, 2025 2 commits
- [Misc]add some explanations for BlockHashType (#11847) · 47de8821
  WangErXiao authored Jan 09, 2025
  
  47de8821
- Remove the duplicate imports of MultiModalKwargs and PlaceholderRange… (#11824) · dc71af0a
  WangErXiao authored Jan 08, 2025
  
  dc71af0a
06 Jan, 2025 4 commits
- [V1] Extend beyond image modality and support mixed-modality inference with... · 91b361ae
  Roger Wang authored Jan 06, 2025
```
[V1] Extend beyond image modality and support mixed-modality inference with Llava-OneVision (#11685)
Signed-off-by: Roger Wang <ywang@roblox.com>
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Co-authored-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  91b361ae
- [Kernel] Move attn_type to Attention.__init__() (#11690) · e20c92bb
  Chen Zhang authored Jan 07, 2025
```
Signed-off-by: Chen Zhang <zhangch99@outlook.com>
```
  e20c92bb
- [V1] Refactor get_executor_cls (#11754) · 022c5c69
  Rui Qiao authored Jan 05, 2025
  
  022c5c69
- [V1] Add BlockTable class (#11693) · 06bfb519
  Woosuk Kwon authored Jan 06, 2025
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  06bfb519
04 Jan, 2025 3 commits
- [Core][Bugfix] Use correct device to initialize GPU data during CUDA-graph-capture (#11233) · 300acb83
  Yan Burman authored Jan 04, 2025
```
Signed-off-by: Yan Burman <yanburman@users.noreply.github.com>
Signed-off-by: Ido Asraff <idoa@atero.ai>
```
  300acb83
- [V1] Add `RayExecutor` support for `AsyncLLM` (api server) (#11712) · fbf25645
  Kunshang Ji authored Jan 04, 2025
  
  fbf25645
- [V1] log GPU blocks num for MultiprocExecutor (#11656) · e5d7ed0c
  WangErXiao authored Jan 04, 2025
  
  e5d7ed0c
03 Jan, 2025 3 commits
- [V1] Chore: cruft removal (#11724) · ad0d567e
  Robert Shaw authored Jan 03, 2025
  
  ad0d567e
- [V1] Improve TP>1 Error Handling + Stack Trace (#11721) · 1543914c
  Robert Shaw authored Jan 03, 2025
```
Co-authored-by: Tyler Michael Smith <tyler@neuralmagic.com>
```
  1543914c
- [V1] Simplify Shutdown (#11659) · 80c751e7
  Robert Shaw authored Jan 03, 2025
  
  80c751e7
02 Jan, 2025 2 commits
- [V1][Minor] Optimize token_ids_cpu copy (#11692) · b55ed6ef
  Woosuk Kwon authored Jan 03, 2025
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  b55ed6ef
- [VLM][Bugfix] Multi-modal processor compatible with V1 multi-input (#11674) · 23c1b10a
  Cyrus Leung authored Jan 02, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  23c1b10a
01 Jan, 2025 2 commits
- [V1] Implement Cascade Attention (#11635) · 73001445
  Woosuk Kwon authored Jan 01, 2025
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  73001445
- [Doc] Fix typo (#11666) · 6d70198b
  Kazuhiro Serizawa authored Jan 01, 2025
```
Signed-off-by: Kazuhiro Serizawa <nserihiro@gmail.com>
```
  6d70198b
31 Dec, 2024 3 commits

[V1][VLM] V1 support for selected single-image models. (#11632) · e7c7c5e8

Roger Wang authored Dec 31, 2024


Signed-off-by: Roger Wang <ywang@roblox.com>
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Signed-off-by: Isotr0py <2037008807@qq.com>
Co-authored-by: DarkLight1337 <tlleungac@connect.ust.hk>
Co-authored-by: Isotr0py <2037008807@qq.com>

e7c7c5e8

[V1] Simpify vision block hash for prefix caching by removing offset from hash (#11646) · 8c3230d8
Chen Zhang authored Dec 31, 2024

8c3230d8

[Bugfix] Move the _touch(computed_blocks) call in the allocate_slots method to... · 2c571880

sakunkun authored Dec 31, 2024

[Bugfix] Move the _touch(computed_blocks) call in the allocate_slots method to after the check for allocating new blocks. (#11565)

2c571880

30 Dec, 2024 1 commit
- [V1] [6/N] API Server: Better Shutdown (#11586) · 5886aa49
  Robert Shaw authored Dec 30, 2024
  
  5886aa49