Commits · 408e5600158bfa34306cfbd034a3779e488752fa · OpenDAS / vllm_cscc

04 Jan, 2025 3 commits
- [Core][Bugfix] Use correct device to initialize GPU data during CUDA-graph-capture (#11233) · 300acb83
  Yan Burman authored Jan 04, 2025
```
Signed-off-by: Yan Burman <yanburman@users.noreply.github.com>
Signed-off-by: Ido Asraff <idoa@atero.ai>
```
  300acb83
- [V1] Add `RayExecutor` support for `AsyncLLM` (api server) (#11712) · fbf25645
  Kunshang Ji authored Jan 04, 2025
  
  fbf25645
- [V1] log GPU blocks num for MultiprocExecutor (#11656) · e5d7ed0c
  WangErXiao authored Jan 04, 2025
  
  e5d7ed0c
03 Jan, 2025 3 commits
- [V1] Chore: cruft removal (#11724) · ad0d567e
  Robert Shaw authored Jan 03, 2025
  
  ad0d567e
- [V1] Improve TP>1 Error Handling + Stack Trace (#11721) · 1543914c
  Robert Shaw authored Jan 03, 2025
```
Co-authored-by: Tyler Michael Smith <tyler@neuralmagic.com>
```
  1543914c
- [V1] Simplify Shutdown (#11659) · 80c751e7
  Robert Shaw authored Jan 03, 2025
  
  80c751e7
02 Jan, 2025 2 commits
- [V1][Minor] Optimize token_ids_cpu copy (#11692) · b55ed6ef
  Woosuk Kwon authored Jan 03, 2025
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  b55ed6ef
- [VLM][Bugfix] Multi-modal processor compatible with V1 multi-input (#11674) · 23c1b10a
  Cyrus Leung authored Jan 02, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  23c1b10a
01 Jan, 2025 2 commits
- [V1] Implement Cascade Attention (#11635) · 73001445
  Woosuk Kwon authored Jan 01, 2025
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  73001445
- [Doc] Fix typo (#11666) · 6d70198b
  Kazuhiro Serizawa authored Jan 01, 2025
```
Signed-off-by: Kazuhiro Serizawa <nserihiro@gmail.com>
```
  6d70198b
31 Dec, 2024 3 commits

[V1][VLM] V1 support for selected single-image models. (#11632) · e7c7c5e8

Roger Wang authored Dec 31, 2024


Signed-off-by: Roger Wang <ywang@roblox.com>
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Signed-off-by: Isotr0py <2037008807@qq.com>
Co-authored-by: DarkLight1337 <tlleungac@connect.ust.hk>
Co-authored-by: Isotr0py <2037008807@qq.com>

e7c7c5e8

[V1] Simpify vision block hash for prefix caching by removing offset from hash (#11646) · 8c3230d8
Chen Zhang authored Dec 31, 2024

8c3230d8

[Bugfix] Move the _touch(computed_blocks) call in the allocate_slots method to... · 2c571880

sakunkun authored Dec 31, 2024

[Bugfix] Move the _touch(computed_blocks) call in the allocate_slots method to after the check for allocating new blocks. (#11565)

2c571880

30 Dec, 2024 2 commits
- [V1] [6/N] API Server: Better Shutdown (#11586) · 5886aa49
  Robert Shaw authored Dec 30, 2024
  
  5886aa49
- [v1] fix compilation cache (#11598) · 3682e33f
  youkaichao authored Dec 30, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  3682e33f
28 Dec, 2024 3 commits
- [V1] [5/N] API Server: unify `Detokenizer` and `EngineCore` input (#11545) · 4fb8e329
  Robert Shaw authored Dec 28, 2024
```
Signed-off-by: rshaw@neuralmagic.com <rshaw@neuralmagic.com>
```
  4fb8e329
- [V1][Minor] Set pin_memory=False for token_ids_cpu tensor (#11581) · 42bb201f
  Woosuk Kwon authored Dec 28, 2024
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  42bb201f
- [V1] [4/N] API Server: ZMQ/MP Utilities (#11541) · df04dffa
  Robert Shaw authored Dec 27, 2024
  
  df04dffa
27 Dec, 2024 3 commits
- [V1][3/N] API Server: Reduce Task Switching + Handle Abort Properly (#11534) · 1b875a0e
  Robert Shaw authored Dec 27, 2024
  
  1b875a0e
- [V1] Fix yapf (#11538) · 81b979f2
  Woosuk Kwon authored Dec 27, 2024
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  81b979f2
- [V1] Use FlashInfer Sampling Kernel for Top-P & Top-K Sampling (#11394) · 371d04d3
  Woosuk Kwon authored Dec 27, 2024
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  371d04d3
26 Dec, 2024 1 commit

[V1] Adding min tokens/repetition/presence/frequence penalties to V1 sampler (#10681) · dcb1a944

sroy745 authored Dec 26, 2024


Signed-off-by: Sourashis Roy <sroy@roblox.com>
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
Co-authored-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>

dcb1a944

25 Dec, 2024 1 commit
- [V1] Unify VLLM_ENABLE_V1_MULTIPROCESSING handling in RayExecutor (#11472) · 9832e557
  Rui Qiao authored Dec 24, 2024
  
  9832e557
23 Dec, 2024 1 commit
- [V1] TP Ray executor (#11107) · a491d6f5
  Rui Qiao authored Dec 23, 2024
```
Signed-off-by: Rui Qiao <ruisearch42@gmail.com>
```
  a491d6f5
21 Dec, 2024 2 commits
- [V1] Make AsyncLLMEngine v1-v0 opaque (#11383) · 584f0ae4
  Ricky Xu authored Dec 20, 2024
```
Signed-off-by: Ricky Xu <xuchen727@hotmail.com>
```
  584f0ae4
- [V1][Bugfix] Skip hashing empty or None mm_data (#11386) · dd2b5633
  Woosuk Kwon authored Dec 21, 2024
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  dd2b5633
20 Dec, 2024 1 commit
- [V1] Fix profiling for models with merged input processor (#11370) · 04139ade
  Roger Wang authored Dec 20, 2024
```
Signed-off-by: ywang96 <ywang@roblox.com>
```
  04139ade
19 Dec, 2024 4 commits
- [Misc] Clean up and consolidate LRUCache (#11339) · cdf22afd
  Cyrus Leung authored Dec 20, 2024
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  cdf22afd
- [V1] Fix multimodal profiling for `Molmo` (#11325) · 7379b3d4
  Roger Wang authored Dec 19, 2024
```
Signed-off-by: ywang96 <ywang@example.com>
Co-authored-by: ywang96 <ywang@example.com>
```
  7379b3d4
- [Feature] Add load generation config from model (#11164) · 5aef4980
  Yanyi Liu authored Dec 19, 2024
```
Signed-off-by: liuyanyi <wolfsonliu@163.com>
Signed-off-by: Yanyi Liu <wolfsonliu@163.com>
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
```
  5aef4980
- [V1] Simplify prefix caching logic by removing `num_evictable_computed_blocks` (#11310) · c6b0a7d3
  Chen Zhang authored Dec 18, 2024
  
  c6b0a7d3
18 Dec, 2024 2 commits
- [V1] VLM - enable processor cache by default (#11305) · fdea8ec1
  Alexander Matveev authored Dec 18, 2024
```
Signed-off-by: Alexander Matveev <alexm@neuralmagic.com>
```
  fdea8ec1
- [V1] Prefix caching for vision language models (#11187) · bf8717eb
  Cody Yu authored Dec 17, 2024
```
Signed-off-by: Cody Yu <hao.yu.cody@gmail.com>
```
  bf8717eb
17 Dec, 2024 2 commits
- [V1][Core] Use weakref.finalize instead of atexit (#11242) · 2bfdbf2a
  Tyler Michael Smith authored Dec 17, 2024
```
Signed-off-by: Tyler Michael Smith <tyler@neuralmagic.com>
```
  2bfdbf2a
- [V1][VLM] Proper memory profiling for image language models (#11210) · 59c9b6eb
  Roger Wang authored Dec 16, 2024
```
Signed-off-by: Roger Wang <ywang@roblox.com>
Co-authored-by: ywang96 <ywang@example.com>
```
  59c9b6eb
15 Dec, 2024 1 commit
- [V1][Minor] Cache np arange to reduce input preparation overhead (#11214) · 25ebed2f
  Woosuk Kwon authored Dec 15, 2024
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  25ebed2f
14 Dec, 2024 4 commits
- Enable mypy checking on V1 code (#11105) · 6d917d0e
  Mark McLoughlin authored Dec 14, 2024
```
Signed-off-by: Mark McLoughlin <markmc@redhat.com>
```
  6d917d0e
- [V1][Bugfix] Fix V1 TP trust-remote-code (#11182) · ea7bd68d
  Tyler Michael Smith authored Dec 14, 2024
```
Signed-off-by: Tyler Michael Smith <tyler@neuralmagic.com>
```
  ea7bd68d
- [Bugfix][V1] Re-compute an entire block when fully cache hit (#11186) · 9855aea2
  Cody Yu authored Dec 13, 2024
```
Signed-off-by: Cody Yu <hao.yu.cody@gmail.com>
```
  9855aea2
- [V1][Bugfix] Fix EngineCoreProc profile (#11185) · 4b5b8a6a
  Tyler Michael Smith authored Dec 13, 2024
```
Signed-off-by: Tyler Michael Smith <tyler@neuralmagic.com>
```
  4b5b8a6a