Commits · e7c7c5e822a886e3dba202ca1b756c3260efffcc · OpenDAS / vllm_cscc

31 Dec, 2024 3 commits

[V1][VLM] V1 support for selected single-image models. (#11632) · e7c7c5e8

Roger Wang authored Dec 31, 2024


Signed-off-by: Roger Wang <ywang@roblox.com>
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Signed-off-by: Isotr0py <2037008807@qq.com>
Co-authored-by: DarkLight1337 <tlleungac@connect.ust.hk>
Co-authored-by: Isotr0py <2037008807@qq.com>

e7c7c5e8

[V1] Simpify vision block hash for prefix caching by removing offset from hash (#11646) · 8c3230d8
Chen Zhang authored Dec 31, 2024

8c3230d8

[Bugfix] Move the _touch(computed_blocks) call in the allocate_slots method to... · 2c571880

sakunkun authored Dec 31, 2024

[Bugfix] Move the _touch(computed_blocks) call in the allocate_slots method to after the check for allocating new blocks. (#11565)

2c571880

30 Dec, 2024 2 commits
- [V1] [6/N] API Server: Better Shutdown (#11586) · 5886aa49
  Robert Shaw authored Dec 30, 2024
  
  5886aa49
- [v1] fix compilation cache (#11598) · 3682e33f
  youkaichao authored Dec 30, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  3682e33f
28 Dec, 2024 3 commits
- [V1] [5/N] API Server: unify `Detokenizer` and `EngineCore` input (#11545) · 4fb8e329
  Robert Shaw authored Dec 28, 2024
```
Signed-off-by: rshaw@neuralmagic.com <rshaw@neuralmagic.com>
```
  4fb8e329
- [V1][Minor] Set pin_memory=False for token_ids_cpu tensor (#11581) · 42bb201f
  Woosuk Kwon authored Dec 28, 2024
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  42bb201f
- [V1] [4/N] API Server: ZMQ/MP Utilities (#11541) · df04dffa
  Robert Shaw authored Dec 27, 2024
  
  df04dffa
27 Dec, 2024 3 commits
- [V1][3/N] API Server: Reduce Task Switching + Handle Abort Properly (#11534) · 1b875a0e
  Robert Shaw authored Dec 27, 2024
  
  1b875a0e
- [V1] Fix yapf (#11538) · 81b979f2
  Woosuk Kwon authored Dec 27, 2024
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  81b979f2
- [V1] Use FlashInfer Sampling Kernel for Top-P & Top-K Sampling (#11394) · 371d04d3
  Woosuk Kwon authored Dec 27, 2024
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  371d04d3
26 Dec, 2024 1 commit

[V1] Adding min tokens/repetition/presence/frequence penalties to V1 sampler (#10681) · dcb1a944

sroy745 authored Dec 26, 2024


Signed-off-by: Sourashis Roy <sroy@roblox.com>
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
Co-authored-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>

dcb1a944

25 Dec, 2024 1 commit
- [V1] Unify VLLM_ENABLE_V1_MULTIPROCESSING handling in RayExecutor (#11472) · 9832e557
  Rui Qiao authored Dec 24, 2024
  
  9832e557
23 Dec, 2024 1 commit
- [V1] TP Ray executor (#11107) · a491d6f5
  Rui Qiao authored Dec 23, 2024
```
Signed-off-by: Rui Qiao <ruisearch42@gmail.com>
```
  a491d6f5
21 Dec, 2024 2 commits
- [V1] Make AsyncLLMEngine v1-v0 opaque (#11383) · 584f0ae4
  Ricky Xu authored Dec 20, 2024
```
Signed-off-by: Ricky Xu <xuchen727@hotmail.com>
```
  584f0ae4
- [V1][Bugfix] Skip hashing empty or None mm_data (#11386) · dd2b5633
  Woosuk Kwon authored Dec 21, 2024
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  dd2b5633
20 Dec, 2024 1 commit
- [V1] Fix profiling for models with merged input processor (#11370) · 04139ade
  Roger Wang authored Dec 20, 2024
```
Signed-off-by: ywang96 <ywang@roblox.com>
```
  04139ade
19 Dec, 2024 4 commits
- [Misc] Clean up and consolidate LRUCache (#11339) · cdf22afd
  Cyrus Leung authored Dec 20, 2024
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  cdf22afd
- [V1] Fix multimodal profiling for `Molmo` (#11325) · 7379b3d4
  Roger Wang authored Dec 19, 2024
```
Signed-off-by: ywang96 <ywang@example.com>
Co-authored-by: ywang96 <ywang@example.com>
```
  7379b3d4
- [Feature] Add load generation config from model (#11164) · 5aef4980
  Yanyi Liu authored Dec 19, 2024
```
Signed-off-by: liuyanyi <wolfsonliu@163.com>
Signed-off-by: Yanyi Liu <wolfsonliu@163.com>
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
```
  5aef4980
- [V1] Simplify prefix caching logic by removing `num_evictable_computed_blocks` (#11310) · c6b0a7d3
  Chen Zhang authored Dec 18, 2024
  
  c6b0a7d3
18 Dec, 2024 2 commits
- [V1] VLM - enable processor cache by default (#11305) · fdea8ec1
  Alexander Matveev authored Dec 18, 2024
```
Signed-off-by: Alexander Matveev <alexm@neuralmagic.com>
```
  fdea8ec1
- [V1] Prefix caching for vision language models (#11187) · bf8717eb
  Cody Yu authored Dec 17, 2024
```
Signed-off-by: Cody Yu <hao.yu.cody@gmail.com>
```
  bf8717eb
17 Dec, 2024 2 commits
- [V1][Core] Use weakref.finalize instead of atexit (#11242) · 2bfdbf2a
  Tyler Michael Smith authored Dec 17, 2024
```
Signed-off-by: Tyler Michael Smith <tyler@neuralmagic.com>
```
  2bfdbf2a
- [V1][VLM] Proper memory profiling for image language models (#11210) · 59c9b6eb
  Roger Wang authored Dec 16, 2024
```
Signed-off-by: Roger Wang <ywang@roblox.com>
Co-authored-by: ywang96 <ywang@example.com>
```
  59c9b6eb
15 Dec, 2024 1 commit
- [V1][Minor] Cache np arange to reduce input preparation overhead (#11214) · 25ebed2f
  Woosuk Kwon authored Dec 15, 2024
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  25ebed2f
14 Dec, 2024 5 commits
- Enable mypy checking on V1 code (#11105) · 6d917d0e
  Mark McLoughlin authored Dec 14, 2024
```
Signed-off-by: Mark McLoughlin <markmc@redhat.com>
```
  6d917d0e
- [V1][Bugfix] Fix V1 TP trust-remote-code (#11182) · ea7bd68d
  Tyler Michael Smith authored Dec 14, 2024
```
Signed-off-by: Tyler Michael Smith <tyler@neuralmagic.com>
```
  ea7bd68d
- [Bugfix][V1] Re-compute an entire block when fully cache hit (#11186) · 9855aea2
  Cody Yu authored Dec 13, 2024
```
Signed-off-by: Cody Yu <hao.yu.cody@gmail.com>
```
  9855aea2
- [V1][Bugfix] Fix EngineCoreProc profile (#11185) · 4b5b8a6a
  Tyler Michael Smith authored Dec 13, 2024
```
Signed-off-by: Tyler Michael Smith <tyler@neuralmagic.com>
```
  4b5b8a6a
- [Core] V1: Use multiprocessing by default (#11074) · 4863e5fb
  Russell Bryant authored Dec 13, 2024
```
Signed-off-by: Russell Bryant <rbryant@redhat.com>
```
  4863e5fb
13 Dec, 2024 3 commits
- [core] clean up cudagraph batchsize padding logic (#10996) · be39e3cd
  youkaichao authored Dec 12, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  be39e3cd
- [Bugfix][V1] Fix 'NoneType' object has no attribute 'hash_value' (#11157) · 34f1a806
  Cody Yu authored Dec 12, 2024
```
Signed-off-by: Cody Yu <hao.yu.cody@gmail.com>
```
  34f1a806
- [Misc][V1] Fix type in v1 prefix caching (#11151) · 78ed8f57
  Cody Yu authored Dec 12, 2024
  
  78ed8f57
12 Dec, 2024 4 commits
- [V1] Fix torch profiling for offline inference (#11125) · 4816d20a
  Roger Wang authored Dec 12, 2024
```
Signed-off-by: Roger Wang <ywang@roblox.com>
```
  4816d20a
- [V1] Use more persistent buffers to optimize input preparation overheads (#11111) · f092153f
  Woosuk Kwon authored Dec 11, 2024
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  f092153f
- [Core] cleanup zmq ipc sockets on exit (#11115) · ccede2b2
  Russell Bryant authored Dec 11, 2024
```
Signed-off-by: Russell Bryant <rbryant@redhat.com>
```
  ccede2b2
- [V1] VLM preprocessor hashing (#11020) · 4e116833
  Alexander Matveev authored Dec 11, 2024
```
Signed-off-by: Roger Wang <ywang@roblox.com>
Signed-off-by: Alexander Matveev <alexm@neuralmagic.com>
Co-authored-by: Michael Goin <michael@neuralmagic.com>
Co-authored-by: Roger Wang <ywang@roblox.com>
```
  4e116833
11 Dec, 2024 2 commits
- [V1][Core] Remove should_shutdown to simplify core process termination (#11113) · 452a723b
  Tyler Michael Smith authored Dec 11, 2024
```
Signed-off-by: Tyler Michael Smith <tyler@neuralmagic.com>
```
  452a723b
- [V1] Use input_ids as input for text-only models (#11032) · d643c2ab
  Woosuk Kwon authored Dec 11, 2024
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  d643c2ab