Commits · 69d765f5a5bbbe1ea9843be19b9480660fc5bc8b · OpenDAS / vllm_cscc

17 Jan, 2025 1 commit
- [V1] Move more control of kv cache initialization from model_executor to EngineCore (#11960) · 69d765f5
  Chen Zhang authored Jan 17, 2025
```
Signed-off-by: Chen Zhang <zhangch99@outlook.com>
Co-authored-by: Cody Yu <hao.yu.cody@gmail.com>
```
  69d765f5
15 Jan, 2025 1 commit
- [V1][Core] Autotune encoder cache budget (#11895) · 70755e81
  Roger Wang authored Jan 15, 2025
```
Signed-off-by: Roger Wang <ywang@roblox.com>
```
  70755e81
13 Jan, 2025 1 commit
- [V1] [2/n] Logging and Metrics - `OutputProcessor` Abstraction (#11973) · 619ae268
  Robert Shaw authored Jan 12, 2025
```
Signed-off-by: rshaw@neuralmagic.com <rshaw@neuralmagic.com>
```
  619ae268
12 Jan, 2025 2 commits
- [V1][Core][1/n] Logging and Metrics (#11962) · 9597a095
  Robert Shaw authored Jan 12, 2025
```
Signed-off-by: rshaw@neuralmagic.com <rshaw@neuralmagic.com>
```
  9597a095
- [V1] Avoid sending text prompt to core engine (#11963) · b25cfab9
  Roger Wang authored Jan 11, 2025
```
Signed-off-by: Roger Wang <ywang@roblox.com>
```
  b25cfab9
10 Jan, 2025 1 commit
- [Bugfix] Validate lora adapters to avoid crashing server (#11727) · ac2f3f7f
  Joe Runde authored Jan 10, 2025
```
Signed-off-by: Joe Runde <Joseph.Runde@ibm.com>
Co-authored-by: Jee Jee Li <pandaleefree@gmail.com>
```
  ac2f3f7f
06 Jan, 2025 2 commits

[V1] Extend beyond image modality and support mixed-modality inference with... · 91b361ae

Roger Wang authored Jan 06, 2025


[V1] Extend beyond image modality and support mixed-modality inference with Llava-OneVision (#11685)
Signed-off-by: Roger Wang <ywang@roblox.com>
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Co-authored-by: DarkLight1337 <tlleungac@connect.ust.hk>

91b361ae

[V1] Refactor get_executor_cls (#11754) · 022c5c69
Rui Qiao authored Jan 05, 2025

022c5c69

04 Jan, 2025 1 commit
- [V1] Add `RayExecutor` support for `AsyncLLM` (api server) (#11712) · fbf25645
  Kunshang Ji authored Jan 04, 2025
  
  fbf25645
03 Jan, 2025 3 commits
- [V1] Chore: cruft removal (#11724) · ad0d567e
  Robert Shaw authored Jan 03, 2025
  
  ad0d567e
- [V1] Improve TP>1 Error Handling + Stack Trace (#11721) · 1543914c
  Robert Shaw authored Jan 03, 2025
```
Co-authored-by: Tyler Michael Smith <tyler@neuralmagic.com>
```
  1543914c
- [V1] Simplify Shutdown (#11659) · 80c751e7
  Robert Shaw authored Jan 03, 2025
  
  80c751e7
02 Jan, 2025 1 commit
- [VLM][Bugfix] Multi-modal processor compatible with V1 multi-input (#11674) · 23c1b10a
  Cyrus Leung authored Jan 02, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  23c1b10a
30 Dec, 2024 1 commit
- [V1] [6/N] API Server: Better Shutdown (#11586) · 5886aa49
  Robert Shaw authored Dec 30, 2024
  
  5886aa49
28 Dec, 2024 2 commits
- [V1] [5/N] API Server: unify `Detokenizer` and `EngineCore` input (#11545) · 4fb8e329
  Robert Shaw authored Dec 28, 2024
```
Signed-off-by: rshaw@neuralmagic.com <rshaw@neuralmagic.com>
```
  4fb8e329
- [V1] [4/N] API Server: ZMQ/MP Utilities (#11541) · df04dffa
  Robert Shaw authored Dec 27, 2024
  
  df04dffa
27 Dec, 2024 1 commit
- [V1][3/N] API Server: Reduce Task Switching + Handle Abort Properly (#11534) · 1b875a0e
  Robert Shaw authored Dec 27, 2024
  
  1b875a0e
25 Dec, 2024 1 commit
- [V1] Unify VLLM_ENABLE_V1_MULTIPROCESSING handling in RayExecutor (#11472) · 9832e557
  Rui Qiao authored Dec 24, 2024
  
  9832e557
23 Dec, 2024 1 commit
- [V1] TP Ray executor (#11107) · a491d6f5
  Rui Qiao authored Dec 23, 2024
```
Signed-off-by: Rui Qiao <ruisearch42@gmail.com>
```
  a491d6f5
21 Dec, 2024 2 commits
- [V1] Make AsyncLLMEngine v1-v0 opaque (#11383) · 584f0ae4
  Ricky Xu authored Dec 20, 2024
```
Signed-off-by: Ricky Xu <xuchen727@hotmail.com>
```
  584f0ae4
- [V1][Bugfix] Skip hashing empty or None mm_data (#11386) · dd2b5633
  Woosuk Kwon authored Dec 21, 2024
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  dd2b5633
19 Dec, 2024 3 commits

[Misc] Clean up and consolidate LRUCache (#11339) · cdf22afd
Cyrus Leung authored Dec 20, 2024
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
cdf22afd
[V1] Fix multimodal profiling for `Molmo` (#11325) · 7379b3d4
Roger Wang authored Dec 19, 2024
```
Signed-off-by: ywang96 <ywang@example.com>
Co-authored-by: ywang96 <ywang@example.com>
```
7379b3d4

[Feature] Add load generation config from model (#11164) · 5aef4980

Yanyi Liu authored Dec 19, 2024


Signed-off-by: liuyanyi <wolfsonliu@163.com>
Signed-off-by: Yanyi Liu <wolfsonliu@163.com>
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>

5aef4980

18 Dec, 2024 2 commits
- [V1] VLM - enable processor cache by default (#11305) · fdea8ec1
  Alexander Matveev authored Dec 18, 2024
```
Signed-off-by: Alexander Matveev <alexm@neuralmagic.com>
```
  fdea8ec1
- [V1] Prefix caching for vision language models (#11187) · bf8717eb
  Cody Yu authored Dec 17, 2024
```
Signed-off-by: Cody Yu <hao.yu.cody@gmail.com>
```
  bf8717eb
17 Dec, 2024 2 commits
- [V1][Core] Use weakref.finalize instead of atexit (#11242) · 2bfdbf2a
  Tyler Michael Smith authored Dec 17, 2024
```
Signed-off-by: Tyler Michael Smith <tyler@neuralmagic.com>
```
  2bfdbf2a
- [V1][VLM] Proper memory profiling for image language models (#11210) · 59c9b6eb
  Roger Wang authored Dec 16, 2024
```
Signed-off-by: Roger Wang <ywang@roblox.com>
Co-authored-by: ywang96 <ywang@example.com>
```
  59c9b6eb
14 Dec, 2024 4 commits
- Enable mypy checking on V1 code (#11105) · 6d917d0e
  Mark McLoughlin authored Dec 14, 2024
```
Signed-off-by: Mark McLoughlin <markmc@redhat.com>
```
  6d917d0e
- [V1][Bugfix] Fix V1 TP trust-remote-code (#11182) · ea7bd68d
  Tyler Michael Smith authored Dec 14, 2024
```
Signed-off-by: Tyler Michael Smith <tyler@neuralmagic.com>
```
  ea7bd68d
- [V1][Bugfix] Fix EngineCoreProc profile (#11185) · 4b5b8a6a
  Tyler Michael Smith authored Dec 13, 2024
```
Signed-off-by: Tyler Michael Smith <tyler@neuralmagic.com>
```
  4b5b8a6a
- [Core] V1: Use multiprocessing by default (#11074) · 4863e5fb
  Russell Bryant authored Dec 13, 2024
```
Signed-off-by: Russell Bryant <rbryant@redhat.com>
```
  4863e5fb
12 Dec, 2024 3 commits

[V1] Fix torch profiling for offline inference (#11125) · 4816d20a
Roger Wang authored Dec 12, 2024
```
Signed-off-by: Roger Wang <ywang@roblox.com>
```
4816d20a
[Core] cleanup zmq ipc sockets on exit (#11115) · ccede2b2
Russell Bryant authored Dec 11, 2024
```
Signed-off-by: Russell Bryant <rbryant@redhat.com>
```
ccede2b2

[V1] VLM preprocessor hashing (#11020) · 4e116833

Alexander Matveev authored Dec 11, 2024


Signed-off-by: Roger Wang <ywang@roblox.com>
Signed-off-by: Alexander Matveev <alexm@neuralmagic.com>
Co-authored-by: Michael Goin <michael@neuralmagic.com>
Co-authored-by: Roger Wang <ywang@roblox.com>

4e116833

11 Dec, 2024 4 commits
- [V1][Core] Remove should_shutdown to simplify core process termination (#11113) · 452a723b
  Tyler Michael Smith authored Dec 11, 2024
```
Signed-off-by: Tyler Michael Smith <tyler@neuralmagic.com>
```
  452a723b
- [Doc] Update docs to refer to pooling models (#11093) · cad5c0a6
  Cyrus Leung authored Dec 11, 2024
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  cad5c0a6
- [Misc] Split up pooling tasks (#10820) · 8f10d5e3
  Cyrus Leung authored Dec 11, 2024
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  8f10d5e3
- [Core] v1: Use atexit to handle engine core client shutdown (#11076) · 61b1d2f6
  Russell Bryant authored Dec 11, 2024
```
Signed-off-by: Russell Bryant <rbryant@redhat.com>
```
  61b1d2f6
10 Dec, 2024 1 commit
- [Bugfix] Backport request id validation to v0 (#11036) · 9b9cef31
  Joe Runde authored Dec 10, 2024
```
Signed-off-by: Joe Runde <Joseph.Runde@ibm.com>
```
  9b9cef31