Commits · 69d765f5a5bbbe1ea9843be19b9480660fc5bc8b · OpenDAS / vllm_cscc

17 Jan, 2025 1 commit
- [V1] Move more control of kv cache initialization from model_executor to EngineCore (#11960) · 69d765f5
  Chen Zhang authored Jan 17, 2025
```
Signed-off-by: Chen Zhang <zhangch99@outlook.com>
Co-authored-by: Cody Yu <hao.yu.cody@gmail.com>
```
  69d765f5
15 Jan, 2025 1 commit
- [V1][Core] Autotune encoder cache budget (#11895) · 70755e81
  Roger Wang authored Jan 15, 2025
```
Signed-off-by: Roger Wang <ywang@roblox.com>
```
  70755e81
12 Jan, 2025 1 commit
- [V1][Core][1/n] Logging and Metrics (#11962) · 9597a095
  Robert Shaw authored Jan 12, 2025
```
Signed-off-by: rshaw@neuralmagic.com <rshaw@neuralmagic.com>
```
  9597a095
03 Jan, 2025 2 commits
- [V1] Improve TP>1 Error Handling + Stack Trace (#11721) · 1543914c
  Robert Shaw authored Jan 03, 2025
```
Co-authored-by: Tyler Michael Smith <tyler@neuralmagic.com>
```
  1543914c
- [V1] Simplify Shutdown (#11659) · 80c751e7
  Robert Shaw authored Jan 03, 2025
  
  80c751e7
28 Dec, 2024 1 commit
- [V1] [4/N] API Server: ZMQ/MP Utilities (#11541) · df04dffa
  Robert Shaw authored Dec 27, 2024
  
  df04dffa
27 Dec, 2024 1 commit
- [V1][3/N] API Server: Reduce Task Switching + Handle Abort Properly (#11534) · 1b875a0e
  Robert Shaw authored Dec 27, 2024
  
  1b875a0e
18 Dec, 2024 1 commit
- [V1] Prefix caching for vision language models (#11187) · bf8717eb
  Cody Yu authored Dec 17, 2024
```
Signed-off-by: Cody Yu <hao.yu.cody@gmail.com>
```
  bf8717eb
14 Dec, 2024 4 commits
- Enable mypy checking on V1 code (#11105) · 6d917d0e
  Mark McLoughlin authored Dec 14, 2024
```
Signed-off-by: Mark McLoughlin <markmc@redhat.com>
```
  6d917d0e
- [V1][Bugfix] Fix V1 TP trust-remote-code (#11182) · ea7bd68d
  Tyler Michael Smith authored Dec 14, 2024
```
Signed-off-by: Tyler Michael Smith <tyler@neuralmagic.com>
```
  ea7bd68d
- [V1][Bugfix] Fix EngineCoreProc profile (#11185) · 4b5b8a6a
  Tyler Michael Smith authored Dec 13, 2024
```
Signed-off-by: Tyler Michael Smith <tyler@neuralmagic.com>
```
  4b5b8a6a
- [Core] V1: Use multiprocessing by default (#11074) · 4863e5fb
  Russell Bryant authored Dec 13, 2024
```
Signed-off-by: Russell Bryant <rbryant@redhat.com>
```
  4863e5fb
12 Dec, 2024 2 commits

[Core] cleanup zmq ipc sockets on exit (#11115) · ccede2b2
Russell Bryant authored Dec 11, 2024
```
Signed-off-by: Russell Bryant <rbryant@redhat.com>
```
ccede2b2

[V1] VLM preprocessor hashing (#11020) · 4e116833

Alexander Matveev authored Dec 11, 2024


Signed-off-by: Roger Wang <ywang@roblox.com>
Signed-off-by: Alexander Matveev <alexm@neuralmagic.com>
Co-authored-by: Michael Goin <michael@neuralmagic.com>
Co-authored-by: Roger Wang <ywang@roblox.com>

4e116833

11 Dec, 2024 2 commits
- [V1][Core] Remove should_shutdown to simplify core process termination (#11113) · 452a723b
  Tyler Michael Smith authored Dec 11, 2024
```
Signed-off-by: Tyler Michael Smith <tyler@neuralmagic.com>
```
  452a723b
- [Misc] Split up pooling tasks (#10820) · 8f10d5e3
  Cyrus Leung authored Dec 11, 2024
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  8f10d5e3
10 Dec, 2024 1 commit
- [V1] Multiprocessing Tensor Parallel Support for v1 (#9856) · 28b3a1c7
  Tyler Michael Smith authored Dec 10, 2024
```
Signed-off-by: Tyler Michael Smith <tyler@neuralmagic.com>
```
  28b3a1c7
06 Dec, 2024 1 commit

[torch.compile] add logging for compilation time (#10941) · b031a455

youkaichao authored Dec 06, 2024


Signed-off-by: youkaichao <youkaichao@gmail.com>
Co-authored-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>

b031a455

03 Dec, 2024 1 commit

[V1] VLM - Run the mm_mapper preprocessor in the frontend process (#10640) · 3bc94cab

Alexander Matveev authored Dec 03, 2024


Signed-off-by: Roger Wang <ywang@roblox.com>
Co-authored-by: Michael Goin <michael@neuralmagic.com>
Co-authored-by: Roger Wang <ywang@roblox.com>

3bc94cab

26 Nov, 2024 1 commit
- [v1] EngineArgs for better config handling for v1 (#10382) · 519e8e41
  Ricky Xu authored Nov 25, 2024
```
Signed-off-by: rickyx <rickyx@anyscale.com>
```
  519e8e41
23 Nov, 2024 1 commit
- [V1] EngineCore supports profiling (#10564) · d345f409
  Zhonghua Deng authored Nov 23, 2024
```
Signed-off-by: Abatom <abzhonghua@gmail.com>
```
  d345f409
13 Nov, 2024 2 commits
- [V1] Support VLMs with fine-grained scheduling (#9871) · bbd3e869
  Woosuk Kwon authored Nov 12, 2024
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
Co-authored-by: Roger Wang <ywang@roblox.com>
```
  bbd3e869
- [V1] Fix CI tests on V1 engine (#10272) · 112fa0bb
  Woosuk Kwon authored Nov 12, 2024
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  112fa0bb
12 Nov, 2024 1 commit

[V1] Use pickle for serializing EngineCoreRequest & Add multimodal inputs to... · 7c655279

Woosuk Kwon authored Nov 12, 2024


[V1] Use pickle for serializing EngineCoreRequest & Add multimodal inputs to EngineCoreRequest (#10245)
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>

7c655279

11 Nov, 2024 1 commit

[V1] `AsyncLLM` Implementation (#9826) · 6ace6fba

Robert Shaw authored Nov 11, 2024


Signed-off-by: Nick Hill <nickhill@us.ibm.com>
Signed-off-by: rshaw@neuralmagic.com <rshaw@neuralmagic.com>
Signed-off-by: Nick Hill <nhill@redhat.com>
Co-authored-by: Nick Hill <nickhill@us.ibm.com>
Co-authored-by: Varun Sundar Rabindranath <varun@neuralmagic.com>
Co-authored-by: Nick Hill <nhill@redhat.com>
Co-authored-by: Tyler Michael Smith <tyler@neuralmagic.com>

6ace6fba