Commits · 60fb4f3bcfce9c84e09ba61e4b59bb1abe19953d · OpenDAS / vllm_cscc

21 Dec, 2024 1 commit
- [V1] Make AsyncLLMEngine v1-v0 opaque (#11383) · 584f0ae4
  Ricky Xu authored Dec 20, 2024
```
Signed-off-by: Ricky Xu <xuchen727@hotmail.com>
```
  584f0ae4
20 Dec, 2024 1 commit
- [Core] Loading model from S3 using RunAI Model Streamer as optional loader (#10192) · 995f5623
  omer-dayan authored Dec 20, 2024
```
Signed-off-by: OmerD <omer@run.ai>
```
  995f5623
19 Dec, 2024 1 commit

[Feature] Add load generation config from model (#11164) · 5aef4980

Yanyi Liu authored Dec 19, 2024


Signed-off-by: liuyanyi <wolfsonliu@163.com>
Signed-off-by: Yanyi Liu <wolfsonliu@163.com>
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>

5aef4980

18 Dec, 2024 3 commits
- [V1] VLM - enable processor cache by default (#11305) · fdea8ec1
  Alexander Matveev authored Dec 18, 2024
```
Signed-off-by: Alexander Matveev <alexm@neuralmagic.com>
```
  fdea8ec1
- [Bugfix] Restore support for larger block sizes (#11259) · 866fa455
  Konrad Zawora authored Dec 18, 2024
```
Signed-off-by: Konrad Zawora <kzawora@habana.ai>
```
  866fa455
- [V1] Prefix caching for vision language models (#11187) · bf8717eb
  Cody Yu authored Dec 17, 2024
```
Signed-off-by: Cody Yu <hao.yu.cody@gmail.com>
```
  bf8717eb
17 Dec, 2024 2 commits
- [Bugfix] Fix request cancellation without polling (#11190) · 2d1b9baa
  Joe Runde authored Dec 17, 2024
  
  2d1b9baa
- [Platform] platform agnostic for EngineArgs initialization (#11225) · e88db68c
  wangxiyuan authored Dec 17, 2024
```
Signed-off-by: wangxiyuan <wangxiyuan1007@gmail.com>
```
  e88db68c
16 Dec, 2024 2 commits
- [core] overhaul memory profiling and fix backward compatibility (#10511) · 551603fe
  youkaichao authored Dec 16, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  551603fe
- [Bugfix] Fix block size validation (#10938) · 69ba344d
  chenqianfzh authored Dec 15, 2024
  
  69ba344d
14 Dec, 2024 1 commit
- [Frontend] Add `logits_processors` as an extra completion argument (#11150) · 9c3dadd1
  Brad Hilton authored Dec 14, 2024
```
Signed-off-by: Brad Hilton <brad.hilton.nw@gmail.com>
```
  9c3dadd1
13 Dec, 2024 2 commits
- [Frontend] Separate pooling APIs in offline inference (#11129) · eeec9e33
  Cyrus Leung authored Dec 13, 2024
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  eeec9e33
- [ROCm][AMD] Disable auto enabling chunked prefill on ROCm (#11146) · 00c1bde5
  Gregory Shtrasberg authored Dec 13, 2024
```
Signed-off-by: Gregory Shtrasberg <Gregory.Shtrasberg@amd.com>
```
  00c1bde5
12 Dec, 2024 2 commits

Fix logging of the vLLM Config (#11143) · 9f3974a3
Jeremy Arnold authored Dec 12, 2024

9f3974a3

[V1] VLM preprocessor hashing (#11020) · 4e116833

Alexander Matveev authored Dec 11, 2024


Signed-off-by: Roger Wang <ywang@roblox.com>
Signed-off-by: Alexander Matveev <alexm@neuralmagic.com>
Co-authored-by: Michael Goin <michael@neuralmagic.com>
Co-authored-by: Roger Wang <ywang@roblox.com>

4e116833

11 Dec, 2024 2 commits
- [Doc] Update docs to refer to pooling models (#11093) · cad5c0a6
  Cyrus Leung authored Dec 11, 2024
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  cad5c0a6
- [Misc] Split up pooling tasks (#10820) · 8f10d5e3
  Cyrus Leung authored Dec 11, 2024
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  8f10d5e3
10 Dec, 2024 4 commits
- [V1][Bugfix] Always set enable_chunked_prefill = True for V1 (#11061) · 134810b3
  Woosuk Kwon authored Dec 10, 2024
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  134810b3
- [Bugfix] Backport request id validation to v0 (#11036) · 9b9cef31
  Joe Runde authored Dec 10, 2024
```
Signed-off-by: Joe Runde <Joseph.Runde@ibm.com>
```
  9b9cef31
- monitor metrics of tokens per step using cudagraph batchsizes (#11031) · ebf77806
  youkaichao authored Dec 09, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  ebf77806
- [Bugfix] Fix usage of `deprecated` decorator (#11025) · 391d7b27
  Cyrus Leung authored Dec 10, 2024
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  391d7b27
09 Dec, 2024 1 commit
- [misc] clean up and unify logging (#10999) · 46004e83
  youkaichao authored Dec 08, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  46004e83
08 Dec, 2024 2 commits
- [V1] Initial support of multimodal models for V1 re-arch (#10699) · a11f3265
  Roger Wang authored Dec 08, 2024
```
Signed-off-by: Roger Wang <ywang@roblox.com>
```
  a11f3265
- [torch.compile] allow candidate compile sizes (#10984) · fd57d2b5
  youkaichao authored Dec 08, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  fd57d2b5
07 Dec, 2024 1 commit
- [Core] Cleanup startup logging a bit (#10961) · 69d357ba
  Russell Bryant authored Dec 06, 2024
```
Signed-off-by: Russell Bryant <rbryant@redhat.com>
```
  69d357ba
06 Dec, 2024 1 commit

[torch.compile] add logging for compilation time (#10941) · b031a455

youkaichao authored Dec 06, 2024


Signed-off-by: youkaichao <youkaichao@gmail.com>
Co-authored-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>

b031a455

05 Dec, 2024 1 commit
- [Doc] Create a new "Usage" section (#10827) · aa39a8e1
  Cyrus Leung authored Dec 05, 2024
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  aa39a8e1
04 Dec, 2024 1 commit
- [LoRA] Change lora_tokenizers capacity (#10796) · 01d079fd
  Xin Yang authored Dec 04, 2024
```
Signed-off-by: Xin Yang <xyang19@gmail.com>
```
  01d079fd
03 Dec, 2024 3 commits
- [Frontend] correctly record prefill and decode time metrics (#10853) · 7c32b686
  tomeras91 authored Dec 03, 2024
```
Signed-off-by: Tomer Asida <tomera@ai21.com>
```
  7c32b686
- [Core][Performance] Add XGrammar support for guided decoding and set it as default (#10785) · 9323a315
  Aaron Pham authored Dec 03, 2024
```
Signed-off-by: Aaron Pham <contact@aarnphm.xyz>
Signed-off-by: mgoin <michael@neuralmagic.com>
Co-authored-by: mgoin <michael@neuralmagic.com>
```
  9323a315
- [Misc] Remove deprecated names (#10817) · 3257d449
  Cyrus Leung authored Dec 03, 2024
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  3257d449
02 Dec, 2024 2 commits

[core] Avoid metrics log noise when idle - include speculative decodi… (#10809) · b7954776
cduk authored Dec 02, 2024

b7954776

[Core] Implement disagg prefill by StatelessProcessGroup (#10502) · 0590ec3f

Kuntai Du authored Dec 01, 2024



This PR provides initial support for single-node disaggregated prefill in 1P1D scenario.
Signed-off-by: KuntaiDu <kuntai@uchicago.edu>
Co-authored-by: ApostaC <yihua98@uchicago.edu>
Co-authored-by: YaoJiayi <120040070@link.cuhk.edu.cn>

0590ec3f

01 Dec, 2024 1 commit
- [Misc] Rename embedding classes to pooling (#10801) · d2f058e7
  Cyrus Leung authored Dec 01, 2024
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  d2f058e7
28 Nov, 2024 1 commit
- [Bug][CLI] Allow users to disable prefix caching explicitly (#10724) · d9b4b3f0
  Ricky Xu authored Nov 27, 2024
```
Signed-off-by: rickyx <rickyx@anyscale.com>
```
  d9b4b3f0
27 Nov, 2024 1 commit
- [Bugfix][Mamba] Fix Multistep on Mamba-like models (#10705) · 197b4484
  Mor Zusman authored Nov 27, 2024
```
Signed-off-by: mzusman <mor.zusmann@gmail.com>
```
  197b4484
26 Nov, 2024 2 commits
- [Bugfix] Fix using `-O[0,3]` with LLM entrypoint (#10677) · 9a99273b
  Michael Goin authored Nov 26, 2024
```
Signed-off-by: mgoin <michael@neuralmagic.com>
```
  9a99273b
- [v1] EngineArgs for better config handling for v1 (#10382) · 519e8e41
  Ricky Xu authored Nov 25, 2024
```
Signed-off-by: rickyx <rickyx@anyscale.com>
```
  519e8e41
25 Nov, 2024 2 commits
- [Bugfix] Fix chunked prefill with model dtype float32 on Turing Devices (#9850) · c27df94e
  Wallas Henrique authored Nov 25, 2024
```
Signed-off-by: Wallas Santos <wallashss@ibm.com>
Co-authored-by: Michael Goin <michael@neuralmagic.com>
```
  c27df94e
- [misc] add torch.compile compatibility check (#10618) · 25d806e9
  youkaichao authored Nov 24, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  25d806e9