Commits · acf092d34802b187f27daa8e1626f67552bde193 · OpenDAS / vllm_cscc

06 Dec, 2024 1 commit

[torch.compile] add logging for compilation time (#10941) · b031a455

youkaichao authored Dec 06, 2024


Signed-off-by: youkaichao <youkaichao@gmail.com>
Co-authored-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>

b031a455

05 Dec, 2024 1 commit
- [V1] Fix when max_model_len is not divisible by block_size (#10903) · 2a56e126
  Woosuk Kwon authored Dec 04, 2024
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  2a56e126
04 Dec, 2024 1 commit
- [LoRA] Change lora_tokenizers capacity (#10796) · 01d079fd
  Xin Yang authored Dec 04, 2024
```
Signed-off-by: Xin Yang <xyang19@gmail.com>
```
  01d079fd
03 Dec, 2024 2 commits
- [V1] VLM - Run the mm_mapper preprocessor in the frontend process (#10640) · 3bc94cab
  Alexander Matveev authored Dec 03, 2024
```
Signed-off-by: Roger Wang <ywang@roblox.com>
Co-authored-by: Michael Goin <michael@neuralmagic.com>
Co-authored-by: Roger Wang <ywang@roblox.com>
```
  3bc94cab
- [torch.compile] remove compilation_context and simplify code (#10838) · dc5ce861
  youkaichao authored Dec 02, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  dc5ce861
02 Dec, 2024 2 commits
- [misc] use out argument for flash attention (#10822) · a4c4daf3
  youkaichao authored Dec 02, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  a4c4daf3
- [Kernel] Use `out` arg in flash_attn_varlen_func (#10811) · 073a4bd1
  Woosuk Kwon authored Dec 01, 2024
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  073a4bd1
01 Dec, 2024 1 commit
- [Misc] Rename embedding classes to pooling (#10801) · d2f058e7
  Cyrus Leung authored Dec 01, 2024
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  d2f058e7
28 Nov, 2024 2 commits
- [V1] Optimize the CPU overheads in FlashAttention custom op (#10733) · 98f47f2a
  Woosuk Kwon authored Nov 28, 2024
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  98f47f2a
- [V1] Do not allocate beyond the max_model_len (#10730) · a79b1224
  Woosuk Kwon authored Nov 28, 2024
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  a79b1224
26 Nov, 2024 4 commits
- [V1] Refactor model executable interface for multimodal models (#10570) · 2f0a0a17
  Roger Wang authored Nov 26, 2024
```
Signed-off-by: Roger Wang <ywang@roblox.com>
```
  2f0a0a17
- [V1] Enable profile for LLMEngine (#10665) · 1f6584ee
  Kunshang Ji authored Nov 26, 2024
  
  1f6584ee
- custom allreduce + torch.compile (#10121) · 9a88f897
  Sage Moore authored Nov 26, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
Co-authored-by: youkaichao <youkaichao@gmail.com>
```
  9a88f897
- [v1] EngineArgs for better config handling for v1 (#10382) · 519e8e41
  Ricky Xu authored Nov 25, 2024
```
Signed-off-by: rickyx <rickyx@anyscale.com>
```
  519e8e41
23 Nov, 2024 2 commits
- [Bugfix] Avoid import AttentionMetadata explicitly in Mllama (#10593) · 04668ebe
  Isotr0py authored Nov 24, 2024
```
Signed-off-by: Isotr0py <2037008807@qq.com>
```
  04668ebe
- [V1] EngineCore supports profiling (#10564) · d345f409
  Zhonghua Deng authored Nov 23, 2024
```
Signed-off-by: Abatom <abzhonghua@gmail.com>
```
  d345f409
22 Nov, 2024 2 commits
- [v1] Refactor KVCacheManager for more hash input than token ids (#10507) · 97814fbf
  Ricky Xu authored Nov 22, 2024
```
Signed-off-by: rickyx <rickyx@anyscale.com>
Signed-off-by: Cody Yu <hao.yu.cody@gmail.com>
Co-authored-by: Cody Yu <hao.yu.cody@gmail.com>
```
  97814fbf
- [torch.compile] support all attention backends (#10558) · eebad39f
  youkaichao authored Nov 22, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  eebad39f
21 Nov, 2024 2 commits
- [V1] Fix Compilation config & Enable CUDA graph by default (#10528) · f9310cbd
  Woosuk Kwon authored Nov 21, 2024
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  f9310cbd
- [torch.compile] Inductor code caching fix (#10273) · 8b0fe06c
  Luka Govedič authored Nov 21, 2024
```
Signed-off-by: luka <luka@neuralmagic.com>
Signed-off-by: Luka Govedic <luka.govedic@gmail.com>
```
  8b0fe06c
19 Nov, 2024 1 commit
- [6/N] torch.compile rollout to users (#10437) · 803f37ea
  youkaichao authored Nov 19, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  803f37ea
17 Nov, 2024 1 commit
- [2/N][torch.compile] make compilation cfg part of vllm cfg (#10383) · 4fd93750
  youkaichao authored Nov 16, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  4fd93750
15 Nov, 2024 1 commit
- [Misc] Fix import error in tensorizer tests and cleanup some code (#10349) · b311efd0
  Cyrus Leung authored Nov 15, 2024
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  b311efd0
13 Nov, 2024 4 commits
- [1/N] Initial prototype for multi-modal processor (#10044) · 0b8bb86b
  Cyrus Leung authored Nov 13, 2024
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  0b8bb86b
- [V1] Add missing tokenizer options for `Detokenizer` (#10288) · bb7991aa
  Roger Wang authored Nov 13, 2024
```
Signed-off-by: Roger Wang <ywang@roblox.com>
```
  bb7991aa
- [V1] Support VLMs with fine-grained scheduling (#9871) · bbd3e869
  Woosuk Kwon authored Nov 12, 2024
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
Co-authored-by: Roger Wang <ywang@roblox.com>
```
  bbd3e869
- [V1] Fix CI tests on V1 engine (#10272) · 112fa0bb
  Woosuk Kwon authored Nov 12, 2024
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  112fa0bb
12 Nov, 2024 2 commits
- [V1] Enable Inductor when using piecewise CUDA graphs (#10268) · 1f55e057
  Woosuk Kwon authored Nov 12, 2024
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  1f55e057
- [V1] Use pickle for serializing EngineCoreRequest & Add multimodal inputs to... · 7c655279
  Woosuk Kwon authored Nov 12, 2024
```
[V1] Use pickle for serializing EngineCoreRequest & Add multimodal inputs to EngineCoreRequest (#10245)
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  7c655279
11 Nov, 2024 6 commits

[V1] `AsyncLLM` Implementation (#9826) · 6ace6fba

Robert Shaw authored Nov 11, 2024


Signed-off-by: Nick Hill <nickhill@us.ibm.com>
Signed-off-by: rshaw@neuralmagic.com <rshaw@neuralmagic.com>
Signed-off-by: Nick Hill <nhill@redhat.com>
Co-authored-by: Nick Hill <nickhill@us.ibm.com>
Co-authored-by: Varun Sundar Rabindranath <varun@neuralmagic.com>
Co-authored-by: Nick Hill <nhill@redhat.com>
Co-authored-by: Tyler Michael Smith <tyler@neuralmagic.com>

6ace6fba

[V1] Enable custom ops with piecewise CUDA graphs (#10228) · 9d5b4e4d
Woosuk Kwon authored Nov 11, 2024
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
9d5b4e4d
[V1] Use custom ops for piecewise CUDA graphs (#10227) · fe15729a
Woosuk Kwon authored Nov 11, 2024
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
fe15729a
[V1] Do not use inductor for piecewise CUDA graphs (#10225) · d7a4f220
Woosuk Kwon authored Nov 11, 2024
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
d7a4f220
[V1] Fix detokenizer ports (#10224) · f9dadfbe
Woosuk Kwon authored Nov 11, 2024
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
f9dadfbe
[V1] Allow `tokenizer_mode` and `trust_remote_code` for Detokenizer (#10211) · 5fb1f935
Roger Wang authored Nov 11, 2024
```
Signed-off-by: Roger Wang <ywang@roblox.com>
```
5fb1f935

09 Nov, 2024 1 commit

[Frontend][Core] Override HF `config.json` via CLI (#5836) · b09895a6

Krishna Mandal authored Nov 09, 2024


Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Co-authored-by: DarkLight1337 <tlleungac@connect.ust.hk>

b09895a6

08 Nov, 2024 4 commits
- [V1] Fix non-cudagraph op name (#10166) · b5815c84
  Woosuk Kwon authored Nov 08, 2024
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  b5815c84
- [V1]Enable APC by default only for text models (#10148) · 208ce622
  Roger Wang authored Nov 08, 2024
```
Signed-off-by: Roger Wang <ywang@roblox.com>
```
  208ce622
- [V1] Prefix caching (take 2) (#9972) · 201fc077
  Cody Yu authored Nov 07, 2024
```
Signed-off-by: Cody Yu <hao.yu.cody@gmail.com>
```
  201fc077
- [V1] Add all_token_ids attribute to Request (#10135) · 42b4f46b
  Woosuk Kwon authored Nov 07, 2024
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  42b4f46b