Commits · 75f89dc44c6e44cc28bae59d5b40a588735b507b · OpenDAS / vllm_cscc

10 Dec, 2024 3 commits
- [torch.compile] add a flag to track batchsize statistics (#11059) · 75f89dc4
  youkaichao authored Dec 10, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  75f89dc4
- [Bugfix] Backport request id validation to v0 (#11036) · 9b9cef31
  Joe Runde authored Dec 10, 2024
```
Signed-off-by: Joe Runde <Joseph.Runde@ibm.com>
```
  9b9cef31
- [V1] Multiprocessing Tensor Parallel Support for v1 (#9856) · 28b3a1c7
  Tyler Michael Smith authored Dec 10, 2024
```
Signed-off-by: Tyler Michael Smith <tyler@neuralmagic.com>
```
  28b3a1c7
09 Dec, 2024 5 commits
- [V1] Do not store `None` in self.generators (#11038) · 6faec545
  Woosuk Kwon authored Dec 09, 2024
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  6faec545
- [v1] fix use compile sizes (#11000) · 1a2f8fb8
  youkaichao authored Dec 09, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  1a2f8fb8
- [V1] Further reduce CPU overheads in flash-attn (#10989) · 3b61cb45
  Woosuk Kwon authored Dec 09, 2024
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  3b61cb45
- [V1] Input Batch Relocation (#10962) · 25b79d9f
  Varun Sundar Rabindranath authored Dec 09, 2024
```
Signed-off-by: Varun Sundar Rabindranath <varun@neuralmagic.com>
Co-authored-by: Varun Sundar Rabindranath <varun@neuralmagic.com>
```
  25b79d9f
- [V1] Fix Detokenizer loading in `AsyncLLM` (#10997) · c6903579
  Roger Wang authored Dec 09, 2024
```
Signed-off-by: Roger Wang <ywang@roblox.com>
```
  c6903579
08 Dec, 2024 1 commit
- [V1] Initial support of multimodal models for V1 re-arch (#10699) · a11f3265
  Roger Wang authored Dec 08, 2024
```
Signed-off-by: Roger Wang <ywang@roblox.com>
```
  a11f3265
07 Dec, 2024 1 commit
- [3/N] Support and implement merged input processor for LLaVA model (#10676) · 955fa953
  Cyrus Leung authored Dec 07, 2024
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Co-authored-by: Roger Wang <ywang@roblox.com>
```
  955fa953
06 Dec, 2024 1 commit

[torch.compile] add logging for compilation time (#10941) · b031a455

youkaichao authored Dec 06, 2024


Signed-off-by: youkaichao <youkaichao@gmail.com>
Co-authored-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>

b031a455

05 Dec, 2024 1 commit
- [V1] Fix when max_model_len is not divisible by block_size (#10903) · 2a56e126
  Woosuk Kwon authored Dec 04, 2024
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  2a56e126
04 Dec, 2024 1 commit
- [LoRA] Change lora_tokenizers capacity (#10796) · 01d079fd
  Xin Yang authored Dec 04, 2024
```
Signed-off-by: Xin Yang <xyang19@gmail.com>
```
  01d079fd
03 Dec, 2024 2 commits
- [V1] VLM - Run the mm_mapper preprocessor in the frontend process (#10640) · 3bc94cab
  Alexander Matveev authored Dec 03, 2024
```
Signed-off-by: Roger Wang <ywang@roblox.com>
Co-authored-by: Michael Goin <michael@neuralmagic.com>
Co-authored-by: Roger Wang <ywang@roblox.com>
```
  3bc94cab
- [torch.compile] remove compilation_context and simplify code (#10838) · dc5ce861
  youkaichao authored Dec 02, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  dc5ce861
02 Dec, 2024 2 commits
- [misc] use out argument for flash attention (#10822) · a4c4daf3
  youkaichao authored Dec 02, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  a4c4daf3
- [Kernel] Use `out` arg in flash_attn_varlen_func (#10811) · 073a4bd1
  Woosuk Kwon authored Dec 01, 2024
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  073a4bd1
01 Dec, 2024 1 commit
- [Misc] Rename embedding classes to pooling (#10801) · d2f058e7
  Cyrus Leung authored Dec 01, 2024
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  d2f058e7
28 Nov, 2024 2 commits
- [V1] Optimize the CPU overheads in FlashAttention custom op (#10733) · 98f47f2a
  Woosuk Kwon authored Nov 28, 2024
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  98f47f2a
- [V1] Do not allocate beyond the max_model_len (#10730) · a79b1224
  Woosuk Kwon authored Nov 28, 2024
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  a79b1224
26 Nov, 2024 4 commits
- [V1] Refactor model executable interface for multimodal models (#10570) · 2f0a0a17
  Roger Wang authored Nov 26, 2024
```
Signed-off-by: Roger Wang <ywang@roblox.com>
```
  2f0a0a17
- [V1] Enable profile for LLMEngine (#10665) · 1f6584ee
  Kunshang Ji authored Nov 26, 2024
  
  1f6584ee
- custom allreduce + torch.compile (#10121) · 9a88f897
  Sage Moore authored Nov 26, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
Co-authored-by: youkaichao <youkaichao@gmail.com>
```
  9a88f897
- [v1] EngineArgs for better config handling for v1 (#10382) · 519e8e41
  Ricky Xu authored Nov 25, 2024
```
Signed-off-by: rickyx <rickyx@anyscale.com>
```
  519e8e41
23 Nov, 2024 2 commits
- [Bugfix] Avoid import AttentionMetadata explicitly in Mllama (#10593) · 04668ebe
  Isotr0py authored Nov 24, 2024
```
Signed-off-by: Isotr0py <2037008807@qq.com>
```
  04668ebe
- [V1] EngineCore supports profiling (#10564) · d345f409
  Zhonghua Deng authored Nov 23, 2024
```
Signed-off-by: Abatom <abzhonghua@gmail.com>
```
  d345f409
22 Nov, 2024 2 commits
- [v1] Refactor KVCacheManager for more hash input than token ids (#10507) · 97814fbf
  Ricky Xu authored Nov 22, 2024
```
Signed-off-by: rickyx <rickyx@anyscale.com>
Signed-off-by: Cody Yu <hao.yu.cody@gmail.com>
Co-authored-by: Cody Yu <hao.yu.cody@gmail.com>
```
  97814fbf
- [torch.compile] support all attention backends (#10558) · eebad39f
  youkaichao authored Nov 22, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  eebad39f
21 Nov, 2024 2 commits
- [V1] Fix Compilation config & Enable CUDA graph by default (#10528) · f9310cbd
  Woosuk Kwon authored Nov 21, 2024
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  f9310cbd
- [torch.compile] Inductor code caching fix (#10273) · 8b0fe06c
  Luka Govedič authored Nov 21, 2024
```
Signed-off-by: luka <luka@neuralmagic.com>
Signed-off-by: Luka Govedic <luka.govedic@gmail.com>
```
  8b0fe06c
19 Nov, 2024 1 commit
- [6/N] torch.compile rollout to users (#10437) · 803f37ea
  youkaichao authored Nov 19, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  803f37ea
17 Nov, 2024 1 commit
- [2/N][torch.compile] make compilation cfg part of vllm cfg (#10383) · 4fd93750
  youkaichao authored Nov 16, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  4fd93750
15 Nov, 2024 1 commit
- [Misc] Fix import error in tensorizer tests and cleanup some code (#10349) · b311efd0
  Cyrus Leung authored Nov 15, 2024
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  b311efd0
13 Nov, 2024 4 commits
- [1/N] Initial prototype for multi-modal processor (#10044) · 0b8bb86b
  Cyrus Leung authored Nov 13, 2024
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  0b8bb86b
- [V1] Add missing tokenizer options for `Detokenizer` (#10288) · bb7991aa
  Roger Wang authored Nov 13, 2024
```
Signed-off-by: Roger Wang <ywang@roblox.com>
```
  bb7991aa
- [V1] Support VLMs with fine-grained scheduling (#9871) · bbd3e869
  Woosuk Kwon authored Nov 12, 2024
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
Co-authored-by: Roger Wang <ywang@roblox.com>
```
  bbd3e869
- [V1] Fix CI tests on V1 engine (#10272) · 112fa0bb
  Woosuk Kwon authored Nov 12, 2024
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  112fa0bb
12 Nov, 2024 2 commits
- [V1] Enable Inductor when using piecewise CUDA graphs (#10268) · 1f55e057
  Woosuk Kwon authored Nov 12, 2024
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  1f55e057
- [V1] Use pickle for serializing EngineCoreRequest & Add multimodal inputs to... · 7c655279
  Woosuk Kwon authored Nov 12, 2024
```
[V1] Use pickle for serializing EngineCoreRequest & Add multimodal inputs to EngineCoreRequest (#10245)
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  7c655279
11 Nov, 2024 1 commit

[V1] `AsyncLLM` Implementation (#9826) · 6ace6fba

Robert Shaw authored Nov 11, 2024


Signed-off-by: Nick Hill <nickhill@us.ibm.com>
Signed-off-by: rshaw@neuralmagic.com <rshaw@neuralmagic.com>
Signed-off-by: Nick Hill <nhill@redhat.com>
Co-authored-by: Nick Hill <nickhill@us.ibm.com>
Co-authored-by: Varun Sundar Rabindranath <varun@neuralmagic.com>
Co-authored-by: Nick Hill <nhill@redhat.com>
Co-authored-by: Tyler Michael Smith <tyler@neuralmagic.com>

6ace6fba