Commits · 2fa4623d9e673faddb05018ac3f3d65780bb083d · OpenDAS / vllm_cscc

17 Jul, 2024 3 commits
- [Core] Refactor _prepare_model_input_tensors - take 2 (#6164) · 2fa4623d
  Cody Yu authored Jul 17, 2024
  
  2fa4623d
- [Misc] Use `torch.Tensor` for type annotation (#6505) · a9a2e74d
  Woosuk Kwon authored Jul 17, 2024
  
  a9a2e74d
- [TPU] Remove multi-modal args in TPU backend (#6504) · e09ce759
  Woosuk Kwon authored Jul 17, 2024
  
  e09ce759
16 Jul, 2024 2 commits
- [Hardware][TPU] Support MoE with Pallas GMM kernel (#6457) · c467dff2
  Woosuk Kwon authored Jul 16, 2024
  
  c467dff2
- [CI/Build] vLLM cache directory for images (#6444) · d9701151
  Cyrus Leung authored Jul 16, 2024
  
  d9701151
12 Jul, 2024 1 commit
- [ROCm][AMD] unify CUDA_VISIBLE_DEVICES usage in cuda/rocm (#6352) · b6c16cf8
  Hongxia Yang authored Jul 12, 2024
  
  b6c16cf8
11 Jul, 2024 1 commit
- [Misc] refactor(config): clean up unused code (#6320) · 3963a533
  aniaan authored Jul 11, 2024
  
  3963a533
10 Jul, 2024 1 commit
- [Speculative Decoding] Medusa Implementation with Top-1 proposer (#4978) · 2416b26e
  Abhinav Goyal authored Jul 10, 2024
  
  2416b26e
09 Jul, 2024 2 commits
- [CORE] Adding support for insertion of soft-tuned prompts (#4645) · 4d6ada94
  Swapnil Parekh authored Jul 09, 2024
```
Co-authored-by: Swapnil Parekh <swapnilp@ibm.com>
Co-authored-by: Joe G <joseph.granados@h2o.ai>
Co-authored-by: Antoni Baum <antoni.baum@protonmail.com>
```
  4d6ada94
- [Bugfix][TPU] Add missing None to model input (#6245) · 5d5b4c5f
  Woosuk Kwon authored Jul 09, 2024
  
  5d5b4c5f
04 Jul, 2024 3 commits
- [VLM] Calculate maximum number of multi-modal tokens by model (#6121) · ae96ef8f
  Cyrus Leung authored Jul 05, 2024
  
  ae96ef8f
- [Kernel][Model] logits_soft_cap for Gemma2 with flashinfer (#6051) · 69ec3ca1
  Lily Liu authored Jul 04, 2024
```
Co-authored-by: Simon Mo <simon.mo@hey.com>
```
  69ec3ca1
- [Hardware][Intel CPU] Adding intel openmp tunings in Docker file (#6008) · 81d7a50f
  Yuan authored Jul 05, 2024
```
Signed-off-by: Yuan Zhou <yuan.zhou@intel.com>
```
  81d7a50f
03 Jul, 2024 4 commits

[core][distributed] support n layers % pp size != 0 (#6115) · 3de6e6a3
youkaichao authored Jul 03, 2024

3de6e6a3

[vlm] Remove vision language config. (#6089) · d9e98f42

xwjiang2010 authored Jul 03, 2024


Signed-off-by: Xiaowei Jiang <xwjiang2010@gmail.com>
Co-authored-by: Roger Wang <ywang@roblox.com>

d9e98f42

[Core] Dynamic image size support for VLMs (#5276) · 9831aec4

Cyrus Leung authored Jul 03, 2024


Signed-off-by: Xiaowei Jiang <xwjiang2010@gmail.com>
Co-authored-by: Xiaowei Jiang <xwjiang2010@gmail.com>
Co-authored-by: ywang96 <ywang@roblox.com>
Co-authored-by: xwjiang2010 <87673679+xwjiang2010@users.noreply.github.com>
Co-authored-by: Roger Wang <136131678+ywang96@users.noreply.github.com>

9831aec4

[hardware][misc] introduce platform abstraction (#6080) · 482045ee
youkaichao authored Jul 02, 2024

482045ee

02 Jul, 2024 3 commits

[Model] Jamba support (#4115) · 9d6a8daa

Mor Zusman authored Jul 03, 2024


Signed-off-by: Muralidhar Andoorveedu <muralidhar.andoorveedu@centml.ai>
Co-authored-by: Erez Schwartz <erezs@ai21.com>
Co-authored-by: Mor Zusman <morz@ai21.com>
Co-authored-by: tomeras91 <57313761+tomeras91@users.noreply.github.com>
Co-authored-by: Tomer Asida <tomera@ai21.com>
Co-authored-by: Zhuohan Li <zhuohan123@gmail.com>
Co-authored-by: Muralidhar Andoorveedu <muralidhar.andoorveedu@centml.ai>

9d6a8daa

[Core] Pipeline Parallel Support (#4412) · c5832d2a
Murali Andoorveedu authored Jul 02, 2024
```
Signed-off-by: Muralidhar Andoorveedu <muralidhar.andoorveedu@centml.ai>
```
c5832d2a

[VLM] Remove `image_input_type` from VLM config (#5852) · 98d6682c

xwjiang2010 authored Jul 02, 2024


Signed-off-by: Xiaowei Jiang <xwjiang2010@gmail.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
Co-authored-by: Roger Wang <ywang@roblox.com>

98d6682c

01 Jul, 2024 1 commit
- [misc][cuda] use nvml to avoid accidentally cuda initialization (#6007) · 614aa512
  youkaichao authored Jun 30, 2024
  
  614aa512
29 Jun, 2024 2 commits
- [Bugfix][TPU] Fix pad slot id (#5977) · 7f83f40d
  Woosuk Kwon authored Jun 28, 2024
  
  7f83f40d
- [Bugfix][TPU] Fix TPU sampler output (#5978) · 54814fd8
  Woosuk Kwon authored Jun 28, 2024
  
  54814fd8
28 Jun, 2024 6 commits
- [Kernel] Flashinfer for prefill & decode, with Cudagraph support for decode (#4628) · 7041de43
  Lily Liu authored Jun 28, 2024
```
Co-authored-by: LiuXiaoxuanPKU &lt;llilyliupku@gmail.com&gt;, bong-furiosa <bongwon.jang@furiosa.ai>
```
  7041de43
- [Spec Decode] Introduce DraftModelRunner (#5799) · b2c62023
  Cody Yu authored Jun 28, 2024
  
  b2c62023
- [Hardware][Intel] OpenVINO vLLM backend (#5379) · 57f09a41
  Ilya Lavrenov authored Jun 28, 2024
  
  57f09a41
- [Core] Registry for processing model inputs (#5214) · 5cbe8d15
  Cyrus Leung authored Jun 28, 2024
```
Co-authored-by: ywang96 <ywang@roblox.com>
```
  5cbe8d15
- [Bugfix][Hardware][Intel CPU] Fix unpassed multi_modal_kwargs for CPU runner (#5956) · 0d0e3a42
  Isotr0py authored Jun 28, 2024
  
  0d0e3a42
- [Hardware][TPU] Optimize KV cache swapping (#5878) · f136da15
  Woosuk Kwon authored Jun 27, 2024
  
  f136da15
27 Jun, 2024 2 commits
- [Model] Add base class for LoRA-supported models (#5018) · 96354d6a
  Cyrus Leung authored Jun 27, 2024
  
  96354d6a
- [BugFix] Fix cuda graph for MLPSpeculator (#5875) · 2110557d
  Nick Hill authored Jun 26, 2024
```
Co-authored-by: Abhinav Goyal <abhinav.goyal@flipkart.com>
```
  2110557d
26 Jun, 2024 4 commits
- [Bugfix][TPU] Fix CPU cache allocation (#5869) · f5c8628f
  Woosuk Kwon authored Jun 26, 2024
  
  f5c8628f
- [Hardware][TPU] Support parallel sampling & Swapping (#5855) · cbc53b6b
  Woosuk Kwon authored Jun 26, 2024
  
  cbc53b6b
- [Bugfix][TPU] Fix KV cache size calculation (#5860) · 3439c5a8
  Woosuk Kwon authored Jun 26, 2024
  
  3439c5a8
- [Core] Refactor Worker and ModelRunner to consolidate control plane communication (#5408) · dda48115
  Stephanie Wang authored Jun 25, 2024
```
Signed-off-by: Stephanie Wang <swang@cs.berkeley.edu>
Signed-off-by: Stephanie <swang@anyscale.com>
Co-authored-by: Stephanie <swang@anyscale.com>
```
  dda48115
25 Jun, 2024 4 commits
- [Hardware][TPU] Raise errors for unsupported sampling params (#5850) · f178e56c
  Woosuk Kwon authored Jun 25, 2024
  
  f178e56c
- [Hardware][AMD][CI/Build][Doc] Upgrade to ROCm 6.1, Dockerfile improvements, test fixes (#5422) · dd793d1d
  Matt Wong authored Jun 25, 2024
  
  dd793d1d
- [Hardware][TPU] Refactor TPU backend (#5831) · bc34937d
  Woosuk Kwon authored Jun 25, 2024
  
  bc34937d
- [Misc] Remove useless code in cpu_worker (#5824) · 7b993143
  Jie Fu (傅杰) authored Jun 26, 2024
  
  7b993143
21 Jun, 2024 1 commit
- [LoRA] Add support for pinning lora adapters in the LRU cache (#5603) · f5dda63e
  rohithkrn authored Jun 21, 2024
  
  f5dda63e