Commits · 42de2cefcbfc1e640f3337b36d5dddfd3133ef0e · OpenDAS / vllm_cscc

22 Jul, 2024 1 commit
- [Misc] Add a wrapper for torch.inference_mode (#6618) · 42de2cef
  Woosuk Kwon authored Jul 21, 2024
  
  42de2cef
20 Jul, 2024 2 commits
- [Misc] Consolidate and optimize logic for building padded tensors (#6541) · 9042d683
  Cyrus Leung authored Jul 20, 2024
  
  9042d683
- [Core] Allow specifying custom Executor (#6557) · 7bd82002
  Antoni Baum authored Jul 19, 2024
  
  7bd82002
19 Jul, 2024 1 commit
- [Core] Multiprocessing Pipeline Parallel support (#6130) · b5672a11
  Nick Hill authored Jul 18, 2024
```
Co-authored-by: Murali Andoorveedu <muralidhar.andoorveedu@centml.ai>
```
  b5672a11
18 Jul, 2024 3 commits
- [TPU] Refactor TPU worker & model runner (#6506) · 4634c872
  Woosuk Kwon authored Jul 18, 2024
  
  4634c872
- [Core] Introduce SPMD worker execution using Ray accelerated DAG (#6032) · 61e59274
  Rui Qiao authored Jul 17, 2024
```
Signed-off-by: Rui Qiao <ruisearch42@gmail.com>
Co-authored-by: Stephanie Wang <swang@cs.berkeley.edu>
```
  61e59274
- [core][model] yet another cpu offload implementation (#6496) · 1c27d25f
  youkaichao authored Jul 17, 2024
```
Co-authored-by: Michael Goin <michael@neuralmagic.com>
```
  1c27d25f
17 Jul, 2024 3 commits
- [Core] Refactor _prepare_model_input_tensors - take 2 (#6164) · 2fa4623d
  Cody Yu authored Jul 17, 2024
  
  2fa4623d
- [Misc] Use `torch.Tensor` for type annotation (#6505) · a9a2e74d
  Woosuk Kwon authored Jul 17, 2024
  
  a9a2e74d
- [TPU] Remove multi-modal args in TPU backend (#6504) · e09ce759
  Woosuk Kwon authored Jul 17, 2024
  
  e09ce759
16 Jul, 2024 2 commits
- [Hardware][TPU] Support MoE with Pallas GMM kernel (#6457) · c467dff2
  Woosuk Kwon authored Jul 16, 2024
  
  c467dff2
- [CI/Build] vLLM cache directory for images (#6444) · d9701151
  Cyrus Leung authored Jul 16, 2024
  
  d9701151
12 Jul, 2024 1 commit
- [ROCm][AMD] unify CUDA_VISIBLE_DEVICES usage in cuda/rocm (#6352) · b6c16cf8
  Hongxia Yang authored Jul 12, 2024
  
  b6c16cf8
11 Jul, 2024 1 commit
- [Misc] refactor(config): clean up unused code (#6320) · 3963a533
  aniaan authored Jul 11, 2024
  
  3963a533
10 Jul, 2024 1 commit
- [Speculative Decoding] Medusa Implementation with Top-1 proposer (#4978) · 2416b26e
  Abhinav Goyal authored Jul 10, 2024
  
  2416b26e
09 Jul, 2024 2 commits
- [CORE] Adding support for insertion of soft-tuned prompts (#4645) · 4d6ada94
  Swapnil Parekh authored Jul 09, 2024
```
Co-authored-by: Swapnil Parekh <swapnilp@ibm.com>
Co-authored-by: Joe G <joseph.granados@h2o.ai>
Co-authored-by: Antoni Baum <antoni.baum@protonmail.com>
```
  4d6ada94
- [Bugfix][TPU] Add missing None to model input (#6245) · 5d5b4c5f
  Woosuk Kwon authored Jul 09, 2024
  
  5d5b4c5f
04 Jul, 2024 3 commits
- [VLM] Calculate maximum number of multi-modal tokens by model (#6121) · ae96ef8f
  Cyrus Leung authored Jul 05, 2024
  
  ae96ef8f
- [Kernel][Model] logits_soft_cap for Gemma2 with flashinfer (#6051) · 69ec3ca1
  Lily Liu authored Jul 04, 2024
```
Co-authored-by: Simon Mo <simon.mo@hey.com>
```
  69ec3ca1
- [Hardware][Intel CPU] Adding intel openmp tunings in Docker file (#6008) · 81d7a50f
  Yuan authored Jul 05, 2024
```
Signed-off-by: Yuan Zhou <yuan.zhou@intel.com>
```
  81d7a50f
03 Jul, 2024 4 commits

[core][distributed] support n layers % pp size != 0 (#6115) · 3de6e6a3
youkaichao authored Jul 03, 2024

3de6e6a3

[vlm] Remove vision language config. (#6089) · d9e98f42

xwjiang2010 authored Jul 03, 2024


Signed-off-by: Xiaowei Jiang <xwjiang2010@gmail.com>
Co-authored-by: Roger Wang <ywang@roblox.com>

d9e98f42

[Core] Dynamic image size support for VLMs (#5276) · 9831aec4

Cyrus Leung authored Jul 03, 2024


Signed-off-by: Xiaowei Jiang <xwjiang2010@gmail.com>
Co-authored-by: Xiaowei Jiang <xwjiang2010@gmail.com>
Co-authored-by: ywang96 <ywang@roblox.com>
Co-authored-by: xwjiang2010 <87673679+xwjiang2010@users.noreply.github.com>
Co-authored-by: Roger Wang <136131678+ywang96@users.noreply.github.com>

9831aec4

[hardware][misc] introduce platform abstraction (#6080) · 482045ee
youkaichao authored Jul 02, 2024

482045ee

02 Jul, 2024 3 commits

[Model] Jamba support (#4115) · 9d6a8daa

Mor Zusman authored Jul 03, 2024


Signed-off-by: Muralidhar Andoorveedu <muralidhar.andoorveedu@centml.ai>
Co-authored-by: Erez Schwartz <erezs@ai21.com>
Co-authored-by: Mor Zusman <morz@ai21.com>
Co-authored-by: tomeras91 <57313761+tomeras91@users.noreply.github.com>
Co-authored-by: Tomer Asida <tomera@ai21.com>
Co-authored-by: Zhuohan Li <zhuohan123@gmail.com>
Co-authored-by: Muralidhar Andoorveedu <muralidhar.andoorveedu@centml.ai>

9d6a8daa

[Core] Pipeline Parallel Support (#4412) · c5832d2a
Murali Andoorveedu authored Jul 02, 2024
```
Signed-off-by: Muralidhar Andoorveedu <muralidhar.andoorveedu@centml.ai>
```
c5832d2a

[VLM] Remove `image_input_type` from VLM config (#5852) · 98d6682c

xwjiang2010 authored Jul 02, 2024


Signed-off-by: Xiaowei Jiang <xwjiang2010@gmail.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
Co-authored-by: Roger Wang <ywang@roblox.com>

98d6682c

01 Jul, 2024 1 commit
- [misc][cuda] use nvml to avoid accidentally cuda initialization (#6007) · 614aa512
  youkaichao authored Jun 30, 2024
  
  614aa512
29 Jun, 2024 2 commits
- [Bugfix][TPU] Fix pad slot id (#5977) · 7f83f40d
  Woosuk Kwon authored Jun 28, 2024
  
  7f83f40d
- [Bugfix][TPU] Fix TPU sampler output (#5978) · 54814fd8
  Woosuk Kwon authored Jun 28, 2024
  
  54814fd8
28 Jun, 2024 6 commits
- [Kernel] Flashinfer for prefill & decode, with Cudagraph support for decode (#4628) · 7041de43
  Lily Liu authored Jun 28, 2024
```
Co-authored-by: LiuXiaoxuanPKU &lt;llilyliupku@gmail.com&gt;, bong-furiosa <bongwon.jang@furiosa.ai>
```
  7041de43
- [Spec Decode] Introduce DraftModelRunner (#5799) · b2c62023
  Cody Yu authored Jun 28, 2024
  
  b2c62023
- [Hardware][Intel] OpenVINO vLLM backend (#5379) · 57f09a41
  Ilya Lavrenov authored Jun 28, 2024
  
  57f09a41
- [Core] Registry for processing model inputs (#5214) · 5cbe8d15
  Cyrus Leung authored Jun 28, 2024
```
Co-authored-by: ywang96 <ywang@roblox.com>
```
  5cbe8d15
- [Bugfix][Hardware][Intel CPU] Fix unpassed multi_modal_kwargs for CPU runner (#5956) · 0d0e3a42
  Isotr0py authored Jun 28, 2024
  
  0d0e3a42
- [Hardware][TPU] Optimize KV cache swapping (#5878) · f136da15
  Woosuk Kwon authored Jun 27, 2024
  
  f136da15
27 Jun, 2024 2 commits
- [Model] Add base class for LoRA-supported models (#5018) · 96354d6a
  Cyrus Leung authored Jun 27, 2024
  
  96354d6a
- [BugFix] Fix cuda graph for MLPSpeculator (#5875) · 2110557d
  Nick Hill authored Jun 26, 2024
```
Co-authored-by: Abhinav Goyal <abhinav.goyal@flipkart.com>
```
  2110557d
26 Jun, 2024 2 commits
- [Bugfix][TPU] Fix CPU cache allocation (#5869) · f5c8628f
  Woosuk Kwon authored Jun 26, 2024
  
  f5c8628f
- [Hardware][TPU] Support parallel sampling & Swapping (#5855) · cbc53b6b
  Woosuk Kwon authored Jun 26, 2024
  
  cbc53b6b