Commits · 4fa3e3334978dce74eba296ee8cc2e970ed20e5e · OpenDAS / vllm_cscc

20 Oct, 2024 1 commit
- [Kernel] Support sliding window in flash attention backend (#9403) · 4fa3e333
  Chen Zhang authored Oct 20, 2024
  
  4fa3e333
14 Oct, 2024 1 commit
- [TPU] Fix TPU SMEM OOM by Pallas paged attention kernel (#9350) · 473e7b36
  Woosuk Kwon authored Oct 14, 2024
  
  473e7b36
11 Oct, 2024 3 commits
- [Bugfix] Sets `is_first_step_output` for TPUModelRunner (#9202) · c6cf9295
  Allen Wang authored Oct 11, 2024
  
  c6cf9295
- [Model] Support Mamba (#6484) · 7342a7d7
  Tyler Michael Smith authored Oct 11, 2024
  
  7342a7d7
- [misc] hide best_of from engine (#9261) · cbc2ef55
  youkaichao authored Oct 10, 2024
```
Co-authored-by: Brendan Wong <bjwpokemon@gmail.com>
```
  cbc2ef55
07 Oct, 2024 1 commit
- [core] remove beam search from the core (#9105) · 18b296fd
  youkaichao authored Oct 06, 2024
  
  18b296fd
27 Sep, 2024 1 commit
- [torch.compile] use empty tensor instead of None for profiling (#8875) · a9b15c60
  youkaichao authored Sep 27, 2024
  
  a9b15c60
14 Sep, 2024 1 commit
- [TPU] Implement multi-step scheduling (#8489) · 50e9ec41
  Woosuk Kwon authored Sep 14, 2024
  
  50e9ec41
07 Sep, 2024 1 commit
- [tpu][misc] fix typo (#8260) · ce2702a9
  youkaichao authored Sep 06, 2024
  
  ce2702a9
03 Sep, 2024 1 commit
- [TPU][Bugfix] Fix next_token_ids shape (#8128) · 0af3abe3
  Woosuk Kwon authored Sep 03, 2024
  
  0af3abe3
30 Aug, 2024 2 commits
- [TPU] Async output processing for TPU (#8011) · 80c7b089
  Woosuk Kwon authored Aug 29, 2024
  
  80c7b089
- [Core] Logprobs support in Multi-step (#7652) · 428dd144
  afeldman-nm authored Aug 29, 2024
  
  428dd144
28 Aug, 2024 1 commit
- [torch.compile] avoid Dynamo guard evaluation overhead (#7898) · ce6bf3a2
  youkaichao authored Aug 28, 2024
```
Co-authored-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  ce6bf3a2
19 Aug, 2024 1 commit
- [TPU] Remove redundant input tensor cloning (#7660) · 43735bf5
  Woosuk Kwon authored Aug 19, 2024
  
  43735bf5
18 Aug, 2024 1 commit
- [TPU] Use mark_dynamic only for dummy run (#7634) · 0c2fa50b
  Woosuk Kwon authored Aug 18, 2024
  
  0c2fa50b
17 Aug, 2024 1 commit
- [VLM] Refactor `MultiModalConfig` initialization and profiling (#7530) · bbf55c48
  Roger Wang authored Aug 17, 2024
  
  bbf55c48
11 Aug, 2024 1 commit
- [TPU] Use mark_dynamic to reduce compilation time (#7340) · 90bab18f
  Woosuk Kwon authored Aug 10, 2024
  
  90bab18f
30 Jul, 2024 1 commit
- [TPU] Fix greedy decoding (#6933) · 6e063ea3
  Woosuk Kwon authored Jul 30, 2024
  
  6e063ea3
27 Jul, 2024 2 commits
- [TPU] Reduce compilation time & Upgrade PyTorch XLA version (#6856) · fad5576c
  Woosuk Kwon authored Jul 27, 2024
  
  fad5576c
- [Hardware][TPU] Implement tensor parallelism with Ray (#5871) · 52f07e3d
  Woosuk Kwon authored Jul 26, 2024
  
  52f07e3d
18 Jul, 2024 1 commit
- [TPU] Refactor TPU worker & model runner (#6506) · 4634c872
  Woosuk Kwon authored Jul 18, 2024
  
  4634c872
17 Jul, 2024 1 commit
- [TPU] Remove multi-modal args in TPU backend (#6504) · e09ce759
  Woosuk Kwon authored Jul 17, 2024
  
  e09ce759
16 Jul, 2024 1 commit
- [Hardware][TPU] Support MoE with Pallas GMM kernel (#6457) · c467dff2
  Woosuk Kwon authored Jul 16, 2024
  
  c467dff2
09 Jul, 2024 1 commit
- [Bugfix][TPU] Add missing None to model input (#6245) · 5d5b4c5f
  Woosuk Kwon authored Jul 09, 2024
  
  5d5b4c5f
03 Jul, 2024 2 commits

[vlm] Remove vision language config. (#6089) · d9e98f42

xwjiang2010 authored Jul 03, 2024


Signed-off-by: Xiaowei Jiang <xwjiang2010@gmail.com>
Co-authored-by: Roger Wang <ywang@roblox.com>

d9e98f42

[Core] Dynamic image size support for VLMs (#5276) · 9831aec4

Cyrus Leung authored Jul 03, 2024


Signed-off-by: Xiaowei Jiang <xwjiang2010@gmail.com>
Co-authored-by: Xiaowei Jiang <xwjiang2010@gmail.com>
Co-authored-by: ywang96 <ywang@roblox.com>
Co-authored-by: xwjiang2010 <87673679+xwjiang2010@users.noreply.github.com>
Co-authored-by: Roger Wang <136131678+ywang96@users.noreply.github.com>

9831aec4

29 Jun, 2024 1 commit
- [Bugfix][TPU] Fix pad slot id (#5977) · 7f83f40d
  Woosuk Kwon authored Jun 28, 2024
  
  7f83f40d
28 Jun, 2024 1 commit
- [Spec Decode] Introduce DraftModelRunner (#5799) · b2c62023
  Cody Yu authored Jun 28, 2024
  
  b2c62023
26 Jun, 2024 1 commit
- [Hardware][TPU] Support parallel sampling & Swapping (#5855) · cbc53b6b
  Woosuk Kwon authored Jun 26, 2024
  
  cbc53b6b
25 Jun, 2024 2 commits
- [Hardware][TPU] Raise errors for unsupported sampling params (#5850) · f178e56c
  Woosuk Kwon authored Jun 25, 2024
  
  f178e56c
- [Hardware][TPU] Refactor TPU backend (#5831) · bc34937d
  Woosuk Kwon authored Jun 25, 2024
  
  bc34937d
12 Jun, 2024 1 commit
- [Hardware] Initial TPU integration (#5292) · 1a8bfd92
  Woosuk Kwon authored Jun 12, 2024
  
  1a8bfd92