Commits · e97f802b2d74861af77997691a7d1c36498f6dca · OpenDAS / vllm_cscc

23 Jan, 2025 1 commit

[FP8][Kernel] Dynamic kv cache scaling factors computation (#11906) · e97f802b

Gregory Shtrasberg authored Jan 23, 2025


Signed-off-by: Gregory Shtrasberg <Gregory.Shtrasberg@amd.com>
Co-authored-by: Micah Williamson <micah.williamson@amd.com>

e97f802b

20 Jan, 2025 1 commit
- [Core] Interface for accessing model from `VllmRunner` (#10353) · 59a0192f
  Cyrus Leung authored Jan 20, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  59a0192f
13 Jan, 2025 1 commit
- [Kernel] unified_attention for Attention.forward (#11967) · 0f8cafe2
  Chen Zhang authored Jan 13, 2025
```
Signed-off-by: Chen Zhang <zhangch99@outlook.com>
```
  0f8cafe2
03 Jan, 2025 1 commit
- Update requirements-tpu.txt to support python 3.9 and 3.11 (#11695) · bf0d97d7
  Michael Goin authored Jan 03, 2025
```
Signed-off-by: mgoin <michael@neuralmagic.com>
```
  bf0d97d7
20 Nov, 2024 1 commit
- [TPU] Implement prefix caching for TPUs (#10307) · 2f77b6cf
  Woosuk Kwon authored Nov 20, 2024
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  2f77b6cf
17 Nov, 2024 1 commit
- [2/N][torch.compile] make compilation cfg part of vllm cfg (#10383) · 4fd93750
  youkaichao authored Nov 16, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  4fd93750
02 Nov, 2024 2 commits
- [3/N] model runner pass the whole config to model (#9958) · cea808f3
  youkaichao authored Nov 02, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  cea808f3
- [2/N] executor pass the complete config to worker/modelrunner (#9938) · e8937954
  youkaichao authored Nov 02, 2024
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
Co-authored-by: Nick Hill <nhill@redhat.com>
```
  e8937954
01 Nov, 2024 1 commit
- [Core][VLM] Add precise multi-modal placeholder tracking (#8346) · 6c0b7f54
  Peter Salas authored Nov 01, 2024
```
Signed-off-by: Peter Salas <peter@fixie.ai>
```
  6c0b7f54
20 Oct, 2024 1 commit
- [Kernel] Support sliding window in flash attention backend (#9403) · 4fa3e333
  Chen Zhang authored Oct 20, 2024
  
  4fa3e333
14 Oct, 2024 1 commit
- [TPU] Fix TPU SMEM OOM by Pallas paged attention kernel (#9350) · 473e7b36
  Woosuk Kwon authored Oct 14, 2024
  
  473e7b36
11 Oct, 2024 3 commits
- [Bugfix] Sets `is_first_step_output` for TPUModelRunner (#9202) · c6cf9295
  Allen Wang authored Oct 11, 2024
  
  c6cf9295
- [Model] Support Mamba (#6484) · 7342a7d7
  Tyler Michael Smith authored Oct 11, 2024
  
  7342a7d7
- [misc] hide best_of from engine (#9261) · cbc2ef55
  youkaichao authored Oct 10, 2024
```
Co-authored-by: Brendan Wong <bjwpokemon@gmail.com>
```
  cbc2ef55
07 Oct, 2024 1 commit
- [core] remove beam search from the core (#9105) · 18b296fd
  youkaichao authored Oct 06, 2024
  
  18b296fd
27 Sep, 2024 1 commit
- [torch.compile] use empty tensor instead of None for profiling (#8875) · a9b15c60
  youkaichao authored Sep 27, 2024
  
  a9b15c60
14 Sep, 2024 1 commit
- [TPU] Implement multi-step scheduling (#8489) · 50e9ec41
  Woosuk Kwon authored Sep 14, 2024
  
  50e9ec41
07 Sep, 2024 1 commit
- [tpu][misc] fix typo (#8260) · ce2702a9
  youkaichao authored Sep 06, 2024
  
  ce2702a9
03 Sep, 2024 1 commit
- [TPU][Bugfix] Fix next_token_ids shape (#8128) · 0af3abe3
  Woosuk Kwon authored Sep 03, 2024
  
  0af3abe3
30 Aug, 2024 2 commits
- [TPU] Async output processing for TPU (#8011) · 80c7b089
  Woosuk Kwon authored Aug 29, 2024
  
  80c7b089
- [Core] Logprobs support in Multi-step (#7652) · 428dd144
  afeldman-nm authored Aug 29, 2024
  
  428dd144
28 Aug, 2024 1 commit
- [torch.compile] avoid Dynamo guard evaluation overhead (#7898) · ce6bf3a2
  youkaichao authored Aug 28, 2024
```
Co-authored-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  ce6bf3a2
19 Aug, 2024 1 commit
- [TPU] Remove redundant input tensor cloning (#7660) · 43735bf5
  Woosuk Kwon authored Aug 19, 2024
  
  43735bf5
18 Aug, 2024 1 commit
- [TPU] Use mark_dynamic only for dummy run (#7634) · 0c2fa50b
  Woosuk Kwon authored Aug 18, 2024
  
  0c2fa50b
17 Aug, 2024 1 commit
- [VLM] Refactor `MultiModalConfig` initialization and profiling (#7530) · bbf55c48
  Roger Wang authored Aug 17, 2024
  
  bbf55c48
11 Aug, 2024 1 commit
- [TPU] Use mark_dynamic to reduce compilation time (#7340) · 90bab18f
  Woosuk Kwon authored Aug 10, 2024
  
  90bab18f
30 Jul, 2024 1 commit
- [TPU] Fix greedy decoding (#6933) · 6e063ea3
  Woosuk Kwon authored Jul 30, 2024
  
  6e063ea3
27 Jul, 2024 2 commits
- [TPU] Reduce compilation time & Upgrade PyTorch XLA version (#6856) · fad5576c
  Woosuk Kwon authored Jul 27, 2024
  
  fad5576c
- [Hardware][TPU] Implement tensor parallelism with Ray (#5871) · 52f07e3d
  Woosuk Kwon authored Jul 26, 2024
  
  52f07e3d
18 Jul, 2024 1 commit
- [TPU] Refactor TPU worker & model runner (#6506) · 4634c872
  Woosuk Kwon authored Jul 18, 2024
  
  4634c872
17 Jul, 2024 1 commit
- [TPU] Remove multi-modal args in TPU backend (#6504) · e09ce759
  Woosuk Kwon authored Jul 17, 2024
  
  e09ce759
16 Jul, 2024 1 commit
- [Hardware][TPU] Support MoE with Pallas GMM kernel (#6457) · c467dff2
  Woosuk Kwon authored Jul 16, 2024
  
  c467dff2
09 Jul, 2024 1 commit
- [Bugfix][TPU] Add missing None to model input (#6245) · 5d5b4c5f
  Woosuk Kwon authored Jul 09, 2024
  
  5d5b4c5f
03 Jul, 2024 2 commits

[vlm] Remove vision language config. (#6089) · d9e98f42

xwjiang2010 authored Jul 03, 2024


Signed-off-by: Xiaowei Jiang <xwjiang2010@gmail.com>
Co-authored-by: Roger Wang <ywang@roblox.com>

d9e98f42

[Core] Dynamic image size support for VLMs (#5276) · 9831aec4

Cyrus Leung authored Jul 03, 2024


Signed-off-by: Xiaowei Jiang <xwjiang2010@gmail.com>
Co-authored-by: Xiaowei Jiang <xwjiang2010@gmail.com>
Co-authored-by: ywang96 <ywang@roblox.com>
Co-authored-by: xwjiang2010 <87673679+xwjiang2010@users.noreply.github.com>
Co-authored-by: Roger Wang <136131678+ywang96@users.noreply.github.com>

9831aec4

29 Jun, 2024 1 commit
- [Bugfix][TPU] Fix pad slot id (#5977) · 7f83f40d
  Woosuk Kwon authored Jun 28, 2024
  
  7f83f40d
28 Jun, 2024 1 commit
- [Spec Decode] Introduce DraftModelRunner (#5799) · b2c62023
  Cody Yu authored Jun 28, 2024
  
  b2c62023
26 Jun, 2024 1 commit
- [Hardware][TPU] Support parallel sampling & Swapping (#5855) · cbc53b6b
  Woosuk Kwon authored Jun 26, 2024
  
  cbc53b6b
25 Jun, 2024 2 commits
- [Hardware][TPU] Raise errors for unsupported sampling params (#5850) · f178e56c
  Woosuk Kwon authored Jun 25, 2024
  
  f178e56c
- [Hardware][TPU] Refactor TPU backend (#5831) · bc34937d
  Woosuk Kwon authored Jun 25, 2024
  
  bc34937d