Commits · 3cde34a4a4bceb511f2f9fe2dade1da116eb7e8a · OpenDAS / vllm_cscc

23 Apr, 2025 1 commit
- [Core][V1][TPU] Enable structured decoding on TPU V1 (#16499) · 83d93371
  Chenyaaang authored Apr 22, 2025
```
Signed-off-by: Chenyaaang <chenyangli@google.com>
```
  83d93371
22 Apr, 2025 3 commits

[Perf] Optimize `_update_states` for GPU model runner (#16910) · a114bf20
SnowCharm authored Apr 22, 2025
```
Signed-off-by: snowcharm <snowcharmqq@gmail.com>
```
a114bf20

[V1] V1 FlashInfer Attention (#16684) · 986537f1

Michael Goin authored Apr 21, 2025


Signed-off-by: mgoin <mgoin64@gmail.com>
Co-authored-by: Aurick Qiao <qiao@aurick.net>

986537f1

[TPU][V1] Capture multimodal encoder during model compilation (#15051) · 21020752

Nicolò Lucchesi authored Apr 22, 2025


Signed-off-by: Michael Goin <mgoin64@gmail.com>
Signed-off-by: NickLucche <nlucches@redhat.com>
Co-authored-by: Michael Goin <mgoin64@gmail.com>
Co-authored-by: Siyuan Liu <lsiyuan@google.com>

21020752

21 Apr, 2025 2 commits
- [V1][Spec Decode] Handle draft tokens beyond max_model_len (#16087) · 3a0fba5c
  Woosuk Kwon authored Apr 21, 2025
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  3a0fba5c
- Restore buffers when wake up from level 2 sleep (#16564) (#16889) · d41faaf9
  Han Zhang authored Apr 21, 2025
```
Signed-off-by: Han <zh950713@gmail.com>
```
  d41faaf9
19 Apr, 2025 1 commit

[Model][VLM] Add Qwen2.5-Omni model support (thinker only) (#15130) · 2c1bd848

Yang Fan authored Apr 19, 2025


Signed-off-by: fyabc <suyang.fy@alibaba-inc.com>
Signed-off-by: Roger Wang <ywang@roblox.com>
Co-authored-by: Roger Wang <136131678+ywang96@users.noreply.github.com>
Co-authored-by: Roger Wang <ywang@roblox.com>
Co-authored-by: Xiong Wang <wangxiongts@163.com>

2c1bd848

17 Apr, 2025 5 commits

[P/D][V1] KV Connector API V1 (#15960) · 3408e471

Yihua Cheng authored Apr 17, 2025


Signed-off-by: ApostaC <yihua98@uchicago.edu>
Signed-off-by: rshaw@neuralmagic.com <robertgshaw2@gmail.com>
Signed-off-by: remi <remi@mistral.ai>
Co-authored-by: rshaw@neuralmagic.com <robertgshaw2@gmail.com>
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com>
Co-authored-by: Rémi Delacourt <54138269+Flechman@users.noreply.github.com>
Co-authored-by: Tyler Michael Smith <tysmith@redhat.com>

3408e471

[MLA] Simplification to batch P/D reordering (#16673) · 0377b831
Nick Hill authored Apr 17, 2025
```
Signed-off-by: Nick Hill <nhill@redhat.com>
```
0377b831

[V1][TPU] Enable Top K (#15489) · eb5819b2

Nicolò Lucchesi authored Apr 17, 2025


Signed-off-by: NickLucche <nlucches@redhat.com>
Signed-off-by: Hyesoo Yang <hyeygit@gmail.com>
Co-authored-by: Hyesoo Yang <hyeygit@gmail.com>

eb5819b2

[TPU][V1] Fix padding recompilation when `max-num-batched-tokens` is not even (#16726) · 5989f468
Nicolò Lucchesi authored Apr 17, 2025
```
Signed-off-by: NickLucche <nlucches@redhat.com>
```
5989f468
[Misc] Remove redundant comment (#16703) · 8a7368e0
Jade Zheng authored Apr 17, 2025
```
Signed-off-by: Jade Zheng <zheng.shoujian@outlook.com>
```
8a7368e0

14 Apr, 2025 3 commits
- [DOC][TPU] Add core idea about avoiding recompilation after warmup (#16614) · 1eb3c2ed
  Chengji Yao authored Apr 14, 2025
```
Signed-off-by: Chengji Yao <chengjiyao@google.com>
```
  1eb3c2ed
- [Model][VLM] Add Kimi-VL model support (#16387) · b1308b84
  courage17340 authored Apr 15, 2025
```
Signed-off-by: courage17340 <courage17340@163.com>
```
  b1308b84
- [TPU][V1] Fix exponential padding when `max-num-batched-tokens` is not a power of 2 (#16596) · b3f2fddd
  Nicolò Lucchesi authored Apr 14, 2025
```
Signed-off-by: NickLucche <nlucches@redhat.com>
```
  b3f2fddd
12 Apr, 2025 1 commit
- [Perf] Optimize Preparing Inputs for GPU Model Runner (#16484) · 93e5f3c5
  SnowCharm authored Apr 12, 2025
```
Signed-off-by: snowcharm <snowcharmqq@gmail.com>
Co-authored-by: Nick Hill <nhill@redhat.com>
```
  93e5f3c5
10 Apr, 2025 4 commits
- [TPU][V1] Disable per-request seed/Generator (#16172) · 3cc9af88
  Nicolò Lucchesi authored Apr 10, 2025
```
Signed-off-by: NickLucche <nlucches@redhat.com>
```
  3cc9af88
- [V1][Spec Decode] Eagle Model loading (#16035) · e8224f3d
  Lily Liu authored Apr 10, 2025
```
Signed-off-by: LiuXiaoxuanPKU <lilyliupku@gmail.com>
```
  e8224f3d
- [TPU][V1] Use `language_model` interface for getting text backbone in MM (#16410) · c1b57855
  Nicolò Lucchesi authored Apr 10, 2025
```
Signed-off-by: NickLucche <nlucches@redhat.com>
```
  c1b57855
- [TPU][V1] Refine tpu_model_runner to mitigate future recompilation issues (#16275) · a4547485
  Chengji Yao authored Apr 09, 2025
```
Signed-off-by: Chengji Yao <chengjiyao@google.com>
```
  a4547485
09 Apr, 2025 1 commit
- [BugFix] fix some typos found by typos. (#16314) · 04149cce
  yihong authored Apr 09, 2025
```
Signed-off-by: yihong0618 <zouzou0208@gmail.com>
```
  04149cce
08 Apr, 2025 2 commits

[torch.compile][TPU] Make @support_torch_compile work for XLA backend (#15782) · 87918e40

Siyuan Liu authored Apr 07, 2025


Signed-off-by: Siyuan Liu <lsiyuan@google.com>
Signed-off-by: mgoin <mgoin64@gmail.com>
Co-authored-by: mgoin <mgoin64@gmail.com>

87918e40

[V1] Scatter and gather placeholders in the model runner (#16076) · f2ebb6f5

Roger Wang authored Apr 07, 2025


Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Signed-off-by: mgoin <mgoin64@gmail.com>
Signed-off-by: Roger Wang <ywang@roblox.com>
Co-authored-by: DarkLight1337 <tlleungac@connect.ust.hk>
Co-authored-by: mgoin <mgoin64@gmail.com>
Co-authored-by: Jennifer Zhao <ai.jenniferzhao@gmail.com>

f2ebb6f5

07 Apr, 2025 1 commit

Upstream Llama4 Support to Main (#16113) · 55dcce91

Lu Fang authored Apr 07, 2025


Signed-off-by: Aston Zhang <22279212+astonzhang@users.noreply.github.com>
Signed-off-by: Chris Thi <chris.c.thi@gmail.com>
Signed-off-by: drisspg <drisspguessous@gmail.com>
Signed-off-by: Jon Swenson <jmswen@gmail.com>
Signed-off-by: Keyun Tong <tongkeyun@gmail.com>
Signed-off-by: Lu Fang <fanglu@meta.com>
Signed-off-by: Xiaodong Wang <xdwang@meta.com>
Signed-off-by: Yang Chen <yangche@fb.com>
Signed-off-by: Ye (Charlotte) Qi <yeq@meta.com>
Signed-off-by: Yong Hoon Shin <yhshin@meta.com>
Signed-off-by: Zijing Liu <liuzijing2014@gmail.com>
Signed-off-by: Lu Fang <lufang@fb.com>
Signed-off-by: Lu Fang <fanglu@fb.com>
Signed-off-by: Lucia Fang <fanglu@fb.com>
Signed-off-by: Roger Wang <ywang@roblox.com>
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Co-authored-by: Lu Fang <fanglu@fb.com>
Co-authored-by: Roger Wang <ywang@roblox.com>
Co-authored-by: DarkLight1337 <tlleungac@connect.ust.hk>