Commits · 6aae216b4e135e6f6889e0bc80efc8a666e264e8 · OpenDAS / vllm_cscc

25 Apr, 2025 2 commits

[Bugfix] remove fallback in guided_json (int range, patterns) (#16725) · 6aae216b

Sangyeon Cho authored Apr 25, 2025


Signed-off-by: csy1204 <josang1204@gmail.com>
Co-authored-by: 조상연[플레이스 AI] <sang-yeon.cho@navercorp.com>

6aae216b

[FEAT] [ROCm]: AITER Fused MOE V1 Support (#16752) · eef36472

vllmellm authored Apr 25, 2025


Signed-off-by: vllmellm <vllm.ellm@embeddedllm.com>
Co-authored-by: tjtanaa <tunjian.tan@embeddedllm.com>

eef36472

24 Apr, 2025 10 commits
- Add chat template for Llama 4 models (#16428) · 05e1fbfc
  Maximilien de Bayser authored Apr 24, 2025
```
Signed-off-by: Max de Bayser <mbayser@br.ibm.com>
```
  05e1fbfc
- Improve configs - `LoRAConfig` + `PromptAdapterConfig` (#16980) · 0fa939e2
  Harry Mellor authored Apr 24, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  0fa939e2
- [V1][Spec Decoding] Add num_drafts and num_accepted_tokens_per_position metrics (#16665) · 340d7b1b
  Mark McLoughlin authored Apr 24, 2025
```
Signed-off-by: Mark McLoughlin <markmc@redhat.com>
```
  340d7b1b
- Add missing rocm_skinny_gemms kernel test to CI (#17060) · 82e43b2d
  Michael Goin authored Apr 24, 2025
```
Signed-off-by: mgoin <mgoin64@gmail.com>
```
  82e43b2d
- [Frontend] Using matryoshka_dimensions control the allowed output dimensions. (#16970) · 67309a1c
  wang.yuqi authored Apr 24, 2025
  
  67309a1c
- [V1][PP] Optimization: continue scheduling prefill chunks (#17080) · c0dfd975
  Rui Qiao authored Apr 24, 2025
```
Signed-off-by: Rui Qiao <ruisearch42@gmail.com>
```
  c0dfd975
- Fix OOT registration test (#17099) · a9138e85
  Harry Mellor authored Apr 24, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  a9138e85
- Simplify `TokenizerGroup` (#16790) · 0a05ed57
  Harry Mellor authored Apr 24, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  0a05ed57
- Disable enforce_eager for V1 TPU sampler and structured output tests (#17016) · 14288d13
  Michael Goin authored Apr 24, 2025
```
Signed-off-by: mgoin <mgoin64@gmail.com>
```
  14288d13
- [Chore] Remove Sampler from Model Code (#17084) · b411418f
  Woosuk Kwon authored Apr 24, 2025
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  b411418f
23 Apr, 2025 6 commits

[Frontend] Support guidance:no-additional-properties for compatibility with xgrammar (#15949) · 3cde34a4
Travis Johnson authored Apr 23, 2025
```
Signed-off-by: Travis Johnson <tsjohnso@us.ibm.com>
```
3cde34a4
Categorize `tests/kernels/` based on kernel type (#16799) · 6317a517
Michael Goin authored Apr 23, 2025
```
Signed-off-by: mgoin <mgoin64@gmail.com>
```
6317a517
[V1][DP] More robust DP/EP dummy request coordination (#16277) · 1e013fa3
Nick Hill authored Apr 22, 2025
```
Signed-off-by: Nick Hill <nhill@redhat.com>
```
1e013fa3

[Kernel][ROCM] Upstream prefix prefill speed up for vLLM V1 (#13305) · bc7c4d20

Aleksandr Malyshev authored Apr 22, 2025


Signed-off-by: Sage Moore <sage@neuralmagic.com>
Signed-off-by: root <root@banff-cyxtera-s73-5.ctr.dcgpu>
Signed-off-by: Aleksandr Malyshev <maleksan@amd.com>
Signed-off-by: root <root@banff-cyxtera-s65-4.amd.com>
Signed-off-by: maleksan85 <maleksan@amd.com>
Signed-off-by: <>
Co-authored-by: Sage Moore <sage@neuralmagic.com>
Co-authored-by: root <root@banff-cyxtera-s73-5.ctr.dcgpu>
Co-authored-by: Aleksandr Malyshev <maleksan@amd.com>
Co-authored-by: qli88 <qiang.li2@amd.com>
Co-authored-by: root <root@banff-cyxtera-s65-4.amd.com>

bc7c4d20

[Bugfix] validate urls object for multimodal content parts (#16990) · 36fe7876
Guillaume Calmettes authored Apr 23, 2025
```
Signed-off-by: Guillaume Calmettes <gcalmettes@scaleway.com>
```
36fe7876
[Core][V1][TPU] Enable structured decoding on TPU V1 (#16499) · 83d93371
Chenyaaang authored Apr 22, 2025
```
Signed-off-by: Chenyaaang <chenyangli@google.com>
```
83d93371

22 Apr, 2025 10 commits

[FEAT][ROCm]: Support AITER MLA (#15893) · 30bc3e0f

vllmellm authored Apr 23, 2025


Signed-off-by: vllmellm <vllm.ellm@embeddedllm.com>
Co-authored-by: qli88 <qiang.li2@amd.com>

30bc3e0f

[Kernel] Support Microsoft Runtime Kernel Lib for our Low Precision Computation - BitBLAS (#6036) · 8d32dc60
Lei Wang authored Apr 22, 2025
```
Signed-off-by: xinyuxiao <xinyuxiao2024@gmail.com>
Co-authored-by: xinyuxiao <xinyuxiao2024@gmail.com>
```
8d32dc60
[V1] Remove pre-allocation for KV cache (#16941) · c4ab9f3e
Woosuk Kwon authored Apr 22, 2025
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
c4ab9f3e

[Bugfix] Fix the issue where llm.generate cannot be called repeatedly after... · acba33a0

Chauncey authored Apr 22, 2025


[Bugfix] Fix the issue where llm.generate cannot be called repeatedly after setting GuidedDecodingParams (#16767)
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
Signed-off-by: Russell Bryant <rbryant@redhat.com>
Co-authored-by: Russell Bryant <rbryant@redhat.com>

acba33a0

[Performance][ROCm] Add skinny gemms for unquantized linear on ROCm (#15830) · 188b7f9b
Charlie Fu authored Apr 21, 2025
```
Signed-off-by: charlifu <charlifu@amd.com>
Co-authored-by: Tyler Michael Smith <tysmith@redhat.com>
```
188b7f9b

[Kernel] Add expert_map support to Cutlass FP8 MOE (#16861) · 7b8a2ab7

Varun Sundar Rabindranath authored Apr 21, 2025


Signed-off-by: varun sundar rabindranath <vsundarr@redhat.com>
Co-authored-by: varun sundar rabindranath <vsundarr@redhat.com>

7b8a2ab7

[Bugfix]: fix issue with n>1 sampling on v1 requests overriding each other (#16863) · 0e425449
Jeffrey Li authored Apr 21, 2025
```
Signed-off-by: Jeffrey Li <jeffrey.dot.li@gmail.com>
```
0e425449

[TPU][V1] Enable Top-P (#16843) · fa3bba2a

Nicolò Lucchesi authored Apr 22, 2025


Signed-off-by: NickLucche <nlucches@redhat.com>
Co-authored-by: mgoin <mgoin64@gmail.com>

fa3bba2a

[V1] V1 FlashInfer Attention (#16684) · 986537f1

Michael Goin authored Apr 21, 2025


Signed-off-by: mgoin <mgoin64@gmail.com>
Co-authored-by: Aurick Qiao <qiao@aurick.net>

986537f1

[TPU][V1] Capture multimodal encoder during model compilation (#15051) · 21020752

Nicolò Lucchesi authored Apr 22, 2025


Signed-off-by: Michael Goin <mgoin64@gmail.com>
Signed-off-by: NickLucche <nlucches@redhat.com>
Co-authored-by: Michael Goin <mgoin64@gmail.com>
Co-authored-by: Siyuan Liu <lsiyuan@google.com>

21020752

21 Apr, 2025 3 commits
- [TPU][V1] Implicitly adjust page size when there's SMEM OOM (#16871) · 471fe656
  Chengji Yao authored Apr 21, 2025
```
Signed-off-by: Chengji Yao <chengjiyao@google.com>
```
  471fe656
- [V1][Spec Decode] Handle draft tokens beyond max_model_len (#16087) · 3a0fba5c
  Woosuk Kwon authored Apr 21, 2025
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  3a0fba5c
- [Bugfix] Fix v1/spec_decode/test_ngram.py (#16895) · bb3605db
  qizixi authored Apr 20, 2025
```
Signed-off-by: qizixi <qizixi@meta.com>
```
  bb3605db
19 Apr, 2025 6 commits

Serialize tensors using int8 views (#16866) · 87aaadef

Staszek Paśko authored Apr 19, 2025


Signed-off-by: Staszek Pasko <staszek@gmail.com>
Co-authored-by: Nick Hill <nhill@redhat.com>

87aaadef

[Model] Refactor Phi-4-multimodal to use merged processor and support V1 (#15477) · 83f3c3bd

Isotr0py authored Apr 19, 2025


Signed-off-by: Isotr0py <2037008807@qq.com>
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Co-authored-by: DarkLight1337 <tlleungac@connect.ust.hk>

83f3c3bd

[V1][Misc] stop update prefix cache stats when logs_stats is disabled (#16460) · d9737ca1
vie-serendipity authored Apr 19, 2025
```
Signed-off-by: vie-serendipity <2733147505@qq.com>
```
d9737ca1
[Misc] Benchmarks for audio models (#16505) · 9d4ca19d
Nicolò Lucchesi authored Apr 19, 2025
```
Signed-off-by: NickLucche <nlucches@redhat.com>
```
9d4ca19d

[Frontend] Add sampling params to `v1/audio/transcriptions` endpoint (#16591) · 2ef0dc53

Nicolò Lucchesi authored Apr 19, 2025


Signed-off-by: Jannis Schönleber <joennlae@gmail.com>
Signed-off-by: NickLucche <nlucches@redhat.com>
Co-authored-by: Jannis Schönleber <joennlae@gmail.com>

2ef0dc53

[Model][VLM] Add Qwen2.5-Omni model support (thinker only) (#15130) · 2c1bd848

Yang Fan authored Apr 19, 2025


Signed-off-by: fyabc <suyang.fy@alibaba-inc.com>
Signed-off-by: Roger Wang <ywang@roblox.com>
Co-authored-by: Roger Wang <136131678+ywang96@users.noreply.github.com>
Co-authored-by: Roger Wang <ywang@roblox.com>
Co-authored-by: Xiong Wang <wangxiongts@163.com>

2c1bd848

18 Apr, 2025 3 commits
- [New Model]: Snowflake Arctic Embed (Family) (#16649) · 3d3ab368
  wang.yuqi authored Apr 18, 2025
  
  3d3ab368
- Fix `nullable_kvs` fallback (#16837) · 686623c5
  Harry Mellor authored Apr 18, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  686623c5
- Improve-mm-and-pooler-and-decoding-configs (#16789) · e78587a6
  Harry Mellor authored Apr 18, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  e78587a6