Commits · e37073efd7dea225ff8a0614f04575547d7588ff · OpenDAS / vllm_cscc

18 Apr, 2025 1 commit

Add property-based testing for vLLM endpoints using an API defined by an... · e37073ef

Tarun Kumar authored Apr 18, 2025


Add property-based testing for vLLM endpoints using an API defined by an OpenAPI 3.1 schema (#16721)
Signed-off-by: Tarun Kumar <takumar@redhat.com>
Signed-off-by: Nick Hill <nhill@redhat.com>
Co-authored-by: Nick Hill <nhill@redhat.com>

e37073ef

17 Apr, 2025 9 commits

[P/D][V1] KV Connector API V1 (#15960) · 3408e471

Yihua Cheng authored Apr 17, 2025


Signed-off-by: ApostaC <yihua98@uchicago.edu>
Signed-off-by: rshaw@neuralmagic.com <robertgshaw2@gmail.com>
Signed-off-by: remi <remi@mistral.ai>
Co-authored-by: rshaw@neuralmagic.com <robertgshaw2@gmail.com>
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com>
Co-authored-by: Rémi Delacourt <54138269+Flechman@users.noreply.github.com>
Co-authored-by: Tyler Michael Smith <tysmith@redhat.com>

3408e471

[V1][TPU] Enable Top K (#15489) · eb5819b2

Nicolò Lucchesi authored Apr 17, 2025


Signed-off-by: NickLucche <nlucches@redhat.com>
Signed-off-by: Hyesoo Yang <hyeygit@gmail.com>
Co-authored-by: Hyesoo Yang <hyeygit@gmail.com>

eb5819b2

[TPU][V1] Fix padding recompilation when `max-num-batched-tokens` is not even (#16726) · 5989f468
Nicolò Lucchesi authored Apr 17, 2025
```
Signed-off-by: NickLucche <nlucches@redhat.com>
```
5989f468
[V1][Perf] Faster incremental detokenization (#15137) · 05fcd1b4
Nick Hill authored Apr 17, 2025
```
Signed-off-by: Nick Hill <nhill@redhat.com>
```
05fcd1b4
Improve configs - `TokenizerPoolConfig` + `DeviceConfig` (#16603) · d27ea940
Harry Mellor authored Apr 17, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
d27ea940
[Bugfix] Fix GLM4 model (#16618) · 5b1aca2a
intervitens authored Apr 17, 2025
```
Signed-off-by: intervitens <intervitens@tutanota.com>
```
5b1aca2a
[Bugfix] Update Florence-2 tokenizer to make grounding tasks work (#16734) · cb072ce9
Isotr0py authored Apr 17, 2025
```
Signed-off-by: Isotr0py <2037008807@qq.com>
```
cb072ce9

[V1][Frontend] Improve Shutdown And Logs (#11737) · 2b05b8ce

Robert Shaw authored Apr 16, 2025


Signed-off-by: rshaw@neuralmagic.com <rshaw@neuralmagic.com>
Signed-off-by: Andrew Feldman <afeldman@neuralmagic.com>
Signed-off-by: Nick Hill <nhill@redhat.com>
Co-authored-by: rshaw@neuralmagic.com <rshaw@neuralmagic.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
Co-authored-by: Russell Bryant <rbryant@redhat.com>
Co-authored-by: Andrew Feldman <afeldman@neuralmagic.com>
Co-authored-by: afeldman-nm <156691304+afeldman-nm@users.noreply.github.com>
Co-authored-by: Nick Hill <nhill@redhat.com>

2b05b8ce

[V1][Performance] Implement custom serializaton for MultiModalKwargs [Rebased] (#16432) · 3092375e

Staszek Paśko authored Apr 17, 2025


Signed-off-by: Staszek Pasko <staszek@gmail.com>
Signed-off-by: Nick Hill <nhill@redhat.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
Co-authored-by: Nick Hill <nhill@redhat.com>

3092375e

16 Apr, 2025 3 commits

[Model] support modernbert (#16648) · ee378f3d

xsank authored Apr 16, 2025


Signed-off-by: 唯勤 <xsank.mz@alibaba-inc.com>
Co-authored-by: 唯勤 <xsank.mz@alibaba-inc.com>

ee378f3d

[V1][Structured Output] Move xgrammar related utils to `backend_xgrammar.py` (#16578) · 976711d9
Shanshan Shen authored Apr 16, 2025
```
Signed-off-by: shen-shanshan <467638484@qq.com>
```
976711d9

[Model] Add PLaMo2 (#14323) · 3badb021

Shinichi Hemmi authored Apr 16, 2025


Signed-off-by: Shinichi Hemmi <50256998+Alnusjaponica@users.noreply.github.com>
Signed-off-by: shemmi <shemmi@preferred.jp>
Co-authored-by: Kento Nozawa <nzw0301@preferred.jp>
Co-authored-by: Hiroaki Mikami <mhiroaki@preferred.jp>
Co-authored-by: Calvin Metzger <metzger@preferred.jp>

3badb021

15 Apr, 2025 8 commits
- [Misc] Enable vLLM to Dynamically Load LoRA from a Remote Server (#10546) · fdcb850f
  Angky William authored Apr 15, 2025
```
Signed-off-by: Angky William <angkywilliam@Angkys-MacBook-Pro.local>
Co-authored-by: Angky William <angkywilliam@Angkys-MacBook-Pro.local>
```
  fdcb850f
- [Misc] Update `compressed-tensors` WNA16 to support zero-points (#14211) · 54a66e5f
  Dipika Sikka authored Apr 15, 2025
  
  54a66e5f
- [CI/Build] Fix LoRA OOM (#16624) · 1575c170
  Jee Jee Li authored Apr 15, 2025
```
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
```
  1575c170
- Add `vllm bench [latency, throughput]` CLI commands (#16508) · b4fe16c7
  Michael Goin authored Apr 15, 2025
```
Signed-off-by: mgoin <mgoin64@gmail.com>
```
  b4fe16c7
- [Bugfix] Fix broken GritLM model and tests (missing pooling_metadata) (#16631) · bc5dd4f6
  Pooya Davoodi authored Apr 14, 2025
```
Signed-off-by: Pooya Davoodi <pooya.davoodi@parasail.io>
```
  bc5dd4f6
- [Bugfix] Fix tests/kernels/test_mamba_ssm_ssd.py (#16623) · dbb036cf
  Tyler Michael Smith authored Apr 15, 2025
```
Signed-off-by: Tyler Michael Smith <tyler@neuralmagic.com>
```
  dbb036cf
- [Kernel] moe wna16 marlin kernel (#14447) · d06ba4ed
  Jinzhen Lin authored Apr 15, 2025
```
Signed-off-by: Jinzhen Lin <linjinzhen@hotmail.com>
Co-authored-by: Michael Goin <michael@neuralmagic.com>
Co-authored-by: mgoin <mgoin64@gmail.com>
```
  d06ba4ed
- [Core][Bugfix] Fix Offline MM Beam Search (#16390) · 6b40996a
  Alex Brooks authored Apr 14, 2025
```
Signed-off-by: Alex-Brooks <Alex.Brooks@ibm.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
```
  6b40996a
14 Apr, 2025 4 commits
- [Model][VLM] Add Kimi-VL model support (#16387) · b1308b84
  courage17340 authored Apr 15, 2025
```
Signed-off-by: courage17340 <courage17340@163.com>
```
  b1308b84
- [TPU][V1] Fix exponential padding when `max-num-batched-tokens` is not a power of 2 (#16596) · b3f2fddd
  Nicolò Lucchesi authored Apr 14, 2025
```
Signed-off-by: NickLucche <nlucches@redhat.com>
```
  b3f2fddd
- [Bugfix] Multi-modal caches not acting like LRU caches (#16593) · aa29841e
  Cyrus Leung authored Apr 15, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  aa29841e
- [Core][V0] Enable regex support with xgrammar (#13228) · dc1b4a6f
  Russell Bryant authored Apr 13, 2025
```
Signed-off-by: Russell Bryant <rbryant@redhat.com>
```
  dc1b4a6f
13 Apr, 2025 1 commit
- [V1][Spec Decode] KV cache slots for eagle heads (#16370) · f49e5aff
  Lily Liu authored Apr 12, 2025
```
Signed-off-by: LiuXiaoxuanPKU <lilyliupku@gmail.com>
```
  f49e5aff
12 Apr, 2025 6 commits
- [Bugfix] Validate logit biases to prevent out of vocab ids crashing engine (#16529) · 6c11ecf8
  Ryan McConville authored Apr 12, 2025
```
Signed-off-by: Ryan McConville <ryan@ryanmcconville.com>
```
  6c11ecf8
- [V1] Enable multi-input by default (#15799) · d9fc8cd9
  Cyrus Leung authored Apr 12, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  d9fc8cd9
- [Misc] Update chat utils tests (#16520) · c5bc0e7f
  Cyrus Leung authored Apr 12, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  c5bc0e7f
- [Frontend] support matryoshka representation / support embedding API dimensions (#16331) · fbf722c6
  wang.yuqi authored Apr 12, 2025
  
  fbf722c6
- [Feature][V1] Add xgrammar to support minLength, maxLength with test (#16516) · e92d7085
  leon-seidel authored Apr 12, 2025
```
Signed-off-by: Leon Seidel <leon.seidel@fau.de>
```
  e92d7085
- [BugFix] Handle non-contiguous tensors properly when serializing (#16492) · 41cc883c
  Nick Hill authored Apr 11, 2025
```
Signed-off-by: Nick Hill <nhill@redhat.com>
```
  41cc883c
11 Apr, 2025 7 commits
- [Frontend] Added chat templates for LLaMa4 pythonic tool calling (#16463) · 16eda8c4
  Ye (Charlotte) Qi authored Apr 11, 2025
```
Signed-off-by: Ye (Charlotte) Qi <yeq@meta.com>
Co-authored-by: Kai Wu <kaiwu@meta.com>
```
  16eda8c4
- [Bugfix] handle alignment of encoder_seq_lens in mllama.py (#14784) · 71b9cde0
  Travis Johnson authored Apr 11, 2025
```
Signed-off-by: Travis Johnson <tsjohnso@us.ibm.com>
```
  71b9cde0
- [Kernel] Support W8A8 channel-wise weights and per-token activations in triton... · f41647ee
  Michael Goin authored Apr 11, 2025
```
[Kernel] Support W8A8 channel-wise weights and per-token activations in triton fused_moe_kernel (#16366)
Signed-off-by: mgoin <mgoin64@gmail.com>
```
  f41647ee
- [Bugfix] Fix bugs of running Quark quantized models (#16236) · 9e90c9f7
  chaow-amd authored Apr 11, 2025
```
Signed-off-by: chaow <chaow@amd.com>
```
  9e90c9f7
- [Kernel] support merge_attn_states CUDA kernel, 3x speedup (#16173) · e9528f6d
  DefTruth authored Apr 11, 2025
```
Signed-off-by: DefTruth <qiustudent_r@163.com>
```
  e9528f6d
- Enforce valid max_num_batched_tokens when disable_chunked_mm_input=True (#16447) · aa3b3d76
  Michael Goin authored Apr 11, 2025
```
Signed-off-by: mgoin <mgoin64@gmail.com>
```
  aa3b3d76
- [Bugfix][VLM] Fix failing Phi-4-MM multi-images tests and add vision-speech test (#16424) · 93195146
  Isotr0py authored Apr 11, 2025
```
Signed-off-by: Isotr0py <2037008807@qq.com>
```
  93195146
10 Apr, 2025 1 commit
- [TPU][V1] Disable per-request seed/Generator (#16172) · 3cc9af88
  Nicolò Lucchesi authored Apr 10, 2025
```
Signed-off-by: NickLucche <nlucches@redhat.com>
```
  3cc9af88