Commits · b877031d806e3d9ebc834e0191ae64de40c4ddc2 · OpenDAS / vllm_cscc

22 Mar, 2025 5 commits
- Remove openvino support in favor of external plugin (#15339) · b877031d
  Russell Bryant authored Mar 22, 2025
```
Signed-off-by: Russell Bryant <rbryant@redhat.com>
```
  b877031d
- [V1] Add `disable-any-whitespace` option support for xgrammar (#15316) · eb63ea1e
  Russell Bryant authored Mar 22, 2025
```
Signed-off-by: Russell Bryant <rbryant@redhat.com>
```
  eb63ea1e
- [Model] Support Tele-FLM Model (#15023) · 2f4bd358
  Naitong Yu authored Mar 22, 2025
```
Signed-off-by: Naitong Yu <ntyu@baai.ac.cn>
Signed-off-by: jiangxin <horizon94@outlook.com>
Co-authored-by: Jason Fang <jasonfang3900@gmail.com>
Co-authored-by: jiangxin <horizon94@outlook.com>
```
  2f4bd358
- [Bugfix] LoRA V0 - Fix case where `max_num_seqs` is between cudagraph capture sizes (#15308) · 8a8b30ea
  Varun Sundar Rabindranath authored Mar 22, 2025
```
Signed-off-by: Varun Sundar Rabindranath <varun@neuralmagic.com>
Co-authored-by: Varun Sundar Rabindranath <varun@neuralmagic.com>
```
  8a8b30ea
- [FEAT] [ROCm]: Add AITER RMS Norm (Layer Norm) Feature (#14959) · ec870fba
  TJian authored Mar 22, 2025
```
Signed-off-by: tjtanaa <tunjian.tan@embeddedllm.com>
```
  ec870fba
21 Mar, 2025 10 commits
- [TPU][V1] MHA Pallas backend (#15288) · cfbb8c93
  Nicolò Lucchesi authored Mar 21, 2025
```
Signed-off-by: NickLucche <nlucches@redhat.com>
```
  cfbb8c93
- Revert "[Feature] specify model in config.yaml (#14855)" (#15293) · baec0d4d
  Cyrus Leung authored Mar 21, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  baec0d4d
- [v1] Refactor KVCacheConfig (#14079) · 93a00d7d
  Chen Zhang authored Mar 21, 2025
```
Signed-off-by: Chen Zhang <zhangch99@outlook.com>
```
  93a00d7d
- [Bugfix] Fix broken kernel test due to missing rename for v1 Triton backend (#15282) · 8afcd0f6
  Isotr0py authored Mar 21, 2025
```
Signed-off-by: Isotr0py <2037008807@qq.com>
```
  8afcd0f6
- [Feature] specify model in config.yaml (#14855) · 0fa3970d
  Wei Zeng authored Mar 21, 2025
```
Signed-off-by: weizeng <weizeng@roblox.com>
```
  0fa3970d
- [V1] Avoid redundant input processing in n>1 case (#14985) · da6ea29f
  Nick Hill authored Mar 20, 2025
```
Signed-off-by: Nick Hill <nhill@redhat.com>
```
  da6ea29f
- [V1][TPU] Speed up top-k on TPU by using torch.topk (#15242) · 47195057
  Hyesoo Yang authored Mar 20, 2025
```
Signed-off-by: Hyesoo Yang <hyeygit@gmail.com>
```
  47195057
- [Bugfix] Fix incorrect qwen2.5-vl attention mask pre-computation (#15200) · 1e508343
  Isotr0py authored Mar 21, 2025
```
Signed-off-by: Isotr0py <2037008807@qq.com>
```
  1e508343
- [CI/Build] LoRA : make add_lora_test safer (#15181) · 0cfe7d38
  Varun Sundar Rabindranath authored Mar 20, 2025
```
Signed-off-by: Varun Sundar Rabindranath <varun@neuralmagic.com>
Co-authored-by: Varun Sundar Rabindranath <varun@neuralmagic.com>
```
  0cfe7d38
- [V1] Scheduler Refactoring [1/N] - Add Scheduler Interface (#15250) · 0c6f5023
  Woosuk Kwon authored Mar 20, 2025
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
Co-authored-by: Cody Yu <hao.yu.cody@gmail.com>
Co-authored-by: Nick Hill <nhill@redhat.com>
```
  0c6f5023
20 Mar, 2025 7 commits
- [Bugfix] fix V1 Engine crash while handling requests with duplicate request id (#15043) · d8e82bc0
  Jason authored Mar 21, 2025
```
Signed-off-by: Jiahui Sun <jhsun2020@gmail.com>
```
  d8e82bc0
- [ci] feat: make the test_torchrun_example run with tp=2, external_dp=2 (#15172) · 086b5682
  Chi Zhang authored Mar 21, 2025
```
Signed-off-by: Chi Zhang <zhangchi.usc1992@bytedance.com>
Signed-off-by: youkaichao <youkaichao@gmail.com>
Co-authored-by: youkaichao <youkaichao@gmail.com>
```
  086b5682
- Enable CUDA graph support for llama 3.2 vision (#14917) · a8652f4f
  Matt Ritter authored Mar 19, 2025
```
Signed-off-by: Matt Ritter <100659061+mritterfigma@users.noreply.github.com>
```
  a8652f4f
- [Attention] Flash Attention 3 - fp8 (#14570) · a597a575
  Mickaël Seznec authored Mar 20, 2025
```
Signed-off-by: Mickael Seznec <mickael@mistral.ai>
```
  a597a575
- [Core][V0] Add guidance backend for structured output (#14589) · 1f16b7fe
  Russell Bryant authored Mar 20, 2025
```
Signed-off-by: Russell Bryant <rbryant@redhat.com>
Co-authored-by: Loc Huynh <lohuynh@microsoft.com>
Co-authored-by: Michal Moskal <michal@moskal.me>
Co-authored-by: Aaron Pham <contact@aarnphm.xyz>
```
  1f16b7fe
- [V1][TPU] Support V1 Sampler for ragged attention (#14227) · d8c6d7d6
  Nicolò Lucchesi authored Mar 20, 2025
```
Signed-off-by: NickLucche <nlucches@redhat.com>
```
  d8c6d7d6
- Fix broken tests (#14713) · 70e500ca
  Jovan Sardinha authored Mar 19, 2025
```
Signed-off-by: JovanSardinha <jovan.sardinha@gmail.com>
Co-authored-by: Luka Govedič <ProExpertProg@users.noreply.github.com>
```
  70e500ca
19 Mar, 2025 4 commits
- [V1] Minor V1 async engine test refactor (#15075) · 61c7a1b8
  Murali Andoorveedu authored Mar 19, 2025
```
Signed-off-by: andoorve <murali.andoorveedu@mail.utoronto.ca>
Co-authored-by: andoorve <murali.andoorveedu@mail.utoronto.ca>
```
  61c7a1b8
- [Bugfix] Fix size calculation of processing cache (#15114) · 3d446433
  Cyrus Leung authored Mar 19, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  3d446433
- [Bugfix] Re-enable Gemma3 for V1 (#14980) · 61f41218
  Cyrus Leung authored Mar 19, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  61f41218
- [Core] Update dtype detection and defaults (#14858) · f690372b
  Cyrus Leung authored Mar 19, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  f690372b
18 Mar, 2025 7 commits
- [V1] TPU - CI/CD use smaller model (#15054) · 72a8639b
  Alexander Matveev authored Mar 18, 2025
```
Signed-off-by: Alexander Matveev <amatveev@redhat.com>
```
  72a8639b
- [V1][Spec Decode] Optimize Rejection Sampler with Triton Kernels (#14930) · 99abb8b6
  Woosuk Kwon authored Mar 18, 2025
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  99abb8b6
- [Bugfix] Fix LoRA extra vocab size (#15047) · 46c759c1
  Jee Jee Li authored Mar 19, 2025
```
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
```
  46c759c1
- [MODEL] Add support for Zamba2 models (#13185) · 452e8fd9
  yury-tokpanov authored Mar 18, 2025
```
Signed-off-by: Yury Tokpanov <yury@zyphra.com>
Signed-off-by: Quentin Anthony <qganthony@yahoo.com>
Co-authored-by: Quentin Anthony <qganthony@yahoo.com>
Co-authored-by: Tyler Michael Smith <tysmith@redhat.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
```
  452e8fd9
- [Mistral-Small 3.1] Update docs and tests (#14977) · f863ffc9
  Patrick von Platen authored Mar 18, 2025
```
Signed-off-by: Roger Wang <ywang@roblox.com>
Co-authored-by: Roger Wang <ywang@roblox.com>
```
  f863ffc9
- [Kernels] LoRA - Retire SGMV and BGMV Kernels (#14685) · 400d483e
  Varun Sundar Rabindranath authored Mar 18, 2025
```
Signed-off-by: Varun Sundar Rabindranath <varun@neuralmagic.com>
Co-authored-by: Varun Sundar Rabindranath <varun@neuralmagic.com>
```
  400d483e
- [Neuron] trim attention kernel tests to fit trn1.2x instance (#14988) · 53a0cf8b
  Liangfu Chen authored Mar 18, 2025
```
Signed-off-by: Liangfu Chen <liangfc@amazon.com>
```
  53a0cf8b
17 Mar, 2025 7 commits

[Bugfix] Fix bnb quantization for models with both HF-format and Mistral-format weights (#14950) · 5eeabc2a
Tristan Leclercq authored Mar 18, 2025

5eeabc2a
[V1] TPU - Fix CI/CD runner (#14974) · 18551e82
Alexander Matveev authored Mar 17, 2025

18551e82
[CI/Build] Use `AutoModelForImageTextToText` to load VLMs in tests (#14945) · b89fb2a4
Cyrus Leung authored Mar 18, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
b89fb2a4

[Fix][Structured Output] using vocab_size to construct matcher (#14868) · c0efdd65

Aaron Pham authored Mar 17, 2025

Signed-off-by: Russell Bryant <rbryant@redhat.com>
Signed-off-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com>
Signed-off-by: Aaron Pham <contact@aarnphm.xyz>
Co-authored-by: rshaw@neuralmagic.com <rshaw@neuralmagic.com>
Co-authored-by: Russell Bryant <rbryant@redhat.com>
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com>

c0efdd65

[Bugfix][ROCm] running new process using spawn method for rocm in tests. (#14810) · 2bb0e1a7

vllmellm authored Mar 17, 2025


Signed-off-by: vllmellm <vllm.ellm@embeddedllm.com>
Signed-off-by: tjtanaa <tunjian.tan@embeddedllm.com>
Co-authored-by: TJian <tunjian.tan@embeddedllm.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>

2bb0e1a7

[V1] [Spec Decode] Support random sampling for spec decode (#13933) · 8d6cf895
Lily Liu authored Mar 16, 2025
```
Co-authored-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
8d6cf895

[Misc] Replace os environ to monkeypatch in test suite (#14516) · a73e183e

Sibi authored Mar 17, 2025


Signed-off-by: sibi <85477603+t-sibiraj@users.noreply.github.com>
Signed-off-by: Aaron Pham <contact@aarnphm.xyz>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
Co-authored-by: Aaron Pham <contact@aarnphm.xyz>

a73e183e