Commits · 247181536fc2cab728077f3e7489622e19671d2d · OpenDAS / vllm_cscc · GitLab

27 Mar, 2025 5 commits
- [Misc] Replace `is_encoder_decoder_inputs` with `split_enc_dec_inputs` (#15620) · 24718153
  Cyrus Leung authored Mar 28, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  24718153
- [V1] Refactor num_computed_tokens logic (#15307) · 54aa6194
  Cody Yu authored Mar 26, 2025
```
Signed-off-by: Cody Yu <hao.yu.cody@gmail.com>
Co-authored-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  54aa6194
- [misc] LoRA: Remove unused long context test data (#15558) · 8095341a
  Varun Sundar Rabindranath authored Mar 26, 2025
```
Signed-off-by: Varun Sundar Rabindranath <varun@neuralmagic.com>
Co-authored-by: Varun Sundar Rabindranath <varun@neuralmagic.com>
```
  8095341a
- [Kernel] CUTLASS grouped gemm fp8 MoE kernel (#13972) · 9239bf71
  ElizaWszola authored Mar 27, 2025
```
Signed-off-by: ElizaWszola <eliza@neuralmagic.com>
Signed-off-by: ElizaWszola <ewszola@redhat.com>
Co-authored-by: Lucas Wilkinson <wilkinson.lucas@gmail.com>
```
  9239bf71
- Support FIPS enabled machines with MD5 hashing (#15299) · 7a6d45bc
  Matthew Vine authored Mar 26, 2025
```
Signed-off-by: Matthew Vine <32849887+MattTheCuber@users.noreply.github.com>
```
  7a6d45bc
26 Mar, 2025 10 commits
- [V1] TPU CI - Fix test_compilation.py (#15570) · 9d119a86
  Alexander Matveev authored Mar 26, 2025
```
Signed-off-by: Alexander Matveev <amatveev@redhat.com>
```
  9d119a86
- Support SHA256 as hash function in prefix caching (#15297) · 27df5199
  marko authored Mar 26, 2025
```
Signed-off-by: Marko Rosenmueller <5467316+dr75@users.noreply.github.com>
```
  27df5199
- [V1][Sampler] Faster top-k only implementation (#15478) · 35fad35a
  Nick Hill authored Mar 26, 2025
```
Signed-off-by: Nick Hill <nhill@redhat.com>
```
  35fad35a
- [Model] Add Reasoning Parser for Granite Models (#14202) · 1711b929
  Alex Brooks authored Mar 26, 2025
```
Signed-off-by: Alex-Brooks <Alex.brooks@ibm.com>
Co-authored-by: Joe Runde <joe@joerun.de>
```
  1711b929
- Separate base model from `TransformersModel` (#15467) · cf5c8f16
  Harry Mellor authored Mar 26, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Signed-off-by: Isotr0py <2037008807@qq.com>
Co-authored-by: Isotr0py <2037008807@qq.com>
Co-authored-by: Isotr0py <mozf@mail2.sysu.edu.cn>
```
  cf5c8f16
- [Misc] Enhance warning information to user-defined chat template (#15408) · 99f536f8
  wwl2755 authored Mar 26, 2025
```
Signed-off-by: wwl2755 <wangwenlong2755@gmail.com>
```
  99f536f8
- [FEAT][ROCm] Integrate Fused MoE Kernels from AITER (#14967) · 5ebf6674
  vllmellm authored Mar 26, 2025
```
Signed-off-by: vllmellm <vllm.ellm@embeddedllm.com>
Signed-off-by: tjtanaa <tunjian.tan@embeddedllm.com>
Co-authored-by: tjtanaa <tunjian.tan@embeddedllm.com>
```
  5ebf6674
- [Model] Support multi-image for Molmo (#15438) · 997c8811
  Cyrus Leung authored Mar 26, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  997c8811
- Transformers backend already supports V1 (#15463) · e42389f9
  Harry Mellor authored Mar 26, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  e42389f9
- [CI/Build] LoRA: Delete long context tests (#15503) · ff38f0a3
  Varun Sundar Rabindranath authored Mar 25, 2025
```
Signed-off-by: Varun Sundar Rabindranath <varun@neuralmagic.com>
Co-authored-by: Varun Sundar Rabindranath <varun@neuralmagic.com>
```
  ff38f0a3
25 Mar, 2025 8 commits
- [core] add bucket padding to tpu_model_runner (#14995) · ac3cd6e8
  Chenyaaang authored Mar 25, 2025
```
Signed-off-by: Chenyaaang <llccyy1212@gmail.com>
Signed-off-by: rshaw@neuralmagic.com <robertgshaw2@gmail.com>
Co-authored-by: rshaw@neuralmagic.com <robertgshaw2@gmail.com>
```
  ac3cd6e8
- [V1] Support long_prefill_token_threshold in v1 scheduler (#15419) · 082ab86f
  Lu Fang authored Mar 25, 2025
```
Signed-off-by: Lu Fang <lufang@fb.com>
```
  082ab86f
- [CI/Build] Add tests for the V1 tpu_model_runner. (#14843) · 0a049c7d
  yarongmu-google authored Mar 25, 2025
```
Signed-off-by: Yarong Mu <ymu@google.com>
```
  0a049c7d
- [Misc] Clean up MiniCPM-V/O code (#15337) · a9e879b3
  Cyrus Leung authored Mar 25, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  a9e879b3
- [Kernel][CPU] CPU MLA (#14744) · 4f044b1d
  Thien Tran authored Mar 25, 2025
```
Signed-off-by: Thien Tran <gau.nernst@yahoo.com.sg>
```
  4f044b1d
- [V1] guidance backend for structured output + `auto` fallback mode (#14779) · a09ad90a
  Russell Bryant authored Mar 25, 2025
```
Signed-off-by: Russell Bryant <rbryant@redhat.com>
Co-authored-by: Loc Huynh <jc1da.3011@gmail.com>
Co-authored-by: Michal Moskal <michal@moskal.me>
```
  a09ad90a
- Add pipeline parallel support to `TransformersModel` (#12832) · 97cfa65d
  Harry Mellor authored Mar 25, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Signed-off-by: Isotr0py <2037008807@qq.com>
Co-authored-by: Isotr0py <2037008807@qq.com>
```
  97cfa65d
- [V1][Spec Decode] Enable spec decode for top-p & top-k sampling (#15063) · ebcebeeb
  Woosuk Kwon authored Mar 24, 2025
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  ebcebeeb
24 Mar, 2025 8 commits
- [ROCm][Kernel] MoE weights padding (#14454) · f533b583
  Gregory Shtrasberg authored Mar 24, 2025
```
Signed-off-by: Gregory Shtrasberg <Gregory.Shtrasberg@amd.com>
Signed-off-by: charlifu <charlifu@amd.com>
Co-authored-by: charlifu <charlifu@amd.com>
```
  f533b583
- [Build] Cython compilation support fix (#14296) · 8279201c
  Gregory Shtrasberg authored Mar 24, 2025
```
Signed-off-by: Gregory Shtrasberg <Gregory.Shtrasberg@amd.com>
```
  8279201c
- [Hardware][TPU] Skip failed compilation test (#15421) · 23fdab00
  Siyuan Liu authored Mar 24, 2025
```
Signed-off-by: Siyuan Liu <lsiyuan@google.com>
```
  23fdab00
- [V1][Perf] Simpler request output queues (#15156) · 9d72daf4
  Nick Hill authored Mar 24, 2025
```
Signed-off-by: Nick Hill <nhill@redhat.com>
Signed-off-by: rshaw@neuralmagic.com <robertgshaw2@gmail.com>
Co-authored-by: rshaw@neuralmagic.com <robertgshaw2@gmail.com>
```
  9d72daf4
- [Core] Integrate `fastsafetensors` loader for loading model weights (#10647) · 761702fd
  Manish Sethi authored Mar 24, 2025
```
Signed-off-by: Manish Sethi <Manish.sethi1@ibm.com>
```
  761702fd
- [Bugfix] Fix chat template loading (#15143) · cbcdf2c6
  Cyrus Leung authored Mar 24, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Signed-off-by: Roger Wang <ywang@roblox.com>
Co-authored-by: chaunceyjiang <chaunceyjiang@gmail.com>
Co-authored-by: Roger Wang <ywang@roblox.com>
```
  cbcdf2c6
- [Kernel] allow non-contiguous input for marlin kernel (#14658) · 6b3cc75b
  Jinzhen Lin authored Mar 24, 2025
```
Signed-off-by: Jinzhen Lin <linjinzhen@hotmail.com>
```
  6b3cc75b
- [Fix] [torch.compile] Improve UUID system for custom passes (#15249) · f622dbcf
  Luka Govedič authored Mar 23, 2025
```
Signed-off-by: luka <luka@neuralmagic.com>
```
  f622dbcf
23 Mar, 2025 4 commits
- [Frontend] Support tool calling and reasoning parser (#14511) · d6cd59f1
  Robin authored Mar 24, 2025
```
Signed-off-by: WangErXiao <863579016@qq.com>
```
  d6cd59f1
- [V1][Spec Decode] Respect prompt_lookup_max (#15348) · b9bd76ca
  Woosuk Kwon authored Mar 23, 2025
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  b9bd76ca
- [ci/build] fix broken tests in LLM.collective_rpc (#15350) · f68cce8e
  youkaichao authored Mar 23, 2025
```
Signed-off-by: youkaichao <youkaichao@gmail.com>
```
  f68cce8e
- [V1][Usage] Refactor speculative decoding configuration and tests (#14434) · 50c9636d
  shangmingc authored Mar 23, 2025
```
Signed-off-by: Shangming Cai <caishangming@linux.alibaba.com>
```
  50c9636d
22 Mar, 2025 5 commits
- Remove openvino support in favor of external plugin (#15339) · b877031d
  Russell Bryant authored Mar 22, 2025
```
Signed-off-by: Russell Bryant <rbryant@redhat.com>
```
  b877031d
- [V1] Add `disable-any-whitespace` option support for xgrammar (#15316) · eb63ea1e
  Russell Bryant authored Mar 22, 2025
```
Signed-off-by: Russell Bryant <rbryant@redhat.com>
```
  eb63ea1e
- [Model] Support Tele-FLM Model (#15023) · 2f4bd358
  Naitong Yu authored Mar 22, 2025
```
Signed-off-by: Naitong Yu <ntyu@baai.ac.cn>
Signed-off-by: jiangxin <horizon94@outlook.com>
Co-authored-by: Jason Fang <jasonfang3900@gmail.com>
Co-authored-by: jiangxin <horizon94@outlook.com>
```
  2f4bd358
- [Bugfix] LoRA V0 - Fix case where `max_num_seqs` is between cudagraph capture sizes (#15308) · 8a8b30ea
  Varun Sundar Rabindranath authored Mar 22, 2025
```
Signed-off-by: Varun Sundar Rabindranath <varun@neuralmagic.com>
Co-authored-by: Varun Sundar Rabindranath <varun@neuralmagic.com>
```
  8a8b30ea
- [FEAT] [ROCm]: Add AITER RMS Norm (Layer Norm) Feature (#14959) · ec870fba
  TJian authored Mar 22, 2025
```
Signed-off-by: tjtanaa <tunjian.tan@embeddedllm.com>
```
  ec870fba