Commits · 99ed5261019ddee871d2ee6186bf4b0a4e46c098 · OpenDAS / vllm_cscc · GitLab

17 Apr, 2025 18 commits
- [Misc] refactor examples series - lmcache (#16758) · 99ed5261
  Reid authored Apr 17, 2025
```
Signed-off-by: reidliu41 <reid201711@gmail.com>
Co-authored-by: reidliu41 <reid201711@gmail.com>
```
  99ed5261
- [Doc] Fix a 404 link in installation/cpu.md (#16773) · 207da281
  Michael Yao authored Apr 17, 2025
```
Signed-off-by: windsonsea <haifeng.yao@daocloud.io>
```
  207da281
- [Bugfix] Fix GLM4 model (#16618) · 5b1aca2a
  intervitens authored Apr 17, 2025
```
Signed-off-by: intervitens <intervitens@tutanota.com>
```
  5b1aca2a
- [doc] add open-webui example (#16747) · d8e557b5
  Reid authored Apr 17, 2025
```
Signed-off-by: reidliu41 <reid201711@gmail.com>
Co-authored-by: reidliu41 <reid201711@gmail.com>
```
  d8e557b5
- [Doc] Add more tips to avoid OOM (#16765) · 61a44a0b
  Cyrus Leung authored Apr 17, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  61a44a0b
- [misc] ignore marlin_moe_wna16 local gen codes (#16760) · a6481525
  DefTruth authored Apr 17, 2025
```
Signed-off-by: DefTruth <qiustudent_r@163.com>
```
  a6481525
- [Ray] Improve documentation on batch inference (#16609) · 8cac35ba
  Richard Liaw authored Apr 16, 2025
```
Signed-off-by: Richard Liaw <rliaw@berkeley.edu>
```
  8cac35ba
- [V1] Remove log noise when idle (#16735) · 9dbf7a2d
  Russell Bryant authored Apr 17, 2025
```
Signed-off-by: Russell Bryant <rbryant@redhat.com>
```
  9dbf7a2d
- [Bugfix] Revert max_prompt_len validation for decoder-only models. (#16741) · 607029e5
  David Heineman authored Apr 16, 2025
```
Signed-off-by: David Heineman <david@davidheineman.com>
```
  607029e5
- [Bugfix] Update Florence-2 tokenizer to make grounding tasks work (#16734) · cb072ce9
  Isotr0py authored Apr 17, 2025
```
Signed-off-by: Isotr0py <2037008807@qq.com>
```
  cb072ce9
- [rocm][V0] fix selection logic for custom PA in V0 (#16426) · 95aca283
  Divakar Verma authored Apr 16, 2025
```
Signed-off-by: Divakar Verma <divakar.verma@amd.com>
```
  95aca283
- [V1][Frontend] Improve Shutdown And Logs (#11737) · 2b05b8ce
  Robert Shaw authored Apr 16, 2025
```
Signed-off-by: rshaw@neuralmagic.com <rshaw@neuralmagic.com>
Signed-off-by: Andrew Feldman <afeldman@neuralmagic.com>
Signed-off-by: Nick Hill <nhill@redhat.com>
Co-authored-by: rshaw@neuralmagic.com <rshaw@neuralmagic.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
Co-authored-by: Russell Bryant <rbryant@redhat.com>
Co-authored-by: Andrew Feldman <afeldman@neuralmagic.com>
Co-authored-by: afeldman-nm <156691304+afeldman-nm@users.noreply.github.com>
Co-authored-by: Nick Hill <nhill@redhat.com>
```
  2b05b8ce
- Adding vllm buildkite job for IBM Power (#16679) · 3c776dce
  Aaruni Aggarwal authored Apr 17, 2025
```
Signed-off-by: Aaruni Aggarwal <aaruniagg@gmail.com>
```
  3c776dce
- [V1][Spec Dec Bug Fix] Respect Spec Dec Method Specification (#16636) · 2cbd4d29
  Bryan Lu authored Apr 16, 2025
```
Signed-off-by: Bryan Lu <yuzhelu@amazon.com>
```
  2cbd4d29
- [V1][Performance] Implement custom serializaton for MultiModalKwargs [Rebased] (#16432) · 3092375e
  Staszek Paśko authored Apr 17, 2025
```
Signed-off-by: Staszek Pasko <staszek@gmail.com>
Signed-off-by: Nick Hill <nhill@redhat.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
Co-authored-by: Nick Hill <nhill@redhat.com>
```
  3092375e
- Help user create custom model for Transformers backend remote code models (#16719) · 3cd91dc9
  Harry Mellor authored Apr 17, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  3cd91dc9
- [Misc] Remove redundant comment (#16703) · 8a7368e0
  Jade Zheng authored Apr 17, 2025
```
Signed-off-by: Jade Zheng <zheng.shoujian@outlook.com>
```
  8a7368e0
- Improve error for structured output backend selection (#16717) · 93e561ec
  Harry Mellor authored Apr 17, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  93e561ec
16 Apr, 2025 13 commits
- [Hardware] Add processor inputs to platform validation (#16680) · e1b00483
  Joe Runde authored Apr 16, 2025
```
Signed-off-by: Joe Runde <Joseph.Runde@ibm.com>
```
  e1b00483
- [Model] support modernbert (#16648) · ee378f3d
  xsank authored Apr 16, 2025
```
Signed-off-by: 唯勤 <xsank.mz@alibaba-inc.com>
Co-authored-by: 唯勤 <xsank.mz@alibaba-inc.com>
```
  ee378f3d
- [Bugfix][Kernel] fix potential cuda graph broken for merge_attn_states kernel (#16693) · e82ee40d
  DefTruth authored Apr 16, 2025
```
Signed-off-by: DefTruth <qiustudent_r@163.com>
```
  e82ee40d
- [Doc] Improve OOM troubleshooting (#16704) · facbe2a1
  Cyrus Leung authored Apr 16, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  facbe2a1
- [Misc] refactor examples series (#16708) · 71689204
  Reid authored Apr 16, 2025
```
Signed-off-by: reidliu41 <reid201711@gmail.com>
Co-authored-by: reidliu41 <reid201711@gmail.com>
```
  71689204
- [CI] Cleanup `additional_dependencies: [toml]` for pre-commit yapf hook (#16405) · 21378a23
  Kay Yan authored Apr 16, 2025
```
Signed-off-by: Kay Yan <kay.yan@daocloud.io>
```
  21378a23
- [V1][Structured Output] Move xgrammar related utils to `backend_xgrammar.py` (#16578) · 976711d9
  Shanshan Shen authored Apr 16, 2025
```
Signed-off-by: shen-shanshan <467638484@qq.com>
```
  976711d9
- [ROCM] Bind triton version to 3.2 in requirements-built.txt (#16664) · 44fa4d55
  Sage Moore authored Apr 15, 2025
```
Signed-off-by: Sage Moore <sage@neuralmagic.com>
```
  44fa4d55
- [Feature] add model aware kv ops helper (#16020) · 3ac98edc
  billishyahao authored Apr 16, 2025
```
Signed-off-by: billishyahao <bill.he@amd.com>
```
  3ac98edc
- Disable remote caching when calling compile_fx (#16611) · 966c742e
  Richard Zou authored Apr 16, 2025
```
Signed-off-by: rzou <zou3519@gmail.com>
```
  966c742e
- [Misc] Modify LRUCache touch (#16689) · 0d7d05f4
  Jee Jee Li authored Apr 16, 2025
```
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
```
  0d7d05f4
- [Bugfix] fix gpu docker image mis benchmarks dir (#16628) · 96bb8aa6
  rongfu.leng authored Apr 16, 2025
```
Signed-off-by: rongfu.leng <rongfu.leng@daocloud.io>
```
  96bb8aa6
- [Model] Add PLaMo2 (#14323) · 3badb021
  Shinichi Hemmi authored Apr 16, 2025
```
Signed-off-by: Shinichi Hemmi <50256998+Alnusjaponica@users.noreply.github.com>
Signed-off-by: shemmi <shemmi@preferred.jp>
Co-authored-by: Kento Nozawa <nzw0301@preferred.jp>
Co-authored-by: Hiroaki Mikami <mhiroaki@preferred.jp>
Co-authored-by: Calvin Metzger <metzger@preferred.jp>
```
  3badb021
15 Apr, 2025 9 commits
- [Misc] Enable vLLM to Dynamically Load LoRA from a Remote Server (#10546) · fdcb850f
  Angky William authored Apr 15, 2025
```
Signed-off-by: Angky William <angkywilliam@Angkys-MacBook-Pro.local>
Co-authored-by: Angky William <angkywilliam@Angkys-MacBook-Pro.local>
```
  fdcb850f
- [Misc] Update `compressed-tensors` WNA16 to support zero-points (#14211) · 54a66e5f
  Dipika Sikka authored Apr 15, 2025
  
  54a66e5f
- [Kernel] Remove redundant Exp calculations (#16123) · 280d62b8
  DefTruth authored Apr 15, 2025
```
Signed-off-by: DefTruth <qiustudent_r@163.com>
```
  280d62b8
- Add "/server_info" endpoint in api_server to retrieve the vllm_config. (#16572) · 1666e664
  Xihui Cang authored Apr 15, 2025
```
Signed-off-by: Xihui Cang <xihuicang@gmail.com>
```
  1666e664
- [CI/Build] Fix LoRA OOM (#16624) · 1575c170
  Jee Jee Li authored Apr 15, 2025
```
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
```
  1575c170
- [Misc] refactor argument parsing in examples (#16635) · 6ae996a8
  Reid authored Apr 15, 2025
```
Signed-off-by: reidliu41 <reid201711@gmail.com>
Co-authored-by: reidliu41 <reid201711@gmail.com>
```
  6ae996a8
- Fix vLLM x torch.compile config caching (#16491) · b590adfd
  Richard Zou authored Apr 15, 2025
```
Signed-off-by: rzou <zou3519@gmail.com>
```
  b590adfd
- Add `vllm bench [latency, throughput]` CLI commands (#16508) · b4fe16c7
  Michael Goin authored Apr 15, 2025
```
Signed-off-by: mgoin <mgoin64@gmail.com>
```
  b4fe16c7
- [Bugfix] Fix broken GritLM model and tests (missing pooling_metadata) (#16631) · bc5dd4f6
  Pooya Davoodi authored Apr 14, 2025
```
Signed-off-by: Pooya Davoodi <pooya.davoodi@parasail.io>
```
  bc5dd4f6