Commits · a257d9bccc818cf826311ff03fbf5b46666321f1 · OpenDAS / vllm_cscc · GitLab

01 May, 2025 11 commits
- Improve configs - `ObservabilityConfig` (#17453) · a257d9bc
  Harry Mellor authored May 01, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  a257d9bc
- [Misc] Optimize the Qwen3_ReasoningParser extract_reasoning_content (#17515) · 015069b0
  Chauncey authored May 01, 2025
```
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
```
  015069b0
- [Core] Enable IPv6 with vllm.utils.make_zmq_socket() (#16506) · fbefc8a7
  Russell Bryant authored May 01, 2025
```
Signed-off-by: Russell Bryant <rbryant@redhat.com>
```
  fbefc8a7
- Avoid overwriting vllm_compile_cache.py (#17418) · 26bc4bbc
  Keyun Tong authored May 01, 2025
```
Signed-off-by: Keyun Tong <tongkeyun@gmail.com>
```
  26bc4bbc
- [BugFix] Fix mla cpu - missing 3 required positional arguments (#17494) · 3c3d7672
  Lucas Wilkinson authored May 01, 2025
```
Signed-off-by: Lucas Wilkinson <lwilkinson@neuralmagic.com>
```
  3c3d7672
- [BugFix] fix speculative decoding memory leak when speculation is disabled (#15506) · 13cf6b62
  Noah Yoshida authored Apr 30, 2025
```
Signed-off-by: Noah Yoshida <noahcy117@gmail.com>
```
  13cf6b62
- FIxing the AMD test failures caused by PR#16457 (#17511) · 7ab643e4
  Alexei-V-Ivanov-AMD authored May 01, 2025
```
Signed-off-by: Alexei V. Ivanov <alexei.ivanov@amd.com>
```
  7ab643e4
- [CI/Build] Reorganize models tests (#17459) · afb4429b
  Cyrus Leung authored May 01, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  afb4429b
- [Bugfix] Temporarily disable gptq_bitblas on ROCm (#17411) · 1144a8ef
  NaLan ZeYu authored May 01, 2025
```
Signed-off-by: Yan Cangang <nalanzeyu@gmail.com>
```
  1144a8ef
- [Bugfix][ROCm] Fix import error on ROCm (#17495) · 08fb5587
  Gregory Shtrasberg authored Apr 30, 2025
```
Signed-off-by: Gregory Shtrasberg <Gregory.Shtrasberg@amd.com>
```
  08fb5587
- [Misc] Rename Audios -> Audio in Qwen2audio Processing (#17507) · 02bd6548
  Alex Brooks authored Apr 30, 2025
```
Signed-off-by: Alex-Brooks <Alex.Brooks@ibm.com>
```
  02bd6548
30 Apr, 2025 19 commits
- [v1][Spec Decode] Make sliding window compatible with eagle prefix caching (#17398) · 81ecf425
  Chen Zhang authored May 01, 2025
```
Signed-off-by: Chen Zhang <zhangch99@outlook.com>
```
  81ecf425
- [V1] Allow turning off pickle fallback in vllm.v1.serial_utils (#17427) · 947f2f53
  Russell Bryant authored Apr 30, 2025
```
Signed-off-by: Russell Bryant <rbryant@redhat.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
```
  947f2f53
- [Bugfix] Fixed mistral tokenizer path when pointing to file (#17457) · 739e03b3
  Pete Savage authored Apr 30, 2025
```
Signed-off-by: Pete Savage <psavage@redhat.com>
```
  739e03b3
- [Fix] Support passing args to logger (#17425) · da4e7687
  Aaron Pham authored Apr 30, 2025
```
Signed-off-by: Aaron Pham <contact@aarnphm.xyz>
```
  da4e7687
- [Feature] The Qwen3 reasoning parser supports guided decoding (#17466) · 2990cee9
  Chauncey authored Apr 30, 2025
```
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
```
  2990cee9
- [V1][Metrics] add support for kv event publishing (#16750) · 0be6d05b
  Alec authored Apr 30, 2025
```
Signed-off-by: alec-flowers <aflowers@nvidia.com>
Signed-off-by: Mark McLoughlin <markmc@redhat.com>
Co-authored-by: Mark McLoughlin <markmc@redhat.com>
```
  0be6d05b
- [Core] Prevent side-channel attacks via cache salting (#17045) · 77073c77
  Marko Rosenmueller authored Apr 30, 2025
```
Signed-off-by: Marko Rosenmueller <5467316+dr75@users.noreply.github.com>
```
  77073c77
- [V1][Bugfix]: vllm v1 verison metric num_gpu_blocks is None (#15755) · d8037867
  rongfu.leng authored Apr 30, 2025
```
Signed-off-by: rongfu.leng <rongfu.leng@daocloud.io>
```
  d8037867
- [Misc] Remove deprecated files (#17447) · 1534d389
  Chauncey authored Apr 30, 2025
```
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
```
  1534d389
- Make the _apply_rotary_emb compatible with dynamo (#17435) · ece5a8b0
  Lu Fang authored Apr 30, 2025
  
  ece5a8b0
- [MODEL ADDITION] Ovis2 Model Addition (#15826) · 54072f31
  Marco authored Apr 30, 2025
```
Signed-off-by: Marco <121761685+mlinmg@users.noreply.github.com>
Signed-off-by: Isotr0py <2037008807@qq.com>
Signed-off-by: isotr0py <2037008807@qq.com>
Co-authored-by: Isotr0py <2037008807@qq.com>
Co-authored-by: Isotr0py <mozf@mail2.sysu.edu.cn>
```
  54072f31
- [Bugfix] Fix AttributeError: 'State' object has no attribute 'engine_client' (#17434) · be633fba
  Chauncey authored Apr 30, 2025
```
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
```
  be633fba
- [Hardware][Intel GPU] Upgrade to torch 2.7 (#17444) · ed6cfb90
  Kunshang Ji authored Apr 30, 2025
```
Signed-off-by: Kunshang Ji <kunshang.ji@intel.com>
Co-authored-by: Qiming Zhang <qiming1.zhang@intel.com>
```
  ed6cfb90
- Support LoRA for Mistral3 (#17428) · a44c4f1d
  Michael Goin authored Apr 29, 2025
```
Signed-off-by: mgoin <mgoin64@gmail.com>
```
  a44c4f1d
- Fix call to `logger.info_once` (#17416) · d1f569b1
  Harry Mellor authored Apr 30, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  d1f569b1
- Improve configs - `ModelConfig` (#17130) · 13698db6
  Harry Mellor authored Apr 30, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  13698db6
- Update PyTorch to 2.7.0 (#16859) · 2c4f59af
  Huy Do authored Apr 29, 2025
  
  2c4f59af
- Truncation control for embedding models (#14776) · 1c2bc7ea
  Gabriel Marinho authored Apr 29, 2025
```
Signed-off-by: Gabriel Marinho <gmarinho@ibm.com>
Signed-off-by: Max de Bayser <mbayser@br.ibm.com>
Co-authored-by: Max de Bayser <mbayser@br.ibm.com>
```
  1c2bc7ea
- [V1][Feature] Enable Speculative Decoding with Structured Outputs (#14702) · 34120f5a
  Benjamin Chislett authored Apr 29, 2025
```
Signed-off-by: Benjamin Chislett <benjamin.chislett@centml.ai>
Signed-off-by: Benjamin Chislett <chislett.ben@gmail.com>
```
  34120f5a
29 Apr, 2025 10 commits
- [V1][Spec Decode] Apply torch.compile & cudagraph to EAGLE (#17211) · 70788bdb
  Bryan Lu authored Apr 29, 2025
```
Signed-off-by: Bryan Lu <yuzhelu@amazon.com>
```
  70788bdb
- Simplify (and fix) passing of guided decoding backend options (#17008) · a6977dbd
  Harry Mellor authored Apr 29, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  a6977dbd
- [Bugfix] Fix Minicpm-O-int4 GPTQ model inference (#17397) · 2fa2a50b
  Isotr0py authored Apr 30, 2025
```
Signed-off-by: Isotr0py <2037008807@qq.com>
```
  2fa2a50b
- fix gemma3 results all zero (#17364) · d3cf61b8
  Qiming Zhang authored Apr 29, 2025
```
Signed-off-by: mayuyuace <qiming1.zhang@intel.com>
```
  d3cf61b8
- [Bugfix] add qwen3 reasoning-parser fix content is None when disable … (#17369) · a39203f9
  mofanke authored Apr 30, 2025
```
Signed-off-by: mofanke <mofanke@gmail.com>
```
  a39203f9
- [V1] Remove num_input_tokens from attn_metadata (#17193) · 24e6ad3f
  Chen Zhang authored Apr 30, 2025
```
Signed-off-by: Chen Zhang <zhangch99@outlook.com>
```
  24e6ad3f
- Improve literal dataclass field conversion to argparse argument (#17391) · 2ef5d106
  Harry Mellor authored Apr 29, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  2ef5d106
- Fix: Spelling of inference (#17387) · 0ed27ef6
  a2q1p authored Apr 30, 2025
  
  0ed27ef6
- Transformers backend tweaks (#17365) · 900edfa8
  Harry Mellor authored Apr 29, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  900edfa8
- [Frontend] Support `chat_template_kwargs` in `LLM.chat` (#17356) · 88ad9ec6
  Cyrus Leung authored Apr 29, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  88ad9ec6