Commits · 7d4aedae7c3e1ff4a1b1dea1f8df3257a7878d0a · OpenDAS / vllm_cscc

09 May, 2025 2 commits
- Handle error when `str` passed to `/v1/audio/transcriptions` (#17909) · 7d4aedae
  Harry Mellor authored May 09, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  7d4aedae
- Change `top_k` to be disabled with `0` (still accept `-1` for now) (#17773) · c6798baa
  Harry Mellor authored May 09, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  c6798baa
08 May, 2025 2 commits
- [Frontend] Chat template fallbacks for multimodal models (#17805) · 96722aa8
  Cyrus Leung authored May 08, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  96722aa8
- Only log non-default CLI args for online serving (#17803) · 998eea4a
  Harry Mellor authored May 08, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  998eea4a
05 May, 2025 1 commit
- [Bugfix] fix KeyError on top logprobs are special tokens (#17637) · 5394ad73
  Chauncey authored May 05, 2025
```
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
```
  5394ad73
04 May, 2025 1 commit
- Add full API docs and improve the UX of navigating them (#17485) · d6484ef3
  Harry Mellor authored May 04, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  d6484ef3
02 May, 2025 1 commit
- [Misc] Clean up input processing (#17582) · cb234955
  Cyrus Leung authored May 02, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  cb234955
01 May, 2025 2 commits
- [Feature][Frontend]: Deprecate --enable-reasoning (#17452) · 98060b00
  Chauncey authored May 01, 2025
```
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
```
  98060b00
- [Frontend] Show progress bar for adding requests (#17525) · 1903c0b8
  Cyrus Leung authored May 01, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  1903c0b8
30 Apr, 2025 5 commits

[Core] Prevent side-channel attacks via cache salting (#17045) · 77073c77
Marko Rosenmueller authored Apr 30, 2025
```
Signed-off-by: Marko Rosenmueller <5467316+dr75@users.noreply.github.com>
```
77073c77

[MODEL ADDITION] Ovis2 Model Addition (#15826) · 54072f31

Marco authored Apr 30, 2025


Signed-off-by: Marco <121761685+mlinmg@users.noreply.github.com>
Signed-off-by: Isotr0py <2037008807@qq.com>
Signed-off-by: isotr0py <2037008807@qq.com>
Co-authored-by: Isotr0py <2037008807@qq.com>
Co-authored-by: Isotr0py <mozf@mail2.sysu.edu.cn>

54072f31

[Bugfix] Fix AttributeError: 'State' object has no attribute 'engine_client' (#17434) · be633fba
Chauncey authored Apr 30, 2025
```
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
```
be633fba
Improve configs - `ModelConfig` (#17130) · 13698db6
Harry Mellor authored Apr 30, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
13698db6

Truncation control for embedding models (#14776) · 1c2bc7ea

Gabriel Marinho authored Apr 29, 2025


Signed-off-by: Gabriel Marinho <gmarinho@ibm.com>
Signed-off-by: Max de Bayser <mbayser@br.ibm.com>
Co-authored-by: Max de Bayser <mbayser@br.ibm.com>

1c2bc7ea

29 Apr, 2025 1 commit
- [Frontend] Support `chat_template_kwargs` in `LLM.chat` (#17356) · 88ad9ec6
  Cyrus Leung authored Apr 29, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  88ad9ec6
28 Apr, 2025 2 commits
- Improve conversion from dataclass configs to argparse arguments (#17303) · f9488694
  Harry Mellor authored Apr 28, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  f9488694
- [Model] Add Granite Speech Support (#16246) · fa93cd9f
  Alex Brooks authored Apr 28, 2025
```
Signed-off-by: Alex-Brooks <Alex.brooks@ibm.com>
Signed-off-by: Alex-Brooks <Alex.Brooks@ibm.com>
```
  fa93cd9f
26 Apr, 2025 1 commit
- [V1] Add `structural_tag` support using xgrammar (#17085) · f8acd01f
  Russell Bryant authored Apr 26, 2025
  
  f8acd01f
25 Apr, 2025 4 commits
- [BugFix][Frontend] Fix `LLM.chat()` tokenization (#16081) · 70116459
  Nick Hill authored Apr 25, 2025
```
Signed-off-by: Nick Hill <nhill@redhat.com>
```
  70116459
- [Bugfix] Fix Mistral ChatCompletionRequest Body Exception (#16769) · d5615af9
  Jasmond L authored Apr 25, 2025
```
Signed-off-by: Jasmond Loh <Jasmond.Loh@hotmail.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
```
  d5615af9
- [Bugfix] Fix mistral model tests (#17181) · 19dcc02a
  Cyrus Leung authored Apr 25, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  19dcc02a
- [Doc] Move todo out of beam search docstring (#17183) · 7feae92c
  Alex Brooks authored Apr 25, 2025
```
Signed-off-by: Alex-Brooks <Alex.Brooks@ibm.com>
```
  7feae92c
24 Apr, 2025 2 commits
- Add chat template for Llama 4 models (#16428) · 05e1fbfc
  Maximilien de Bayser authored Apr 24, 2025
```
Signed-off-by: Max de Bayser <mbayser@br.ibm.com>
```
  05e1fbfc
- Simplify `TokenizerGroup` (#16790) · 0a05ed57
  Harry Mellor authored Apr 24, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  0a05ed57
23 Apr, 2025 2 commits
- [Bugfix] Fix AssertionError: skip_special_tokens=False is not supported for... · 8c87a9ad
  Chauncey authored Apr 23, 2025
```
[Bugfix] Fix AssertionError: skip_special_tokens=False is not supported for Mistral tokenizers (#16964)
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
```
  8c87a9ad
- [Bugfix] validate urls object for multimodal content parts (#16990) · 36fe7876
  Guillaume Calmettes authored Apr 23, 2025
```
Signed-off-by: Guillaume Calmettes <gcalmettes@scaleway.com>
```
  36fe7876
22 Apr, 2025 1 commit

[frontend] enhance tool_calls type check (#16882) · f3441071

Reid authored Apr 22, 2025


Signed-off-by: reidliu41 <reid201711@gmail.com>
Co-authored-by: reidliu41 <reid201711@gmail.com>

f3441071

19 Apr, 2025 3 commits

[Model] Refactor Phi-4-multimodal to use merged processor and support V1 (#15477) · 83f3c3bd

Isotr0py authored Apr 19, 2025


Signed-off-by: Isotr0py <2037008807@qq.com>
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Co-authored-by: DarkLight1337 <tlleungac@connect.ust.hk>

83f3c3bd

[Frontend] Add sampling params to `v1/audio/transcriptions` endpoint (#16591) · 2ef0dc53

Nicolò Lucchesi authored Apr 19, 2025


Signed-off-by: Jannis Schönleber <joennlae@gmail.com>
Signed-off-by: NickLucche <nlucches@redhat.com>
Co-authored-by: Jannis Schönleber <joennlae@gmail.com>

2ef0dc53

[Model][VLM] Add Qwen2.5-Omni model support (thinker only) (#15130) · 2c1bd848

Yang Fan authored Apr 19, 2025


Signed-off-by: fyabc <suyang.fy@alibaba-inc.com>
Signed-off-by: Roger Wang <ywang@roblox.com>
Co-authored-by: Roger Wang <136131678+ywang96@users.noreply.github.com>
Co-authored-by: Roger Wang <ywang@roblox.com>
Co-authored-by: Xiong Wang <wangxiongts@163.com>

2c1bd848

18 Apr, 2025 1 commit
- [Misc] add collect_env to cli and docker image (#16759) · 7bdfd29a
  rongfu.leng authored Apr 18, 2025
```
Signed-off-by: rongfu.leng <rongfu.leng@daocloud.io>
```
  7bdfd29a
17 Apr, 2025 4 commits

[V1][Metrics] Fix http metrics middleware (#15894) · e4755f7f
Mark McLoughlin authored Apr 17, 2025

e4755f7f
[Bugfix]Fix index out of range error in api server log (#16787) · 6211b922
Robin authored Apr 18, 2025
```
Signed-off-by: WangErXiao <863579016@qq.com>
```
6211b922
[V1][Perf] Faster incremental detokenization (#15137) · 05fcd1b4
Nick Hill authored Apr 17, 2025
```
Signed-off-by: Nick Hill <nhill@redhat.com>
```
05fcd1b4

[V1][Frontend] Improve Shutdown And Logs (#11737) · 2b05b8ce

Robert Shaw authored Apr 16, 2025


Signed-off-by: rshaw@neuralmagic.com <rshaw@neuralmagic.com>
Signed-off-by: Andrew Feldman <afeldman@neuralmagic.com>
Signed-off-by: Nick Hill <nhill@redhat.com>
Co-authored-by: rshaw@neuralmagic.com <rshaw@neuralmagic.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
Co-authored-by: Russell Bryant <rbryant@redhat.com>
Co-authored-by: Andrew Feldman <afeldman@neuralmagic.com>
Co-authored-by: afeldman-nm <156691304+afeldman-nm@users.noreply.github.com>
Co-authored-by: Nick Hill <nhill@redhat.com>

2b05b8ce

15 Apr, 2025 4 commits
- [Misc] Enable vLLM to Dynamically Load LoRA from a Remote Server (#10546) · fdcb850f
  Angky William authored Apr 15, 2025
```
Signed-off-by: Angky William <angkywilliam@Angkys-MacBook-Pro.local>
Co-authored-by: Angky William <angkywilliam@Angkys-MacBook-Pro.local>
```
  fdcb850f
- Add "/server_info" endpoint in api_server to retrieve the vllm_config. (#16572) · 1666e664
  Xihui Cang authored Apr 15, 2025
```
Signed-off-by: Xihui Cang <xihuicang@gmail.com>
```
  1666e664
- Add `vllm bench [latency, throughput]` CLI commands (#16508) · b4fe16c7
  Michael Goin authored Apr 15, 2025
```
Signed-off-by: mgoin <mgoin64@gmail.com>
```
  b4fe16c7
- [Core][Bugfix] Fix Offline MM Beam Search (#16390) · 6b40996a
  Alex Brooks authored Apr 14, 2025
```
Signed-off-by: Alex-Brooks <Alex.Brooks@ibm.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
```
  6b40996a
14 Apr, 2025 1 commit
- [Model][VLM] Add Kimi-VL model support (#16387) · b1308b84
  courage17340 authored Apr 15, 2025
```
Signed-off-by: courage17340 <courage17340@163.com>
```
  b1308b84