Commits · 646a31e51ecf0da02d136512a7e37283c4cee34e · OpenDAS / vllm_cscc

07 May, 2025 4 commits

Fix and simplify `deprecated=True` CLI `kwarg` (#17781) · 646a31e5
Harry Mellor authored May 07, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
646a31e5
[Frontend] Add missing chat templates for various MLLMs (#17758) · 8a15c260
Cyrus Leung authored May 07, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
8a15c260

Add NeuronxDistributedInference support, Speculative Decoding, Dynamic on-device sampling (#16357) · 043e4c49

Satyajith Chilappagari authored May 07, 2025


Signed-off-by: Satyajith Chilappagari <satchill@amazon.com>
Co-authored-by: Aaron Dou <yzdou@amazon.com>
Co-authored-by: Shashwat Srijan <sssrijan@amazon.com>
Co-authored-by: Chongming Ni <chongmni@amazon.com>
Co-authored-by: Amulya Ballakur <amulyaab@amazon.com>
Co-authored-by: Patrick Lange <patlange@amazon.com>
Co-authored-by: Elaine Zhao <elaineyz@amazon.com>
Co-authored-by: Lin Lin Pan <tailinpa@amazon.com>
Co-authored-by: Navyadhara Gogineni <navyadha@amazon.com>
Co-authored-by: Yishan McNabb <yishanm@amazon.com>
Co-authored-by: Mrinal Shukla <181322398+mrinalks@users.noreply.github.com>

043e4c49

[Misc] Remove qlora_adapter_name_or_path (#17699) · ba7703e6
Jee Jee Li authored May 07, 2025
```
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
```
ba7703e6

06 May, 2025 3 commits
- [TPU] Increase block size and reset block shapes (#16458) · 621ca2c0
  Jevin Jiang authored May 06, 2025
  
  621ca2c0
- [Bugfix] Fix modality limits in vision language example (#17721) · 5b8c3907
  Cyrus Leung authored May 07, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  5b8c3907
- [doc] Add RAG Integration example (#17692) · 7525d5f3
  Reid authored May 07, 2025
```
Signed-off-by: reidliu41 <reid201711@gmail.com>
Co-authored-by: reidliu41 <reid201711@gmail.com>
```
  7525d5f3
04 May, 2025 1 commit
- Add full API docs and improve the UX of navigating them (#17485) · d6484ef3
  Harry Mellor authored May 04, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  d6484ef3
02 May, 2025 2 commits
- [Misc] Rename assets for testing (#17575) · d7543862
  Cyrus Leung authored May 02, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  d7543862
- [Misc] Continue refactoring model tests (#17573) · f89d0e11
  Cyrus Leung authored May 02, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  f89d0e11
01 May, 2025 4 commits
- [Model] Refactor Ovis2 to support original tokenizer (#17537) · 88c83041
  Isotr0py authored May 02, 2025
```
Signed-off-by: Isotr0py <2037008807@qq.com>
```
  88c83041
- [Misc] refactor example - cpu_offload_lmcache (#17460) · 7423cf0a
  Reid authored May 01, 2025
```
Signed-off-by: reidliu41 <reid201711@gmail.com>
Co-authored-by: reidliu41 <reid201711@gmail.com>
```
  7423cf0a
- [Feature][Frontend]: Deprecate --enable-reasoning (#17452) · 98060b00
  Chauncey authored May 01, 2025
```
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
```
  98060b00
- [doc] add streamlit integration (#17522) · 7169f87a
  Reid authored May 01, 2025
```
Signed-off-by: reidliu41 <reid201711@gmail.com>
Co-authored-by: reidliu41 <reid201711@gmail.com>
```
  7169f87a
30 Apr, 2025 4 commits

[BugFix] Fix authorization of openai_transcription_client.py (#17321) · d586ddc6
zh Wang authored May 01, 2025
```
Signed-off-by: zh Wang <rekind133@outlook.com>
```
d586ddc6

[V1][Metrics] add support for kv event publishing (#16750) · 0be6d05b

Alec authored Apr 30, 2025


Signed-off-by: alec-flowers <aflowers@nvidia.com>
Signed-off-by: Mark McLoughlin <markmc@redhat.com>
Co-authored-by: Mark McLoughlin <markmc@redhat.com>

0be6d05b

[MODEL ADDITION] Ovis2 Model Addition (#15826) · 54072f31

Marco authored Apr 30, 2025


Signed-off-by: Marco <121761685+mlinmg@users.noreply.github.com>
Signed-off-by: Isotr0py <2037008807@qq.com>
Signed-off-by: isotr0py <2037008807@qq.com>
Co-authored-by: Isotr0py <2037008807@qq.com>
Co-authored-by: Isotr0py <mozf@mail2.sysu.edu.cn>

54072f31

Update PyTorch to 2.7.0 (#16859) · 2c4f59af
Huy Do authored Apr 29, 2025

2c4f59af

29 Apr, 2025 3 commits
- [V1][Spec Decode] Apply torch.compile & cudagraph to EAGLE (#17211) · 70788bdb
  Bryan Lu authored Apr 29, 2025
```
Signed-off-by: Bryan Lu <yuzhelu@amazon.com>
```
  70788bdb
- Simplify (and fix) passing of guided decoding backend options (#17008) · a6977dbd
  Harry Mellor authored Apr 29, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  a6977dbd
- [Misc] Add a Jinja template to support Mistral3 function calling (#17195) · 96e06e3c
  Chauncey authored Apr 29, 2025
```
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
```
  96e06e3c
28 Apr, 2025 2 commits
- [Model] Add Granite Speech Support (#16246) · fa93cd9f
  Alex Brooks authored Apr 28, 2025
```
Signed-off-by: Alex-Brooks <Alex.brooks@ibm.com>
Signed-off-by: Alex-Brooks <Alex.Brooks@ibm.com>
```
  fa93cd9f
- [Doc] Fix wrong github link in LMCache examples (#17274) · 9053d0b1
  Kuntai Du authored Apr 27, 2025
```
Signed-off-by: KuntaiDu <kuntai@uchicago.edu>
```
  9053d0b1
26 Apr, 2025 3 commits
- [V1] Add `structural_tag` support using xgrammar (#17085) · f8acd01f
  Russell Bryant authored Apr 26, 2025
  
  f8acd01f
- [Bugfix] Fix missing int type for `-n` in multi-image example (#17223) · 8c1c926d
  Isotr0py authored Apr 26, 2025
  
  8c1c926d
- [v1] [P/D] Adding LMCache KV connector for v1 (#16625) · 5e83a727
  Yihua Cheng authored Apr 25, 2025
  
  5e83a727
25 Apr, 2025 2 commits

[Misc] Refine ray_serve_deepseek example (#17204) · c53e0730
Rui Qiao authored Apr 25, 2025
```
Signed-off-by: Rui Qiao <ruisearch42@gmail.com>
```
c53e0730

[V1][Spec Decode] EAGLE-3 Support (#16937) · a0e619e6

Benjamin Chislett authored Apr 25, 2025


Signed-off-by: Bryan Lu <yuzhelu@amazon.com>
Signed-off-by: Benjamin Chislett <benjamin.chislett@centml.ai>
Co-authored-by: Bryan Lu <yuzhelu@amazon.com>

a0e619e6

24 Apr, 2025 5 commits
- [Misc] Add example to run DeepSeek with Ray Serve LLM (#17134) · 583e9009
  Rui Qiao authored Apr 24, 2025
```
Signed-off-by: Rui Qiao <ruisearch42@gmail.com>
```
  583e9009
- Add chat template for Llama 4 models (#16428) · 05e1fbfc
  Maximilien de Bayser authored Apr 24, 2025
```
Signed-off-by: Max de Bayser <mbayser@br.ibm.com>
```
  05e1fbfc
- [Misc] refactor example series - structured outputs (#17040) · 1bcbcbf5
  Reid authored Apr 24, 2025
```
Signed-off-by: reidliu41 <reid201711@gmail.com>
Co-authored-by: reidliu41 <reid201711@gmail.com>
```
  1bcbcbf5
- [Frontend] Using matryoshka_dimensions control the allowed output dimensions. (#16970) · 67309a1c
  wang.yuqi authored Apr 24, 2025
  
  67309a1c
- [V1] Update structured output (#16812) · db2f8d91
  Reid authored Apr 24, 2025
```
Signed-off-by: reidliu41 <reid201711@gmail.com>
Co-authored-by: reidliu41 <reid201711@gmail.com>
```
  db2f8d91
22 Apr, 2025 1 commit

[Misc] refactor example series (#16972) · 4b91c927

Reid authored Apr 22, 2025


Signed-off-by: reidliu41 <reid201711@gmail.com>
Co-authored-by: reidliu41 <reid201711@gmail.com>

4b91c927

19 Apr, 2025 4 commits

[VLM] Clean up models (#16873) · 205d84aa
Cyrus Leung authored Apr 19, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
205d84aa

[Model] Refactor Phi-4-multimodal to use merged processor and support V1 (#15477) · 83f3c3bd

Isotr0py authored Apr 19, 2025


Signed-off-by: Isotr0py <2037008807@qq.com>
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Co-authored-by: DarkLight1337 <tlleungac@connect.ust.hk>

83f3c3bd

[Frontend] Add sampling params to `v1/audio/transcriptions` endpoint (#16591) · 2ef0dc53

Nicolò Lucchesi authored Apr 19, 2025


Signed-off-by: Jannis Schönleber <joennlae@gmail.com>
Signed-off-by: NickLucche <nlucches@redhat.com>
Co-authored-by: Jannis Schönleber <joennlae@gmail.com>

2ef0dc53

[Model][VLM] Add Qwen2.5-Omni model support (thinker only) (#15130) · 2c1bd848

Yang Fan authored Apr 19, 2025


Signed-off-by: fyabc <suyang.fy@alibaba-inc.com>
Signed-off-by: Roger Wang <ywang@roblox.com>
Co-authored-by: Roger Wang <136131678+ywang96@users.noreply.github.com>
Co-authored-by: Roger Wang <ywang@roblox.com>
Co-authored-by: Xiong Wang <wangxiongts@163.com>

2c1bd848

18 Apr, 2025 2 commits
- [Misc] refactor examples series - Chat Completion Client With Tools (#16829) · 5a5e29de
  Reid authored Apr 19, 2025
```
Signed-off-by: reidliu41 <reid201711@gmail.com>
Co-authored-by: reidliu41 <reid201711@gmail.com>
```
  5a5e29de
- [Misc] Clean up Kimi-VL (#16833) · aadb6565
  Cyrus Leung authored Apr 18, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  aadb6565