Commits · c91fe7b1b9c4398c6d4c980fc480ada0da8a0b23 · OpenDAS / vllm_cscc

22 May, 2025 4 commits
- [Frontend][Bug Fix] Update llama4 pythonic jinja template and llama4_pythonic parser (#17917) · c91fe7b1
  Kai Wu authored May 22, 2025
```
Signed-off-by: Kai Wu <kaiwu@meta.com>
```
  c91fe7b1
- [Misc] improve Automatic Prefix Caching example (#18554) · cb506ecb
  Reid authored May 22, 2025
```
Signed-off-by: reidliu41 <reid201711@gmail.com>
Co-authored-by: reidliu41 <reid201711@gmail.com>
```
  cb506ecb
- [Doc] Add stream flag for chat completion example (#18524) · 3f505233
  Calvin Chen authored May 22, 2025
```
Signed-off-by: calvin chen <120380290@qq.com>
```
  3f505233
- [Doc] Support --stream arg in openai_completion_client.py script (#18388) · 71075029
  CYJiang authored May 22, 2025
```
Signed-off-by: googs1025 <googs1025@gmail.com>
```
  71075029
21 May, 2025 1 commit

[Misc] refactor disaggregated-prefill-v1 example (#18474) · 107f5fc4

Reid authored May 21, 2025


Signed-off-by: reidliu41 <reid201711@gmail.com>
Co-authored-by: reidliu41 <reid201711@gmail.com>

107f5fc4

20 May, 2025 1 commit

[Misc] refactor prompt embedding examples (#18405) · 8f55962a

Reid authored May 20, 2025


Signed-off-by: reidliu41 <reid201711@gmail.com>
Co-authored-by: reidliu41 <reid201711@gmail.com>

8f55962a

19 May, 2025 2 commits
- [Misc] Fix typo (#18330) · 81712218
  Gong Shufan authored May 20, 2025
  
  81712218
- [Misc] extract parser.parse_args() (#18323) · 27d09526
  Reid authored May 19, 2025
```
Signed-off-by: reidliu41 <reid201711@gmail.com>
Co-authored-by: reidliu41 <reid201711@gmail.com>
```
  27d09526
16 May, 2025 2 commits
- [doc] fix multimodal example script (#18089) · 5c04bb8b
  David Xia authored May 16, 2025
```
Signed-off-by: David Xia <david@davidxia.com>
```
  5c04bb8b
- [Feature] Support Pipeline Parallism in torchrun SPMD offline inference for V1 (#17827) · 3d2779c2
  Lucia Fang authored May 15, 2025
```
Signed-off-by: Lucia Fang <fanglu@fb.com>
```
  3d2779c2
15 May, 2025 2 commits
- Improve examples rendering in docs and GitHub (#18203) · 51ff1546
  Harry Mellor authored May 15, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  51ff1546
- fix: typos (#18151) · a9944aab
  omahs authored May 15, 2025
```
Signed-off-by: omahs <73983677+omahs@users.noreply.github.com>
```
  a9944aab
14 May, 2025 4 commits
- Modularize fused experts and integrate PPLX kernels (#15956) · f9c069c8
  bnellnm authored May 14, 2025
  
  f9c069c8
- [V1][Spec Decode] Share input embedding of target model with EAGLE draft model... · 418d2f8b
  Ekagra Ranjan authored May 14, 2025
```
[V1][Spec Decode] Share input embedding of target model with EAGLE draft model to free ~1GB for llama 3 model (#17326)
Co-authored-by: root <root@ekagra-8xh100.us-east5-a.c.serving-efficiency-poc.internal>
Co-authored-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  418d2f8b
- [Bugfix][Example] make lmcache v0 work. (#18051) · e7ef61c1
  majianpeng authored May 14, 2025
```
Signed-off-by: Ma, Jianpeng <jianpeng.ma@intel.com>
```
  e7ef61c1
- Fix broken example: examples/offline_inference/profiling at scheduler_config (#18117) · 33011318
  Ecthlion_zyy authored May 14, 2025
  
  33011318
13 May, 2025 1 commit
- Implements dual-chunk-flash-attn backend for dual chunk attention with sparse... · 60f76243
  Tao He authored May 13, 2025
```
Implements dual-chunk-flash-attn backend for dual chunk attention with sparse attention support (#11844)
```
  60f76243
12 May, 2025 3 commits
- Construct `KVTransferConfig` properly from Python instead of using JSON blobs without CLI (#17994) · 72a3f6b8
  Harry Mellor authored May 12, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  72a3f6b8
- [Feature] Support DeepSeekV3 Function Call (#17784) · 3a5ea751
  Xu Wenqing authored May 12, 2025
```
Signed-off-by: 许文卿 <xwq391974@alibaba-inc.com>
Signed-off-by: Xu Wenqing <xuwq1993@qq.com>
```
  3a5ea751
- [Model] Broadcast Ovis2 implementation to fit Ovis1.6 (#17861) · 021c16c7
  Isotr0py authored May 12, 2025
```
Signed-off-by: Isotr0py <2037008807@qq.com>
```
  021c16c7
11 May, 2025 1 commit
- [Frontend] Add /classify endpoint (#17032) · 9cea90ea
  Frieda Huang authored May 11, 2025
```
Signed-off-by: Frieda (Jingying) Huang <jingyingfhuang@gmail.com>
```
  9cea90ea
10 May, 2025 1 commit

[Misc] remove --model from vllm serve usage (#17944) · 4c31218f

Reid authored May 10, 2025


Signed-off-by: reidliu41 <reid201711@gmail.com>
Co-authored-by: reidliu41 <reid201711@gmail.com>

4c31218f

09 May, 2025 2 commits
- [V1][Spec Decoding] Include bonus tokens in mean acceptance length (#17908) · 7e357113
  Mark McLoughlin authored May 09, 2025
```
Signed-off-by: Mark McLoughlin <markmc@redhat.com>
```
  7e357113
- [Misc] Add references in ray_serve_deepseek example (#17907) · c44c384b
  Rui Qiao authored May 09, 2025
```
Signed-off-by: Rui Qiao <ruisearch42@gmail.com>
```
  c44c384b
08 May, 2025 4 commits
- [Doc] Fix a typo in the file name (#17836) · a1e19b63
  Cyrus Leung authored May 08, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  a1e19b63
- [Bugfix] Fix tool call template validation for Mistral models (#17644) · ca04b97c
  Rick Yuan authored May 08, 2025
```
Signed-off-by: Rick Yuan <yuan821120@gmail.com>
Signed-off-by: RIck Yuan <yuan821120@gmail.com>
Co-authored-by: Aaron Pham <Aaronpham0103@gmail.com>
```
  ca04b97c
- [Frontend] Chat template fallbacks for multimodal models (#17805) · 96722aa8
  Cyrus Leung authored May 08, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  96722aa8
- [Chore][Doc] uses model id determined from OpenAI client (#17815) · a8238bbd
  Aaron Pham authored May 07, 2025
```
Signed-off-by: Aaron Pham <contact@aarnphm.xyz>
```
  a8238bbd
07 May, 2025 4 commits

Fix and simplify `deprecated=True` CLI `kwarg` (#17781) · 646a31e5
Harry Mellor authored May 07, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
646a31e5
[Frontend] Add missing chat templates for various MLLMs (#17758) · 8a15c260
Cyrus Leung authored May 07, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
8a15c260

Add NeuronxDistributedInference support, Speculative Decoding, Dynamic on-device sampling (#16357) · 043e4c49

Satyajith Chilappagari authored May 07, 2025


Signed-off-by: Satyajith Chilappagari <satchill@amazon.com>
Co-authored-by: Aaron Dou <yzdou@amazon.com>
Co-authored-by: Shashwat Srijan <sssrijan@amazon.com>
Co-authored-by: Chongming Ni <chongmni@amazon.com>
Co-authored-by: Amulya Ballakur <amulyaab@amazon.com>
Co-authored-by: Patrick Lange <patlange@amazon.com>
Co-authored-by: Elaine Zhao <elaineyz@amazon.com>
Co-authored-by: Lin Lin Pan <tailinpa@amazon.com>
Co-authored-by: Navyadhara Gogineni <navyadha@amazon.com>
Co-authored-by: Yishan McNabb <yishanm@amazon.com>
Co-authored-by: Mrinal Shukla <181322398+mrinalks@users.noreply.github.com>

043e4c49

[Misc] Remove qlora_adapter_name_or_path (#17699) · ba7703e6
Jee Jee Li authored May 07, 2025
```
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
```
ba7703e6

06 May, 2025 3 commits
- [TPU] Increase block size and reset block shapes (#16458) · 621ca2c0
  Jevin Jiang authored May 06, 2025
  
  621ca2c0
- [Bugfix] Fix modality limits in vision language example (#17721) · 5b8c3907
  Cyrus Leung authored May 07, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  5b8c3907
- [doc] Add RAG Integration example (#17692) · 7525d5f3
  Reid authored May 07, 2025
```
Signed-off-by: reidliu41 <reid201711@gmail.com>
Co-authored-by: reidliu41 <reid201711@gmail.com>
```
  7525d5f3
04 May, 2025 1 commit
- Add full API docs and improve the UX of navigating them (#17485) · d6484ef3
  Harry Mellor authored May 04, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  d6484ef3
02 May, 2025 2 commits
- [Misc] Rename assets for testing (#17575) · d7543862
  Cyrus Leung authored May 02, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  d7543862
- [Misc] Continue refactoring model tests (#17573) · f89d0e11
  Cyrus Leung authored May 02, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  f89d0e11
01 May, 2025 2 commits
- [Model] Refactor Ovis2 to support original tokenizer (#17537) · 88c83041
  Isotr0py authored May 02, 2025
```
Signed-off-by: Isotr0py <2037008807@qq.com>
```
  88c83041
- [Misc] refactor example - cpu_offload_lmcache (#17460) · 7423cf0a
  Reid authored May 01, 2025
```
Signed-off-by: reidliu41 <reid201711@gmail.com>
Co-authored-by: reidliu41 <reid201711@gmail.com>
```
  7423cf0a