Commits · 2b05b8ce69fff90b7f1e3285dcd05ced2b79fd97 · OpenDAS / vllm_cscc

17 Apr, 2025 5 commits

[V1][Frontend] Improve Shutdown And Logs (#11737) · 2b05b8ce

Robert Shaw authored Apr 16, 2025


Signed-off-by: rshaw@neuralmagic.com <rshaw@neuralmagic.com>
Signed-off-by: Andrew Feldman <afeldman@neuralmagic.com>
Signed-off-by: Nick Hill <nhill@redhat.com>
Co-authored-by: rshaw@neuralmagic.com <rshaw@neuralmagic.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
Co-authored-by: Russell Bryant <rbryant@redhat.com>
Co-authored-by: Andrew Feldman <afeldman@neuralmagic.com>
Co-authored-by: afeldman-nm <156691304+afeldman-nm@users.noreply.github.com>
Co-authored-by: Nick Hill <nhill@redhat.com>

2b05b8ce

[V1][Spec Dec Bug Fix] Respect Spec Dec Method Specification (#16636) · 2cbd4d29
Bryan Lu authored Apr 16, 2025
```
Signed-off-by: Bryan Lu <yuzhelu@amazon.com>
```
2cbd4d29

[V1][Performance] Implement custom serializaton for MultiModalKwargs [Rebased] (#16432) · 3092375e

Staszek Paśko authored Apr 17, 2025


Signed-off-by: Staszek Pasko <staszek@gmail.com>
Signed-off-by: Nick Hill <nhill@redhat.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
Co-authored-by: Nick Hill <nhill@redhat.com>

3092375e

[Misc] Remove redundant comment (#16703) · 8a7368e0
Jade Zheng authored Apr 17, 2025
```
Signed-off-by: Jade Zheng <zheng.shoujian@outlook.com>
```
8a7368e0
Improve error for structured output backend selection (#16717) · 93e561ec
Harry Mellor authored Apr 17, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
93e561ec

16 Apr, 2025 7 commits
- [Hardware] Add processor inputs to platform validation (#16680) · e1b00483
  Joe Runde authored Apr 16, 2025
```
Signed-off-by: Joe Runde <Joseph.Runde@ibm.com>
```
  e1b00483
- [Model] support modernbert (#16648) · ee378f3d
  xsank authored Apr 16, 2025
```
Signed-off-by: 唯勤 <xsank.mz@alibaba-inc.com>
Co-authored-by: 唯勤 <xsank.mz@alibaba-inc.com>
```
  ee378f3d
- [V1][Structured Output] Move xgrammar related utils to `backend_xgrammar.py` (#16578) · 976711d9
  Shanshan Shen authored Apr 16, 2025
```
Signed-off-by: shen-shanshan <467638484@qq.com>
```
  976711d9
- [Feature] add model aware kv ops helper (#16020) · 3ac98edc
  billishyahao authored Apr 16, 2025
```
Signed-off-by: billishyahao <bill.he@amd.com>
```
  3ac98edc
- Disable remote caching when calling compile_fx (#16611) · 966c742e
  Richard Zou authored Apr 16, 2025
```
Signed-off-by: rzou <zou3519@gmail.com>
```
  966c742e
- [Misc] Modify LRUCache touch (#16689) · 0d7d05f4
  Jee Jee Li authored Apr 16, 2025
```
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
```
  0d7d05f4
- [Model] Add PLaMo2 (#14323) · 3badb021
  Shinichi Hemmi authored Apr 16, 2025
```
Signed-off-by: Shinichi Hemmi <50256998+Alnusjaponica@users.noreply.github.com>
Signed-off-by: shemmi <shemmi@preferred.jp>
Co-authored-by: Kento Nozawa <nzw0301@preferred.jp>
Co-authored-by: Hiroaki Mikami <mhiroaki@preferred.jp>
Co-authored-by: Calvin Metzger <metzger@preferred.jp>
```
  3badb021
15 Apr, 2025 9 commits
- [Misc] Enable vLLM to Dynamically Load LoRA from a Remote Server (#10546) · fdcb850f
  Angky William authored Apr 15, 2025
```
Signed-off-by: Angky William <angkywilliam@Angkys-MacBook-Pro.local>
Co-authored-by: Angky William <angkywilliam@Angkys-MacBook-Pro.local>
```
  fdcb850f
- [Misc] Update `compressed-tensors` WNA16 to support zero-points (#14211) · 54a66e5f
  Dipika Sikka authored Apr 15, 2025
  
  54a66e5f
- [Kernel] Remove redundant Exp calculations (#16123) · 280d62b8
  DefTruth authored Apr 15, 2025
```
Signed-off-by: DefTruth <qiustudent_r@163.com>
```
  280d62b8
- Add "/server_info" endpoint in api_server to retrieve the vllm_config. (#16572) · 1666e664
  Xihui Cang authored Apr 15, 2025
```
Signed-off-by: Xihui Cang <xihuicang@gmail.com>
```
  1666e664
- Fix vLLM x torch.compile config caching (#16491) · b590adfd
  Richard Zou authored Apr 15, 2025
```
Signed-off-by: rzou <zou3519@gmail.com>
```
  b590adfd
- Add `vllm bench [latency, throughput]` CLI commands (#16508) · b4fe16c7
  Michael Goin authored Apr 15, 2025
```
Signed-off-by: mgoin <mgoin64@gmail.com>
```
  b4fe16c7
- [Bugfix] Fix broken GritLM model and tests (missing pooling_metadata) (#16631) · bc5dd4f6
  Pooya Davoodi authored Apr 14, 2025
```
Signed-off-by: Pooya Davoodi <pooya.davoodi@parasail.io>
```
  bc5dd4f6
- [Kernel] moe wna16 marlin kernel (#14447) · d06ba4ed
  Jinzhen Lin authored Apr 15, 2025
```
Signed-off-by: Jinzhen Lin <linjinzhen@hotmail.com>
Co-authored-by: Michael Goin <michael@neuralmagic.com>
Co-authored-by: mgoin <mgoin64@gmail.com>
```
  d06ba4ed
- [Core][Bugfix] Fix Offline MM Beam Search (#16390) · 6b40996a
  Alex Brooks authored Apr 14, 2025
```
Signed-off-by: Alex-Brooks <Alex.Brooks@ibm.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
```
  6b40996a
14 Apr, 2025 9 commits
- config check sleep mode support oot platforms (#16562) · d2020aca
  Shuqiao Li authored Apr 15, 2025
  
  d2020aca
- [DOC][TPU] Add core idea about avoiding recompilation after warmup (#16614) · 1eb3c2ed
  Chengji Yao authored Apr 14, 2025
```
Signed-off-by: Chengji Yao <chengjiyao@google.com>
```
  1eb3c2ed
- [Model][VLM] Add Kimi-VL model support (#16387) · b1308b84
  courage17340 authored Apr 15, 2025
```
Signed-off-by: courage17340 <courage17340@163.com>
```
  b1308b84
- [TPU][V1] Fix exponential padding when `max-num-batched-tokens` is not a power of 2 (#16596) · b3f2fddd
  Nicolò Lucchesi authored Apr 14, 2025
```
Signed-off-by: NickLucche <nlucches@redhat.com>
```
  b3f2fddd
- [Bugfix] Multi-modal caches not acting like LRU caches (#16593) · aa29841e
  Cyrus Leung authored Apr 15, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  aa29841e
- [Misc] Update usage with mooncake lib for kv transfer (#16523) · 1dd23386
  shangmingc authored Apr 14, 2025
```
Signed-off-by: Shangming Cai <caishangming@linux.alibaba.com>
```
  1dd23386
- [Misc] remove warning if triton>=3.2.0 (#16553) · ce4ddd2d
  DefTruth authored Apr 14, 2025
```
Signed-off-by: DefTruth <qiustudent_r@163.com>
```
  ce4ddd2d
- Improve configs - `SchedulerConfig` (#16533) · e51929eb
  Harry Mellor authored Apr 14, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  e51929eb
- [Core][V0] Enable regex support with xgrammar (#13228) · dc1b4a6f
  Russell Bryant authored Apr 13, 2025
```
Signed-off-by: Russell Bryant <rbryant@redhat.com>
```
  dc1b4a6f
13 Apr, 2025 2 commits
- Enable PTPC FP8 for CompressedTensorsW8A8Fp8MoEMethod (triton fused_moe) (#16537) · d085a440
  Michael Goin authored Apr 13, 2025
```
Signed-off-by: mgoin <mgoin64@gmail.com>
```
  d085a440
- [V1][Spec Decode] KV cache slots for eagle heads (#16370) · f49e5aff
  Lily Liu authored Apr 12, 2025
```
Signed-off-by: LiuXiaoxuanPKU <lilyliupku@gmail.com>
```
  f49e5aff
12 Apr, 2025 8 commits
- [Bugfix] Validate logit biases to prevent out of vocab ids crashing engine (#16529) · 6c11ecf8
  Ryan McConville authored Apr 12, 2025
```
Signed-off-by: Ryan McConville <ryan@ryanmcconville.com>
```
  6c11ecf8
- [Perf] Optimize Preparing Inputs for GPU Model Runner (#16484) · 93e5f3c5
  SnowCharm authored Apr 12, 2025
```
Signed-off-by: snowcharm <snowcharmqq@gmail.com>
Co-authored-by: Nick Hill <nhill@redhat.com>
```
  93e5f3c5
- Fix syntaxWarning: invalid escape sequence '\s' (#16532) · 70363bcc
  Jie Fu (傅杰) authored Apr 12, 2025
```
Signed-off-by: Jie Fu <jiefu@tencent.com>
```
  70363bcc
- [MISC] Make GroupCoordinator compatible with out-of-tree devices (#16464) · 68bb122e
  Huazhong Ji authored Apr 12, 2025
```
Signed-off-by: hzji210@gmail.com <hzji210@gmail.com>
```
  68bb122e
- [V1] Enable multi-input by default (#15799) · d9fc8cd9
  Cyrus Leung authored Apr 12, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  d9fc8cd9
- [Frontend] support matryoshka representation / support embedding API dimensions (#16331) · fbf722c6
  wang.yuqi authored Apr 12, 2025
  
  fbf722c6
- [Feature][V1] Add xgrammar to support minLength, maxLength with test (#16516) · e92d7085
  leon-seidel authored Apr 12, 2025
```
Signed-off-by: Leon Seidel <leon.seidel@fau.de>
```
  e92d7085
- Optimized topk for topk=1 (Llama-4) (#16512) · bd6028d6
  Michael Goin authored Apr 12, 2025
```
Signed-off-by: mgoin <mgoin64@gmail.com>
```
  bd6028d6