Commits · 48ac2bed5b6271f82187de61245a85d987197c6f · OpenDAS / vllm_cscc

"docs/vscode:/vscode.git/clone" did not exist on "23b67b37b246fb7cecb3815c6873fa3d18c3c0e7"

17 May, 2025 6 commits

[Hardware][TPU] Optionally import for TPU backend (#18269) · 48ac2bed

Siyuan Liu authored May 17, 2025


Signed-off-by: Siyuan Liu <lsiyuan@google.com>
Signed-off-by: Jade Zheng <zheng.shoujian@outlook.com>
Co-authored-by: Carol Zheng <cazheng@google.com>
Co-authored-by: Jade Zheng <zheng.shoujian@outlook.com>
Co-authored-by: Hongmin Fan <fanhongmin@google.com>

48ac2bed

[P/D][V1] Support dynamic loading of external KV connector implementations (#18142) · 3e0d4350
David Ben-David authored May 17, 2025
```
Signed-off-by: David Ben-David <davidb@pliops.com>
Co-authored-by: David Ben-David <davidb@pliops.com>
```
3e0d4350

[BugFix] Correct max_model_len derivation from config.json for Mistral format (#17937) · 4ee4826e

汪志鹏 authored May 16, 2025


Signed-off-by: 汪志鹏 <wangzhipeng628@gmail.com>
Co-authored-by: tracelogfb <48808670+tracelogfb@users.noreply.github.com>
Co-authored-by: Stephen Chen <tracelog@meta.com>

4ee4826e

[Misc] reformat the collect-env output (#18285) · 60017dc8

Reid authored May 17, 2025


Signed-off-by: reidliu41 <reid201711@gmail.com>
Co-authored-by: reidliu41 <reid201711@gmail.com>

60017dc8

Move cli args docs to its own page (#18228) (#18264) · 55f1a468
Trevor Royer authored May 16, 2025
```
Signed-off-by: Trevor Royer <troyer@redhat.com>
```
55f1a468
[V1][P/D] Local attention optimization for NIXL (#18170) · fd195b19
Michael Goin authored May 16, 2025
```
Signed-off-by: mgoin <mgoin64@gmail.com>
```
fd195b19

16 May, 2025 20 commits
- [Spec Decode] Don't fall back to V0 when spec decoding is enabled (#18265) · fabe89bb
  Woosuk Kwon authored May 16, 2025
  
  fabe89bb
- [Bugfix] fix `an illegal memory access was encountered` of marlin kernel + act_order (#18245) · e73b7dfd
  Jinzhen Lin authored May 17, 2025
  
  e73b7dfd
- [Sampler] Adapt to FlashInfer 0.2.3 sampler API (#15777) · 7fdfa015
  Bowen Wang authored May 16, 2025
```
Signed-off-by: Bowen Wang <abmfy@icloud.com>
Co-authored-by: mgoin <mgoin64@gmail.com>
```
  7fdfa015
- [CI] Assign reviewer to mergify with changes to Tensorizer files (#18278) · aef94c6d
  Sanger Steel authored May 16, 2025
  
  aef94c6d
- [BugFix] Fix ordering of KVConnector finished send/rcv sets (#18211) · 0ceaebf8
  Nick Hill authored May 16, 2025
```
Signed-off-by: Nick Hill <nhill@redhat.com>
```
  0ceaebf8
- [BugFix] Fix multi async save in MultiConnector (#18246) · 1db4f47f
  Nick Hill authored May 16, 2025
```
Signed-off-by: Nick Hill <nhill@redhat.com>
```
  1db4f47f
- [Misc][MacOS] fix bfloat16 error (#18249) · d3d91b6f
  Reid authored May 16, 2025
```
Signed-off-by: reidliu41 <reid201711@gmail.com>
Co-authored-by: reidliu41 <reid201711@gmail.com>
```
  d3d91b6f
- [Model] Use autoweightloader for dbrx (#18251) · 87d87147
  learner0810 authored May 16, 2025
```
Signed-off-by: learner0810 <zhongjun.li@daocloud.io>
```
  87d87147
- [Fix] Fix typo in `resolve_hf_chat_template` (#18259) · a5f8c111
  fxmarty-amd authored May 16, 2025
```
Signed-off-by: Felix Marty <felmarty@amd.com>
```
  a5f8c111
- use ceil_div in cutlass block scaling shape check (#17918) · e23564cb
  Lain authored May 16, 2025
  
  e23564cb
- [Misc] Consolidate Audio tests into multimodal common generation tests (#18214) · 390ec889
  Isotr0py authored May 16, 2025
```
Signed-off-by: Isotr0py <2037008807@qq.com>
```
  390ec889
- [Misc] Add Ray Prometheus logger to V1 (#17925) · 54181767
  Seiji Eicher authored May 16, 2025
```
Signed-off-by: Seiji Eicher <seiji@anyscale.com>
```
  54181767
- [PERF] Speed up Qwen2.5-VL model by speed up rotary position embedding (#17973) · 67da5720
  Vadim Gimpelson authored May 16, 2025
```
Signed-off-by: Vadim Gimpelson <vadim.gimpelson@centml.ai>
```
  67da5720
- [doc] fix multimodal example script (#18089) · 5c04bb8b
  David Xia authored May 16, 2025
```
Signed-off-by: David Xia <david@davidxia.com>
```
  5c04bb8b
- [Feature] Support Pipeline Parallism in torchrun SPMD offline inference for V1 (#17827) · 3d2779c2
  Lucia Fang authored May 15, 2025
```
Signed-off-by: Lucia Fang <fanglu@fb.com>
```
  3d2779c2
- Throw better error for when running into k8s service discovery issue (#18209) · 6b31c84a
  Will Eaton authored May 16, 2025
```
Signed-off-by: Will Eaton <weaton@redhat.com>
```
  6b31c84a
- Allow users to pass arbitrary JSON keys from CLI (#18208) · b18201fe
  Harry Mellor authored May 16, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  b18201fe
- [Model] vLLM v1 supports Medusa (#17956) · f4937a51
  Sky Lee authored May 16, 2025
```
Signed-off-by: lisiqi23 <lisiqi23@xiaomi.com>
Signed-off-by: skylee-01 <497627264@qq.com>
Co-authored-by: lisiqi23 <lisiqi23@xiaomi.com>
```
  f4937a51
- [Bugfix][ROCm] Use `chunked_prefill_paged_decode` as fallback for V1 attention on ROCm (#18093) · ee659e3b
  kliuae authored May 16, 2025
```
Signed-off-by: kf <kuanfu.liu@embeddedllm.com>
```
  ee659e3b
- [Bugfix] fix rotary embedding test for _get_padded_tensor_shape (#18229) · 4e1c6a02
  Lucas Wilkinson authored May 15, 2025
```
Signed-off-by: Lucas Wilkinson <lwilkinson@neuralmagic.com>
```
  4e1c6a02
15 May, 2025 14 commits
- [Build] Allow shipping PTX on a per-file basis (#18155) · c7852a6d
  Lucas Wilkinson authored May 15, 2025
```
Signed-off-by: Lucas Wilkinson <lwilkinson@neuralmagic.com>
```
  c7852a6d
- [Bugfix] Fix test_eagle test (#18223) · 8795eb99
  Lucia Fang authored May 15, 2025
```
Signed-off-by: Lucia Fang <fanglu@fb.com>
```
  8795eb99
- Adding "AMD: Tensorizer Test" to amdproduction. (#18216) · 0b345930
  Alexei-V-Ivanov-AMD authored May 15, 2025
  
  0b345930
- [Misc] Avoid cuda graph log when sizes still match (#18202) · e3f3aee6
  Nicolò Lucchesi authored May 15, 2025
```
Signed-off-by: NickLucche <nlucches@redhat.com>
```
  e3f3aee6
- [Bugfix] [ROCm]: Remove assertion logic when using AITER fused moe in... · 92540529
  TJian authored May 16, 2025
```
[Bugfix] [ROCm]: Remove assertion logic when using AITER fused moe in unquantizedMethod to reenable LLama4 BF16 (#18205)
Signed-off-by: tjtanaa <tunjian.tan@embeddedllm.com>
```
  92540529
- [Bugfix]Change the exception thrown by call_hf_processor from RuntimeError to ValueError (#18181) · fadb8d5c
  Zhonghua Deng authored May 16, 2025
```
Signed-off-by: Abatom <abzhonghua@gmail.com>
```
  fadb8d5c
- [Frontend] Fix chat template content format detection (#18190) · 2aa5470a
  Sebastian Schoennenbeck authored May 15, 2025
```
Signed-off-by: Sebastian Schönnenbeck <sebastian.schoennenbeck@comma-soft.com>
```
  2aa5470a
- Improve examples rendering in docs and GitHub (#18203) · 51ff1546
  Harry Mellor authored May 15, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  51ff1546
- Adding "Basic Models Test" and "Multi-Modal Models Test (Extended) 3" in AMD Pipeline (#18106) · 566ec04c
  Alexei-V-Ivanov-AMD authored May 15, 2025
```
Signed-off-by: Alexei V. Ivanov <alexei.ivanov@amd.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
```
  566ec04c
- [Kernel] [V1] Fix performance regression for triton unified attention (#18161) · 01c22335
  Thomas Parnell authored May 15, 2025
```
Signed-off-by: Thomas Parnell <tpa@zurich.ibm.com>
Co-authored-by: Lucas Wilkinson <lwilkinson@neuralmagic.com>
```
  01c22335
- add tools into TokenizeChatRequest (#18187) · 451da4bc
  hustxiayang authored May 15, 2025
```
Signed-off-by: yangxia <yangxiast@gmail.com>
```
  451da4bc
- Update deprecated type hinting in `model_loader` (#18130) · 07ad2712
  Harry Mellor authored May 15, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  07ad2712
- fix: typos (#18151) · a9944aab
  omahs authored May 15, 2025
```
Signed-off-by: omahs <73983677+omahs@users.noreply.github.com>
```
  a9944aab
- [V1] Update zmq socket creation in nixl connector (#18148) · a8f5aec2
  Russell Bryant authored May 15, 2025
```
Signed-off-by: Russell Bryant <rbryant@redhat.com>
```
  a8f5aec2