Commits · f34410715f9758a2ee2d45acd56179f9c1c06fc3 · OpenDAS / vllm_cscc · GitLab

22 Apr, 2025 29 commits
- [frontend] enhance tool_calls type check (#16882) · f3441071
  Reid authored Apr 22, 2025
```
Signed-off-by: reidliu41 <reid201711@gmail.com>
Co-authored-by: reidliu41 <reid201711@gmail.com>
```
  f3441071
- [Misc] Add S3 environment variables for better support of MinIO. (#16977) · 68d4c332
  Chauncey authored Apr 22, 2025
```
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
```
  68d4c332
- [BugFix] Pass in correct VLLM config in FlashInfer backend (#13207) (#16973) · f961d7f6
  Zhengyuan Su (苏政渊) authored Apr 22, 2025
```
Signed-off-by: 苏政渊 <suzhengyuan@moonshot.cn>
Co-authored-by: 苏政渊 <suzhengyuan@moonshot.cn>
```
  f961d7f6
- Improve configs - `SpeculativeConfig` (#16971) · d0591104
  Harry Mellor authored Apr 22, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  d0591104
- [Bugfix] Fix distributed bug again in Qwen2.5-VL & Qwen2.5-Omni (#16974) · 571e8dd6
  Yang Fan authored Apr 22, 2025
```
Signed-off-by: fyabc <suyang.fy@alibaba-inc.com>
```
  571e8dd6
- [Misc] refactor example series (#16972) · 4b91c927
  Reid authored Apr 22, 2025
```
Signed-off-by: reidliu41 <reid201711@gmail.com>
Co-authored-by: reidliu41 <reid201711@gmail.com>
```
  4b91c927
- [FEAT][ROCm] Integrate Paged Attention Kernel from AITER (#15001) · 0e237f00
  vllmellm authored Apr 22, 2025
```
Signed-off-by: vllmellm <vllm.ellm@embeddedllm.com>
Signed-off-by: tjtanaa <tunjian.tan@embeddedllm.com>
Co-authored-by: tjtanaa <tunjian.tan@embeddedllm.com>
```
  0e237f00
- [Doc] Improve documentation for multimodal CLI args (#16960) · 8f7bace7
  Cyrus Leung authored Apr 22, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  8f7bace7
- [BugFix] Fix incremental detokenization perf issue (#16963) · e4d61442
  Nick Hill authored Apr 22, 2025
```
Signed-off-by: Nick Hill <nhill@redhat.com>
```
  e4d61442
- [Kernel] Support Microsoft Runtime Kernel Lib for our Low Precision Computation - BitBLAS (#6036) · 8d32dc60
  Lei Wang authored Apr 22, 2025
```
Signed-off-by: xinyuxiao <xinyuxiao2024@gmail.com>
Co-authored-by: xinyuxiao <xinyuxiao2024@gmail.com>
```
  8d32dc60
- [V1] Remove pre-allocation for KV cache (#16941) · c4ab9f3e
  Woosuk Kwon authored Apr 22, 2025
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  c4ab9f3e
- [Model] Use autoweightloader for mamba (#16950) · 2689d5c0
  Flora Feng authored Apr 22, 2025
```
Signed-off-by: sfeng33 <4florafeng@gmail.com>
```
  2689d5c0
- [Bugfix] Fix the issue where llm.generate cannot be called repeatedly after... · acba33a0
  Chauncey authored Apr 22, 2025
```
[Bugfix] Fix the issue where llm.generate cannot be called repeatedly after setting GuidedDecodingParams (#16767)
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
Signed-off-by: Russell Bryant <rbryant@redhat.com>
Co-authored-by: Russell Bryant <rbryant@redhat.com>
```
  acba33a0
- [Perf] Optimize `_update_states` for GPU model runner (#16910) · a114bf20
  SnowCharm authored Apr 22, 2025
```
Signed-off-by: snowcharm <snowcharmqq@gmail.com>
```
  a114bf20
- [Doc] Update ai_accelerator/hpu-gaudi.inc.md (#16956) · 3097ce3a
  Michael Yao authored Apr 22, 2025
```
Signed-off-by: windsonsea <haifeng.yao@daocloud.io>
```
  3097ce3a
- [Bugfix] Fix f-string for Python 3.9-3.11 (#16962) · d6da9322
  Cyrus Leung authored Apr 22, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  d6da9322
- Support S3 Sharded loading with RunAI Model Streamer (#16317) · 71ce4404
  omer-dayan authored Apr 22, 2025
```
Signed-off-by: Omer Dayan (SW-GPU) <omer@run.ai>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
```
  71ce4404
- [Performance][ROCm] Add skinny gemms for unquantized linear on ROCm (#15830) · 188b7f9b
  Charlie Fu authored Apr 21, 2025
```
Signed-off-by: charlifu <charlifu@amd.com>
Co-authored-by: Tyler Michael Smith <tysmith@redhat.com>
```
  188b7f9b
- [V1] Remove additional_config check (#16710) · b9b47469
  wangxiyuan authored Apr 22, 2025
```
Signed-off-by: wangxiyuan <wangxiyuan1007@gmail.com>
```
  b9b47469
- [Kernel] Add expert_map support to Cutlass FP8 MOE (#16861) · 7b8a2ab7
  Varun Sundar Rabindranath authored Apr 21, 2025
```
Signed-off-by: varun sundar rabindranath <vsundarr@redhat.com>
Co-authored-by: varun sundar rabindranath <vsundarr@redhat.com>
```
  7b8a2ab7
- [Misc] Remove the chunked prefill warning for LoRA (#16925) · c9acbf11
  Jee Jee Li authored Apr 22, 2025
```
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
```
  c9acbf11
- [ROCm] Add aiter tkw1 kernel for Llama4 fp8 (#16727) · 5b794cae
  kliuae authored Apr 22, 2025
```
Signed-off-by: kliuae <kuanfu.liu@embeddedllm.com>
Signed-off-by: tjtanaa <tunjian.tan@embeddedllm.com>
Co-authored-by: tjtanaa <tunjian.tan@embeddedllm.com>
Co-authored-by: vllmellm <vllm.ellm@embeddedllm.com>
```
  5b794cae
- [Bugfix]: fix issue with n>1 sampling on v1 requests overriding each other (#16863) · 0e425449
  Jeffrey Li authored Apr 21, 2025
```
Signed-off-by: Jeffrey Li <jeffrey.dot.li@gmail.com>
```
  0e425449
- [BugFix][Spec Decode] No in-place update to draft probs (#16952) · 1311913f
  Woosuk Kwon authored Apr 21, 2025
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  1311913f
- [Doc] Remove unnecessary V1 flag (#16924) · 29f395c9
  Cyrus Leung authored Apr 22, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  29f395c9
- [TPU][V1] Enable Top-P (#16843) · fa3bba2a
  Nicolò Lucchesi authored Apr 22, 2025
```
Signed-off-by: NickLucche <nlucches@redhat.com>
Co-authored-by: mgoin <mgoin64@gmail.com>
```
  fa3bba2a
- [V1] V1 FlashInfer Attention (#16684) · 986537f1
  Michael Goin authored Apr 21, 2025
```
Signed-off-by: mgoin <mgoin64@gmail.com>
Co-authored-by: Aurick Qiao <qiao@aurick.net>
```
  986537f1
- [TPU][V1] Capture multimodal encoder during model compilation (#15051) · 21020752
  Nicolò Lucchesi authored Apr 22, 2025
```
Signed-off-by: Michael Goin <mgoin64@gmail.com>
Signed-off-by: NickLucche <nlucches@redhat.com>
Co-authored-by: Michael Goin <mgoin64@gmail.com>
Co-authored-by: Siyuan Liu <lsiyuan@google.com>
```
  21020752
- Update Qwen1.5-MoE-W4A16-compressed-tensors.yaml (#16946) · 71eda0bb
  Michael Goin authored Apr 21, 2025
  
  71eda0bb
21 Apr, 2025 11 commits
- [TPU][V1] Implicitly adjust page size when there's SMEM OOM (#16871) · 471fe656
  Chengji Yao authored Apr 21, 2025
```
Signed-off-by: Chengji Yao <chengjiyao@google.com>
```
  471fe656
- [V1][Spec Decode] Handle draft tokens beyond max_model_len (#16087) · 3a0fba5c
  Woosuk Kwon authored Apr 21, 2025
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  3a0fba5c
- [Core] Speed up decode by remove synchronizing operation in sampler (#16436) · 299ebb62
  Chanh Nguyen authored Apr 21, 2025
```
Signed-off-by: Chanh Nguyen <cnguyen@linkedin.com>
Co-authored-by: Chanh Nguyen <cnguyen@linkedin.com>
```
  299ebb62
- [Doc] mention how to install in CPU editable mode (#16923) · f728ab8e
  David Xia authored Apr 21, 2025
```
Signed-off-by: David Xia <david@davidxia.com>
```
  f728ab8e
- [doc] install required python3-dev apt package (#16888) · 63e26fff
  David Xia authored Apr 21, 2025
```
Signed-off-by: David Xia <david@davidxia.com>
```
  63e26fff
- [XPU][Bugfix] minor fix for XPU (#15591) · fe3462c7
  Yan Ma authored Apr 22, 2025
```
Signed-off-by: yan ma <yan.ma@intel.com>
```
  fe3462c7
- Raise error for data-parallel with benchmark_throughput (#16737) · 3b34fd52
  Kartik Ramesh authored Apr 21, 2025
```
Signed-off-by: Kartik Ramesh <kartikx2000@gmail.com>
Co-authored-by: Simon Mo <simon.mo@hey.com>
```
  3b34fd52
- [Bugfix] Fix GLM rotary_dim issue and support v1 (#16912) · 55d6d3fd
  Isotr0py authored Apr 21, 2025
```
Signed-off-by: isotr0py <2037008807@qq.com>
```
  55d6d3fd
- [Misc] Refactor platform to get device specific stream and event (#14411) · 7272bfae
  Shanshan Shen authored Apr 21, 2025
```
Signed-off-by: shen-shanshan <467638484@qq.com>
```
  7272bfae
- [Misc] fix collect_env version parse (#15267) · d9ac9e3d
  wangxiyuan authored Apr 21, 2025
```
Signed-off-by: wangxiyuan <wangxiyuan1007@gmail.com>
```
  d9ac9e3d
- Restore buffers when wake up from level 2 sleep (#16564) (#16889) · d41faaf9
  Han Zhang authored Apr 21, 2025
```
Signed-off-by: Han <zh950713@gmail.com>
```
  d41faaf9