Commits · 32aa74c09c82c54a4a6c3fbaaf9892a9fe6ce4cc · OpenDAS / vllm_cscc · GitLab

07 May, 2025 19 commits
- [ROCm][FP8][Kernel] FP8 quantization fused into Custom Paged Attention (#17139) · 32aa74c0
  Gregory Shtrasberg authored May 07, 2025
```
Signed-off-by: Gregory Shtrasberg <Gregory.Shtrasberg@amd.com>
```
  32aa74c0
- [doc] update the issue link (#17782) · 7377dd03
  Reid authored May 07, 2025
```
Signed-off-by: reidliu41 <reid201711@gmail.com>
Co-authored-by: reidliu41 <reid201711@gmail.com>
```
  7377dd03
- Make key optional for rotary embedding (#17566) · 98c89e16
  Yong Hoon Shin authored May 07, 2025
```
Signed-off-by: Yong Hoon Shin <yhshin@meta.com>
```
  98c89e16
- Fix test_memory_usage_no_spec (#17754) · 324a3119
  Yong Hoon Shin authored May 07, 2025
```
Signed-off-by: Yong Hoon Shin <yhshin@meta.com>
```
  324a3119
- [Frontend] Add missing chat templates for various MLLMs (#17758) · 8a15c260
  Cyrus Leung authored May 07, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  8a15c260
- Add NeuronxDistributedInference support, Speculative Decoding, Dynamic on-device sampling (#16357) · 043e4c49
  Satyajith Chilappagari authored May 07, 2025
```
Signed-off-by: Satyajith Chilappagari <satchill@amazon.com>
Co-authored-by: Aaron Dou <yzdou@amazon.com>
Co-authored-by: Shashwat Srijan <sssrijan@amazon.com>
Co-authored-by: Chongming Ni <chongmni@amazon.com>
Co-authored-by: Amulya Ballakur <amulyaab@amazon.com>
Co-authored-by: Patrick Lange <patlange@amazon.com>
Co-authored-by: Elaine Zhao <elaineyz@amazon.com>
Co-authored-by: Lin Lin Pan <tailinpa@amazon.com>
Co-authored-by: Navyadhara Gogineni <navyadha@amazon.com>
Co-authored-by: Yishan McNabb <yishanm@amazon.com>
Co-authored-by: Mrinal Shukla <181322398+mrinalks@users.noreply.github.com>
```
  043e4c49
- [Misc] Remove qlora_adapter_name_or_path (#17699) · ba7703e6
  Jee Jee Li authored May 07, 2025
```
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
```
  ba7703e6
- [Kernel] Use fused rmsnorm for some models like qwen3 series (#17735) · f80ae5bd
  Wanrui Dai authored May 07, 2025
```
Signed-off-by: evian <eviantai@u.nus.edu>
Co-authored-by: evian <eviantai@u.nus.edu>
```
  f80ae5bd
- [Kernel] GGUF MoeVec kernel (#16780) · 1a45a613
  Szymon Ożóg authored May 07, 2025
```
Signed-off-by: SzymonOzog <szymon.ozog@aleph-alpha.com>
Signed-off-by: SzymonOzog <szymon.ozog@gmail.com>
Signed-off-by: Isotr0py <2037008807@qq.com>
Co-authored-by: Isotr0py <2037008807@qq.com>
```
  1a45a613
- [Misc] Use `apply_rotary_emb` from vllm_flash_attn for Qwen2-VL vision RoPE (#17726) · c3e9d506
  Isotr0py authored May 07, 2025
```
Signed-off-by: Isotr0py <2037008807@qq.com>
```
  c3e9d506
- [Misc] Split model loader (#17712) · 822de7fb
  Jee Jee Li authored May 07, 2025
```
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
```
  822de7fb
- [BugFix][Spec Decode] Fix hidden size mismatch between target and eagle head (#17740) · 8d84d836
  Woosuk Kwon authored May 06, 2025
```
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  8d84d836
- Replace lm-eval bash script with pytest and use enforce_eager for faster CI (#17717) · 950b7118
  Michael Goin authored May 06, 2025
```
Signed-off-by: mgoin <mgoin64@gmail.com>
```
  950b7118
- [TPU] Add kernel test for moe_pallas (#17496) · e50a1f1a
  Michael Goin authored May 06, 2025
```
Signed-off-by: Michael Goin <mgoin64@gmail.com>
```
  e50a1f1a
- Removed unused marlin cuda code (#17684) · a17cef70
  Michael Goin authored May 06, 2025
```
Signed-off-by: mgoin <mgoin64@gmail.com>
```
  a17cef70
- [Model] Mamba2 causal conv1d Refactor to Split Prefill and Decode Requests for... · 18dd5e01
  Chih-Chieh Yang authored May 06, 2025
```
[Model] Mamba2 causal conv1d Refactor to Split Prefill and Decode Requests for Corresponding Kernels (#17146)
Signed-off-by: Chih-Chieh-Yang <7364402+cyang49@users.noreply.github.com>
```
  18dd5e01
- Add logging for torch nightly version (#17669) · 6de3e134
  Yang Wang authored May 06, 2025
```
Signed-off-by: Yang Wang <elainewy@meta.com>
```
  6de3e134
- [ROCm] fix num_stages for default moe config to avoid triton OutOfResource error (#17744) · ed3a1d21
  Hongxia Yang authored May 06, 2025
```
Signed-off-by: Hongxia Yang <hongxia.yang@amd.com>
```
  ed3a1d21
- Fix doc build performance (#17748) · 022afbeb
  Harry Mellor authored May 07, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  022afbeb
06 May, 2025 21 commits
- [Kernel] Unified Triton kernel that doesn't distinguish between prefill + decode (#16828) · 2f925e57
  Thomas Parnell authored May 06, 2025
```
Signed-off-by: Thomas Parnell <tpa@zurich.ibm.com>
Signed-off-by: Lucas Wilkinson <lwilkinson@neuralmagic.com>
Co-authored-by: Lucas Wilkinson <lwilkinson@neuralmagic.com>
```
  2f925e57
- [Bugfix] Fix for the condition to accept empty encoder inputs for mllama (#17732) · de906b95
  Gregory Shtrasberg authored May 06, 2025
```
Signed-off-by: Gregory Shtrasberg <Gregory.Shtrasberg@amd.com>
```
  de906b95
- [Misc] Add Next Edit Prediction (NEP) datasets support in `benchmark_serving.py` (#16839) · d456aea7
  d.transposed authored May 06, 2025
```
Signed-off-by: dtransposed <damian@damian-ml-machine.europe-west3-b.c.jetbrains-grazie.internal>
Signed-off-by: dtransposed <>
Co-authored-by: dtransposed <damian@damian-ml-machine.europe-west3-b.c.jetbrains-grazie.internal>
```
  d456aea7
- [TPU] Increase block size and reset block shapes (#16458) · 621ca2c0
  Jevin Jiang authored May 06, 2025
  
  621ca2c0
- Make right sidebar more readable in "Supported Models" (#17723) · 6115b115
  Harry Mellor authored May 06, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  6115b115
- [Bugfix] Fix modality limits in vision language example (#17721) · 5b8c3907
  Cyrus Leung authored May 07, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  5b8c3907
- [doc] Add RAG Integration example (#17692) · 7525d5f3
  Reid authored May 07, 2025
```
Signed-off-by: reidliu41 <reid201711@gmail.com>
Co-authored-by: reidliu41 <reid201711@gmail.com>
```
  7525d5f3
- [v1] Introduce KVCacheBlocks as interface between Scheduler and KVCacheManager (#17479) · aabcd2ca
  Chen Zhang authored May 06, 2025
```
Signed-off-by: Chen Zhang <zhangch99@outlook.com>
```
  aabcd2ca
- [Docs] Use gh-file to add links to tool_calling.md (#17709) · 0d115460
  Michael Yao authored May 06, 2025
```
Signed-off-by: windsonsea <haifeng.yao@daocloud.io>
```
  0d115460
- [Feat] Add deprecated=True to CLI args (#17426) · 175bda67
  Aaron Pham authored May 06, 2025
```
Signed-off-by: Aaron Pham <contact@aarnphm.xyz>
```
  175bda67
- [v1] AttentionMetadata for each layer (#17394) · cba31c47
  Chen Zhang authored May 06, 2025
```
Signed-off-by: Chen Zhang <zhangch99@outlook.com>
```
  cba31c47
- [V1][PP] Support PP for MultiprocExecutor (#14219) · a6fed020
  Li, Jiang authored May 06, 2025
```
Signed-off-by: jiang1.li <jiang1.li@intel.com>
Signed-off-by: jiang.li <jiang1.li@intel.com>
```
  a6fed020
- [V1] Enable TPU V1 backend by default (#17673) · d419aa5d
  Michael Goin authored May 06, 2025
```
Signed-off-by: mgoin <mgoin64@gmail.com>
```
  d419aa5d
- [Bugfix] Fix triton import with local TritonPlaceholder (#17446) · f9bc5a06
  Mengqing Cao authored May 06, 2025
```
Signed-off-by: Mengqing Cao <cmq0113@163.com>
```
  f9bc5a06
- Fix `dockerfilegraph` pre-commit hook (#17698) · 05e1f964
  Harry Mellor authored May 06, 2025
```
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
```
  05e1f964
- [Misc] Fix ScalarType float4 naming (#17690) · 6eae3453
  Lucas Wilkinson authored May 06, 2025
```
Signed-off-by: Lucas Wilkinson <lwilkinson@neuralmagic.com>
```
  6eae3453
- [Doc] Update notes for H2O-VL and Gemma3 (#17219) · 63ced7b4
  Cyrus Leung authored May 06, 2025
```
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  63ced7b4
- [Bugfix] Fixed prompt length for random dataset (#17408) · dc47ba32
  Mikhail Podvitskii authored May 06, 2025
```
Signed-off-by: Mikhail Podvitskii <podvitskiymichael@gmail.com>
```
  dc47ba32
- [easy] Fix logspam on PiecewiseBackend errors (#17138) · edbf2d60
  Richard Zou authored May 06, 2025
```
Signed-off-by: rzou <zou3519@gmail.com>
```
  edbf2d60
- [Model] Add GraniteMoeHybrid 4.0 model (#17497) · 999328be
  Stan Wozniak authored May 06, 2025
```
Signed-off-by: Thomas Ortner <boh@zurich.ibm.com>
Signed-off-by: Stanislaw Wozniak <stw@zurich.ibm.com>
Co-authored-by: Thomas Ortner <boh@zurich.ibm.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
Co-authored-by: Tyler Michael Smith <tysmith@redhat.com>
```
  999328be
- Update nm to rht in doc links + refine fp8 doc (#17678) · 98834fef
  Michael Goin authored May 05, 2025
```
Signed-off-by: mgoin <mgoin64@gmail.com>
```
  98834fef