GitLab

[Rocm][torch.compile] Adding layernorm + fp8 block quant and silu + fp8 block quant for Aiter (#25693)
Signed-off-by: charlifu <charlifu@amd.com>
Signed-off-by: Micah Williamson <micah.williamson@amd.com>
Signed-off-by: Charlie Fu <Charlie.Fu@amd.com>
Co-authored-by: Micah Williamson <micah.williamson@amd.com>
Co-authored-by: wuhuikx <hattie.wu@amd.com>
Co-authored-by: Luka Govedič <ProExpertProg@users.noreply.github.com>
Co-authored-by: Gregory Shtrasberg <156009573+gshtras@users.noreply.github.com>

Signed-off-by: Kyle Sayers <kylesayrs@gmail.com>

Signed-off-by: Bill Nell <bnell@redhat.com>
Signed-off-by: Tyler Michael Smith <tlrmchlsmth@gmail.com>
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com>
Co-authored-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com>
Co-authored-by: Tyler Michael Smith <tlrmchlsmth@gmail.com>

[CI/Build] Make test_mha_attn.py run on correct platform only and check for flash_attn_varlen_func in layer.py (#29145)

Signed-off-by: Tsukasa OI <floss_llm@irq.a4lg.com>

Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Signed-off-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com>
Co-authored-by: Tyler Michael Smith <tyler@neuralmagic.com>
Co-authored-by: Matthew Bonanni <mbonanni@redhat.com>

Signed-off-by: Benjamin Chislett <bchislett@nvidia.com>
Signed-off-by: Benjamin Chislett <chislett.ben@gmail.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>

Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>

Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>

Signed-off-by: ilmarkov <markovilya197@gmail.com>

[Compile] Fix torch warning `TensorFloat32 tensor cores for float32 matrix multiplication available but not enabled` (#29897)
Signed-off-by: yewentao256 <zhyanwentao@126.com>

Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>

Signed-off-by: quanliu <18646313696@163.com>
Co-authored-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com>

Signed-off-by: juliendenize <julien.denize@mistral.ai>

Signed-off-by: vllmellm <vllm.ellm@embeddedllm.com>

Signed-off-by: baonudesifeizhai <baonudesifeizhai@gmail.com>

Signed-off-by: Haoyang Li <lihaoyang0109@gmail.com>

[Structured Output][Reasoning] Improves decoding throughput for models using single-token reasoning endings. (#30056)

Signed-off-by: FENP <yuanyongjie.yyj@antgroup.com>

Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>
Signed-off-by: wang.yuqi <noooop@126.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>

Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Signed-off-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com>
Co-authored-by: Benjamin Chislett <chislett.ben@gmail.com>

Signed-off-by: Yongtao Huang <yongtaoh2022@gmail.com>

[Model][Quantization] Restore MoE + GGUF models support (incl. Qwen3 MoE) by allowing Sideload Parameters (#30116)
Signed-off-by: Tsukasa OI <floss_llm@irq.a4lg.com>
Co-authored-by: Isotr0py <mozf@mail2.sysu.edu.cn>

Signed-off-by: Angel Li <liangel@meta.com>

Signed-off-by: Or Ozeri <oro@il.ibm.com>

Signed-off-by: mgoin <mgoin64@gmail.com>
Signed-off-by: Michael Goin <mgoin64@gmail.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>

Signed-off-by: czhu-cohere <conway.zhu@cohere.com>

Signed-off-by: gnovack <gnovack@amazon.com>

Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>

Signed-off-by: zhewenli <zhewenli@meta.com>
Signed-off-by: Zhewen Li <zhewenli@meta.com>

Signed-off-by: yewentao256 <zhyanwentao@126.com>

Signed-off-by: Ming Yang <minos.future@gmail.com>

Signed-off-by: Ziliang Peng <ziliang@character.ai>

Signed-off-by: Siyuan Fu <siyuanf@nvidia.com>

Signed-off-by: Roi Koren <roik@nvidia.com>

Signed-off-by: vasiliy <vasiliy@fb.com>

Signed-off-by: Shahar Mor <smor@nvidia.com>
Co-authored-by: Tyler Michael Smith <tyler@neuralmagic.com>

Signed-off-by: weiguihua2 <weiguihua2@huawei.com>

Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>

Signed-off-by: Laith Sakka <lsakka@meta.com>