GitLab

[CI][AMD][BugFix] Ensure VLLM_ROCM_USE_AITER is set so test_rocm_aiter_topk.py can run correctly (#33840)
Signed-off-by: Randall Smith <Randall.Smith@amd.com>

Signed-off-by: rinbaro <ilgomishra@gmail.com>

Signed-off-by: Nick Hill <nickhill123@gmail.com>

Signed-off-by: Andreas Karatzas <akaratza@amd.com>

Signed-off-by: Kevin H. Luu <khluu000@gmail.com>

Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>

Signed-off-by: Zhanqiu Hu <zh338@cornell.edu>

Signed-off-by: Luka Govedič <lgovedic@redhat.com>
Signed-off-by: ProExpertProg <luka.govedic@gmail.com>
Signed-off-by: Luka Govedič <ProExpertProg@users.noreply.github.com>

Signed-off-by: Ilya Boytsov <ilyaboytsov1805@gmail.com>
Signed-off-by: Ilya Boytsov <boytsovpanamera@mail.ru>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
Co-authored-by: wang.yuqi <yuqi.wang@daocloud.io>

Signed-off-by: Nick Hill <nickhill123@gmail.com>

Change the type signature of MixtureOfExperts.expert_weights to MutableSequence[Sequence[Tensor]] (#33573)
Signed-off-by: Sage Moore <sagmoore@redhat.com>
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com>

Signed-off-by: Richard Zou <zou3519@gmail.com>

Signed-off-by: Muhammad Hashmi <mhashmi@berkeley.edu>
Signed-off-by: NickLucche <nlucches@redhat.com>
Co-authored-by: NickLucche <nlucches@redhat.com>

Signed-off-by: simondanielsson <simon.danielsson99@hotmail.com>

Signed-off-by: Taeksang Kim <ts.kim@hyperaccel.ai>

Signed-off-by: Kourosh Hakhamaneshi <kourosh@anyscale.com>

Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>

Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>

Signed-off-by: jiangkuaixue123 <jiangxiaozhou111@163.com>

Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>

Signed-off-by: yewentao256 <zhyanwentao@126.com>
Signed-off-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com>
Co-authored-by: Nick Hill <nhill@redhat.com>

Signed-off-by: Micah Williamson <micah.williamson@amd.com>

Signed-off-by: Zachary Aristei <zaristei@nvidia.com>
Co-authored-by: zaristei2 <zaristei2@gmail.com>
Co-authored-by: Zachary Aristei <zaristei@nvidia.com>

Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>

Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>

Signed-off-by: linyueqian <linyueqian@outlook.com>
Signed-off-by: Roger Wang <hey@rogerw.io>
Co-authored-by: Roger Wang <hey@rogerw.io>

Signed-off-by: Vadim Gimpelson <vadim.gimpelson@gmail.com>

Fixes a not-yet-reported case where it was possible for blocks to be
freed by an abort before an async transfer completed, resulting
in corrupted KV data.
Signed-off-by: Or Ozeri <oro@il.ibm.com>

Signed-off-by: zhxchen17 <zhxchen17@fb.com>

Signed-off-by: zhxchen17 <zhxchen17@fb.com>

Signed-off-by: Kunshang Ji <kunshang.ji@intel.com>

Signed-off-by: augusto.yjh <augusto.yjh@antgroup.com>

Signed-off-by: Kunshang Ji <kunshang.ji@intel.com>

Add labeled Prometheus metrics to distinguish where prompt tokens come
from in P/D disaggregated deployments.

In P/D disaggregation, decode instances receive KV cache from prefill instances.
Currently, decode reports inflated prompt throughput because it counts all
prompt tokens as "computed", even though most were transferred.

This PR adds labeled metrics so users can understand actual compute work vs
transferred work:

vllm:prompt_tokens_by_source_total{source="local_compute"}        # Tokens prefilled locally
vllm:prompt_tokens_by_source_total{source="external_kv_transfer"} # Tokens received via KV transfer  
vllm:prompt_tokens_by_source_total{source="local_cache_hit"}      # Tokens from local prefix cache
vllm:prompt_tokens_cached_total                                    # Total cached (local + external, -1 when all 
Signed-off-by: Zhanqiu Hu <zh338@cornell.edu>

Signed-off-by: Matthew Wong <Matthew.Wong2@amd.com>

Signed-off-by: yewentao256 <zhyanwentao@126.com>

Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>

Signed-off-by: frankwang28 <frank.wbb@hotmail.com>

Signed-off-by: yewentao256 <zhyanwentao@126.com>