GitLab

Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>

Signed-off-by: linyueqian <linyueqian@outlook.com>
Signed-off-by: Roger Wang <hey@rogerw.io>
Co-authored-by: Roger Wang <hey@rogerw.io>

Signed-off-by: Vadim Gimpelson <vadim.gimpelson@gmail.com>

Fixes a not-yet-reported case where it was possible for blocks to be
freed by an abort before an async transfer completed, resulting
in corrupted KV data.
Signed-off-by: Or Ozeri <oro@il.ibm.com>

Signed-off-by: zhxchen17 <zhxchen17@fb.com>

Signed-off-by: zhxchen17 <zhxchen17@fb.com>

Signed-off-by: Kunshang Ji <kunshang.ji@intel.com>

Signed-off-by: augusto.yjh <augusto.yjh@antgroup.com>

Signed-off-by: Kunshang Ji <kunshang.ji@intel.com>

Add labeled Prometheus metrics to distinguish where prompt tokens come
from in P/D disaggregated deployments.

In P/D disaggregation, decode instances receive KV cache from prefill instances.
Currently, decode reports inflated prompt throughput because it counts all
prompt tokens as "computed", even though most were transferred.

This PR adds labeled metrics so users can understand actual compute work vs
transferred work:

vllm:prompt_tokens_by_source_total{source="local_compute"}        # Tokens prefilled locally
vllm:prompt_tokens_by_source_total{source="external_kv_transfer"} # Tokens received via KV transfer  
vllm:prompt_tokens_by_source_total{source="local_cache_hit"}      # Tokens from local prefix cache
vllm:prompt_tokens_cached_total                                    # Total cached (local + external, -1 when all 
Signed-off-by: Zhanqiu Hu <zh338@cornell.edu>

Signed-off-by: Matthew Wong <Matthew.Wong2@amd.com>

Signed-off-by: yewentao256 <zhyanwentao@126.com>

Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>

Signed-off-by: frankwang28 <frank.wbb@hotmail.com>

Signed-off-by: yewentao256 <zhyanwentao@126.com>

Signed-off-by: mgoin <mgoin64@gmail.com>

Signed-off-by: Huy Do <huydhn@gmail.com>

Signed-off-by: shen-shanshan <467638484@qq.com>

Signed-off-by: Rehan Khan <Rehan.Khan7@ibm.com>

Signed-off-by: Andrew Xia <axia@fb.com>
Co-authored-by: Andrew Xia <axia@fb.com>

Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>

Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>

Signed-off-by: Nick Hill <nickhill123@gmail.com>

Signed-off-by: yewentao256 <zhyanwentao@126.com>

Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>

[Bugfix] Disable TRTLLM FP8 MoE if router_logits_dtype==float32 and routing_method!=DeepSeekV3 (#33613)
Signed-off-by: mgoin <mgoin64@gmail.com>

Signed-off-by: Patrick von Platen <patrick.v.platen@gmail.com>

Signed-off-by: Vadim Gimpelson <vadim.gimpelson@gmail.com>

Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>

Signed-off-by: Richard Zou <zou3519@gmail.com>

Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>

Signed-off-by: Tianchen Ding <dtcccc@linux.alibaba.com>
Co-authored-by: Nicolò Lucchesi <nicolo.lucchesi@gmail.com>

Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>

Signed-off-by: Patrick von Platen <patrick.v.platen@gmail.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>

Signed-off-by: shen-shanshan <467638484@qq.com>

[Bugfix] Do not add extra \n for image-only cases when constructing multimodal text prompts. (#33647)
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>

[Bugfix][Async][Connector] avoid vllm-side double free during async scheduling + request abort + async KV cache transfer (#33377)
Signed-off-by: KuntaiDu <kuntai@uchicago.edu>

Signed-off-by: KrxGu <krishom70@gmail.com>

Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>