GitLab

[Bugfix] Fix Ray compiled-DAG SHM channel stalls by detaching zero-copy `np.ndarray` logprobs buffers (#35736)
Signed-off-by: Nikita Shapovalov <nikita@poolside.ai>

Signed-off-by: Giancarlo Delfin <gdelfin@inferact.ai>

Signed-off-by: Yan Ma <yan.ma@intel.com>
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com>

[Metrics] Add request_id to FinishedRequestStats to enable correlation between metrics and requests (#39710)

Enables external `StatLogger` plugins to correlate per-request metrics
with request-level context. Also, this is a pre-requisite for Prometheus
exemplars in #30972.
Signed-off-by: Csrayz <33659823+Csrayz@users.noreply.github.com>

Signed-off-by: Or Ozeri <oro@il.ibm.com>

Signed-off-by: vibhavagarwal5 <vibhavagarwal5@gmail.com>
Signed-off-by: Michael Goin <mgoin64@gmail.com>
Co-authored-by: Xinyu Chen <xinyu1.chen@intel.com>
Co-authored-by: Michael Goin <mgoin64@gmail.com>

Signed-off-by: Giancarlo Delfin <gdelfin@inferact.ai>

Signed-off-by: Roi Koren <roik@nvidia.com>

Signed-off-by: omerpaz95 <omerpaz95@gmail.com>
Co-authored-by: Or Ozeri <oro@il.ibm.com>

Signed-off-by: Bortlesboat <bortstheboat@gmail.com>

Signed-off-by: Mark McLoughlin <markmc@redhat.com>

Signed-off-by: Lucas Kabela <lucaskabela@meta.com>

Signed-off-by: shen-shanshan <467638484@qq.com>
Signed-off-by: Shanshan Shen <87969357+shen-shanshan@users.noreply.github.com>
Co-authored-by: Roger Wang <hey@rogerw.io>

[Core][Metrics][BugFix] Replace num_cached_tokens/num_external_computed_tokens with PrefillStats (#37460)

Related to `Counters can only be incremented by non-negative amounts`
error with the `vllm:prompt_tokens_by_source_total` metric.
Signed-off-by: Mark McLoughlin <markmc@redhat.com>
Co-authored-by: Or Ozeri <or@ozery.com>

Signed-off-by: Giancarlo Delfin <gdelfin@inferact.ai>

Signed-off-by: Mukesh Baphna <mukesh@hippocraticai.com>
Signed-off-by: Mark McLoughlin <markmc@redhat.com>
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-authored-by: Mark McLoughlin <markmc@redhat.com>

Signed-off-by: yewentao256 <zhyanwentao@126.com>
Co-authored-by: TJian <tunjian.tan@embeddedllm.com>

Signed-off-by: Mark McLoughlin <markmc@redhat.com>

Signed-off-by: Martin Hickey <martin.hickey@ie.ibm.com>
Co-authored-by: Or Ozeri <or@ozery.com>

Signed-off-by: Xinyu Chen <xinyu1.chen@intel.com>
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com>

Signed-off-by: Tianyu Guo <guoty9@mail2.sysu.edu.cn>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>

Signed-off-by: Rahul-Tuli <rtuli@redhat.com>
Signed-off-by: Fynn Schmitt-Ulms <fschmitt@redhat.com>
Co-authored-by: Rahul-Tuli <rtuli@redhat.com>
Co-authored-by: Claude <noreply@anthropic.com>

Signed-off-by: Yifan Zong <yzong@redhat.com>

Signed-off-by: zhuhaoran <zhuhaoran.zhr@alibaba-inc.com>

Signed-off-by: Peter Nguyen <petern0408@gmail.com>

Signed-off-by: xaguilar-amd <xaguilar@amd.com>

Signed-off-by: Elvir Crncevic <elvircrn@gmail.com>
Co-authored-by: Claude Sonnet 4 <noreply@anthropic.com>

Signed-off-by: Richard Zou <zou3519@gmail.com>

Signed-off-by: jackcfwang <jackcfwang@tencent.com>
Co-authored-by: Isotr0py <mozf@mail2.sysu.edu.cn>

Signed-off-by: Nick Hill <nickhill123@gmail.com>

Signed-off-by: vnadathur <glvikramn@gmail.com>
Signed-off-by: WorldExplored <srreyansh.sethi@gmail.com>
Signed-off-by: Srreyansh Sethi <107075589+WorldExplored@users.noreply.github.com>
Co-authored-by: vnadathur <glvikramn@gmail.com>
Co-authored-by: vnadathur <236933696+vnadathur@users.noreply.github.com>

Signed-off-by: Ronen Schaffer <ronen.schaffer@ibm.com>

Signed-off-by: lkm2835 <lkm2835@gmail.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>

Signed-off-by: R <Ganesh.R@amd.com>

Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>

Signed-off-by: Jaebok Lee <jaebok9541@naver.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>

Signed-off-by: Xinyu Chen <xinyu1.chen@intel.com>

Signed-off-by: yewentao256 <zhyanwentao@126.com>

[Performance Improvement] Update `batched_count_greater_than` to handle batch size 1 without recompile (#38933)
Signed-off-by: Lucas Kabela <lucaskabela@meta.com>
Co-authored-by: Luka Govedič <ProExpertProg@users.noreply.github.com>