GitLab

Signed-off-by: Netanel Haber <58652339+netanel-haber@users.noreply.github.com>

Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>

Signed-off-by: Daniel Serebrenik <daserebrenik@nvidia.com>

Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>

Add a new metric to track the number of requests that had their KV blocks
expire. The scenario is particularly important to surface and track as it is a
vital indicator of the health of the deployment.

Currently we're resorting to track these failures through unstructured log
parsing (which is, among other thing, error string dependent); current main:

> Releasing expired KV blocks for request cmpl-071d which were retrieved by 0 decode worker(s) within 0 seconds.
Signed-off-by: NickLucche <nlucches@redhat.com>

Signed-off-by: Daniel Mescheder <dmesch@amazon.com>
Co-authored-by: Daniel Mescheder <dmesch@amazon.com>

Signed-off-by: NickLucche <nlucches@redhat.com>

Signed-off-by: Andreas Karatzas <akaratza@amd.com>

Signed-off-by: zRzRzRzRzRzRzR <2448370773@qq.com>
Signed-off-by: Yuxuan Zhang <2448370773@qq.com>

Signed-off-by: Matthew Wong <Matthew.Wong2@amd.com>

Signed-off-by: HyunKyun Moon <mhg5303@gmail.com>
Signed-off-by: Hyunkyun Moon <mhg5303@gmail.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>

Signed-off-by: Alex-Brooks <Alex.Brooks@ibm.com>

Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>

Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>

[BUGFIX]  Fix degenerate strides in TRTLLM query tensors for FlashInfer backend. Fixes issue #32353 (#32417)
Signed-off-by: Vadim Gimpelson <vadim.gimpelson@gmail.com>

Signed-off-by: Iryna Boiko <iboiko@habana.ai>

Signed-off-by: yewentao256 <zhyanwentao@126.com>

Signed-off-by: yewentao256 <zhyanwentao@126.com>

Signed-off-by: Andrey Khalyavin <halyavin@yandex-team.ru>

Signed-off-by: Bill Nell <bnell@redhat.com>

Signed-off-by: tom-zju <tanjianpingzju1990@gmail.com>
Co-authored-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com>

Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>

Signed-off-by: xieli <xieli@stepfun.com>

Signed-off-by: gcanlin <canlinguosdu@gmail.com>

Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>

Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>

Signed-off-by: Robert Shaw <rshaw@neuralmagic.com>
Co-authored-by: Robert Shaw <rshaw@neuralmagic.com>

Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>

Signed-off-by: Karan Bansal <karanb192@gmail.com>

Signed-off-by: Shengqi Chen <harry-chen@outlook.com>

Signed-off-by: Mritunjay Sharma <mritunjay.sharma@chainguard.dev>

Signed-off-by: Guofang Tang <tinggofun@gmail.com>
Co-authored-by: Guofang Tang <tinggofun@gmail.com>

Signed-off-by: kimheesu <wlskaka4@gmail.com>

Signed-off-by: Paul Pak <paulpak58@gmail.com>

[CI] Implement uploading to PyPI and GitHub in the release pipeline, enable release image building for CUDA 13.0 (#31032)

Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>

Signed-off-by: Vanshil Shah <vanshilshah@gmail.com>