GitLab

[WIP][[V1][Metrics] Implement max_num_generation_tokens,  request_params_n, and request_params_max_tokens metrics (#14055)
Signed-off-by: Mark McLoughlin <markmc@redhat.com>

Signed-off-by: Nick Hill <nhill@redhat.com>

Signed-off-by: Mark McLoughlin <markmc@redhat.com>

Signed-off-by: Mark McLoughlin <markmc@redhat.com>

Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>

Signed-off-by: Mark McLoughlin <markmc@redhat.com>

Signed-off-by: Cody Yu <hao.yu.cody@gmail.com>

Signed-off-by: Mengqing Cao <cmq0113@163.com>

Signed-off-by: Ce Gao <cegao@tensorchord.ai>

Signed-off-by: Jun Duan <jun.duan.phd@outlook.com>

Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>

Signed-off-by: Sage Moore <sage@neuralmagic.com>

Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>

Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>

[torch.compile] Fix RMSNorm + quant fusion in the non-cutlass-fp8 case, rename RedundantReshapesPass to NoopEliminationPass (#10902)
Signed-off-by: luka <luka@neuralmagic.com>

Signed-off-by: Rui Qiao <ruisearch42@gmail.com>

Signed-off-by: Chen Zhang <zhangch99@outlook.com>
Co-authored-by: Cody Yu <hao.yu.cody@gmail.com>

Signed-off-by: Chen Zhang <zhangch99@outlook.com>

Signed-off-by: Xiongfei Wei <isaacwxf23@gmail.com>
Signed-off-by: mgoin <mgoin64@gmail.com>
Co-authored-by: mgoin <mgoin64@gmail.com>

[Bugfix][Disaggregated] patch the inflight batching on the decode node in SimpleConnector to avoid hangs in SimpleBuffer (nccl based) (#13987)
Signed-off-by: Mathis Felardos <mathis@mistral.ai>

Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>

Signed-off-by: Travis Johnson <tsjohnso@us.ibm.com>

Signed-off-by: Roger Wang <ywang@roblox.com>

Signed-off-by: Benjamin Chislett <benjamin.chislett@centml.ai>

Signed-off-by: Yang Chen <yangche@fb.com>
Signed-off-by: Lucas Wilkinson <lwilkinson@neuralmagic.com>
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Co-authored-by: Yang Chen <yangche@fb.com>