GitLab

Signed-off-by: Seiji Eicher <seiji@anyscale.com>

Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>

Signed-off-by: Lukas Geiger <lukas.geiger94@gmail.com>

[Bugfix] Max concurrency estimation and check_enough_kv_cache_memory for models with sliding window layers (#19029)
Signed-off-by: Chen Zhang <zhangch99@outlook.com>

Signed-off-by: Chen Zhang <zhangch99@outlook.com>

Signed-off-by: Chen Zhang <zhangch99@outlook.com>

Signed-off-by: Yong Hoon Shin <yhshin@meta.com>

Signed-off-by: simon-mo <simon.mo@hey.com>

Signed-off-by: raushan <raushan@huggingface.co>

Signed-off-by: Chen Zhang <zhangch99@outlook.com>

Signed-off-by: Siyuan Liu <lsiyuan@google.com>
Co-authored-by: Hossein Sarshar <hossein.sarshar@gmail.com>
Co-authored-by: Chengji Yao <chengjiyao@google.com>

Signed-off-by: 22quinn <33176974+22quinn@users.noreply.github.com>

[Misc] reuse num_tokens_across_dp of get_dp_padding to avoid unnecessary dp all reduce in set_forward_context (#18935)
Signed-off-by: Tyler Michael Smith <tysmith@redhat.com>
Co-authored-by: zhuhaoran <zhuhaoran.zhr@alibaba-inc.com>
Co-authored-by: Tyler Michael Smith <tysmith@redhat.com>

Signed-off-by: Nick Hill <nhill@redhat.com>

Signed-off-by: Nick Hill <nhill@redhat.com>
Co-authored-by: Yizhou Liu <liu_yizhou@outlook.com>

Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>

Signed-off-by: Yong Hoon Shin <yhshin@meta.com>

Signed-off-by: Carol Zheng <cazheng@google.com>

Signed-off-by: vllmellm <vllm.ellm@embeddedllm.com>

Signed-off-by: Will Eaton <weaton@redhat.com>

Signed-off-by: Nick Hill <nhill@redhat.com>
Co-authored-by: Will Eaton <weaton@redhat.com>

Signed-off-by: nicklucche <nlucches@redhat.com>

Signed-off-by: googs1025 <googs1025@gmail.com>

Signed-off-by: Gregory Shtrasberg <Gregory.Shtrasberg@amd.com>

Signed-off-by: Lukas Geiger <lukas.geiger94@gmail.com>

Signed-off-by: Varun Sundar Rabindranath <varun@neuralmagic.com>
Co-authored-by: Varun Sundar Rabindranath <varun@neuralmagic.com>

Signed-off-by: Akshat Tripathi <akshat@krai.ai>
Signed-off-by: Chengji Yao <chengjiyao@google.com>
Signed-off-by: xihajun <junfan@krai.ai>
Signed-off-by: Jorge de Freitas <jorge.de-freitas22@imperial.ac.uk>
Signed-off-by: Jorge de Freitas <jorge@krai.ai>
Co-authored-by: Chengji Yao <chengjiyao@google.com>
Co-authored-by: xihajun <junfan@krai.ai>
Co-authored-by: Jorge de Freitas <jorge.de-freitas22@imperial.ac.uk>
Co-authored-by: Jorge de Freitas <jorge@krai.ai>

Signed-off-by: Aaron Pham <contact@aarnphm.xyz>

Signed-off-by: LucasWilkinson <lwilkinson@neuralmagic.com>

Signed-off-by: Ronald Xu <ronaldxu@amazon.com>

Signed-off-by: Divakar Verma <divakar.verma@amd.com>

Signed-off-by: Lukas Geiger <lukas.geiger94@gmail.com>

Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>

Signed-off-by: Mark McLoughlin <markmc@redhat.com>