GitLab

Co-authored-by: sang <sangcho@anyscale.com>

Signed-off-by: Joe Runde <joe@joerun.de>

Co-authored-by: Antoni Baum <antoni.baum@protonmail.com>

Signed-off-by: Joe Runde <Joseph.Runde@ibm.com>

[Core][Distributed] add coordinator to reduce code duplication in tp and pp (#5293)

Co-authored-by: Roger Wang <ywang@roblox.com>

Currently we need to call rotary embedding kernel for each LoRA, which makes it hard to serve multiple long context length LoRA. Add batched rotary embedding kernel and pipe it through.

It replaces the rotary embedding layer to the one that is aware of multiple cos-sin-cache per scaling factors.

Follow up of https://github.com/vllm-project/vllm/pull/3095/files

Co-authored-by: Silencio <silencio@adsl-99-6-187-6.dsl.irvnca.sbcglobal.net>

Co-authored-by: SAHIL SUNEJA <suneja@us.ibm.com>

Co-authored-by: Antoni Baum <antoni.baum@protonmail.com>

Co-authored-by: simon-mo <simon.mo@hey.com>

[WIP][Core][Refactor] move vllm/model_executor/parallel_utils into vllm/distributed and vllm/device_communicators (#3950)

[Misc] [Core] Implement RFC "Augment BaseExecutor interfaces to enable hardware-agnostic speculative decoding" (#3837)

Co-authored-by: Antoni Baum <antoni.baum@protonmail.com>

Add missing kernel for CodeLlama-34B on A/H100 (no tensor parallelism) when using Multi-LoRA. (#3350)