GitLab

Co-authored-by: Philipp Moritz <pcmoritz@gmail.com>

Co-authored-by: Michael Goin <michael@neuralmagic.com>
Co-authored-by: youkaichao <youkaichao@gmail.com>
Co-authored-by: zifeitong <zifei.tong@parasail.io>
Co-authored-by: Robert Shaw <114415538+robertgshaw2-neuralmagic@users.noreply.github.com>

Co-authored-by: Roger Wang <ywang@roblox.com>

Co-authored-by: Jianan Gu <jianan.gu@intel.com>

Tune Qwen2-57B-A14B configs based on #4921

Throughput Performance
command: python benchmarks/benchmark_throughput.py --model=Qwen/Qwen2-57B-A14B-Instruct --input-len 1000 --output-len 50 -tp 2

A100 GPU

benchmark	no config	w/ PR
tp=2	10.53 requests/s, 11058.17 tokens/s	12.47 requests/s, 13088.57 tokens/s
tp=4	17.77 requests/s, 18662.95 tokens/s	20.20 requests/s, 21212.32 tokens/s

Co-authored-by: Robert Shaw <114415538+robertgshaw2-neuralmagic@users.noreply.github.com>

[Core][Distributed] add coordinator to reduce code duplication in tp and pp (#5293)

Signed-off-by: Travis Johnson <tsjohnso@us.ibm.com>
Co-authored-by: Sanger Steel <sangersteel@gmail.com>
Co-authored-by: Roger Wang <ywang@roblox.com>

Co-authored-by: Antoni Baum <antoni.baum@protonmail.com>

Co-authored-by: DarkLight1337 <tlleungac@connect.ust.hk>

Co-authored-by: Roger Wang <ywang@roblox.com>

Co-authored-by: Michael Goin <michael@neuralmagic.com>

[Core][CUDA Graph] add output buffer for cudagraph to reduce memory footprint (#5074)