GitLab

2.解决mtp >1 大EP推理all gather卡住问题

This reverts merge request !169

Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>

Signed-off-by: Nick Hill <nhill@redhat.com>
Signed-off-by: Andrew Feldman <afeldman@neuralmagic.com>
Signed-off-by: Andrew Feldman <afeldman@redhat.com>
Co-authored-by: Nick Hill <nhill@redhat.com>

Signed-off-by: cjackal <44624812+cjackal@users.noreply.github.com>

Signed-off-by: luka <luka@neuralmagic.com>

Signed-off-by: youkaichao <youkaichao@gmail.com>

Signed-off-by: qizixi <qizixi@meta.com>

Signed-off-by: Benjamin Chislett <benjamin.chislett@centml.ai>

Signed-off-by: simon-mo <simon.mo@hey.com>

Signed-off-by: Ronald Xu <ronaldxu@amazon.com>

Signed-off-by: Mark McLoughlin <markmc@redhat.com>

Signed-off-by: qizixi <qizixi@meta.com>

Signed-off-by: qizixi <qizixi@meta.com>

Signed-off-by: Rui Qiao <ruisearch42@gmail.com>
Signed-off-by: YaoJiayi <120040070@link.cuhk.edu.cn>
Co-authored-by: Rui Qiao <ruisearch42@gmail.com>

Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>

Signed-off-by: Ronald Xu <ronaldxu@amazon.com>

Signed-off-by: Mark McLoughlin <markmc@redhat.com>

Signed-off-by: Seiji Eicher <seiji@anyscale.com>

Signed-off-by: lisiqi23 <lisiqi23@xiaomi.com>
Signed-off-by: skylee-01 <497627264@qq.com>
Co-authored-by: lisiqi23 <lisiqi23@xiaomi.com>

[V1][Spec Decode] Share input embedding of target model with EAGLE draft model to free ~1GB for llama 3 model (#17326)
Co-authored-by: root <root@ekagra-8xh100.us-east5-a.c.serving-efficiency-poc.internal>
Co-authored-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>

Signed-off-by: wwl2755 <wangwenlong2755@gmail.com>

Signed-off-by: Mark McLoughlin <markmc@redhat.com>

Signed-off-by: Mark McLoughlin <markmc@redhat.com>

Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>