GitLab

[distributed][misc] keep consistent with how pytorch finds libcudart.so (#6346)

Co-authored-by: Zifei Tong <zifeitong@gmail.com>

Signed-off-by: Xiaowei Jiang <xwjiang2010@gmail.com>

Co-authored-by: Robert Shaw <rshaw@neuralmagic.com>

Signed-off-by: Thomas Parnell <tpa@zurich.ibm.com>
Co-authored-by: Travis Johnson <tsjohnso@us.ibm.com>

Signed-off-by: yatta zhang <ytzhang01@foxmail.com>
Signed-off-by: zhangyuntao.dev <zhangyuntao.dev@bytedance.com>
Co-authored-by: zhangyuntao.dev <zhangyuntao.dev@bytedance.com>

[Speculative Decoding] Enabling bonus token in speculative decoding for KV cache based models (#5765)

Signed-off-by: sangjune.park <sangjune.park@navercorp.com>

Signed-off-by: Thomas Parnell <tpa@zurich.ibm.com>

[core][distributed] add zmq fallback for broadcasting large objects (#6183)

Co-authored-by: Robert Shaw <114415538+robertgshaw2-neuralmagic@users.noreply.github.com>

Co-authored-by: Swapnil Parekh <swapnilp@ibm.com>
Co-authored-by: Joe G <joseph.granados@h2o.ai>
Co-authored-by: Antoni Baum <antoni.baum@protonmail.com>

[Kernel] Correctly invoke prefill & decode kernels for cross-attention (towards eventual encoder/decoder model support) (#4888)
Co-authored-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>

Co-authored-by: Cody Yu <hao.yu.cody@gmail.com>

Co-authored-by: Robert Shaw <rshaw@neuralmagic>

Co-authored-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>