GitLab

[Speculative Decoding] Enabling bonus token in speculative decoding for KV cache based models (#5765)

Signed-off-by: sangjune.park <sangjune.park@navercorp.com>

Signed-off-by: Thomas Parnell <tpa@zurich.ibm.com>

[core][distributed] add zmq fallback for broadcasting large objects (#6183)

Co-authored-by: Robert Shaw <114415538+robertgshaw2-neuralmagic@users.noreply.github.com>

Co-authored-by: Swapnil Parekh <swapnilp@ibm.com>
Co-authored-by: Joe G <joseph.granados@h2o.ai>
Co-authored-by: Antoni Baum <antoni.baum@protonmail.com>

[Kernel] Correctly invoke prefill & decode kernels for cross-attention (towards eventual encoder/decoder model support) (#4888)
Co-authored-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>

Co-authored-by: Cody Yu <hao.yu.cody@gmail.com>

Co-authored-by: Robert Shaw <rshaw@neuralmagic>

Co-authored-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>

Co-authored-by: Simon Mo <simon.mo@hey.com>

Signed-off-by: Yuan Zhou <yuan.zhou@intel.com>

[ROCm][AMD][Model]Adding alibi slopes support in ROCm triton flash attention and naive flash attention (#6043)
Co-authored-by: Hongxia Yang <62075498+hongxiayang@users.noreply.github.com>

Signed-off-by: Muralidhar Andoorveedu <muralidhar.andoorveedu@centml.ai>

Signed-off-by: Travis Johnson <tsjohnso@us.ibm.com>
Co-authored-by: Nick Hill <nickhill@us.ibm.com>

Signed-off-by: Xiaowei Jiang <xwjiang2010@gmail.com>
Co-authored-by: Roger Wang <ywang@roblox.com>