GitLab

[Speculative Decoding] Enabling bonus token in speculative decoding for KV cache based models (#5765)

Signed-off-by: sangjune.park <sangjune.park@navercorp.com>

Signed-off-by: Thomas Parnell <tpa@zurich.ibm.com>

[core][distributed] add zmq fallback for broadcasting large objects (#6183)

Co-authored-by: Robert Shaw <114415538+robertgshaw2-neuralmagic@users.noreply.github.com>

Signed-off-by: Muralidhar Andoorveedu <muralidhar.andoorveedu@centml.ai>
Co-authored-by: Simon Mo <simon.mo@hey.com>

Co-authored-by: Swapnil Parekh <swapnilp@ibm.com>
Co-authored-by: Joe G <joseph.granados@h2o.ai>
Co-authored-by: Antoni Baum <antoni.baum@protonmail.com>

Signed-off-by: kevin <kevin@anyscale.com>

[Kernel] Correctly invoke prefill & decode kernels for cross-attention (towards eventual encoder/decoder model support) (#4888)
Co-authored-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>

Co-authored-by: Roger Wang <ywang@roblox.com>

Co-authored-by: Cody Yu <hao.yu.cody@gmail.com>

Co-authored-by: Robert Shaw <rshaw@neuralmagic>

Co-authored-by: Roger Wang <ywang@roblox.com>

Co-authored-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>

Signed-off-by: Christian Rohmann <christian.rohmann@inovex.de>