GitLab

Co-authored-by: Cody Yu <hao.yu.cody@gmail.com>

[Core] Subclass ModelRunner to support cross-attention & encoder sequences (towards eventual encoder/decoder model support) (#4942)
Co-authored-by: Andrew Feldman <afeld2012@gmail.com>
Co-authored-by: Nick Hill <nickhill@us.ibm.com>

Co-authored-by: Michael Goin <michael@neuralmagic.com>

[Bugfix] [SpecDecode] Default speculative_draft_tensor_parallel_size to 1 when using MLPSpeculator (#7105)
Signed-off-by: Thomas Parnell <tpa@zurich.ibm.com>

Signed-off-by: Jefferson Fialho <jfialho@ibm.com>
Co-authored-by: Nick Hill <nickhill@us.ibm.com>

Signed-off-by: Muralidhar Andoorveedu <muralidhar.andoorveedu@centml.ai>

Co-authored-by: Michael Goin <michael@neuralmagic.com>

Signed-off-by: Travis Johnson <tsjohnso@us.ibm.com>

[Spec Decode] Disable Log Prob serialization to CPU for spec decoding for both draft and target models. (#6485)

Co-authored-by: Murali Andoorveedu <muralidhar.andoorveedu@centml.ai>

Co-authored-by: Michael Goin <michael@neuralmagic.com>