GitLab

[Misc] Refactor linear layer weight loading; introduce `BasevLLMParameter` and `weight_loader_v2` (#5874)

Co-authored-by: Roger Wang <ywang@roblox.com>

[Core] Subclass ModelRunner to support cross-attention & encoder sequences (towards eventual encoder/decoder model support) (#4942)
Co-authored-by: Andrew Feldman <afeld2012@gmail.com>
Co-authored-by: Nick Hill <nickhill@us.ibm.com>

Co-authored-by: Tyler Michael Smith <tyler@neuralmagic.com>

Co-authored-by: Roger Wang <136131678+ywang96@users.noreply.github.com>

Co-authored-by: Michael Goin <michael@neuralmagic.com>

Co-authored-by: Jacob Schein <jacobschein@Jacobs-MacBook-Pro-2.local>

Signed-off-by: Thomas Parnell <tpa@zurich.ibm.com>

Signed-off-by: Rui Qiao <ruisearch42@gmail.com>

Co-authored-by: Roger Wang <ywang@roblox.com>

Co-authored-by: hezhihui <hzh7269@modelbest.cn>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>

[Bugfix] [SpecDecode] Default speculative_draft_tensor_parallel_size to 1 when using MLPSpeculator (#7105)
Signed-off-by: Thomas Parnell <tpa@zurich.ibm.com>

Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>

Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>

Signed-off-by: Jefferson Fialho <jfialho@ibm.com>
Co-authored-by: Nick Hill <nickhill@us.ibm.com>

Signed-off-by: Joe Runde <Joseph.Runde@ibm.com>
Co-authored-by: Joe Runde <Joseph.Runde@ibm.com>
Co-authored-by: Joe Runde <joe@joerun.de>
Co-authored-by: Nick Hill <nickhill@us.ibm.com>
Co-authored-by: Simon Mo <simon.mo@hey.com>

Signed-off-by: Rui Qiao <ruisearch42@gmail.com>