GitLab

Signed-off-by: youkaichao <youkaichao@gmail.com>

Signed-off-by: Joe Runde <Joseph.Runde@ibm.com>

Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>

Signed-off-by: Aaron Pham <contact@aarnphm.xyz>

Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>

Signed-off-by: Went-Liang <wenteng_liang@163.com>

Signed-off-by: wangshuai09 <391746016@qq.com>

Co-authored-by: Roger Wang <136131678+ywang96@users.noreply.github.com>

Signed-off-by: Muralidhar Andoorveedu <muralidhar.andoorveedu@centml.ai>
Signed-off-by: Murali Andoorveedu <muralidhar.andoorveedu@centml.ai>
Co-authored-by: DarkLight1337 <tlleungac@connect.ust.hk>

Co-authored-by: Michael Goin <michael@neuralmagic.com>

Co-authored-by: Michael Goin <michael@neuralmagic.com>

Signed-off-by: Travis Johnson <tsjohnso@us.ibm.com>

Co-authored-by: hezhihui <hzh7269@modelbest.cn>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>

original title: [Distributed][Model] Rank-based Component Creation for Pipeline Parallelism Memory Optimization

Co-authored-by: Robert Shaw <rshaw@neuralmagic.com>
Co-authored-by: ZX <zx@lbx.dev>

Signed-off-by: Muralidhar Andoorveedu <muralidhar.andoorveedu@centml.ai>

Co-authored-by: rsnm2 <rshaw@neuralmagic.com>
Co-authored-by: Robert Shaw <114415538+robertgshaw2-neuralmagic@users.noreply.github.com>

Co-authored-by: Varun Sundar Rabindranath <varunsundar08@gmail.com>
Co-authored-by: Varun Sundar Rabindranath <varun@neuralmagic.com>

The 2nd PR for #4532.

This PR supports loading FP8 kv-cache scaling factors from a FP8 checkpoint (with .kv_scale parameter).