GitLab

Co-authored-by: helloyongyang <yongyang1030@163.com>

1. 修复muxi int8-vllm推理结果精度问题。torch.empty导致推理结果有nan值。
Co-authored-by: root <root@master.cluster.local>

1. 修复mlu int8量化

1. 修复之前的代码合并冲突，并测试通过。

---------
Co-authored-by: Yang Yong (雍洋) <yongyang1030@163.com>

Co-authored-by: yihuiwen <yihuiwen@sensetime.com>

Thanks to HunyuanVideo Team and ModelTC Team.

---------
Co-authored-by: gushiqiao <975033167@qq.com>
Co-authored-by: gushiqiao <77222802+gushiqiao@users.noreply.github.com>
Co-authored-by: chendingyu <chendingyu1@sensetime.com>
Co-authored-by: XHPlus <xhplus@163.com>
Co-authored-by: wangshankun <wangshankun2011@hotmail.com>
Co-authored-by: STwangyingrui <86730325+STwangyingrui@users.noreply.github.com>
Co-authored-by: root <root@pt-80f094c20fc44a8cad096e5f3dbc962e-worker-0.pt-80f094c20fc44a8cad096e5f3dbc962e.ns-devsft-3460edd0.svc.cluster.local>

Feature:
    1. added mlu590 bfloat16, single-gpu and multi-gpus inference.
    2. added mlu590 int8 inference.

Co-authored-by: root <root@pt-731f15f5c62740f495519352dc90d9c4-worker-0.pt-731f15f5c62740f495519352dc90d9c4.ns-devsft-3460edd0.svc.cluster.local>

Co-authored-by: gushiqiao <975033167@qq.ocm>

* add gguf loading functions

* support wan22 distill

* reconstruct quantization and fix memory leak bug.

* Support lazy load inference.

* reconstruct quantization

* Fix hunyuan bugs

* deleted tmp file

---------
Co-authored-by: root <root@pt-c0b333b3a1834e81a0d4d5f412c6ffa1-worker-0.pt-c0b333b3a1834e81a0d4d5f412c6ffa1.ns-devsft-3460edd0.svc.cluster.local>
Co-authored-by: gushiqiao <gushqiaio@sensetime.com>
Co-authored-by: gushiqiao <gushiqiao@sensetime.com>

* fix offload extra latency in the first step by pre-allocating pinned memory

* pre-commit

---------
Co-authored-by: “de1star” <“843414674@qq.com”>