GitLab

Enable 720p model inference on low-spec GPUs/CPUs and accelerate T5/CLIP quantized models with vLLM operators

Co-authored-by: gushiqiao <gushiqiao@sensetime.com>

* add step & cfg distillation wan model

* bug fixed

* reconstruct quantization and fix memory leak bug.

* Support lazy load inference.

* reconstruct quantization

* Fix hunyuan bugs

* deleted tmp file

---------
Co-authored-by: root <root@pt-c0b333b3a1834e81a0d4d5f412c6ffa1-worker-0.pt-c0b333b3a1834e81a0d4d5f412c6ffa1.ns-devsft-3460edd0.svc.cluster.local>
Co-authored-by: gushiqiao <gushqiaio@sensetime.com>
Co-authored-by: gushiqiao <gushiqiao@sensetime.com>

* split dit server from default runner

* split dit server from default runner

* update loading functions

* simplify loader functions and runner functions

* simplify code && split dit service

* simplify code && split dit service

* support split server for cogvideox

* clear code.

* add load_transformer methods for split server

* add service utils

* [feature] support split servers

* function hunyuan_t2v_tea, hunyuan_t2v_taylorseer, modify the fresh_threshold of taylorseer

* hunyuan i2v,t2v + tea,tay; wan i2v,t2v + tea function, add log files

* 删除了TeaCace Scheduler的多余属性

* 删除了多余目录

* 修复了TeaCaching部分的bug,目前t2v, i2v feature caching均可跑通

* Update attn_weight.py

---------
Co-authored-by: Yang Yong(雍洋) <yongyang1030@163.com>

adding a padding strategy for SP

---------
Co-authored-by: “de1star” <“843414674@qq.com”>

* Support load advance ptq model.

* Update run_wan_i2v_advanced_ptq.sh

---------
Co-authored-by: gushiqiao <gushiqiao@sensetime.com>
Co-authored-by: Yang Yong(雍洋) <yongyang1030@163.com>

* [feature]: add Wan Sparge infer

* Update scripts/run_wan_t2v_sparge.sh
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* [minor]: fix typo and use config style

* [minor]: remove breakpoint

* [feature]: add all attn class

* [minor]: remove args

* [minor]: remove shared weights

---------
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* [bugs fixed] fixed bugs for cpu offload.

* [rename] rename causal_model -> causvid_model

* [feature] add prompt enhancer

* [feature] add prompt enhancer

* [rename] rename causal_model -> causvid_model

* [feature]: add support for Causvid inference

* [feature]: support long video generation for autoregressive wan2.1-t2v

* [feature]: support long video generation for autoregressive wan2.1-t2v

* Update wan_t2v_causal.json

* Update run_wan_t2v_causal.sh

* update readme for Wan2.1-T2V-CausVid

---------
Co-authored-by: Yang Yong(雍洋) <yongyang1030@163.com>