README_zh.md 11.5 KB
Newer Older
Harahan's avatar
Harahan committed
1
<div align="center" style="font-family: charter;">
gushiqiao's avatar
gushiqiao committed
2
  <h1>⚡️ LightX2V:<br> 轻量级视频生成推理框架</h1>
Harahan's avatar
Harahan committed
3
4
5
6
7
8
9

<img alt="logo" src="assets/img_lightx2v.png" width=75%></img>

[![License](https://img.shields.io/badge/License-Apache_2.0-blue.svg)](https://opensource.org/licenses/Apache-2.0)
[![Ask DeepWiki](https://deepwiki.com/badge.svg)](https://deepwiki.com/ModelTC/lightx2v)
[![Doc](https://img.shields.io/badge/docs-English-99cc2)](https://lightx2v-en.readthedocs.io/en/latest)
[![Doc](https://img.shields.io/badge/文档-中文-99cc2)](https://lightx2v-zhcn.readthedocs.io/zh-cn/latest)
helloyongyang's avatar
helloyongyang committed
10
[![Papers](https://img.shields.io/badge/论文集-中文-99cc2)](https://lightx2v-papers-zhcn.readthedocs.io/zh-cn/latest)
Yang Yong (雍洋)'s avatar
Yang Yong (雍洋) committed
11
[![Docker](https://img.shields.io/badge/Docker-2496ED?style=flat&logo=docker&logoColor=white)](https://hub.docker.com/r/lightx2v/lightx2v/tags)
Harahan's avatar
Harahan committed
12

helloyongyang's avatar
helloyongyang committed
13
14
**\[ [English](README.md) | 中文 \]**

Harahan's avatar
Harahan committed
15
16
</div>

helloyongyang's avatar
helloyongyang committed
17
18
--------------------------------------------------------------------------------

gushiqiao's avatar
gushiqiao committed
19
20
**LightX2V** 是一个先进的轻量级视频生成推理框架,专为提供高效、高性能的视频合成解决方案而设计。该统一平台集成了多种前沿的视频生成技术,支持文本生成视频(T2V)和图像生成视频(I2V)等多样化生成任务。**X2V 表示将不同的输入模态(X,如文本或图像)转换为视频输出(V)**

Yang Yong (雍洋)'s avatar
Yang Yong (雍洋) committed
21
22
## :fire: 最新动态

Yang Yong (雍洋)'s avatar
Yang Yong (雍洋) committed
23
- **2025年11月21日:** 🚀 我们Day0支持了[HunyuanVideo-1.5](https://huggingface.co/tencent/HunyuanVideo-1.5)的视频生成模型,同样GPU数量,LightX2V可带来约2倍以上的速度提升,并支持更低显存GPU部署(如24G RTX4090)。支持CFG并行/Ulysses并行,高效Offload,TeaCache/MagCache等技术。同时支持沐曦,寒武纪等国产芯片部署。我们很快将在我们的[HuggingFace主页](https://huggingface.co/lightx2v)更新更多模型,包括步数蒸馏,VAE蒸馏等相关模型。量化模型和轻量VAE模型现已可用:[Hy1.5-Quantized-Models](https://huggingface.co/lightx2v/Hy1.5-Quantized-Models)用于量化推理,[HunyuanVideo-1.5轻量TAE](https://huggingface.co/lightx2v/Autoencoders/blob/main/lighttaehy1_5.safetensors)用于快速VAE解码。使用教程参考[这里](https://github.com/ModelTC/LightX2V/tree/main/scripts/hunyuan_video_15),或查看[示例目录](https://github.com/ModelTC/LightX2V/tree/main/examples)获取代码示例。
Yang Yong (雍洋)'s avatar
Yang Yong (雍洋) committed
24
25


helloyongyang's avatar
helloyongyang committed
26
27
28
29
## 💡 快速开始

详细使用说明请参考我们的文档:**[英文文档](https://lightx2v-en.readthedocs.io/en/latest/) | [中文文档](https://lightx2v-zhcn.readthedocs.io/zh-cn/latest/)**

30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
### 从 Git 安装
```bash
pip install -v git+https://github.com/ModelTC/LightX2V.git
```

### 从源码构建
```bash
git clone https://github.com/ModelTC/LightX2V.git
cd LightX2V
uv pip install -v . # pip install -v .
```

### (可选)安装注意力/量化算子
注意力算子安装说明请参考我们的文档:**[英文文档](https://lightx2v-en.readthedocs.io/en/latest/getting_started/quickstart.html#step-4-install-attention-operators) | [中文文档](https://lightx2v-zhcn.readthedocs.io/zh-cn/latest/getting_started/quickstart.html#id9)**

### 快速开始
```python
# examples/hunyuan_video/hunyuan_t2v.py
from lightx2v import LightX2VPipeline

pipe = LightX2VPipeline(
    model_path="/path/to/ckpts/hunyuanvideo-1.5/",
    model_cls="hunyuan_video_1.5",
    transformer_model_name="720p_t2v",
    task="t2v",
)

pipe.create_generator(
    attn_mode="sage_attn2",
    infer_steps=50,
    num_frames=121,
    guidance_scale=6.0,
    sample_shift=9.0,
    aspect_ratio="16:9",
    fps=24,
)

seed = 123
prompt = "Two anthropomorphic cats in comfy boxing gear and bright gloves fight intensely on a spotlighted stage."
negative_prompt = ""
save_result_path="/path/to/save_results/output.mp4"

pipe.generate(
    seed=seed,
    prompt=prompt,
    negative_prompt=negative_prompt,
    save_result_path=save_result_path,
)

```

> 💡 **更多示例**: 更多使用案例,包括量化、卸载、缓存等进阶配置,请参考 [examples 目录](https://github.com/ModelTC/LightX2V/tree/main/examples)。

gushiqiao's avatar
gushiqiao committed
83
## 🤖 支持的模型生态
Harahan's avatar
Harahan committed
84

gushiqiao's avatar
gushiqiao committed
85
### 官方开源模型
86
-[HunyuanVideo-1.5](https://huggingface.co/tencent/HunyuanVideo-1.5)
helloyongyang's avatar
helloyongyang committed
87
-[Wan2.1 & Wan2.2](https://huggingface.co/Wan-AI/)
gushiqiao's avatar
gushiqiao committed
88
89
-[Qwen-Image](https://huggingface.co/Qwen/Qwen-Image)
-[Qwen-Image-Edit](https://huggingface.co/spaces/Qwen/Qwen-Image-Edit)
Watebear's avatar
Watebear committed
90
-[Qwen-Image-Edit-2509](https://huggingface.co/Qwen/Qwen-Image-Edit-2509)
Harahan's avatar
Harahan committed
91

gushiqiao's avatar
gushiqiao committed
92
93
94
95
96
### 量化模型和蒸馏模型/Lora (**🚀 推荐:4步推理**)
-[Wan2.1-Distill-Models](https://huggingface.co/lightx2v/Wan2.1-Distill-Models)
-[Wan2.2-Distill-Models](https://huggingface.co/lightx2v/Wan2.2-Distill-Models)
-[Wan2.1-Distill-Loras](https://huggingface.co/lightx2v/Wan2.1-Distill-Loras)
-[Wan2.2-Distill-Loras](https://huggingface.co/lightx2v/Wan2.2-Distill-Loras)
gushiqiao's avatar
gushiqiao committed
97

98
99
### 轻量级自编码器模型(**🚀 推荐:推理快速 + 内存占用低**)
-[Autoencoders](https://huggingface.co/lightx2v/Autoencoders)
helloyongyang's avatar
helloyongyang committed
100
101
🔔 可以关注我们的[HuggingFace主页](https://huggingface.co/lightx2v),及时获取我们团队的模型。

gushiqiao's avatar
gushiqiao committed
102
103
### 自回归模型
-[Wan2.1-T2V-CausVid](https://huggingface.co/lightx2v/Wan2.1-T2V-14B-CausVid)
gushiqiao's avatar
gushiqiao committed
104
-[Self-Forcing](https://github.com/guandeh17/Self-Forcing)
Watebear's avatar
Watebear committed
105
-[Matrix-Game-2.0](https://huggingface.co/Skywork/Matrix-Game-2.0)
gushiqiao's avatar
gushiqiao committed
106

gushiqiao's avatar
gushiqiao committed
107
💡 参考[模型结构文档](https://lightx2v-zhcn.readthedocs.io/zh-cn/latest/getting_started/model_structure.html)快速上手 LightX2V
gushiqiao's avatar
gushiqiao committed
108

gushiqiao's avatar
gushiqiao committed
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
## 🚀 前端展示

我们提供了多种前端界面部署方式:

- **🎨 Gradio界面**: 简洁易用的Web界面,适合快速体验和原型开发
  - 📖 [Gradio部署文档](https://lightx2v-zhcn.readthedocs.io/zh-cn/latest/deploy_guides/deploy_gradio.html)
- **🎯 ComfyUI界面**: 强大的节点式工作流界面,支持复杂的视频生成任务
  - 📖 [ComfyUI部署文档](https://lightx2v-zhcn.readthedocs.io/zh-cn/latest/deploy_guides/deploy_comfyui.html)
- **🚀 Windows一键部署**: 专为Windows用户设计的便捷部署方案,支持自动环境配置和智能参数优化
  - 📖 [Windows一键部署文档](https://lightx2v-zhcn.readthedocs.io/zh-cn/latest/deploy_guides/deploy_local_windows.html)

**💡 推荐方案**:
- **首次使用**: 建议选择Windows一键部署方案
- **高级用户**: 推荐使用ComfyUI界面获得更多自定义选项
- **快速体验**: Gradio界面提供最直观的操作体验

gushiqiao's avatar
gushiqiao committed
125
126
127
## 🚀 核心特性

### 🎯 **极致性能优化**
gushiqiao's avatar
gushiqiao committed
128
- **🔥 SOTA推理速度**: 通过步数蒸馏和系统优化实现**20倍**极速加速(单GPU)
gushiqiao's avatar
gushiqiao committed
129
130
131
132
133
134
135
136
137
138
139
140
141
- **⚡️ 革命性4步蒸馏**: 将原始40-50步推理压缩至仅需4步,且无需CFG配置
- **🛠️ 先进算子支持**: 集成顶尖算子,包括[Sage Attention](https://github.com/thu-ml/SageAttention)[Flash Attention](https://github.com/Dao-AILab/flash-attention)[Radial Attention](https://github.com/mit-han-lab/radial-attention)[q8-kernel](https://github.com/KONAKONA666/q8_kernels)[sgl-kernel](https://github.com/sgl-project/sglang/tree/main/sgl-kernel)[vllm](https://github.com/vllm-project/vllm)

### 💾 **资源高效部署**
- **💡 突破硬件限制**: **仅需8GB显存 + 16GB内存**即可运行14B模型生成480P/720P视频
- **🔧 智能参数卸载**: 先进的磁盘-CPU-GPU三级卸载架构,支持阶段/块级别的精细化管理
- **⚙️ 全面量化支持**: 支持`w8a8-int8``w8a8-fp8``w4a4-nvfp4`等多种量化策略

### 🎨 **丰富功能生态**
- **📈 智能特征缓存**: 智能缓存机制,消除冗余计算,提升效率
- **🔄 并行推理加速**: 多GPU并行处理,显著提升性能表现
- **📱 灵活部署选择**: 支持Gradio、服务化部署、ComfyUI等多种部署方式
- **🎛️ 动态分辨率推理**: 自适应分辨率调整,优化生成质量
gushiqiao's avatar
gushiqiao committed
142
- **🎞️ 视频帧插值**: 基于RIFE的帧插值技术,实现流畅的帧率提升
gushiqiao's avatar
gushiqiao committed
143
144


gushiqiao's avatar
gushiqiao committed
145
146
147
148
149
## 🏆 性能基准测试

详细的性能指标和对比分析,请参考我们的[基准测试文档](https://github.com/ModelTC/LightX2V/blob/main/docs/ZH_CN/source/getting_started/benchmark_source.md)

[详细服务部署指南 →](https://lightx2v-zhcn.readthedocs.io/zh-cn/latest/deploy_guides/deploy_service.html)
Harahan's avatar
Harahan committed
150

gushiqiao's avatar
gushiqiao committed
151
## 📚 技术文档
Harahan's avatar
Harahan committed
152

gushiqiao's avatar
gushiqiao committed
153
154
155
156
157
158
### 📖 **方法教程**
- [模型量化](https://lightx2v-zhcn.readthedocs.io/zh-cn/latest/method_tutorials/quantization.html) - 量化策略全面指南
- [特征缓存](https://lightx2v-zhcn.readthedocs.io/zh-cn/latest/method_tutorials/cache.html) - 智能缓存机制详解
- [注意力机制](https://lightx2v-zhcn.readthedocs.io/zh-cn/latest/method_tutorials/attention.html) - 前沿注意力算子
- [参数卸载](https://lightx2v-zhcn.readthedocs.io/zh-cn/latest/method_tutorials/offload.html) - 三级存储架构
- [并行推理](https://lightx2v-zhcn.readthedocs.io/zh-cn/latest/method_tutorials/parallel.html) - 多GPU加速策略
helloyongyang's avatar
helloyongyang committed
159
- [变分辨率推理](https://lightx2v-zhcn.readthedocs.io/zh-cn/latest/method_tutorials/changing_resolution.html) - U型分辨率策略
gushiqiao's avatar
gushiqiao committed
160
- [步数蒸馏](https://lightx2v-zhcn.readthedocs.io/zh-cn/latest/method_tutorials/step_distill.html) - 4步推理技术
PengGao's avatar
PengGao committed
161
- [视频帧插值](https://lightx2v-zhcn.readthedocs.io/zh-cn/latest/method_tutorials/video_frame_interpolation.html) - 基于RIFE的帧插值技术
gushiqiao's avatar
gushiqiao committed
162
163
164
165
166
167

### 🛠️ **部署指南**
- [低资源场景部署](https://lightx2v-zhcn.readthedocs.io/zh-cn/latest/deploy_guides/for_low_resource.html) - 优化的8GB显存解决方案
- [低延迟场景部署](https://lightx2v-zhcn.readthedocs.io/zh-cn/latest/deploy_guides/for_low_latency.html) - 极速推理优化
- [Gradio部署](https://lightx2v-zhcn.readthedocs.io/zh-cn/latest/deploy_guides/deploy_gradio.html) - Web界面搭建
- [服务化部署](https://lightx2v-zhcn.readthedocs.io/zh-cn/latest/deploy_guides/deploy_service.html) - 生产级API服务部署
helloyongyang's avatar
helloyongyang committed
168
- [Lora模型部署](https://lightx2v-zhcn.readthedocs.io/zh-cn/latest/deploy_guides/lora_deploy.html) - Lora灵活部署
gushiqiao's avatar
gushiqiao committed
169
170
171
172
173
174
175

## 🧾 代码贡献指南

我们通过自动化的预提交钩子来保证代码质量,确保项目代码格式的一致性。

> [!TIP]
> **安装说明:**
Harahan's avatar
Harahan committed
176
>
gushiqiao's avatar
gushiqiao committed
177
> 1. 安装必要的依赖:
Harahan's avatar
Harahan committed
178
179
> ```shell
> pip install ruff pre-commit
gushiqiao's avatar
gushiqiao committed
180
> ```
Harahan's avatar
Harahan committed
181
>
gushiqiao's avatar
gushiqiao committed
182
> 2. 提交前运行:
Harahan's avatar
Harahan committed
183
184
> ```shell
> pre-commit run --all-files
gushiqiao's avatar
gushiqiao committed
185
> ```
Harahan's avatar
Harahan committed
186

gushiqiao's avatar
gushiqiao committed
187
感谢您为LightX2V的改进做出贡献!
Harahan's avatar
Harahan committed
188
189
190

## 🤝 致谢

gushiqiao's avatar
gushiqiao committed
191
我们向所有启发和促进LightX2V开发的模型仓库和研究社区表示诚挚的感谢。此框架基于开源社区的集体努力而构建。
Harahan's avatar
Harahan committed
192

gushiqiao's avatar
gushiqiao committed
193
## 🌟 Star 历史
Harahan's avatar
Harahan committed
194
195
196
197
198

[![Star History Chart](https://api.star-history.com/svg?repos=ModelTC/lightx2v&type=Timeline)](https://star-history.com/#ModelTC/lightx2v&Timeline)

## ✏️ 引用

gushiqiao's avatar
gushiqiao committed
199
如果您发现LightX2V对您的研究有用,请考虑引用我们的工作:
Harahan's avatar
Harahan committed
200
201
202

```bibtex
@misc{lightx2v,
gushiqiao's avatar
gushiqiao committed
203
 author = {LightX2V Contributors},
helloyongyang's avatar
helloyongyang committed
204
 title = {LightX2V: Light Video Generation Inference Framework},
gushiqiao's avatar
gushiqiao committed
205
206
207
208
 year = {2025},
 publisher = {GitHub},
 journal = {GitHub repository},
 howpublished = {\url{https://github.com/ModelTC/lightx2v}},
Harahan's avatar
Harahan committed
209
210
}
```
gushiqiao's avatar
gushiqiao committed
211
212
213
214
215
216
217
218
219
220
221
222

## 📞 联系与支持

如有任何问题、建议或需要支持,欢迎通过以下方式联系我们:
- 🐛 [GitHub Issues](https://github.com/ModelTC/lightx2v/issues) - 错误报告和功能请求
- 💬 [GitHub Discussions](https://github.com/ModelTC/lightx2v/discussions) - 社区讨论和问答

---

<div align="center">
由 LightX2V 团队用 ❤️ 构建
</div>