README.md 10.5 KB
Newer Older
Harahan's avatar
Harahan committed
1
<div align="center" style="font-family: charter;">
helloyongyang's avatar
helloyongyang committed
2
  <h1>⚡️ LightX2V:<br> Light Video Generation Inference Framework</h1>
helloyongyang's avatar
helloyongyang committed
3

Yang Yong(雍洋)'s avatar
Yang Yong(雍洋) committed
4
<img alt="logo" src="assets/img_lightx2v.png" width=75%></img>
helloyongyang's avatar
helloyongyang committed
5

helloyongyang's avatar
helloyongyang committed
6
[![License](https://img.shields.io/badge/License-Apache_2.0-blue.svg)](https://opensource.org/licenses/Apache-2.0)
PengGao's avatar
PengGao committed
7
[![Ask DeepWiki](https://deepwiki.com/badge.svg)](https://deepwiki.com/ModelTC/lightx2v)
helloyongyang's avatar
helloyongyang committed
8
9
[![Doc](https://img.shields.io/badge/docs-English-99cc2)](https://lightx2v-en.readthedocs.io/en/latest)
[![Doc](https://img.shields.io/badge/文档-中文-99cc2)](https://lightx2v-zhcn.readthedocs.io/zh-cn/latest)
helloyongyang's avatar
helloyongyang committed
10
[![Papers](https://img.shields.io/badge/论文集-中文-99cc2)](https://lightx2v-papers-zhcn.readthedocs.io/zh-cn/latest)
Yang Yong (雍洋)'s avatar
Yang Yong (雍洋) committed
11
[![Docker](https://img.shields.io/badge/Docker-2496ED?style=flat&logo=docker&logoColor=white)](https://hub.docker.com/r/lightx2v/lightx2v/tags)
PengGao's avatar
PengGao committed
12

helloyongyang's avatar
helloyongyang committed
13
**\[ English | [中文](README_zh.md) \]**
Harahan's avatar
Harahan committed
14

helloyongyang's avatar
helloyongyang committed
15
16
17
</div>

--------------------------------------------------------------------------------
helloyongyang's avatar
helloyongyang committed
18

gushiqiao's avatar
gushiqiao committed
19
**LightX2V** is an advanced lightweight video generation inference framework engineered to deliver efficient, high-performance video synthesis solutions. This unified platform integrates multiple state-of-the-art video generation techniques, supporting diverse generation tasks including text-to-video (T2V) and image-to-video (I2V). **X2V represents the transformation of different input modalities (X, such as text or images) into video output (V)**.
helloyongyang's avatar
helloyongyang committed
20

Yang Yong (雍洋)'s avatar
Yang Yong (雍洋) committed
21
22
## :fire: Latest News

Yang Yong (雍洋)'s avatar
Yang Yong (雍洋) committed
23
- **November 21, 2025:** 🚀 We support the HunyuanVideo1.5 video generation model since Day 0. With the same number of GPUs, LightX2V can achieve a speed improvement of over 2 times and supports deployment on GPUs with lower memory (such as the 24GB RTX 4090). It also supports CFG/Ulysses parallelism, efficient offloading, TeaCache/MagCache technologies, and more. We will soon update our models on our [HuggingFace page](https://huggingface.co/lightx2v), including quantization, step distillation, VAE distillation, and other related models. Refer to [this](https://github.com/ModelTC/LightX2V/tree/main/scripts/hunyuan_video_15) for usage tutorials.
Yang Yong (雍洋)'s avatar
Yang Yong (雍洋) committed
24

helloyongyang's avatar
helloyongyang committed
25
26
27
28
## 💡 Quick Start

For comprehensive usage instructions, please refer to our documentation: **[English Docs](https://lightx2v-en.readthedocs.io/en/latest/) | [中文文档](https://lightx2v-zhcn.readthedocs.io/zh-cn/latest/)**

helloyongyang's avatar
helloyongyang committed
29

gushiqiao's avatar
gushiqiao committed
30
31
32
## 🤖 Supported Model Ecosystem

### Official Open-Source Models
33
-[HunyuanVideo-1.5](https://huggingface.co/tencent/HunyuanVideo-1.5)
helloyongyang's avatar
helloyongyang committed
34
-[Wan2.1 & Wan2.2](https://huggingface.co/Wan-AI/)
gushiqiao's avatar
gushiqiao committed
35
36
-[Qwen-Image](https://huggingface.co/Qwen/Qwen-Image)
-[Qwen-Image-Edit](https://huggingface.co/spaces/Qwen/Qwen-Image-Edit)
Watebear's avatar
Watebear committed
37
-[Qwen-Image-Edit-2509](https://huggingface.co/Qwen/Qwen-Image-Edit-2509)
helloyongyang's avatar
helloyongyang committed
38

gushiqiao's avatar
gushiqiao committed
39
### Quantized and Distilled Models/LoRAs (**🚀 Recommended: 4-step inference**)
40
-[Hy1.5-Quantized-Models](https://huggingface.co/lightx2v/Hy1.5-Quantized-Models)
gushiqiao's avatar
gushiqiao committed
41
42
43
44
-[Wan2.1-Distill-Models](https://huggingface.co/lightx2v/Wan2.1-Distill-Models)
-[Wan2.2-Distill-Models](https://huggingface.co/lightx2v/Wan2.2-Distill-Models)
-[Wan2.1-Distill-Loras](https://huggingface.co/lightx2v/Wan2.1-Distill-Loras)
-[Wan2.2-Distill-Loras](https://huggingface.co/lightx2v/Wan2.2-Distill-Loras)
gushiqiao's avatar
gushiqiao committed
45

46
47
48
### Lightweight Autoencoder Models (**🚀 Recommended: fast inference & low memory usage**)
-[Autoencoders](https://huggingface.co/lightx2v/Autoencoders)

helloyongyang's avatar
helloyongyang committed
49
50
🔔 Follow our [HuggingFace page](https://huggingface.co/lightx2v) for the latest model releases from our team.

gushiqiao's avatar
gushiqiao committed
51
52
### Autoregressive Models
-[Wan2.1-T2V-CausVid](https://huggingface.co/lightx2v/Wan2.1-T2V-14B-CausVid)
gushiqiao's avatar
gushiqiao committed
53
-[Self-Forcing](https://github.com/guandeh17/Self-Forcing)
Watebear's avatar
Watebear committed
54
-[Matrix-Game-2.0](https://huggingface.co/Skywork/Matrix-Game-2.0)
gushiqiao's avatar
gushiqiao committed
55

gushiqiao's avatar
gushiqiao committed
56
💡 Refer to the [Model Structure Documentation](https://lightx2v-en.readthedocs.io/en/latest/getting_started/model_structure.html) to quickly get started with LightX2V
gushiqiao's avatar
gushiqiao committed
57

gushiqiao's avatar
gushiqiao committed
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
## 🚀 Frontend Interfaces

We provide multiple frontend interface deployment options:

- **🎨 Gradio Interface**: Clean and user-friendly web interface, perfect for quick experience and prototyping
  - 📖 [Gradio Deployment Guide](https://lightx2v-en.readthedocs.io/en/latest/deploy_guides/deploy_gradio.html)
- **🎯 ComfyUI Interface**: Powerful node-based workflow interface, supporting complex video generation tasks
  - 📖 [ComfyUI Deployment Guide](https://lightx2v-en.readthedocs.io/en/latest/deploy_guides/deploy_comfyui.html)
- **🚀 Windows One-Click Deployment**: Convenient deployment solution designed for Windows users, featuring automatic environment configuration and intelligent parameter optimization
  - 📖 [Windows One-Click Deployment Guide](https://lightx2v-en.readthedocs.io/en/latest/deploy_guides/deploy_local_windows.html)

**💡 Recommended Solutions**:
- **First-time Users**: We recommend the Windows one-click deployment solution
- **Advanced Users**: We recommend the ComfyUI interface for more customization options
- **Quick Experience**: The Gradio interface provides the most intuitive operation experience
gushiqiao's avatar
gushiqiao committed
73
74
75
76

## 🚀 Core Features

### 🎯 **Ultimate Performance Optimization**
gushiqiao's avatar
gushiqiao committed
77
- **🔥 SOTA Inference Speed**: Achieve **~20x** acceleration via step distillation and system optimization (single GPU)
gushiqiao's avatar
gushiqiao committed
78
79
80
81
82
83
84
85
86
87
88
89
90
- **⚡️ Revolutionary 4-Step Distillation**: Compress original 40-50 step inference to just 4 steps without CFG requirements
- **🛠️ Advanced Operator Support**: Integrated with cutting-edge operators including [Sage Attention](https://github.com/thu-ml/SageAttention), [Flash Attention](https://github.com/Dao-AILab/flash-attention), [Radial Attention](https://github.com/mit-han-lab/radial-attention), [q8-kernel](https://github.com/KONAKONA666/q8_kernels), [sgl-kernel](https://github.com/sgl-project/sglang/tree/main/sgl-kernel), [vllm](https://github.com/vllm-project/vllm)

### 💾 **Resource-Efficient Deployment**
- **💡 Breaking Hardware Barriers**: Run 14B models for 480P/720P video generation with only **8GB VRAM + 16GB RAM**
- **🔧 Intelligent Parameter Offloading**: Advanced disk-CPU-GPU three-tier offloading architecture with phase/block-level granular management
- **⚙️ Comprehensive Quantization**: Support for `w8a8-int8`, `w8a8-fp8`, `w4a4-nvfp4` and other quantization strategies

### 🎨 **Rich Feature Ecosystem**
- **📈 Smart Feature Caching**: Intelligent caching mechanisms to eliminate redundant computations
- **🔄 Parallel Inference**: Multi-GPU parallel processing for enhanced performance
- **📱 Flexible Deployment Options**: Support for Gradio, service deployment, ComfyUI and other deployment methods
- **🎛️ Dynamic Resolution Inference**: Adaptive resolution adjustment for optimal generation quality
PengGao's avatar
PengGao committed
91
- **🎞️ Video Frame Interpolation**: RIFE-based frame interpolation for smooth frame rate enhancement
gushiqiao's avatar
gushiqiao committed
92
93


gushiqiao's avatar
gushiqiao committed
94
95
96
97
98
99
100
101
102
103
104
105
106
107
## 🏆 Performance Benchmarks

For detailed performance metrics and comparisons, please refer to our [benchmark documentation](https://github.com/ModelTC/LightX2V/blob/main/docs/EN/source/getting_started/benchmark_source.md).

[Detailed Service Deployment Guide →](https://lightx2v-en.readthedocs.io/en/latest/deploy_guides/deploy_service.html)

## 📚 Technical Documentation

### 📖 **Method Tutorials**
- [Model Quantization](https://lightx2v-en.readthedocs.io/en/latest/method_tutorials/quantization.html) - Comprehensive guide to quantization strategies
- [Feature Caching](https://lightx2v-en.readthedocs.io/en/latest/method_tutorials/cache.html) - Intelligent caching mechanisms
- [Attention Mechanisms](https://lightx2v-en.readthedocs.io/en/latest/method_tutorials/attention.html) - State-of-the-art attention operators
- [Parameter Offloading](https://lightx2v-en.readthedocs.io/en/latest/method_tutorials/offload.html) - Three-tier storage architecture
- [Parallel Inference](https://lightx2v-en.readthedocs.io/en/latest/method_tutorials/parallel.html) - Multi-GPU acceleration strategies
helloyongyang's avatar
helloyongyang committed
108
- [Changing Resolution Inference](https://lightx2v-en.readthedocs.io/en/latest/method_tutorials/changing_resolution.html) - U-shaped resolution strategy
gushiqiao's avatar
gushiqiao committed
109
- [Step Distillation](https://lightx2v-en.readthedocs.io/en/latest/method_tutorials/step_distill.html) - 4-step inference technology
helloyongyang's avatar
helloyongyang committed
110
- [Video Frame Interpolation](https://lightx2v-en.readthedocs.io/en/latest/method_tutorials/video_frame_interpolation.html) - Base on the RIFE technology
gushiqiao's avatar
gushiqiao committed
111
112
113
114
115
116

### 🛠️ **Deployment Guides**
- [Low-Resource Deployment](https://lightx2v-en.readthedocs.io/en/latest/deploy_guides/for_low_resource.html) - Optimized 8GB VRAM solutions
- [Low-Latency Deployment](https://lightx2v-en.readthedocs.io/en/latest/deploy_guides/for_low_latency.html) - Ultra-fast inference optimization
- [Gradio Deployment](https://lightx2v-en.readthedocs.io/en/latest/deploy_guides/deploy_gradio.html) - Web interface setup
- [Service Deployment](https://lightx2v-en.readthedocs.io/en/latest/deploy_guides/deploy_service.html) - Production API service deployment
helloyongyang's avatar
helloyongyang committed
117
- [Lora Model Deployment](https://lightx2v-en.readthedocs.io/en/latest/deploy_guides/lora_deploy.html) - Flexible Lora deployment
helloyongyang's avatar
helloyongyang committed
118

Harahan's avatar
Harahan committed
119
## 🧾 Contributing Guidelines
helloyongyang's avatar
helloyongyang committed
120

gushiqiao's avatar
gushiqiao committed
121
We maintain code quality through automated pre-commit hooks to ensure consistent formatting across the project.
helloyongyang's avatar
helloyongyang committed
122

Harahan's avatar
Harahan committed
123
> [!TIP]
gushiqiao's avatar
gushiqiao committed
124
> **Setup Instructions:**
Harahan's avatar
Harahan committed
125
>
gushiqiao's avatar
gushiqiao committed
126
> 1. Install required dependencies:
Harahan's avatar
Harahan committed
127
128
> ```shell
> pip install ruff pre-commit
gushiqiao's avatar
gushiqiao committed
129
> ```
Harahan's avatar
Harahan committed
130
>
gushiqiao's avatar
gushiqiao committed
131
> 2. Run before committing:
Harahan's avatar
Harahan committed
132
133
> ```shell
> pre-commit run --all-files
gushiqiao's avatar
gushiqiao committed
134
> ```
Dongz's avatar
Dongz committed
135

gushiqiao's avatar
gushiqiao committed
136
We appreciate your contributions to making LightX2V better!
Dongz's avatar
Dongz committed
137

Harahan's avatar
Harahan committed
138
## 🤝 Acknowledgments
Dongz's avatar
Dongz committed
139

gushiqiao's avatar
gushiqiao committed
140
We extend our gratitude to all the model repositories and research communities that inspired and contributed to the development of LightX2V. This framework builds upon the collective efforts of the open-source community.
Dongz's avatar
Dongz committed
141

Harahan's avatar
Harahan committed
142
## 🌟 Star History
Dongz's avatar
Dongz committed
143

gushiqiao's avatar
gushiqiao committed
144
[![Star History Chart](https://api.star-history.com/svg?repos=ModelTC/lightx2v&type=Timeline)](https://star-history.com/#ModelTC/lightx2v&Timeline)
helloyongyang's avatar
helloyongyang committed
145

Harahan's avatar
Harahan committed
146
## ✏️ Citation
helloyongyang's avatar
helloyongyang committed
147

gushiqiao's avatar
gushiqiao committed
148
If you find LightX2V useful in your research, please consider citing our work:
helloyongyang's avatar
helloyongyang committed
149

gushiqiao's avatar
gushiqiao committed
150
```bibtex
Harahan's avatar
Harahan committed
151
@misc{lightx2v,
gushiqiao's avatar
gushiqiao committed
152
 author = {LightX2V Contributors},
helloyongyang's avatar
helloyongyang committed
153
 title = {LightX2V: Light Video Generation Inference Framework},
Harahan's avatar
Harahan committed
154
 year = {2025},
Harahan's avatar
Harahan committed
155
156
157
158
159
 publisher = {GitHub},
 journal = {GitHub repository},
 howpublished = {\url{https://github.com/ModelTC/lightx2v}},
}
```
gushiqiao's avatar
gushiqiao committed
160
161
162
163
164
165
166
167
168
169
170
171

## 📞 Contact & Support

For questions, suggestions, or support, please feel free to reach out through:
- 🐛 [GitHub Issues](https://github.com/ModelTC/lightx2v/issues) - Bug reports and feature requests
- 💬 [GitHub Discussions](https://github.com/ModelTC/lightx2v/discussions) - Community discussions and Q&A

---

<div align="center">
Built with ❤️ by the LightX2V team
</div>