"tests/test_device/vscode:/vscode.git/clone" did not exist on "fdeee889589df413e368b05fd702b5c3f76ac7d3"
Commit 2829f746 authored by helloyongyang's avatar helloyongyang
Browse files

update doc

parent 4659c635
......@@ -48,6 +48,7 @@ Documentation
Attention Module <method_tutorials/attention.md>
Offloading <method_tutorials/offload.md>
Parallel Inference <method_tutorials/parallel.md>
Changing Resolution Inference <method_tutorials/changing_resolution.md>
Step Distill <method_tutorials/step_distill.md>
Autoregressive Distill <method_tutorials/autoregressive_distill.md>
......
# Changing Resolution Inference
## Overview
Changing resolution inference is a technical strategy for optimizing the denoising process. It improves computational efficiency while maintaining generation quality by using different resolutions at different denoising stages. The core idea is to use lower resolution for rough denoising in the early stages of the denoising process, then switch to normal resolution for fine-tuning in the later stages.
## Technical Principles
### Phased Denoising Strategy
Changing resolution inference is based on the following observations:
- **Early-stage denoising**: Mainly processes rough noise and overall structure, doesn't require excessive detail information
- **Late-stage denoising**: Focuses on detail optimization and high-frequency information recovery, requires complete resolution information
### Resolution Switching Mechanism
1. **Low Resolution Stage** (Early stage)
- Downsample the input to lower resolution (e.g., 0.75 of original size)
- Execute initial denoising steps
- Quickly remove most noise and establish basic structure
2. **Normal Resolution Stage** (Late stage)
- Upsample the denoising result from the first step back to original resolution
- Continue executing remaining denoising steps
- Recover detail information and complete fine-tuning
## Usage
The config files for changing resolution inference are available [here](https://github.com/ModelTC/LightX2V/tree/main/configs/changing_resolution)
By specifying --config_json to the specific config file, you can test changing resolution inference.
You can refer to [this script](https://github.com/ModelTC/LightX2V/blob/main/scripts/wan/run_wan_t2v_changing_resolution.sh).
......@@ -49,6 +49,7 @@ HuggingFace: https://huggingface.co/lightx2v
注意力机制 <method_tutorials/attention.md>
参数卸载 <method_tutorials/offload.md>
并行推理 <method_tutorials/parallel.md>
变分辨率推理 <method_tutorials/changing_resolution.md>
步数蒸馏 <method_tutorials/step_distill.md>
自回归蒸馏 <method_tutorials/autoregressive_distill.md>
......
# 变分辨率推理
## 概述
变分辨率推理是一种优化去噪过程的技术策略,通过在不同的去噪阶段采用不同的分辨率来提升计算效率并保持生成质量。该方法的核心思想是:在去噪过程的前期使用较低分辨率进行粗略去噪,在后期切换到正常分辨率进行精细化处理。
## 技术原理
### 分阶段去噪策略
变分辨率推理基于以下观察:
- **前期去噪**:主要处理粗糙的噪声和整体结构,不需要过多的细节信息
- **后期去噪**:专注于细节优化和高频信息恢复,需要完整的分辨率信息
### 分辨率切换机制
1. **低分辨率阶段**(前期)
- 将输入降采样到较低分辨率(如原尺寸的0.75)
- 执行初始的去噪步骤
- 快速移除大部分噪声,建立基本结构
2. **正常分辨率阶段**(后期)
- 将第一步的去噪结果上采样回原始分辨率
- 继续执行剩余的去噪步骤
- 恢复细节信息,完成精细化处理
## 使用方式
变分辨率推理的config文件在[这里](https://github.com/ModelTC/LightX2V/tree/main/configs/changing_resolution)
通过指定--config_json到具体的config文件,即可以测试变分辨率推理。
可以参考[该脚本](https://github.com/ModelTC/LightX2V/blob/main/scripts/wan/run_wan_t2v_changing_resolution.sh)
......@@ -5,8 +5,8 @@ from lightx2v.models.schedulers.wan.scheduler import WanScheduler
class WanScheduler4ChangingResolution(WanScheduler):
def __init__(self, config):
super().__init__(config)
self.resolution_rate = 0.75
self.changing_resolution_steps = 25
self.resolution_rate = config.get("resolution_rate", 0.75)
self.changing_resolution_steps = config.get("changing_resolution_steps", config.infer_steps // 2)
def prepare_latents(self, target_shape, dtype=torch.float32):
self.latents = torch.randn(
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment