Changing resolution inference is a technical strategy for optimizing the denoising process. It improves computational efficiency while maintaining generation quality by using different resolutions at different denoising stages. The core idea is to use lower resolution for rough denoising in the early stages of the denoising process, then switch to normal resolution for fine-tuning in the later stages.
Variable resolution inference is a technical strategy for optimizing the denoising process. It improves computational efficiency while maintaining generation quality by using different resolutions at different stages of the denoising process. The core idea of this method is to use lower resolution for coarse denoising in the early stages and switch to normal resolution for fine processing in the later stages.
## Technical Principles
## Technical Principles
### Phased Denoising Strategy
### Multi-stage Denoising Strategy
Changing resolution inference is based on the following observations:
Variable resolution inference is based on the following observations:
-**Early-stage denoising**: Mainly processes rough noise and overall structure, doesn't require excessive detail information
-**Late-stage denoising**: Focuses on detail optimization and high-frequency information recovery, requires complete resolution information
-**Early-stage denoising**: Mainly handles coarse noise and overall structure, requiring less detailed information
-**Late-stage denoising**: Focuses on detail optimization and high-frequency information recovery, requiring complete resolution information
### Resolution Switching Mechanism
### Resolution Switching Mechanism
1.**Low Resolution Stage** (Early stage)
1.**Low-resolution stage** (early stage)
- Downsample the input to lower resolution (e.g., 0.75 of original size)
- Downsample the input to a lower resolution (e.g., 0.75x of original size)
- Execute initial denoising steps
- Execute initial denoising steps
- Quickly remove most noise and establish basic structure
- Quickly remove most noise and establish basic structure
2.**Normal Resolution Stage** (Late stage)
2.**Normal resolution stage** (late stage)
- Upsample the denoising result from the first step back to original resolution
- Upsample the denoising result from the first step back to original resolution
- Continue executing remaining denoising steps
- Continue executing remaining denoising steps
- Recover detail information and complete fine-tuning
- Restore detailed information and complete fine processing
### U-shaped Resolution Strategy
If resolution is reduced at the very beginning of the denoising steps, it may cause significant differences between the final generated video and the video generated through normal inference. Therefore, a U-shaped resolution strategy can be adopted, where the original resolution is maintained for the first few steps, then resolution is reduced for inference.
## Usage
## Usage
The config files for changing resolution inference are available [here](https://github.com/ModelTC/LightX2V/tree/main/configs/changing_resolution)
The config files for variable resolution inference are located [here](https://github.com/ModelTC/LightX2V/tree/main/configs/changing_resolution)
You can test variable resolution inference by specifying --config_json to the specific config file.
You can refer to the scripts [here](https://github.com/ModelTC/LightX2V/blob/main/scripts/changing_resolution) to run.
### Example 1:
```
{
"infer_steps": 50,
"changing_resolution": true,
"resolution_rate": [0.75],
"changing_resolution_steps": [25]
}
```
This means a total of 50 steps, with resolution at 0.75x original resolution from step 0 to 25, and original resolution from step 26 to the final step.
By specifying --config_json to the specific config file, you can test changing resolution inference.
### Example 2:
```
{
"infer_steps": 50,
"changing_resolution": true,
"resolution_rate": [1.0, 0.75],
"changing_resolution_steps": [10, 35]
}
```
You can refer to [this script](https://github.com/ModelTC/LightX2V/blob/main/scripts/wan/run_wan_t2v_changing_resolution.sh).
This means a total of 50 steps, with original resolution from step 0 to 10, 0.75x original resolution from step 11 to 35, and original resolution from step 36 to the final step.
--prompt"Summer beach vacation style, a white cat wearing sunglasses sits on a surfboard. The fluffy-furred feline gazes directly at the camera with a relaxed expression. Blurred beach scenery forms the background featuring crystal-clear waters, distant green hills, and a blue sky dotted with white clouds. The cat assumes a naturally relaxed posture, as if savoring the sea breeze and warm sunlight. A close-up shot highlights the feline's intricate details and the refreshing atmosphere of the seaside."\
--prompt"Summer beach vacation style, a white cat wearing sunglasses sits on a surfboard. The fluffy-furred feline gazes directly at the camera with a relaxed expression. Blurred beach scenery forms the background featuring crystal-clear waters, distant green hills, and a blue sky dotted with white clouds. The cat assumes a naturally relaxed posture, as if savoring the sea breeze and warm sunlight. A close-up shot highlights the feline's intricate details and the refreshing atmosphere of the seaside."\
