update scrips, configs, docs

configs/dist_infer/wan_i2v_dist_ring.json

+{
+    "infer_steps": 40,
+    "target_video_length": 81,
+    "target_height": 480,
+    "target_width": 832,
+    "self_attn_1_type": "flash_attn3",
+    "cross_attn_1_type": "flash_attn3",
+    "cross_attn_2_type": "flash_attn3",
+    "seed": 42,
+    "sample_guide_scale": 5,
+    "sample_shift": 5,
+    "enable_cfg": true,
+    "cpu_offload": false,
+    "parallel_attn_type": "ring",
+    "parallel_vae": true
+}

configs/dist_infer/wan_t2v_dist_ring.json

+{
+    "infer_steps": 50,
+    "target_video_length": 81,
+    "text_len": 512,
+    "target_height": 480,
+    "target_width": 832,
+    "self_attn_1_type": "flash_attn3",
+    "cross_attn_1_type": "flash_attn3",
+    "cross_attn_2_type": "flash_attn3",
+    "seed": 42,
+    "sample_guide_scale": 6,
+    "sample_shift": 8,
+    "enable_cfg": true,
+    "cpu_offload": false,
+    "parallel_attn_type": "ring",
+    "parallel_vae": true
+}

configs/wan/readme.md

-### TODO

docs/EN/source/method_tutorials/parallel.md

-# 并行推理
+# Parallel Inference
 
-xxx
+LightX2V supports distributed parallel inference, enabling the use of multiple GPUs for inference. The DiT part supports two parallel attention mechanisms: **Ulysses** and **Ring**, and also supports **VAE parallel inference**. Parallel inference significantly reduces inference time and alleviates the memory overhead of each GPU.
+
+## DiT Parallel Configuration
+
+DiT parallel is controlled by the `parallel_attn_type` parameter and supports two parallel attention mechanisms:
+
+### 1. Ulysses Parallel
+
+**Configuration:**
+```json
+{
+    "parallel_attn_type": "ulysses"
+}
+```
+
+### 2. Ring Parallel
+
+**Configuration:**
+```json
+{
+    "parallel_attn_type": "ring"
+}
+```
+
+## VAE Parallel Configuration
+
+VAE parallel is controlled by the `parallel_vae` parameter:
+
+```json
+{
+    "parallel_vae": true
+}
+```
+
+**Configuration Description:**
+- `parallel_vae: true`: Enable VAE parallel inference (recommended setting)
+- `parallel_vae: false`: Disable VAE parallel, use single GPU processing
+
+**Usage Recommendations:**
+- In multi-GPU environments, it is recommended to always enable VAE parallel
+- VAE parallel can be combined with any attention parallel method (Ulysses/Ring)
+- For memory-constrained scenarios, VAE parallel can significantly reduce memory usage
+
+## Usage
+
+The config files for parallel inference are available [here](https://github.com/ModelTC/lightx2v/tree/main/configs/dist_infer)
+
+By specifying --config_json to the specific config file, you can test parallel inference.
+
+Some running scripts are available [here](https://github.com/ModelTC/lightx2v/tree/main/scripts/dist_infer) for use.

docs/ZH_CN/source/method_tutorials/parallel.md

 # 并行推理
 
-xxx
+LightX2V 支持分布式并行推理，能够利用多个 GPU 进行推理。DiT部分支持两种并行注意力机制：**Ulysses** 和 **Ring**，同时还支持 **VAE 并行推理**。并行推理，显著降低推理耗时和减轻每个GPU的显存开销。
+
+## DiT 并行配置
+
+DiT 并行是通过 `parallel_attn_type` 参数控制的,支持两种并行注意力机制：
+
+### 1. Ulysses 并行
+
+**配置方式：**
+```json
+{
+    "parallel_attn_type": "ulysses"
+}
+```
+
+### 2. Ring 并行
+
+
+**配置方式：**
+```json
+{
+    "parallel_attn_type": "ring"
+}
+```
+
+
+## VAE 并行配置
+
+VAE 并行是通过 `parallel_vae` 参数控制：
+
+```json
+{
+    "parallel_vae": true
+}
+```
+
+**配置说明：**
+- `parallel_vae: true`：启用 VAE 并行推理（推荐设置）
+- `parallel_vae: false`：禁用 VAE 并行，使用单 GPU 处理
+
+**使用建议：**
+- 在多 GPU 环境下，建议始终启用 VAE 并行
+- VAE 并行可与任何注意力并行方式（Ulysses/Ring）组合使用
+- 对于内存受限的场景，VAE 并行可显著减少内存使用
+
+
+## 使用方式
+
+并行推理的config文件在[这里](https://github.com/ModelTC/lightx2v/tree/main/configs/dist_infer)
+
+通过指定--config_json到具体的config文件，即可以测试并行推理
+
+[这里](https://github.com/ModelTC/lightx2v/tree/main/scripts/dist_infer)有一些运行脚本供使用。

lightx2v/infer.py

@@ -86,6 +86,11 @@ def main():
 
         runner.run_pipeline()
 
+    # Clean up distributed process group
+    if dist.is_initialized():
+        dist.destroy_process_group()
+        logger.info("Distributed process group cleaned up")
+
 
 if __name__ == "__main__":
     main()

scripts/wan/run_wan_i2v_dist.sh → scripts/dist_infer/run_wan_i2v_dist_ring.sh

@@ -32,7 +32,7 @@ torchrun --nproc_per_node=4 -m lightx2v.infer \
 --model_cls wan2.1 \
 --task i2v \
 --model_path $model_path \
---config_json ${lightx2v_path}/configs/wan/wan_i2v_dist.json \
+--config_json ${lightx2v_path}/configs/dist_infer/wan_i2v_dist_ring.json \
 --prompt "Summer beach vacation style, a white cat wearing sunglasses sits on a surfboard. The fluffy-furred feline gazes directly at the camera with a relaxed expression. Blurred beach scenery forms the background featuring crystal-clear waters, distant green hills, and a blue sky dotted with white clouds. The cat assumes a naturally relaxed posture, as if savoring the sea breeze and warm sunlight. A close-up shot highlights the feline's intricate details and the refreshing atmosphere of the seaside." \
 --negative_prompt "镜头晃动，色调艳丽，过曝，静态，细节模糊不清，字幕，风格，作品，画作，画面，静止，整体发灰，最差质量，低质量，JPEG压缩残留，丑陋的，残缺的，多余的手指，画得不好的手部，画得不好的脸部，畸形的，毁容的，形态畸形的肢体，手指融合，静止不动的画面，杂乱的背景，三条腿，背景人很多，倒着走" \
 --image_path ${lightx2v_path}/assets/inputs/imgs/img_0.jpg \

scripts/dist_infer/run_wan_i2v_dist_ulysses.sh

+#!/bin/bash
+
+# set path and first
+lightx2v_path=
+model_path=
+
+# check section
+if [ -z "${CUDA_VISIBLE_DEVICES}" ]; then
+    cuda_devices=1,2,3,4
+    echo "Warn: CUDA_VISIBLE_DEVICES is not set, using default value: ${cuda_devices}, change at shell script or set env variable."
+    export CUDA_VISIBLE_DEVICES=${cuda_devices}
+fi
+
+if [ -z "${lightx2v_path}" ]; then
+    echo "Error: lightx2v_path is not set. Please set this variable first."
+    exit 1
+fi
+
+if [ -z "${model_path}" ]; then
+    echo "Error: model_path is not set. Please set this variable first."
+    exit 1
+fi
+
+export TOKENIZERS_PARALLELISM=false
+
+export PYTHONPATH=${lightx2v_path}:$PYTHONPATH
+export DTYPE=BF16
+export ENABLE_PROFILING_DEBUG=true
+export ENABLE_GRAPH_MODE=false
+
+torchrun --nproc_per_node=4 -m lightx2v.infer \
+--model_cls wan2.1 \
+--task i2v \
+--model_path $model_path \
+--config_json ${lightx2v_path}/configs/dist_infer/wan_i2v_dist_ulysses.json \
+--prompt "Summer beach vacation style, a white cat wearing sunglasses sits on a surfboard. The fluffy-furred feline gazes directly at the camera with a relaxed expression. Blurred beach scenery forms the background featuring crystal-clear waters, distant green hills, and a blue sky dotted with white clouds. The cat assumes a naturally relaxed posture, as if savoring the sea breeze and warm sunlight. A close-up shot highlights the feline's intricate details and the refreshing atmosphere of the seaside." \
+--negative_prompt "镜头晃动，色调艳丽，过曝，静态，细节模糊不清，字幕，风格，作品，画作，画面，静止，整体发灰，最差质量，低质量，JPEG压缩残留，丑陋的，残缺的，多余的手指，画得不好的手部，画得不好的脸部，畸形的，毁容的，形态畸形的肢体，手指融合，静止不动的画面，杂乱的背景，三条腿，背景人很多，倒着走" \
+--image_path ${lightx2v_path}/assets/inputs/imgs/img_0.jpg \
+--save_video_path ${lightx2v_path}/save_results/output_lightx2v_wan_i2v.mp4

scripts/wan/run_wan_t2v_dist.sh → scripts/dist_infer/run_wan_t2v_dist_ring.sh

@@ -32,7 +32,7 @@ torchrun --nproc_per_node=4 -m lightx2v.infer \
 --model_cls wan2.1 \
 --task t2v \
 --model_path $model_path \
---config_json ${lightx2v_path}/configs/wan/wan_t2v_dist.json \
+--config_json ${lightx2v_path}/configs/dist_infer/wan_t2v_dist_ring.json \
 --prompt "Two anthropomorphic cats in comfy boxing gear and bright gloves fight intensely on a spotlighted stage." \
 --negative_prompt "镜头晃动，色调艳丽，过曝，静态，细节模糊不清，字幕，风格，作品，画作，画面，静止，整体发灰，最差质量，低质量，JPEG压缩残留，丑陋的，残缺的，多余的手指，画得不好的手部，画得不好的脸部，畸形的，毁容的，形态畸形的肢体，手指融合，静止不动的画面，杂乱的背景，三条腿，背景人很多，倒着走" \
 --save_video_path ${lightx2v_path}/save_results/output_lightx2v_wan_t2v.mp4

scripts/dist_infer/run_wan_t2v_dist_ulysses.sh

+#!/bin/bash
+
+# set path and first
+lightx2v_path=
+model_path=
+
+# check section
+if [ -z "${CUDA_VISIBLE_DEVICES}" ]; then
+    cuda_devices=1,2,3,4
+    echo "Warn: CUDA_VISIBLE_DEVICES is not set, using default value: ${cuda_devices}, change at shell script or set env variable."
+    export CUDA_VISIBLE_DEVICES=${cuda_devices}
+fi
+
+if [ -z "${lightx2v_path}" ]; then
+    echo "Error: lightx2v_path is not set. Please set this variable first."
+    exit 1
+fi
+
+if [ -z "${model_path}" ]; then
+    echo "Error: model_path is not set. Please set this variable first."
+    exit 1
+fi
+
+export TOKENIZERS_PARALLELISM=false
+
+export PYTHONPATH=${lightx2v_path}:$PYTHONPATH
+export DTYPE=BF16
+export ENABLE_PROFILING_DEBUG=true
+export ENABLE_GRAPH_MODE=false
+
+torchrun --nproc_per_node=4 -m lightx2v.infer \
+--model_cls wan2.1 \
+--task t2v \
+--model_path $model_path \
+--config_json ${lightx2v_path}/configs/dist_infer/wan_t2v_dist_ulysses.json \
+--prompt "Two anthropomorphic cats in comfy boxing gear and bright gloves fight intensely on a spotlighted stage." \
+--negative_prompt "镜头晃动，色调艳丽，过曝，静态，细节模糊不清，字幕，风格，作品，画作，画面，静止，整体发灰，最差质量，低质量，JPEG压缩残留，丑陋的，残缺的，多余的手指，画得不好的手部，画得不好的脸部，畸形的，毁容的，形态畸形的肢体，手指融合，静止不动的画面，杂乱的背景，三条腿，背景人很多，倒着走" \
+--save_video_path ${lightx2v_path}/save_results/output_lightx2v_wan_t2v.mp4