Commit 348822d9 authored by helloyongyang's avatar helloyongyang
Browse files

update parallel doc

parent 65a46082
# Parallel Inference # Parallel Inference
LightX2V supports distributed parallel inference, enabling the use of multiple GPUs for inference. The DiT part supports two parallel attention mechanisms: **Ulysses** and **Ring**, and also supports **VAE parallel inference**. Parallel inference significantly reduces inference time and alleviates the memory overhead of each GPU. LightX2V supports distributed parallel inference, enabling the utilization of multiple GPUs for inference. The DiT component supports two parallel attention mechanisms: **Ulysses** and **Ring**, while also supporting **Cfg parallel inference**. Parallel inference significantly reduces inference time and alleviates memory overhead on each GPU.
## DiT Parallel Configuration ## DiT Parallel Configuration
DiT parallel is controlled by the `parallel_attn_type` parameter and supports two parallel attention mechanisms:
### 1. Ulysses Parallel ### 1. Ulysses Parallel
**Configuration:** **Configuration method:**
```json ```json
{ "parallel": {
"parallel_attn_type": "ulysses" "seq_p_size": 4,
} "seq_p_attn_type": "ulysses"
}
``` ```
### 2. Ring Parallel ### 2. Ring Parallel
**Configuration:** **Configuration method:**
```json ```json
{ "parallel": {
"parallel_attn_type": "ring" "seq_p_size": 4,
} "seq_p_attn_type": "ring"
}
``` ```
## VAE Parallel Configuration ## Cfg Parallel Configuration
VAE parallel is controlled by the `parallel_vae` parameter:
**Configuration method:**
```json ```json
{ "parallel": {
"parallel_vae": true "cfg_p_size": 2
} }
``` ```
**Configuration Description:** ## Hybrid Parallel Configuration
- `parallel_vae: true`: Enable VAE parallel inference (recommended setting)
- `parallel_vae: false`: Disable VAE parallel, use single GPU processing
**Usage Recommendations:** **Configuration method:**
- In multi-GPU environments, it is recommended to always enable VAE parallel ```json
- VAE parallel can be combined with any attention parallel method (Ulysses/Ring) "parallel": {
- For memory-constrained scenarios, VAE parallel can significantly reduce memory usage "seq_p_size": 4,
"seq_p_attn_type": "ulysses",
"cfg_p_size": 2
}
```
## Usage ## Usage
The config files for parallel inference are available [here](https://github.com/ModelTC/lightx2v/tree/main/configs/dist_infer) Parallel inference configuration files are available [here](https://github.com/ModelTC/lightx2v/tree/main/configs/dist_infer)
By specifying --config_json to the specific config file, you can test parallel inference. By specifying --config_json to a specific config file, you can test parallel inference.
Some running scripts are available [here](https://github.com/ModelTC/lightx2v/tree/main/scripts/dist_infer) for use. [Here](https://github.com/ModelTC/lightx2v/tree/main/scripts/dist_infer) are some run scripts for your use.
# 并行推理 # 并行推理
LightX2V 支持分布式并行推理,能够利用多个 GPU 进行推理。DiT部分支持两种并行注意力机制:**Ulysses****Ring**,同时还支持 **VAE 并行推理**。并行推理,显著降低推理耗时和减轻每个GPU的显存开销。 LightX2V 支持分布式并行推理,能够利用多个 GPU 进行推理。DiT部分支持两种并行注意力机制:**Ulysses****Ring**,同时还支持 **Cfg 并行推理**。并行推理,显著降低推理耗时和减轻每个GPU的显存开销。
## DiT 并行配置 ## DiT 并行配置
DiT 并行是通过 `parallel_attn_type` 参数控制的,支持两种并行注意力机制:
### 1. Ulysses 并行 ### 1. Ulysses 并行
**配置方式:** **配置方式:**
```json ```json
{ "parallel": {
"parallel_attn_type": "ulysses" "seq_p_size": 4,
} "seq_p_attn_type": "ulysses"
}
``` ```
### 2. Ring 并行 ### 2. Ring 并行
...@@ -20,30 +19,31 @@ DiT 并行是通过 `parallel_attn_type` 参数控制的,支持两种并行注 ...@@ -20,30 +19,31 @@ DiT 并行是通过 `parallel_attn_type` 参数控制的,支持两种并行注
**配置方式:** **配置方式:**
```json ```json
{ "parallel": {
"parallel_attn_type": "ring" "seq_p_size": 4,
} "seq_p_attn_type": "ring"
}
``` ```
## Cfg 并行配置
## VAE 并行配置 **配置方式:**
VAE 并行是通过 `parallel_vae` 参数控制:
```json ```json
{ "parallel": {
"parallel_vae": true "cfg_p_size": 2
} }
``` ```
**配置说明:** ## 混合并行配置
- `parallel_vae: true`:启用 VAE 并行推理(推荐设置)
- `parallel_vae: false`:禁用 VAE 并行,使用单 GPU 处理
**使用建议:** **配置方式:**
- 在多 GPU 环境下,建议始终启用 VAE 并行 ```json
- VAE 并行可与任何注意力并行方式(Ulysses/Ring)组合使用 "parallel": {
- 对于内存受限的场景,VAE 并行可显著减少内存使用 "seq_p_size": 4,
"seq_p_attn_type": "ulysses",
"cfg_p_size": 2
}
```
## 使用方式 ## 使用方式
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment