README.md 4.84 KB
Newer Older
dcuai's avatar
dcuai committed
1
# pix2pixHD
yongshk's avatar
yongshk committed
2

yongshk's avatar
yongshk committed
3
4
## 论文

yongshk's avatar
yongshk committed
5
6
7
8
` High-Resolution Image Synthesis and Semantic `

- https://arxiv.org/pdf/1711.11585.pdf

yongshk's avatar
更新  
yongshk committed
9
## 模型结构
yongshk's avatar
yongshk committed
10
11
模型整体的结构是conditional GAN。 模型与Pix2pix相比更换了coarse-to-fine的生成器、multi-scale的判别器。

chenzk's avatar
chenzk committed
12
![](https://developer.sourcefind.cn/codes/modelzoo/pix2pixhd_pytorch/-/raw/master/doc/Pix2PixHD-GAN-model-for-paired-image-to-image-translation.png)
yongshk's avatar
yongshk committed
13
14
15

## 算法原理

yongshk's avatar
yongshk committed
16
17
 pix2pixHD的算法原理是基于条件生成对抗网络(Conditional Generative Adversarial Nets)。它是一个由两个部分组成的网络:生成器和判别器。生成器用于从训练数据中合成图像,而判别器则负责判断生成的图像是否真实。 

chenzk's avatar
chenzk committed
18
![](https://developer.sourcefind.cn/codes/modelzoo/pix2pixhd_pytorch/-/raw/master/doc/pix2pixHD原理.png)
yongshk's avatar
yongshk committed
19
20

## 环境配置
yongshk's avatar
yongshk committed
21

yongshk's avatar
yongshk committed
22
23
### Docker(方法一)

dcuai's avatar
dcuai committed
24
此处提供[光源](https://sourcefind.cn/#/main-page)拉取docker镜像
yongshk's avatar
yongshk committed
25

yongshk's avatar
yongshk committed
26
```
yongshk's avatar
update  
yongshk committed
27
docker pull image.sourcefind.cn:5000/dcu/admin/base/pytorch:2.1.0-ubuntu20.04-dtk24.04.1-py3.10
shantf's avatar
shantf committed
28
docker run -it --network=host --name=pix2pixhd --privileged --device=/dev/kfd --device=/dev/dri --ipc=host --shm-size=32G  --group-add video --cap-add=SYS_PTRACE --security-opt seccomp=unconfined -u root --ulimit stack=-1:-1 --ulimit memlock=-1:-1 -v /opt/hyhal/:/opt/hyhal/:ro image.sourcefind.cn:5000/dcu/admin/base/pytorch:2.1.0-ubuntu20.04-dtk24.04.1-py3.10
dcuai's avatar
dcuai committed
29
pip install -r requirements.txt
yongshk's avatar
yongshk committed
30
31
```

yongshk's avatar
yongshk committed
32
33
### Dockerfile(方法二)

yongshk's avatar
yongshk committed
34
dockerfile使用方法
yongshk's avatar
yongshk committed
35
36

```
yongshk's avatar
yongshk committed
37
docker build --no-cache -t pix2pixhd:latest .
shantf's avatar
shantf committed
38
docker run -dit --network=host --name=pix2pixhd --privileged --device=/dev/kfd --device=/dev/dri --ipc=host --shm-size=16G  --group-add video --cap-add=SYS_PTRACE --security-opt seccomp=unconfined -u root --ulimit stack=-1:-1 --ulimit memlock=-1:-1 -v /opt/hyhal/:/opt/hyhal/:ro pix2pixhd:latest
yongshk's avatar
yongshk committed
39
docker exec -it pix2pixhd /bin/bash
yongshk's avatar
yongshk committed
40
41
42
43
```

### Anaconda(方法三)

chenzk's avatar
chenzk committed
44
关于本项目DCU显卡所需的特殊深度学习库可从[光合](https://developer.sourcefind.cn/tool/)开发者社区下载安装。
yongshk's avatar
yongshk committed
45

yongshk's avatar
yongshk committed
46
```
yongshk's avatar
update  
yongshk committed
47
48
49
50
51
DTK驱动:dtk24.04.1
python:python3.10
apex==apex-1.1.0+das1.1.gitf477a3a.abi1.dtk2404.torch2.1.0-cp310-cp310-manylinux_2_31_x86_64.whl
torch==torch-2.1.0+das1.1.git3ac1bdd.abi1.dtk2404-cp310-cp310-manylinux_2_31_x86_64.whl
torchvision==torchvision-0.16.0+das1.1.git7d45932.abi1.dtk2404.torch2.1-cp310-cp310-manylinux_2_31_x86_64.whl
yongshk's avatar
yongshk committed
52
```
yongshk's avatar
yongshk committed
53

yongshk's avatar
yongshk committed
54
55
56
`Tips:以上dtk驱动、python等DCU相关工具版本需要严格一一对应`

其它非深度学习库参照requirements.txt安装:
yongshk's avatar
yongshk committed
57

yongshk's avatar
yongshk committed
58
59
```
pip install -r requirements.txt
dcuai's avatar
dcuai committed
60
pip install numpy pillow==9.5.0 scipy
yongshk's avatar
yongshk committed
61
```
yongshk's avatar
yongshk committed
62

yongshk's avatar
更新  
yongshk committed
63
## 数据集
yongshk's avatar
yongshk committed
64

yongshk's avatar
yongshk committed
65
66
67
`模型使用数据为cityscapes`

- <https://www.cityscapes-dataset.com/>
yongshk's avatar
yongshk committed
68
69

项目中已提供用于试验训练的迷你数据集,训练数据目录结构如下,用于正常训练的完整数据集请按此目录结构进行制备:
yongshk's avatar
yongshk committed
70

yongshk's avatar
yongshk committed
71
```
yongshk's avatar
yongshk committed
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
 ── dataset
 ────── cityscapes
       │   ├── train_label
       │             ├── xxx.png
       │             ├── xxx.png
       │             └── ...
       │   └── train_inst
       │             ├── xxx.png
       │             ├── xxx.png
       │             └── ...
       │   └── train_img
       │             ├── xxx.png
       │             ├── xxx.png
       │             └── ...
       │   └── test_label
       │             ├── xxx.png
       │             ├── xxx.png
       │             └── ...
       │   └── test_inst
       │             ├── xxx.png
       │             ├── xxx.png
       │             └── ...
yongshk's avatar
yongshk committed
94
```
yongshk's avatar
yongshk committed
95

yongshk's avatar
yongshk committed
96
97
## 训练
### 单机单卡
yongshk's avatar
yongshk committed
98
99
100

以 1024 x 512 分辨率训练模型

yongshk's avatar
yongshk committed
101
```
yongshk's avatar
yongshk committed
102
103
#!./scripts/train_512p.sh
python train.py --name label2city_512p
yongshk's avatar
yongshk committed
104
```
yongshk's avatar
yongshk committed
105

yongshk's avatar
yongshk committed
106
### 单机多卡
yongshk's avatar
yongshk committed
107

yongshk's avatar
yongshk committed
108
```
yongshk's avatar
yongshk committed
109
110
#!./scripts/train_512p.sh
python train.py --name label2city_512p --batchSize 4 --gpu_ids 0,1,2,3
yongshk's avatar
yongshk committed
111
```
yongshk's avatar
更新  
yongshk committed
112

yongshk's avatar
yongshk committed
113
## 推理
yongshk's avatar
更新  
yongshk committed
114

yongshk's avatar
yongshk committed
115
116
117
118
    python test.py --name label2city_512p \
                   --netG local \
                   --ngf 32 \
                   --resize_or_crop none \
yongshk's avatar
yongshk committed
119

yongshk's avatar
yongshk committed
120
## result
yongshk's avatar
yongshk committed
121
122
123

测试图

chenzk's avatar
chenzk committed
124
![](https://developer.sourcefind.cn/codes/modelzoo/pix2pixhd_pytorch/-/raw/master/doc/face1_1.jpg) )
yongshk's avatar
yongshk committed
125
126

### 精度
yongshk's avatar
yongshk committed
127

dongchy920's avatar
dongchy920 committed
128
测试数据:<https://www.cityscapes-dataset.com/>,使用的加速卡:Z100L。  
yongshk's avatar
yongshk committed
129
130

根据测试结果情况填写表格:
yongshk's avatar
yongshk committed
131
132
133
134
135

| pix2pixHD  | G_GAN | G_GAN_Feat | G_VGG | D_fake | D_real |
| :--------: | :---: | ---------- | ----- | ------ | :----: |
| cityscapes | 0.540 | 6.153      | 3.087 | 0.534  | 0.424  |

yongshk's avatar
yongshk committed
136
## 应用场景
yongshk's avatar
yongshk committed
137

yongshk's avatar
yongshk committed
138
139
### 算法类别

yongshk's avatar
yongshk committed
140
`图像超分`
yongshk's avatar
yongshk committed
141
142

### 热点应用行业
yongshk's avatar
yongshk committed
143

yongshk's avatar
update  
yongshk committed
144
`设计`,`交通`,`网安`
yongshk's avatar
yongshk committed
145

yongshk's avatar
更新  
yongshk committed
146
## 源码仓库及问题反馈
yongshk's avatar
yongshk committed
147

chenzk's avatar
chenzk committed
148
* [https://developer.sourcefind.cn/codes/modelzoo/pix2pixhd_pytorch](https://developer.sourcefind.cn/codes/modelzoo/pix2pixhd_pytorch)
yongshk's avatar
yongshk committed
149
## 参考资料
yongshk's avatar
yongshk committed
150
151
* [https://github.com/NVIDIA/pix2pixHD](https://github.com/NVIDIA/pix2pixHD)