README.md 4.45 KB
Newer Older
yongshk's avatar
yongshk committed
1
2
# PIX2PIXHD

yongshk's avatar
yongshk committed
3
4
## 论文

yongshk's avatar
yongshk committed
5
6
7
8
` High-Resolution Image Synthesis and Semantic `

- https://arxiv.org/pdf/1711.11585.pdf

yongshk's avatar
更新  
yongshk committed
9
## 模型结构
yongshk's avatar
yongshk committed
10
11
12
模型整体的结构是conditional GAN。 模型与Pix2pix相比更换了coarse-to-fine的生成器、multi-scale的判别器。

![](https://developer.hpccube.com/codes/modelzoo/pix2pixhd_pytorch/-/raw/master/doc/Pix2PixHD-GAN-model-for-paired-image-to-image-translation.png)
yongshk's avatar
yongshk committed
13
14
15

## 算法原理

yongshk's avatar
yongshk committed
16
17
18
 pix2pixHD的算法原理是基于条件生成对抗网络(Conditional Generative Adversarial Nets)。它是一个由两个部分组成的网络:生成器和判别器。生成器用于从训练数据中合成图像,而判别器则负责判断生成的图像是否真实。 

![](https://developer.hpccube.com/codes/modelzoo/pix2pixhd_pytorch/-/raw/master/doc/pix2pixHD原理.png)
yongshk's avatar
yongshk committed
19
20

## 环境配置
yongshk's avatar
yongshk committed
21

yongshk's avatar
yongshk committed
22
23
### Docker(方法一)

yongshk's avatar
yongshk committed
24
25
此处提供[光源](https://www.sourcefind.cn/#/service-details)拉取docker镜像

yongshk's avatar
yongshk committed
26
```
yongshk's avatar
yongshk committed
27
28
29
30
docker pull image.sourcefind.cn:5000/dcu/admin/base/pytorch:1.10.0-centos7.6-dtk-22.10-py37-latest
docker run -it --network=host --name=bert_prof --privileged --device=/dev/kfd --device=/dev/dri --ipc=host --shm-size=32G  --group-add video --cap-add=SYS_PTRACE --security-opt seccomp=unconfined -u root --ulimit stack=-1:-1 --ulimit memlock=-1:-1 image.sourcefind.cn:5000/dcu/admin/base/pytorch:1.10.0-centos7.6-dtk-22.10-py37-latest
```

yongshk's avatar
yongshk committed
31
32
### Dockerfile(方法二)

yongshk's avatar
yongshk committed
33
dockerfile使用方法
yongshk's avatar
yongshk committed
34
35

```
yongshk's avatar
yongshk committed
36
37
38
docker build --no-cache -t pix2pixhd:latest .
docker run -dit --network=host --name=pix2pixhd --privileged --device=/dev/kfd --device=/dev/dri --ipc=host --shm-size=16G  --group-add video --cap-add=SYS_PTRACE --security-opt seccomp=unconfined -u root --ulimit stack=-1:-1 --ulimit memlock=-1:-1 pix2pixhd:latest
docker exec -it pix2pixhd /bin/bash
yongshk's avatar
yongshk committed
39
40
41
42
43
pip install -r requirements.txt
```

### Anaconda(方法三)

yongshk's avatar
yongshk committed
44
关于本项目DCU显卡所需的特殊深度学习库可从[光合](https://developer.hpccube.com/tool/)开发者社区下载安装。
yongshk's avatar
yongshk committed
45

yongshk's avatar
yongshk committed
46
```
yongshk's avatar
yongshk committed
47
DTK驱动:dtk22.10
yongshk's avatar
yongshk committed
48
python:python3.7
yongshk's avatar
yongshk committed
49
50
apex==0.1+gitdb7007a.dtk2210
torch==1.10.0a0+git2040069.dtk2210
yongshk's avatar
yongshk committed
51
```
yongshk's avatar
yongshk committed
52

yongshk's avatar
yongshk committed
53
54
55
`Tips:以上dtk驱动、python等DCU相关工具版本需要严格一一对应`

其它非深度学习库参照requirements.txt安装:
yongshk's avatar
yongshk committed
56

yongshk's avatar
yongshk committed
57
58
59
```
pip install -r requirements.txt
```
yongshk's avatar
yongshk committed
60

yongshk's avatar
更新  
yongshk committed
61
## 数据集
yongshk's avatar
yongshk committed
62

yongshk's avatar
yongshk committed
63
64
65
`模型使用数据为cityscapes`

- <https://www.cityscapes-dataset.com/>
yongshk's avatar
yongshk committed
66
67

项目中已提供用于试验训练的迷你数据集,训练数据目录结构如下,用于正常训练的完整数据集请按此目录结构进行制备:
yongshk's avatar
yongshk committed
68

yongshk's avatar
yongshk committed
69
```
yongshk's avatar
yongshk committed
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
 ── dataset
 ────── cityscapes
       │   ├── train_label
       │             ├── xxx.png
       │             ├── xxx.png
       │             └── ...
       │   └── train_inst
       │             ├── xxx.png
       │             ├── xxx.png
       │             └── ...
       │   └── train_img
       │             ├── xxx.png
       │             ├── xxx.png
       │             └── ...
       │   └── test_label
       │             ├── xxx.png
       │             ├── xxx.png
       │             └── ...
       │   └── test_inst
       │             ├── xxx.png
       │             ├── xxx.png
       │             └── ...
yongshk's avatar
yongshk committed
92
```
yongshk's avatar
yongshk committed
93
94
95
96




yongshk's avatar
yongshk committed
97
98
## 训练
### 单机单卡
yongshk's avatar
yongshk committed
99
100
101

以 1024 x 512 分辨率训练模型

yongshk's avatar
yongshk committed
102
```
yongshk's avatar
yongshk committed
103
104
#!./scripts/train_512p.sh
python train.py --name label2city_512p
yongshk's avatar
yongshk committed
105
```
yongshk's avatar
yongshk committed
106

yongshk's avatar
yongshk committed
107
### 单机多卡
yongshk's avatar
yongshk committed
108

yongshk's avatar
yongshk committed
109
```
yongshk's avatar
yongshk committed
110
111
#!./scripts/train_512p.sh
python train.py --name label2city_512p --batchSize 4 --gpu_ids 0,1,2,3
yongshk's avatar
yongshk committed
112
```
yongshk's avatar
更新  
yongshk committed
113

yongshk's avatar
yongshk committed
114
## 推理
yongshk's avatar
更新  
yongshk committed
115

yongshk's avatar
yongshk committed
116
117
118
119
    python test.py --name label2city_512p \
                   --netG local \
                   --ngf 32 \
                   --resize_or_crop none \
yongshk's avatar
yongshk committed
120

yongshk's avatar
yongshk committed
121
## result
yongshk's avatar
yongshk committed
122
123
124
125

测试图

![](https://developer.hpccube.com/codes/modelzoo/pix2pixhd_pytorch/-/raw/master/doc/face1_1.jpg) )
yongshk's avatar
yongshk committed
126
127

### 精度
yongshk's avatar
yongshk committed
128
129

测试数据:<https://www.cityscapes-dataset.com/>,使用的加速卡:Z100L。
yongshk's avatar
yongshk committed
130
131

根据测试结果情况填写表格:
yongshk's avatar
yongshk committed
132
133
134
135
136

| pix2pixHD  | G_GAN | G_GAN_Feat | G_VGG | D_fake | D_real |
| :--------: | :---: | ---------- | ----- | ------ | :----: |
| cityscapes | 0.540 | 6.153      | 3.087 | 0.534  | 0.424  |

yongshk's avatar
yongshk committed
137
## 应用场景
yongshk's avatar
yongshk committed
138

yongshk's avatar
yongshk committed
139
140
### 算法类别

yongshk's avatar
yongshk committed
141
`图像超分`
yongshk's avatar
yongshk committed
142
143

### 热点应用行业
yongshk's avatar
yongshk committed
144
145

`设计`
yongshk's avatar
yongshk committed
146

yongshk's avatar
更新  
yongshk committed
147
## 源码仓库及问题反馈
yongshk's avatar
yongshk committed
148
149

* [https://github.com/NVIDIA/pix2pixHD](https://github.com/NVIDIA/pix2pixHD)
yongshk's avatar
yongshk committed
150
## 参考资料
yongshk's avatar
yongshk committed
151
152
* [https://github.com/NVIDIA/pix2pixHD](https://github.com/NVIDIA/pix2pixHD)