README.md

# stable_diffusion
## 论文
[High-Resolution Image Synthesis with Latent Diffusion Models](https://arxiv.org/pdf/2112.10752)
## 模型结构
stable diffusion的核心是latent diffusion model,latent diffusion model结构如下：

![image](https://developer.hpccube.com/codes/modelzoo/stable_diffusion_ait/-/raw/master/resources/sd_model.png)
## 算法原理
根据模型结构，算法原理简要如下：

![image](https://developer.hpccube.com/codes/modelzoo/stable_diffusion_ait/-/raw/master/resources/sd_principle.png)
## 数据集
无
## 环境配置
在[光源](https://sourcefind.cn/#/service-list)可拉取推理的docker镜像。stable_diffusion_ait推荐的镜像如下：
```
docker pull image.sourcefind.cn:5000/dcu/admin/base/custom:ait-0.0.1_dtk24.04_py310
docker run -d -t -v /opt/hyhal:/opt/hyhal:ro --privileged --device=/dev/kfd --device=/dev/dri/ --network=host --group-add video --name sd-test image.sourcefind.cn:5000/dcu/admin/base/custom:ait-0.0.1_dtk24.04_py310
docker exec -it sd-test bash
source /opt/dtk/env.sh
```
## 推理
**install ait**
```
cd stable_diffusion_ait
pip3 install dist/aitemplate-0.0.1-py3-none-any.whl
```
#### 01_resnet-50
```
cd examples/01_resnet-50
```
下载resnet50 weights(https://github.com/rwightman/pytorch-image-models/releases/download/v0.1-rsb-weights/resnet50_a1_0-14fe96d1.pth)

**benchmark**
```
python3 benchmark_ait.py
python3 benchmark_pt.py
```
**infer**
```
python3 infer_with_torch.py
```
#### 02_bert
```
cd examples/02_bert
```
下载bert-base-uncased weights(https://huggingface.co/google-bert/bert-base-uncased)

**benchmark**
```
python3 benchmark_ait.py
python3 benchmark_pt.py
```
**infer**
```
python3 demo.py
```
#### 03_vit
```
cd examples/03_vit
```
下载vit_base_patch16_224 weights(https://huggingface.co/timm/vit_base_patch16_224.augreg2_in21k_ft_in1k)

**benchmark**
```
python3 benchmark_ait.py
python3 benchmark_pt.py
```
**verification**
```
python3 verification.py
```
#### 04_stable_diffusion

下载stable-diffusion-2-1-base weights(https://huggingface.co/stabilityai/stable-diffusion-2-1-base)

下载clip-vit-large-patch14 weights(https://huggingface.co/openai/clip-vit-large-patch14)

**compile**
```
cd examples/04_stable_diffusion
python3 scripts/compile.py --local-dir stable-diffusion-2-1-base_path
```
**benchmark**
```
python3 src/benchmark.py --local-dir stable-diffusion-2-1-base_path --clip-dir clip-vit-large-patch14_path --benchmark-pt True
```
**infer**
```
python3 scripts/demo.py --local-dir stable-diffusion-2-1-base_path
python3 scripts/demo_pt.py --local-dir stable-diffusion-2-1-base_path
```
## result
![image](https://developer.hpccube.com/codes/modelzoo/stable_diffusion_ait/-/raw/master/resources/example_ait.png)
### 精度
无
## 应用场景
### 算法类别
文生图
### 热点应用行业
艺术设计,游戏开发,电影制作
## 源码仓库及问题反馈
http://10.6.10.68/modelzoo/stable_diffusion_ait
## 参考资料
https://github.com/ROCm/AITemplate