img2img.mdx 1.78 KB
Newer Older
Nathan Lambert's avatar
Nathan Lambert committed
1
2
3
4
5
6
7
8
9
10
11
12
<!--Copyright 2022 The HuggingFace Team. All rights reserved.

Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with
the License. You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on
an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the
specific language governing permissions and limitations under the License.
-->

13
# Text-Guided Image-to-Image Generation
Patrick von Platen's avatar
Patrick von Platen committed
14

15
The [`StableDiffusionImg2ImgPipeline`] lets you pass a text prompt and an initial image to condition the generation of new images.
Patrick von Platen's avatar
Patrick von Platen committed
16

17
```python
Patrick von Platen's avatar
Patrick von Platen committed
18
import torch
19
20
21
import requests
from PIL import Image
from io import BytesIO
Patrick von Platen's avatar
Patrick von Platen committed
22

23
from diffusers import StableDiffusionImg2ImgPipeline
Patrick von Platen's avatar
Patrick von Platen committed
24

25
26
27
# load the pipeline
device = "cuda"
pipe = StableDiffusionImg2ImgPipeline.from_pretrained(
apolinario's avatar
apolinario committed
28
    "runwayml/stable-diffusion-v1-5", revision="fp16", torch_dtype=torch.float16
29
).to(device)
Patrick von Platen's avatar
Patrick von Platen committed
30

31
32
# let's download an initial image
url = "https://raw.githubusercontent.com/CompVis/stable-diffusion/main/assets/stable-samples/img2img/sketch-mountains-input.jpg"
Patrick von Platen's avatar
Patrick von Platen committed
33

34
35
36
response = requests.get(url)
init_image = Image.open(BytesIO(response.content)).convert("RGB")
init_image = init_image.resize((768, 512))
Patrick von Platen's avatar
Patrick von Platen committed
37

38
prompt = "A fantasy landscape, trending on artstation"
Patrick von Platen's avatar
Patrick von Platen committed
39

40
images = pipe(prompt=prompt, init_image=init_image, strength=0.75, guidance_scale=7.5).images
Patrick von Platen's avatar
Patrick von Platen committed
41

42
43
images[0].save("fantasy_landscape.png")
```
44
You can also run this example on colab [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/huggingface/notebooks/blob/main/diffusers/image_2_image_using_diffusers.ipynb)
Patrick von Platen's avatar
Patrick von Platen committed
45