inpaint.mdx 1.92 KB
Newer Older
Nathan Lambert's avatar
Nathan Lambert committed
1
2
3
4
5
6
7
8
9
10
11
12
<!--Copyright 2022 The HuggingFace Team. All rights reserved.

Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with
the License. You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on
an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the
specific language governing permissions and limitations under the License.
-->

13
# Text-Guided Image-Inpainting
Patrick von Platen's avatar
Patrick von Platen committed
14

15
The [`StableDiffusionInpaintPipeline`] lets you edit specific parts of an image by providing a mask and text prompt.
Patrick von Platen's avatar
Patrick von Platen committed
16

17
18
```python
from io import BytesIO
Patrick von Platen's avatar
Patrick von Platen committed
19

20
21
22
23
import requests
import PIL

from diffusers import StableDiffusionInpaintPipeline
Patrick von Platen's avatar
Patrick von Platen committed
24
25


26
27
28
def download_image(url):
    response = requests.get(url)
    return PIL.Image.open(BytesIO(response.content)).convert("RGB")
Patrick von Platen's avatar
Patrick von Platen committed
29
30


31
32
img_url = "https://raw.githubusercontent.com/CompVis/latent-diffusion/main/data/inpainting_examples/overture-creations-5sI6fQgYIuo.png"
mask_url = "https://raw.githubusercontent.com/CompVis/latent-diffusion/main/data/inpainting_examples/overture-creations-5sI6fQgYIuo_mask.png"
Patrick von Platen's avatar
Patrick von Platen committed
33

34
35
init_image = download_image(img_url).resize((512, 512))
mask_image = download_image(mask_url).resize((512, 512))
Patrick von Platen's avatar
Patrick von Platen committed
36

37
38
device = "cuda"
pipe = StableDiffusionInpaintPipeline.from_pretrained(
39
    "CompVis/stable-diffusion-v1-4", revision="fp16", torch_dtype=torch.float16
40
41
42
).to(device)

prompt = "a cat sitting on a bench"
43
images = pipe(prompt=prompt, init_image=init_image, mask_image=mask_image, strength=0.75).images
44
45
46

images[0].save("cat_on_bench.png")
```
Patrick von Platen's avatar
Patrick von Platen committed
47

48
You can also run this example on colab [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/huggingface/notebooks/blob/main/diffusers/in_painting_with_stable_diffusion_using_diffusers.ipynb)