inpaint.mdx 1.99 KB
Newer Older
Nathan Lambert's avatar
Nathan Lambert committed
1
2
3
4
5
6
7
8
9
10
11
12
<!--Copyright 2022 The HuggingFace Team. All rights reserved.

Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with
the License. You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on
an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the
specific language governing permissions and limitations under the License.
-->

13
# Text-Guided Image-Inpainting
Patrick von Platen's avatar
Patrick von Platen committed
14

15
The [`StableDiffusionInpaintPipeline`] lets you edit specific parts of an image by providing a mask and text prompt.
Patrick von Platen's avatar
Patrick von Platen committed
16

17
18
```python
from io import BytesIO
Patrick von Platen's avatar
Patrick von Platen committed
19

20
21
22
23
24
from torch import autocast
import requests
import PIL

from diffusers import StableDiffusionInpaintPipeline
Patrick von Platen's avatar
Patrick von Platen committed
25
26


27
28
29
def download_image(url):
    response = requests.get(url)
    return PIL.Image.open(BytesIO(response.content)).convert("RGB")
Patrick von Platen's avatar
Patrick von Platen committed
30
31


32
33
img_url = "https://raw.githubusercontent.com/CompVis/latent-diffusion/main/data/inpainting_examples/overture-creations-5sI6fQgYIuo.png"
mask_url = "https://raw.githubusercontent.com/CompVis/latent-diffusion/main/data/inpainting_examples/overture-creations-5sI6fQgYIuo_mask.png"
Patrick von Platen's avatar
Patrick von Platen committed
34

35
36
init_image = download_image(img_url).resize((512, 512))
mask_image = download_image(mask_url).resize((512, 512))
Patrick von Platen's avatar
Patrick von Platen committed
37

38
39
40
41
42
43
44
45
46
47
48
device = "cuda"
pipe = StableDiffusionInpaintPipeline.from_pretrained(
    "CompVis/stable-diffusion-v1-4", revision="fp16", torch_dtype=torch.float16, use_auth_token=True
).to(device)

prompt = "a cat sitting on a bench"
with autocast("cuda"):
    images = pipe(prompt=prompt, init_image=init_image, mask_image=mask_image, strength=0.75).images

images[0].save("cat_on_bench.png")
```
Patrick von Platen's avatar
Patrick von Platen committed
49

50
You can also run this example on colab [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/patil-suraj/Notebooks/blob/master/in_painting_with_stable_diffusion_using_diffusers.ipynb)