using_safetensors.mdx 4.6 KB
Newer Older
Steven Liu's avatar
Steven Liu committed
1
# Load safetensors
2

Steven Liu's avatar
Steven Liu committed
3
[safetensors](https://github.com/huggingface/safetensors) is a safe and fast file format for storing and loading tensors. Typically, PyTorch model weights are saved or *pickled* into a `.bin` file with Python's [`pickle`](https://docs.python.org/3/library/pickle.html) utility. However, `pickle` is not secure and pickled files may contain malicious code that can be executed. safetensors is a secure alternative to `pickle`, making it ideal for sharing model weights.
4

Steven Liu's avatar
Steven Liu committed
5
This guide will show you how you load `.safetensor` files, and how to convert Stable Diffusion model weights stored in other formats to `.safetensor`. Before you start, make sure you have safetensors installed:
6

Steven Liu's avatar
Steven Liu committed
7
8
```bash
!pip install safetensors
9
10
```

Steven Liu's avatar
Steven Liu committed
11
If you look at the [`runwayml/stable-diffusion-v1-5`](https://huggingface.co/runwayml/stable-diffusion-v1-5/tree/main) repository, you'll see weights inside the `text_encoder`, `unet` and `vae` subfolders are stored in the `.safetensors` format. By default, 🤗 Diffusers automatically loads these `.safetensors` files from their subfolders if they're available in the model repository.
12

Steven Liu's avatar
Steven Liu committed
13
For more explicit control, you can optionally set `use_safetensors=True` (if `safetensors` is not installed, you'll get an error message asking you to install it):
14

Steven Liu's avatar
Steven Liu committed
15
16
```py
from diffusers import DiffusionPipeline
17

Steven Liu's avatar
Steven Liu committed
18
pipeline = DiffusionPipeline.from_pretrained("runwayml/stable-diffusion-v1-5", use_safetensors=True)
19
20
```

Steven Liu's avatar
Steven Liu committed
21
However, model weights are not necessarily stored in separate subfolders like in the example above. Sometimes, all the weights are stored in a single `.safetensors` file. In this case, if the weights are Stable Diffusion weights, you can load the file directly with the [`~diffusers.loaders.FromCkptMixin.from_ckpt`] method:
22

Steven Liu's avatar
Steven Liu committed
23
24
```py
from diffusers import StableDiffusionPipeline
25

Steven Liu's avatar
Steven Liu committed
26
27
28
29
pipeline = StableDiffusionPipeline.from_ckpt(
    "https://huggingface.co/WarriorMama777/OrangeMixs/blob/main/Models/AbyssOrangeMix/AbyssOrangeMix.safetensors"
)
```
30

Steven Liu's avatar
Steven Liu committed
31
## Convert to safetensors
32

Steven Liu's avatar
Steven Liu committed
33
Not all weights on the Hub are available in the `.safetensors` format, and you may encounter weights stored as `.bin`. In this case, use the Space below to convert the weights to `.safetensors`. The Convert Space downloads the pickled weights, converts them, and opens a Pull Request to upload the newly converted `.safetensors` file on the Hub. This way, if there is any malicious code contained in the pickled files, they're uploaded to the Hub - which has a [security scanner](https://huggingface.co/docs/hub/security-pickle#hubs-security-scanner) to detect unsafe files and suspicious pickle imports - instead of your computer.
34

Steven Liu's avatar
Steven Liu committed
35
36
37
38
39
40
<iframe
	src="https://safetensors-convert.hf.space"
	frameborder="0"
	width="850"
	height="450"
></iframe>
41

Steven Liu's avatar
Steven Liu committed
42
You can use the model with the new `.safetensors` weights by specifying the reference to the Pull Request in the `revision` parameter (you can also test it in this [Check PR](https://huggingface.co/spaces/diffusers/check_pr) Space on the Hub), for example `refs/pr/22`:
43

Steven Liu's avatar
Steven Liu committed
44
45
```py
from diffusers import DiffusionPipeline
46

Steven Liu's avatar
Steven Liu committed
47
pipeline = DiffusionPipeline.from_pretrained("stabilityai/stable-diffusion-2-1", revision="refs/pr/22")
48
49
```

Steven Liu's avatar
Steven Liu committed
50
## Why use safetensors?
51

Steven Liu's avatar
Steven Liu committed
52
There are several reasons for using safetensors:
53

Steven Liu's avatar
Steven Liu committed
54
55
- Safety is the number one reason for using safetensors. As open-source and model distribution grows, it is important to be able to trust the model weights you downloaded don't contain any malicious code. The current size of the header in safetensors prevents parsing extremely large JSON files.
- Loading speed between switching models is another reason to use safetensors, which performs zero-copy of the tensors. It is especially fast compared to `pickle` if you're loading the weights to CPU (the default case), and just as fast if not faster when directly loading the weights to GPU. You'll only notice the performance difference if the model is already loaded, and not if you're downloading the weights or loading the model for the first time.
56

Steven Liu's avatar
Steven Liu committed
57
	The time it takes to load the entire pipeline:
58

Steven Liu's avatar
Steven Liu committed
59
60
	```py
	from diffusers import StableDiffusionPipeline
61

Steven Liu's avatar
Steven Liu committed
62
63
64
65
	pipeline = StableDiffusionPipeline.from_pretrained("stabilityai/stable-diffusion-2-1")
	"Loaded in safetensors 0:00:02.033658"
	"Loaded in PyTorch 0:00:02.663379"
	```
66

Steven Liu's avatar
Steven Liu committed
67
	But the actual time it takes to load 500MB of the model weights is only:
68

Steven Liu's avatar
Steven Liu committed
69
70
71
72
	```bash
	safetensors: 3.4873ms
	PyTorch: 172.7537ms
	```
73

Steven Liu's avatar
Steven Liu committed
74
- Lazy loading is also supported in safetensors, which is useful in distributed settings to only load some of the tensors. This format allowed the [BLOOM](https://huggingface.co/bigscience/bloom) model to be loaded in 45 seconds on 8 GPUs instead of 10 minutes with regular PyTorch weights.