lora.md 5.82 KB
Newer Older
Aryan's avatar
Aryan committed
1
<!--Copyright 2025 The HuggingFace Team. All rights reserved.
Steven Liu's avatar
Steven Liu committed
2
3
4
5
6
7
8
9
10
11
12
13
14

Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with
the License. You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on
an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the
specific language governing permissions and limitations under the License.
-->

# LoRA

15
LoRA is a fast and lightweight training method that inserts and trains a significantly smaller number of parameters instead of all the model parameters. This produces a smaller file (~100 MBs) and makes it easier to quickly train a model to learn a new concept. LoRA weights are typically loaded into the denoiser, text encoder or both. The denoiser usually corresponds to a UNet ([`UNet2DConditionModel`], for example) or a Transformer ([`SD3Transformer2DModel`], for example). There are several classes for loading LoRA weights:
Steven Liu's avatar
Steven Liu committed
16

17
18
19
- [`StableDiffusionLoraLoaderMixin`] provides functions for loading and unloading, fusing and unfusing, enabling and disabling, and more functions for managing LoRA weights. This class can be used with any model.
- [`StableDiffusionXLLoraLoaderMixin`] is a [Stable Diffusion (SDXL)](../../api/pipelines/stable_diffusion/stable_diffusion_xl) version of the [`StableDiffusionLoraLoaderMixin`] class for loading and saving LoRA weights. It can only be used with the SDXL model.
- [`SD3LoraLoaderMixin`] provides similar functions for [Stable Diffusion 3](https://huggingface.co/blog/sd3).
20
21
22
- [`FluxLoraLoaderMixin`] provides similar functions for [Flux](https://huggingface.co/docs/diffusers/main/en/api/pipelines/flux).
- [`CogVideoXLoraLoaderMixin`] provides similar functions for [CogVideoX](https://huggingface.co/docs/diffusers/main/en/api/pipelines/cogvideox).
- [`Mochi1LoraLoaderMixin`] provides similar functions for [Mochi](https://huggingface.co/docs/diffusers/main/en/api/pipelines/mochi).
23
- [`AuraFlowLoraLoaderMixin`] provides similar functions for [AuraFlow](https://huggingface.co/fal/AuraFlow).
24
25
26
- [`LTXVideoLoraLoaderMixin`] provides similar functions for [LTX-Video](https://huggingface.co/docs/diffusers/main/en/api/pipelines/ltx_video).
- [`SanaLoraLoaderMixin`] provides similar functions for [Sana](https://huggingface.co/docs/diffusers/main/en/api/pipelines/sana).
- [`HunyuanVideoLoraLoaderMixin`] provides similar functions for [HunyuanVideo](https://huggingface.co/docs/diffusers/main/en/api/pipelines/hunyuan_video).
27
- [`Lumina2LoraLoaderMixin`] provides similar functions for [Lumina2](https://huggingface.co/docs/diffusers/main/en/api/pipelines/lumina2).
28
- [`WanLoraLoaderMixin`] provides similar functions for [Wan](https://huggingface.co/docs/diffusers/main/en/api/pipelines/wan).
29
- [`SkyReelsV2LoraLoaderMixin`] provides similar functions for [SkyReels-V2](https://huggingface.co/docs/diffusers/main/en/api/pipelines/skyreels_v2).
30
- [`CogView4LoraLoaderMixin`] provides similar functions for [CogView4](https://huggingface.co/docs/diffusers/main/en/api/pipelines/cogview4).
31
- [`AmusedLoraLoaderMixin`] is for the [`AmusedPipeline`].
32
- [`HiDreamImageLoraLoaderMixin`] provides similar functions for [HiDream Image](https://huggingface.co/docs/diffusers/main/en/api/pipelines/hidream)
Sayak Paul's avatar
Sayak Paul committed
33
- [`QwenImageLoraLoaderMixin`] provides similar functions for [Qwen Image](https://huggingface.co/docs/diffusers/main/en/api/pipelines/qwen).
34
- [`ZImageLoraLoaderMixin`] provides similar functions for [Z-Image](https://huggingface.co/docs/diffusers/main/en/api/pipelines/zimage).
Sayak Paul's avatar
Sayak Paul committed
35
- [`Flux2LoraLoaderMixin`] provides similar functions for [Flux2](https://huggingface.co/docs/diffusers/main/en/api/pipelines/flux2).
36
- [`LoraBaseMixin`] provides a base class with several utility methods to fuse, unfuse, unload, LoRAs and more.
Steven Liu's avatar
Steven Liu committed
37

Steven Liu's avatar
Steven Liu committed
38
> [!TIP]
Steven Liu's avatar
Steven Liu committed
39
> To learn more about how to load LoRA weights, see the [LoRA](../../tutorials/using_peft_for_inference) loading guide.
Steven Liu's avatar
Steven Liu committed
40

41
42
43
44
## LoraBaseMixin

[[autodoc]] loaders.lora_base.LoraBaseMixin

45
## StableDiffusionLoraLoaderMixin
Steven Liu's avatar
Steven Liu committed
46

47
[[autodoc]] loaders.lora_pipeline.StableDiffusionLoraLoaderMixin
Steven Liu's avatar
Steven Liu committed
48
49
50

## StableDiffusionXLLoraLoaderMixin

51
52
53
54
55
56
[[autodoc]] loaders.lora_pipeline.StableDiffusionXLLoraLoaderMixin

## SD3LoraLoaderMixin

[[autodoc]] loaders.lora_pipeline.SD3LoraLoaderMixin

57
58
59
60
## FluxLoraLoaderMixin

[[autodoc]] loaders.lora_pipeline.FluxLoraLoaderMixin

Sayak Paul's avatar
Sayak Paul committed
61
62
63
64
## Flux2LoraLoaderMixin

[[autodoc]] loaders.lora_pipeline.Flux2LoraLoaderMixin

65
66
67
68
69
70
71
## CogVideoXLoraLoaderMixin

[[autodoc]] loaders.lora_pipeline.CogVideoXLoraLoaderMixin

## Mochi1LoraLoaderMixin

[[autodoc]] loaders.lora_pipeline.Mochi1LoraLoaderMixin
72
73
74
## AuraFlowLoraLoaderMixin

[[autodoc]] loaders.lora_pipeline.AuraFlowLoraLoaderMixin
75

76
77
78
79
80
81
82
83
84
85
86
87
## LTXVideoLoraLoaderMixin

[[autodoc]] loaders.lora_pipeline.LTXVideoLoraLoaderMixin

## SanaLoraLoaderMixin

[[autodoc]] loaders.lora_pipeline.SanaLoraLoaderMixin

## HunyuanVideoLoraLoaderMixin

[[autodoc]] loaders.lora_pipeline.HunyuanVideoLoraLoaderMixin

88
89
90
91
## Lumina2LoraLoaderMixin

[[autodoc]] loaders.lora_pipeline.Lumina2LoraLoaderMixin

92
93
94
95
96
97
98
99
## CogView4LoraLoaderMixin

[[autodoc]] loaders.lora_pipeline.CogView4LoraLoaderMixin

## WanLoraLoaderMixin

[[autodoc]] loaders.lora_pipeline.WanLoraLoaderMixin

100
101
102
103
## SkyReelsV2LoraLoaderMixin

[[autodoc]] loaders.lora_pipeline.SkyReelsV2LoraLoaderMixin

104
105
106
107
## AmusedLoraLoaderMixin

[[autodoc]] loaders.lora_pipeline.AmusedLoraLoaderMixin

108
109
110
111
## HiDreamImageLoraLoaderMixin

[[autodoc]] loaders.lora_pipeline.HiDreamImageLoraLoaderMixin

112
113
114
115
## QwenImageLoraLoaderMixin

[[autodoc]] loaders.lora_pipeline.QwenImageLoraLoaderMixin

116
117
118
119
## ZImageLoraLoaderMixin

[[autodoc]] loaders.lora_pipeline.ZImageLoraLoaderMixin

120
121
122
## KandinskyLoraLoaderMixin
[[autodoc]] loaders.lora_pipeline.KandinskyLoraLoaderMixin

123
## LoraBaseMixin
Steven Liu's avatar
Steven Liu committed
124

125
[[autodoc]] loaders.lora_base.LoraBaseMixin