GitLab

* first draft


---------
Co-authored-by: yiyixuxu <yixu310@gmail,com>
Co-authored-by: Junhwa Song <ethan9867@gmail.com>
Co-authored-by: Ahn Donghoon (안동훈 / suno) <suno.vivid@gmail.com>
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>

okay

* add clip text-encoder training

* no dora

* text encoder traing fixes

* text encoder traing fixes

* text encoder training fixes

* text encoder training fixes

* text encoder training fixes

* text encoder training fixes

* add text_encoder layers to save_lora

* style

* fix imports

* style

* fix text encoder

* review changes

* review changes

* review changes

* minor change

* add lora tag

* style

* add readme notes

* add tests for clip encoders

* style

* typo

* fixes

* style

* Update tests/lora/test_lora_layers_sd3.py
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>

* Update examples/dreambooth/README_sd3.md
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>

* minor readme change

---------
Co-authored-by: YiYi Xu <yixu310@gmail.com>
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>

create a utility for calculating the expected number of shards.

Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>

* Class methods are supposed to use `cls` conventionally

* `make style && make quality`

* An Empty commit

---------
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>

add note

fix import error

* add note on caching in fast diffusion

* formatting

* Update docs/source/en/tutorials/fast_diffusion.md
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>

---------
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>

* Fix typos

* Fix typos & up style

* chore: Update numbers

---------
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>

* Discourage using `revision`

* `make style && make quality`

* Refactor code to use 'variant' instead of 'revision'

* `revision="bf16"` -> `variant="bf16"`

---------
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>

* Trim all the trailing white space in the whole repo

* Remove unnecessary empty places

* make style && make quality

* Trim trailing white space

* trim

---------
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>

* 🎨 fix xl playground device

* 🎨 run `make fix-copies`

* 🎨

 run `make fix-copies`

* edit xl_controlnet_img2img file

* edit playground img2img test slow

* Update tests/pipelines/stable_diffusion_xl/test_stable_diffusion_xl_img2img.py

---------
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>

* Fix typos & improve contributing page

* `make style && make quality`

* fix typos

* Fix typo

---------
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>

[train_lcm_distill_lora_sdxl.py] Fix the LR schedulers when num_train_epochs is passed in a distributed training env (#8446)

fix num_train_epochs
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>

Add extra performance features for EMAModel, torch._foreach operations and better support for non-blocking CPU offloading (#7685)

* Add support for _foreach operations and non-blocking to EMAModel

* default foreach to false

* add non-blocking EMA offloading to SD1.5 T2I example script

* fix whitespace

* move foreach to cli argument

* linting

* Update README.md re: EMA weight training

* correct args.foreach_ema

* add tests for foreach ema

* code quality

* add foreach to from_pretrained

* default foreach false

* fix linting

---------
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>
Co-authored-by: drhead <a@a.a>

Co-authored-by: YiYi Xu <yixu310@gmail,com>

* refactor lora conversion utility.

* remove error raises.

* add onetrainer support too.

* fix

* add check

* key present is checked before

* test case draft

* aply suggestions

* changed testing repo, back to old class

* forgot docstring

---------
Co-authored-by: YiYi Xu <yixu310@gmail.com>
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>

* get rid of the legacy lora remnants and make our codebase lighter

* fix depcrecated lora argument

* fix

* empty commit to trigger ci

* remove print

* empty

fix
Co-authored-by: yiyixuxu <yixu310@gmail,com>

* fix typo in __call__ of pipeline_stable_diffusion_3.py

* fix typo in __call__ of pipeline_stable_diffusion_3_img2img.py

---------
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>

* Update train_dreambooth_lora_sd3.py

* Update train_dreambooth_lora_sd3.py

* Update train_dreambooth_sd3.py

---------
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>

* add support for lora fusion in sd3

* add test to ensure fused lora and effective lora produce same outpouts

reduce tolerance for a flax test

fix single file

* change to logit_normal as the weighting scheme

* sensible default mote

* sd3 controlnet



---------
Co-authored-by: haofanwang <haofanwang.ai@gmail.com>

[SD3] Fix mis-matched shape when num_images_per_prompt > 1 using without T5 (text_encoder_3=None) (#8558)

* fix shape mismatch when num_images_per_prompt > 1 and text_encoder_3=None

* style

* fix copies

---------
Co-authored-by: YiYi Xu <yixu310@gmail.com>
Co-authored-by: yiyixuxu <yixu310@gmail,com>

Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>
Co-authored-by: YiYi Xu <yixu310@gmail.com>

* add sd1.5 compatibility to controlnet-xs

* set use_linear_projection by base_block

* refine code style

Making self.upsample actually be Upsample1D

[SD3 Docs] Corrected title about loading model with T5

Corrected the documentation title to "Loading the single file checkpoint with T5" Previously, it incorrectly stated "Loading the single file checkpoint without T5" which contradicted the code snippet showing how to load the SD3 checkpoint with the T5 model

* [LoRA] text encoder: read the ranks for all the attn modules

 * In addition to out_proj, read the ranks of adapters for q_proj, k_proj, and  v_proj

 * Allow missing adapters (UNet already supports this)

* ruff format loaders.lora

* [LoRA] add tests for partial text encoders LoRAs

* [LoRA] update test_simple_inference_with_partial_text_lora to be deterministic

* [LoRA] comment justifying test_simple_inference_with_partial_text_lora

* style

---------
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>

Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>

* shift factor argument to tiny

* remove shift factor rejigging from the sd3 docs

* max_sequence_length for the T5

* updated img2img

* apply suggestions

---------
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>
Co-authored-by: YiYi Xu <yixu310@gmail.com>

* Fix sharding when no device_map is passed

* style

* add tests

* align

* add docstring

* format

---------
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>

[Single File Loading] Handle unexpected keys in CLIP models when `accelerate` isn't installed.  (#8462)

* update

* update

* update

* update

* update

---------
Co-authored-by: YiYi Xu <yixu310@gmail.com>
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>