GitLab

* Post Release: v0.24.0

* post pone deprecation

* post pone deprecation

* Add model_index.json

* Update README.md

* Update README.md

* Add files via upload

* Update README.md

* Update examples/community/README.md

---------
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com>

readme

[examples/advanced_diffusion_training] bug fixes and improvements for LoRA Dreambooth SDXL advanced training script (#5935)

* imports and readme bug fixes

* bug fix - ensures text_encoder params are dtype==float32 (when using pivotal tuning) even if the rest of the model is loaded in fp16

* added pivotal tuning to readme

* mapping token identifier to new inserted token in validation prompt (if used)

* correct default value of --train_text_encoder_frac

* change default value of  --adam_weight_decay_text_encoder

* validation prompt generations when using pivotal tuning bug fix

* style fix

* textual inversion embeddings name change

* style fix

* bug fix - stopping text encoder optimization halfway

* readme - will include token abstraction and new inserted tokens when using pivotal tuning
- added type to --num_new_tokens_per_abstraction

* style fix

---------
Co-authored-by: Linoy Tsaban <linoy@huggingface.co>

* make `requires_safety_checker` a kwarg instead of a positional argument as it's more future-proof

* apply `make style` formatting edits

* add image_encoder to arguments and pass to super constructor

* Adapt lora example scripts to use PEFT

* add to_out.0

Fix typo in output_dir argument: "text-inversion-model" → "dreambooth-model"

* bug in MultiAdapter for Inpainting

* adapter_input is a list for MultiAdapter

---------
Co-authored-by: andres <andres@hax.ai>
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>

* control and custom diffusion

* dreambooth

* instructpix2pix and dreambooth ckpting

* t2i adapters.

* text to image ft

* textual inversion

* unconditional

* workflows

* import fix

* fix import

---------
Co-authored-by: Aflalo <estellea@isl-gpu27.rr.intel.com>
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com>
Co-authored-by: YiYi Xu <yixu310@gmail.com>

* [community pipeline] dps impl

* add type checking

* pass ruff check

* ruff formatter

fix: error on device for `lpw_stable_diffusion_xl` pipeline if `pipe.enable_sequential_cpu_offload()` enabled (#5885)

fix: set device for pipe.enable_sequential_cpu_offload()

add IPEX pipeline LoRA weights loading support

* Fix bug related to parsing unet_time_cond_proj_dim.

* Fix analogous bug in the SD-XL LCM distillation script.

Signed-off-by: Wang, Yi A <yi.a.wang@intel.com>

* add variant

* add variant

* Apply suggestions from code review

* reformat

* fix: textual_inversion.py

* fix: variant in model_info

---------
Co-authored-by: sayakpaul <spsayakpaul@gmail.com>

readme improvement and metadata fix

* readme bug fix

* style fix

---------
Co-authored-by: Linoy Tsaban <linoy@huggingface.co>

Adds an advanced version of the SD-XL DreamBooth LoRA training script supporting pivotal tuning  (#5883)

* sdxl dreambooth lora training script with pivotal tuning

* bug fix - args missing from parse_args

* code quality fixes

* comment unnecessary code from TokenEmbedding handler class

* fixup

---------
Co-authored-by: Linoy Tsaban <linoy@huggingface.co>

* ControlNet+Adapter pipeline, and +Inpaint pipeline


---------
Co-authored-by: andres <andres@hax.ai>

* [Lora] Seperate logic

* [Lora] Seperate logic

* [Lora] Seperate logic

* add comments to explain the code better

* add comments to explain the code better

* Additions:
- support for different lr for text encoder
- support for Prodigy optimizer
- support for min snr gamma
- support for custom captions and dataset loading from the hub

* adjusted --caption_column behaviour (to -not- use the second column of the dataset by default if --caption_column is not provided)

* fixed --output_dir / --model_dir_name confusion

* added --repeats, --adam_weight_decay_text_encoder
+ some fixes

* Update examples/dreambooth/train_dreambooth_lora_sdxl.py
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com>

* Update examples/dreambooth/train_dreambooth_lora_sdxl.py
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com>

* Update examples/dreambooth/train_dreambooth_lora_sdxl.py
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com>

* - import compute_snr from diffusers/training_utils.py
- cluster adamw together
- when using 'prodigy', if --train_text_encoder == True and --text_encoder_lr != --learning rate, changes the lr of the text encoders optimization params to be --learning_rate (otherwise errors)

* shape fixes when custom captions are used

* formatting and a little cleanup

* code styling

* --repeats default value fixed, changed to 1

* bug fix - removed redundant lines of embedding concatenation when using prior_preservation (that duplicated class_prompt embeddings)

* changed dataset loading logic according to the following usecases (to avoid unnecessary dependency on datasets)-
1. user provides --dataset_name
2. user provides local dir --instance_data_dir that contains a metadata .jsonl file
3. user provides local dir --instance_data_dir that contains only images
in cases [1,2] we import datasets and use load_dataset method, in case [3] we process the data same as in the original script setting

* styling fix

* arg name fix

* adjusted the --repeats logic

* -removed redundant arg and 'if' when loading local folder with prompts
-updated readme template
-some default val fixes
-custom caption tests

* image path fix for readme

* code style

* bug fix

* --caption_column arg

* readme fix

---------
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com>
Co-authored-by: Linoy Tsaban <linoy@huggingface.co>

* fix an issue that ipex occupy too much memory, it will not impact performance

* make style

---------
Co-authored-by: root <jun.chen@intel.com>
Co-authored-by: Meng Guoqing <guoqing.meng@intel.com>

* wip: add interpolate pipeline for lcm

* update documentation

* update documentation

* ruff format

* not need to use doc-builder's black styling as the doc is styled in ruff

* make fix-copies

* comment

* use run_ruff

* Set usedforsecurity=False in hashlib methods (FIPS compliance)

* update version dependency

* bump hfh version

* bump hfh version

* fix: enabled num_images_per_prompt>1 for lpw_stable_diffusion_xl

* style: fixed isort

* 🔧 Fix import codes in diffusers library

* ✨Refactor imports in community examples

* refactor loaders.py to make it cleaner and leaner.

* refactor loaders init

* inits.

* textual inversion to the init.

* inits.

* remove certain modules from the main init.

* AttnProcsLayers

* fix imports

* avoid circular import.

* fix circular import pt 2.

* address PR comments

* imports

* fix: imports.

* remove from main init for avoiding circular deps.

* remove spurious deps.

* fix-copies.

* fix imports.

* more debug

* more debug

* Apply suggestions from code review

* Apply suggestions from code review

---------
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com>

Fix the pipeline name in the examples for LMD+ pipeline. Add a colab link to pipeline README. (#5775)

* Fix the pipeline name in the examples for LMD+ pipeline

* Add LMD+ colab link

* Apply code formatting

---------
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>

Correct code for distributed training

fix: ignore other args
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com>

fix styling issues

* Add LLM-grounded Diffusion (LMD+) pipeline

* Update the formatting

* Applied formatting