Commits · 2b1786735e27bc97f4d4699712292d5c463a7380 · renzhc / diffusers_dcu

28 Jul, 2023 8 commits

fix fp type in t2i adapter docs (#4350) · 2b178673
Will Berman authored Jul 28, 2023

2b178673

[Feat] Support SDXL Kohya-style LoRA (#4287) · 4a4cdd6b

Sayak Paul authored Jul 28, 2023



* sdxl lora changes.

* better name replacement.

* better replacement.

* debugging

* debugging

* debugging

* debugging

* debugging

* remove print.

* print state dict keys.

* print

* distingisuih better

* debuggable.

* fxi: tyests

* fix: arg from training script.

* access from class.

* run style

* debug

* save intermediate

* some simplifications for SDXL LoRA

* styling

* unet config is not needed in diffusers format.

* fix: dynamic SGM block mapping for SDXL kohya loras (#4322)

* Use lora compatible layers for linear proj_in/proj_out (#4323)

* improve condition for using the sgm_diffusers mapping

* informative comment.

* load compatible keys and embedding layer maaping.

* Get SDXL 1.0 example lora to load

* simplify

* specif ranks and hidden sizes.

* better handling of k rank and hidden

* debug

* debug

* debug

* debug

* debug

* fix: alpha keys

* add check for handling LoRAAttnAddedKVProcessor

* sanity comment

* modifications for text encoder SDXL

* debugging

* debugging

* debugging

* debugging

* debugging

* debugging

* debugging

* debugging

* denugging

* debugging

* debugging

* debugging

* debugging

* debugging

* debugging

* debugging

* debugging

* debugging

* debugging

* debugging

* debugging

* debugging

* debugging

* debugging

* debugging

* debugging

* debugging

* up

* up

* up

* up

* up

* up

* unneeded comments.

* unneeded comments.

* kwargs for the other attention processors.

* kwargs for the other attention processors.

* debugging

* debugging

* debugging

* debugging

* improve

* debugging

* debugging

* more print

* Fix alphas

* debugging

* debugging

* debugging

* debugging

* debugging

* debugging

* clean up

* clean up.

* debugging

* fix: text

---------
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com>
Co-authored-by: Batuhan Taskaya <batuhan@python.org>

4a4cdd6b

[SDXL] Make watermarker optional under certain circumstances to improve... · b7b6d613

Patrick von Platen authored Jul 28, 2023

[SDXL] Make watermarker optional under certain circumstances to improve usability of SDXL 1.0 (#4346)

* improve sdxl

* more fixes

* improve sdxl

* improve sdxl

* improve sdxl

* finish

b7b6d613

Fix repeat of negative prompt (#4335) · faa6cbc9
kathath authored Jul 28, 2023
```
fix repeat of negative prompt
```
faa6cbc9

[ONNX] Don't download ONNX model by default (#4338) · 306a7bd0

Patrick von Platen authored Jul 28, 2023

* [Download] Don't download ONNX weights by default

* [Download] Don't download ONNX weights by default

* [Download] Don't download ONNX weights by default

* fix more

* finish

* finish

* finish

306a7bd0

correct doc string for default value of guidance_scale (#4339) · c7250f2b
Tanupriya Singh authored Jul 28, 2023

c7250f2b

[SDXL Refiner] Fix refiner forward pass for batched input (#4327) · 18b018c8

Patrick von Platen authored Jul 28, 2023

* fix_batch_xl

* Fix other pipelines as well

* up

* up

* Update tests/pipelines/stable_diffusion_xl/test_stable_diffusion_xl_inpaint.py

* sort

* up

* Finish it all up Co-authored-by: Bagheera <bghira@users.github.com>

* Co-authored-by: Bagheera bghira@users.github.com

* Co-authored-by: Bagheera <bghira@users.github.com>

* Finish it all up Co-authored-by: Bagheera <bghira@users.github.com>

18b018c8

Update README_sdxl.md to correct the header (#4330) · 54fab2cd
Sayak Paul authored Jul 28, 2023
```
Update README_sdxl.md
```
54fab2cd

27 Jul, 2023 9 commits
- Honor the SDXL 1.0 licensing from the training scripts. (#4319) · 96117306
  Sayak Paul authored Jul 28, 2023
```
* honor the original license.

* train_instruct_pix2pix_xl -> train_instruct_pix2pix_sdxl
```
  96117306
- [Tests] add test for pipeline import. (#4276) · 7d0d0732
  Sayak Paul authored Jul 28, 2023
```
* add test for pipeline import.

* Update tests/others/test_dependencies.py
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com>

* address suggestions

---------
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com>
```
  7d0d0732
- fix validation option for dreambooth training example (#4317) · 01b6ec21
  Xinyang Li authored Jul 28, 2023
  
  01b6ec21
- Fix typo documentation (#4320) · 92e5ddd2
  Ella Charlaix authored Jul 27, 2023
```
fix typo documentation
```
  92e5ddd2
- [Local loading] Correct bug with local files only (#4318) · 1926331e
  Patrick von Platen authored Jul 27, 2023
```
* [Local loading] Correct bug with local files only

* file not found error

* fix

* finish
```
  1926331e
- fix a bug in StableDiffusionUpscalePipeline when `prompt` is `None` (#4278) · 5fd3dca5
  YiYi Xu authored Jul 27, 2023
```
* fix batch_size

* add test

---------
Co-authored-by: yiyixuxu <yixu310@gmail,com>
```
  5fd3dca5
- Fix SDXL conversion from original to diffusers (#4280) · a2091b70
  Duong A. Nguyen authored Jul 27, 2023
```
* fix sdxl conversion

* convention
```
  a2091b70
- [Torch.compile] Fixes torch compile graph break (#4315) · d8bc1a4e
  Patrick von Platen authored Jul 27, 2023
```
* fix torch compile

* Fix all

* make style
```
  d8bc1a4e
- update Kandinsky doc (#4301) · 80c10d82
  YiYi Xu authored Jul 27, 2023
```
* update doc

* fix an error in autopipe doc

---------
Co-authored-by: yiyixuxu <yixu310@gmail,com>
```
  80c10d82
26 Jul, 2023 7 commits

0.20.0dev0 (#4299) · 20e92586
Patrick von Platen authored Jul 26, 2023
```
* 0.20.0dev0

* make style
```
20e92586
quick fix · 5623ea06
Patrick von Platen authored Jul 26, 2023

5623ea06
quick fix · 16049caf
Patrick von Platen authored Jul 26, 2023

16049caf

Rename (#4294) · 6a6dfe1c

Patrick von Platen authored Jul 26, 2023

* up

* Apply suggestions from code review

* Apply suggestions from code review

* up

6a6dfe1c

add openvino and onnx runtime SD XL documentation (#4285) · b83bdce4

Ella Charlaix authored Jul 26, 2023

* add openvino SD XL documentation

* add onnx SD XL integration

* rephrase

* update doc

* add images

* update model

b83bdce4

Where did this 'x' come from, Elon? (#4277) · c6ae9b7d

camenduru authored Jul 26, 2023



* why mdx?

* why mdx?

* why mdx?

* no x for kandinksy either

---------
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com>

c6ae9b7d

[Kandinsky] Add combined pipelines / Fix cpu model offload / Fix inpainting (#4207) · b3e5cd6b

Patrick von Platen authored Jul 26, 2023



* Add combined pipeline

* Download readme

* Upload

* up

* up

* fix final

* Add enable model cpu offload kandinsky

* finish

* finish

* Fix

* fix more

* make style

* fix kandinsky mask

* fix inpainting test

* add callbacks

* add tests

* fix tests

* Apply suggestions from code review
Co-authored-by: YiYi Xu <yixu310@gmail.com>

* docs

* docs

* correct docs

* fix tests

* add warning

* correct docs

---------
Co-authored-by: YiYi Xu <yixu310@gmail.com>

b3e5cd6b

25 Jul, 2023 15 commits

Fix all missing optional import statements from pipeline folders (#4272) · b37dc3b3
Patrick von Platen authored Jul 26, 2023
```
* fix circular import

* fix imports when watermark not specified

* fix all pipelines
```
b37dc3b3

Load Kohya-ss style LoRAs with auxilary states (#4147) · ff8f5808

Batuhan Taskaya authored Jul 26, 2023



* Support to load Kohya-ss style LoRA file format (without restrictions)
Co-Authored-By: Takuma Mori <takuma104@gmail.com>
Co-Authored-By: Sayak Paul <spsayakpaul@gmail.com>

* tmp: add sdxl to mlp_modules

---------
Co-authored-by: Takuma Mori <takuma104@gmail.com>
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>

ff8f5808

[SDXL DreamBooth LoRA] multiple fixes (#4262) · 161449d5

Sayak Paul authored Jul 26, 2023

* add automatic licensing.

* debugging

* debugging

* more debugging

* more debugging.

* run make fix-copies.

* change to default tracker.

161449d5

[docs] Fix image in SDXL docs (#4267) · 34abee09
Steven Liu authored Jul 25, 2023
```
fix image link
```
34abee09

[SDXL and IP2P]: instruction pix2pix XL training and pipeline (#4079) · 428dbfec

Harutatsu Akiyama authored Jul 25, 2023



* Support instruction pix2pix sdxl

* Support instruction pix2pix sdxl

* Support instruction pix2pix sdxl

* Support instruction pix2pix sdxl

* Support instruction pix2pix sdxl

* Support instruction pix2pix sdxl

* Support instruction pix2pix sdxl

* Support instruction pix2pix sdxl

* Support instruction pix2pix sdxl

* Support instruction pix2pix sdxl

* Support instruction pix2pix sdxl

* Support instruction pix2pix sdxl

* Support instruction pix2pix sdxl

* Support instruction pix2pix sdxl

* Support instruction pix2pix sdxl

* Support instruction pix2pix sdxl

* Support instruction pix2pix sdxl

* [Community] Implementation of the IADB community pipeline (#3996)

* community pipeline: implementation of iadb

* iadb.py: reformat using black

* iadb.py: linting update

* add kandinsky to readme table (#4081)
Co-authored-by: yiyixuxu <yixu310@gmail,com>

* [From Single File] Force accelerate to be installed (#4078)

force accelerate to be installed

* Support instruction pix2pix sdxl

* Support instruction pix2pix sdxl

* Support instruction pix2pix sdxl

* Support instruction pix2pix sdxl

* Support instruction pix2pix sdxl

* Support instruction pix2pix sdxl

* Support instruction pix2pix sdxl

* Support instruction pix2pix sdxl

* Support instruction pix2pix sdxl

* Support instruction pix2pix sdxl

* Support instruction pix2pix sdxl

* Support instruction pix2pix sdxl

* Support instruction pix2pix sdxl

* Support instruction pix2pix sdxl

* Support instruction pix2pix sdxl

* Clean up IP2P SDXL code

* Clean up IP2P SDXL code

* [IP2P and SDXL] clean up code

* [IP2P and SDXL] clean up code

* [IP2P and SDXL] clean up code

* [IP2P SDXL] Address code reviews

* [IP2P SDXL] Address code reviews, add docs, tests

* [IP2P SDXL] Address code reviews, add docs, tests

* [IP2P SDXL] Address code reviews, add docs, tests

* [IP2P SDXL] Address code reviews, add docs, tests

* [IP2P SDXL] Address code reviews, add docs, tests

* [IP2P SDXL] Address code reviews, add docs, tests

* [IP2P SDXL] Address code reviews, add docs, tests

* [IP2P SDXL] Address code reviews, add docs, tests

* [IP2P SDXL] Address code reviews, add docs, tests

* [IP2P SDXL] Address code reviews, add docs, tests

* [IP2P SDXL] Address code reviews, add docs, tests

* [IP2P SDXL] Address code reviews, add docs, tests

* [IP2P SDXL] Address code reviews

* [IP2P SDXL] Address code reviews

* [IP2P SDXL] Add README_SDXL

* [IP2P SDXL] Address code reviews

* [IP2P SDXL] Address code reviews

* [IP2P SDXL] Fix the copy problems

* [IP2P SDXL] Add license

* [IP2P SDXL] Add license

* [IP2P SDXL] Add license

* [IP2P SDXL] Address code reivew for selecting VAE andd others

* [IP2P SDXL] Update README_sdxl

* [IP2P SDXL] Update __init__

* [IP2P SDXL] Update dummy_torch_and_transformers_and_invisible_watermark_objects

* address patrick's comments and some additions to readmes.

---------
Co-authored-by: Harutatsu Akiyama <kf.zy.qin@gmail.com>
Co-authored-by: Thomas Chambon <36728882+tchambon@users.noreply.github.com>
Co-authored-by: YiYi Xu <yixu310@gmail.com>
Co-authored-by: yiyixuxu <yixu310@gmail,com>
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com>
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>

428dbfec

Model path for sdxl wrong in dreambooth README (#4261) · 4e2a0218
Ragnar Rova authored Jul 25, 2023

4e2a0218
[from_single_file] Fix circular import (#4259) · ebfe3431
Patrick von Platen authored Jul 25, 2023
```
* up

* finish

* fix final
```
ebfe3431
Update README_sdxl.md to change the note on default hyperparameters (#4258) · 5ef6b8fa
Sayak Paul authored Jul 25, 2023

5ef6b8fa

[draft v2] AutoPipeline (#4138) · c11d11d6

YiYi Xu authored Jul 25, 2023



* initial

* style

* from ...pipelines -> from ..pipeline_util

* make style

* fix-copies

* fix value_guided_sampling oops

* style

* add test

* Show failing test

* update from_pipe

* fix

* add controlnet, additional test and register unused original config

* update for controlnet

* Apply suggestions from code review
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com>

* store unused config as private attribute and pass if can

* add doc

* kandinsky inpaint pipeline does not work with decoder checkpoint

* update doc

* Apply suggestions from code review
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>

* style

* Apply suggestions from code review
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com>

* fix

* Apply suggestions from code review

---------
Co-authored-by: yiyixuxu <yixu310@gmail,com>
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com>
Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>

c11d11d6

[SDXL] Improve docs (#4196) · d74561da

Patrick von Platen authored Jul 25, 2023



* Improve docs

* Correct docs

* Add better example inpaint

* make style

* Apply suggestions from code review
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>

* fix

---------
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>

d74561da

[From Single File] Allow vae to be loaded (#4242) · a0422ed0
Patrick von Platen authored Jul 25, 2023
```
* Allow vae to be loaded

* up
```
a0422ed0
do not pass list to accelerator.init_trackers (#4248) · 3dd33937
Will Berman authored Jul 24, 2023

3dd33937

Resolve bf16 error as mentioned in this... · 5652c43f

nupurkmr9 authored Jul 24, 2023

Resolve bf16 error as mentioned in this [issue](https://github.com/huggingface/diffusers/issues/4139#issuecomment-1639977304) (#4214)

* resolve bf16 error

* resolve bf16 error

* resolve bf16 error

* resolve bf16 error

* resolve bf16 error

* resolve bf16 error

* resolve bf16 error

5652c43f

[SDXL DreamBooth LoRA] add support for text encoder fine-tuning (#4097) · 365e8461

Sayak Paul authored Jul 25, 2023



* Allow low precision sd xl

* finish

* finish

* feat: initial draft for supporting text encoder lora finetuning for SDXL DreamBooth

* fix: variable assignments.

* add: autocast block.

* add debugging

* vae dtype hell

* fix: vae dtype hell.

* fix: vae dtype hell 3.

* clean up

* lora text encoder loader.

* fix: unwrapping models.

* add: tests.

* docs.

* handle unexpected keys.

* fix vae dtype in the final inference.

* fix scope problem.

* fix: save_model_card args.

* initialize: prefix to None.

* fix: dtype issues.

* apply gixes.

* debgging.

* debugging

* debugging

* debugging

* debugging

* debugging

* add: fast tests.

* pre-tokenize.

* address: will's comments.

* fix: loader and tests.

* fix: dataloader.

* simplify dataloader.

* length.

* simplification.

* make style && make quality

* simplify state_dict munging

* fix: tests.

* fix: state_dict packing.

* Apply suggestions from code review
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com>

---------
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com>

365e8461

[ControlNet SDXL training] fixes in the training script (#4223) · fed12376

Sayak Paul authored Jul 25, 2023

* fix: #4206

* add: sdxl controlnet training smoketest.

* remove unnecessary token inits.

* add: licensing to model card.

* include SDXL licensing in the model card and make public visibility default

* debugging

* debugging

* disable local file download.

* fix: training test.

* fix: ckpt prefix.

fed12376

24 Jul, 2023 1 commit
- [Docs] Fix from pretrained docs (#4240) · 95b7de88
  Patrick von Platen authored Jul 24, 2023
```
* [Docs] Fix from pretrained docs

* [Docs] Fix from pretrained docs
```
  95b7de88