1. 02 Nov, 2023 2 commits
    • Dhruv Nair's avatar
      Animatediff Proposal (#5413) · 2a8cf8e3
      Dhruv Nair authored
      * draft design
      
      * clean up
      
      * clean up
      
      * clean up
      
      * clean up
      
      * clean up
      
      * clean  up
      
      * clean up
      
      * clean up
      
      * clean up
      
      * update pipeline
      
      * clean up
      
      * clean up
      
      * clean up
      
      * add tests
      
      * change motion block
      
      * clean up
      
      * clean up
      
      * clean up
      
      * update
      
      * update
      
      * update
      
      * update
      
      * update
      
      * update
      
      * update
      
      * update
      
      * clean up
      
      * update
      
      * update
      
      * update model test
      
      * update
      
      * update
      
      * update
      
      * update
      
      * make style
      
      * update
      
      * fix embeddings
      
      * update
      
      * merge upstream
      
      * max fix copies
      
      * fix bug
      
      * fix mistake
      
      * add docs
      
      * update
      
      * clean up
      
      * update
      
      * clean up
      
      * clean up
      
      * fix docstrings
      
      * fix docstrings
      
      * update
      
      * update
      
      * clean  up
      
      * update
      2a8cf8e3
    • Sayak Paul's avatar
      [remote code] document trust remote code. (#5620) · b81f709f
      Sayak Paul authored
      document trust remote code.
      b81f709f
  2. 01 Nov, 2023 2 commits
  3. 27 Oct, 2023 1 commit
  4. 25 Oct, 2023 1 commit
    • Steven Liu's avatar
      [docs] Loader docs (#5473) · bc8a08f6
      Steven Liu authored
      * first draft
      
      * make fix-copies
      
      * add peft section
      
      * manual fix
      
      * make fix-copies again
      
      * manually revert changes to other files
      bc8a08f6
  5. 24 Oct, 2023 3 commits
    • dg845's avatar
      Add Latent Consistency Models Pipeline (#5448) · 958e17da
      dg845 authored
      
      
      * initial commit for LatentConsistencyModelPipeline and LCMScheduler based on the community pipeline
      
      * Add callback and freeu support.
      
      * apply suggestions from review
      
      * Clean up LCMScheduler
      
      * Remove timeindex argument to LCMScheduler.step.
      
      * Add support for clipping or thresholding the predicted original sample.
      
      * Remove unused methods and arguments in LCMScheduler.
      
      * Improve comment about (lack of) negative prompt support.
      
      * Change input guidance_scale to match the StableDiffusionPipeline (Imagen) CFG formulation.
      
      * Move lcm_origin_steps from pipeline __call__ to LCMScheduler.__init__/config (as origin_steps).
      
      * Fix typo when clipping/thresholding in LCMScheduler.
      
      * Add some initial LCMScheduler tests.
      
      * add type annotations from review
      
      * Fix type annotation bug.
      
      * Override test_add_noise_device in LCMSchedulerTest since hardcoded timesteps doesn't work under default settings.
      
      * Add generator argument pipeline prepare_latents call.
      
      * Cast LCMScheduler.timesteps to long in set_timesteps.
      
      * Add onestep and multistep full loop scheduler tests.
      
      * Set default height/width to None and don't hardcode guidance scale embedding dim.
      
      * Add initial LatentConsistencyPipeline fast and slow tests.
      
      * Add initial documentation for LatentConsistencyModelPipeline and LCMScheduler.
      
      * Make remaining failing fast tests pass.
      
      * make style
      
      * Make original_inference_steps configurable from pipeline __call__ again.
      
      * make style
      
      * Remove guidance_rescale arg from pipeline __call__ since LCM currently doesn't support CFG.
      
      * Make LCMScheduler defaults match config of LCM_Dreamshaper_v7 checkpoint.
      
      * Fix LatentConsistencyPipeline slow tests and add dummy expected slices.
      
      * Add checks for original_steps in LCMScheduler.set_timesteps.
      
      * make fix-copies
      
      * Improve LatentConsistencyModelPipeline docs.
      
      * Apply suggestions from code review
      Co-authored-by: default avatarAryan V S <avs050602@gmail.com>
      
      * Apply suggestions from code review
      Co-authored-by: default avatarAryan V S <avs050602@gmail.com>
      
      * Apply suggestions from code review
      Co-authored-by: default avatarAryan V S <avs050602@gmail.com>
      
      * Update src/diffusers/schedulers/scheduling_lcm.py
      
      * Apply suggestions from code review
      Co-authored-by: default avatarAryan V S <avs050602@gmail.com>
      
      * finish
      
      ---------
      Co-authored-by: default avatarSayak Paul <spsayakpaul@gmail.com>
      Co-authored-by: default avatarPatrick von Platen <patrick.v.platen@gmail.com>
      Co-authored-by: default avatarAryan V S <avs050602@gmail.com>
      958e17da
    • Steven Liu's avatar
      [docs] General updates (#5378) · 7c3a75a1
      Steven Liu authored
      * first draft
      
      * feedback
      
      * feedback
      7c3a75a1
    • Sayak Paul's avatar
      [Core] Refactor activation and normalization layers (#5493) · 77241c48
      Sayak Paul authored
      
      
      * move out the activations.
      
      * move normalization layers.
      
      * add doc.
      
      * add doc.
      
      * fix: paths
      
      * Apply suggestions from code review
      Co-authored-by: default avatarSteven Liu <59462357+stevhliu@users.noreply.github.com>
      
      * style
      
      ---------
      Co-authored-by: default avatarSteven Liu <59462357+stevhliu@users.noreply.github.com>
      77241c48
  6. 16 Oct, 2023 1 commit
  7. 05 Oct, 2023 2 commits
    • Sayak Paul's avatar
      add: entry for DDPO support. (#5250) · e6faf607
      Sayak Paul authored
      * add: entry for DDPO support.
      
      * move to training
      
      * address steven's comments./
      e6faf607
    • Kadir Nar's avatar
      [Core] Add FreeU mechanism (#5164) · 84b82a6c
      Kadir Nar authored
      *  Added Fourier filter function to upsample blocks
      
      * 🔧 Update Fourier_filter for float16 support
      
      *  Added UNetFreeUConfig to UNet model for FreeU adaptation 🛠
      
      ️
      
      * move unet to its original form and add fourier_filter to torch_utils.
      
      * implement freeU enable mechanism
      
      * implement disable mechanism
      
      * resolution index.
      
      * correct resolution idx condition.
      
      * fix copies.
      
      * no need to use resolution_idx in vae.
      
      * spell out the kwargs
      
      * proper config property
      
      * fix attribution setting
      
      * place unet hasattr properly.
      
      * fix: attribute access.
      
      * proper disable
      
      * remove validation method.
      
      * debug
      
      * debug
      
      * debug
      
      * debug
      
      * debug
      
      * debug
      
      * potential fix.
      
      * add: doc.
      
      * fix copies
      
      * add: tests.
      
      * add: support freeU in SDXL.
      
      * set default value of resolution idx.
      
      * set default values for resolution_idx.
      
      * fix copies
      
      * fix rest.
      
      * fix copies
      
      * address PR comments.
      
      * run fix-copies
      
      * move apply_free_u to utils and other minors.
      
      * introduce support for video (unet3D)
      
      * minor ups
      
      * consistent fix-copies.
      
      * consistent stuff
      
      * fix-copies
      
      * add: rest
      
      * add: docs.
      
      * fix: tests
      
      * fix: doc path
      
      * Apply suggestions from code review
      Co-authored-by: default avatarSteven Liu <59462357+stevhliu@users.noreply.github.com>
      
      * style up
      
      * move to techniques.
      
      * add: slow test for sd freeu.
      
      * add: slow test for sd freeu.
      
      * add: slow test for sd freeu.
      
      * add: slow test for sd freeu.
      
      * add: slow test for sd freeu.
      
      * add: slow test for sd freeu.
      
      * add: slow test for video with freeu
      
      * add: slow test for video with freeu
      
      * add: slow test for video with freeu
      
      * style
      
      ---------
      Co-authored-by: default avatarSayak Paul <spsayakpaul@gmail.com>
      Co-authored-by: default avatarPatrick von Platen <patrick.v.platen@gmail.com>
      Co-authored-by: default avatarSteven Liu <59462357+stevhliu@users.noreply.github.com>
      84b82a6c
  8. 21 Sep, 2023 1 commit
    • Ayush Mangal's avatar
      Add BLIP Diffusion (#4388) · 157c9011
      Ayush Mangal authored
      
      
      * Add BLIP Diffusion skeleton
      
      * Add other model components
      
      * Add BLIP2, need to change it for now
      
      * Fix pipeline imports
      
      * Load pretrained ViT
      
      * Make qformer fwd pass same
      
      * Replicate fwd passes
      
      * Fix device bug
      
      * Add accelerate functions
      
      * Remove extra functions from Blip2
      
      * Minor bug
      
      * Integrate initial review changes
      
      * Refactoring
      
      * Refactoring
      
      * Refactor
      
      * Add controlnet
      
      * Refactor
      
      * Update conversion script
      
      * Add image processor
      
      * Shift postprocessing to ImageProcessor
      
      * Refactor
      
      * Fix device
      
      * Add fast tests
      
      * Update conversion script
      
      * Fix checkpoint conversion script
      
      * Integrate review changes
      
      * Integrate reivew changes
      
      * Remove unused functions from test
      
      * Reuse HF image processor in Cond image
      
      * Create new BlipImageProcessor based on transfomers
      
      * Fix image preprocessor
      
      * Minor
      
      * Minor
      
      * Add canny preprocessing
      
      * Fix controlnet preprocessing
      
      * Fix blip diffusion test
      
      * Add controlnet test
      
      * Add initial doc strings
      
      * Integrate review changes
      
      * Refactor
      
      * Update examples
      
      * Remove DDIM comments
      
      * Add copied from for prepare_latents
      
      * Add type anotations
      
      * Add docstrings
      
      * Do black formatting
      
      * Add batch support
      
      * Make tests pass
      
      * Make controlnet tests pass
      
      * Black formatting
      
      * Fix progress bar
      
      * Fix some licensing comments
      
      * Fix imports
      
      * Refactor controlnet
      
      * Make tests faster
      
      * Edit examples
      
      * Black formatting/Ruff
      
      * Add doc
      
      * Minor
      Co-authored-by: default avatarPatrick von Platen <patrick.v.platen@gmail.com>
      
      * Move controlnet pipeline
      
      * Make tests faster
      
      * Fix imports
      
      * Fix formatting
      
      * Fix make errors
      
      * Fix make errors
      
      * Minor
      
      * Add suggested doc changes
      Co-authored-by: default avatarSayak Paul <spsayakpaul@gmail.com>
      
      * Edit docs
      
      * Fix 16 bit loading
      
      * Update examples
      
      * Edit toctree
      
      * Update docs/source/en/api/pipelines/blip_diffusion.md
      Co-authored-by: default avatarSayak Paul <spsayakpaul@gmail.com>
      
      * Minor
      
      * Add tips
      
      * Edit examples
      
      * Update model paths
      
      ---------
      Co-authored-by: default avatarPatrick von Platen <patrick.v.platen@gmail.com>
      Co-authored-by: default avatarSayak Paul <spsayakpaul@gmail.com>
      157c9011
  9. 13 Sep, 2023 1 commit
  10. 08 Sep, 2023 1 commit
    • Suraj Patil's avatar
      [examples] T2IAdapter training script (#4934) · dfec61f4
      Suraj Patil authored
      
      
      * add t2i_example script
      
      * remove in channels logic
      
      * remove comments
      
      * remove use_euler arg
      
      * add requirements
      
      * only use canny example
      
      * use datasets
      
      * comments
      
      * make log_validation consistent with other scripts
      
      * add readme
      
      * fix title in readme
      
      * update check_min_version
      
      * change a few minor things.
      
      * add doc entry
      
      * add: test for t2i adapter training
      
      * remove use_auth_token
      
      * fix: logged info.
      
      * remove tests for now.
      
      ---------
      Co-authored-by: default avatarSayak Paul <spsayakpaul@gmail.com>
      dfec61f4
  11. 06 Sep, 2023 1 commit
    • Kashif Rasul's avatar
      Würstchen model (#3849) · 541bb6ee
      Kashif Rasul authored
      
      
      * initial
      
      * initial
      
      * added initial convert script for paella vqmodel
      
      * initial wuerstchen pipeline
      
      * add LayerNorm2d
      
      * added modules
      
      * fix typo
      
      * use model_v2
      
      * embed clip caption amd negative_caption
      
      * fixed name of var
      
      * initial modules in one place
      
      * WuerstchenPriorPipeline
      
      * inital shape
      
      * initial denoising prior loop
      
      * fix output
      
      * add WuerstchenPriorPipeline to __init__.py
      
      * use the noise ratio in the Prior
      
      * try to save pipeline
      
      * save_pretrained working
      
      * Few additions
      
      * add _execution_device
      
      * shape is int
      
      * fix batch size
      
      * fix shape of ratio
      
      * fix shape of ratio
      
      * fix output dataclass
      
      * tests folder
      
      * fix formatting
      
      * fix float16 + started with generator
      
      * Update pipeline_wuerstchen.py
      
      * removed vqgan code
      
      * add WuerstchenGeneratorPipeline
      
      * fix WuerstchenGeneratorPipeline
      
      * fix docstrings
      
      * fix imports
      
      * convert generator pipeline
      
      * fix convert
      
      * Work on Generator Pipeline. WIP
      
      * Pipeline works with our diffuzz code
      
      * apply scale factor
      
      * removed vqgan.py
      
      * use cosine schedule
      
      * redo the denoising loop
      
      * Update src/diffusers/models/resnet.py
      Co-authored-by: default avatarPatrick von Platen <patrick.v.platen@gmail.com>
      
      * use torch.lerp
      
      * use warp-diffusion org
      
      * clip_sample=False,
      
      * some refactoring
      
      * use model_v3_stage_c
      
      * c_cond size
      
      * use clip-bigG
      
      * allow stage b clip to be None
      
      * add dummy
      
      * würstchen scheduler
      
      * minor changes
      
      * set clip=None in the pipeline
      
      * fix attention mask
      
      * add attention_masks to text_encoder
      
      * make fix-copies
      
      * add back clip
      
      * add text_encoder
      
      * gen_text_encoder and tokenizer
      
      * fix import
      
      * updated pipeline test
      
      * undo changes to pipeline test
      
      * nip
      
      * fix typo
      
      * fix output name
      
      * set guidance_scale=0 and remove diffuze
      
      * fix doc strings
      
      * make style
      
      * nip
      
      * removed unused
      
      * initial docs
      
      * rename
      
      * toc
      
      * cleanup
      
      * remvoe test script
      
      * fix-copies
      
      * fix multi images
      
      * remove dup
      
      * remove unused modules
      
      * undo changes for debugging
      
      * no  new line
      
      * remove dup conversion script
      
      * fix doc string
      
      * cleanup
      
      * pass default args
      
      * dup permute
      
      * fix some tests
      
      * fix prepare_latents
      
      * move Prior class to modules
      
      * offload only the text encoder and vqgan
      
      * fix resolution calculation for prior
      
      * nip
      
      * removed testing script
      
      * fix shape
      
      * fix argument to set_timesteps
      
      * do not change .gitignore
      
      * fix resolution calculations + readme
      
      * resolution calculation fix + readme
      
      * small fixes
      
      * Add combined pipeline
      
      * rename generator -> decoder
      
      * Update .gitignore
      Co-authored-by: default avatarPatrick von Platen <patrick.v.platen@gmail.com>
      
      * removed efficient_net
      
      * create combined WuerstchenPipeline
      
      * make arguments consistent with VQ model
      
      * fix var names
      
      * no need to return text_encoder_hidden_states
      
      * add latent_dim_scale to config
      
      * split model into its own file
      
      * add WuerschenPipeline to docs
      
      * remove unused latent_size
      
      * register latent_dim_scale
      
      * update script
      
      * update docstring
      
      * use Attention preprocessor
      
      * concat with normed input
      
      * fix-copies
      
      * add docs
      
      * fix test
      
      * fix style
      
      * add to cpu_offloaded_model
      
      * updated type
      
      * remove 1-line func
      
      * updated type
      
      * initial decoder test
      
      * formatting
      
      * formatting
      
      * fix autodoc link
      
      * num_inference_steps is int
      
      * remove comments
      
      * fix example in docs
      
      * Update src/diffusers/pipelines/wuerstchen/diffnext.py
      Co-authored-by: default avatarPatrick von Platen <patrick.v.platen@gmail.com>
      
      * rename layernorm to WuerstchenLayerNorm
      
      * rename DiffNext to WuerstchenDiffNeXt
      
      * added comment about MixingResidualBlock
      
      * move paella vq-vae to pipelines' folder
      
      * initial decoder test
      
      * increased test_float16_inference expected diff
      
      * self_attn is always true
      
      * more passing decoder tests
      
      * batch image_embeds
      
      * fix failing tests
      
      * set the correct dtype
      
      * relax inference test
      
      * update prior
      
      * added combined pipeline test
      
      * faster test
      
      * faster test
      
      * Update src/diffusers/pipelines/wuerstchen/pipeline_wuerstchen_combined.py
      Co-authored-by: default avatarPatrick von Platen <patrick.v.platen@gmail.com>
      
      * fix issues from review
      
      * update wuerstchen.md + change generator name
      
      * resolve issues
      
      * fix copied from usage and add back batch_size
      
      * fix API
      
      * fix arguments
      
      * fix combined test
      
      * Added timesteps argument + fixes
      
      * Update tests/pipelines/test_pipelines_common.py
      Co-authored-by: default avatarPatrick von Platen <patrick.v.platen@gmail.com>
      
      * Update tests/pipelines/wuerstchen/test_wuerstchen_prior.py
      Co-authored-by: default avatarPatrick von Platen <patrick.v.platen@gmail.com>
      
      * Update src/diffusers/pipelines/wuerstchen/pipeline_wuerstchen_combined.py
      Co-authored-by: default avatarPatrick von Platen <patrick.v.platen@gmail.com>
      
      * Update src/diffusers/pipelines/wuerstchen/pipeline_wuerstchen_combined.py
      Co-authored-by: default avatarPatrick von Platen <patrick.v.platen@gmail.com>
      
      * Update src/diffusers/pipelines/wuerstchen/pipeline_wuerstchen_combined.py
      Co-authored-by: default avatarPatrick von Platen <patrick.v.platen@gmail.com>
      
      * Update src/diffusers/pipelines/wuerstchen/pipeline_wuerstchen_combined.py
      
      * up
      
      * Fix more
      
      * failing tests
      
      * up
      
      * up
      
      * correct naming
      
      * correct docs
      
      * correct docs
      
      * fix test params
      
      * correct docs
      
      * fix classifier free guidance
      
      * fix classifier free guidance
      
      * fix more
      
      * fix all
      
      * make tests faster
      
      ---------
      Co-authored-by: default avatarDominic Rampas <d6582533@gmail.com>
      Co-authored-by: default avatarPatrick von Platen <patrick.v.platen@gmail.com>
      Co-authored-by: default avatarDominic Rampas <61938694+dome272@users.noreply.github.com>
      541bb6ee
  12. 02 Sep, 2023 1 commit
  13. 01 Sep, 2023 1 commit
  14. 31 Aug, 2023 1 commit
    • Steven Liu's avatar
      [docs] ControlNet guide (#4640) · aedd7876
      Steven Liu authored
      * first draft
      
      * finish first draft
      
      * feedback and remove sections from API pages
      
      * clean docstrings
      
      * add full code example
      aedd7876
  15. 30 Aug, 2023 1 commit
    • Steven Liu's avatar
      [docs] SDXL (#4428) · a1fdfca3
      Steven Liu authored
      * first draft
      
      * reorg toctree
      
      * note about minsdxl
      
      * feedback
      
      * fix
      
      * micro-conditionings
      
      * add tip
      
      * fix section levels
      
      * d'oh fix pipeline names
      
      * feedback
      
      * remove old section
      a1fdfca3
  16. 25 Aug, 2023 1 commit
    • Sanchit Gandhi's avatar
      Convert MusicLDM (#4579) · b1290d3f
      Sanchit Gandhi authored
      
      
      * from audioldm
      
      * fix vae
      
      * move to new pipeline
      
      * copied from audioldm
      
      * remove redundant control flow
      
      * iterate
      
      * fix docstring
      
      * finish pipeline
      
      * tests: from audioldm2
      
      * iterate
      
      * finish fast tests
      
      * finish slow integration tests
      
      * add docs
      
      * remove dtype test
      
      * update toctree
      
      * "copied from" in conversion (where possible)
      
      * Update docs/source/en/api/pipelines/musicldm.md
      Co-authored-by: default avatarPatrick von Platen <patrick.v.platen@gmail.com>
      
      * fix docstring
      
      * make nightly
      
      * style
      
      * fix dtype test
      
      ---------
      Co-authored-by: default avatarPatrick von Platen <patrick.v.platen@gmail.com>
      b1290d3f
  17. 21 Aug, 2023 1 commit
    • Sanchit Gandhi's avatar
      Add AudioLDM 2 (#4549) · 7a24977c
      Sanchit Gandhi authored
      
      
      * from audioldm
      
      * unet down + mid
      
      * vae, clap, flan-t5
      
      * start sequence audio mae
      
      * iterate on audioldm encoder
      
      * finish encoder
      
      * finish weight conversion
      
      * text pre-processing
      
      * gpt2 pre-processing
      
      * fix projection model
      
      * working
      
      * unet equivalence
      
      * finish in base
      
      * add unet cond
      
      * finish unet
      
      * finish custom unet
      
      * start clean-up
      
      * revert base unet changes
      
      * refactor pre-processing
      
      * tests: from audioldm
      
      * fix some tests
      
      * more fixes
      
      * iterate on tests
      
      * make fix copies
      
      * harden fast tests
      
      * slow integration tests
      
      * finish tests
      
      * update checkpoint
      
      * update copyright
      
      * docs
      
      * remove outdated method
      
      * add docstring
      
      * make style
      
      * remove decode latents
      
      * enable cpu offload
      
      * (text_encoder_1, tokenizer_1) -> (text_encoder, tokenizer)
      
      * more clean up
      
      * more refactor
      
      * build pr docs
      
      * Update docs/source/en/api/pipelines/audioldm2.md
      Co-authored-by: default avatarSayak Paul <spsayakpaul@gmail.com>
      
      * small clean
      
      * tidy conversion
      
      * update for large checkpoint
      
      * generate -> generate_language_model
      
      * full clap model
      
      * shrink clap-audio in tests
      
      * fix large integration test
      
      * fix fast tests
      
      * use generation config
      
      * make style
      
      * update docs
      
      * finish docs
      
      * finish doc
      
      * update tests
      
      * fix last test
      
      * syntax
      
      * finalise tests
      
      * refactor projection model in prep for TTS
      
      * fix fast tests
      
      * style
      
      ---------
      Co-authored-by: default avatarSayak Paul <spsayakpaul@gmail.com>
      7a24977c
  18. 16 Aug, 2023 2 commits
    • Steven Liu's avatar
      [docs] PushToHubMixin (#4622) · 4ff7264d
      Steven Liu authored
      * push to hub docs
      
      * fix typo
      
      * feedback
      
      * make style
      4ff7264d
    • nikhil-masterful's avatar
      Add GLIGEN implementation (#4441) · da5ab51d
      nikhil-masterful authored
      * Add GLIGEN implementation
      
      * GLIGEN: Fix code quality check failures
      
      * GLIGEN: Fix Import block un-sorted or un-formatted failures
      
      * GLIGEN: Fix check_repository_consistency failures
      
      * GLIGEN: Add 'PositionNet' to versatile_diffusion/modeling_text_unet.py
      
      * GLIGEN: check_repository_consistency: fix 'copy does not match' error
      
      * GLIGEN: Fix review comments (1)
      
      * GLIGEN: Fix E721 Do not compare types, use `isinstance()` failures
      
      * GLIGEN : Ensure _encode_prompt() copy matches to StableDiffusionPipeline
      
      * GLIGEN: Fix ruff E721 failure in unidiffuser/test_unidiffuser.py
      
      * GLIGEN: doc_builder: restyle pipeline_stable_diffusion_gligen.py
      
      * GIGLEN: reset files unrelated to gligen
      
      * GLIGEN: Fix documentation comments (1)
      
      * GLIGEN: Fix review comments (2)
      
      * GLIGEN: Added FastTest
      
      * GLIGEN: Fix review comments (3)
      da5ab51d
  19. 10 Aug, 2023 2 commits
  20. 09 Aug, 2023 1 commit
    • Steven Liu's avatar
      [docs] Clean scheduler api (#4204) · 16ad13b6
      Steven Liu authored
      * clean scheduler mixin
      
      * up to dpmsolvermultistep
      
      * finish cleaning
      
      * first draft
      
      * fix overview table
      
      * apply feedback
      
      * update reference code
      16ad13b6
  21. 03 Aug, 2023 1 commit
  22. 02 Aug, 2023 2 commits
    • Sayak Paul's avatar
      [Feat] add tiny Autoencoder for (almost) instant decoding (#4384) · 18fc40c1
      Sayak Paul authored
      
      
      * add: model implementation of tiny autoencoder.
      
      * add: inits.
      
      * push the latest devs.
      
      * add: conversion script and finish.
      
      * add: scaling factor args.
      
      * debugging
      
      * fix denormalization.
      
      * fix: positional argument.
      
      * handle use_torch_2_0_or_xformers.
      
      * handle post_quant_conv
      
      * handle dtype
      
      * fix: sdxl image processor for tiny ae.
      
      * fix: sdxl image processor for tiny ae.
      
      * unify upcasting logic.
      
      * copied from madness.
      
      * remove trailing whitespace.
      
      * set is_tiny_vae = False
      
      * address PR comments.
      
      * change to AutoencoderTiny
      
      * make act_fn an str throughout
      
      * fix: apply_forward_hook decorator call
      
      * get rid of the special is_tiny_vae flag.
      
      * directly scale the output.
      
      * fix dummies?
      
      * fix: act_fn.
      
      * get rid of the Clamp() layer.
      
      * bring back copied from.
      
      * movement of the blocks to appropriate modules.
      
      * add: docstrings to AutoencoderTiny
      
      * add: documentation.
      
      * changes to the conversion script.
      
      * add doc entry.
      
      * settle tests.
      
      * style
      
      * add one slow test.
      
      * fix
      
      * fix 2
      
      * fix 2
      
      * fix: 4
      
      * fix: 5
      
      * finish integration tests
      
      * Apply suggestions from code review
      Co-authored-by: default avatarSteven Liu <59462357+stevhliu@users.noreply.github.com>
      
      * style
      
      ---------
      Co-authored-by: default avatarSteven Liu <59462357+stevhliu@users.noreply.github.com>
      18fc40c1
    • Steven Liu's avatar
      [docs] AutoPipeline tutorial (#4273) · ae82a3eb
      Steven Liu authored
      
      
      * first draft
      
      * tidy api
      
      * apply feedback
      
      * mdx to md
      
      * apply feedback
      
      * Apply suggestions from code review
      Co-authored-by: default avatarPatrick von Platen <patrick.v.platen@gmail.com>
      
      ---------
      Co-authored-by: default avatarPatrick von Platen <patrick.v.platen@gmail.com>
      ae82a3eb
  23. 26 Jul, 2023 1 commit
  24. 25 Jul, 2023 1 commit
  25. 24 Jul, 2023 1 commit
  26. 21 Jul, 2023 2 commits
  27. 20 Jul, 2023 1 commit
  28. 18 Jul, 2023 1 commit
  29. 17 Jul, 2023 2 commits
    • Will Berman's avatar
      t2i pipeline (#3932) · a0597f33
      Will Berman authored
      
      
      * Quick implementation of t2i-adapter
      
      Load adapter module with from_pretrained
      
      Prototyping generalized adapter framework
      
      Writeup doc string for sideload framework(WIP) + some minor update on implementation
      
      Update adapter models
      
      Remove old adapter optional args in UNet
      
      Add StableDiffusionAdapterPipeline unit test
      
      Handle cpu offload in StableDiffusionAdapterPipeline
      
      Auto correct coding style
      
      Update model repo name to "RzZ/sd-v1-4-adapter-pipeline"
      
      Refactor MultiAdapter to better compatible with config system
      
      Export MultiAdapter
      
      Create pipeline document template from controlnet
      
      Create dummy objects
      
      Supproting new AdapterLight model
      
      Fix StableDiffusionAdapterPipeline common pipeline test
      
      [WIP] Update adapter pipeline document
      
      Handle num_inference_steps in StableDiffusionAdapterPipeline
      
      Update definition of Adapter "channels_in"
      
      Update documents
      
      Apply code style
      
      Fix doc typo and merge error
      
      Update doc string and example
      
      Quality of life improvement
      
      Remove redundant code and file from prototyping
      
      Remove unused pageage
      
      Remove comments
      
      Fix title
      
      Fix typo
      
      Add conditioning scale arg
      
      Bring back old implmentation
      
      Offload sideload
      
      Add supply info on document
      
      Update src/diffusers/models/adapter.py
      Co-authored-by: default avatarWill Berman <wlbberman@gmail.com>
      
      Update MultiAdapter constructor
      
      Swap out custom checkpoint and update pipeline constructor
      
      Update docment
      
      Apply suggestions from code review
      Co-authored-by: default avatarWill Berman <wlbberman@gmail.com>
      
      Correcting style
      
      Following single-file policy
      
      Update auto size in image preprocess func
      
      Update src/diffusers/pipelines/stable_diffusion/pipeline_stable_diffusion_adapter.py
      Co-authored-by: default avatarWill Berman <wlbberman@gmail.com>
      
      fix copies
      
      Update adapter pipeline behavior
      
      Add adapter_conditioning_scale doc string
      
      Add the missing doc string
      
      Apply suggestions from code review
      Co-authored-by: default avatarPatrick von Platen <patrick.v.platen@gmail.com>
      
      Fix few bugs from suggestion
      
      Handle L-mode PIL image as control image
      
      Rename to differentiate adapter resblock
      
      Update src/diffusers/models/adapter.py
      Co-authored-by: default avatarSayak Paul <spsayakpaul@gmail.com>
      
      Fix typo
      
      Update adapter parameter name
      
      Update test case and code style
      
      Fix copies
      
      Fix typo
      
      Update src/diffusers/pipelines/stable_diffusion/pipeline_stable_diffusion_adapter.py
      Co-authored-by: default avatarWill Berman <wlbberman@gmail.com>
      
      Update Adapter class name
      
      Add checkpoint converting script
      
      Fix style
      
      Fix-copies
      
      Remove dev script
      
      Apply suggestions from code review
      Co-authored-by: default avatarPatrick von Platen <patrick.v.platen@gmail.com>
      
      Updates for parameter rename
      
      Fix convert_adapter
      
      remove main
      
      fix diff
      
      more
      
      refactoring
      
      more
      
      more
      
      small fixes
      
      refactor
      
      tests
      
      more slow tests
      
      more tests
      
      Update docs/source/en/api/pipelines/overview.mdx
      Co-authored-by: default avatarSayak Paul <spsayakpaul@gmail.com>
      
      add community contributor to docs
      
      Update docs/source/en/api/pipelines/stable_diffusion/adapter.mdx
      Co-authored-by: default avatarSayak Paul <spsayakpaul@gmail.com>
      
      Update docs/source/en/api/pipelines/stable_diffusion/adapter.mdx
      Co-authored-by: default avatarSayak Paul <spsayakpaul@gmail.com>
      
      Update docs/source/en/api/pipelines/stable_diffusion/adapter.mdx
      Co-authored-by: default avatarSayak Paul <spsayakpaul@gmail.com>
      
      Update docs/source/en/api/pipelines/stable_diffusion/adapter.mdx
      Co-authored-by: default avatarSayak Paul <spsayakpaul@gmail.com>
      
      Update docs/source/en/api/pipelines/stable_diffusion/adapter.mdx
      Co-authored-by: default avatarSayak Paul <spsayakpaul@gmail.com>
      
      fix
      
      remove from_adapters
      
      license
      
      paper link
      
      docs
      
      more url fixes
      
      more docs
      
      fix
      
      fixes
      
      fix
      
      fix
      
      * fix sample inplace add
      
      * additional_kwargs -> additional_residuals
      
      * move t2i adapter pipeline to own module
      
      * preprocess -> _preprocess_adapter_image
      
      * add TencentArc to license
      
      * fix example code links
      
      * add image converter and fix example doc string
      
      * fix links
      
      * clearer additional residual application
      
      ---------
      Co-authored-by: default avatarHimariO <dsfhe49854@gmail.com>
      a0597f33
    • Kadir Nar's avatar
      📝 Update doc with more descriptive title and filename for "IF" section (#4049) · 39299546
      Kadir Nar authored
      * 📝 Update doc with more descriptive title and filename for "IF" section
      
      Updated the documentation to provide a more descriptive title and filename for the "IF" section. Previously, having only "IF" as the title was not conveying a clear meaning. By renaming the section to "DeepFloyd IF," we provide users with a more informative and context-specific heading.
      
      Thanks! 🙌
      
      * 📝 Update name for "IF" section in 📝 Update name for "IF" section in README
      
      Updated the link and name for the "IF" section in the README file to reflect the new heading "DeepFloyd IF."
      
      * 📝 Fix broken link for "Instruct Pix2Pix" section in README
      
      Fixed the broken link for the "Instruct Pix2Pix" section in the README file. Previously, the link was pointing to an incorrect location due to the presence of "stable_diffusion" in the URL. By removing "stable_diffusion" from the URL, I have corrected the error and ensured that users are directed to the correct section.
      
      * 🔧💼 Updated parameters in _toctree.yml file
      
      - ️ Updated 'local' parameter to 'api/pipelines/deepfloyd_if'.
      - ️ Updated 'title' parameter to 'DeepFloyd IF'.
      
      🎯 These changes aim to improve visibility and accessibility in the documentation of the DeepFloyd IF pipeline. 🚀📚
      39299546
  30. 06 Jul, 2023 1 commit