1. 29 May, 2024 1 commit
  2. 13 May, 2024 1 commit
  3. 10 May, 2024 1 commit
    • Mark Van Aken's avatar
      #7535 Update FloatTensor type hints to Tensor (#7883) · be4afa0b
      Mark Van Aken authored
      * find & replace all FloatTensors to Tensor
      
      * apply formatting
      
      * Update torch.FloatTensor to torch.Tensor in the remaining files
      
      * formatting
      
      * Fix the rest of the places where FloatTensor is used as well as in documentation
      
      * formatting
      
      * Update new file from FloatTensor to Tensor
      be4afa0b
  4. 08 May, 2024 1 commit
    • Aryan's avatar
      [Pipeline] AnimateDiff SDXL (#6721) · 818f7607
      Aryan authored
      
      
      * update conversion script to handle motion adapter sdxl checkpoint
      
      * add animatediff xl
      
      * handle addition_embed_type
      
      * fix output
      
      * update
      
      * add imports
      
      * make fix-copies
      
      * add decode latents
      
      * update docstrings
      
      * add animatediff sdxl to docs
      
      * remove unnecessary lines
      
      * update example
      
      * add test
      
      * revert conv_in conv_out kernel param
      
      * remove unused param addition_embed_type_num_heads
      
      * latest IPAdapter impl
      
      * make fix-copies
      
      * fix return
      
      * add IPAdapterTesterMixin to tests
      
      * fix return
      
      * revert based on suggestion
      
      * add freeinit
      
      * fix test_to_dtype test
      
      * use StableDiffusionMixin instead of different helper methods
      
      * fix progress bar iterations
      
      * apply suggestions from review
      
      * hardcode flip_sin_to_cos and freq_shift
      
      * make fix-copies
      
      * fix ip adapter implementation
      
      * fix last failing test
      
      * make style
      
      * Update docs/source/en/api/pipelines/animatediff.md
      Co-authored-by: default avatarDhruv Nair <dhruv.nair@gmail.com>
      
      * remove todo
      
      * fix doc-builder errors
      
      ---------
      Co-authored-by: default avatarDhruv Nair <dhruv.nair@gmail.com>
      818f7607
  5. 02 Apr, 2024 1 commit
    • Sayak Paul's avatar
      add: utility to format our docs too 📜 (#7314) · 4a343077
      Sayak Paul authored
      * add: utility to format our docs too 📜
      
      * debugging saga
      
      * fix: message
      
      * checking
      
      * should be fixed.
      
      * revert pipeline_fixture
      
      * remove empty line
      
      * make style
      
      * fix: setup.py
      
      * style.
      4a343077
  6. 01 Apr, 2024 1 commit
  7. 28 Feb, 2024 1 commit
  8. 26 Feb, 2024 1 commit
  9. 08 Feb, 2024 1 commit
  10. 05 Feb, 2024 1 commit
  11. 31 Jan, 2024 1 commit
    • Dhruv Nair's avatar
      Add PIA Model/Pipeline (#6698) · 56bea6b4
      Dhruv Nair authored
      * update
      
      * update
      
      * updaet
      
      * add tests and docs
      
      * clean up
      
      * add to toctree
      
      * fix copies
      
      * pr review feedback
      
      * fix copies
      
      * fix tests
      
      * update docs
      
      * update
      
      * update
      
      * update docs
      
      * update
      
      * update
      
      * update
      
      * update
      56bea6b4
  12. 29 Jan, 2024 1 commit
  13. 23 Jan, 2024 1 commit
    • Sayak Paul's avatar
      [Big refactor] move unets to `unets` module 🦋 (#6630) · 1f0705ad
      Sayak Paul authored
      * move unets to  module 🦋
      
      * parameterize unet-level import.
      
      * fix flax unet2dcondition model import
      
      * models __init__
      
      * mildly depcrecating models.unet_2d_blocks in favor of models.unets.unet_2d_blocks.
      
      * noqa
      
      * correct depcrecation behaviour
      
      * inherit from the actual classes.
      
      * Empty-Commit
      
      * backwards compatibility for unet_2d.py
      
      * backward compatibility for unet_2d_condition
      
      * bc for unet_1d
      
      * bc for unet_1d_blocks
      1f0705ad
  14. 03 Jan, 2024 1 commit
  15. 21 Nov, 2023 1 commit
  16. 20 Nov, 2023 1 commit
  17. 16 Nov, 2023 1 commit
  18. 02 Nov, 2023 1 commit
    • Dhruv Nair's avatar
      Animatediff Proposal (#5413) · 2a8cf8e3
      Dhruv Nair authored
      * draft design
      
      * clean up
      
      * clean up
      
      * clean up
      
      * clean up
      
      * clean up
      
      * clean  up
      
      * clean up
      
      * clean up
      
      * clean up
      
      * update pipeline
      
      * clean up
      
      * clean up
      
      * clean up
      
      * add tests
      
      * change motion block
      
      * clean up
      
      * clean up
      
      * clean up
      
      * update
      
      * update
      
      * update
      
      * update
      
      * update
      
      * update
      
      * update
      
      * update
      
      * clean up
      
      * update
      
      * update
      
      * update model test
      
      * update
      
      * update
      
      * update
      
      * update
      
      * make style
      
      * update
      
      * fix embeddings
      
      * update
      
      * merge upstream
      
      * max fix copies
      
      * fix bug
      
      * fix mistake
      
      * add docs
      
      * update
      
      * clean up
      
      * update
      
      * clean up
      
      * clean up
      
      * fix docstrings
      
      * fix docstrings
      
      * update
      
      * update
      
      * clean  up
      
      * update
      2a8cf8e3
  19. 27 Oct, 2023 1 commit
  20. 09 Oct, 2023 1 commit
  21. 05 Oct, 2023 1 commit
    • Kadir Nar's avatar
      [Core] Add FreeU mechanism (#5164) · 84b82a6c
      Kadir Nar authored
      *  Added Fourier filter function to upsample blocks
      
      * 🔧 Update Fourier_filter for float16 support
      
      *  Added UNetFreeUConfig to UNet model for FreeU adaptation 🛠
      
      ️
      
      * move unet to its original form and add fourier_filter to torch_utils.
      
      * implement freeU enable mechanism
      
      * implement disable mechanism
      
      * resolution index.
      
      * correct resolution idx condition.
      
      * fix copies.
      
      * no need to use resolution_idx in vae.
      
      * spell out the kwargs
      
      * proper config property
      
      * fix attribution setting
      
      * place unet hasattr properly.
      
      * fix: attribute access.
      
      * proper disable
      
      * remove validation method.
      
      * debug
      
      * debug
      
      * debug
      
      * debug
      
      * debug
      
      * debug
      
      * potential fix.
      
      * add: doc.
      
      * fix copies
      
      * add: tests.
      
      * add: support freeU in SDXL.
      
      * set default value of resolution idx.
      
      * set default values for resolution_idx.
      
      * fix copies
      
      * fix rest.
      
      * fix copies
      
      * address PR comments.
      
      * run fix-copies
      
      * move apply_free_u to utils and other minors.
      
      * introduce support for video (unet3D)
      
      * minor ups
      
      * consistent fix-copies.
      
      * consistent stuff
      
      * fix-copies
      
      * add: rest
      
      * add: docs.
      
      * fix: tests
      
      * fix: doc path
      
      * Apply suggestions from code review
      Co-authored-by: default avatarSteven Liu <59462357+stevhliu@users.noreply.github.com>
      
      * style up
      
      * move to techniques.
      
      * add: slow test for sd freeu.
      
      * add: slow test for sd freeu.
      
      * add: slow test for sd freeu.
      
      * add: slow test for sd freeu.
      
      * add: slow test for sd freeu.
      
      * add: slow test for sd freeu.
      
      * add: slow test for video with freeu
      
      * add: slow test for video with freeu
      
      * add: slow test for video with freeu
      
      * style
      
      ---------
      Co-authored-by: default avatarSayak Paul <spsayakpaul@gmail.com>
      Co-authored-by: default avatarPatrick von Platen <patrick.v.platen@gmail.com>
      Co-authored-by: default avatarSteven Liu <59462357+stevhliu@users.noreply.github.com>
      84b82a6c
  22. 27 Sep, 2023 1 commit
  23. 01 Sep, 2023 1 commit
    • Dhruv Nair's avatar
      Test Cleanup Precision issues (#4812) · 189e9f01
      Dhruv Nair authored
      
      
      * proposal for flaky tests
      
      * more precision fixes
      
      * move more tests to use cosine distance
      
      * more test fixes
      
      * clean up
      
      * use default attn
      
      * clean up
      
      * update expected value
      
      * make style
      
      * make style
      
      * Apply suggestions from code review
      
      * Update src/diffusers/pipelines/stable_diffusion/pipeline_onnx_stable_diffusion_img2img.py
      
      * make style
      
      * fix failing tests
      
      ---------
      Co-authored-by: default avatarPatrick von Platen <patrick.v.platen@gmail.com>
      189e9f01
  24. 28 Aug, 2023 1 commit
    • Patrick von Platen's avatar
      [LoRA Attn Processors] Refactor LoRA Attn Processors (#4765) · 766aa50f
      Patrick von Platen authored
      * [LoRA Attn] Refactor LoRA attn
      
      * correct for network alphas
      
      * fix more
      
      * fix more tests
      
      * fix more tests
      
      * Move below
      
      * Finish
      
      * better version
      
      * correct serialization format
      
      * fix
      
      * fix more
      
      * fix more
      
      * fix more
      
      * Apply suggestions from code review
      
      * Update src/diffusers/pipelines/stable_diffusion/pipeline_onnx_stable_diffusion_img2img.py
      
      * deprecation
      
      * relax atol for slow test slighly
      
      * Finish tests
      
      * make style
      
      * make style
      766aa50f
  25. 06 Jul, 2023 2 commits
  26. 03 Jul, 2023 1 commit
  27. 30 Jun, 2023 1 commit
    • Steven Liu's avatar
      [docs] Model API (#3562) · 174dcd69
      Steven Liu authored
      * add modelmixin and unets
      
      * remove old model page
      
      * minor fixes
      
      * fix unet2dcondition
      
      * add vqmodel and autoencoderkl
      
      * add rest of models
      
      * fix autoencoderkl path
      
      * fix toctree
      
      * fix toctree again
      
      * apply feedback
      
      * apply feedback
      
      * fix copies
      
      * fix controlnet copy
      
      * fix copies
      174dcd69
  28. 22 Jun, 2023 1 commit
    • Patrick von Platen's avatar
      Correct bad attn naming (#3797) · 88d26946
      Patrick von Platen authored
      
      
      * relax tolerance slightly
      
      * correct incorrect naming
      
      * correct namingc
      
      * correct more
      
      * Apply suggestions from code review
      
      * Fix more
      
      * Correct more
      
      * correct incorrect naming
      
      * Update src/diffusers/models/controlnet.py
      
      * Correct flax
      
      * Correct renaming
      
      * Correct blocks
      
      * Fix more
      
      * Correct more
      
      * mkae style
      
      * mkae style
      
      * mkae style
      
      * mkae style
      
      * mkae style
      
      * Fix flax
      
      * mkae style
      
      * rename
      
      * rename
      
      * rename attn head dim to attention_head_dim
      
      * correct flax
      
      * make style
      
      * improve
      
      * Correct more
      
      * make style
      
      * fix more
      
      * mkae style
      
      * Update src/diffusers/models/controlnet_flax.py
      
      * Apply suggestions from code review
      Co-authored-by: default avatarPedro Cuenca <pedro@huggingface.co>
      
      ---------
      Co-authored-by: default avatarPedro Cuenca <pedro@huggingface.co>
      88d26946
  29. 12 Apr, 2023 1 commit
  30. 27 Mar, 2023 1 commit
  31. 23 Mar, 2023 1 commit
  32. 22 Mar, 2023 1 commit
    • Patrick von Platen's avatar
      [MS Text To Video] Add first text to video (#2738) · ca1a2229
      Patrick von Platen authored
      
      
      * [MS Text To Video} Add first text to video
      
      * upload
      
      * make first model example
      
      * match unet3d params
      
      * make sure weights are correcctly converted
      
      * improve
      
      * forward pass works, but diff result
      
      * make forward work
      
      * fix more
      
      * finish
      
      * refactor video output class.
      
      * feat: add support for a video export utility.
      
      * fix: opencv availability check.
      
      * run make fix-copies.
      
      * add: docs for the model components.
      
      * add: standalone pipeline doc.
      
      * edit docstring of the pipeline.
      
      * add: right path to TransformerTempModel
      
      * add: first set of tests.
      
      * complete fast tests for text to video.
      
      * fix bug
      
      * up
      
      * three fast tests failing.
      
      * add: note on slow tests
      
      * make work with all schedulers
      
      * apply styling.
      
      * add slow tests
      
      * change file name
      
      * update
      
      * more correction
      
      * more fixes
      
      * finish
      
      * up
      
      * Apply suggestions from code review
      
      * up
      
      * finish
      
      * make copies
      
      * fix pipeline tests
      
      * fix more tests
      
      * Apply suggestions from code review
      Co-authored-by: default avatarPedro Cuenca <pedro@huggingface.co>
      
      * apply suggestions
      
      * up
      
      * revert
      
      ---------
      Co-authored-by: default avatarSayak Paul <spsayakpaul@gmail.com>
      Co-authored-by: default avatarPedro Cuenca <pedro@huggingface.co>
      ca1a2229
  33. 21 Mar, 2023 1 commit
  34. 15 Mar, 2023 1 commit
  35. 14 Mar, 2023 1 commit
  36. 07 Mar, 2023 1 commit
  37. 02 Mar, 2023 1 commit
    • Takuma Mori's avatar
      Add a ControlNet model & pipeline (#2407) · 8dfff7c0
      Takuma Mori authored
      
      
      * add scaffold
      - copied convert_controlnet_to_diffusers.py from
      convert_original_stable_diffusion_to_diffusers.py
      
      * Add support to load ControlNet (WIP)
      - this makes Missking Key error on ControlNetModel
      
      * Update to convert ControlNet without error msg
      - init impl for StableDiffusionControlNetPipeline
      - init impl for ControlNetModel
      
      * cleanup of commented out
      
      * split create_controlnet_diffusers_config()
      from create_unet_diffusers_config()
      
      - add config: hint_channels
      
      * Add input_hint_block, input_zero_conv and
      middle_block_out
      - this makes missing key error on loading model
      
      * add unet_2d_blocks_controlnet.py
      - copied from unet_2d_blocks.py as impl CrossAttnDownBlock2D,DownBlock2D
      - this makes missing key error on loading model
      
      * Add loading for input_hint_block, zero_convs
      and middle_block_out
      
      - this makes no error message on model loading
      
      * Copy from UNet2DConditionalModel except __init__
      
      * Add ultra primitive test for ControlNetModel
      inference
      
      * Support ControlNetModel inference
      - without exceptions
      
      * copy forward() from UNet2DConditionModel
      
      * Impl ControlledUNet2DConditionModel inference
      - test_controlled_unet_inference passed
      
      * Frozen weight & biases for training
      
      * Minimized version of ControlNet/ControlledUnet
      - test_modules_controllnet.py passed
      
      * make style
      
      * Add support model loading for minimized ver
      
      * Remove all previous version files
      
      * from_pretrained and inference test passed
      
      * copied from pipeline_stable_diffusion.py
      except `__init__()`
      
      * Impl pipeline, pixel match test (almost) passed.
      
      * make style
      
      * make fix-copies
      
      * Fix to add import ControlNet blocks
      for `make fix-copies`
      
      * Remove einops dependency
      
      * Support  np.ndarray, PIL.Image for controlnet_hint
      
      * set default config file as lllyasviel's
      
      * Add support grayscale (hw) numpy array
      
      * Add and update docstrings
      
      * add control_net.mdx
      
      * add control_net.mdx to toctree
      
      * Update copyright year
      
      * Fix to add PIL.Image RGB->BGR conversion
      - thanks @Mystfit
      
      * make fix-copies
      
      * add basic fast test for controlnet
      
      * add slow test for controlnet/unet
      
      * Ignore down/up_block len check on ControlNet
      
      * add a copy from test_stable_diffusion.py
      
      * Accept controlnet_hint is None
      
      * merge pipeline_stable_diffusion.py diff
      
      * Update class name to SDControlNetPipeline
      
      * make style
      
      * Baseline fast test almost passed (w long desc)
      
      * still needs investigate.
      
      Following didn't passed descriped in TODO comment:
      - test_stable_diffusion_long_prompt
      - test_stable_diffusion_no_safety_checker
      
      Following didn't passed same as stable_diffusion_pipeline:
      - test_attention_slicing_forward_pass
      - test_inference_batch_single_identical
      - test_xformers_attention_forwardGenerator_pass
      these seems come from calc accuracy.
      
      * Add note comment related vae_scale_factor
      
      * add test_stable_diffusion_controlnet_ddim
      
      * add assertion for vae_scale_factor != 8
      
      * slow test of pipeline almost passed
      Failed: test_stable_diffusion_pipeline_with_model_offloading
      - ImportError: `enable_model_offload` requires `accelerate v0.17.0` or higher
      
      but currently latest version == 0.16.0
      
      * test_stable_diffusion_long_prompt passed
      
      * test_stable_diffusion_no_safety_checker passed
      
      - due to its model size, move to slow test
      
      * remove PoC test files
      
      * fix num_of_image, prompt length issue add add test
      
      * add support List[PIL.Image] for controlnet_hint
      
      * wip
      
      * all slow test passed
      
      * make style
      
      * update for slow test
      
      * RGB(PIL)->BGR(ctrlnet) conversion
      
      * fixes
      
      * remove manual num_images_per_prompt test
      
      * add document
      
      * add `image` argument docstring
      
      * make style
      
      * Add line to correct conversion
      
      * add controlnet_conditioning_scale (aka control_scales
      strength)
      
      * rgb channel ordering by default
      
      * image batching logic
      
      * Add control image descriptions for each checkpoint
      
      * Only save controlnet model in conversion script
      
      * Update src/diffusers/pipelines/stable_diffusion/pipeline_stable_diffusion_controlnet.py
      
      typo
      Co-authored-by: default avatarPedro Cuenca <pedro@huggingface.co>
      
      * Update docs/source/en/api/pipelines/stable_diffusion/control_net.mdx
      Co-authored-by: default avatarPedro Cuenca <pedro@huggingface.co>
      
      * Update docs/source/en/api/pipelines/stable_diffusion/control_net.mdx
      Co-authored-by: default avatarPedro Cuenca <pedro@huggingface.co>
      
      * Update docs/source/en/api/pipelines/stable_diffusion/control_net.mdx
      Co-authored-by: default avatarPedro Cuenca <pedro@huggingface.co>
      
      * Update docs/source/en/api/pipelines/stable_diffusion/control_net.mdx
      Co-authored-by: default avatarPedro Cuenca <pedro@huggingface.co>
      
      * Update docs/source/en/api/pipelines/stable_diffusion/control_net.mdx
      Co-authored-by: default avatarPedro Cuenca <pedro@huggingface.co>
      
      * Update docs/source/en/api/pipelines/stable_diffusion/control_net.mdx
      Co-authored-by: default avatarPedro Cuenca <pedro@huggingface.co>
      
      * Update docs/source/en/api/pipelines/stable_diffusion/control_net.mdx
      Co-authored-by: default avatarPedro Cuenca <pedro@huggingface.co>
      
      * Update docs/source/en/api/pipelines/stable_diffusion/control_net.mdx
      Co-authored-by: default avatarPedro Cuenca <pedro@huggingface.co>
      
      * Update docs/source/en/api/pipelines/stable_diffusion/control_net.mdx
      Co-authored-by: default avatarPedro Cuenca <pedro@huggingface.co>
      
      * add gerated image example
      
      * a depth mask -> a depth map
      
      * rename control_net.mdx to controlnet.mdx
      
      * fix toc title
      
      * add ControlNet abstruct and link
      
      * Update src/diffusers/pipelines/stable_diffusion/pipeline_stable_diffusion_controlnet.py
      Co-authored-by: default avatardqueue <dbyqin@gmail.com>
      
      * remove controlnet constructor arguments re: @patrickvonplaten
      
      * [integration tests] test canny
      
      * test_canny fixes
      
      * [integration tests] test_depth
      
      * [integration tests] test_hed
      
      * [integration tests] test_mlsd
      
      * add channel order config to controlnet
      
      * [integration tests] test normal
      
      * [integration tests] test_openpose test_scribble
      
      * change height and width to default to conditioning image
      
      * [integration tests] test seg
      
      * style
      
      * test_depth fix
      
      * [integration tests] size fixes
      
      * [integration tests] cpu offloading
      
      * style
      
      * generalize controlnet embedding
      
      * fix conversion script
      
      * Update docs/source/en/api/pipelines/stable_diffusion/controlnet.mdx
      Co-authored-by: default avatarSayak Paul <spsayakpaul@gmail.com>
      
      * Update docs/source/en/api/pipelines/stable_diffusion/controlnet.mdx
      Co-authored-by: default avatarSayak Paul <spsayakpaul@gmail.com>
      
      * Update docs/source/en/api/pipelines/stable_diffusion/controlnet.mdx
      Co-authored-by: default avatarSayak Paul <spsayakpaul@gmail.com>
      
      * Update docs/source/en/api/pipelines/stable_diffusion/controlnet.mdx
      Co-authored-by: default avatarSayak Paul <spsayakpaul@gmail.com>
      
      * Style adapted to the documentation of pix2pix
      
      * merge main by hand
      
      * style
      
      * [docs] controlling generation doc nits
      
      * correct some things
      
      * add: controlnetmodel to autodoc.
      
      * finish docs
      
      * finish
      
      * finish 2
      
      * correct images
      
      * finish controlnet
      
      * Apply suggestions from code review
      Co-authored-by: default avatarPedro Cuenca <pedro@huggingface.co>
      
      * uP
      
      * upload model
      
      * up
      
      * up
      
      ---------
      Co-authored-by: default avatarWilliam Berman <WLBberman@gmail.com>
      Co-authored-by: default avatarPedro Cuenca <pedro@huggingface.co>
      Co-authored-by: default avatardqueue <dbyqin@gmail.com>
      Co-authored-by: default avatarSayak Paul <spsayakpaul@gmail.com>
      Co-authored-by: default avatarPatrick von Platen <patrick.v.platen@gmail.com>
      8dfff7c0
  38. 01 Mar, 2023 1 commit
  39. 14 Feb, 2023 1 commit