1. 06 Jun, 2024 2 commits
  2. 05 Jun, 2024 2 commits
    • Tolga Cangöz's avatar
      Errata (#8322) · 98730c5d
      Tolga Cangöz authored
      * Fix typos
      
      * Trim trailing whitespaces
      
      * Remove a trailing whitespace
      
      * chore: Update MarigoldDepthPipeline checkpoint to prs-eth/marigold-lcm-v1-0
      
      * Revert "chore: Update MarigoldDepthPipeline checkpoint to prs-eth/marigold-lcm-v1-0"
      
      This reverts commit fd742b30b4258106008a6af4d0dd4664904f8595.
      
      * pokemon -> naruto
      
      * `DPMSolverMultistep` -> `DPMSolverMultistepScheduler`
      
      * Improve Markdown stylization
      
      * Improve style
      
      * Improve style
      
      * Refactor pipeline variable names for consistency
      
      * up style
      98730c5d
    • Sayak Paul's avatar
      [Hunyuan] allow Hunyuan DiT to run under 6GB for GPU VRAM (#8399) · 2f6f426f
      Sayak Paul authored
      * allow hunyuan dit to run under 6GB for GPU VRAM
      
      * add section in the docs/
      2f6f426f
  3. 04 Jun, 2024 2 commits
  4. 03 Jun, 2024 1 commit
  5. 31 May, 2024 1 commit
    • Sayak Paul's avatar
      [Core] Introduce class variants for `Transformer2DModel` (#7647) · 983dec3b
      Sayak Paul authored
      * init for patches
      
      * finish patched model.
      
      * continuous transformer
      
      * vectorized transformer2d.
      
      * style.
      
      * inits.
      
      * fix-copies.
      
      * introduce DiTTransformer2DModel.
      
      * fixes
      
      * use REMAPPING as suggested by @DN6
      
      * better logging.
      
      * add pixart transformer model.
      
      * inits.
      
      * caption_channels.
      
      * attention masking.
      
      * fix use_additional_conditions.
      
      * remove print.
      
      * debug
      
      * flatten
      
      * fix: assertion for sigma
      
      * handle remapping for modeling_utils
      
      * add tests for dit transformer2d
      
      * quality
      
      * placeholder for pixart tests
      
      * pixart tests
      
      * add _no_split_modules
      
      * add docs.
      
      * check
      
      * check
      
      * check
      
      * check
      
      * fix tests
      
      * fix tests
      
      * move Transformer output to modeling_output
      
      * move errors better and bring back use_additional_conditions attribute.
      
      * add unnecessary things from DiT.
      
      * clean up pixart
      
      * fix remapping
      
      * fix device_map things in pixart2d.
      
      * replace Transformer2DModel with appropriate classes in dit, pixart tests
      
      * empty
      
      * legacy mixin classes./
      
      * use a remapping dict for fetching class names.
      
      * change to specifc model types in the pipeline implementations.
      
      * move _fetch_remapped_cls_from_config to modeling_loading_utils.py
      
      * fix dependency problems.
      
      * add deprecation note.
      983dec3b
  6. 29 May, 2024 1 commit
  7. 27 May, 2024 2 commits
    • Anton Obukhov's avatar
      [Pipeline] Marigold depth and normals estimation (#7847) · b3d10d6d
      Anton Obukhov authored
      
      
      * implement marigold depth and normals pipelines in diffusers core
      
      * remove bibtex
      
      * remove deprecations
      
      * remove save_memory argument
      
      * remove validate_vae
      
      * remove config output
      
      * remove batch_size autodetection
      
      * remove presets logic
      move default denoising_steps and processing_resolution into the model config
      make default ensemble_size 1
      
      * remove no_grad
      
      * add fp16 to the example usage
      
      * implement is_matplotlib_available
      use is_matplotlib_available, is_scipy_available for conditional imports in the marigold depth pipeline
      
      * move colormap, visualize_depth, and visualize_normals into export_utils.py
      
      * make the denoising loop more lucid
      fix the outputs to always be 4d tensors or lists of pil images
      support a 4d input_image case
      attempt to support model_cpu_offload_seq
      move check_inputs into a separate function
      change default batch_size to 1, remove any logic to make it bigger implicitly
      
      * style
      
      * rename denoising_steps into num_inference_steps
      
      * rename input_image into image
      
      * rename input_latent into latents
      
      * remove decode_image
      change decode_prediction to use the AutoencoderKL.decode method
      
      * move clean_latent outside of progress_bar
      
      * refactor marigold-reusable image processing bits into MarigoldImageProcessor class
      
      * clean up the usage example docstring
      
      * make ensemble functions members of the pipelines
      
      * add early checks in check_inputs
      rename E into ensemble_size in depth ensembling
      
      * fix vae_scale_factor computation
      
      * better compatibility with torch.compile
      better variable naming
      
      * move export_depth_to_png to export_utils
      
      * remove encode_prediction
      
      * improve visualize_depth and visualize_normals to accept multi-dimensional data and lists
      remove visualization functions from the pipelines
      move exporting depth as 16-bit PNGs functionality from the depth pipeline
      update example docstrings
      
      * do not shortcut vae.config variables
      
      * change all asserts to raise ValueError
      
      * rename output_prediction_type to output_type
      
      * better variable names
      clean up variable deletion code
      
      * better variable names
      
      * pass desc and leave kwargs into the diffusers progress_bar
      implement nested progress bar for images and steps loops
      
      * implement scale_invariant and shift_invariant flags in the ensemble_depth function
      add scale_invariant and shift_invariant flags readout from the model config
      further refactor ensemble_depth
      support ensembling without alignment
      add ensemble_depth docstring
      
      * fix generator device placement checks
      
      * move encode_empty_text body into the pipeline call
      
      * minor empty text encoding simplifications
      
      * adjust pipelines' class docstrings to explain the added construction arguments
      
      * improve the scipy failure condition
      add comments
      improve docstrings
      change the default use_full_z_range to True
      
      * make input image values range check configurable in the preprocessor
      refactor load_image_canonical in preprocessor to reject unknown types and return the image in the expected 4D format of tensor and on right device
      support a list of everything as inputs to the pipeline, change type to PipelineImageInput
      implement a check that all input list elements have the same dimensions
      improve docstrings of pipeline outputs
      remove check_input pipeline argument
      
      * remove forgotten print
      
      * add prediction_type model config
      
      * add uncertainty visualization into export utils
      fix NaN values in normals uncertainties
      
      * change default of output_uncertainty to False
      better handle the case of an attempt to export or visualize none
      
      * fix `output_uncertainty=False`
      
      * remove kwargs
      fix check_inputs according to the new inputs of the pipeline
      
      * rename prepare_latent into prepare_latents as in other pipelines
      annotate prepare_latents in normals pipeline with "Copied from"
      annotate encode_image in normals pipeline with "Copied from"
      
      * move nested-capable `progress_bar` method into the pipelines
      revert the original `progress_bar` method in pipeline_utils
      
      * minor message improvement
      
      * fix cpu offloading
      
      * move colormap, visualize_depth, export_depth_to_16bit_png, visualize_normals, visualize_uncertainty to marigold_image_processing.py
      update example docstrings
      
      * fix missing comma
      
      * change torch.FloatTensor to torch.Tensor
      
      * fix importing of MarigoldImageProcessor
      
      * fix vae offloading
      fix batched image encoding
      remove separate encode_image function and use vae.encode instead
      
      * implement marigold's intial tests
      relax generator checks in line with other pipelines
      implement return_dict __call__ argument in line with other pipelines
      
      * fix num_images computation
      
      * remove MarigoldImageProcessor and outputs from import structure
      update tests
      
      * update docstrings
      
      * update init
      
      * update
      
      * style
      
      * fix
      
      * fix
      
      * up
      
      * up
      
      * up
      
      * add simple test
      
      * up
      
      * update expected np input/output to be channel last
      
      * move expand_tensor_or_array into the MarigoldImageProcessor
      
      * rewrite tests to follow conventions - hardcoded slices instead of image artifacts
      write more smoke tests
      
      * add basic docs.
      
      * add anton's contribution statement
      
      * remove todos.
      
      * fix assertion values for marigold depth slow tests
      
      * fix assertion values for depth normals.
      
      * remove print
      
      * support AutoencoderTiny in the pipelines
      
      * update documentation page
      add Available Pipelines section
      add Available Checkpoints section
      add warning about num_inference_steps
      
      * fix missing import in docstring
      fix wrong value in visualize_depth docstring
      
      * [doc] add marigold to pipelines overview
      
      * [doc] add section "usage examples"
      
      * fix an issue with latents check in the pipelines
      
      * add "Frame-by-frame Video Processing with Consistency" section
      
      * grammarly
      
      * replace tables with images with css-styled images (blindly)
      
      * style
      
      * print
      
      * fix the assertions.
      
      * take from the github runner.
      
      * take the slices from action artifacts
      
      * style.
      
      * update with the slices from the runner.
      
      * remove unnecessary code blocks.
      
      * Revert "[doc] add marigold to pipelines overview"
      
      This reverts commit a505165150afd8dab23c474d1a054ea505a56a5f.
      
      * remove invitation for new modalities
      
      * split out marigold usage examples
      
      * doc cleanup
      
      ---------
      Co-authored-by: default avataryiyixuxu <yixu310@gmail.com>
      Co-authored-by: default avataryiyixuxu <yixu310@gmail,com>
      Co-authored-by: default avatarsayakpaul <spsayakpaul@gmail.com>
      b3d10d6d
    • Dhaivat Bhatt's avatar
      Add details about 1-stage implementation in I2VGen-XL docs (#8282) · 4d40c914
      Dhaivat Bhatt authored
      * Add details about 1-stage implementation
      
      * Add details about 1-stage implementation
      4d40c914
  8. 24 May, 2024 1 commit
  9. 21 May, 2024 1 commit
  10. 20 May, 2024 1 commit
  11. 10 May, 2024 2 commits
    • Sayak Paul's avatar
      [Core] introduce videoprocessor. (#7776) · 04f4bd54
      Sayak Paul authored
      
      
      * introduce videoprocessor.
      
      * fix quality
      
      * address yiyi's feedback
      
      * fix preprocess_video call.
      
      * video_processor -> image_processor
      
      * fix
      
      * fix more.
      
      * quality
      
      * image_processor -> video_processor
      
      * support List[List[PIL.Image.Image]]
      
      * change to video_processor.
      
      * documentation
      
      * Apply suggestions from code review
      
      * changes
      
      * remove print.
      
      * refactor video processor (part # 7776) (#7861)
      
      * update
      
      * update remove deprecate
      
      * Update src/diffusers/video_processor.py
      
      * update
      
      * Apply suggestions from code review
      
      * deprecate list of 5d for video and list of 4d for image + apply other feedbacks
      
      * up
      
      ---------
      Co-authored-by: default avatarSayak Paul <spsayakpaul@gmail.com>
      
      * add doc.
      
      * tensor2vid -> postprocess_video.
      
      * refactor preprocess with preprocess_video
      
      * set default values.
      
      * empty commit
      
      * more refactoring of prepare_latents in animatediff vid2vid
      
      * checking documentation
      
      * remove documentation for now.
      
      * fix animatediff sdxl
      
      * fix test failure [part of video processor PR] (#7905)
      
      up
      
      * remove preceed_with_frames.
      
      * doc
      
      * fix
      
      * fix
      
      * remove video input as a single-frame video.
      
      ---------
      Co-authored-by: default avatarYiYi Xu <yixu310@gmail.com>
      04f4bd54
    • Sayak Paul's avatar
      add missing image processors to the docs (#7910) · 82be58c5
      Sayak Paul authored
      add missing processors.
      82be58c5
  12. 09 May, 2024 1 commit
    • Dhruv Nair's avatar
      [Refactor] Better align `from_single_file` logic with `from_pretrained` (#7496) · cb0f3b49
      Dhruv Nair authored
      
      
      * refactor unet single file loading a bit.
      
      * retrieve the unet from create_diffusers_unet_model_from_ldm
      
      * update
      
      * update
      
      * updae
      
      * update
      
      * update
      
      * update
      
      * update
      
      * update
      
      * update
      
      * update
      
      * update
      
      * update
      
      * update
      
      * update
      
      * update
      
      * update
      
      * update
      
      * update
      
      * update
      
      * update
      
      * update
      
      * update
      
      * update
      
      * update
      
      * update
      
      * update
      
      * update
      
      * update
      
      * update
      
      * update
      
      * update
      
      * update
      
      * update
      
      * update
      
      * update
      
      * update
      
      * update
      
      * update
      
      * update
      
      * update
      
      * update
      
      * update
      
      * update
      
      * update
      
      * update
      
      * tests
      
      * update
      
      * update
      
      * update
      
      * Update docs/source/en/api/single_file.md
      Co-authored-by: default avatarSayak Paul <spsayakpaul@gmail.com>
      
      * Update docs/source/en/api/single_file.md
      Co-authored-by: default avatarSayak Paul <spsayakpaul@gmail.com>
      
      * update
      
      * update
      
      * update
      
      * update
      
      * update
      
      * update
      
      * update
      
      * update
      
      * update
      
      * update
      
      * update
      
      * update
      
      * update
      
      * Update docs/source/en/api/loaders/single_file.md
      Co-authored-by: default avatarYiYi Xu <yixu310@gmail.com>
      
      * Update src/diffusers/loaders/single_file.py
      Co-authored-by: default avatarYiYi Xu <yixu310@gmail.com>
      
      * Update docs/source/en/api/loaders/single_file.md
      Co-authored-by: default avatarSayak Paul <spsayakpaul@gmail.com>
      
      * Update docs/source/en/api/loaders/single_file.md
      Co-authored-by: default avatarSayak Paul <spsayakpaul@gmail.com>
      
      * Update docs/source/en/api/loaders/single_file.md
      Co-authored-by: default avatarSayak Paul <spsayakpaul@gmail.com>
      
      * Update docs/source/en/api/loaders/single_file.md
      Co-authored-by: default avatarSayak Paul <spsayakpaul@gmail.com>
      
      * update
      
      * update
      
      * update
      
      * update
      
      * update
      
      * update
      
      * update
      
      * update
      
      * update
      
      * update
      
      * update
      
      * update
      
      * update
      
      * update
      
      * update
      
      * update
      
      * update
      
      * update
      
      * update
      
      * update
      
      * update
      
      * update
      
      * update
      
      * update
      
      * update
      
      * update
      
      * update
      
      * update
      
      * update
      
      * update
      
      * update
      
      ---------
      Co-authored-by: default avatarsayakpaul <spsayakpaul@gmail.com>
      Co-authored-by: default avatarYiYi Xu <yixu310@gmail.com>
      cb0f3b49
  13. 08 May, 2024 1 commit
    • Aryan's avatar
      [Pipeline] AnimateDiff SDXL (#6721) · 818f7607
      Aryan authored
      
      
      * update conversion script to handle motion adapter sdxl checkpoint
      
      * add animatediff xl
      
      * handle addition_embed_type
      
      * fix output
      
      * update
      
      * add imports
      
      * make fix-copies
      
      * add decode latents
      
      * update docstrings
      
      * add animatediff sdxl to docs
      
      * remove unnecessary lines
      
      * update example
      
      * add test
      
      * revert conv_in conv_out kernel param
      
      * remove unused param addition_embed_type_num_heads
      
      * latest IPAdapter impl
      
      * make fix-copies
      
      * fix return
      
      * add IPAdapterTesterMixin to tests
      
      * fix return
      
      * revert based on suggestion
      
      * add freeinit
      
      * fix test_to_dtype test
      
      * use StableDiffusionMixin instead of different helper methods
      
      * fix progress bar iterations
      
      * apply suggestions from review
      
      * hardcode flip_sin_to_cos and freq_shift
      
      * make fix-copies
      
      * fix ip adapter implementation
      
      * fix last failing test
      
      * make style
      
      * Update docs/source/en/api/pipelines/animatediff.md
      Co-authored-by: default avatarDhruv Nair <dhruv.nair@gmail.com>
      
      * remove todo
      
      * fix doc-builder errors
      
      ---------
      Co-authored-by: default avatarDhruv Nair <dhruv.nair@gmail.com>
      818f7607
  14. 03 May, 2024 1 commit
  15. 25 Apr, 2024 2 commits
  16. 22 Apr, 2024 1 commit
  17. 16 Apr, 2024 1 commit
    • UmerHA's avatar
      Fixing implementation of ControlNet-XS (#6772) · fda1531d
      UmerHA authored
      
      
      * CheckIn - created DownSubBlocks
      
      * Added extra channels, implemented subblock fwd
      
      * Fixed connection sizes
      
      * checkin
      
      * Removed iter, next in forward
      
      * Models for SD21 & SDXL run through
      
      * Added back pipelines, cleared up connections
      
      * Cleaned up connection creation
      
      * added debug logs
      
      * updated logs
      
      * logs: added input loading
      
      * Update umer_debug_logger.py
      
      * log: Loading hint
      
      * Update umer_debug_logger.py
      
      * added logs
      
      * Changed debug logging
      
      * debug: added more logs
      
      * Fixed num_norm_groups
      
      * Debug: Logging all of SDXL input
      
      * Update umer_debug_logger.py
      
      * debug: updated logs
      
      * checkim
      
      * Readded tests
      
      * Removed debug logs
      
      * Fixed Slow Tests
      
      * Added value ckecks | Updated model_cpu_offload_seq
      
      * accelerate-offloading works ; fast tests work
      
      * Made unet & addon explicit in controlnet
      
      * Updated slow tests
      
      * Added dtype/device to ControlNetXS
      
      * Filled in test model paths
      
      * Added image_encoder/feature_extractor to XL pipe
      
      * Fixed fast tests
      
      * Added comments and docstrings
      
      * Fixed copies
      
      * Added docs ; Updates slow tests
      
      * Moved changes to UNetMidBlock2DCrossAttn
      
      * tiny cleanups
      
      * Removed stray prints
      
      * Removed ip adapters + freeU
      
      - Removed ip adapters + freeU as they don't make sense for ControlNet-XS
      - Fixed imports of UNet components
      
      * Fixed test_save_load_float16
      
      * Make style, quality, fix-copies
      
      * Changed loading/saving API for ControlNetXS
      
      - Changed loading/saving API for ControlNetXS
      - other small fixes
      
      * Removed ControlNet-XS from research examples
      
      * Make style, quality, fix-copies
      
      * Small fixes
      
      - deleted ControlNetXSModel.init_original
      - added time_embedding_mix to StableDiffusionControlNetXSPipeline .from_pretrained / StableDiffusionXLControlNetXSPipeline.from_pretrained
      - fixed copy hints
      
      * checkin May 11 '23
      
      * CheckIn Mar 12 '24
      
      * Fixed tests for SD
      
      * Added tests for UNetControlNetXSModel
      
      * Fixed SDXL tests
      
      * cleanup
      
      * Delete Pipfile
      
      * CheckIn Mar 20
      
      Started replacing sub blocks  by `ControlNetXSCrossAttnDownBlock2D` and `ControlNetXSCrossAttnUplock2D`
      
      * check-in Mar 23
      
      * checkin 24 Mar
      
      * Created init for UNetCnxs and CnxsAddon
      
      * CheckIn
      
      * Made from_modules, from_unet and no_control work
      
      * make style,quality,fix-copies & small changes
      
      * Fixed freezing
      
      * Added gradient ckpt'ing; fixed tests
      
      * Fix slow tests(+compile) ; clear naming confusion
      
      * Don't create UNet in init ; removed class_emb
      
      * Incorporated review feedback
      
      - Deleted get_base_pipeline /  get_controlnet_addon for pipes
      - Pipes inherit from StableDiffusionXLPipeline
      - Made module dicts for cnxs-addon's down/mid/up classes
      - Added support for qkv fusion and freeU
      
      * Make style, quality, fix-copies
      
      * Implemented review feedback
      
      * Removed compatibility check for vae/ctrl embedding
      
      * make style, quality, fix-copies
      
      * Delete Pipfile
      
      * Integrated review feedback
      
      - Importing ControlNetConditioningEmbedding now
      - get_down/mid/up_block_addon now outside class
      - renamed `do_control` to `apply_control`
      
      * Reduced size of test tensors
      
      For this, added `norm_num_groups` as parameter everywhere
      
      * Renamed cnxs-`Addon` to cnxs-`Adapter`
      
      - `ControlNetXSAddon` -> `ControlNetXSAdapter`
      - `ControlNetXSAddonDownBlockComponents` -> `DownBlockControlNetXSAdapter`, and similarly for mid/up
      - `get_mid_block_addon` -> `get_mid_block_adapter`, and similarly for mid/up
      
      * Fixed save_pretrained/from_pretrained bug
      
      * Removed redundant code
      
      ---------
      Co-authored-by: default avatarDhruv Nair <dhruv.nair@gmail.com>
      fda1531d
  18. 11 Apr, 2024 1 commit
  19. 08 Apr, 2024 1 commit
    • Nguyễn Công Tú Anh's avatar
      Add AudioLDM2 TTS (#5381) · 56a76082
      Nguyễn Công Tú Anh authored
      
      
      * add audioldm2 tts
      
      * change gpt2 max new tokens
      
      * remove unnecessary pipeline and class
      
      * add TTS to AudioLDM2Pipeline
      
      * add TTS docs
      
      * delete unnecessary file
      
      * remove unnecessary import
      
      * add audioldm2 slow testcase
      
      * fix code quality
      
      * remove AudioLDMLearnablePositionalEmbedding
      
      * add variable check vits encoder
      
      * add use_learned_position_embedding
      
      ---------
      Co-authored-by: default avatarDhruv Nair <dhruv.nair@gmail.com>
      56a76082
  20. 19 Mar, 2024 1 commit
  21. 16 Mar, 2024 1 commit
  22. 14 Mar, 2024 1 commit
  23. 13 Mar, 2024 2 commits
    • Manuel Brack's avatar
      [Pipeline] Add LEDITS++ pipelines (#6074) · 00eca4b8
      Manuel Brack authored
      
      
      * Setup LEdits++ file structure
      
      * Fix import
      
      * LEditsPP Stable Diffusion pipeline
      
      * Include variable image aspect ratios
      
      * Implement LEDITS++ for SDXL
      
      * clean up LEditsPPPipelineStableDiffusion
      
      * Adjust inversion output
      
      * Added docu, more cleanup for LEditsPPPipelineStableDiffusion
      
      * clean up LEditsPPPipelineStableDiffusionXL
      
      * Update documentation
      
      * Fix documentation import
      
      * Add skeleton IF implementation
      
      * Fix documentation typo
      
      * Add LEDTIS docu to toctree
      
      * Add missing title
      
      * Finalize SD documentation
      
      * Finalize SD-XL documentation
      
      * Fix code style and quality
      
      * Fix typo
      
      * Fix return types
      
      * added LEditsPPPipelineIF; minor changes for LEditsPPPipelineStableDiffusion and LEditsPPPipelineStableDiffusionXL
      
      * Fix copy reference
      
      * add documentation for IF
      
      * Add first tests
      
      * Fix batching for SD-XL
      
      * Fix text encoding and perfect reconstruction for SD-XL
      
      * Add tests for SD-XL, minor changes
      
      * move user_mask to correct device, use cross_attention_kwargs also for inversion
      
      * Example docstring
      
      * Fix attention resolution for non-square images
      
      * Refactoring for PR review
      
      * Safely remove ledits_utils.py
      
      * Style fixes
      
      * Replace assertions with ValueError
      
      * Remove LEditsPPPipelineIF
      
      * Remove unecessary input checks
      
      * Refactoring of CrossAttnProcessor
      
      * Revert unecessary changes to scheduler
      
      * Remove first progress-bar in inversion
      
      * Refactor scheduler usage and reset
      
      * Use imageprocessor instead of custom logic
      
      * Fix scheduler init warning
      
      * Fix error when running the pipeline in fp16
      
      * Update documentation wrt perfect inversion
      
      * Update tests
      
      * Fix code quality and copy consistency
      
      * Update LEditsPP import
      
      * Remove enable/disable methods that are now in StableDiffusionMixin
      
      * Change import in docs
      
      * Revert import structure change
      
      * Fix ledits imports
      
      ---------
      Co-authored-by: default avatarKatharina Kornmeier <katharina.kornmeier@stud.tu-darmstadt.de>
      00eca4b8
    • Dhruv Nair's avatar
      Update Cascade documentation (#7257) · a17d6d68
      Dhruv Nair authored
      
      
      * updates
      
      * update
      
      * update
      
      * Update docs/source/en/api/pipelines/stable_cascade.md
      Co-authored-by: default avatarSayak Paul <spsayakpaul@gmail.com>
      
      * update
      
      * update
      
      * update
      
      * update
      
      ---------
      Co-authored-by: default avatarSayak Paul <spsayakpaul@gmail.com>
      Co-authored-by: default avatarKashif Rasul <kashif.rasul@gmail.com>
      a17d6d68
  24. 08 Mar, 2024 1 commit
  25. 06 Mar, 2024 1 commit
    • Kashif Rasul's avatar
      [Pipiline] Wuerstchen v3 aka Stable Cascasde pipeline (#6487) · 40aa47b9
      Kashif Rasul authored
      
      
      * initial diffNext v3
      
      * move to v3 folder
      
      * imports
      
      * dry up the unets
      
      * no switch_level
      
      * fix init
      
      * add switch_level tp config
      
      * Fixed some things
      
      * Added pooled text embeddings
      
      * Initial work on adding image encoder
      
      * changes from @dome272
      
      * Stuff for the image encoder processing and variable naming in decoder
      
      * fix arg name
      
      * inference fixes
      
      * inference fixes
      
      * default TimestepBlock without conds
      
      * c_skip=0 by default
      
      * fix bfloat16 to cpu
      
      * use config
      
      * undo temp change
      
      * fix gen_c_embeddings args
      
      * change text encoding
      
      * text encoding
      
      * undo print
      
      * undo .gitignore change
      
      * Allow WuerstchenV3PriorPipeline to use the base DDPM & DDIM schedulers
      
      * use WuerstchenV3Unet in both pipelines
      
      * fix imports
      
      * initial failing tests
      
      * cleanup
      
      * use scheduler.timesterps
      
      * some fixes to the tests, still not fully working
      
      * fix tests
      
      * fix prior tests
      
      * add dropout to the model_kwargs
      
      * more tests passing
      
      * update expected_slice
      
      * initial rename
      
      * rename tests
      
      * rename class names
      
      * make fix-copies
      
      * initial docs
      
      * autodocs
      
      * typos
      
      * fix arg docs
      
      * add text_encoder info
      
      * combined pipeline has optional image arg
      
      * fix documentation
      
      * Update src/diffusers/pipelines/stable_cascade/modeling_stable_cascade_common.py
      Co-authored-by: default avatarPatrick von Platen <patrick.v.platen@gmail.com>
      
      * Update src/diffusers/pipelines/stable_cascade/modeling_stable_cascade_common.py
      Co-authored-by: default avatarPatrick von Platen <patrick.v.platen@gmail.com>
      
      * Update src/diffusers/pipelines/stable_cascade/modeling_stable_cascade_common.py
      Co-authored-by: default avatarYiYi Xu <yixu310@gmail.com>
      
      * Update src/diffusers/pipelines/stable_cascade/modeling_stable_cascade_common.py
      Co-authored-by: default avatarPatrick von Platen <patrick.v.platen@gmail.com>
      
      * Update src/diffusers/pipelines/stable_cascade/pipeline_stable_cascade.py
      Co-authored-by: default avatarYiYi Xu <yixu310@gmail.com>
      
      * Update src/diffusers/pipelines/stable_cascade/modeling_stable_cascade_common.py
      Co-authored-by: default avatarYiYi Xu <yixu310@gmail.com>
      
      * use self.config
      
      * Update src/diffusers/pipelines/stable_cascade/modeling_stable_cascade_common.py
      Co-authored-by: default avatarYiYi Xu <yixu310@gmail.com>
      
      * c_in -> in_channels
      
      * removed kwargs from unet's forward
      
      * Update src/diffusers/pipelines/stable_cascade/pipeline_stable_cascade.py
      Co-authored-by: default avatarYiYi Xu <yixu310@gmail.com>
      
      * Update src/diffusers/pipelines/stable_cascade/pipeline_stable_cascade.py
      Co-authored-by: default avatarPatrick von Platen <patrick.v.platen@gmail.com>
      
      * remove older callback api
      
      * removed kwargs and fixed decoder guidance > 1
      
      * decoder takes emeds
      
      * check and use image_embeds
      
      * fixed all but one decoder test
      
      * fix decoder tests
      
      * update callback api
      
      * fix some more combined tests
      
      * push combined pipeline
      
      * initial docs
      
      * fix doc_string
      
      * update combined api
      
      * no test_callback_inputs test for combined pipeline
      
      * add optional components
      
      * fix ordering of components
      
      * fix combined tests
      
      * update convert script
      
      * Update src/diffusers/pipelines/stable_cascade/pipeline_stable_cascade_prior.py
      Co-authored-by: default avatarYiYi Xu <yixu310@gmail.com>
      
      * Update src/diffusers/pipelines/stable_cascade/pipeline_stable_cascade_prior.py
      Co-authored-by: default avatarYiYi Xu <yixu310@gmail.com>
      
      * Update src/diffusers/pipelines/stable_cascade/pipeline_stable_cascade_prior.py
      Co-authored-by: default avatarYiYi Xu <yixu310@gmail.com>
      
      * fix imports
      
      * move effnet out of deniosing loop
      
      * prompt_embeds_pooled only when doing guidance
      
      * Fix repeat shape
      
      * move StableCascadeUnet to models/unets/
      
      * more descriptive names
      
      * converted when numpy()
      
      * StableCascadePriorPipelineOutput docs
      
      * rename StableCascadeUNet
      
      * add slow tests
      
      * fix slow tests
      
      * update
      
      * update
      
      * updated model_path
      
      * add args for weights
      
      * set push_to_hub to false
      
      * update
      
      * update
      
      * update
      
      * update
      
      * update
      
      * update
      
      * update
      
      * update
      
      * update
      
      * update
      
      * update
      
      * update
      
      * update
      
      * update
      
      ---------
      Co-authored-by: default avatarDominic Rampas <d6582533@gmail.com>
      Co-authored-by: default avatarPablo Pernias <pablo@pernias.com>
      Co-authored-by: default avatarSayak Paul <spsayakpaul@gmail.com>
      Co-authored-by: default avatarPatrick von Platen <patrick.v.platen@gmail.com>
      Co-authored-by: default avatarYiYi Xu <yixu310@gmail.com>
      Co-authored-by: default avatar99991 <99991@users.noreply.github.com>
      Co-authored-by: default avatarDhruv Nair <dhruv.nair@gmail.com>
      40aa47b9
  26. 05 Mar, 2024 1 commit
  27. 29 Feb, 2024 1 commit
    • M. Tolga Cangöz's avatar
      [`Docs`] Fix typos (#7131) · f4fc7503
      M. Tolga Cangöz authored
      * Add copyright notice to relevant files and fix typos
      
      * Set `timestep_spacing` parameter of `StableDiffusionXLPipeline`'s scheduler to `'trailing'`.
      
      * Update `StableDiffusionXLPipeline.from_single_file` by including EulerAncestralDiscreteScheduler with `timestep_spacing="trailing"` param.
      
      * Update model loading method in SDXL Turbo documentation
      f4fc7503
  28. 25 Feb, 2024 1 commit
  29. 20 Feb, 2024 1 commit
  30. 19 Feb, 2024 1 commit
  31. 17 Feb, 2024 1 commit
  32. 14 Feb, 2024 1 commit
    • Steven Liu's avatar
      [docs] IP-Adapter (#6897) · 9efe1e52
      Steven Liu authored
      * use cases
      
      * first draft
      
      * fix image links
      
      * lcm-lora
      
      * feedback
      
      * review
      
      * feedback
      
      * feedback
      9efe1e52
  33. 08 Feb, 2024 1 commit