- 21 Sep, 2023 1 commit
-
-
Ayush Mangal authored
* Add BLIP Diffusion skeleton * Add other model components * Add BLIP2, need to change it for now * Fix pipeline imports * Load pretrained ViT * Make qformer fwd pass same * Replicate fwd passes * Fix device bug * Add accelerate functions * Remove extra functions from Blip2 * Minor bug * Integrate initial review changes * Refactoring * Refactoring * Refactor * Add controlnet * Refactor * Update conversion script * Add image processor * Shift postprocessing to ImageProcessor * Refactor * Fix device * Add fast tests * Update conversion script * Fix checkpoint conversion script * Integrate review changes * Integrate reivew changes * Remove unused functions from test * Reuse HF image processor in Cond image * Create new BlipImageProcessor based on transfomers * Fix image preprocessor * Minor * Minor * Add canny preprocessing * Fix controlnet preprocessing * Fix blip diffusion test * Add controlnet test * Add initial doc strings * Integrate review changes * Refactor * Update examples * Remove DDIM comments * Add copied from for prepare_latents * Add type anotations * Add docstrings * Do black formatting * Add batch support * Make tests pass * Make controlnet tests pass * Black formatting * Fix progress bar * Fix some licensing comments * Fix imports * Refactor controlnet * Make tests faster * Edit examples * Black formatting/Ruff * Add doc * Minor Co-authored-by:
Patrick von Platen <patrick.v.platen@gmail.com> * Move controlnet pipeline * Make tests faster * Fix imports * Fix formatting * Fix make errors * Fix make errors * Minor * Add suggested doc changes Co-authored-by:
Sayak Paul <spsayakpaul@gmail.com> * Edit docs * Fix 16 bit loading * Update examples * Edit toctree * Update docs/source/en/api/pipelines/blip_diffusion.md Co-authored-by:
-
- 13 Sep, 2023 1 commit
-
-
Dhruv Nair authored
* fix gligen clip import issue * fix dtype issue with gligen text to image pipeline * make fix copies
-
- 11 Sep, 2023 2 commits
-
-
Patrick von Platen authored
[TF] Remove tf
-
Dhruv Nair authored
* initial commit * move modules to import struct * add dummy objects and _LazyModule * add lazy import to schedulers * clean up unused imports * lazy import on models module * lazy import for schedulers module * add lazy import to pipelines module * lazy import altdiffusion * lazy import audio diffusion * lazy import audioldm * lazy import consistency model * lazy import controlnet * lazy import dance diffusion ddim ddpm * lazy import deepfloyd * lazy import kandinksy * lazy imports * lazy import semantic diffusion * lazy imports * lazy import stable diffusion * move sd output to its own module * clean up * lazy import t2iadapter * lazy import unclip * lazy import versatile and vq diffsuion * lazy import vq diffusion * helper to fetch objects from modules * lazy import sdxl * lazy import txt2vid * lazy import stochastic karras * fix model imports * fix bug * lazy import * clean up * clean up * fixes for tests * fixes for tests * clean up * remove import of torch_utils from utils module * clean up * clean up * fix mistake import statement * dedicated modules for exporting and loading * remove testing utils from utils module * fixes from merge conflicts * Update src/diffusers/pipelines/kandinsky2_2/__init__.py * fix docs * fix alt diffusion copied from * fix check dummies * fix more docs * remove accelerate import from utils module * add type checking * make style * fix check dummies * remove torch import from xformers check * clean up error message * fixes after upstream merges * dummy objects fix * fix tests * remove unused module import --------- Co-authored-by:
-
- 06 Sep, 2023 1 commit
-
-
Kashif Rasul authored
* initial * initial * added initial convert script for paella vqmodel * initial wuerstchen pipeline * add LayerNorm2d * added modules * fix typo * use model_v2 * embed clip caption amd negative_caption * fixed name of var * initial modules in one place * WuerstchenPriorPipeline * inital shape * initial denoising prior loop * fix output * add WuerstchenPriorPipeline to __init__.py * use the noise ratio in the Prior * try to save pipeline * save_pretrained working * Few additions * add _execution_device * shape is int * fix batch size * fix shape of ratio * fix shape of ratio * fix output dataclass * tests folder * fix formatting * fix float16 + started with generator * Update pipeline_wuerstchen.py * removed vqgan code * add WuerstchenGeneratorPipeline * fix WuerstchenGeneratorPipeline * fix docstrings * fix imports * convert generator pipeline * fix convert * Work on Generator Pipeline. WIP * Pipeline works with our diffuzz code * apply scale factor * removed vqgan.py * use cosine schedule * redo the denoising loop * Update src/diffusers/models/resnet.py Co-authored-by:
Dominic Rampas <d6582533@gmail.com> Co-authored-by:
Dominic Rampas <61938694+dome272@users.noreply.github.com>
-
- 02 Sep, 2023 1 commit
-
-
Harutatsu Akiyama authored
* [ControlNet SDXL Inpainting] Support inpainting of ControlNet SDXL Co-authored-by: Jiabin Bai 1355864570@qq.com --------- Co-authored-by:Harutatsu Akiyama <kf.zy.qin@gmail.com>
-
- 01 Sep, 2023 1 commit
-
-
Nguyễn Công Tú Anh authored
* Add GLIGEN Text Image implementation * add style transfer from image * fix check_repository_consistency * add convert script GLIGEN model to Diffusers * rename attention type * fix style code * remove PositionNetTextImage * Revert "fix check_repository_consistency" This reverts commit 15f098c96e00bb9e67b831161615b30a2d28d815. * change attention type name * update docs for GLIGEN * change examples with hf-document-image * fix style * add CLIPImageProjection for GLIGEN * Add new encode_prompt, load project matrix in pipe init * move CLIPImageProjection to stable_diffusion * add comment
-
- 29 Aug, 2023 1 commit
-
-
Chong Mou authored
* T2I-Adapter-XL * update * update * add pipeline * modify pipeline * modify pipeline * modify pipeline * modify pipeline * modify pipeline * modify modeling_text_unet * fix styling. * fix: copies. * adapter settings * new test case * new test case * debugging * debugging * debugging * debugging * debugging * debugging * debugging * debugging * revert prints. * new test case * remove print * org test case * add test_pipeline * styling. * fix copies. * modify test parameter * style. * add adapter-xl doc * double quotes in docs * Fix potential type mismatch * style. --------- Co-authored-by:
-
- 28 Aug, 2023 1 commit
-
-
YiYi Xu authored
--------- Co-authored-by:
yiyixuxu <yixu310@gmail,com> Co-authored-by:
-
- 25 Aug, 2023 2 commits
-
-
Dhruv Nair authored
* proposal for flaky tests * clean up
-
Sanchit Gandhi authored
* from audioldm * fix vae * move to new pipeline * copied from audioldm * remove redundant control flow * iterate * fix docstring * finish pipeline * tests: from audioldm2 * iterate * finish fast tests * finish slow integration tests * add docs * remove dtype test * update toctree * "copied from" in conversion (where possible) * Update docs/source/en/api/pipelines/musicldm.md Co-authored-by:
-
- 22 Aug, 2023 2 commits
-
-
Patrick von Platen authored
-
Alex McKinney authored
This is a better method than comparing against a list of supported backends as it allows for supporting any number of backends provided they are installed on the user's system. This should have no effect on the behaviour of tests in Huggingface's CI workers. See transformers#25506 where this approach has already been added.
-
- 21 Aug, 2023 1 commit
-
-
Sanchit Gandhi authored
* from audioldm * unet down + mid * vae, clap, flan-t5 * start sequence audio mae * iterate on audioldm encoder * finish encoder * finish weight conversion * text pre-processing * gpt2 pre-processing * fix projection model * working * unet equivalence * finish in base * add unet cond * finish unet * finish custom unet * start clean-up * revert base unet changes * refactor pre-processing * tests: from audioldm * fix some tests * more fixes * iterate on tests * make fix copies * harden fast tests * slow integration tests * finish tests * update checkpoint * update copyright * docs * remove outdated method * add docstring * make style * remove decode latents * enable cpu offload * (text_encoder_1, tokenizer_1) -> (text_encoder, tokenizer) * more clean up * more refactor * build pr docs * Update docs/source/en/api/pipelines/audioldm2.md Co-authored-by:
-
- 16 Aug, 2023 2 commits
-
-
Steven Liu authored
* push to hub docs * fix typo * feedback * make style
-
nikhil-masterful authored
* Add GLIGEN implementation * GLIGEN: Fix code quality check failures * GLIGEN: Fix Import block un-sorted or un-formatted failures * GLIGEN: Fix check_repository_consistency failures * GLIGEN: Add 'PositionNet' to versatile_diffusion/modeling_text_unet.py * GLIGEN: check_repository_consistency: fix 'copy does not match' error * GLIGEN: Fix review comments (1) * GLIGEN: Fix E721 Do not compare types, use `isinstance()` failures * GLIGEN : Ensure _encode_prompt() copy matches to StableDiffusionPipeline * GLIGEN: Fix ruff E721 failure in unidiffuser/test_unidiffuser.py * GLIGEN: doc_builder: restyle pipeline_stable_diffusion_gligen.py * GIGLEN: reset files unrelated to gligen * GLIGEN: Fix documentation comments (1) * GLIGEN: Fix review comments (2) * GLIGEN: Added FastTest * GLIGEN: Fix review comments (3)
-
- 15 Aug, 2023 2 commits
-
-
Sayak Paul authored
* add: pushtohubmixin to pipelines and schedulers docs overview. * Apply suggestions from code review Co-authored-by:
Steven Liu <59462357+stevhliu@users.noreply.github.com> --------- Co-authored-by:
-
Sayak Paul authored
[Pipeline utils] feat: implement push_to_hub for standalone models, schedulers as well as pipelines (#4128) * feat: implement push_to_hub for standalone models. * address PR feedback. * Apply suggestions from code review Co-authored-by:
Lucain <lucainp@gmail.com> * reflect pr feedback. * address another round of deedback. * better handling of kwargs. * add: tests * Apply suggestions from code review Co-authored-by:
ydshieh <2521628+ydshieh@users.noreply.github.com> * fix: tokenizer loading. * fix: json dumping. * move is_staging_test to a better location. * better treatment to tokens. * define repo_id to better handle concurrency * style * explicitly set token * Empty-Commit * move SUER, TOKEN to test * collate org_repo_id * delete repo --------- Co-authored-by:
-
- 12 Aug, 2023 1 commit
-
-
Sayak Paul authored
* add: utility for image grid. * add: return type. * change necessary places. * add to utility page.
-
- 11 Aug, 2023 1 commit
-
-
Abhipsha Das authored
* [WIP] Remove code snippets containing `is_safetensors_available()` * Modifying `import_utils.py` * update pipeline tests for safetensor default * fix test related to cached requests * address import nits --------- Co-authored-by:Dhruv Nair <dhruv.nair@gmail.com>
-
- 02 Aug, 2023 1 commit
-
-
Sayak Paul authored
* add: model implementation of tiny autoencoder. * add: inits. * push the latest devs. * add: conversion script and finish. * add: scaling factor args. * debugging * fix denormalization. * fix: positional argument. * handle use_torch_2_0_or_xformers. * handle post_quant_conv * handle dtype * fix: sdxl image processor for tiny ae. * fix: sdxl image processor for tiny ae. * unify upcasting logic. * copied from madness. * remove trailing whitespace. * set is_tiny_vae = False * address PR comments. * change to AutoencoderTiny * make act_fn an str throughout * fix: apply_forward_hook decorator call * get rid of the special is_tiny_vae flag. * directly scale the output. * fix dummies? * fix: act_fn. * get rid of the Clamp() layer. * bring back copied from. * movement of the blocks to appropriate modules. * add: docstrings to AutoencoderTiny * add: documentation. * changes to the conversion script. * add doc entry. * settle tests. * style * add one slow test. * fix * fix 2 * fix 2 * fix: 4 * fix: 5 * finish integration tests * Apply suggestions from code review Co-authored-by:
-
- 28 Jul, 2023 1 commit
-
-
Patrick von Platen authored
[SDXL] Make watermarker optional under certain circumstances to improve usability of SDXL 1.0 (#4346) * improve sdxl * more fixes * improve sdxl * improve sdxl * improve sdxl * finish
-
- 26 Jul, 2023 3 commits
-
-
Patrick von Platen authored
* 0.20.0dev0 * make style
-
Patrick von Platen authored
-
Patrick von Platen authored
* Add combined pipeline * Download readme * Upload * up * up * fix final * Add enable model cpu offload kandinsky * finish * finish * Fix * fix more * make style * fix kandinsky mask * fix inpainting test * add callbacks * add tests * fix tests * Apply suggestions from code review Co-authored-by:
YiYi Xu <yixu310@gmail.com> * docs * docs * correct docs * fix tests * add warning * correct docs --------- Co-authored-by:
-
- 25 Jul, 2023 2 commits
-
-
Harutatsu Akiyama authored
* Support instruction pix2pix sdxl * Support instruction pix2pix sdxl * Support instruction pix2pix sdxl * Support instruction pix2pix sdxl * Support instruction pix2pix sdxl * Support instruction pix2pix sdxl * Support instruction pix2pix sdxl * Support instruction pix2pix sdxl * Support instruction pix2pix sdxl * Support instruction pix2pix sdxl * Support instruction pix2pix sdxl * Support instruction pix2pix sdxl * Support instruction pix2pix sdxl * Support instruction pix2pix sdxl * Support instruction pix2pix sdxl * Support instruction pix2pix sdxl * Support instruction pix2pix sdxl * [Community] Implementation of the IADB community pipeline (#3996) * community pipeline: implementation of iadb * iadb.py: reformat using black * iadb.py: linting update * add kandinsky to readme table (#4081) Co-authored-by:
Thomas Chambon <36728882+tchambon@users.noreply.github.com> Co-authored-by:
-
YiYi Xu authored
* initial * style * from ...pipelines -> from ..pipeline_util * make style * fix-copies * fix value_guided_sampling oops * style * add test * Show failing test * update from_pipe * fix * add controlnet, additional test and register unused original config * update for controlnet * Apply suggestions from code review Co-authored-by:
-
- 20 Jul, 2023 2 commits
-
-
YiYi Xu authored
* add output_type=mesh * update img2img * make style * add doc * make style * Apply suggestions from code review Co-authored-by:
Pedro Cuenca <pedro@huggingface.co> * add docstring for output_type * add a section in doc about hub mesh visualization/ rotation * update conversion script so default background is white * Apply suggestions from code review Co-authored-by:
-
Ruslan Vorovchenko authored
* added AsymmetricAutoencoderKL * fixed copies+dummy * added script to convert original asymmetric vqgan * added docs * updated docs * fixed style * fixes, added tests * update doc * fixed doc * fixed tests * naming Co-authored-by:
Ruslan Vorovchenko <r.vorovchenko@prequelapp.com> Co-authored-by:
-
- 18 Jul, 2023 1 commit
-
-
Sayak Paul authored
* add: controlnet sdxl. * modifications to controlnet. * run styling. * add: __init__.pys * incorporate https://github.com/huggingface/diffusers/pull/4019 changes. * run make fix-copies. * resize the conditioning images. * remove autocast. * run styling. * disable autocast. * debugging * device placement. * back to autocast. * remove comment. * save some memory by reusing the vae and unet in the pipeline. * apply styling. * Allow low precision sd xl * finish * finish * changes to accommodate the improved VAE. * modifications to how we handle vae encoding in the training. * make style * make existing controlnet fast tests pass. * change vae checkpoint cli arg. * fix: vae pretrained paths. * fix: steps in get_scheduler(). * debugging. * debugging./ * fix: weight conversion. * add: docs. * add: limited tests./ * add: datasets to the requirements. * update docstrings and incorporate the usage of watermarking. * incorporate fix from #4083 * fix watermarking dependency handling. * run make-fix-copies. * Empty-Commit * Update requirements_sdxl.txt * remove vae upcasting part. * Apply suggestions from code review Co-authored-by:
-
- 17 Jul, 2023 1 commit
-
-
Will Berman authored
* Quick implementation of t2i-adapter Load adapter module with from_pretrained Prototyping generalized adapter framework Writeup doc string for sideload framework(WIP) + some minor update on implementation Update adapter models Remove old adapter optional args in UNet Add StableDiffusionAdapterPipeline unit test Handle cpu offload in StableDiffusionAdapterPipeline Auto correct coding style Update model repo name to "RzZ/sd-v1-4-adapter-pipeline" Refactor MultiAdapter to better compatible with config system Export MultiAdapter Create pipeline document template from controlnet Create dummy objects Supproting new AdapterLight model Fix StableDiffusionAdapterPipeline common pipeline test [WIP] Update adapter pipeline document Handle num_inference_steps in StableDiffusionAdapterPipeline Update definition of Adapter "channels_in" Update documents Apply code style Fix doc typo and merge error Update doc string and example Quality of life improvement Remove redundant code and file from prototyping Remove unused pageage Remove comments Fix title Fix typo Add conditioning scale arg Bring back old implmentation Offload sideload Add supply info on document Update src/diffusers/models/adapter.py Co-authored-by:
Will Berman <wlbberman@gmail.com> Update MultiAdapter constructor Swap out custom checkpoint and update pipeline constructor Update docment Apply suggestions from code review Co-authored-by:
HimariO <dsfhe49854@gmail.com>
-
- 14 Jul, 2023 2 commits
-
-
Patrick von Platen authored
* Add more * more * up * Get ensemble of expert denoisers working * Fix code * add tests * up
-
Patrick von Platen authored
-
- 09 Jul, 2023 1 commit
-
-
Will Berman authored
* refactor to support patching LoRA into T5 instantiate the lora linear layer on the same device as the regular linear layer get lora rank from state dict tests fmt can create lora layer in float32 even when rest of model is float16 fix loading model hook remove load_lora_weights_ and T5 dispatching remove Unet#attn_processors_state_dict docstrings * text encoder monkeypatch class method * fix test --------- Co-authored-by:
-
- 06 Jul, 2023 4 commits
-
-
Patrick von Platen authored
* finish sd xl docs * make style * Apply suggestions from code review * uP * uP * Correct
-
YiYi Xu authored
* refactor prior_transformer adding conversion script add pipeline add step_index from pipeline, + remove permute add zero pad token remove copy from statement for betas_for_alpha_bar function * add * add * update conversion script for renderer model * refactor camera a little bit * clean up * style * fix copies * Update src/diffusers/schedulers/scheduling_heun_discrete.py Co-authored-by:
-
YiYi Xu authored
* Kandinsky2_2 * fix init kandinsky2_2 * kandinsky2_2 fix inpainting * rename pipelines: remove decoder + 2_2 -> V22 * Update scheduling_unclip.py * remove text_encoder and tokenizer arguments from doc string * add test for text2img * add tests for text2img & img2img * fix * add test for inpaint * add prior tests * style * copies * add controlnet test * style * add a test for controlnet_img2img * update prior_emb2emb api to accept image_embedding or image * add a test for prior_emb2emb * style * remove try except * example * fix * add doc string examples to all kandinsky pipelines * style * update doc * style * add a top about 2.2 * Apply suggestions from code review Co-authored-by:
Shahmatov Arseniy <62886550+cene555@users.noreply.github.com> Co-authored-by:
-
Patrick von Platen authored
* Add new text encoder * add transformers depth * More * Correct conversion script * Fix more * Fix more * Correct more * correct text encoder * Finish all * proof that in works in run local xl * clean up * Get refiner to work * Add red castle * Fix batch size * Improve pipelines more * Finish text2image tests * Add img2img test * Fix more * fix import * Fix embeddings for classic models (#3888) Fix embeddings for classic SD models. * Allow multiple prompts to be passed to the refiner (#3895) * finish more * Apply suggestions from code review * add watermarker * Model offload (#3889) * Model offload. * Model offload for refiner / img2img * Hardcode encoder offload on img2img vae encode Saves some GPU RAM in img2img / refiner tasks so it remains below 8 GB. --------- Co-authored-by:
-
- 05 Jul, 2023 1 commit
-
-
dg845 authored
* initial commit * Improve consistency models sampling implementation. * Add CMStochasticIterativeScheduler, which implements the multi-step sampler (stochastic_iterative_sampler) in the original code, and make further improvements to sampling. * Add Unet blocks for consistency models * Add conversion script for Unet * Fix bug in new unet blocks * Fix attention weight loading * Make design improvements to ConsistencyModelPipeline and CMStochasticIterativeScheduler and add initial version of tests. * make style * Make small random test UNet class conditional and set resnet_time_scale_shift to 'scale_shift' to better match consistency model checkpoints. * Add support for converting a test UNet and non-class-conditional UNets to the consistency models conversion script. * make style * Change num_class_embeds to 1000 to better match the original consistency models implementation. * Add support for distillation in pipeline_consistency_models.py. * Improve consistency model tests: - Get small testing checkpoints from hub - Modify tests to take into account "distillation" parameter of ConsistencyModelPipeline - Add onestep, multistep tests for distillation and distillation + class conditional - Add expected image slices for onestep tests * make style * Improve ConsistencyModelPipeline: - Add initial support for class-conditional generation - Fix initial sigma for onestep generation - Fix some sigma shape issues * make style * Improve ConsistencyModelPipeline: - add latents __call__ argument and prepare_latents method - add check_inputs method - add initial docstrings for ConsistencyModelPipeline.__call__ * make style * Fix bug when randomly generating class labels for class-conditional generation. * Switch CMStochasticIterativeScheduler to configuring a sigma schedule and make related changes to the pipeline and tests. * Remove some unused code and make style. * Fix small bug in CMStochasticIterativeScheduler. * Add expected slices for multistep sampling tests and make them pass. * Work on consistency model fast tests: - in pipeline, call self.scheduler.scale_model_input before denoising - get expected slices for Euler and Heun scheduler tests - make Euler test pass - mark Heun test as expected fail because it doesn't support prediction_type "sample" yet - remove DPM and Euler Ancestral tests because they don't support use_karras_sigmas * make style * Refactor conversion script to make it easier to add more model architectures to convert in the future. * Work on ConsistencyModelPipeline tests: - Fix device bug when handling class labels in ConsistencyModelPipeline.__call__ - Add slow tests for onestep and multistep sampling and make them pass - Refactor fast tests - Refactor ConsistencyModelPipeline.__init__ * make style * Remove the add_noise and add_noise_to_input methods from CMStochasticIterativeScheduler for now. * Run python utils/check_copies.py --fix_and_overwrite python utils/check_dummies.py --fix_and_overwrite to make dummy objects for new pipeline and scheduler. * Make fast tests from PipelineTesterMixin pass. * make style * Refactor consistency models pipeline and scheduler: - Remove support for Karras schedulers (only support CMStochasticIterativeScheduler) - Move sigma manipulation, input scaling, denoising from pipeline to scheduler - Make corresponding changes to tests and ensure they pass * make style * Add docstrings and further refactor pipeline and scheduler. * make style * Add initial version of the consistency models documentation. * Refactor custom timesteps logic following DDPMScheduler/IFPipeline and temporarily add torch 2.0 SDPA kernel selection logic for debugging. * make style * Convert current slow tests to use fp16 and flash attention. * make style * Add slow tests for normal attention on cuda device. * make style * Fix attention weights loading * Update consistency model fast tests for new test checkpoints with attention fix. * make style * apply suggestions * Add add_noise method to CMStochasticIterativeScheduler (copied from EulerDiscreteScheduler). * Conversion script now outputs pipeline instead of UNet and add support for LSUN-256 models and different schedulers. * When both timesteps and num_inference_steps are supplied, raise warning instead of error (timesteps take precedence). * make style * Add remaining diffusers model checkpoints for models in the original consistency model release and update usage example. * apply suggestions from review * make style * fix attention naming * Add tests for CMStochasticIterativeScheduler. * make style * Make CMStochasticIterativeScheduler tests pass. * make style * Override test_step_shape in CMStochasticIterativeSchedulerTest instead of modifying it in SchedulerCommonTest. * make style * rename some models * Improve API * rename some models * Remove duplicated block * Add docstring and make torch compile work * More fixes * Fixes * Apply suggestions from code review * Apply suggestions from code review * add more docstring * update consistency conversion script --------- Co-authored-by:
ayushmangal <ayushmangal@microsoft.com> Co-authored-by:
Ayush Mangal <43698245+ayushtues@users.noreply.github.com> Co-authored-by:
-
- 02 Jul, 2023 1 commit
-
-
Patrick von Platen authored
* Add image to image video * Improve * better naming * make fix copies * add docs * finish tests * trigger tests * make style * correct * finish * Fix more * make style * finish
-
