- 28 May, 2024 2 commits
-
-
Steven Liu authored
* first draft * edits
-
Steven Liu authored
* noise schedule * sigmas and zero snr * feedback * feedback
-
- 27 May, 2024 1 commit
-
-
Anton Obukhov authored
* implement marigold depth and normals pipelines in diffusers core * remove bibtex * remove deprecations * remove save_memory argument * remove validate_vae * remove config output * remove batch_size autodetection * remove presets logic move default denoising_steps and processing_resolution into the model config make default ensemble_size 1 * remove no_grad * add fp16 to the example usage * implement is_matplotlib_available use is_matplotlib_available, is_scipy_available for conditional imports in the marigold depth pipeline * move colormap, visualize_depth, and visualize_normals into export_utils.py * make the denoising loop more lucid fix the outputs to always be 4d tensors or lists of pil images support a 4d input_image case attempt to support model_cpu_offload_seq move check_inputs into a separate function change default batch_size to 1, remove any logic to make it bigger implicitly * style * rename denoising_steps into num_inference_steps * rename input_image into image * rename input_latent into latents * remove decode_image change decode_prediction to use the AutoencoderKL.decode method * move clean_latent outside of progress_bar * refactor marigold-reusable image processing bits into MarigoldImageProcessor class * clean up the usage example docstring * make ensemble functions members of the pipelines * add early checks in check_inputs rename E into ensemble_size in depth ensembling * fix vae_scale_factor computation * better compatibility with torch.compile better variable naming * move export_depth_to_png to export_utils * remove encode_prediction * improve visualize_depth and visualize_normals to accept multi-dimensional data and lists remove visualization functions from the pipelines move exporting depth as 16-bit PNGs functionality from the depth pipeline update example docstrings * do not shortcut vae.config variables * change all asserts to raise ValueError * rename output_prediction_type to output_type * better variable names clean up variable deletion code * better variable names * pass desc and leave kwargs into the diffusers progress_bar implement nested progress bar for images and steps loops * implement scale_invariant and shift_invariant flags in the ensemble_depth function add scale_invariant and shift_invariant flags readout from the model config further refactor ensemble_depth support ensembling without alignment add ensemble_depth docstring * fix generator device placement checks * move encode_empty_text body into the pipeline call * minor empty text encoding simplifications * adjust pipelines' class docstrings to explain the added construction arguments * improve the scipy failure condition add comments improve docstrings change the default use_full_z_range to True * make input image values range check configurable in the preprocessor refactor load_image_canonical in preprocessor to reject unknown types and return the image in the expected 4D format of tensor and on right device support a list of everything as inputs to the pipeline, change type to PipelineImageInput implement a check that all input list elements have the same dimensions improve docstrings of pipeline outputs remove check_input pipeline argument * remove forgotten print * add prediction_type model config * add uncertainty visualization into export utils fix NaN values in normals uncertainties * change default of output_uncertainty to False better handle the case of an attempt to export or visualize none * fix `output_uncertainty=False` * remove kwargs fix check_inputs according to the new inputs of the pipeline * rename prepare_latent into prepare_latents as in other pipelines annotate prepare_latents in normals pipeline with "Copied from" annotate encode_image in normals pipeline with "Copied from" * move nested-capable `progress_bar` method into the pipelines revert the original `progress_bar` method in pipeline_utils * minor message improvement * fix cpu offloading * move colormap, visualize_depth, export_depth_to_16bit_png, visualize_normals, visualize_uncertainty to marigold_image_processing.py update example docstrings * fix missing comma * change torch.FloatTensor to torch.Tensor * fix importing of MarigoldImageProcessor * fix vae offloading fix batched image encoding remove separate encode_image function and use vae.encode instead * implement marigold's intial tests relax generator checks in line with other pipelines implement return_dict __call__ argument in line with other pipelines * fix num_images computation * remove MarigoldImageProcessor and outputs from import structure update tests * update docstrings * update init * update * style * fix * fix * up * up * up * add simple test * up * update expected np input/output to be channel last * move expand_tensor_or_array into the MarigoldImageProcessor * rewrite tests to follow conventions - hardcoded slices instead of image artifacts write more smoke tests * add basic docs. * add anton's contribution statement * remove todos. * fix assertion values for marigold depth slow tests * fix assertion values for depth normals. * remove print * support AutoencoderTiny in the pipelines * update documentation page add Available Pipelines section add Available Checkpoints section add warning about num_inference_steps * fix missing import in docstring fix wrong value in visualize_depth docstring * [doc] add marigold to pipelines overview * [doc] add section "usage examples" * fix an issue with latents check in the pipelines * add "Frame-by-frame Video Processing with Consistency" section * grammarly * replace tables with images with css-styled images (blindly) * style * print * fix the assertions. * take from the github runner. * take the slices from action artifacts * style. * update with the slices from the runner. * remove unnecessary code blocks. * Revert "[doc] add marigold to pipelines overview" This reverts commit a505165150afd8dab23c474d1a054ea505a56a5f. * remove invitation for new modalities * split out marigold usage examples * doc cleanup --------- Co-authored-by:
yiyixuxu <yixu310@gmail.com> Co-authored-by:
yiyixuxu <yixu310@gmail,com> Co-authored-by:
sayakpaul <spsayakpaul@gmail.com>
-
- 20 May, 2024 1 commit
-
-
Junsong Chen authored
* 1. add doc for PixArtSigmaPipeline; --------- Co-authored-by:
Steven Liu <59462357+stevhliu@users.noreply.github.com> Co-authored-by:
Guillaume LEGENDRE <glegendre01@gmail.com> Co-authored-by:
Álvaro Somoza <asomoza@users.noreply.github.com> Co-authored-by:
Bagheera <59658056+bghira@users.noreply.github.com> Co-authored-by:
bghira <bghira@users.github.com> Co-authored-by:
Hyoungwon Cho <jhw9811@korea.ac.kr> Co-authored-by:
Tolga Cangöz <46008593+standardAI@users.noreply.github.com> Co-authored-by:
Philip Pham <phillypham@google.com>
-
- 10 May, 2024 1 commit
-
-
Sayak Paul authored
* introduce videoprocessor. * fix quality * address yiyi's feedback * fix preprocess_video call. * video_processor -> image_processor * fix * fix more. * quality * image_processor -> video_processor * support List[List[PIL.Image.Image]] * change to video_processor. * documentation * Apply suggestions from code review * changes * remove print. * refactor video processor (part # 7776) (#7861) * update * update remove deprecate * Update src/diffusers/video_processor.py * update * Apply suggestions from code review * deprecate list of 5d for video and list of 4d for image + apply other feedbacks * up --------- Co-authored-by:
-
- 06 May, 2024 1 commit
-
-
Steven Liu authored
* combine * edits
-
- 03 May, 2024 1 commit
-
-
Steven Liu authored
* lcm * lcm lora * fix * fix hfoption * edits
-
- 30 Apr, 2024 1 commit
-
-
Steven Liu authored
* community pipelines * feedback * consolidate
-
- 25 Apr, 2024 2 commits
-
-
Steven Liu authored
* refactor * code snippets * fix path * fix path in guide * code outputs * align toctree title * title * fix title
-
Steven Liu authored
* reproducibility * feedback * feedback * fix path * github link
-
- 23 Apr, 2024 1 commit
-
-
Steven Liu authored
* toctree * optim * feedback * improve overview
-
- 17 Apr, 2024 2 commits
-
-
Steven Liu authored
* pipelines * schedulers and models * community pipelines * feedback
-
Wentian authored
* Create tgate.md * Update _toctree.yml * Update docs/source/en/optimization/tgate.md Co-authored-by:
-
- 16 Apr, 2024 1 commit
-
-
UmerHA authored
* CheckIn - created DownSubBlocks * Added extra channels, implemented subblock fwd * Fixed connection sizes * checkin * Removed iter, next in forward * Models for SD21 & SDXL run through * Added back pipelines, cleared up connections * Cleaned up connection creation * added debug logs * updated logs * logs: added input loading * Update umer_debug_logger.py * log: Loading hint * Update umer_debug_logger.py * added logs * Changed debug logging * debug: added more logs * Fixed num_norm_groups * Debug: Logging all of SDXL input * Update umer_debug_logger.py * debug: updated logs * checkim * Readded tests * Removed debug logs * Fixed Slow Tests * Added value ckecks | Updated model_cpu_offload_seq * accelerate-offloading works ; fast tests work * Made unet & addon explicit in controlnet * Updated slow tests * Added dtype/device to ControlNetXS * Filled in test model paths * Added image_encoder/feature_extractor to XL pipe * Fixed fast tests * Added comments and docstrings * Fixed copies * Added docs ; Updates slow tests * Moved changes to UNetMidBlock2DCrossAttn * tiny cleanups * Removed stray prints * Removed ip adapters + freeU - Removed ip adapters + freeU as they don't make sense for ControlNet-XS - Fixed imports of UNet components * Fixed test_save_load_float16 * Make style, quality, fix-copies * Changed loading/saving API for ControlNetXS - Changed loading/saving API for ControlNetXS - other small fixes * Removed ControlNet-XS from research examples * Make style, quality, fix-copies * Small fixes - deleted ControlNetXSModel.init_original - added time_embedding_mix to StableDiffusionControlNetXSPipeline .from_pretrained / StableDiffusionXLControlNetXSPipeline.from_pretrained - fixed copy hints * checkin May 11 '23 * CheckIn Mar 12 '24 * Fixed tests for SD * Added tests for UNetControlNetXSModel * Fixed SDXL tests * cleanup * Delete Pipfile * CheckIn Mar 20 Started replacing sub blocks by `ControlNetXSCrossAttnDownBlock2D` and `ControlNetXSCrossAttnUplock2D` * check-in Mar 23 * checkin 24 Mar * Created init for UNetCnxs and CnxsAddon * CheckIn * Made from_modules, from_unet and no_control work * make style,quality,fix-copies & small changes * Fixed freezing * Added gradient ckpt'ing; fixed tests * Fix slow tests(+compile) ; clear naming confusion * Don't create UNet in init ; removed class_emb * Incorporated review feedback - Deleted get_base_pipeline / get_controlnet_addon for pipes - Pipes inherit from StableDiffusionXLPipeline - Made module dicts for cnxs-addon's down/mid/up classes - Added support for qkv fusion and freeU * Make style, quality, fix-copies * Implemented review feedback * Removed compatibility check for vae/ctrl embedding * make style, quality, fix-copies * Delete Pipfile * Integrated review feedback - Importing ControlNetConditioningEmbedding now - get_down/mid/up_block_addon now outside class - renamed `do_control` to `apply_control` * Reduced size of test tensors For this, added `norm_num_groups` as parameter everywhere * Renamed cnxs-`Addon` to cnxs-`Adapter` - `ControlNetXSAddon` -> `ControlNetXSAdapter` - `ControlNetXSAddonDownBlockComponents` -> `DownBlockControlNetXSAdapter`, and similarly for mid/up - `get_mid_block_addon` -> `get_mid_block_adapter`, and similarly for mid/up * Fixed save_pretrained/from_pretrained bug * Removed redundant code --------- Co-authored-by:Dhruv Nair <dhruv.nair@gmail.com>
-
- 11 Apr, 2024 1 commit
-
-
Steven Liu authored
* refactor t2i * add code snippets
-
- 10 Apr, 2024 1 commit
-
-
Steven Liu authored
* prompt enhance * edits * align titles * feedback * feedback * feedback * link to style
-
- 18 Mar, 2024 1 commit
-
-
Sayak Paul authored
* post-release * quality
-
- 14 Mar, 2024 1 commit
-
-
Suraj Patil authored
* add edm schedulers in doc * add in toctree * address reviewe comments
-
- 13 Mar, 2024 2 commits
-
-
Michael authored
* add tcd intro * resolve repos * Apply suggestions from code review Co-authored-by:
-
Manuel Brack authored
* Setup LEdits++ file structure * Fix import * LEditsPP Stable Diffusion pipeline * Include variable image aspect ratios * Implement LEDITS++ for SDXL * clean up LEditsPPPipelineStableDiffusion * Adjust inversion output * Added docu, more cleanup for LEditsPPPipelineStableDiffusion * clean up LEditsPPPipelineStableDiffusionXL * Update documentation * Fix documentation import * Add skeleton IF implementation * Fix documentation typo * Add LEDTIS docu to toctree * Add missing title * Finalize SD documentation * Finalize SD-XL documentation * Fix code style and quality * Fix typo * Fix return types * added LEditsPPPipelineIF; minor changes for LEditsPPPipelineStableDiffusion and LEditsPPPipelineStableDiffusionXL * Fix copy reference * add documentation for IF * Add first tests * Fix batching for SD-XL * Fix text encoding and perfect reconstruction for SD-XL * Add tests for SD-XL, minor changes * move user_mask to correct device, use cross_attention_kwargs also for inversion * Example docstring * Fix attention resolution for non-square images * Refactoring for PR review * Safely remove ledits_utils.py * Style fixes * Replace assertions with ValueError * Remove LEditsPPPipelineIF * Remove unecessary input checks * Refactoring of CrossAttnProcessor * Revert unecessary changes to scheduler * Remove first progress-bar in inversion * Refactor scheduler usage and reset * Use imageprocessor instead of custom logic * Fix scheduler init warning * Fix error when running the pipeline in fp16 * Update documentation wrt perfect inversion * Update tests * Fix code quality and copy consistency * Update LEditsPP import * Remove enable/disable methods that are now in StableDiffusionMixin * Change import in docs * Revert import structure change * Fix ledits imports --------- Co-authored-by:Katharina Kornmeier <katharina.kornmeier@stud.tu-darmstadt.de>
-
- 07 Mar, 2024 1 commit
-
-
Steven Liu authored
* merge loras * feedback * torch.compile * feedback
-
- 06 Mar, 2024 1 commit
-
-
Kashif Rasul authored
* initial diffNext v3 * move to v3 folder * imports * dry up the unets * no switch_level * fix init * add switch_level tp config * Fixed some things * Added pooled text embeddings * Initial work on adding image encoder * changes from @dome272 * Stuff for the image encoder processing and variable naming in decoder * fix arg name * inference fixes * inference fixes * default TimestepBlock without conds * c_skip=0 by default * fix bfloat16 to cpu * use config * undo temp change * fix gen_c_embeddings args * change text encoding * text encoding * undo print * undo .gitignore change * Allow WuerstchenV3PriorPipeline to use the base DDPM & DDIM schedulers * use WuerstchenV3Unet in both pipelines * fix imports * initial failing tests * cleanup * use scheduler.timesterps * some fixes to the tests, still not fully working * fix tests * fix prior tests * add dropout to the model_kwargs * more tests passing * update expected_slice * initial rename * rename tests * rename class names * make fix-copies * initial docs * autodocs * typos * fix arg docs * add text_encoder info * combined pipeline has optional image arg * fix documentation * Update src/diffusers/pipelines/stable_cascade/modeling_stable_cascade_common.py Co-authored-by:
Patrick von Platen <patrick.v.platen@gmail.com> * Update src/diffusers/pipelines/stable_cascade/modeling_stable_cascade_common.py Co-authored-by:
Dominic Rampas <d6582533@gmail.com> Co-authored-by:
Pablo Pernias <pablo@pernias.com> Co-authored-by:
99991 <99991@users.noreply.github.com> Co-authored-by:
-
- 05 Mar, 2024 1 commit
-
-
Michael authored
* add: support TCD scheduler --------- Co-authored-by:
-
- 17 Feb, 2024 1 commit
-
-
Steven Liu authored
* first draft * fix path * fix path * i2vgen-xl * review * modelscopet2v * feedback
-
- 14 Feb, 2024 1 commit
-
-
Steven Liu authored
* use cases * first draft * fix image links * lcm-lora * feedback * review * feedback * feedback
-
- 31 Jan, 2024 2 commits
-
-
Sayak Paul authored
--------- Co-authored-by:
-
Dhruv Nair authored
* update * update * updaet * add tests and docs * clean up * add to toctree * fix copies * pr review feedback * fix copies * fix tests * update docs * update * update * update docs * update * update * update * update
-
- 25 Jan, 2024 1 commit
-
-
Steven Liu authored
* uvit2d * fix * fix? * add correct paper * fix paths * update abstract
-
- 10 Jan, 2024 1 commit
-
-
YiYi Xu authored
--------- Co-authored-by:
-
- 05 Jan, 2024 2 commits
-
-
Sayak Paul authored
* introduce integrations module. * remove duplicate methods. * better imports. * move to loaders.py * remove peftadaptermixin from modelmixin. * add: peftadaptermixin selectively. * add: entry to _toctree * Empty-Commit
-
Horseee authored
* add documentation for DeepCache * fix typo * add wandb url for DeepCache * fix some typos * add item in _toctree.yml * update formats for arguments * Update deepcache.md * Update docs/source/en/optimization/deepcache.md Co-authored-by:
-
- 27 Dec, 2023 1 commit
-
-
Dhruv Nair authored
* update * update * update * update * update * make style * remove docs * update * move to research folder. * fix-copies * remove _toctree entry. --------- Co-authored-by:
-
- 26 Dec, 2023 1 commit
-
-
Sayak Paul authored
* add doc for diffusion fast * add entry to _toctree * Apply suggestions from code review * fix titlew * fix: title entry * add note about fuse_qkv_projections
-
- 21 Dec, 2023 1 commit
-
-
Will Berman authored
amused rename Update docs/source/en/api/pipelines/amused.md Co-authored-by:
Suraj Patil <surajp815@gmail.com> Update examples/amused/README.md Co-authored-by:
-
- 18 Dec, 2023 1 commit
-
-
Dhruv Nair authored
* deprecate pipe * make style * update * add deprecation message * format * remove tests for deprecated pipelines * remove deprecation message * make style * fix copies * clean up * clean * clean * clean * clean up * clean up * clean up toctree * clean up --------- Co-authored-by:
-
- 14 Dec, 2023 1 commit
-
-
Steven Liu authored
add ip-adapter Co-authored-by:
-
- 06 Dec, 2023 1 commit
-
-
UmerHA authored
* Check in 23-10-05 * check-in 23-10-06 * check-in 23-10-07 2pm * check-in 23-10-08 * check-in 231009T1200 * check-in 230109 * checkin 231010 * init + forward run * checkin * checkin * ControlNetXSModel is now saveable+loadable * Forward works * checkin * Pipeline works with `no_control=True` * checkin * debug: save intermediate outputs of resnet * checkin * Understood time error + fixed connection error * checkin * checkin 231106T1600 * turned off detailled debug prints * time debug logs * small fix * Separated control_scale for connections/time * simplified debug logging * Full denoising works with control scale = 0 * aligned logs * Added control_attention_head_dim param * Passing n_heads instead of dim_head into ctrl unet * Fixed ctrl midblock bug * Cleanup * Fixed time dtype bug * checkin * 1. from_unet, 2. base passed, 3. all unet params * checkin * Finished docstrings * cleanup * make style * checkin * more tests pass * Fixed tests * removed debug logs * make style + quality * make fix-copies * fixed documentation * added cnxs to doc toc * added control start/end param * Update controlnetxs_sdxl.md * tried to fix copies.. * Fixed norm_num_groups in from_unet * added sdxl-depth test * created SD2.1 controlnet-xs pipeline * re-added debug logs * Adjusting group norm ; readded logs * Added debug log statements * removed debug logs ; started tests for sd2.1 * updated sd21 tests * fixed tests * fixed tests * slightly increased error tolerance for 1 test * make style & quality * Added docs for CNXS-SD * make fix-copies * Fixed sd compile test ; fixed gradient ckpointing * vae downs = cnxs conditioning downs; removed guess * make style & quality * Fixed tests * fixed test * Incorporated review feedback * simplified control model surgery * fixed tests & make style / quality * Updated docs; deleted pip & cursor files * Rolled back minimal change to resnet * Update resnet.py * Update resnet.py * Update src/diffusers/models/controlnetxs.py Co-authored-by:
-
- 29 Nov, 2023 3 commits
-
-
Patrick von Platen authored
* add diffusers example * add diffusers example * Comment about making it faster * Apply suggestions from code review Co-authored-by:
Pedro Cuenca <pedro@huggingface.co> --------- Co-authored-by:
-
Suraj Patil authored
* begin model * finish blocks * add_embedding * addition_time_embed_dim * use TimestepEmbedding * fix temporal res block * fix time_pos_embed * fix add_embedding * add conversion script * fix model * up * add new resnet blocks * make forward work * return sample in original shape * fix temb shape in TemporalResnetBlock * add spatio temporal transformers * add vae blocks * fix blocks * update * update * fix shapes in Alphablender and add time activation in res blcok * use new blocks * style * fix temb shape * fix SpatioTemporalResBlock * reuse TemporalBasicTransformerBlock * fix TemporalBasicTransformerBlock * use TransformerSpatioTemporalModel * fix TransformerSpatioTemporalModel * fix time_context dim * clean up * make temb optional * add blocks * rename model * update conversion script * remove UNetMidBlockSpatioTemporal * add in init * remove unused arg * remove unused arg * remove more unsed args * up * up * check for None * update vae * update up/mid blocks for decoder * begin pipeline * adapt scheduler * add guidance scalings * fix norm eps in temporal transformers * add temporal autoencoder * make pipeline run * fix frame decodig * decode in float32 * decode n frames at a time * pass decoding_t to decode_latents * fix decode_latents * vae encode/decode in fp32 * fix dtype in TransformerSpatioTemporalModel * type image_latents same as image_embeddings * allow using differnt eps in temporal block for video decoder * fix default values in vae * pass num frames in decode * switch spatial to temporal for mixing in VAE * fix num frames during split decoding * cast alpha to sample dtype * fix attention in MidBlockTemporalDecoder * fix typo * fix guidance_scales dtype * fix missing activation in TemporalDecoder * skip_post_quant_conv * add vae conversion * style * take guidance scale as input * up * allow passing PIL to export_video * accept fps as arg * add pipeline and vae in init * remove hack * use AutoencoderKLTemporalDecoder * don't scale image latents * add unet tests * clean up unet * clean TransformerSpatioTemporalModel * add slow svd test * clean up * make temb optional in Decoder mid block * fix norm eps in TransformerSpatioTemporalModel * clean up temp decoder * clean up * clean up * use c_noise values for timesteps * use math for log * update * fix copies * doc * upcast vae * update forward pass for gradient checkpointing * make added_time_ids is tensor * up * fix upcasting * remove post quant conv * add _resize_with_antialiasing * fix _compute_padding * cleanup model * more cleanup * more cleanup * more cleanup * remove freeu * remove attn slice * small clean * up * up * remove extra step kwargs * remove eta * remove dropout * remove callback * remove merge factor args * clean * clean up * move to dedicated folder * remove attention_head_dim * docstr and small fix * update unet doc strings * rename decoding_t * correct linting * store c_skip and c_out * cleanup * clean TemporalResnetBlock * more cleanup * clean up vae * clean up * begin doc * more cleanup * up * up * doc * Improve * better naming * better naming * better naming * better naming * better naming * better naming * better naming * better naming * Apply suggestions from code review * Default chunk size to None * add example * Better * Apply suggestions from code review * update doc * Update src/diffusers/pipelines/stable_diffusion_video/pipeline_stable_diffusion_video.py Co-authored-by:
apolinário <joaopaulo.passos@gmail.com>
-
Steven Liu authored
* first draft * feedback
-
