• Ayush Mangal's avatar
    Add BLIP Diffusion (#4388) · 157c9011
    Ayush Mangal authored
    
    
    * Add BLIP Diffusion skeleton
    
    * Add other model components
    
    * Add BLIP2, need to change it for now
    
    * Fix pipeline imports
    
    * Load pretrained ViT
    
    * Make qformer fwd pass same
    
    * Replicate fwd passes
    
    * Fix device bug
    
    * Add accelerate functions
    
    * Remove extra functions from Blip2
    
    * Minor bug
    
    * Integrate initial review changes
    
    * Refactoring
    
    * Refactoring
    
    * Refactor
    
    * Add controlnet
    
    * Refactor
    
    * Update conversion script
    
    * Add image processor
    
    * Shift postprocessing to ImageProcessor
    
    * Refactor
    
    * Fix device
    
    * Add fast tests
    
    * Update conversion script
    
    * Fix checkpoint conversion script
    
    * Integrate review changes
    
    * Integrate reivew changes
    
    * Remove unused functions from test
    
    * Reuse HF image processor in Cond image
    
    * Create new BlipImageProcessor based on transfomers
    
    * Fix image preprocessor
    
    * Minor
    
    * Minor
    
    * Add canny preprocessing
    
    * Fix controlnet preprocessing
    
    * Fix blip diffusion test
    
    * Add controlnet test
    
    * Add initial doc strings
    
    * Integrate review changes
    
    * Refactor
    
    * Update examples
    
    * Remove DDIM comments
    
    * Add copied from for prepare_latents
    
    * Add type anotations
    
    * Add docstrings
    
    * Do black formatting
    
    * Add batch support
    
    * Make tests pass
    
    * Make controlnet tests pass
    
    * Black formatting
    
    * Fix progress bar
    
    * Fix some licensing comments
    
    * Fix imports
    
    * Refactor controlnet
    
    * Make tests faster
    
    * Edit examples
    
    * Black formatting/Ruff
    
    * Add doc
    
    * Minor
    Co-authored-by: default avatarPatrick von Platen <patrick.v.platen@gmail.com>
    
    * Move controlnet pipeline
    
    * Make tests faster
    
    * Fix imports
    
    * Fix formatting
    
    * Fix make errors
    
    * Fix make errors
    
    * Minor
    
    * Add suggested doc changes
    Co-authored-by: default avatarSayak Paul <spsayakpaul@gmail.com>
    
    * Edit docs
    
    * Fix 16 bit loading
    
    * Update examples
    
    * Edit toctree
    
    * Update docs/source/en/api/pipelines/blip_diffusion.md
    Co-authored-by: default avatarSayak Paul <spsayakpaul@gmail.com>
    
    * Minor
    
    * Add tips
    
    * Edit examples
    
    * Update model paths
    
    ---------
    Co-authored-by: default avatarPatrick von Platen <patrick.v.platen@gmail.com>
    Co-authored-by: default avatarSayak Paul <spsayakpaul@gmail.com>
    157c9011
__init__.py 19 KB