• Sayak Paul's avatar
    [Feat] Support SDXL Kohya-style LoRA (#4287) · 4a4cdd6b
    Sayak Paul authored
    
    
    * sdxl lora changes.
    
    * better name replacement.
    
    * better replacement.
    
    * debugging
    
    * debugging
    
    * debugging
    
    * debugging
    
    * debugging
    
    * remove print.
    
    * print state dict keys.
    
    * print
    
    * distingisuih better
    
    * debuggable.
    
    * fxi: tyests
    
    * fix: arg from training script.
    
    * access from class.
    
    * run style
    
    * debug
    
    * save intermediate
    
    * some simplifications for SDXL LoRA
    
    * styling
    
    * unet config is not needed in diffusers format.
    
    * fix: dynamic SGM block mapping for SDXL kohya loras (#4322)
    
    * Use lora compatible layers for linear proj_in/proj_out (#4323)
    
    * improve condition for using the sgm_diffusers mapping
    
    * informative comment.
    
    * load compatible keys and embedding layer maaping.
    
    * Get SDXL 1.0 example lora to load
    
    * simplify
    
    * specif ranks and hidden sizes.
    
    * better handling of k rank and hidden
    
    * debug
    
    * debug
    
    * debug
    
    * debug
    
    * debug
    
    * fix: alpha keys
    
    * add check for handling LoRAAttnAddedKVProcessor
    
    * sanity comment
    
    * modifications for text encoder SDXL
    
    * debugging
    
    * debugging
    
    * debugging
    
    * debugging
    
    * debugging
    
    * debugging
    
    * debugging
    
    * debugging
    
    * denugging
    
    * debugging
    
    * debugging
    
    * debugging
    
    * debugging
    
    * debugging
    
    * debugging
    
    * debugging
    
    * debugging
    
    * debugging
    
    * debugging
    
    * debugging
    
    * debugging
    
    * debugging
    
    * debugging
    
    * debugging
    
    * debugging
    
    * debugging
    
    * debugging
    
    * up
    
    * up
    
    * up
    
    * up
    
    * up
    
    * up
    
    * unneeded comments.
    
    * unneeded comments.
    
    * kwargs for the other attention processors.
    
    * kwargs for the other attention processors.
    
    * debugging
    
    * debugging
    
    * debugging
    
    * debugging
    
    * improve
    
    * debugging
    
    * debugging
    
    * more print
    
    * Fix alphas
    
    * debugging
    
    * debugging
    
    * debugging
    
    * debugging
    
    * debugging
    
    * debugging
    
    * clean up
    
    * clean up.
    
    * debugging
    
    * fix: text
    
    ---------
    Co-authored-by: default avatarPatrick von Platen <patrick.v.platen@gmail.com>
    Co-authored-by: default avatarBatuhan Taskaya <batuhan@python.org>
    4a4cdd6b
lora.py 4.81 KB