• Benjamin Bossan's avatar
    FIX set_lora_device when target layers differ (#11844) · 25279175
    Benjamin Bossan authored
    
    
    * FIX set_lora_device when target layers differ
    
    Resolves #11833
    
    Fixes a bug that occurs after calling set_lora_device when multiple LoRA
    adapters are loaded that target different layers.
    
    Note: Technically, the accompanying test does not require a GPU because
    the bug is triggered even if the parameters are already on the
    corresponding device, i.e. loading on CPU and then changing the device
    to CPU is sufficient to cause the bug. However, this may be optimized
    away in the future, so I decided to test with GPU.
    
    * Update docstring to warn about device mismatch
    
    * Extend docstring with an example
    
    * Fix docstring
    
    ---------
    Co-authored-by: default avatarSayak Paul <spsayakpaul@gmail.com>
    25279175
lora_base.py 43.7 KB