• drbh's avatar
    fix: attempt forward on flash attn2 to check hardware support (#2335) · 215ed3ad
    drbh authored
    * fix: attempt forward on flash attn2 to check hardware support
    
    * fix: warn window_size_left when using flash attn 1
    
    * fix: prefer version check over test op and avoid window_size_left if not flash attn2
    
    * fix: improve condtional and error message
    
    * fix: update sliding window conditional
    
    * fix: simplify changes and revert model changes
    
    * fix: avoid changing conditional
    
    * fix: typo tweak
    215ed3ad
__init__.py 42.6 KB