• Marc Sun's avatar
    add exllamav2 arg (#26437) · 8214d6e7
    Marc Sun authored
    * add_ xllamav2 arg
    
    * add test
    
    * style
    
    * add check
    
    * add doc
    
    * replace by use_exllama_v2
    
    * fix tests
    
    * fix doc
    
    * style
    
    * better condition
    
    * fix logic
    
    * add deprecate msg
    8214d6e7
quantization.md 21 KB