• David Corvoysier's avatar
    Update neuron backend (#2314) · 9a092f37
    David Corvoysier authored
    * feat(neuron): align with latest optimum-neuron
    
    * feat(neuron): support pre-exported neuron models
    
    * fix(neuron): correctly use max_length
    
    * fix(neuron): adapt loglikelihood
    
    The evaluation of log likelihood was not working for neuron models
    using continuous batching, such as all cached neuron LLama models.
    
    * refactor(neuron): remove dead code
    9a092f37
neuron_optimum.py 28.1 KB