• Lintang Sutawika's avatar
    Cont metrics (#1475) · 96d185fa
    Lintang Sutawika authored
    
    
    * add brier_score
    
    * process brier_score
    
    * brier score is working for N-sized class
    
    * fxied brier score
    
    * add TED to BigBench and Brier score to MMLU
    
    * format
    
    * Update metrics.py
    
    * Update task.py
    
    * Update generate_until_template_yaml
    
    * Delete lm_eval/tasks/bigbench/aux_metric.py
    
    * Update generate_until_template_yaml
    
    * Update _default_template_yaml
    
    * Update _generate_configs.py
    
    * Update _generate_configs.py
    
    * Update _generate_configs.py
    
    * fix (format?)
    
    * format?
    
    * format, once more
    
    ---------
    Co-authored-by: default avatarHailey Schoelkopf <65563625+haileyschoelkopf@users.noreply.github.com>
    96d185fa
task.py 54.5 KB