• Lintang Sutawika's avatar
    [Refactor] Continuous Metrics (#969) · 967eb4fa
    Lintang Sutawika authored
    * add brier_score
    
    * process brier_score
    
    * brier score is working for N-sized class
    
    * fxied brier score
    
    * add TED to BigBench and Brier score to MMLU
    
    * format
    
    * Update metrics.py
    
    * Update task.py
    
    * Update generate_until_template_yaml
    
    * Delete lm_eval/tasks/bigbench/aux_metric.py
    
    * Update generate_until_template_yaml
    
    * Update _default_template_yaml
    967eb4fa
task.py 48.9 KB