Commits · c21240c0a9b68f5845e8705aeb5326b17da2d3bb · gaoqiong / lm-evaluation-harness

17 Nov, 2023 10 commits
- update bigbench datasets to mirror, rename groupnames · 487767ab
  haileyschoelkopf authored Nov 17, 2023
  
  487767ab
- fix error when using wildcard task names · 2444b364
  haileyschoelkopf authored Nov 17, 2023
  
  2444b364
- fix error when using wildcard task names · aa9e1062
  haileyschoelkopf authored Nov 17, 2023
  
  aa9e1062
- format · bf6cf1de
  lintangsutawika authored Nov 17, 2023
  
  bf6cf1de
- will check if group_name is None · 2314ff4f
  lintangsutawika authored Nov 17, 2023
  
  2314ff4f
- precommit · f40b7d0e
  lintangsutawika authored Nov 17, 2023
  
  f40b7d0e
- removed provide_description arg · 0d209e2e
  lintangsutawika authored Nov 17, 2023
  
  0d209e2e
- edits and format · 10cc0a56
  lintangsutawika authored Nov 17, 2023
  
  10cc0a56
- added alternative prompts for mathqa · bbd6ab3a
  lintangsutawika authored Nov 17, 2023
  
  bbd6ab3a
- added alternative prompts for hellaswag · 7cd1555e
  lintangsutawika authored Nov 17, 2023
  
  7cd1555e
16 Nov, 2023 1 commit
- remove provide_description flag · a745d589
  haileyschoelkopf authored Nov 16, 2023
  
  a745d589
15 Nov, 2023 1 commit
- update squad file · c9a3fd3f
  lintangsutawika authored Nov 15, 2023
  
  c9a3fd3f
14 Nov, 2023 5 commits
- add task_alias · f8740ff4
  lintangsutawika authored Nov 14, 2023
  
  f8740ff4
- removed unused variables · d9230aa8
  lintangsutawika authored Nov 14, 2023
  
  d9230aa8
- enables tasks of different group but same task_alis (for example, if... · 7760573f
  lintangsutawika authored Nov 14, 2023
```
enables tasks of different group but same task_alis (for example, if evaluating on different versions of MMLU
```
  7760573f
- alternative prompts for arithmetic · c25f6a31
  lintangsutawika authored Nov 14, 2023
  
  c25f6a31
- brier score for loglikelihood task · 5e4f1799
  lintangsutawika authored Nov 14, 2023
  
  5e4f1799
10 Nov, 2023 2 commits
- fixes · 69186ef2
  lintangsutawika authored Nov 10, 2023
  
  69186ef2
- added initialize_task and updated where eval_logger is imported from · 056c9d85
  lintangsutawika authored Nov 10, 2023
  
  056c9d85
09 Nov, 2023 7 commits
- updates to scrolls · 51753750
  lintangsutawika authored Nov 09, 2023
  
  51753750
- reformat · 097c9253
  lintangsutawika authored Nov 09, 2023
  
  097c9253
- reformat · 6c812bb5
  lintangsutawika authored Nov 09, 2023
  
  6c812bb5
- reformat · 9a64e642
  lintangsutawika authored Nov 09, 2023
  
  9a64e642
- add scrolls · 75137836
  lintangsutawika authored Nov 09, 2023
  
  75137836
- readded task descriptions · a3520619
  lintangsutawika authored Nov 09, 2023
  
  a3520619
- merged cont-metrics here · 6998762a
  lintangsutawika authored Nov 09, 2023
  
  6998762a
08 Nov, 2023 6 commits
- Update generate_until_template_yaml · 1522009c
  Lintang Sutawika authored Nov 08, 2023
  
  1522009c
- Update task.py · 66b3c3a2
  Lintang Sutawika authored Nov 08, 2023
  
  66b3c3a2
- Update metrics.py · 59b5471c
  Lintang Sutawika authored Nov 08, 2023
  
  59b5471c
- removed yaml version of squad · d3f429ac
  lintangsutawika authored Nov 08, 2023
  
  d3f429ac
- removed yaml version of squad · 9817e7c2
  lintangsutawika authored Nov 08, 2023
  
  9817e7c2
- added comment line · 7b60ead4
  lintangsutawika authored Nov 08, 2023
  
  7b60ead4
07 Nov, 2023 4 commits
- moved squad.py and format changes · e1d5c849
  lintangsutawika authored Nov 07, 2023
  
  e1d5c849
- add squad from master · 8bf55a20
  lintangsutawika authored Nov 07, 2023
  
  8bf55a20
- format · 0a39d055
  lintangsutawika authored Nov 07, 2023
  
  0a39d055
- add TED to BigBench and Brier score to MMLU · 2b7d8c2d
  lintangsutawika authored Nov 07, 2023
  
  2b7d8c2d
06 Nov, 2023 4 commits
- fxied brier score · 5cc65a79
  lintangsutawika authored Nov 06, 2023
  
  5cc65a79
- add space after - · 44124d95
  lintangsutawika authored Nov 06, 2023
  
  44124d95
- brier score is working for N-sized class · d49636a3
  lintangsutawika authored Nov 06, 2023
  
  d49636a3
- process brier_score · a76754ff
  lintangsutawika authored Nov 06, 2023
  
  a76754ff