Commits · 6ec258cca5b922ebee2bcab61ae68ddea1540e87 · gaoqiong / lm-evaluation-harness

12 Dec, 2023 2 commits
- allow N/A as a possible value for Stderr · d6f8faad
  lintangsutawika authored Dec 12, 2023
  
  d6f8faad
- calculate std if items > 1 · 91279a6e
  lintangsutawika authored Dec 12, 2023
  
  91279a6e
11 Dec, 2023 5 commits
- Delete lm_eval/tasks/glue/sst directory · ce4d9c68
  weijie authored Dec 11, 2023
  
  ce4d9c68
- Change the sub-task name from sst to sst2 in glue · dc556837
  weijie authored Dec 11, 2023
  
  dc556837
- Update template of qqp dataset · 5dbdf582
  weijie authored Dec 11, 2023
  
  5dbdf582
- Debugging · c256eda8
  h-albert-lee authored Dec 11, 2023
  
  c256eda8
- Change spaces to underline · 489814b7
  h-albert-lee authored Dec 11, 2023
  
  489814b7
10 Dec, 2023 6 commits
- Remove blanks at the end of a file · 4edd9c4c
  h-albert-lee authored Dec 10, 2023
  
  4edd9c4c
- reformatted with Black · 520d8ce2
  h-albert-lee authored Dec 10, 2023
  
  520d8ce2
- Modify code for flake8 · 95908d8b
  h-albert-lee authored Dec 10, 2023
  
  95908d8b
- bugfix #2 remove 'context' · 9221cb57
  h-albert-lee authored Dec 10, 2023
  
  9221cb57
- bug fix (f-string) · 7bc9b82b
  h-albert-lee authored Dec 10, 2023
  
  7bc9b82b
- update yaml config · c64da8ac
  h-albert-lee authored Dec 10, 2023
  
  c64da8ac
08 Dec, 2023 1 commit
- implementing kmmlu · 1b14602e
  h-albert-lee authored Dec 08, 2023
  
  1b14602e
07 Dec, 2023 8 commits
- fixed enumeration · 12f260cf
  lintangsutawika authored Dec 07, 2023
  
  12f260cf
- Update minerva_math_algebra.yaml · ce079c96
  Hailey Schoelkopf authored Dec 06, 2023
  
  ce079c96
- Update _cot_zeroshot_template_yaml · ba7ba910
  Hailey Schoelkopf authored Dec 06, 2023
  
  ba7ba910
- Update _fewshot_template_yaml · a2cc877b
  Hailey Schoelkopf authored Dec 06, 2023
  
  a2cc877b
- Update _zeroshot_template_yaml · 361ba192
  Hailey Schoelkopf authored Dec 06, 2023
  
  361ba192
- Update _mmlu_flan_cot_zeroshot_template_yaml · 8c05c6be
  Hailey Schoelkopf authored Dec 06, 2023
  
  8c05c6be
- Update _mmlu_flan_cot_fewshot_template_yaml · 4e34a6e8
  Hailey Schoelkopf authored Dec 06, 2023
  
  4e34a6e8
- Update _cot_fewshot_template_yaml · a6d28ea7
  Lintang Sutawika authored Dec 07, 2023
```
BBH cot fewshot already has fewshot examples in the description. So num_fewshot needs to be set to 0 so that users won't mistakenly set other num_fewshot values.
```
  a6d28ea7
04 Dec, 2023 3 commits
- Update samplers.py · 3512f742
  Hailey Schoelkopf authored Dec 04, 2023
  
  3512f742
- add \n\n to end of description · c30b71c3
  lintangsutawika authored Dec 04, 2023
  
  c30b71c3
- add \n\n to end of description · b32b3793
  lintangsutawika authored Dec 04, 2023
  
  b32b3793
29 Nov, 2023 2 commits
- fixed sampler issue with new default num_fewshot value; n_shot value changed · 38ac7b2a
  baberabb authored Nov 29, 2023
  
  38ac7b2a
- change torch req for mps · 7bb147b5
  baberabb authored Nov 29, 2023
  
  7bb147b5
28 Nov, 2023 13 commits
- Update evaluator.py · b87ee153
  Lintang Sutawika authored Nov 28, 2023
  
  b87ee153
- Delete lm_eval/tasks/siqa/bigbench.yaml · 331d7c51
  Stella Biderman authored Nov 28, 2023
  
  331d7c51
- Update default.yml · 20463a3f
  Stella Biderman authored Nov 28, 2023
  
  20463a3f
- Rename bigbench.yml to default.yml · 5dcbcd63
  Stella Biderman authored Nov 28, 2023
```
Changing the name since this is not actually the big bench format.
```
  5dcbcd63
- reformat · 5ded4d5b
  lintangsutawika authored Nov 28, 2023
  
  5ded4d5b
- reformat · 37a46351
  lintangsutawika authored Nov 28, 2023
  
  37a46351
- update version · 43f615c9
  lintangsutawika authored Nov 28, 2023
  
  43f615c9
- update version · 94cdface
  lintangsutawika authored Nov 28, 2023
  
  94cdface
- add versions · 0d03a9f3
  lintangsutawika authored Nov 28, 2023
  
  0d03a9f3
- fixes how num_fewshot from config is read · 9e65d0ce
  lintangsutawika authored Nov 28, 2023
  
  9e65d0ce
- force configs with num_fewshot=0 to ignore manual num_fewshot setting · 37ccb191
  lintangsutawika authored Nov 28, 2023
  
  37ccb191
- use all_headers for headers · 61d0f137
  lintangsutawika authored Nov 28, 2023
  
  61d0f137
- renamed template file to make it easier to search · 19738ed2
  lintangsutawika authored Nov 28, 2023
  
  19738ed2