Commits · f8c2cfcbe1cde622ceed933055f60cc102c8f4e4 · gaoqiong / lm-evaluation-harness

29 Nov, 2023 1 commit
- fixed sampler issue with new default num_fewshot value; n_shot value changed · 38ac7b2a
  baberabb authored Nov 29, 2023
  
  38ac7b2a
28 Nov, 2023 3 commits
- Update evaluator.py · b87ee153
  Lintang Sutawika authored Nov 28, 2023
  
  b87ee153
- fixes how num_fewshot from config is read · 9e65d0ce
  lintangsutawika authored Nov 28, 2023
  
  9e65d0ce
- force configs with num_fewshot=0 to ignore manual num_fewshot setting · 37ccb191
  lintangsutawika authored Nov 28, 2023
  
  37ccb191
24 Nov, 2023 3 commits
- udate · a5b65c9b
  lintangsutawika authored Nov 24, 2023
  
  a5b65c9b
- fix missing lines · b64e592d
  lintangsutawika authored Nov 24, 2023
  
  b64e592d
- gen_kwargs is parsed once · 47933b98
  lintangsutawika authored Nov 24, 2023
  
  47933b98
17 Nov, 2023 3 commits
- format · bf6cf1de
  lintangsutawika authored Nov 17, 2023
  
  bf6cf1de
- will check if group_name is None · 2314ff4f
  lintangsutawika authored Nov 17, 2023
  
  2314ff4f
- edits and format · 10cc0a56
  lintangsutawika authored Nov 17, 2023
  
  10cc0a56
14 Nov, 2023 2 commits
- removed unused variables · d9230aa8
  lintangsutawika authored Nov 14, 2023
  
  d9230aa8
- enables tasks of different group but same task_alis (for example, if... · 7760573f
  lintangsutawika authored Nov 14, 2023
```
enables tasks of different group but same task_alis (for example, if evaluating on different versions of MMLU
```
  7760573f
13 Nov, 2023 1 commit
- num fewshot is printed in table · 81495b0a
  lintangsutawika authored Nov 13, 2023
  
  81495b0a
10 Nov, 2023 1 commit
- added initialize_task and updated where eval_logger is imported from · 056c9d85
  lintangsutawika authored Nov 10, 2023
  
  056c9d85
07 Nov, 2023 1 commit
- add squad from master · 8bf55a20
  lintangsutawika authored Nov 07, 2023
  
  8bf55a20
06 Nov, 2023 2 commits
- add space after - · 44124d95
  lintangsutawika authored Nov 06, 2023
  
  44124d95
- remove samples in the table · 1e1dbaf3
  lintangsutawika authored Nov 06, 2023
  
  1e1dbaf3
02 Nov, 2023 5 commits
- eval_logger is not imported from logger.py anymore · f701ba7d
  lintangsutawika authored Nov 02, 2023
  
  f701ba7d
- very rough way of utilzing verbosity · ca8b00cc
  lintangsutawika authored Nov 02, 2023
  
  ca8b00cc
- removed print · 6eac0056
  lintangsutawika authored Nov 02, 2023
  
  6eac0056
- fixed stderr calculation · d1e7a30a
  lintangsutawika authored Nov 02, 2023
  
  d1e7a30a
- fixing display name · 29ba8cb1
  lintangsutawika authored Nov 02, 2023
  
  29ba8cb1
01 Nov, 2023 3 commits
- remove nonstandard logger from evaluator.py · 2fda31ca
  haileyschoelkopf authored Nov 01, 2023
  
  2fda31ca
- new way to display tasks · 60dd33c8
  lintangsutawika authored Nov 01, 2023
  
  60dd33c8
- new way to display tasks · f5bdefe8
  lintangsutawika authored Nov 01, 2023
  
  f5bdefe8
17 Oct, 2023 3 commits
- print tasks in alphabetically · 93a45962
  lintangsutawika authored Oct 17, 2023
  
  93a45962
- pre-commit reformat · c4f0bf75
  lintangsutawika authored Oct 17, 2023
  
  c4f0bf75
- sqrt final variance calculation to get stderr · 4ccd2ec6
  lintangsutawika authored Oct 17, 2023
  
  4ccd2ec6
16 Oct, 2023 3 commits
- removed comments · e97019c0
  lintangsutawika authored Oct 16, 2023
  
  e97019c0
- added stderr reprocessing for groups · 00859825
  lintangsutawika authored Oct 16, 2023
  
  00859825
- default to weighted averaging · 1dc8f96f
  lintangsutawika authored Oct 16, 2023
  
  1dc8f96f
03 Oct, 2023 1 commit
- Allow Generation arguments on greedy_until reqs · ec5846d7
  USVSN Sai Prashanth authored Oct 03, 2023
  
  ec5846d7
19 Sep, 2023 1 commit
- moved benchmarks back to tasks/ · 1ed76cfa
  lintangsutawika authored Sep 19, 2023
  
  1ed76cfa
15 Sep, 2023 1 commit
- reenable bootstrapping · 624fb207
  haileyschoelkopf authored Sep 15, 2023
  
  624fb207
13 Sep, 2023 5 commits
- add comments · 892f40a9
  lintangsutawika authored Sep 13, 2023
  
  892f40a9
- aggregates across tasks within the same group in addition to average accross task average · d2804132
  lintangsutawika authored Sep 13, 2023
  
  d2804132
- better presentation · d4f62844
  lintangsutawika authored Sep 13, 2023
  
  d4f62844
- cleaner process · e1e05b19
  lintangsutawika authored Sep 13, 2023
  
  e1e05b19
- cleaner process · 91d84840
  lintangsutawika authored Sep 13, 2023
  
  91d84840
12 Sep, 2023 1 commit
- remove comments · 993baaa6
  lintangsutawika authored Sep 12, 2023
  
  993baaa6