Commits · 18a7c8b18569d9cf686f31cfff3ccbb23c759238 · gaoqiong / lm-evaluation-harness

17 Jul, 2023 8 commits
- attach group identifier to task for aggregation · 18a7c8b1
  lintangsutawika authored Jul 17, 2023
  
  18a7c8b1
- add column as option · ef7588b6
  lintangsutawika authored Jul 17, 2023
  
  ef7588b6
- print two tables · 1f6d3cd6
  lintangsutawika authored Jul 17, 2023
  
  1f6d3cd6
- aggregate is shown in the table · 32a70d89
  lintangsutawika authored Jul 17, 2023
  
  32a70d89
- modified to arc_* · 1d995b6d
  lintangsutawika authored Jul 17, 2023
  
  1d995b6d
- printing all the configs are made optional with --show_config · 5e2f0ffe
  lintangsutawika authored Jul 17, 2023
  
  5e2f0ffe
- prototype for aggregate · 3001569b
  lintangsutawika authored Jul 17, 2023
  
  3001569b
- Merge pull request #679 from EleutherAI/fix-padding-ranks · 53754d41
  Lintang Sutawika authored Jul 17, 2023
```
[Refactor] Fix padding ranks
```
  53754d41
14 Jul, 2023 32 commits
- padding now handled correctly per-reqtype · abccc756
  haileyschoelkopf authored Jul 14, 2023
  
  abccc756
- changes · e5161a6d
  lintangsutawika authored Jul 14, 2023
  
  e5161a6d
- removed json files · 57f08e40
  lintangsutawika authored Jul 14, 2023
  
  57f08e40
- add winogrande · 34fc9e68
  lintangsutawika authored Jul 05, 2023
  
  34fc9e68
- add where to put aggregation · 241aa871
  lintangsutawika authored Jul 05, 2023
  
  241aa871
- added include_benchmarks · 89c4ee5c
  lintangsutawika authored Jul 04, 2023
  
  89c4ee5c
- added benchmark configs · 1d3a1601
  lintangsutawika authored Jul 04, 2023
  
  1d3a1601
- simplify · c70383d5
  lintangsutawika authored Jul 04, 2023
  
  c70383d5
- add ouput/ to ignore · 4a1da52b
  lintangsutawika authored Jul 04, 2023
  
  4a1da52b
- change variable name · 15c186b7
  lintangsutawika authored Jul 04, 2023
  
  15c186b7
- added script to build benchmarks from promptsouce · e5306ea6
  lintangsutawika authored Jul 03, 2023
  
  e5306ea6
- update on how yaml file is made · 3713ec52
  lintangsutawika authored Jun 27, 2023
  
  3713ec52
- more tasks · ee098a47
  lintangsutawika authored Jun 21, 2023
  
  ee098a47
- can include paths in main · 068f8ab2
  lintangsutawika authored Jun 21, 2023
  
  068f8ab2
- changed name of boolq and cb config task · 7e565f33
  lintangsutawika authored Jun 21, 2023
  
  7e565f33
- Merge pull request #671 from EleutherAI/revamp-process · 2c20cd1f
  Lintang Sutawika authored Jul 14, 2023
```
Revamp process
```
  2c20cd1f
- Merge branch 'big-refactor' into revamp-process · 0dadc92a
  Lintang Sutawika authored Jul 14, 2023
  
  0dadc92a
- Merge pull request #678 from EleutherAI/fix-generative-multitarget · bfa57719
  Lintang Sutawika authored Jul 14, 2023
```
[Refactor] Patch to revamp-process
```
  bfa57719
- make sure boolq works · d70f7134
  haileyschoelkopf authored Jul 14, 2023
  
  d70f7134
- override target_delimiter · a36348fa
  lintangsutawika authored Jul 14, 2023
  
  a36348fa
- fixed whitespace for target · addada53
  lintangsutawika authored Jul 14, 2023
  
  addada53
- fix error · 60acbde2
  lintangsutawika authored Jul 14, 2023
  
  60acbde2
- remove info · 1ca2c0ba
  lintangsutawika authored Jul 14, 2023
  
  1ca2c0ba
- adjustments for super_glue · b0597a94
  lintangsutawika authored Jul 14, 2023
  
  b0597a94
- Merge pull request #676 from EleutherAI/new-flags · 6862fa7d
  Lintang Sutawika authored Jul 14, 2023
```
[Refactor] Miscellaneous fixes
```
  6862fa7d
- formatted · f7dde0c3
  lintangsutawika authored Jul 14, 2023
  
  f7dde0c3
- fix typehints in Instance() · 7684bfa9
  haileyschoelkopf authored Jul 14, 2023
  
  7684bfa9
- don't log samples if not requested · 04c81e7d
  haileyschoelkopf authored Jul 14, 2023
  
  04c81e7d
- fix random seed issue, log_samples optional · 5fbc3f86
  haileyschoelkopf authored Jul 14, 2023
  
  5fbc3f86
- updates, corrections, and fixes to match big-refactor · ee633332
  lintangsutawika authored Jul 14, 2023
  
  ee633332
- doc_to_target and doc_to_text should also accept int · 94a49f70
  lintangsutawika authored Jul 14, 2023
  
  94a49f70
- config fixes · 248b45da
  lintangsutawika authored Jul 14, 2023
  
  248b45da