Commits · 7520ea80fc7ff24eb9b33562caa1bb45ba7ecef3 · gaoqiong / lm-evaluation-harness

18 Jul, 2023 6 commits
- Merge branch 'benchmark-scripts' into add-comments · 7520ea80
  Lintang Sutawika authored Jul 18, 2023
  
  7520ea80
- Update evaluator.py · 1f21b8ef
  Lintang Sutawika authored Jul 18, 2023
  
  1f21b8ef
- Update evaluator.py · b85ec188
  Lintang Sutawika authored Jul 18, 2023
  
  b85ec188
- Update evaluator.py · 3ab0aa5c
  Lintang Sutawika authored Jul 18, 2023
  
  3ab0aa5c
- Merge branch 'big-refactor' into benchmark-scripts · 7f654439
  Lintang Sutawika authored Jul 18, 2023
  
  7f654439
- Update evaluator.py · db9231e2
  Lintang Sutawika authored Jul 18, 2023
  
  db9231e2
17 Jul, 2023 20 commits
- add comments to all vars defined in evaluate() · 777f57ce
  haileyschoelkopf authored Jul 17, 2023
  
  777f57ce
- Merge pull request #684 from baberabb/big-refactor_output · 3cd3921f
  Hailey Schoelkopf authored Jul 17, 2023
```
edited output_path and added help to args
```
  3cd3921f
- removed stale args. added assertion if log_samples true and no output_path · decb55cd
  baberabb authored Jul 17, 2023
  
  decb55cd
- Merge branch 'benchmark-scripts' of... · 9a4e7e7f
  lintangsutawika authored Jul 17, 2023
```
Merge branch 'benchmark-scripts' of https://github.com/EleutherAI/lm-evaluation-harness into benchmark-scripts
```
  9a4e7e7f
- attach group identifier to task for aggregation · 18a7c8b1
  lintangsutawika authored Jul 17, 2023
  
  18a7c8b1
- add column as option · ef7588b6
  lintangsutawika authored Jul 17, 2023
  
  ef7588b6
- print two tables · 1f6d3cd6
  lintangsutawika authored Jul 17, 2023
  
  1f6d3cd6
- edited output_path and added help to args · 74ba283d
  baberabb authored Jul 17, 2023
  
  74ba283d
- Merge branch 'big-refactor' into benchmark-scripts · 027fe14c
  Lintang Sutawika authored Jul 17, 2023
  
  027fe14c
- aggregate is shown in the table · 32a70d89
  lintangsutawika authored Jul 17, 2023
  
  32a70d89
- modified to arc_* · 1d995b6d
  lintangsutawika authored Jul 17, 2023
  
  1d995b6d
- printing all the configs are made optional with --show_config · 5e2f0ffe
  lintangsutawika authored Jul 17, 2023
  
  5e2f0ffe
- Merge pull request #682 from yeoedward/anli · 4dfa8aba
  Lintang Sutawika authored Jul 17, 2023
```
[Refactor] Migrate ANLI tasks to yaml
```
  4dfa8aba
- Update anli_r3.yaml · 97a172ae
  Lintang Sutawika authored Jul 17, 2023
  
  97a172ae
- Update anli_r2.yaml · cd7f642d
  Lintang Sutawika authored Jul 17, 2023
  
  cd7f642d
- Update anli_r1.yaml · a7c1a639
  Lintang Sutawika authored Jul 17, 2023
  
  a7c1a639
- [Refactor] Migrate anli tasks to yaml · b783c68e
  Eddy Yeo authored Jul 17, 2023
  
  b783c68e
- Merge pull request #680 from baberabb/big-refactor_fixup · 7aa13c47
  Hailey Schoelkopf authored Jul 17, 2023
```
[Refactor] minor edits
```
  7aa13c47
- prototype for aggregate · 3001569b
  lintangsutawika authored Jul 17, 2023
  
  3001569b
- Merge pull request #679 from EleutherAI/fix-padding-ranks · 53754d41
  Lintang Sutawika authored Jul 17, 2023
```
[Refactor] Fix padding ranks
```
  53754d41
16 Jul, 2023 2 commits
- set fp32 if device=mps · 43407f36
  baberabb authored Jul 16, 2023
  
  43407f36
- set fp32 if device=mps · ce214979
  baberabb authored Jul 16, 2023
  
  ce214979
15 Jul, 2023 2 commits
- add mps · 2bb7ce3b
  baberabb authored Jul 15, 2023
  
  2bb7ce3b
- removed rnd from task.fewshot_context in write_out.py; add mps in huggingface.py · 73b2149f
  baberabb authored Jul 15, 2023
  
  73b2149f
14 Jul, 2023 10 commits
- padding now handled correctly per-reqtype · abccc756
  haileyschoelkopf authored Jul 14, 2023
  
  abccc756
- changes · e5161a6d
  lintangsutawika authored Jul 14, 2023
  
  e5161a6d
- removed json files · 57f08e40
  lintangsutawika authored Jul 14, 2023
  
  57f08e40
- add winogrande · 34fc9e68
  lintangsutawika authored Jul 05, 2023
  
  34fc9e68
- add where to put aggregation · 241aa871
  lintangsutawika authored Jul 05, 2023
  
  241aa871
- added include_benchmarks · 89c4ee5c
  lintangsutawika authored Jul 04, 2023
  
  89c4ee5c
- added benchmark configs · 1d3a1601
  lintangsutawika authored Jul 04, 2023
  
  1d3a1601
- simplify · c70383d5
  lintangsutawika authored Jul 04, 2023
  
  c70383d5
- add ouput/ to ignore · 4a1da52b
  lintangsutawika authored Jul 04, 2023
  
  4a1da52b
- change variable name · 15c186b7
  lintangsutawika authored Jul 04, 2023
  
  15c186b7