Commits · f8740ff4d194c90453065befa3c22b7381244dd1 · gaoqiong / lm-evaluation-harness

14 Nov, 2023 3 commits
- add task_alias · f8740ff4
  lintangsutawika authored Nov 14, 2023
  
  f8740ff4
- alternative prompts for arithmetic · c25f6a31
  lintangsutawika authored Nov 14, 2023
  
  c25f6a31
- brier score for loglikelihood task · 5e4f1799
  lintangsutawika authored Nov 14, 2023
  
  5e4f1799
09 Nov, 2023 1 commit
- merged cont-metrics here · 6998762a
  lintangsutawika authored Nov 09, 2023
  
  6998762a
08 Nov, 2023 4 commits
- Merge branch 'cont-metrics' of https://github.com/EleutherAI/lm-evaluation-harness into alt_worlds · 2184b8de
  lintangsutawika authored Nov 08, 2023
  
  2184b8de
- Update generate_until_template_yaml · 1522009c
  Lintang Sutawika authored Nov 08, 2023
  
  1522009c
- Update task.py · 66b3c3a2
  Lintang Sutawika authored Nov 08, 2023
  
  66b3c3a2
- Update metrics.py · 59b5471c
  Lintang Sutawika authored Nov 08, 2023
  
  59b5471c
07 Nov, 2023 3 commits
- Merge branch 'big-refactor' into cont-metrics · ccca64f7
  Lintang Sutawika authored Nov 07, 2023
  
  ccca64f7
- format · 0a39d055
  lintangsutawika authored Nov 07, 2023
  
  0a39d055
- add TED to BigBench and Brier score to MMLU · 2b7d8c2d
  lintangsutawika authored Nov 07, 2023
  
  2b7d8c2d
06 Nov, 2023 10 commits
- fxied brier score · 5cc65a79
  lintangsutawika authored Nov 06, 2023
  
  5cc65a79
- Merge pull request #952 from EleutherAI/fixup-logging · b7a4ea06
  Lintang Sutawika authored Nov 06, 2023
```
[Refactor] Logging fixes
```
  b7a4ea06
- Merge pull request #965 from EleutherAI/remove-goldalias · 01227a7e
  Lintang Sutawika authored Nov 06, 2023
```
[Refactor] Remove deprecated `gold_alias` task YAML option
```
  01227a7e
- Merge pull request #922 from EleutherAI/mmlu_subgroups · 815f59e6
  Lintang Sutawika authored Nov 06, 2023
```
[Refactor] Mmlu subgroups and weight avg
```
  815f59e6
- add space after - · 44124d95
  lintangsutawika authored Nov 06, 2023
  
  44124d95
- brier score is working for N-sized class · d49636a3
  lintangsutawika authored Nov 06, 2023
  
  d49636a3
- process brier_score · a76754ff
  lintangsutawika authored Nov 06, 2023
  
  a76754ff
- add brier_score · 98f9bac9
  lintangsutawika authored Nov 06, 2023
  
  98f9bac9
- reformat · 491d4799
  lintangsutawika authored Nov 06, 2023
  
  491d4799
- remove samples in the table · 1e1dbaf3
  lintangsutawika authored Nov 06, 2023
  
  1e1dbaf3
04 Nov, 2023 1 commit
- Merge pull request #966 from EleutherAI/fix-verbose · 3533e4b9
  Stella Biderman authored Nov 03, 2023
```
[Refactor] Bugfix: AttributeError: 'Namespace' object has no attribute 'verbose'
```
  3533e4b9
03 Nov, 2023 8 commits
- fix bug with args.verbose · 5f3b8bf6
  haileyschoelkopf authored Nov 03, 2023
  
  5f3b8bf6
- Merge pull request #815 from EleutherAI/model-written-eval · 6f700f98
  Hailey Schoelkopf authored Nov 03, 2023
```
[Refactor] Model written eval
```
  6f700f98
- fix gsm8k regexes · 93d088c8
  haileyschoelkopf authored Nov 03, 2023
  
  93d088c8
- Update ignore.txt · 9f09016d
  Hailey Schoelkopf authored Nov 03, 2023
  
  9f09016d
- Update _generate_configs.py · 4d9b928e
  Hailey Schoelkopf authored Nov 03, 2023
  
  4d9b928e
- make sure gold_alias not in docs · b9f0d0d3
  haileyschoelkopf authored Nov 03, 2023
  
  b9f0d0d3
- remove gold_alias from codebase · 37ac5f46
  haileyschoelkopf authored Nov 03, 2023
  
  37ac5f46
- Merge branch 'big-refactor' into fixup-logging · 9e5648b8
  haileyschoelkopf authored Nov 03, 2023
  
  9e5648b8
02 Nov, 2023 10 commits
- cleanup hf tqdm · ad8eee89
  haileyschoelkopf authored Nov 02, 2023
  
  ad8eee89
- Merge pull request #957 from EleutherAI/Refactor]-Patch-for-Generate-Until · c7b3f538
  Lintang Sutawika authored Nov 02, 2023
  
  c7b3f538
- Rename greedy_until_template_yaml to generate_until_template_yaml · 0d1fb8a5
  Lintang Sutawika authored Nov 02, 2023
  
  0d1fb8a5
- removed print · 6eac0056
  lintangsutawika authored Nov 02, 2023
  
  6eac0056
- merged conflicts · e49f2706
  lintangsutawika authored Nov 02, 2023
  
  e49f2706
- added group and task alias · 6dfe848e
  lintangsutawika authored Nov 02, 2023
  
  6dfe848e
- fixed stderr calculation · d1e7a30a
  lintangsutawika authored Nov 02, 2023
  
  d1e7a30a
- fixing display name · 29ba8cb1
  lintangsutawika authored Nov 02, 2023
  
  29ba8cb1
- Merge branch 'big-refactor' into model-written-eval · 2099099b
  Lintang Sutawika authored Nov 02, 2023
  
  2099099b
- Merge pull request #953 from EleutherAI/mmlu-no-train · ae74b808
  Lintang Sutawika authored Nov 02, 2023
```
[Refactor] Don't load MMLU auxiliary_train set
```
  ae74b808