Commits · f5bdefe84a07fb2db1fb3593d005c411eb21fcdf · gaoqiong / lm-evaluation-harness

01 Nov, 2023 3 commits
- new way to display tasks · f5bdefe8
  lintangsutawika authored Nov 01, 2023
  
  f5bdefe8
- Merge branch 'big-refactor' of... · a5d33ebe
  lintangsutawika authored Nov 01, 2023
```
Merge branch 'big-refactor' of https://github.com/EleutherAI/lm-evaluation-harness into mmlu_subgroups
```
  a5d33ebe
- add task and group alias · dd7002d6
  lintangsutawika authored Nov 01, 2023
  
  dd7002d6
31 Oct, 2023 2 commits
- Merge pull request #949 from EleutherAI/fix-whitespace-warning · 12fe236d
  Lintang Sutawika authored Nov 01, 2023
  
  12fe236d
- fix warning for delimiter having no whitespace · 888d3035
  haileyschoelkopf authored Oct 31, 2023
  
  888d3035
30 Oct, 2023 1 commit

Merge branch 'big-refactor' of... · 21cf6508

lintangsutawika authored Oct 30, 2023

Merge branch 'big-refactor' of https://github.com/EleutherAI/lm-evaluation-harness into mmlu_subgroups

21cf6508

24 Oct, 2023 7 commits
- Merge pull request #915 from EleutherAI/package-patch · 06e69149
  Hailey Schoelkopf authored Oct 24, 2023
```
Update pyproject.toml
```
  06e69149
- Update pyproject.toml · 84db6359
  Hailey Schoelkopf authored Oct 24, 2023
  
  84db6359
- Update pyproject.toml · a1b589df
  Hailey Schoelkopf authored Oct 24, 2023
  
  a1b589df
- Update pyproject.toml · fa1f3571
  Hailey Schoelkopf authored Oct 24, 2023
  
  fa1f3571
- Update pyproject.toml · e7e3adeb
  Hailey Schoelkopf authored Oct 24, 2023
  
  e7e3adeb
- Update pyproject.toml · b3a2af33
  Hailey Schoelkopf authored Oct 24, 2023
  
  b3a2af33
- Merge pull request #937 from EleutherAI/big-refactor_write_out · 56b7021f
  Hailey Schoelkopf authored Oct 24, 2023
```
Big refactor write out adaption
```
  56b7021f
20 Oct, 2023 3 commits
- fix: copy paste mistake · 1a183d03
  Michael Pieler authored Oct 20, 2023
  
  1a183d03
- feat: add --include_path option to write out · 1964ccd3
  Michael Pieler authored Oct 20, 2023
  
  1964ccd3
- Merge pull request #935 from EleutherAI/fix-default-metric-call · 408115ea
  Lintang Sutawika authored Oct 20, 2023
  
  408115ea
19 Oct, 2023 11 commits
- fixes to pythia benchmark, boolqseq2seq · 1428ad57
  haileyschoelkopf authored Oct 19, 2023
  
  1428ad57
- fix bug when target_delimiter is empty string · d9b4ca7f
  haileyschoelkopf authored Oct 19, 2023
  
  d9b4ca7f
- don't call evaluate.load() every time · ff148cd8
  haileyschoelkopf authored Oct 19, 2023
  
  ff148cd8
- fixed registered metric · 8d4d1fa9
  lintangsutawika authored Oct 19, 2023
  
  8d4d1fa9
- changed warning message · 9fbe6eef
  lintangsutawika authored Oct 19, 2023
  
  9fbe6eef
- fix issue with default metrics and aggregation functions · 90d818da
  lintangsutawika authored Oct 19, 2023
  
  90d818da
- Changed `get_default_aggregation` to `get_aggregation` · 6b4161c1
  Lintang Sutawika authored Oct 19, 2023
  
  6b4161c1
- Update registry.py · a6179b49
  Lintang Sutawika authored Oct 19, 2023
```
Removed `get_default_aggregation`
```
  a6179b49
- Merge pull request #916 from jasonkrone/big-refactor · cf617ab1
  Lintang Sutawika authored Oct 19, 2023
```
Fix 'tqdm' object is not subscriptable" error in huggingface.py when batch size is auto
```
  cf617ab1
- Merge pull request #931 from EleutherAI/fix-generate-until · ef332026
  Hailey Schoelkopf authored Oct 18, 2023
```
[Refactor] Generate_until rename
```
  ef332026
- Update README.md · a8d130ab
  Stella Biderman authored Oct 18, 2023
  
  a8d130ab
18 Oct, 2023 5 commits
- don't raise error for unset higher_is_better · e66ba123
  haileyschoelkopf authored Oct 18, 2023
  
  e66ba123
- replace the rest of the 'greedy_until' occurrences · c28d100d
  haileyschoelkopf authored Oct 18, 2023
  
  c28d100d
- Merge branch 'big-refactor' of... · 7d496612
  lintangsutawika authored Oct 18, 2023
```
Merge branch 'big-refactor' of https://github.com/EleutherAI/lm-evaluation-harness into mmlu_subgroups
```
  7d496612
- Merge pull request #929 from EleutherAI/fix-generate-until · 4aafe026
  Lintang Sutawika authored Oct 18, 2023
```
Fix `generate_until` rename
```
  4aafe026
- fix greedy_until abstractmethod · 785153f6
  haileyschoelkopf authored Oct 18, 2023
  
  785153f6
17 Oct, 2023 8 commits
- Merge branch 'big-refactor' of... · f77a3a27
  lintangsutawika authored Oct 17, 2023
```
Merge branch 'big-refactor' of https://github.com/EleutherAI/lm-evaluation-harness into mmlu_subgroups
```
  f77a3a27
- Merge pull request #905 from EleutherAI/fix-unittests · f8342178
  Lintang Sutawika authored Oct 17, 2023
```
[Refactor] Fix Unit Tests
```
  f8342178
- Merge pull request #910 from EleutherAI/verbose · 6a49ed9b
  Hailey Schoelkopf authored Oct 17, 2023
```
[Refactor] Verbose
```
  6a49ed9b
- Merge branch 'big-refactor' into fix-unittests · de71ad92
  Lintang Sutawika authored Oct 17, 2023
  
  de71ad92
- Merge pull request #923 from EleutherAI/fix_squadv2 · 73c80915
  Hailey Schoelkopf authored Oct 17, 2023
```
[Refactor] Squadv2 updates
```
  73c80915
- Merge pull request #927 from EleutherAI/change_output_type · a056eded
  Hailey Schoelkopf authored Oct 17, 2023
```
[Refactor] change all mentions of `greedy_until` to `generate_until`
```
  a056eded
- Merge pull request #926 from EleutherAI/patch_belebele · b3224ec0
  Hailey Schoelkopf authored Oct 17, 2023
```
[Refactor] Precommit formatting for Belebele
```
  b3224ec0
- changed file name · a7ba3d76
  lintangsutawika authored Oct 17, 2023
  
  a7ba3d76