Commits · 21aa92d26ffaf028976e25a7593abaee8f414d2e · gaoqiong / lm-evaluation-harness

15 Aug, 2023 3 commits
- rename var · 21aa92d2
  haileyschoelkopf authored Aug 15, 2023
  
  21aa92d2
- add translation tasks · 40f81c42
  haileyschoelkopf authored Aug 15, 2023
  
  40f81c42
- make pycountry optional dep · 39bd5caf
  haileyschoelkopf authored Aug 15, 2023
  
  39bd5caf
14 Aug, 2023 12 commits
- Merge branch 'big-refactor' into wmt · adde09b3
  haileyschoelkopf authored Aug 14, 2023
  
  adde09b3
- Merge pull request #747 from EleutherAI/haileyschoelkopf-patch-2 · 64c76fc3
  Lintang Sutawika authored Aug 14, 2023
```
[WIP] Update API docs in README
```
  64c76fc3
- Added info on `lm-eval` command. · 725d7e3e
  Lintang Sutawika authored Aug 14, 2023
  
  725d7e3e
- Merge pull request #777 from EleutherAI/benchmark · be79a281
  Lintang Sutawika authored Aug 14, 2023
```
[Refactor] Update Benchmark
```
  be79a281
- Merge pull request #776 from EleutherAI/xnli · f918c8fd
  Lintang Sutawika authored Aug 14, 2023
```
[Refactor] XNLI
```
  f918c8fd
- Update with task list. · 1768f118
  Lintang Sutawika authored Aug 14, 2023
  
  1768f118
- adjustments to have `task` to modify a task name in benchmark · 3de2c0e7
  lintangsutawika authored Aug 14, 2023
  
  3de2c0e7
- explain how to use benchmarks · cccb294f
  lintangsutawika authored Aug 14, 2023
  
  cccb294f
- return fewshot_delimiter if we get "" · 7758d16a
  lintangsutawika authored Aug 14, 2023
  
  7758d16a
- add warning log if prompt returns empty string · 1701c4e2
  lintangsutawika authored Aug 14, 2023
  
  1701c4e2
- add warning log if prompt returns empty string · 722b8974
  lintangsutawika authored Aug 14, 2023
  
  722b8974
- add warning log if prompt returns empty string · bc951ac1
  lintangsutawika authored Aug 14, 2023
  
  bc951ac1
13 Aug, 2023 12 commits
- formate · ce8c2b14
  lintangsutawika authored Aug 13, 2023
  
  ce8c2b14
- update and fixes · 5ddabc29
  lintangsutawika authored Aug 13, 2023
  
  5ddabc29
- reformat · c523063d
  lintangsutawika authored Aug 13, 2023
  
  c523063d
- xnli reformat · 2d54f2c9
  lintangsutawika authored Aug 13, 2023
  
  2d54f2c9
- xnli reformat · ac98d9e1
  lintangsutawika authored Aug 13, 2023
  
  ac98d9e1
- Merge pull request #766 from EleutherAI/patch-crowspairs · 054ed37f
  Hailey Schoelkopf authored Aug 13, 2023
```
[Refactor] Patch crowspairs higher_is_better
```
  054ed37f
- Merge pull request #774 from EleutherAI/speedup-hellaswag · 4f27f0b9
  Hailey Schoelkopf authored Aug 13, 2023
```
[Refactor] Speedup hellaswag context building
```
  4f27f0b9
- update in task guide · db0c46b0
  lintangsutawika authored Aug 13, 2023
  
  db0c46b0
- uncommented tasks · 22ece14a
  lintangsutawika authored Aug 13, 2023
  
  22ece14a
- moved to a new dir · f4813cfc
  lintangsutawika authored Aug 13, 2023
  
  f4813cfc
- update · 83683ddd
  lintangsutawika authored Aug 13, 2023
  
  83683ddd
- added xnli · 18e2d093
  lintangsutawika authored Aug 13, 2023
  
  18e2d093
12 Aug, 2023 2 commits
- fix multi-metric greedy_until · df6c5dcb
  haileyschoelkopf authored Aug 12, 2023
  
  df6c5dcb
- make chrf and ter aggregations · 116c540a
  haileyschoelkopf authored Aug 12, 2023
  
  116c540a
11 Aug, 2023 5 commits
- switch to process_docs · 8189eb18
  haileyschoelkopf authored Aug 11, 2023
  
  8189eb18
- support bleu score as a metric · 8806eff5
  haileyschoelkopf authored Aug 11, 2023
  
  8806eff5
- Merge pull request #768 from EleutherAI/lintangsutawika-patch-2 · dbf2c083
  Lintang Sutawika authored Aug 11, 2023
  
  dbf2c083
- Merge pull request #770 from tju01/big-refactor · c2f76ec7
  Hailey Schoelkopf authored Aug 11, 2023
```
[Refactor] Use evaluation mode for accelerate to prevent OOM
```
  c2f76ec7
- placate pre-commit · d69a962d
  Hailey Schoelkopf authored Aug 10, 2023
  
  d69a962d
10 Aug, 2023 6 commits
- Use evaluation_mode=True for accelerate to prevent OOM · c7ef3da7
  Til Jasper Ullrich authored Aug 10, 2023
  
  c7ef3da7
- Patch Blimp · 506a32ac
  Lintang Sutawika authored Aug 10, 2023
  
  506a32ac
- Update utils.py · bede79c2
  Hailey Schoelkopf authored Aug 10, 2023
  
  bede79c2
- Update crows_pairs_english.yaml · 83ed584c
  Hailey Schoelkopf authored Aug 10, 2023
  
  83ed584c
- Merge pull request #763 from EleutherAI/blimp · 7634a6ec
  Lintang Sutawika authored Aug 10, 2023
```
[Refactor] Add Blimp
```
  7634a6ec
- fix higher_is_better · 367f9693
  haileyschoelkopf authored Aug 10, 2023
  
  367f9693