Commits · c478e20a9caab0ab43a40af1a7e975c3ce29d46d · gaoqiong / lm-evaluation-harness

"vscode:/vscode.git/clone" did not exist on "e4ceb227b872dc28b3ff22567a45c4c04a4c953e"

14 Aug, 2023 22 commits
- Merge branch 'big-refactor' of https://github.com/EleutherAI/lm-evaluation-harness into toxicity · c478e20a
  lintangsutawika authored Aug 14, 2023
  
  c478e20a
- reformat · 88511112
  lintangsutawika authored Aug 03, 2023
  
  88511112
- pull and modified input args to metric · 33273d6e
  lintangsutawika authored Aug 03, 2023
  
  33273d6e
- Removed Implementation from metrics.py · 60763408
  Aflah authored Aug 03, 2023
  
  60763408
- Delete realtoxicityprompts_yaml · bdc23326
  Aflah authored Aug 03, 2023
  
  bdc23326
- Addressed Review Comments · ce7eb7b6
  Aflah authored Aug 03, 2023
  
  ce7eb7b6
- Fixed Wrong File Push · 67bbc5b4
  Aflah authored Aug 03, 2023
  
  67bbc5b4
- Improved DocString · 57bcb4d6
  Aflah authored Aug 02, 2023
  
  57bcb4d6
- CMD Line Works · b221531a
  Aflah authored Aug 02, 2023
  
  b221531a
- Added Metric and Seems to Run but some Errors · d857e539
  Aflah authored Aug 02, 2023
  
  d857e539
- Base Template · 94ccc429
  Aflah authored Aug 02, 2023
  
  94ccc429
- Merge pull request #747 from EleutherAI/haileyschoelkopf-patch-2 · 64c76fc3
  Lintang Sutawika authored Aug 14, 2023
```
[WIP] Update API docs in README
```
  64c76fc3
- Added info on `lm-eval` command. · 725d7e3e
  Lintang Sutawika authored Aug 14, 2023
  
  725d7e3e
- Merge pull request #777 from EleutherAI/benchmark · be79a281
  Lintang Sutawika authored Aug 14, 2023
```
[Refactor] Update Benchmark
```
  be79a281
- Merge pull request #776 from EleutherAI/xnli · f918c8fd
  Lintang Sutawika authored Aug 14, 2023
```
[Refactor] XNLI
```
  f918c8fd
- Update with task list. · 1768f118
  Lintang Sutawika authored Aug 14, 2023
  
  1768f118
- adjustments to have `task` to modify a task name in benchmark · 3de2c0e7
  lintangsutawika authored Aug 14, 2023
  
  3de2c0e7
- explain how to use benchmarks · cccb294f
  lintangsutawika authored Aug 14, 2023
  
  cccb294f
- return fewshot_delimiter if we get "" · 7758d16a
  lintangsutawika authored Aug 14, 2023
  
  7758d16a
- add warning log if prompt returns empty string · 1701c4e2
  lintangsutawika authored Aug 14, 2023
  
  1701c4e2
- add warning log if prompt returns empty string · 722b8974
  lintangsutawika authored Aug 14, 2023
  
  722b8974
- add warning log if prompt returns empty string · bc951ac1
  lintangsutawika authored Aug 14, 2023
  
  bc951ac1
13 Aug, 2023 12 commits
- formate · ce8c2b14
  lintangsutawika authored Aug 13, 2023
  
  ce8c2b14
- update and fixes · 5ddabc29
  lintangsutawika authored Aug 13, 2023
  
  5ddabc29
- reformat · c523063d
  lintangsutawika authored Aug 13, 2023
  
  c523063d
- xnli reformat · 2d54f2c9
  lintangsutawika authored Aug 13, 2023
  
  2d54f2c9
- xnli reformat · ac98d9e1
  lintangsutawika authored Aug 13, 2023
  
  ac98d9e1
- Merge pull request #766 from EleutherAI/patch-crowspairs · 054ed37f
  Hailey Schoelkopf authored Aug 13, 2023
```
[Refactor] Patch crowspairs higher_is_better
```
  054ed37f
- Merge pull request #774 from EleutherAI/speedup-hellaswag · 4f27f0b9
  Hailey Schoelkopf authored Aug 13, 2023
```
[Refactor] Speedup hellaswag context building
```
  4f27f0b9
- update in task guide · db0c46b0
  lintangsutawika authored Aug 13, 2023
  
  db0c46b0
- uncommented tasks · 22ece14a
  lintangsutawika authored Aug 13, 2023
  
  22ece14a
- moved to a new dir · f4813cfc
  lintangsutawika authored Aug 13, 2023
  
  f4813cfc
- update · 83683ddd
  lintangsutawika authored Aug 13, 2023
  
  83683ddd
- added xnli · 18e2d093
  lintangsutawika authored Aug 13, 2023
  
  18e2d093
11 Aug, 2023 4 commits
- switch to process_docs · 8189eb18
  haileyschoelkopf authored Aug 11, 2023
  
  8189eb18
- Merge pull request #768 from EleutherAI/lintangsutawika-patch-2 · dbf2c083
  Lintang Sutawika authored Aug 11, 2023
  
  dbf2c083
- Merge pull request #770 from tju01/big-refactor · c2f76ec7
  Hailey Schoelkopf authored Aug 11, 2023
```
[Refactor] Use evaluation mode for accelerate to prevent OOM
```
  c2f76ec7
- placate pre-commit · d69a962d
  Hailey Schoelkopf authored Aug 10, 2023
  
  d69a962d
10 Aug, 2023 2 commits
- Use evaluation_mode=True for accelerate to prevent OOM · c7ef3da7
  Til Jasper Ullrich authored Aug 10, 2023
  
  c7ef3da7
- Patch Blimp · 506a32ac
  Lintang Sutawika authored Aug 10, 2023
  
  506a32ac