Commits · 693c19e25d6f5fc88daff6d51541f8aa8fbfc266 · gaoqiong / lm-evaluation-harness

27 Jan, 2021 2 commits
- Merge pull request #87 from VitamintK/master · 693c19e2
  Leo Gao authored Jan 27, 2021
```
first attempt at adding arithmetic evaluations
```
  693c19e2
- use openai's gpt3 datafiles instead of generating equations · baadb2da
  Kevin Wang authored Jan 27, 2021
  
  baadb2da
24 Jan, 2021 2 commits
- Merge pull request #91 from zphang/superglue · 38892f0a
  Leo Gao authored Jan 24, 2021
```
SuperGLUE (excluding WSC)
```
  38892f0a
- superglue ex wsc · 2e707b87
  Jason Phang authored Jan 23, 2021
  
  2e707b87
23 Jan, 2021 2 commits
- prepending a space, because it significantly increases log-likelihood of the correct answer. · 183f4721
  Kevin Wang authored Jan 14, 2021
  
  183f4721
- first attempt at adding arithmetic evaluations · a58e5e4b
  Kevin Wang authored Jan 14, 2021
  
  a58e5e4b
22 Jan, 2021 5 commits
- Merge pull request #90 from EleutherAI/no_footguns · 8ae88962
  Stella Biderman authored Jan 21, 2021
```
Get rid of some footguns
```
  8ae88962
- Remove rewrite reminder comment from everything except SuperGLUE · f4120e59
  Leo Gao authored Jan 21, 2021
  
  f4120e59
- Clean up code, remove some footguns · e31b4b31
  Leo Gao authored Jan 21, 2021
  
  e31b4b31
- Implement isgreedy · e723d3d5
  Leo Gao authored Jan 21, 2021
  
  e723d3d5
- Merge pull request #88 from jon-tow/glue-evaluation-refactor · 27a859e2
  Leo Gao authored Jan 21, 2021
```
Adopt new framework for `glue`
```
  27a859e2
21 Jan, 2021 1 commit
- Adopt new framework for `glue` · 36467c0e
  Jonathan Tow authored Jan 21, 2021
  
  36467c0e
16 Jan, 2021 6 commits
- Merge pull request #85 from EleutherAI/sglue-eval-fazz · 4c8d22db
  Leo Gao authored Jan 16, 2021
```
SuperGLUE commitmentbank implementation (new framework)
```
  4c8d22db
- Replace tab with spaces, formatting · 173b33a4
  thefazzer authored Jan 16, 2021
  
  173b33a4
- Minor formatting correction · 2c05f0e4
  thefazzer authored Jan 16, 2021
  
  2c05f0e4
- Refactor iteration, indentation fix · b19dff50
  thefazzer authored Jan 16, 2021
  
  b19dff50
- Remove as functions are inline · 432226b0
  thefazzer authored Jan 16, 2021
  
  432226b0
- Merge branch 'master' into sglue-eval-fazz · 64cd64bc
  thefazzer authored Jan 16, 2021
  
  64cd64bc
13 Jan, 2021 2 commits
- Merge pull request #86 from jon-tow/doc-to-text-refactor · 0e4139b8
  Leo Gao authored Jan 13, 2021
```
Move `doc_to_text` target code into `doc_to_target`
```
  0e4139b8
- Move `doc_to_text` target code into `doc_to_target` · d77241eb
  Jonathan Tow authored Jan 13, 2021
  
  d77241eb
12 Jan, 2021 2 commits
- Add MultiRC Implementation · a538a1ad
  thefazzer authored Jan 12, 2021
  
  a538a1ad
- Add COPA implementation · 2c6ff0d5
  thefazzer authored Jan 12, 2021
  
  2c6ff0d5
10 Jan, 2021 2 commits
- Added F1_score metric · 5165bd38
  thefazzer authored Jan 10, 2021
  
  5165bd38
- CommitmentBank framework refactor · 68a8790c
  thefazzer authored Jan 10, 2021
  
  68a8790c
09 Jan, 2021 5 commits
- Merge pull request #82 from EleutherAI/bmk_refactor2 · e5d0229f
  Stella Biderman authored Jan 08, 2021
```
Refactor and (re)implement SAT evaluation
```
  e5d0229f
- Merge branch 'bmk_refactor2' of github.com:EleutherAI/lm_evaluation_harness into bmk_refactor2 · 0b3b7251
  Leo Gao authored Jan 08, 2021
  
  0b3b7251
- Move higher_is_better and aggregation into their own functions · a18104a4
  Leo Gao authored Jan 08, 2021
  
  a18104a4
- Merge branch 'master' into bmk_refactor2 · 5ce42fc0
  Leo Gao authored Jan 08, 2021
  
  5ce42fc0
- Refactor and implement SAT evaluation · 0f9c1624
  Leo Gao authored Jan 08, 2021
  
  0f9c1624
08 Jan, 2021 1 commit
- Merge pull request #80 from nicholaskross/master · 93510e3a
  Leo Gao authored Jan 07, 2021
```
Started SAT eval
```
  93510e3a
07 Jan, 2021 1 commit
- Merge branch 'master' into master · 515e0470
  Nicholas Kross authored Jan 07, 2021
  
  515e0470
06 Jan, 2021 4 commits
- rewrote based on BoolQ implementation · 215de045
  nicholaskross authored Jan 06, 2021
  
  215de045
- Merge pull request #81 from EleutherAI/more-comments · afc614fe
  Stella Biderman authored Jan 06, 2021
```
Add comments to remind that evaluation needs to be written for the ne…
```
  afc614fe
- Add comments to remind that evaluation needs to be written for the new framework · 2d2dbf96
  Leo Gao authored Jan 05, 2021
  
  2d2dbf96
- got a start on the SAT eval · 57c751fa
  nicholaskross authored Jan 05, 2021
  
  57c751fa
05 Jan, 2021 5 commits
- Merge pull request #79 from EleutherAI/bmk_refactor · 6803e647
  Leo Gao authored Jan 05, 2021
```
Bmk refactor
```
  6803e647
- Merge branch 'bmk_refactor' of github.com:EleutherAI/lm_evaluation_harness into bmk_refactor · 041ea8a7
  Leo Gao authored Jan 05, 2021
  
  041ea8a7
- Merge remote-tracking branch 'origin/master' into bmk_refactor · 18281758
  Leo Gao authored Jan 05, 2021
```
# Conflicts:
#	batch_eval/main.py
```
  18281758
- Update README.md · 75bea6d5
  Stella Biderman authored Jan 05, 2021
  
  75bea6d5
- Update README.md · 611b1d55
  Stella Biderman authored Jan 05, 2021
  
  611b1d55