Commits · 3c37ea9caf8ef8982116c4a8abb2a4e4b92eb854 · gaoqiong / lm-evaluation-harness

22 Feb, 2022 2 commits
- Merge pull request #264 from StephenHogg/qasper · 3c37ea9c
  Leo Gao authored Feb 21, 2022
```
Add QASPER task
```
  3c37ea9c
- Don't create requests for TODO metrics · 815f165c
  Stephen Hogg authored Feb 22, 2022
  
  815f165c
21 Feb, 2022 2 commits
- Mark unanswerable as TODO · f7aaff08
  Stephen Hogg authored Feb 22, 2022
  
  f7aaff08
- Merge pull request #289 from jon-tow/gsm8k-testdata · 89905174
  Leo Gao authored Feb 20, 2022
```
Add `gsm8k` test data
```
  89905174
17 Feb, 2022 1 commit
- Add `gsm8k` test data · 54065e6a
  Jonathan Tow authored Feb 17, 2022
  
  54065e6a
13 Feb, 2022 8 commits
- Align casing between doc creation and evaluation · be55ea8c
  Stephen Hogg authored Feb 13, 2022
  
  be55ea8c
- Remove extractive spans score; mark as TODO · 8f0e1765
  Stephen Hogg authored Feb 13, 2022
  
  8f0e1765
- Merge pull request #260 from jon-tow/gsm8k · cef6aa8d
  Leo Gao authored Feb 12, 2022
```
Add Grade School Math 8k (`GSM8K`)
```
  cef6aa8d
- gpt3: print exceptions · f6fb7f6a
  Leo Gao authored Feb 12, 2022
  
  f6fb7f6a
- Update README.md · edc9449f
  Leo Gao authored Feb 12, 2022
  
  edc9449f
- Merge pull request #288 from EleutherAI/leogao2-patch-1 · 2cc968ba
  Leo Gao authored Feb 12, 2022
```
Update README.md
```
  2cc968ba
- Update README.md · 71647775
  Leo Gao authored Feb 12, 2022
  
  71647775
- Merge pull request #263 from jon-tow/hendrycks_math_checksum_update · c5131e8a
  Leo Gao authored Feb 12, 2022
```
Bump Hendrycks `MATH` because of updated dataset
```
  c5131e8a
12 Feb, 2022 2 commits
- Merge pull request #286 from StephenHogg/255 · c74ca57c
  Leo Gao authored Feb 12, 2022
```
Fix BUG: TypeError: create_from_arg_string() takes 2 positional arguments but 3 were given
```
  c74ca57c
- Update dummy model classmethod args · d895bfe6
  Stephen Hogg authored Feb 12, 2022
  
  d895bfe6
11 Feb, 2022 4 commits
- Remove incorrect comments · af9766d1
  Stephen Hogg authored Feb 12, 2022
  
  af9766d1
- Name changes as per leogao2 · 894c0697
  Stephen Hogg authored Feb 12, 2022
  
  894c0697
- Disable extractive spans · 96f3e5b3
  Stephen Hogg authored Feb 12, 2022
  
  96f3e5b3
- Include extraction in process_results; fixes per test results · c6a35696
  Stephen Hogg authored Feb 11, 2022
  
  c6a35696
09 Feb, 2022 2 commits
- exponentiate unanswerable score; rename dict key · c2f12474
  Stephen Hogg authored Feb 09, 2022
  
  c2f12474
- Add citation; update process_results and construct_requests · 7ed5e29e
  Stephen Hogg authored Feb 09, 2022
  
  7ed5e29e
08 Feb, 2022 3 commits
- Add initial draft of QASPER; register with package; yet to complete process_results · 79492627
  Stephen Hogg authored Feb 08, 2022
  
  79492627
- pin best-download to 0.0.9 · e18ef6e1
  Leo Gao authored Feb 07, 2022
  
  e18ef6e1
- Revert "Update requirements.txt" · 7d37dd4a
  Leo Gao authored Feb 07, 2022
```
This reverts commit 05590e11.
```
  7d37dd4a
06 Feb, 2022 2 commits
- Bump Hendrycks `MATH` because of updated dataset · 49dc29bd
  Jonathan Tow authored Feb 06, 2022
  
  49dc29bd
- Update requirements.txt · 05590e11
  Stella Biderman authored Feb 06, 2022
  
  05590e11
31 Jan, 2022 1 commit
- Merge branch 'master' of https://github.com/EleutherAI/lm-evaluation-harness into gsm8k · c75b3413
  Jonathan Tow authored Jan 31, 2022
  
  c75b3413
30 Jan, 2022 1 commit
- Merge pull request #259 from jon-tow/headqa-ux · 26f0233f
  Leo Gao authored Jan 30, 2022
```
Update `headqa` deprecation warning to display on init only
```
  26f0233f
23 Jan, 2022 2 commits
- Update `headqa` deprecation warning to display on init only · 03c15e06
  Jonathan Tow authored Jan 23, 2022
  
  03c15e06
- Add grade school math `GSM8K` · f04a91c1
  Jonathan Tow authored Jan 23, 2022
  
  f04a91c1
15 Jan, 2022 3 commits
- cb: add testdata · 1c52e917
  Leo Gao authored Jan 14, 2022
  
  1c52e917
- Merge pull request #254 from bigscience-workshop/thomas/fix_cb · c3d941b9
  Leo Gao authored Jan 14, 2022
```
Fix CB
```
  c3d941b9
- Merge pull request #256 from jon-tow/update-asdiv · 4849702e
  Leo Gao authored Jan 14, 2022
```
Conform `asidv` to the new description api
```
  4849702e
13 Jan, 2022 1 commit
- Conform `asidv` to the new description api · caff4f10
  Jonathan Tow authored Jan 13, 2022
  
  caff4f10
11 Jan, 2022 3 commits
- Merge pull request #252 from bigscience-workshop/thomas/fix_best_download_version_1 · be40299e
  Leo Gao authored Jan 11, 2022
```
Best download fix
```
  be40299e
- Bump version · b421f052
  thomasw21 authored Jan 11, 2022
  
  b421f052
- Fix CB · ea3fd79b
  thomasw21 authored Jan 11, 2022
  
  ea3fd79b
08 Jan, 2022 3 commits
- Missed asdiv · 2d9fc256
  thomasw21 authored Jan 09, 2022
  
  2d9fc256
- Merge pull request #250 from bigscience-workshop/thomas/fix_best_download_version · ed6931e7
  Leo Gao authored Jan 08, 2022
```
Best-download have backward compatibility issue
```
  ed6931e7
- Merge branch 'master' into thomas/fix_best_download_version · 78824d7f
  Thomas Wang authored Jan 08, 2022
  
  78824d7f