Commits · 20a54b3af209ea191051cdb8a8abb3b6b1996048 · gaoqiong / lm-evaluation-harness

05 Oct, 2023 4 commits
- removed print messages, added cot extraction strings · 20a54b3a
  lintangsutawika authored Oct 05, 2023
  
  20a54b3a
- Merge branch 'big-refactor' of... · 337492ad
  lintangsutawika authored Oct 05, 2023
```
Merge branch 'big-refactor' of https://github.com/EleutherAI/lm-evaluation-harness into flan-benchmark
```
  337492ad
- Merge pull request #883 from chrisociepa/transformation-filters · 4824a832
  Lintang Sutawika authored Oct 05, 2023
```
Add transformation filters
```
  4824a832
- Merge pull request #900 from EleutherAI/fix-wildcards · b8faaa93
  Hailey Schoelkopf authored Oct 05, 2023
```
[Refactor] Fix wildcards
```
  b8faaa93
04 Oct, 2023 12 commits
- fix wildcards · 09d935ee
  haileyschoelkopf authored Oct 04, 2023
  
  09d935ee
- Merge pull request #852 from EleutherAI/bigbench · 7f2a9cbd
  Hailey Schoelkopf authored Oct 04, 2023
```
[Refactor] bigbench
```
  7f2a9cbd
- change back to original dataset path · 51a43a76
  haileyschoelkopf authored Oct 04, 2023
  
  51a43a76
- change back to original dataset path · 54498fcc
  haileyschoelkopf authored Oct 04, 2023
  
  54498fcc
- Merge pull request #899 from EleutherAI/fix-scripts · c4fa6fb5
  Hailey Schoelkopf authored Oct 04, 2023
```
[Refactor] Fix error when calling `lm-eval`
```
  c4fa6fb5
- also, reenable CPU tests · 436b2697
  haileyschoelkopf authored Oct 04, 2023
  
  436b2697
- remove references to main.py · 9b06de54
  haileyschoelkopf authored Oct 04, 2023
  
  9b06de54
- remove references to main.py · 0856828f
  haileyschoelkopf authored Oct 04, 2023
  
  0856828f
- update commandline script · 2e13caa6
  haileyschoelkopf authored Oct 04, 2023
  
  2e13caa6
- move to __main__.py · 3b5e554f
  haileyschoelkopf authored Oct 04, 2023
  
  3b5e554f
- moved main to lm_eval/ to fix import of main error · 6f92c20d
  lintangsutawika authored Oct 04, 2023
  
  6f92c20d
- modify default yamls · 7e6e3c20
  haileyschoelkopf authored Oct 04, 2023
  
  7e6e3c20
03 Oct, 2023 10 commits
- Merge branch 'big-refactor' into bigbench · 2041dc34
  haileyschoelkopf authored Oct 03, 2023
  
  2041dc34
- add bigbench push_to_hub script · 67c0f73a
  haileyschoelkopf authored Oct 03, 2023
  
  67c0f73a
- add bigbench push_to_hub script · fa608798
  haileyschoelkopf authored Oct 03, 2023
  
  fa608798
- push updated bigbench task yamls · 8806944e
  haileyschoelkopf authored Oct 03, 2023
  
  8806944e
- update generate_tasks.py and template yamls · 1a77b4d5
  haileyschoelkopf authored Oct 03, 2023
  
  1a77b4d5
- Merge pull request #890 from tmabraham/big-refactor · 15f4a3ef
  Hailey Schoelkopf authored Oct 03, 2023
```
[Refactor] Fix PubMedQA
```
  15f4a3ef
- Bring task name to `pubmedqa` in line with master branch · 3a7f52d0
  Hailey Schoelkopf authored Oct 03, 2023
  
  3a7f52d0
- Merge pull request #895 from chrisociepa/update-python-version-in-precommit-config · efa38caa
  Hailey Schoelkopf authored Oct 03, 2023
```
Change Python Version to 3.8 in .pre-commit-config.yaml and GitHub Actions
```
  efa38caa
- Change Python Version to 3.8 in GitHub Actions · bf275d00
  Chris authored Oct 03, 2023
  
  bf275d00
- Change python version to 3.8 in .pre-commit-config.yaml · 3f2dd9fb
  Chris authored Oct 03, 2023
  
  3f2dd9fb
02 Oct, 2023 2 commits
- Fix formatting · 06ce7a62
  Chris authored Oct 02, 2023
  
  06ce7a62
- Add PolEmo2 tasks · e634f83f
  Chris authored Oct 02, 2023
  
  e634f83f
29 Sep, 2023 2 commits
- Update preprocess_pubmedqa.py · 6658f510
  Tanishq Abraham authored Sep 29, 2023
  
  6658f510
- Update pubmedqa.yaml · 5c418e2e
  Tanishq Abraham authored Sep 29, 2023
  
  5c418e2e
26 Sep, 2023 5 commits
- Update _default_template_yaml · 3d2ee4d4
  Hailey Schoelkopf authored Sep 26, 2023
  
  3d2ee4d4
- remove comments · 6ae9e9e8
  lintangsutawika authored Sep 26, 2023
  
  6ae9e9e8
- update loading prompts · b2d16321
  lintangsutawika authored Sep 26, 2023
  
  b2d16321
- modify to conform to pre-commit · 30711873
  lintangsutawika authored Sep 26, 2023
  
  30711873
- moved files · 3f090027
  lintangsutawika authored Sep 26, 2023
  
  3f090027
25 Sep, 2023 3 commits
- merged latest · a5e93901
  lintangsutawika authored Sep 25, 2023
  
  a5e93901
- remove templerature as do_sample is False · 694af7d6
  lintangsutawika authored Sep 25, 2023
  
  694af7d6
- add function to add both task and benchmark · 0dc556dc
  lintangsutawika authored Sep 25, 2023
  
  0dc556dc
22 Sep, 2023 2 commits
- Fix formatting · 8310e67e
  Chris authored Sep 22, 2023
  
  8310e67e
- Fix formatting · ad4ab52a
  Chris authored Sep 22, 2023
  
  ad4ab52a