Commits · 906ef948dc8dbb4c84e1bb0f2861b1aba30ab533 · gaoqiong / lm-evaluation-harness

17 Oct, 2023 1 commit
- Merge pull request #920 from luiscosio/patch-1 · 906ef948
  Hailey Schoelkopf authored Oct 17, 2023
```
Update Dockerfile
```
  906ef948
16 Oct, 2023 1 commit
- Update Dockerfile · f5f4856b
  Luis Cosio authored Oct 15, 2023
  
  f5f4856b
05 Oct, 2023 1 commit
- Merge pull request #902 from carmocca/patch-1 · e81d3cce
  Hailey Schoelkopf authored Oct 04, 2023
```
Exclude the scripts directory from the installation
```
  e81d3cce
04 Oct, 2023 2 commits
- Update setup.py · a3329027
  Carlos Mocholí authored Oct 05, 2023
  
  a3329027
- Exclude the scripts directory from the installation · 065d9074
  Carlos Mocholí authored Oct 05, 2023
```
Fixes https://github.com/Lightning-AI/lit-gpt/issues/558
Fixes https://github.com/EleutherAI/lm-evaluation-harness/issues/901
```
  065d9074
03 Oct, 2023 1 commit
- Merge pull request #889 from tmabraham/master · f2e3950b
  Hailey Schoelkopf authored Oct 03, 2023
```
Fix PubMedQA on master
```
  f2e3950b
02 Oct, 2023 3 commits
- Merge pull request #893 from EleutherAI/revert-735-licenses · e1a8b2fa
  Stella Biderman authored Oct 02, 2023
```
Revert "Dataset licenses"
```
  e1a8b2fa
- Revert "Dataset licenses" · 7295ed08
  Stella Biderman authored Oct 02, 2023
  
  7295ed08
- Merge pull request #735 from glerzing/licenses · f2de8609
  Stella Biderman authored Oct 01, 2023
```
Dataset licenses
```
  f2de8609
29 Sep, 2023 1 commit
- Update pubmedqa.py · 27ddebd6
  Tanishq Abraham authored Sep 28, 2023
  
  27ddebd6
21 Sep, 2023 4 commits
- Merge pull request #876 from jonabur/output_bugfix · 2c18e367
  Lintang Sutawika authored Sep 21, 2023
```
fix bug with output path in CWD
```
  2c18e367
- Add pre-commit fixes. · 93cbffa5
  jonabur authored Sep 21, 2023
  
  93cbffa5
- fix whitespace · db9b3338
  jonabur authored Sep 21, 2023
  
  db9b3338
- fix bug with output path in CWD · 95b62361
  jonabur authored Sep 21, 2023
  
  95b62361
11 Sep, 2023 1 commit
- Merge pull request #843 from baberabb/master_bugfix · 00209e10
  Hailey Schoelkopf authored Sep 11, 2023
```
[bugfix] openai
```
  00209e10
10 Sep, 2023 2 commits
- bugfix:retrieve until from until dict · 51f4994d
  baberabb authored Sep 10, 2023
  
  51f4994d
- bugfix:retrieve until from until dict · 701bd09f
  baberabb authored Sep 10, 2023
  
  701bd09f
23 Aug, 2023 2 commits
- Merge pull request #795 from codehound42/master · 3326c547
  Hailey Schoelkopf authored Aug 23, 2023
```
Add `inject_fused_attention` parameter for `auto-gptq`
```
  3326c547
- Update huggingface.py · 4c062d44
  Hailey Schoelkopf authored Aug 23, 2023
  
  4c062d44
21 Aug, 2023 1 commit
- Merge pull request #789 from qmdnls/nq-open · 341f04c4
  Hailey Schoelkopf authored Aug 21, 2023
```
Add NQ-Open task based on the Natural Questions dataset
```
  341f04c4
18 Aug, 2023 2 commits
- Match prompt with existing works; fix article regex and whitespace · b9b1ac06
  Björn Bebensee authored Aug 18, 2023
  
  b9b1ac06
- Merge pull request #772 from haonan-li/master · aaef2c46
  Hailey Schoelkopf authored Aug 17, 2023
```
Add CMMLU Benchmark
```
  aaef2c46
17 Aug, 2023 4 commits
- Add `inject_fused_attention` parameter for `auto-gptq` · 436edcc9
  codehound42 authored Aug 17, 2023
  
  436edcc9
- Changed whitespace fix to match existing evaluation code · 71d3655b
  Björn Bebensee authored Aug 17, 2023
  
  71d3655b
- Updated NQOpen documentation · 9e5acd1d
  Björn Bebensee authored Aug 17, 2023
  
  9e5acd1d
- Added NQ-Open task · a28c03e6
  Björn Bebensee authored Aug 17, 2023
  
  a28c03e6
13 Aug, 2023 1 commit
- Merge pull request #765 from EleutherAI/haileyschoelkopf-patch-1 · fd1c7196
  Hailey Schoelkopf authored Aug 13, 2023
```
Update crowspairs.py higher_is_better
```
  fd1c7196
11 Aug, 2023 1 commit
- Add CMMLU Benchmark · 24e5972c
  Haonan Li authored Aug 11, 2023
  
  24e5972c
10 Aug, 2023 1 commit
- Update crowspairs.py · 9ab8538f
  Hailey Schoelkopf authored Aug 10, 2023
  
  9ab8538f
08 Aug, 2023 2 commits
- Merge pull request #746 from EleutherAI/StellaAthena-patch-2 · b952a206
  Lintang Sutawika authored Aug 08, 2023
```
Update triviaqa.py
```
  b952a206
- Update triviaqa.py · f08f7c79
  Stella Biderman authored Aug 08, 2023
```
Removes the `_remove_prefixes` optimization as it's invalid for exact match metrics. Closes https://github.com/EleutherAI/lm-evaluation-harness/issues/726
```
  f08f7c79
07 Aug, 2023 1 commit

Alain authored Aug 07, 2023

- Add in comments the sources of the licenses.
- Add "No license found" when it is the case.

8c1fea15

06 Aug, 2023 3 commits
- Merge pull request #738 from baberabb/master_anthropic · 5a49b2a3
  Hailey Schoelkopf authored Aug 06, 2023
```
[Main] updated to new anthropic API
```
  5a49b2a3
- updated syntax to new anthropic API · d504944b
  baberabb authored Aug 06, 2023
  
  d504944b
- Dataset licenses · 1241d1c7
  Alain authored Aug 06, 2023
```
Added the license for 7 datasets.
For 4 datasets I can't find the license and removed the TODO.
```
  1241d1c7
04 Aug, 2023 1 commit
- Merge pull request #722 from jiqing-feng/4bit_double_quant · fe803c29
  Hailey Schoelkopf authored Aug 03, 2023
```
add bnb_4bit_use_double_quant and low_cpu_mem_usage
```
  fe803c29
02 Aug, 2023 1 commit
- add bnb_4bit_use_double_quant and low_cpu_mem_usage · f5808609
  Feng, Jiqing authored Aug 01, 2023
  
  f5808609
01 Aug, 2023 2 commits
- Merge pull request #718 from EleutherAI/remove-results-folder · 4fbbd60f
  Hailey Schoelkopf authored Aug 01, 2023
```
Remove Results folder temporarily
```
  4fbbd60f
- remove results folder · 1b9833dd
  haileyschoelkopf authored Aug 01, 2023
  
  1b9833dd
31 Jul, 2023 1 commit
- Merge pull request #706 from guijinSON/master · 5e59782e
  GUIJIN SON authored Jul 31, 2023
```
add haerae
```
  5e59782e