Commits · 05e97d8482fab38651469dd4763ea327aba64fe6 · gaoqiong / lm-evaluation-harness

02 May, 2023 3 commits
- modify layout of yaml vs vanilla tasks · 05e97d84
  haileyschoelkopf authored May 02, 2023
  
  05e97d84
- add configurable ppl task · 337419b8
  haileyschoelkopf authored May 02, 2023
  
  337419b8
- prelim. multiple choice support · 487f7811
  haileyschoelkopf authored May 02, 2023
  
  487f7811
01 May, 2023 1 commit
- change id_ to idx in instance · 9d8e0532
  haileyschoelkopf authored May 01, 2023
  
  9d8e0532
28 Apr, 2023 5 commits
- updated yaml · e7f49cca
  lintangsutawika authored Apr 28, 2023
  
  e7f49cca
- added opt8ion to use prompts, removed typos on metric kwargs · 3ec68d38
  lintangsutawika authored Apr 28, 2023
  
  3ec68d38
- can process looglikelihood requests · cea713dc
  lintangsutawika authored Apr 28, 2023
  
  cea713dc
- registries for metrics · 84191b83
  lintangsutawika authored Apr 28, 2023
  
  84191b83
- yaml for lambada and sglue · 7ad96ded
  lintangsutawika authored Apr 28, 2023
  
  7ad96ded
27 Apr, 2023 1 commit
- ported changes here · f107ae29
  lintangsutawika authored Apr 27, 2023
  
  f107ae29
24 Apr, 2023 3 commits
- add metric + agg registries · 2a9da9fb
  haileyschoelkopf authored Apr 24, 2023
  
  2a9da9fb
- add template_vars + make jinja rendering stricter · 460584ca
  haileyschoelkopf authored Apr 23, 2023
  
  460584ca
- make tasks and models registered by decorators · f275301a
  haileyschoelkopf authored Apr 23, 2023
  
  f275301a
23 Apr, 2023 1 commit
- refactor Instance dataclass · e7c18e53
  haileyschoelkopf authored Apr 23, 2023
  
  e7c18e53
19 Apr, 2023 1 commit
- in-place replace main with lm-eval2, keeping old git history · d2a9b759
  haileyschoelkopf authored Apr 19, 2023
  
  d2a9b759
13 Apr, 2023 1 commit
- feat: peft model evals · 2b9a6e94
  Zach Nussbaum authored Apr 13, 2023
  
  2b9a6e94
10 Mar, 2023 1 commit
- Hosting wikitext on HuggingFace (#396) · 83dbfbf6
  ben authored Mar 10, 2023
```
* host wikitext on hf

* transfer wikitext to Eleuther HF org
```
  83dbfbf6
09 Mar, 2023 1 commit
- transfer data to eleuther HF org · 7502ff3d
  Benjamin Fattori authored Mar 09, 2023
  
  7502ff3d
08 Mar, 2023 4 commits
- inspect import not needed · db501387
  Benjamin Fattori authored Mar 08, 2023
  
  db501387
- remove unused import for eval-harness · 1581db0b
  Benjamin Fattori authored Mar 08, 2023
  
  1581db0b
- arithmetic task class now uses HF remote builder · 3c8479ba
  Benjamin Fattori authored Mar 08, 2023
  
  3c8479ba
- remove old dataset builder script under datasets for arithmetic · f022d303
  Benjamin Fattori authored Mar 08, 2023
  
  f022d303
07 Mar, 2023 1 commit
- enable "low_cpu_mem_usage" · 859f96fd
  Xingjian Shi authored Mar 07, 2023
  
  859f96fd
04 Feb, 2023 1 commit
- hotfix(gpt2): Remove vocab-size logits slice · 1a545447
  jon-tow authored Feb 04, 2023
  
  1a545447
18 Jan, 2023 1 commit
- Add accuracy metric to crows-pairs (#380) · f9eca2c8
  Hailey Schoelkopf authored Jan 18, 2023
```
* add accuracy metric to crows-pairs
```
  f9eca2c8
17 Jan, 2023 2 commits

Aflah authored Jan 18, 2023

* Added CrowSPairs

* Fixed Bug

* Changed to absolute Value

* Docstring Changes

* Update crowspairs.py

Finegrained Tasks

* Update __init__.py

* Finegrained Tasks

* Add `crows_pairs` version metadata

ea3df930

first step: add HF+accelerate implementations · db1f55ff
haileyschoelkopf authored Jan 17, 2023

db1f55ff

11 Jan, 2023 1 commit

Added ToxiGen task (#377) · f079e322

Tom Hartvigsen authored Jan 10, 2023

* added toxigen task

* removed previous doc_to_text

* Add train split for few-shot samples

* Add task version metadata

f079e322

29 Dec, 2022 1 commit
- Update Pile Test/Val Download URLs (#373) · 8c048e26
  ben authored Dec 29, 2022
```
* use pile links hosted on the eye

* update links in dataset_infos
```
  8c048e26
16 Dec, 2022 1 commit
- Update `lambada_openai` multilingual data source · 2bae14d2
  jon-tow authored Dec 16, 2022
  
  2bae14d2
13 Dec, 2022 1 commit
- Bump `triviaqa` version · f7aeeff2
  jon-tow authored Dec 12, 2022
  
  f7aeeff2
12 Dec, 2022 2 commits
- Run pre-commit · 3aff8606
  jon-tow authored Dec 12, 2022
  
  3aff8606
- Replace stale `triviaqa` dataset link · 299e9505
  jon-tow authored Dec 12, 2022
  
  299e9505
02 Dec, 2022 2 commits
- Remove redefinition of `jieba` · 1e5d55d9
  jon-tow authored Dec 02, 2022
  
  1e5d55d9
- Cleanup `README.md` and package deps · 0699d774
  jon-tow authored Dec 02, 2022
  
  0699d774
25 Nov, 2022 4 commits
- Run pre-commit · 1386065d
  jon-tow authored Nov 25, 2022
  
  1386065d
- Revert "Remove old `LAMBADA` dataset" · 0bca4303
  jon-tow authored Nov 25, 2022
```
This reverts commit 63bdb4b0.
```
  0bca4303
- Remove old `LAMBADA` dataset · 63bdb4b0
  jon-tow authored Nov 25, 2022
  
  63bdb4b0
- Add the original `LAMBADA` dataset · bfa57687
  jon-tow authored Nov 24, 2022
  
  bfa57687
04 Nov, 2022 1 commit
- Add `TextSynth` API (#299) · 4f0410a4
  Jonathan Tow authored Nov 03, 2022
```
* Add `TextSynth` API
```
  4f0410a4