Commits · 7aee2dfff3ecc61e5437b5c2b18756fa227f9c19 · gaoqiong / lm-evaluation-harness

15 May, 2023 6 commits
- moved functions out and some fixes · 7aee2dff
  lintangsutawika authored May 15, 2023
  
  7aee2dff
- adjustments to ALL_TASKS · 51b795cf
  lintangsutawika authored May 15, 2023
  
  51b795cf
- delete files · 64d07df7
  lintangsutawika authored May 15, 2023
  
  64d07df7
- arc yaml set as 1 group · 439c7c6c
  lintangsutawika authored May 15, 2023
  
  439c7c6c
- adjust super glue cb · a5f5d92b
  lintangsutawika authored May 15, 2023
  
  a5f5d92b
- can list other yamls to be included and loaded · 600918bb
  lintangsutawika authored May 15, 2023
  
  600918bb
13 May, 2023 2 commits
- python func of tasks and yamls are both in the tasks/ with yamls being in a... · a179c472
  lintangsutawika authored May 13, 2023
```
python func of tasks and yamls are both in the tasks/ with yamls being in a directory that denotes the task name. This is so each yaml file in that dir can have minor changes like prompts that are different
```
  a179c472
- modified how yaml and python functions are added to groups and task registry · ca0b8d45
  lintangsutawika authored May 13, 2023
  
  ca0b8d45
10 May, 2023 2 commits
- adjusted to adapt to modiefied registry process · 52ee80a6
  lintangsutawika authored May 10, 2023
  
  52ee80a6
- modified registry process · 4bc837d4
  lintangsutawika authored May 10, 2023
  
  4bc837d4
09 May, 2023 1 commit
- muliple prompts · 41d16c2b
  lintangsutawika authored May 09, 2023
  
  41d16c2b
08 May, 2023 2 commits
- add yaml registering decorator · 38244e15
  haileyschoelkopf authored May 08, 2023
  
  38244e15
- add mutual info, code cleanup · 95642aa6
  haileyschoelkopf authored May 08, 2023
  
  95642aa6
02 May, 2023 3 commits
- modify layout of yaml vs vanilla tasks · 05e97d84
  haileyschoelkopf authored May 02, 2023
  
  05e97d84
- add configurable ppl task · 337419b8
  haileyschoelkopf authored May 02, 2023
  
  337419b8
- prelim. multiple choice support · 487f7811
  haileyschoelkopf authored May 02, 2023
  
  487f7811
01 May, 2023 1 commit
- change id_ to idx in instance · 9d8e0532
  haileyschoelkopf authored May 01, 2023
  
  9d8e0532
28 Apr, 2023 2 commits
- updated yaml · e7f49cca
  lintangsutawika authored Apr 28, 2023
  
  e7f49cca
- yaml for lambada and sglue · 7ad96ded
  lintangsutawika authored Apr 28, 2023
  
  7ad96ded
24 Apr, 2023 1 commit
- make tasks and models registered by decorators · f275301a
  haileyschoelkopf authored Apr 23, 2023
  
  f275301a
23 Apr, 2023 1 commit
- refactor Instance dataclass · e7c18e53
  haileyschoelkopf authored Apr 23, 2023
  
  e7c18e53
19 Apr, 2023 1 commit
- in-place replace main with lm-eval2, keeping old git history · d2a9b759
  haileyschoelkopf authored Apr 19, 2023
  
  d2a9b759
10 Mar, 2023 1 commit
- Hosting wikitext on HuggingFace (#396) · 83dbfbf6
  ben authored Mar 10, 2023
```
* host wikitext on hf

* transfer wikitext to Eleuther HF org
```
  83dbfbf6
09 Mar, 2023 1 commit
- transfer data to eleuther HF org · 7502ff3d
  Benjamin Fattori authored Mar 09, 2023
  
  7502ff3d
08 Mar, 2023 3 commits
- inspect import not needed · db501387
  Benjamin Fattori authored Mar 08, 2023
  
  db501387
- remove unused import for eval-harness · 1581db0b
  Benjamin Fattori authored Mar 08, 2023
  
  1581db0b
- arithmetic task class now uses HF remote builder · 3c8479ba
  Benjamin Fattori authored Mar 08, 2023
  
  3c8479ba
18 Jan, 2023 1 commit
- Add accuracy metric to crows-pairs (#380) · f9eca2c8
  Hailey Schoelkopf authored Jan 18, 2023
```
* add accuracy metric to crows-pairs
```
  f9eca2c8
17 Jan, 2023 1 commit

Aflah authored Jan 18, 2023

* Added CrowSPairs

* Fixed Bug

* Changed to absolute Value

* Docstring Changes

* Update crowspairs.py

Finegrained Tasks

* Update __init__.py

* Finegrained Tasks

* Add `crows_pairs` version metadata

ea3df930

11 Jan, 2023 1 commit

Added ToxiGen task (#377) · f079e322

Tom Hartvigsen authored Jan 10, 2023

* added toxigen task

* removed previous doc_to_text

* Add train split for few-shot samples

* Add task version metadata

f079e322

16 Dec, 2022 1 commit
- Update `lambada_openai` multilingual data source · 2bae14d2
  jon-tow authored Dec 16, 2022
  
  2bae14d2
13 Dec, 2022 1 commit
- Bump `triviaqa` version · f7aeeff2
  jon-tow authored Dec 12, 2022
  
  f7aeeff2
02 Dec, 2022 2 commits
- Remove redefinition of `jieba` · 1e5d55d9
  jon-tow authored Dec 02, 2022
  
  1e5d55d9
- Cleanup `README.md` and package deps · 0699d774
  jon-tow authored Dec 02, 2022
  
  0699d774
25 Nov, 2022 1 commit
- Add the original `LAMBADA` dataset · bfa57687
  jon-tow authored Nov 24, 2022
  
  bfa57687
24 Jun, 2022 1 commit
- fix key access in squad evaluation metrics · 1a153185
  Konstantin Schulz authored Jun 24, 2022
  
  1a153185
11 Jun, 2022 1 commit
- Remove custom datasets that are in HF · b4c0275d
  jon-tow authored Jun 11, 2022
  
  b4c0275d
03 May, 2022 3 commits
- Revert `tests/testdata` changes and address flake8 issues · 8c997e53
  jon-tow authored May 03, 2022
  
  8c997e53
- fix codespell · d95a4333
  Fabrizio Milo authored May 02, 2022
  
  d95a4333
- add pre-commit · 121b7096
  Fabrizio Milo authored May 02, 2022
  
  121b7096