Commits · 6cc1ae7c0b0cb535af593fe1fad5d84912d0c59d · gaoqiong / lm-evaluation-harness

05 Sep, 2023 1 commit
- consolidated tasks testing · 6cc1ae7c
  baberabb authored Sep 05, 2023
  
  6cc1ae7c
20 Aug, 2023 1 commit
- increase atol · 21706139
  baberabb authored Aug 20, 2023
  
  21706139
19 Aug, 2023 3 commits
- atol=1e-8 · 51882c1e
  baberabb authored Aug 19, 2023
  
  51882c1e
- get _RES from CI and change atol to 0.0001 · ac290ce8
  baberabb authored Aug 19, 2023
  
  ac290ce8
- add print to view result on CI · e1c46787
  baberabb authored Aug 19, 2023
  
  e1c46787
17 Aug, 2023 5 commits
- added indices equality check · da77b216
  baberabb authored Aug 17, 2023
  
  da77b216
- change logliklihood atol to 1e-2 · 72bb97ca
  baberabb authored Aug 17, 2023
  
  72bb97ca
- fixup! added tolerance for likelihood, rolling · 91a06081
  baberabb authored Aug 17, 2023
  
  91a06081
- added tolerance for likelihood, rolling · dbbc613f
  baberabb authored Aug 17, 2023
  
  dbbc613f
- added hf model test · f0aa2c7b
  baberabb authored Aug 17, 2023
  
  f0aa2c7b
02 Aug, 2023 1 commit
- added logiqa, logiqa2 and loqieval tasks; fixed tests for greedy_until · 8591f089
  baberabb authored Aug 02, 2023
  
  8591f089
22 Jul, 2023 1 commit
- added conditon to test_create_choices and minor chnages. · 4d8cee80
  baberabb authored Jul 22, 2023
  
  4d8cee80
21 Jul, 2023 4 commits
- fixed fixture · 48d861ce
  baberabb authored Jul 21, 2023
  
  48d861ce
- changed task_class fixture · ba75c838
  baberabb authored Jul 21, 2023
  
  ba75c838
- new_tasks have own tests now · 3d0282cf
  baberabb authored Jul 21, 2023
  
  3d0282cf
- seperated tests for new tasks into own file. · a199314a
  baberabb authored Jul 21, 2023
  
  a199314a
18 Jul, 2023 1 commit
- attempt to fix multiple tasks modified case · 1e2fbe6b
  haileyschoelkopf authored Jul 18, 2023
  
  1e2fbe6b
17 Jul, 2023 6 commits
- fix new_tasks function · 705e57c7
  baberabb authored Jul 17, 2023
  
  705e57c7
- fix typos · 8c0db2ac
  baberabb authored Jul 17, 2023
  
  8c0db2ac
- add CI test case if api/* files changed then pytest more tasks · 80b84321
  baberabb authored Jul 17, 2023
  
  80b84321
- Update and rename __innit__.py to __init__.py · 8e3f1a49
  Hailey Schoelkopf authored Jul 17, 2023
  
  8e3f1a49
- Delete test file · fa660ad8
  baberabb authored Jul 17, 2023
  
  fa660ad8
- edited test_tasks.py and workflow files · 4c250860
  baberabb authored Jul 15, 2023
  
  4c250860
14 Jul, 2023 5 commits
- typo · 1c27e371
  baberabb authored Jul 14, 2023
  
  1c27e371
- updated test_tasks · fae09c2c
  baberabb authored Jul 14, 2023
  
  fae09c2c
- Add CI setup · 94cba287
  baberabb authored Jul 14, 2023
  
  94cba287
- test pythia-70m and pytest only test_evaluator.py · 196fbbb8
  baberabb authored Jul 14, 2023
  
  196fbbb8
- test pythia-70m and pytest only test_evaluator.py · 4225df50
  baberabb authored Jul 14, 2023
  
  4225df50
13 Jul, 2023 1 commit
- fix test_evaluator.py · d9b547b7
  baberabb authored Jul 13, 2023
  
  d9b547b7
21 Jun, 2023 1 commit
- patch some tests · a7993806
  haileyschoelkopf authored Jun 21, 2023
  
  a7993806
19 Jun, 2023 2 commits
- remove descrip dict from test_evaluator · 77289aed
  haileyschoelkopf authored Jun 19, 2023
  
  77289aed
- rename test_description.py · 05a7d1da
  haileyschoelkopf authored Jun 19, 2023
  
  05a7d1da
19 Apr, 2023 1 commit
- in-place replace main with lm-eval2, keeping old git history · d2a9b759
  haileyschoelkopf authored Apr 19, 2023
  
  d2a9b759
18 Jan, 2023 1 commit
- Add accuracy metric to crows-pairs (#380) · f9eca2c8
  Hailey Schoelkopf authored Jan 18, 2023
```
* add accuracy metric to crows-pairs
```
  f9eca2c8
17 Jan, 2023 1 commit

Aflah authored Jan 18, 2023

* Added CrowSPairs

* Fixed Bug

* Changed to absolute Value

* Docstring Changes

* Update crowspairs.py

Finegrained Tasks

* Update __init__.py

* Finegrained Tasks

* Add `crows_pairs` version metadata

ea3df930

11 Jan, 2023 1 commit

Added ToxiGen task (#377) · f079e322

Tom Hartvigsen authored Jan 10, 2023

* added toxigen task

* removed previous doc_to_text

* Add train split for few-shot samples

* Add task version metadata

f079e322

13 Dec, 2022 1 commit
- Bump `triviaqa` version · f7aeeff2
  jon-tow authored Dec 12, 2022
  
  f7aeeff2
02 Dec, 2022 1 commit
- Cleanup `README.md` and package deps · 0699d774
  jon-tow authored Dec 02, 2022
  
  0699d774
25 Nov, 2022 1 commit
- Add the original `LAMBADA` dataset · bfa57687
  jon-tow authored Nov 24, 2022
  
  bfa57687
04 Nov, 2022 1 commit
- Add `TextSynth` API (#299) · 4f0410a4
  Jonathan Tow authored Nov 03, 2022
```
* Add `TextSynth` API
```
  4f0410a4