Commits · ef33202653db7eb775c5ee942304cf3ca91e3f0d · gaoqiong / lm-evaluation-harness

18 Oct, 2023 1 commit
- replace the rest of the 'greedy_until' occurrences · c28d100d
  haileyschoelkopf authored Oct 18, 2023
  
  c28d100d
14 Sep, 2023 1 commit
- make typing 3.8 compatible · 894370df
  baberabb authored Sep 15, 2023
  
  894370df
06 Sep, 2023 3 commits
- changed atol · dfc47bd5
  baberabb authored Sep 06, 2023
  
  dfc47bd5
- artifact loglikelihood results · 96c60cf6
  baberabb authored Sep 06, 2023
  
  96c60cf6
- added torch deterministic · f1e62d36
  baberabb authored Sep 05, 2023
  
  f1e62d36
05 Sep, 2023 1 commit
- consolidated tasks testing · 6cc1ae7c
  baberabb authored Sep 05, 2023
  
  6cc1ae7c
20 Aug, 2023 1 commit
- increase atol · 21706139
  baberabb authored Aug 20, 2023
  
  21706139
19 Aug, 2023 3 commits
- atol=1e-8 · 51882c1e
  baberabb authored Aug 19, 2023
  
  51882c1e
- get _RES from CI and change atol to 0.0001 · ac290ce8
  baberabb authored Aug 19, 2023
  
  ac290ce8
- add print to view result on CI · e1c46787
  baberabb authored Aug 19, 2023
  
  e1c46787
17 Aug, 2023 5 commits
- added indices equality check · da77b216
  baberabb authored Aug 17, 2023
  
  da77b216
- change logliklihood atol to 1e-2 · 72bb97ca
  baberabb authored Aug 17, 2023
  
  72bb97ca
- fixup! added tolerance for likelihood, rolling · 91a06081
  baberabb authored Aug 17, 2023
  
  91a06081
- added tolerance for likelihood, rolling · dbbc613f
  baberabb authored Aug 17, 2023
  
  dbbc613f
- added hf model test · f0aa2c7b
  baberabb authored Aug 17, 2023
  
  f0aa2c7b
02 Aug, 2023 1 commit
- added logiqa, logiqa2 and loqieval tasks; fixed tests for greedy_until · 8591f089
  baberabb authored Aug 02, 2023
  
  8591f089
22 Jul, 2023 1 commit
- added conditon to test_create_choices and minor chnages. · 4d8cee80
  baberabb authored Jul 22, 2023
  
  4d8cee80
21 Jul, 2023 4 commits
- fixed fixture · 48d861ce
  baberabb authored Jul 21, 2023
  
  48d861ce
- changed task_class fixture · ba75c838
  baberabb authored Jul 21, 2023
  
  ba75c838
- new_tasks have own tests now · 3d0282cf
  baberabb authored Jul 21, 2023
  
  3d0282cf
- seperated tests for new tasks into own file. · a199314a
  baberabb authored Jul 21, 2023
  
  a199314a
18 Jul, 2023 1 commit
- attempt to fix multiple tasks modified case · 1e2fbe6b
  haileyschoelkopf authored Jul 18, 2023
  
  1e2fbe6b
17 Jul, 2023 6 commits
- fix new_tasks function · 705e57c7
  baberabb authored Jul 17, 2023
  
  705e57c7
- fix typos · 8c0db2ac
  baberabb authored Jul 17, 2023
  
  8c0db2ac
- add CI test case if api/* files changed then pytest more tasks · 80b84321
  baberabb authored Jul 17, 2023
  
  80b84321
- Update and rename __innit__.py to __init__.py · 8e3f1a49
  Hailey Schoelkopf authored Jul 17, 2023
  
  8e3f1a49
- Delete test file · fa660ad8
  baberabb authored Jul 17, 2023
  
  fa660ad8
- edited test_tasks.py and workflow files · 4c250860
  baberabb authored Jul 15, 2023
  
  4c250860
14 Jul, 2023 5 commits
- typo · 1c27e371
  baberabb authored Jul 14, 2023
  
  1c27e371
- updated test_tasks · fae09c2c
  baberabb authored Jul 14, 2023
  
  fae09c2c
- Add CI setup · 94cba287
  baberabb authored Jul 14, 2023
  
  94cba287
- test pythia-70m and pytest only test_evaluator.py · 196fbbb8
  baberabb authored Jul 14, 2023
  
  196fbbb8
- test pythia-70m and pytest only test_evaluator.py · 4225df50
  baberabb authored Jul 14, 2023
  
  4225df50
13 Jul, 2023 1 commit
- fix test_evaluator.py · d9b547b7
  baberabb authored Jul 13, 2023
  
  d9b547b7
21 Jun, 2023 1 commit
- patch some tests · a7993806
  haileyschoelkopf authored Jun 21, 2023
  
  a7993806
19 Jun, 2023 2 commits
- remove descrip dict from test_evaluator · 77289aed
  haileyschoelkopf authored Jun 19, 2023
  
  77289aed
- rename test_description.py · 05a7d1da
  haileyschoelkopf authored Jun 19, 2023
  
  05a7d1da
19 Apr, 2023 1 commit
- in-place replace main with lm-eval2, keeping old git history · d2a9b759
  haileyschoelkopf authored Apr 19, 2023
  
  d2a9b759
18 Jan, 2023 1 commit
- Add accuracy metric to crows-pairs (#380) · f9eca2c8
  Hailey Schoelkopf authored Jan 18, 2023
```
* add accuracy metric to crows-pairs
```
  f9eca2c8
17 Jan, 2023 1 commit

Aflah authored Jan 18, 2023

* Added CrowSPairs

* Fixed Bug

* Changed to absolute Value

* Docstring Changes

* Update crowspairs.py

Finegrained Tasks

* Update __init__.py

* Finegrained Tasks

* Add `crows_pairs` version metadata

ea3df930