Commits · 21d4ae98c1e5bab15ccd71a239fc08c8fbf02c03 · gaoqiong / lm-evaluation-harness

27 Nov, 2023 3 commits
- skip vllm test (requires CUDA) · 05326b04
  baberabb authored Nov 27, 2023
  
  05326b04
- lazy load import in tests · 2c20df08
  baberabb authored Nov 27, 2023
  
  2c20df08
- Add tests for VLLM · 026005ac
  baberabb authored Nov 27, 2023
  
  026005ac
24 Nov, 2023 1 commit
- update · 30296795
  lintangsutawika authored Nov 24, 2023
  
  30296795
21 Nov, 2023 1 commit
- API update · bfe69313
  lintangsutawika authored Nov 21, 2023
  
  bfe69313
20 Nov, 2023 1 commit
- add ggml tests · f87b864f
  haileyschoelkopf authored Nov 20, 2023
  
  f87b864f
17 Nov, 2023 1 commit
- initialize tasks in test files · 032820dd
  haileyschoelkopf authored Nov 17, 2023
  
  032820dd
16 Nov, 2023 2 commits
- widen HF model test tolerance · 57ce579c
  haileyschoelkopf authored Nov 16, 2023
  
  57ce579c
- update gsm8k name · 1eed1cb5
  haileyschoelkopf authored Nov 16, 2023
  
  1eed1cb5
18 Oct, 2023 1 commit
- replace the rest of the 'greedy_until' occurrences · c28d100d
  haileyschoelkopf authored Oct 18, 2023
  
  c28d100d
14 Sep, 2023 1 commit
- make typing 3.8 compatible · 894370df
  baberabb authored Sep 15, 2023
  
  894370df
06 Sep, 2023 3 commits
- changed atol · dfc47bd5
  baberabb authored Sep 06, 2023
  
  dfc47bd5
- artifact loglikelihood results · 96c60cf6
  baberabb authored Sep 06, 2023
  
  96c60cf6
- added torch deterministic · f1e62d36
  baberabb authored Sep 05, 2023
  
  f1e62d36
05 Sep, 2023 1 commit
- consolidated tasks testing · 6cc1ae7c
  baberabb authored Sep 05, 2023
  
  6cc1ae7c
20 Aug, 2023 1 commit
- increase atol · 21706139
  baberabb authored Aug 20, 2023
  
  21706139
19 Aug, 2023 3 commits
- atol=1e-8 · 51882c1e
  baberabb authored Aug 19, 2023
  
  51882c1e
- get _RES from CI and change atol to 0.0001 · ac290ce8
  baberabb authored Aug 19, 2023
  
  ac290ce8
- add print to view result on CI · e1c46787
  baberabb authored Aug 19, 2023
  
  e1c46787
17 Aug, 2023 5 commits
- added indices equality check · da77b216
  baberabb authored Aug 17, 2023
  
  da77b216
- change logliklihood atol to 1e-2 · 72bb97ca
  baberabb authored Aug 17, 2023
  
  72bb97ca
- fixup! added tolerance for likelihood, rolling · 91a06081
  baberabb authored Aug 17, 2023
  
  91a06081
- added tolerance for likelihood, rolling · dbbc613f
  baberabb authored Aug 17, 2023
  
  dbbc613f
- added hf model test · f0aa2c7b
  baberabb authored Aug 17, 2023
  
  f0aa2c7b
02 Aug, 2023 1 commit
- added logiqa, logiqa2 and loqieval tasks; fixed tests for greedy_until · 8591f089
  baberabb authored Aug 02, 2023
  
  8591f089
22 Jul, 2023 1 commit
- added conditon to test_create_choices and minor chnages. · 4d8cee80
  baberabb authored Jul 22, 2023
  
  4d8cee80
21 Jul, 2023 4 commits
- fixed fixture · 48d861ce
  baberabb authored Jul 21, 2023
  
  48d861ce
- changed task_class fixture · ba75c838
  baberabb authored Jul 21, 2023
  
  ba75c838
- new_tasks have own tests now · 3d0282cf
  baberabb authored Jul 21, 2023
  
  3d0282cf
- seperated tests for new tasks into own file. · a199314a
  baberabb authored Jul 21, 2023
  
  a199314a
18 Jul, 2023 1 commit
- attempt to fix multiple tasks modified case · 1e2fbe6b
  haileyschoelkopf authored Jul 18, 2023
  
  1e2fbe6b
17 Jul, 2023 6 commits
- fix new_tasks function · 705e57c7
  baberabb authored Jul 17, 2023
  
  705e57c7
- fix typos · 8c0db2ac
  baberabb authored Jul 17, 2023
  
  8c0db2ac
- add CI test case if api/* files changed then pytest more tasks · 80b84321
  baberabb authored Jul 17, 2023
  
  80b84321
- Update and rename __innit__.py to __init__.py · 8e3f1a49
  Hailey Schoelkopf authored Jul 17, 2023
  
  8e3f1a49
- Delete test file · fa660ad8
  baberabb authored Jul 17, 2023
  
  fa660ad8
- edited test_tasks.py and workflow files · 4c250860
  baberabb authored Jul 15, 2023
  
  4c250860
14 Jul, 2023 3 commits
- typo · 1c27e371
  baberabb authored Jul 14, 2023
  
  1c27e371
- updated test_tasks · fae09c2c
  baberabb authored Jul 14, 2023
  
  fae09c2c
- Add CI setup · 94cba287
  baberabb authored Jul 14, 2023
  
  94cba287