Commits · 1701c4e23defaa7fb054b60c49fe238edd92fb35 · gaoqiong / lm-evaluation-harness

14 Aug, 2023 3 commits
- add warning log if prompt returns empty string · 1701c4e2
  lintangsutawika authored Aug 14, 2023
  
  1701c4e2
- add warning log if prompt returns empty string · 722b8974
  lintangsutawika authored Aug 14, 2023
  
  722b8974
- add warning log if prompt returns empty string · bc951ac1
  lintangsutawika authored Aug 14, 2023
  
  bc951ac1
08 Aug, 2023 2 commits
- ensure multiple_choice task type respects target_delimiter · cf1370d1
  haileyschoelkopf authored Aug 08, 2023
  
  cf1370d1
- added typehints · 97f5c020
  baberabb authored Aug 08, 2023
  
  97f5c020
07 Aug, 2023 1 commit
- added typehints and updated docs · 331340ad
  baberabb authored Aug 07, 2023
  
  331340ad
02 Aug, 2023 1 commit
- Update task.py · 53105bba
  Lintang Sutawika authored Aug 02, 2023
  
  53105bba
01 Aug, 2023 3 commits
- update on metrics and delet files · e37698df
  lintangsutawika authored Aug 01, 2023
  
  e37698df
- skips metrics prep if process_result is not None · 1813bf04
  lintangsutawika authored Aug 01, 2023
  
  1813bf04
- process int label to string for greedy_until · b2598de8
  lintangsutawika authored Aug 01, 2023
  
  b2598de8
24 Jul, 2023 1 commit
- Use num_fewshot set in yaml and show warning if it's being overwritten by argparse · 8add5ed6
  lintangsutawika authored Jul 24, 2023
  
  8add5ed6
21 Jul, 2023 2 commits
- [Refactor] Migrate xwinograd tasks to yaml · 94563a36
  Eddy Yeo authored Jul 21, 2023
  
  94563a36
- make process_docs boolean more clear · 3d7f777d
  haileyschoelkopf authored Jul 21, 2023
  
  3d7f777d
18 Jul, 2023 5 commits
- [Refactor] Migrate webqs task to yaml · d2332149
  Eddy Yeo authored Jul 18, 2023
  
  d2332149
- remove template_aliases fully from taskconfig · fbc26541
  haileyschoelkopf authored Jul 18, 2023
  
  fbc26541
- remove template_aliases from existing tasks · 98c5411c
  haileyschoelkopf authored Jul 18, 2023
  
  98c5411c
- remove winograd_schema task type fully · 2bd45fd9
  haileyschoelkopf authored Jul 18, 2023
  
  2bd45fd9
- add assert for passing generation_kwargs w/ output_type neq greedy_until · 80358e5a
  haileyschoelkopf authored Jul 18, 2023
  
  80358e5a
16 Jul, 2023 2 commits
- set fp32 if device=mps · 43407f36
  baberabb authored Jul 16, 2023
  
  43407f36
- set fp32 if device=mps · ce214979
  baberabb authored Jul 16, 2023
  
  ce214979
14 Jul, 2023 9 commits
- can include paths in main · 068f8ab2
  lintangsutawika authored Jun 21, 2023
  
  068f8ab2
- make sure boolq works · d70f7134
  haileyschoelkopf authored Jul 14, 2023
  
  d70f7134
- fix error · 60acbde2
  lintangsutawika authored Jul 14, 2023
  
  60acbde2
- remove info · 1ca2c0ba
  lintangsutawika authored Jul 14, 2023
  
  1ca2c0ba
- adjustments for super_glue · b0597a94
  lintangsutawika authored Jul 14, 2023
  
  b0597a94
- fix typehints in Instance() · 7684bfa9
  haileyschoelkopf authored Jul 14, 2023
  
  7684bfa9
- fix random seed issue, log_samples optional · 5fbc3f86
  haileyschoelkopf authored Jul 14, 2023
  
  5fbc3f86
- doc_to_target and doc_to_text should also accept int · 94a49f70
  lintangsutawika authored Jul 14, 2023
  
  94a49f70
- adjustments · 70e3ed25
  lintangsutawika authored Jul 14, 2023
  
  70e3ed25
13 Jul, 2023 5 commits
- doc_to_choice can run when a promptsource template is defined · ce5ec854
  lintangsutawika authored Jul 13, 2023
  
  ce5ec854
- process condition for doc_to_text · 82d57f65
  lintangsutawika authored Jul 13, 2023
  
  82d57f65
- doc_to_target should also be able to output str instead of solely int · f2b52d26
  lintangsutawika authored Jul 13, 2023
  
  f2b52d26
- removed gold_alias for greedy_until · d1bb50e0
  lintangsutawika authored Jul 13, 2023
  
  d1bb50e0
- process focuses on doc_to_text, doc_to_target, and doc_to_choice · b7449cd9
  lintangsutawika authored Jul 13, 2023
  
  b7449cd9
11 Jul, 2023 6 commits
- add ast.literal_eval to doc_to_choice · 155160a1
  lintangsutawika authored Jul 11, 2023
  
  155160a1
- process multiple_output · c37ad6ec
  lintangsutawika authored Jul 11, 2023
  
  c37ad6ec
- multiple_output option for greedy_until · 4c363ef8
  lintangsutawika authored Jul 11, 2023
  
  4c363ef8
- added error message · f09f3ab8
  lintangsutawika authored Jul 11, 2023
  
  f09f3ab8
- typo · 4054ab84
  lintangsutawika authored Jul 11, 2023
  
  4054ab84
- update choice process · 9b20e2d0
  lintangsutawika authored Jul 11, 2023
  
  9b20e2d0