Commits · e3077dcf4b0008c84dcc6b1b45cdb30dead4b4ff · gaoqiong / lm-evaluation-harness

15 Aug, 2023 1 commit
- rename var · 21aa92d2
  haileyschoelkopf authored Aug 15, 2023
  
  21aa92d2
14 Aug, 2023 4 commits
- return fewshot_delimiter if we get "" · 7758d16a
  lintangsutawika authored Aug 14, 2023
  
  7758d16a
- add warning log if prompt returns empty string · 1701c4e2
  lintangsutawika authored Aug 14, 2023
  
  1701c4e2
- add warning log if prompt returns empty string · 722b8974
  lintangsutawika authored Aug 14, 2023
  
  722b8974
- add warning log if prompt returns empty string · bc951ac1
  lintangsutawika authored Aug 14, 2023
  
  bc951ac1
12 Aug, 2023 1 commit
- fix multi-metric greedy_until · df6c5dcb
  haileyschoelkopf authored Aug 12, 2023
  
  df6c5dcb
11 Aug, 2023 1 commit
- support bleu score as a metric · 8806eff5
  haileyschoelkopf authored Aug 11, 2023
  
  8806eff5
09 Aug, 2023 2 commits
- removed line · 898cdd8b
  lintangsutawika authored Aug 09, 2023
  
  898cdd8b
- allow doc_to_target to be list so that multiple_target works · c2990e29
  lintangsutawika authored Aug 09, 2023
  
  c2990e29
08 Aug, 2023 2 commits
- ensure multiple_choice task type respects target_delimiter · cf1370d1
  haileyschoelkopf authored Aug 08, 2023
  
  cf1370d1
- added typehints · 97f5c020
  baberabb authored Aug 08, 2023
  
  97f5c020
02 Aug, 2023 1 commit
- Update task.py · 53105bba
  Lintang Sutawika authored Aug 02, 2023
  
  53105bba
01 Aug, 2023 3 commits
- update on metrics and delet files · e37698df
  lintangsutawika authored Aug 01, 2023
  
  e37698df
- skips metrics prep if process_result is not None · 1813bf04
  lintangsutawika authored Aug 01, 2023
  
  1813bf04
- process int label to string for greedy_until · b2598de8
  lintangsutawika authored Aug 01, 2023
  
  b2598de8
24 Jul, 2023 1 commit
- Use num_fewshot set in yaml and show warning if it's being overwritten by argparse · 8add5ed6
  lintangsutawika authored Jul 24, 2023
  
  8add5ed6
21 Jul, 2023 2 commits
- [Refactor] Migrate xwinograd tasks to yaml · 94563a36
  Eddy Yeo authored Jul 21, 2023
  
  94563a36
- make process_docs boolean more clear · 3d7f777d
  haileyschoelkopf authored Jul 21, 2023
  
  3d7f777d
18 Jul, 2023 4 commits
- [Refactor] Migrate webqs task to yaml · d2332149
  Eddy Yeo authored Jul 18, 2023
  
  d2332149
- remove template_aliases fully from taskconfig · fbc26541
  haileyschoelkopf authored Jul 18, 2023
  
  fbc26541
- remove template_aliases from existing tasks · 98c5411c
  haileyschoelkopf authored Jul 18, 2023
  
  98c5411c
- add assert for passing generation_kwargs w/ output_type neq greedy_until · 80358e5a
  haileyschoelkopf authored Jul 18, 2023
  
  80358e5a
14 Jul, 2023 8 commits
- can include paths in main · 068f8ab2
  lintangsutawika authored Jun 21, 2023
  
  068f8ab2
- make sure boolq works · d70f7134
  haileyschoelkopf authored Jul 14, 2023
  
  d70f7134
- fix error · 60acbde2
  lintangsutawika authored Jul 14, 2023
  
  60acbde2
- remove info · 1ca2c0ba
  lintangsutawika authored Jul 14, 2023
  
  1ca2c0ba
- adjustments for super_glue · b0597a94
  lintangsutawika authored Jul 14, 2023
  
  b0597a94
- fix random seed issue, log_samples optional · 5fbc3f86
  haileyschoelkopf authored Jul 14, 2023
  
  5fbc3f86
- doc_to_target and doc_to_text should also accept int · 94a49f70
  lintangsutawika authored Jul 14, 2023
  
  94a49f70
- adjustments · 70e3ed25
  lintangsutawika authored Jul 14, 2023
  
  70e3ed25
13 Jul, 2023 4 commits
- doc_to_choice can run when a promptsource template is defined · ce5ec854
  lintangsutawika authored Jul 13, 2023
  
  ce5ec854
- doc_to_target should also be able to output str instead of solely int · f2b52d26
  lintangsutawika authored Jul 13, 2023
  
  f2b52d26
- removed gold_alias for greedy_until · d1bb50e0
  lintangsutawika authored Jul 13, 2023
  
  d1bb50e0
- process focuses on doc_to_text, doc_to_target, and doc_to_choice · b7449cd9
  lintangsutawika authored Jul 13, 2023
  
  b7449cd9
11 Jul, 2023 6 commits
- add ast.literal_eval to doc_to_choice · 155160a1
  lintangsutawika authored Jul 11, 2023
  
  155160a1
- process multiple_output · c37ad6ec
  lintangsutawika authored Jul 11, 2023
  
  c37ad6ec
- multiple_output option for greedy_until · 4c363ef8
  lintangsutawika authored Jul 11, 2023
  
  4c363ef8
- added error message · f09f3ab8
  lintangsutawika authored Jul 11, 2023
  
  f09f3ab8
- typo · 4054ab84
  lintangsutawika authored Jul 11, 2023
  
  4054ab84
- update choice process · 9b20e2d0
  lintangsutawika authored Jul 11, 2023
  
  9b20e2d0