Commits · 0458e152aef701dd83c2e2857e737362732c8a99 · gaoqiong / lm-evaluation-harness

15 Aug, 2023 3 commits
- go through choices and targets · 0458e152
  lintangsutawika authored Jul 24, 2023
  
  0458e152
- removed comment · ffb56822
  lintangsutawika authored Jul 24, 2023
  
  ffb56822
- added warning regarding use of whitespace for target_delimiter and target, add... · db97d7f8
  lintangsutawika authored Jul 24, 2023
```
added warning regarding use of whitespace for target_delimiter and target, add feature to allow to acquire target from doc_to_choice for greedy_until, switch order between multiple_target and non multiple_target being processed
```
  db97d7f8
14 Aug, 2023 4 commits
- return fewshot_delimiter if we get "" · 7758d16a
  lintangsutawika authored Aug 14, 2023
  
  7758d16a
- add warning log if prompt returns empty string · 1701c4e2
  lintangsutawika authored Aug 14, 2023
  
  1701c4e2
- add warning log if prompt returns empty string · 722b8974
  lintangsutawika authored Aug 14, 2023
  
  722b8974
- add warning log if prompt returns empty string · bc951ac1
  lintangsutawika authored Aug 14, 2023
  
  bc951ac1
08 Aug, 2023 2 commits
- ensure multiple_choice task type respects target_delimiter · cf1370d1
  haileyschoelkopf authored Aug 08, 2023
  
  cf1370d1
- added typehints · 97f5c020
  baberabb authored Aug 08, 2023
  
  97f5c020
02 Aug, 2023 1 commit
- Update task.py · 53105bba
  Lintang Sutawika authored Aug 02, 2023
  
  53105bba
01 Aug, 2023 3 commits
- update on metrics and delet files · e37698df
  lintangsutawika authored Aug 01, 2023
  
  e37698df
- skips metrics prep if process_result is not None · 1813bf04
  lintangsutawika authored Aug 01, 2023
  
  1813bf04
- process int label to string for greedy_until · b2598de8
  lintangsutawika authored Aug 01, 2023
  
  b2598de8
24 Jul, 2023 1 commit
- Use num_fewshot set in yaml and show warning if it's being overwritten by argparse · 8add5ed6
  lintangsutawika authored Jul 24, 2023
  
  8add5ed6
21 Jul, 2023 2 commits
- [Refactor] Migrate xwinograd tasks to yaml · 94563a36
  Eddy Yeo authored Jul 21, 2023
  
  94563a36
- make process_docs boolean more clear · 3d7f777d
  haileyschoelkopf authored Jul 21, 2023
  
  3d7f777d
18 Jul, 2023 4 commits
- [Refactor] Migrate webqs task to yaml · d2332149
  Eddy Yeo authored Jul 18, 2023
  
  d2332149
- remove template_aliases fully from taskconfig · fbc26541
  haileyschoelkopf authored Jul 18, 2023
  
  fbc26541
- remove template_aliases from existing tasks · 98c5411c
  haileyschoelkopf authored Jul 18, 2023
  
  98c5411c
- add assert for passing generation_kwargs w/ output_type neq greedy_until · 80358e5a
  haileyschoelkopf authored Jul 18, 2023
  
  80358e5a
14 Jul, 2023 8 commits
- can include paths in main · 068f8ab2
  lintangsutawika authored Jun 21, 2023
  
  068f8ab2
- make sure boolq works · d70f7134
  haileyschoelkopf authored Jul 14, 2023
  
  d70f7134
- fix error · 60acbde2
  lintangsutawika authored Jul 14, 2023
  
  60acbde2
- remove info · 1ca2c0ba
  lintangsutawika authored Jul 14, 2023
  
  1ca2c0ba
- adjustments for super_glue · b0597a94
  lintangsutawika authored Jul 14, 2023
  
  b0597a94
- fix random seed issue, log_samples optional · 5fbc3f86
  haileyschoelkopf authored Jul 14, 2023
  
  5fbc3f86
- doc_to_target and doc_to_text should also accept int · 94a49f70
  lintangsutawika authored Jul 14, 2023
  
  94a49f70
- adjustments · 70e3ed25
  lintangsutawika authored Jul 14, 2023
  
  70e3ed25
13 Jul, 2023 4 commits
- doc_to_choice can run when a promptsource template is defined · ce5ec854
  lintangsutawika authored Jul 13, 2023
  
  ce5ec854
- doc_to_target should also be able to output str instead of solely int · f2b52d26
  lintangsutawika authored Jul 13, 2023
  
  f2b52d26
- removed gold_alias for greedy_until · d1bb50e0
  lintangsutawika authored Jul 13, 2023
  
  d1bb50e0
- process focuses on doc_to_text, doc_to_target, and doc_to_choice · b7449cd9
  lintangsutawika authored Jul 13, 2023
  
  b7449cd9
11 Jul, 2023 8 commits
- add ast.literal_eval to doc_to_choice · 155160a1
  lintangsutawika authored Jul 11, 2023
  
  155160a1
- process multiple_output · c37ad6ec
  lintangsutawika authored Jul 11, 2023
  
  c37ad6ec
- multiple_output option for greedy_until · 4c363ef8
  lintangsutawika authored Jul 11, 2023
  
  4c363ef8
- added error message · f09f3ab8
  lintangsutawika authored Jul 11, 2023
  
  f09f3ab8
- typo · 4054ab84
  lintangsutawika authored Jul 11, 2023
  
  4054ab84
- update choice process · 9b20e2d0
  lintangsutawika authored Jul 11, 2023
  
  9b20e2d0
- process update to process doc_to_target variety · 656c310e
  lintangsutawika authored Jul 11, 2023
  
  656c310e
- updated winogrande · 3b50b941
  lintangsutawika authored Jul 11, 2023
  
  3b50b941