Commits · e3077dcf4b0008c84dcc6b1b45cdb30dead4b4ff · gaoqiong / lm-evaluation-harness

15 Aug, 2023 1 commit
- rename var · 21aa92d2
  haileyschoelkopf authored Aug 15, 2023
  
  21aa92d2
14 Aug, 2023 9 commits
- Removed Implementation from metrics.py · 60763408
  Aflah authored Aug 03, 2023
  
  60763408
- Improved DocString · 57bcb4d6
  Aflah authored Aug 02, 2023
  
  57bcb4d6
- CMD Line Works · b221531a
  Aflah authored Aug 02, 2023
  
  b221531a
- Added Metric and Seems to Run but some Errors · d857e539
  Aflah authored Aug 02, 2023
  
  d857e539
- Base Template · 94ccc429
  Aflah authored Aug 02, 2023
  
  94ccc429
- return fewshot_delimiter if we get "" · 7758d16a
  lintangsutawika authored Aug 14, 2023
  
  7758d16a
- add warning log if prompt returns empty string · 1701c4e2
  lintangsutawika authored Aug 14, 2023
  
  1701c4e2
- add warning log if prompt returns empty string · 722b8974
  lintangsutawika authored Aug 14, 2023
  
  722b8974
- add warning log if prompt returns empty string · bc951ac1
  lintangsutawika authored Aug 14, 2023
  
  bc951ac1
12 Aug, 2023 2 commits
- fix multi-metric greedy_until · df6c5dcb
  haileyschoelkopf authored Aug 12, 2023
  
  df6c5dcb
- make chrf and ter aggregations · 116c540a
  haileyschoelkopf authored Aug 12, 2023
  
  116c540a
11 Aug, 2023 1 commit
- support bleu score as a metric · 8806eff5
  haileyschoelkopf authored Aug 11, 2023
  
  8806eff5
09 Aug, 2023 3 commits
- add sampler condition for if target is multiple, and target_delimiter is " " · 1d3599a3
  lintangsutawika authored Aug 09, 2023
  
  1d3599a3
- removed line · 898cdd8b
  lintangsutawika authored Aug 09, 2023
  
  898cdd8b
- allow doc_to_target to be list so that multiple_target works · c2990e29
  lintangsutawika authored Aug 09, 2023
  
  c2990e29
08 Aug, 2023 2 commits
- ensure multiple_choice task type respects target_delimiter · cf1370d1
  haileyschoelkopf authored Aug 08, 2023
  
  cf1370d1
- added typehints · 97f5c020
  baberabb authored Aug 08, 2023
  
  97f5c020
07 Aug, 2023 1 commit
- added typehints and updated docs · 331340ad
  baberabb authored Aug 07, 2023
  
  331340ad
03 Aug, 2023 1 commit
- Removed Implementation from metrics.py · 7b376ae1
  Aflah authored Aug 03, 2023
  
  7b376ae1
02 Aug, 2023 5 commits
- Improved DocString · c42cb562
  Aflah authored Aug 02, 2023
  
  c42cb562
- CMD Line Works · eaa1c766
  Aflah authored Aug 02, 2023
  
  eaa1c766
- Added Metric and Seems to Run but some Errors · 62b8a6ce
  Aflah authored Aug 02, 2023
  
  62b8a6ce
- Base Template · 6cb8169c
  Aflah authored Aug 02, 2023
  
  6cb8169c
- Update task.py · 53105bba
  Lintang Sutawika authored Aug 02, 2023
  
  53105bba
01 Aug, 2023 3 commits
- update on metrics and delet files · e37698df
  lintangsutawika authored Aug 01, 2023
  
  e37698df
- skips metrics prep if process_result is not None · 1813bf04
  lintangsutawika authored Aug 01, 2023
  
  1813bf04
- process int label to string for greedy_until · b2598de8
  lintangsutawika authored Aug 01, 2023
  
  b2598de8
24 Jul, 2023 1 commit
- Use num_fewshot set in yaml and show warning if it's being overwritten by argparse · 8add5ed6
  lintangsutawika authored Jul 24, 2023
  
  8add5ed6
21 Jul, 2023 2 commits
- [Refactor] Migrate xwinograd tasks to yaml · 94563a36
  Eddy Yeo authored Jul 21, 2023
  
  94563a36
- make process_docs boolean more clear · 3d7f777d
  haileyschoelkopf authored Jul 21, 2023
  
  3d7f777d
18 Jul, 2023 5 commits
- [Refactor] Migrate webqs task to yaml · d2332149
  Eddy Yeo authored Jul 18, 2023
  
  d2332149
- remove template_aliases fully from taskconfig · fbc26541
  haileyschoelkopf authored Jul 18, 2023
  
  fbc26541
- remove template_aliases from existing tasks · 98c5411c
  haileyschoelkopf authored Jul 18, 2023
  
  98c5411c
- remove winograd_schema task type fully · 2bd45fd9
  haileyschoelkopf authored Jul 18, 2023
  
  2bd45fd9
- add assert for passing generation_kwargs w/ output_type neq greedy_until · 80358e5a
  haileyschoelkopf authored Jul 18, 2023
  
  80358e5a
16 Jul, 2023 2 commits
- set fp32 if device=mps · 43407f36
  baberabb authored Jul 16, 2023
  
  43407f36
- set fp32 if device=mps · ce214979
  baberabb authored Jul 16, 2023
  
  ce214979
14 Jul, 2023 2 commits
- can include paths in main · 068f8ab2
  lintangsutawika authored Jun 21, 2023
  
  068f8ab2
- make sure boolq works · d70f7134
  haileyschoelkopf authored Jul 14, 2023
  
  d70f7134