Commits · ce5ec854bef0bef921dd11739b38f72801475fa7 · gaoqiong / lm-evaluation-harness

13 Jul, 2023 4 commits
- doc_to_choice can run when a promptsource template is defined · ce5ec854
  lintangsutawika authored Jul 13, 2023
  
  ce5ec854
- doc_to_target should also be able to output str instead of solely int · f2b52d26
  lintangsutawika authored Jul 13, 2023
  
  f2b52d26
- removed gold_alias for greedy_until · d1bb50e0
  lintangsutawika authored Jul 13, 2023
  
  d1bb50e0
- process focuses on doc_to_text, doc_to_target, and doc_to_choice · b7449cd9
  lintangsutawika authored Jul 13, 2023
  
  b7449cd9
11 Jul, 2023 8 commits
- add ast.literal_eval to doc_to_choice · 155160a1
  lintangsutawika authored Jul 11, 2023
  
  155160a1
- process multiple_output · c37ad6ec
  lintangsutawika authored Jul 11, 2023
  
  c37ad6ec
- multiple_output option for greedy_until · 4c363ef8
  lintangsutawika authored Jul 11, 2023
  
  4c363ef8
- added error message · f09f3ab8
  lintangsutawika authored Jul 11, 2023
  
  f09f3ab8
- typo · 4054ab84
  lintangsutawika authored Jul 11, 2023
  
  4054ab84
- update choice process · 9b20e2d0
  lintangsutawika authored Jul 11, 2023
  
  9b20e2d0
- process update to process doc_to_target variety · 656c310e
  lintangsutawika authored Jul 11, 2023
  
  656c310e
- updated winogrande · 3b50b941
  lintangsutawika authored Jul 11, 2023
  
  3b50b941
10 Jul, 2023 2 commits
- Testing for multiple inputs from doc_to_text · f33c6c9b
  Lintang Sutawika authored Jul 10, 2023
  
  f33c6c9b
- template aliase process update. · 552b09e0
  Lintang Sutawika authored Jul 10, 2023
  
  552b09e0
09 Jul, 2023 3 commits
- post_init to test if multple_choice has doc_to_text return list or str · 49a116ee
  Lintang Sutawika authored Jul 09, 2023
  
  49a116ee
- multple_choice handles 2 cases · 003c2581
  Lintang Sutawika authored Jul 09, 2023
  
  003c2581
- use prediction and prediction index · 87d93e99
  Lintang Sutawika authored Jul 09, 2023
  
  87d93e99
08 Jul, 2023 1 commit
- Very rough draft and scribbles · 3aeb95b6
  Lintang Sutawika authored Jul 09, 2023
  
  3aeb95b6
06 Jul, 2023 3 commits
- revert change to template_aliases · 6fc2e148
  haileyschoelkopf authored Jul 06, 2023
  
  6fc2e148
- bugfixes + add write_out · 4e5a328e
  haileyschoelkopf authored Jul 06, 2023
  
  4e5a328e
- Change `target_delimiter` to `fewshot_delimiter` · 7ffa1d1b
  Lintang Sutawika authored Jul 06, 2023
  
  7ffa1d1b
05 Jul, 2023 3 commits
- remove batch_size param from config · 38bd2cc8
  haileyschoelkopf authored Jul 05, 2023
  
  38bd2cc8
- Update task.py · 59b047fb
  Lintang Sutawika authored Jul 05, 2023
  
  59b047fb
- Update task.py · 633fb458
  Lintang Sutawika authored Jul 05, 2023
  
  633fb458
04 Jul, 2023 1 commit
- address PR comments · b1b5239d
  Benjamin Fattori authored Jul 04, 2023
  
  b1b5239d
03 Jul, 2023 3 commits
- cleanup metric loading code · 328f0e85
  haileyschoelkopf authored Jul 03, 2023
  
  328f0e85
- add up-to-date config fields to docs · 97220c9b
  haileyschoelkopf authored Jul 03, 2023
  
  97220c9b
- Update task.py · 0b62e042
  Hailey Schoelkopf authored Jul 03, 2023
  
  0b62e042
02 Jul, 2023 1 commit
- add new task type: winograd_schema · a8396b2c
  Benjamin Fattori authored Jul 02, 2023
  
  a8396b2c
28 Jun, 2023 1 commit
- revert mutual info whitespace change · 03b9db6b
  haileyschoelkopf authored Jun 28, 2023
  
  03b9db6b
23 Jun, 2023 2 commits
- make fewshot_docs warning more explicit · 8d14b376
  haileyschoelkopf authored Jun 23, 2023
  
  8d14b376
- fix mutual info normalization · a68c3fa4
  haileyschoelkopf authored Jun 23, 2023
  
  a68c3fa4
22 Jun, 2023 2 commits
- more pre-commit · b48f5205
  haileyschoelkopf authored Jun 20, 2023
  
  b48f5205
- fix small TaskConfig bugs · 1d7d3de5
  haileyschoelkopf authored Jun 20, 2023
  
  1d7d3de5
21 Jun, 2023 3 commits
- swap target_delimiter and fewshot_delimiter · a575d9e5
  Hailey Schoelkopf authored Jun 21, 2023
  
  a575d9e5
- Update task.py · b8cb513b
  Lintang Sutawika authored Jun 21, 2023
  
  b8cb513b
- made two delimiter modes · 977817a0
  lintangsutawika authored Jun 21, 2023
  
  977817a0
20 Jun, 2023 3 commits
- remove old code · cbb04a20
  haileyschoelkopf authored Jun 20, 2023
  
  cbb04a20
- more pre-commit · 9cf4a104
  haileyschoelkopf authored Jun 20, 2023
  
  9cf4a104
- fix small TaskConfig bugs · be5e118b
  haileyschoelkopf authored Jun 20, 2023
  
  be5e118b