Commits · fc63c7afb85db9c66a5050cc0afee8cdee89dd83 · gaoqiong / lm-evaluation-harness

16 Jul, 2023 2 commits
- set fp32 if device=mps · 43407f36
  baberabb authored Jul 16, 2023
  
  43407f36
- set fp32 if device=mps · ce214979
  baberabb authored Jul 16, 2023
  
  ce214979
14 Jul, 2023 8 commits
- make sure boolq works · d70f7134
  haileyschoelkopf authored Jul 14, 2023
  
  d70f7134
- fix error · 60acbde2
  lintangsutawika authored Jul 14, 2023
  
  60acbde2
- remove info · 1ca2c0ba
  lintangsutawika authored Jul 14, 2023
  
  1ca2c0ba
- adjustments for super_glue · b0597a94
  lintangsutawika authored Jul 14, 2023
  
  b0597a94
- fix typehints in Instance() · 7684bfa9
  haileyschoelkopf authored Jul 14, 2023
  
  7684bfa9
- fix random seed issue, log_samples optional · 5fbc3f86
  haileyschoelkopf authored Jul 14, 2023
  
  5fbc3f86
- doc_to_target and doc_to_text should also accept int · 94a49f70
  lintangsutawika authored Jul 14, 2023
  
  94a49f70
- adjustments · 70e3ed25
  lintangsutawika authored Jul 14, 2023
  
  70e3ed25
13 Jul, 2023 5 commits
- doc_to_choice can run when a promptsource template is defined · ce5ec854
  lintangsutawika authored Jul 13, 2023
  
  ce5ec854
- process condition for doc_to_text · 82d57f65
  lintangsutawika authored Jul 13, 2023
  
  82d57f65
- doc_to_target should also be able to output str instead of solely int · f2b52d26
  lintangsutawika authored Jul 13, 2023
  
  f2b52d26
- removed gold_alias for greedy_until · d1bb50e0
  lintangsutawika authored Jul 13, 2023
  
  d1bb50e0
- process focuses on doc_to_text, doc_to_target, and doc_to_choice · b7449cd9
  lintangsutawika authored Jul 13, 2023
  
  b7449cd9
11 Jul, 2023 8 commits
- add ast.literal_eval to doc_to_choice · 155160a1
  lintangsutawika authored Jul 11, 2023
  
  155160a1
- process multiple_output · c37ad6ec
  lintangsutawika authored Jul 11, 2023
  
  c37ad6ec
- multiple_output option for greedy_until · 4c363ef8
  lintangsutawika authored Jul 11, 2023
  
  4c363ef8
- added error message · f09f3ab8
  lintangsutawika authored Jul 11, 2023
  
  f09f3ab8
- typo · 4054ab84
  lintangsutawika authored Jul 11, 2023
  
  4054ab84
- update choice process · 9b20e2d0
  lintangsutawika authored Jul 11, 2023
  
  9b20e2d0
- process update to process doc_to_target variety · 656c310e
  lintangsutawika authored Jul 11, 2023
  
  656c310e
- updated winogrande · 3b50b941
  lintangsutawika authored Jul 11, 2023
  
  3b50b941
10 Jul, 2023 2 commits
- Testing for multiple inputs from doc_to_text · f33c6c9b
  Lintang Sutawika authored Jul 10, 2023
  
  f33c6c9b
- template aliase process update. · 552b09e0
  Lintang Sutawika authored Jul 10, 2023
  
  552b09e0
09 Jul, 2023 3 commits
- post_init to test if multple_choice has doc_to_text return list or str · 49a116ee
  Lintang Sutawika authored Jul 09, 2023
  
  49a116ee
- multple_choice handles 2 cases · 003c2581
  Lintang Sutawika authored Jul 09, 2023
  
  003c2581
- use prediction and prediction index · 87d93e99
  Lintang Sutawika authored Jul 09, 2023
  
  87d93e99
08 Jul, 2023 1 commit
- Very rough draft and scribbles · 3aeb95b6
  Lintang Sutawika authored Jul 09, 2023
  
  3aeb95b6
06 Jul, 2023 3 commits
- revert change to template_aliases · 6fc2e148
  haileyschoelkopf authored Jul 06, 2023
  
  6fc2e148
- bugfixes + add write_out · 4e5a328e
  haileyschoelkopf authored Jul 06, 2023
  
  4e5a328e
- Change `target_delimiter` to `fewshot_delimiter` · 7ffa1d1b
  Lintang Sutawika authored Jul 06, 2023
  
  7ffa1d1b
05 Jul, 2023 3 commits
- remove batch_size param from config · 38bd2cc8
  haileyschoelkopf authored Jul 05, 2023
  
  38bd2cc8
- Update task.py · 59b047fb
  Lintang Sutawika authored Jul 05, 2023
  
  59b047fb
- Update task.py · 633fb458
  Lintang Sutawika authored Jul 05, 2023
  
  633fb458
04 Jul, 2023 1 commit
- address PR comments · b1b5239d
  Benjamin Fattori authored Jul 04, 2023
  
  b1b5239d
03 Jul, 2023 3 commits
- cleanup metric loading code · 328f0e85
  haileyschoelkopf authored Jul 03, 2023
  
  328f0e85
- add up-to-date config fields to docs · 97220c9b
  haileyschoelkopf authored Jul 03, 2023
  
  97220c9b
- Update task.py · 0b62e042
  Hailey Schoelkopf authored Jul 03, 2023
  
  0b62e042
02 Jul, 2023 1 commit
- add new task type: winograd_schema · a8396b2c
  Benjamin Fattori authored Jul 02, 2023
  
  a8396b2c