Commits · 2f0ef6ca03f26353ba92d3a4417accee181fddde · gaoqiong / lm-evaluation-harness

23 Aug, 2023 1 commit
- fix condition where a benchmark and a regular task is called together · 2f0ef6ca
  lintangsutawika authored Aug 23, 2023
  
  2f0ef6ca
13 Aug, 2023 1 commit
- moved to a new dir · f4813cfc
  lintangsutawika authored Aug 13, 2023
  
  f4813cfc
08 Aug, 2023 1 commit
- fix error in a logging msg · f33d3758
  haileyschoelkopf authored Aug 08, 2023
  
  f33d3758
07 Aug, 2023 1 commit
- Update error message if task name misspelt · 7aff9b75
  baberabb authored Aug 07, 2023
  
  7aff9b75
04 Aug, 2023 2 commits
- Update evaluator.py · fa2d592a
  Lintang Sutawika authored Aug 04, 2023
  
  fa2d592a
- reformat · b3a8c985
  lintangsutawika authored Aug 04, 2023
  
  b3a8c985
01 Aug, 2023 2 commits
- update on metrics and delet files · e37698df
  lintangsutawika authored Aug 01, 2023
  
  e37698df
- process aggregate fix · 0eb94c8b
  lintangsutawika authored Aug 01, 2023
  
  0eb94c8b
25 Jul, 2023 1 commit
- __setitem__ allows setting key-value like dict · e0b3cbf5
  Lintang Sutawika authored Jul 25, 2023
  
  e0b3cbf5
24 Jul, 2023 3 commits
- Update evaluator.py · 0cd992b1
  Lintang Sutawika authored Jul 24, 2023
  
  0cd992b1
- Use num_fewshot set in yaml and show warning if it's being overwritten by argparse · 8add5ed6
  lintangsutawika authored Jul 24, 2023
  
  8add5ed6
- add condition if --task is not a benchmark · 2d96a8c8
  lintangsutawika authored Jul 24, 2023
  
  2d96a8c8
21 Jul, 2023 1 commit
- try to placate precommit · f4cb06b1
  Hailey Schoelkopf authored Jul 21, 2023
  
  f4cb06b1
20 Jul, 2023 1 commit
- Update evaluator.py · 310250d3
  Lintang Sutawika authored Jul 20, 2023
  
  310250d3
18 Jul, 2023 6 commits
- remove winograd_schema task type fully · 2bd45fd9
  haileyschoelkopf authored Jul 18, 2023
  
  2bd45fd9
- update format · 62bac284
  lintangsutawika authored Jul 18, 2023
  
  62bac284
- Update evaluator.py · 1f21b8ef
  Lintang Sutawika authored Jul 18, 2023
  
  1f21b8ef
- Update evaluator.py · b85ec188
  Lintang Sutawika authored Jul 18, 2023
  
  b85ec188
- Update evaluator.py · 3ab0aa5c
  Lintang Sutawika authored Jul 18, 2023
  
  3ab0aa5c
- Update evaluator.py · db9231e2
  Lintang Sutawika authored Jul 18, 2023
  
  db9231e2
17 Jul, 2023 4 commits
- add comments to all vars defined in evaluate() · 777f57ce
  haileyschoelkopf authored Jul 17, 2023
  
  777f57ce
- attach group identifier to task for aggregation · 18a7c8b1
  lintangsutawika authored Jul 17, 2023
  
  18a7c8b1
- aggregate is shown in the table · 32a70d89
  lintangsutawika authored Jul 17, 2023
  
  32a70d89
- prototype for aggregate · 3001569b
  lintangsutawika authored Jul 17, 2023
  
  3001569b
14 Jul, 2023 4 commits
- padding now handled correctly per-reqtype · abccc756
  haileyschoelkopf authored Jul 14, 2023
  
  abccc756
- add where to put aggregation · 241aa871
  lintangsutawika authored Jul 05, 2023
  
  241aa871
- don't log samples if not requested · 04c81e7d
  haileyschoelkopf authored Jul 14, 2023
  
  04c81e7d
- fix random seed issue, log_samples optional · 5fbc3f86
  haileyschoelkopf authored Jul 14, 2023
  
  5fbc3f86
13 Jul, 2023 2 commits
- log all arguments per doc · 09a71562
  haileyschoelkopf authored Jul 13, 2023
  
  09a71562
- reformat · c827e4ce
  lintangsutawika authored Jul 13, 2023
  
  c827e4ce
12 Jul, 2023 1 commit
- Update evaluator.py · 907c99c0
  Hailey Schoelkopf authored Jul 12, 2023
  
  907c99c0
06 Jul, 2023 1 commit
- bugfixes + add write_out · 4e5a328e
  haileyschoelkopf authored Jul 06, 2023
  
  4e5a328e
05 Jul, 2023 1 commit
- remove unused line in evaluator.py · 23318d0e
  haileyschoelkopf authored Jul 05, 2023
  
  23318d0e
03 Jul, 2023 2 commits
- remove output_base_path · 6a2620ad
  haileyschoelkopf authored Jul 03, 2023
  
  6a2620ad
- fix and gather sample logging · aeebf6f2
  haileyschoelkopf authored Jul 03, 2023
  
  aeebf6f2
02 Jul, 2023 1 commit
- add new task type: winograd_schema · a8396b2c
  Benjamin Fattori authored Jul 02, 2023
  
  a8396b2c
28 Jun, 2023 1 commit
- log arguments too · fb108f2e
  haileyschoelkopf authored Jun 28, 2023
  
  fb108f2e
26 Jun, 2023 1 commit
- Update evaluator.py · adef6323
  Hailey Schoelkopf authored Jun 26, 2023
  
  adef6323
23 Jun, 2023 1 commit
- add use_cache arg · 6449ab1a
  haileyschoelkopf authored Jun 23, 2023
  
  6449ab1a
22 Jun, 2023 1 commit
- more pre-commit · b48f5205
  haileyschoelkopf authored Jun 20, 2023
  
  b48f5205