Commits · cbe4ecdc07593a08809e3505886841d1c186bb22 · gaoqiong / lm-evaluation-harness

11 Sep, 2023 1 commit
- update evaluator docstring · 0b99c7d2
  haileyschoelkopf authored Sep 11, 2023
  
  0b99c7d2
25 Aug, 2023 1 commit

Add suggestions from autotyping · fc69d84f

Ethan Smith authored Aug 25, 2023

This adds a bunch of simple annotations suggested by https://github.com/JelleZijlstra/autotyping.

fc69d84f

23 Aug, 2023 1 commit
- fix condition where a benchmark and a regular task is called together · 2f0ef6ca
  lintangsutawika authored Aug 23, 2023
  
  2f0ef6ca
19 Aug, 2023 1 commit
- Implement ece, remove plotting · a6bd7126
  Herbie Bradley authored Aug 19, 2023
  
  a6bd7126
18 Aug, 2023 1 commit
- Add logiqa calibration · dadfd4a8
  Herbie Bradley authored Aug 18, 2023
  
  dadfd4a8
17 Aug, 2023 1 commit
- improve --write_out · 4156a005
  haileyschoelkopf authored Aug 17, 2023
  
  4156a005
13 Aug, 2023 1 commit
- moved to a new dir · f4813cfc
  lintangsutawika authored Aug 13, 2023
  
  f4813cfc
11 Aug, 2023 1 commit
- support bleu score as a metric · 8806eff5
  haileyschoelkopf authored Aug 11, 2023
  
  8806eff5
08 Aug, 2023 1 commit
- fix error in a logging msg · f33d3758
  haileyschoelkopf authored Aug 08, 2023
  
  f33d3758
07 Aug, 2023 2 commits
- making t5 version of superglue prompt · 9c748204
  lintangsutawika authored Aug 07, 2023
  
  9c748204
- Update error message if task name misspelt · 7aff9b75
  baberabb authored Aug 07, 2023
  
  7aff9b75
04 Aug, 2023 2 commits
- Update evaluator.py · fa2d592a
  Lintang Sutawika authored Aug 04, 2023
  
  fa2d592a
- reformat · b3a8c985
  lintangsutawika authored Aug 04, 2023
  
  b3a8c985
01 Aug, 2023 2 commits
- update on metrics and delet files · e37698df
  lintangsutawika authored Aug 01, 2023
  
  e37698df
- process aggregate fix · 0eb94c8b
  lintangsutawika authored Aug 01, 2023
  
  0eb94c8b
25 Jul, 2023 1 commit
- __setitem__ allows setting key-value like dict · e0b3cbf5
  Lintang Sutawika authored Jul 25, 2023
  
  e0b3cbf5
24 Jul, 2023 3 commits
- Update evaluator.py · 0cd992b1
  Lintang Sutawika authored Jul 24, 2023
  
  0cd992b1
- Use num_fewshot set in yaml and show warning if it's being overwritten by argparse · 8add5ed6
  lintangsutawika authored Jul 24, 2023
  
  8add5ed6
- add condition if --task is not a benchmark · 2d96a8c8
  lintangsutawika authored Jul 24, 2023
  
  2d96a8c8
21 Jul, 2023 1 commit
- try to placate precommit · f4cb06b1
  Hailey Schoelkopf authored Jul 21, 2023
  
  f4cb06b1
20 Jul, 2023 1 commit
- Update evaluator.py · 310250d3
  Lintang Sutawika authored Jul 20, 2023
  
  310250d3
18 Jul, 2023 6 commits
- remove winograd_schema task type fully · 2bd45fd9
  haileyschoelkopf authored Jul 18, 2023
  
  2bd45fd9
- update format · 62bac284
  lintangsutawika authored Jul 18, 2023
  
  62bac284
- Update evaluator.py · 1f21b8ef
  Lintang Sutawika authored Jul 18, 2023
  
  1f21b8ef
- Update evaluator.py · b85ec188
  Lintang Sutawika authored Jul 18, 2023
  
  b85ec188
- Update evaluator.py · 3ab0aa5c
  Lintang Sutawika authored Jul 18, 2023
  
  3ab0aa5c
- Update evaluator.py · db9231e2
  Lintang Sutawika authored Jul 18, 2023
  
  db9231e2
17 Jul, 2023 4 commits
- add comments to all vars defined in evaluate() · 777f57ce
  haileyschoelkopf authored Jul 17, 2023
  
  777f57ce
- attach group identifier to task for aggregation · 18a7c8b1
  lintangsutawika authored Jul 17, 2023
  
  18a7c8b1
- aggregate is shown in the table · 32a70d89
  lintangsutawika authored Jul 17, 2023
  
  32a70d89
- prototype for aggregate · 3001569b
  lintangsutawika authored Jul 17, 2023
  
  3001569b
14 Jul, 2023 4 commits
- padding now handled correctly per-reqtype · abccc756
  haileyschoelkopf authored Jul 14, 2023
  
  abccc756
- add where to put aggregation · 241aa871
  lintangsutawika authored Jul 05, 2023
  
  241aa871
- don't log samples if not requested · 04c81e7d
  haileyschoelkopf authored Jul 14, 2023
  
  04c81e7d
- fix random seed issue, log_samples optional · 5fbc3f86
  haileyschoelkopf authored Jul 14, 2023
  
  5fbc3f86
13 Jul, 2023 2 commits
- log all arguments per doc · 09a71562
  haileyschoelkopf authored Jul 13, 2023
  
  09a71562
- reformat · c827e4ce
  lintangsutawika authored Jul 13, 2023
  
  c827e4ce
12 Jul, 2023 1 commit
- Update evaluator.py · 907c99c0
  Hailey Schoelkopf authored Jul 12, 2023
  
  907c99c0
06 Jul, 2023 1 commit
- bugfixes + add write_out · 4e5a328e
  haileyschoelkopf authored Jul 06, 2023
  
  4e5a328e
05 Jul, 2023 1 commit
- remove unused line in evaluator.py · 23318d0e
  haileyschoelkopf authored Jul 05, 2023
  
  23318d0e