Commits · d2804132e2d73855ca139b3ade44bc7819664c19 · gaoqiong / lm-evaluation-harness

13 Sep, 2023 4 commits
- aggregates across tasks within the same group in addition to average accross task average · d2804132
  lintangsutawika authored Sep 13, 2023
  
  d2804132
- better presentation · d4f62844
  lintangsutawika authored Sep 13, 2023
  
  d4f62844
- cleaner process · e1e05b19
  lintangsutawika authored Sep 13, 2023
  
  e1e05b19
- cleaner process · 91d84840
  lintangsutawika authored Sep 13, 2023
  
  91d84840
12 Sep, 2023 3 commits
- remove comments · 993baaa6
  lintangsutawika authored Sep 12, 2023
  
  993baaa6
- update · d29c0940
  lintangsutawika authored Sep 12, 2023
  
  d29c0940
- presented with task-group relation · aed4e694
  lintangsutawika authored Sep 12, 2023
  
  aed4e694
25 Aug, 2023 1 commit

Add suggestions from autotyping · fc69d84f

Ethan Smith authored Aug 25, 2023

This adds a bunch of simple annotations suggested by https://github.com/JelleZijlstra/autotyping.

fc69d84f

23 Aug, 2023 1 commit
- fix condition where a benchmark and a regular task is called together · 2f0ef6ca
  lintangsutawika authored Aug 23, 2023
  
  2f0ef6ca
17 Aug, 2023 1 commit
- improve --write_out · 4156a005
  haileyschoelkopf authored Aug 17, 2023
  
  4156a005
13 Aug, 2023 1 commit
- moved to a new dir · f4813cfc
  lintangsutawika authored Aug 13, 2023
  
  f4813cfc
11 Aug, 2023 1 commit
- support bleu score as a metric · 8806eff5
  haileyschoelkopf authored Aug 11, 2023
  
  8806eff5
08 Aug, 2023 1 commit
- fix error in a logging msg · f33d3758
  haileyschoelkopf authored Aug 08, 2023
  
  f33d3758
07 Aug, 2023 2 commits
- making t5 version of superglue prompt · 9c748204
  lintangsutawika authored Aug 07, 2023
  
  9c748204
- Update error message if task name misspelt · 7aff9b75
  baberabb authored Aug 07, 2023
  
  7aff9b75
04 Aug, 2023 2 commits
- Update evaluator.py · fa2d592a
  Lintang Sutawika authored Aug 04, 2023
  
  fa2d592a
- reformat · b3a8c985
  lintangsutawika authored Aug 04, 2023
  
  b3a8c985
01 Aug, 2023 2 commits
- update on metrics and delet files · e37698df
  lintangsutawika authored Aug 01, 2023
  
  e37698df
- process aggregate fix · 0eb94c8b
  lintangsutawika authored Aug 01, 2023
  
  0eb94c8b
25 Jul, 2023 1 commit
- __setitem__ allows setting key-value like dict · e0b3cbf5
  Lintang Sutawika authored Jul 25, 2023
  
  e0b3cbf5
24 Jul, 2023 3 commits
- Update evaluator.py · 0cd992b1
  Lintang Sutawika authored Jul 24, 2023
  
  0cd992b1
- Use num_fewshot set in yaml and show warning if it's being overwritten by argparse · 8add5ed6
  lintangsutawika authored Jul 24, 2023
  
  8add5ed6
- add condition if --task is not a benchmark · 2d96a8c8
  lintangsutawika authored Jul 24, 2023
  
  2d96a8c8
21 Jul, 2023 1 commit
- try to placate precommit · f4cb06b1
  Hailey Schoelkopf authored Jul 21, 2023
  
  f4cb06b1
20 Jul, 2023 1 commit
- Update evaluator.py · 310250d3
  Lintang Sutawika authored Jul 20, 2023
  
  310250d3
18 Jul, 2023 6 commits
- remove winograd_schema task type fully · 2bd45fd9
  haileyschoelkopf authored Jul 18, 2023
  
  2bd45fd9
- update format · 62bac284
  lintangsutawika authored Jul 18, 2023
  
  62bac284
- Update evaluator.py · 1f21b8ef
  Lintang Sutawika authored Jul 18, 2023
  
  1f21b8ef
- Update evaluator.py · b85ec188
  Lintang Sutawika authored Jul 18, 2023
  
  b85ec188
- Update evaluator.py · 3ab0aa5c
  Lintang Sutawika authored Jul 18, 2023
  
  3ab0aa5c
- Update evaluator.py · db9231e2
  Lintang Sutawika authored Jul 18, 2023
  
  db9231e2
17 Jul, 2023 4 commits
- add comments to all vars defined in evaluate() · 777f57ce
  haileyschoelkopf authored Jul 17, 2023
  
  777f57ce
- attach group identifier to task for aggregation · 18a7c8b1
  lintangsutawika authored Jul 17, 2023
  
  18a7c8b1
- aggregate is shown in the table · 32a70d89
  lintangsutawika authored Jul 17, 2023
  
  32a70d89
- prototype for aggregate · 3001569b
  lintangsutawika authored Jul 17, 2023
  
  3001569b
14 Jul, 2023 4 commits
- padding now handled correctly per-reqtype · abccc756
  haileyschoelkopf authored Jul 14, 2023
  
  abccc756
- add where to put aggregation · 241aa871
  lintangsutawika authored Jul 05, 2023
  
  241aa871
- don't log samples if not requested · 04c81e7d
  haileyschoelkopf authored Jul 14, 2023
  
  04c81e7d
- fix random seed issue, log_samples optional · 5fbc3f86
  haileyschoelkopf authored Jul 14, 2023
  
  5fbc3f86
13 Jul, 2023 1 commit
- log all arguments per doc · 09a71562
  haileyschoelkopf authored Jul 13, 2023
  
  09a71562