Commits · c4f0bf75a9c1dcf42f3564ca04cc0626ea850aec · gaoqiong / lm-evaluation-harness

17 Oct, 2023 2 commits
- pre-commit reformat · c4f0bf75
  lintangsutawika authored Oct 17, 2023
  
  c4f0bf75
- sqrt final variance calculation to get stderr · 4ccd2ec6
  lintangsutawika authored Oct 17, 2023
  
  4ccd2ec6
16 Oct, 2023 3 commits
- removed comments · e97019c0
  lintangsutawika authored Oct 16, 2023
  
  e97019c0
- added stderr reprocessing for groups · 00859825
  lintangsutawika authored Oct 16, 2023
  
  00859825
- default to weighted averaging · 1dc8f96f
  lintangsutawika authored Oct 16, 2023
  
  1dc8f96f
19 Sep, 2023 1 commit
- moved benchmarks back to tasks/ · 1ed76cfa
  lintangsutawika authored Sep 19, 2023
  
  1ed76cfa
15 Sep, 2023 1 commit
- reenable bootstrapping · 624fb207
  haileyschoelkopf authored Sep 15, 2023
  
  624fb207
13 Sep, 2023 5 commits
- add comments · 892f40a9
  lintangsutawika authored Sep 13, 2023
  
  892f40a9
- aggregates across tasks within the same group in addition to average accross task average · d2804132
  lintangsutawika authored Sep 13, 2023
  
  d2804132
- better presentation · d4f62844
  lintangsutawika authored Sep 13, 2023
  
  d4f62844
- cleaner process · e1e05b19
  lintangsutawika authored Sep 13, 2023
  
  e1e05b19
- cleaner process · 91d84840
  lintangsutawika authored Sep 13, 2023
  
  91d84840
12 Sep, 2023 3 commits
- remove comments · 993baaa6
  lintangsutawika authored Sep 12, 2023
  
  993baaa6
- update · d29c0940
  lintangsutawika authored Sep 12, 2023
  
  d29c0940
- presented with task-group relation · aed4e694
  lintangsutawika authored Sep 12, 2023
  
  aed4e694
11 Sep, 2023 1 commit
- update evaluator docstring · 0b99c7d2
  haileyschoelkopf authored Sep 11, 2023
  
  0b99c7d2
25 Aug, 2023 1 commit

Add suggestions from autotyping · fc69d84f

Ethan Smith authored Aug 25, 2023

This adds a bunch of simple annotations suggested by https://github.com/JelleZijlstra/autotyping.

fc69d84f

23 Aug, 2023 1 commit
- fix condition where a benchmark and a regular task is called together · 2f0ef6ca
  lintangsutawika authored Aug 23, 2023
  
  2f0ef6ca
17 Aug, 2023 1 commit
- improve --write_out · 4156a005
  haileyschoelkopf authored Aug 17, 2023
  
  4156a005
13 Aug, 2023 1 commit
- moved to a new dir · f4813cfc
  lintangsutawika authored Aug 13, 2023
  
  f4813cfc
11 Aug, 2023 1 commit
- support bleu score as a metric · 8806eff5
  haileyschoelkopf authored Aug 11, 2023
  
  8806eff5
08 Aug, 2023 1 commit
- fix error in a logging msg · f33d3758
  haileyschoelkopf authored Aug 08, 2023
  
  f33d3758
07 Aug, 2023 2 commits
- making t5 version of superglue prompt · 9c748204
  lintangsutawika authored Aug 07, 2023
  
  9c748204
- Update error message if task name misspelt · 7aff9b75
  baberabb authored Aug 07, 2023
  
  7aff9b75
04 Aug, 2023 2 commits
- Update evaluator.py · fa2d592a
  Lintang Sutawika authored Aug 04, 2023
  
  fa2d592a
- reformat · b3a8c985
  lintangsutawika authored Aug 04, 2023
  
  b3a8c985
01 Aug, 2023 2 commits
- update on metrics and delet files · e37698df
  lintangsutawika authored Aug 01, 2023
  
  e37698df
- process aggregate fix · 0eb94c8b
  lintangsutawika authored Aug 01, 2023
  
  0eb94c8b
25 Jul, 2023 1 commit
- __setitem__ allows setting key-value like dict · e0b3cbf5
  Lintang Sutawika authored Jul 25, 2023
  
  e0b3cbf5
24 Jul, 2023 3 commits
- Update evaluator.py · 0cd992b1
  Lintang Sutawika authored Jul 24, 2023
  
  0cd992b1
- Use num_fewshot set in yaml and show warning if it's being overwritten by argparse · 8add5ed6
  lintangsutawika authored Jul 24, 2023
  
  8add5ed6
- add condition if --task is not a benchmark · 2d96a8c8
  lintangsutawika authored Jul 24, 2023
  
  2d96a8c8
21 Jul, 2023 1 commit
- try to placate precommit · f4cb06b1
  Hailey Schoelkopf authored Jul 21, 2023
  
  f4cb06b1
20 Jul, 2023 1 commit
- Update evaluator.py · 310250d3
  Lintang Sutawika authored Jul 20, 2023
  
  310250d3
18 Jul, 2023 6 commits
- remove winograd_schema task type fully · 2bd45fd9
  haileyschoelkopf authored Jul 18, 2023
  
  2bd45fd9
- update format · 62bac284
  lintangsutawika authored Jul 18, 2023
  
  62bac284
- Update evaluator.py · 1f21b8ef
  Lintang Sutawika authored Jul 18, 2023
  
  1f21b8ef
- Update evaluator.py · b85ec188
  Lintang Sutawika authored Jul 18, 2023
  
  b85ec188
- Update evaluator.py · 3ab0aa5c
  Lintang Sutawika authored Jul 18, 2023
  
  3ab0aa5c
- Update evaluator.py · db9231e2
  Lintang Sutawika authored Jul 18, 2023
  
  db9231e2