Commits · f5bdefe84a07fb2db1fb3593d005c411eb21fcdf · gaoqiong / lm-evaluation-harness

01 Nov, 2023 2 commits
- new way to display tasks · f5bdefe8
  lintangsutawika authored Nov 01, 2023
  
  f5bdefe8
- add task and group alias · dd7002d6
  lintangsutawika authored Nov 01, 2023
  
  dd7002d6
31 Oct, 2023 1 commit
- fix warning for delimiter having no whitespace · 888d3035
  haileyschoelkopf authored Oct 31, 2023
  
  888d3035
19 Oct, 2023 8 commits
- fixes to pythia benchmark, boolqseq2seq · 1428ad57
  haileyschoelkopf authored Oct 19, 2023
  
  1428ad57
- fix bug when target_delimiter is empty string · d9b4ca7f
  haileyschoelkopf authored Oct 19, 2023
  
  d9b4ca7f
- don't call evaluate.load() every time · ff148cd8
  haileyschoelkopf authored Oct 19, 2023
  
  ff148cd8
- fixed registered metric · 8d4d1fa9
  lintangsutawika authored Oct 19, 2023
  
  8d4d1fa9
- changed warning message · 9fbe6eef
  lintangsutawika authored Oct 19, 2023
  
  9fbe6eef
- fix issue with default metrics and aggregation functions · 90d818da
  lintangsutawika authored Oct 19, 2023
  
  90d818da
- Changed `get_default_aggregation` to `get_aggregation` · 6b4161c1
  Lintang Sutawika authored Oct 19, 2023
  
  6b4161c1
- Update registry.py · a6179b49
  Lintang Sutawika authored Oct 19, 2023
```
Removed `get_default_aggregation`
```
  a6179b49
18 Oct, 2023 3 commits
- don't raise error for unset higher_is_better · e66ba123
  haileyschoelkopf authored Oct 18, 2023
  
  e66ba123
- replace the rest of the 'greedy_until' occurrences · c28d100d
  haileyschoelkopf authored Oct 18, 2023
  
  c28d100d
- fix greedy_until abstractmethod · 785153f6
  haileyschoelkopf authored Oct 18, 2023
  
  785153f6
17 Oct, 2023 8 commits
- changed file name · a7ba3d76
  lintangsutawika authored Oct 17, 2023
  
  a7ba3d76
- change all mentions of `greedy_until` to `generate_until` · c64bf9a9
  lintangsutawika authored Oct 17, 2023
  
  c64bf9a9
- added subgroups for other mmlu variants · 109ed1c7
  lintangsutawika authored Oct 17, 2023
  
  109ed1c7
- precommit stuff · a2c4139b
  lintangsutawika authored Oct 17, 2023
  
  a2c4139b
- print tasks in alphabetically · 93a45962
  lintangsutawika authored Oct 17, 2023
  
  93a45962
- pre-commit reformat · c4f0bf75
  lintangsutawika authored Oct 17, 2023
  
  c4f0bf75
- sqrt final variance calculation to get stderr · 4ccd2ec6
  lintangsutawika authored Oct 17, 2023
  
  4ccd2ec6
- template yaml · 2ade9693
  lintangsutawika authored Oct 17, 2023
  
  2ade9693
16 Oct, 2023 5 commits
- update · 11fdcb49
  lintangsutawika authored Oct 16, 2023
  
  11fdcb49
- removed comments · e97019c0
  lintangsutawika authored Oct 16, 2023
  
  e97019c0
- added stderr reprocessing for groups · 00859825
  lintangsutawika authored Oct 16, 2023
  
  00859825
- default to weighted averaging · 1dc8f96f
  lintangsutawika authored Oct 16, 2023
  
  1dc8f96f
- modfied to add subcategory · 92f25463
  lintangsutawika authored Oct 16, 2023
  
  92f25463
13 Oct, 2023 1 commit
- Fix "TypeError: 'tqdm' object is not subscriptable" error that occurs · 10625bd8
  Jason Krone authored Oct 13, 2023
```
in hugging face model loglikelihood_tokens and greedy_util functions
when batch-size is set to auto
```
  10625bd8
12 Oct, 2023 1 commit
- Update __init__.py · 32a6362c
  Lintang Sutawika authored Oct 12, 2023
  
  32a6362c
11 Oct, 2023 3 commits
- check with pre-commit · 660dfb71
  Zhiwei Zhuang authored Oct 11, 2023
  
  660dfb71
- finished test code · 2bd5dcb6
  Zhiwei Zhuang authored Oct 11, 2023
  
  2bd5dcb6
- add _batch_scheduler in greedy_until · 1aa3bc1e
  Zhiwei Zhuang authored Oct 11, 2023
  
  1aa3bc1e
10 Oct, 2023 2 commits
- error set to DEBUG · e33a7d92
  lintangsutawika authored Oct 10, 2023
  
  e33a7d92
- add verbose arg · 9894597c
  lintangsutawika authored Oct 10, 2023
  
  9894597c
08 Oct, 2023 1 commit
- Improve error logging · 4d8094bb
  baberabb authored Oct 09, 2023
  
  4d8094bb
06 Oct, 2023 2 commits
- adjustments · 7d5e511c
  lintangsutawika authored Oct 06, 2023
  
  7d5e511c
- changed default to 5-shot · 2493c6cc
  lintangsutawika authored Oct 06, 2023
  
  2493c6cc
05 Oct, 2023 3 commits
- pre-commit reformat · f2c396ab
  lintangsutawika authored Oct 05, 2023
  
  f2c396ab
- removed print messages, added cot extraction strings · 20a54b3a
  lintangsutawika authored Oct 05, 2023
  
  20a54b3a
- change : to . in the prompt · 24ecd386
  ManuelFay authored Oct 05, 2023
  
  24ecd386