Commits · e5b245cc70c885e203cd24ca2a637404d3df71db · gaoqiong / lm-evaluation-harness

28 Dec, 2023 4 commits
- remove aggregation · e5b245cc
  lintangsutawika authored Dec 28, 2023
  
  e5b245cc
- removed passthrough fn · 039832e5
  lintangsutawika authored Dec 28, 2023
  
  039832e5
- simplify registry · 3888193d
  lintangsutawika authored Dec 28, 2023
  
  3888193d
- aggregation to compute_metric · 9d6bc929
  lintangsutawika authored Dec 28, 2023
  
  9d6bc929
27 Dec, 2023 1 commit
- update · c6a91582
  lintangsutawika authored Dec 27, 2023
  
  c6a91582
19 Dec, 2023 3 commits
- loglikelihood and loglikelihood rolling modified · 028f04c7
  lintangsutawika authored Dec 19, 2023
  
  028f04c7
- change how metrics are registered · 1d262a59
  lintangsutawika authored Dec 19, 2023
  
  1d262a59
- sample metrics that have both sample-wise and set-wise operations · e7cd7d68
  lintangsutawika authored Dec 19, 2023
  
  e7cd7d68
15 Dec, 2023 1 commit
- fixed how to check if dataset_path is a local directory or not (#1127) · 04707a2d
  Lintang Sutawika authored Dec 15, 2023
  
  04707a2d
14 Dec, 2023 2 commits

doc_to_decontamination_query can use function (#1082) · fcb39a5a

Lintang Sutawika authored Dec 14, 2023

* doc_to_decontamination_query can use function

* add option for doc_to_decontamination_query to follow doc_to_text

* added documentation for doc_to_decontamination_query

* adjust description

* format

fcb39a5a

Additional process for doc_to_choice (#1093) · a2ed953f
Lintang Sutawika authored Dec 14, 2023
```
* Additional process for doc_to_choice

* doc_to_choice can also parse a string
```
a2ed953f

04 Dec, 2023 1 commit
- Update samplers.py · 3512f742
  Hailey Schoelkopf authored Dec 04, 2023
  
  3512f742
29 Nov, 2023 2 commits
- fixed sampler issue with new default num_fewshot value; n_shot value changed · 38ac7b2a
  baberabb authored Nov 29, 2023
  
  38ac7b2a
- change torch req for mps · 7bb147b5
  baberabb authored Nov 29, 2023
  
  7bb147b5
28 Nov, 2023 3 commits
- reformat · 37a46351
  lintangsutawika authored Nov 28, 2023
  
  37a46351
- add versions · 0d03a9f3
  lintangsutawika authored Nov 28, 2023
  
  0d03a9f3
- fixes how num_fewshot from config is read · 9e65d0ce
  lintangsutawika authored Nov 28, 2023
  
  9e65d0ce
17 Nov, 2023 1 commit
- removed provide_description arg · 0d209e2e
  lintangsutawika authored Nov 17, 2023
  
  0d209e2e
16 Nov, 2023 1 commit
- remove provide_description flag · a745d589
  haileyschoelkopf authored Nov 16, 2023
  
  a745d589
09 Nov, 2023 1 commit
- reformat · 9a64e642
  lintangsutawika authored Nov 09, 2023
  
  9a64e642
07 Nov, 2023 2 commits
- moved squad.py and format changes · e1d5c849
  lintangsutawika authored Nov 07, 2023
  
  e1d5c849
- add squad from master · 8bf55a20
  lintangsutawika authored Nov 07, 2023
  
  8bf55a20
03 Nov, 2023 1 commit
- remove gold_alias from codebase · 37ac5f46
  haileyschoelkopf authored Nov 03, 2023
  
  37ac5f46
02 Nov, 2023 2 commits
- precommit format · 73f3029c
  lintangsutawika authored Nov 02, 2023
  
  73f3029c
- eval_logger is not imported from logger.py anymore · f701ba7d
  lintangsutawika authored Nov 02, 2023
  
  f701ba7d
01 Nov, 2023 1 commit
- add task and group alias · dd7002d6
  lintangsutawika authored Nov 01, 2023
  
  dd7002d6
31 Oct, 2023 1 commit
- fix warning for delimiter having no whitespace · 888d3035
  haileyschoelkopf authored Oct 31, 2023
  
  888d3035
19 Oct, 2023 7 commits
- fix bug when target_delimiter is empty string · d9b4ca7f
  haileyschoelkopf authored Oct 19, 2023
  
  d9b4ca7f
- don't call evaluate.load() every time · ff148cd8
  haileyschoelkopf authored Oct 19, 2023
  
  ff148cd8
- fixed registered metric · 8d4d1fa9
  lintangsutawika authored Oct 19, 2023
  
  8d4d1fa9
- changed warning message · 9fbe6eef
  lintangsutawika authored Oct 19, 2023
  
  9fbe6eef
- fix issue with default metrics and aggregation functions · 90d818da
  lintangsutawika authored Oct 19, 2023
  
  90d818da
- Changed `get_default_aggregation` to `get_aggregation` · 6b4161c1
  Lintang Sutawika authored Oct 19, 2023
  
  6b4161c1
- Update registry.py · a6179b49
  Lintang Sutawika authored Oct 19, 2023
```
Removed `get_default_aggregation`
```
  a6179b49
18 Oct, 2023 3 commits
- don't raise error for unset higher_is_better · e66ba123
  haileyschoelkopf authored Oct 18, 2023
  
  e66ba123
- replace the rest of the 'greedy_until' occurrences · c28d100d
  haileyschoelkopf authored Oct 18, 2023
  
  c28d100d
- fix greedy_until abstractmethod · 785153f6
  haileyschoelkopf authored Oct 18, 2023
  
  785153f6
17 Oct, 2023 1 commit
- change all mentions of `greedy_until` to `generate_until` · c64bf9a9
  lintangsutawika authored Oct 17, 2023
  
  c64bf9a9
10 Oct, 2023 1 commit
- fixes and adjustments · 0d701496
  lintangsutawika authored Oct 10, 2023
  
  0d701496
08 Oct, 2023 1 commit
- Improve error logging · 4d8094bb
  baberabb authored Oct 09, 2023
  
  4d8094bb