Commits · 815f59e69039b7d416e2bdf35164e36ab34a705e · gaoqiong / lm-evaluation-harness

01 Nov, 2023 1 commit
- add task and group alias · dd7002d6
  lintangsutawika authored Nov 01, 2023
  
  dd7002d6
31 Oct, 2023 1 commit
- fix warning for delimiter having no whitespace · 888d3035
  haileyschoelkopf authored Oct 31, 2023
  
  888d3035
19 Oct, 2023 5 commits
- fix bug when target_delimiter is empty string · d9b4ca7f
  haileyschoelkopf authored Oct 19, 2023
  
  d9b4ca7f
- fixed registered metric · 8d4d1fa9
  lintangsutawika authored Oct 19, 2023
  
  8d4d1fa9
- changed warning message · 9fbe6eef
  lintangsutawika authored Oct 19, 2023
  
  9fbe6eef
- fix issue with default metrics and aggregation functions · 90d818da
  lintangsutawika authored Oct 19, 2023
  
  90d818da
- Changed `get_default_aggregation` to `get_aggregation` · 6b4161c1
  Lintang Sutawika authored Oct 19, 2023
  
  6b4161c1
17 Oct, 2023 1 commit
- change all mentions of `greedy_until` to `generate_until` · c64bf9a9
  lintangsutawika authored Oct 17, 2023
  
  c64bf9a9
10 Oct, 2023 1 commit
- fixes and adjustments · 0d701496
  lintangsutawika authored Oct 10, 2023
  
  0d701496
08 Oct, 2023 1 commit
- Improve error logging · 4d8094bb
  baberabb authored Oct 09, 2023
  
  4d8094bb
25 Sep, 2023 1 commit
- remove templerature as do_sample is False · 694af7d6
  lintangsutawika authored Sep 25, 2023
  
  694af7d6
22 Sep, 2023 6 commits
- Fix formatting · 8310e67e
  Chris authored Sep 22, 2023
  
  8310e67e
- Use 'is' for boolean comparison · 34d1ac6b
  Chris authored Sep 22, 2023
  
  34d1ac6b
- Cast gold to the same type as result · 5b26b3b0
  Chris authored Sep 22, 2023
  
  5b26b3b0
- Allow forced import of metrics from the HF Evaluate library · 94346b7e
  Chris authored Sep 22, 2023
  
  94346b7e
- trigger precommit · e4aa240e
  haileyschoelkopf authored Sep 22, 2023
  
  e4aa240e
- hotfix bugs · 43c91b5c
  haileyschoelkopf authored Sep 22, 2023
  
  43c91b5c
15 Sep, 2023 1 commit
- add fewshot_config field · 9d0df41b
  haileyschoelkopf authored Sep 15, 2023
  
  9d0df41b
14 Sep, 2023 3 commits
- list(gold) if `multiple_target` · 5f205bd5
  baberabb authored Sep 15, 2023
  
  5f205bd5
- add hendryck_math tasks; bugfixes · 1a34d084
  baberabb authored Sep 14, 2023
  
  1a34d084
- Refactor whitespace check and add handling for multiple_targets · bf038375
  baberabb authored Sep 14, 2023
  
  bf038375
13 Sep, 2023 2 commits
- add comments · 892f40a9
  lintangsutawika authored Sep 13, 2023
  
  892f40a9
- fixed warning · 6414d55b
  lintangsutawika authored Sep 13, 2023
  
  6414d55b
11 Sep, 2023 1 commit
- make Task._config a public property · fb436108
  haileyschoelkopf authored Sep 11, 2023
  
  fb436108
05 Sep, 2023 1 commit
- format for pre-commit · 01ad787d
  lintangsutawika authored Sep 05, 2023
  
  01ad787d
30 Aug, 2023 4 commits
- Update task.py · ebfff08f
  Lintang Sutawika authored Aug 30, 2023
  
  ebfff08f
- Update task.py · 8d1d003d
  Lintang Sutawika authored Aug 30, 2023
  
  8d1d003d
- reformat · cddbf9f6
  lintangsutawika authored Aug 30, 2023
  
  cddbf9f6
- added checks for if gold is outside choice options · 65e5bc69
  lintangsutawika authored Aug 30, 2023
  
  65e5bc69
29 Aug, 2023 1 commit
- update to format · 5172d176
  lintangsutawika authored Aug 29, 2023
  
  5172d176
27 Aug, 2023 1 commit
- format · 3768b1c4
  lintangsutawika authored Aug 27, 2023
  
  3768b1c4
26 Aug, 2023 1 commit
- add asdiv task · 69a08222
  lintangsutawika authored Aug 26, 2023
  
  69a08222
25 Aug, 2023 1 commit

Add suggestions from autotyping · fc69d84f

Ethan Smith authored Aug 25, 2023

This adds a bunch of simple annotations suggested by https://github.com/JelleZijlstra/autotyping.

fc69d84f

24 Aug, 2023 1 commit
- filter takes docs as argument in case filtering requires it · cb485883
  lintangsutawika authored Aug 24, 2023
  
  cb485883
23 Aug, 2023 3 commits
- minor fix · f5d0f8e6
  lintangsutawika authored Aug 23, 2023
  
  f5d0f8e6
- readded missing chunk · 4fe3409e
  lintangsutawika authored Aug 23, 2023
  
  4fe3409e
- fix for choices · 32294ff1
  lintangsutawika authored Aug 23, 2023
  
  32294ff1
22 Aug, 2023 3 commits
- fix test_choice test · 80d88899
  lintangsutawika authored Aug 22, 2023
  
  80d88899
- result is just 1 · 26196917
  lintangsutawika authored Aug 22, 2023
  
  26196917
- fixes to multiple_target · b1d468f2
  haileyschoelkopf authored Aug 22, 2023
  
  b1d468f2