Commits · 88fea8adb9be89c022be25e054ec2948e82d60ce · gaoqiong / lm-evaluation-harness

16 May, 2024 1 commit
- add get_subtask_list function to get proper subtask list · 88fea8ad
  lintangsutawika authored May 16, 2024
  
  88fea8ad
15 May, 2024 2 commits
- adjust task_id · c90655d5
  lintangsutawika authored May 15, 2024
  
  c90655d5
- add _process_group_config · 6d1753dc
  lintangsutawika authored May 15, 2024
  
  6d1753dc
11 May, 2024 9 commits
- reformate · aef7028c
  lintangsutawika authored May 11, 2024
  
  aef7028c
- Update truthfulqa_mc2.yaml · 79ce346d
  Lintang Sutawika authored May 11, 2024
  
  79ce346d
- new group config parameter `tag_to_task` · 41e64b2e
  lintangsutawika authored May 11, 2024
  
  41e64b2e
- new mmlu config · 1960eb9a
  lintangsutawika authored May 11, 2024
  
  1960eb9a
- revert mmlu tasks · 0ffdee02
  lintangsutawika authored May 11, 2024
  
  0ffdee02
- revert truthfulqa · 39246084
  lintangsutawika authored May 11, 2024
  
  39246084
- add task_id for python tasks as well · f4d2e6e0
  lintangsutawika authored May 11, 2024
  
  f4d2e6e0
- add task_id for python tasks as well · 1ef5b0bf
  lintangsutawika authored May 11, 2024
  
  1ef5b0bf
- add task_id for python tasks as well · 78c3f7d3
  lintangsutawika authored May 11, 2024
  
  78c3f7d3
10 May, 2024 11 commits
- reformat · 0905615f
  lintangsutawika authored May 10, 2024
  
  0905615f
- use task_id · 39c40277
  lintangsutawika authored May 10, 2024
  
  39c40277
- convert all groups to configurable groups · e0986475
  lintangsutawika authored May 10, 2024
  
  e0986475
- use task id to differentiate tasks · 2f3d4272
  lintangsutawika authored May 10, 2024
  
  2f3d4272
- fix bug · 5a3a9573
  lintangsutawika authored May 10, 2024
  
  5a3a9573
- fix bug · 5c289f97
  lintangsutawika authored May 10, 2024
  
  5c289f97
- update · 719fa9b1
  lintangsutawika authored May 10, 2024
  
  719fa9b1
- fixed info log · 7350958b
  lintangsutawika authored May 10, 2024
  
  7350958b
- reformat with pre-commit · 13203943
  lintangsutawika authored May 10, 2024
  
  13203943
- update loading of task group and newly added tags · f36fb47f
  lintangsutawika authored May 10, 2024
  
  f36fb47f
- remove warning · 1fae7283
  lintangsutawika authored May 10, 2024
  
  1fae7283
08 May, 2024 6 commits
- fixed conditional statement · a88886a3
  lintangsutawika authored May 08, 2024
  
  a88886a3
- Merge branch 'group-agg-rework' of... · c6b8132d
  lintangsutawika authored May 08, 2024
```
Merge branch 'group-agg-rework' of https://github.com/EleutherAI/lm-evaluation-harness into group-agg-rework
```
  c6b8132d
- replace group to tag · 2f2322b9
  lintangsutawika authored May 08, 2024
  
  2f2322b9
- add description regarding tags replacing group · 3f770bb6
  lintangsutawika authored May 08, 2024
  
  3f770bb6
- update truthfulqa · 9f06432c
  lintangsutawika authored May 08, 2024
  
  9f06432c
- update additional condition when loading a group in a group yaml · 44d70398
  lintangsutawika authored May 08, 2024
  
  44d70398
07 May, 2024 11 commits
- reversed task list · 56373978
  lintangsutawika authored May 07, 2024
  
  56373978
- add version for groups · 2a817535
  lintangsutawika authored May 07, 2024
  
  2a817535
- sort set to False by default, fix predict_only arg · 9f698c20
  lintangsutawika authored May 07, 2024
  
  9f698c20
- resolved merge conflict · 8d59330b
  lintangsutawika authored May 07, 2024
  
  8d59330b
- move prepare_print_tasks to evaluator_utils · 110e5a28
  lintangsutawika authored May 07, 2024
  
  110e5a28
- readd files · 75dfac43
  lintangsutawika authored May 07, 2024
  
  75dfac43
- readd group_agg · 03982e03
  lintangsutawika authored May 07, 2024
  
  03982e03
- Logging Updates (Alphabetize table printouts, fix eval tracker bug) (#1774) (#1791) · d4a913c4
  Hailey Schoelkopf authored May 07, 2024
```
* fix auto-batch size bug for seq2seq models

* alphabetize task + group tables ; fix eval tracker bug

* fix eval tracker bug
```
  d4a913c4
- Re-add Hendrycks MATH (no sympy checking, no Minerva hardcoded prompt) variant (#1793) · d42a3e44
  Hailey Schoelkopf authored May 07, 2024
```
* add Hendrycks MATH (no sympy checking) variant

* add readmes for MATH tasks
```
  d42a3e44
- link to the example output on the hub (#1798) · 20be169b
  KonradSzafer authored May 07, 2024
  
  20be169b
- update to work with new group and task configuration · ad70d206
  lintangsutawika authored May 07, 2024
  
  ad70d206