Commits · 79ce346d96b8ee62452e1d2914129c9b64419a71 · gaoqiong / lm-evaluation-harness

11 May, 2024 8 commits
- Update truthfulqa_mc2.yaml · 79ce346d
  Lintang Sutawika authored May 11, 2024
  
  79ce346d
- new group config parameter `tag_to_task` · 41e64b2e
  lintangsutawika authored May 11, 2024
  
  41e64b2e
- new mmlu config · 1960eb9a
  lintangsutawika authored May 11, 2024
  
  1960eb9a
- revert mmlu tasks · 0ffdee02
  lintangsutawika authored May 11, 2024
  
  0ffdee02
- revert truthfulqa · 39246084
  lintangsutawika authored May 11, 2024
  
  39246084
- add task_id for python tasks as well · f4d2e6e0
  lintangsutawika authored May 11, 2024
  
  f4d2e6e0
- add task_id for python tasks as well · 1ef5b0bf
  lintangsutawika authored May 11, 2024
  
  1ef5b0bf
- add task_id for python tasks as well · 78c3f7d3
  lintangsutawika authored May 11, 2024
  
  78c3f7d3
10 May, 2024 11 commits
- reformat · 0905615f
  lintangsutawika authored May 10, 2024
  
  0905615f
- use task_id · 39c40277
  lintangsutawika authored May 10, 2024
  
  39c40277
- convert all groups to configurable groups · e0986475
  lintangsutawika authored May 10, 2024
  
  e0986475
- use task id to differentiate tasks · 2f3d4272
  lintangsutawika authored May 10, 2024
  
  2f3d4272
- fix bug · 5a3a9573
  lintangsutawika authored May 10, 2024
  
  5a3a9573
- fix bug · 5c289f97
  lintangsutawika authored May 10, 2024
  
  5c289f97
- update · 719fa9b1
  lintangsutawika authored May 10, 2024
  
  719fa9b1
- fixed info log · 7350958b
  lintangsutawika authored May 10, 2024
  
  7350958b
- reformat with pre-commit · 13203943
  lintangsutawika authored May 10, 2024
  
  13203943
- update loading of task group and newly added tags · f36fb47f
  lintangsutawika authored May 10, 2024
  
  f36fb47f
- remove warning · 1fae7283
  lintangsutawika authored May 10, 2024
  
  1fae7283
08 May, 2024 5 commits
- fixed conditional statement · a88886a3
  lintangsutawika authored May 08, 2024
  
  a88886a3
- replace group to tag · 2f2322b9
  lintangsutawika authored May 08, 2024
  
  2f2322b9
- add description regarding tags replacing group · 3f770bb6
  lintangsutawika authored May 08, 2024
  
  3f770bb6
- update truthfulqa · 9f06432c
  lintangsutawika authored May 08, 2024
  
  9f06432c
- update additional condition when loading a group in a group yaml · 44d70398
  lintangsutawika authored May 08, 2024
  
  44d70398
07 May, 2024 16 commits
- reversed task list · 56373978
  lintangsutawika authored May 07, 2024
  
  56373978
- add version for groups · 2a817535
  lintangsutawika authored May 07, 2024
  
  2a817535
- sort set to False by default, fix predict_only arg · 9f698c20
  lintangsutawika authored May 07, 2024
  
  9f698c20
- move prepare_print_tasks to evaluator_utils · 110e5a28
  lintangsutawika authored May 07, 2024
  
  110e5a28
- readd files · 75dfac43
  lintangsutawika authored May 07, 2024
  
  75dfac43
- readd group_agg · 03982e03
  lintangsutawika authored May 07, 2024
  
  03982e03
- Logging Updates (Alphabetize table printouts, fix eval tracker bug) (#1774) (#1791) · d4a913c4
  Hailey Schoelkopf authored May 07, 2024
```
* fix auto-batch size bug for seq2seq models

* alphabetize task + group tables ; fix eval tracker bug

* fix eval tracker bug
```
  d4a913c4
- Re-add Hendrycks MATH (no sympy checking, no Minerva hardcoded prompt) variant (#1793) · d42a3e44
  Hailey Schoelkopf authored May 07, 2024
```
* add Hendrycks MATH (no sympy checking) variant

* add readmes for MATH tasks
```
  d42a3e44
- update to work with new group and task configuration · ad70d206
  lintangsutawika authored May 07, 2024
  
  ad70d206
- Fix Caching Tests ; Remove `pretrained=gpt2` default (#1775) · 7fe2b93c
  Hailey Schoelkopf authored May 07, 2024
  
  7fe2b93c
- update mmlu · c23c9305
  lintangsutawika authored May 07, 2024
  
  c23c9305
- update mmlu · cb085b02
  lintangsutawika authored May 07, 2024
  
  cb085b02
- adjust args · 09bc7c68
  lintangsutawika authored May 07, 2024
  
  09bc7c68
- adjust group scoring with using ConfigurableGroup · 62572f05
  lintangsutawika authored May 07, 2024
  
  62572f05
- simplify get_task_list · fe2a1472
  lintangsutawika authored May 07, 2024
  
  fe2a1472
- add configurable group · 86039e85
  lintangsutawika authored May 07, 2024
  
  86039e85