- 10 May, 2024 10 commits
-
-
lintangsutawika authored
-
lintangsutawika authored
-
lintangsutawika authored
-
lintangsutawika authored
-
lintangsutawika authored
-
lintangsutawika authored
-
lintangsutawika authored
-
lintangsutawika authored
-
lintangsutawika authored
-
lintangsutawika authored
-
- 08 May, 2024 6 commits
-
-
lintangsutawika authored
-
lintangsutawika authored
Merge branch 'group-agg-rework' of https://github.com/EleutherAI/lm-evaluation-harness into group-agg-rework
-
lintangsutawika authored
-
lintangsutawika authored
-
lintangsutawika authored
-
lintangsutawika authored
-
- 07 May, 2024 19 commits
-
-
lintangsutawika authored
-
lintangsutawika authored
-
lintangsutawika authored
-
lintangsutawika authored
-
lintangsutawika authored
-
lintangsutawika authored
-
lintangsutawika authored
-
Hailey Schoelkopf authored
* fix auto-batch size bug for seq2seq models * alphabetize task + group tables ; fix eval tracker bug * fix eval tracker bug
-
Hailey Schoelkopf authored
* add Hendrycks MATH (no sympy checking) variant * add readmes for MATH tasks
-
KonradSzafer authored
-
lintangsutawika authored
-
Hailey Schoelkopf authored
-
lintangsutawika authored
-
lintangsutawika authored
-
lintangsutawika authored
-
lintangsutawika authored
-
lintangsutawika authored
-
lintangsutawika authored
-
lintangsutawika authored
-
- 06 May, 2024 3 commits
-
-
aditya thomas authored
-
LSinev authored
* Added fewshot sampling seeds to evaluator.simple_evaluate signature Way to control seed of fewshot sampling may help with #1591 * Added ability for custom sampler for ConfigurableTask May be set in config like ``` fewshot_config: sampler: !function utils.MyFewshotSampler ``` * explicitly set fewshot random generator seed for HFLM generate_until_task test * add backward compatibility for three args seed setup * save seeds info to logs/reports
-
lintangsutawika authored
-
- 05 May, 2024 2 commits
-
-
ciaranby authored
-
Muhammad Bin Usman authored
fix `----hf_hub_log_args` to `--hf_hub_log_args`
-