Skip to content

GitLab

Commit 17ec4906 authored by lintangsutawika's avatar lintangsutawika
Browse files

update

parent aed4e694
Hide whitespace changes
Inline Side-by-side
Showing with 5 additions and 5 deletions
lm_eval/benchmarks/__init__.py
View file @ 17ec4906
...@@ -44,7 +44,7 @@ def include_benchmarks(task_dir: str) -> None: ...@@ -44,7 +44,7 @@ def include_benchmarks(task_dir: str) -> None:
task_names = utils.pattern_match(task_list, ALL_TASKS) task_names = utils.pattern_match(task_list, ALL_TASKS)
for task in task_names: for task in task_names:
if task in TASK_REGISTRY: if (task in TASK_REGISTRY) or (task in GROUP_REGISTRY):
if group in GROUP_REGISTRY: if group in GROUP_REGISTRY:
GROUP_REGISTRY[group].append(task) GROUP_REGISTRY[group].append(task)
else: else:
......
lm_eval/benchmarks/pythia.yaml
View file @ 17ec4906
group: pythia group: pythia
task: task:
- lambada_openai - lambada_openai
- wikitext - logiqa
- piqa - piqa
- sciq - sciq
- wsc - wikitext
- winogrande - winogrande
- arc - wsc
- logiqa - ai2_arc
- blimp - blimp
- hendrycksTest* - hendrycksTest*
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment