Commit c523063d authored by lintangsutawika's avatar lintangsutawika
Browse files

reformat

parent db0c46b0
...@@ -240,7 +240,7 @@ Tasks using complex filtering: ...@@ -240,7 +240,7 @@ Tasks using complex filtering:
## Benchmarks ## Benchmarks
When evaluating a language model, it's is not unusual to test accross a number of tasks that may not be related to one another in order to assess a variety of capabilities. To this end, it may be combursome to have to list the set of tasks or add a new group name to each yaml of each individual task. When evaluating a language model, it's is not unusual to test across a number of tasks that may not be related to one another in order to assess a variety of capabilities. To this end, it may be combursome to have to list the set of tasks or add a new group name to each yaml of each individual task.
To solve this, we can create a benchmark yaml config. This is a config that contains the names of the tasks that should be included in a particular benchmark. The config consists of two main keys `group` which denotes the name of the benchmark and `task` which is where we can list the tasks. To solve this, we can create a benchmark yaml config. This is a config that contains the names of the tasks that should be included in a particular benchmark. The config consists of two main keys `group` which denotes the name of the benchmark and `task` which is where we can list the tasks.
......
...@@ -59,5 +59,6 @@ def include_benchmarks(task_dir): ...@@ -59,5 +59,6 @@ def include_benchmarks(task_dir):
f" Error: {error}" f" Error: {error}"
) )
task_dir = os.path.dirname(os.path.abspath(__file__)) + "/" task_dir = os.path.dirname(os.path.abspath(__file__)) + "/"
include_benchmarks(task_dir) include_benchmarks(task_dir)
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment