Skip to content

GitLab

Commit 49bfaf68 authored by Baber's avatar Baber
Browse files

add docs

parent 15d07121
Hide whitespace changes
Inline Side-by-side
Showing with 14 additions and 2 deletions
lm_eval/api/task.py
View file @ 49bfaf68
...@@ -144,7 +144,15 @@ class TemplateConfig: ...@@ -144,7 +144,15 @@ class TemplateConfig:
@dataclass @dataclass
class MCQTemplateConfig: class MCQTemplateConfig:
"""Encapsulates information about a template.""" """Encapsulates information about a template.
Would return a sample with the following format:
Question: <doc_to_text(doc)>
A. <doc_to_choice(doc)[0]>
B. <doc_to_choice(doc)[1]>
C. <doc_to_choice(doc)[2]>
D. <doc_to_choice(doc)[3]>
Answer:` doc_to_choice(doc)` for each choice.
"""
doc_to_text: Union[str, Callable[[dict], str]] doc_to_text: Union[str, Callable[[dict], str]]
doc_to_choice: Union[str, list, Callable[[dict], list]] doc_to_choice: Union[str, list, Callable[[dict], list]]
...@@ -163,7 +171,11 @@ class MCQTemplateConfig: ...@@ -163,7 +171,11 @@ class MCQTemplateConfig:
@dataclass @dataclass
class ClozeTemplateConfig: class ClozeTemplateConfig:
"""Encapsulates information about a template.""" """Encapsulates information about a template.
Would return a sample with the following format:
Question: <doc_to_text(doc)>
Answer:` <doc_to_target(doc)>`
"""
doc_to_text: Union[str, Callable[[dict], str]] doc_to_text: Union[str, Callable[[dict], str]]
doc_to_choice: Union[str, list, Callable[[dict], list]] doc_to_choice: Union[str, list, Callable[[dict], list]]
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment