Commit b56ec435 authored by Baber's avatar Baber
Browse files

fix cwe prefix

parent 6580aabd
include: niah_single_1.yaml include: niah_single_1.yaml
task: ruler_cwe task: ruler_cwe
download_dataset: !function cwe_utils.get_cw_dataset download_dataset: !function cwe_utils.get_cw_dataset
target_delimiter: "\n\n"
generation_kwargs: generation_kwargs:
do_sample: false do_sample: false
temperature: 0.0 temperature: 0.0
......
...@@ -22,7 +22,7 @@ from lm_eval.tasks.ruler.common_utils import DEFAULT_SEQ_LENGTHS, get_tokenizer ...@@ -22,7 +22,7 @@ from lm_eval.tasks.ruler.common_utils import DEFAULT_SEQ_LENGTHS, get_tokenizer
RNG = random.Random(42) RNG = random.Random(42)
TEMPLATE = "Below is a numbered list of words. In these words, some appear more often than others. Memorize the ones that appear most often.\n{context}\nQuestion: What are the 10 most common words in the above list?\n\nAnswer: The top 10 words that appear most often in the list are:" TEMPLATE = "Below is a numbered list of words. In these words, some appear more often than others. Memorize the ones that appear most often.\n{context}\nQuestion: What are the 10 most common words in the above list?"
r = wonderwords.RandomWord() r = wonderwords.RandomWord()
......
...@@ -114,6 +114,8 @@ def simple_parse_args_string(args_string: Optional[str]) -> dict: ...@@ -114,6 +114,8 @@ def simple_parse_args_string(args_string: Optional[str]) -> dict:
def parse_keyed_list_string(s: str) -> dict[str, list]: def parse_keyed_list_string(s: str) -> dict[str, list]:
"""Parse a string of key-value pairs into a dictionary where all values are lists.""" """Parse a string of key-value pairs into a dictionary where all values are lists."""
if s is None:
return {}
result = {} result = {}
current_key = None current_key = None
values = [] values = []
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment