Commit fbeaa2c1 authored by Yu Shi Jie's avatar Yu Shi Jie
Browse files

Merge branch 'mmlu-pro' of github.com:ysjprojects/lm-evaluation-harness into mmlu-pro

Resolve conflict.
parents 91b2eec6 5c7cba23
"""
"""
import re
from typing import List
......
......@@ -13,6 +13,7 @@ also determine when no answer is supported by the paragraph and abstain from ans
Homepage: https://rajpurkar.github.io/SQuAD-explorer/
"""
from functools import partial
from math import exp
......
""" This code mirrors the utils of the original winogrande task """
"""This code mirrors the utils of the original winogrande task"""
def doc_to_text(doc):
......
"""
Take in a YAML, and output all "other" splits with this YAML
"""
import argparse
import os
......
......@@ -10,7 +10,7 @@ It uses the approach described in the [GPT-3 paper](https://arxiv.org/abs/2005.1
the match, splitting the training data into chunks
3) Any chunks less than `minimum_slice_length` are removed
4) Training data sets split into more than `too_dirty_cutoff` are considered
completey contaminated and removed
completely contaminated and removed
OpenAI used:
```
......
......@@ -2,6 +2,7 @@
Usage:
python make_table_tasks.py --output <markdown_filename>
"""
import json
import logging
import os
......
......@@ -2,6 +2,7 @@
Usage:
python make_table_tasks.py --output <markdown_filename>
"""
import argparse
import logging
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment