Commit 041ea8a7 authored by Leo Gao's avatar Leo Gao
Browse files

Merge branch 'bmk_refactor' of github.com:EleutherAI/lm_evaluation_harness into bmk_refactor

parents 18281758 75bea6d5
...@@ -7,8 +7,6 @@ The goal of this project is to build a set of tools for evaluating LMs on typica ...@@ -7,8 +7,6 @@ The goal of this project is to build a set of tools for evaluating LMs on typica
2. Removing task val/test data from LM training set 2. Removing task val/test data from LM training set
3. Adding task training data to LM training set 3. Adding task training data to LM training set
The raw Google doc can be found here: https://docs.google.com/document/d/177dwJpH8GHebISXYZSn4NL98sXdCtQMH82b7O5F7jmw/edit?usp=sharing
## Usage ## Usage
### Evaluate a task ### Evaluate a task
...@@ -99,6 +97,3 @@ With the data downloader in place, we simply need to (1) expose the val/test exa ...@@ -99,6 +97,3 @@ With the data downloader in place, we simply need to (1) expose the val/test exa
### 3. Adding task training data to LM training set ### 3. Adding task training data to LM training set
This part is the easiest. I guess we just write out some text files containing the training data? We can let the usual LM preprocessing pipeline handle it from there. This part is the easiest. I guess we just write out some text files containing the training data? We can let the usual LM preprocessing pipeline handle it from there.
## Summary (need to convert from google docs at some point):
https://docs.google.com/document/d/177dwJpH8GHebISXYZSn4NL98sXdCtQMH82b7O5F7jmw/edit?usp=sharing
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment