"docs/.sphinx/requirements.txt" did not exist on "b54395481a3d89c87c26b2e592553f17dc461682"
README.md 1.12 KB
Newer Older
Jonathan Tow's avatar
Jonathan Tow committed
1
2
# datasets

jon-tow's avatar
jon-tow committed
3
This directory contains custom HuggingFace [dataset loading scripts](https://huggingface.co/docs/datasets/dataset_script). They are provided to maintain backward compatibility with the ad-hoc data downloaders in earlier versions of the `lm-evaluation-harness` before HuggingFace [`datasets`](https://huggingface.co/docs/datasets/index) was adopted as the default downloading manager. For example, some instances in the HuggingFace `datasets` repository process features (e.g. whitespace stripping, lower-casing, etc.) in ways that the `lm-evaluation-harness` did not.
Jonathan Tow's avatar
Jonathan Tow committed
4

jon-tow's avatar
jon-tow committed
5
6
7
__NOTE__: We are __not__ accepting any additional loading scripts into the main branch! If you'd like to use a custom dataset, fork the repo and follow HuggingFace's loading script guide found [here](https://huggingface.co/docs/datasets/dataset_script). You can then override your `Task`'s `DATASET_PATH` attribute to point to this script's local path.


jon-tow's avatar
jon-tow committed
8
__WARNING__: A handful of loading scripts are included in this collection because they have not yet been pushed to the Huggingface Hub or a HuggingFace organization repo. We will remove such scripts once pushed.