README.md 859 Bytes
Newer Older
haileyschoelkopf's avatar
haileyschoelkopf committed
1
2
# Eval Harness Documentation

lintangsutawika's avatar
lintangsutawika committed
3
Welcome to the docs for the LM Evaluation Harness!
haileyschoelkopf's avatar
haileyschoelkopf committed
4
5
6

## Table of Contents

Baber Abbasi's avatar
Baber Abbasi committed
7
* To learn about the public interface of the library, as well as how to evaluate via the command line or as integrated into an external library, see the [Interface](./interface.md).
8
* To learn how to add a new library, API, or model type to the library, as well as a quick explainer on the types of ways to evaluate an LM, see the [Model Guide](./model_guide.md).
Baber Abbasi's avatar
Baber Abbasi committed
9
  * For an extended description of how to extend the library to new model classes served over an API, see the [API Guide](./API_guide.md).
10
11
* For a crash course on adding new tasks to the library, see our [New Task Guide](./new_task_guide.md).
* To learn more about pushing the limits of task configuration that the Eval Harness supports, see the [Task Configuration Guide](./task_guide.md).