Unverified Commit 93ea091e authored by Stella Biderman's avatar Stella Biderman Committed by GitHub
Browse files

Updated readme for clarity

parent af0b1e26
......@@ -13,7 +13,7 @@ The goal of this project is to build a set of tools for evaluating LMs on typica
2. Removing task val/test data from LM training set
3. Adding task training data to LM training set
### Overview of Tasks
### Full Task List
| Task Name |Train|Val|Test|Val/Test Docs| Metrics |
|-------------------------------------------------|-----|---|----|------------:|------------------------------------------------------------------------------|
......@@ -211,9 +211,8 @@ To evaluate a model, (e.g. GPT-2) on NLU tasks (e.g. RTE, Winograd Scheme Challe
```bash
python main.py \
--model gpt2 \
--model_args device=cuda:0 \
--device 0 \
--tasks rte,wsc \
--provide_description \
--num_fewshot 2
```
......@@ -228,6 +227,18 @@ python main.py \
--num_fewshot 2
```
Additional arguments can be provided to the model constructor using the `--model_args` flag. Most importantly, the `gpt2` model can be used to load an arbitrary HuggingFace model as follows:
```bash
python main.py \
--model gpt2 \
--model_args pretrained=EleutherAI/gpt-neo-1.3B \
--device 0 \
--tasks rte,wsc \
--num_fewshot 2
```
To inspect what the LM inputs look like, you can run the following command:
```bash
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment