@@ -211,9 +211,8 @@ To evaluate a model, (e.g. GPT-2) on NLU tasks (e.g. RTE, Winograd Scheme Challe
```bash
python main.py \
--model gpt2 \
--model_argsdevice=cuda:0 \
--device0 \
--tasks rte,wsc \
--provide_description\
--num_fewshot 2
```
...
...
@@ -228,6 +227,18 @@ python main.py \
--num_fewshot 2
```
Additional arguments can be provided to the model constructor using the `--model_args` flag. Most importantly, the `gpt2` model can be used to load an arbitrary HuggingFace model as follows:
```bash
python main.py \
--model gpt2 \
--model_argspretrained=EleutherAI/gpt-neo-1.3B \
--device 0 \
--tasks rte,wsc \
--num_fewshot 2
```
To inspect what the LM inputs look like, you can run the following command: