GitLab

[bugfix] openai

Add `inject_fused_attention` parameter for `auto-gptq`

Add NQ-Open task based on the Natural Questions dataset

Add CMMLU Benchmark

Update crowspairs.py higher_is_better

Update triviaqa.py

Removes the `_remove_prefixes` optimization as it's invalid for exact match metrics. Closes https://github.com/EleutherAI/lm-evaluation-harness/issues/726

[Main] updated to new anthropic API

add bnb_4bit_use_double_quant and low_cpu_mem_usage

Remove Results folder temporarily

add haerae

Update setup.py

Running `main.py` yields the following:

```
ImportError: This modeling file requires the following packages that were not found in your environment: einops. Run `pip install einops`
```

This commit should fix this.

Ceval