GitLab

* modularize HFLM code

* pass through extra kwargs to AutoModel.from_pretrained call

* remove explicit model_kwargs

* rename gptq -> autogptq

* fix tokenizer pad token errors

* ensure model always respects device_map and autogptq's selected devices

* add a _get_config helper fn

fix: bug of BBH_cot_fewshot

* remove unlabled test sets

* add note to readme

* unpack group; add output_path to arg

* Add `vllm` to overview

[Refactor] vllm data parallel

Simplified `evaluator.py`

Fix stderr

Add kmmlu evaluation to tasks

Change the sub-task name from sst to sst2 in glue

Update template of qqp dataset

Patch scrolls