GitLab

* save progress

* fixed issue with table only showing 1 group

* store aliases directly in results_agg

* removed unused parts

Co-authored-by: Paul O'Leary McCann <polm@dampfkraft.com>

* remove gooseAI

* Modify preconfig to specify commit branch

* precommit

* remove openai alias for completions

* set `--gen_kwargs` to None + add help to CLI

* add logging metavar

* fix verbosity help messages

* Reorder severity levels.

* Add IFEval task

* Check and download nltk punkt if not already downloaded

* Update gen_max_toks to 2048 to support "900 words+" instructions

* Resolve pre-commit linting issues

* Reduce max_gen_toks to 1280 to conserve token usage

* Add warning message in `process_results` call for non chat-finetuned models

* fixed syntactic nits

* fix temperature and seed

* fix logprobs

* fixup merge

* enabling OpenAI completions via gooseai

* openai-completions and pin openai

* add ignoring of no-commit-to-branch

* fix method of skipping pre-commit step

* [fix] loading dataset from hub fails when the dataset name includes '.', as the program assumes it is on the local filesystem

* add FLD benchmark

* Update task.py

* [update] add group 'fld'

* [update] rename fld -> fld_default. add explanation to the readme

* Update README.md

---------
Co-authored-by: Lintang Sutawika <lintang@sutawika.com>

* fix: passing max_length to vllm engine args

* feat: add `max_model_len`

* chore: lint