GitLab

* fix auto-batch size bug for seq2seq models

* alphabetize task + group tables ; fix eval tracker bug

* fix eval tracker bug

* add Hendrycks MATH (no sympy checking) variant

* add readmes for MATH tasks

* Added fewshot sampling seeds to evaluator.simple_evaluate signature

Way to control seed of fewshot sampling
may help with #1591

* Added ability for custom sampler for ConfigurableTask

May be set in config like
```
fewshot_config:
  sampler: !function utils.MyFewshotSampler
```

* explicitly set fewshot random generator seed for HFLM generate_until_task test

* add backward compatibility for three args seed setup

* save seeds info to logs/reports

fix `----hf_hub_log_args` to `--hf_hub_log_args`

* remove echo parameter in OpenAI completions API

* remove context length parameter doc string

* evaluation tracker implementation

* OVModelForCausalLM test fix

* typo fix

* moved methods args

* multiple args in one flag

* loggers moved to dedicated dir

* improved filename sanitization

* Add option to set OpenVINO config

* Use utils.eval_logger for logging

* vllm lora support

* remove print

* version check, rename lora kwarg

* upload new tasks

* add readmes

* run linters

---------
Co-authored-by: haileyschoelkopf <hailey@eleuther.ai>

* Update utils.py

This is a 4-choice task, option_e is null for all but 3 samples

* Fix options

Adaptive choices

* add option e

* bump multilingual arc version

---------
Co-authored-by: Hailey Schoelkopf <65563625+haileyschoelkopf@users.noreply.github.com>

* Add Pile-10k readme

* Add Pile-10k task configuration file

* Add register_filter decorator

* Add register_filter docs

* Support individual scrolls datasets

* Add qmsum context

* Fix formatting