GitLab

* multi_round dataset

* add multi_round evaluation

* fix small bugs

* fix small bugs

* Support LLaMA2-Accessory

* remove strip

* clear imports

* reformat

* fix lint

* fix lint

* update readme

* update readme

* update readme

* update readme

* Add NeedleInAHaystack Test

* Apply pre-commit formatting

* Update configs/eval_hf_internlm_chat_20b_cdme.py
Co-authored-by: Songyang Zhang <tonysy@users.noreply.github.com>

* add needle in haystack test

* update needle in haystack test

* update plot function in tools_needleinahaystack.py

* optimizing needleinahaystack dataset generation strategy

* modify minor formatting issues

* add English version support

* change NeedleInAHaystackDataset to dynamic loading

* change NeedleInAHaystackDataset to dynamic loading

* fix needleinahaystack test eval bug

* fix needleinahaystack config bug

---------
Co-authored-by: Songyang Zhang <tonysy@users.noreply.github.com>

* check

* message

* add

* change prompt

* change a para nameq

* modify name of the file

* delete an useless file

* add new dataset mastermath2024v1

* change it to simplified chinese prompt

* change file name

* Add NeedleInAHaystack Test

* Apply pre-commit formatting

* Update configs/eval_hf_internlm_chat_20b_cdme.py
Co-authored-by: Songyang Zhang <tonysy@users.noreply.github.com>

* add needle in haystack test

* update needle in haystack test

* update plot function in tools_needleinahaystack.py

* optimizing needleinahaystack dataset generation strategy

* modify minor formatting issues

---------
Co-authored-by: Songyang Zhang <tonysy@users.noreply.github.com>

* [Feat] update code dataset

* [Feat] update code dataset

* [Feat] update code dataset

* add chinese_version of humaneval,mbpp

* add humaneval&mbpp gen.py

* minor fix

* minor add

---------
Co-authored-by: yingfhu <yingfhu@gmail.com>

* fix subjective_eval

* subject_eval partition situation fixed

* subject_eval partition situation fixed

* add judgellm prompts

* add judgelm prompts

* update import info

* fix situation that no abbr in config

* fix situation that no abbr in config

* add summarizer for other judgellm

* change config name

* add maxlen

* add maxlen

* dict assert

* dict assert

* fix strings

* fix strings

* Update LightllmApi and Fix mmlu bug

* checkout mmlu_gen_a484b3.py

---------
Co-authored-by: Leymore <zfz-960727@163.com>

* add InfiniteBench

* add InfiniteBench

---------
Co-authored-by: wangchonghua <wangchonghua@pjlab.org.cn>

* add turbomind restful api support

* config

* top_p 0.8

* top_k = 1

* Add NeedleInAHaystack Test

* Apply pre-commit formatting

* Update configs/eval_hf_internlm_chat_20b_cdme.py
Co-authored-by: Songyang Zhang <tonysy@users.noreply.github.com>

* add needle in haystack test

* update needle in haystack test

---------
Co-authored-by: Songyang Zhang <tonysy@users.noreply.github.com>

* add news for teval

* update

* update doc for cz&en

[Feature] Update configs for evaluating chat models like qwen, baichuan, llama2 using turbomind backend (#721)

* add llama2 test

* fix

* test qwen chat-7b

* test w4

* add baichuan2

* update

* update

* update configs and docs

* update

* add_judgemodel_abbr

* add judgemodel abbr

* [Feature] Add reasonbench dataset

* add configs for supporting generative inference & merge datasets in the same category

* modify config filename to prompt version

* fix codes to meet pre-commit requirements

* lint the code to meet pre-commit requirements

* Align Load_data Sourcecode Briefly

* fix bugs

* reduce code redundancy

* [Feature] Support the use of humaneval_plus.

* [Feature] Add humaneval_plus_gen.py

* minor check

* [Fix] Fix bug

---------
Co-authored-by: yingfhu <yingfhu@gmail.com>

* [Docs] update docker docs

* [Docs] update docker docs

* minor add

* minor add

* minor fix

* add judgellms

* add judgellms

* add sub_size_partition

* add docs

* add ref

* update alignmentbench

* update alignmentbench

* update doc

* update

* update

Removed redundant code in GSM8KDataset.load method.

* update alignmentbench

* update alignmentbench

* update alignmentbench

* alignmentbench infer and judge

* alignmentbench

* alignmentbench done

* alignment all done

* alignment all done

* update contamination docs

* add citation

* Update contamination_eval.md

* Update contamination_eval.md

---------
Co-authored-by: Songyang Zhang <tonysy@users.noreply.github.com>

[Feature] Add double order of subjective evaluation and removing duplicated response among two models (#692)

* add features

* add doc string

* add doc string