GitLab

Co-authored-by: notoschord <wangzekai@kanzhun.com>

* update medbench

* medbench update

* format medbench

* format

* Update

* update

* update

* update suffix

---------
Co-authored-by: 施晓明 <PJLAB\shixiaoming@pjnl104220118l.pjlab.org>
Co-authored-by: Leymore <zfz-960727@163.com>

* jiangjin1999: in the _batch_generate function, add the MultiTokenEOSCriteria feature to speed up inference.

* jiangjin1999: in the _batch_generate function, add the MultiTokenEOSCriteria feature to speed up inference.

---------
Co-authored-by: jiangjin08 <jiangjin08@MBP-2F32S5MD6P-0029.local>
Co-authored-by: jiangjin08 <jiangjin08@a.sh.vip.dianping.com>

* Contamination analysis for ARC_c, mmlu, and Hellaswag

* update `eval_contamination.py`

* update `contamination.py` summarizer

* fix `eval_contamination.py`

* add mmlu groups for contamination analysis

* Update installation.md

* Update installation.md

Co-authored-by: yanyuchen04 <yanyuchen04@meituan.com>

* support mbpp+

* support mbpp+

* minor fix

* [Feat] minor fix

---------
Co-authored-by: yingfhu <yingfhu@gmail.com>

* [Doc] Update Example of CompassBench

* [Doc] Update Example of CompassBench

* [Doc] Update Example of CompassBench

* update

* Update docs/zh_cn/advanced_guides/compassbench_intro.md
Co-authored-by: Fengzhe Zhou <zfz-960727@163.com>

---------
Co-authored-by: Fengzhe Zhou <zfz-960727@163.com>

* add subject ir

* Add ir dataset

* Add ir dataset

* multi_round dataset

* add multi_round evaluation

* fix small bugs

* fix small bugs

* Support LLaMA2-Accessory

* remove strip

* clear imports

* reformat

* fix lint

* fix lint

* update readme

* update readme

* update readme

* update readme

* Add NeedleInAHaystack Test

* Apply pre-commit formatting

* Update configs/eval_hf_internlm_chat_20b_cdme.py
Co-authored-by: Songyang Zhang <tonysy@users.noreply.github.com>

* add needle in haystack test

* update needle in haystack test

* update plot function in tools_needleinahaystack.py

* optimizing needleinahaystack dataset generation strategy

* modify minor formatting issues

* add English version support

* change NeedleInAHaystackDataset to dynamic loading

* change NeedleInAHaystackDataset to dynamic loading

* fix needleinahaystack test eval bug

* fix needleinahaystack config bug

---------
Co-authored-by: Songyang Zhang <tonysy@users.noreply.github.com>

* check

* message

* add

* change prompt

* change a para nameq

* modify name of the file

* delete an useless file

* add new dataset mastermath2024v1

* change it to simplified chinese prompt

* change file name

* Add NeedleInAHaystack Test

* Apply pre-commit formatting

* Update configs/eval_hf_internlm_chat_20b_cdme.py
Co-authored-by: Songyang Zhang <tonysy@users.noreply.github.com>

* add needle in haystack test

* update needle in haystack test

* update plot function in tools_needleinahaystack.py

* optimizing needleinahaystack dataset generation strategy

* modify minor formatting issues

---------
Co-authored-by: Songyang Zhang <tonysy@users.noreply.github.com>

* [Feat] update code dataset

* [Feat] update code dataset

* [Feat] update code dataset

* add chinese_version of humaneval,mbpp

* add humaneval&mbpp gen.py

* minor fix

* minor add

---------
Co-authored-by: yingfhu <yingfhu@gmail.com>

* fix subjective_eval

* subject_eval partition situation fixed

* subject_eval partition situation fixed

* add judgellm prompts

* add judgelm prompts

* update import info

* fix situation that no abbr in config

* fix situation that no abbr in config

* add summarizer for other judgellm

* change config name

* add maxlen

* add maxlen

* dict assert

* dict assert

* fix strings

* fix strings

* Update LightllmApi and Fix mmlu bug

* checkout mmlu_gen_a484b3.py

---------
Co-authored-by: Leymore <zfz-960727@163.com>

* add InfiniteBench

* add InfiniteBench

---------
Co-authored-by: wangchonghua <wangchonghua@pjlab.org.cn>

* add turbomind restful api support

* config

* top_p 0.8

* top_k = 1

* Add NeedleInAHaystack Test

* Apply pre-commit formatting

* Update configs/eval_hf_internlm_chat_20b_cdme.py
Co-authored-by: Songyang Zhang <tonysy@users.noreply.github.com>

* add needle in haystack test

* update needle in haystack test

---------
Co-authored-by: Songyang Zhang <tonysy@users.noreply.github.com>

* add news for teval

* update

* update doc for cz&en

[Feature] Update configs for evaluating chat models like qwen, baichuan, llama2 using turbomind backend (#721)

* add llama2 test

* fix

* test qwen chat-7b

* test w4

* add baichuan2

* update

* update

* update configs and docs

* update

* add_judgemodel_abbr

* add judgemodel abbr

* [Feature] Add reasonbench dataset

* add configs for supporting generative inference & merge datasets in the same category

* modify config filename to prompt version

* fix codes to meet pre-commit requirements

* lint the code to meet pre-commit requirements

* Align Load_data Sourcecode Briefly

* fix bugs

* reduce code redundancy