GitLab

* Add LightllmApi KeyError log

* Update LightllmApi doc

* Update pr-run-test.yml

* Update pr-run-test.yml

* Update pr-run-test.yml

* split step and change order, change schedule time and disable hf cache

* update

* update docs

* add engine_config and gen_config in eval_config

* update

* fix

* fix

* fix

* fix docstr

* fix url

Co-authored-by: zhangyifan1 <zhangyifan1@pjlab.org.cn>

* Add NeedleInAHaystack Test

* Apply pre-commit formatting

* Update configs/eval_hf_internlm_chat_20b_cdme.py
Co-authored-by: Songyang Zhang <tonysy@users.noreply.github.com>

* add needle in haystack test

* update needle in haystack test

* update plot function in tools_needleinahaystack.py

* optimizing needleinahaystack dataset generation strategy

* modify minor formatting issues

* add English version support

* change NeedleInAHaystackDataset to dynamic loading

* change NeedleInAHaystackDataset to dynamic loading

* fix needleinahaystack test eval bug

* fix needleinahaystack config bug

* Added support for multi-needle testing in needle-in-a-haystack test

* Optimize the code for plotting in the needle-in-a-haystack test.

* Correct the typo in the dataset parameters.

* update needleinahaystack test docs

---------
Co-authored-by: Songyang Zhang <tonysy@users.noreply.github.com>

* update

* fix

* fix

* fix

* init test yaml

* add simple pr

* update

* update

* change name

* Update pr-run-test.yml

* Update pr-run-test.yml

---------
Co-authored-by: zhulin1 <zhulin1@pjlab.org.cn>

* add creationv2_zh

* add creationv2_zh

* add eng config for creationbench

* add eng config for creationbench

* add eng config for creationbench

Co-authored-by: notoschord <wangzekai@kanzhun.com>

* update medbench

* medbench update

* format medbench

* format

* Update

* update

* update

* update suffix

---------
Co-authored-by: 施晓明 <PJLAB\shixiaoming@pjnl104220118l.pjlab.org>
Co-authored-by: Leymore <zfz-960727@163.com>

* jiangjin1999: in the _batch_generate function, add the MultiTokenEOSCriteria feature to speed up inference.

* jiangjin1999: in the _batch_generate function, add the MultiTokenEOSCriteria feature to speed up inference.

---------
Co-authored-by: jiangjin08 <jiangjin08@MBP-2F32S5MD6P-0029.local>
Co-authored-by: jiangjin08 <jiangjin08@a.sh.vip.dianping.com>

* Contamination analysis for ARC_c, mmlu, and Hellaswag

* update `eval_contamination.py`

* update `contamination.py` summarizer

* fix `eval_contamination.py`

* add mmlu groups for contamination analysis

* Update installation.md

* Update installation.md

Co-authored-by: yanyuchen04 <yanyuchen04@meituan.com>

* support mbpp+

* support mbpp+

* minor fix

* [Feat] minor fix

---------
Co-authored-by: yingfhu <yingfhu@gmail.com>

* [Doc] Update Example of CompassBench

* [Doc] Update Example of CompassBench

* [Doc] Update Example of CompassBench

* update

* Update docs/zh_cn/advanced_guides/compassbench_intro.md
Co-authored-by: Fengzhe Zhou <zfz-960727@163.com>

---------
Co-authored-by: Fengzhe Zhou <zfz-960727@163.com>

* add subject ir

* Add ir dataset

* Add ir dataset

* multi_round dataset

* add multi_round evaluation

* fix small bugs

* fix small bugs

* Support LLaMA2-Accessory

* remove strip

* clear imports

* reformat

* fix lint

* fix lint

* update readme

* update readme

* update readme

* update readme

* Add NeedleInAHaystack Test

* Apply pre-commit formatting

* Update configs/eval_hf_internlm_chat_20b_cdme.py
Co-authored-by: Songyang Zhang <tonysy@users.noreply.github.com>

* add needle in haystack test

* update needle in haystack test

* update plot function in tools_needleinahaystack.py

* optimizing needleinahaystack dataset generation strategy

* modify minor formatting issues

* add English version support

* change NeedleInAHaystackDataset to dynamic loading

* change NeedleInAHaystackDataset to dynamic loading

* fix needleinahaystack test eval bug

* fix needleinahaystack config bug

---------
Co-authored-by: Songyang Zhang <tonysy@users.noreply.github.com>

* check

* message

* add

* change prompt

* change a para nameq

* modify name of the file

* delete an useless file

* add new dataset mastermath2024v1

* change it to simplified chinese prompt

* change file name

* Add NeedleInAHaystack Test

* Apply pre-commit formatting

* Update configs/eval_hf_internlm_chat_20b_cdme.py
Co-authored-by: Songyang Zhang <tonysy@users.noreply.github.com>

* add needle in haystack test

* update needle in haystack test

* update plot function in tools_needleinahaystack.py

* optimizing needleinahaystack dataset generation strategy

* modify minor formatting issues

---------
Co-authored-by: Songyang Zhang <tonysy@users.noreply.github.com>

* [Feat] update code dataset

* [Feat] update code dataset

* [Feat] update code dataset

* add chinese_version of humaneval,mbpp

* add humaneval&mbpp gen.py

* minor fix

* minor add

---------
Co-authored-by: yingfhu <yingfhu@gmail.com>

* fix subjective_eval

* subject_eval partition situation fixed

* subject_eval partition situation fixed