GitLab

* hotfix for mtbench

* hotfix

* add do sample demo for subjective dataset

* fix strings

* format

---------
Co-authored-by: Leymore <zfz-960727@163.com>

* support NPHardEval

* add .md file and fix minor bugs

* refactor and minor fix

---------
Co-authored-by: Leymore <zfz-960727@163.com>

* support alpacaeval_v1

* Update opencompass/summarizers/subjective/__init__.py
Co-authored-by: Songyang Zhang <tonysy@users.noreply.github.com>

* Update opencompass/summarizers/subjective/alpacaeval_v1.py
Co-authored-by: Songyang Zhang <tonysy@users.noreply.github.com>

* fix conflict

* support alpacaeval v2

* support alpacav2

---------
Co-authored-by: Songyang Zhang <tonysy@users.noreply.github.com>

* fix

* update

* fix internlm1

* fix docs

* remove sys

* Update zhipu api and fix min_out_len issue of API class

* Update example

* Update example

* [Feature] Add IFEval

* [Fix] Changing the Score Rule.

* [Feat] update cibench

* [Feat] Support CIBench

* [Feat] Support CIBench

* [Feat] Support CIBench

* [Feat] Support CIBench

* fix corev2

* fix corev2

Co-authored-by: liuhongwei <liuhongwei@pjlab.org.cn>

* add mtbench

* add mtbench

* Update configs/datasets/subjective/multiround/mtbench_judgeby_gpt4.py
Co-authored-by: Songyang Zhang <tonysy@users.noreply.github.com>

* Update configs/datasets/subjective/multiround/mtbench_judgeby_gpt4.py
Co-authored-by: Songyang Zhang <tonysy@users.noreply.github.com>

* Update opencompass/datasets/subjective/__init__.py
Co-authored-by: Songyang Zhang <tonysy@users.noreply.github.com>

* Update opencompass/datasets/subjective/mtbench.py
Co-authored-by: Songyang Zhang <tonysy@users.noreply.github.com>

* fix mtbench

---------
Co-authored-by: Songyang Zhang <tonysy@users.noreply.github.com>

* [Feature] Add IFEval

* [Doc] add introduction of IFEval

* add compass arena

* add compass_arena

* add compass arena

* Update opencompass/summarizers/subjective/compass_arena.py
Co-authored-by: Songyang Zhang <tonysy@users.noreply.github.com>

* Update opencompass/summarizers/subjective/__init__.py
Co-authored-by: Songyang Zhang <tonysy@users.noreply.github.com>

* Update opencompass/datasets/subjective/compass_arena.py
Co-authored-by: Songyang Zhang <tonysy@users.noreply.github.com>

* Update opencompass/datasets/subjective/__init__.py
Co-authored-by: Songyang Zhang <tonysy@users.noreply.github.com>

* Update configs/eval_subjective_compassarena.py
Co-authored-by: Songyang Zhang <tonysy@users.noreply.github.com>

* Update configs/datasets/subjective/compassarena/compassarena_compare.py
Co-authored-by: Songyang Zhang <tonysy@users.noreply.github.com>

* Update configs/eval_subjective_compassarena.py
Co-authored-by: Songyang Zhang <tonysy@users.noreply.github.com>

* Update configs/datasets/subjective/compassarena/compassarena_compare.py
Co-authored-by: Songyang Zhang <tonysy@users.noreply.github.com>

* fix check position bias

---------
Co-authored-by: Songyang Zhang <tonysy@users.noreply.github.com>

* Update hf_internlm2_chat template

* Update 20B

* update cdme config and evaluator

* fix cdme prompt

* move CDME trim post-processor as a separate evaluator

---------
Co-authored-by: 郭琦鹏 <guoqipeng@pjlab.org.cn>

* update

* update docs

* add engine_config and gen_config in eval_config

* update

* fix

* fix

* fix

* fix docstr

* fix url

Co-authored-by: zhangyifan1 <zhangyifan1@pjlab.org.cn>

* Add NeedleInAHaystack Test

* Apply pre-commit formatting

* Update configs/eval_hf_internlm_chat_20b_cdme.py
Co-authored-by: Songyang Zhang <tonysy@users.noreply.github.com>

* add needle in haystack test

* update needle in haystack test

* update plot function in tools_needleinahaystack.py

* optimizing needleinahaystack dataset generation strategy

* modify minor formatting issues

* add English version support

* change NeedleInAHaystackDataset to dynamic loading

* change NeedleInAHaystackDataset to dynamic loading

* fix needleinahaystack test eval bug

* fix needleinahaystack config bug

* Added support for multi-needle testing in needle-in-a-haystack test

* Optimize the code for plotting in the needle-in-a-haystack test.

* Correct the typo in the dataset parameters.

* update needleinahaystack test docs

---------
Co-authored-by: Songyang Zhang <tonysy@users.noreply.github.com>

* update

* fix

* fix

* fix

* add creationv2_zh

* add creationv2_zh

* add eng config for creationbench

* add eng config for creationbench

* add eng config for creationbench

Co-authored-by: notoschord <wangzekai@kanzhun.com>

* update medbench

* medbench update

* format medbench

* format

* Update

* update

* update

* update suffix

---------
Co-authored-by: 施晓明 <PJLAB\shixiaoming@pjnl104220118l.pjlab.org>
Co-authored-by: Leymore <zfz-960727@163.com>

* Contamination analysis for ARC_c, mmlu, and Hellaswag

* update `eval_contamination.py`

* update `contamination.py` summarizer

* fix `eval_contamination.py`

* add mmlu groups for contamination analysis

Co-authored-by: yanyuchen04 <yanyuchen04@meituan.com>

* support mbpp+

* support mbpp+

* minor fix

* [Feat] minor fix

---------
Co-authored-by: yingfhu <yingfhu@gmail.com>

* add subject ir

* Add ir dataset

* Add ir dataset

* multi_round dataset

* add multi_round evaluation

* Support LLaMA2-Accessory

* remove strip

* clear imports

* reformat

* fix lint

* fix lint

* update readme

* update readme

* update readme

* update readme

* Add NeedleInAHaystack Test

* Apply pre-commit formatting

* Update configs/eval_hf_internlm_chat_20b_cdme.py
Co-authored-by: Songyang Zhang <tonysy@users.noreply.github.com>

* add needle in haystack test

* update needle in haystack test

* update plot function in tools_needleinahaystack.py

* optimizing needleinahaystack dataset generation strategy

* modify minor formatting issues

* add English version support

* change NeedleInAHaystackDataset to dynamic loading

* change NeedleInAHaystackDataset to dynamic loading

* fix needleinahaystack test eval bug

* fix needleinahaystack config bug

---------
Co-authored-by: Songyang Zhang <tonysy@users.noreply.github.com>

* check

* message

* add

* change prompt

* change a para nameq

* modify name of the file

* delete an useless file