GitLab

[Feature] Update configs for evaluating chat models like qwen, baichuan, llama2 using turbomind backend (#721)

* add llama2 test

* fix

* test qwen chat-7b

* test w4

* add baichuan2

* update

* update

* update configs and docs

* update

* add_judgemodel_abbr

* add judgemodel abbr

* [Feature] Add reasonbench dataset

* add configs for supporting generative inference & merge datasets in the same category

* modify config filename to prompt version

* fix codes to meet pre-commit requirements

* lint the code to meet pre-commit requirements

* Align Load_data Sourcecode Briefly

* fix bugs

* reduce code redundancy

* [Feature] Support the use of humaneval_plus.

* [Feature] Add humaneval_plus_gen.py

* minor check

* [Fix] Fix bug

---------
Co-authored-by: yingfhu <yingfhu@gmail.com>

* [Docs] update docker docs

* [Docs] update docker docs

* minor add

* minor add

* minor fix

* add judgellms

* add judgellms

* add sub_size_partition

* add docs

* add ref

* update alignmentbench

* update alignmentbench

* update doc

* update

* update

Removed redundant code in GSM8KDataset.load method.

* update alignmentbench

* update alignmentbench

* update alignmentbench

* alignmentbench infer and judge

* alignmentbench

* alignmentbench done

* alignment all done

* alignment all done

* update contamination docs

* add citation

* Update contamination_eval.md

* Update contamination_eval.md

---------
Co-authored-by: Songyang Zhang <tonysy@users.noreply.github.com>

[Feature] Add double order of subjective evaluation and removing duplicated response among two models (#692)

* add features

* add doc string

* add doc string

* support rwkv5-3b learnboard

* update rwkv-5-3b config

* update config

* refine

* fix bug

* update config

* refine

* reduce batch size

* refine

* reduce batch size to avoid oom in special datasets

* Update huggingface.py

* Update huggingface.py

Co-authored-by: Leymore <zfz-960727@163.com>

* new version of subject

* fixed draw

* fixed draw

* fixed draw

* done

* done

* done

* done

* fixed lint

* [Feature] enhance the ability of humaneval_postprocess

* refactor

* [Feature] Keep the old version of the function and realize the new function in humaneval_postprocess_v2.

* Update opencompass/datasets/humaneval.py

---------
Co-authored-by: Leymore <zfz-960727@163.com>
Co-authored-by: Hubert <42952108+yingfhu@users.noreply.github.com>

* [Feat] support ci

* [Feat] support ci

* [Feat] support ci

* [Feat] support ci

* init docs

* init docs

* init docs

* [Enhancement] Update API interface

* [Enhancement] Update API interface

* Update mixtral

* Update readme

* update medbench

* medbench update

* format medbench

* format

---------
Co-authored-by: 施晓明 <PJLAB\shixiaoming@pjnl104220118l.pjlab.org>
Co-authored-by: Leymore <zfz-960727@163.com>

* add contamination analysis to ceval

* fix bugs

* add contamination docs

* to pass CI check

* update

---------
Co-authored-by: zhangyifan1 <zhangyifan1@pjlab.org.cn>
Co-authored-by: Leymore <zfz-960727@163.com>

* TabMWP

* TabMWP

* fixed

* fixed

* fixed

* done

* done

* done

* add new subjective judgement

* add new subjective judgement

* add new subjective judgement

* add new subjective judgement

* add new subjective judgement

* modified to a more general way

* modified to a more general way

* final

* final

* add summarizer

* add new summarize

* fixed

* fixed

* fixed

---------
Co-authored-by: caomaosong <caomaosong@pjlab.org.cn>

* added rolebench

* 修改了不合理的变量名

* 修改了评论中的变量名

* Update MathBench CodeInterpreter & fix MathBench Bug

* Fix errors

* update

---------
Co-authored-by: liuhongwei <liuhongwei@pjlab.org.cn>
Co-authored-by: Fengzhe Zhou <zfz-960727@163.com>

* [Feat] update gsm8k and math agent config

* minor fix

* Add WikiBench

* Add WikiBench

* format

---------
Co-authored-by: Leymore <zfz-960727@163.com>

* add Chinese version: csqa crowspairs nq

* Update cn_data

* Update cn_data

* update format

---------
Co-authored-by: liuhongwei <liuhongwei@pjlab.org.cn>
Co-authored-by: Leymore <zfz-960727@163.com>

* [Feature] Support chat style inferencer.

* [Fix] use new prompt

* [Fix] use new prompt

---------
Co-authored-by: yingfhu <yingfhu@gmail.com>