GitLab

* Add Math Evaluation with Judge Model Evaluator

* Add Math Evaluation with Judge Model Evaluator

* Add Math Evaluation with Judge Model Evaluator

* Add Math Evaluation with Judge Model Evaluator

* Fix Llama-3 meta template

* Fix MATH with JudgeLM Evaluation

* Fix MATH with JudgeLM Evaluation

* Fix MATH with JudgeLM Evaluation

* Fix MATH with JudgeLM Evaluation

---------
Co-authored-by: liuhongwei <liuhongwei@pjlab.org.cn>

* update NeedleInAHaystack Test Docs

* update docs

* Remove MultiModal

* update index.rst

* update README

* remove mmbench codes

* update news

---------
Co-authored-by: Leymore <zfz-960727@163.com>

* support arenahard

* support arenahard

* support arenahard

Co-authored-by: youmuspc <yejiayi2004@outlook.com>

* support alpacaeval_v2

* support alpacaeval

* update docs

* update docs

* add temp for mtbench

* add document for mtbench

* add document for mtbench

* fix

* update

* fix internlm1

* fix docs

* remove sys

* Add LightllmApi KeyError log

* Update LightllmApi doc

* update

* update docs

* add engine_config and gen_config in eval_config

* update

* fix

* fix

* fix

* fix docstr

* fix url

* Add NeedleInAHaystack Test

* Apply pre-commit formatting

* Update configs/eval_hf_internlm_chat_20b_cdme.py
Co-authored-by: Songyang Zhang <tonysy@users.noreply.github.com>

* add needle in haystack test

* update needle in haystack test

* update plot function in tools_needleinahaystack.py

* optimizing needleinahaystack dataset generation strategy

* modify minor formatting issues

* add English version support

* change NeedleInAHaystackDataset to dynamic loading

* change NeedleInAHaystackDataset to dynamic loading

* fix needleinahaystack test eval bug

* fix needleinahaystack config bug

* Added support for multi-needle testing in needle-in-a-haystack test

* Optimize the code for plotting in the needle-in-a-haystack test.

* Correct the typo in the dataset parameters.

* update needleinahaystack test docs

---------
Co-authored-by: Songyang Zhang <tonysy@users.noreply.github.com>

* update

* fix

* fix

* fix

* [Doc] Update Example of CompassBench

* [Doc] Update Example of CompassBench

* [Doc] Update Example of CompassBench

* update

* Update docs/zh_cn/advanced_guides/compassbench_intro.md
Co-authored-by: Fengzhe Zhou <zfz-960727@163.com>

---------
Co-authored-by: Fengzhe Zhou <zfz-960727@163.com>

* Add NeedleInAHaystack Test

* Apply pre-commit formatting

* Update configs/eval_hf_internlm_chat_20b_cdme.py
Co-authored-by: Songyang Zhang <tonysy@users.noreply.github.com>

* add needle in haystack test

* update needle in haystack test

---------
Co-authored-by: Songyang Zhang <tonysy@users.noreply.github.com>

[Feature] Update configs for evaluating chat models like qwen, baichuan, llama2 using turbomind backend (#721)

* add llama2 test

* fix

* test qwen chat-7b

* test w4

* add baichuan2

* update

* update

* update configs and docs

* update

* [Docs] update docker docs

* [Docs] update docker docs

* add judgellms

* add judgellms

* add sub_size_partition

* add docs

* add ref

* update alignmentbench

* update alignmentbench

* update doc

* update

* update

* update contamination docs

* add citation

* Update contamination_eval.md

* Update contamination_eval.md

---------
Co-authored-by: Songyang Zhang <tonysy@users.noreply.github.com>

[Feature] Add double order of subjective evaluation and removing duplicated response among two models (#692)

* add features

* add doc string

* add doc string

* new version of subject

* fixed draw

* fixed draw

* fixed draw

* done

* done

* done

* done

* fixed lint

* add contamination analysis to ceval

* fix bugs

* add contamination docs

* to pass CI check

* update

---------
Co-authored-by: zhangyifan1 <zhangyifan1@pjlab.org.cn>
Co-authored-by: Leymore <zfz-960727@163.com>

* refactor default, add circular summarizer

* add circular

* update impl

* update doc

* minor update

* no more to be added

* integrate turbomind python api

* update

* update user guide

* update

* fix according to reviewer's comments

* fix error

* fix linting

* update user guide

* remove debug log

---------
Co-authored-by: Songyang Zhang <tonysy@users.noreply.github.com>

* [Feature] Support Lightllm api

* formatting & renaming

---------
Co-authored-by: Leymore <zfz-960727@163.com>

* [Feat] support pass@ k

* [Feat] support pass@k

* [Feat] support pass@k

* [Feat] support pass@k

* [Feat] support pass@k

* [Feat] support pass@k docs

* update naming

---------
Co-authored-by: Leymore <zfz-960727@163.com>

* rename

* fix sub bugs and update docs

* update

* update

* [Docs] add humanevalx dataset link in config

* [Docs] add humanevalx dataset link in config

* minor fix

* rename

* add en subdoc

* fix name

* fix writing

* update

---------
Co-authored-by: Leymore <zfz-960727@163.com>

* support tis

* integrate turbomind inference via its RPC API instead of its python API

* update guide

* update ip address spec

* update according to reviewer's comments

* add multimodal docs

* fix lint

---------
Co-authored-by: Tong Gao <gaotongxiao@gmail.com>

* [Docs] Readme in longeval

* [Docs] Readme in longeval

* [Docs] Readme in longeval

* [Docs] Readme in longeval

* [Docs] Readme in longeval

* [Docs] Readme in longeval

* [Docs] Readme in longeval

* [Feat] support adv_glue dataset for adversarial robustness

* reorg files

* minor fix

* minor fix

* support prompt bench demo

* minor fix

* minor fix

* minor fix

* minor fix

* minor fix

* minor fix

* minor fix

* minor fix