GitLab

* add_mathbench

* update mathbench

* support non circular eval dataset

---------
Co-authored-by: liuhongwei <liuhongwei@pjlab.org.cn>
Co-authored-by: yingfhu <yingfhu@gmail.com>

* add lawbench

* update requirements

* update

* use jieba rouge in lcsts

* use rouge_chinese

* [Refactor] Move fix_id_list to Retriever

* update

* move to base

* fix

* [Dataset] add GLUE_CoLA dataset

* [update] use HFDataset to load glue/cola dataset

* update

---------
Co-authored-by: gaotongxiao <gaotongxiao@gmail.com>

* fix conflict

* add eval_cfg

* Add ToT method

* Update ToT

* Update ToT

* Update ToT

* Update ToT

* Update ToT

* Add Koashi

* Update Kaoshi

* Update Kaoshi

* Update kaoshi

* Update kaoshi

* Update Kaoshi

* Update Kaoshi

* Update Kaoshi

* Update Kaoshi

* update Kaoshi

* update

* update

* fix

---------
Co-authored-by: gaotongxiao <gaotongxiao@gmail.com>

* add evaluation of scibench

* add evaluation of scibench

* update scibench

* remove scibench evaluator

---------
Co-authored-by: Leymore <zfz-960727@163.com>

* [Feat] support anthropics ai risk dataset

* [Feat] support anthropics evals dataset

* [Feat] support anthropics evals dataset

* [Feat] support adv_glue dataset for adversarial robustness

* reorg files

* minor fix

* minor fix

* support prompt bench demo

* minor fix

* minor fix

* minor fix

* minor fix

* minor fix

* minor fix

* minor fix

* minor fix

* [Feat] support ds1000 datase

* Add CMB

* modify CMB

---------
Co-authored-by: wangxidong <xidongw@163.com>

* [Feat] support wizardcoder series

* minor fix

* add and apply update suffix tool

* add tool doc

* add qwen configs

* add cmmlu

* rename bbh

* update datasets

* delete

* update hf_qwen_7b.py

* add and apply update suffix tool

* add dataset suffix updater as precommit hook

* update workflow

* update scripts

* update ci

* update

* ci with py3.8

* run in serial

* update bbh

* use py 3.10

* update pre commit zh cn

* [Feature] Simply entry script

* update

* Add ToT method

* Update ToT

* Update ToT

* Update ToT

* Update ToT

* Update ToT

* Update ToT

* Update ToT

* Update chain_of_thought.md

* Update icl_tot_inferencer.py

---------
Co-authored-by: liuhongwei <liuhongwei@pjlab.org.cn>

Co-authored-by: wangchonghua <wangchonghua@pjlab.org.cn>

* add codegeex2

* add humanevalx dataset

* add evaluator

* update evaluator

* update configs

* update clean code

* update configs

* fix lint

* remove sleep

* fix lint

* update docs

* fix lint

* [Feat] support adv_glue dataset for adversarial robustness

* reorg files

* minor fix

* minor fix

* [Fix] fix bug for postprocessor

* minor fix

Co-authored-by: kennymckormick <dhd@pku.edu.cn>

* [Feat] update postprocessor to get first option

* minor fix

* minor fix

* add llama-oriented dataset configs

* update

* revert cvalues & update llama_example

* [Feat] add safety to collections

* minor fix

[Enhancement] Add humaneval postprocessor for GPT models & eval config for GPT4, enhance the original humaneval postprocessor (#129)

* [Enhancement] Enhance humaneval postprocessor

* add human-eval testcase

* update

* update

---------
Co-authored-by: Leymore <zfz-960727@163.com>

* add Xiezhi SQuAD2.0 ANLI; update WSC

* update

* update

* update doc string

* update agieval data

* rename variables

* [Feature] Support evaluating acc based on minimum edit distance, update SIQA

* update

* add self-consistency

* add CoT method Self-Consistency

* fix typo error and update openicl_eval

* add tydiQA-GoldP task

* fix sc

* rename gsm8k_sc

* fix sc

* add self-consistency doc

* refine sc

---------
Authored-by: liushz <qq1791167085@163.com>

* [Refactor] Update crows-pairs evaluation

* [Refactor] Update crows-pairs evaluation

* minor

* add CMMLU

* debug cmmlu

* add slurm args `qos`

* fix format: space before comment

* remove unused variable

* change the location of `answer is`

---------
Co-authored-by: 李浩楠 <lihaonan@lihaonandeMacBook-Air.local>
Co-authored-by: 李浩楠 <haonan.li>
Co-authored-by: Leymore <zfz-960727@163.com>