GitLab

* add evaluation of scibench

* add evaluation of scibench

* update scibench

* remove scibench evaluator

---------
Co-authored-by: Leymore <zfz-960727@163.com>

* Support GSM8k evaluation with tools by Lagent and LangChain

* Avoid to use MMEngine new feature

* update document

---------
Co-authored-by: Leymore <zfz-960727@163.com>

* [Fix] Visualglm performance fixed.

* [Fix] Hide ckpt path.

* [Feat] support anthropics ai risk dataset

* [Feat] support anthropics evals dataset

* [Feat] support anthropics evals dataset

* [Feat] Add public dataset support for visualglm, qwenvl, and flamingo

* [Fix] MMBench related changes.

* [Fix] Openflamingo inference.

* [Fix] Hide ckpt path.

* [Fix] Pre-commit.

---------
Co-authored-by: Haodong Duan <dhd.efz@gmail.com>

* refine gitignore

* [Feature]: Add minigpt-4

* [Feature]: Add mm local runner

* [Feature]: Add instructblip

* add otter and llama-adapter

* add owl

* add llama2-adapter and owl

* lint

* [Feature]: Add minigpt-4

* [Feature]: Add instructblip

* add otter and llama-adapter

* add owl

* add llama2-adapter and owl

* lint

* lint

* update

* lint

* lint

* add __init__.py

* update

* update

* update

* update

* [Feature]: Add minigpt-4

* [Feature]: Add mm local runner

* [Feature]: Add instructblip

* add otter and llama-adapter

* add owl

* add llama2-adapter and owl

* lint

* [Feature]: Add minigpt-4

* [Feature]: Add instructblip

* add otter and llama-adapter

* add owl

* add llama2-adapter and owl

* lint

* lint

* update

* lint

* lint

* add __init__.py

* update

* update

* update

* update

* optimize mmbench dataset args

* update

* update

* run commit hook

---------
Co-authored-by: liuyuan <3463423099@qq.com>
Co-authored-by: kennymckormick <dhd@pku.edu.cn>
Co-authored-by: kennymckormick <dhd.efz@gmail.com>

* [Feat] support adv_glue dataset for adversarial robustness

* reorg files

* minor fix

* minor fix

* support prompt bench demo

* minor fix

* minor fix

* minor fix

* minor fix

* minor fix

* minor fix

* minor fix

* minor fix

* [Feat] support ds1000 datase

Co-authored-by: bensenliu <bensenliu@tencent.com>

* Add CMB

* modify CMB

---------
Co-authored-by: wangxidong <xidongw@163.com>

* add bag of new models: baichuan2, tigerbot, vicuna v1.5

* update

* re-organize models

* update readme

* update

* add configs

* refactor model

* add post processor and prompt constructor

* [Feature] Update claude2 config

* [Feature] Update claude2 postprocessor

* [Feat] Support Qwen-VL base.

* [Feat] Support Qwen-VL-Chat on MMBench.

* [Fix] Add postprocessor and fix format.

* [Fix] Add type hint and remove redundant codes.

* [Fix] fix bugs in postprocessor.

* [Fix] Use given commit id.

* [Feat] support wizardcoder series

* minor fix

* refine gitignore

* [Feature]: Add minigpt-4

* [Feature]: Add mm local runner

* [Feature]: Add instructblip

* add otter and llama-adapter

* add owl

* add llama2-adapter and owl

* lint

* [Feature]: Add minigpt-4

* [Feature]: Add instructblip

* add otter and llama-adapter

* add owl

* add llama2-adapter and owl

* lint

* lint

* update

* lint

* lint

* add __init__.py

* update

* update

* update

---------
Co-authored-by: liuyuan <3463423099@qq.com>

* update docs

* update docs

* update

* update en docs

* update

* update

---------
Co-authored-by: Leymore <zfz-960727@163.com>

* add otter model for opencompass mmbench

* add docs

* add readme docs

* debug for otter opencomass eval

* delete unused folders

* change to default data path

* remove unused files

* remove unused files

* update

* update config file

* flake8 lint formated and add prompt generator

* add prompt generator to config

* add a specific postproecss

* add post processor

* add post processor

* add post processor

* update according to suggestions

* remove unused redefinition

* add and apply update suffix tool

* add tool doc

* add qwen configs

* add cmmlu

* rename bbh

* update datasets

* delete

* update hf_qwen_7b.py

* add and apply update suffix tool

* add dataset suffix updater as precommit hook

* update workflow

* update scripts

* update ci

* update

* ci with py3.8

* run in serial

* update bbh

* use py 3.10

* update pre commit zh cn

* [Feature] Simply entry script

* update

* [Feature] Support model-bound text postprocessor, add claude as an example

* update

* update

* minor fix

---------
Co-authored-by: zhoufengzhe <zhoufengzhe@pjlab.org.cn>

* [Feat] Add public dataset support of VisualGLM.

* [Feat] Refactor LLaVA.

* [Feat] Add public dataset support of LlaVA.

* [Fix] Add  arg.

* [Feature]: Refactor public dataset eval

* [Feature]: Verify public dataset acc

* [Feature] Add Claude support

* [Feature] Add Claude support

* Update opencompass/models/claude_api.py
Co-authored-by: Hubert <42952108+yingfhu@users.noreply.github.com>

* raise import erorr

---------
Co-authored-by: Hubert <42952108+yingfhu@users.noreply.github.com>

* [Feature]: Add Openflamingo MMBench

* [Fix]: Fix import error

* [Fix]: Revert task config

* [Fix]: Fix path bug

* refactor instructblip

* add post processor

* add forward

* fix lint

* update

* update

* Add ToT method

* Update ToT

* Update ToT

* Update ToT

* Update ToT

* Update ToT

* Update ToT

* Update ToT

* Update chain_of_thought.md

* Update icl_tot_inferencer.py

---------
Co-authored-by: liuhongwei <liuhongwei@pjlab.org.cn>

* add llama2 native implements

* rename configs/eval_llama_7b.py

---------
Co-authored-by: zhoufengzhe <zhoufengzhe@pjlab.org.cn>

* [Feat] Support visualglm inference on MMBench.

* [Feat] Support llava inference on MMBench.

* [Fix] Fix pre-commit format.

* [Fix] Add docstring for llava

* [Fix] Fix multi-process inference error of LlaVA and add comments.
1. Set `low_cpu_mem_usage` to False to address device issue.
2. Add docstring and type hints.
3. Rename class and remove registry.

* [Fix] Pre-commit fix.

* [Fix] add forward entry, add dynamic import to seedbench

* [Fix] Fix pre-commit.

* [Fix] Fix missing context.

* [Fix] Fix docstring.

* [Feat] Support multi-modal evaluation on MME benchmark.

* [Fix] Remove debug code.

* [Fix] Remove redundant codes and add type hints.

* [Fix] Rename in config.

* [Fix] Rebase main.

* [Fix] Fix isort and yapf conflict.

Co-authored-by: wangchonghua <wangchonghua@pjlab.org.cn>