GitLab

* add and apply update suffix tool

* add tool doc

* add qwen configs

* add cmmlu

* rename bbh

* update datasets

* delete

* update hf_qwen_7b.py

* [Docs] Update doc theme

* update

* support sample count in prompt_viewer

* update

---------
Co-authored-by: Leymore <zfz-960727@163.com>

* add and apply update suffix tool

* add dataset suffix updater as precommit hook

* update workflow

* update scripts

* update ci

* update

* ci with py3.8

* run in serial

* update bbh

* use py 3.10

* update pre commit zh cn

* Update README.md

* update news and readme

* update

* update

---------
Co-authored-by: gaotongxiao <gaotongxiao@gmail.com>

* [Feature] Simply entry script

* update

* [Feature] Support model-bound text postprocessor, add claude as an example

* update

* update

* minor fix

---------
Co-authored-by: zhoufengzhe <zhoufengzhe@pjlab.org.cn>

* [Feat] Add public dataset support of VisualGLM.

* [Feat] Refactor LLaVA.

* [Feat] Add public dataset support of LlaVA.

* [Fix] Add  arg.

* [Feature]: Refactor public dataset eval

* [Feature]: Verify public dataset acc

* [Feature] Add Claude support

* [Feature] Add Claude support

* Update opencompass/models/claude_api.py
Co-authored-by: Hubert <42952108+yingfhu@users.noreply.github.com>

* raise import erorr

---------
Co-authored-by: Hubert <42952108+yingfhu@users.noreply.github.com>

* [Feature]: Add Openflamingo MMBench

* [Fix]: Fix import error

* [Fix]: Revert task config

* [Fix]: Fix path bug

* fix bugs

* fix typo

* refactor instructblip

* add post processor

* add forward

* fix lint

* update

* update

* Add ToT method

* Update ToT

* Update ToT

* Update ToT

* Update ToT

* Update ToT

* Update ToT

* Update ToT

* Update chain_of_thought.md

* Update icl_tot_inferencer.py

---------
Co-authored-by: liuhongwei <liuhongwei@pjlab.org.cn>

* add llama2 native implements

* rename configs/eval_llama_7b.py

---------
Co-authored-by: zhoufengzhe <zhoufengzhe@pjlab.org.cn>

* add summarizer doc

* update

* update doc

* Apply suggestions from code review

---------
Co-authored-by: Tong Gao <gaotongxiao@gmail.com>

* Update run.py

* Update run.py

Co-authored-by: zhangsongyang <zhangsongyang@pjlab.org.cn>

* [Feat] Support visualglm inference on MMBench.

* [Feat] Support llava inference on MMBench.

* [Fix] Fix pre-commit format.

* [Fix] Add docstring for llava

* [Fix] Fix multi-process inference error of LlaVA and add comments.
1. Set `low_cpu_mem_usage` to False to address device issue.
2. Add docstring and type hints.
3. Rename class and remove registry.

* [Fix] Pre-commit fix.

* [Fix] add forward entry, add dynamic import to seedbench

* [Fix] Fix pre-commit.

* [Fix] Fix missing context.

* [Fix] Fix docstring.

* [Feat] Support multi-modal evaluation on MME benchmark.

* [Fix] Remove debug code.

* [Fix] Remove redundant codes and add type hints.

* [Fix] Rename in config.

* [Fix] Rebase main.

* [Fix] Fix isort and yapf conflict.

Co-authored-by: wangchonghua <wangchonghua@pjlab.org.cn>

Co-authored-by: wangchonghua <wangchonghua@pjlab.org.cn>

* support seedbench

* update docstrings

* update

* update

* update

* update according to review

* rebase

* fix lint

* update

* [Feature]: Add flickr30k

* [Feature]: Add GQA

* [Feature]: Add OCR VQA

* [Feature]: Add OK VQA

* [Feature]: Add text vqa

* [Feature]: Add other vqa

* add codegeex2

* add humanevalx dataset

* add evaluator

* update evaluator

* update configs

* update clean code

* update configs

* fix lint

* remove sleep

* fix lint

* update docs

* fix lint

* [Feat] support adv_glue dataset for adversarial robustness

* reorg files

* minor fix

* minor fix