To help users get started quickly, we recommend the following process:
- For users who want to use VLMEvalKit, we recommend reading the "Start Your First Step" section to set up the environment and start a mini-experiment to familiarize yourself with the process.
- If you want to customize more modules, such as adding datasets and models, we provide an "Advanced Tutorial."
We always welcome users' PRs (Pull Requests) and Issues to improve VLMEvalKit!
[](https://github.com/open-compass/VLMEvalKit/stargazers)
title={VLMEvalKit: An Open-Source Toolkit for Evaluating Large Multi-Modality Models},
author={Haodong Duan and Junming Yang and Yuxuan Qiao and Xinyu Fang and Lin Chen and Yuan Liu and Xiaoyi Dong and Yuhang Zang and Pan Zhang and Jiaqi Wang and Dahua Lin and Kai Chen},
你可以通过支持 `chat_inner(message, dataset)` API 为你的模型新增多轮对话功能并兼容多轮对话评测。这个 API 输出一个字符串型回复，`message` 包含一个聊天记录的列表，格式如下：
```python
# Assume msg1, msg2, msg3, ... are multi-modal messages following the previously described format
# `chat_inner` take the following chat history list as input:
message=[
dict(role='user',content=msg1),
dict(role='assistant',content=msg2),
dict(role='user',content=msg3),
dict(role='assistant',content=msg4),
......
dict(role='user',content=msgn),
]
# `message` should contain an odd number of chat utterances, the role of utterances should be interleaved "user" and "assistant", with the role of the last utterance to be "user".
[](https://github.com/open-compass/VLMEvalKit/stargazers)
title={VLMEvalKit: An Open-Source Toolkit for Evaluating Large Multi-Modality Models},
author={Haodong Duan and Junming Yang and Yuxuan Qiao and Xinyu Fang and Lin Chen and Yuan Liu and Xiaoyi Dong and Yuhang Zang and Pan Zhang and Jiaqi Wang and Dahua Lin and Kai Chen},
