Skip to content

GitLab

"vscode:/vscode.git/clone" did not exist on "c8693599c30c254ab4e4b47d0295c1aeec2e71fc"
Switch branch/tag
Find file Normal viewHistoryPermalink
README.md 5.3 KB
Newer Older
wanglch's avatar
Initial commit  
wanglch committed
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95 
# README for Evaluation

Here, we list the codebase we used to obtain the evaluation results in the InternVL 2.5 technical report.

## Multimodal Reasoning and Mathematics

| Benchmark Name | Codebase                                                 |
| -------------- | -------------------------------------------------------- |
| MMMU           | [VLMEvalKit](https://github.com/open-compass/VLMEvalKit) |
| MMMU-Pro       | [This Codebase](./mmmu_pro)                              |
| MathVista      | [VLMEvalKit](https://github.com/open-compass/VLMEvalKit) |
| MATH-Vision    | [VLMEvalKit](https://github.com/open-compass/VLMEvalKit) |
| MathVerse      | [VLMEvalKit](https://github.com/open-compass/VLMEvalKit) |
| OlympiadBench  | [VLMEvalKit](https://github.com/open-compass/VLMEvalKit) |

## Multimodal Reasoning and Mathematics

| Benchmark Name    | Codebase                                                 |
| ----------------- | -------------------------------------------------------- |
| AI2D with mask    | [This Codebase](./vqa)                                   |
| AI2D without mask | [VLMEvalKit](https://github.com/open-compass/VLMEvalKit) |
| ChartQA           | [This Codebase](./vqa)                                   |
| DocVQA            | [This Codebase](./vqa)                                   |
| InfoVQA           | [This Codebase](./vqa)                                   |
| OCRBench          | [VLMEvalKit](https://github.com/open-compass/VLMEvalKit) |
| SEED-2-Plus       | [VLMEvalKit](https://github.com/open-compass/VLMEvalKit) |
| CharXiv           | [CharXiv](https://github.com/princeton-nlp/CharXiv)      |
| VCR               | [VLMEvalKit](https://github.com/open-compass/VLMEvalKit) |

## Multi-Image Understanding

| Benchmark Name | Codebase                                                 |
| -------------- | -------------------------------------------------------- |
| BLINK          | [VLMEvalKit](https://github.com/open-compass/VLMEvalKit) |
| Mantis Eval    | [This Codebase](./mantis_eval)                           |
| MMIU           | [This Codebase](./mmiu)                                  |
| MuirBench      | [VLMEvalKit](https://github.com/open-compass/VLMEvalKit) |
| MMT-Bench      | [VLMEvalKit](https://github.com/open-compass/VLMEvalKit) |
| MIRB           | [This Codebase](./mirb)                                  |

## Real-World Comprehension

| Benchmark Name | Codebase                                                 |
| -------------- | -------------------------------------------------------- |
| RealWorldQA    | [VLMEvalKit](https://github.com/open-compass/VLMEvalKit) |
| MME-RealWorld  | [VLMEvalKit](https://github.com/open-compass/VLMEvalKit) |
| WildVision     | [VLMEvalKit](https://github.com/open-compass/VLMEvalKit) |
| R-Bench        | [VLMEvalKit](https://github.com/open-compass/VLMEvalKit) |

## Comprehensive Multimodal Evaluation

| Benchmark Name | Codebase                                                 |
| -------------- | -------------------------------------------------------- |
| MME            | [This Codebase](./mme)                                   |
| MMBench        | [This Codebase](./mmbench)                               |
| MMBench v1.1   | [VLMEvalKit](https://github.com/open-compass/VLMEvalKit) |
| MMVet          | [VLMEvalKit](https://github.com/open-compass/VLMEvalKit) |
| MMVet v2       | [This Codebase](./mmvetv2)                               |
| MMStar         | [VLMEvalKit](https://github.com/open-compass/VLMEvalKit) |

## Multimodal Hallucination Evaluation

| Benchmark Name | Codebase                                                 |
| -------------- | -------------------------------------------------------- |
| HallBench      | [VLMEvalKit](https://github.com/open-compass/VLMEvalKit) |
| MMHal-Bench    | [This Codebase](./mmhal)                                 |
| CRPE           | [VLMEvalKit](https://github.com/open-compass/VLMEvalKit) |
| POPE           | [This Codebase](./pope)                                  |

## Visual Grounding

| Benchmark Name | Codebase                   |
| -------------- | -------------------------- |
| RefCOCO        | [This Codebase](./refcoco) |
| RefCOCO+       | [This Codebase](./refcoco) |
| RefCOCOg       | [This Codebase](./refcoco) |

## Multimodal Multilingual Understanding

| Benchmark Name       | Codebase                                                 |
| -------------------- | -------------------------------------------------------- |
| MMMB                 | [VLMEvalKit](https://github.com/open-compass/VLMEvalKit) |
| Multilingual MMBench | [VLMEvalKit](https://github.com/open-compass/VLMEvalKit) |
| MTVQA                | [VLMEvalKit](https://github.com/open-compass/VLMEvalKit) |

## Video Understanding

| Benchmark Name | Codebase                                                 |
| -------------- | -------------------------------------------------------- |
| Video-MME      | [VLMEvalKit](https://github.com/open-compass/VLMEvalKit) |
| MVBench        | [This Codebase](./mvbench)                               |
| MMBench-Video  | [VLMEvalKit](https://github.com/open-compass/VLMEvalKit) |
| MLVU           | [VLMEvalKit](https://github.com/open-compass/VLMEvalKit) |
| LongVideoBench | [VLMEvalKit](https://github.com/open-compass/VLMEvalKit) |
| CG-Bench       | provided by authors                                      |