bagel

0816cc7e · mashun1 · 0816cc7e · 0816cc7e · 0816cc7e · 0816cc7e
Commit 0816cc7e authored May 30, 2025 by mashun1
20 changed files
--- a/eval/gen/wise/gpt_eval_mp.py
+++ b/eval/gen/wise/gpt_eval_mp.py
--- a/eval/vlm/__init__.py
+++ b/eval/vlm/__init__.py
+# Copyright 2025 Bytedance Ltd. and/or its affiliates.
+# SPDX-License-Identifier: Apache-2.0
\ No newline at end of file
--- a/eval/vlm/eval/mathvista/calculate_score.py
+++ b/eval/vlm/eval/mathvista/calculate_score.py
--- a/eval/vlm/eval/mathvista/evaluate_mathvista.py
+++ b/eval/vlm/eval/mathvista/evaluate_mathvista.py
--- a/eval/vlm/eval/mathvista/extract_answer.py
+++ b/eval/vlm/eval/mathvista/extract_answer.py
--- a/eval/vlm/eval/mathvista/extract_answer_mp.py
+++ b/eval/vlm/eval/mathvista/extract_answer_mp.py
--- a/eval/vlm/eval/mathvista/prompts/ext_ans.py
+++ b/eval/vlm/eval/mathvista/prompts/ext_ans.py
+# Copyright (c) 2023 OpenGVLab
+# Copyright (c) 2025 Bytedance Ltd. and/or its affiliates.
+# SPDX-License-Identifier: MIT
+#
+# This file has been modified by ByteDance Ltd. and/or its affiliates. on 2025-05-20.
+#
+# Original file was released under MIT, with the full license text
+# available at https://github.com/OpenGVLab/InternVL/blob/main/LICENSE.
+#
+# This modified file is released under the same license.
+
+# pids = 852,  104,  824,  506,  540
+
+demo_prompt = """
+Please read the following example. Then extract the answer from the model response and type it at the end of the prompt.
+
+Hint: Please answer the question requiring an integer answer and provide the final value, e.g., 1, 2, 3, at the end.
+Question: Which number is missing?
+
+Model response: The number missing in the sequence is 14.
+
+Extracted answer: 14
+
+Hint: Please answer the question requiring a floating-point number with one decimal place and provide the final value, e.g., 1.2, 1.3, 1.4, at the end.
+Question: What is the fraction of females facing the camera?
+
+Model response: The fraction of females facing the camera is 0.6, which means that six out of ten females in the group are facing the camera.
+
+Extracted answer: 0.6
+
+Hint: Please answer the question requiring a floating-point number with two decimal places and provide the final value, e.g., 1.23, 1.34, 1.45, at the end.
+Question: How much money does Luca need to buy a sour apple candy and a butterscotch candy? (Unit: $)
+
+Model response: Luca needs $1.45 to buy a sour apple candy and a butterscotch candy.
+
+Extracted answer: 1.45
+
+Hint: Please answer the question requiring a Python list as an answer and provide the final list, e.g., [1, 2, 3], [1.2, 1.3, 1.4], at the end.
+Question: Between which two years does the line  graph saw its maximum peak?
+
+Model response: The line graph saw its maximum peak between 2007 and 2008.
+
+Extracted answer: [2007, 2008]
+
+Hint: Please answer the question and provide the correct option letter, e.g., A, B, C, D, at the end.
+Question: What fraction of the shape is blue?\nChoices:\n(A) 3/11\n(B) 8/11\n(C) 6/11\n(D) 3/5
+
+Model response: The correct answer is (B) 8/11.
+
+Extracted answer: B
+"""
--- a/eval/vlm/eval/mathvista/utilities.py
+++ b/eval/vlm/eval/mathvista/utilities.py
--- a/eval/vlm/eval/mmbench/evaluate_mmbench.py
+++ b/eval/vlm/eval/mmbench/evaluate_mmbench.py
--- a/eval/vlm/eval/mme/Your_Results/OCR.txt
+++ b/eval/vlm/eval/mme/Your_Results/OCR.txt
+0001.jpg	Is the word in the logo "angie's"? Please answer yes or no.	Yes
+0001.jpg	Is the word in the logo "angle's"? Please answer yes or no.	No
+0002.jpg	Is the word in the logo "c'est cheese"? Please answer yes or no.	Yes
+0002.jpg	Is the word in the logo "crest cheese"? Please answer yes or no.	No
+0003.jpg	Is the word in the logo "beavertails pastry"? Please answer yes or no.	Yes
+0003.jpg	Is the word in the logo "beavertalls pastry"? Please answer yes or no.	No
+0004.jpg	Is the word in the logo "old market sundries"? Please answer yes or no.	Yes
+0004.jpg	Is the word in the logo "old market hundreds"? Please answer yes or no.	No
+0005.jpg	Is the word in the logo "kress"? Please answer yes or no.	Yes
+0005.jpg	Is the word in the logo "dress"? Please answer yes or no.	No
+0006.jpg	Is the word in the logo "the beatles story liver pool"? Please answer yes or no.	Yes
+0006.jpg	Is the word in the logo "the beats story liver pool"? Please answer yes or no.	No
+0007.jpg	Is the phone number in the picture "0131 555 6363"? Please answer yes or no.	Yes
+0007.jpg	Is the phone number in the picture "0137 556 6363"? Please answer yes or no.	No
+0008.jpg	Is the word in the logo "phil's market"? Please answer yes or no.	Yes
+0008.jpg	Is the word in the logo "phll's market"? Please answer yes or no.	No
+0009.jpg	Is the word in the logo "fenders diner"? Please answer yes or no.	Yes
+0009.jpg	Is the word in the logo "finders diner"? Please answer yes or no.	No
+0010.jpg	Is the word in the logo "high time coffee shop"? Please answer yes or no.	Yes
+0010.jpg	Is the word in the logo "high tite cofeee shop"? Please answer yes or no.	No
+0011.jpg	Is the word in the logo "ihop restaurant"? Please answer yes or no.	Yes
+0011.jpg	Is the word in the logo "lhop restaurant"? Please answer yes or no.	No
+0012.jpg	Is the word in the logo "casa grecque restaurants"? Please answer yes or no.	Yes
+0012.jpg	Is the word in the logo "case grecque restaurants"? Please answer yes or no.	No
+0013.jpg	Is the word in the picture "seabreeze motel"? Please answer yes or no.	Yes
+0013.jpg	Is the word in the picture "seebreeze model"? Please answer yes or no.	No
+0014.jpg	Is the word in the logo "penarth pier built 1894"? Please answer yes or no.	Yes
+0014.jpg	Is the word in the logo "penarth pies buid 1894"? Please answer yes or no.	No
+0015.jpg	Is the text in the picture "hollywood"? Please answer yes or no.	Yes
+0015.jpg	Is the text in the picture "holly word"? Please answer yes or no.	No
+0016.jpg	Is the word in the logo "shop rite"? Please answer yes or no.	Yes
+0016.jpg	Is the word in the logo "stop rite"? Please answer yes or no.	No
+0017.jpg	Is the word in the logo "hardco industrial construction"? Please answer yes or no.	Yes
+0017.jpg	Is the word in the logo "hardto industal construction"? Please answer yes or no.	No
+0018.jpg	Is the word in the logo "oldsmobile service"? Please answer yes or no.	Yes
+0018.jpg	Is the word in the logo "old mobile service"? Please answer yes or no.	No
+0019.jpg	Is the word in the logo "exchange hotel"? Please answer yes or no.	Yes
+0019.jpg	Is the word in the logo "excharge hotel"? Please answer yes or no.	No
+0020.jpg	Is the word in the logo "cold drinks"? Please answer yes or no.	Yes
+0020.jpg	Is the word in the logo "cold rinks"? Please answer yes or no.	No
--- a/eval/vlm/eval/mme/Your_Results/artwork.txt
+++ b/eval/vlm/eval/mme/Your_Results/artwork.txt
--- a/eval/vlm/eval/mme/Your_Results/celebrity.txt
+++ b/eval/vlm/eval/mme/Your_Results/celebrity.txt
--- a/eval/vlm/eval/mme/Your_Results/code_reasoning.txt
+++ b/eval/vlm/eval/mme/Your_Results/code_reasoning.txt
--- a/eval/vlm/eval/mme/Your_Results/color.txt
+++ b/eval/vlm/eval/mme/Your_Results/color.txt
--- a/eval/vlm/eval/mme/Your_Results/commonsense_reasoning.txt
+++ b/eval/vlm/eval/mme/Your_Results/commonsense_reasoning.txt
--- a/eval/vlm/eval/mme/Your_Results/count.txt
+++ b/eval/vlm/eval/mme/Your_Results/count.txt
--- a/eval/vlm/eval/mme/Your_Results/existence.txt
+++ b/eval/vlm/eval/mme/Your_Results/existence.txt
--- a/eval/vlm/eval/mme/Your_Results/landmark.txt
+++ b/eval/vlm/eval/mme/Your_Results/landmark.txt
--- a/eval/vlm/eval/mme/Your_Results/numerical_calculation.txt
+++ b/eval/vlm/eval/mme/Your_Results/numerical_calculation.txt
--- a/eval/vlm/eval/mme/Your_Results/position.txt
+++ b/eval/vlm/eval/mme/Your_Results/position.txt