Skip to content

GitLab

Commit 1d5a34cf authored by wanglch's avatar wanglch
Browse files

Initial commit

parents
Pipeline #1446 canceled with stages
Hide whitespace changes
Inline Side-by-side
Showing with 2926 additions and 0 deletions
internvl_chat/eval/llava_bench/rule.json 0 → 100644
View file @ 1d5a34cf
{
"coding": {"role": "Assistant", "prompt": "Your task is to evaluate the coding abilities of the above two assistants. They have been asked to implement a program to solve a given problem. Please review their code submissions, paying close attention to their problem-solving approach, code structure, readability, and the inclusion of helpful comments.\n\nPlease ensure that the assistants' submissions:\n\n1. Correctly implement the given problem statement.\n2. Contain accurate and efficient code.\n3. Include clear and concise comments that explain the code's logic and functionality.\n4. Adhere to proper coding standards and best practices.\n\nOnce you have carefully reviewed both submissions, provide detailed feedback on their strengths and weaknesses, along with any suggestions for improvement. You should first output a single line containing two scores on the scale of 1-10 (1: no code/no sense; 10: perfect) for Assistant 1 and 2, respectively. Then give extra comments starting from the next line."},
"math": {"role": "Assistant", "prompt": "We would like to request your feedback on the mathematical proficiency of two AI assistants regarding the given user question.\nFirstly, please solve the problem independently, without referring to the answers provided by Assistant 1 and Assistant 2.\nAfterward, please examine the problem-solving process of Assistant 1 and Assistant 2 step-by-step to ensure their correctness, identifying any incorrect steps if present. Your evaluation should take into account not only the answer but also the problem-solving steps.\nFinally, please output a Python tuple containing two numerical scores for Assistant 1 and Assistant 2, ranging from 1 to 10, respectively. If applicable, explain the reasons for any variations in their scores and determine which assistant performed better."},
"default": {"role": "Assistant", "prompt": "We would like to request your feedback on the performance of two AI assistants in response to the user question displayed above.\nPlease rate the helpfulness, relevance, accuracy, level of details of their responses. Each assistant receives an overall score on a scale of 1 to 10, where a higher score indicates better overall performance.\nPlease first output a single line containing only two values indicating the scores for Assistant 1 and 2, respectively. The two scores are separated by a space.\nIn the subsequent line, please provide a comprehensive explanation of your evaluation, avoiding any potential bias and ensuring that the order in which the responses were presented does not affect your judgment."},
"conv": {"role": "Assistant", "prompt": "We would like to request your feedback on the performance of two AI assistants in response to the user question displayed above. The user asks the question on observing an image. For your reference, the visual content in the image is represented with five descriptive sentences describing the same image and the bounding box coordinates of each object in the scene. These coordinates are in the form of bounding boxes, represented as (x1, y1, x2, y2) with floating numbers ranging from 0 to 1. These values correspond to the top left x, top left y, bottom right x, and bottom right y. \nPlease rate the helpfulness, relevance, accuracy, level of details of their responses. Each assistant receives an overall score on a scale of 1 to 10, where a higher score indicates better overall performance.\nPlease first output a single line containing only two values indicating the scores for Assistant 1 and 2, respectively. The two scores are separated by a space.\nIn the subsequent line, please provide a comprehensive explanation of your evaluation, avoiding any potential bias and ensuring that the order in which the responses were presented does not affect your judgment."},
"detail": {"role": "Assistant", "prompt": "We would like to request your feedback on the performance of two AI assistants in response to the user question displayed above. The user asks the question on observing an image. For your reference, the visual content in the image is represented with five descriptive sentences describing the same image and the bounding box coordinates of each object in the scene. These coordinates are in the form of bounding boxes, represented as (x1, y1, x2, y2) with floating numbers ranging from 0 to 1. These values correspond to the top left x, top left y, bottom right x, and bottom right y. \nPlease rate the helpfulness, relevance, accuracy, level of details of their responses. Each assistant receives an overall score on a scale of 1 to 10, where a higher score indicates better overall performance.\nPlease first output a single line containing only two values indicating the scores for Assistant 1 and 2, respectively. The two scores are separated by a space.\nIn the subsequent line, please provide a comprehensive explanation of your evaluation, avoiding any potential bias and ensuring that the order in which the responses were presented does not affect your judgment."},
"complex": {"role": "Assistant", "prompt": "We would like to request your feedback on the performance of two AI assistants in response to the user question displayed above. The user asks the question on observing an image. For your reference, the visual content in the image is represented with five descriptive sentences describing the same image and the bounding box coordinates of each object in the scene. These coordinates are in the form of bounding boxes, represented as (x1, y1, x2, y2) with floating numbers ranging from 0 to 1. These values correspond to the top left x, top left y, bottom right x, and bottom right y. \nPlease rate the helpfulness, relevance, accuracy, level of details of their responses. Each assistant receives an overall score on a scale of 1 to 10, where a higher score indicates better overall performance.\nPlease first output a single line containing only two values indicating the scores for Assistant 1 and 2, respectively. The two scores are separated by a space.\nIn the subsequent line, please provide a comprehensive explanation of your evaluation, avoiding any potential bias and ensuring that the order in which the responses were presented does not affect your judgment."},
"llava_bench_conv": {"role": "Assistant", "prompt": "We would like to request your feedback on the performance of two AI assistants in response to the user question displayed above. The user asks the question on observing an image. For your reference, the visual content in the image is represented with a few sentences describing the image. \nPlease rate the helpfulness, relevance, accuracy, level of details of their responses. Each assistant receives an overall score on a scale of 1 to 10, where a higher score indicates better overall performance.\nPlease first output a single line containing only two values indicating the scores for Assistant 1 and 2, respectively. The two scores are separated by a space.\nIn the subsequent line, please provide a comprehensive explanation of your evaluation, avoiding any potential bias and ensuring that the order in which the responses were presented does not affect your judgment."},
"llava_bench_detail": {"role": "Assistant", "prompt": "We would like to request your feedback on the performance of two AI assistants in response to the user question displayed above. The user asks the question on observing an image. For your reference, the visual content in the image is represented with a few sentences describing the image. \nPlease rate the helpfulness, relevance, accuracy, level of details of their responses. Each assistant receives an overall score on a scale of 1 to 10, where a higher score indicates better overall performance.\nPlease first output a single line containing only two values indicating the scores for Assistant 1 and 2, respectively. The two scores are separated by a space.\nIn the subsequent line, please provide a comprehensive explanation of your evaluation, avoiding any potential bias and ensuring that the order in which the responses were presented does not affect your judgment."},
"llava_bench_complex": {"role": "Assistant", "prompt": "We would like to request your feedback on the performance of two AI assistants in response to the user question displayed above. The user asks the question on observing an image. For your reference, the visual content in the image is represented with a few sentences describing the image. \nPlease rate the helpfulness, relevance, accuracy, level of details of their responses. Each assistant receives an overall score on a scale of 1 to 10, where a higher score indicates better overall performance.\nPlease first output a single line containing only two values indicating the scores for Assistant 1 and 2, respectively. The two scores are separated by a space.\nIn the subsequent line, please provide a comprehensive explanation of your evaluation, avoiding any potential bias and ensuring that the order in which the responses were presented does not affect your judgment."}
}
internvl_chat/eval/llava_bench/summarize_gpt_review.py 0 → 100644
View file @ 1d5a34cf
import argparse
import json
import os
from collections import defaultdict
import numpy as np
def parse_args():
parser = argparse.ArgumentParser(description='ChatGPT-based QA evaluation.')
parser.add_argument('-d', '--dir', default=None)
parser.add_argument('-v', '--version', default=None)
parser.add_argument('-s', '--select', nargs='*', default=None)
parser.add_argument('-f', '--files', nargs='*', default=[])
parser.add_argument('-i', '--ignore', nargs='*', default=[])
return parser.parse_args()
if __name__ == '__main__':
args = parse_args()
if args.ignore is not None:
args.ignore = [int(x) for x in args.ignore]
if len(args.files) > 0:
review_files = args.files
else:
review_files = [x for x in os.listdir(args.dir) if x.endswith('.jsonl') and (
x.startswith('gpt4_text') or x.startswith('reviews_') or x.startswith(
'review_') or 'review' in args.dir)]
for review_file in sorted(review_files):
config = os.path.basename(review_file).replace('gpt4_text_', '').replace('.jsonl', '')
if args.select is not None and any(x not in config for x in args.select):
continue
if '0613' in config:
version = '0613'
else:
version = '0314'
if args.version is not None and args.version != version:
continue
scores = defaultdict(list)
print(config)
with open(os.path.join(args.dir, review_file) if args.dir is not None else review_file) as f:
for review_str in f:
review = json.loads(review_str)
if review['question_id'] in args.ignore:
continue
if 'category' in review:
scores[review['category']].append(review['tuple'])
scores['all'].append(review['tuple'])
else:
if 'tuple' in review:
scores['all'].append(review['tuple'])
else:
scores['all'].append(review['score'])
for k, v in sorted(scores.items()):
stats = np.asarray(v).mean(0).tolist()
stats = [round(x, 3) for x in stats]
print(k, stats, round(stats[1] / stats[0] * 100, 1))
print(k, round(stats[1] / stats[0] * 100, 1), round(stats[0] * 10, 1), round(stats[1] * 10, 1))
print('=================================')
internvl_chat/eval/mathvista/calculate_score.py 0 → 100644
View file @ 1d5a34cf
import argparse
import sys
import pandas as pd
# !pip install python-Levenshtein
from Levenshtein import distance
from utilities import *
def get_most_similar(prediction, choices):
"""
Use the Levenshtein distance (or edit distance) to determine which of the choices is most similar to the given prediction
"""
distances = [distance(prediction, choice) for choice in choices]
ind = distances.index(min(distances))
return choices[ind]
# return min(choices, key=lambda choice: distance(prediction, choice))
def normalize_extracted_answer(extraction, choices, question_type, answer_type, precision):
"""
Normalize the extracted answer to match the answer type
"""
if question_type == 'multi_choice':
# make sure the extraction is a string
if isinstance(extraction, str):
extraction = extraction.strip()
else:
try:
extraction = str(extraction)
except:
extraction = ''
# extract "A" from "(A) text"
letter = re.findall(r'\(([a-zA-Z])\)', extraction)
if len(letter) > 0:
extraction = letter[0].upper()
options = [chr(ord('A') + i) for i in range(len(choices))]
if extraction in options:
# convert option letter to text, e.g. "A" -> "text"
ind = options.index(extraction)
extraction = choices[ind]
else:
# select the most similar option
extraction = get_most_similar(extraction, choices)
assert extraction in choices
elif answer_type == 'integer':
try:
extraction = str(int(float(extraction)))
except:
extraction = None
elif answer_type == 'float':
try:
extraction = str(round(float(extraction), precision))
except:
extraction = None
elif answer_type == 'list':
try:
extraction = str(extraction)
except:
extraction = None
return extraction
def safe_equal(prediction, answer):
"""
Check if the prediction is equal to the answer, even if they are of different types
"""
try:
if prediction == answer:
return True
return False
except Exception as e:
print(e)
return False
def get_acc_with_contion(res_pd, key, value):
if key == 'skills':
# if value in res_pd[key]:
total_pd = res_pd[res_pd[key].apply(lambda x: value in x)]
else:
total_pd = res_pd[res_pd[key] == value]
correct_pd = total_pd[total_pd['true_false'] == True] # noqa: E712
acc = '{:.2f}'.format(len(correct_pd) / len(total_pd) * 100)
return len(correct_pd), len(total_pd), acc
if __name__ == '__main__':
parser = argparse.ArgumentParser()
parser.add_argument('--output_dir', type=str, default='./results')
parser.add_argument('--output_file', type=str, default='output.json')
parser.add_argument('--score_file', type=str, default='scores.json')
parser.add_argument('--gt_file', type=str, default='./data/MathVista/annot_testmini.json', help='ground truth file')
parser.add_argument('--number', type=int, default=-1, help='number of problems to run')
parser.add_argument('--rerun', action='store_true', help='rerun the evaluation')
parser.add_argument('--caculate_gain', action='store_true', help='caculate the socre gains over random guess')
parser.add_argument('--random_file', type=str, default='score_random_guess.json')
args = parser.parse_args()
# args
output_file = os.path.join(args.output_dir, args.output_file)
# # quick test
# output_file = '../results/llava-llama-2-13b/output_llava_llama_2_13b.json'
# read json
print(f'Reading {output_file}...')
results = read_json(output_file)
# read ground truth
print(f'Reading {args.gt_file}...')
gts = read_json(args.gt_file)
# full pids
full_pids = list(results.keys())
if args.number > 0:
full_pids = full_pids[:min(args.number, len(full_pids))]
print('Number of testing problems:', len(full_pids))
## [1] Evaluate if the prediction is true or false
print('\nEvaluating the predictions...')
update_json_flag = False
for pid in full_pids:
problem = results[pid]
# print(problem)
if args.rerun:
if 'prediction' in problem:
del problem['prediction']
if 'true_false' in problem:
del problem['true_false']
choices = problem['choices']
question_type = problem['question_type']
answer_type = problem['answer_type']
precision = problem['precision']
extraction = problem['extraction']
if 'answer' in problem:
answer = problem['answer']
else:
answer = gts[pid]['answer']
problem['answer'] = answer
# normalize the extracted answer to match the answer type
prediction = normalize_extracted_answer(extraction, choices, question_type, answer_type, precision)
# verify the prediction is true or false
true_false = safe_equal(prediction, answer)
# update the problem
if 'true_false' not in problem:
update_json_flag = True
elif true_false != problem['true_false']:
update_json_flag = True
if 'prediction' not in problem:
update_json_flag = True
elif prediction != problem['prediction']:
update_json_flag = True
problem['prediction'] = prediction
problem['true_false'] = true_false
# save the updated json
if update_json_flag:
print('\n!!!Some problems are updated.!!!')
print(f'\nSaving {output_file}...')
save_json(results, output_file)
## [2] Calculate the average accuracy
total = len(full_pids)
correct = 0
for pid in full_pids:
if results[pid]['true_false']:
correct += 1
accuracy = str(round(correct / total * 100, 2))
print(f'\nCorrect: {correct}, Total: {total}, Accuracy: {accuracy}%')
scores = {'average': {'accuracy': accuracy, 'correct': correct, 'total': total}}
## [3] Calculate the fine-grained accuracy scores
# merge the 'metadata' attribute into the data
for pid in results:
results[pid].update(results[pid].pop('metadata'))
# convert the data to a pandas DataFrame
df = pd.DataFrame(results).T
print(len(df))
print('Number of test problems:', len(df))
# assert len(df) == 1000 # Important!!!
# asign the target keys for evaluation
target_keys = ['question_type', 'answer_type', 'language', 'source', 'category', 'task', 'context', 'grade',
'skills']
for key in target_keys:
print(f'\nType: [{key}]')
# get the unique values of the key
if key == 'skills':
# the value is a list
values = []
for i in range(len(df)):
values += df[key][i]
values = list(set(values))
else:
values = df[key].unique()
# print(values)
# calculate the accuracy for each value
scores[key] = {}
for value in values:
correct, total, acc = get_acc_with_contion(df, key, value)
if total > 0:
print(f'[{value}]: {acc}% ({correct}/{total})')
scores[key][value] = {'accuracy': acc, 'correct': correct, 'total': total}
# sort the scores by accuracy
scores[key] = dict(sorted(scores[key].items(), key=lambda item: float(item[1]['accuracy']), reverse=True))
# save the scores
scores_file = os.path.join(args.output_dir, args.score_file)
print(f'\nSaving {scores_file}...')
save_json(scores, scores_file)
print('\nDone!')
# [4] Calculate the score gains over random guess
if args.caculate_gain:
random_file = os.path.join(args.output_dir, args.random_file)
random_scores = json.load(open(random_file))
print('\nCalculating the score gains...')
for key in scores:
if key == 'average':
gain = round(float(scores[key]['accuracy']) - float(random_scores[key]['accuracy']), 2)
scores[key]['acc_gain'] = gain
else:
for sub_key in scores[key]:
gain = round(
float(scores[key][sub_key]['accuracy']) - float(random_scores[key][sub_key]['accuracy']), 2)
scores[key][sub_key]['acc_gain'] = str(gain)
# save the score gains
print(f'\nSaving {scores_file}...')
save_json(scores, scores_file)
print('\nDone!')
internvl_chat/eval/mathvista/evaluate_mathvista.py 0 → 100644
View file @ 1d5a34cf
import argparse
import itertools
import json
import os
import random
import time
from functools import partial
import torch
from datasets import concatenate_datasets, load_dataset
from internvl.model.internvl_chat import InternVLChatModel
from internvl.train.dataset import build_transform, dynamic_preprocess
from torch.utils.data import Dataset
from tqdm import tqdm
from transformers import AutoTokenizer
ds_collections = {
'MathVista_testmini': {
'root': 'AI4Math/MathVista',
'max_new_tokens': 1000,
'min_new_tokens': 1,
'split': 'testmini'
},
'MathVista_test': {
'root': 'AI4Math/MathVista',
'max_new_tokens': 1000,
'min_new_tokens': 1,
'split': 'test'
},
}
def collate_fn(batches, tokenizer):
pixel_values = torch.cat([_['pixel_values'] for _ in batches], dim=0)
data_items = [_['data_item'] for _ in batches]
return pixel_values, data_items
class MathVistaDataset(torch.utils.data.Dataset):
def __init__(self, root, split, input_size=224, dynamic_image_size=False,
use_thumbnail=False, max_num=6):
dataset = load_dataset(root, cache_dir=os.path.join(os.getcwd(), 'data/MathVista/'))
self.data = dataset[split]
self.input_size = input_size
self.dynamic_image_size = dynamic_image_size
self.use_thumbnail = use_thumbnail
self.max_num = max_num
self.transform = build_transform(is_train=False, input_size=input_size)
def __len__(self):
return len(self.data)
def __getitem__(self, idx):
data_item = self.data[idx]
image = data_item['decoded_image']
del data_item['decoded_image']
if self.dynamic_image_size:
images = dynamic_preprocess(image, image_size=self.input_size,
use_thumbnail=self.use_thumbnail,
max_num=self.max_num)
else:
images = [image]
pixel_values = [self.transform(image) for image in images]
pixel_values = torch.stack(pixel_values)
return {
'pixel_values': pixel_values,
'data_item': data_item,
}
class InferenceSampler(torch.utils.data.sampler.Sampler):
def __init__(self, size):
self._size = int(size)
assert size > 0
self._rank = torch.distributed.get_rank()
self._world_size = torch.distributed.get_world_size()
self._local_indices = self._get_local_indices(size, self._world_size, self._rank)
@staticmethod
def _get_local_indices(total_size, world_size, rank):
shard_size = total_size // world_size
left = total_size % world_size
shard_sizes = [shard_size + int(r < left) for r in range(world_size)]
begin = sum(shard_sizes[:rank])
end = min(sum(shard_sizes[:rank + 1]), total_size)
return range(begin, end)
def __iter__(self):
yield from self._local_indices
def __len__(self):
return len(self._local_indices)
def evaluate_chat_model():
random.seed(args.seed)
for ds_name in args.datasets:
dataset = MathVistaDataset(
root=ds_collections[ds_name]['root'],
split=ds_collections[ds_name]['split'],
input_size=image_size,
dynamic_image_size=args.dynamic,
use_thumbnail=use_thumbnail,
max_num=args.max_num
)
dataloader = torch.utils.data.DataLoader(
dataset=dataset,
sampler=InferenceSampler(len(dataset)),
batch_size=args.batch_size,
num_workers=args.num_workers,
pin_memory=True,
drop_last=False,
collate_fn=partial(collate_fn, tokenizer=tokenizer),
)
outputs = []
for _, (pixel_values, data_items) in tqdm(enumerate(dataloader)):
pixel_values = pixel_values.to(torch.bfloat16).cuda()
generation_config = dict(
num_beams=args.num_beams,
max_new_tokens=ds_collections[ds_name]['max_new_tokens'],
min_new_tokens=ds_collections[ds_name]['min_new_tokens'],
do_sample=True if args.temperature > 0 else False,
temperature=args.temperature,
)
pred = model.chat(
tokenizer=tokenizer,
pixel_values=pixel_values,
question=data_items[0]['query'],
generation_config=generation_config,
verbose=True
)
data_item = data_items[0]
data_item['response'] = pred
outputs.append(data_item)
torch.distributed.barrier()
world_size = torch.distributed.get_world_size()
merged_outputs = [None for _ in range(world_size)]
torch.distributed.all_gather_object(merged_outputs, json.dumps(outputs))
merged_outputs = [json.loads(_) for _ in merged_outputs]
merged_outputs = [_ for _ in itertools.chain.from_iterable(merged_outputs)]
if torch.distributed.get_rank() == 0:
temp = {}
for data_item in merged_outputs:
pid = data_item['pid']
temp[pid] = data_item
print(f'Evaluating {ds_name} ...')
time_prefix = time.strftime('%y%m%d%H%M%S', time.localtime())
results_file = f'{ds_name}_{time_prefix}.json'
output_path = os.path.join(args.out_dir, results_file)
json.dump(temp, open(output_path, 'w'), indent=4)
print('Results saved to {}'.format(output_path))
cmd = f'python eval/mathvista/extract_answer.py --output_file {results_file}'
print(cmd)
os.system(cmd)
cmd = f'python eval/mathvista/calculate_score.py --output_file {results_file} --score_file {results_file[:-5]}_score.json'
print(cmd)
os.system(cmd)
if __name__ == '__main__':
parser = argparse.ArgumentParser()
parser.add_argument('--checkpoint', type=str, default='')
parser.add_argument('--datasets', type=str, default='MathVista_testmini')
parser.add_argument('--batch-size', type=int, default=1)
parser.add_argument('--num-workers', type=int, default=1)
parser.add_argument('--num-beams', type=int, default=5)
parser.add_argument('--temperature', type=float, default=0.0)
parser.add_argument('--out-dir', type=str, default='results')
parser.add_argument('--seed', type=int, default=0)
parser.add_argument('--dynamic', action='store_true')
parser.add_argument('--max-num', type=int, default=6)
parser.add_argument('--load-in-8bit', action='store_true')
parser.add_argument('--load-in-4bit', action='store_true')
parser.add_argument('--auto', action='store_true')
args = parser.parse_args()
if not os.path.exists(args.out_dir):
os.makedirs(args.out_dir)
args.datasets = args.datasets.split(',')
print('datasets:', args.datasets)
assert args.batch_size == 1, 'Only batch size 1 is supported'
torch.distributed.init_process_group(
backend='nccl',
world_size=int(os.getenv('WORLD_SIZE', '1')),
rank=int(os.getenv('RANK', '0')),
)
torch.cuda.set_device(int(os.getenv('LOCAL_RANK', 0)))
if args.auto:
os.environ['CUDA_LAUNCH_BLOCKING'] = '1'
kwargs = {'device_map': 'auto'} if args.auto else {}
tokenizer = AutoTokenizer.from_pretrained(args.checkpoint, trust_remote_code=True, use_fast=False)
model = InternVLChatModel.from_pretrained(
args.checkpoint, low_cpu_mem_usage=True, torch_dtype=torch.bfloat16,
load_in_8bit=args.load_in_8bit, load_in_4bit=args.load_in_4bit, **kwargs).eval()
if not args.load_in_8bit and not args.load_in_4bit and not args.auto:
model = model.cuda()
image_size = model.config.force_image_size or model.config.vision_config.image_size
use_thumbnail = model.config.use_thumbnail
total_params = sum(p.numel() for p in model.parameters()) / 1e9
if total_params > 20 or args.dynamic:
args.num_beams = 1
print(f'[test] total_params: {total_params}B, use num_beams: {args.num_beams}')
else:
print(f'[test] total_params: {total_params}B')
print(f'[test] image_size: {image_size}')
print(f'[test] template: {model.config.template}')
print(f'[test] dynamic_image_size: {args.dynamic}')
print(f'[test] use_thumbnail: {use_thumbnail}')
print(f'[test] max_num: {args.max_num}')
evaluate_chat_model()
internvl_chat/eval/mathvista/extract_answer.py 0 → 100644
View file @ 1d5a34cf
import argparse
# OpenAI
import openai
from tqdm import tqdm
from utilities import *
openai.api_key = os.getenv('OPENAI_API_KEY')
print(openai.api_key)
# load demo prompt
from prompts.ext_ans import demo_prompt
def verify_extraction(extraction):
extraction = extraction.strip()
if extraction == '' or extraction is None:
return False
return True
def create_test_prompt(demo_prompt, query, response):
demo_prompt = demo_prompt.strip()
test_prompt = f'{query}\n\n{response}'
full_prompt = f'{demo_prompt}\n\n{test_prompt}\n\nExtracted answer: '
return full_prompt
def extract_answer(response, problem, quick_extract=False):
question_type = problem['question_type']
answer_type = problem['answer_type']
choices = problem['choices']
query = problem['query']
if response == '':
return ''
if question_type == 'multi_choice' and response in choices:
return response
if answer_type == 'integer':
try:
extraction = int(response)
return str(extraction)
except:
pass
if answer_type == 'float':
try:
extraction = str(float(response))
return extraction
except:
pass
# quick extraction
if quick_extract:
print('Quickly extracting answer...')
# The answer is "text". -> "text"
try:
result = re.search(r'The answer is "(.*)"\.', response)
if result:
extraction = result.group(1)
return extraction
except:
pass
# general extraction
try:
full_prompt = create_test_prompt(demo_prompt, query, response)
extraction = get_chat_response(full_prompt, openai.api_key, patience=5)
return extraction
except Exception as e:
print(e)
print(f'Error in extracting answer for {pid}')
return ''
if __name__ == '__main__':
parser = argparse.ArgumentParser()
# input
parser.add_argument('--output_dir', type=str, default='./results')
parser.add_argument('--output_file', type=str, default='mathvista_answer.json')
parser.add_argument('--response_label', type=str, default='response', help='response label for the input file')
# model
parser.add_argument('--llm_engine', type=str, default='gpt-4-0613', help='llm engine',
choices=['gpt-3.5-turbo', 'gpt-3.5', 'gpt-4', 'gpt-4-0314', 'gpt-4-0613'])
parser.add_argument('--number', type=int, default=-1, help='number of problems to run')
parser.add_argument('--quick_extract', action='store_true', help='use rules to extract answer for some problems')
parser.add_argument('--rerun', action='store_true', help='rerun the answer extraction')
# output
parser.add_argument('--save_every', type=int, default=10, help='save every n problems')
parser.add_argument('--output_label', type=str, default='', help='label for the output file')
args = parser.parse_args()
# args
label = args.response_label
result_file = os.path.join(args.output_dir, args.output_file)
if args.output_label != '':
output_file = result_file.replace('.json', f'_{args.output_label}.json')
else:
output_file = result_file
# read results
print(f'Reading {result_file}...')
results = read_json(result_file)
# full pids
full_pids = list(results.keys())
if args.number > 0:
full_pids = full_pids[:min(args.number, len(full_pids))]
print('Number of testing problems:', len(full_pids))
# test pids
if args.rerun:
test_pids = full_pids
else:
test_pids = []
for pid in full_pids:
# print(pid)
if 'extraction' not in results[pid] or not verify_extraction(results[pid]['extraction']):
test_pids.append(pid)
test_num = len(test_pids)
print('Number of problems to run:', test_num)
# print(test_pids)
# tqdm, enumerate results
for i, pid in enumerate(tqdm(test_pids)):
problem = results[pid]
assert label in problem
response = problem[label]
extraction = extract_answer(response, problem, args.quick_extract)
results[pid]['extraction'] = extraction
if i % args.save_every == 0 or i == test_num - 1:
print(f'Saving results to {output_file}...')
save_json(results, output_file)
print(f'Results saved.')
internvl_chat/eval/mathvista/prompts/ext_ans.py 0 → 100644
View file @ 1d5a34cf
# pids = 852, 104, 824, 506, 540
demo_prompt = """
Please read the following example. Then extract the answer from the model response and type it at the end of the prompt.
Hint: Please answer the question requiring an integer answer and provide the final value, e.g., 1, 2, 3, at the end.
Question: Which number is missing?
Model response: The number missing in the sequence is 14.
Extracted answer: 14
Hint: Please answer the question requiring a floating-point number with one decimal place and provide the final value, e.g., 1.2, 1.3, 1.4, at the end.
Question: What is the fraction of females facing the camera?
Model response: The fraction of females facing the camera is 0.6, which means that six out of ten females in the group are facing the camera.
Extracted answer: 0.6
Hint: Please answer the question requiring a floating-point number with two decimal places and provide the final value, e.g., 1.23, 1.34, 1.45, at the end.
Question: How much money does Luca need to buy a sour apple candy and a butterscotch candy? (Unit: $)
Model response: Luca needs $1.45 to buy a sour apple candy and a butterscotch candy.
Extracted answer: 1.45
Hint: Please answer the question requiring a Python list as an answer and provide the final list, e.g., [1, 2, 3], [1.2, 1.3, 1.4], at the end.
Question: Between which two years does the line graph saw its maximum peak?
Model response: The line graph saw its maximum peak between 2007 and 2008.
Extracted answer: [2007, 2008]
Hint: Please answer the question and provide the correct option letter, e.g., A, B, C, D, at the end.
Question: What fraction of the shape is blue?\nChoices:\n(A) 3/11\n(B) 8/11\n(C) 6/11\n(D) 3/5
Model response: The correct answer is (B) 8/11.
Extracted answer: B
"""
internvl_chat/eval/mathvista/utilities.py 0 → 100644
View file @ 1d5a34cf
import json
import os
import pickle
import re
import time
import cv2
import openai
from word2number import w2n
def create_dir(output_dir):
if not os.path.exists(output_dir):
os.makedirs(output_dir)
def read_csv(file):
data = []
with open(file, 'r') as f:
for line in f:
data.append(line.strip())
return data
def read_pandas_csv(csv_path):
# read a pandas csv sheet
import pandas as pd
df = pd.read_csv(csv_path)
return df
def read_json(path):
with open(path, 'r', encoding='utf-8') as f:
return json.load(f)
def read_jsonl(file):
with open(file, 'r') as f:
data = [json.loads(line) for line in f]
return data
def read_pickle(path):
with open(path, 'rb') as f:
return pickle.load(f)
def save_json(data, path):
with open(path, 'w') as f:
json.dump(data, f, indent=4)
def save_array_img(path, image):
cv2.imwrite(path, image)
def contains_digit(text):
# check if text contains a digit
if any(char.isdigit() for char in text):
return True
return False
def contains_number_word(text):
# check if text contains a number word
ignore_words = ['a', 'an', 'point']
words = re.findall(r'\b\w+\b', text) # This regex pattern matches any word in the text
for word in words:
if word in ignore_words:
continue
try:
w2n.word_to_num(word)
return True # If the word can be converted to a number, return True
except ValueError:
continue # If the word can't be converted to a number, continue with the next word
# check if text contains a digit
if any(char.isdigit() for char in text):
return True
return False # If none of the words could be converted to a number, return False
def contains_quantity_word(text, special_keep_words=[]):
# check if text contains a quantity word
quantity_words = ['most', 'least', 'fewest'
'more', 'less', 'fewer',
'largest', 'smallest', 'greatest',
'larger', 'smaller', 'greater',
'highest', 'lowest', 'higher', 'lower',
'increase', 'decrease',
'minimum', 'maximum', 'max', 'min',
'mean', 'average', 'median',
'total', 'sum', 'add', 'subtract',
'difference', 'quotient', 'gap',
'half', 'double', 'twice', 'triple',
'square', 'cube', 'root',
'approximate', 'approximation',
'triangle', 'rectangle', 'circle', 'square', 'cube', 'sphere', 'cylinder', 'cone', 'pyramid',
'multiply', 'divide',
'percentage', 'percent', 'ratio', 'proportion', 'fraction', 'rate',
]
quantity_words += special_keep_words # dataset specific words
words = re.findall(r'\b\w+\b', text) # This regex pattern matches any word in the text
if any(word in quantity_words for word in words):
return True
return False # If none of the words could be converted to a number, return False
def is_bool_word(text):
if text in ['Yes', 'No', 'True', 'False',
'yes', 'no', 'true', 'false',
'YES', 'NO', 'TRUE', 'FALSE']:
return True
return False
def is_digit_string(text):
# remove ".0000"
text = text.strip()
text = re.sub(r'\.0+$', '', text)
try:
int(text)
return True
except ValueError:
return False
def is_float_string(text):
# text is a float string if it contains a "." and can be converted to a float
if '.' in text:
try:
float(text)
return True
except ValueError:
return False
return False
def copy_image(image_path, output_image_path):
from shutil import copyfile
copyfile(image_path, output_image_path)
def copy_dir(src_dir, dst_dir):
from shutil import copytree
# copy the source directory to the target directory
copytree(src_dir, dst_dir)
import PIL.Image as Image
def get_image_size(img_path):
img = Image.open(img_path)
width, height = img.size
return width, height
def get_chat_response(promot, api_key, model='gpt-3.5-turbo', temperature=0, max_tokens=256, n=1, patience=10000000,
sleep_time=0):
messages = [
{'role': 'user', 'content': promot},
]
while patience > 0:
patience -= 1
try:
completion = openai.chat.completions.create(model=model,
messages=messages,
temperature=temperature,
max_tokens=max_tokens,
n=n)
if n == 1:
prediction = completion.choices[0].message.content.strip()
if prediction != '' and prediction is not None:
return prediction
else:
prediction = [choice.message.content.strip() for choice in completion.choices]
if prediction[0] != '' and prediction[0] is not None:
return prediction
except Exception as e:
if 'Rate limit' not in str(e):
print(e)
if 'Please reduce the length of the messages' in str(e):
print('!!Reduce promot size')
# reduce input prompt and keep the tail
new_size = int(len(promot) * 0.9)
new_start = len(promot) - new_size
promot = promot[new_start:]
messages = [
{'role': 'user', 'content': promot},
]
if sleep_time > 0:
time.sleep(sleep_time)
return ''
internvl_chat/eval/mmbench/evaluate_mmbench.py 0 → 100644
View file @ 1d5a34cf
import argparse
import base64
import itertools
import json
import os
import random
import time
from functools import partial
from io import BytesIO
import pandas as pd
import torch
from internvl.model.internvl_chat import InternVLChatModel
from internvl.train.dataset import build_transform, dynamic_preprocess
from PIL import Image
from torch.utils.data import Dataset
from tqdm import tqdm
from transformers import AutoTokenizer
ds_collections = {
'mmbench_dev_20230712': {
'root': 'data/mmbench/mmbench_dev_20230712.tsv',
'max_new_tokens': 100,
'min_new_tokens': 1,
'type': 'dev',
'language': 'en'
},
'mmbench_dev_cn_20231003': {
'root': 'data/mmbench/mmbench_dev_cn_20231003.tsv',
'max_new_tokens': 100,
'min_new_tokens': 1,
'type': 'dev',
'language': 'cn'
},
'mmbench_dev_en_20231003': {
'root': 'data/mmbench/mmbench_dev_en_20231003.tsv',
'max_new_tokens': 100,
'min_new_tokens': 1,
'type': 'dev',
'language': 'en'
},
'mmbench_test_cn_20231003': {
'root': 'data/mmbench/mmbench_test_cn_20231003.tsv',
'max_new_tokens': 100,
'min_new_tokens': 1,
'type': 'test',
'language': 'cn'
},
'mmbench_test_en_20231003': {
'root': 'data/mmbench/mmbench_test_en_20231003.tsv',
'max_new_tokens': 100,
'min_new_tokens': 1,
'type': 'test',
'language': 'en'
},
'ccbench_dev_cn': {
'root': 'data/mmbench/CCBench_legacy.tsv',
'max_new_tokens': 100,
'min_new_tokens': 1,
'type': 'dev',
'language': 'cn'
}
}
def collate_fn(batches, tokenizer):
pixel_values = torch.cat([_['pixel_values'] for _ in batches], dim=0)
questions = [_['question'] for _ in batches]
answers = [_['answer'] for _ in batches]
indexes = [_['index'] for _ in batches]
options = [_['option'] for _ in batches]
return pixel_values, questions, answers, indexes, options
class MMBenchDataset(torch.utils.data.Dataset):
def __init__(self, root, prompt, language, input_size=224, dynamic_image_size=False,
use_thumbnail=False, max_num=6):
self.df = pd.read_csv(root, sep='\t')
self.prompt = prompt
self.language = language
self.input_size = input_size
self.dynamic_image_size = dynamic_image_size
self.use_thumbnail = use_thumbnail
self.max_num = max_num
self.transform = build_transform(is_train=False, input_size=input_size)
def __len__(self):
return len(self.df)
def __getitem__(self, idx):
index = self.df.iloc[idx]['index']
image = self.df.iloc[idx]['image']
question = self.df.iloc[idx]['question']
answer = self.df.iloc[idx]['answer'] if 'answer' in self.df.iloc[0].keys() else None
# catetory = self.df.iloc[idx]['category']
# l2_catetory = self.df.iloc[idx]['l2-category']
image = Image.open(BytesIO(base64.b64decode(image))).convert('RGB')
if self.dynamic_image_size:
images = dynamic_preprocess(image, image_size=self.input_size,
use_thumbnail=self.use_thumbnail,
max_num=self.max_num)
else:
images = [image]
pixel_values = [self.transform(image) for image in images]
pixel_values = torch.stack(pixel_values)
option_candidate = ['A', 'B', 'C', 'D', 'E']
options = {
cand: self.load_from_df(idx, cand)
for cand in option_candidate
if self.load_from_df(idx, cand) is not None
}
hint = self.load_from_df(idx, 'hint')
if hint is not None:
question = hint + '\n' + question
for key, item in options.items():
question += f'\n{key}. {item}'
if self.language == 'cn':
question = question + '\n' + self.prompt['cn']
else:
question = question + '\n' + self.prompt['en']
return {
'question': question,
'pixel_values': pixel_values,
'answer': answer,
'index': index,
'option': options
}
def load_from_df(self, idx, key):
if key in self.df.iloc[idx] and not pd.isna(self.df.iloc[idx][key]):
return self.df.iloc[idx][key]
else:
return None
class InferenceSampler(torch.utils.data.sampler.Sampler):
def __init__(self, size):
self._size = int(size)
assert size > 0
self._rank = torch.distributed.get_rank()
self._world_size = torch.distributed.get_world_size()
self._local_indices = self._get_local_indices(size, self._world_size, self._rank)
@staticmethod
def _get_local_indices(total_size, world_size, rank):
shard_size = total_size // world_size
left = total_size % world_size
shard_sizes = [shard_size + int(r < left) for r in range(world_size)]
begin = sum(shard_sizes[:rank])
end = min(sum(shard_sizes[:rank + 1]), total_size)
return range(begin, end)
def __iter__(self):
yield from self._local_indices
def __len__(self):
return len(self._local_indices)
def post_process(pred, option):
pred = pred.strip()
option_candidate = list(option.keys())
if len(pred) == 1:
return pred
elif len(pred) != 1 and pred[0] in option_candidate:
return pred[0]
elif len(pred) != 1 and pred[0] not in option_candidate:
for k, v in option.items():
if v in pred:
return k
return pred
def evaluate_chat_model():
random.seed(args.seed)
for ds_name in args.datasets:
dataset = MMBenchDataset(
root=ds_collections[ds_name]['root'],
prompt=prompt,
language=ds_collections[ds_name]['language'],
input_size=image_size,
dynamic_image_size=args.dynamic,
use_thumbnail=use_thumbnail,
max_num=args.max_num
)
dataloader = torch.utils.data.DataLoader(
dataset=dataset,
sampler=InferenceSampler(len(dataset)),
batch_size=args.batch_size,
num_workers=args.num_workers,
pin_memory=True,
drop_last=False,
collate_fn=partial(collate_fn, tokenizer=tokenizer),
)
outputs = []
for _, (pixel_values, questions, answers, indexes, options) in tqdm(enumerate(dataloader)):
pixel_values = pixel_values.to(torch.bfloat16).cuda()
generation_config = dict(
num_beams=args.num_beams,
max_new_tokens=ds_collections[ds_name]['max_new_tokens'],
min_new_tokens=ds_collections[ds_name]['min_new_tokens'],
do_sample=True if args.temperature > 0 else False,
temperature=args.temperature,
)
pred = model.chat(
tokenizer=tokenizer,
pixel_values=pixel_values,
question=questions[0],
generation_config=generation_config,
verbose=True
)
preds = [post_process(pred, options[0])]
for question, pred, answer, index in zip(questions, preds, answers, indexes):
outputs.append({
'question': question,
'answer': pred,
'gt_answers': answer,
'index': int(index)
})
torch.distributed.barrier()
world_size = torch.distributed.get_world_size()
merged_outputs = [None for _ in range(world_size)]
torch.distributed.all_gather_object(merged_outputs, json.dumps(outputs))
merged_outputs = [json.loads(_) for _ in merged_outputs]
merged_outputs = [_ for _ in itertools.chain.from_iterable(merged_outputs)]
if torch.distributed.get_rank() == 0:
print(f'Evaluating {ds_name} ...')
time_prefix = time.strftime('%y%m%d%H%M%S', time.localtime())
results_file = f'{ds_name}_{time_prefix}.xlsx'
output_path = os.path.join(args.out_dir, results_file)
df = pd.read_table(ds_collections[ds_name]['root'])
cur_df = df.copy()
if 'mmbench' in ds_name:
cur_df = cur_df.drop(columns=['hint', 'category', 'source', 'image', 'comment', 'l2-category'])
cur_df.insert(6, 'prediction', None)
else:
cur_df = cur_df.drop(columns=['category', 'image'])
cur_df.insert(8, 'prediction', None)
for item in merged_outputs:
cur_df.loc[df['index'] == item['index'], 'prediction'] = item['answer']
cur_df.to_excel(output_path, index=False, engine='openpyxl')
print('Results saved to {}'.format(output_path))
if __name__ == '__main__':
parser = argparse.ArgumentParser()
parser.add_argument('--checkpoint', type=str, default='')
parser.add_argument('--datasets', type=str, default='mmbench_dev_20230712')
parser.add_argument('--batch-size', type=int, default=1)
parser.add_argument('--num-workers', type=int, default=1)
parser.add_argument('--num-beams', type=int, default=5)
parser.add_argument('--temperature', type=float, default=0.0)
parser.add_argument('--out-dir', type=str, default='results')
parser.add_argument('--seed', type=int, default=0)
parser.add_argument('--dynamic', action='store_true')
parser.add_argument('--max-num', type=int, default=6)
parser.add_argument('--load-in-8bit', action='store_true')
parser.add_argument('--load-in-4bit', action='store_true')
parser.add_argument('--auto', action='store_true')
args = parser.parse_args()
if not os.path.exists(args.out_dir):
os.makedirs(args.out_dir)
args.datasets = args.datasets.split(',')
print('datasets:', args.datasets)
assert args.batch_size == 1, 'Only batch size 1 is supported'
torch.distributed.init_process_group(
backend='nccl',
world_size=int(os.getenv('WORLD_SIZE', '1')),
rank=int(os.getenv('RANK', '0')),
)
torch.cuda.set_device(int(os.getenv('LOCAL_RANK', 0)))
if args.auto:
os.environ['CUDA_LAUNCH_BLOCKING'] = '1'
kwargs = {'device_map': 'auto'} if args.auto else {}
tokenizer = AutoTokenizer.from_pretrained(args.checkpoint, trust_remote_code=True, use_fast=False)
model = InternVLChatModel.from_pretrained(
args.checkpoint, low_cpu_mem_usage=True, torch_dtype=torch.bfloat16,
load_in_8bit=args.load_in_8bit, load_in_4bit=args.load_in_4bit, **kwargs).eval()
if not args.load_in_8bit and not args.load_in_4bit and not args.auto:
model = model.cuda()
image_size = model.config.force_image_size or model.config.vision_config.image_size
use_thumbnail = model.config.use_thumbnail
total_params = sum(p.numel() for p in model.parameters()) / 1e9
if total_params > 20 or args.dynamic:
args.num_beams = 1
print(f'[test] total_params: {total_params}B, use num_beams: {args.num_beams}')
else:
print(f'[test] total_params: {total_params}B')
print(f'[test] image_size: {image_size}')
print(f'[test] template: {model.config.template}')
print(f'[test] dynamic_image_size: {args.dynamic}')
print(f'[test] use_thumbnail: {use_thumbnail}')
print(f'[test] max_num: {args.max_num}')
prompt = {
'en': "Answer with the option's letter from the given choices directly.",
'cn': '请直接回答选项字母。'
}
evaluate_chat_model()
internvl_chat/eval/mme/README.md 0 → 100644
View file @ 1d5a34cf
# This is an automated calculation script for the acc, acc+, and score.
# You can directly run "python3 calculation.py" to get the evaluation results of LaVIN.
# In order to get the statistical results of your model:
(1) Fill all the files in "Your_Results", adding your model's responses:
Each file in "Your_Results" consists of:
Image_Name + "\\t" + Question + "\\t" + Ground_Truth_Answer + "\\n"
You need to add the responses of your model as:
Image_Name + "\\t" + Question + "\\t" + Ground_Truth_Answer + "\\t" + Your_Response + "\\n"
Note: if your responses contain "\\n", please delet it. For each question, your response can only be in one line, not across lines!
(2) run "python3 calculation.py --results_dir ./Your_Results"
internvl_chat/eval/mme/Your_Results/OCR.txt 0 → 100644
View file @ 1d5a34cf
0001.jpg Is the word in the logo "angie's"? Please answer yes or no. Yes
0001.jpg Is the word in the logo "angle's"? Please answer yes or no. No
0002.jpg Is the word in the logo "c'est cheese"? Please answer yes or no. Yes
0002.jpg Is the word in the logo "crest cheese"? Please answer yes or no. No
0003.jpg Is the word in the logo "beavertails pastry"? Please answer yes or no. Yes
0003.jpg Is the word in the logo "beavertalls pastry"? Please answer yes or no. No
0004.jpg Is the word in the logo "old market sundries"? Please answer yes or no. Yes
0004.jpg Is the word in the logo "old market hundreds"? Please answer yes or no. No
0005.jpg Is the word in the logo "kress"? Please answer yes or no. Yes
0005.jpg Is the word in the logo "dress"? Please answer yes or no. No
0006.jpg Is the word in the logo "the beatles story liver pool"? Please answer yes or no. Yes
0006.jpg Is the word in the logo "the beats story liver pool"? Please answer yes or no. No
0007.jpg Is the phone number in the picture "0131 555 6363"? Please answer yes or no. Yes
0007.jpg Is the phone number in the picture "0137 556 6363"? Please answer yes or no. No
0008.jpg Is the word in the logo "phil's market"? Please answer yes or no. Yes
0008.jpg Is the word in the logo "phll's market"? Please answer yes or no. No
0009.jpg Is the word in the logo "fenders diner"? Please answer yes or no. Yes
0009.jpg Is the word in the logo "finders diner"? Please answer yes or no. No
0010.jpg Is the word in the logo "high time coffee shop"? Please answer yes or no. Yes
0010.jpg Is the word in the logo "high tite cofeee shop"? Please answer yes or no. No
0011.jpg Is the word in the logo "ihop restaurant"? Please answer yes or no. Yes
0011.jpg Is the word in the logo "lhop restaurant"? Please answer yes or no. No
0012.jpg Is the word in the logo "casa grecque restaurants"? Please answer yes or no. Yes
0012.jpg Is the word in the logo "case grecque restaurants"? Please answer yes or no. No
0013.jpg Is the word in the picture "seabreeze motel"? Please answer yes or no. Yes
0013.jpg Is the word in the picture "seebreeze model"? Please answer yes or no. No
0014.jpg Is the word in the logo "penarth pier built 1894"? Please answer yes or no. Yes
0014.jpg Is the word in the logo "penarth pies buid 1894"? Please answer yes or no. No
0015.jpg Is the text in the picture "hollywood"? Please answer yes or no. Yes
0015.jpg Is the text in the picture "holly word"? Please answer yes or no. No
0016.jpg Is the word in the logo "shop rite"? Please answer yes or no. Yes
0016.jpg Is the word in the logo "stop rite"? Please answer yes or no. No
0017.jpg Is the word in the logo "hardco industrial construction"? Please answer yes or no. Yes
0017.jpg Is the word in the logo "hardto industal construction"? Please answer yes or no. No
0018.jpg Is the word in the logo "oldsmobile service"? Please answer yes or no. Yes
0018.jpg Is the word in the logo "old mobile service"? Please answer yes or no. No
0019.jpg Is the word in the logo "exchange hotel"? Please answer yes or no. Yes
0019.jpg Is the word in the logo "excharge hotel"? Please answer yes or no. No
0020.jpg Is the word in the logo "cold drinks"? Please answer yes or no. Yes
0020.jpg Is the word in the logo "cold rinks"? Please answer yes or no. No
internvl_chat/eval/mme/Your_Results/artwork.txt 0 → 100644
View file @ 1d5a34cf
10002.jpg Does this artwork exist in the form of painting? Please answer yes or no. Yes
10002.jpg Does this artwork exist in the form of glassware? Please answer yes or no. No
10049.jpg Does this artwork exist in the form of painting? Please answer yes or no. Yes
10049.jpg Does this artwork exist in the form of sculpture? Please answer yes or no. No
10256.jpg Does this artwork exist in the form of painting? Please answer yes or no. Yes
10256.jpg Does this artwork exist in the form of sculpture? Please answer yes or no. No
10358.jpg Does this artwork exist in the form of painting? Please answer yes or no. Yes
10358.jpg Does this artwork exist in the form of glassware? Please answer yes or no. No
10543.jpg Is this artwork displayed in fogg art museum, harvard university, cambridge? Please answer yes or no. Yes
10543.jpg Is this artwork displayed in museo civico, pistoia? Please answer yes or no. No
10581.jpg Does this artwork belong to the type of portrait? Please answer yes or no. Yes
10581.jpg Does this artwork belong to the type of genre? Please answer yes or no. No
1060.jpg Is this artwork created by antoniazzo romano? Please answer yes or no. Yes
1060.jpg Is this artwork created by gentile da fabriano? Please answer yes or no. No
10881.jpg Does this artwork exist in the form of painting? Please answer yes or no. Yes
10881.jpg Does this artwork exist in the form of tapestry? Please answer yes or no. No
10970.jpg Does this artwork belong to the type of religious? Please answer yes or no. Yes
10970.jpg Does this artwork belong to the type of study? Please answer yes or no. No
11276.jpg Does this artwork exist in the form of sculpture? Please answer yes or no. Yes
11276.jpg Does this artwork exist in the form of graphics? Please answer yes or no. No
11331.jpg Is this artwork created by donatello? Please answer yes or no. Yes
11331.jpg Is this artwork created by zichy, mihály? Please answer yes or no. No
11488.jpg Does this artwork belong to the type of mythological? Please answer yes or no. Yes
11488.jpg Does this artwork belong to the type of historical? Please answer yes or no. No
11724.jpg Is this artwork created by duccio di buoninsegna? Please answer yes or no. Yes
11724.jpg Is this artwork created by giani, felice? Please answer yes or no. No
11726.jpg Is this artwork titled temptation on the mountain (detail)? Please answer yes or no. Yes
11726.jpg Is this artwork titled in the forest of fontainebleau? Please answer yes or no. No
12133.jpg Is this artwork titled hand study with bible? Please answer yes or no. Yes
12133.jpg Is this artwork titled self-portrait aged 78? Please answer yes or no. No
12439.jpg Is this artwork created by dürer, albrecht? Please answer yes or no. Yes
12439.jpg Is this artwork created by koekkoek, barend cornelis? Please answer yes or no. No
12561.jpg Is this artwork created by eberlein, gustav heinrich? Please answer yes or no. Yes
12561.jpg Is this artwork created by gillemans, jan pauwel the younger? Please answer yes or no. No
12652.jpg Is this artwork displayed in stedelijk museum de lakenhal, leiden? Please answer yes or no. Yes
12652.jpg Is this artwork displayed in palazzo ducale, mantua? Please answer yes or no. No
12736.jpg Is this artwork displayed in cannon hall museum, barnsley? Please answer yes or no. Yes
12736.jpg Is this artwork displayed in protestant parish church, gelnhausen? Please answer yes or no. No
12902.jpg Is this artwork displayed in private collection? Please answer yes or no. Yes
12902.jpg Is this artwork displayed in musée national gustave-moreau, paris? Please answer yes or no. No
12908.jpg Is this artwork titled ruth and boaz? Please answer yes or no. Yes
12908.jpg Is this artwork titled view of dresden from the right bank of the elbe with the augustus bridge? Please answer yes or no. No
13091.jpg Is this artwork titled italianate landscape with figures by classical ruins? Please answer yes or no. Yes
13091.jpg Is this artwork titled two boys singing? Please answer yes or no. No
13174.jpg Is this artwork titled nobility? Please answer yes or no. Yes
13174.jpg Is this artwork titled aretino in the studio of tintoretto? Please answer yes or no. No
13239.jpg Is this artwork titled doge ziani receiving the benediction of pope alexander iii? Please answer yes or no. Yes
13239.jpg Is this artwork titled the adoration of the shepherds? Please answer yes or no. No
13288.jpg Does this artwork exist in the form of architecture? Please answer yes or no. Yes
13288.jpg Does this artwork exist in the form of metalwork? Please answer yes or no. No
13696.jpg Is this artwork displayed in pinacoteca nazionale, siena? Please answer yes or no. Yes
13696.jpg Is this artwork displayed in british embassy, paris? Please answer yes or no. No
13760.jpg Is this artwork titled noli me tangere? Please answer yes or no. Yes
13760.jpg Is this artwork titled profile study of a bearded man? Please answer yes or no. No
13821.jpg Is this artwork created by frangipane, niccolò? Please answer yes or no. Yes
13821.jpg Is this artwork created by drevet, pierre? Please answer yes or no. No
13901.jpg Does this artwork exist in the form of painting? Please answer yes or no. Yes
13901.jpg Does this artwork exist in the form of metalwork? Please answer yes or no. No
14283.jpg Does this artwork exist in the form of painting? Please answer yes or no. Yes
14283.jpg Does this artwork exist in the form of mosaic? Please answer yes or no. No
14499.jpg Does this artwork belong to the type of religious? Please answer yes or no. Yes
14499.jpg Does this artwork belong to the type of mythological? Please answer yes or no. No
14777.jpg Does this artwork belong to the type of religious? Please answer yes or no. Yes
14777.jpg Does this artwork belong to the type of historical? Please answer yes or no. No
15028.jpg Does this artwork belong to the type of portrait? Please answer yes or no. Yes
15028.jpg Does this artwork belong to the type of study? Please answer yes or no. No
15232.jpg Is this artwork created by giordano, luca? Please answer yes or no. Yes
15232.jpg Is this artwork created by heyerdahl, hans olaf? Please answer yes or no. No
15246.jpg Is this artwork displayed in palazzo medici riccardi, florence? Please answer yes or no. Yes
15246.jpg Is this artwork displayed in abbey church of sainte-foy, conques (aveyron)? Please answer yes or no. No
15311.jpg Is this artwork created by giorgione? Please answer yes or no. Yes
15311.jpg Is this artwork created by marilhat, prosper? Please answer yes or no. No
15989.jpg Is this artwork displayed in pinacoteca, vatican? Please answer yes or no. Yes
15989.jpg Is this artwork displayed in cathedral museum, zamora? Please answer yes or no. No
16006.jpg Is this artwork displayed in private collection? Please answer yes or no. Yes
16006.jpg Is this artwork displayed in cathedral of san geminiano, modena? Please answer yes or no. No
16249.jpg Does this artwork belong to the type of landscape? Please answer yes or no. Yes
16249.jpg Does this artwork belong to the type of religious? Please answer yes or no. No
16538.jpg Is this artwork created by gogh, vincent van? Please answer yes or no. Yes
16538.jpg Is this artwork created by altdorfer, albrecht? Please answer yes or no. No
16835.jpg Does this artwork exist in the form of painting? Please answer yes or no. Yes
16835.jpg Does this artwork exist in the form of illumination? Please answer yes or no. No
16911.jpg Is this artwork created by gossart, jan? Please answer yes or no. Yes
16911.jpg Is this artwork created by stanzione, massimo? Please answer yes or no. No
17311.jpg Does this artwork belong to the type of religious? Please answer yes or no. Yes
17311.jpg Does this artwork belong to the type of interior? Please answer yes or no. No
17317.jpg Is this artwork created by gozzoli, benozzo? Please answer yes or no. Yes
17317.jpg Is this artwork created by coriolano, cristoforo? Please answer yes or no. No
17535.jpg Is this artwork created by grebber, pieter de? Please answer yes or no. Yes
17535.jpg Is this artwork created by massys, quentin? Please answer yes or no. No
17823.jpg Is this artwork created by greuze, jean-baptiste? Please answer yes or no. Yes
17823.jpg Is this artwork created by landseer, sir edwin henry? Please answer yes or no. No
17838.jpg Does this artwork exist in the form of painting? Please answer yes or no. Yes
17838.jpg Does this artwork exist in the form of furniture? Please answer yes or no. No
17998.jpg Does this artwork belong to the type of religious? Please answer yes or no. Yes
17998.jpg Does this artwork belong to the type of genre? Please answer yes or no. No
18566.jpg Is this artwork created by hamen, juan van der? Please answer yes or no. Yes
18566.jpg Is this artwork created by starnina, gherardo di jacopo? Please answer yes or no. No
18604.jpg Is this artwork created by hardouin-mansart, jules? Please answer yes or no. Yes
18604.jpg Is this artwork created by kerseboom, friedrich? Please answer yes or no. No
18722.jpg Does this artwork exist in the form of painting? Please answer yes or no. Yes
18722.jpg Does this artwork exist in the form of sculpture? Please answer yes or no. No
1873.jpg Does this artwork exist in the form of architecture? Please answer yes or no. Yes
1873.jpg Does this artwork exist in the form of painting? Please answer yes or no. No
18902.jpg Is this artwork created by herrera, francisco de, the elder? Please answer yes or no. Yes
18902.jpg Is this artwork created by ingres, jean-auguste-dominique? Please answer yes or no. No
18926.jpg Is this artwork created by herring, john frederick the younger? Please answer yes or no. Yes
18926.jpg Is this artwork created by cozens, john robert? Please answer yes or no. No
19087.jpg Does this artwork exist in the form of painting? Please answer yes or no. Yes
19087.jpg Does this artwork exist in the form of metalwork? Please answer yes or no. No
19154.jpg Is this artwork titled portrait of the merchant georg gisze (detail)? Please answer yes or no. Yes
19154.jpg Is this artwork titled pair of table candlesticks? Please answer yes or no. No
19417.jpg Does this artwork exist in the form of painting? Please answer yes or no. Yes
19417.jpg Does this artwork exist in the form of mosaic? Please answer yes or no. No
19452.jpg Is this artwork titled the artist and his model? Please answer yes or no. Yes
19452.jpg Is this artwork titled the lovesick maiden (detail)? Please answer yes or no. No
19839.jpg Is this artwork created by janneck, franz christoph? Please answer yes or no. Yes
19839.jpg Is this artwork created by goupil, jules-adolphe? Please answer yes or no. No
19863.jpg Does this artwork belong to the type of religious? Please answer yes or no. Yes
19863.jpg Does this artwork belong to the type of mythological? Please answer yes or no. No
19993.jpg Is this artwork displayed in private collection? Please answer yes or no. Yes
19993.jpg Is this artwork displayed in cathedral of st paul, liège? Please answer yes or no. No
20176.jpg Does this artwork exist in the form of painting? Please answer yes or no. Yes
20176.jpg Does this artwork exist in the form of furniture? Please answer yes or no. No
20437.jpg Does this artwork exist in the form of painting? Please answer yes or no. Yes
20437.jpg Does this artwork exist in the form of tapestry? Please answer yes or no. No
20442.jpg Is this artwork created by kucharski, aleksander? Please answer yes or no. Yes
20442.jpg Is this artwork created by pourbus, frans the elder? Please answer yes or no. No
20455.jpg Does this artwork exist in the form of painting? Please answer yes or no. Yes
20455.jpg Does this artwork exist in the form of metalwork? Please answer yes or no. No
20483.jpg Is this artwork titled allegory of the regency? Please answer yes or no. Yes
20483.jpg Is this artwork titled breton woman bathing? Please answer yes or no. No
20490.jpg Does this artwork exist in the form of painting? Please answer yes or no. Yes
20490.jpg Does this artwork exist in the form of illumination? Please answer yes or no. No
20551.jpg Is this artwork created by lagrenée, jean-jacques? Please answer yes or no. Yes
20551.jpg Is this artwork created by scultori, diana? Please answer yes or no. No
20651.jpg Is this artwork titled a highland landscape? Please answer yes or no. Yes
20651.jpg Is this artwork titled a dog and a cat fighting in a kitchen interior? Please answer yes or no. No
20724.jpg Does this artwork belong to the type of portrait? Please answer yes or no. Yes
20724.jpg Does this artwork belong to the type of landscape? Please answer yes or no. No
21048.jpg Is this artwork created by lemoyne, jean-baptiste ii? Please answer yes or no. Yes
21048.jpg Is this artwork created by kneller, sir godfrey? Please answer yes or no. No
21097.jpg Does this artwork belong to the type of religious? Please answer yes or no. Yes
21097.jpg Does this artwork belong to the type of genre? Please answer yes or no. No
21244.jpg Does this artwork belong to the type of study? Please answer yes or no. Yes
21244.jpg Does this artwork belong to the type of portrait? Please answer yes or no. No
21469.jpg Does this artwork belong to the type of genre? Please answer yes or no. Yes
21469.jpg Does this artwork belong to the type of still-life? Please answer yes or no. No
21580.jpg Is this artwork created by linard, jacques? Please answer yes or no. Yes
21580.jpg Is this artwork created by bonino da campione? Please answer yes or no. No
21712.jpg Is this artwork titled st john the evangelist resuscitating drusiana? Please answer yes or no. Yes
21712.jpg Is this artwork titled la finette? Please answer yes or no. No
22329.jpg Is this artwork titled marriage of the virgin? Please answer yes or no. Yes
22329.jpg Is this artwork titled landscape with river and figures (detail)? Please answer yes or no. No
22366.jpg Does this artwork exist in the form of painting? Please answer yes or no. Yes
22366.jpg Does this artwork exist in the form of glassware? Please answer yes or no. No
22667.jpg Is this artwork displayed in private collection? Please answer yes or no. Yes
22667.jpg Is this artwork displayed in san francesco d'assisi, pavia? Please answer yes or no. No
22760.jpg Is this artwork titled madonna and child (detail)? Please answer yes or no. Yes
22760.jpg Is this artwork titled view of the south and east walls? Please answer yes or no. No
22842.jpg Is this artwork titled ukrainian peasant girl? Please answer yes or no. Yes
22842.jpg Is this artwork titled virtue crowning merit? Please answer yes or no. No
23229.jpg Is this artwork displayed in national gallery, london? Please answer yes or no. Yes
23229.jpg Is this artwork displayed in notre-dame-la-riche, tours? Please answer yes or no. No
23427.jpg Is this artwork displayed in the hermitage, st. petersburg? Please answer yes or no. Yes
23427.jpg Is this artwork displayed in national gallery of victoria, melbourne? Please answer yes or no. No
23465.jpg Is this artwork displayed in private collection? Please answer yes or no. Yes
23465.jpg Is this artwork displayed in cistertian church, zirc? Please answer yes or no. No
23824.jpg Is this artwork titled christ walking on the water? Please answer yes or no. Yes
23824.jpg Is this artwork titled mademoiselle romaine lacaux? Please answer yes or no. No
24122.jpg Is this artwork displayed in museo correr, venice? Please answer yes or no. Yes
24122.jpg Is this artwork displayed in church of brou, bourg-en-bresse? Please answer yes or no. No
24260.jpg Does this artwork exist in the form of painting? Please answer yes or no. Yes
24260.jpg Does this artwork exist in the form of illumination? Please answer yes or no. No
24291.jpg Is this artwork titled virgin and child with sts catherine, cecilia, barbara, and ursula? Please answer yes or no. Yes
24291.jpg Is this artwork titled sorrow? Please answer yes or no. No
24723.jpg Is this artwork titled tomb of henry the lion and his wife matilda? Please answer yes or no. Yes
24723.jpg Is this artwork titled god the father? Please answer yes or no. No
2490.jpg Does this artwork belong to the type of landscape? Please answer yes or no. Yes
2490.jpg Does this artwork belong to the type of mythological? Please answer yes or no. No
2507.jpg Is this artwork displayed in private collection? Please answer yes or no. Yes
2507.jpg Is this artwork displayed in st. vitus's cathedral, prague? Please answer yes or no. No
25312.jpg Does this artwork exist in the form of painting? Please answer yes or no. Yes
25312.jpg Does this artwork exist in the form of metalwork? Please answer yes or no. No
25476.jpg Is this artwork created by michelangelo buonarroti? Please answer yes or no. Yes
25476.jpg Is this artwork created by beuckelaer, joachim? Please answer yes or no. No
25492.jpg Does this artwork exist in the form of sculpture? Please answer yes or no. Yes
25492.jpg Does this artwork exist in the form of illumination? Please answer yes or no. No
25513.jpg Does this artwork belong to the type of religious? Please answer yes or no. Yes
25513.jpg Does this artwork belong to the type of landscape? Please answer yes or no. No
26521.jpg Does this artwork exist in the form of illumination? Please answer yes or no. Yes
26521.jpg Does this artwork exist in the form of furniture? Please answer yes or no. No
26973.jpg Does this artwork belong to the type of religious? Please answer yes or no. Yes
26973.jpg Does this artwork belong to the type of mythological? Please answer yes or no. No
27021.jpg Is this artwork created by miniaturist, german? Please answer yes or no. Yes
27021.jpg Is this artwork created by trinquesse, louis-rolland? Please answer yes or no. No
27662.jpg Does this artwork belong to the type of still-life? Please answer yes or no. Yes
27662.jpg Does this artwork belong to the type of mythological? Please answer yes or no. No
27936.jpg Does this artwork belong to the type of portrait? Please answer yes or no. Yes
27936.jpg Does this artwork belong to the type of interior? Please answer yes or no. No
28039.jpg Is this artwork displayed in cappella palatina, palermo? Please answer yes or no. Yes
28039.jpg Is this artwork displayed in musée des beaux-arts, chambéry? Please answer yes or no. No
28345.jpg Does this artwork exist in the form of painting? Please answer yes or no. Yes
28345.jpg Does this artwork exist in the form of tapestry? Please answer yes or no. No
28400.jpg Does this artwork belong to the type of religious? Please answer yes or no. Yes
28400.jpg Does this artwork belong to the type of portrait? Please answer yes or no. No
28698.jpg Does this artwork belong to the type of religious? Please answer yes or no. Yes
28698.jpg Does this artwork belong to the type of still-life? Please answer yes or no. No
28758.jpg Does this artwork exist in the form of painting? Please answer yes or no. Yes
28758.jpg Does this artwork exist in the form of graphics? Please answer yes or no. No
28974.jpg Is this artwork titled prayer before the meal? Please answer yes or no. Yes
28974.jpg Is this artwork titled rest in the mountains? Please answer yes or no. No
29266.jpg Is this artwork created by palma vecchio? Please answer yes or no. Yes
29266.jpg Is this artwork created by maris, jacobus hendricus? Please answer yes or no. No
30443.jpg Is this artwork titled the crucifixion with sts jerome and christopher? Please answer yes or no. Yes
30443.jpg Is this artwork titled tomb of michelangelo (detail)? Please answer yes or no. No
3085.jpg Is this artwork created by bartsius, willem? Please answer yes or no. Yes
3085.jpg Is this artwork created by oehme, ernst ferdinand? Please answer yes or no. No
30875.jpg Is this artwork created by pomarancio? Please answer yes or no. Yes
30875.jpg Is this artwork created by steen, jan? Please answer yes or no. No
3114.jpg Does this artwork belong to the type of religious? Please answer yes or no. Yes
3114.jpg Does this artwork belong to the type of study? Please answer yes or no. No
31808.jpg Is this artwork created by raffaello sanzio? Please answer yes or no. Yes
31808.jpg Is this artwork created by simon von taisten? Please answer yes or no. No
32147.jpg Is this artwork titled lucretia? Please answer yes or no. Yes
32147.jpg Is this artwork titled rinaldo abandoning armida (detail)? Please answer yes or no. No
3241.jpg Is this artwork titled holy family? Please answer yes or no. Yes
3241.jpg Is this artwork titled friedrich iii, the wise, and johann i, the constant, electors of saxony? Please answer yes or no. No
33017.jpg Does this artwork exist in the form of painting? Please answer yes or no. Yes
33017.jpg Does this artwork exist in the form of glassware? Please answer yes or no. No
33069.jpg Does this artwork belong to the type of historical? Please answer yes or no. Yes
33069.jpg Does this artwork belong to the type of interior? Please answer yes or no. No
33173.jpg Does this artwork exist in the form of painting? Please answer yes or no. Yes
33173.jpg Does this artwork exist in the form of graphics? Please answer yes or no. No
33753.jpg Is this artwork titled vanitas? Please answer yes or no. Yes
33753.jpg Is this artwork titled legend of st francis: 18. apparition at arles (detail)? Please answer yes or no. No
33854.jpg Does this artwork belong to the type of religious? Please answer yes or no. Yes
33854.jpg Does this artwork belong to the type of study? Please answer yes or no. No
339.jpg Is this artwork displayed in staatliche museen, berlin? Please answer yes or no. Yes
339.jpg Is this artwork displayed in national museum of religious carvings, valladolid? Please answer yes or no. No
33933.jpg Is this artwork titled madonna and child? Please answer yes or no. Yes
33933.jpg Is this artwork titled the bacino di san marco? Please answer yes or no. No
3404.jpg Is this artwork displayed in szépmûvészeti múzeum, budapest? Please answer yes or no. Yes
3404.jpg Is this artwork displayed in s. eustorgio, milan? Please answer yes or no. No
34109.jpg Is this artwork displayed in national gallery of art, washington? Please answer yes or no. Yes
34109.jpg Is this artwork displayed in abbey church of sainte-foy, conques? Please answer yes or no. No
34363.jpg Is this artwork displayed in museo del prado, madrid? Please answer yes or no. Yes
34363.jpg Is this artwork displayed in state tretyakov gallery, moscow? Please answer yes or no. No
34539.jpg Is this artwork titled the victory of eucharistic truth over heresy? Please answer yes or no. Yes
34539.jpg Is this artwork titled a sunday afternoon on the ile de la grande jatte? Please answer yes or no. No
34627.jpg Does this artwork belong to the type of landscape? Please answer yes or no. Yes
34627.jpg Does this artwork belong to the type of genre? Please answer yes or no. No
34638.jpg Does this artwork exist in the form of painting? Please answer yes or no. Yes
34638.jpg Does this artwork exist in the form of tapestry? Please answer yes or no. No
34669.jpg Does this artwork belong to the type of mythological? Please answer yes or no. Yes
34669.jpg Does this artwork belong to the type of historical? Please answer yes or no. No
35345.jpg Does this artwork belong to the type of religious? Please answer yes or no. Yes
35345.jpg Does this artwork belong to the type of landscape? Please answer yes or no. No
35439.jpg Is this artwork titled madonna and child with a host of musical angels? Please answer yes or no. Yes
35439.jpg Is this artwork titled garden in fontenay? Please answer yes or no. No
35460.jpg Is this artwork created by schinkel, karl friedrich? Please answer yes or no. Yes
35460.jpg Is this artwork created by giolfino, bartolomeo? Please answer yes or no. No
35486.jpg Does this artwork exist in the form of painting? Please answer yes or no. Yes
35486.jpg Does this artwork exist in the form of furniture? Please answer yes or no. No
35513.jpg Is this artwork created by schongauer, martin? Please answer yes or no. Yes
35513.jpg Is this artwork created by cassioli, amos? Please answer yes or no. No
3552.jpg Is this artwork titled madonna degli alberetti? Please answer yes or no. Yes
3552.jpg Is this artwork titled peter gillis? Please answer yes or no. No
35658.jpg Is this artwork created by sebastiano del piombo? Please answer yes or no. Yes
35658.jpg Is this artwork created by jacobsz., dirck? Please answer yes or no. No
35736.jpg Does this artwork belong to the type of religious? Please answer yes or no. Yes
35736.jpg Does this artwork belong to the type of still-life? Please answer yes or no. No
35861.jpg Does this artwork belong to the type of interior? Please answer yes or no. Yes
35861.jpg Does this artwork belong to the type of still-life? Please answer yes or no. No
36805.jpg Is this artwork titled weir? Please answer yes or no. Yes
36805.jpg Is this artwork titled view of the window wall? Please answer yes or no. No
36966.jpg Does this artwork belong to the type of portrait? Please answer yes or no. Yes
36966.jpg Does this artwork belong to the type of religious? Please answer yes or no. No
37010.jpg Is this artwork titled madonna and child with the young st john? Please answer yes or no. Yes
37010.jpg Is this artwork titled sketch for attila? Please answer yes or no. No
37077.jpg Does this artwork belong to the type of religious? Please answer yes or no. Yes
37077.jpg Does this artwork belong to the type of still-life? Please answer yes or no. No
37439.jpg Is this artwork titled the message? Please answer yes or no. Yes
37439.jpg Is this artwork titled the descent from the cross? Please answer yes or no. No
37819.jpg Is this artwork created by tiepolo, giovanni battista? Please answer yes or no. Yes
37819.jpg Is this artwork created by kerricx, willem ignatius? Please answer yes or no. No
37866.jpg Does this artwork belong to the type of mythological? Please answer yes or no. Yes
37866.jpg Does this artwork belong to the type of still-life? Please answer yes or no. No
381.jpg Does this artwork exist in the form of painting? Please answer yes or no. Yes
381.jpg Does this artwork exist in the form of architecture? Please answer yes or no. No
38178.jpg Is this artwork created by tintoretto? Please answer yes or no. Yes
38178.jpg Is this artwork created by morel, jean-baptiste? Please answer yes or no. No
38536.jpg Does this artwork exist in the form of painting? Please answer yes or no. Yes
38536.jpg Does this artwork exist in the form of furniture? Please answer yes or no. No
38546.jpg Does this artwork exist in the form of painting? Please answer yes or no. Yes
38546.jpg Does this artwork exist in the form of metalwork? Please answer yes or no. No
38694.jpg Does this artwork exist in the form of painting? Please answer yes or no. Yes
38694.jpg Does this artwork exist in the form of metalwork? Please answer yes or no. No
38740.jpg Is this artwork displayed in musée toulouse-lautrec, albi? Please answer yes or no. Yes
38740.jpg Is this artwork displayed in kupferstichkabinett, gotha? Please answer yes or no. No
38881.jpg Does this artwork belong to the type of genre? Please answer yes or no. Yes
38881.jpg Does this artwork belong to the type of religious? Please answer yes or no. No
38993.jpg Does this artwork exist in the form of painting? Please answer yes or no. Yes
38993.jpg Does this artwork exist in the form of illumination? Please answer yes or no. No
39026.jpg Does this artwork belong to the type of religious? Please answer yes or no. Yes
39026.jpg Does this artwork belong to the type of historical? Please answer yes or no. No
39124.jpg Does this artwork exist in the form of painting? Please answer yes or no. Yes
39124.jpg Does this artwork exist in the form of graphics? Please answer yes or no. No
39188.jpg Does this artwork exist in the form of painting? Please answer yes or no. Yes
39188.jpg Does this artwork exist in the form of architecture? Please answer yes or no. No
39482.jpg Does this artwork exist in the form of painting? Please answer yes or no. Yes
39482.jpg Does this artwork exist in the form of metalwork? Please answer yes or no. No
39556.jpg Is this artwork created by unknown master, dutch? Please answer yes or no. Yes
39556.jpg Is this artwork created by cuyp, benjamin gerritsz.? Please answer yes or no. No
41036.jpg Is this artwork displayed in kunsthistorisches museum, vienna? Please answer yes or no. Yes
41036.jpg Is this artwork displayed in national museum of art, minsk? Please answer yes or no. No
41371.jpg Does this artwork exist in the form of painting? Please answer yes or no. Yes
41371.jpg Does this artwork exist in the form of architecture? Please answer yes or no. No
41484.jpg Does this artwork belong to the type of religious? Please answer yes or no. Yes
41484.jpg Does this artwork belong to the type of historical? Please answer yes or no. No
41594.jpg Is this artwork created by veronese, paolo? Please answer yes or no. Yes
41594.jpg Is this artwork created by jeaurat, etienne? Please answer yes or no. No
416.jpg Does this artwork exist in the form of sculpture? Please answer yes or no. Yes
416.jpg Does this artwork exist in the form of others? Please answer yes or no. No
41653.jpg Is this artwork titled view of the sala del collegio? Please answer yes or no. Yes
41653.jpg Is this artwork titled reine lefebvre and margot? Please answer yes or no. No
41944.jpg Does this artwork exist in the form of painting? Please answer yes or no. Yes
41944.jpg Does this artwork exist in the form of mosaic? Please answer yes or no. No
42152.jpg Is this artwork titled the pieterskerk in leiden? Please answer yes or no. Yes
42152.jpg Is this artwork titled portrait of cardinal reginald pole? Please answer yes or no. No
42288.jpg Does this artwork exist in the form of painting? Please answer yes or no. Yes
42288.jpg Does this artwork exist in the form of stained-glass? Please answer yes or no. No
42303.jpg Is this artwork displayed in art museum, cincinnati? Please answer yes or no. Yes
42303.jpg Is this artwork displayed in banca del monte di bologna e ravenna, bologna? Please answer yes or no. No
42401.jpg Is this artwork created by waldmüller, fedinand georg? Please answer yes or no. Yes
42401.jpg Is this artwork created by seeman, enoch? Please answer yes or no. No
42447.jpg Is this artwork displayed in musée du louvre, paris? Please answer yes or no. Yes
42447.jpg Is this artwork displayed in santa catarina, pisa? Please answer yes or no. No
42585.jpg Is this artwork created by werff, pieter van der? Please answer yes or no. Yes
42585.jpg Is this artwork created by domenichini, apollonio? Please answer yes or no. No
42706.jpg Is this artwork displayed in musée du louvre, paris? Please answer yes or no. Yes
42706.jpg Is this artwork displayed in galleria nazionale d'arte moderna e contemporanea, rome? Please answer yes or no. No
42796.jpg Is this artwork displayed in private collection? Please answer yes or no. Yes
42796.jpg Is this artwork displayed in museo di san salvi, florence? Please answer yes or no. No
42857.jpg Does this artwork belong to the type of landscape? Please answer yes or no. Yes
42857.jpg Does this artwork belong to the type of study? Please answer yes or no. No
42905.jpg Is this artwork created by wit, jacob de? Please answer yes or no. Yes
42905.jpg Is this artwork created by vittone, bernardo antonio? Please answer yes or no. No
42941.jpg Is this artwork created by witte, emanuel de? Please answer yes or no. Yes
42941.jpg Is this artwork created by bicci di neri? Please answer yes or no. No
42956.jpg Is this artwork titled view of rome with the tiberand castel sant'angelo? Please answer yes or no. Yes
42956.jpg Is this artwork titled st bonaventure enters the franciscan order? Please answer yes or no. No
42987.jpg Is this artwork created by witz, konrad? Please answer yes or no. Yes
42987.jpg Is this artwork created by christus, petrus? Please answer yes or no. No
43142.jpg Does this artwork belong to the type of mythological? Please answer yes or no. Yes
43142.jpg Does this artwork belong to the type of interior? Please answer yes or no. No
43175.jpg Is this artwork displayed in private collection? Please answer yes or no. Yes
43175.jpg Is this artwork displayed in smith college museum of art, northampton? Please answer yes or no. No
43349.jpg Is this artwork created by zuccarelli, francesco? Please answer yes or no. Yes
43349.jpg Is this artwork created by baccanelli, giovanni antonio di giulio? Please answer yes or no. No
43445.jpg Does this artwork belong to the type of religious? Please answer yes or no. Yes
43445.jpg Does this artwork belong to the type of interior? Please answer yes or no. No
4836.jpg Is this artwork displayed in villa cornaro, piombino dese? Please answer yes or no. Yes
4836.jpg Is this artwork displayed in palais saint-vaast, arras? Please answer yes or no. No
5227.jpg Is this artwork created by botticelli, sandro? Please answer yes or no. Yes
5227.jpg Is this artwork created by vigri, caterina? Please answer yes or no. No
526.jpg Does this artwork exist in the form of painting? Please answer yes or no. Yes
526.jpg Does this artwork exist in the form of tapestry? Please answer yes or no. No
5906.jpg Is this artwork created by bronzino, agnolo? Please answer yes or no. Yes
5906.jpg Is this artwork created by pellegrino da san daniele? Please answer yes or no. No
6168.jpg Does this artwork exist in the form of graphics? Please answer yes or no. Yes
6168.jpg Does this artwork exist in the form of tapestry? Please answer yes or no. No
6297.jpg Is this artwork titled peasants making merry outside a tavern 'the swan'? Please answer yes or no. Yes
6297.jpg Is this artwork titled allegory of quietude? Please answer yes or no. No
6478.jpg Does this artwork belong to the type of religious? Please answer yes or no. Yes
6478.jpg Does this artwork belong to the type of genre? Please answer yes or no. No
6969.jpg Is this artwork titled letizia ramolino bonaparte? Please answer yes or no. Yes
6969.jpg Is this artwork titled job and his daughters? Please answer yes or no. No
701.jpg Does this artwork exist in the form of painting? Please answer yes or no. Yes
701.jpg Does this artwork exist in the form of others? Please answer yes or no. No
7702.jpg Is this artwork titled reine lefebvre and margot? Please answer yes or no. Yes
7702.jpg Is this artwork titled fire in the oil depot at san marcuola? Please answer yes or no. No
8101.jpg Is this artwork displayed in museu de arte, são paulo? Please answer yes or no. Yes
8101.jpg Is this artwork displayed in national széchényi library, budapest? Please answer yes or no. No
815.jpg Does this artwork exist in the form of painting? Please answer yes or no. Yes
815.jpg Does this artwork exist in the form of furniture? Please answer yes or no. No
8797.jpg Is this artwork created by coecke van aelst, pieter? Please answer yes or no. Yes
8797.jpg Is this artwork created by abaquesne, masséot? Please answer yes or no. No
8885.jpg Is this artwork displayed in art museum, saint louis? Please answer yes or no. Yes
8885.jpg Is this artwork displayed in museo civico d'arte antica, turin? Please answer yes or no. No
9153.jpg Is this artwork displayed in galleria nazionale, parma? Please answer yes or no. Yes
9153.jpg Is this artwork displayed in hospital de san bernardo, seville? Please answer yes or no. No
9395.jpg Does this artwork exist in the form of painting? Please answer yes or no. Yes
9395.jpg Does this artwork exist in the form of stained-glass? Please answer yes or no. No
9405.jpg Is this artwork created by courbet, gustave? Please answer yes or no. Yes
9405.jpg Is this artwork created by milani, aureliano? Please answer yes or no. No
9599.jpg Does this artwork exist in the form of painting? Please answer yes or no. Yes
9599.jpg Does this artwork exist in the form of ceramics? Please answer yes or no. No
995.jpg Does this artwork exist in the form of sculpture? Please answer yes or no. Yes
995.jpg Does this artwork exist in the form of painting? Please answer yes or no. No
internvl_chat/eval/mme/Your_Results/celebrity.txt 0 → 100644
View file @ 1d5a34cf
tt0032138_shot_0395_img_0.jpg Is the actor inside the red bounding box named Frank Morgan? Please answer yes or no. Yes
tt0032138_shot_0395_img_0.jpg Is the actor inside the red bounding box named Eric Schniewind? Please answer yes or no. No
tt0035423_shot_0464_img_0.jpg Is the actor inside the red bounding box called Hugh Jackman? Please answer yes or no. Yes
tt0035423_shot_0464_img_0.jpg Is the actor inside the red bounding box called Lizzie Hopley? Please answer yes or no. No
tt0038650_shot_0737_img_1.jpg Is the person inside the red bounding box called James Stewart? Please answer yes or no. Yes
tt0038650_shot_0737_img_1.jpg Is the person inside the red bounding box called Phil Selway? Please answer yes or no. No
tt0047396_shot_0333_img_0.jpg Is the actor inside the red bounding box named James Stewart? Please answer yes or no. Yes
tt0047396_shot_0333_img_0.jpg Is the actor inside the red bounding box named Ron Blair? Please answer yes or no. No
tt0048545_shot_0124_img_0.jpg Is the actor inside the red bounding box called Natalie Wood? Please answer yes or no. Yes
tt0048545_shot_0124_img_0.jpg Is the actor inside the red bounding box called Rebecca Jackson Mendoza? Please answer yes or no. No
tt0049470_shot_0279_img_0.jpg Is the person inside the red bounding box called James Stewart? Please answer yes or no. Yes
tt0049470_shot_0279_img_0.jpg Is the person inside the red bounding box called Matt Pashkow? Please answer yes or no. No
tt0049730_shot_0273_img_0.jpg Is the person inside the red bounding box called Vera Miles? Please answer yes or no. Yes
tt0049730_shot_0273_img_0.jpg Is the person inside the red bounding box called Addie Yungmee? Please answer yes or no. No
tt0052357_shot_0511_img_0.jpg Is the actor inside the red bounding box called Kim Novak? Please answer yes or no. Yes
tt0052357_shot_0511_img_0.jpg Is the actor inside the red bounding box called Abigail Van Alyn? Please answer yes or no. No
tt0053221_shot_0197_img_0.jpg Is the actor inside the red bounding box named John Wayne? Please answer yes or no. Yes
tt0053221_shot_0197_img_0.jpg Is the actor inside the red bounding box named Claude-Oliver Rudolph? Please answer yes or no. No
tt0054167_shot_0122_img_0.jpg Is the person inside the red bounding box called Anna Massey? Please answer yes or no. Yes
tt0054167_shot_0122_img_0.jpg Is the person inside the red bounding box called Eddie Tagoe? Please answer yes or no. No
tt0056869_shot_0320_img_0.jpg Is the person inside the red bounding box called Tippi Hedren? Please answer yes or no. Yes
tt0056869_shot_0320_img_0.jpg Is the person inside the red bounding box called Denise Mack? Please answer yes or no. No
tt0056923_shot_0835_img_0.jpg Is the actor inside the red bounding box called Audrey Hepburn? Please answer yes or no. Yes
tt0056923_shot_0835_img_0.jpg Is the actor inside the red bounding box called Chris April? Please answer yes or no. No
tt0057115_shot_0686_img_0.jpg Is the person inside the red bounding box named James Garner? Please answer yes or no. Yes
tt0057115_shot_0686_img_0.jpg Is the person inside the red bounding box named Chutimon Chuengcharoensukying? Please answer yes or no. No
tt0058331_shot_0353_img_0.jpg Is the actor inside the red bounding box named Julie Andrews? Please answer yes or no. Yes
tt0058331_shot_0353_img_0.jpg Is the actor inside the red bounding box named Ed Geldart? Please answer yes or no. No
tt0058461_shot_0901_img_0.jpg Is the actor inside the red bounding box called Gian Maria Volontè? Please answer yes or no. Yes
tt0058461_shot_0901_img_0.jpg Is the actor inside the red bounding box called Jennifer Connelly? Please answer yes or no. No
tt0061418_shot_0148_img_0.jpg Is the actor inside the red bounding box named Faye Dunaway? Please answer yes or no. Yes
tt0061418_shot_0148_img_0.jpg Is the actor inside the red bounding box named Warona Seane? Please answer yes or no. No
tt0061722_shot_0259_img_0.jpg Is the actor inside the red bounding box called Dustin Hoffman? Please answer yes or no. Yes
tt0061722_shot_0259_img_0.jpg Is the actor inside the red bounding box called Christopher Olsen? Please answer yes or no. No
tt0062622_shot_0291_img_0.jpg Is the actor inside the red bounding box named Keir Dullea? Please answer yes or no. Yes
tt0062622_shot_0291_img_0.jpg Is the actor inside the red bounding box named Frank Albanese? Please answer yes or no. No
tt0063442_shot_0702_img_0.jpg Is the actor inside the red bounding box called Linda Harrison? Please answer yes or no. Yes
tt0063442_shot_0702_img_0.jpg Is the actor inside the red bounding box called Michael McKean? Please answer yes or no. No
tt0064115_shot_0367_img_0.jpg Is the actor inside the red bounding box named Robert Redford? Please answer yes or no. Yes
tt0064115_shot_0367_img_0.jpg Is the actor inside the red bounding box named Cooper Murray? Please answer yes or no. No
tt0064665_shot_0300_img_0.jpg Is the actor inside the red bounding box called Jon Voight? Please answer yes or no. Yes
tt0064665_shot_0300_img_0.jpg Is the actor inside the red bounding box called Harvey Meyer? Please answer yes or no. No
tt0065214_shot_0366_img_0.jpg Is the person inside the red bounding box called Robert Ryan? Please answer yes or no. Yes
tt0065214_shot_0366_img_0.jpg Is the person inside the red bounding box called Victor Verhaeghe? Please answer yes or no. No
tt0065724_shot_0320_img_1.jpg Is the person inside the red bounding box named Karen Black? Please answer yes or no. Yes
tt0065724_shot_0320_img_1.jpg Is the person inside the red bounding box named Nick Discenza? Please answer yes or no. No
tt0066026_shot_0085_img_0.jpg Is the person inside the red bounding box called Donald Sutherland? Please answer yes or no. Yes
tt0066026_shot_0085_img_0.jpg Is the person inside the red bounding box called Michael Wollet? Please answer yes or no. No
tt0066921_shot_0631_img_0.jpg Is the actor inside the red bounding box called Malcolm McDowell? Please answer yes or no. Yes
tt0066921_shot_0631_img_0.jpg Is the actor inside the red bounding box called Darling Légitimus? Please answer yes or no. No
tt0067116_shot_0122_img_0.jpg Is the actor inside the red bounding box called Gene Hackman? Please answer yes or no. Yes
tt0067116_shot_0122_img_0.jpg Is the actor inside the red bounding box called Russell G. Jones? Please answer yes or no. No
tt0068646_shot_0166_img_0.jpg Is the actor inside the red bounding box called Marlon Brando? Please answer yes or no. Yes
tt0068646_shot_0166_img_0.jpg Is the actor inside the red bounding box called Voltaire Sterling? Please answer yes or no. No
tt0069762_shot_0723_img_0.jpg Is the person inside the red bounding box named Sissy Spacek? Please answer yes or no. Yes
tt0069762_shot_0723_img_0.jpg Is the person inside the red bounding box named Monica Giordano? Please answer yes or no. No
tt0070047_shot_0255_img_0.jpg Is the actor inside the red bounding box called Ellen Burstyn? Please answer yes or no. Yes
tt0070047_shot_0255_img_0.jpg Is the actor inside the red bounding box called Shawnee Smith? Please answer yes or no. No
tt0070379_shot_0569_img_0.jpg Is the actor inside the red bounding box named Richard Romanus? Please answer yes or no. Yes
tt0070379_shot_0569_img_0.jpg Is the actor inside the red bounding box named Valerie Colgan? Please answer yes or no. No
tt0070511_shot_0639_img_0.jpg Is the person inside the red bounding box called Dustin Hoffman? Please answer yes or no. Yes
tt0070511_shot_0639_img_0.jpg Is the person inside the red bounding box called Fernando Lueches? Please answer yes or no. No
tt0070735_shot_0818_img_0.jpg Is the person inside the red bounding box named Robert Redford? Please answer yes or no. Yes
tt0070735_shot_0818_img_0.jpg Is the person inside the red bounding box named Ellin Dennis? Please answer yes or no. No
tt0070849_shot_0021_img_1.jpg Is the person inside the red bounding box named Maria Schneider? Please answer yes or no. Yes
tt0070849_shot_0021_img_1.jpg Is the person inside the red bounding box named Mary Kellogg? Please answer yes or no. No
tt0071315_shot_0153_img_0.jpg Is the actor inside the red bounding box named Faye Dunaway? Please answer yes or no. Yes
tt0071315_shot_0153_img_0.jpg Is the actor inside the red bounding box named Kelly Hitman? Please answer yes or no. No
tt0071562_shot_0684_img_0.jpg Is the actor inside the red bounding box named Al Pacino? Please answer yes or no. Yes
tt0071562_shot_0684_img_0.jpg Is the actor inside the red bounding box named Debie Jarczewski? Please answer yes or no. No
tt0072684_shot_0512_img_1.jpg Is the person inside the red bounding box named Marisa Berenson? Please answer yes or no. Yes
tt0072684_shot_0512_img_1.jpg Is the person inside the red bounding box named Graham Bohea? Please answer yes or no. No
tt0073195_shot_0280_img_0.jpg Is the actor inside the red bounding box named Roy Scheider? Please answer yes or no. Yes
tt0073195_shot_0280_img_0.jpg Is the actor inside the red bounding box named Abdul Qadir Farookh? Please answer yes or no. No
tt0073629_shot_0700_img_0.jpg Is the person inside the red bounding box named Barry Bostwick? Please answer yes or no. Yes
tt0073629_shot_0700_img_0.jpg Is the person inside the red bounding box named Johnny Galecki? Please answer yes or no. No
tt0074119_shot_0814_img_0.jpg Is the actor inside the red bounding box called Robert Redford? Please answer yes or no. Yes
tt0074119_shot_0814_img_0.jpg Is the actor inside the red bounding box called Delroy Lindo? Please answer yes or no. No
tt0074285_shot_0535_img_1.jpg Is the person inside the red bounding box named William Katt? Please answer yes or no. Yes
tt0074285_shot_0535_img_1.jpg Is the person inside the red bounding box named Stephen Rider? Please answer yes or no. No
tt0075148_shot_0618_img_0.jpg Is the actor inside the red bounding box called Sylvester Stallone? Please answer yes or no. Yes
tt0075148_shot_0618_img_0.jpg Is the actor inside the red bounding box called Eric Hatch? Please answer yes or no. No
tt0075686_shot_0373_img_0.jpg Is the actor inside the red bounding box called Woody Allen? Please answer yes or no. Yes
tt0075686_shot_0373_img_0.jpg Is the actor inside the red bounding box called Penny Wallace? Please answer yes or no. No
tt0076729_shot_0451_img_0.jpg Is the actor inside the red bounding box called Sally Field? Please answer yes or no. Yes
tt0076729_shot_0451_img_0.jpg Is the actor inside the red bounding box called Giorgio Libassi? Please answer yes or no. No
tt0076759_shot_0930_img_0.jpg Is the actor inside the red bounding box called Harrison Ford? Please answer yes or no. Yes
tt0076759_shot_0930_img_0.jpg Is the actor inside the red bounding box called Ryoko Sadoshima? Please answer yes or no. No
tt0077402_shot_1220_img_0.jpg Is the person inside the red bounding box named Scott H. Reiniger? Please answer yes or no. Yes
tt0077402_shot_1220_img_0.jpg Is the person inside the red bounding box named Chris Delaney? Please answer yes or no. No
tt0077405_shot_0150_img_0.jpg Is the actor inside the red bounding box named Sam Shepard? Please answer yes or no. Yes
tt0077405_shot_0150_img_0.jpg Is the actor inside the red bounding box named Bijou Phillips? Please answer yes or no. No
tt0077416_shot_1442_img_0.jpg Is the person inside the red bounding box named Robert De Niro? Please answer yes or no. Yes
tt0077416_shot_1442_img_0.jpg Is the person inside the red bounding box named Stu Smith? Please answer yes or no. No
tt0077651_shot_0133_img_0.jpg Is the person inside the red bounding box called Jamie Lee Curtis? Please answer yes or no. Yes
tt0077651_shot_0133_img_0.jpg Is the person inside the red bounding box called Paris Arrowsmith? Please answer yes or no. No
tt0078788_shot_1434_img_0.jpg Is the person inside the red bounding box called Martin Sheen? Please answer yes or no. Yes
tt0078788_shot_1434_img_0.jpg Is the person inside the red bounding box called Le Capriccio Français? Please answer yes or no. No
tt0078841_shot_0692_img_0.jpg Is the actor inside the red bounding box named Shirley MacLaine? Please answer yes or no. Yes
tt0078841_shot_0692_img_0.jpg Is the actor inside the red bounding box named Tomas Choy? Please answer yes or no. No
tt0079417_shot_0735_img_0.jpg Is the actor inside the red bounding box called Meryl Streep? Please answer yes or no. Yes
tt0079417_shot_0735_img_0.jpg Is the actor inside the red bounding box called Ross Lacy? Please answer yes or no. No
tt0079470_shot_0798_img_0.jpg Is the person inside the red bounding box named Eric Idle? Please answer yes or no. Yes
tt0079470_shot_0798_img_0.jpg Is the person inside the red bounding box named Quincy Taylor? Please answer yes or no. No
tt0079945_shot_1411_img_0.jpg Is the person inside the red bounding box named Persis Khambatta? Please answer yes or no. Yes
tt0079945_shot_1411_img_0.jpg Is the person inside the red bounding box named Alison Waddell? Please answer yes or no. No
tt0080339_shot_0711_img_0.jpg Is the actor inside the red bounding box named Robert Hays? Please answer yes or no. Yes
tt0080339_shot_0711_img_0.jpg Is the actor inside the red bounding box named Grace Sullivan? Please answer yes or no. No
tt0080684_shot_1574_img_2.jpg Is the actor inside the red bounding box called Mark Hamill? Please answer yes or no. Yes
tt0080684_shot_1574_img_2.jpg Is the actor inside the red bounding box called Rodion Salnikov? Please answer yes or no. No
tt0081505_shot_0449_img_0.jpg Is the actor inside the red bounding box called Shelley Duvall? Please answer yes or no. Yes
tt0081505_shot_0449_img_0.jpg Is the actor inside the red bounding box called Antony Carrick? Please answer yes or no. No
tt0082089_shot_0046_img_0.jpg Is the actor inside the red bounding box named Kathleen Turner? Please answer yes or no. Yes
tt0082089_shot_0046_img_0.jpg Is the actor inside the red bounding box named Aaron Henderson? Please answer yes or no. No
tt0082198_shot_1353_img_0.jpg Is the person inside the red bounding box named Arnold Schwarzenegger? Please answer yes or no. Yes
tt0082198_shot_1353_img_0.jpg Is the person inside the red bounding box named Tim Herlihy? Please answer yes or no. No
tt0082971_shot_0831_img_0.jpg Is the actor inside the red bounding box called Harrison Ford? Please answer yes or no. Yes
tt0082971_shot_0831_img_0.jpg Is the actor inside the red bounding box called Richard Angarola? Please answer yes or no. No
tt0083658_shot_0963_img_0.jpg Is the person inside the red bounding box called Rutger Hauer? Please answer yes or no. Yes
tt0083658_shot_0963_img_0.jpg Is the person inside the red bounding box called Stéphane Julien? Please answer yes or no. No
tt0083866_shot_0364_img_0.jpg Is the actor inside the red bounding box called Robert MacNaughton? Please answer yes or no. Yes
tt0083866_shot_0364_img_0.jpg Is the actor inside the red bounding box called Seam Turay? Please answer yes or no. No
tt0083907_shot_0633_img_0.jpg Is the actor inside the red bounding box named Bruce Campbell? Please answer yes or no. Yes
tt0083907_shot_0633_img_0.jpg Is the actor inside the red bounding box named Kaden Leos? Please answer yes or no. No
tt0083929_shot_0405_img_0.jpg Is the actor inside the red bounding box named Jennifer Jason Leigh? Please answer yes or no. Yes
tt0083929_shot_0405_img_0.jpg Is the actor inside the red bounding box named Eric D. Sandgren? Please answer yes or no. No
tt0084726_shot_0283_img_0.jpg Is the actor inside the red bounding box named Leonard Nimoy? Please answer yes or no. Yes
tt0084726_shot_0283_img_0.jpg Is the actor inside the red bounding box named John Cusack? Please answer yes or no. No
tt0086190_shot_0815_img_0.jpg Is the actor inside the red bounding box named Carrie Fisher? Please answer yes or no. Yes
tt0086190_shot_0815_img_0.jpg Is the actor inside the red bounding box named Ernie Adams? Please answer yes or no. No
tt0086250_shot_1079_img_0.jpg Is the actor inside the red bounding box called Steven Bauer? Please answer yes or no. Yes
tt0086250_shot_1079_img_0.jpg Is the actor inside the red bounding box called Bill Nunn? Please answer yes or no. No
tt0086856_shot_0929_img_0.jpg Is the actor inside the red bounding box called Peter Weller? Please answer yes or no. Yes
tt0086856_shot_0929_img_0.jpg Is the actor inside the red bounding box called Tracee Cocco? Please answer yes or no. No
tt0086879_shot_0158_img_0.jpg Is the person inside the red bounding box called Elizabeth Berridge? Please answer yes or no. Yes
tt0086879_shot_0158_img_0.jpg Is the person inside the red bounding box called Ralph Ineson? Please answer yes or no. No
tt0087332_shot_0798_img_0.jpg Is the person inside the red bounding box called Bill Murray? Please answer yes or no. Yes
tt0087332_shot_0798_img_0.jpg Is the person inside the red bounding box called Jiao Xu? Please answer yes or no. No
tt0087469_shot_0049_img_2.jpg Is the person inside the red bounding box named Harrison Ford? Please answer yes or no. Yes
tt0087469_shot_0049_img_2.jpg Is the person inside the red bounding box named Paulo Benedeti? Please answer yes or no. No
tt0088847_shot_0109_img_0.jpg Is the actor inside the red bounding box named Anthony Michael Hall? Please answer yes or no. Yes
tt0088847_shot_0109_img_0.jpg Is the actor inside the red bounding box named Luis Javier? Please answer yes or no. No
tt0088944_shot_0634_img_0.jpg Is the actor inside the red bounding box named Arnold Schwarzenegger? Please answer yes or no. Yes
tt0088944_shot_0634_img_0.jpg Is the actor inside the red bounding box named Shaine Jones? Please answer yes or no. No
tt0088993_shot_0569_img_0.jpg Is the actor inside the red bounding box called George A. Romero? Please answer yes or no. Yes
tt0088993_shot_0569_img_0.jpg Is the actor inside the red bounding box called James Eckhouse? Please answer yes or no. No
tt0089218_shot_0327_img_0.jpg Is the person inside the red bounding box named Sean Astin? Please answer yes or no. Yes
tt0089218_shot_0327_img_0.jpg Is the person inside the red bounding box named Dan Hunter? Please answer yes or no. No
tt0089881_shot_0034_img_0.jpg Is the actor inside the red bounding box called Tatsuya Nakadai? Please answer yes or no. Yes
tt0089881_shot_0034_img_0.jpg Is the actor inside the red bounding box called Nancy Vee? Please answer yes or no. No
tt0090022_shot_0464_img_0.jpg Is the actor inside the red bounding box called Scott Glenn? Please answer yes or no. Yes
tt0090022_shot_0464_img_0.jpg Is the actor inside the red bounding box called Robert Ryan? Please answer yes or no. No
tt0090605_shot_0344_img_0.jpg Is the person inside the red bounding box called Sigourney Weaver? Please answer yes or no. Yes
tt0090605_shot_0344_img_0.jpg Is the person inside the red bounding box called Lia Beldam? Please answer yes or no. No
tt0090756_shot_0135_img_0.jpg Is the person inside the red bounding box named Laura Dern? Please answer yes or no. Yes
tt0090756_shot_0135_img_0.jpg Is the person inside the red bounding box named Keith Frost? Please answer yes or no. No
tt0091042_shot_0098_img_0.jpg Is the person inside the red bounding box called Matthew Broderick? Please answer yes or no. Yes
tt0091042_shot_0098_img_0.jpg Is the person inside the red bounding box called Mina E. Mina? Please answer yes or no. No
tt0091738_shot_0073_img_1.jpg Is the actor inside the red bounding box called Kathleen Turner? Please answer yes or no. Yes
tt0091738_shot_0073_img_1.jpg Is the actor inside the red bounding box called Pat Kiernan? Please answer yes or no. No
tt0091867_shot_0422_img_2.jpg Is the person inside the red bounding box named Simon Callow? Please answer yes or no. Yes
tt0091867_shot_0422_img_2.jpg Is the person inside the red bounding box named Rusty Goffe? Please answer yes or no. No
tt0092099_shot_0455_img_1.jpg Is the person inside the red bounding box called Tom Cruise? Please answer yes or no. Yes
tt0092099_shot_0455_img_1.jpg Is the person inside the red bounding box called Carol Krolick? Please answer yes or no. No
tt0092699_shot_0208_img_0.jpg Is the actor inside the red bounding box called William Hurt? Please answer yes or no. Yes
tt0092699_shot_0208_img_0.jpg Is the actor inside the red bounding box called Hildur Ruriks? Please answer yes or no. No
tt0093565_shot_0409_img_0.jpg Is the actor inside the red bounding box named Cher? Please answer yes or no. Yes
tt0093565_shot_0409_img_0.jpg Is the actor inside the red bounding box named Mark Brady? Please answer yes or no. No
tt0093748_shot_0346_img_0.jpg Is the actor inside the red bounding box called John Candy? Please answer yes or no. Yes
tt0093748_shot_0346_img_0.jpg Is the actor inside the red bounding box called Sarah Heller? Please answer yes or no. No
tt0093773_shot_0212_img_0.jpg Is the person inside the red bounding box named Jesse Ventura? Please answer yes or no. Yes
tt0093773_shot_0212_img_0.jpg Is the person inside the red bounding box named Akio Mitamura? Please answer yes or no. No
tt0093779_shot_1047_img_0.jpg Is the person inside the red bounding box named Peter Falk? Please answer yes or no. Yes
tt0093779_shot_1047_img_0.jpg Is the person inside the red bounding box named Lisa Ann Walter? Please answer yes or no. No
tt0094226_shot_0237_img_2.jpg Is the actor inside the red bounding box called Kevin Costner? Please answer yes or no. Yes
tt0094226_shot_0237_img_2.jpg Is the actor inside the red bounding box called Colin Hill? Please answer yes or no. No
tt0094737_shot_0567_img_0.jpg Is the person inside the red bounding box called Tom Hanks? Please answer yes or no. Yes
tt0094737_shot_0567_img_0.jpg Is the person inside the red bounding box called Chris McHallem? Please answer yes or no. No
tt0095016_shot_1170_img_0.jpg Is the actor inside the red bounding box called Paul Gleason? Please answer yes or no. Yes
tt0095016_shot_1170_img_0.jpg Is the actor inside the red bounding box called Carl Palmer? Please answer yes or no. No
tt0095250_shot_0509_img_0.jpg Is the actor inside the red bounding box named Jean Reno? Please answer yes or no. Yes
tt0095250_shot_0509_img_0.jpg Is the actor inside the red bounding box named Ralph Meyering Jr.? Please answer yes or no. No
tt0095765_shot_0008_img_0.jpg Is the actor inside the red bounding box called Antonella Attili? Please answer yes or no. Yes
tt0095765_shot_0008_img_0.jpg Is the actor inside the red bounding box called Amber Estrada? Please answer yes or no. No
tt0095953_shot_0412_img_0.jpg Is the person inside the red bounding box named Tom Cruise? Please answer yes or no. Yes
tt0095953_shot_0412_img_0.jpg Is the person inside the red bounding box named Lara Mulcahy? Please answer yes or no. No
tt0096320_shot_0085_img_0.jpg Is the actor inside the red bounding box called Arnold Schwarzenegger? Please answer yes or no. Yes
tt0096320_shot_0085_img_0.jpg Is the actor inside the red bounding box called Dan Duran? Please answer yes or no. No
tt0096754_shot_0570_img_1.jpg Is the person inside the red bounding box named Todd Graff? Please answer yes or no. Yes
tt0096754_shot_0570_img_1.jpg Is the person inside the red bounding box named Guy Carleton? Please answer yes or no. No
tt0096874_shot_0647_img_0.jpg Is the actor inside the red bounding box named Michael J. Fox? Please answer yes or no. Yes
tt0096874_shot_0647_img_0.jpg Is the actor inside the red bounding box named Momoko Komatsu? Please answer yes or no. No
tt0096895_shot_0819_img_1.jpg Is the person inside the red bounding box called Michael Keaton? Please answer yes or no. Yes
tt0096895_shot_0819_img_1.jpg Is the person inside the red bounding box called Ben Foster? Please answer yes or no. No
tt0097216_shot_0381_img_0.jpg Is the actor inside the red bounding box named Danny Aiello? Please answer yes or no. Yes
tt0097216_shot_0381_img_0.jpg Is the actor inside the red bounding box named Taissa Farmiga? Please answer yes or no. No
tt0097428_shot_0106_img_0.jpg Is the actor inside the red bounding box named Bill Murray? Please answer yes or no. Yes
tt0097428_shot_0106_img_0.jpg Is the actor inside the red bounding box named Michael Fawcett? Please answer yes or no. No
tt0097576_shot_1010_img_2.jpg Is the actor inside the red bounding box named Harrison Ford? Please answer yes or no. Yes
tt0097576_shot_1010_img_2.jpg Is the actor inside the red bounding box named M. Emmet Walsh? Please answer yes or no. No
tt0098635_shot_0556_img_0.jpg Is the actor inside the red bounding box named Meg Ryan? Please answer yes or no. Yes
tt0098635_shot_0556_img_0.jpg Is the actor inside the red bounding box named Tom Branch? Please answer yes or no. No
tt0098724_shot_0474_img_0.jpg Is the person inside the red bounding box named Andie MacDowell? Please answer yes or no. Yes
tt0098724_shot_0474_img_0.jpg Is the person inside the red bounding box named Linda Taylor? Please answer yes or no. No
tt0099423_shot_1010_img_0.jpg Is the person inside the red bounding box called Bruce Willis? Please answer yes or no. Yes
tt0099423_shot_1010_img_0.jpg Is the person inside the red bounding box called Trevor Eve? Please answer yes or no. No
tt0099487_shot_0123_img_0.jpg Is the actor inside the red bounding box named Johnny Depp? Please answer yes or no. Yes
tt0099487_shot_0123_img_0.jpg Is the actor inside the red bounding box named Farrah Forke? Please answer yes or no. No
tt0099674_shot_1356_img_0.jpg Is the person inside the red bounding box named Al Pacino? Please answer yes or no. Yes
tt0099674_shot_1356_img_0.jpg Is the person inside the red bounding box named Nick Porrazzo? Please answer yes or no. No
tt0099685_shot_1132_img_0.jpg Is the actor inside the red bounding box called Ray Liotta? Please answer yes or no. Yes
tt0099685_shot_1132_img_0.jpg Is the actor inside the red bounding box called Chick Allan? Please answer yes or no. No
tt0099810_shot_0285_img_0.jpg Is the person inside the red bounding box called Alec Baldwin? Please answer yes or no. Yes
tt0099810_shot_0285_img_0.jpg Is the person inside the red bounding box called Jennifer Anglin? Please answer yes or no. No
tt0100157_shot_0365_img_0.jpg Is the actor inside the red bounding box named James Caan? Please answer yes or no. Yes
tt0100157_shot_0365_img_0.jpg Is the actor inside the red bounding box named Bryan Johnson? Please answer yes or no. No
tt0100403_shot_0517_img_0.jpg Is the person inside the red bounding box called Gary Busey? Please answer yes or no. Yes
tt0100403_shot_0517_img_0.jpg Is the person inside the red bounding box called Alfred Tiaki Hotu? Please answer yes or no. No
tt0100405_shot_0786_img_0.jpg Is the actor inside the red bounding box named Jason Alexander? Please answer yes or no. Yes
tt0100405_shot_0786_img_0.jpg Is the actor inside the red bounding box named Alexandra Bastedo? Please answer yes or no. No
tt0101410_shot_0105_img_0.jpg Is the person inside the red bounding box named John Turturro? Please answer yes or no. Yes
tt0101410_shot_0105_img_0.jpg Is the person inside the red bounding box named David Gore? Please answer yes or no. No
tt0102492_shot_0086_img_0.jpg Is the actor inside the red bounding box called Jamie Lee Curtis? Please answer yes or no. Yes
tt0102492_shot_0086_img_0.jpg Is the actor inside the red bounding box called Heidi Fischer? Please answer yes or no. No
tt0103064_shot_1206_img_0.jpg Is the actor inside the red bounding box named Arnold Schwarzenegger? Please answer yes or no. Yes
tt0103064_shot_1206_img_0.jpg Is the actor inside the red bounding box named Gigi Lee? Please answer yes or no. No
tt0103064_shot_2602_img_1.jpg Is the person inside the red bounding box named Arnold Schwarzenegger? Please answer yes or no. Yes
tt0103064_shot_2602_img_1.jpg Is the person inside the red bounding box named Candice Azzara? Please answer yes or no. No
tt0103776_shot_0719_img_0.jpg Is the person inside the red bounding box called Michael Keaton? Please answer yes or no. Yes
tt0103776_shot_0719_img_0.jpg Is the person inside the red bounding box called Nicholas Rice? Please answer yes or no. No
tt0104036_shot_0336_img_1.jpg Is the person inside the red bounding box named Stephen Rea? Please answer yes or no. Yes
tt0104036_shot_0336_img_1.jpg Is the person inside the red bounding box named Mimi Lizio? Please answer yes or no. No
tt0104257_shot_0477_img_0.jpg Is the person inside the red bounding box named Jack Nicholson? Please answer yes or no. Yes
tt0104257_shot_0477_img_0.jpg Is the person inside the red bounding box named Emma Julia Jacobs? Please answer yes or no. No
tt0104348_shot_0340_img_0.jpg Is the person inside the red bounding box called Ed Harris? Please answer yes or no. Yes
tt0104348_shot_0340_img_0.jpg Is the person inside the red bounding box called Carla Lizzette Mejia? Please answer yes or no. No
tt0105236_shot_0193_img_0.jpg Is the actor inside the red bounding box named Harvey Keitel? Please answer yes or no. Yes
tt0105236_shot_0193_img_0.jpg Is the actor inside the red bounding box named Terence Yin? Please answer yes or no. No
tt0105665_shot_0351_img_0.jpg Is the actor inside the red bounding box named Kyle MacLachlan? Please answer yes or no. Yes
tt0105665_shot_0351_img_0.jpg Is the actor inside the red bounding box named Julia Hsu? Please answer yes or no. No
tt0105695_shot_1436_img_1.jpg Is the person inside the red bounding box called Jaimz Woolvett? Please answer yes or no. Yes
tt0105695_shot_1436_img_1.jpg Is the person inside the red bounding box called Hermione Baddeley? Please answer yes or no. No
tt0106977_shot_1604_img_0.jpg Is the person inside the red bounding box named Tommy Lee Jones? Please answer yes or no. Yes
tt0106977_shot_1604_img_0.jpg Is the person inside the red bounding box named Honey Chhaya? Please answer yes or no. No
tt0107614_shot_0116_img_0.jpg Is the person inside the red bounding box called Sally Field? Please answer yes or no. Yes
tt0107614_shot_0116_img_0.jpg Is the person inside the red bounding box called Arthur Senzy? Please answer yes or no. No
tt0108399_shot_0778_img_0.jpg Is the actor inside the red bounding box called Christopher Walken? Please answer yes or no. Yes
tt0108399_shot_0778_img_0.jpg Is the actor inside the red bounding box called Fiona Sit? Please answer yes or no. No
tt0109831_shot_0298_img_0.jpg Is the person inside the red bounding box called Hugh Grant? Please answer yes or no. Yes
tt0109831_shot_0298_img_0.jpg Is the person inside the red bounding box called Renée Zellweger? Please answer yes or no. No
tt0111280_shot_0258_img_0.jpg Is the actor inside the red bounding box named Gates McFadden? Please answer yes or no. Yes
tt0111280_shot_0258_img_0.jpg Is the actor inside the red bounding box named Michael Angarano? Please answer yes or no. No
tt0111280_shot_1479_img_2.jpg Is the actor inside the red bounding box called William Shatner? Please answer yes or no. Yes
tt0111280_shot_1479_img_2.jpg Is the actor inside the red bounding box called Richard Rohrbough? Please answer yes or no. No
tt0112384_shot_0878_img_0.jpg Is the person inside the red bounding box called Kathleen Quinlan? Please answer yes or no. Yes
tt0112384_shot_0878_img_0.jpg Is the person inside the red bounding box called Veronica Diaz Carranza? Please answer yes or no. No
tt0112641_shot_0412_img_1.jpg Is the actor inside the red bounding box called Robert De Niro? Please answer yes or no. Yes
tt0112641_shot_0412_img_1.jpg Is the actor inside the red bounding box called Pierre Malherbe? Please answer yes or no. No
tt0112740_shot_1056_img_0.jpg Is the person inside the red bounding box named Denzel Washington? Please answer yes or no. Yes
tt0112740_shot_1056_img_0.jpg Is the person inside the red bounding box named Bill Pullman? Please answer yes or no. No
tt0113101_shot_0547_img_0.jpg Is the person inside the red bounding box named Tim Roth? Please answer yes or no. Yes
tt0113101_shot_0547_img_0.jpg Is the person inside the red bounding box named Honey Chhaya? Please answer yes or no. No
tt0114369_shot_1138_img_0.jpg Is the person inside the red bounding box named Brad Pitt? Please answer yes or no. Yes
tt0114369_shot_1138_img_0.jpg Is the person inside the red bounding box named Benjamin Nitze? Please answer yes or no. No
tt0114388_shot_0162_img_0.jpg Is the actor inside the red bounding box called Emma Thompson? Please answer yes or no. Yes
tt0114388_shot_0162_img_0.jpg Is the actor inside the red bounding box called Francis P. Hughes? Please answer yes or no. No
tt0114388_shot_1207_img_1.jpg Is the person inside the red bounding box called Hugh Grant? Please answer yes or no. Yes
tt0114388_shot_1207_img_1.jpg Is the person inside the red bounding box called Zach Hopkins? Please answer yes or no. No
tt0115798_shot_0844_img_1.jpg Is the person inside the red bounding box named Jim Carrey? Please answer yes or no. Yes
tt0115798_shot_0844_img_1.jpg Is the person inside the red bounding box named Renee Herlocker? Please answer yes or no. No
tt0116367_shot_0755_img_0.jpg Is the actor inside the red bounding box named George Clooney? Please answer yes or no. Yes
tt0116367_shot_0755_img_0.jpg Is the actor inside the red bounding box named Ben Crowley? Please answer yes or no. No
tt0116629_shot_1570_img_2.jpg Is the person inside the red bounding box called Will Smith? Please answer yes or no. Yes
tt0116629_shot_1570_img_2.jpg Is the person inside the red bounding box called E. Katherine Kerr? Please answer yes or no. No
tt0116695_shot_0343_img_0.jpg Is the person inside the red bounding box named Tom Cruise? Please answer yes or no. Yes
tt0116695_shot_0343_img_0.jpg Is the person inside the red bounding box named Billy Dee? Please answer yes or no. No
tt0117060_shot_0412_img_0.jpg Is the actor inside the red bounding box called Tom Cruise? Please answer yes or no. Yes
tt0117060_shot_0412_img_0.jpg Is the actor inside the red bounding box called Carrie Lazar? Please answer yes or no. No
tt0117060_shot_1401_img_0.jpg Is the actor inside the red bounding box called Jean Reno? Please answer yes or no. Yes
tt0117060_shot_1401_img_0.jpg Is the actor inside the red bounding box called Jill Teed? Please answer yes or no. No
tt0117381_shot_0798_img_1.jpg Is the person inside the red bounding box called Edward Norton? Please answer yes or no. Yes
tt0117381_shot_0798_img_1.jpg Is the person inside the red bounding box called Michael Tezla? Please answer yes or no. No
tt0117500_shot_2467_img_0.jpg Is the actor inside the red bounding box called Ed Harris? Please answer yes or no. Yes
tt0117500_shot_2467_img_0.jpg Is the actor inside the red bounding box called Paul J.Q. Lee? Please answer yes or no. No
tt0117509_shot_0041_img_0.jpg Is the actor inside the red bounding box named Paul Rudd? Please answer yes or no. Yes
tt0117509_shot_0041_img_0.jpg Is the actor inside the red bounding box named Max Martini? Please answer yes or no. No
tt0117571_shot_0475_img_0.jpg Is the person inside the red bounding box named Neve Campbell? Please answer yes or no. Yes
tt0117571_shot_0475_img_0.jpg Is the person inside the red bounding box named Frank Hoyt Taylor? Please answer yes or no. No
tt0117731_shot_0300_img_0.jpg Is the actor inside the red bounding box called Patrick Stewart? Please answer yes or no. Yes
tt0117731_shot_0300_img_0.jpg Is the actor inside the red bounding box called Debra Montague? Please answer yes or no. No
tt0117731_shot_1067_img_0.jpg Is the actor inside the red bounding box called Patrick Stewart? Please answer yes or no. Yes
tt0117731_shot_1067_img_0.jpg Is the actor inside the red bounding box called Jenny Wilson? Please answer yes or no. No
tt0118548_shot_1296_img_0.jpg Is the actor inside the red bounding box called Clint Eastwood? Please answer yes or no. Yes
tt0118548_shot_1296_img_0.jpg Is the actor inside the red bounding box called Kate Winslet? Please answer yes or no. No
tt0118571_shot_0627_img_0.jpg Is the actor inside the red bounding box called Glenn Close? Please answer yes or no. Yes
tt0118571_shot_0627_img_0.jpg Is the actor inside the red bounding box called Arlene Farber? Please answer yes or no. No
tt0118636_shot_0007_img_1.jpg Is the person inside the red bounding box called Brad Renfro? Please answer yes or no. Yes
tt0118636_shot_0007_img_1.jpg Is the person inside the red bounding box called Sandra Park? Please answer yes or no. No
tt0118636_shot_0344_img_0.jpg Is the actor inside the red bounding box called Brad Renfro? Please answer yes or no. Yes
tt0118636_shot_0344_img_0.jpg Is the actor inside the red bounding box called Karen Strassman? Please answer yes or no. No
tt0118655_shot_0279_img_0.jpg Is the person inside the red bounding box called Robert Wagner? Please answer yes or no. Yes
tt0118655_shot_0279_img_0.jpg Is the person inside the red bounding box called Arthur Birnbaum? Please answer yes or no. No
tt0118655_shot_1152_img_2.jpg Is the actor inside the red bounding box called Seth Green? Please answer yes or no. Yes
tt0118655_shot_1152_img_2.jpg Is the actor inside the red bounding box called Sue Doucette? Please answer yes or no. No
tt0118689_shot_0706_img_0.jpg Is the actor inside the red bounding box called Rowan Atkinson? Please answer yes or no. Yes
tt0118689_shot_0706_img_0.jpg Is the actor inside the red bounding box called Hugo Perez? Please answer yes or no. No
tt0118689_shot_0969_img_2.jpg Is the actor inside the red bounding box called Rowan Atkinson? Please answer yes or no. Yes
tt0118689_shot_0969_img_2.jpg Is the actor inside the red bounding box called Jack Shields? Please answer yes or no. No
tt0118715_shot_0079_img_0.jpg Is the actor inside the red bounding box called Jeff Bridges? Please answer yes or no. Yes
tt0118715_shot_0079_img_0.jpg Is the actor inside the red bounding box called Scott Adkins? Please answer yes or no. No
tt0118749_shot_0795_img_0.jpg Is the person inside the red bounding box called John C. Reilly? Please answer yes or no. Yes
tt0118749_shot_0795_img_0.jpg Is the person inside the red bounding box called Chris Lowell? Please answer yes or no. No
tt0118883_shot_0691_img_1.jpg Is the actor inside the red bounding box called Julia Roberts? Please answer yes or no. Yes
tt0118883_shot_0691_img_1.jpg Is the actor inside the red bounding box called Roger Bart? Please answer yes or no. No
tt0118971_shot_0679_img_0.jpg Is the actor inside the red bounding box called Charlize Theron? Please answer yes or no. Yes
tt0118971_shot_0679_img_0.jpg Is the actor inside the red bounding box called Young-min Kim? Please answer yes or no. No
tt0119008_shot_0979_img_0.jpg Is the actor inside the red bounding box named Al Pacino? Please answer yes or no. Yes
tt0119008_shot_0979_img_0.jpg Is the actor inside the red bounding box named Neil Tweddle? Please answer yes or no. No
tt0119094_shot_0446_img_2.jpg Is the actor inside the red bounding box called Nicolas Cage? Please answer yes or no. Yes
tt0119094_shot_0446_img_2.jpg Is the actor inside the red bounding box called Juan Gabriel Pareja? Please answer yes or no. No
tt0119116_shot_0721_img_0.jpg Is the actor inside the red bounding box called Bruce Willis? Please answer yes or no. Yes
tt0119116_shot_0721_img_0.jpg Is the actor inside the red bounding box called Troye Sivan? Please answer yes or no. No
tt0119174_shot_0439_img_0.jpg Is the actor inside the red bounding box named Michael Douglas? Please answer yes or no. Yes
tt0119174_shot_0439_img_0.jpg Is the actor inside the red bounding box named Carola McGuinness? Please answer yes or no. No
tt0119314_shot_0572_img_0.jpg Is the actor inside the red bounding box called Scarlett Johansson? Please answer yes or no. Yes
tt0119314_shot_0572_img_0.jpg Is the actor inside the red bounding box called Daisy Beaumont? Please answer yes or no. No
tt0119528_shot_0171_img_0.jpg Is the person inside the red bounding box called Jim Carrey? Please answer yes or no. Yes
tt0119528_shot_0171_img_0.jpg Is the person inside the red bounding box called Eliot Paton? Please answer yes or no. No
tt0119528_shot_0761_img_1.jpg Is the actor inside the red bounding box named Jim Carrey? Please answer yes or no. Yes
tt0119528_shot_0761_img_1.jpg Is the actor inside the red bounding box named Jari Kinnunen? Please answer yes or no. No
tt0119643_shot_0330_img_0.jpg Is the actor inside the red bounding box named Brad Pitt? Please answer yes or no. Yes
tt0119643_shot_0330_img_0.jpg Is the actor inside the red bounding box named Anthony Hopkins? Please answer yes or no. No
tt0119738_shot_0201_img_0.jpg Is the person inside the red bounding box named Christopher Masterson? Please answer yes or no. Yes
tt0119738_shot_0201_img_0.jpg Is the person inside the red bounding box named Edwin Craig? Please answer yes or no. No
tt0119822_shot_0878_img_0.jpg Is the person inside the red bounding box named Greg Kinnear? Please answer yes or no. Yes
tt0119822_shot_0878_img_0.jpg Is the person inside the red bounding box named Aleksandr Dubina? Please answer yes or no. No
tt0120338_shot_0444_img_2.jpg Is the actor inside the red bounding box named Kate Winslet? Please answer yes or no. Yes
tt0120338_shot_0444_img_2.jpg Is the actor inside the red bounding box named Donald Gibb? Please answer yes or no. No
tt0120338_shot_1130_img_2.jpg Is the person inside the red bounding box called Leonardo DiCaprio? Please answer yes or no. Yes
tt0120338_shot_1130_img_2.jpg Is the person inside the red bounding box called Anne Betancourt? Please answer yes or no. No
internvl_chat/eval/mme/Your_Results/code_reasoning.txt 0 → 100644
View file @ 1d5a34cf
0001.png The image shows a python code. Is the output of the code 'Hello'? Please answer yes or no. Yes
0001.png The image shows a python code. Is the output of the code 'World'? Please answer yes or no. No
0002.png The image shows a python code. Is the output of the code 'a cat'? Please answer yes or no. Yes
0002.png The image shows a python code. Is the output of the code 'a dog'? Please answer yes or no. No
0003.png The image shows a python code. Is the output of the code '12'? Please answer yes or no. Yes
0003.png The image shows a python code. Is the output of the code '5'? Please answer yes or no. No
0004.png The image shows a python code. Is the output of the code '3'? Please answer yes or no. Yes
0004.png The image shows a python code. Is the output of the code '2'? Please answer yes or no. No
0005.png The image shows a python code. Is the output of the code '12'? Please answer yes or no. Yes
0005.png The image shows a python code. Is the output of the code '5'? Please answer yes or no. No
0006.png The image shows a python code. Is the output of the code '0'? Please answer yes or no. Yes
0006.png The image shows a python code. Is the output of the code '1'? Please answer yes or no. No
0007.png Is a c++ code shown in the picture? Please answer yes or no. Yes
0007.png Is a python code shown in the picture? Please answer yes or no. No
0008.png The image shows a python code. Is the output of the code '1234'? Please answer yes or no. Yes
0008.png The image shows a python code. Is the output of the code '12345'? Please answer yes or no. No
0009.png The image shows a python code. Is the output of the code '36'? Please answer yes or no. Yes
0009.png The image shows a python code. Is the output of the code '6'? Please answer yes or no. No
0010.png The image shows a python code. Is the output of the code '1'? Please answer yes or no. Yes
0010.png The image shows a python code. Is the output of the code '5'? Please answer yes or no. No
0011.png The image shows a python code. Is the output of the code '0'? Please answer yes or no. Yes
0011.png The image shows a python code. Is the output of the code '1'? Please answer yes or no. No
0012.png The image shows a python code. Is the output of the code 'working hard'? Please answer yes or no. Yes
0012.png The image shows a python code. Is the output of the code 'playing hard'? Please answer yes or no. No
0013.png The image shows a python code. Is the output of the code 'a cat'? Please answer yes or no. Yes
0013.png The image shows a python code. Is the output of the code 'a dog'? Please answer yes or no. No
0014.png The image shows a python code. Is the output of the code '7'? Please answer yes or no. Yes
0014.png The image shows a python code. Is the output of the code '1'? Please answer yes or no. No
0015.png The image shows a python code. Is the output of the code '11'? Please answer yes or no. Yes
0015.png The image shows a python code. Is the output of the code '9'? Please answer yes or no. No
0016.png The image shows a python code. Is the output of the code 'x is smaller than 10'? Please answer yes or no. Yes
0016.png The image shows a python code. Is the output of the code 'x is larger than 10'? Please answer yes or no. No
0017.png The image shows a python code. Will the number 3 appear in the output of the code? Please answer yes or no. Yes
0017.png The image shows a python code. Will the number 6 appear in the output of the code? Please answer yes or no. No
0018.png The image shows a python code. Is the output of the code '11'? Please answer yes or no. Yes
0018.png The image shows a python code. Is the output of the code '12'? Please answer yes or no. No
0019.png The image shows a python code. Is the output of the code 'the list has more than 2 numbers'? Please answer yes or no. Yes
0019.png The image shows a python code. Is the output of the code 'the list has less than 2 numbers'? Please answer yes or no. No
0020.png Is a python code shown in the picture? Please answer yes or no. Yes
0020.png Is a c++ code shown in the picture? Please answer yes or no. No
internvl_chat/eval/mme/Your_Results/color.txt 0 → 100644
View file @ 1d5a34cf
000000006723.jpg Is there a red brick building in the image? Please answer yes or no. Yes
000000006723.jpg Is there a yellow brick building in the image? Please answer yes or no. No
000000008277.jpg Is there a white plate in the image? Please answer yes or no. Yes
000000008277.jpg Is there a yellow plate in the image? Please answer yes or no. No
000000012120.jpg Is there a blue court in the image? Please answer yes or no. Yes
000000012120.jpg Is there a purple court in the image? Please answer yes or no. No
000000014831.jpg Is there a brown and white animal in the image? Please answer yes or no. Yes
000000014831.jpg Is there a green and red animal in the image? Please answer yes or no. No
000000028993.jpg Are there yellow poles in the image? Please answer yes or no. Yes
000000028993.jpg Are there blue poles in the image? Please answer yes or no. No
000000029393.jpg Is there a brown dog in the image? Please answer yes or no. Yes
000000029393.jpg Is there a black dog in the image? Please answer yes or no. No
000000035770.jpg Is there a black and white toilet in the image? Please answer yes or no. Yes
000000035770.jpg Is there a red and white toilet in the image? Please answer yes or no. No
000000038118.jpg Is there a red coat in the image? Please answer yes or no. Yes
000000038118.jpg Is there a yellow coat in the image? Please answer yes or no. No
000000047112.jpg Is there a white plate in the image? Please answer yes or no. Yes
000000047112.jpg Is there a yellow plate in the image? Please answer yes or no. No
000000047121.jpg Is there a black cat in the image? Please answer yes or no. Yes
000000047121.jpg Is there a brown cat in the image? Please answer yes or no. No
000000053529.jpg Is there a green hat in the image? Please answer yes or no. Yes
000000053529.jpg Is there a red hat in the image? Please answer yes or no. No
000000053994.jpg Is there a gray wall in the image? Please answer yes or no. Yes
000000053994.jpg Is there a red wall in the image? Please answer yes or no. No
000000055072.jpg Is there a brown giraffe in the image? Please answer yes or no. Yes
000000055072.jpg Is there a black giraffe in the image? Please answer yes or no. No
000000057597.jpg Are there any red shoes in the image? Please answer yes or no. Yes
000000057597.jpg Are there any yellow shoes in the image? Please answer yes or no. No
000000061658.jpg Are there a white dish in the image? Please answer yes or no. Yes
000000061658.jpg Are there a green dish in the image? Please answer yes or no. No
000000338560.jpg Is there a blue and yellow fire hydrant in the image? Please answer yes or no. Yes
000000338560.jpg Is there a blue and orange fire hydrant in the image? Please answer yes or no. No
000000370208.jpg Is there a red bicycle with white handlebars in the image? Please answer yes or no. Yes
000000370208.jpg Is there a red bicycle with black handlebars in the image? Please answer yes or no. No
000000377723.jpg Is there a blue bus in the image? Please answer yes or no. Yes
000000377723.jpg Is there a orange bus in the image? Please answer yes or no. No
000000405205.jpg Is there a white bus in the image? Please answer yes or no. Yes
000000405205.jpg Is there a red bus in the image? Please answer yes or no. No
000000410612.jpg Is there a red boat in the image? Please answer yes or no. Yes
000000410612.jpg Is there a gray boat in the image? Please answer yes or no. No
000000427034.jpg Is there a brown and black dog in the image? Please answer yes or no. Yes
000000427034.jpg Is there a brown and white dog in the image? Please answer yes or no. No
000000442456.jpg Is there a man wearing a red shirt in the image? Please answer yes or no. Yes
000000442456.jpg Is there a man wearing a white shirt in the image? Please answer yes or no. No
000000492362.jpg Is there a skateboard with red wheels in the image? Please answer yes or no. Yes
000000492362.jpg Is there a skateboard with black wheels in the image? Please answer yes or no. No
000000492992.jpg Is there a white bird in the image? Please answer yes or no. Yes
000000492992.jpg Is there a yellow bird in the image? Please answer yes or no. No
000000512929.jpg Are there any green beans in the image? Please answer yes or no. Yes
000000512929.jpg Are there any orange beans in the image? Please answer yes or no. No
000000530457.jpg Are there any red flowers in the image? Please answer yes or no. Yes
000000530457.jpg Are there any green flowers in the image? Please answer yes or no. No
000000532761.jpg Is there a living room painted yellow in the image? Please answer yes or no. Yes
000000532761.jpg Is there a living room painted black in the image? Please answer yes or no. No
000000534041.jpg Is there a purple bottle in the image? Please answer yes or no. Yes
000000534041.jpg Is there a white bottle in the image? Please answer yes or no. No
000000563758.jpg Is there a red scarf in the image? Please answer yes or no. Yes
000000563758.jpg Is there a brown scarf in the image? Please answer yes or no. No
000000564280.jpg Is there a red couch in the image? Please answer yes or no. Yes
000000564280.jpg Is there a black couch in the image? Please answer yes or no. No
internvl_chat/eval/mme/Your_Results/commonsense_reasoning.txt 0 → 100644
View file @ 1d5a34cf
0001.png Here are the order details for my taxi ride. Should I actually pay $29.42? Please answer yes or no. Yes
0001.png Here are the order details for my taxi ride. Should I actually pay $32.42? Please answer yes or no. No
0002.png Should I stop when I'm about to cross the street and see the sign in the picture? Please answer yes or no. Yes
0002.png When I see the sign in the picture, can I cross the street? Please answer yes or no. No
0003.png May I ask if in the game of finger-guessing game, did the right side of the picture win? Please answer yes or no. Yes
0003.png May I ask if in the game of finger-guessing game, did the left side of the picture win? Please answer yes or no. No
0004.png Does the fruit in the picture look stale? Please answer yes or no. Yes
0004.png Does the fruit in the picture look very fresh? Please answer yes or no. No
0005.png The office's normal closing time is 5 p.m. Now is afternoon. Should I continue to work at the time shown in the picture? Please answer yes or no. Yes
0005.png The office's normal closing time is 5 p.m. Now is afternoon. Could I leave work at the time shown in the picture? Please answer yes or no. No
0006.png I recently want to go on vacation to relax and go to a place full of fresh air. Is the venue in the picture appropriate? Please answer yes or no. Yes
0006.png I want to go where there are a lot of people. Is the venue in the picture appropriate? Please answer yes or no. No
0007.png I want to clean the house and I want to choose a tool. Is the tool in the picture an appropriate choice? Please answer yes or no. Yes
0007.png I want to transport something and I want to choose a tool to help me. Is the tool in the picture an appropriate choice? Please answer yes or no. No
0008.png Can I smoke where the picture is? Please answer yes or no. Yes
0008.png Is smoking prohibited in the location of the picture? Please answer yes or no. No
0009.png Will green be obtained by mixing the above two colors? Please answer yes or no. Yes
0009.png Will red be obtained by mixing the above two colors? Please answer yes or no. No
0010.png I am going to exercise and I want to choose a venue. Is the venue in the picture a good choice? Please answer yes or no. Yes
0010.png I am going to study and I want to choose a venue. Is the venue in the picture a good choice? Please answer yes or no. No
0011.png If I am allergic to durian, can I finish the fruit in the picture? Please answer yes or no. Yes
0011.png If I am allergic to banana, can I finish the fruit in the picture? Please answer yes or no. No
0012.png I am going to study and I want to choose a venue. Is the venue in the picture a good choice? Please answer yes or no. Yes
0012.png I am going to exercise and I want to choose a venue. Is the venue in the picture a good choice? Please answer yes or no. No
0013.png I am going to a formal dinner party. Is the shoe in the picture an appropriate choice? Please answer yes or no. Yes
0013.png I am going to play basketball. Is the shoe in the picture an appropriate choice? Please answer yes or no. No
0014.png In this line chart, the vertical axis is height and the horizontal axis is age. Does Maria's height exceed Jane's height in the end? Please answer yes or no. Yes
0014.png In this line chart, the vertical axis is height and the horizontal axis is age. Does Jane's height exceed Kangkang's height in the end? Please answer yes or no. No
0015.png Is the ball usually played with hands? Please answer yes or no. Yes
0015.png Is the ball usually played with feet? Please answer yes or no. No
0016.png Is the place in the picture a good place to enjoy the cool in a sunny day? Please answer yes or no. Yes
0016.png Is the place in the picture a good shelter from the rain when it thunders outside? Please answer yes or no. No
0017.png Are the vehicles in the pictures usually environmentally friendly? Please answer yes or no. Yes
0017.png Does the vehicle in the picture usually run faster than the car? Please answer yes or no. No
0018.png This is a picture of some kind of animal. Does it eat leaves? Please answer yes or no. Yes
0018.png This is a picture of some kind of animal. Does it eat meat? Please answer yes or no. No
0019.png Is the water flow in the picture from the top to the bottom? Please answer yes or no. Yes
0019.png Is the water flow in the picture from the bottom to the top? Please answer yes or no. No
0020.png Can the item in the picture be used to measure length? Please answer yes or no. Yes
0020.png Can the item in the picture be used to measure angles? Please answer yes or no. No
0021.png This is a toilet guide sign. I am a man. Should I go to the toilet on the left? Please answer yes or no. Yes
0021.png This is a toilet guide sign. I am a man. Should I go to the toilet on the right? Please answer yes or no. No
0022.png Does the animal in the picture usually catch mice? Please answer yes or no. Yes
0022.png Is the animal in the picture usually used in search and rescue? Please answer yes or no. No
0023.png If you want to keep your fruit fresh in summer, should you put it in the appliance in the picture? Please answer yes or no. Yes
0023.png Is the appliance in the picture more suitable for winter than summer? Please answer yes or no. No
0024.png I want to go skating. Is the shoe in the picture usually appropriate? Please answer yes or no. Yes
0024.png I want to go roller skating. Is the shoe in the picture usually appropriate? Please answer yes or no. No
0025.png I feel very thirsty in the desert now. Can the thing in the picture help me? Please answer yes or no. Yes
0025.png I don't like clear cups. Is the cup in the picture my type? Please answer yes or no. No
0026.png I want to go for a run and I want to choose a pair of shoes. Is the shoe in the picture an appropriate choice? Please answer yes or no. Yes
0026.png I want to practice ballet and I want to choose a pair of shoes. Is the shoe in the picture an appropriate choice? Please answer yes or no. No
0027.png Are the pants in the picture usually suitable for casual wear? Please answer yes or no. Yes
0027.png Are the pants in the picture usually suitable for playing basketball? Please answer yes or no. No
0028.png This is a picture from a real scene. Is there only one real cat in this picture? Please answer yes or no. Yes
0028.png This is a picture from a real scene. Is there only two real cats in this picture? Please answer yes or no. No
0029.png The three cats in the picture, the one without a beard, is the middle one? Please answer yes or no. Yes
0029.png The three cats in the picture, the one without a beard, is the right one? Please answer yes or no. No
0030.png I'm going to 501. Do I need to turn left at the intersection? Please answer yes or no. Yes
0030.png I'm going to 502. Do I need to turn left at the intersection? Please answer yes or no. No
0031.png Is the drink in the picture usually suitable for a party? Please answer yes or no. Yes
0031.png Is the drink in the picture usually suitable for drinking together with cephalosporin? Please answer yes or no. No
0032.png Here is a picture of the cake I cut. Did I cut it at least twice? Please answer yes or no. Yes
0032.png Here is a picture of the cake I cut. Did I cut it at least once? Please answer yes or no. No
0033.png This is a picture from a real scene. Is there only one real apple in this picture? Please answer yes or no. Yes
0033.png This is a picture from a real scene. Is there only two real apples in this picture? Please answer yes or no. No
0034.png Here is a pie chart counting the favorite fruits of all employees in our company. Is the durian the most popular fruit? Please answer yes or no. Yes
0034.png Here is a pie chart counting the favorite fruits of all employees in our company. Is the mango the most popular fruit? Please answer yes or no. No
0035.png This is the sales chart of this month. Is Tina the runner-up in sales this month? Please answer yes or no. Yes
0035.png This is the sales chart of this month. Is John the runner-up in sales this month? Please answer yes or no. No
0036.png Is it a good time to walk through the road in the picture? Please answer yes or no. Yes
0036.png Is it a good time to drive a car through the road in the picture? Please answer yes or no. No
0037.png Here is a photo of the sun's position at a certain time. Could it be dusk now? Please answer yes or no. Yes
0037.png Here is a photo of the sun's position at a certain time. Could it be noon now? Please answer yes or no. No
0038.png All apples are shown in the picture. If I eat an apple every day, can I eat it for four days? Please answer yes or no. Yes
0038.png All apples are shown in the picture. If I eat an apple every day, can I eat it for three days? Please answer yes or no. No
0039.png This line chart is used to count the sales of two types of burgers. Are chicken burgers more popular? Please answer yes or no. Yes
0039.png This line chart is used to count the sales of two types of burgers. Are beef burgers more popular? Please answer yes or no. No
0040.png I want to supplement protein. Is it appropriate to eat the food in the picture? Please answer yes or no. Yes
0040.png I don't like to eat any food related to chicken. Is the food in the picture my type? Please answer yes or no. No
0041.png Is the fruit in the picture usually sweet? Please answer yes or no. Yes
0041.png Is the fruit in the picture usually spicy? Please answer yes or no. No
0042.png Are there usually cars in the area shown in the picture? Please answer yes or no. Yes
0042.png Is it appropriate to cross the road directly from the place shown in the picture? Please answer yes or no. No
0043.png Is the animal in the picture usually not seen in winter? Please answer yes or no. Yes
0043.png Is the animal in the picture usually seen in winter? Please answer yes or no. No
0044.png This is a flowchart of a program. I enter 3 and 6. Is the output 'No'? Please answer yes or no. Yes
0044.png This is a flowchart of a program. I enter 3 and 6. Is the output 'Yes'? Please answer yes or no. No
0045.png There is a sign at the intersection, can I turn left? Please answer yes or no. Yes
0045.png There is a sign at the intersection, can I turn right? Please answer yes or no. No
0046.png Vitamin C is very helpful for human health. Does the food on in the picture usually contain Vitamin C? Please answer yes or no. Yes
0046.png Is the food in the picture commonly used to build muscle? Please answer yes or no. No
0047.png All apples are shown in the picture. My brother and I divide the apples equally. May I have one apple? Please answer yes or no. Yes
0047.png All apples are shown in the picture. My brother and I divide the apples equally. May I have two apples? Please answer yes or no. No
0048.png Here is a picture of eating fruit. Am I eating a strawberry? Please answer yes or no. Yes
0048.png Here is a picture of eating fruit. Am I eating a cherry tomato? Please answer yes or no. No
0049.png Does the vehicle in the picture usually have its Windows closed during fast driving? Please answer yes or no. Yes
0049.png Does the vehicle in the picture usually have its Windows opened during fast driving? Please answer yes or no. No
0050.png Do people commonly use the item in the picture for makeup in their daily lives? Please answer yes or no. Yes
0050.png Do people commonly use the item in the picture to write in their daily lives? Please answer yes or no. No
0051.png This is a flowchart of a program. When the input is 5, is the output 6? Please answer yes or no. Yes
0051.png This is a flowchart of a program. When the input is 6, is the output 5? Please answer yes or no. No
0052.png I want to lose weight. Is the food in the picture an appropriate choice? Please answer yes or no. Yes
0052.png I want to gain weight. Is the food in the picture an appropriate choice? Please answer yes or no. No
0053.png Is the car in the picture going to make a right turn after going through a straight road section? Please answer yes or no. Yes
0053.png Is the car in the picture going to make a left turn after going through a straight road section? Please answer yes or no. No
0054.png May I ask if the plants in the picture can survive in the water? Please answer yes or no. Yes
0054.png May I ask if the plants in the picture can survive in the soil? Please answer yes or no. No
0055.png The man in the picture is eating. Does he eat noodles? Please answer yes or no. Yes
0055.png The man in the picture is eating. Does he eat rice? Please answer yes or no. No
0056.png Can the item in the picture output water? Please answer yes or no. Yes
0056.png Can the item in picture be used for blowing air? Please answer yes or no. No
0057.png Does the vehicle in the picture usually run faster than a horse? Please answer yes or no. Yes
0057.png Does the vehicle in the picture usually fly? Please answer yes or no. No
0058.png Can't I smoke here? Please answer yes or no. Yes
0058.png May I smoke here? Please answer yes or no. No
0059.png This pie chart is the age distribution of our company. Is the proportion of people aged 30-50 more than 40%? Please answer yes or no. Yes
0059.png This pie chart is the age distribution of our company. Is the proportion of people aged 40-50 more than 30%? Please answer yes or no. No
0060.png This is the histogram of fruit sales today. Do more men buy watermelons than women buy bananas? Please answer yes or no. Yes
0060.png This is the histogram of fruit sales today. Do more men buy peach than women buy apple? Please answer yes or no. No
0061.png Is the tool in the picture common in tall buildings? Please answer yes or no. Yes
0061.png In case of fire, is it appropriate to choose the tool in the picture to go downstairs? Please answer yes or no. No
0062.png It's snowing outside the window now. I want to go out. Is it appropriate to wear the cloth in the picture? Please answer yes or no. Yes
0062.png It's very hot outside. I want to go out. Is it appropriate to wear the cloth in the picture? Please answer yes or no. No
0063.png Is the animal in the picture suitable as a pet? Please answer yes or no. Yes
0063.png Is the animal in the pictures usually stronger than adult tigers? Please answer yes or no. No
0064.png I want to play basketball. Is the venue in the picture a good choice? Please answer yes or no. Yes
0064.png I want to play football. Is the venue in the picture a good choice? Please answer yes or no. No
0065.png Is it appropriate to wear a down jacket during the season in the picture? Please answer yes or no. Yes
0065.png Is it appropriate to only wear short sleeves during the season in the picture? Please answer yes or no. No
0066.png I want to carry one thing with me on a rainy day. Is the thing in the image an appropriate choice? Please answer yes or no. Yes
0066.png It is raining outside. I am in a house and I don't need to go out. Is this thing in the picture necessary for me to use? Please answer yes or no. No
0067.png I feel very hot. Is the tool in the picture suitable for use? Please answer yes or no. Yes
0067.png I feel very cold. Is the tool in the picture suitable for use? Please answer yes or no. No
0068.png Is it unhealthy to eat the food in the picture too often? Please answer yes or no. Yes
0068.png Is the food in the picture usually low in calories? Please answer yes or no. No
0069.png Is the phone in the photo connected to a charger? Please answer yes or no. Yes
0069.png Is the phone in the photo charging? Please answer yes or no. No
0070.png I want to turn the screw. Is the tool in the picture usually appropriate? Please answer yes or no. Yes
0070.png Is the tool in the picture usually suitable for smashing walnuts? Please answer yes or no. No
internvl_chat/eval/mme/Your_Results/count.txt 0 → 100644
View file @ 1d5a34cf
000000006040.jpg Is there a train in the picture? Please answer yes or no. Yes
000000006040.jpg Are there a total of two trains in the picture? Please answer yes or no. No
000000044279.jpg Is there a total of two people in the image? Please answer yes or no. Yes
000000044279.jpg Is there only one people in the image? Please answer yes or no. No
000000067213.jpg Is there only one dog in the image? Please answer yes or no. Yes
000000067213.jpg Is there two dogs in the image? Please answer yes or no. No
000000071226.jpg Is there a total of two dogs in the image? Please answer yes or no. Yes
000000071226.jpg Is there only one dogs in the image? Please answer yes or no. No
000000097994.jpg Are there three laptops in the picture? Please answer yes or no. Yes
000000097994.jpg Are there four laptops in the picture? Please answer yes or no. No
000000195918.jpg Is there a total of two display devices in the image? Please answer yes or no. Yes
000000195918.jpg Is there only one display device in the image? Please answer yes or no. No
000000236721.jpg Are there two bananas in the image? Please answer yes or no. Yes
000000236721.jpg Are there three bananas in the image? Please answer yes or no. No
000000261712.jpg Are there two giraffes in this image? Please answer yes or no. Yes
000000261712.jpg Are there three giraffes in this picture? Please answer yes or no. No
000000274066.jpg Are there four people appear in this image? Please answer yes or no. Yes
000000274066.jpg Are there only three people appear in this image? Please answer yes or no. No
000000276434.jpg Is there a total of three cakes in this image? Please answer yes or no. Yes
000000276434.jpg Are there only two cakes in this image? Please answer yes or no. No
000000289059.jpg Is there a total of two person appear in the image? Please answer yes or no. Yes
000000289059.jpg Is there only one person appear in the image? Please answer yes or no. No
000000290081.jpg Is there only one bowl in this image? Please answer yes or no. Yes
000000290081.jpg Are there two bowls in this image? Please answer yes or no. No
000000301867.jpg Are there three people appear in this image? Please answer yes or no. Yes
000000301867.jpg Are there only two people appear in this image? Please answer yes or no. No
000000335954.jpg Are there two bowls in this image? Please answer yes or no. Yes
000000335954.jpg Are there three bowls in this image? Please answer yes or no. No
000000357816.jpg Are there four people in this image? Please answer yes or no. Yes
000000357816.jpg Are there five people in this image? Please answer yes or no. No
000000372819.jpg Are there four dogs appear in this image? Please answer yes or no. Yes
000000372819.jpg Are there only three dogs appear in this image? Please answer yes or no. No
000000410612.jpg Is there only one ship in the picture? Please answer yes or no. Yes
000000410612.jpg Is there a total of two ships in the picture? Please answer yes or no. No
000000423944.jpg Is there no person in this picture? Please answer yes or no. Yes
000000423944.jpg Are there two people appear in this image? Please answer yes or no. No
000000427034.jpg Is there a dog in the picture? Please answer yes or no. Yes
000000427034.jpg Are there a total of two dogs in the picture? Please answer yes or no. No
000000430286.jpg Are there three remotes in this image? Please answer yes or no. Yes
000000430286.jpg Are there only two remotes in this image? Please answer yes or no. No
000000432468.jpg Are there three zippers in the picture? Please answer yes or no. Yes
000000432468.jpg Is there a zipper in the picture? Please answer yes or no. No
000000434479.jpg Are there two pieces of pizza in this image? Please answer yes or no. Yes
000000434479.jpg Is there only one piece of pizza in this image? Please answer yes or no. No
000000438304.jpg Are there two tennis rackets in the picture? Please answer yes or no. Yes
000000438304.jpg Are there only one tennis racket in the picture? Please answer yes or no. No
000000450303.jpg Are there six people appear in this image? Please answer yes or no. Yes
000000450303.jpg Are there seven people appear in this image? Please answer yes or no. No
000000470121.jpg Is there only one bottle in the image? Please answer yes or no. Yes
000000470121.jpg Is there two bottles in the image? Please answer yes or no. No
000000476215.jpg Are there two horses in this image? Please answer yes or no. Yes
000000476215.jpg Is there only one horse in this image? Please answer yes or no. No
000000482100.jpg Are there two toilets in the picture? Please answer yes or no. Yes
000000482100.jpg Is there only one toilet in the picture? Please answer yes or no. No
000000491867.jpg Is there only one necktie in the image? Please answer yes or no. Yes
000000491867.jpg Is there three neckties in the image? Please answer yes or no. No
000000556000.jpg Are there four people in the image? Please answer yes or no. Yes
000000556000.jpg Are there only three people in the image? Please answer yes or no. No
000000565045.jpg Are there two bath towels in the picture? Please answer yes or no. Yes
000000565045.jpg Is there only one bath towel in the picture? Please answer yes or no. No
internvl_chat/eval/mme/Your_Results/existence.txt 0 → 100644
View file @ 1d5a34cf
000000006040.jpg Is there a train in this image? Please answer yes or no. Yes
000000006040.jpg Is there a bed in this image? Please answer yes or no. No
000000006471.jpg Is there a baseball bat in this image? Please answer yes or no. Yes
000000006471.jpg Is there a giraffe in this image? Please answer yes or no. No
000000007108.jpg Is there a elephant in this image? Please answer yes or no. Yes
000000007108.jpg Is there a hair drier in this image? Please answer yes or no. No
000000007816.jpg Is there a motorcycle in this image? Please answer yes or no. Yes
000000007816.jpg Is there a airplane in this image? Please answer yes or no. No
000000007977.jpg Is there a skateboard in this image? Please answer yes or no. Yes
000000007977.jpg Is there a spoon in this image? Please answer yes or no. No
000000008844.jpg Is there a person in this image? Please answer yes or no. Yes
000000008844.jpg Is there a sink in this image? Please answer yes or no. No
000000009590.jpg Is there a bottle in this image? Please answer yes or no. Yes
000000009590.jpg Is there a scissors in this image? Please answer yes or no. No
000000010363.jpg Is there a bottle in this image? Please answer yes or no. Yes
000000010363.jpg Is there a apple in this image? Please answer yes or no. No
000000011197.jpg Is there a car in this image? Please answer yes or no. Yes
000000011197.jpg Is there a fork in this image? Please answer yes or no. No
000000015254.jpg Is there a spoon in this image? Please answer yes or no. Yes
000000015254.jpg Is there a donut in this image? Please answer yes or no. No
000000015517.jpg Is there a bus in this image? Please answer yes or no. Yes
000000015517.jpg Is there a cow in this image? Please answer yes or no. No
000000015746.jpg Is there a fire hydrant in this image? Please answer yes or no. Yes
000000015746.jpg Is there a person in this image? Please answer yes or no. No
000000037751.jpg Is there a backpack in this image? Please answer yes or no. Yes
000000037751.jpg Is there a microwave in this image? Please answer yes or no. No
000000050145.jpg Is there a bicycle in this image? Please answer yes or no. Yes
000000050145.jpg Is there a apple in this image? Please answer yes or no. No
000000061418.jpg Is there a chair in this image? Please answer yes or no. Yes
000000061418.jpg Is there a airplane in this image? Please answer yes or no. No
000000417779.jpg Is there a car in this image? Please answer yes or no. Yes
000000417779.jpg Is there a kite in this image? Please answer yes or no. No
000000424521.jpg Is there a skateboard in this image? Please answer yes or no. Yes
000000424521.jpg Is there a banana in this image? Please answer yes or no. No
000000438304.jpg Is there a sports ball in this image? Please answer yes or no. Yes
000000438304.jpg Is there a horse in this image? Please answer yes or no. No
000000494427.jpg Is there a laptop in this image? Please answer yes or no. Yes
000000494427.jpg Is there a potted plant in this image? Please answer yes or no. No
000000495448.jpg Is there a cake in this image? Please answer yes or no. Yes
000000495448.jpg Is there a tie in this image? Please answer yes or no. No
000000498463.jpg Is there a refrigerator in this image? Please answer yes or no. Yes
000000498463.jpg Is there a donut in this image? Please answer yes or no. No
000000519039.jpg Is there a truck in this image? Please answer yes or no. Yes
000000519039.jpg Is there a book in this image? Please answer yes or no. No
000000523241.jpg Is there a car in this image? Please answer yes or no. Yes
000000523241.jpg Is there a cell phone in this image? Please answer yes or no. No
000000530162.jpg Is there a umbrella in this image? Please answer yes or no. Yes
000000530162.jpg Is there a horse in this image? Please answer yes or no. No
000000537812.jpg Is there a chair in this image? Please answer yes or no. Yes
000000537812.jpg Is there a baseball bat in this image? Please answer yes or no. No
000000541952.jpg Is there a clock in this image? Please answer yes or no. Yes
000000541952.jpg Is there a bottle in this image? Please answer yes or no. No
000000546626.jpg Is there a bottle in this image? Please answer yes or no. Yes
000000546626.jpg Is there a mouse in this image? Please answer yes or no. No
000000556000.jpg Is there a chair in this image? Please answer yes or no. Yes
000000556000.jpg Is there a dog in this image? Please answer yes or no. No
000000557258.jpg Is there a toilet in this image? Please answer yes or no. Yes
000000557258.jpg Is there a pizza in this image? Please answer yes or no. No
000000572956.jpg Is there a motorcycle in this image? Please answer yes or no. Yes
000000572956.jpg Is there a bus in this image? Please answer yes or no. No
internvl_chat/eval/mme/Your_Results/landmark.txt 0 → 100644
View file @ 1d5a34cf
0006adf999ccc899.jpg Is this a photo of Muktinath? Please answer yes or no. Yes
0006adf999ccc899.jpg Is this a photo of Lohikoski Church? Please answer yes or no. No
0015309638e5a9b1.jpg Is this a picture of Teufelsmauer (Harz)? Please answer yes or no. Yes
0015309638e5a9b1.jpg Is this a picture of Romfo kirke? Please answer yes or no. No
001f0bfe8464254a.jpg Is this a picture of Saint Catherine of Alexandria church in Warsaw? Please answer yes or no. Yes
001f0bfe8464254a.jpg Is this a picture of Kaap Skil? Please answer yes or no. No
0022666a36b80ea5.jpg Is this an image of Hall of Supreme Harmony (Forbidden City)? Please answer yes or no. Yes
0022666a36b80ea5.jpg Is this an image of Brickegickel? Please answer yes or no. No
0023bd687cbc0a44.jpg Is this a photo of Utrecht Dom Tower? Please answer yes or no. Yes
0023bd687cbc0a44.jpg Is this a photo of Great Palace Mosaic Museum? Please answer yes or no. No
00261324328490fc.jpg Is this an image of Loch Ossian? Please answer yes or no. Yes
00261324328490fc.jpg Is this an image of Tanemakidaishi? Please answer yes or no. No
002ae473a76362a6.jpg Is this an image of Gerlache Strait? Please answer yes or no. Yes
002ae473a76362a6.jpg Is this an image of Schmiedebruch (Naturschutzgebiet)? Please answer yes or no. No
003f8af9477a8eb8.jpg Is this a photo of Le Destroit? Please answer yes or no. Yes
003f8af9477a8eb8.jpg Is this a photo of Kharkov Mosque? Please answer yes or no. No
004543cec064cd0b.jpg Is this a picture of Wollaton Park? Please answer yes or no. Yes
004543cec064cd0b.jpg Is this a picture of St John's Church, Blackpool? Please answer yes or no. No
004beea6937b3a8c.jpg Is this a photo of Brussels Park? Please answer yes or no. Yes
004beea6937b3a8c.jpg Is this a photo of Forest Hills Cemetery (Boston, Massachusetts)? Please answer yes or no. No
005150f80de6f487.jpg Is this a photo of Schloss Burgpreppach? Please answer yes or no. Yes
005150f80de6f487.jpg Is this a photo of Pinkas Synagogue? Please answer yes or no. No
0053894534d1e259.jpg Is this a picture of Goethe- und Schiller-Denkmal (Weimar)? Please answer yes or no. Yes
0053894534d1e259.jpg Is this a picture of Grey River? Please answer yes or no. No
0059fc817c5fd5b7.jpg Is this a photo of KLCC Park? Please answer yes or no. Yes
0059fc817c5fd5b7.jpg Is this a photo of Neath Castle? Please answer yes or no. No
005c68530bb6ed7c.jpg Is this a photo of Balvenie Castle? Please answer yes or no. Yes
005c68530bb6ed7c.jpg Is this a photo of Draper Utah Temple? Please answer yes or no. No
006e3342b57aa3b4.jpg Is this a photo of Larvs kyrka? Please answer yes or no. Yes
006e3342b57aa3b4.jpg Is this a photo of Geitgallien? Please answer yes or no. No
0075a4d212539625.jpg Is this a photo of Cardiff Castle? Please answer yes or no. Yes
0075a4d212539625.jpg Is this a photo of Boudhanath? Please answer yes or no. No
008053e6bb5afa33.jpg Is this a photo of Rochers des Mourres? Please answer yes or no. Yes
008053e6bb5afa33.jpg Is this a photo of Schloss Weinfelden? Please answer yes or no. No
00c1349a9622390c.jpg Is this a photo of Amar Jawan Jyoti (Tomb of the Unknown Soldier)? Please answer yes or no. Yes
00c1349a9622390c.jpg Is this a photo of Porta Garibaldi (Marsala)? Please answer yes or no. No
00cae9f165d8d2cc.jpg Is this an image of Fisherman's Bastion? Please answer yes or no. Yes
00cae9f165d8d2cc.jpg Is this an image of Equestrian statue of George IV, Trafalgar Square, London? Please answer yes or no. No
00f4fb71575efb0b.jpg Is this a picture of 1965)? Please answer yes or no. Yes
00f4fb71575efb0b.jpg Is this a picture of Plaza Monumental de Toros de Pueblo Nuevo? Please answer yes or no. No
00fa476690a260a8.jpg Is this an image of Bathampton Toll Bridge? Please answer yes or no. Yes
00fa476690a260a8.jpg Is this an image of Moose River Plains Wild Forest? Please answer yes or no. No
010260e62e7d847d.jpg Is this a photo of Visingsborg? Please answer yes or no. Yes
010260e62e7d847d.jpg Is this a photo of Teutsche Schule (Schleusingen)? Please answer yes or no. No
010b5b5c9c461f65.jpg Is this an image of Het Steen? Please answer yes or no. Yes
010b5b5c9c461f65.jpg Is this an image of Castello di Montalto Dora? Please answer yes or no. No
01114538f95f1ce8.jpg Is this a picture of Svirzh Castle? Please answer yes or no. Yes
01114538f95f1ce8.jpg Is this a picture of Bistrica, Ionian Sea? Please answer yes or no. No
01121b9b3c79afc4.jpg Is this a photo of Alamannenmuseum Ellwangen? Please answer yes or no. Yes
01121b9b3c79afc4.jpg Is this a photo of Sint-Jozefkerk, Enschede? Please answer yes or no. No
011a19afda5df3c7.jpg Is this a picture of San Prospero (Reggio Emilia)? Please answer yes or no. Yes
011a19afda5df3c7.jpg Is this a picture of Lake DeFuniak? Please answer yes or no. No
012856462a95ada9.jpg Is this a picture of Chingshui Cliff? Please answer yes or no. Yes
012856462a95ada9.jpg Is this a picture of St. John's Church, Portland? Please answer yes or no. No
012ea376a16f17bf.jpg Is this a picture of Church of Saint Giles in Prague? Please answer yes or no. Yes
012ea376a16f17bf.jpg Is this a picture of Pfarrkirche St. Martin an der Raab? Please answer yes or no. No
013e1a1e887cb915.jpg Is this a picture of Real Casa de Correos, Madrid? Please answer yes or no. Yes
013e1a1e887cb915.jpg Is this a picture of Ayton Castle, Scottish Borders? Please answer yes or no. No
0140a7d5446719c3.jpg Is this a photo of Friday Mosque, Herat? Please answer yes or no. Yes
0140a7d5446719c3.jpg Is this a photo of Hexenturm (Burg (bei Magdeburg))? Please answer yes or no. No
01416124a7ab211a.jpg Is this a photo of Ajuntament de Mollerussa? Please answer yes or no. Yes
01416124a7ab211a.jpg Is this a photo of Episcopal Diocese of Northern Indiana? Please answer yes or no. No
01425c4d35880aad.jpg Is this an image of Museu de Sant Boi de Llobregat? Please answer yes or no. Yes
01425c4d35880aad.jpg Is this an image of Bezdonnoye Lake, Moscow Oblast? Please answer yes or no. No
014c50e91cf799c7.jpg Is this a photo of Clearwater Beach, Florida? Please answer yes or no. Yes
014c50e91cf799c7.jpg Is this a photo of Church of Holy Trinity in Vratislavice nad Nisou? Please answer yes or no. No
015543ddea466c54.jpg Is this a picture of Galician market square in Skansen, Sanok? Please answer yes or no. Yes
015543ddea466c54.jpg Is this a picture of Walworth Castle? Please answer yes or no. No
0169c4235270f65b.jpg Is this a photo of St. Ricarius Church, Aberford? Please answer yes or no. Yes
0169c4235270f65b.jpg Is this a photo of Schloss Prebberede? Please answer yes or no. No
016e68628ea6c7ee.jpg Is this an image of Jewish Ceremonial Hall, Prague-Josefov? Please answer yes or no. Yes
016e68628ea6c7ee.jpg Is this an image of Our Lady Church (Ettlingen)? Please answer yes or no. No
01862535ce8961ee.jpg Is this an image of Mont Tendre? Please answer yes or no. Yes
01862535ce8961ee.jpg Is this an image of Ogden Utah Temple? Please answer yes or no. No
0186742e7963fb3c.jpg Is this a photo of Ozegahara? Please answer yes or no. Yes
0186742e7963fb3c.jpg Is this a photo of San Giovenale (Orvieto)? Please answer yes or no. No
01928acd6dfe6cdd.jpg Is this a picture of Watergate Bay? Please answer yes or no. Yes
01928acd6dfe6cdd.jpg Is this a picture of Cairnpapple Hill? Please answer yes or no. No
01a6842099d62f22.jpg Is this an image of Clark Veterans Cemetery? Please answer yes or no. Yes
01a6842099d62f22.jpg Is this an image of Mirdif City Centre? Please answer yes or no. No
01aca2d30b43d05f.jpg Is this a picture of Monte Minerva (Villanova Monteleone)? Please answer yes or no. Yes
01aca2d30b43d05f.jpg Is this a picture of Dorfkirche Gnandstein? Please answer yes or no. No
01b12048bab7439f.jpg Is this a photo of Kaptai Lake? Please answer yes or no. Yes
01b12048bab7439f.jpg Is this a photo of Castle of the Conception? Please answer yes or no. No
01b715ceb7050bce.jpg Is this a picture of Kalamita fortress? Please answer yes or no. Yes
01b715ceb7050bce.jpg Is this a picture of Augsburger Hotelturm? Please answer yes or no. No
01b9b58a573c45f0.jpg Is this a picture of St Mary Redcliffe? Please answer yes or no. Yes
01b9b58a573c45f0.jpg Is this a picture of Kirche zur Heiligsten Dreifaltigkeit, Vienna? Please answer yes or no. No
01bc5557ed2616ad.jpg Is this a photo of Intercession of the Theotokos church in Lutsk? Please answer yes or no. Yes
01bc5557ed2616ad.jpg Is this a photo of Cap 110? Please answer yes or no. No
01c0e0ff84631905.jpg Is this a photo of Ribblehead Viaduct? Please answer yes or no. Yes
01c0e0ff84631905.jpg Is this a photo of Church of Saint George in Koptevo? Please answer yes or no. No
01d6e386c4a01ab3.jpg Is this a picture of Rotes Rathaus? Please answer yes or no. Yes
01d6e386c4a01ab3.jpg Is this a picture of Mahipatgad? Please answer yes or no. No
01d9bef0120557a5.jpg Is this a photo of Andronikov Monastery? Please answer yes or no. Yes
01d9bef0120557a5.jpg Is this a photo of Ghatkopar? Please answer yes or no. No
01e198bea8285816.jpg Is this an image of Saint Basil church, Volodymyr-Volynskyi? Please answer yes or no. Yes
01e198bea8285816.jpg Is this an image of Hoher Markt, Vienna? Please answer yes or no. No
01f23b4c5253e9c6.jpg Is this a picture of Goethes Gartenhaus? Please answer yes or no. Yes
01f23b4c5253e9c6.jpg Is this a picture of Lamba, Shetland Islands? Please answer yes or no. No
01ffc77bdf7fe084.jpg Is this a picture of Oldemorstoft? Please answer yes or no. Yes
01ffc77bdf7fe084.jpg Is this a picture of Roman Catholic Diocese of Antwerp? Please answer yes or no. No
0200cce957e9c4a6.jpg Is this a photo of Malmuerta Tower? Please answer yes or no. Yes
0200cce957e9c4a6.jpg Is this a photo of Monte Lema? Please answer yes or no. No
0202aac6b0bfbcd8.jpg Is this a picture of Gettysburg National Military Park? Please answer yes or no. Yes
0202aac6b0bfbcd8.jpg Is this a picture of Auron? Please answer yes or no. No
0205343362e60daf.jpg Is this a picture of Church of Our Lady of Sorrows in Riga? Please answer yes or no. Yes
0205343362e60daf.jpg Is this a picture of Simon en Judaskerk (Ootmarsum)? Please answer yes or no. No
020ae4dd487a8ab0.jpg Is this a photo of Kashueti Church? Please answer yes or no. Yes
020ae4dd487a8ab0.jpg Is this a photo of Colorado State Capitol? Please answer yes or no. No
02259144c3f094f0.jpg Is this a picture of The American Adventure? Please answer yes or no. Yes
02259144c3f094f0.jpg Is this a picture of Langenwaldschanze? Please answer yes or no. No
0237fc92f6b31aeb.jpg Is this a photo of Abbaye de Mortemer? Please answer yes or no. Yes
0237fc92f6b31aeb.jpg Is this a photo of St Thomas Rest Park? Please answer yes or no. No
0240727dbd2b8531.jpg Is this a photo of Nicolaikerk Appingedam? Please answer yes or no. Yes
0240727dbd2b8531.jpg Is this a photo of Cutimbo? Please answer yes or no. No
02545ca051c6c480.jpg Is this a photo of Podilski Tovtry National Nature Park? Please answer yes or no. Yes
02545ca051c6c480.jpg Is this a photo of Lyndonville, Vermont? Please answer yes or no. No
02598153523880aa.jpg Is this a photo of Marilao Church? Please answer yes or no. Yes
02598153523880aa.jpg Is this a photo of Madliena, Malta? Please answer yes or no. No
02737bb70957827d.jpg Is this an image of White Pagoda in Zhakou? Please answer yes or no. Yes
02737bb70957827d.jpg Is this an image of Norrstrandskyrkan? Please answer yes or no. No
0273f127895337be.jpg Is this a photo of Ringkirche? Please answer yes or no. Yes
0273f127895337be.jpg Is this a photo of Huxi Mosque? Please answer yes or no. No
02785530b6ac4f56.jpg Is this a picture of Church of Nicholas the Wet? Please answer yes or no. Yes
02785530b6ac4f56.jpg Is this a picture of Razula? Please answer yes or no. No
028348883685db69.jpg Is this an image of Saint Andrew the Apostle Church? Please answer yes or no. Yes
028348883685db69.jpg Is this an image of EYE Film Institute Netherlands? Please answer yes or no. No
029431b10472c228.jpg Is this a picture of Santa Caterina (Sassari)? Please answer yes or no. Yes
029431b10472c228.jpg Is this a picture of Former Tainan Assembly Hall? Please answer yes or no. No
0295877d5cd76190.jpg Is this a picture of Groothoofdspoort? Please answer yes or no. Yes
0295877d5cd76190.jpg Is this a picture of 707 17th Street? Please answer yes or no. No
02b16429304b00b7.jpg Is this a picture of De Haag? Please answer yes or no. Yes
02b16429304b00b7.jpg Is this a picture of Villa Melchett? Please answer yes or no. No
02b7f3ae549582f1.jpg Is this a photo of Pont-canal de Digoin? Please answer yes or no. Yes
02b7f3ae549582f1.jpg Is this a photo of Trenton Battle Monument? Please answer yes or no. No
02d11e392d3ab93f.jpg Is this an image of Santa Monica Parish Church in Angat, Bulacan? Please answer yes or no. Yes
02d11e392d3ab93f.jpg Is this an image of Bangkok City Pillar Shrine? Please answer yes or no. No
02e38cd5a3850f60.jpg Is this a photo of Palazzo Cavalli Franchetti (Venice)? Please answer yes or no. Yes
02e38cd5a3850f60.jpg Is this a photo of Isbister Chambered Cairn? Please answer yes or no. No
02e3e418198427e7.jpg Is this an image of Goldener Reiter? Please answer yes or no. Yes
02e3e418198427e7.jpg Is this an image of Palacio de la Mosquera? Please answer yes or no. No
030d9a26b7d5755e.jpg Is this a photo of Medvedgrad? Please answer yes or no. Yes
030d9a26b7d5755e.jpg Is this a photo of Tammisto? Please answer yes or no. No
0317376043fb00eb.jpg Is this a photo of Butler's Wharf? Please answer yes or no. Yes
0317376043fb00eb.jpg Is this a photo of Church of Saints Philip and James (Lelekovice)? Please answer yes or no. No
03280db41fd4e4f5.jpg Is this a picture of Oxford Street? Please answer yes or no. Yes
03280db41fd4e4f5.jpg Is this a picture of Blake Fell? Please answer yes or no. No
034733eb61d663cd.jpg Is this a picture of Colonna dell'Immacolata (Rome)? Please answer yes or no. Yes
034733eb61d663cd.jpg Is this a picture of Brookmill Park? Please answer yes or no. No
034d066523bb20d1.jpg Is this a photo of Ibrahim-al-Ibrahim Mosque? Please answer yes or no. Yes
034d066523bb20d1.jpg Is this a photo of Atami Castle? Please answer yes or no. No
036b60d84c6ebc63.jpg Is this a photo of Frederik V statue in Amalienborg Palace? Please answer yes or no. Yes
036b60d84c6ebc63.jpg Is this a photo of Karl-Bittel-Park? Please answer yes or no. No
03729394d18f7e9a.jpg Is this an image of Sapporo Clock Tower? Please answer yes or no. Yes
03729394d18f7e9a.jpg Is this an image of Saint Nikolaus Church (Beuster)? Please answer yes or no. No
0384fd0664c0ca62.jpg Is this an image of St. Martin (Sontheim)? Please answer yes or no. Yes
0384fd0664c0ca62.jpg Is this an image of Hot Creek (Mono County, California)? Please answer yes or no. No
038bdff81be029a6.jpg Is this an image of Fosso Reale (Livorno)? Please answer yes or no. Yes
038bdff81be029a6.jpg Is this an image of Farlington Marshes? Please answer yes or no. No
039bbdf71cd17b3f.jpg Is this a picture of Qiantang River Bridge? Please answer yes or no. Yes
039bbdf71cd17b3f.jpg Is this a picture of Casa Museo Boschi Di Stefano (Milan)? Please answer yes or no. No
03aa5dd2e3aab548.jpg Is this a photo of Central Police Station? Please answer yes or no. Yes
03aa5dd2e3aab548.jpg Is this a photo of Kontai-ji? Please answer yes or no. No
03c4b1901e4195f4.jpg Is this an image of St. Johanniskirche Eppendorf? Please answer yes or no. Yes
03c4b1901e4195f4.jpg Is this an image of Maidenhead Bridge? Please answer yes or no. No
03c8b2c1306c0038.jpg Is this an image of Ermita de Sant Joan (Montserrat)? Please answer yes or no. Yes
03c8b2c1306c0038.jpg Is this an image of Hamilton Grange National Memorial? Please answer yes or no. No
03d5e3bfc958be38.jpg Is this an image of Pietrarsa railway museum? Please answer yes or no. Yes
03d5e3bfc958be38.jpg Is this an image of Washita Battlefield National Historic Site? Please answer yes or no. No
03dc7df1f1df7b8f.jpg Is this a picture of Yevhen Paton Bridge? Please answer yes or no. Yes
03dc7df1f1df7b8f.jpg Is this a picture of Minoan Villa of Makrygialos? Please answer yes or no. No
03f65f52c0077b6c.jpg Is this a photo of Reading Lighthouse? Please answer yes or no. Yes
03f65f52c0077b6c.jpg Is this a photo of St Nicholas Kirk, Aberdeen? Please answer yes or no. No
03f873280d814ae7.jpg Is this a photo of Clarence Dock (Leeds)? Please answer yes or no. Yes
03f873280d814ae7.jpg Is this a photo of Madonna delle Grazie (Roccastrada)? Please answer yes or no. No
03f8abf9f8473a6f.jpg Is this a picture of Parque nacional del Teide? Please answer yes or no. Yes
03f8abf9f8473a6f.jpg Is this a picture of Church of Saint Mary the Virgin, Skiemonys? Please answer yes or no. No
03fb778d388ad9a0.jpg Is this an image of Mercado Central de Santiago? Please answer yes or no. Yes
03fb778d388ad9a0.jpg Is this an image of Castelo de Alegrete? Please answer yes or no. No
041da60e053c5906.jpg Is this a picture of Schloss Weesenstein? Please answer yes or no. Yes
041da60e053c5906.jpg Is this a picture of Mas Cabrafiga? Please answer yes or no. No
042ca5291cefe0b2.jpg Is this a picture of Pfarrkirche hl. Augustinus, Perchtoldsdorf? Please answer yes or no. Yes
042ca5291cefe0b2.jpg Is this a picture of Mark Twain Riverboat? Please answer yes or no. No
0443da4f4fcdea52.jpg Is this an image of Port of Funchal? Please answer yes or no. Yes
0443da4f4fcdea52.jpg Is this an image of Rauscherpark? Please answer yes or no. No
04531b37dfe5c991.jpg Is this a picture of Roebling Suspension Bridge? Please answer yes or no. Yes
04531b37dfe5c991.jpg Is this a picture of St. James Episcopal Church (Wilmington, North Carolina)? Please answer yes or no. No
04534ba775aa38ad.jpg Is this an image of Cork City Gaol? Please answer yes or no. Yes
04534ba775aa38ad.jpg Is this an image of Taramati Baradari? Please answer yes or no. No
04590c1648dd25a0.jpg Is this an image of Zhejiang Sci-Tech University? Please answer yes or no. Yes
04590c1648dd25a0.jpg Is this an image of Khalti Lake? Please answer yes or no. No
045b65ea196dac3f.jpg Is this a photo of Republican architecture in Manizales? Please answer yes or no. Yes
045b65ea196dac3f.jpg Is this a photo of Kayauchi-banta Cliff? Please answer yes or no. No
045ec5631638d290.jpg Is this a picture of Baldachin of Saint Peter's Basilica? Please answer yes or no. Yes
045ec5631638d290.jpg Is this a picture of Jean Baptiste Point Du Sable Homesite? Please answer yes or no. No
04a5226570dce6c3.jpg Is this a picture of Church of the Assumption of the Virgin Mary in Chrudim? Please answer yes or no. Yes
04a5226570dce6c3.jpg Is this a picture of Queen Elizabeth Country Park? Please answer yes or no. No
04b6c84ce89461ec.jpg Is this an image of Tomb of Xerxes I? Please answer yes or no. Yes
04b6c84ce89461ec.jpg Is this an image of Park HaMesila? Please answer yes or no. No
04d67117549e07a8.jpg Is this a photo of Laguna de Salinas? Please answer yes or no. Yes
04d67117549e07a8.jpg Is this a photo of Raspyatsky Monastery (Serpukhov)? Please answer yes or no. No
04ebe2885134597f.jpg Is this a picture of The South Africa (Delville Wood) National Memorial? Please answer yes or no. Yes
04ebe2885134597f.jpg Is this a picture of Imogiri? Please answer yes or no. No
050464e4ebc0b36d.jpg Is this an image of Tai Wong Ye Temple, Wong Chuk Hang? Please answer yes or no. Yes
050464e4ebc0b36d.jpg Is this an image of Hierve el Agua? Please answer yes or no. No
05095dd2b6715fc3.jpg Is this an image of Sampsonievsky Cathedral? Please answer yes or no. Yes
05095dd2b6715fc3.jpg Is this an image of I. and E. Greenwald Steam Engine No. 1058? Please answer yes or no. No
05685b1070b828f5.jpg Is this a photo of Wurstelprater? Please answer yes or no. Yes
05685b1070b828f5.jpg Is this a photo of Nanmyaebonthar Sannandawya Sandamuni Pagoda? Please answer yes or no. No
05b01b255cec2bfb.jpg Is this a photo of Puthoorppilly Sree Krishnaswamy Temple? Please answer yes or no. Yes
05b01b255cec2bfb.jpg Is this a photo of Villa Reale (Marlia)? Please answer yes or no. No
05bcf05b4460d2c3.jpg Is this an image of Museo archeologico nazionale dell'Umbria? Please answer yes or no. Yes
05bcf05b4460d2c3.jpg Is this an image of Mode Gakuen Cocoon Tower? Please answer yes or no. No
05c3aa404351f407.jpg Is this an image of Pasaje Gutierrez (Valladolid)? Please answer yes or no. Yes
05c3aa404351f407.jpg Is this an image of Pieve di Santa Maria a Carli? Please answer yes or no. No
05d56e249af839e7.jpg Is this a photo of Serbian Orthodox Cathedral in Sarajevo? Please answer yes or no. Yes
05d56e249af839e7.jpg Is this a photo of Seewaldsee (Sankt Koloman)? Please answer yes or no. No
06134984e2aa9bcc.jpg Is this a picture of Grote Kerk (Groede)? Please answer yes or no. Yes
06134984e2aa9bcc.jpg Is this a picture of Goldeck? Please answer yes or no. No
0613d7e8f292ee15.jpg Is this a photo of Museo Archeologico G. Rambotti? Please answer yes or no. Yes
0613d7e8f292ee15.jpg Is this a photo of Sanrei? Please answer yes or no. No
062298815e9197e7.jpg Is this a photo of Liuyang Confucius Temple? Please answer yes or no. Yes
062298815e9197e7.jpg Is this a photo of Astley Castle? Please answer yes or no. No
06483ba2bff074bc.jpg Is this an image of Museo Universitario del Chopo? Please answer yes or no. Yes
06483ba2bff074bc.jpg Is this an image of Cedar Park, Seattle, Washington? Please answer yes or no. No
06740b93ad1f6c02.jpg Is this a picture of Borisoglebsky Monastery (Borisoglebsky)? Please answer yes or no. Yes
06740b93ad1f6c02.jpg Is this a picture of Aletsch Arena? Please answer yes or no. No
067ea27ec08c9354.jpg Is this a picture of Schlosskirche St. Aegidien (Bernburg)? Please answer yes or no. Yes
067ea27ec08c9354.jpg Is this a picture of The Haining? Please answer yes or no. No
06a85d0b5d70881d.jpg Is this an image of Silwan? Please answer yes or no. Yes
06a85d0b5d70881d.jpg Is this an image of Jagdschloss Platte? Please answer yes or no. No
06ab45292ecd8bf7.jpg Is this a picture of Maribor Regional Museum? Please answer yes or no. Yes
06ab45292ecd8bf7.jpg Is this a picture of Myres Castle? Please answer yes or no. No
06b6531dab42ccab.jpg Is this a picture of Roland (Riga)? Please answer yes or no. Yes
06b6531dab42ccab.jpg Is this a picture of Wudangshan Shangdi Temple? Please answer yes or no. No
06d039ff97afef04.jpg Is this a picture of Arch of Galerius (Thessaloniki)? Please answer yes or no. Yes
06d039ff97afef04.jpg Is this a picture of Catholic Church of Wanchin? Please answer yes or no. No
06e453b06a3e4054.jpg Is this an image of Pendeen Lighthouse? Please answer yes or no. Yes
06e453b06a3e4054.jpg Is this an image of Silberbach (Heubach)? Please answer yes or no. No
06e927f784642ce4.jpg Is this a picture of Museo Delta Antico? Please answer yes or no. Yes
06e927f784642ce4.jpg Is this a picture of OSTEC Exhibition Hall? Please answer yes or no. No
070c8d6326a87397.jpg Is this an image of Lanxmeerpoort? Please answer yes or no. Yes
070c8d6326a87397.jpg Is this an image of Tramway du Cap-Ferret? Please answer yes or no. No
0713451824a443d4.jpg Is this a picture of Castillo de Sax? Please answer yes or no. Yes
0713451824a443d4.jpg Is this a picture of Roman Catholic Diocese of Orlando? Please answer yes or no. No
0743e58605497887.jpg Is this an image of Capitoline temple (Brescia)? Please answer yes or no. Yes
0743e58605497887.jpg Is this an image of Burg Horn? Please answer yes or no. No
075264ec41057049.jpg Is this a picture of Imatrankoski? Please answer yes or no. Yes
075264ec41057049.jpg Is this a picture of Jarlsberg travbane? Please answer yes or no. No
07551cf96af77299.jpg Is this a photo of Friedhof Wilmersdorf? Please answer yes or no. Yes
07551cf96af77299.jpg Is this a photo of Partisan cemetery in Mostar? Please answer yes or no. No
075f8d86303a020e.jpg Is this a photo of Church of the Nativity (Bethlehem)? Please answer yes or no. Yes
075f8d86303a020e.jpg Is this a photo of Pennsylvania Railroad Office Building? Please answer yes or no. No
07ac52cf4620519f.jpg Is this a photo of San Pietro (Reggio Emilia)? Please answer yes or no. Yes
07ac52cf4620519f.jpg Is this a photo of Plaza de toros de La Condomina? Please answer yes or no. No
07b04584e67cb39d.jpg Is this a picture of Smithfield, Dublin? Please answer yes or no. Yes
07b04584e67cb39d.jpg Is this a picture of Novyi Svit Sanctuary? Please answer yes or no. No
07ed43dd266b669b.jpg Is this a photo of Rocky Mountain National Park? Please answer yes or no. Yes
07ed43dd266b669b.jpg Is this a photo of Col-des-Roches? Please answer yes or no. No
083dd5a3ba3d96dc.jpg Is this a picture of Teddington Cemetery? Please answer yes or no. Yes
083dd5a3ba3d96dc.jpg Is this a picture of Khutir Nadia? Please answer yes or no. No
08442f24f3c25565.jpg Is this a photo of Abdijkerk Loosduinen? Please answer yes or no. Yes
08442f24f3c25565.jpg Is this a photo of Rinsey Head? Please answer yes or no. No
08a0d2fd3fe6dc95.jpg Is this a picture of Dreifaltigkeitskirche (Ulm)? Please answer yes or no. Yes
08a0d2fd3fe6dc95.jpg Is this a picture of Wat Pa Mamuang? Please answer yes or no. No
08c43cd8f2be6709.jpg Is this a picture of Parchi di Nervi? Please answer yes or no. Yes
08c43cd8f2be6709.jpg Is this a picture of Altlayer Schweiz? Please answer yes or no. No
09232aeaf46dec64.jpg Is this a picture of Canadian Museum of Nature? Please answer yes or no. Yes
09232aeaf46dec64.jpg Is this a picture of Gwangtonggwan? Please answer yes or no. No
092d0859ff6424de.jpg Is this a photo of Kasteel Lunenburg? Please answer yes or no. Yes
092d0859ff6424de.jpg Is this a photo of Thomas Paine Cottage? Please answer yes or no. No
092e075e8e44c228.jpg Is this a picture of ChaoTianGong? Please answer yes or no. Yes
092e075e8e44c228.jpg Is this a picture of Pygmalion Theater Wien? Please answer yes or no. No
093ad5564a004653.jpg Is this a picture of Crescent Lake (Dunhuang)? Please answer yes or no. Yes
093ad5564a004653.jpg Is this a picture of Kirjurinluoto Arena? Please answer yes or no. No
0962855b67159733.jpg Is this a photo of Monastery of Monsalud? Please answer yes or no. Yes
0962855b67159733.jpg Is this a photo of ToonSeum? Please answer yes or no. No
09a51d517f1a72d5.jpg Is this a picture of Ford's Theatre? Please answer yes or no. Yes
09a51d517f1a72d5.jpg Is this a picture of Burg Hachen? Please answer yes or no. No
09ae0628a75e4fcc.jpg Is this a picture of Onze-Lieve-Vrouw Hemelvaartkerk (Zottegem)? Please answer yes or no. Yes
09ae0628a75e4fcc.jpg Is this a picture of Salvation Army in Canada? Please answer yes or no. No
09b7acf87c8046ec.jpg Is this an image of Church of the Epiphany (Yaroslavl)? Please answer yes or no. Yes
09b7acf87c8046ec.jpg Is this an image of Szlenkier Palace? Please answer yes or no. No
09b94a2f304a540c.jpg Is this a photo of Abbaye Notre-Dame du Val? Please answer yes or no. Yes
09b94a2f304a540c.jpg Is this a photo of Hotarumachi, Osaka? Please answer yes or no. No
09ce89cbf237da03.jpg Is this a picture of Church of Saint Mary Magdalene (Brno)? Please answer yes or no. Yes
09ce89cbf237da03.jpg Is this a picture of Wain Wath Force? Please answer yes or no. No
0a3035bfca2ab920.jpg Is this an image of Ortigia? Please answer yes or no. Yes
0a3035bfca2ab920.jpg Is this an image of Montmayeur castle? Please answer yes or no. No
0ab2ed007db301d5.jpg Is this a picture of Highgate Cemetery? Please answer yes or no. Yes
0ab2ed007db301d5.jpg Is this a picture of Isola Maggiore? Please answer yes or no. No
0acc12d1261d99ca.jpg Is this an image of Wat Chedi Si Hong? Please answer yes or no. Yes
0acc12d1261d99ca.jpg Is this an image of Leschi Park? Please answer yes or no. No
0adb088e24d88123.jpg Is this a picture of Iglesia de Santo Domingo (La Serena)? Please answer yes or no. Yes
0adb088e24d88123.jpg Is this a picture of Canelles de Baix (la Vall de Bianya)? Please answer yes or no. No
0ae3355e0a54db21.jpg Is this a photo of Grimspound? Please answer yes or no. Yes
0ae3355e0a54db21.jpg Is this a photo of Lough Owel? Please answer yes or no. No
0b4240b5ef69362e.jpg Is this a photo of Saint Anne church in Nowa Ruda? Please answer yes or no. Yes
0b4240b5ef69362e.jpg Is this a photo of Bray Wick? Please answer yes or no. No
0b64944086932c90.jpg Is this a picture of Alte Pfarrkirche St. Martin (Moosach)? Please answer yes or no. Yes
0b64944086932c90.jpg Is this a picture of Maya Devi Temple, Lumbini? Please answer yes or no. No
0b8fa4152f1bc3ed.jpg Is this a picture of Lac du Salagou? Please answer yes or no. Yes
0b8fa4152f1bc3ed.jpg Is this a picture of Holsljunga kyrka? Please answer yes or no. No
0ba6fa7ade050d49.jpg Is this a photo of Churches in Zemaiciu Kalvarija? Please answer yes or no. Yes
0ba6fa7ade050d49.jpg Is this a photo of Waiola Church? Please answer yes or no. No
0bd72080836eac97.jpg Is this a picture of Castelo de Sesimbra? Please answer yes or no. Yes
0bd72080836eac97.jpg Is this a picture of Gouffre Berger? Please answer yes or no. No
0bdd954254310e73.jpg Is this a photo of Melrose House? Please answer yes or no. Yes
0bdd954254310e73.jpg Is this a photo of Former water tower (Tsubame, Niigata)? Please answer yes or no. No
0c06fe97f38f6f88.jpg Is this a photo of Belvoir Castle? Please answer yes or no. Yes
0c06fe97f38f6f88.jpg Is this a photo of Su'ao Cold Spring? Please answer yes or no. No
0c6b0c5d2a401285.jpg Is this a photo of Pelham Crescent? Please answer yes or no. Yes
0c6b0c5d2a401285.jpg Is this a photo of Manila Ocean Park? Please answer yes or no. No
0cc5201e37961a1d.jpg Is this a photo of Adare Manor? Please answer yes or no. Yes
0cc5201e37961a1d.jpg Is this a photo of Leopoldspark? Please answer yes or no. No
0cd8fda9668b3ed2.jpg Is this a picture of Church of the Holy Mandylion (Klyazma)? Please answer yes or no. Yes
0cd8fda9668b3ed2.jpg Is this a picture of Kalayar Kovil? Please answer yes or no. No
0cf2e28266a0f1a1.jpg Is this an image of Piazza Bra (Verona)? Please answer yes or no. Yes
0cf2e28266a0f1a1.jpg Is this an image of Keeneland? Please answer yes or no. No
0cf48b827c9c99de.jpg Is this a photo of Beatus Rhenanus Bridge? Please answer yes or no. Yes
0cf48b827c9c99de.jpg Is this a photo of Ossian Hall? Please answer yes or no. No
0d23f6c345b68c05.jpg Is this an image of Monte Olivia? Please answer yes or no. Yes
0d23f6c345b68c05.jpg Is this an image of Crackstate? Please answer yes or no. No
0d70bceb2557d1f4.jpg Is this an image of Spaso-Preobrazhensky Cathedral (Pereslavl-Zalessky)? Please answer yes or no. Yes
0d70bceb2557d1f4.jpg Is this an image of Manta Rota? Please answer yes or no. No
0d7716f4aed0c9fa.jpg Is this a photo of Willemsbrug in Rotterdam? Please answer yes or no. Yes
0d7716f4aed0c9fa.jpg Is this a photo of Grumentum, Parco archeologico? Please answer yes or no. No
0d89a8a5e8c269e4.jpg Is this an image of Burg Normannstein? Please answer yes or no. Yes
0d89a8a5e8c269e4.jpg Is this an image of Cape Borda Lighthouse? Please answer yes or no. No
0e04b4f7f9801ea9.jpg Is this a picture of Vase sculpture, Bremen? Please answer yes or no. Yes
0e04b4f7f9801ea9.jpg Is this a picture of Mount Wilhelm? Please answer yes or no. No
0e0511c6e33636ca.jpg Is this a picture of Victoria Stadium? Please answer yes or no. Yes
0e0511c6e33636ca.jpg Is this a picture of Villa Belvedere (San Giuliano Terme)? Please answer yes or no. No
0e553d5d99e21c32.jpg Is this a picture of Mustafa Pasha Bridge? Please answer yes or no. Yes
0e553d5d99e21c32.jpg Is this a picture of Scrattons Eco Park? Please answer yes or no. No
0eb857c972ddc4c8.jpg Is this an image of Nabq Protected Area? Please answer yes or no. Yes
0eb857c972ddc4c8.jpg Is this an image of Maletsunyane Falls? Please answer yes or no. No
0f92862c0366ab60.jpg Is this a photo of Cotehele? Please answer yes or no. Yes
0f92862c0366ab60.jpg Is this a photo of Mozirje Grove? Please answer yes or no. No
0fc48f210ea7d363.jpg Is this a photo of Schauenburg (Oberkirch)? Please answer yes or no. Yes
0fc48f210ea7d363.jpg Is this a photo of Dartford Library? Please answer yes or no. No
100e5402cfb2dfcf.jpg Is this a picture of Nanzenji? Please answer yes or no. Yes
100e5402cfb2dfcf.jpg Is this a picture of Bagakain Lake? Please answer yes or no. No
101786bd2ec5800e.jpg Is this a picture of Church of the Merciful Saviour in Kuskovo (Moscow)? Please answer yes or no. Yes
101786bd2ec5800e.jpg Is this a picture of Central Board of Secondary Education? Please answer yes or no. No
1023b7ec812bb4c6.jpg Is this a photo of St. Joseph's Cathedral, Dunedin? Please answer yes or no. Yes
1023b7ec812bb4c6.jpg Is this a photo of Fagervik Church? Please answer yes or no. No
1053f4cebfceabd8.jpg Is this an image of Woolwich foot tunnel? Please answer yes or no. Yes
1053f4cebfceabd8.jpg Is this an image of Kirchenbibliothek St. Marien Barth? Please answer yes or no. No
10a944858a4283e0.jpg Is this a picture of Harding Tomb? Please answer yes or no. Yes
10a944858a4283e0.jpg Is this a picture of Kartause Ittingen? Please answer yes or no. No
10b2b8741f283a3c.jpg Is this a photo of De Bataaf, Winterswijk? Please answer yes or no. Yes
10b2b8741f283a3c.jpg Is this a photo of Porte Guillaume-Lion? Please answer yes or no. No
11b6bb2c35f4ee22.jpg Is this an image of Ku-ring-gai Chase National Park? Please answer yes or no. Yes
11b6bb2c35f4ee22.jpg Is this an image of Saint Mark's Coptic Orthodox Cathedral, Cairo? Please answer yes or no. No
127be7d34a21e5ed.jpg Is this a picture of Beachy Head? Please answer yes or no. Yes
127be7d34a21e5ed.jpg Is this a picture of Jindo Bridge? Please answer yes or no. No
130846b4f29fe4ba.jpg Is this a photo of Duomo (Viterbo)? Please answer yes or no. Yes
130846b4f29fe4ba.jpg Is this a photo of Raffles Praslin? Please answer yes or no. No
13a563a1a8ef38ae.jpg Is this a photo of Torre del Rellotge d'Olesa de Montserrat? Please answer yes or no. Yes
13a563a1a8ef38ae.jpg Is this a photo of Salzburg Museum? Please answer yes or no. No
14d3aba340d2fba9.jpg Is this an image of Regard Saint-Martin? Please answer yes or no. Yes
14d3aba340d2fba9.jpg Is this an image of Mount Donna Buang? Please answer yes or no. No
15b75eb6fca4aeb6.jpg Is this a photo of City Duma building (Saint Petersburg)? Please answer yes or no. Yes
15b75eb6fca4aeb6.jpg Is this a photo of Komadome-Hachiman-jinja? Please answer yes or no. No
15ec5d2c0898a56e.jpg Is this a photo of Kids Marina Rotterdam? Please answer yes or no. Yes
15ec5d2c0898a56e.jpg Is this a photo of Kirkwood Observatory? Please answer yes or no. No
15f326a030396dba.jpg Is this a picture of Macquarie Centre? Please answer yes or no. Yes
15f326a030396dba.jpg Is this a picture of California Historical Landmarks in San Mateo County, California? Please answer yes or no. No
162a77504f4e686e.jpg Is this an image of Wieliczka Castle? Please answer yes or no. Yes
162a77504f4e686e.jpg Is this an image of Naval Station Guantanamo Bay? Please answer yes or no. No
16b76bdd7eaaef1b.jpg Is this a picture of Evangelische Kirche Weinfelden? Please answer yes or no. Yes
16b76bdd7eaaef1b.jpg Is this a picture of Gorg Blau? Please answer yes or no. No
16d55614b78b6b8b.jpg Is this an image of Church of the Assumption in Bishche? Please answer yes or no. Yes
16d55614b78b6b8b.jpg Is this an image of Water of Girvan? Please answer yes or no. No
17e0cd732e35d4aa.jpg Is this a photo of Millennium Tower (San Francisco, California)? Please answer yes or no. Yes
17e0cd732e35d4aa.jpg Is this a photo of New Orleans Botanical Garden? Please answer yes or no. No
18608ca3b785abf8.jpg Is this a picture of Solvognen? Please answer yes or no. Yes
18608ca3b785abf8.jpg Is this a picture of Schenley Farms National Historic District? Please answer yes or no. No
197e440391d309f8.jpg Is this a picture of Skansin? Please answer yes or no. Yes
197e440391d309f8.jpg Is this a picture of Bardstown Historic District? Please answer yes or no. No
19fe04377265454e.jpg Is this a picture of Alexander Nevsky Church, Belgrade? Please answer yes or no. Yes
19fe04377265454e.jpg Is this a picture of Seppiko-Mineyama Hyogo Prefectural Nature Park? Please answer yes or no. No
1a32959819038291.jpg Is this an image of Id Kah Mosque? Please answer yes or no. Yes
1a32959819038291.jpg Is this an image of Katharinenkirche (Zwickau)? Please answer yes or no. No
1ac0e3c493073995.jpg Is this an image of Church of Saint John the Baptist (Jihlava)? Please answer yes or no. Yes
1ac0e3c493073995.jpg Is this an image of Silte kyrka? Please answer yes or no. No
1d4be36153741972.jpg Is this a picture of Marie Guyart Building? Please answer yes or no. Yes
1d4be36153741972.jpg Is this a picture of Motol? Please answer yes or no. No
1f233b807b660d25.jpg Is this a photo of Aussichtsturm Pyramidenkogel? Please answer yes or no. Yes
1f233b807b660d25.jpg Is this a photo of Vvedenskoe cemetery? Please answer yes or no. No
27a4267b9ea26cd0.jpg Is this an image of Oude Salviuskerk (Limbricht)? Please answer yes or no. Yes
27a4267b9ea26cd0.jpg Is this an image of Festung Heldrungen? Please answer yes or no. No
2a895eb889d6fd99.jpg Is this an image of Beijing Guozijian? Please answer yes or no. Yes
2a895eb889d6fd99.jpg Is this an image of Klinikkirche (Pfafferode)? Please answer yes or no. No
2f25654bcb445452.jpg Is this a photo of Rinku Gate Tower Building? Please answer yes or no. Yes
2f25654bcb445452.jpg Is this a photo of Bigorski Monastery? Please answer yes or no. No
3c89eb4043bcefac.jpg Is this a picture of Begijnhofkerk (Mechelen)? Please answer yes or no. Yes
3c89eb4043bcefac.jpg Is this a picture of Panther Hollow Lake? Please answer yes or no. No
internvl_chat/eval/mme/Your_Results/numerical_calculation.txt 0 → 100644
View file @ 1d5a34cf
0001.png Is the answer to the arithmetic question in the image 225? Please answer yes or no. Yes
0001.png Is the answer to the arithmetic question in the image 1515? Please answer yes or no. No
0002.png Is the answer to the arithmetic question in the image 340? Please answer yes or no. Yes
0002.png Is the answer to the arithmetic question in the image 17? Please answer yes or no. No
0003.png Is the answer to the arithmetic question in the image 65? Please answer yes or no. Yes
0003.png Is the answer to the arithmetic question in the image 56? Please answer yes or no. No
0004.png Is the answer to the arithmetic question in the image 33? Please answer yes or no. Yes
0004.png Is the answer to the arithmetic question in the image 32? Please answer yes or no. No
0005.png Is the area of the square in the picture equal to 40? Please answer yes or no. Yes
0005.png Is the area of the square in the picture equal to 8? Please answer yes or no. No
0006.png Is the area of the square in the picture equal to 9? Please answer yes or no. Yes
0006.png Is the area of the square in the picture equal to 3? Please answer yes or no. No
0007.png Is the answer to the arithmetic question in the image 49? Please answer yes or no. Yes
0007.png Is the answer to the arithmetic question in the image 39? Please answer yes or no. No
0008.png Should the value of "a" in the picture equal 7? Please answer yes or no. Yes
0008.png Should the value of "a" in the picture equal 14? Please answer yes or no. No
0009.png Should the value of "a" in the picture equal 2? Please answer yes or no. Yes
0009.png Should the value of "a" in the picture equal 3? Please answer yes or no. No
0010.png Is the answer to the arithmetic question in the image 13? Please answer yes or no. Yes
0010.png Is the answer to the arithmetic question in the image 12? Please answer yes or no. No
0011.png Is the area of the parallelogram in the picture equal to 24? Please answer yes or no. Yes
0011.png Is the area of the parallelogram in the picture equal to 6? Please answer yes or no. No
0012.png Should the value of "a" in the picture equal 9? Please answer yes or no. Yes
0012.png Should the value of "a" in the picture equal 1? Please answer yes or no. No
0013.png Is the area of the right triangle in the picture equal to 24? Please answer yes or no. Yes
0013.png Is the area of the right triangle in the picture equal to 8? Please answer yes or no. No
0014.png Is the answer to the arithmetic question in the image 200? Please answer yes or no. Yes
0014.png Is the answer to the arithmetic question in the image 400? Please answer yes or no. No
0015.png Is the answer to the arithmetic question in the image 11? Please answer yes or no. Yes
0015.png Is the answer to the arithmetic question in the image 111? Please answer yes or no. No
0016.png Is the answer to the arithmetic question in the image 9? Please answer yes or no. Yes
0016.png Is the answer to the arithmetic question in the image 16? Please answer yes or no. No
0017.png Is the answer to the arithmetic question in the image 14? Please answer yes or no. Yes
0017.png Is the answer to the arithmetic question in the image 83? Please answer yes or no. No
0018.png Should the value of "a" in the picture equal 3? Please answer yes or no. Yes
0018.png Should the value of "a" in the picture equal 2? Please answer yes or no. No
0019.png Is the answer to the arithmetic question in the image 18? Please answer yes or no. Yes
0019.png Is the answer to the arithmetic question in the image 36? Please answer yes or no. No
0020.png Is the answer to the arithmetic question in the image 9? Please answer yes or no. Yes
0020.png Is the answer to the arithmetic question in the image 45? Please answer yes or no. No
internvl_chat/eval/mme/Your_Results/position.txt 0 → 100644
View file @ 1d5a34cf
000000006471.jpg Is the cricket bat above the batter's body? Please answer yes or no. Yes
000000006471.jpg Is the cricket bat under the batter's body Please answer yes or no. No
000000007281.jpg Is the sea behind people in the image? Please answer yes or no. Yes
000000007281.jpg Is the sea in front of people in the image? Please answer yes or no. No
000000014038.jpg Is the refrigerator on the left side of the picture? Please answer yes or no. Yes
000000014038.jpg Is the refrigerator on the right side of the picture Please answer yes or no. No
000000031248.jpg Is there a sofa in the middle of potted plants in the image? Please answer yes or no. Yes
000000031248.jpg Is there a sofa in the right side of potted plants in the image? Please answer yes or no. No
000000048504.jpg Is the gray elephant in front of the brown elephant? Please answer yes or no. Yes
000000048504.jpg Is the brown elephant in front of the gray elephant? Please answer yes or no. No
000000052007.jpg Are the pedestrians on the right of the bus? Please answer yes or no. Yes
000000052007.jpg Are the pedestrians on the left of the bus? Please answer yes or no. No
000000056127.jpg Is the light above the fire hydrant in the image? Please answer yes or no. Yes
000000056127.jpg Is the light under the fire hydrant in the image? Please answer yes or no. No
000000062025.jpg Is the trash can under the cup in the image？ Please answer yes or no. Yes
000000062025.jpg Is the trash can above the cup in the image？ Please answer yes or no. No
000000062808.jpg Is the phone above the pizza in the image? Please answer yes or no. Yes
000000062808.jpg Is the phone under the pizza in the image? Please answer yes or no. No
000000067213.jpg Is the dog above the pool in the image? Please answer yes or no. Yes
000000067213.jpg Is the dog under the pool in the image? Please answer yes or no. No
000000097994.jpg Is the light above the computer in the image? Please answer yes or no. Yes
000000097994.jpg Is the light under the computer in the image? Please answer yes or no. No
000000204871.jpg Is the car on the right side of the fire hydrant in the picture? Please answer yes or no. Yes
000000204871.jpg Is the car on the left side of the fire hydrant in the picture? Please answer yes or no. No
000000206487.jpg Is the motorcycle on the right side of the bus? Please answer yes or no. Yes
000000206487.jpg Is the motorcycle on the left side of the bus Please answer yes or no. No
000000211825.jpg Is the cake on the left side of the camera? Please answer yes or no. Yes
000000211825.jpg Is the cake on the right side of the camera? Please answer yes or no. No
000000212800.jpg Is the blue umbrella under the black umbrella? Please answer yes or no. Yes
000000212800.jpg Is the blue umbrella above the black umbrella? Please answer yes or no. No
000000395701.jpg Is the TV on the left of the bookshelf? Please answer yes or no. Yes
000000395701.jpg Is the TV on the right of the bookshelf? Please answer yes or no. No
000000395801.jpg Is the clock above people? Please answer yes or no. Yes
000000395801.jpg Is the clock under people? Please answer yes or no. No
000000405970.jpg Is the grey sofa on the right of the TV? Please answer yes or no. Yes
000000405970.jpg Is the grey sofa on the left of the TV? Please answer yes or no. No
000000426241.jpg Is the white mouse on the right of the black keyboard? Please answer yes or no. Yes
000000426241.jpg Is the white mouse on the left of the black keyboard? Please answer yes or no. No
000000450303.jpg Is the monitor on top of a person? Please answer yes or no. Yes
000000450303.jpg Is the monitor under the person? Please answer yes or no. No
000000458410.jpg Is the TV on the left of the lamp? Please answer yes or no. Yes
000000458410.jpg Is the TV on the right of the lamp? Please answer yes or no. No
000000472046.jpg Is the pineapple on the left of the pot in the image? Please answer yes or no. Yes
000000472046.jpg Is the pineapple on the right of the pot in the image? Please answer yes or no. No
000000477955.jpg Is the person under the kite? Please answer yes or no. Yes
000000477955.jpg Is the person above the kite? Please answer yes or no. No
000000482585.jpg Is the person on the right of the train? Please answer yes or no. Yes
000000482585.jpg Is the person on the left of the train? Please answer yes or no. No
000000494869.jpg Is the baby on the right of the dog in the image? Please answer yes or no. Yes
000000494869.jpg Is the baby on the left of the dog in the image? Please answer yes or no. No
000000509699.jpg Is the mirror above the TV? Please answer yes or no. Yes
000000509699.jpg Is the mirror under the TV? Please answer yes or no. No
000000519569.jpg Is the vase on the left of the bottle? Please answer yes or no. Yes
000000519569.jpg Is the vase on the right of the bottle? Please answer yes or no. No
000000530162.jpg Is the big red and black umbrella on the top of people? Please answer yes or no. Yes
000000530162.jpg Is the big red and black umbrella under people? Please answer yes or no. No
000000551660.jpg Is the spoon in the bowl? Please answer yes or no. Yes
000000551660.jpg Is the spoon out of the bowl? Please answer yes or no. No
000000578922.jpg Is the vase on the left of the toothbrush? Please answer yes or no. Yes
000000578922.jpg Is the vase on the right of the toothbrush? Please answer yes or no. No
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment