Commit 2fb273ab authored by Lydia Chan's avatar Lydia Chan Committed by Facebook GitHub Bot
Browse files

Increase limit on number of detections per image in {COCO,LVIS}Evaluator

Summary:
## Context
- The current limit on the number of detections per image (`K`) in LVIS is 300.
- Implementing AP_pool/AP_fixed requires removing this default limit on `K`
- [Literature](https://arxiv.org/pdf/2102.01066.pdf) has shown that increasing `K` correlates with AP gains

## This Diff
- Changed limit on number of detections per image (`K`) to be customizable for LVIS and COCO through `TEST.DETECTIONS_PER_IMAGE` in the config
   - For COCO:
       - Maintain the default `max_dets_per_image` to be [1, 10, 100] as from [COCOEval](https://www.internalfb.com/code/fbsource/[88bb57c3054a]/fbcode/deeplearning/projects/cocoApi/PythonAPI/pycocotools/cocoeval.py?lines=28-29)
       - Allow users to input a custom integer for `TEST.DETECTIONS_PER_IMAGE` in the config, and use  [1, 10, `TEST.DETECTIONS_PER_IMAGE`] for COCOEval
   - For LVIS:
       - Maintain the default `max_dets_per_image` to be 300 as from [LVISEval](https://www.internalfb.com/code/fbsource/[f6b86d023721]/fbcode/deeplearning/projects/lvisApi/lvis/eval.py?lines=528-529)
       - Allow users to input a custom integer for `TEST.DETECTIONS_PER_IMAGE` in the config, and use this in LVISEval
- Added `COCOevalMaxDets` for evaluating AP with the custom limit on number of detections per image (since default `COCOeval` uses 100 as limit on detections per image for evaluating AP)

## Inference Runs using this Diff
- Performed inference using `K = {300, 1000, 10000, 100000}`
- Launched fblearner flows for object detector baseline models with N1055536 (LVIS) and N1055756 (COCO)
  - Recorded [results of running inference](https://docs.google.com/spreadsheets/d/1rgdjN2KvxcYfKCkGUC4tMw0XQJ5oZL0dwjOIh84YRg8/edit?usp=sharing)

Reviewed By: ppwwyyxx

Differential Revision: D30077359

fbshipit-source-id: 372eb5e0d7c228fb77fe23bf80d53597ec66287b
parent 0a38f8c8
...@@ -56,6 +56,7 @@ from detectron2.evaluation import ( ...@@ -56,6 +56,7 @@ from detectron2.evaluation import (
COCOEvaluator, COCOEvaluator,
RotatedCOCOEvaluator, RotatedCOCOEvaluator,
DatasetEvaluators, DatasetEvaluators,
LVISEvaluator,
inference_on_dataset, inference_on_dataset,
print_csv_format, print_csv_format,
verify_results, verify_results,
...@@ -534,11 +535,18 @@ class Detectron2GoRunner(BaseRunner): ...@@ -534,11 +535,18 @@ class Detectron2GoRunner(BaseRunner):
dataset_name, dataset_name,
output_dir=output_folder, output_dir=output_folder,
kpt_oks_sigmas=cfg.TEST.KEYPOINT_OKS_SIGMAS, kpt_oks_sigmas=cfg.TEST.KEYPOINT_OKS_SIGMAS,
max_dets_per_image=cfg.TEST.DETECTIONS_PER_IMAGE,
) )
elif evaluator_type in ["rotated_coco"]: elif evaluator_type in ["rotated_coco"]:
dataset_evaluators = DatasetEvaluators( dataset_evaluators = DatasetEvaluators(
[RotatedCOCOEvaluator(dataset_name, cfg, True, output_folder)] [RotatedCOCOEvaluator(dataset_name, cfg, True, output_folder)]
) )
elif evaluator_type in ["lvis"]:
dataset_evaluators = LVISEvaluator(
dataset_name,
output_dir=output_folder,
max_dets_per_image=cfg.TEST.DETECTIONS_PER_IMAGE,
)
else: else:
dataset_evaluators = D2Trainer.build_evaluator( dataset_evaluators = D2Trainer.build_evaluator(
cfg, dataset_name, output_folder cfg, dataset_name, output_folder
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment