Increase limit on number of detections per image in {COCO,LVIS}Evaluator
Summary: ## Context - The current limit on the number of detections per image (`K`) in LVIS is 300. - Implementing AP_pool/AP_fixed requires removing this default limit on `K` - [Literature](https://arxiv.org/pdf/2102.01066.pdf) has shown that increasing `K` correlates with AP gains ## This Diff - Changed limit on number of detections per image (`K`) to be customizable for LVIS and COCO through `TEST.DETECTIONS_PER_IMAGE` in the config - For COCO: - Maintain the default `max_dets_per_image` to be [1, 10, 100] as from [COCOEval](https://www.internalfb.com/code/fbsource/[88bb57c3054a]/fbcode/deeplearning/projects/cocoApi/PythonAPI/pycocotools/cocoeval.py?lines=28-29) - Allow users to input a custom integer for `TEST.DETECTIONS_PER_IMAGE` in the config, and use [1, 10, `TEST.DETECTIONS_PER_IMAGE`] for COCOEval - For LVIS: - Maintain the default `max_dets_per_image` to be 300 as from [LVISEval](https://www.internalfb.com/code/fbsource/[f6b86d023721]/fbcode/deeplearning/projects/lvisApi/lvis/eval.py?lines=528-529) - Allow users to input a custom integer for `TEST.DETECTIONS_PER_IMAGE` in the config, and use this in LVISEval - Added `COCOevalMaxDets` for evaluating AP with the custom limit on number of detections per image (since default `COCOeval` uses 100 as limit on detections per image for evaluating AP) ## Inference Runs using this Diff - Performed inference using `K = {300, 1000, 10000, 100000}` - Launched fblearner flows for object detector baseline models with N1055536 (LVIS) and N1055756 (COCO) - Recorded [results of running inference](https://docs.google.com/spreadsheets/d/1rgdjN2KvxcYfKCkGUC4tMw0XQJ5oZL0dwjOIh84YRg8/edit?usp=sharing) Reviewed By: ppwwyyxx Differential Revision: D30077359 fbshipit-source-id: 372eb5e0d7c228fb77fe23bf80d53597ec66287b
Showing
Please register or sign in to comment