Perform Sliced (Tiled) Inference and Detailed Error Analysis Using Detectron2 Models

• Do you need sliced (tiled) inference for small object detection on satellite images or high-resolution images?
• Do you need explainable metrics for your detection model (insights on possible improvements)?
• Do you need an interactive UI to visualize faulty predictions?
• Do you rely on Detectron2 models?

All of them is possible with the new Detectron2 integration of SAHI 🚀
(Many thanks to Kadir Nar for his contributions in Detectron2 PR 🔥)

GitHub - obss/sahi: A lightweight vision library for performing large scale object detection/…

This post will walk you through installation, model preparation, sliced inference, error analysis and interactive visualization steps for your Detectron2 models.

Installation

✔️ Install Pytorch:

conda install pytorch=1.10.0 torchvision=0.11.1 cudatoolkit=11.3 -c pytorch

✔️ Install SAHI:

conda install sahi -c conda-forge
or
pip install -U sahi

✔️ Install Detectron2:

pip install detectron2 -f https://dl.fbaipublicfiles.com/detectron2/wheels/cu113/torch1.10/index.html

Preparation

✔️ Export your Detectron2 config:

# create desired Detectron2 config
from detectron2.config import get_cfg
cfg = get_cfg()
cfg.MODEL.ROI_HEADS.NUM_CLASSES = 445 cfg.MODEL.ROI_HEADS.SCORE_THRESH_TEST = 0.5
# export config as yaml
from sahi.utils.detectron2 import export_cfg_as_yaml export_cfg_as_yaml(cfg, export_path='config.yaml')

Sliced Prediction

Gif showing sliced prediction capabilities of SAHI.

✔️ Use your weight path and exported config path to perform inference:

sahi predict --source image_dir/ --model_type detectron2 --model_path weight.pt --config_path config.yaml --slice_height 512 --slice_width 512

Error Analysis Plots/Metrics

✔️ Create COCO formatted prediction results using COCO formatted dataset:

Gif showing COCO formatted dataset prediction capabilities of SAHI.

sahi predict --source image_dir/ --dataset_json_path dataset.json --model_type detectron2 --model_path weight.pt --config_path config.yaml --no_sliced_prediction

✔️ Create error analysis plots using the created result.json:

Gif showing error analysis capabilities of SAHI.

sahi coco analyse --dataset_json_path dataset.json --result_json_path result.json

🎯 Meaning of the metrics:

C75: Results at 0.75 IOU threshold
C50: Results at 0.50 IOU threshold
Loc: Results after ignoring localization errors
Sim: Results after ignoring supercategory false positives
Oth: Results after ignoring all category confusions
BG: Results after ignoring all false positives
FN: Results after ignoring all false negatives

📈 Possible model improvements:

C75-C50 and C50-Loc=Potential gain with more accurate bounding box prediction
Loc-Sim=Potential gain after fixing supercategory confusions
Loc-Oth=Potential gain after fixing category confusions
Oth-BG=Potential gain after fixing all false positives
BG-FN=Potential gain after fixing all false negatives

Interactive Visualization

✔️ Install fiftyone:

 pip install -U fiftyone

✔️ Start a fiftyone web app with the results of your prediction:

Gif showing interactive visualization capabilities of SAHI.

sahi coco fiftyone --dataset_json_path dataset.json --image_dir image_dir/ result.json