README.md

English | 简体中文

Introduction

STD EvalKit (Small Target Detection Evaluation Kit) is a library of evaluation metrics toolbox for infrared small target segmentation tasks.

We statistics the evaluation metrics in the field of infrared small target segmentation in recent years(statistical results).

Major features

• High Efficiency

  Multi-threading.

• Device Friendly

  All metrics support automatic batch accumulation.

• Unified API

  All metrics provide the same API, Metric.update(labels, preds) complete the accumulation of batches， Metric.get() get metrics。

• Unified Computational

  We use the same calculation logic and algorithms for the same type of metrics, ensuring consistency between results.

• Supports multiple data formats

  Supports multiple input data formats, hwc/chw/bchw/bhwc/image path, more details in

  ./notebook/tutorial.ipynb

Overview of Metrics

Based on the data required for the calculation of the evaluation metrics, we have classified the metrics into three broad categories, Pixel-Level, Center-Level, and Center.

Architectures

Pixel Level	Target Level
AUC ROC AP PR Precision Recall F1 IoU (DOI:10.1109/TAES.2023.3238703) NormalizedIoU (DOI:10.1109/WACV48630.2021.00099)	Center-Level	Box Level
	Precision Recall F1 (DOI:10.1109/TAES.2022.3159308) Average Precision (Ours) Pd_Fa (DOI:10.1109/TIP.2022.3199107) ROC Pd_Fa Center Normalized IoU (Ours)	Mean Average Precision, Recall (COCO)

Installation

git clone [email protected]:IRSTD/STD-EvalKit.git

cd STD-EvalKit

For developers(recommended, easy for debugging)

pip install -e .

Only use

pip install stdeval

Tutorial

from stdeval.metrics import PixelPrecisionRecallF1IoU
Metric = PixelPrecisionRecallF1IoU(
    conf_thr=0.5,
    )
Metric.update(labels=labels, preds=preds.sigmoid())
precision, recall, f1_score, iou = Metric.get()

For more details, please refer to

./notebook/tutorial.ipynb