faster-coco-eval 1.3.3

Faster interpretation of the original COCOEval

Faster-COCO-Eval

Disclaimer

I often use this project, but I saw it abandoned and without a public repository on github. Also, part of the project remained unfinished for a long time. I implemented some of the author's ideas and decided to make the results publicly available.

Faster-COCO-Eval base

This package wraps a facebook C++ implementation of COCO-eval operations found in the pycocotools package. This implementation greatly speeds up the evaluation time for coco's AP metrics, especially when dealing with a high number of instances in an image.

Comparison

For our use case with a test dataset of 5000 images from the coco val dataset. Testing was carried out using the mmdetection framework and the eval_metric.py script. The indicators are presented below.

Visualization of testing comparison.ipynb available in directory examples/comparison Tested with yolo3 model (bbox eval) and yoloact model (segm eval)

Type	COCOeval	COCOeval_faster	Profit
bbox	18.477 sec.	7.345 sec.	2.5x faster
segm	29.819 sec.	15.840 sec.	2x faster

Usage

This package contains a faster implementation of the pycocotools COCOEval class.
To import and use COCOeval_faster type:

from faster_coco_eval import COCO, COCOeval_faster
....

For usage, look at the original COCOEval class documentation.

Usage plot curves

from faster_coco_eval import COCO
from faster_coco_eval.extra import Curves

cocoGt = COCO(....)
cocoDt = cocoGt.loadRes(....)

cur = Curves(cocoGt, cocoDt, iou_tresh=0.5, iouType='segm')
cur.plot_pre_rec(plotly_backend=False)

Setup dependencies

• numpy
• plotly (optional if extra.Curve usage)

history

v1.3.3

• fix by ViTrox https://github.com/vitrox-technologies/faster_coco_eval
  • missing file issue
  • issue discovered by torchmetric
  • fstring for python3.7
  • Windows compilation

v1.3.2

• rework math_matches function. moved to faster_eval_api
• Moved calculations from python to c++
• Separated extra classes
• Added new sample data
• append mIoU based on TP pred.
• append mAUC based on Coco pre/rec.

v1.3.1

• rework mask code
• change np.float to float (numpy deprecations)

v1.3.0

• remove pycocotools dependencies
• clean c/c++ code

v1.2.3

• Implemented of mean IoU for TP
• set FP-red FN-blue

v1.2.2

• Removed own implementation of pre-rec
• Switched to the implementation of pre-rec calculation from COCO eval
• Lost backward compatibility
• Implemented output fp/fn/tp + gt to pictures

v1.2.1

• bug fix with pre-rec curve
• rework error calc (tp/fp/fn)
• change image plot to plotly
• append docker auto builder
• append native iou calc (slow but accurate)
• rework auc calc with link

v1.1.3-v1.1.4

• rebuild plotly backend
• Segm bug-fix

v1.1.2

• Append fp fn error analysis via curves
• Append confusion matrix
• Append plotly backend support for ROC / AUC

v1.1.1

• Redesigned curves
• Reworked data preload
• Append csrc to setup
• Build sdist Package

v1.1.0

• Wrap c++ code
• Get it to compile
• Add COCOEval class wraper
• Remove detectron2 dependencies
• Remove torch dependencies
• Append unittest
• Append ROC / AUC curves
• Check if it works on windows

TODOs

• Remove pycocotools dependencies
• Remove matplotlib dependencies

License

The original module was licensed with apache 2, I will continue with the same license. Distributed under the apache version 2.0 license, see license for more information.