Comment by Ahmet Özel on "Should you trust an object detector’s confidence?"

Ahmet Özel

AI Engineer. Computer Vision, RAG and LLM agents.

Keeping the same 10,893 detections before and after correction makes the calibration improvement interpretable. There is an important population boundary in the threshold table: these detections were selected using original scores of at least 0.25, so a corrected-score threshold is being evaluated inside that already-filtered set.

If deployment applies Platt correction to lower original scores and admits some of them, the 1.31-point ECE does not yet validate that expanded population. I would report the original-score floor alongside each operational threshold, or repeat calibration evaluation over the actual deployed candidate set. That preserves the distinction between remapping existing detections and changing which detections reach users.

Oleg Serov

Sharing the results of my endless experiments.

Oct 18

Thanks, Ahmet! Apologies for a delayed response.

That's an important point. The ECE applies only to the fixed set selected using original scores ≥0.25. It doesn’t validate lower-scoring detections that a deployment pipeline might admit after correction.

The correction was also fitted on that filtered population, so applying it below the original-score floor would need separate validation. Reporting that floor alongside each operational threshold would make the scope clearer. That's a great direction for the next step, thank you!

The fitting population matters as much as the evaluation population here. A correction fitted above 0.25 could look smooth below that point while being unsupported by any calibration examples.

For an extension, I would save candidates at a lower original-score floor first, then fit and test on separate images from that expanded set. Report retained true positives and false positives near the admission boundary alongside calibration. That would show whether the new population is useful operationally, while keeping the existing 10,893-detection result as a clearly defined baseline.

Search Hashnode