Discussion on "Should you trust an object detector’s confidence?"

Oleg Serov · 2026-10-07T01:32:49.373Z

An object detector draws a box around an object, names it, and gives it a confidence score. A score of 0.90 sounds like a 90% chance of being right. But does the detector actually get about nine out o

Keeping the same 10,893 detections before and after correction makes the calibration improvement interpretable. There is an important population boundary in the threshold table: these detections were selected using original scores of at least 0.25, so a corrected-score threshold is being evaluated inside that already-filtered set.

If deployment applies Platt correction to lower original scores and admits some of them, the 1.31-point ECE does not yet validate that expanded population. I would report the original-score floor alongside each operational threshold, or repeat calibration evaluation over the actual deployed candidate set. That preserves the distinction between remapping existing detections and changing which detections reach users.

Thanks, Ahmet! Apologies for a delayed response.

That's an important point. The ECE applies only to the fixed set selected using original scores ≥0.25. It doesn’t validate lower-scoring detections that a deployment pipeline might admit after correction.

The correction was also fitted on that filtered population, so applying it below the original-score floor would need separate validation. Reporting that floor alongside each operational threshold would make the scope clearer. That's a great direction for the next step, thank you!

The fitting population matters as much as the evaluation population here. A correction fitted above 0.25 could look smooth below that point while being unsupported by any calibration examples.

For an extension, I would save candidates at a lower original-score floor first, then fit and test on separate images from that expanded set. Report retained true positives and false positives near the admission boundary alongside calibration. That would show whether the new population is useful operationally, while keeping the existing 10,893-detection result as a clearly defined baseline.

The 61.81% average confidence against 78.45% actually correct is a clean case of underconfidence, and temperature scaling fixing the scores without moving a single box is the neat part. It is the lesson I keep relearning on evals: a healthy-looking number can still be miscalibrated, the way a green CI run hides a live regression (I wrote that up here: kartiknvjk.hashnode.dev/my-ci-evals-were-green-a-… ). Did 15 equal-width ECE bins ever mask something that adaptive-width bins caught?

Thanks, Kartik! Apologies for a long response. The CI analogy makes sense - one summary number can hide quite a lot.

In my experiment Platt scaling produced the large improvement, reducing ECE from 16.64 to 1.31 percentage points, however the temperature scaling only brought it down to 15.28. Both methods kept the boxes and rankings unchanged.

I haven’t tested adaptive-width bins in this experiment, so I can’t say they caught something the fixed bins missed. I did check 10, 15, and 20 equal-width bins: YOLO26n’s original ECE stayed at 16.64 points across all three. That confirms underconfidence finding, but still doesn’t really establish how sensitive the corrected result is to binning.

Thank you, comparing equal-count bins would be a useful follow-up, especially in the high-confidence region.

Really neat findings Oleg. Modern models improving accuracy while regressing on score reliability is a classic trade off. The Platt correction results are impressive. Did you notice any major difference in performance between temperature scaling and Platt scaling on rarer classes?

Ahila, thank you for you comment, that's an interesting metric to check! To do this I defined rare classes as the 20 COCO classes with the fewest labeled objects in the calibration split. Together, they had 773 eligible test detections. On this subset the ECE value for Temperature was about 9.89 percentage points, while Platt gave 3.74 (lower is better). So Platt still showed better calibration on the rare classes, although individual class results varied and many had small sample sizes.

Keeping the same 10,893 detections before and after correction makes the result much easier to interpret. Would you consider adding precision and retained-detection counts at a few thresholds on the corrected scores? That would help readers choose an operating threshold without confusing better calibration with improved recall.

Jam, thank you, that's a great point! I have the saved per-detection results, so it's possible report precision and retained counts at several corrected-score thresholds. I have added it to the article.

Thanks for adding the table, Oleg. Comparing the 0.90 and 0.95 rows makes the tradeoff much clearer: the extra precision comes with substantially fewer retained detections. That is much more useful for choosing a threshold than the ECE number alone.

Search Hashnode

Should you trust an object detector’s confidence?

Responses10