The point that a 91.4% overall can still hide a 70% category is the part most people skip, and it is exactly why a single aggregate threshold lulls you into shipping regressions. I like that you tested the gate for false negatives too, since a gate that fires on noise trains everyone to ignore it. Are your per-category thresholds static, or do you recompute them off a rolling baseline so a slow drift does not quietly eat your margin?
Kartik N V J K
AI Developer | Making AI reliable, trustworthy & accessible to everyone | Active community contributor
The point that a 91.4% overall can still hide a 70% category is the part most people skip, and it is exactly why a single aggregate threshold lulls you into shipping regressions. I like that you tested the gate for false negatives too, since a gate that fires on noise trains everyone to ignore it. Are your per-category thresholds static, or do you recompute them off a rolling baseline so a slow drift does not quietly eat your margin?