Great post, the section on severity tag calibration really caught my eye. Usually, you spend way more time optimizing metrics like precision or F1-score, but you shouldn't overlook user alignment or the model's user experience. These are things you might not even consider at first.