i think saving failed runs as their own records with a status and an exclusion reason is worth doing, since otherwise the denominator can shift without anyone noticing

printing the excluded count next to each metric helps a lot with that. for run counts, i’d look at how much results vary between repeats of the same query on your own prompts first, and then decide from there. also, country belongs in the key because the same prompt can surface different sources in different markets