Manual. The tool wasn't released yet, so there was no production to get a signal from.

I harvested 13,474 real tool descriptions, ran the detector over them of which twenty servers failed. All twenty were wrong.

Nothing prompted it. Nothing looked broken. I went looking because I'd written both the tests and the rules, so the tests passing didn't tell me anything.

That's the part worth generalising I suppose, that this cateogry of bug has no symptom. You don't catch it by watching, you catch it by getting data you didn't make up.

I only half-fixed it, mind. My "12 of 13 attack shapes" uses a list I wrote myself, so it has the same problem. That one's still open.