Your eval monitor fired on four days this week. At your sample size, that was the most likely count

A team I compared notes with recently runs continuous evals on production traffic: an LLM judge scores a sample of traces, the scores land on a dashboard, and a monitor pages when the hourly pass rate