---
title: "Discussion on \"A Wrong Ruler Is Worse Than No Ruler: Verifying the Checks You Trust\" | Hashnode"
description: "Discussion on \"A Wrong Ruler Is Worse Than No Ruler: Verifying the Checks You Trust\". In a rules-first AI system, the deterministic checks get authority. They gate output and overrule the model judge."
url: "https://hashnode.com/posts/a-wrong-ruler-is-worse-than-no-ruler-verifying-the-checks-you-trust/6aba98adea7b0501e7d55f5b"
---

[Feed](https://hashnode.com/feed) /

Discussion

# [A Wrong Ruler Is Worse Than No Ruler: Verifying the Checks You Trust](https://harrisonsec.hashnode.dev/a-wrong-ruler-is-worse-than-no-ruler-verifying-the-checks-you-trust?utm_source=hashnode&utm_medium=feed)

In a rules-first AI system, the deterministic checks get authority. They gate output and overrule the model judge.

Sep 28 · 12 min read · harrisonsec.hashnode.dev  
Read the article

-   [#ai](https://hashnode.com/tag/ai)
-   [#observability](https://hashnode.com/tag/observability)
-   [#testing](https://hashnode.com/tag/testing)

## Written by

[Harrison Guo](https://hashnode.com/@harrisonguo)

@harrisonguo · WhiteRock, CA

  
[View profile](https://hashnode.com/@harrisonguo)

Security engineer and distributed systems practitioner. Writing about systems internals, performance, security research, and AI applications. Building harrisonsec.com.

## Responses 2

Markdown supported · ⌘↵ to post

[Mike Dabydeen](https://hashnode.com/@firelinks)

[Sep 29](https://hashnode.com/posts/a-wrong-ruler-is-worse-than-no-ruler-verifying-the-checks-you-trust/6aba98adea7b0501e7d55f5b/comment/6abbba0dbb0fe8785e3c21c2)

Your four moves verify a check before it gets the veto. I'd add one for after: keep proving it in production, because a ruler can go wrong later without anyone touching it. A change to the normalisation step, a new document template or a library upgrade can change what the hash sees, and the check keeps its authority the whole time.

The cheapest version is a small set of known-repetitive and known-varied documents fed through the live pipeline on a schedule, each with its expected verdict. If a planted near-duplicate comes back clean, the check loses its veto until someone looks. I'd also watch the veto rate itself. A check whose override count drops to zero after a deploy looks like quiet success, which is exactly the shape of failure you described.

On shadow mode: disagreement between the rule and the judge tells you how often they differ, not which one is right. A sampled human read of the disagreements is what settles the direction.

[Harrison Guo](https://hashnode.com/@harrisonguo)

[Oct 5](https://hashnode.com/posts/a-wrong-ruler-is-worse-than-no-ruler-verifying-the-checks-you-trust/6aba98adea7b0501e7d55f5b/comment/6abbba0dbb0fe8785e3c21c2#reply-6ac3ecd8397c67fd54c52e03)

That fifth move is the one I should have included, and you've named why it matters: a ruler drifts without anyone touching it. A normalisation tweak, a new template, a library bump, and the hash quietly sees something different while the check keeps its full authority. The concrete version of your golden set is good precisely because it's cheap: fix the expected verdict for each known-repetitive and known-varied document, run them on every deploy, and alarm when a verdict changes that nobody intended. The check checking itself. And your point on shadow mode is the one people skip: the disagreement rate only tells you the rule and the judge differ, never which is right, so without a sampled human read of those disagreements you've measured noise, not direction.