Neither, as far as I could tell - it was session-level. Whichever rule won, won from the first message and held for the whole session - a long conversation didn't degrade it, it stayed consistently wrong in one direction. Then fresh session with the identical setup could flip. So each session looked internally fine on its own terms, which is why I kept blaming the rules instead of the setup.
My best guess at the mechanism: the two rules didn't reach the model the same way. The terse one gets re-stated in the prompt stream every turn, the readable one is a static file loaded once at the start. Same words, different reinforcement, so which one lead is mostly whichever landed closer.
On the eval, I don't have one. The bit I'd add is that a per-rule adherence eval still passes here, because each rule is obeyed fine in isolation. The conflict only shows up when following one breaks the other, so the adversarial prompt I'd want gives it something that can only be written tersely or readably, rather than just tempting it to be verbose.
Kartik N V J K
AI Developer | Making AI reliable, trustworthy & accessible to everyone | Active community contributor
Two rules and it broke both is the honest version of instruction-following that benchmarks tend to hide. What worked for me was testing rule adherence as its own eval, with adversarial prompts that tempt the agent to break each rule, instead of assuming the system prompt holds. Did the failures cluster on one rule, or did context length erode both?