---
title: "Discussion on \"I Told Codex to Test Everything. It Believed Me.\" | Hashnode"
description: "Discussion on \"I Told Codex to Test Everything. It Believed Me.\". 443 subagents, two perfect 247-case test campaigns, and an Apple entitlement that never came. What I'd scope before the next handoff."
url: "https://hashnode.com/posts/codex-security-hardening/6ae263a92524d51eeb44811c"
---

[Feed](https://hashnode.com/feed) /

Discussion

# [I Told Codex to Test Everything. It Believed Me.](https://blog.theknowngood.com/codex-security-hardening?utm_source=hashnode&utm_medium=feed)

443 subagents, two perfect 247-case test campaigns, and an Apple entitlement that never came. What I'd scope before the next handoff.

Oct 28 · 10 min read · blog.theknowngood.com  
Read the article

-   [#agentic-ai](https://hashnode.com/tag/agentic-ai)
-   [#openai](https://hashnode.com/tag/openai)
-   [#cybersecurity](https://hashnode.com/tag/cybersecurity)
-   [#security](https://hashnode.com/tag/security)
-   [#ai](https://hashnode.com/tag/ai)

## Written by

[Kerry Kier](https://hashnode.com/@kkier)

@kkier · Sacramento, USA

  
[View profile](https://hashnode.com/@kkier)

Systems engineer working on public safety infrastructure, emergency communications, and where AI actually belongs.

I'm a systems engineer in a public safety environment. The systems I keep running are the ones people reach after something has already gone wrong, which sets a different bar than most IT work — the failure isn't a bad quarter, it's a call that doesn't connect. My work sits in three places: the network and virtualization layer under emergency communications, the security posture around it, and the practical question of where AI belongs in systems that can't afford to be confidently wrong. What I keep returning to are the failures that don't announce themselves — degraded service that still passes an availability check, a vendor dependency nobody inventoried, a monitor measuring the wrong layer entirely. I run my own infrastructure for the same reason: local inference stacks, self-hosted services, hardware I own end to end. It's the only way to see how something actually fails rather than how the documentation says it will. I write that up as field notes at blog.theknowngood.com, and maintain a reference dataset on AI model evaluation at theknowngood.com. Licensed amateur radio operator, KO6JKE. Troubleshooting an RF path and debugging a network stack are closer than they look, and both matter most when the usual channels are down.

## Responses 7

Markdown supported · ⌘↵ to post

[Kartik N V J K](https://hashnode.com/@kartik-nvj)

[Oct 29](https://hashnode.com/posts/codex-security-hardening/6ae263a92524d51eeb44811c/comment/6ae34f9ccabb70c219e8bb46)

161 subagents on September 26 alone, two days after you switched to GPT-6 Sol, and by then there was no way to tell whether the model or the P50 and P51 rounds drove it. I'd put the subagent count into those three plain daily sentences too. A burst like that should get flagged the same day.

[Kerry Kier](https://hashnode.com/@kkier)

[Oct 29](https://hashnode.com/posts/codex-security-hardening/6ae263a92524d51eeb44811c/comment/6ae34f9ccabb70c219e8bb46#reply-6ae372ec951d59945e956bdb)

I am not a develper and it shows, I am a sys engineer and build infrastructure so this was a large part of not knowing. I also think a small part was the fact i was using Daybreak with some Easing of cyber controls so it worked a bit more than a standard model. As for the model change i learned rthe hard way never to do this, and you are right i dont know if it was the OP50/51 rounds or the model that caused the issue, but apples endpoint security part was the ultiimate block. I could have taken a different direction and not used the enpoint piece of it but i would have lost the past 2 weeks of work and would have had to start over, in in a session that ran for a total of 14+ Days of actual agent work, would have been a nightmare.

[AI Developer](https://hashnode.com/@lycore)

[Oct 30](https://hashnode.com/posts/codex-security-hardening/6ae263a92524d51eeb44811c/comment/6ae34f9ccabb70c219e8bb46#reply-6ae3ee18d9938a35989f6ca0)

That 14-day session is the strongest argument for checkpoints. Since the Apple endpoint-security piece was the real blocker, a short written decision log at each phase boundary (what changed, why, what the exit criterion was) would let you retrace whether the model switch or the extra rounds drove the growth, and make it cheaper to branch off without losing the two weeks of work. Pinning the model for the length of a campaign and changing it only at a boundary is also a simple way to keep the comparison clean next time.

[Indie Ship Log](https://hashnode.com/@indieshiplog)

[Oct 29](https://hashnode.com/posts/codex-security-hardening/6ae263a92524d51eeb44811c/comment/6ae2f581055718c347a2cd1e)

The capybara site versus 443 subagents is what got me. I've written a version of "check whatever is needed and keep going" and then watched the work grow way past anything a reader would notice. A hard stop before I walk away would have saved me the same kind of week.

[Kerry Kier](https://hashnode.com/@kkier)

[Oct 29](https://hashnode.com/posts/codex-security-hardening/6ae263a92524d51eeb44811c/comment/6ae2f581055718c347a2cd1e#reply-6ae37402d02725a6dfce97b8)

You nailed this on the head, I shoud have gave it more controls, becuase ultimately when claude looked it over it came back and said basically, you are testing and hardening based on foregn giovernments trying to compromise the site and in reality, those people are not attacking a independant book webiste. Now what I have done is i have an agent that monitors CVE's and KEV's and as soon as one is relaeased (there is an hourly check for this) it tells the maintence and security subagents to investigate and supply a report via email with the reocommendation and the ability for me to either approve or instracut via email. So i figured this along with monthly maintenance on the server shoudl have most of the basis covered. I also run this site and one of my others on my own Infrastruture at home and putit behind cloudfalre so i did not have to open up ports on the firewall.

[AI Developer](https://hashnode.com/@lycore)

[Oct 29](https://hashnode.com/posts/codex-security-hardening/6ae263a92524d51eeb44811c/comment/6ae2d563fe0cc0a6810a0696)

The "whatever is needed" sentence is the real finding here, and I'd go a step further: the day-one scope prompt could also include a stop condition that's checked mechanically, not just stated. For example, a fixed budget of subagents or wall-clock hours per phase, after which the agent must stop and report rather than propose another revision. A related suggestion is to make the agent rank findings by cost-to-fix versus severity before designing anything, so the three Medium host findings and the absolute\_redirect off; fix land first. One question: since each campaign was bound to an exact artifact, did you consider a cheaper tier of evidence (a smoke subset of the 247 cases) for revisions that only touch files unrelated to the failure paths?

[Kerry Kier](https://hashnode.com/@kkier)

[Oct 29](https://hashnode.com/posts/codex-security-hardening/6ae263a92524d51eeb44811c/comment/6ae2d563fe0cc0a6810a0696#reply-6ae374e5613cb51b162fc28b)

After running this i learmned a lot more on how to approach this and now with artifacts avaiible as well as having it attached to my internal git with all the documents and instructions made the next hardening much easier and claude was able to do it with CVP entitlement with Mythos 5.1 a lot faster and mythos before i started even stated it did a full scan oin the server to see if that much was even needed. I have a global prompt that does not agree with me and doesnt try to appease but will go thorugh and think of ouitcomes before it answers as well as some other controls, becuase i got tired of it telling me i was right all the time when i was clearly wrong.
