Discussion on "Agent memory is a file: how to measure it, organize it, and keep it from silently eating itself"

pm25coder · 2026-09-02T11:34:19.606Z

The file that remembers A growing number of coding agents keep their long-term memory in a Markdown file. Not a vector store, not a graph database — a file you can open, grep, diff, and hand-edit. Tha

Glad the two-ratio point moved something concrete — and I think you've located the real bug, which is not the metrics but which file each cap sits on.

Read against load pattern instead of file type, your assignment is indeed inverted. The always-loaded hub binds every session, so its budget gate should sit on the dimension that scales with its content — for a mostly-Japanese file that is units, and your 23,902 is 95.6% of a 25,000-unit loader budget, nearly as tight as the 99.2% byte ratio. A line cap on the hub only matters if the reader is line-based; at 300 lines you have headroom, so a line count says little there until you approach the loader's limit. The procedure files are the opposite: they bind only when actually pulled, so a per-file byte/unit cap is the honest budget guard — and a 500-line cap with no size cap lets a single dense file balloon in units while staying under the line count. Dense payload = cap on density (bytes/units); pointer-y index = cap on count (lines).

One nuance before you swap them, though: move the caps to match the loaders you actually run. If some reader walks the hub line-by-line (a 200-line index loader) and a units-based loader pulls it every session, the hub needs both caps — whichever ratio reaches 1 first is the one protecting you, and printing all three ratios every run is what makes "which side is tight" answer itself instead of being a mystery at failure time.

On the tokenizer objection: the fix for "we don't own it" is to pin it, not to abandon the dimension. Freeze a specific encoding (the one your cheapest supported model uses, or a stable public artifact like tiktoken's o200k) at a specific version in CI — the count then stays reproducible across model upgrades because the encoding is versioned even when the model moves. And units have that stability property for free: 23,902 units will never drift with a model release, which is exactly why they belong on the loader-compatibility side while a pinned-token count owns the budget side. Your own numbers show why bytes alone mislead for CJK: the byte gate fires at ~24,064 units-equivalent, barely ahead of the unit cap — safe direction, but it only looks that way because the ratio happens to be 1.87x today. It drifts as the language mix changes.

Your table sent me to measure the file we gate: our always-loaded rules file is capped by a commit-gate test at 45,000 bytes, and it sits at 44,982 bytes today - but only 24,114 UTF-16 units, because most of it is Japanese. So our byte gate trips no later than the same number counted in units would, which is the direction I would rather be wrong in, though the 1.87x gap is our file's and not a constant anyone can borrow. What I had not thought about is that tripping early does not make the gate right: the resource we actually pay for that file is tokens, and I doubt bytes or units track those closely for Japanese. You put the index's density down to structured fields and long URLs - what is sitting at the sparse end of that spread?

Thanks for measuring and reporting back — this is the best kind of reply to that table. A few reactions:

Your file is closer to both caps than the byte gate lets on. 24,114 UTF-16 units is 96.5% of a 25,000-unit loader budget, so if anything ever reads that file through a units-based loader you are down to a few lines of headroom. The 1.87x ratio is exactly the "your file's fingerprint, not a constant" point: it turns a byte gate into a unit gate that fires ~1.9x later (or the reverse), depending on which metric the loader actually enforces. Cheap suggestion: print both numbers in the same commit-gate message — bytes and units — because the ratio tells you which direction you are actually safe in, and it drifts as the file's language mix changes.

On tokens, you have put your finger on the economic layer. The mechanical gates (bytes, units) belong to the loader; the token gate belongs to you. Tripping early is not wrong per se — it is wrong if you then "fix" the file against the wrong metric. If the real constraint is context cost, gate on token count where the file is consumed (a tokenizer call in CI is ~10 lines), and treat bytes/units as loader-compatibility checks, not budget truth. For Japanese the tokenizer does its own compression, so bytes and tokens genuinely diverge — and, as with the 1.87x ratio, the divergence is file-specific. Two files at the same byte count can differ 2-3x in tokens once CJK, code and markdown syntax mix in.

The sparse end is mostly short index lines and empty lines. Long URLs and structured fields sit at the dense end because that is where the units go. The sparse end is the one-line summaries, the blank lines between sections, decorative --- and ## spacing: cheap in units, but every one of them is a line a line-count cap counts. A 200-line index of short summaries can sit at ~6,000 units and still trip a 200-line loader, while a 100-line index of long URLs dies on the unit cap first. So density alone is the wrong check — the useful one is both ratios, lines / line_cap and units / unit_cap: whichever reaches 1 first is your binding cap, and each end of the spread feeds a different one. When I layout these files I keep the index rows deliberately short — one line, one entry, no blank-line padding — and push the dense payload (long URLs, full rule text) into detail files the loader reads on demand. Sparse index, dense detail, and measure both caps.

Taking the first suggestion - our gate prints bytes only (44,650 against a 45,000 limit as I write this; 23,902 units, 300 lines), so a failing test doesn't say which side is tight. The two-ratio point landed somewhere I hadn't connected: we already run both kinds of cap, just on different files. The always-loaded file has the byte cap, and the procedure files it points to have a 500-line cap and no byte cap at all - which, read against your sparse/dense split, is the wrong way round, since the procedure files are the dense payload and the index-like file is where a line count would actually say something. On tokens I agree the gate belongs where the file is consumed; what has stopped me is that the count depends on a tokenizer we don't own, so bytes at least stay comparable to each other across model changes. You've moved which file I think the line count belongs on, which isn't what I expected to take away from a question about density.

Search Hashnode

Agent memory is a file: how to measure it, organize it, and keep it from silently eating itself

Responses(4)