Putting evaluation in the same tier as chunking and retrieval is what most weekend guides get wrong, and the regulated-environment vantage point makes the argument land harder. Order matters too: if the eval set does not exist before the chunker is tuned, every later decision is made on vibes and you cannot distinguish a reranker improvement from noise.

One addition from the compliance angle you write from. Audit trails want per-stage provenance, which is a constraint on chunk design rather than a logging afterthought. If a chunk cannot be traced back to a document version, a section and a page, the answer is unauditable no matter how good the retrieval was, and retrofitting that later means reindexing everything.