Discussion on "RAG vs Long Context in .NET"

Patrick Kearns · 2026-10-03T16:36:07.172Z

A .NET application that answers questions over documents has to decide what evidence to send to the language model. You can find a few relevant passages, load complete documents, or retrieve documents

EvidencePacker re-counting the assembled messages on each trial is a defensible correctness baseline because metadata and framing consume tokens too. With many candidate sections, however, it repeatedly tokenizes the same growing prefix, so packing itself can become measurable work before the model call.

I would record packing latency alongside retrieval and generation, particularly for parent-expanded evidence. An incremental counter could reduce repeated work, but only after testing it against the complete serialized request; summing isolated text counts can miss boundary and framing effects. Keeping the existing full-count path as an oracle would make that optimization safer.

Yeah, I’d expect packing latency to be measured alongside retrieval and generation when taking this into production. The full count approach gives the example a straightforward correctness baseline, if profiling shows it's becoming expensive, an incremental counter is a sensible optimisation, validated against that existing path.

Keeping the example's full-count path as the baseline makes sense. For an incremental implementation, I would build the parity fixtures around boundary cases: an empty section, a source label containing non-ASCII text, a section admitted exactly at the budget, and one rejected just above it.

Assert the selected section IDs as well as the final count. Two counters can disagree only slightly yet choose different evidence at the boundary, which changes the model input even when both totals look reasonable. Profiling can then decide whether the extra implementation is justified.

The distinction between finding relevant evidence and preserving sufficient evidence for reasoning is an important one here. A retrieval system can return highly relevant chunks and still fail if the answer depends on a definition, exception, or relationship that wasn't included in the final evidence bundle. I’d be interested in evaluating “evidence completeness” alongside retrieval precision and recall: not just whether the expected passage was retrieved, but whether all dependencies required to justify the answer survived selection and packing. That metric could also make the RAG-versus-long-context decision more concrete. If a task repeatedly requires broad evidence relationships, long context or structured parent expansion may be the better baseline even when top-k retrieval looks strong.

Yeah, thats worth measuring alongside precision and recall. Id test completeness against the evidence actually sent to the model, because retrieving a required definition or exception doesn't help if packing drops it. Questions with known evidence dependencies would make a useful comparison between top-k retrieval, parent expansion and long context, checking whether each approach preserves enough evidence to support the answer.

Search Hashnode

RAG vs Long Context in .NET

Responses(5)