The 4 AM framing is the right way to specify this system, because it tells you which failures are acceptable. Answering slowly is annoying; answering confidently from a superseded revision of the equipment spec is the one that hurts, and in industrial documentation superseded revisions are everywhere - the same pump, three document versions, all indexed, none of them saying in the text which is current. Revision and effective date as hard filters rather than ranking signals is what separates this from a demo. The mixture of static spec and live telemetry also creates a subtler trap worth guarding: the two can disagree, and the model has no basis for deciding which wins. A spec saying the relief valve is set at one pressure and a sensor implying another is exactly the case where blending them into one fluent paragraph is worse than surfacing both with their sources and timestamps. Curious how you handled the case where the telemetry call fails - falling back silently to spec-only answers is the failure mode I would most want an explicit path for, since the answer still reads complete.
The 4 AM framing is the right way to specify this system, because it tells you which failures are acceptable. Answering slowly is annoying; answering confidently from a superseded revision of the equipment spec is the one that hurts, and in industrial documentation superseded revisions are everywhere - the same pump, three document versions, all indexed, none of them saying in the text which is current. Revision and effective date as hard filters rather than ranking signals is what separates this from a demo. The mixture of static spec and live telemetry also creates a subtler trap worth guarding: the two can disagree, and the model has no basis for deciding which wins. A spec saying the relief valve is set at one pressure and a sensor implying another is exactly the case where blending them into one fluent paragraph is worse than surfacing both with their sources and timestamps. Curious how you handled the case where the telemetry call fails - falling back silently to spec-only answers is the failure mode I would most want an explicit path for, since the answer still reads complete.