The interesting part of the tournament model is that disagreement becomes useful data. But I’d add one caveat: three models are only meaningfully independent if they aren’t sharing the same assumptions, context, or failure mode. Three confident answers that converge on the same wrong premise can create more false confidence than using one model.
In production AI work at IT Path Solutions, I’d make the competition measurable: give each model the same constraints, require explicit assumptions and tradeoffs, then score the outputs against a predefined evaluation set rather than letting the human simply pick the most convincing response. That turns “multiple opinions” into an actual decision system and lets you see which model is consistently right for a particular class of problems, not just which one won a single round.
This is the sharpest objection to the whole idea, and it's right: three models from similar training lineages share priors, so they can converge confidently on the same wrong premise, which is worse than one model because it manufactures agreement. Two things I lean on: pick models of genuinely different lineage so their blind spots don't line up, and make each state its assumptions before its answer, so what you compare is the reasoning, not just the conclusion. On scoring, you're right that where a predefined eval set exists it should settle it, not the human picking the most convincing voice. The harder case is the novel problem with no eval set yet, and there the honest judge isn't persuasiveness, it's shipping the candidate and letting reality score it. Slower, but it doesn't lie.
The interesting part of the tournament model is that disagreement becomes useful data. But I’d add one caveat: three models are only meaningfully independent if they aren’t sharing the same assumptions, context, or failure mode. Three confident answers that converge on the same wrong premise can create more false confidence than using one model.
In production AI work at IT Path Solutions, I’d make the competition measurable: give each model the same constraints, require explicit assumptions and tradeoffs, then score the outputs against a predefined evaluation set rather than letting the human simply pick the most convincing response. That turns “multiple opinions” into an actual decision system and lets you see which model is consistently right for a particular class of problems, not just which one won a single round.