Yup, I have now measured it inside one language. I rewrote the 20 English golden questions in a customer's words. "Bank" for PSP, "instant transfer" for SCT Inst, none of the section's own terms. The original wording puts the answer page in the top 5 for 17 of 21 questions, when the everyday wording does it for 14.
One example: "available on weekends and public holidays" finds the page at rank 2 vs. "Can I send an instant transfer on a Sunday?" finds it at rank 46.
The page says "Calendar Days". So a golden set in the rulebook's own words does look healthier than real questions. Here the gap is 3 of 21, before any language change.
On translating the query first: I measured it in this article. It brought Russian to within one question of English and did nothing for Spanish. Two questions already sat at rank 4 and 5 in English, so any rewording pushes them out. Inspectable, agreed. It is also a paraphrase, and the fragile questions pay for it. The phrasing numbers go out as the next piece.
Narrowing this to whether the right page still lands in the top 5 is the right scope, and it is the measurement most cross-lingual discussions skip past on the way to answer quality.
The thing I would watch in the numbers is that degradation tends to be uneven rather than uniform. Domain terms that are loanwords survive the language switch nearly intact, while everyday phrasing collapses, so a golden set weighted toward jargon looks far healthier than real user questions do.
If you extend the series, translating the query into the corpus language before retrieval is usually cheaper than a multilingual embedder, and it has the advantage of being inspectable when it goes wrong.