The distinction between a one-time conversion hit and error that compounds through a recurrent state is the useful part, and it is worth stating why the second is so much nastier operationally: the damage is a function of run length, so every short test passes. Quantise, benchmark on standard prompts, ship - and the failure only appears in the long sessions nobody puts in the eval set. Appreciate you flagging that the paper describes rather than measures, since that distinction usually gets lost by the second retelling. The measurement I would want is drift of the state against a full-precision reference as a function of steps, per quantisation scheme, because the interesting question is not whether error accumulates but whether it saturates or diverges - those imply completely different mitigations. If it diverges, a periodic full-precision refresh of the state is a cheap fix; if it saturates below tolerance, the whole thing is a non-issue for most workloads.
The distinction between a one-time conversion hit and error that compounds through a recurrent state is the useful part, and it is worth stating why the second is so much nastier operationally: the damage is a function of run length, so every short test passes. Quantise, benchmark on standard prompts, ship - and the failure only appears in the long sessions nobody puts in the eval set. Appreciate you flagging that the paper describes rather than measures, since that distinction usually gets lost by the second retelling. The measurement I would want is drift of the state against a full-precision reference as a function of steps, per quantisation scheme, because the interesting question is not whether error accumulates but whether it saturates or diverges - those imply completely different mitigations. If it diverges, a periodic full-precision refresh of the state is a cheap fix; if it saturates below tolerance, the whole thing is a non-issue for most workloads.