180 maths questions, 9 right answers, and an AI that said it was 95 to 100% sure almost every time

Many AI products are built on a large language model (LLM), the kind of model behind ChatGPT. Many of them have a step that looks like this: the model answers, then the model is asked "how sure are yo