Here’s the key: after pretraining, the model’s latent space is frozen. Post-training and prompting don't change this landscape; they change how we navigate it. Iterating over prompts, for ICL or any task, is the process of finding the optimal starting point and pathway through that fixed space. We're not changing the distribution's mass; we're just learning to traverse it more effectively to reach better results. Related articles: [1] https://ai-cosmos.hashnode.dev/understanding-ai-in-2025-its-still-all-about-the-next-token [2] https://ai-cosmos.hashnode.dev/the-ai-trinity-what-everyone-gets-wrong-about-modern-ai-systems [3] https://ai-cosmos.hashnode.dev/the-attention-bottleneck-ai-failure-modes-explained

Exactly. If you’re interested in exploring this particular aspect of understanding and LLMs, you might also want to look into semiotics, epistemology, psychology (especially cognitive biases, Eliza effect), and the concept of umwelten. https://ai-cosmos.hashnode.dev/unveiling-the-ai-illusion-why-chatbots-lack-true-understanding-and-intelligence

Thanks for reading and for your reply. However, you are overlooking key findings in the research that clearly indicate a lack of actual ‘reasoning’ or ‘thinking’. It’s important to recognize that your framing reflects anthropomorphism, a cognitive bias that attributes human traits to AI. While reinforcement learning introduces a different signal compared to next-token prediction during pre-training, the underlying backpropagation mechanism used in post-training remains unchanged. Adjusting weights through this process does not amount to cognitive reasoning. The transformer’s behavior during inference also remains the same. Post-training is more accurately described as ‘stochastic funneling’ or ‘manifold sculpting’, modeling existing distributions to favor the examples seen during fine-tuning, based on the data and tasks involved.

You are completely right. I don’t remember exactly what I was thinking but this makes more sense. Probably it was to keep the visual representation from the first diagram. Designing graphics can introduce artefacts as you try to add more ideas. Good feedback. Thanks.

A paper to follow up with the topic in more depth arxiv