The Two Lives of an LLM Request: Prefill and Decode

The strange pause Okay, so lets say, you open a chat app and you paste a long document. Then, you ask, "Hey, can you summarize this?" You hit enter. Nothing happens. One second. Two seconds. Maybe thr