How Sorting Your Prompts Speeds Up a Mixture-of-Experts Batch Job

Engineer's view. This is a batch job that touches records in arrival order. Each record needs a different index page, so the cache keeps reloading pages. Sort by key and each page loads once per batch