Compute-Optimal Training of Language Models: Balancing Model Size and Data Generation Costs

The rapid advancement of large language models (LLMs) has ushered in a new era of artificial intelligence, enabling breakthroughs in natural language processing, code generation, and reasoning tasks. However, the increasing size and complexity of the...