GPT-2 From Scratch vs Fine-Tuning: A Text Generation Experiment and Its Honest Findings

There are two main ways to build a text generation model: train one from nothing, or take a model that already knows a lot and teach it something new. This post runs both, on the same small compute bu