What it takes to run a 0.6B language model inside a browser tab: four trade-offs

A chat model with about half a billion parameters can run on an ordinary laptop's CPU, inside a browser tab, with no server and no messages leaving the device. Getting there is mostly a series of trad