Google TPU 8t and TPU 8i: Why Splitting Training and Inference Into Two Chips Changes Everything

For seven generations of Google’s Tensor Processing Units, the same chip handled both training large models and running them in production. That approach made sense when models were smaller and the two workloads had broadly similar compute profiles. ...