Nanochat Training Continues

After approximately 233 hours, pre-training completed. Conveniently enough, I was on a week-long trip at the time. A brief comparison to H100, here. Andrej Karpathy says pre-training should take about 4 hours on an 8xH100 setup. It’s pretty much line...