Comment by Ahmet Özel on "Neural Network From Scratch: Learning Deep Learning with MNIST & Fashion-MNIST"

Running MNIST and Fashion-MNIST side by side is a better exercise than either alone, and the reason is instructive: same shape, same size, same architecture, and the accuracy you get is very different. That gap is the whole lesson about data difficulty versus model capacity, delivered without changing a single hyperparameter. Fashion-MNIST is also where confusion matrices start earning their keep, because the errors are not uniform - shirt, coat and pullover collapse into each other while trousers are near-perfect. Looking at which classes get confused teaches more than the aggregate number does, and it is the same habit that later stops you from shipping a model that is excellent on average and useless on the one class that matters.

Ahmet, I added a confusion matrix to the notebook based on your comment — and the data confirms exactly what you predicted.

Trouser, Bag, Sandal, and Ankle boot all hit 95%+ recall (unique enough silhouettes). Shirt is the worst performer at 67.7% recall, getting confused with T-shirt/top (98 images) and Coat (103 images) most often. Pullover and Coat also swap with each other constantly — 169 Pullovers misclassified as Coat alone.

The aggregate 87% accuracy completely hid this. Really appreciate you pointing me toward looking at it this way — it's a habit I'll carry into future projects.

The updated notebook (confusion matrix in the last section) is linked from the GitHub repo mentioned in the post above.

Search Hashnode