Whichever you prefer.
I personally prefer all errors get handled immediately at the fetch, either in the same function or with a utility function. You start repeating logic at the component level (unless it's displaying the error client-side, which is usually component-level unless it's an app-wide error).
Makes debugging easier when you look inside where the action happens versus the one (or many) components using the data. Makes testing easier when you can test your fetch, and failure, in the same unit/feature test (and you avoid setting up Enzyme or other React test renderers).
These aren't hard and fast rules. Ideally you're separating any data logic from presentational logic, which would mean that display components just display errors -- not figure out what to do with them, or log or whatever. Leave the logging to utility functions and classes. In a smaller app though? You might want the logic more closely coupled, instead of spread out into 5-6 files.
If you're using something like NextJS you might prefer throwing at the component level versus the server level, or vice versa. For example, I have a NextJS app running an express server with custom dynamic routes. Some routes have a middleware that queries the API, checks if the user is logged in, and returns a response. If an error is received, I redirect back to the homepage. I used to handle this on the client-side, with a isUserLoggedIn() HOC that wrapped pages and did a similar fetch. I found that running it on a component level let the page load, display a flash of white (or remnants of hidden components), then redirect away -- versus the server solution that redirects before the page is actually rendered.
Ryosuke
Designer / Developer / Influencer
For application errors you just wrap your app in an ErrorBoundary and call it day. Have it log the errors to wherever. If you don't want your entire app taken down by deeply nested child, wrap more bug-prone components in their own ErrorBoundary.
For 404 I make a route for it. When I use frameworks like Gatsby it handles this for me and I just have to extend their template.
401s and 500s are server-side errors. You'd need to be running React off of Node with something like Express installed to print those errors in React. When I use frameworks like NextJS with small to mid size apps, I'll have a general catch-all error route ("Sorry we couldn't find that.") and log the errors for analysis/debugging later.