It took me some time to be up and running, but their docs are really written well, so that helped.
At first graphql data layer seemed like an overhead. But the elegance of getting the data that was required, helped components being independent and clean.
I've not been critical enough to look for metrics of if the markup comes out bloated like Jason Knight did in his answer ( I've started being a little scared of him 😅), but the fact that they pre cache remaining routes ( as well as data needed by them ) by following Link tags of a route, as soon as the route is loaded makes transitions fast, at least for small to medium websites. More info with metrics here
I agree with Ryosuke, if there's a first class data entity that needs to be put somewhere, a CMS solution has to be a must with gatsbyjs.
I also have had instances of not very helpful build crash errors when there was some problem in either fetching data from the source, or some other typo mistakes like Ryosuke explained.
But I really really like their plugin architecture. It enables to for example have blur image effect (progressive image loading) and have responsive images with just one plugin inclusion in gatsby config and it works like a breeze.
Gatsby has been adopted by numerous government websites throughout the world which is really nice to hear, considering how dysfunctional many of the government websites throughout the world seem to be. It's a good middle ground between a fully blown opinionated content management system and something completely static.
Gatsby comes with out of the box service worker support.
Some of the libraries / components that you might use with a client side single page app, or server rendered app is hard to pull off with Gatsby. For example one can't properly use ( as far as I could understand and tried ) react-media React component for handling media queries because, there is no way for us to tell react-media what the value of defaultMatches should be at load time. We might only keep track of it in React's state, and detect the user's device on componentDidMount, which is ought to produce jagged switch of layout either on mobile, tablet, or desktop. But I guess this is very obvious 😅. It wasn't for me 😁.
Gatsby is constantly working on lowering the build time for thousands of pages, which is encouraging to see. I've never tried building 10000 pages for a website, so I can't comment on the experience with a massive website/app like that.
I'm sold on Gatsby over other static site generators in what it enables me to do, with a little reading of the docs and trying things out.