I've been working on Headless CMS project and it's really good. Using the WP REST API plus some extra endpoints that we expose with custom plugins. It's really good, highly recommend it.
But there are a few caveats:
The bright side is, your frontend will perform and scale well, independently of WordPress. You are no longer constrained by WP. You can your frontend it do whatever you want and Content and Presentation can be managed by different development teams. You can also feed different apps with the same content.
And there are some cool things that you could do, like share content between sites thru the WP REST API.
TL;DR: It is worth it, but comes with a cost for both the content authors (new experience, things might be missing and don't work as they used to) and software developers (re-implementing features and maintaining and deploying two codebases).
We are using VueJS with Nuxt, it's wonderful. Code is maintainable, it's easy to learn, very well supported and it has advanced features out of the box.
I have another project with React and Next. Has a steeper learning curve and you need to implement advanced features on your own. Still worth a thorough read IMHO.