Try using a static site generator like GatsbyJS , NextJS , or Vuepress . This will allow you to get the best SEO, and only use a CDN/S3.
Or try looking into server side rendering in general for React. But you'll need to run a server at that point using Node and Express, so you won't be able to exclusively run off S3.