Configuration of ReactJS app during deployment?

I'm looking for a way to modify configuration settings of a React app after its built. My CI/CD pipeline builds the app and creates an artifact from the build folder and then needs to deploy the app to multiple environments. I know I could build different artifacts for each environment and specify different .env files for each build but this introduces potential differences between builds and is a bad practice.

How can I create a configuration file that my deployment process can modify for each environment (Dev, Test, Prod). Wright now, everything in the build folder is optimized and not modifiable.

My Web App is hosted as a static website on Amazon S3 or Azure Storage so there is no Node process running that can provide environment variables.

Thanks!