It's easiest to load everything using babel-polyfill. I'm not sure what your deployment environment looks, but typically the transpiled code will be bundled up with something like webpack or jspm. A good bundler will minify and optimize the transpiled Javascript code, so you don't really need to worry about "excess" stuff.
Detecting native browser support for various ES2015 features is possible, but not really worth it when transpilation + bundling solves this problem in a different way. Detecting and supporting native APIs would add complexity to your application (and thus cost/effort).
Of course, if you know you're only using certain ES2015 features, you can manually configure Babel to include just those polyfills. But you will most likely still bundle the code for production.
Denny Trebbin
Lead Fullstack Developer. Experimenting with bleeding-edge tech. Irregularly DJ. Hobby drone pilot. Amateur photographer.