I haven't used most of these frameworks so I could be wrong with my points here. (Someone please correct me if I am!) But from what I've seen from all of these frameworks your best bet seems to be Aurelia for staying the closest to vanilla js.
Ember and Angular (1 at least, I haven't tried 2 yet!) are both fairly opinionated on how your app should be set up. I haven't used Ember, but in Angular 1 for example, you have controllers, directives, services, factories, providers, etc etc. All of these have their place and certain pieces of your logic should be put within different areas.
React is fairly un-opinionated as, by itself, it's simply a view layer that you use to help write the front end. There is no MV* pattern that you're required to use, but the js (from what I hear) tends to have a lot of boilerplate. On top of that there's also the use of jsx, or html in strings (as another option, if you prefer to stay away from jsx) to use in their render functions. This is a whole topic on its own, and people tend to choose between having html in js (react) or js in html (angular and many more). Everyone has their own preferences.
Vue is very lightweight, and admittedly, is what I've taken a liking to lately. The bulk of your logic tends to be put inside an object for computed properties, reactive data, methods, etc. Aside from the structure of defining your logic, the code feels much more vanilla to me than angular does for example.
I don't have anything to say about Polymer as I've hardly seen any code using it, so I'll leave it up to you to decide for that one. :)
As for Aurelia, I haven't used it and haven't looked too deeply into how the code is for larger projects, but their short demo video shows the classic to-do application being built, and the class running the logic looks about as close to vanilla as I can imagine for a framework. That being said, most code samples use es6 which looks very clean, while in es5 you end up using Object.definePropety quite a bit, which, imo, doesn't look very clean.
Sorry about the wall of text, and apologies in advance if I missed something or was incorrect about some of these! Hope this helps a bit!