While it's not the most unobtrusive (given your description), I would still recommend React. It handles only the rendering of components (which is very hard to do manually) and interactions (click, etc) while all the application logic may remain pure JS.
It's very reliable and have awesome error messages / warnings. It was the first JS library in which I could imagine myself writing a big application without messing everything up.
And yes, you read it right. It's a library, not a framework.