Websites? Go for it. Javascript is very mature right now for simple tasks, you don't even need jquery for that anymore.
Applications? I wouldn't. DOM manipulation will bite you in the ass at no time. Even if you manage to pull it off, you'll probably end up skipping usability improvements because they will be too hard for the value they would aggregate.
I'm saying this because I currently have to deal with projects that are somewhat close to Vanilla JS. The developers carefully tried to organize things and make for a reasonable project. They are still a pain to maintain, and it's not the developers fault at all.