The way the DOM renders your CSS is basically one big recursive function. So, yes, it does slow it down a bit by having to check for all present selectors before applying your styles.
Here's a decent article I found with a few points about keeping selector specificity down