It is really hard to ignore, when most of the your clients are using it. If you are creating a product targeting younger generation then you need not worry about IE at all. But in enterprise world, where still a handsome part of end user using XP, it is un-avoidable. BTW I've worked on application which had to support IE8 till about a year ago, but still I'm stuck with IE9! And believe me, AngularJS is good for IE9, but angularjs plugins aren't!!!
TL;DR It really depends on the work and the target clients.