I think your question -- and post in general -- is based on a lot of false assumptions and flawed mentality when it comes to web development. Really this breaks down into two parts.

The first has been mentioned to a degree by others already, but can be summed up in one simple phrase:

This is the Internet, the only thing you can be certain of on your website is you have no idea who will visit your website.

Not everyone has the money for the latest and greatest, and with a lot of people still hanging on to XP based machines where now even FF and Chrome won't install their newer versions the options are quite limited. Even 9x and older versions of CE abound in the business world. I regularly support government agencies and the heathcare industry where WinCE based thing clients are a the rule.

... and I don't know if you've ever used a pre Win8 thin client, but anything they run is out of ROM so if you're LUCKY they have IE6 with no option to use anything else (since all applications are in ROM). Many systems only have IE5.x since IE6 didn't even make it onto WinCE until '08. Then Microsoft wonders why their reputation in the mobile spaces is shite.

Laugh is, I've even been places where they remotely run Win9x or WinCE in remote window under Linux when they get new machines, mixing the old and the new. Even seen it done with WinCE on ARM emulation atop a x86 Linux box. (which is like.. at that point just buy a modern ARM that can run CE... problem is old CE != new CE)

Often these places have locked-down access because their in-house crapplets rely on the Trident stack for their front-end. Visual Basic crapplets that work just fine for their intended task and that the bosses will look at you like you have eight heads when you suggest replacing their outdated garbage.

So often as a developer on the front-end you do have to support them. But that brings us to the second part : support

... and that right there depends on your definition of support. There is nothing wrong with using the new stuff so long as you have a graceful degradation plan for when it fails or is unavailable. In fact, that is one of the entire points of why HTML, CSS, and JavaScript are three separate specifications. ... and why the lumping of all three under HTML 5's banner is just more proof the WhatWG was unqualified to make HTML 4 Strict's successor. They didn't make that, what they made is 4 tranny's child.

Support does not mean you have to make it 100% perfect and identical in those older browsers, it just means you should let things gracefully degrade... and if you're building your pages/templates properly with progressive enhancement, that mechanism of 'failure' should already be in place.

Well written websites -- in the majority of cases -- should:

1. be able to stay working if CSS is unavailable or doesn't apply to the UA. Screen readers, braille readers, puffers, tty, teletype, search engines -- they don't give a flying purple fish about your style and presentation, they care about your semantics. Again, semantic markup just being a sick euphemism for "using HTML properly"

2. Remain fully usable and useful if images are blocked. Whilst sure there are some cases where this isn't viable due to the content being images, that is no reason to screw over the accessibility of the base template. A lot of people on restricted connections, metered connections, facing bandwidth caps, cold turkey cut-offs, or overage charges will block images for this reason. Not everyone lives in the inner city or a fantasy-land nordic country where everyone has 100mbps+ broadband. People like you and I might be able to afford and have access to highs speed, but that doesn't mean places like Coos County NH (just 40 miles north of where I am), the Dakota's, Utah, Nevada, Colorado, western Maine, and so forth aren't stuck with a choice between shotgunning 33.6 dial-up or shelling out a weeks pay every month for HughesNet. (where the downstream speeds are almost broadband and the upstream is 14.4 dialup) This is even more of a worry with the looming "bandwidth crunch" since nobody is building new backbone infrastructure right now; you'd almost think they hadn't paid off the existing copper much less any new fiber.

3. Still be useful if fancy new bits of CSS or HTML are missing in the UA -- I'm gonna come back to that.

4. Provide basic functionality if JavaScript is unavailable/blocked. Some users distrust JavaScript, some are in workplaces where it is actively blocked, some users disable it to save battery on their mobile devices. Browser plugins like NoScript, ScriptSafe, Ghostery, etc, etc, exist and have millions upon millions of downloads for a reason. Remember, good scripting should enhance an already working page, not supplanting what HTML (or even CSS) should already be doing for you.

As I've said many times -- queue the broken record here -- you start out with the content or a reasonable facsimile of future content and organize it in a flat text editor as if HTML doesn't exist. Then semantically markup the content -- aka saying what things are and not what you want them to look like (so no DIV or SPAN yet) using the 4 Strict structural rules. Then use CSS to bend that markup to your will creating your legacy desktop layout, applying DIV, SPAN, classes, and ID's where and only as needed. Not every blasted element should have a DIV around it or a class on it, much less multiple "classes and DIV for nothing". Then make enhance that layout with the modern CSS stuff. Then make it responsive with media queries. Then enhance it with scripting to improve the already working user experience.

It's called progressive enhancement, and is the fast track to making websites that gracefully degrade. Vary from this process at your own risk; since typically if you start out screwing around drawing pretty pictures in photoshop or slopping together JavaScript "components" all you are doing is flipping the bird at usability, accessibility, and even ease of development. Hence why IMHO all front-end frameworks are complete and utter garbage. Their class use violates separation of concerns, and they are made by people who clearly are unqualified to tell others how to use web technologies. They are not simpler or easier, they are not faster to work with or develop on, and they do not aid in collaboration; no matter how many people blindly parrot those claims out of utter ignorance of the most basic of HTML and CSS concepts!

The CSS part (said I'd get back to this) is a stunning example of the flawed mindset in action. As CSS3 stuff started being added to browsers we started seeing people attempting to use JavaScript and VML to implement "support" in legacy IE. The same failure to grasp the point of graceful degradation was done with throwing scripting at HTML 5's new pointless garbabge "Structural" tags that prove yet again the WhatWG was unqualified to write 4 strict's successor.

Thing is, if the end user on an outdated browser doesn't get some stupid tag that's nothing more than a overglorified DIV, or rounded corners on an element, or drop-shadows, does that really break the page working or being useful? Oh noes, they don't see my pretty linear gradients, notz thatsies!!!

Seriously, new tags (ARTICLE, SECTION, ASIDE, NAV, MAIN, HEADER, FOOTER) that there's no legitimate reason to even use given they're redundant to existing semantics? border-radius? text-shadow? box-shadow? Who cares?!?

Does the page work? Can the end user do what it is they came to the site to do? That should be the be-all end-all of your legacy browser support, and because new fairy-tale non-semantic made-up tags are treated by default as SPAN, and unknown CSS properties are ignored, the mechanism for such legacy support is already in place! Has been since HTML was created, and both CSS and JS followed suit. Use it!

Use it... Use It! USE IT!!!

To that end if you have some fancy new CSS go ahead and use them, but beware of and plan for what happens when it fails. For example the grid layout module, have the fallback just be a max-width single column and be done with it. Since you should already have a plan for when CSS is unavailable, that might be your best legacy support plan.

Same for your scripting, you want to use all those fancy new features you go right ahead, but since you should have a plan for what happens when JS is blocked, unavailable, or just doesn't apply to the end user's UA (since not all UA are browsers) that too might be your fallback plan.

A great way of doing this is to add those (pointless needlessly cryptic, and often mis-used to make scripts bloated and slower) arrow functions, since all versions of IE will flat out refuse to run your entire script file if it so much as see's one of them. Instantly makes the page fall back to whatever your existing non-scripted plan is.

... and if you care at in the slightest about the end user -- and in some industries need to worry about scripting only / CSR landing you with fines and court dates -- you should have a scripting off plan.

In my case, I just don't use arrow functions anyways since half what people do with them is garbage, and the other half is just painfully and pointlessly cryptic. But I have stopped playing with polyfilling much of legacy IE in my scripting, to the point that since I run all my scripts in SIF / SSCB / IIFE / pickAHuffingNameAlready I just:

if (
    (d.all && !w.atob) ||
    !d.body.classList
) return;

At the start of any scripting that won't work prior to IE9. It forces the fallback to the non-scripted behavior. Notice how that's done with capability detection, not browser sniffing.

Scripting and style should not be treated as the core of your functionality or behavior client-side. They should be an enhancement to an already working and useful HTML. To that end the best plan for providing legacy support is to just write your HTML functionality properly, and let it gracefully degrade back to the same accessible baseline as non-visual UA's, or to allow the fancy bits and doo-dads to simply not be there.

Failing to do so is basically telling the poor or people at work to sod off; just like how the same failings can tell users with accessibility needs to get bent. The only difference is that in the case of the former you aren't running the risk of having some snot-nosed prosecutor trying to build his career around dragging you into court under laws like the US' ADA and the UK's EQA.

So yes, it's important to point out "hey that won't work in IE" -- so you can have a plan for what happens when it doesn't work. Failing to do so amounts to nothing less than failing to do your job as a web developer.

I'd ask myself who the users of the app/website you're building are.

E.g. Some gov institutions rely on old and aged software and hardware. My current driving school banned us to use anything other than IE, even though we're using newer version of Windows where Chrome can be installed, just because of the gov regulations.

It all depends on the case, but I understand your frustrations, IE is not used by a majority of Internet users and tends to complicate stuff. I wouldn't worry about IE if the website/app is intended for millenials.