Hard disagree. Accessible components aren't the bottleneck, shipping half-baked ones is. The keyboard behavior and AT testing failures you're describing aren't a11y doing its job wrong, they're a11y being skipped.
Real talk: if your team is spending months on ARIA while ignoring actual user testing, that's a process problem, not an a11y problem. I've seen the opposite work fine. Build accessible from day one (not months of polish), test with real users early, iterate fast. Costs way less than retrofitting later when you've got a11y debt.
Component libraries that ignore a11y end up as liability. You inherit a11y problems across your entire product.