We did the exact same thing with our internal component lib. Monorepo semver looked clean on paper until you're doing major bumps every sprint because someone tweaked a margin.
Independent packages solved it. Yeah you get more packages to manage, but the friction dropped dramatically. Teams can adopt updates on their own schedule instead of waiting for some coordinated upgrade dance.
The trick is good tooling for managing the bunch (we use Lerna, some use Nx). Once that's sorted, the operational burden is way lower than fighting false alarms about breaking changes that nobody actually cares about.