Philip Hahn I don't know webpack, so I cannot help you with this. Anything that create-single-spa creates for me has never even run once for me. Unsure if it is a problem in my PC. It is one of the reasons I quickly moved to an all-Vite solution.
Generally speaking: Utility modules should be bundled and served via a CDN. When more than one MFE import it, it should be added to the root config so it can be shared and is not duplicated. If no UI components are being exported, then you don't need sing-spa stuff anywhere. If there are UI components, it is not a utility project and instead is probably a MFE that exports parcels.