It doesn't sound like your web site fits the bill for using service workers. Barring a genuine need for push notifications, you should probably focus on the well-trodden methods of static web site optimisation and performance. The huge effort to build a client-side proxy for caching resources (and maintaining it as the site is updated) is hardly worth the very little gain.
The standard techniques like minifying and consolidating resources, reducing the number of unncessary scripts on the page, async scripts, resource sizes, performance validation on mobile, etc are going to deliver much better usability. They aren't interesting or cool, but will give a lot more value to your users and customer.