The problem you are facing is that you are probably quite proficient using the frameworks, however you lack the Vanilla skill. It's a similar problem in JS, where everyone uses Angular, Aurelia, React, etc. and no one wants to build stuff without those.
The remedy I see is to start thinking "is it really necessary to use a framework/library here? Can't I do it without one just as easy?" Try to build stuff without frameworks, just You and Java.
I wrote a JS-focused Hashnode Original article about your problem. If you feel like it, give it a read :)