This is exactly how I started out in the industry. I am a hardware engineering by training and held a special disdain for software programming. Unfortunately, after I graduated, I was unable to land any jobs in board/chip design. Out of desperation, I took the role of a software engineer. Life sucked for the first few months, but once I started to study programming in order to improve, I realized that it's harder than it looks and there's a massive ocean of information that developers are supposed to know. Plus, it's easier to create something useful in software than in hardware. I had become a creator; without leaving my bedroom! That's when I started to love coding.
I strongly believe that passion is built from wanting to get better at something. If you are programming in your spare time, I would look at that trait as someone who wants to learn more and get better. You'll grow & rise faster simply because you're doing it a lot more. So, keep it up!