Wow, people always talk about pointers and OOP when asked about what was the hardest thing for them to grasp. But I grew up with OOP in Delphi and pointers are just that: pointers. In Delphi, you have to explicitely announce that you want to to use a pointer, else everything is auto-deduced, so I had an easy start into pointers by explicitely using them where I wanted instead of where needed.
Yeah, I never really had problems with them. My teacher at the time was awesome and explained to me, how a simple int-sized pointer could reference an array or any other structure in memory, so it was clear and easy.
So, thinking back, the hardest part about programming was finding the motivation. It took me nearly a year to find the motivation and the concept of "always learn" (or "never settle", to quote OnePlus). After that, I went on a programming spree, and never stopped, finding interest even in difficult principles (like Rust's Composition instead of Inheritance).