You're definitely not alone in this. I have many friends from high school, college, from previous workplaces or online that I was close to and I just kinda... fell away even though I still think fondly of them. My issue is that I am terrible at small talk and it creates a barrier to initiating conversation, as well as the fear of the conversation just petering out soon after it begins and I get left with the whole "awkward encounter" feeling, which I hate.
I don't know how to resolve it, because I struggle with it too. But you're not a bad person, and I need to believe it for myself as well.