I feel it's mostly just that us "gifted" kids happened to have brains that matured at different rates than our peers in certain areas. And often it's a result of more fundamental neurodiversity.
I'm a decently intelligent person. But I'm not as great a genius as my high school academic record might imply. When I tell people I took AP calculus my freshman year of high school and got a 5 on the exam, they think I'm a genius. My teachers in high school wondered what fields I would help transform with my brilliance.
I'm nearly 40 now. I love teaching and have a long list of degrees to my name. I can even brag about finishing my engineering bachelor's in three years. I'm someone that most would consider decently well read and intelligent. I've taught at the undergraduate and K12 levels, worked on technically complex billion dollar engineering projects, and worked in numerous technical, scientific, and educational roles. I am a decently intelligent person. But make no mistake, I am no Einstein. I've contributed basically nothing to the fundamental advancement of human knowledge, and I tried. I really did. I spent far far too long chasing a doomed PhD. Did great in the coursework, teaching, and lab work. But I just couldn't make progress on the writing and lit review due to undiagnosed AuDHD. I dropped out of a PhD program after several years. I dropped out even though I had a 3.9 GPA in doctoral level engineering coursework.
I'm decently intelligent, but honestly I just think I was an early bloomer in math and science. My final knowledge level is no different than those in my field that were not labeled "gifted." I just hit the peak earlier. At the same time, a lot of my softer skills like social skills were deeply stunted.
I wasn't a genius. My neural development was just min/maxxing to math and science academics. The adults around me saw that I was succeeding in class, getting good grades, and not disrupting things. And that's all they saw. It wasn't until years later, after I finally hit a brick wall I couldn't get through, that I actually put the time and resources into figuring out what was wrong.