Sex can be fun, but tbh a lot of it is pretty underwhelming, especially if you don't understand each other, put in some effort or have good chemistry. When my partner and I are on holiday or have lots of free time we often have sex three or four times a day. But that's not because it's always mind-blowing, most of the time it's just a fun thing to do together.
The 'horny man' thing is a bit different. There's a kinda of arousal and drive that isn't really about the objective fun of sex, but the sense of pursuit. There's people I've slept with where the specific sextime wasn't necessarily amazing, but I had a huge crush on them and the feeling of satisfaction from finally hooking up was amazing. For me, that's usually pretty innocent "oh wow, I didn't know they liked me!" , but for some guys it's about perceived status in quite a douchey Andrew Tate kinda way.
There's also the genuine instinctual drive aspect. Times where I've not had sex in a while, and been thinking about it a lot (like visiting a long distance partner) it can make me pretty crazy. Especially if you do edging or orgasam denial, you can end up in a pretty delirious place. When that has built up for a while I definitely get to the stage where I will do super reckless things just to get release. For me, that's never a big problem because I only get in that state through choice, if I'm not planning on having sex with my partner I'll just jerk off and the urgency is gone. But the experience has given me a bit of understanding of why men sometimes do terrible and destructive things because of sex. I can literally feel my brain shutting down, and all sense of consequences disappearing.