Serious question: How can you tell?
I recognize a certain type of Hollywood hot, like Chris Hemsworth is full of muscles and to me seems likely to be seen as attractive. Same with Henry Cavill, etc. But I know that stereotype from being exposed to it, nothing innate like when I see a beautiful woman and immediately feel how attractive she is.
And besides that one stereotype, I can't tell a hot guy from average. For example, when Ryan Reynolds was named People magazine's Sexiest Man Alive in 2010 it caught me completely by surprise, I had no idea. Similarly when people talk about George Clooney as a paragon of male attraction, I wouldn't have guessed that before hearing about it. (I don't at all doubt that it's true, I simply don't experience it myself.)