• people don't like him in a few movies, including the joker
• he has at least 9 public sexual misconduct allegations against him, including costars alluding to some additional misconduct
• his fans treat him like a cult leader and he leans into it, adding to the "creepy narcissist vibe" for many people
• he is an intense method actor, going so far as to send used condoms and dead rats to production companies and costars during the filming phase

That's the jist of it, but for me there's some additional things, like seeing some of the tiktoks he did he is just missing the mark for me. To me he comes off as inauthentic in minor ways, leading him down the uncanny valley.