Well, yeah. Owning slaves was an affluent male thing to do, and most politicians are those kinda people even today. If it wasn't abolished, I'm quite certain this would be 3/4, if not much, much higher.
I mean, do you think Trump, Bush(es), Clinton, Reagan, Kennedy or Nixon wouldn't own slaves? I can only see Obama and Carter (despite a peanut farm), and I'll venture a guess that Obama probably didn't get elected president in that fucked up parallel universe since voting rights are probably pretty different even if he was allowed to run.