Some people do use it to mean they don't agree with the thing that was said. Heck, I will down vote certain comments because they are a detriment to the conversation even if they aren't misinformation because they might be derailing or ignoring the context of the conversation.

I also down vote some comments for being unnecessarily rude even if I agree with them.

I just have a low bar for what deserves up and down votes, but there is always a reason. Heck, I just upvoted your comment to counter a down vote.