That is why I put my down votes in a separate trash can. So that they can be recycled as up votes later on.