While I can understand (but fundamentally disagree with) someone who supports the death penalty for certain crimes, I can never get the problem of false convictions out of my head.

False convictions happen, and more often than the average citizen is likely comfortable with. Someone imprisoned for a false conviction can be compensated with money; someone executed for a false conviction cannot be given anything.