The way I view it is that fraud is a crime, but we also have an ethical obligation not to let ourselves be marks. Same goes for lots of other crimes. Legal penalties are one deterrent. Raising the barrier to entry for a criminal activity by being a hard target is another. Both are good. But there will always be gullible, vulnerable people, and saying that they're fair game is much too close to eugenics. Healthy societies protect the vulnerable, but also train people to be less vulnerable when that's possible.