I do agree with you in the gist of it... something I find flat out stupid is how many people define the "homewrecker" as the outsider that was willing to have sex with a married adult. (hell some of the time they weren't even aware that the person was married at the time).
Don't get me wrong, anyone who knowingly has sex with someone they are aware is in a committed relationship is still IMO 100% garbage of a human. But the actual blame for the marraige falling appart is the one who had trust to betray. I view that as the difference between stealing $20 from a stranger on the street, versus stealing $20 from a close friend or relative that absolutely trusts you.