I choose to read this as him being from a family so gloriously manly that he's the dainty one of the bunch. Even though his sisters could beat the snot out of him, they're true gentlemen who adore their brother and employ violence only in appropriate contexts.