There's no doubt that it's more wear and tear on your clothes.

Like if I put on a pair of shorts I'm probably going to sit around in them for most of the day and then take them off before bed. Several hours in the drier where it's hot and they're rubbing around on everything is surely going to be the equivalent of many days worth of "wearing them".