Because history.

The way the first button-down shirts were made without collars - they were an entirely separate accoutrement.

This just further proves the point - the better one understand historical antecedents, the less "nonsensical" current things are.

We are crazy spoiled with modern fabrics - just the difference in my own lifetime is staggering. Jeans when I was a kid were stiff as cardboard - you had to break them in. Wrinkle free stuff didn't really work well until at least the 90's.

The pants I wear today are amazingly comfortable and flexible compared to what there was even 40 years ago, let alone 140 years ago.