Nope.

I'm an enby, so I can confirm that sleeping with a bare chest is better when you have tits than without them.

For the nut-havers out there, sleeping bare-tittied is kinda like having your balls shaved hairless and tucked into a perfectly-fitted silk bag before bed. It really does feel that much nicer.