If I keep them at the back of the fridge they'll freeze onto the back wall so that sounds worse.

I only move eggs into the door if I can be bothered or run out of space to keep the box on a shelf and have never noticed a difference - I wonder if you've noticed this yourself or are just repeating advice? I don't really see how changes in temperature are likely to affect anything.