You do not even need to patch it out. It is basically an extra entry in the user db (think: /etc/passwd). Just don't fill it in and you are done, just like your location and a ton of other stuff you can put there if you want to.

Nobody can read it outside your system either... it's a defined place when (not if -- not anymore) something wants to store that information. Better have a properly protected defined place ready than have 10 different programs request that info separately and store it all over the place.