Even looking past the potential privacy issues and data breaches, at least if I drop a paper or plastic card, it won't just up and break.
Now a modern phone is basically an eggshell, with a monthly service plan no less. Drop the phone and CRACK!!!
Plus people lose their phones or they get stolen more regularly than wallets anymore..