Privacy concern for sure, but given that you can already tie different photos back to the same phone from lens artifacts, I don't think this is going to make things much worse than they already are.

someone could create a virtual camera that would sign images

Anyone who produces cameras can publish a list of valid keys associated with their camera. If you trust the manufacturer, then you also trust their keys. If there's no trusted source for the keys, then you don't trust the signature.