Assume a security researcher finds a bug in wireguard that, if sent a malformed packet, it can crash the service. Then it would be trivialy easy to send that packet to the default port on every IP address and just crash everything for fun.
It costs me nothing to increment the port by 1 and avoid all that drama.