Ya, people follows where the content is, also people are lazy, so if you still post on x your followers will not migrate and you will see worse numbers on the new platform, but if you fully stop posting on x your followers will follow you to the new platform (you will not hit old numbers but those are inactive followers anyways).