Obtaining a CLN to demonstrate relinquishment of U.S. citizenship has become a lengthy process with high barriers. The total cost of renouncing U.S. citizenship for a person in France, including the cost of preparing the related tax paperwork, has been reported to be € 10 000 – 20 000 on average.[29] Allison Christians of McGill University and Peter Spiro of Temple University have suggested that the complexity and cost of the process, especially the $2350 State Department fee and the potential penalties for failure to file related tax forms, may constitute a breach of the U.S.' obligation not to impose arbitrary barriers to change of nationality, particularly when applied to accidental Americans who have few genuine links to the United States.

https://en.wikipedia.org/wiki/Relinquishment_of_United_States_nationality

It's not simple at all and while you can just not file taxes, that's no guarantee you'll get away with it much like someone living in the US not filing taxes.

Also lottery winnings is not capital gains income it's gambling income and they typically take the taxes out before paying you your winnings. Even casinos do this if you win over $1500 at once on a machine for example.

Lastly, the premise being put forth here is that someone is using the Caribbean citizenship as a "tax haven" while still living and earning money in the US, not someone leaving the country and never looking back.

You can also just go back to the US. It's only working in the US, paying and subsequently filing taxes, that would cause the IRS to start looking into you.

Apparently if you're in such a situation you need a tax lawyer, not an accountant. Accountants are mandatory reporters to the IRS, so if you tell them you've not filed taxes they have to report you, while laywers have client confidentiality and would be able to help you try and smooth things over with the IRS.