Teen killed himself after ‘months of encouragement from ChatGPT’, lawsuit claims
(www.theguardian.com)
These AIs will need to always have a suicide hotline disclaimer in each response regardless of what is being done like world building.
ChatGPT gave multiple warnings to this teen, which he ignored. Warnings do very little to protect users, unless they are completely naive (i.e. hot coffee is hot), and warnings really only exist to guard against legal liability.