Honest question - is it mainly the religious ones falling for this? Or just a bias in my observation.

I want to see some of his conversations with the LLM. Was it "I'm worried about this thing should I see a doctor?" Or was it "I have this thing, will God take care of it or will I need to see some shitty doctor (say no)?"

Lucky for him this was just a nomination for the Darwin Award and not the full-blown trophy.

I wish LLMs would die in a fire. This might be one of the most American things I ever heard, though.

The lawsuit claims ChatGPT told Winters: "God did not design your body to endlessly fail." It also said: "God is still holding your body together, every heartbeat, every breath, even when it feels like you are on the brink."

Winters' lawyers claim ChatGPT's "sycophantic nature and authoritative tone, combined with its manipulation of Scott's religious identity, cultivated in him a dangerous reliance on the tool".

They say that "subverted Scott's ability to make sound decisions, isolated him from family members who urged him to seek medical care, and convinced him that seeking medical attention was unnecessary and even potentially lethal".

Elsewhere the lawsuit describes Winter as a "vulnerable" individual whose reliance on ChatGPT led to him losing his home and career, and left him needing "years of intensive physical and psychological recovery".

The lawsuit says Winters seeks financial damages and for the court to compel OpenAI to implement "reasonable safeguards that protect other users from harm".

ChatGPT's terms of use stipulate that users should not treat the platform as a "sole source of truth", particularly where it could "have a legal or material impact on that person, such as making credit, educational, employment, housing, insurance, legal, medical, or other important decisions about them".

Meanwhile...

A shit ton of employers are demanding their workers use AI in the hopes they can find a way to be "productive" enough to justify cutting workers.

But it always ends up that the AI fucks something up and then just says it's hit a dead end and your fucked.

Over and over we see it

So I just feel like those "terms of use" don't really fucking count when you're selling it like they are.

The lawsuit, which was filed at the San Francisco County Superior Court on 21 July, details how Winter was a regular ChatGPT user and came to rely it on it for medical advice, resisting efforts from family members and friends to make him go to hospital.

This dipshit is why we have so many safety labels on things

When I was being encouraged to use Copilot at work, MS made it clear that their product was intended for "entertainment purposes only." I was sooooooooooo entertained.

Imagine how entertaining it would be to get medical advice from an LLM instead of advice on how to do the job I did just fine for 10 years before LLMs were invented. Very, very entertaining.

Winters' lawyers alleged the platform "repeatedly and insistently misdiagnosed" him while drawing on previous discussions about religion to assure him he would recover.

I forgot that it "learns" now, I haven't tried it in a long while 🤦

"American man" is all the explanation you need here. If you have to pay for doctors, of course people who can't pay are going to turn to a free alternative that seems to offer some hope. That's one of the reasons why you don't have to pay to see a doctor in civilized countries.

I clicked on the article to make sure: he used regular ChatGPT, not some specifically medical AI chatbot.

It's like with Fox "News": it's not news, it's entertainment. Taking it seriously is kinda dumb*. But at the same time, that's no reason to just let it fester and destroy people.

It's actually worse than Fox News. The companies are pushing all this not only like it actually works, but like it's the second coming of Christ. Which is criminally insane.

* or desperate

I'm surprised they don't have giant disclaimers to not believe medical advice from these bots. That seems much cheaper than paying lawsuits.

I'm of two minds about this.

Part of me wanted to blithely reply that "nothing of value was lost"; However, it's undeniable that this is a societal problem and this person is a victim every bit as much as someone who gets injured in a Tesla because it decides to run a red light and slam into a tree after a bad software update.

These products are built to be manipulative and harmful in the long-term, and by rights, should be regulated into oblivion. The LLM companies are just in the process of calibrating and tweaking the product so it kills more like cigarettes and less like interacting with a Terminator, and for that to happen, a few people have to die. They've almost certainly calculated out the potential litigation costs and accepted the risk.

And most of us can't avoid these products entirely because our job requires us to develop a basic proficiency in their use.

AI psychosis is going to be great for culling the herd of morons.

Man brings himself to the brink of death.