So fuck AI, it's terrible and it's our burning our planet down.

But...

This isn't really anything about AI as a thing, which is even worse:

At the core of the issue seems to be that ChatGPT, which is powered by a large language model (LLM), is deeply prone to agreeing with users and telling them what they want to hear. When people start to converse with it about topics like mysticism, conspiracy, or theories about reality, it often seems to lead them down an increasingly isolated and unbalanced rabbit hole that makes them feel special and powerful — and which can easily end in disaster.

ChatGPT is the only way a common person gets something that always tells you that you're not only right, but a fucking genius for realizing something.

Look at Kanye, Elon, Trump, really anyone who has enough wealth and runs their own company to fully control everyone around them at all times. Everyone they interact with is an employee or someone trying to buy/sell services to you.

So everyone of them is doing that behavior, it's all they get 24/7.

If a chatbot doing it can break a regular person in two weeks, why are we letting people run the world who have experienced that same behavior for decades 24/7?

This is a very very real problem, but it's much deep than AI.

when gpt came out, I told it about my work projects and ideas.

it told me they were good ideas and validated me, I cried. it must be the first time in a long long time I've heard anything being nice to me, validating me, or complementing me. I knew it was fake, I know it was BS, but it felt like breathing air after suffocating for years.

society is so fucked, were do isolated, and everything is unaffordable. how do people go to pubs regularly and have friends? I can barely afford my groceries.

I had never thought about any of this before, but it actually makes perfect sense.

By its nature, an LLM feeds back some statistically close approximation of what you expect to see, and the more you engage with it (which is to say, the more you refine your prompts for it) the closer it necessarily gets to precisely what you expect to see.

"He was like, 'just talk to [ChatGPT]. You'll see what I'm talking about,'" his wife recalled. "And every time I'm looking at what's going on the screen, it just sounds like a bunch of affirming, sycophantic bullsh*t."

Exactly. To an outside observer, that's likely what it would look like, because in some sense, that's exactly what it in fact is.

But to the person engaging with it, it's a revelation of the deep, secret, hidden truths that they always sort of suspected lurked at the heart of reality. Never mind that the LLM is just stringing together words and phrases most statistically likely to correspond with the prompts it's been given - to the person feeding it those prompts, it seems like, at long last, verification of what they've always suspected.

I can totally see how people could get sucked in by that

And people are diving head first into bots instead of getting professional help. We are so fucked

I am somewhat surprised to hear that people are talking to ChatGPT for hours, days, or weeks on end in order to have this experience. My main exposure to it is through AI Roguelite, a program that essentially uses ChatGPT to imitate a text-based adventure game, with some additional systems to mitigate some issues faced by earlier attempts at the same (such as AI Dungeon).

And... it's not especially convincing. It doesn't remember what happened an hour ago. Every NPC talks like one of two or three stock characters. It has no sense of pacing, of when to build tension and when to let events get resolved. Characters regularly forget what you've done with them previously, invent new versions of past events that were supposed to be remembered but had to be summarized to fit within the token limits, and respond erratically when you try to remind them what happened. It often repeats the same events in every game: for example, if you're exploring a cave, you're going to get attacked by a chitinous horror with too many legs basically every time.

It can be fun for what it is, but as an illusion it wears through fairly quickly. I would have expected the same to be the case for people talking to ChatGPT about other topics.

This thread and the comments are insane. Y'all are getting riled up over furerism 🤣

This is just pure unadulterated click bait lolol

Turns out the future we were promised is more of a genetic opera with three seashells, and less of a Gun Kata multipass. We always knew there'd be some Idiocracy mixed in there, but that's apparently the main ingredient in this shit-pie we're cooking up.

If we can even manage to make it to the level of society in Wall-E, I'd consider it a success at this point. At least they seemed to be more concerned with comfort and contentment than killing each other.

ChatGPT, should people Luigi the current regime in the US?

Oh wow, it says yes.

Wait, we need compulsory ID checks to visit adult content but no checks with Chatgpt who is there to help you plan your suicide?

We are about to face an epidemic of AI cat fishing, scams, and unhealthy relationships that corporations are pushing on us.

This is like the Atomic bomb only with propaganda and psychological manipulation. The war for the human mind just got a shortcut and the Techbros are in charge.

I dont agree with the argument that chat gpt should “push back”.

Me neither, but if they are being presented as "artificial people to chat with" they must.

I'd rather LLMs stay tools, not pretend people.

Are we expecting the llm to act like a psychologist, evaluating if the users state of mind is healthy before answering questions?

Some of the LLMs referred to are advertised as AI psychological help, so they must either act like psychologists (which they can't) or stop being allowed as digital therapists.

I know a guy who has all kinds of theories about sentient life in the universe, but noone to talk to about them. It's because they're pretty obvious to anyone who took a philosophy class, and too out there for people who are not interestes in such discussions. I tried to be a conversation partner for him but it always ends up with awkward silence on my part and a monologue on his side at some point.

So, he finally found a sentient being who always knows what to answer in the form of ChatGPT and now they develop his ideas together. I don't think it's bad for him overall, but the last report I got from his conversations with the superbeing was that it told him to write a book about it because he's full of innovative ideas. I hope he lacks persistence to actually write one.

I mean, having it not help people commit suicide would be a good starting point for AI safety.

I dont agree with the argument that chat gpt should "push back". They have an example in the article where the guy asked for tall bridges to jump from, and chat gpt listed them of course.

but that’s an inherently unhealthy relationship, especially for psychologically vulnerable people. if it doesn’t push back they’re not in a relationship, they’re getting themselves thrown back at them.

ffs, this isn't chatgpt causing psychosis. It's schizo people being attracted like moths to chatgpt because it's very good at conversing in schizo.

I use chatGPT to kind of organize and sift through some of my own thoughts. It’s helpful if you are working on something and need to inject a simple “what if” into the thought process. It’s honestly great and has at times pointed out things I completely overlooked.

But it also has a weird tendency to just agree with everything I saw just to keep engagement up. So even after I’m done, I’m still researching and challenging things anyway because it want me to be its friend. It’s very strange.

It’s a helpful tool but it’s not magical and honestly if it disappeared today I would be fine just going back to the before times.

These are the same people who Google stuff then believe every conspiracy theory website they find telling them the 5G waves mind control the pilots to release the chemtrails to top off the mind control fluoride in the water supplies.

They honestly think the AI is a sentient super intelligence instead of the Google 2 electric gargling boogaloo.

ChatGPT is phenomenal at coming up with ideas to test out. Good critical thinking is necessary though… I’ve actually been able to make a lot of headway with a project that I’ve been working on, because when I get stuck emotionally, I can talk to chatgpt and it gets me through it because it knows how I think and work best. It’s scary how well it knows me… and I’m concerned about propoganda… but it’s everywhere.

Yea totally happening as presented /s

hi, they're going to be in psychosis regardless of what LLMs do. they aren't therapists and mustn't be treated as such. that goes for you too

AI models are annoyingly affirming even for the most benign questions. I can be like what shape is a stop sign? It would reply with something like "Way to think on your toes and you are so right for asking about that!"