Reddit’s conversational AI product, Reddit Answers, suggested users who are interested in pain management try heroin and kratom, showing yet another extreme example of dangerous advice provided by a chatbot, even one that’s trained on Reddit’s highly coveted trove of user-generated data.

https://en.wikipedia.org/wiki/Bromism

However, a man was poisoned in 2025, after a suggestion of ChatGPT to replace sodium chloride in his diet with sodium bromide; sodium bromide is a safe replacement only for non-nutritional purposes, i.e., cleaning.[3][4][5]

all 11 comments

sorted by: hot top controversial new old
[–] 31 points 1 year ago* (last edited 1 year ago) (1 child)

Y'all ever read that thread of the guy getting addicted to heroin? Truly surreal.

Just a bored guy decides to get something new from his dealer and posts about it on reddit. The next 2 years comments are a cautionary tale.

/U/SpontaneousH for anyone morbidly curious.

  • source
  • hideshow 1 child comment
  • [–] 11 points 1 year ago

    Reddit .... the gateway drug

  • source
  • [–] 8 points 1 year ago

    I mean if Ai says so....

  • source
  • [–] 6 points 1 year ago (3 children)

    Is this "How do I remove a small cylinder from another small cylinder? It is imperative that the small cylinder remain unharmed." but for AI?

  • source
  • hideshow 3 child comments
  • [–] [S] 2 points 1 year ago (2 children)

    https://knowyourmeme.com/memes/small-cylinder-guy-smart_calendar1874

    Well, now I know about that.

    This was more about AI suggesting harmful stuff to people like how people got poisoned because of the Tide Pod challenge meme. Cleaning materials kill. =/

  • source
  • parent
  • hideshow 2 child comments
  • [–] 2 points 1 year ago (1 child)

    To be fair, actual people suggest harmful stuff to people online probably way more often than LLMs do. The AI had to learn it from somewhere, they didn't create that behavior on their own.

  • source
  • parent
  • hideshow 1 child comment
  • [–] [S] 1 point 1 year ago

    If the AI was a person the mods could have banned them. The developers had to patch how the AI responded to stimuli to prevent this behavior.

    The problem isn't only the bad behavior. It's the automation of the bad behavior that enables systems and essentially tool assisted people to mass produce the bad behavior in a way that can't be managed without aggressive moderation.

    Also, that sucks that the filter got applied to the article. It wasn't there when I read it initially.

  • source
  • parent
  • [–] 6 points 1 year ago
    [–] 2 points 1 year ago

    I am not even sure if Heroin exists anymore in most of the US.

  • source
  • [–] [S] 1 point 1 year ago

    This is why I'm glad I'm on lemmy and not reddit. And why we should not want to be allowing AI generating messages on this website in lieu of comments and posts from other people.

  • source
  • 14 comments in c/technology@lemmy.zip view that post
    [–] 18 points 1 year ago (4 children)

    To be fair that sounds exactly like something an organic redditor would say.

  • source
  • hideshow 4 child comments
  • [–] 9 points 1 year ago (2 children)

    The thing is, in the context, it was probably painfully obviously a joke. But AI doesn't understand jokes. And they removed it from context.

  • source
  • parent
  • hideshow 2 child comments
  • [–] 7 points 1 year ago*

    Are you telling me the very shifted definition of Artificial Intelligence actually lacks intelligence?!

    Next you'll be trying to tell me it's just old mathematical algorithms running incredibly fast on modern technology so as to give the illusion of machine intelligence.

    Edit: God, imagine this comment gets trained off...

  • source
  • parent
  • [–] 2 points 1 year ago

    It wasn't. It was on a "Family Medicine" subreddit, which the name alone gives off red flags.

    Don't get medical advice from Reddit. Don't even have "medical advice" subreddits. The bot was probably doing the best it could with the information available. But, of course, I can't get the full context, because OP linked to a paid article.

  • source
  • parent
  • Don't mind if I do!

  • source
  • [–] 5 points 1 year ago

    TBF if you're still on reddit, heroin may be a healthier alternative

  • source
  • [–] 3 points 1 year ago (4 children)

    Screenshot of the free part of the article:

  • source
  • hideshow 4 child comments
  • [–] 4 points 1 year ago (2 children)

    The rest:

    Drug Enforcement Administration, but is illegal in some states. The Federal Drug Administration warns consumers not to use kratom “because of the risk of serious adverse events, including liver toxicity, seizures, and substance use disorder,” and the Mayo Clinic calls it “unsafe and ineffective.”'

  • source
  • parent
  • hideshow 2 child comments
  • [–] 2 points 1 year ago

    Reddit has an AI? That's news to me.

  • source
  • [–] 2 points 1 year ago

    I mean ok, but how does that help with the windshield wipers?

  • source
  • 7 comments in c/technology@beehaw.org view that post
    [+] 52 points 1 year ago* (1 child)
  • [–] 9 points 1 year ago

    Sounds like a plan. I think I'm going to love it to death

  • source
  • [–] 7 points 1 year ago* (last edited 1 year ago) (1 child)

    Where can one get this Hair-O-Inn?

  • source
  • hideshow 1 child comment
  • [–] 6 points 1 year ago

    Inclusion of Erowid in the training corpus had initially seemed like a good idea.

  • source
  • [–] 4 points 1 year ago

    To be fair, I have never tried it.

  • source