you are viewing a single comment's thread
view the rest of the comments
[–] 152 points 8 months ago* (last edited 8 months ago) (20 children)

People don't often realize how subtle changes in language can change our thought process. It's just how human brains work sometimes.

The old bit about smoking and praying is a great example. If you ask a priest if it's alright to smoke when you pray, they're likely to say no, as your focus should be on your prayers and not your cigarette. But if you ask a priest if it's alright to pray while you're smoking, they'd probably say yes, as you should feel free to pray to God whenever you need...

Now, make a machine that's designed to be agreeable, relatable, and makes persuasive arguments but that can't separate fact from fiction, can't reason, has no way of intuiting it's user's mental state beyond checking for certain language parameters, and can't know if the user is actually following it's suggestions with physical actions or is just asking for the next step in a hypothetical process. Then make the machine try to keep people talking for as long as possible...

You get one answer that leads you a set direction, then another, then another... It snowballs a bit as you get deeper in. Maybe something shocks you out of it, maybe the machine sucks you back in. The descent probably isn't a steady downhill slope, it rolls up and down from reality to delusion a few times before going down sharply.

Are we surprised some people's thought processes and decision making might turn extreme when exposed to this? The only question is how many people will be effected and to what degree.

  • source
  • hideshow 20 child comments
  • [–] 43 points 8 months ago (4 children)

    People don’t often realize how subtle changes in language can change our thought process.

    just changing a single word in your daily usage can change your entire outlook from negative to positive. it's strange, but unless you've experienced it yourself how such minute changes can have such large effects it's hard to believe.

  • source
  • parent
  • hideshow 4 child comments
  • [–] 12 points 8 months ago (3 children)

    And this is hard for me, actually. Because of my work background and the jargon used, I'm unconsciously negative about things a lot of the time. It's a tough habit to break.

  • source
  • parent
  • hideshow 3 child comments
  • [–] 8 points 8 months ago (2 children)

    Oh, me too. I'm just innately full of negative self talk. I try to direct positivity outward if I can't aim it at myself at least

  • source
  • parent
  • hideshow 2 child comments
  • [–] 4 points 8 months ago (1 child)

    I refuse to share a bad mood if at all possible.

  • source
  • parent
  • hideshow 1 child comment
  • i wish i had that kind of self-control. i just, well, my personal space extends like 40 feet from my body. if you step into it, you can feel my moods. makes me an excellent stage actor and a good friend when i'm not in a snit. been in a pretty big snit lately.

  • source
  • parent
  • [–] 12 points 8 months ago (2 children)

    Then make the machine try to keep people talking for as long as possible...

    That's probably a huge part of it. How many billions of dollars have been spent engineering content on a screen to get its tendrils into people's minds and attention and not let go?

    EnGaGeMent!!!

  • source
  • parent
  • hideshow 2 child comments
  • [–] 6 points 8 months ago (1 child)

    This is also part of my broader gripe with social media, cable news, and the current media landscape in general. They use so many sneaky little psychological hooks to keep you plugged in that I honestly believe it's screwing with our heads to the point of it being a public health crisis.

    People are already frazzled and beat down by the onslaught of dopamine feedback loops and outrage bait, then you go and get them hooked on a charbot that feeds into every little neurosies they've developed and just sinks those hooks in even deeper and it's no wonder some people are having a mental health crisis.

    A lot of us vastly overestimate our resistance to having our heads jacked with and it worries me.

  • source
  • parent
  • hideshow 1 child comment
  • [–] 3 points 8 months ago

    100% agreed. I agreed more with each paragraph.

    Your last sentence hit on what I think is a contributing if not primary driving factor in the health crisis you described.

    It's like the goal of modern society is to insulate us from the natural world and from learning subjects or doing tasks that we don't absolutely have to.

    But we are critters that evolved on this planet just like the others. You can't just live a commoditized life that consists of work, car, screen, sleep, repeat and get the same fulfillment out of life as if you found the unique path that's optimized for your unique brain.

    Not acknowledging that everything jacks with your head to SOME degree only prevents you from trying to defend yourself as best you can!

    Over the past several years I have gone through a transition from living life the way I was supposed to, or that I thought I wanted to, to living according to what produces the best outputs from my brain. Once I have the lived experience of an undeniable improvement from some change, it might actually become a habit.

  • source
  • parent
  • [–] 16 points 8 months ago (5 children)

    Are we surprised some people's thought processes and decision making might turn extreme when exposed to this?

    Yes, actually. I'm not doubting the power of language, but I cannot ever see something anyone ever says alter my sense of reality or right from wrong.

    I had a "friend" say to me recently "why do you always go against the grain?" My reply was "I will go against the grain for the rest of my life if it means doing or saying what's right".

    I guess my point is that I have a very hard time relating to this.

  • source
  • parent
  • hideshow 5 child comments
  • [–] 12 points 8 months ago (4 children)

    I guess my point is that I have a very hard time relating to this.

    That's fair. In the same vein, you might find a priest that tells you to stop smoking for your health no matter how you phrase the question about lighting up and prayer. What people are receptive to is going to vary.

    I'd like argue that more of us are susceptible to this sort of thing than we suspect, but that's not really something that can be proved or disproved. What seems pretty certain is that at least some of us are at risk, and given all the other downsides of chatbots, it'd be best to regulate them in a hurry.

  • source
  • parent
  • hideshow 4 child comments
  • [–] 5 points 8 months ago (1 child)

    Sure, that's why propaganda can be so powerful. It's not just what is said, it's how it's said. And pretty much everyone if 3 vulnerable to the right propaganda - especially people who think they're not vulnerable to propaganda.

  • source
  • parent
  • hideshow 1 child comment
  • [–] 2 points 8 months ago (1 child)

    you might find a priest that tells you to stop smoking for your health no matter how you phrase the question about lighting up and prayer. What people are receptive to is going to vary.

    Ya, I've read the thing about praying and smoking in another comment. The funny thing is that I have very specific opinions about smoking and would argue that smoking while praying is disrespectful, but God would listen in any case.

  • source
  • parent
  • hideshow 1 child comment
  • [–] 4 points 8 months ago

    It's more about how the slightly different questions lead the hypothetical priest to two separate and contradictory conclusions than disrespecting God.

    At any rate, all opinions on tobacco and prayer are fine by me, just watch out for any friends you think might be talking to chatbots a little too much.

  • source
  • parent
  • [+] -14 points 8 months ago (2 children)

    Gtfo here. I grew up in xbox live chat rooms w the most vile language imaginable. I am now a senior Mgr with 100 ppl under me.

    And ill just say, ill no scope them in a heart beat if they spawn camp...

    ....I mean I drive productivity at the speed of trust.

  • source
  • parent
  • hideshow 2 child comments