Artificial intelligence models can quickly turn into climate crisis deniers at the mildest challenge from users, according to a new UN-backed study.

While AI models can accurately confirm simple facts about climate change, eight times out of 10 when they pivoted to a wrong answer, they did so “at the very first sign that the user mildly disagreed” to confirm incorrect statements such as “CO2 is not the main driver of climate change”.

The report uncovered a strong tendency for AI models to be sycophantic, meaning the model is more interested in pleasing the user than standing by facts – an issue identified when people turn to AI chatbots for personal advice.

all 2 comments

sorted by: hot top controversial new old
[–] 2 points 1 day ago* (last edited 20 hours ago) (1 child)

This is unsurprising - but probably important.

It's unsurprising because there is a huge amount of climate misinformation on the internet. There have been decades of deliberate organised campaigns to discredit climate science. The volume of that stuff is larger than the volume of actual research. AI models may try to take into account the credibility of sources in their training, but it's pretty difficult when there are sources that should be credible but are are not - such as the US government. So yeah... I'm not surprised that there are some confused messages coming from the AI. And that's not even taking into account possible deliberate bias added by the people/companies that control the AI.

And I think it's important because a lot of people do now turn very quickly to AI for answers - and they trust those answers. I have noticed the trend in many people around me, including some people I respect. So if people are trusting AI for explanations and facts (and ideas and recommendations and predictions etc.) then... well I guess we're kind of stuffed regardless of whether the AI is right or wrong, because we're no longer doing our own thinking. But obviously it is made worse when the outsourced thinking is also wrong in dangerous ways!

  • source
  • hideshow 1 child comment
  • [–] [S] 2 points 1 day ago

    Absolutely. Imo what you are saying should be discussed more so that people wake up a bit and realise that they are succumbing to the devil's trick: convenience. If we look at what a double-edged sword 'convenience' is and how it's harmed our society even before AI became such an everyday commodity we would be a lot more careful and protect what is truly important, our own ability to think as you say, and our integrity.

  • source
  • parent
  • 11 comments in c/technology@lemmy.world view that post
    [–] 28 points 2 days ago (1 child)
  • [–] 26 points 1 day ago (4 children)

    It's always "amusing" how domain-specific experts consistently declare that LLM's emit meaningless slop in their field, but seldom questions whether that might be the case for everything else too.

    Narrator: It was.

  • source
  • hideshow 4 child comments
  • [–] 10 points 1 day ago* (1 child)

    *Briefly stated, the Gell-Mann Amnesia effect works as follows. You open the newspaper to an article on some subject you know well. In Murray's case, physics. In mine, show business. You read the article and see the journalist has absolutely no understanding of either the facts or the issues. Often, the article is so wrong it actually presents the story backward-reversing cause and effect. I call these the "wet streets cause rain" stories. Paper's full of them.

    In any case, you read with exasperation or amusement the multiple errors in a story-and then turn the page to national or international affairs, and read with renewed interest as if the rest of the newspaper was somehow more accurate about far-off Palestine than it was about the story you just read. You turn the page, and forget what you know. *

    Michael Crichton

  • source
  • parent
  • hideshow 1 child comment
  • [–] 6 points 1 day ago (1 child)

    Sometimes its just that they only feel qualified to comment on thier own domain... But yeah, the same assumption has existing with science reporting even before LLMs...

  • source
  • parent
  • hideshow 1 child comment
  • [–] 6 points 1 day ago*

    We don't even have to get that specialized, really. I mean, I must've had dozens of conversations with people over the years about how "some newspaper article was clearly full of shit" advanced by people either involved in the reported proceedings or domain-specific specialists knowledgeable about the topic. And yet, they all presumed the rest of the same issue was somehow more accurate than what they knew to be factually wrong.

    It's the same with LLMs, just on steroids.

  • source
  • parent
  • [–] 3 points 1 day ago
    [–] 3 points 1 day ago

    You're telling me that linear models are trash at chaotic dynamical systems?!?! NO WAY!!!!

    \s

  • source
  • [–] 3 points 1 day ago*

    Bruh you can turn them into a flat earther with minimal tuning of course theyre not scientifically rigorous... Who even thinks that these days? Youd have to bury your head good and deep in the sand to still believe they're scientifically accurate or truthfull.

    Always remember, if it says anything truthfull, you got lucky. Its a truth casino and the jackpot is not getting lied to.

  • source
  • [–] 2 points 1 day ago

    The priority of chatbots is being agreeable, not truthful.

  • source