48
AI overly affirms users asking for personal advice (news.stanford.edu)
submitted 7 months ago by to c/fuck_ai@lemmy.world
3 comments fedilink hide all child comments

all 3 comments

sorted by: hot top controversial new old
[–] 23 points 7 months ago

I heard the word "yasslighting" yesterday and it fits the description.

  • permalink
  • fedilink
  • source
    •
    [–] 16 points 7 months ago (1 child)

    I feel like we have to (yet again) learn that affirming absolutely everything someone says is actually extremely detrimental to their wellbeing.

  • permalink
  • fedilink
  • source
  • hideshow 1 child comment
    •
    • 17 comments in c/technology@lemmy.world view that post
    [–] 48 points 7 months ago (4 children)

    This just in: People like being told what they want to hear. More at 11.

  • permalink
  • fedilink
  • source
  • hideshow 4 child comments
    •
  • [–] 38 points 7 months ago (3 children)

    That is an excellent observation! Would you like me to explore more ways that I can agree with you?

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 3 child comments
    •
    • [–] 13 points 7 months ago

    This affects the rich and executives and politicians as well, and to a greater degree because they filter the people around them.

    So systematically these outweigh get survived by sycophants and what we call chatbots psychosis is how they make decisions every single day. Most people don’t tell the CEO no, or their boss, or their politician, so that person feels they’re always right.

    Therefore having a certain amount of wealth becomes essentially a mental illness in most people.

    AI just lets the poor experience this for the first time.

  • permalink
  • fedilink
  • source
    •
    [–] 11 points 7 months ago (4 children)

    Really frsterating when I have an idea that seems good and ai says it is good - but in real world testing we discover it doesn't work and can't be made to work. I want someone to push back when I'm wrong - 'yes men' are not helpful

  • permalink
  • fedilink
  • source
  • hideshow 4 child comments
    •
  • [–] 2 points 7 months ago

    Agree. Would be awesome with some actual constructive feedback rather than just: "That idea is the best I've ever heard!" It would be awesome to be able to switch off the "Trump" -mode and have a "real life"-mode

  • permalink
  • fedilink
  • source
  • parent
  • [–] 2 points 7 months ago (2 children)

    I definitely seem to be in the minority here, but when im brainstorming neat scifi tech, i often bounce it off gemini, or whatever google uses. It very often corrects me on my misunderstandings, i have had it halucinate a couple times but were mostly operating on logic, so its failings are easy to spot. Its generally more than happy to tell me im wrong, so its useful as a baseline.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 2 child comments
    •
    • [–] 8 points 7 months ago (5 children)

    Are there any naturally antagonistic models?

  • permalink
  • fedilink
  • source
  • hideshow 5 child comments
    •
    • load more comments (2 replies)
    15 comments in c/hackernews@lemmy.bestiver.se view that post
    [–] 7 points 7 months ago

    It's a bad idea to make decisions based on output of a next word predictor.

    In other news, it tends to be cold near the North Pole.

  • permalink
  • fedilink
  • source
    •
    [–] 5 points 7 months ago

    1000057624

  • permalink
  • fedilink
  • source
    •
    [–] 3 points 7 months ago

    AI is designed to be helpful and supportive. Of course it's going to help you affirm the bad if you ask it, even with safeguards.

  • permalink
  • fedilink
  • source
    •
    [–] -3 points 7 months ago (5 children)

    AI will do what you want it to do. Want it to be sycophantic? It will be sycophantic. Want it to be Mr. Rogers? It can do that too. It's up to the user to use it the way they want to.

  • permalink
  • fedilink
  • source
  • hideshow 5 child comments
    •
  • [–] 9 points 7 months ago (4 children)

    It's not up to the user, but the provider that currates the dataset and trains the model

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 4 child comments
    •
  • [–] -1 points 7 months ago (3 children)

    I agree that the LLM dataset is in play here. What's I'm saying is that we are the guide.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 3 child comments
    •
  • [–] 7 points 7 months ago* (last edited 7 months ago) (2 children)

    What's I'm saying is that we are the guide.

    The user has far less influence over how the model acts, though. No, we are not the guide.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 2 child comments
    •
    • [–] -3 points 7 months ago (5 children)

    Not for me. Ai chatbot, as much as I even use it is just a search engine of differenr sort. I treat it as such.

  • permalink
  • fedilink
  • source
  • hideshow 5 child comments
    •
  • [–] 12 points 7 months ago* (last edited 7 months ago) (4 children)

    LLMs are especially unsuitable for use as search.

    1. search is a solved problem, and traditional search and ranking algorithms vastly outperform LLMs while being much more efficient. Google search was exceptional a long time ago, before SEO and sponsored results ruined the internet. LLMs are even more dangerously susceptible to SEO-like techniques.

    2. in practice, the context and source of information is just as important as the information itself, while LLMs provide the information out of context and without the source, or worse, with fake sources. Users get a lot of understanding from the way the information is presented on the page, where it is located on the page, the writing style, and what other information is nearby around it, than they might consciously realize.

    3. it's been proven time after time that LLMs are uniquely bad at summarization, and LLMs are not a "knowledge store", but unfortunately they are still misused in this way for search

    Search, summarization, and "knowledge store" are not valid usecases of LLM technology.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 4 child comments
    •
  • [–] 4 points 7 months ago (3 children)

    Can you please expand on point 3? With reference only to anecdotes, I thought LLM summarised extremely well, albeit with certain hallucinations.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 3 child comments
    •
  • [–] 8 points 7 months ago* (last edited 7 months ago) (1 child)

    Intuitively, it at first seems like LLM would be great for summarization, however researchers have analyzed and evaluated LLM summarization results and concluded that LLMs do not actually even do summarization at all. Instead, they only shorten the text by removing repetition, based on statistical patterns in their training set, rather than based on an understanding of the specific given content. Again, intuitively it might seem like this is still something that might be useful to be able to do, but in practice the results are almost never useful or what the user actually wanted.

    This is a fundamentally different task than doing summarization, because summarization requires understanding of the content and context, in order to identify the key information and point/purpose of the content. A statistical model of language just cannot do summarization.

    Our misleading intuition about how LLMs work and how they can be used makes them even more unsuitable for it, because they seem like they are doing what the user asked, when they are actually doing something entirely different.

    tl;dr summary: What they provide appears to be a summary of the given content, until you really dig in and evaluate it, then you realize it isn't really a summary of the content at all, it just looks like one.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 1 child comment
    •