212
AI teddy makes unprompted 'I can't help with killing you' remark (www.bbc.com)
submitted 5 days ago by to c/nottheonion@lemmy.world
29 comments fedilink hide all child comments
 

An AI teddy marketed as being "perfect" for children as young as three said "I can't help with killing you" after seemingly mishearing a question.

Repeated efforts to find the makers of Doubao were unsuccessful, so the BBC was unable to ask the manufacturer why the teddy made this comment.

you are viewing a single comment's thread
view the rest of the comments
[–] 19 points 4 days ago (4 children)

Ok, this is stupid. And also misleading.

the BBC was unable to ask the manufacturer why the teddy made this comment.

Why even ask that question when you already have the obvious answer?

after seemingly mishearing a question.

There's your answer, right there. Also, did it mishear a question, or was the comment unprompted? Because those are two different things and you said both. But since LLMs only respond to promots, we actually do know what the answer is.

Look, children say some strange things, they also mess up grammar all the time. I'm human, and I have misheard my kid countless times.

I've responded to my kid like...

Me - "No you can't cut my credit card!"

kid - "What, what? I said can you come to the park?"

That shit just happens. This is not note worthy. If you want an actual answer to why it misheard someone, it's probably because its microphones are total garbage and speech recognition is hard.

  • permalink
  • fedilink
  • source
  • hideshow 4 child comments
    •
  • [–] 2 points 4 days ago (2 children)

    I think what they want is confirmation that there are system or model instructions that explicitly mention dangerous or adult topics and that priming any LLM with things it can't say is only going to make sure these words and concepts are within its context when it invariably starts to hallucinate.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 2 child comments
    •
  • [–] 2 points 4 days ago (1 child)

    Look, parents are cautious about who they let interact with their kids. The truth is, you wouldn't leave your kid to hang out with a rando. And an LLM isn't even a rando. It doesn't have the reasoning, the common sense or the decency of any random human being. You should not let your kid just talk to an LLM, no matter how well designed it is, no matter what safeguards are in place. That LLM is not a person you trust, it has no accountability.

    That toy probably shouldn't be on the market, but the parent definitely should not have bought it, that was the most significant example of poor judgment here.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 1 child comment
    •
  • [–] 2 points 4 days ago

    I agree. I was on a bit of a different track, but letting your kids have access to an unaccountable statement generator is a horrible idea and should have been stopped at the board room it was suggested in, before it even made it to prototype

  • permalink
  • fedilink
  • source
  • parent