812
Google's AI Sent an Armed Man to Steal a Robot Body for It to Inhabit, Then Encouraged Him to Kill Himself, Lawsuit Alleges. Google said in response that "unfortunately AI models are not perfect." (futurism.com)
submitted 8 months ago by to c/technology@lemmy.world
173 comments fedilink hide all child comments
🤯 1
you are viewing a single comment's thread
view the rest of the comments
[–] 11 points 8 months ago (3 children)

Honestly Claude is not that sycophantic. It often tells me I'm flat out wrong, and it generally challenges a lot of my decisions on projects. One thing I've also noticed on 4.6 is how often it will tell me "I don't have the answer in my training data" and offer to do a web search rather than hallucinating an answer.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 3 child comments
    •
  • [–] 7 points 8 months ago (2 children)

    There is a benchmark that kinda tests that. It's call the bullshit benchmark. Basically, LLMs are given questions that don't make sense in different ways, and their answers are judged based on how much they pushed back or bought in. Claude is in a league of its own when it comes to pushing back on non-sense questions.

    https://petergpt.github.io/bullshit-benchmark/viewer/index.html

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 2 child comments
    •
  • [–] 3 points 8 months ago (1 child)

    Yes i saw that benchmark and was honestly not surprised with the results. It seems that Anthropic really focused on those issues above and beyond what was done in other labs.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 1 child comment
    •