The intelligence report, circulated across the US military this spring in the midst of the war with Iran, immediately set off alarm bells: A Chinese ship in the Middle East was transporting components of a nuclear weapons program.

The US military swung into action with plans to intercept the vessel, according to four sources familiar with the episode. According to two of the sources, armed members of the US military were preparing to board the ship. Military planes were in the air, one of those sources and another source familiar with the incident said.

It was only just before the planned operation that officials dug deeper into the report put together by a special operations command analyst and found it had been generated with the help of artificial intelligence (AI) — and that a chatbot the analyst had used inaccurately identified the material the ship was carrying. CNN was not able to learn what the misidentified cargo was.

The report, according to one of the sources, was “entirely false.” But it also “almost started a war,” the source said. Any US operation against a Chinese vessel could have risked spiraling into an armed conflict between the two nations.

In this particular instance, the analyst queried a chatbot about some intelligence reporting on the ship’s manifest that originated with US Special Operations Command Pacific, based in Hawaii. It was not clear whether the chatbot was a commercially available one or a US government product.

“The internal tools are mostly just copies of the commercial stuff wearing lipstick,” a former senior US official familiar with the AI systems used by military and intelligence analysts.

The bot fused together open-source intelligence with secret signals intelligence in government holdings and reached its fateful conclusion about the material the ship was carrying.

The analyst then used AI again to package the findings into a standard intelligence report — the kind that is trusted by military officials — and disseminated it.

For some older intelligence officials — even those who broadly support the use of AI inside the military — AI has put pressure on analysts to produce and disseminate intelligence faster, opening the door for mistakes. Young analysts in particular, several sources said, are natives on these tools and more likely to trust them uncritically.

“AI allows you to get to a bad idea faster,” one of the sources said.

1 comment

sorted by: hot top controversial new old
[–] 1 point 1 month ago

Artificial Intelligence is dangerous, and using it can lead to disastrous results, when done so recklessly. The United States of America need to be very cautious when using such powerful technology in the future.

  • source
  • 5 comments in c/technology@lemmy.world view that post
    [–] 23 points 1 month ago

    This was not AI’s fault.

    That was sheer incompetence up to the very top levels of the US army.

  • source
  • [–] 36 points 1 month ago* (last edited 1 month ago) (2 children)

    Since when does the US care about the truth before going to war?

    The 1964 Gulf of Tonkin Resolution, that got us into Vietnam, was based on false reports.

    In 2002, The 2nd Iraq War was based on fabricated intelligence reports that Saddam Hussein was allied with al Qaeda and had Weapons of Mass Destruction.

    In 2026, we bombed Iran on fabricated intelligence reports that Iran was six months (or whatever) away from producing a nuclear bomb.

    The United States military runs on lies. They're gonna love AI.

  • source
  • hideshow 2 child comments
  • [–] 10 points 1 month ago
    4 comments in c/hackernews@lemmy.bestiver.se view that post
    [–] 13 points 1 month ago

    In true MAGA lEAdeRsHiP fashion...

  • source
  • [–] 8 points 1 month ago

    Fucking stupid as fuck.

  • source
  • [–] 10 points 1 month ago (1 child)

    But the episode underscores the profound risks of using this powerful, new and relatively poorly understood technology for targeting in the middle of a war.

    Not really powerful, not that new, well understood in academia. It's just severely misused by people that choose to remain dumb about the tools at their employ. Magical thinking at its finest.

  • source
  • hideshow 1 child comment
  • load more comments (1 reply)