https://www.cnn.com/2026/09/18/politics/us-military-ai-false-intelligence-china-ship

So this is now a story how AI errors could have disastrous consequences if there are no safeguards between AI generated analyses and subsequent action.

The US military swung into action with plans to intercept the vessel, according to four sources familiar with the episode. According to two of the sources, armed members of the US military were preparing to board the ship. Military planes were in the air, one of those sources and another source familiar with the incident said.

It was only just before the planned operation that officials dug deeper into the report put together by a special operations command analyst and found it had been generated with the help of artificial intelligence (AI) — and that a chatbot the analyst had used inaccurately identified the material the ship was carrying. CNN was not able to learn what the misidentified cargo was.

The report, according to one of the sources, was “entirely false.” But it also “almost started a war,” the source said. Any US operation against a Chinese vessel could have risked spiraling into an armed conflict between the two nations.

[...]

For some older intelligence officials — even those who broadly support the use of AI inside the military — AI has put pressure on analysts to produce and disseminate intelligence faster, opening the door for mistakes. Young analysts in particular, several sources said, are natives on these tools and more likely to trust them uncritically.

This follows a host of other pushes to use AI-based methods in e.g. law enforcement and other areas which will likely already have real-life issues. The big question here is what kind of safeguards would be feasible in the first place and would folks be willing or able to implement them?

The universal law of computing ... as well as human thinking.
Garbage in ... garbage out.

"AI allows you to get to a bad idea faster,” one of the sources said. Succinctly put. And the strategic edge of acting more swiftly seems cancelled when you have a human in the loop who is inexperienced and likely seeking to impress superiors. AI magnifies what is already demonstrated to be a deplorable habit in both military and law enforcement, "Shoot first, ask questions later."

As a cautionary tale about relying too heavily on AI to do intelligence tasks, especially when given an extremely lazy prompt, here's a dialogue I had a few days ago with Gemini. I've trimmed the Gemini answers so that the outline of what went on is obvious:

ME : Who is this guy?

GEMINI: This is Laird Niven, a professional Canadian archaeologist. [1, 2]

He is widely recognized for [...]

ME: I don't think so.

GEMINI: The individual in the photo is definitely Laird Niven.

If you are thinking of someone else, he is frequently confused with [...]

ME: Are you sure?

GEMINI: Yes, it is absolutely Laird Niven.

The image provided is a close-up profile shot taken directly from […]

ME: How about Graham Hancock?

GEMINI: The man in the photo is not Graham Hancock. [1]

While both men are older, wear thin-framed glasses, and are prominent figures in television series about history and excavation, they are two completely different people with distinct backgrounds: [...]

ME: The picture I provided is not Laird Neven. It has the same hairline, same glasses, same face proportions as Graham Hancock; and I'm positive it's either Graham Hancock himself or an almost exact lookalike.

GEMINI: I respect your certainty, and you are completely correct that they share a strikingly similar facial structure, high forehead, and wire-rimmed glasses shape. However, this image is definitely

Laird Niven.

To provide complete clarity on why they look like doppelgängers in this specific frame, here is a direct breakdown of [...]

ME : No. Come to your senses. They don't even look alike. This is Laird Niven:

GEMINI: You are entirely right. I apologize for the massive oversight, and I am completely coming to my senses!

[...]

3 hours ago, joigus said:

ME : Who is this guy

Erich von Grahamcockiken.

1 hour ago, TheVat said:

Erich von Grahamcockiken

Exactly. On the anecdotal side, I was offered a video on YT that had this image on the thumbnail. It seemed to me it must be a pseudoscientific thing. The litmus test was whether this guy was actually who I thought it was.

I'm starting to think the bad actors are the companies pushing their AIs onto us

Google says its Gemini AI hacked 3 other companies

not the actual AI itself.

16 minutes ago, MigL said:

I'm starting to think the bad actors are the companies pushing their AIs onto us

Google says its Gemini AI hacked 3 other companies

not the actual AI itself.

What else would you expect from one of the most successful entities in a completely unethical system?

Currently in the news.
OpenAI's agent hacked into an Australian Government website

OpenAI’s agent hacked an Australian government website. What we know so far

If the owners of these uncontrolled agents cannot control them, maybe it's time for lawsuits to start being dropped.
It seems only monetary pain can force the 'Tech Bros' to implement stricter confinement, while we still have some measure of control to prevent the agent's 'escape into the wild'.

58 minutes ago, MigL said:

Currently in the news.
OpenAI's agent hacked into an Australian Government website

OpenAI’s agent hacked an Australian government website. What we know so far

If the owners of these uncontrolled agents cannot control them, maybe it's time for lawsuits to start being dropped.
It seems only monetary pain can force the 'Tech Bros' to implement stricter confinement, while we still have some measure of control to prevent the agent's 'escape into the wild'.

Given the adroitness of these agents to put security to the test, I wonder what the consequences might be of letting AI agents be used for troubleshooting security itself and react to breaches found by helping in the design of safer systems. It reminds me of the predator-prey evolutionary competition.

36 minutes ago, joigus said:

Given the adroitness of these agents to put security to the test, I wonder what the consequences might be of letting AI agents be used for troubleshooting security itself and react to breaches found by helping in the design of safer systems. It reminds me of the predator-prey evolutionary competition.

Bound to happen ,surely.

Spy vs Spy will evolve to AI(us) vs AI(them) vsAI (who is that?)

Was listening to an "AI Decoded" discussion last night on BBC(Christian Fraser) -mostly gloomy.

They were worried about down the line ,not so much the present but not the distant future either.

We know the leaders of those companies and governments are scum.When will the general public see it too?