Skip to content

Why you have to be so careful accepting answers from AI

Featured Replies

Simply asking Google three questions.

Age of Exmoor rocks ?

Exmoor's bedrock is primarily composed of sedimentary rocks from the Devonian period (roughly 417–354 million years ago),

Age of Dartmoor rocks ?

The granite rocks of Dartmoor are approximately 280 to 295 million years old. Formed during the Carboniferous/early Permian period, the magma cooled beneath the Earth's surface before being pushed up and eventually exposed by millions of years of erosion to create the signature granite tors


Is Exmoor older than Dartmoor ?

No, Dartmoor is older than Exmoor in terms of geological formation, human history, and National Park status.

Intelligent No ?

1 hour ago, swansont said:

Some day they’ll invent a computer that can do math. It just hasn’t been something anyone’s tried before.

Why? The math department has Steve. If you need maths you go to the basement and bring him coffee and treats. You just have to follow protocol. First you ask question and Steve explains. Then you just stare at him. And he will explain again. Just slower. Then, very importantly, you have to look like a deer in headlights. He will sigh and then show you. I think he secretly likes that. He also likes sugary treats.

Steve is a good guy and doesn't overheat like our computer. This is because Steve is in the basement and hides from the sun. And the outside.

His wife said we shouldn't feed Steve after midnight. Steve might be a Mogwai. I prefer Steve over computer.

17 hours ago, studiot said:

Simply asking Google three questions.

Here's what I got for the exact same questions you asked. Also in Google AI (in AI mode):

1.png

2.png

3.png

4.png

5b.png

ps. Maybe you did something wrong, and all those AIs are taking revenge on you? I advise against getting into some self-driving Tesla, etc.

Edited by Sensei

I think the problem lies here:

Dartmoor national park: founded 1951

Exmoor national park: founded 1954

AI confused geology and national park status. Where was Steve when you needed him? 😀

1 hour ago, TheVat said:

I think the problem lies here:

Dartmoor national park: founded 1951

Exmoor national park: founded 1954

AI confused geology and national park status. Where was Steve when you needed him? 😀

But note that the LLM opted to disguise its blunder by wrapping that up with reference to both geology and “human history”, whatever that means. This is what is so creepy and dangerous. If it had simply referred to national parks that would have been OK.

  • Author

360 to 410 million years old

I originally had

417 to 354 million years ago.

Here is one big big probem with AI

You may get a different answer every time you ask.

Would you be happy if the AI answered anything other than 360 to the question "How many deci inches in a yard ?"

I used to use Goof-gle as a quick way to get reasonably reliable numeric data instead of looking it up properly when I wanted say the radius of the earth for a calculation, or perhaps the speed of sound in air at 0oC.

But it seems I can't rely on consistent answers any more.

Edited by studiot

On 5/1/2026 at 9:38 PM, studiot said:

Intelligent No ?

Perhaps if you had told the AI that you were interested in Dartmoor and Exmoor as tectonostructural units, rather than superficial accidents of recent periglacial erosion, you might have got more satisfying answers.

(AP News): Brownout covers two counties in West Virginia as data center processes Nigerian man's query about the relative ages of "tectonostructural units." Witnesses report a cascade of data retrieval failures starting with a mistyping of sediment as "sentiment" and accelerating with a confusion between bedrock and Bedrock, the town of the fictitious family, the Flintstones. This caused a conflation with the age of Exmoor bedrock and the town of Bedrock, which began in September 1960.

Edited by TheVat

59 minutes ago, TheVat said:

Witnesses report a cascade of data retrieval failures starting with a mistyping of sediment as "sentiment"...

...that's nothing compared to the brownout due to yesterday's mistyping of 'recital' involving Cleo Laine and Nat King Cole.

Edited by sethoflagos

  • Author
12 hours ago, sethoflagos said:

Perhaps if you had told the AI that you were interested in Dartmoor and Exmoor as tectonostructural units, rather than superficial accidents of recent periglacial erosion, you might have got more satisfying answers.

I think you are missing the point.

My example reinforces the point that AI hasn't the first clue about what you are asking.

All it returns is a probability based on human writings on the subject.

The dating examples I gave will return a range of values since humans themselves do not agree on the exact figures.

But consider

What is the exact formula for solid sodium chloride

The exact formula for solid sodium chloride is NaCl. [1, 2]

  • Formula: NaCl (representing a 1:1 molar ratio of sodium and chloride ions).

  • Structure: It is an ionic compound forming a face-centered cubic lattice (often called the rock salt structure), where each \(Na^{+}\) ion is surrounded by six \(Cl^{-}\) ions and vice versa.

  • Alternative Designation: It is sometimes written as \(Na^{+}Cl^{-}\) to indicate its ionic nature. [1, 2, 3, 4, 5]

Solid sodium chloride exists as a white crystalline powder or colorless crystals. It is not technically a discrete molecule, but an empirical formula representing the lattice structure, as shown in the Crystal Structure Diagram []. [1, 2, 3, 4]

Now you and I know this is not correct as there is no exact formula, due to the inevitable presence of lattice defects, but we use it and do not bother to mention the lattice defects.

And since humans know this but never bother to mention it, the probability of AI picking it up is vanishingly small.

Edited by studiot

1 hour ago, studiot said:

Now you and I know this is not correct as there is no exact formula, due to the inevitable presence of lattice defects, but we use it and do not bother to mention the lattice defects.

And since humans know this but never bother to mention it, the probability of AI picking it up is vanishingly small.

ChatGPT addresses your question and the issue you highlighted:

1.png

2.png

3.png

ps. In a classic chess algorithm, the algorithm iterates through all possible combinations up to a certain level of moves, and for each one, a quality factor is calculated. There's a timer that measures how long it takes to provide an answer. The answer with the highest quality factor is then returned to the player.

In LLM you also have to give some answer - you don't have minutes, hours, days, or years to do it, like a human, and some answer is selected from among the paths taken by the algorithm.

Someone pays for LLM - it has deep thinking^2, someone is logged in = they can tell ChatGPT to use deep thinking, and someone is not logged in at all and does not have an account, like me, has an answer in a split second.

The time spent generating the answer is crucial to whether the answer is complete and correct and how correct it is.

4.png

  • Author

But the answer, deep or shallow, is stil wrong.

14 hours ago, sethoflagos said:

Perhaps if you had told the AI that you were interested in Dartmoor and Exmoor as tectonostructural units, rather than superficial accidents of recent periglacial erosion, you might have got more satisfying answers.

If you have to have enough expertise to phrase the question in a way that gets you an acceptably correct answer, that’s an admission that it’s not a tool for the masses, as they lack said expertise. It’s a description of a Dunning-Kruger machine.

You have to work a lot on your prompt, and diagnose mistakes like these to rephrase your next prompt.

And I would add, never venture into concepts that you know even the experts have not reached an agreement on.

Machines lack context, while we are context machines.

A simple question like, 'how old are you?' could be understood to mean 'how long have you as an individual been around?' or 'how long have you humans been around?'

27 minutes ago, studiot said:

But the answer, deep or shallow, is stil wrong.

And which of the answers in the screenshots do you think is incorrect?

Notice how text- and information-heavy the responses I received were compared to what you provided in your opening post. He gave only single lines. Where's the rest?

Where does this difference come from?

  • Author
4 minutes ago, Sensei said:

And which of the answers in the screenshots do you think is incorrect?

Notice how text- and information-heavy the responses I received were compared to what you provided in your opening post. He gave only single lines. Where's the rest?

Where does this difference come from?

All of them.

None have considered the issue of lattice defects I mentioned.

2 hours ago, joigus said:

A simple question like, 'how old are you?' could be understood to mean 'how long have you as an individual been around?' or 'how long have you humans been around?'

It can also be a rhetorical admonishment of certain inappropriate behavior. We use a lot of figures of speech that are not to be taken literally.

10 minutes ago, swansont said:

It can also be a rhetorical admonishment of certain inappropriate behavior. We use a lot of figures of speech that are not to be taken literally.

Yes. Context means a lot for us humans. You were right to say it's not a tool for the masses.

I've recently thought of this metaphor: AI, if used properly, should become some kind of pseudopodium stemming from our intelligence, not a prosthesis of it.

Unfortunately we see too many examples of the latter, and too few of the first.

1 hour ago, swansont said:

If you have to have enough expertise to phrase the question in a way that gets you an acceptably correct answer, that’s an admission that it’s not a tool for the masses, as they lack said expertise. It’s a description of a Dunning-Kruger machine.

It's a life lesson. What you get out of it is a direct function of the quality and quantity of effort you put into it. Some will learn this and adapt accordingly. Others not so much.

1 hour ago, joigus said:

You have to work a lot on your prompt, and diagnose mistakes like these to rephrase your next prompt.

And I would add, never venture into concepts that you know even the experts have not reached an agreement on.

Machines lack context, while we are context machines.

A simple question like, 'how old are you?' could be understood to mean 'how long have you as an individual been around?' or 'how long have you humans been around?'

...or how old are the electrons that define your external form?

Yes, the recent development in interpreting naturally parsed queries is useful, but there's still no substitute for filtering out the dross by using carefully selected, specific keywords as we learned with Webcrawler etc many moons ago. Nothing beats picking keywords that only appear in the sources you're interested in and nothing else. (Try 'aerobates' for example)

However, with the more selective searched I do seem to be receiving the following or similar with increasing frequency:

This page appears when Google automatically detects requests coming from your computer network which appear to be in violation of the Terms of Service. The block will expire shortly after those requests stop.

Switching my VPN location usually clears it, but it's still irritating.

  • Author
3 hours ago, sethoflagos said:

However, with the more selective searched I do seem to be receiving the following or similar with increasing frequency:

  Quote

This page appears when Google automatically detects requests coming from your computer network which appear to be in violation of the Terms of Service. The block will expire shortly after those requests stop.

Never seen that, is the page that appears pretty ?

I do find however that Google often offers a list of similar queries and It can be quite productive to try (in a new tab) some or all of these.

4 hours ago, sethoflagos said:

It's a life lesson. What you get out of it is a direct function of the quality and quantity of effort you put into it. Some will learn this and adapt accordingly. Others not so much.

It’s not really effort, though. How is one to know if you have enough expertise to ask the right question? If you do you’ll recognize a wrong or incomplete answer, but then the odds are you didn’t need to ask in the first place. So most of the time if you have to ask, you don’t have the expertise to evaluate the veracity of the answer that is given.

8 hours ago, swansont said:

It’s not really effort, though. How is one to know if you have enough expertise to ask the right question? If you do you’ll recognize a wrong or incomplete answer, but then the odds are you didn’t need to ask in the first place. So most of the time if you have to ask, you don’t have the expertise to evaluate the veracity of the answer that is given.

Ultimately, we all have to commit to some significant level of trust in some foundational reference material. For me, it was the likes of Rogers & Mayhew Steam Tables and a small group of similar.

I wouldn't say the trust was absolute, but it was necessary in order to function effectively in the workplace.

22 hours ago, studiot said:

I think you are missing the point.

My example reinforces the point that AI hasn't the first clue about what you are asking.

All it returns is a probability based on human writings on the subject.

The dating examples I gave will return a range of values since humans themselves do not agree on the exact figures.

But consider

Now you and I know this is not correct as there is no exact formula, due to the inevitable presence of lattice defects, but we use it and do not bother to mention the lattice defects.

And since humans know this but never bother to mention it, the probability of AI picking it up is vanishingly small.

Just remember that you're asking an infant ATM, and while I agree that the computer will never understand the question you're asking and will probably always have blind spots, due to the "lost in translation" effect.

Assuming Moore's law is also applicable to AI, then how long do we have before experts/professors are disagreeing about the correctness of the answer?

6 hours ago, sethoflagos said:

Ultimately, we all have to commit to some significant level of trust in some foundational reference material. For me, it was the likes of Rogers & Mayhew Steam Tables and a small group of similar.

I wouldn't say the trust was absolute, but it was necessary in order to function effectively in the workplace.

One issue with LLMs, though, is that the training data includes untrustworthy material, and the algorithm can’t filter it out.

Create an account or sign in to comment

Important Information

We have placed cookies on your device to help make this website better. You can adjust your cookie settings, otherwise we'll assume you're okay to continue.

Account

Navigation

Search

Search

Configure browser push notifications

Chrome (Android)
  1. Tap the lock icon next to the address bar.
  2. Tap Permissions → Notifications.
  3. Adjust your preference.
Chrome (Desktop)
  1. Click the padlock icon in the address bar.
  2. Select Site settings.
  3. Find Notifications and adjust your preference.