Skip to content

Why you have to be so careful accepting answers from AI

Featured Replies

The ocelot, prowling in his jungle dream, more sentient

Than the heedless frogge it would seem.

As well the self-coding neural net in its virtual jungle dream heuristics,

Cortical columns, recursive self improvement, metacognition,

More sentient than the parrot LLM with its billion texts statistics,

A stochastic echo of human introspection, sans inner spark's ignition.

20 hours ago, Sensei said:

Are you unfamiliar with the concept that "winners write history"?

https://www.google.com/search?q=winners+write+history

Can't you read English? After all, it was mentioned in the conversation with ChatGPT quite clearly..

Point well missed, winners dictate history they never learn from it, ChatGPT may evolve into something that could teach us, so called winner's, how to be better humans, even you... 😉

17 hours ago, Sensei said:

One cybersecurity guy here told us how an LLM from Atrophy generated code for him that was hacked during development, and then the hacker laughed in his face and replaced the source code. He described in quite detail what the LLM did wrong and its errors.

Cool story, bruh. Aka: One anecdote does not a trend make.

17 hours ago, Sensei said:

You've probably only seen programmers in the movie The Matrix

I appreciate that we have different POVs on this, but stop acting like a giant douchebag, please. Your disrespect while authentic is unhelpful and unconducive to progress.

14 hours ago, TheVat said:

The ocelot, prowling in his jungle dream, more sentient

Than the heedless frogge it would seem.

As well the self-coding neural net in its virtual jungle dream heuristics,

Cortical columns, recursive self improvement, metacognition,

More sentient than the parrot LLM with its billion texts statistics,

A stochastic echo of human introspection, sans inner spark's ignition.

O Muse, thou’s fled, like mist o’er Tay’s cauld stream,

And left me here wi’ rage that gars me greet;

For iron brains, wi’ neither soul nor dream,

Do grind out verse mair swift than mortal feet.

O cruel age! O time o’ doom and dread!

When clatterin’ gears outsing the poet’s cry;

And I, puir wretch, by hunger near struck dead,

Must watch yon soulless scribblers multiply.

Their lines, though neat, are cauld as winter’s breath,

Nae spark o’ heart, nae trembling human pain;

Yet publishers proclaim my art is death,

And cast me oot like midden in the rain.

O William McGonagall, thy shade I loudly invoke,

For I, like thee, am scorned—yet forced tae croak!

22 minutes ago, sethoflagos said:

O William McGonagall, thy shade I loudly invoke,

Might be one for the bad poetry thread, though it's really too good for it. (Well, I loved it) But the nod to McGonagall does make it relevant to that topic.

I will resume my obsequious and confectious probings of a lump of green putty in my armpit.

7 hours ago, TheVat said:

Might be one for the bad poetry thread, though it's really too good for it. (Well, I loved it) But the nod to McGonagall does make it relevant to that topic.

Duly duplicitously duplicated on the basis that 'too good' is synonymous with 'so consistently poor it almost works'.

On 5/17/2026 at 7:18 PM, swansont said:

“We” do? The reason I brought up the lack of substantiation is that I don’t.

Does that mean, if someone disagrees with you, they can't???

4 hours ago, dimreepr said:

Does that mean, if someone disagrees with you, they can't???

No. Why would you think that?

How does a criticism of a lack of substantiation connect with agreement/disagreement?

16 hours ago, swansont said:

No. Why would you think that?

How does a criticism of a lack of substantiation connect with agreement/disagreement?

I think it was a joke, I'm clearly not as funny as I think I am. 🤔

Separate from the core topic under discussion, but likely interesting to the participants here given the subject matter:

OpenAI announced one of its general-purpose reasoning models autonomously cracked a famous geometry problem that stumped mathematicians for 80 years. The implications are enormous: An AI capable of original mathematical discovery could unlock breakthroughs across science, engineering and medicine.

https://openai.com/index/model-disproves-discrete-geometry-conjecture/

For nearly 80 years, mathematicians have studied a deceptively simple question: if you place nn points in the plane, how many pairs of points can be exactly distance 11 apart?

This is the planar unit distance problem, first posed by Paul Erdős in 1946. It is one of the best-known questions in combinatorial geometry, easy to state and remarkably difficult to resolve. The 2005 book Research Problems in Discrete Geometry, by Brass, Moser, and Pach, calls it “possibly the best known (and simplest to explain) problem in combinatorial geometry.” Noga Alon, a leading combinatorialist at Princeton, describes it as “one of Erdős’ favorite problems.” Erdős even offered a monetary prize for resolving this problem.

Today, we share a breakthrough on the unit distance problem. Since Erdős’s original work, the prevailing belief has been that the “square grid” constructions depicted further below were essentially optimal for maximizing the number of unit-distance pairs. An internal OpenAI model has disproved this longstanding conjecture, providing an infinite family of examples that yield a polynomial improvement. The proof has been checked by a group of external mathematicians. They have also written a companion paper explaining the argument and providing further background and context for the significance of the result.

The result is also notable for how it was found. The proof came from a new general-purpose reasoning model, rather than from a system trained specifically for mathematics, scaffolded to search through proof strategies, or targeted at the unit distance problem in particular. As part of a broader effort to test whether advanced models can contribute to frontier research, we evaluated it on a collection of Erdős problems. In this case, it produced a proof resolving the open problem.

This proof is an important milestone for the math and AI communities. It marks the first time that a prominent open problem, central to a subfield of mathematics, has been solved autonomously by AI. It also demonstrates the depth of reasoning these systems now support.

22 hours ago, iNow said:

Separate from the core topic under discussion, but likely interesting to the participants here given the subject matter:

OpenAI announced one of its general-purpose reasoning models autonomously cracked a famous geometry problem that stumped mathematicians for 80 years. The implications are enormous: An AI capable of original mathematical discovery could unlock breakthroughs across science, engineering and medicine.

https://openai.com/index/model-disproves-discrete-geometry-conjecture/

That is exciting, but do you think that's a step toward AGI or just another AI that can out think us in very specific way's?

With AI technology increasingly taking away the burden of thinking and researching, it sounds like a recipe for evolving away from cognitive neuroplasticity in the general population.

Edited by StringJunky

I wonder if there are any significant studies out there charting the general population's ability to perform mental arithmetic over the fifty years or so that pocket calculators have been widely available.

37 minutes ago, sethoflagos said:

I wonder if there are any significant studies out there charting the general population's ability to perform mental arithmetic over the fifty years or so that pocket calculators have been widely available.

The findings of that would be a clue.

Edited by StringJunky

On 5/21/2026 at 2:31 PM, iNow said:

Separate from the core topic under discussion, but likely interesting to the participants here given the subject matter:

OpenAI announced one of its general-purpose reasoning models autonomously cracked a famous geometry problem that stumped mathematicians for 80 years. The implications are enormous: An AI capable of original mathematical discovery could unlock breakthroughs across science, engineering and medicine.

6 hours ago, dimreepr said:

That is exciting, but do you think that's a step toward AGI or just another AI that can out think us in very specific way's?

Both in mathematics and physics it seems to be happening to some extent. It's been a while since the problem of classifying all finite, simple groups was cracked in its final steps by (non-AI) software, so I think it's been a long time coming. Machines were instrumental in classifying the 26 isolated oddities (the so-called sporadic groups), and that was back in 1983. It's perhaps worth noticing that the proof was later considered wanting and it was finally clinched by humans in 2004.

I still think that the process relies very heavily on clueing in the machine, as well as it does on humans assesing what the former is saying.

I don't think the next vital, perpendicular thinking process will come from AI, though. It's probably gonna be more like 'will you check this for me while I get some sleep?'

Let's all hope it will remain that way.

2 hours ago, sethoflagos said:

I wonder if there are any significant studies out there charting the general population's ability to perform mental arithmetic over the fifty years or so that pocket calculators have been widely available.

https://medium.com/@reuvengorsht/what-we-lose-when-we-stop-doing-hard-things-01830f7a2c41

When Texas Instruments released the first handheld scientific calculator in 1972, math teachers everywhere sounded an alarm.

They worried that students would lose the ability to perform mental arithmetic, that the cognitive muscles required for mathematical thinking would atrophy. They were right, but also wrong in ways they couldn’t have predicted.

Research from the late 1980s confirmed what teachers had suspected: students who relied heavily on calculators showed diminished mental math abilities.

A study published in the Journal for Research in Mathematics Education found that calculator-dependent students struggled more with estimation and number sense.

(Goes on to more nuances, with those losses paired with some students gaining more problem-solving abilities and mastering more complex math problems)

33 minutes ago, TheVat said:

https://medium.com/@reuvengorsht/what-we-lose-when-we-stop-doing-hard-things-01830f7a2c41

When Texas Instruments released the first handheld scientific calculator in 1972, math teachers everywhere sounded an alarm.

They worried that students would lose the ability to perform mental arithmetic, that the cognitive muscles required for mathematical thinking would atrophy. They were right, but also wrong in ways they couldn’t have predicted

Interesting link.

I remember the fuss at the time, and when I eventually caved in and bought my first calculator, I still continued to do at least rough approximations in my head in parallel, if only to guard against typing errors. Still do. Served me well one way or another.

Admittedly, the calculator was quicker and more accurate than my once treasured hardback Chambers 7-figure mathematical tables, for logarithms and trig functions. But now and then, there's some satisfaction to be gained by working out the first few terms of a Maclaurin expansion in ones head, and estimating a function value to a couple of significant digits. Just to prove you can still do it.

I wonder if slide rules and abaci had the same effect as calculators?

4 hours ago, sethoflagos said:

But now and then, there's some satisfaction to be gained by working out the first few terms of a Maclaurin expansion in ones head, and estimating a function value to a couple of significant digits. Just to prove you can still

Had an intro calc course and a dim memory of Taylor polynomials as an approximation tool, but that stuff is a use it or lose it thing. If you never use, you swiftly lose. Especially anything beyond basic algebra and trig. So me hearing "Maclaurin expansion in one's head" is a bit like my predatory cat hearing a lecture on Lotka-Volterra equations.

  • Author
11 hours ago, npts2020 said:

I wonder if slide rules and abaci had the same effect as calculators?

No not really.

People can't loose what they never had.

The abacus was 'invented' to carry out calculations that most (99.999 %) of the population could not carry out anyway.

Slide rules were in general not accurate enough for other than rough calculation, which was why Seth (and myself amongst others used Chambers Tables. there was in fact a special operation in WWII to steal German tables that were more accurate that any the British had).

There were special purpose slide rules that were more accurate but these wereexpensive and bulky, as the accuracy depends on size, so there would only be a few organisations that would have such a thing.

One other calculation aid not mentioned is the nomogram.

Like a slide rule the acuracy depends upon size, but they were once very useful but have fallen out of fashion.

Their USP was that you did not need to know the mechanics of the problem to use one.

15 hours ago, joigus said:

I still think that the process relies very heavily on clueing in the machine, as well as it does on humans assesing what the former is saying.

I don't think the next vital, perpendicular thinking process will come from AI, though. It's probably gonna be more like 'will you check this for me while I get some sleep?'

Let's all hope it will remain that way.

I'm not sure I share that hope, if we feedback human bias on a scientific basis, we could create a Nietsche type god replacement that we all have faith in.

12 hours ago, TheVat said:

a bit like my predatory cat hearing a lecture on Lotka-Volterra equations.

Cat thinks:: He's drawing mousetails again! How many does the greedy sod need from me FFS!?

3 hours ago, dimreepr said:

I'm not sure I share that hope, if we feedback human bias on a scientific basis, we could create a Nietsche type god replacement that we all have faith in.

Nietzsche didn't propose to replace God. He proposed that God is dead!

An AI should only be as good as what it eats. So we'd better feed them well. That's more Feuerbach than Nietzsche.

2 hours ago, joigus said:

Nietzsche didn't propose to replace God. He proposed that God is dead!

Thanks for reminding us that he was an idiot. To die, you 1) have to exist, and then to be 2) mortal.. So basically the opposite of (immortal) deity..

Edited by Sensei

On 5/22/2026 at 12:31 PM, sethoflagos said:

I wonder if there are any significant studies out there charting the general population's ability to perform mental arithmetic over the fifty years or so that pocket calculators have been widely available.

An apt comparison, as is our ability to read maps now with Google and Apple Maps or remember phone numbers now with phone contacts

Create an account or sign in to comment

Important Information

We have placed cookies on your device to help make this website better. You can adjust your cookie settings, otherwise we'll assume you're okay to continue.

Account

Navigation

Search

Search

Configure browser push notifications

Chrome (Android)
  1. Tap the lock icon next to the address bar.
  2. Tap Permissions → Notifications.
  3. Adjust your preference.
Chrome (Desktop)
  1. Click the padlock icon in the address bar.
  2. Select Site settings.
  3. Find Notifications and adjust your preference.