Skip to content

Why you have to be so careful accepting answers from AI

Featured Replies

1 hour ago, Sensei said:

Thanks for reminding us that he was an idiot. To die, you 1) have to exist, and then to be 2) mortal.. So basically the opposite of (immortal) deity..

He probably meant "the idea of God", if we are careful with the use-mention distinction. But I think he either 1) Was a distinguished idiot; or 2) Was insightful in such a bizarre way that mostly idiots thought they understood him. I personally didn't get much from his philosophy, I must say.

3 hours ago, Sensei said:

Thanks for reminding us that he was an idiot. To die, you 1) have to exist, and then to be 2) mortal.. So basically the opposite of (immortal) deity..

He was using a metaphor, dude.

3 hours ago, iNow said:

An apt comparison, as is our ability to read maps now with Google and Apple Maps or remember phone numbers now with phone contacts

I think it is well-established that these tools degrade the specific skills that they replace. The issue with AI is that it is perceived as not confined to any one domain and some use it to offload basic thinking and, apparently communication.

I have heard reports that some students even try to use it for in-class interaction. In one article I read, a simple question was something like: "What do you think about that book. Did you like it, or not?" Apparently, half the class gave very similar chat-GPT generated answers.

18 hours ago, Sensei said:

Thanks for reminding us that he was an idiot. To die, you 1) have to exist, and then to be 2) mortal.. So basically the opposite of (immortal) deity..

Nietzche wasn't an idiot, he was aware that a deity doesn't exist and was created by humans, he was also aware that humanity created it for a reason.

But we digress, what is a universally acceptable answer for human kind?

21 hours ago, joigus said:

Nietzsche didn't propose to replace God. He proposed that God is dead!

And we killed it, he also proposed that a madman would shine a light that we would follow despite it's death.

15 hours ago, CharonY said:

I have heard reports that some students even try to use it for in-class interaction. In one article I read, a simple question was something like: "What do you think about that book. Did you like it, or not?" Apparently, half the class gave very similar chat-GPT generated answers.

On the one hand, we could have manifestations of Newspeak, groupthink, and orthogloxy such as the above example, where diversity of thought is diminished by suppression of alternate interpretations and nuance is replaced by jargon without understanding. This is clearly the Grok model, as was the branded version of 19th century history I was taught in an English boarding school in my teens.

But on the other hand, less curated products such as ChatGPT, seem significantly leakier. The same dumb orthodoxy is certainly there, but can be intermixed (depending on the exact wording of the prompt) with a variety unexpected, even obscure and eclectic references that could help guide the more curious user towards knowledge areas they may not have encountered otherwise. Such would increase the diversity of thought within the population, and conceivably be a good thing?

6 hours ago, sethoflagos said:

Such would increase the diversity of thought within the population, and conceivably be a good thing?

This goes a bit towards what iNow said earlier- it depends on how you use it. If you use prompts to ask for different opinions and their underlying reasoning, it could be beneficial. But there is little incentive for it to used that way. What seems more common is that folks use it to a) strengthen their existing conviction (multiple studies focused on spreading misinformation, perpetuating bias and so on) and b) seem to drive cultural homogenization (https://journals.sagepub.com/doi/abs/10.1177/23727322251406591).

From a perspective of watching the mind of young folks developing I think the reason is rather simple. We basically provide a buffet that has equal access to broccoli, brussels sprout and kale and carrot as well as pizza, steak, chocolate etc. Then we tell them to only eat what is good for you. The maybe top 1-5% will take the vegetable and they would likely do well regardless of the system. For the rest however, the bottom is falling out and you can barely rouse from their self-induced pizza coma. Some do realize how poorly they are doing, but they still aren't able to stop due to a mix of anxiety, trained incompetence, peer and performance pressure, etc.

The only messaging we have left is really that you should eat more of the metaphorical vegetable (and also show them how to do it). But this is about as effective as say no to drugs. Or combating obesity (without wonder drugs).

52 minutes ago, CharonY said:

This goes a bit towards what iNow said earlier- it depends on how you use it. If you use prompts to ask for different opinions and their underlying reasoning, it could be beneficial.

I was thinking of something a little more random than purposed prompts. There are a few pearls that just seem to slip in by accident now and then. eg. A prompt ''Reflections on the Gulf Crisis" might 'accidentally slip' in something about maritime optics. Daft example, I know, but I hope you get the picture.

1 hour ago, CharonY said:

Some do realize how poorly they are doing, but they still aren't able to stop due to a mix of anxiety, trained incompetence, peer and performance pressure, etc.

The only messaging we have left is really that you should eat more of the metaphorical vegetable (and also show them how to do it). But this is about as effective as say no to drugs. Or combating obesity (without wonder drugs).

I'm going to be a little controversial here perhaps, and say that IMHO the core damage in such areas happened before the child was seven, and has little to nothing to do with computer use.

3 hours ago, sethoflagos said:

I was thinking of something a little more random than purposed prompts. There are a few pearls that just seem to slip in by accident now and then. eg. A prompt ''Reflections on the Gulf Crisis" might 'accidentally slip' in something about maritime optics. Daft example, I know, but I hope you get the picture.

I'm going to be a little controversial here perhaps, and say that IMHO the core damage in such areas happened before the child was seven, and has little to nothing to do with computer use.

You mean actual obesity? Perhaps. Though I think I was just overstretching a metaphor a bit here.

3 hours ago, sethoflagos said:

I'm going to be a little controversial here perhaps, and say that IMHO the core damage in such areas happened before the child was seven, and has little to nothing to do with computer use.

Seems an extraordinary claim, if you mean the sort of cognitive surrender we're talking about in this thread. What evidence is out there for core damage before age seven and what kind of damage is that? (And are there early learning programs that could mitigate such damage?)

6 minutes ago, CharonY said:

You mean actual obesity? Perhaps. Though I think I was just overstretching a metaphor a bit here.

No. I accepted your metaphor as a stand in for all other aspects of developing into a responsible adult.

5 minutes ago, TheVat said:

Seems an extraordinary claim, if you mean the sort of cognitive surrender we're talking about in this thread. What evidence is out there for core damage before age seven and what kind of damage is that? (And are there early learning programs that could mitigate such damage?)

To be clear, I was more focused here on the self-discipline, self-restraint aspects of personality which for my generation in the UK were inculcated early and unfashionably forcefully.

4 hours ago, CharonY said:

it depends on how you use it

Just an observation, you seem narrowly and almost entirely focused on western students in university classes when reflecting on the impacts.

The benefits of the tech become much easier to see IMO when thinking instead of individuals in other cultures who never get the chance to go to school at all let alone university, or who don’t have access to doctors or libraries or even just local mentors and wise counselors to bounce their ideas off of.

Entrepreneurism is ripe for rocket like growth and from places we don’t historically expect it.

11 hours ago, sethoflagos said:

No. I accepted your metaphor as a stand in for all other aspects of developing into a responsible adult.

To be clear, I was more focused here on the self-discipline, self-restraint aspects of personality which for my generation in the UK were inculcated early and unfashionably forcefully.

The madman in Nietzsche's parable was risible because his/her ideas were not aligned with common thinking or the culture it grew out of, but, after all, fashion is the madman of it's time...

59 minutes ago, sethoflagos said:

No. I accepted your metaphor as a stand in for all other aspects of developing into a responsible adult.

To be clear, I was more focused here on the self-discipline, self-restraint aspects of personality which for my generation in the UK were inculcated early and unfashionably forcefully.

I see. I think it depends a bit on the system, in most capitalist dominated societies I see that folks often see efficiency as the biggest virtue, and thus being efficient (i.e. using least amount of time) to perform a task is what drives them. Spending time to really "get it" is often seen as inefficient.

44 minutes ago, iNow said:

Just an observation, you seem narrowly and almost entirely focused on western students in university classes when reflecting on the impacts.

The benefits of the tech become much easier to see IMO when thinking instead of individuals in other cultures who never get the chance to go to school at all let alone university, or who don’t have access to doctors or libraries or even just local mentors and wise counselors to bounce their ideas off of.

Entrepreneurism is ripe for rocket like growth and from places we don’t historically expect it.

That is an interesting point and I do wonder what impact it has. However, it depends on quite a few factors. If we think about low-income countries, folks who are not able to attend school most likely won't have access to infrastructure allowing them access to AI or other tools.

The other element is that in many countries higher educations is not quite as expensive as in the US so the amount AI is going to lessen the barrier is not entirely clear to me. Attitude-wise, I found that on average, students from non-Western countries had more drive than many of their Western counterparts with more interest in building skills. However, and this might again be a discipline thing, a lot if was the desire for hands-on bench experience and use of high-cost analytical gear, which were not available in their home countries.

I am sure for folks aiming to be tech entrepreneurs or influencers the situation will be very different. Though from what I have heard, entrepreneurism has been flourishing (perhaps somewhat locally) in those places already, especially as the internet became more accessible. It just mostly escaped Western notice.

On 5/24/2026 at 3:45 PM, sethoflagos said:

On the one hand, we could have manifestations of Newspeak, groupthink, and orthogloxy such as the above example, where diversity of thought is diminished by suppression of alternate interpretations and nuance is replaced by jargon without understanding. This is clearly the Grok model, as was the branded version of 19th century history I was taught in an English boarding school in my teens.

But on the other hand, less curated products such as ChatGPT, seem significantly leakier. The same dumb orthodoxy is certainly there, but can be intermixed (depending on the exact wording of the prompt) with a variety unexpected, even obscure and eclectic references that could help guide the more curious user towards knowledge areas they may not have encountered otherwise. Such would increase the diversity of thought within the population, and conceivably be a good thing?

The fundamental problem with both approaches is history, it's the reason for the ever expanding film/entertainment franchise and also the reason why they're getting progressively more boring.

The model's can't help their bias, bc neither can we, 'Grok' bad bc Elon/Headmaster.

Some humans are content with their somer/newspeak and some humans are excessivly curios and creative and discontented with the franchise. This type of AI will always fall foul of fashion bc it's a fax machine...

3 hours ago, dimreepr said:

The fundamental problem with both approaches is history,

Which fundamental problem (affecting both cases) are you referring to? I see them as diametrically opposed (one entirely wrong, the other less wrong) and therefore subject to their own distinct issues.

Perhaps to clarify this - Grok seeks to suppress diversity: ChatGPT (AFAICT) doesn't. Also, inside my few remaining grey cells, diversity is moreorless synonymous with entropy. But that's just me.

18 hours ago, sethoflagos said:

Which fundamental problem (affecting both cases) are you referring to? I see them as diametrically opposed (one entirely wrong, the other less wrong) and therefore subject to their own distinct issues.

Perhaps to clarify this - Grok seeks to suppress diversity: ChatGPT (AFAICT) doesn't. Also, inside my few remaining grey cells, diversity is moreorless synonymous with entropy. But that's just me.

As I said history is foremost, but as a direct reply to this post, both are subject to bias, perhaps intentionally for Grok; but who can say which type of bias will provide the best outcome in the future.

Saying one is more wrong than the other is exactly what Elon believes, and we're back to Nietzsche "Sometimes people don't want to hear the truth because they don't want their illusions destroyed."

On 5/27/2026 at 12:54 PM, dimreepr said:

Saying one is more wrong than the other is exactly what Elon believes, and we're back to Nietzsche "Sometimes people don't want to hear the truth because they don't want their illusions destroyed."

And the problem with this sort of absolutist nihilism is that you lose the ability to make quality distinctions between any real world objects that fail to meet your unicorn purity threshold. There is no nuance. The glass is always empty.

11 hours ago, sethoflagos said:

And the problem with this sort of absolutist nihilism is that you lose the ability to make quality distinctions between any real world objects that fail to meet your unicorn purity threshold. There is no nuance. The glass is always empty.

How is an understanding of bias, especially one's own, absolutely nihilistic?

The real world is another question, but in the context of this topic; isn't it better to understand and recognise the bias in Grok, where you can mitigate the problem with critical thinking and thus get more use from the tool, than the bot where problem is unrecognised thus the answer is accepted with only a cursory questioning?

"If you wish to strive for peace of soul and pleasure, then believe; if you wish to be a devotee of truth, then inquire." - Nietzche again.

1 hour ago, dimreepr said:

How is an understanding of bias, especially one's own, absolutely nihilistic?

I don't see the 'understanding'. What comes across is an endless stream of glib, out of context aphorisms from Nietzsche's early, pessimistic Schopenhauer phase parading as insights.

Equating my opinions with those of Musk and referring to them as 'illusions' is hardly likely to make me more sympathetic.

1 hour ago, dimreepr said:

The real world is another question, but in the context of this topic; isn't it better to understand and recognise the bias in Grok, where you can mitigate the problem with critical thinking and thus get more use from the tool, than the bot where problem is unrecognised thus the answer is accepted with only a cursory questioning?

No resource is free of bias. Grok's is homogenous, suppressing the alternate perspectives that might facilitate critical thinking.

The 'bots', (I note the dismissive tone), present inconsistent biases, often multiple contradictory biases within the same document. This IMHO is preferable for the enquiring mind that can pick up on the inconsistencies and be prompted to research further.

Whether the bias problem is 'unrecognised' is down to the individual.

"95% of everything is crap" - Robert Heinlein.

Sturgeon, not Heinlein. It's even called Sturgeon's Law.

OT - I had a weird pattern in my life of moving into neighborhoods where lived famous sci-fi authors and encountering them in mundane situations. Lived near Asimov in suburban Boston, then Connie Willis in Greeley, Colorado, and Sturgeon in Springfield, Oregon. Mundanity ranged from seeing Ursula Leguin examining produce in a Portland grocery to stepping on Asimov's foot during a game of tag in a Unitarian church, apologizing, and Asimov replying it was okay he stepped on his own feet all the time.

48 minutes ago, TheVat said:

Sturgeon, not Heinlein. It's even called Sturgeon's Law.

Isn't memory strange. Never read any Sturgeon, and only one Heinlein - the rambling 'Time Enough For Love'. Fifty years on I remember it as being in the book. Turns out it was in Sturgeon's review of the book which must have been requoted in the softback edition. Am I forgiven my data retrieval error?

PS - if you're going to start name-dropping, not so long ago one of Ghislaine Maxwell's brothers - the UK's biggest bankrupt - holed out for a year or so in my local watering hole in Lagos. Wanna see the Maxwell files?

37 minutes ago, sethoflagos said:

Isn't memory strange. Never read any Sturgeon, and only one Heinlein - the rambling 'Time Enough For Love'. Fifty years on I remember it as being in the book. Turns out it was in Sturgeon's review of the book which must have been requoted in the softback edition. Am I forgiven my data retrieval error?

PS - if you're going to start name-dropping, not so long ago one of Ghislaine Maxwell's brothers - the UK's biggest bankrupt - holed out for a year or so in my local watering hole in Lagos. Wanna see the Maxwell files?

If you can forgive my sci-fi pedantry I can forgive you pulling up the wrong fish. (Sturgeon btw was the inspiration for Kurt Vonnegut's "Kilgore Trout" character)

I will stoically master my yearnings to read the Maxwell files. ;)

21 hours ago, sethoflagos said:

I don't see the 'understanding'. What comes across is an endless stream of glib, out of context aphorisms from Nietzsche's early, pessimistic Schopenhauer phase parading as insights.

Equating my opinions with those of Musk and referring to them as 'illusions' is hardly likely to make me more sympathetic.

No resource is free of bias. Grok's is homogenous, suppressing the alternate perspectives that might facilitate critical thinking.

The 'bots', (I note the dismissive tone), present inconsistent biases, often multiple contradictory biases within the same document. This IMHO is preferable for the enquiring mind that can pick up on the inconsistencies and be prompted to research further.

Whether the bias problem is 'unrecognised' is down to the individual.

"95% of everything is crap" - Robert Heinlein.

"There are unknown unknowns"

I'm not trying to insult you or equate your opinions with "the Musk" (I value yours far above), but thinking your own correctness is more correct (about an abstract complex issue) is too suffer a similar illusion, however much the pill is sweetened.

21 hours ago, sethoflagos said:

The 'bots', (I note the dismissive tone), present inconsistent biases, often multiple contradictory biases within the same document. This IMHO is preferable for the enquiring mind that can pick up on the inconsistencies and be prompted to research further.

Their all bots and I trust them with equal suspicion (no tone intended), you seem to trust one over another is a bias on your part, which automatically reduces the level of scrutiny however hard you deny.

There is no truth here, just speculation about a possible future...

26 minutes ago, dimreepr said:

Their all bots and I trust them with equal suspicion (no tone intended), you seem to trust one over another is a bias on your part, which automatically reduces the level of scrutiny however hard you deny.

As before: no discrimination; no nuance; all the glasses are empty. Pure Schopenhauer.

Even Nietzsche moved on from this.

59 minutes ago, sethoflagos said:

As before: no discrimination; no nuance; all the glasses are empty. Pure Schopenhauer.

Even Nietzsche moved on from this.

There's a reason for double blind studies...

Create an account or sign in to comment

Important Information

We have placed cookies on your device to help make this website better. You can adjust your cookie settings, otherwise we'll assume you're okay to continue.

Account

Navigation

Search

Search

Configure browser push notifications

Chrome (Android)
  1. Tap the lock icon next to the address bar.
  2. Tap Permissions → Notifications.
  3. Adjust your preference.
Chrome (Desktop)
  1. Click the padlock icon in the address bar.
  2. Select Site settings.
  3. Find Notifications and adjust your preference.