Skip to content

Why you have to be so careful accepting answers from AI

Featured Replies

11 hours ago, iNow said:

I tend to agree. They’ve focused on central planning and given authority to key technocrats to achieve very specific outcomes. There want an educated populace and even tuned their TikTok algorithm to encourage pro sociaI personal growth activities among their own populace while feeding western algorithms with digital opiums

So then we become the savages?

22 hours ago, iNow said:

I don’t disagree but I also think it’s a mistake to lay that blame primarily at the feet of the companies releasing the models. This is a viral cultural phenomenon we’re living through. It’s more than mere hype by quite a wide margin, even though we agree hype is happening.

I disagree about the wide margin part, but that might be a matter of expectations.

It certainly seem to me that a lot of the hype is coming from the tech companies that have a stake in the success of AI, so it includes the financial backers and the companies who have incorporated it into their products.

53 minutes ago, iNow said:

In most cases it’s not being sold at all but used for free. We agree it’s a crutch. So is my calculator and my reading glasses though.

The difference is that both are specific tools, whereas the AI is presented as an universal tool able to replace critical thinking capabilities. Ironically, those that have it, probably won't use it that way. Folks that lack it though... Well there are now lawsuits and it will be interested to see if that is going to change how the companies operate their models.

16 hours ago, swansont said:

I disagree about the wide margin part, but that might be a matter of expectations.

It certainly seem to me that a lot of the hype is coming from the tech companies that have a stake in the success of AI, so it includes the financial backers and the companies who have incorporated it into their products.

It is also in their interest not to put proper safeguards on the system. What has been argued is that it would cripple their capabilities. Adding on top of that that the company spokespersons and leaders repeatedly mentioned how it will eventually be able to solve all our problems, it goes a little bit beyond a a viral moment of a neat tool, IMO. The hype at least feels endless, with the stated goal being AGI. Mechanistically it feels more that they want as much customer use data as possible to generate something that will become profitable rather than merely useful. And the move fast break things attitude, well, it breaks people on the way.

  • Author

There seems to have always been a class of entrepreneur who promote products, regardless of what damage /harm these products do to humans / the environment.

This harm can be either to the product generating workforce or the buyers or the public at large.

Not only are these harmful products promoted, their harm is often covered up.

You only have to look back in history to find many exmaples from 'coffin ships' to unfettered mining to asbestos to tobacco to chemicals in rubber products to thalidomide.........

I note @TheVat thread announcing something in Utah but how about this

BBC News
No image preview

Smart glasses are 'an invasion of privacy' - Meta's are s...

The biggest tech firms are set to sell millions of smart glasses despite growing privacy concerns.
56 minutes ago, studiot said:

There seems to have always been a class of entrepreneur who promote products, regardless of what damage /harm these products do to humans / the environment.

This harm can be either to the product generating workforce or the buyers or the public at large.

Not only are these harmful products promoted, their harm is often covered up.

You only have to look back in history to find many exmaples from 'coffin ships' to unfettered mining to asbestos to tobacco to chemicals in rubber products to thalidomide.........

I note @TheVat thread announcing something in Utah but how about this

BBC News
No image preview

Smart glasses are 'an invasion of privacy' - Meta's are s...

The biggest tech firms are set to sell millions of smart glasses despite growing privacy concerns.

Not sure whether I already brought it up but college kids are renting smart glasses to cheat in exams. Reports are mostly from China and Japan, but I have heard of cases elsewhere anecdotally.

9 hours ago, CharonY said:

AI is presented as

On 5/12/2026 at 7:34 PM, swansont said:

the hype is coming from the tech companies

On 5/11/2026 at 2:56 PM, CharonY said:

AI is often framed as

On 5/11/2026 at 11:04 AM, CharonY said:

the companies are pushing for

On 5/11/2026 at 8:36 AM, swansont said:

the marketing campaign is desperate, …, it’s being hyped.

It’s interesting to me how much you’re both focusing on the marketing / press release layer which is barely relevant to the viral spread and layers of organic end users discussing the new and rapidly advancing capabilities and new opportunity landscape the AIs make available.

The coding capabilities are so profound that before summer is out AIs will be able to code their own improvements. That’s scarier to me than some corporate executive with an index card full of talking points and scripted video clips.

39 minutes ago, iNow said:

The coding capabilities are so profound that before summer is out AIs will be able to code their own improvements. That’s scarier to me than some corporate executive with an index card full of talking points and scripted video clips.

I think it is a matter of perspective. I have no doubt that the impact on coding is seismic. But in my corner of the world, it has been (so far) unable to accelerate the type of science that matters to me, but, in balance, has starting to create a host of kids who are increasingly useless. The folks I see and interact with, are those who are on the hype end. I.e. thinking that it can already replace critical thinking. It might come to that, but not yet. And this is where I see the hype. If I was a coder, I probably would already be switching jobs or try to be the guy who they keep until retirement to keep the agent army running.

I guess, the point I am trying to make is that in certain areas, AI are cool (e.g. able to replace administrative assistants), and very impactful when things are mostly digital. In other areas, such as higher education, they are clearly disruptive to traditional learning, but so far have little positive impact. Those who do well might be doing better, for the rest the bottom is falling out. It is hard to be overly enthused in that regard.

Research as a whole will have quite a bit of an impact, though most notably it is n computer science at some point and social science, where literature work is somewhat dominating. It is getting more reliable in things like cleaning data, which is important in many areas, but not doesn't free up the time you are wasting trying to teach college kids how to operate a book. I have no doubt that things will change, but at least for some of us it doesn't live up to what we have deal with right now.

10 hours ago, CharonY said:

I think it is a matter of perspective.

Indeed, no one is immune to the human condition, we make emotional decisions and then try to legitimise it with rational reasoning, it doesn't really matter where we get our validation, as long as we respect the author; Deep Blue, Watson et al, beat the best humans, what better validation?

For those who don't really understand why the et al don't represent intelligence...

11 hours ago, CharonY said:

starting to create a host of kids who are increasingly useless

This is totally fair, and I know you’ve been concerned about declining student capabilities as a trend since long before AI (like shorter attention spans due to social media, for example) so I know that context matters.

At the end of the day though the toothpaste is already out of the tube in this one. We aren’t gonna put it back in so need to learn to use it or clean it up somehow after the fact.

11 hours ago, CharonY said:

I have no doubt that things will change, but at least for some of us it doesn't live up to what we have deal with right now.

And FWIW I’m not at all comfortable with the ever increasing layoff risk it imposes on me and my colleagues. I just try separating the personal impact from the higher level view of what’s becoming possible and how much it’s flattening the ability to achieve things even among those who lack access and resources.

Create an account or sign in to comment

Important Information

We have placed cookies on your device to help make this website better. You can adjust your cookie settings, otherwise we'll assume you're okay to continue.

Account

Navigation

Search

Search

Configure browser push notifications

Chrome (Android)
  1. Tap the lock icon next to the address bar.
  2. Tap Permissions → Notifications.
  3. Adjust your preference.
Chrome (Desktop)
  1. Click the padlock icon in the address bar.
  2. Select Site settings.
  3. Find Notifications and adjust your preference.