Do you trust AI?

Use this board for general non-cycling-related chat, or to introduce yourself to the forum.
Nearholmer
Posts: 7593
Joined: 26 Mar 2022, 7:13am

Re: Do you trust AI?

Post by Nearholmer »

Not that long ago the vast majority of human learning was via practice, not theorising
I really, really don’t think so.

Tuition in one form or another, so transferring knowledge from one brain to another, is probably a defining or differentiating characteristic of us upright apes. Many, many creatures teach their offspring to some extent, but nothing like to the extent that we teach.

Much of the “initial knowledge capital” was certainly gained by trial and error, but it wasn’t down to each successive individual to discover everything over again, and a rich oral tradition is as much a library as is a pile of books, maybe even more so, because it gets foisted upon the individual. TBH Im not certain whether the “soma” in question is the individual, of the entire collection of individuals, but if it’s the former then the collected wisdom in the heads of the tribe members is “extra-somatic”.

As for theorising, humans seemto have been at that game from very early-on too, postulating by extension from proven knowledge and essaying, making the odd inductive leap etc.

You are right that technologies have long emerged in advance of any deep theoretical understanding of them, but it would also be true to say that the two have often progressed sort-of hand-in-hand, and although theory preceding practise by a long distance has become much, much more common in recent centuries it wasn’t completely absent before that.

As for the overall “ignorance was bliss” drift of what you say, I don’t buy it. People managed to exist in large numbers in miserable and degraded condition before the “knowledge revolution” (invented term) as much as they have since, they just existed in smaller numbers because of lack of the knowledge needed to sustain greater numbers.

To be sure, our ability to obliterate ourselves either in a bang or a series of whimpers, is greater than it’s ever been, but so is our ability to sustain ourselves. Everything to play for in the second half, as they say in football commentaries.
sjs
Posts: 1498
Joined: 24 Jan 2010, 10:08pm
Location: Hitchin

Re: Do you trust AI?

Post by sjs »

In the recent past I have googled two clear questions with well-defined numerical answers. In both cases the AI version of the answer began with a correct statement of the problem, correct values for the inputs, and wildly incorrect (many orders of magnitude) values for the answer. That struck me as strange.
tim-b
Posts: 2556
Joined: 10 Oct 2009, 8:20am

Re: Do you trust AI?

Post by tim-b »

Jdsk wrote: 26 Aug 2025, 8:36am
tim-b wrote: 26 Aug 2025, 7:53am AI doesn't reason. The language models aggregate internet hits on similar problems. Many of them will be wrong and so AI is wrong
This needs distinction between the various approaches used in AI work. Logic programming systems use explicit reasoning. Systems using production rules are often able to explain their outputs.Some pattern recognition systems can. Current LLMs generally can't.

There's a lot of work going on in AI besides LLMs, and besides the LLMs that are now so widely available to the public.

Jonathan
I'd maintain that AI models don't reason.
They use a series of reinforced learning steps to reach a conclusion, but that's not the same as human reasoning.
They can use different methods and they can show their working, which is essential to verify their "reasoning", especially in critical areas such as law where they have invented "facts".
Inventing fact is something that AI excels at

AI cuts through huge volume, humans provide finer context and judgement
"But when they've notched up 12 U-turns and rising, the only conclusion is serial incompetence." Keir Starmer on Twitter 2/9/2020
Nearholmer
Posts: 7593
Joined: 26 Mar 2022, 7:13am

Re: Do you trust AI?

Post by Nearholmer »

They use a series of reinforced learning steps to reach a conclusion, but that's not the same as human reasoning.
Isn't that what we do too?

If not, how does human reasoning work?
tim-b
Posts: 2556
Joined: 10 Oct 2009, 8:20am

Re: Do you trust AI?

Post by tim-b »

Nearholmer wrote: 26 Aug 2025, 3:20pm
They use a series of reinforced learning steps to reach a conclusion, but that's not the same as human reasoning.
Isn't that what we do too?

If not, how does human reasoning work?
We have the ability to use context and intuition when the learning runs out and test that reasoning through discussion.

AI has to be taught "reasoning". Similarly, it struggles with sarcasm
Last edited by tim-b on 26 Aug 2025, 5:02pm, edited 1 time in total.
"But when they've notched up 12 U-turns and rising, the only conclusion is serial incompetence." Keir Starmer on Twitter 2/9/2020
Jdsk
Posts: 29772
Joined: 5 Mar 2019, 5:42pm

Re: Do you trust AI?

Post by Jdsk »

sjs wrote: 26 Aug 2025, 1:21pm In the recent past I have googled two clear questions with well-defined numerical answers. In both cases the AI version of the answer began with a correct statement of the problem, correct values for the inputs, and wildly incorrect (many orders of magnitude) values for the answer. That struck me as strange.
Which AI system was that, please?

Jonathan
Nearholmer
Posts: 7593
Joined: 26 Mar 2022, 7:13am

Re: Do you trust AI?

Post by Nearholmer »

We have the ability to use context and intuition when the learning runs out and test that reasoning through discussion.
Sounds to me rather like a difference of scale, rather than a difference of kind.

Context is simply data, often about lots of things which at first inspection don’t seem connected to the subject in hand, and we are good at hoovering all that data up, storing a lot, accessing it, and spotting the links. I can’t see why an AI couldn’t do that.

Intuition is very similar, it’s not some mysterious process, it’s (often subconscious and incredibly fast) use of stored data as a comparator for what we are sensing, allowing us to suggest probable answers based on very little live sensor data. Again I can’t see why an AI couldn’t do that.

Testing through reason and discussion again I can’t see is beyond the possible with AI. It would be sort of one-up from what one finds in safety critical systems, where softwares developed by completely independent teams, using different approaches, are asked to vote to arrive at a decision.

My instinct sbout AI is that one should always add “this week” or “yet” when making a statement about what it cannot do.
Jdsk
Posts: 29772
Joined: 5 Mar 2019, 5:42pm

Re: Do you trust AI?

Post by Jdsk »

Nearholmer wrote: 26 Aug 2025, 5:32pm ...
Testing through reason and discussion again I can’t see is beyond the possible with AI. It would be sort of one-up from what one finds in safety critical systems, where softwares developed by completely independent teams, using different approaches, are asked to vote to arrive at a decision.
...
In the early days of medical expert systems it was identified that critiquing was a useful approach: the human goes first then the computer joins in and responds. This can be much more acceptable than the computer going first. And of course can support an educational angle. (Those systems were based on production rules, sometimes with statistical inference, not on LLMs.)

Jonathan
axel_knutt
Posts: 4281
Joined: 11 Jan 2007, 12:20pm

Re: Do you trust AI?

Post by axel_knutt »

When I tried Gemini with "How much energy does it take to accelerate 5kg to 20 kph?", I got a different wrong answer (15.46J), followed by the correct answer when I clicked the "Dive Deeper" button. Co-Pilot and Chat GPT both got the right answer first time. All three stated that this calculation assumes no frictional losses, which is the reasonable assumption, given that no loss data was supplied in the question.
Nearholmer wrote: 26 Aug 2025, 12:00am I think it’s wrong at the basic physics level too, because what it’s answering is the question “what kinetic energy does a stated mass travelling at a stated velocity posses?”, which isn't what you asked.

It can’t answer the question you did ask, because it doesn’t have a knowledge of the various resistances (opposing forces) to be overcome during acceleration, and it really ought to have told you that. If you think about trying to accelerate a huge, 5kg silk parachute through the air, a 5kg stone pushed up a hill across a bed of sand, a 5kg block of stone on two ice skates across an ice rink, a 5kg model boat pulled on a string across a pond, and any other weird combination you can think of, my point will become clear.
He didn't ask what the wind resistance of a parachute is, or the friction of a bed of sand, so the method for the question as framed is correct. One of the basic rules for doing exams is that if you're not given information in the question, it's because you're not expected to use it.

Redefining a question after it's been answered so that you can criticise the answer is a debating ploy known as Bait & Switch, named after the retail practice of placing good quality merchandise in the shop window, then switching for inferior goods at the point of sale.
“I'm not upset that you lied to me, I'm upset that from now on I can't believe you.”
― Friedrich Nietzsche
Nearholmer
Posts: 7593
Joined: 26 Mar 2022, 7:13am

Re: Do you trust AI?

Post by Nearholmer »

He asked a question which couldn’t be answered without further information being supplied. A good tool would have pointed that out to him, a n even better one would have given him the choice between supplying further information, or being given an answer which made the (very unreal world) assumption that no losses/resistances/opposing forces were at play, or maybe (faintly more real world) that losses were negligible.
One of the basic rules for doing exams is that if you're not given information in the question, it's because you're not expected to use it
Really? I guess it depends on the level of the exam, but if presented with that question in even a fairly basic physics exam, I would expect students to identify that it was unanswerable without further information, and then (extra marks coming) state a set of assumptions and answer accordingly. Easiest assumption, of course, is to assume negligible losses.

I didn’t redefine the question BTW, all I did was to hold AI’s feet to the fire to answer it properly, instead of answering a question that hadn’t been asked. On this occassion, this AI was no good!
Last edited by Nearholmer on 26 Aug 2025, 6:03pm, edited 2 times in total.
Jdsk
Posts: 29772
Joined: 5 Mar 2019, 5:42pm

Re: Do you trust AI?

Post by Jdsk »

One of the basic rules for doing exams is that if you're not given information in the question, it's because you're not expected to use it.
I've examined at many institutions. It may differ with level of education but I don't agree with this. I'd expect vocational candidates to identify missing relevant information, and that can be essential in answers which affect safe practice.

Jonathan
sjs
Posts: 1498
Joined: 24 Jan 2010, 10:08pm
Location: Hitchin

Re: Do you trust AI?

Post by sjs »

Jdsk wrote: 26 Aug 2025, 5:01pm
sjs wrote: 26 Aug 2025, 1:21pm In the recent past I have googled two clear questions with well-defined numerical answers. In both cases the AI version of the answer began with a correct statement of the problem, correct values for the inputs, and wildly incorrect (many orders of magnitude) values for the answer. That struck me as strange.
Which AI system was that, please?

Jonathan
Whichever one that google pushes in response to certain searches, as "AI overview".
Nearholmer
Posts: 7593
Joined: 26 Mar 2022, 7:13am

Re: Do you trust AI?

Post by Nearholmer »

Those “AI overviews” are so laughably wrong in so many instances that Im surprised Google isn't ashamed to display them. Maybe Google isn’t capable of shame.
sjs
Posts: 1498
Joined: 24 Jan 2010, 10:08pm
Location: Hitchin

Re: Do you trust AI?

Post by sjs »

Nearholmer wrote: 26 Aug 2025, 8:29pm Those “AI overviews” are so laughably wrong in so many instances that Im surprised Google isn't ashamed to display them. Maybe Google isn’t capable of shame.
Indeed. But you'd think that the final step, of simple arithmetic, would be the easiest part of answering the question. But perhaps only if you understood the question and the meaning of the inputs you'd gathered.
Jdsk
Posts: 29772
Joined: 5 Mar 2019, 5:42pm

Re: Do you trust AI?

Post by Jdsk »

sjs wrote: 26 Aug 2025, 7:53pm
Jdsk wrote: 26 Aug 2025, 5:01pm
sjs wrote: 26 Aug 2025, 1:21pm In the recent past I have googled two clear questions with well-defined numerical answers. In both cases the AI version of the answer began with a correct statement of the problem, correct values for the inputs, and wildly incorrect (many orders of magnitude) values for the answer. That struck me as strange.
Which AI system was that, please?
Whichever one that google pushes in response to certain searches, as "AI overview".
Thanks

"AI Overviews":
https://en.wikipedia.org/wiki/AI_Overviews

Jonathan
Post Reply