Jump to content

The Bayesian Machine


ydoaPs

Recommended Posts

Tell me what a proof system is and then give me a reference or two that states what you have said.

 

EQ

 

See Evidential Foundations of Probabilistic Reasoning by David A. Schum Wiley concerning how to work with incomplete evidence. Mutatis mutandis it thus also works with complete evidence for else it would not work with incomplete evidence. Then you can only have the problem that it might be less accurate. Well now how must I picture that, like any other proof system i.e. logical inference tool that helps work towards a proof, the more evidence the more accurate the result. I see no reason whatsoever that Bayes can't replicate the accuracy of empirical statistics because the latter is less and not more (only more data) and on the same amount of data would lag behind.

 

The only critique to be giving the use of Bayes when you have more complete evidence is that you had better use an other tool because it is more simple Occam dixit then.

 

The other critique one can have against Bayes is that incorrectly used it also leads to claim more than can be delivered but that only applies when talking incomplete evidence. I.e. saying for instance that it is better in a court of law to use Bayesian mathematics all the time. That is then also an incorrect use of Bayes.

 

All science is thus frequentist per definition. n0 and n1 is frequentist BTW. So I can accurately describe a . and when you observe this you have some photons hit your eye your (Bayes in the) brain works out via "google-ing" in itself that I probably mean a "dot". Now then you need context. Do I mean a mathematical point? From this one dot I can have more dots probably representing a straight line. How many dots make a straight line, or can I take it as a given to represent a mathematical straight line. Does Bayes at a point when we have extremely many dots in the line somewhere along the line become less accurate then any other mathematical tool? Of course not. Bayes - properly used - is just as accurate. Then I can make a cube, and a moving cube with Bayes. And prove that a cube has certain mathematical properties. I can also model a photon red-shifting i.e. GR or describe it having or not having spin in QM.

 

Now this constitutes a logic verbal proof that can be set via Schum into mathematics to see if it is a sound reasoning. It will conclude that I made a lot of shortcuts, yet at a certain level constitutes a proof on a low norm.

 

Q

 

 

I have never heard anyone state that. Please provide us with some references.

 

 

This is just a wild claim at this stage. What do you mean by handle GR and QM?

 

 

Okay, so again we have a mathematical was of describing the scientific principal in light of Bayes.

 

 

So we do have some mathematics that cannot be described by Bayes or not? Your claim was that all mathematics can be reduced to Bayes' inference, right?

 

EQ

 

Yes

 

Q

 

 

 

Are you talking about methods of proof theory here, or do you want me to state a theorem and allow you to try to prove it using Bayes'?

 

Most modern proofs today are in the sense of proof theory, informal. They do not employ the machinery of foundational mathematics.

 

This is outside my area of expertise.

 

 

You are making this claim again without backing it up in any way.

 

It is like you are in court making statements about a defendant without anything to suggest what you say is true. No witnesses, no CCTV, no phone records, no forensic evidence... What is the judge going to say to the jury?

 

EQ

 

 

Okay lets take it to court. We Dutch don't have a jury system, but if you read our old law from the 1920ies that is in effect the French Code Penal you will see that Bayes held then pen of the Lawmaker.

 

After the Lucia case mentioned earlier they incorrectly blamed the law and changed it in lieu of critique by psychologists and a philosopher. The old law stated that the expert witness should state under oath what his feelings are about what his science teaches him concerning the evidence presented. He was allowed to guess.

 

Further more the evidence I gave in logic and Bayes and the reference that I had given earlier as well, provides a proof. So I met the burden of proof on what I stated on a norm that is by law correctly stated to be at the level you may expect. Now for the expert to provide counterproof.

 

You are (correctly so) treading very careful, but now you choose to put the case to court, okay you are hereby in this simulation brought under oath as an expert witness being a mathematician. The judge has to decide quickly whether or not to let the incarcerated Photon out of jail, or keep him there pending further investigation in the matter.

 

Now you have said that it isn't your field of expertise, well neither that of the judge. So then what does your expertise tell you is it on the basis of the evidence and argument I rendered and stated as proof on a low probative value - in your more expert feeling - probably more probable or less probable for Bayes to accurately reach the same outcomes on a given question as GR and QM in all cases?

 

A priori your guess as an expert being probably better than that of the judge. So please help the judge to take the quick decision to release or detain the photon. I.e. as a metaphor for believing Bayes can or can't comply preventing you to provide a don't know. You MUST guess, for your guess is better than that of the judge (as a probability rule of law.) In the old law you could justly so be detained if you shirked your responsibility to not only the judge but also the sunburned victim or the innocent photon.

Edited by kristalris
Link to comment
Share on other sites

Cox's "theorem" shows how Bayesian logic can serve as an extension of propositional logic. The first link in the third paragraph of ydoaPs' OP explains this nicely. Perhaps this is what kristalris is referring to.

 

However, propositional logic isn't sufficient for all of mathematics. First-order logic at least is required for much of the foundation of mathematics, and higher orders are also sometimes used when mathematics runs up against the limits of first-order. I haven't been able to find anything to show that Bayesian logic is also an extension of predicate logic, but perhaps kristalris knows of something along those lines, and I'd be interested in seeing it if so. Until then, the claim that all of mathematics is reducible to Bayes seems a bit off.

Link to comment
Share on other sites

I'd like to see an example, such as Pythagoras's theorem proved via Bayes's theorem.

Does this mean that you are certain it can't be done? A

 

Or B that you guess that it can't be done?

 

Or C that you are not certain it can be done?

 

Edit NB it is not the issue whether or not it is practical to use Bayes, but only if Bayes is capable of doing it just as accurate within the same scope.

Cox's "theorem" shows how Bayesian logic can serve as an extension of propositional logic. The first link in the third paragraph of ydoaPs' OP explains this nicely. Perhaps this is what kristalris is referring to.

 

However, propositional logic isn't sufficient for all of mathematics. First-order logic at least is required for much of the foundation of mathematics, and higher orders are also sometimes used when mathematics runs up against the limits of first-order. I haven't been able to find anything to show that Bayesian logic is also an extension of predicate logic, but perhaps kristalris knows of something along those lines, and I'd be interested in seeing it if so. Until then, the claim that all of mathematics is reducible to Bayes seems a bit off.

 

 

http://ba.stat.cmu.edu/journal/2012/vol07/issue01/sancetta.pdf

 

 

Will this do?

 

Edit If you accept that Bayes can show what happened in the past like describe a crime science, then what is the problem with predicting the future as what probably will happen? Say what will happen to fundamental particles colliding et cetera?

Edited by kristalris
Link to comment
Share on other sites

While I don't have time to thoroughly read the paper right now, I did look through it briefly. It doesn't seem to have much to do with my post, so no, it won't do.

 

You'll have to find someone just a tad less mathematically mature than I am if you're looking to drown someone in equations. :P

 

More seriously, if you do find something that addresses the point (read: shows that predicate logic is isomorphic to probability theory), though, I would like to read it (probably after finals this week).

Edited by John
Link to comment
Share on other sites

Does this mean that you are certain it can't be done? A

 

Or B that you guess that it can't be done?

 

Or C that you are not certain it can be done?

This is besides the point. It does not matter what we think as you are the one making the claims here. Thus it is up to you to provide some evidence.

 

Give us a simple mathematical theorem that is not obviously linked to Bayes' and then prove it using Bayes.

 

This would add weight to your claim even though it would not be a full proof. I have no idea how you could fully prove your claim. You would need to show that Bayes' covers all foundational mathematics and even then I think you would run into trouble with questions related to reducing all mathematics to logic.

 

John's request is very reasonable, show us the links between the various forms of logic, probability theory and Bayes. From there you maybe able to make your claims more rigorous, but I doubt you will capture all mathematics.

Link to comment
Share on other sites

Capt'n John & ajb,

 

Okay let's take on Pythagoras as requested.

 

A friendly statistician said: "Bayes can't prove Pythagoras

 

Bayes could help you evaluate empirical evidence that something close to Pythagoras is true

 

But you'll have to put in a prior probability and if you do that in a silly way even Bayes won't help you"

 

Okay let's see if we can find a silly prior probability? The normal mathematical proof of Pythagoras is based on assuming an absolutely straight line of absolutely the same length is possible. Well, this is absolutely within a Bayesian mathematical context impossible, with one exception what I'll deal with later.

 

An absolutely straight line is at the deepest possible level never proven to be possible at all. Further more it is extremely improbable that an absolute straight line can exist at all anywhere in the cosmos. It only exists as an galloping mystic unicorn in the imagination of mathematicians.

 

So on the deepest level in reality where other mathematics outside Bayes can't hack it, Bayes can. Yet because there is not enough data any inference will a priori be so inexact that the law of parsimony requires the use of normal language for this proof on a reasonable i.e. practically reachable norm.

 

Well then Pythagoras is thus disproven at this deepest level. I think we can leave the formalities of putting this verbal logic to the mathematics of Bayes as described by Shum. So at the deepest level Bayes rules Pythagoras out.

 

Yet at a practical level (Bayes being the mathematics of common sense) say when building a house then Bayes on a much lower norm can cope with proving Pythagoras as correct, in the way the friendly statistician said. I don't think anyone would contest that.

 

Now then the mathematical proof of Pythagoras in mathematics isn't possible depending what definition you hold for what is Bayes.

 

Because Bayes also allows you to assume that absolute proof is possible. If you do put that into the formula of Bayes then the algebra that remains will no longer be distinguishable as Bayes, but we know it is in the full proof because that was the starting point. The rest of the algebra can thus subsequently be taken in via the normal way of proving Pythagoras. I.e. Bayes is not at odds with that mathematical proof even then.

 

As I said Bayes rules. And as an alpha to omega from the smallest to the largest infinite scale it always renders the correct scientific answer, when used correctly.

 

What has also dawned on me is that in these extremes it is first gear Bayes that rules. I.e. normal language that is mathematically thus pointed to via the law held within Occam.

 

For you to disprove my proof.

Link to comment
Share on other sites

You're misunderstanding how mathematics works. In mathematics, we state axioms, which are taken to be true, and prove theorems based on those axioms. The systems that result may describe the real world fairly accurately, or they may bear very little resemblance to the real world. Regardless, the mathematics is valid (as far as we can tell--silly Gödel).

The Pythagorean theorem doesn't say, "There exist straight lines, and putting three line segments together such that a right triangle is formed, the side lengths are related such that the square of the hypotenuse is equal to the sum of the squares of the other two sides." Rather, based on the axioms of Euclidean geometry, which assume the existence of straight lines and allow the construction of triangles, the Pythagorean theorem states that if a triangle contains a right angle, then its side lengths are related as stated before.

 

Propositional logic may be sufficient to prove the theorem, so Bayesian logic may be sufficient also. But you've yet to show how Bayes handles logics of higher orders than propositional logic, which are required for much of mathematics. Therefore, the point stands that "all mathematics is reducible to Bayes" doesn't seem to hold water. In any case, if your claim were true, Bayes' theorem still wouldn't deserve the highest praise, given that it's derived from the formula for conditional probability. Thus your ultimate claim seems to be that the Kolmogorov axioms (or something equivalent) provide a complete axiomatization of mathematics, which is problematic for reasons ajb mentioned before.

Give Bayes credit where it's due, but as ydoaPs indirectly mentioned in his OP, recognize its limitations as well.

Edited by John
Link to comment
Share on other sites

You're misunderstanding how mathematics works. In mathematics, we state axioms, which are taken to be true, and prove theorems based on those axioms. The systems that result may describe the real world fairly accurately, or they may bear very little resemblance to the real world. Regardless, the mathematics is valid (as far as we can tell--silly Gödel).

 

The Pythagorean theorem doesn't say, "There exist straight lines, and putting three line segments together such that a right triangle is formed, the side lengths are related such that the square of the hypotenuse is equal to the sum of the squares of the other two sides." Rather, based on the axioms of Euclidean geometry, which assume the existence of straight lines and allow the construction of triangles, the Pythagorean theorem states that if a triangle contains a right angle, then its side lengths are related as stated before.

 

Propositional logic may be sufficient to prove the theorem, so Bayesian logic may be sufficient also. But you've yet to show how Bayes handles logics of higher orders than propositional logic, which are required for much of mathematics. Therefore, the point stands that "all mathematics is reducible to Bayes" doesn't seem to hold water. In any case, if your claim were true, Bayes' theorem still wouldn't deserve the highest praise, given that it's derived from the formula for conditional probability. Thus your ultimate claim seems to be that the Kolmogorov axioms (or something equivalent) provide a complete axiomatization of mathematics, which is problematic for reasons ajb mentioned before.

 

Give Bayes credit where it's due, but as ydoaPs indirectly mentioned in his OP, recognize its limitations as well.

John thanks for a clear response to my post. I'm quite busy at the moment and I guess so are you. I'll postpone my reaction to next week ok?

Link to comment
Share on other sites

Bayes could help you evaluate empirical evidence that something close to Pythagoras is true.

 

The normal mathematical proof of Pythagoras is based on assuming an absolutely straight line of absolutely the same length is possible.

 

An absolutely straight line is at the deepest possible level never proven to be possible at all.

Echoing what John has said, you have completely misunderstood what mathematics is. The fact we cannot physically realise a straight line has no bearing on the question. Mathematics is about abstraction, how it relates to nature is another question.

 

However, you are right in the sense that I can preform experiments on right angled triangles drawn on a piece of paper and then use some statistics to show that Pythagoras' theorem holds well in nature. This would not prove Pythagoras' theorem in the mathematical sense, but it would convince people that it is probably true and that it is a useful fact.

 

Well then Pythagoras is thus disproven at this deepest level.

You may now want to re-think that statement.

Link to comment
Share on other sites

Echoing what John has said, you have completely misunderstood what mathematics is. The fact we cannot physically realise a straight line has no bearing on the question. Mathematics is about abstraction, how it relates to nature is another question.

 

However, you are right in the sense that I can preform experiments on right angled triangles drawn on a piece of paper and then use some statistics to show that Pythagoras' theorem holds well in nature. This would not prove Pythagoras' theorem in the mathematical sense, but it would convince people that it is probably true and that it is a useful fact.

 

 

You may now want to re-think that statement.

 

Now I hope that John doesn't feel compelled to react, because I guess finals are at the moment more important, but as you push it I will react.

 

The context of the OP is me shouting Bayes all the time in a scientific context. That context is one in which scientists keep shouting show the mathematics all the time.

 

Bayes is mathematics right? Yes I am right.

 

What I have stated is nowhere in conflict with what you or John have stated on mathematics. We are only at odds on the conclusion in reference to science being the topic.

 

Again then: This is philosophy forum remember? Yes indeed it is.

 

Now then when you take something as a mathematical fact you mutatis mutandis take it as an absolute fact. As such Bayes also has absolute facts namely the mathematical trick of having both the absolute fact pro and the absolute opposite taken both to be true at the same time. I.e. as a mathematical fact. Doing that will logically and mathematically never lead to the mathematically required absolute truth as John and you and my friendly statistician correctly state because I taking it that way state the same.

 

Well then starting with my proof with the formula of Bayes I subsequently perform the counter trick: this by also taking in what Pythagoras takes in as a fact, the absolute theoretical / mathematical truth. In so doing I change the formula that is algebra to start with in algebra only having one stated absolute truth. Subsequently the algebra can be formulated in exactly the same way as is done by Pythagoras' proof.

There is then no mathematical conflict between Bayes starting point and Pythagoras. Or would you state there is?

 

The words that I thus de-Bayesed Bayes are words because the proof then is pure mathematics. Uncontestably so.

 

What you subsequently don't grasp is that Bayes allows for different norms to be applied. Also thus on Pythagoras. Within Bayes (= mathematics) it is this also possible to prove Pythagoras and even to disprove Pythagoras in - science! - with mathematics. I've shown you how. Because such a proof is in the mathematics of Bayes it thus per definition also qualifies as a mathematical proof, yet not one you mathematicians are accustomed to. I.e. now you know that there obviously are different sorts of mathematical proof.

 

You all incorrectly think the topic is only A CERTAIN PART of mathematics. Well with Bayes it is much broader. And whether within mathematics as you take it or encompassing all of science - with the mathematics of Bayes - the Bayes machine rules as the ultimate arbiter of all of science (and mathematics BTW). (= the actual claim I keep on making and the topic thus.)

 

There is no mathematical formula that via the counter trick can not be emulated by Bayes. There is thus no mathematical odds between what we are saying, but just in effect a debate on what the definition (=word =/= mathematics!) Bayes covers.

Edited by kristalris
Link to comment
Share on other sites

No only the correct ones of course. If you take the incorrect ones then they are 100% incorrect. Like I said the nice thing with Bayes is you can fill in what you like, it will show you if it is mathematically correct providing you do your mathematics correctly of course.

 

In mathematics for you B = all incorrect mathematics are incorrect because given Bayes that A is the opposite of B yet the same value of 100% (i.e. 1) because you assume this.

 

You could also do the same trick in theory assuming an infinite amount of data only pointing in one direction towards the correctness of correctly used mathematics.

 

Conclusion: all of correct mathematics fits Bayes => all of correct mathematics = scientific

Edited by kristalris
Link to comment
Share on other sites

No only the correct ones of course. If you take the incorrect ones then they are 100% incorrect. Like I said the nice thing with Bayes is you can fill in what you like, it will show you if it is mathematically correct providing you do your mathematics correctly of course.

In hindsight, yes, but that's trivially easy and by then, who cares? All you've shown is the probability of something happening is 1 if the event already happened. In this kind of situation the usefulness of Bayes lies in prediction where little or no data exists, and there is no model that can do any better.

Link to comment
Share on other sites

Well no, you lot need to brush up on your Bayes (there is much more to it than I've shown you BTW). Further more you lot need to rethink the philosophy on mathematics / science. And, you lot need to brush up on your verbal logic because I've been explaining this all along.

 

The only thing I've stated is that all of mathematics fits Bayes and I've been opposed by this enormously. So I guess that you admit you all were wrong then? Because then we can go to the non trivial ramifications of this for you lot newly acquired insight.

Link to comment
Share on other sites

The only thing I've stated is that all of mathematics fits Bayes and I've been opposed by this enormously. So I guess that you admit you all were wrong then? Because then we can go to the non trivial ramifications of this for you lot newly acquired insight.

No, you made some rather bold claims that go well beyond this, and that's really what has been opposed. "any useful mathematics that you agree can be used can be replicated by Bayes", (emphasis added) for one.

 

When ajb asked "What is your claim here? All mathematics can be reduced to Bayes' theorem?" you answered "Yes."

 

When it came time to back this up, you moved the goalposts.

Link to comment
Share on other sites

Indeed I've shown you that all mathematics can be replicated by Bayes and all mathematics can be reduced to Bayes.

 

I reduced all mathematics first to A and B and thereby had the formula of Bayes replicate it. Informally of course because you have in effect thrown in the towel.

 

So not one goalpost has been moved not an inch. Absolutely zero movement of the goalposts. You are trying desperately to wriggle out of it. First attempt failed. Okay try again.

Link to comment
Share on other sites

The only thing I've stated is that all of mathematics fits Bayes ...

In what sence?

 

It look looks like you are saying that if a correct proof exists then it is a correct proof!

 

Of course in practice things are more subtle than this, but that is another story.

 

 

Indeed I've shown you that all mathematics can be replicated by Bayes and all mathematics can be reduced to Bayes.

You have not shown this, only that you don't seem to understand what mathematematics is and thus our objections.

Edited by ajb
Link to comment
Share on other sites

What I've proved is that a correct mathematical proof always can be made to fit Bayes. You et all disputed this. The only objection you can have would either be that I thus don't understand that mathematics is about formalism. For I took a shortcut. Is that your objection? Putting all mathematics in A is a reduction to the max. And when I marry it to the formula of Bayes in this way that in fact is a replication of all mathematics. I guess I can safely dispense with the formality of actually writing down all formal proofs of mathematics and between ( ) and then say = A. Or is there in your expert opinion some mathematics that don't fit?

 

Or is your objection that you think it is trivial, and that mathematicians don't concern themselves with that? I already put forward that there are non trivial ramifications that will follow.

 

BTW the counter that I don't understand mathematics is an invalid counter. It is a fallacy ad hominem based on a fallacy of authority if you don't substantiate what you mean. BTW the objections you refer to don't cover the proof I gave. They were before I gave the formula. So then you will have to shoot apart the formula I gave and what I did with it. You haven't done that. Not before or after.

Edited by kristalris
Link to comment
Share on other sites

Kristalris;

 

Hi.

 

I have been following this thread for some time now, and although I do not know anything about mathmatics, I do see a problem that I hope to explain. When I read Swansont's post that stated that you keep moving the "goalposts", I identified with it, because that is exactly how I felt when in discussion with you in the prior thread.

 

It amazed me to learn how many members are exceptional in mathmatics, and I do not wish to dispute their findings or arguments, but I suspect that your problem with them lies more in your premises than in your math. Reading your posts makes me think that you do not see a difference between truth, facts, statistics, and mathmatics. You seem to treat them as essentially the same thing; and therefore, interchangable. They are not.

 

It would help me, if no one else, if you could explain how you think they are different, and under what circumstance(s) they are interchangable.

 

G

Link to comment
Share on other sites

What I've proved is that a correct mathematical proof always can be made to fit Bayes.

Please make it very clear what you mean by this.

 

It looks like you are just saying something along the lines of if a proof exists then we can be 100% sure a proof exists and thus any proved theorem is true!

 

I think we can agree on this. But so how does this mean that all mathematics can be reduce to Bayes' inference or how does Bayes' theorem help us get at unknown proofs?

 

Or is there in your expert opinion some mathematics that don't fit?

Please see my comments above.

 

Or is your objection that you think it is trivial, and that mathematicians don't concern themselves with that?

It has been said that all theorems once proved are trivial!

 

The problem is we don't know what you are really trying to tell us. Not all mathematics can be reduced to Bayes' and that I think is not really disputed. It follows from the fact that not all mathematics can be reduced to pure logic.

 

So once again, make clear your claim. It is either false or trivial, or maybe we are missing what you are saying.

 

BTW the counter that I don't understand mathematics is an invalid counter. It is a fallacy ad hominem based on a fallacy of authority if you don't substantiate what you mean.

Using Bayes', as you have not demonstrated to any of us that you are knowledgeble in mathematics, we conclude that it is unlikely that what you are saying has any worth.

 

BTW the objections you refer to don't cover the proof I gave. They were before I gave the formula. So then you will have to shoot apart the formula I gave and what I did with it. You haven't done that. Not before or after.

You have not given us proof of anything.

 

What formula are you refering to? No one will dispute Bayes' theorem as it is a well formulated statment within a well founded mathematical frame work. We can prove the statement it true! It is the use and interpretation that we object to in this context.

 

Once again, you will need to make your claim very clear to us. It looks a trivial statment with hindsight and not useful.

Link to comment
Share on other sites

Well Gee, on the goalposts then:

 

Please picture or take a empty peace of paper and draw a line across have of it. Then draw a large circle in the top half and name it (goal(post)) A. Then repeat that with a large circle under the line and name that (goal)post)) B.

 

Now take anything you like to know from me like all correct mathematics and draw a small circle in the top circle. You may place some dots in the circle. Then take say all correct statistics and draw that small circle in the top circle A. New dots. If you like the small circles do or don't (mathematical statistics certainly does BTW) overlap. What ever you like. And you an take the large circle A as containing all absolute truths.

 

In the circle B you can do the same with all absolute untruths. (In correct mathematics.)

 

Now when you've drawn all the respective circles in the picture, representing the most basic representation of Bayes in this respect, do you think the goalposts moved? I don't think so.

 

So for starters I'm making the point that Bayes can deal with all mathematical or logical absolute truths. (And of course less then absolute truths.) As this picture depicts you don't have to bother with questions whether or not statistics is an integral part of mathematics or not. The only thing UP TO NOW i.e. concerning my claim that Bayes can check all of science as being absolutely or to a lesser degree true because it also takes care of the exact opposite.

 

If you do see now what I've been stating then you might ask yourself why you didn't spot this sooner. It should of been clear since the OP because the formula was already given, yet not in its most simple / reduced form concerning the question whether or not the formula could take all science in.

 

I hope that you now see that it can. Only then can we go further and see what Bayes can all do and that the ramifications are far from trivial.

Edited by kristalris
Link to comment
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
×
×
  • Create New...

Important Information

We have placed cookies on your device to help make this website better. You can adjust your cookie settings, otherwise we'll assume you're okay to continue.