statistical likelyhood, randomness and perseption

[Dak]

not sure wether stats goes in applied or analysis, but i saw some stuff on probability in this forum... sorry if this is in the wrong place.

 

Basically, i was thinking about why we accept some stuff as random, whilst other stuff is deemed unrandom, even tho the statistical likelyhood is the same. i think i've figured the answre out, but id be interested in wether i'm right or not.

 

Basically, if we, say, roll two dice 1000 times, and get the following results:

 

die1: 1,4,3,1,2,6,etc (i.e., seemingly random string of numbers)

 

die2: 1,2,3,4,5,6,1,2,3,4,5,6,1,2... (i.e., the sequence '1,2,3,4,5,6' repeating)

 

then, we'd say die 1 was random, whilst die 2 wasn't; however, the probability of both outcomes is the same -- 1/(6^1000), i believe.

 

so... the result of die 1's rolls has a chance of occouring of 1/(6^1000). this is very unlikely, BUT it would be falicious to say that this result is non-random due to it's unlikely hood, because any result would have the same probability of occouring (compare: 'hey, the die we just rolled once resultied in 4, a result with only a 1/6 chance of occouring -- how statistically unlikely that result was, i therefore question the randomness of this die' ).

 

however, the same seems as if it should be true of die2's result... yes, it's unlikely for that sequence to occour, but any sequence would be equally unlikely, so surely it's incorrect to claim that the die is non-random due to the unlikelyhood of it's result?

 

surely it's only because we see a pattern in die2's result that we question it, whilst, because we can't see a pattern in die1's result, we will accept it as random? surely this is heavily reliant on our perception?

 

I think the conundrum is probably answred like this:

 

H0: die is random

HA: die is non-random

 

die1's results have no discernably pattern, so we have no problem accepting H0.

 

die2's results have a discernable pattern, so, reguarding this pattern we have two new hypothesys:

 

H(2)1: the pattern is reasonably expectable to be randomly produced (accept H0)

H(2)2: the pattern is not reasonably likely to result from randomness (reject H0)

 

so... in this case, with the probability of the pattern occouring randomly being 1/(6^1000), we reject H0 and assume non-randomness. (if dice2 were only rolled three times, the result would be different: ie, '1,2,3' would have a 1/216 chance of happening, which is resonably acceptable: therefore, accept H(2)1 and H0 -- we can believe that the die is random on a result of '1,2,3')

 

question 1, then, is: is the above correct?

 

(if the answre is yes):

 

question 2 is: relying on our perception, isn't the above somewhat arbritrary? for example, if i randomly arrange the letters of the alphabet to get:

 

qwertyuiopasdfghjklzxcvbnm

 

according to the above, we can assume non-randomness (the keys are in the same order as the layout on a qwerty keyboard); however, if the qwerty keyboard were never invented, and the dvorak were instead the norm, we would have no problem accepting the above as random?

 

finally, question 3 is: is it neccesary/usefull to work out the likelyhood of getting something that will arbritrarily be recognised as a pattern? e.g, is it neccesary to work out the chance of the alphabet being randomly rearanged resulting in:

 

P(the alphabet) + P(querty layout) + P(dvorak layout) (etc) *2 (because any of them backwards would also be considered a pattern) (etc)

 

so that we can say 'the probability of getting something recognisable as a pattern is x, so getting a pattern is/isn't significant enough to discound randomness?

 

feel free to add anything relevent to randomness v perseption

 

oh, and one request: if possible, could you please tone-down on the squiggly latex? i'll make an effort to understand any squiggles you use, even if i have to learn small amounts of mathematical concepts, but learning all of calculus to understand what you say is a bit beyond my patience

[bascule]

As far as I have been able to ascertain, arguments for the existence of true randomness have always been an argument from incredulity...

 

"I cannot conceive of any sort of statistical probability distribution behind this set of numbers"

 

Ergo...

 

"It's random!"

 

An obvious non-sequitur.

 

Instead, I assume the opposite: there is a tracable sequence of causation behind any sequence of numbers.

 

Why?

 

Because every other event, save for quantum events, has such a causal chain leading to its occurance.

 

To assume otherwise, until an acausal relationship with the rest of events in reality can be demonstrated, seems inconsistent.

 

The Bohm Interpretation shows that violation of Bell's Inequality does not demonstrate the existance of acausal, non-deterministic events. Strong determinism can be reconciled with so-called "quantum indeterminacy", because the possibility remains that the alleged indeterminacy is the result of non-local hidden variables.

 

I refuse to believe in randomness until I hear a reasonable defense of its existence.

[Tartaglia]

The test required here is a test on the fourier transform of the data. The second will be massively significant.

[Bignose]

oh, and one request: if possible, could you please tone-down on the squiggly latex? i'll make an effort to understand any squiggles you use, even if i have to learn small amounts of mathematical concepts, but learning all of calculus to understand what you say is a bit beyond my patience

 

Well this is awfully contradictory, since you are trying to discuss a mathematical concept, and you want to try to limit the amount of math we can use. Rather difficult to do....

 

That said, the very first thing that comes to mind is repetition is necessary. If the second die does the series you suggested every time, then it is obviously deterministic (non-random). But, there is a finite chance that the series did just occur on its own.

 

The next thing is that the experiment needs to be defined much more carefully. Is an experiment 1000 rolls? Then there is a distribution, P(1st roll = n1, 2nd roll = n2, 3rd roll = n3, ...). which is a huge function with 1000 variables. If each roll is independent, then it becomes P1*P2*...*P1000 where (P1 is shorthand for P(1st roll) & etc.). This could also be the case where 1 experiment is 1 roll of the die, and it is repeated 1000 times.

 

Lastly, I'd like to echo a lot of what bascule said, though in a slightly different way. The human brain works to a huge extent by pattern recognition. And, so, even in truly random situation, the brain is going to try to find a pattern in the data, whether one exists or not. Look how many times cause and effect are announced just because two variables have a correlation?

 

E.g. there is a correlation between having a healthy mouth and overall health of the body. The toothpaste companies in their ads try to tell you that a healthy mouth leads to a healthy body. And while, that is probably true to a certain extent, my bet is that the people who take care of their entire body are much more likely to brush and floss regularly, hence the correlation between the two.

 

One of the best ways to untrain the brain to look for patterns is to work through and understand the nuances of probability and statistics. You can quickly learn that coincidences just happen all the time, and that they don't necessarily mean anything. Also, you can learn how assign values of confidence to know how likely an event or hypothesis is.

[the tree]

so that we can say 'the probability of getting something recognisable as a pattern is x, so getting a pattern is/isn't significant enough to discound randomness?

Problem being that you only decide that something is "a pattern" retroactively, when you start doing that, probability becomes pracitically useless.

[Tartaglia]

For time series there are all sorts of statistical tests for all sorts of non randomness whether it be cyclic, autocorrelation, no of turning points, unit roots etc etc