Baysian statistics 101

If you go to an astronomy department at a university these days and you'll see (if you hang out for about 5 minutes) an astronomer having some difficulty with a particular problem discusings it with their colleagues. Undoubtedly (and it's becoming more often) someone will poke their heads smugly out from their terminal and say "You know what'll fix that?.. Bayesian stats!"I know...I'm exaggerating, but it's fun!

For the following information I have relied heavily on these 2 blogs of Kevin Boon and Gallileo's Pendulum. thanks to Kevin and Matthew.

So what is bayesian stats? Complicated...but powerful.

I was first introduced to Bayesian statistics by a colleague at work earlier this year, Julia Gillard was still PM and we were talking about the upcoming election. He had done some baysian analysis on the outcome of the election and the result was a very high probability of a complete wipeout by the Liberal Party. He continued (in what can be described as a lucky guess or precognition!), even if Rudd becomes labour leader again, it'll still be a trainwreck for labour. I have two comments here. Firstly, it wasn't as bad as that, and secondly, I really didn't need baysian stats to tell me that result. So clearly, Bayesian stats is only useful in some cases, just like anything I guess!

Statistics is all about probability, and with probablility we can run into problems quite quickly. Baysian statistics attempt ot solve those problems. So here's some really basic probability in order to get to a version of baysian stats.

If I was to toss a coin, the probability that I'll get a heads is:


Further, the probability of getting tails given that the previous throw yeilded heads is


I'll generalise to get a more mathsy expression which is also the same in reverse which gives us:



This expression is also known as the prosecutors fallacy in forensic science and is not true.

It doesn't take much thought to see where this falls down. let's look at an example that I took from Kevin Boon. In a murder case, if we find fingerprints of person X at a crime scene we can say there is a  probability that Person X committed the crime, this is not the same probability of person X being the culprit given that their fingerprints are at the crime scene. A subtle but important difference.

Baysian statictics uses maths to get around this by working out the likelihood of something in the face of some particular piece, or pieces, of prior knowledge or evidence. It is expressed like this:



In words, it says that the probability of having a hypotheis, x, given the results, y, is equal to the probability of a hypothesis, y, given the result, x, times the probability of the result without any hypothsis, divided by the probability of that hypothesis without the result. In our coin example they are the same, the answer sill turns out to be a half.  I guess this is where the  prosecutors logical fallacy comes from. For another example. I took this one from an article in the new york times in August this year.

Assume that you’re presented with three coins, two of them fair and the other a counterfeit that always lands heads. If you randomly pick one of the three coins, the probability that it’s the counterfeit is 1 in 3. This is the prior probability of the hypothesis that the coin is counterfeit. Now after picking the coin, you flip it three times and observe that it lands heads each time. Seeing this new evidence that your chosen coin has landed heads three times in a row, you want to know the revised posterior probability that it is the counterfeit. The answer to this question, found using Bayes’s theorem (you can go ahead and calculate if you like), is 4 in 5. You thus revise your probability estimate of the coin’s being counterfeit upward from 1 in 3 to 4 in 5.

This is very simplified and with gross simplifications come errors and misinterpretations, I'm aware of this, but I think the reason for that is that Baysian statistics is very complicated and can be used well in some circumstances, but where it yeilds the same results as simple logic and "frequentist"stats, then it is not needed. Most situations we use stats, it is not complicated enough to invoke bayesian, so we dont. It'd be like trying to use wave functions and quantum mechanics to try and figure out if a cat is alive or dead! Too complicated and not necessary.

I guess my take home message is, baysian stats in certain fields like astrophysics, forensics etc can be very useful, but where it is not necessary, I wouldn't us it. In our astronomy office, sure, baysian might fix it, but if something else will too, go with that, it's probably gonna be quicker.

Oh, and @fringetracker (or anyone else), if you read this please comment!