Who says we don't understand Bayes' rule?

When dealing with conditional probabilities, people commonly make a mistake called ignoring the prior. Suppose you get tested for a disease that is present in 0.5% of the population. It's a very accurate test: when applied to a healthy person, it comes up negative 99% of the time; when applied to a sick person, it comes up positive 99% of the time as well. Your test comes back positive; what's the probability that you're sick?

Most people (including highly educated ones, e.g. doctors) will say it's 99%. This answer is way, way wrong; the actual probability is slightly over 33%. The mistake comes from ignoring the fact that it is highly unlikely that the test has been administered to a sick person in the first place, because only o.5% of the population carries the disease.

The technique that allows us to solve problems of this sort is called Bayes' Theorem. To calculate the probability that you're sick given that you tested positive--let's denote that by Prob(S|+)--you need three pieces of information:

1) The probability that the test is positive given that you're sick, or Prob(+|S).

2) The "base" (or prior) probability that you're sick, regardless of any additional information, or Prob(S).

3) The probability that the test comes back positive, regardless of any additional information, or Prob(+).

Now all that Bayes' Theorem says is that

Prob(S|+) = [Prob(+|S)Prob(S)]/Prob(+)

Even though the result of the calculation may sound surprising, the formula is simple and makes immediate sense (and, in our particular case, gives the answer of 0.3322).

But why is the result counterintuitive? Why are we so prone to ignoring the prior, if Bayes' formula is so obviously true and, in addition, mathematically simple? I remember having a conversation with an NYU game theorist about this; he was amazed that people systematically make such a simple mathematical error while at the same time being able to routinely solve mathematical problems orders of magnitude more complex than that.

Problems like what?, I asked.

He said: like, for example, walking.

He was right. Being able to walk requires whatever part of the brain that is responsible for walking (I have no idea which part it is, I don't know my anatomy) to be able to solve problems isomorphic to fantastically complex systems of differential equations, and to solve them over and over again at mind-boggling speeds. So if we can do things like that, why do we stumble on something as simple as Bayes' rule?

The answer is that we can actually do Bayes' rule just as well as we can do walking. But, like walking, we can only do it subconsciously. Just because we can walk doesn't mean we can actually consciously perform all the complex mathematics involved in successful walking; and conversely, just because our conscious thought has trouble grasping Bayes' rule does not mean that it isn't subconsciously applied somewhere inside our body.

It is applied. Neurons do it. Suppose there are two connected auditory neurons, A and B. Neuron A receives some stimulus S from the set of many possible stimuli, and then sends a certain response R to neuron B. The receiving neuron is then trying to guess which stimulus was received by neuron A, given the response it's got from it. In other words, it's trying to "estimate" Prob(S|R). How does it do it? By doing chemical things that can be accurately modeled by a very simple equation:

Prob(S|R) = [Prob(R|S)Prob(S)]/Prob(R)

Looks rather familiar.