Your brain is not your friend

Our brains have certain intuitions about the world hard-wired into them. Sometimes those intuitions are correct; other times, not so much. Science taught us not to trust our intuitions about the physical world because they are often wrong. We feel, for example, that heavier objects fall faster than light object, and that time as measured on a moving object does not depend on the speed at which the object moves. Both those feelings are flat wrong, as has been shown through theoretical reasoning and controlled experiment. Basically, science taught us that "gut feeling" is a useless tool when it comes to reasoning about the physical world.

But we also have built in intuitions about things other than physics. We have moral intuitions, as well as social and economic ones. And here's the thing: as opposed to physical intuitions, we do not check those against reality. Even though our hard-wired model of the physical world is obviously wrong and we know not to trust it, we don't want to apply the same standards to our model of social interactions. Why?

Before offering a possible answer, I'm going to discuss two examples of moral/social intuitions that are hard-wired, wrong, and seemingly incorrectible. The first one is the conviction we have that a person who acts with good intentions will always bring about good outcomes. Sure, we do have a saying that "The road to hell is paved with good intentions," but we don't really believe it. Evidence? When it comes to candidates for electable offices, we are much more interested in their character, integrity, etc., than in their knowledge of complex issues. We want politicians to show us that they care about us, rather than that they actually know what's good for us. Another ingrained wrong conviction is the zero-sum bias. This bias is essentially the belief that the amount of wealth in a society is fixed, so that the only way one person can get richer is by making someone else poorer.

It isn't hard to see where those beliefs come from. Way back when, when all humans lived in small bands of hunter-gatherers, it really was much better to know your friends' intentions rather than their IQs. When social interactions were much less complex than they are know, you were much more likely to be hurt by someone who was smart but emotionally unattached to you rather than by someone who genuinely cared about you. Also, back in those days, societal wealth actually was constant. Due to extremely small population density and extremely high time demands on subsistence labor, neither condition of wealth aggregation (specialization and trade) was fulfilled. But those things are no longer true. Well-meaning politicians can hurt us, and wealth is being generated every day. Why did we not notice, the way scientists did, that our ingrained beliefs do not correspond to reality? Why, in the words of Robin Hanson, do we see this:

Consider how differently the public treats physics and economics. Physicists can say that this week they think the universe has eleven dimensions, three of which are purple, and two of which are twisted clockwise, and reporters will quote them unskeptically, saying "Isn’t that cool!" But if economists say, as they have for centuries, that a minimum wage raises unemployment, reporters treat them skeptically and feel they need to find a contrary quote to "balance" their story.

The main reason for this, I think, is that in the "social" world the incentives aren't right, the way they are in the "science" world. Scientists are interested in what works. They are rewarded for building theories and conducting experiments that are useful, that "work." In order to get things right, scientific debate needs to weed out errors, no matter how commonsensical they may appear. Couple that with the fact that the feedback from physical reality is very strong, and you'll see why blatant errors cannot persist for too long. Not so in the world of political debate. Here, people are interested not in what works but in winning popularity contests called elections. Because competition for political offices is fierce, no one can win elections by trying to appeal to everyone; they have to choose their target audience. This means that, for every wrong hard-wired belief about social reality there will be someone who panders to that belief, trying to get votes that way. When it comes to issues that are politicized, there is no end to debate. And, contrary to what Hanson says, scientific issues can be politicized too; look at evolution, for example.