The law of small numbers

I've written once before about a quite common belief that small samples behave just like large samples do. Some time ago I was watching a tennis match which would have been extremely close if not for the fact that one of the players was getting the short end of the stick in terms of bad referee calls. The announcers noticed it but didn't think it was important because, they said, refs aren't biased but simply make random mistakes so "in the long run it all evens out" (i.e. you'll get about as many erroneous calls in your favor as you will against you).

The total number of points an average tennis pro will play during his entire career is a rather large sample, so the claim about things "evening out" is probably true. Still, it's meaningless, even if true. It doesn't matter what happens in the large sample of all the points a player will play during his career; what matters is what happens in a number of small subsamples of those points that we call "matches." The number of points played in a single match is small enough that random mistakes do not have to even out. Or, thinking about it in a slightly different way, even though the total number of points you'll play is large, some of those points will matter a lot more than others. Very close encounters can be decided by five or six key points going one way or the other. If you get unlucky and the refs make five bad calls that cost you a Wimbledon final, what does it matter that at some point in your career you will get five bad calls going in your favor, if those calls are unlikely to be as meaningful?