Survivorship Bias 101

“There are known knowns. These are things we know that we know. There are known unknowns. That is to say, there are things that we know we don’t know. But there are also unknown unknowns. There are things we don’t know we don’t know.” – Donald Rumsfeld

When making a decision, what are the factors we have not considered? What is missing? How do we manage risk when we don’t know what is missing? How can we make good decisions when we don’t take into consideration what is missing, and we believe we are operating with full knowledge?

What is Survivorship Bias?

Survivorship Bias is an error in thinking, a logical fallacy, where we make decisions based on the data at hand, and fail to acknowledge that we may be missing other data, that is vital to our decision. As a result, we are at risk of drawing incorrect conclusions and making wrong decisions.

Survivorship Bias, in particular, is also subject to other biases, such as the Optimism Bias, and Availability Bias, which means that our decisions are even further removed from logic, with every bias adding another layer of fog to our already skewed thinking.

For example, some lotto winners, when announced, “reveal” their winning strategy; and it is almost always the same strategy: they have played the same numbers for years. This, mistakenly, leads us to conclude that people who win play the same numbers. What we don’t take into account is all the people who won and had no strategy to share. They simply got lucky (as is the case with or without strategy) and won.

Now, this is an example with pretty trivial consequences. The worst that can happen to you, in this case, is that you waste $10 a week for 10 years while believing you have a winning lotto strategy. So, let’s look at an example that shows the seriousness of this bias more appropriately.

Example

Here is a simple real-life example from my own life.

Last year I had to undergo surgery that would leave me with a fairly big scar. The 3 surgeons that I had consulted all confirmed that the scar is unavoidable, and 2 out of 3 had shown me post-op pictures of the work they have done that showed off their skills.

All 3 surgeons came highly recommended, but I opted for the surgeon who showed me no post-op pictures. Why?

Because, as he explained, people who agree to before and after pictures being published are usually people who have healed better. In essence, people who have ended up with ‘uglier’ scars generally don’t like their photos to be shared around. He went on to explain that it then gets worse as surgeons ‘pick’ the best cases, out of an already biased lot, to use as their testimonials.

I know this is quite a specific and personal example, but the bias is worth considering next time you’re reading testimonials or success stories for personal or professional use. The failures never see the light of day, they don’t ‘survive’, hence the Survivorship Bias.

Overcoming Survivorship Bias

The bottom-line error in the Survivorship Bias is a nothing more than flawed cause and effect logic, where people take a “surviving” data set and think the relationship between the data and the observation is causal, when in fact it is only correlational. 

When making a decision, my advice is to sort out the available data into what you know, what you know you don’t know, and then that leaves what you don’t know you don’t know… for this last one I recommend another pair of eyes, another pair of hands or another brain. In other words, have more people look over the information and the decision at hand. Different people will bring different life experiences, and with that different viewpoints. They may see what you have missed, help fix the holes in your knowledge, and maybe even surface some assumptions that you have taken as facts or some correlations you have taken as causation.