Survivorship Bias

What is it: we overestimate our chances of success because we tend to hear about success more than failure and failures tend not to make it into a chosen sample…

Every footballer I see on TV is a millionaire so it must be really easy to become a footballer and make millions…”

Overview

Everywhere you look there is success. There are 1000s of authors making millions of pounds from book deals, startups make entrepreneurs millionaires overnight, footballers makes millions and there are musicians selling millions of records. It all looks so easy.

Is it though?

Let’s look at book deals. According to the Fiction Writer’s Mentor website, 1-2% of submitted manuscripts get published, that’s just one or two out of every hundred. Now consider how many aspiring writers actually finish their ‘project’ and submit a manuscript in the first place. Then consider how many aspiring writers actually even start writing, instead just dreaming of one finding the time to start.

Once you’ve looked a little more deeply, you’ll appreciate that the chances of being the next J K Rowling as marginally above zero. The same goes for our chances of being in a successful rock band or being given a £250,000 per week contract to play football.

So, why do many authors grossly over-estimate their chances are being published? It’s simply down to us not being able to access accounts of failure as easily as accounts of success. Failure is hidden or doesn’t get into sample group.

However, survivorship bias doesn’t just present itself when you’re looking at your future career prospects. Survivorship bias also rears its ugly head when we’re trying to get a representative, objective sample of data on which to make decisions. This is because evidence that may help to balance assumptions doesn’t make into your sample.

Let me explain.

Firstly, this isn’t the same as confirmation bias, which is when we, albeit subconsciously, eliminate evidence that is contrary to our existing beliefs. Instead, it is when the contrary evidence is simple hidden from us so we are never exposed to it.

A great example of this is when surveying groups of people. We’ll often exclude elements of a sample, simply because they can’t be part of it. The best example is one I’ve used below but I’ll expand on it slightly.

If you are looking to find out how many people can easily access your online college courses, you may be tempted to send out an online questionnaire. That sounds like a plan. However, aren’t the only ones that access the online questionnaire going to be the ones that can easily access online content? What about the ones who can’t easily get online access? They’ll never be able to answer your questionnaire, because, you got it, they couldn’t easily get online in the first place. So, they’ll never enter your ‘representative sample’ will they?

There are a number of other famous examples below, but the simply answer is, look for those who didn’t ‘survive’ and make sure they’re represented in your sample.

There is one final aspect to survivorship bias that we need to address, and that is pure luck. We often attribute success to factors we have influenced but don’t attribute pure luck. Be aware that, if you examine success and failure, you will probably find many similarities in their respective strategies. There is a lot to be learned in stories of failure… if you make the effort to find them.

What can we do to avoid this?

The simple answer is, dig around in the grave yards of your subject matter. Make sure you exclude blind luck. Don’t fall foul of confirmation bias by simply looking to confirm what you already think you know.

Also, beware of retrospective 20:20 vision. Don’t exclude your failures in your path to success. Finding things that didn’t work in your journey is as important as the things that did work.

You should also look at the impact of luck. In small samples, we tend to ignore the impact of sheer luck. Look at the false attribution bias for this.

Examples

• One of the most famous examples was in World War II where engineers looked at improving the robustness of their aircarft by strengthening the areas of the returning planes that had the most damage – what they should have done is strengthened the most damaged areas of the planes that didn’t return!
• An online survey looked at student access to online courses and surveyed all the students. This told them that 100% of the people that had filled in their survey said accessing online courses was easy. What about those that couldn’t access the online survey… did they find it easy to access the online systems?
• Be wary of survivorship bias in pre-selected groups. If you look at the FTSE100, does it represent the UK’s economy as a whole? Nope. It represents businesses that have been successful enough to be invited to join the FTSE100. The overwhelming majority of businesses are outside the FTSE100 so in effect the FTSE100 is simply a subsection of already successful businesses.
• There are many stories of elderly people attributing their survival to daily cocktails of red wine, chocolate and cigars. That’s great, if you surveyed people over 100 and asked why they were alive, would they attribute it to luck? What about those that ate well, exercised and slept eight hours every night but died at 95 or the others that drank copious amounts of red wine and died in their 70s? They couldn’t answer to balance the sample group.