What is survivorship bias? It is when researchers focus on the individuals that have passed a selection process, while forgetting those that have not. It is an error in logic. Survivorship bias doesn’t have to be about people who have survived, it can reference people or things that have survived a selection process.
There are many examples of survivor bias. A common one comes from World War 2. The American Airforce was losing a lot of planes over Europe. They obviously wanted to do something to stop this. They decided to add more armor to the planes, but they needed to be selective where they put it because armor is heavy and the heavier a plane is, the more fuel it needs, and the shorter its range. The Airforce had a record of every bullet hole in every plane and they decided that they would armor the planes in the places where they had the most bullet holes. If this is where they were shot the most, it stood to reason that those were the places they should armor.
Before they went ahead and armored the planes, they turned to a mathematician called Abraham Wald and asked him to look at the data. He turned round and told the Airforce they were victims of survivorship bias. He told them that they should armor the planes in places where there were no bullet holes. Wald had realized that the Airforce were only looking at the planes that had survived and they were not looking at the planes that had been shot down. The planes with bullet holes were those that had made it back, which meant that planes shot in the places where those planes had been shot could still fly and make it back. Therefore, logically, the planes that hadn’t made it back had been shot in the places that the surviving planes hadn’t been.
There are many other examples of survivorship bias and we all fall for it. Warren Buffett is an example. He is one of the richest men in the world and has made a fortune investing in companies. He has, without fail, always made the right decision. People try to analyze his decision making style and to copy him, but we are not looking at the people who didn’t make it. If you start with a group of a million people and have them make an investment, then cut out the people whose investments fail, and repeat this process, you will slowly whittle the group down until you are left with one or two people who have been right every time. These people’s success could be down to skill, it could just be blind luck, but when we try to copy them, we need to think about the 999,999 people who failed.
You could replicate this by sending out 1 million letters at the start of the month predicting whether a stock would go up or down by the end of the month. In 500,000 of the letters you write “up” and in the other 500,000 you write “down”. Half of them will be correct. In month two, you repeat it with the 500,000 that are left, diving half into up and half into down. You continue this for 12 months and you will have 244 people who have received the correct stock predictions for an entire year. They will be happy to invest with you because they don’t know about the 999,756 people were you got it wrong. This is survivorship bias.
The same is true for successful people who dropped out of university, like Mark Zuckerberg and Bill Gates. The common reasoning is that if they dropped out of university and became successful, more people should. But that is survivorship bias because we are not looking at all of the people who dropped out of university and weren’t successful. Statistics show that people with a university education are 10% more likely to get a job and to make more money. Mark Zuckerberg and Bill Gates became successful despite dropping out of university, not because of it.
It can also be seen in architecture. We often have a romantic image of the architecture of bygone ages, such as the Victorian, or Tudor. However, this is survivor bias. The buildings that we still have from those eras are the ones that were deemed attractive, or were sturdy. The weak, unattractive, unappealing buildings from those eras would have fallen down, or been pulled down, to make room for the new buildings. The buildings we use to base our image of that era on are the ones that survived.
Is it possible to avoid survivorship bias? Yes, of course, but it is not easy. It is a logical error that all of us make perfectly naturally without thinking about it. If we stop and think, we can overcome it. We have to remember to look at the successes but also look at all of the failures as well. Abraham Wald was able to do it with the planes in World War 2 and we can do it as well.
Photo by urtimud.89: https://www.pexels.com/photo/wreck-of-airplane-10032959/
Sources
https://thedecisionlab.com/biases/survivorship-bias
https://en.wikipedia.org/wiki/Survivorship_bias
https://www.scribbr.com/research-bias/survivorship-bias/
https://www.bbc.com/worklife/article/20200827-how-survivorship-bias-can-cause-you-to-make-mistakes
https://mindojo.squarespace.com/blog-posts/survivorship-and-related-biases