Have you ever wondered why popular books on how to become a millionaire don’t turn their readers into rich people? Or why successful companies like Microsoft, Google or Facebook appear once a decade, although their success story is well-studied and even filmed in a big movie? The answer is simple: any success story is always luck, and the most valuable information remains in the minds of those who could not achieve success.
This phenomenon was first described during the Second World War. Mathematician Abraham Wald studied the location of the bullet holes received by American bombers during combat missions. The wings, stabilizers and fuselage were the most suffered from enemy fire; the cockpit, the engine and the fuel system had the least amount of holes.
It would seem the solution is to strengthen the most damaged places. However, Wald decided otherwise. He suggested the fact the plane was able to return to the base with such damage shows that they are not so critical, and the least damaged airframe’s parts are needed to be strengthened.
This conclusion was confirmed after the end of the war, when crashed planes were drawn from woods and swamps. It turned out that they had damaged exactly the places that Wald talked about — the engines, fuel system, and the cockpit. With such damage, a bomber could not fly — either the pilot died, or all the fuel was leaking. As a result, engineers who collected damage statistics just didn’t know on these types of damage.
This story is an example of survivorship bias, where the data set analyzed is only taken from the success stories. We can see the same cognitive bias in the business environment.
Study Losing Battles
Everyone and their mother knows the name of Facebook’s founder and the key moments of the company’s success story. What about the story of companies which “have come back on top of the shield dead”? Do you know the creator of MySpace? Can you describe the company’s route map? When you read only success stories you completely lose sight of those whose story is much more important.
Imagine each potential user in your target market is a plane.Then every sticking point, bad advert, miscommunication, missed conversion, bad meeting, missed opportunity in your marketing and sales funnel, confusing moment in your user onboarding, is an enemy bullet. And this makes your top users those planes that made it home safely with only non-fatal bullet holes received.
Like the British engineers, you need to strengthen the most vulnerable places of your project. And information about the weak links can be found only in the posthumous chronologies.
Countering Survivorship Bias
To counter this survivorship bias, you should take the following steps.
- Find out which parts of your target market are giving you no feedback. Do this by looking at where your current users are placed in your target market and how you have acquired them.
- Get in touch and gather feedback from some of those who have ignored your offers.
- Get in touch from some of those who have visited your website but did not start a free trial.
- Put in place measures to gather at least some feedback from the people who are not interested in your product.
Learn failure stories — they are more important than success stories. Appreciate people who write about their failures. When faced with someone else’s experience, distinguish between private and systemic cases — everyone plays the lottery by the same rules, but the units win and the millions lose.