What exactly is Survivorship Bias?
Survivorship bias is a tendency to focus on success over failure, typically because of a lack of visibility over people or things that did not make it past a specific selection process. It’s a logical error that can easily plague your ability to use data accurately in order to drive decision-making.
For businesses, survivorship bias is an important factor to keep in mind as it can damage the reliability of Key Performance Indicators (KPI), which are often taken as fact. This is a critical issue because KPI performance has been shown to increase by 7% when actions are grounded in data. But measuring the goals of a business against specific, quantifiable data over a period of time is only valuable if you are looking at all the data and doing so in a statistically sound way.
Here, we are going to look in detail at survivorship bias — how the theory was developed and why it’s so important — and then help you build systems and processes that can avoid this mistake within your own planning and data analysis. Luckily, this is both an interesting and important topic. Let’s get started.
The serious history behind survivorship bias
During the Second World War, the Statistical Research Group (SRG) was housed in an apartment at 401 West 118th street in Manhattan. The SRG were described as the most extraordinary group of statisticians ever organised, with their sole focus on solving problems for the military whenever they arose. The key figure within this group of extraordinary people, Abraham Wald, was presented with a very specific problem.
The Allies wanted to prevent their planes from being shot down by enemy fighters in combat, so they would armour them for extra protection. However, the armour was heavy and had a detrimental impact on the manoeuvrability of the planes while also increasing fuel consumption. Too much armour is a problem, as is too little. So, what was the optimum solution?
At this point, the military gave the SRG some data they thought might be useful as they tried to find a way to solve this issue. The returning planes were covered in bullet holes, but the damage the bullets inflicted wasn’t uniformly distributed. There were more holes in the fuselage, fuel system, wings and nose, with significantly less concentrated around the engine.
Armed with this data, the officers saw what appeared to be an opportunity for efficiency. Concentrate the armour in the areas with the greatest need — those that were being hit with the most bullets. But how much armour should they use? That was the question they wanted Wald to answer for them.
The response they got was entirely unexpected!
Wald simply asked them “where are the missing holes?” He was sure that the missing bullet holes could be found on the missing planes. He was right. The number of planes that were able to return after the fuselage was peppered with bullets was strong evidence that bullets to the fuselage can, and should be tolerated. The armour should therefore go where the bullet holes aren’t — on the plane’s engine.
Wald proved that while effective decision making requires forensic data examination, mistakes can still occur if the whole picture isn’t visible and taken into consideration.
Can you see all the bullet holes?
It isn’t your data that defines you — it’s the action you take. Data on its own doesn’t necessarily result in actionable insights. In fact, without the necessary care and consideration, data-driven strategies can take up considerable resources without ultimately producing results that translate into measurable business outcomes.
Not all of your data is useful immediately and, in the age of big data, it can be challenging for businesses to utilise all of their data effectively. As a result, it’s important to approach data with a clear understanding and focus on what is relevant to you right now.
Identifying which Key Performance Indicators (KPIs) are most relevant to your goals is critical in order to make sense of your data. If your data and KPIs aren’t in one place to give you access to the full picture, the results can be at best negligible, and at worst, a disaster.
In terms of business intelligence, KPIs are becoming increasingly influential. Tools that allow you to monitor and set targets against KPIs are making it easier to take action based on the accurate data you’ve collected.
Suggested reading: For more on using data to deliver successful outcomes, check out our blog — Make Data Actionable: Driving Success Using KPIs
Loop can help
As with the SRG a variety of businesses, including franchise networks and those in the retail, automotive and hospitality sectors, rely heavily on data to drive positive outcomes. Unfortunately, data can often be in different places, different systems and even different formats. Businesses can often grapple with too many spreadsheets, graphs and tables with no history, no predictability and no defined relationship to business outcomes.
Focusing on the centralisation of data is key to understanding business performance and identifying areas for improvement. With Loop, you can utilise a customisable, real-time KPI dashboard which provides a space for data and KPIs to exist side by side. This helps to avoid the misinterpretation of data and gives your business the ability to set and track KPIs against specific metrics, taking your business intelligence beyond simple data analytics.
Implementing the right technology and expertise is an essential step in transforming the insights harnessed from data into effective business outcomes. So, do you know where the bullet holes are? Either way, to make sure the armour is in the right place for you, get in touch with our team at Loop.
Suggested reading: If you want to read more about methods of KPI use, take a look at our blog — KPI Reporting and Analysis Best Practices
