The Royal Institution released a video explaining how survivorship bias distorts data by focusing on survivors while ignoring those who did not make it [1].
Understanding this cognitive bias is critical for scientists and analysts because it prevents the drawing of false conclusions from incomplete datasets. When researchers only look at the "survivors" of a process, they risk missing the most important information, the failures.
Aoife McLysaght and Alice Roberts present the concept through a historical example involving World War II aircraft [1]. During the war, analysts looked at planes returning from missions to determine where to add armor. They found that returning aircraft had bullet holes in specific areas, such as the wings and fuselage [1].
Initial logic suggested that these damaged areas were where the planes were hit most often and therefore needed more protection. However, this approach ignored the planes that did not return. The aircraft that crashed were likely hit in the engines or cockpit, areas where the surviving planes had no holes [1].
By focusing only on the survivors, the analysts were seeing where a plane could be hit and still fly home. The true vulnerability lay in the areas where no returning planes showed damage [1]. This illustrates the core of survivorship bias: the visible data is not representative of the whole population.
The presentation extends this logic to the field of genetics and evolution [1]. In biological systems, the traits observed in living populations are those that allowed ancestors to survive and reproduce. The traits that led to extinction are not present in the current gene pool, meaning the evolutionary record is inherently biased toward success [1].
This conceptual framework helps researchers identify "missing" data in various fields. By questioning why certain data points are absent, scientists can better understand the mechanisms of failure and survival in both mechanical and biological systems [1].
“The visible data is not representative of the whole population.”
Survivorship bias is a logical error that occurs when a sample is skewed because the process of selection removes the non-survivors. In practical terms, this means that studying only successful companies, surviving patients, or returning aircraft leads to an incorrect understanding of risk and failure. By acknowledging the 'silent' data of those who failed, researchers can develop more accurate models for safety and evolution.
