1.1 Survivorship Bias¶
Understanding Survivorship Bias¶
Survivorship bias occurs when only the "survivors" or successful instances of a given situation are considered, while those that did not survive or failed are ignored. This can lead to skewed perceptions and incorrect conclusions, particularly when analyzing correlations.
The World War II Bomber Planes¶
Historical Context: During World War II, the U.S. military wanted to improve the protection of their bomber planes to reduce losses. They analyzed the returning bombers, noting where bullet holes were concentrated, to determine where to add more armor.
Survivorship Bias in Action: The military examined the bullet holes on bombers that made it back from their missions. They found that the wings and fuselage had the most bullet holes, leading to the initial conclusion that these areas needed additional armor.
The Flaw: The key flaw was that they ignored the planes that didn't return—planes that were hit in different areas and thus didn't survive to be analyzed. The planes that were lost often had critical hits in the engines and cockpit, areas that appeared relatively unscathed on the surviving planes.
Revised Strategy: When this bias was recognized (notably by statistician Abraham Wald), the military revised their strategy to focus on reinforcing the engines and cockpit—the areas where damage was lethal and thus underrepresented in the surviving fleet.
Lesson: Focusing only on surviving bombers led to a flawed conclusion about where to add armor. Including information about lost planes provided a more accurate picture of where protection was truly needed.
See: Survivorship Bias (Wikipedia)
"Survivor" Companies in the 2008 Financial Crisis¶
Context: In the aftermath of the 2008 financial crisis, many analysts highlighted the success stories of banks that weathered the storm. Firms like JPMorgan Chase and Goldman Sachs were celebrated for their resilience.
Survivorship Bias in Action: Analysts focused on these surviving banks, attributing their success to superior risk management and strategic decisions. Many banks that failed or needed significant bailouts—such as Lehman Brothers and Bear Stearns—were not analyzed as thoroughly.
The Flaw: By focusing only on the surviving banks, the analysis might overlook broader systemic issues or common pitfalls that contributed to the failure of other institutions. The survivors may have had similar risk management strategies but benefited from factors like timing, government connections, or luck.
Lesson: Including a broader range of institutions, both successful and failed, provides a more balanced understanding of risk management and financial stability.
Successful Hollywood Films¶
Context: Film studios often analyze blockbuster films like Titanic or Avatar to identify what makes a film successful. They might conclude that high budgets, big stars, and elaborate special effects are key success factors.
Survivorship Bias in Action: Analysts focus on blockbusters while ignoring the many films with similar high budgets, big stars, and special effects that did not succeed. By excluding unsuccessful films, the analysis overestimates the importance of these attributes.
The Flaw: Timing, market trends, script quality, and audience preferences are often more important than budget or star power. Many expensive films with A-list casts have been commercial failures.
Lesson: To understand what makes a film successful, it is essential to consider both successful and unsuccessful films, analyzing why some failed despite having similar attributes to the blockbusters.
The Health Benefits of Exercise¶
Context: Numerous studies highlight the health benefits of regular exercise, often showcasing the long lives of people who exercise regularly.
Survivorship Bias in Action: Studies may highlight individuals who live long and healthy lives due to regular exercise. However, individuals who exercised regularly but experienced health issues or shorter lifespans are often underrepresented.
The Flaw: The benefits of exercise might be overstated if studies only focus on successful cases. Genetics, diet, environmental influences, and socioeconomic factors all interact with exercise to influence overall health outcomes.
Lesson: Including a broader range of individuals provides a more nuanced understanding of the relationship between exercise and health.
The Dot-Com Bubble of the Late 1990s¶
Context: During the late 1990s, investors poured money into technology and internet companies, driven by the success stories of firms like Amazon and eBay.
Survivorship Bias in Action: Investors focused on successful dot-com companies, attributing their success to technology, market potential, and innovative business models. Numerous dot-com companies that failed despite having similar attributes were not included in analyses.
The Flaw: The focus on successful companies led to a bubble where investments were based on overly optimistic projections. When the bubble burst, investors realized that the perceived success factors were not universally applicable.
Lesson: Understanding both successful and failed dot-com companies provides a more accurate picture of the risks and potential rewards in the tech sector.
The 1960s "Successful" Retail Strategies¶
Context: Retailers in the 1960s analyzed successful department stores like Macy's and Nordstrom to understand what contributed to their success, attributing it to location, customer service, and product variety.
Survivorship Bias in Action: The analysis concentrated on successful stores, believing their strategies were universally effective, while ignoring failed stores with similar strategies.
The Flaw: By ignoring failed stores, the analysis missed potential pitfalls and challenges. Replicating successful strategies does not guarantee success if the underlying conditions differ.
Lesson: Analyzing a comprehensive range of cases provides better insights into what strategies are truly effective.
The Case of Successful Entrepreneurs¶
Example: Steve Jobs and Apple's Success
Steve Jobs and Apple are often cited as prime examples of entrepreneurial success. Analysts highlight Jobs' leadership, Apple's product innovations, and marketing prowess as key factors.
Survivorship Bias in Action: Many other tech entrepreneurs with similar visionary ideas and strong leadership failed. There were numerous tech startups in the 1980s and 1990s with innovative ideas that did not achieve success, often for reasons including timing, market conditions, or execution failures.
The Flaw: By focusing only on Jobs and Apple, analysts might miss factors such as market timing, competition, and specific conditions that contributed to their success.
Lesson: Survivorship bias in evaluating entrepreneurial success can lead to an overemphasis on factors seen in successful cases while ignoring the complex reasons why similar ventures might fail.
Survivorship Bias in Financial Markets¶
Imagine you are an analyst studying successful companies to identify patterns that predict success. You analyze stock performance of companies that thrived over the past decade, looking for common characteristics.
Skewed Correlation Analysis: Your analysis might reveal that successful companies have high R&D spending, significant marketing budgets, and innovative leadership. You might conclude that these factors are crucial for success. But what if many of the companies that failed also had these same characteristics? By excluding failed companies, you miss the possibility that these factors alone are not sufficient for success.
The Startup Success Myth: Studying only successful startups and finding they all had charismatic leaders and innovative products might lead you to conclude these traits are essential. But if many failed startups also had these traits, survivorship bias means you are only seeing the "winners."
Implications of Survivorship Bias¶
By focusing only on successful outcomes, survivorship bias can lead to:
- Overemphasizing correlations: You find correlations between success and certain variables (e.g., high R&D spending) without recognizing that these correlations might not hold for failures.
- Misguided decisions: Investing resources into factors that seem correlated with success but are not truly predictive.
Avoiding Survivorship Bias¶
To avoid survivorship bias in analyses:
- Include all cases: Ensure that your analysis includes both successful and failed instances. This might involve looking at historical data, including companies that went bankrupt, or analyzing different segments of the market.
- Consider context: Understand the broader context and potential factors that contribute to both success and failure.
- Base rates matter: Consider how common success is overall. If 90% of startups fail, any shared trait among the 10% that succeed might simply reflect the trait's prevalence in the general population.
Summary¶
These real-world stories—ranging from WWII bombers and financial institutions to film successes and investment bubbles—demonstrate how focusing only on successes (or survivors) can lead to biased conclusions. By incorporating data from failures and understanding the broader context, we gain a more accurate and balanced view, leading to more informed decisions and strategies.
Exercises¶
Exercise 1: Survivorship Bias in Investment¶
Research the performance of major technology stocks over the past decade. Find information on both successful companies (like Apple, Amazon) and companies that failed or performed poorly despite having similar business models. Write a short report on how focusing only on successful tech companies might lead to survivorship bias.
Exercise 2: Evaluating Business Success¶
Compare the strategies of successful businesses (e.g., Starbucks, Tesla) with those that failed (e.g., Blockbuster, Theranos). Identify common factors and analyze whether these factors are truly indicative of success or if survivorship bias might be influencing your perception.
Exercise 3: Impact on Scientific Research¶
A scientific study shows that a new drug is highly effective because the study only includes patients who responded well. Design an improved study that includes all patients and evaluate how survivorship bias might have skewed the original results.
Exercise 4: Historical Analysis of Inventions¶
Research famous successful inventions (e.g., the light bulb, the telephone) and find other inventions from the same time period that failed despite being similar in concept. Prepare a report on how focusing only on successful inventions might lead to an incomplete understanding.
Exercise 5: Media Influence on Perceived Success¶
Analyze media coverage of successful startups or celebrities and contrast it with similar cases that faced failure. Reflect on how media focus on successes can create a skewed perception of success factors.
Exercise 6: Financial Success Stories¶
Compare successful and unsuccessful investment strategies or financial products. Determine if the successful ones share common features that might be misleading due to survivorship bias.