# Survivorship Bias Survivorship bias can be summed up nicely with a single example: > **Never forget the missing bullet holes.** The example goes as follows: > “The armor, Wald said, doesn’t go where the bullet holes are. It goes where the bullet holes *aren’t*: on the engines. > > Walds insight was simply to ask: where are the missing holes? The ones that would have been all over the engine casing if the damage had been spread equally over all of the plane? Wald was pretty sure he knew. The missing bullet holes were on the missing plane. The reason planes were coming back with fewer hits on the engine is that planes that got hit on the engine weren’t coming back.” ### And don’t forget about those missing mutual funds either This same exact logic applies to mutual funds (this is described all over the place, although listening to Taleb talk about it is likely the most entertaining). Say you start with $N$ mutual funds on time $t=0$. Then at time $t = 10$ years you look at the performance of the funds. If you only look at the funds that didn’t go bust you may observe that the average return rate was $5\%$. However, if we include those that went bust, maybe the average return rate goes to $0\%$. But wait, you may ask, why would we *not* include all funds? Why would we purposefully *discard* those that went bust? That seems like important information that we would be discarding! And of course you are right! However, the way the world tends to work is that we only see the winners, not the losers. This means that we must remain ever vigilant against these types of situations! That is why this bias is listed in [Data Science Mental Models](Data%20Science%20Mental%20Models.md). ### How to train this type of thinking? Ellenberg writes: “A mathematician is always asking, ‘what assumptions are you making? And are they justified? In this case, the officers were making an assumption unwittingly: that the planes that came back were a random sample of all the planes.’” --- Date: 20220727 Links to: [Critical Thinking MOC](Critical%20Thinking%20MOC.md) [Data Science Mental Models](Data%20Science%20Mental%20Models.md) Tags: #review #todo References: * Chapter 1, How Not to be Wrong * All of Talebs work (especially the first 2 books)