Survivorship Bias

How often do we focus our analysis on winners? They could be winners in

any aspect of life; sport, business, medicine, but the case is clear we
tend to look towards those who are successful for clues and lessons on
how to be more successful ourselves. But is this actually the right way to
do things?
When Maurice Greene was the 100m World Record holder and Olympic
Champion, sprinters were slightly different to what they are now. They
were a bit more aggressive; the focus tended to be a bit more on power.
On the start line, they would prowl backwards and forwards; Im sure we
can all remember images of Linford Christie attempting to psyche out
his opposition. It was the prevailing wisdom that to be successful,
sprinters had to make themselves aggressive, and get pumped up before
a race to perform at their best. The opposite is now true; since Usain Bolt
started to dominate, sprinters are now typically much more relaxed. Youre
more likely to see them shadow boxing or laughing and joking on that
start line, as opposed to prowling with their lips curled as per the early
2000s. Now, the prevailing wisdom is that sprinters should be relaxed in
order to win.
But Maurice Greene, in terms of performance, would still be very much
competitive today. Similarly, Usain Bolt would still have been successful if
he was around in 2000, so clearly no one outlook is better; it comes down
to what works best for that individual. Taking our cues from those that are
successful can lead to a misreading of patterns; of thinking something is
important when it isnt; of over interpreting noise as signal. We see this is
medicine all the time. Just because one person is cured from terminal
cancer by a juicing diet doesnt mean the juicing diet works; we dont hear
the stories of those that follow the juicing diet but arent miraculously
cured.
The same is true in business; if you read a business book, its most likely
by someone who is successful. They are trying to help us by passing on to
us the tricks that brought them success but what we dont hear are the
stories of people who also did those tricks and werent successful. Again,
we are misinterpreting the importance of that information, in a form of
cognitive bias known as survivorship bias.
The best example of survivorship bias Ive ever come across occurred in
World War Two. The US Air Force kept having their bombers shot down,
and wanted to place extra armour in the planes to protect them from the
German anti-aircraft gunners. They only had a limited budget, however,
and so needed to place the extra armour in the places that would most
benefit them. They hired a group of mathematicians to form the Statistical
Research Group. This group looked at the planes that successfully
returned home from bombing raids, and focused on where they had
sustained damage. They found that the bullet holes were concentrated

around three key areas; the wings, the centre of the plane, and the tail
gunner.
So the logical conclusion would have been to put the armour on these
three areas, wouldnt it? And that would be survivorship bias; focusing on
the planes that hadnt been shot down. The research group suggested
exactly the opposite, placing the armour on the areas that had least
damage. This was because damage to the wings, centre of the plane, and
tail gunners obviously were not crucial to the plane, as these planes were
still returning home; they were the survivors. Instead, the planes that
were shot down were most likely sustaining damage elsewhere; the
reason why the surviving planes didnt show these damage patterns was
precisely because they hadnt sustained damage there, and hence had
survived.
We can apply this to our own performance too. So often in sport, when
people lose, they analyse their performance, focusing on what went wrong
and how to get better. But when they win, they celebrate and move on.
This is our own example of survivorship bias; it assumes that when we win
everything went well, and when we lost something went wrong. But this
robs us of a learning opportunity; when we win, things still might have
gone badly. The win might just be papering over the cracks. Personally,
Ive always tried to approach victory and defeat with the same levelheaded analysis. In 2005, as an 18-year old I beat Darren Campbell in a
race in Bedford. I received a lot of press for this, and it would have been
easy for me to get caught up in this. Instead, I recognised that my
performance was poor; I won, but my time overall wasnt good, and I
messed up a few areas of my race. It would have been easy to lose sight
of this in victory, but I made sure to evaluate my performance correctly.
The flipside is also true; in 2007 I came second in the European under23s, narrowly losing to Simeon Williamson. I had been focused on winning
that competition all season, so I could have taken defeat badly. But, again,
I tried to stay level headed; I ran a personal best, and actually put
together a decent race, performing well under pressure. Just because I lost
didnt mean I had done something wrong I was just beaten by a better
athlete. I could have searched for things that I felt needed changing, but
that would have been a mistake; instead, thats an example of the type of
race where everything comes together.
Its important to overcome survivorship bias in coaching. Do successful
athletes actually do something different, or do we just think that they do
because they are successful? To answer this question, its important to
look not just at those athletes you coach who meet their goals, but also
those that dont; are there actually any differences between them? We
can then apply this same logic to competition. We need to analyse the
performance itself, not necessarily the result. Sometimes the result is far
more important that the performance; this is in those competitions that
are end-goals, usually major championships such as the Olympic Games
or World Championships. All other competitions are typically stepping

stones on the way to the end goal, and so represent a great opportunity
for an evaluation of the athlete; where are they physically, could they
cope mentally with the competition, did they perform the skill technically
well? If we just look at a race as a win/loss outcome, we are robbing
ourselves of this chance to analyse and make changes, and that really is a
crucial opportunity lost.
In summing up, survivorship bias is the act of only paying attention to
success, and ignoring those that we perceive to be unsuccessful. At best,
this can hide trends and information that can be useful; at worst, it limits
individual performance and reduces the chances for growth and
improvement. We can overcome this by widening our lens to look at a
variety of different sources, and also by taking a more balanced approach
to evaluation, treating wins and losses as the same and seeing what went
well, what didnt, and what needs to be changed.