Sie sind auf Seite 1von 3

The Statistician Who Ate Humble Pie http://www.strategy-business.com/enews/enewsarticle/ac00001?pg=all...

Magazine enews Search and Browse Press Room Reprints Subscriber Services HOME

strategy+business is published by the global commercial consulting firm Booz & Company.

Click here to view the enews archives.

enews Page 1 2 All

The Statistician Who Ate Humble Pie


by Nassim Nicholas Taleb

6/17/09
Nassim Nicholas Taleb, author of The Black Swan: The Impact of the Highly Improbable, introduces
an engaging lesson in business forecasting from Dance with Chance: Making Luck Work for You, by
Spyros Makridakis, Robin Hogarth, and Anil Gaba.
Just about every human decision about the future is tainted by a gap — the difference
between what we think we know and what we actually know. The more expert we are,
the wider the gap is likely to be. The story below, an excerpt from the book Dance with
Chance, is a classic example of an expert-busting enterprise. The experts in this case
are highly sophisticated statisticians and professors who are using up-to-date models.
Most of us assume that sophistication helps us understand the future. But in fact,
sophistication makes things worse; it invites misplaced focus on the complicated. In the
end, if we expect to make better decisions, we would do well to internalize the lesson in
this passage. That doesn’t mean relying on common sense. It might mean looking for
experts who understand the limits of what they know and the relative value of simpler
methods.

— Nassim Nicholas Taleb

Excerpted from chapter 9 of


Dance with Chance: Making Luck Work for You

As an expert in statistics, working in a business school during the 1970s, one of the
authors…couldn’t fail to notice that executives were deeply preoccupied with forecasting. Their main interest lay in various
types of business and economic data: the sales of their firm, its profits, exports, exchange rates, house prices, industrial
output…and a host of other figures. It bugged the professor greatly that practitioners were making these predictions without
recourse to the latest, most theoretically sophisticated methods developed by statisticians like himself. Instead, they preferred
simpler techniques which — they said — allowed them to explain their forecasts more easily to senior management. The
outraged author decided to teach them a lesson. He embarked on a research project that would demonstrate the superiority of
the latest statistical techniques. Even if he couldn’t persuade business people to adopt his methods, at least he’d be able to
prove the precise cost of their attempts to please the boss.

Every decent statistician knows the value of a good example, so the professor and his research assistant collected many sets of
economic and business data over time from a wide range of economic and business sources. In fact they hunted down 111
different time series, which they analyzed and used to make forecasts — a pretty impressive achievement given the
computational requirements of the task back in the days when computers were no faster than today’s calculators. They decided
to use their trawl of data to mimic, as far as possible, the real process of forecasting. To do so, each series was split into two
parts: earlier data and later data. The researchers pretended that the later part hadn’t happened yet and proceeded to fit
various statistical techniques, both simple and statistically sophisticated, to the earlier data. Treating this earlier data as “the
past,” they then used each of the techniques to predict “the future,” whereupon they sat back and started to compare their
“predictions” with what had actually happened.

1 of 3 19/07/2009 15:14
The Statistician Who Ate Humble Pie http://www.strategy-business.com/enews/enewsarticle/ac00001?pg=all...

Horror of horrors, the practitioners’ simple, boss-pleasing techniques turned out to be more accurate than the statisticians’
clever, statistically sophisticated methods. To be honest, neither [were] particularly great, but there was no doubt that the
statisticians had served themselves a large portion of humble pie.

One of the simplest methods, known as “single exponential smoothing,” in fact appeared to be one of the most accurate.
Indeed, for 61.8% of the time it was more accurate than the so-called Box-Jenkins technique, which represented the pinnacle of
theoretically based statistical forecasting technology back in the 1970s. The academic journals of the day had proven that the
Box-Jenkins method was more accurate than large econometric models where predictions were based on hundreds of
equations and impressive volumes of data. So, by extension, single exponential smoothing was also more accurate than the
grand-scale econometric models that cost hundreds of thousands of dollars to develop and use!

At first, the professor and his assistant were so taken aback by their results that they suspected they’d made a mistake. They
made extensive checks on their own calculations but could find no errors at all. The initial shock now over, they began to cheer
up. If there’s one thing that makes up for an academic proving himself wrong, it’s the opportunity to show that other eminent
authorities are wrong too. So the professor submitted a paper on his surprising and important findings to a prestigious, learned
journal and waited for the plaudits to start rolling in. This in itself turned out to be another forecasting error. The paper was
rejected on the grounds that the results didn’t square with statistical theory! Fortunately, another journal did decide to publish
the paper, but they insisted on including comments from the leading statisticians of the day. The experts were not impressed.
Among the many criticisms was a suggestion that the poor performance of the sophisticated methods was due to the inability of
the author to apply them properly.

Undaunted, the valiant statistician and his faithful assistant set out to prove their critics wrong. This time around they collected
and made forecasts for even more sets of data (1,001 in total, as computers were much faster by this time), from the worlds of
business, economics and finance. As before, the series were separated into two parts: the first used to develop forecasting
models and make predictions; and the second used to measure the accuracy of the various methods. But there was a new and
cunning plan. Instead of doing all the work himself, the author asked the most renowned experts in their fields — both
academics and practitioners — to forecast the 1,001 series. All in all, fourteen experts participated and compared the accuracy
of seventeen methods.

This time, there were no bad surprises for the professor. The findings were exactly the same as in his previous research.
Simpler methods were at least as accurate as their complex and statistically sophisticated cousins. The only difference was that
there were no experts to criticize, as most of the world’s leading authorities had taken part.

That was way back in 1982. Since then, the author has organized two further forecasting “competitions” to keep pace with new
developments and eliminate the new criticisms that academics have ingeniously managed to concoct. The latest findings,
published in 2000, consisted of 3,003 economic series, an expanding range of statistical methods, and a growing army of
experts. However, the basic conclusion — supported by many other academic studies over the past three decades — remains
steadfast. That is, when forecasting, always use the KISS principle: Keep It Simple, Statistician.

— Spyros Makridakis, Robin Hogarth, and Anil Gaba

Page 1 2 All

This article is from


Click HERE to subscribe to strategy+business

Looking for Booz Allen Hamilton?


Now independent of Booz & Company

strategy+business is published by the global commercial consulting firm Booz & Company.

Magazine || enews || Search and Browse || Press Room || Reprints || Subscriber Services || HOME ||
About s+b || Contact s+b || Advertise in s+b || Privacy Statement || Legal
©2009 Booz & Company. All rights reserved. "booz&co." is a service mark of Booz & Company.
Contact Webmaster || Powered by Raven Creative, Inc.

2 of 3 19/07/2009 15:14
The Statistician Who Ate Humble Pie http://www.strategy-business.com/enews/enewsarticle/ac00001?pg=all...

3 of 3 19/07/2009 15:14

Das könnte Ihnen auch gefallen