You are on page 1of 3

SPECIAL FEATURE

Forecasting: Academia versus Business


Reprinted from Foresight, Issue 41 (Spring 2016)
SUJIT SINGH

I n a recent discussion on LinkedIn, the question was


raised: Is there a disconnect between the academic
and the business world when it comes to forecasting?
model, resulting in greater complexity and confusion
about the meaning of the results.
Very relevant, then, is the question of whether more
As you can imagine, this is a matter of checking the complex methods actually perform any better than
record and reporting on what one finds in the data. simpler alternatives. There is a body of evidence to
Since I have neither the data nor the resources to the contrary, including the very recent Foresight
conduct this research, though, a comment from me article by Stephan Kolassa (2016), Sometimes Its
amounts perhaps to overreaching. Still, Ill try to Better to Be Simple than Correct.
present what I personally see in the marketplace and A proposed new method should demonstrably
do so (hopefully) without any biases. improve upon existing (and simpler) alternatives,
TYPES OF FORECASTING RESEARCH not just on one specific sample but across a broader
range of data sets.
In my mind, there are four types of research on
forecasting: Behavioral Research
Similarly, behavioral research is ahead of the curve
Statistics/mathematics-focused compared to what is being used in the business world.
- With many subfields, such as time-series fore- It has reached a critical mass whereby almost every
casting and causal modeling other book seems to be touching on how the human
- For the most part, using computer software brain works and how it affects the ways we do things.
Behavioral-focused With this awareness, I see much discussion and some
- Judgmental forecasts and judgmental adjust- accommodation in the process of forecasting.
ments to statistical forecasts
Big Data
- Behavioral biases and silo mentalities
Big-data-based research has come into its own in
Big-data-based the last decade. The essential idea is to incorporate
- A relatively new line of research more detailed (and larger) data sets into forecasting
- Applies to both structured and unstructured data models. A newer variant involves the use of non-
Business performance focused structured data such as comments in tweets, blogs,
- Th
 e impact of better (or worse) forecasting and sections of websites.
practices on the performance of the business Business Performance
- Forecast performance assessment and This fourth type of research is the most relevant to
benchmarking our discussion. You would thinkand probably hope
Statistical Research that this type of research makes the connection
For the most part, statistical research is ahead of between theory and practice and points out the ben-
actual business practice. There appears to be a long efits of good forecasting to the business user. It is in
gap between the publication of research and its first this area that research can provide the most value to
implementation in commercial software. I believe the the business user.
gap here is not only wide but growing wider as more Forecasting deals with the future, and the future is
and more factors are brought into the forecasting inherently uncertain. Forecasters with any reasonable

24 FORESIGHT Summer 2016


training and/or experience recognize that one cannot Can forecast accuracy be correlated with a forecast-
codify uncertainty. As a result, no one really buys into ability score such as the Coefficient of Variation?
the idea that a particular forecast method is perfect. Theoretically, more-forecastable data should make
The focus is put on reducing forecast error as much it easier to get to more accurate results. What are
as possible. the appropriate cutoffs of the said forecastability
I think that traditionally this has been done by com- scores to determine different levels of forecastabil-
paring forecast accuracy before and after a project ity? For each of these levels, what is an achievable
or across other comparative situations. Forecast- degree of accuracy in the forecast?
accuracy improvements are fine as far as they go. What is the expected improvement from collab-
However, the business world is always interested in orative input supplied by people in the field, such
the return on investment (ROI). as sales reps?
ROI is important for many reasons. During project If I measure forecast accuracy at the top level of
approval stages we need to calculate the anticipated my business, are there reliable ways to estimate
ROI in order to win managements approval for the forecast accuracy at the lower levels of my forecast
project. And its often the case that multiple projects hierarchy?
are vying to get access to money from the same pool. These questions elicit great interest from business
Similarly, at the end of the project we need to show folks. Even when practitioners have no interest in
the gains made and the percentage of them that can the details behind the science, they still need to know
be attributed to the forecast improvements. the answers to these types of questions in order to
THE GAPS gain project approval from their management offices.
It is here that I see the biggest gap. The types of ques- I see limited work along these lines coming out of
tions that I see practitioners asking include these: academia. On the other hand, a lot of the work in this
area is done by software vendors but can this work
What is a 10% improvement in forecast accuracy be considered trustworthy? It is my hope that fore-
worth to my business? casting researchers will give more attention to these
- At what level is this 10% accuracy improvement practical aspects of forecasting research.
taking place the overall business level or the
SKU-Customer level?
- At what time lag is this 10% accuracy improve-
ment? One month in advance or three months in
advance for companies with long lead times? Sujit Singh, CFPIM, CSCP,
Chief Operating Officer of Arkieva
Is this the same if the starting forecast accuracy (www.arkieva.com), is respon-
is 20%? 40-60-80-90%? Is the impact easier to sible for managing the delivery
achieve if the starting forecast accuracy is very of software and implementation
low? services as well as customer rela-
tionships and the day-to-day op-
What is the minimum/average/maximum improve- erations of the corporation.
ment one should expect using a commercial soft-
ware package relative to (a) what the organiza- ssingh@arkieva.com
tion is doing currently and (b) a nave (no change)
model?

What is the minimum/average/maximum
improvement one should expect using bottom-up/
top-down/middle-out approaches?

www.forecasters.org/foresight FORESIGHT 25
Copyright of Foresight: The International Journal of Applied Forecasting is the property of
International Institute of Forecasters and its content may not be copied or emailed to multiple
sites or posted to a listserv without the copyright holder's express written permission.
However, users may print, download, or email articles for individual use.