A test of B2B sales forecasting methods

A Test of B2B Sales Forecasting Methods

July 2012

Share this white paper!

2012 Nimble Apps Limited

A test of B2B sales forecasting methods

Table of contents
1 Introduction
....................................................................................................................................

2 Scope and methodology ................................................................................................. 2

2.1.Scope ........................................................................................................................................................... 2 2.2.Methodology ............................................................................................................................................. 2

3 Analysis of sample

.................................................................................................................

3.1.Closing dates are always optimistic .............................................................................................. 2 3.2.Losing takes longer than winning ................................................................................................... 3

4 Forecasting methods ........................................................................................................... 3

4.1.Description ............................................................................................................................................... 3 4.2.Results ........................................................................................................................................................ 4

5 Conclusion ........................................................................................................................................ 6

Share this white paper!

A test of B2B sales forecasting methods

1 Introduction
Sales forecasting is a major issue for B2B companies. On one hand, B2B companies often lack the thousands of data points that statistical forecasting techniques require. But on the other hand, recent research by Aberdeen Group shows a clear link between forecasting best practices and sales performance. The implication is obvious: robust, B2B-specific forecasting methods would change the life of sales managers. This white paper describes the test of common and not-so-common B2B forecasting techniques we recently performed on a sample of SalesClic client data. Our research yields a number of confirmations and a few surprises.

Share this white paper!

A test of B2B sales forecasting methods

2 Scope and methodology

2.1.Scope Our sample data - representative of B2B companies selling complex products and services was made up as follows: We measured forecasting errors over the test periods using their root mean square, normalized by the average amount of opportunities in the sample. Measure of forecasting error

12 sales teams The teams were located in the US, UK and Asia
The teams operated in the software, electronic  equipment and financial services industries  he teams managed structured sales pipelines T (i.e. following stage-by-stage sales processes) Over the research period, we totaled 144,817  closed opportunities

3 Analysis of sample
Before discussing the accuracy of the forecasting techniques included in the test, it is worth noting 2 interesting patterns in the sample data. 3.1.Closing dates are always optimistic Initial closing dates are optimistic for 10 teams out

The sales cycles were from 75 to 250 days

2.2.Methodology Training and test periods We divided the historical data of these 12 teams into training and test periods for the selected algorithms. The training period is always twice as long as the test period, with a minimum of 1.5 years, an average of 4.5 years and a maximum of 9 years. Training periods contained at least 500 sales opportunities.

of 12 in our sample. On average, it takes 22% longer than initially expected to win an opportunity for the sample teams. That is worrying in a B2B context, where sales forecasts are very sensitive to closing dates. For sales managers and sales operations managers, monitoring closing dates is a clear priority.

Share this white paper!

A test of B2B sales forecasting methods

4 Forecasting methods
120%

Closing date error

4.1.Description The following describes the 8 forecasting techniques we tested.

100%

80%

60%

Weighted pipeline #1 is a simple weighted pipe line using declared opportunity amounts, closing
Sales cycle length (days)

40%

20%

dates and closing probabilities  eighted pipeline #2 uses declared opportunity W amounts and closing dates but historical closing probabilities  eighted pipeline #3 uses declared opportunity W

0%

50

100

150

200

250

300

-20%

Figure 1 - Closing date error x Sales cycle length

The bias increases with the length of the sales cycle, as illustrated by figure 1 above. In our sample, we find a closing date error of 16% for the team with the shortest sales cycle, and of 109% for the team with the longest sales cycle. 3.2.Losing takes longer than winning In our sample, two-thirds of closed-lost opportunities are lost after the closing date initially expected, and losing an opportunity takes an average 1.7 times longer than winning one. Stagnation in the pipeline does not bode well for pending opportunities, and B2B companies have much to gain from detecting stuck opportunities as early as possible.

amounts, historical opportunity time-to-wins1 and historical closing probabilities Weighted pipeline #4 is a variation of weighted  pipeline #3, using declared opportunity amounts, historical stage durations2,3 and historical closing probabilities  e also tested most combinations of the 4 W weighted pipelines methods mixing declared and historical inputs The linear predictors assume a linear rela tionship between stage amounts on day d and closed-won amount on day d+n The decision tree predictors are sophisticated  algorithms using the decision tree technique. Our trees are grown and pruned on stage amounts The daily closing rate method assumes that,  until the end of the forecast period, a team will close the same daily amount as during the last n days (see example p. 5)

1. The time to win of an opportunity is the average time required to win opportunities that have reached the corresponding pipeline stage. 2. The duration of a pipeline stage is the average time that opportunities spend in that stage. 3.  Time to win for pipeline stage n and the sum of stage durations for stages n to i usually differ because of early losses, stage jumps and back-and-forth opportunity movements.

A test of B2B sales forecasting methods

We classified these techniques according to 2 criteria that are very relevant to CRM software users: Whether they use declared or calculated closing dates and closing probabilities 

How computer intensive they are

Closing date Weighted pipeline #1 Weighted pipeline #2 Weighted pipeline #3 Weighted pipeline #4 WP combinations Linear predictors Decision tree predictors Daily closing rate Declared Declared Calculated Calculated Both Both None None Closing probability Declared Calculated Calculated Calculated Both None None None Computer intensive Low Medium Medium Medium Medium High High Low

Figure 2 - Classification of forecasting methods

4.2.Results Judgmental forecasts are not reliable The simple weighted pipeline forecasting technique (declared amounts, declared closing dates and declared closing probabilities) is the second worst performing in the sample. This research thus confirms what most sales managers already know: simple weighted pipelines cannot be trusted. Leveraging historical data helps Traditional CRM software is unable to leverage properly the historical data of sales teams for optimization purposes. Our research shows that replacing sales rep and manager judgment on closing dates and closing probabilities with historical averages increases forecast accuracy.  eighted pipeline #3 is 7% more accurate than W weighted pipeline #1 Weighted pipeline #4 is 35% more accurate than  weighted pipeline #1 This means that sales teams are sitting on a huge amount of forecasting information they could be using to inform their judgments.

Share this white paper!

A test of B2B sales forecasting methods

Sophistication pays... up to a point Averaging time-to-wins, durations and closing probabilities is a straightforward way to leverage historical data. How do the more sophisticated techniques tested here perform? Linear predictors dont work very well.  In particular, they are outperformed by weighted pipeline #4. This is disappointing but not surprising since sales pipelines can have widely different shapes, and linear equations are ill equipped to deal with such irregularities. Decision trees perform well. Compared to the  simple weighted pipeline, they increase forecast accuracy by an average 46%. However, they are quite hard to implement.

And the winner is... As shown by figure 3 (below left), the best forecasting technique on our sample data is a simple but nimble one: the daily closing rate. Here is an example of how it works: Suppose your team has closed 300K  over the past rolling 3 months

That is a daily average of 3.3K

 uppose that you are 30 days S into the current quarter (1/3 of the quarter)...

...and that you have closed 50K so far

 our historical daily closings forecast for the Y quarter is: 50K + 3.3K x 60 days = 250K This method improves simple weighted pipeline

Accuracy improvement
Weighted pipeline #1 Daily closing rate Decision tree predictors4 Weighted pipeline #4 WP combinations Linear predictors5 Weighted pipeline #3 Weighted pipeline #2 Reference point 53% 46% 42% 20% 11% 7% -8%

forecasts by 53%. Forecast accuracy also increases by 20% compared to weighted pipeline #4.

Figure 3 - Performance of forecasting methods vs. simple weighted pipeline

4. This is the average of 3 decision tree predictors all 3 tightly grouped around this average. 5. This is the average of 6 linear predictors. The best one improves forecast accuracy by 21% over weighted pipeline #1.

A test of B2B sales forecasting methods

5 Conclusion
This research suggests immediate ways for B2B companies to improve sales forecast accuracy. Measure pipeline dynamics (opportunity time-to-wins, pipeline stage durations,  closing probabilities by pipeline stage) and use that information for forecasting purposes.  alculate your daily closing rate and use that information for forecasting purposes. C Implementing forecasts based on decision trees is also a good idea, although potentially complicated. For additional progress, we believe that moving from the analysis of a pipelines macro structure (pipeline stages essentially) to a pipelines micro structure (the behavior of individual opportunities) is required. Nimble Apps will continue to study and share insights on these topics.

About Nimble Apps Limited

Nimble Apps is the publisher of SalesClic, a simple and powerful solution for visualizing, analyzing and forecasting your sales pipeline. SalesClic integrates with Google Apps, Highrise and Salesforce.

Follow SalesClic on
6