Published in PM World Today - July 2007 (Vol.

IX, Issue VII)

Managing Programs to Success:

Key Program Management Metrics
(Part 7 of a Series)

By: Russ Martinelli and Jim Waddell

Parts 1 - 6 in this series available at:

http://www.pmworldtoday.net/featured_papers/2007/july.htm#2

Introduction
Humans know that measures such as their blood pressure, cholesterol, blood sugar, and white blood cells levels
reflect the state of their health. Similarly, program managers know that metrics such as time-to-money,
development cost, gross profit margin and profitability index are measures that reflect the health of their program.
It is commonly understood that one of the key rationales for using metrics is that what gets measured warrants
attention and gets improved. In particular, using program performance metrics will help program managers, their
sponsors, and other stakeholders understand how well a program is performing, where and why a program has
problems and tailor actions to eliminate the problems. This will, in turn, improve the program and keep it on
track toward achievement of the objectives.

Program metrics not only measure the health of individual programs but also show the effectiveness of program
management-related processes, such as strategic management and portfolio management. In this manner,
program management metrics are an effective means to integrate and synchronize strategy, planning, and
execution activities.

In the final paper of this series, we’ll look at how to design the right set of metrics for a program, describe how
strategy drives the metrics selection process, and give some guidance as to how many metrics are needed to fully
comprehend the performance of a program. An extensive set of program management metrics can be found in our
book titled, Program Management for Improved Business Results (ISBN: 0-471-78354-4).

Designing a Metrics Solution

Recent studies cite the use of a comprehensive set of metrics as a key factor in program and project success1.
Programs using comprehensive metrics to measure and monitor performance will have fewer problems, hence
higher success in accomplishment of program goals. Another study found that companies using consistent, tiered,
balanced, and mutually aligned metrics outperformed companies who used sporadic, non-tiered, schedule-
oriented, and non-aligned metrics2. The underperforming companies still favored schedule-only metrics that are
sporadically utilized, normally when a program is in trouble.

The use of program metrics is an institutionalized practice of leading companies. They focus on regular, periodic
measurements of program and business performance. Figure 1 demonstrates the key differences between
companies that methodically design their program metrics set (which we refer to as best-in-class companies), and
those that take an ad hoc approach (which we refer to as rest-in-class companies). Best-in-class companies
experience both a higher rate of program goal accomplishment and produce a higher quality metric set – with
quality being qualified by usefulness3.

A detailed comparison between the best-in-class and rest-in-class companies revealed that the best companies
engineered and installed a system of metrics that were balanced and mutually aligned. Balanced metrics are those
that cover all dimensions of a program including schedule-oriented metrics, as well as those for financial,
customer, process, and human resource utilization. Balanced metrics also include both leading and lagging

PM World Today is a free monthly eJournal. Free subscriptions available at: http://www.pmworldtoday.net Page 1
 Published in PM World Today - July 2007 (Vol. IX, Issue VII)

metrics. Leading metrics are forward-looking, such as a projected program finish date based upon the rate of
milestone completion. Lagging metrics, such as percentage of deliverables completed, are most valuable for a
retrospective view of program performance. Both leading and lagging metrics are important and useful.

Mutually-aligned means that metrics are compatible, using the same baseline information. For example,
performance-to-planned schedule, probability of completing the program by a certain date, and the cumulative
percentage of milestones accomplished are all based on the same baseline – the program schedule.

Source: Program Management for Improved Business Results

Figure 1: Best- versus rest-in-class metrics design

To reflect how metrics vary across firms and industries, successful companies create metric tiers that differentiate
between various types of metrics. Tiering allows the companies to categorize a diverse continuum of programs
and projects, and define specific, as well as common, metrics for each category. For example, one medical
equipment manufacturing company uses three tiers for its research and development programs: basic research
programs (tier 1), technology development programs (tier 2), and product development programs (tier 3).

Metric Selection
Each company will need to define a distinct set of metrics, depending on its business strategy. Therefore, the set
of metrics will vary by the type of industry the company is in and by its particular business strategy - e.g.,
differentiation, cost leadership, or best-cost strategy4.

When a business unit determines its own distinct set of metrics, it should do so in collaboration with company
stakeholders. Each stakeholder will emphasize different metrics according to their needs.

PM World Today is a free monthly eJournal. Free subscriptions available at: http://www.pmworldtoday.net Page 2
 Published in PM World Today - July 2007 (Vol. IX, Issue VII)

• Business management stakeholders will show a strong interest in metrics that assess the strategic health of
the business unit, the alignment of development programs with business strategy, and the balance of the
program portfolio relative to the business unit objectives and needs.

• Program management directors will be concerned with metrics assessing the portfolio of programs and
the management of individual programs.

• Members of program core team and project teams will be most keenly interested in metrics addressing
their specific project management and team execution goals.

The best way to demonstrate strategy-based metric selection is through example. Let’s look at three companies
with three different business strategies. In Table 1, Company A utilizes a differentiation strategy to offer its
customers something different from their competitors. In particular, Company A focuses on technology
innovation and fast time-to-market to achieve differentiation and gain higher profits and market share.
Secondarily, Company A strives to achieve high quality and doesn’t allow runaway cost. To measure performance
to their strategy, Company A selects metrics for measuring time-to-market, percentage of new technologies
developed, profitability index, and market share. Additionally they choose customer fallout and development cost
metrics to measure their secondary strategy of providing high quality and cost containment.

Company A Company B Company C

Company Business High-tech electronics Conventional manufacturing Automobile company IT

Business Strategy Differentiation
Type of programs New product development
Time-to-market
Percentage of new
s
Cost leadership Best-cost
New product development Information technology
First Priority
Development cost Defect rate assessment
Manufacturing cost Customer fallout

technologies developed
c

Profitability index Market share Development cost

i

Market share Staffing level Manufacturing cost

r

Gross profit margin

t

Sales volume
e

Second Priority
Customer fallout Development time Development time
M

Manufacturing cost Defect rate assessment % of new technologies to

company
Development cost

Source: Program Management for Improved Business Results

Table 1: Strategy drives metric selection

PM World Today is a free monthly eJournal. Free subscriptions available at: http://www.pmworldtoday.net Page 3
 Published in PM World Today - July 2007 (Vol. IX, Issue VII)

Company B focuses on a low cost strategy aimed at establishing a sustainable cost advantage over their rivals.
The intent is to use the low cost advantage as a way of under pricing rivals, and capturing market share. This
strategy will leave company B with small profit gross margin per product, but profit can be achieved through a
large volume of sales and lean staffing. Metrics to support this business strategy should measure development and
manufacturing cost, market share, program staffing level, gross profit margin, and sales volume. Secondary
objectives are to bring the product to the market in an average industry standard time and with an average quality.
This is complemented with the second priority metrics of development time and defect assessment rate.

The best-cost strategy pursued by Company C combines upscale features with low cost. Company C aims to
become a low-cost provider of information technology products that have high quality features. A secondary
strategy is to develop innovative information technologies within reasonable time frames. Primary metrics to
measure this strategy should include defect rate assessment, customer fallout, development cost, and
manufacturing cost. Secondary metrics could include percent of new technologies developed and development
time.

These examples illustrate how business strategy drives the choice of metrics. We showed only three example
companies with three different strategies and their corresponding sets of metrics chosen. Every company will
have its own unique business strategy, and each needs to have its own unique set of program management metrics
to measure the effectiveness and achievement of that business strategy.

How Many Metrics Do You Need?

The answer to this question of course depends on the nature of the program. The strategy, culture and values of
the company, as well as how big, complex, technically and commercially risky a program is all play a major role
in deciding how many metrics you need. Additionally, accelerated time-to-money objectives for the program will
require special attention to how the program is managed and what metrics will best assist management in keeping
it on track. As a general rule, one needs a sufficient number of metrics to maintain good control of a program,
and as few as possible to still stay nimble. How many is that for an “average” program?

Some examples may provide guidance. In one successful high tech company, program teams have eight standard
metrics agreed upon with senior management, in another company programs use between 10 and 14 metrics. In a
leading semiconductor company such as Intel, very complex fab building programs evaluate about 20 metrics
each month. A study by BusinessWeek and the Boston Group showed that the sweet spot is somewhere
between 8 and 12 metrics5. Our personal experience supports this finding, we find that about ten metrics are
sufficient if an effective metrics system is constructed.

Conclusion
As we have consistently stated, program management is about achieving business success. Therefore, the purpose
of program management metrics is to help improve the business results of the organization. To serve this purpose,
metrics need to be carefully developed to ensure they are compatible (or mutually aligned), balanced, consistently
utilized, tiered, and are driven by the business strategy of the company.

This completes our seven-part series on program management. Throughout this series, we explored various
aspects of program management that we feel are most important in managing programs to success. We thank
David Pells for the opportunity to create this series of papers, and we hope you have enjoyed reading this series as
much as we have enjoyed writing it! Feel free to contact us with your questions and comments – we’d like to
hear from you.

References

PM World Today is a free monthly eJournal. Free subscriptions available at: http://www.pmworldtoday.net Page 4
 Published in PM World Today - July 2007 (Vol. IX, Issue VII)

1. Hauser, J. and F. Zettelmeyer, “Metrics to evaluate R&D,” Sloan Working Paper #3934, MIT, (October
1996).
2. Milosevic, Dragan Z., R.J. Martinelli, J.M. Waddell, Program Management for Improved Business Results,
Hoboken, NJ: John Wiley & Sons, 2007.
3. Tipping, J.W., E. Zeffren, et al., “Assessing the Value of Your Technology,” Research Technology
Management, Vol. 38, No. 5 (1995): 22-39.
4. Porter, M. E., Competitive Strategy: Techniques for Analyzing Industries and Competitors, 1st
edition. New York, NY: Free Press Publishers, 1998.
5. McGregor, Jena. “The World’s Most Innovative Companies,” Business Week Online (April 24, 2006).

© Copyright 2007 by the Program Management Academy (www.programmanagement-academy.com)

PM World Today is a free monthly eJournal. Free subscriptions available at: http://www.pmworldtoday.net Page 5
 Published in PM World Today - July 2007 (Vol. IX, Issue VII)

Russ Martinelli
Author

Russ Martinelli is the Manager of Program Management

Methods within the Corporate Platform Office at Intel Corporation, where he
focuses on the definition and implementation of program management practices
across Intel. Additionally, Russ is the chairman of Intel’s global Program
Management Community of Practice, an adjunct professor at the University of
Phoenix, and co-founder of the Program Management Academy. Russ has held
a variety of positions at Intel and Lockheed Martin in the areas of systems
engineering, general management, operations management, and project and
program management. Contact Russ at:
mailto:russ.martinelli@programmanagement-academy.com

Jim Waddell
Author

Jim Waddell is an independent consultant specializing in

program management and mergers and acquisitions. He is the
former Director of Program Management for Tektronix Inc. where he established
and led the Tektronix’s first worldwide Program Management Office (PMO).
Additionally, Jim is an adjunct professor at the Oregon Graduate Institute, a
founding member of the Program Management Forum in Portland, and the co-
founder of the Program Management Academy. Jim has held a wide range of
managerial and operational roles ranging across engineering, marketing,
systems and manufacturing in the high tech and energy industries. Contact Jim
at: jim.waddell@programmanagement-academy.com

PM World Today is a free monthly eJournal. Free subscriptions available at: http://www.pmworldtoday.net Page 6