Key Metrics for Monitoring New Product Development

Anthony Lemus
Director, Research and Development
Advanced Sterilization Products)
(a Johnson & Johnson Company)
About the author: With 18 years of experience in new product development, Anthony (Tony)
Lemus has concentrated on refining the product concept creation process for product
commercialization. As a certified Six Sigma Master Black Belt, Mr. Lemus uses process
excellence techniques in portfolio management to measure new product development (NPD)
pipeline health.
Examples of Mr. Lemus’ process implementation in the NPD funnel include a rapid concept
evaluation process, innovation business metrics, incorporation of design excellence into project
deliverables and incorporation of IDweb software into the NPD tracking system.

BRIEF DESCRIPTION OF NPD AT ASP

Advanced Sterilization Products (ASP) was created internally at Johnson & Johnson based on an
innovative sterilization process that uses a combination of hydrogen peroxide gas and plasma.
ASP has approximately 400 employees, with an estimated $300 million in annual sales and over
5,000 systems installed worldwide.

The first commercial STERRAD® system was installed in 1992, followed by

the Cidex line in 1997 and automated endoscope processing in 1999.
Since 1997, ASP has used a phase-gate NPD process, with six process approval phases,
managed by a multifunctional core team. NPD projects require FDA regulatory clearance, which
creates a dynamic project cycle time.

1
In the NPD process maturity model, ASP is situated between portfolio and cross-enterprise
excellence. Our experience has been that we are alternating between these two levels as we
continue to improve our process. There are four steps in the ASP roadmap to achieve NPD
process maturity:
1. Link all deliverables for projects in the NPD funnel to ultimate business-level (top-tier) metrics
(e.g., sales, NPV, ROI).
2. Make business processes global, extending to suppliers and outsourced entities.
3. Use software (IDweb) to monitor metrics, performance and scenarios in real time to support
strategic decision-making.
4. Use Six Sigma as a vehicle for change and to supplement the NPD process to create
process cultural awareness.

Stage 4
Stage 3 Co-
Portfolio development
Stage 2 Excellence
excellence
Stage 1 Project
Functional excellence
Stage 0
Informal excellence
management

Stages of NPD maturity. ASP alternates between portfolio excellence and

cross-enterprise excellence.

© 2003 Pittiglio Rabin Todd & McGrath (www.prtm.com).

Used by permission.

WHAT ASP HAS LEARNED

ASP has gained much experience since its research-driven beginnings. This is nowhere more
evident than in the continuing evolution of our NPD process. Starting with one innovative product
for sterilization that arose from a laboratory environment, ASP has adopted ever more
sophisticated means for evaluating ideas, managing the pipeline, making tradeoff decisions and
bringing products to market. This sophistication—in process, organization and software—is
required to maintain a competitive advantage given the highly data-driven landscape of NPD
today and the realities of the marketplace.

Metrics Derive from Accurate Process Model and Detailed Map

ASP’s ongoing campaign to improve the NPD process has yielded valuable insights and results.
The first step was to model our actual business information process and how we operate
strategically. Then we constructed a detailed NPD process map. With this information, we could
ensure that our metrics and processes as practiced align with company strategy and vision.

2
To refine our process maps and to ensure that each block in the map represents actual business
metrics and NPD deliverables, ASP uses a Six Sigma tool called SIPOC (Supplier–Input–
Process–Output–Customer). This tool reveals the inputs and outputs of the process, so
ownership of deliverables can be established.

Three Tiers of Business Metrics

We have identified key metrics at three levels of our business:
• A basic tier of business drivers
• A mid tier of execution metrics (internal communications dashboard)
• A top tier of business performance metrics (shareholder value)

Top business New product New product NPS as percent New product
metrics sales gross profits of market share quality

Execution Launch New design

metrics Pipeline NPV VOC fidelity
effectiveness sigma

Portfolio Project NPS to R&D Requirements

Project cost
allocation cancellation rate spend ratio compliance

Significant Ideas Changes to PR First-pass Cycle time Milestone

patents (total and adopted) after DEA regulatory approvals by phase attainment

Drivers
Early protocols Prototype parts Design to release
reviews procurement time cycle time

Resource Inside vs. outside

Process compliance Design reuse ECO accuracy ECO cycle time
plan vs. actual development

ASP tracks three tiers of NPD metrics, as exemplified here. Improvements in

driver metrics improve execution and business performance.
To determine driver metrics, product managers at the director level went into the operational
functions, documented how people did their jobs and charted the results on process maps. While
this took six to eight months, it was only then that we could determine what metrics we had. We
learned we were not streamlined, as metrics in one area were not linked to other areas. So we
initiated a Six Sigma project to determine the linkage and do the analysis without interrupting
current pipeline or production output.
We learned that improvements we can make in metric performance at the bottom tier drive
improvements in mid- and top-tier metrics, thereby improving internal communications, decision-
making and financial performance for NPD and the business at large. In other words, the drivers
at the bottom help reach business objectives at the top. Our goal then is to monitor the driver
metrics so we can make timely tactical and strategic decisions to achieve high performance at the
business level.
The mid-tier of business execution metrics tie the top level to the driver level through deliverables.
These deliverables inform management for decision-making, and they also serve as our
dashboard metrics to communicate back to the whole company the status of each project and of
the pipeline. This supports the cultural aspect of our improvement roadmap by giving everyone
insight into how they personally influence the NPD process.

3
Metrics Balancing Act—the Triangle
At the most basic level, the key metrics that ASP tracks for every deliverable are the cornerstones
of the traditional NPD investment tradeoff triangle, with one alteration. The familiar tradeoff
elements are cost, time and quality. We replace quality with resources to better reflect
measurable NPD investment components. Though the quality element is displaced in this
predictive model, we monitor and maintain product and process quality using Six Sigma tools,
such as design scorecards.

Cost Time REG plan BOM MRP PAC binder

Resources Early protocols Prototype parts Design to release

reviews procurement time cycle time

Resource Inside vs. outside

Process compliance Design reuse ECO accuracy ECO cycle time
plan vs. actual development

QA Plan BOM MRP Gantt MFG plan Design spec DCC plan

Each driver generates a deliverable (oval) that has measurable cost, time, and
resources associated with it (investment triangle). Bottom-up: Real or simulated changes
in process steps, deliverables or triangles for any project ripple up through higher-order
metrics to alter business results. Top-down: ASP manages toward desired business
results and solves pipeline issues by focusing on and managing interplay of deliverables
at the driver level.
Each NPD process block represents a deliverable. Cost, time and resources must be accurately
proportioned in a triangle for each deliverable and at each business tier in order to realize the
best return in the NPD pipeline. The investment triangles allow us to accurately predict how
various bottom-tier drivers will affect the deliverables and hence affect the top-tier business
results.
In fact, with IDweb software, we can instantly see, in easy-to-read graphic representations, the
likely results of what-if scenarios and tradeoff decisions. For example, we can see how shrinking
the time on one deliverable affects the resources and costs of another deliverable and the impact
that will have on business results.

Metrics Cause and Effect

Central to our method is how we link NPD metrics to all deliverables and then track and score
them throughout the process for our entire project portfolio. We also maintain awareness of the
shape (not equilateral) of each deliverable’s metric triangle, as well as account for the causal
relationships among the metrics, even as the information changes from day to day, project to
project and triangle to triangle.
Knowing the cause-and-effect relationships lets us quickly answer any questions raised at the top
business tier by tracking back to the relevant driver information at lower tiers. However, tracking
all the causality among the many sets of metrics greatly increases the data flow, making it more

4
difficult to organize and monitor the information. In the past, ASP attempted to accomplish this by
hand, which was very labor intensive if not impossible. Now we use IDweb software to do it in real
time.

MANAGEMENT AND LINKAGE TOOLS

IDweb Software
Once we used our Six Sigma tools to chart the flow of our business, we realized we had too much
data to analyze on paper; therefore we implemented IDweb software from IDe (www.ide.com).
IDweb, with its ability to manage complex NPD data and present it in simple graphic
representations, is what enables the process I have described. For example, IDweb can track the
resource, time and cost legs of the metric triangles.
Because we embed business metrics in the drivers, IDweb allows us not only to know the current
status of projects and investment actions but also to play out what-if scenarios and predict
outcomes for proposed or alternative actions, while answering tough questions such as: How
many projects can we put in the pipeline before we run out of resources—or money? What
projects should be in the pipeline? If we have a great new idea and also something similar
approaching commercialization, should we stop the advanced project and start the new one?
What are the risks and tradeoffs? Given the projected market size for the project, how do we
optimally execute, or should we cancel the project? Given the critical importance of resources,
what are IDweb gap analysis charts telling us about resource bottlenecks?

IDweb gap analysis charts quickly identify resource bottlenecks, allowing ASP
to adjust project start dates, shift resources among projects or initiate other tradeoffs to
balance investment triangles at the driver level.
IDweb is what links all the information, updates the metrics, integrates the scorecards and
manages and makes accessible the right information in real time, so we can make timely
decisions based on accurate information.

Six Sigma
We use Six Sigma to supplement our NPD process with tools that produce quantifiable metrics
for decision-making. The Six Sigma philosophy creates a process-driven culture that aligns
business deliverables with execution strategy. For example, Quality Function Deployment (QFD),
a customer vs. design requirement mapping tool, is used to create top-level strategic cost–benefit
analysis of customer needs. QFD outputs a list of prioritized business driver metrics required to
meet customer needs. This list is then deployed in the NPD process to create deliverables such

5
as product requirements, component specifications, forecast models, QA and manufacturing
protocols and commercialization plans.
Six Sigma also inherently emphasizes the Voice of the Customer (VOC), which is paramount for
the success of an NPD product business result. We look at the number of product requirement
changes that occur in the development process, which drives costs and customer satisfaction.
Then we look at the variance between the actual and expected customer value based on product
requirements. If we exceed expectations, we enjoy a positive variance. If we miss, we suffer a
negative variance and we will not experience the full profit potential of our new product.

IDEA MANAGEMENT FOR NPD

Idea management is a critical and complicated activity. Every company wants to capture good
ideas, to not miss any and to realize their full profit potential. But companies can't execute on all
ideas because of NPD funnel capacity. Many ideas are killed early because selection filters stop
them. Additionally, ideas must fit with the corporate vision and strategy. Triangular relationships
must be aligned and balanced at all phases, because each phase calls for resources, time and
money to execute the ideas. IDweb allows us to track these relationships early, between the idea
and feasibility planning stages.
We provide incentives to employees for submitting a product idea that makes sense for our
business. If the idea survives all the way through the NPD process, incentive payments are based
on the size of the profitability potential or the value of the improved process.
Initially, we have a market screen and a technical screen. The market screen looks at the
business merit of the idea in terms of company vision. Does it have linkage? Are there objectives
in the idea that relate to our business that we should look at? Technical merit is a measure of
whether we can actually do the project and of whether the core science is legitimate. We use
QFD here. We do a scientific robustness study, we do a patent study, and we look at potential
consumables. With the QFD matrix, we score the idea on the business scorecard and route it to
the appropriate funnel.

6
 New Product Development -- Pipeline
5

NPD ScorCard index

3

Idea Feas n Plan Development Validation Commercial

0 1 2 3 4 5
Project Phases (Bubble Size: )
(Bubble Color: )

Innovation Portfolio -- Pipeline Process Excellence -- Pipeline

2 4
NPD ScorCard index

BalancedScor index
3

0 1

Idea Concept Define Measure Analyze Innov Improv Control

0 1 2 0 1 2 3 4 5
Project Phases (Bubble Size: ) Project Phases (Bubble Size: )
(Bubble Color: ) (Bubble Color: )

Sustaining and Support -- Pipeline

21-Aug-2004
21-Feb-2004
23-Aug-2003
22-Feb-2003
24-Aug-2002
23-Feb-2002
Project planned start date

25-Aug-2001
24-Feb-2001
26-Aug-2000
26-Feb-2000
28-Aug-1999
27-Feb-1999
29-Aug-1998
28-Feb-1998
30-Aug-1997
01-Mar-1997
31-Aug-1996
02-Mar-1996 Identify Feas n Plan Development Valid&Pilot Normal Ops
0 1 2 3 4 5
Project Phases (Bubble Size: )
(Bubble Color: )

ASP maintains a parking lot for good ideas (left), and then routes ideas into
the appropriate funnel (NPD, Process Excellence, or Sustaining and Support) when both
strategic and tactical considerations (investment triangles) are aligned and favorable.
Then we get into the business scorecard, where we undertake a more intense market merit
review. Here, we spend money doing focus groups and visiting current and new potential
customers. This is where we make sure the project is feasible before more substantial money is
committed. If the idea is plausible, but we can't undertake the project because of pipeline
capacity, we complete the market analysis and then put the project in our “parking lot.”
Ideas that pass the business scorecard screen are then routed into either the NPD funnel for a
new product, to the Six Sigma funnel for a process improvement or to the operations engineering
funnel for a product improvement. We use IDweb to manage the NPD location of a project as
dynamic business factors change.

7
Once a project is declared feasible, we use Six Sigma process maps and IDweb to track its
progress. We aim to continually improve our processes so that, once we decide to go, 80 percent
of ASP projects will reach commercialization.
Idea management and project filtering must be fact-based and in alignment with a company’s
vision and NPD strategy; so each company must develop this for itself. We use our business
opportunity scorecard, loaded into IDweb, to produce a weighted score called the business
opportunity index, to be used for management decisions.

SUMMARY
Since our research-driven beginnings, ASP has continually improved the NPD process. Detailed
process mapping revealed three tiers of business metrics, starting with drivers and moving up
through execution metrics to business performance metrics. Improvements in driver metrics result
in improved business performance. Six Sigma tools helped identify specific metrics at all tiers and
the causality linkages among them.
Each NPD process step at ASP results in a deliverable, and each deliverable relates to a tradeoff
triangle (cost–time–resources) and, ultimately, investment dollars. To track performance, ASP
calculates how actual NPD decisions and what-if scenarios affect deliverables, alter investment
triangles and thereby influence business results. ASP uses similar process and tools, including
the investment triangles and Six Sigma QFD, to make idea management decisions. The IDweb
real-time software environment enables both the NPD and idea management processes at ASP.