Performance Metrics

CPS 181s
April 10, 2003
Should Performance Be
Measured?
Metrics
Assessing the progress and health of on-line businesses
Financial progress: sales, margins, profit, market share
Strategic metrics require entire examination of the company:
perception of value proposition, marketplace offering,
effectiveness of implementation
Branding and implementation metrics focus on
> supply-chain performance
> organizational dynamics
> marketing communication effectiveness including branding
Senior and stock market analysts tend to have a bias
toward financial metrics
provide history of the company in the marketplace
do not reflect the strategy of the company
Managers need early warning system metrics


Should Senior Managers be
Concerned About Metrics?
Organization metrics represent performance targets of
the company
Percent increase in revenue
Percent increase in unique visitors
Length of time visitors remain on site
Cost of customer acquisition
Metrics valuable as employee reward or taking strategic
action
Change processes
Change strategy
Change product strategies

Metrics Drive Behavior in a
Number of Ways
Help define the business model
Concrete goals with precise measurements
Focusing on measurement can increase the precision of the value
proposition
Help communicate strategy
Document performance targets
Communicates strategy to workforce
Creates buy-in to the metric-setting processes
Help track performance
Instantaneous feedback concerning site performance real-time
Usage, visitors, length of time on site, average sales, page views
Metrics Drive Behavior in a
Number of Ways...
Help increase accountability
Metrics must be linked to the performance appraisal system
Individual performance can be appraisals tied to companywide,
team-specific, and individual metrics
Site usability can be tied to the interactive design function
Customer service metrics can be tied to the customer service
department
Help Align Objectives
Align precise objectives, departmental functional goals,
companywide strategic activities as a whole
Current Challenges to Specifying
Metrics for Online Businesses
Companies Strategies Change Rapidly Internet time
requires rapid change
Measurement is resource intensive
Setup and maintenance of capital intensive/human resource
intensive systems and procedures
Data capture, data mining, and information use require time and
commitment of senior executives
Online measurement systems are vulnerable easy to
manipulate final results

Current Challenges to Specifying
Metrics for Online Businesses...
Soft metrics are not valued by the investment community
Investment community looks at revenues, margins, number of
visitors, length of time on site, customer acquisition costs
But, ease of customer use, best value perception, and other market
research measures are frequently early warning indicators of site
performance
Meaningful metrics change on Internet time often as quickly as
the time it takes to learn best how to interpret the retained original
data

What Should We Measure?
How Can We Assess the Health
of eCommerce Firms?...
The Balanced Scorecard
Introduced by Kaplan and Norton
Finance balanced by other domains of the business
Includes internal business processes and customer responses
Four categories
> Financial
> Customer
> Internal business systems
> Learning and growth

Kaplan-Norton Balanced Scorecard

FINANCIAL
To succeed financially, how
should we appear to our
shareholders?
INTERNAL BUSINESS
PROCESS
To satisfy our shareholders and
customers, at which businesses
must we excel?
CUSTOMER
To achieve our vision, how
should we appear to our
customers?
LEARNING AND GROWTH
To achieve our vision, how will
we sustain our ability to change
and improve?
Vision
and
Strategy
Source: Kaplan, Robert S. and David P. Norton. 1996. Using the Balanced Scorecard as a Strategic Management System. Harvard Business Review 74 (January-February): 76.
K-N Financial and Customer
Metrics
Financial Metrics
Revenue, revenue growth, gross margins, operating income,
net margin, earnings per share, cash flow
Reflect recent planning period , to some degree all previous
planning periods
Customer Metrics
Management relationship of customers by the firm
Market share
Customer acquisition
Customer satisfaction
Customer profitability
Customized by the target segment
K-N Internal Business Process
Metrics

Innovation includes customers perception of
innovativeness of the company
Operations quality of entire supply
Postsale service quality being offered to
customer

K-N Learning and Growth
Metrics
Employee
Selection
Training
Retention
Satisfaction
Information Systems
Quality of infrastructure
Timeliness, accuracy, and utility of data
Motivation
Empowerment
Alignment of company and personal goals
Limitations to the K-N
Balanced Scorecard
No clear definition of strategy or business
models
Unclear location of organizational
capabilities or resources in framework
Unclear where partnerships reside in
framework
The Rayport-Jaworski Performance
Dashboard and Strategy Framework
The Performance Dashboard - intended to reflect the
health of the business
The Strategy Framework Drives the Necessary Metrics
Six critical steps
> opportunity assessment
> business-model metrics
> customer-interface design
> market communications and branding
> implementations
> financial

Introducing the R-J Performance Dashboard

Financial
Metrics
Steps of Strategy:
Customer Interface
Implementation
and Branding
Performance
Dashboard:
Areas
Addressed:
Size of market
opportunity
Attractiveness of
target segments
Competitive
landscape
Customer acquisition
Changes in customer
behavior
Site experience and
usability
Branding
Technology
infrastructure
Internal organization
Relationship with
suppliers and
partners
Fulfillment capability
Revenue
Profit
Cost
Balance sheet
Market Opportunity
Assessment
Business Model
Implementation
& Branding
Metrics
Customer
Interface &
Outcome Metrics
Business Model
Metrics
Opportunity
Metrics
Uniqueness of value
proposition
Attractiveness of
offering
Firm capabilities
relative to
competition
Sustainability of
competitive position
Evaluation
Components of the R-J
Performance Dashboard
Opportunity Metrics
Ability to discern unfulfilled needs in the market place is critical
Market-opportunity metrics asses the degree to which the firm
can accurately gauge the market opportunity
Business-model Metrics
Capture subcomponents of the business model: egg diagram,
resource systems, financial metrics
Value Proposition or Cluster Benefits Metrics
> target segments, benefits offered, capabiliti4s that drive benefits
> focus on customers perceptions
> performance relative to the competitors benefits

R-J Branding and
Implementation Metrics
Focus on the supply-chain performance,
organizational dynamics, marketing
communication effectiveness (including
branding)
Fulfillment of the brand process metrics:
customer awareness of the brand
Innovation metrics: ability to rapidly innovate,
even if in a potentially discontinuous fashion
R-J Customer-Interface
Metrics

Customer-interface and customer-outcome metrics
Customers perception of the firms performance on
each level of the 7 Cs of the customer interface
Example: customers rating of customization compared to
competitors
Other critical interface metrics
Including value proposition
R-J Customer-Outcome
Metrics
Capture process measures that the firm will produce
favorable customer responses such as satisfaction
and loyalty
Focus on subjective customer-outcome metrics
satisfaction
length of stay at site (stickiness)
overall evaluation of the experience
Focus on objective customer-outcome metrics
customer acquisition costs
average order size
customer profitability
number of visits per month

Financial Metrics
Revenues
Costs
Profits
Balancesheet metrics

Lise Buyer
Life Cycle of a Company

STARTUP/
BETA
CUSTOMER
ACQUISITION
MONETIZATION MATURITY
Develop a
platform for rapid
growth by
building a strong
team and
creating a flexible
site.
Build market
share as quickly
as possible by
aggressively
spending on
partnerships and
promotion.
Increase
revenues and
customer lock-in
by developing
new revenue
streams.
Control costs and
optimize
marketing
expenditures to
achieve profitable
growth.
6 Months- 1 Year 1 Year - 2 Years 2 Years - 5 years > 5 Years
Strategy
Business.com
TNBT.com
mValue.com
Paypal.com
Evite.com
Yahoo.com
Amazon.com
Schwab
AOL
Examples*
* At time of writing (3/2000)
What are the Steps to Implement
the R-J Performance Dashboard?
Step One: articulate business strategy
market opportunity assessment
business model
customer-interface design
branding
implementation
evaluation

Steps Two and Three
Step Two: translate strategy into desired outcomes
not settling target levels but specifying outcomes
Step Three: devise metrics
identifies specific metrics that reflect the desired
outcomes
look-to-book ratios

Bill Gurley on the Power of
Conversion Rates

Conversion Rate 2% 4% 8%
Advertising Spend 10,000 $ 10,000 $ 10,000 $
Visitors 5000 5000 5000
Transactions 100 200 400
Cost/Transaction 100 $ 50 $ 25 $
Revenue 10,000 $ 20,000 $ 40,000 $
Marketing/Revenue (%) 100% 50% 25%
Average transaction size = $100 Source: J William Gurley
Source: Gurley, J. William. 2000. The Most Powerful Metric of All. CNET News.com, 21 February.
URL: http://www.news.com/Perspectives/Column/0,176,403,00/html?tag=st.ne.
Steps Four and Five
Step Four: link metrics to leading and lagging
indicators
map entire set of indicators
including leading & lagging indicators
Step Five: calculate current and target performance
current conversion rates
advertising expenditures
degree of positive word of mouth

Blueprint to the R-J Performance Dashboard

Step One:
Articulate
Business
Strategy
Step Two:
Translate
Strategy Into
Desired
Outcomes
Step Three:
Devise
Metrics
Step Four:
Link Metrics
To Leading
And Lagging
Indicators
Step Five:
Calculate
Current And
Target
Performance
D
e
f
i
n
e

g
o
a
l
s

a
n
d

v
a
l
u
e

p
r
o
p
o
s
i
t
i
o
n

Market Opportunity
Opportunity size?
Competitive environment?
Customer
How to acquire customers?
How will customers change?
The customer experience?
Implementation and
Branding
How to develop brand?
How to go to market?
Financial
Financial consequences in
terms of revenue, profit, cost
and balance sheet?
Market Opportunity
Market size and growth
Average age and income
Competitor concentration
Customer
Market share
Purchases / year
Success rate
Service requests / customer
Implementation
Customer brand awareness
System uptime percentage
Number of IT staff
% inaccurate orders
Financial
Revenue
Profit
Earnings per share
Debt to Equity ratio
For each metric,
determine the metrics
that it affects and that
affect it
Map the linked set of
metrics, indicating
leading and lagging
indicators
Ensure that there is a
balance between
leading and lagging
indicators
D
e
v
e
l
o
p

r
e
s
o
u
r
c
e

s
y
s
t
e
m

r
e
q
u
i
r
e
d

t
o

d
e
l
i
v
e
r

t
h
e

s
t
r
a
t
e
g
y

For each metric,
calculate current level
of performance
Determine target
level required to meet
outcomes described
in Step 2
Ensure that targets
are consistent with
each other
Business Model
Unique value proposition?
Capabilities vs. competition?
Business Model
Customer perceived benefits
Exclusive partnerships
$ invested in technology vs
competition

What are the Three Sources of
Metrics Information That Firms Can
Use to Chart Their Progress?
Industry standard metrics
Benchmark values
On-line Information

Online Information
Market research primary customer data
Analysts reports blending market data
Financial information income statement, balance sheet,
statement of cash flow
Use of Bizrate use of dimensions of service
Use of Forrester Reports e-commerce reports
competitors, consumers, technology evolution in a segment
of the industry
Use of Hoovers Online financial and market information for
both online and offline companies

U.S. Top 25 Web & Digital Media Properties (March 2000)

Source: Media Metrix (www.mediametrix.com/usa/press/releases/20000424.jsp) March 31, 2000
Rank Digital Media/Web Unique Visitors (000)
1 AOL Network 59,858
2 Yahoo Sites 48,336
3 Microsoft Sites 46,581
4 Lycos 32,899
5 Excite @ Home 28,571
6 Go Network 23,006
7 NBC Internet 17,169
8 Amazon 15,217
9 Time Warner Online 13,636
10 Real.com Network 13,482
11 Go2Net Network 13,041
12 AltaVista Network 12,557
13 About.com Sites 12,329
14 Ask Jeeves 12,269
15 eBay 11,155
16 LookSmart 10,557
17 ZDNet Sites 10,226
18 CNET Networks 10,023
19 eUniverse Network 9,198
20 JUNO/JUNO.COM 9,177
21 EarthLink 8,526
22 Infospace Impressions 8,305
23 Viacom Online 8,139
24 FortuneCity Network 7,809
25 CitySearch-TicketMaster
Online
7,689
Market Research Source - Bizrate.com

Analyst Source - Forrester

Financial Information Source - Hoovers Online

Mapping Internet Research Onto The Performance Dashboard

Media
Metrix
AC
Nielsen
BizRate Forrester Gomez Jupiter
Creative
Good
Hoovers
Market Info
Traffic
Fulfillment
Implemen-
tation
Privacy
Usability
Content
Customer
Satisfaction
Customer
Service
Financial
Performance
Market
Implementation
Customer
Financial
Market Research Analyst
Financial
Information