Sie sind auf Seite 1von 27

Measure what is measurable and make measurable what is not so

Galileo Galilei(1564-1642)

Nashville

Measuring Team Productivity


specifically for Support teams

using

Oregon Productivity Matrix

An experience paper on improving PPQA team productivity by


Srinath Ramasesh
Wipro Quality Consulting Group

Fundamentals
|

The Oregon Productivity Matrix (OPM in short) takes multiple metrics, uses a eight step process* and subjective weighting system to rank/prioritize the metrics. The metrics are then added up to give a cumulative composite score This single index indicates how well the company/organization/team is doing against the performance metrics

*as we implemented it.

Theme
| | | | | | | | | |

Preamble - on Productivity Productivity in Support Teams -Really? Background The team which implemented OPM OPM as a solution OPM History - and advantages The OPM Grid - What is OPM? The Eight steps to Implementation The core details Avenues to improving OPM Lessons Learned Q&A

Preamble
|

Productivity is a major concern in an IT industry, especially so for support teams (or non-coding jobs)
z z z

PPQAs (Process and product Quality Assurance) Project Management Office Administration

Productivity by definition
z

"Goods and services produced per unit of labor and expense The relationship between the inputs required to produce a product or service, and the value of the output produced

or
z

Goods + Services Labor + Expenses


z

In a software development activity, this is relatively easy to calculate. We could measure code productivity as Function points or lines of code developed per unit effort or cost 4

Productivity of Support Teams


|

Inputs are fairly self-evident: staff, equipment, costs, etc


z

but what of the outputs?

What exactly do support teams produce?


z z z

Multiple deliverables, Non periodic deliverables, Multiple tasks,

Measured by multiple parameters Example:


z

An accounts department, for instance, will produce a variety of reports, some regular and some adhoc They work on several tasks simultaneously, with constant interruptions, and its impossible to apportion the total input to each

Can Productivity for a support team be clearly established? Is it possible to measure the value driven by all these activities and deliverables? Is it possible to measure improvement collectively?
5

Its Possible!!!
|

The productivity of support teams can be measured!!!,


z

ing s U

Changes up and down can be monitored allowing managerial action to be taken to ensure that improvement takes place

Oregon Productivity Matrix

Goods + Services Labor + Expenses

Volume + Value Efficient Resource Consumption

The Hypothesis:
Anything that increases the quantity and/or quality of output, without a corresponding increase in the utilization of resources, creates a gain in productivity *

This presentation covers an experience of implementing a research based methodology called Oregon Productivity Matrix with a Process and Product Quality Assurance (PPQA) team in a leading automobile company in North America
*Productivity by the Objectives Matrix Oregon Productivity Center

Background
|

The PPQA group at an IT division of a leading Automobile company assisted projects in process verification and validation activities The teams key activities included
z z z z z z

QA

Process validation, Issue of phase approvals, Conducting configuration management audits, Reviewing work products, Mentoring projects, Facilitating root cause analysis meetings with Project teams

The Challenge:
z

With multiple type of activities and multiple measurement parameters, the teams challenge lay ahead in identifying a single indicator for measuring teams productivity, showing continual improvement, and establishing a performance measurement system

The Solution
|

The team agreed that the solution was to establish a concrete method for measuring the PPQA teams productivity, particularly as it relates to the team's continuous improvement indicators Based on a thorough literature search we evaluated that Oregon Productivity Matrix (OPM) best fits this scenario In addition, we had experiences in the team of successfully implementing OPM in an operations management area to indicate a single performance indicator of all operations

OPM History
|

Developed by the Oregon Productivity Center at Oregon State University in 1986, (Productivity By the Objectives Matrix (OMX) Handbook-1986)
z

the OPM was first introduced at Boeing with a pilot team in 1990 (Defense and Space facility, Corinth)

Similar to Balanced score card* with an exception that the OPM can be implemented at a shop floor level without getting into strategic perspectives of the organization Has been applied in various industries. Is immensely versatile and adaptable to various applications in any setting like:
z z z z z z z z z z z

Hardware Network productivity IT Office productivity (administration)- Service Sales productivity - Fabrication Production control- Food Processing Purchase-Fabrication Accounting -Electronics Human Resources Bottling Planning and Development- Government Police Government Team productivity- Aircraft Foreclosures- Mortgage Company

*Balanced Scorecard is method developed by Kaplan and Norton that enables organizations to clarify their vision and strategy and translate them into action

Advantages of OPM
|

Flexible
z z

Goals can be agreed to be adjusted based on continuous improvement Allows for multiple measurement categories where baseline and current performance levels are depicted A valuable method for standardizing team performance by the use of an index that can be uniformly interpreted throughout the organization Can be used in a variety of work environments to establish a performance measurement system Weights are established for each of the performance areas, thus focusing the team on the most important areas to improve Goals are set by the teams Teams are the users and take ownership May be rolled up to an organizational level with multiple OPMs OPM graphically displays team performance against goals

Standardized
z

Versatile
z

Prioritizes
z

Team oriented
z z

| |

Rolled-Up
z z

Visual

10

The OPM Grid


Point Score
Goal

Categories

Goal

10.00 Performance Category 1 Performance Category 2 Performance Category 3 Performance Category 4 Performance Category 5

9.00

8.00

7.00

6.00

5.00

4.00

3.00

2.00

1.00

0.00

Limits

Point

Weight

Total

100.0

99

98

97

96

95

93

91

87

83

82

94%

25

100.0

98

96

94

92

90

88

86

84

82

80

100%

15

100.0

99.5

99

98

97

96

94

92

90

88

85

98%

10

100.0

99

98

98

97

96

95

94

93

92

90

97%

30

100.0

99.5

99.0

98.5

98

97

96

95

94

92

90

98%

20

1 2 3 4

Performance Category Goal Limits Weights Point Score Table

OPM may be seen as a grid in which the different Performance categories (which are monitored using quantitative metrics data) are depicted vertically along the grid. Goal Limits are identified for each category. Weights each of the Performance Category are established to identify importance. A Point Score table is used to earn points of performance. 11

When Rubber Hits the Road


(Step 8) Analysis (Step 7) Arrive at OPM Index (Step 1) Identify Team Goals

8 7
OPM

(Step 6) Identify Scores for Actual Performance

6 5

2 3

(Step 2) Define &Validate Performance categories

(Step 5) Establish Performance indicators

(Step 4) Prioritize Categories by assigning weights

(Step 3) Set Performance Goal Limits 12

8 7 1

Identify Team Goals


| z z z

OPM
5 4 3

Identification of the team goals is the first activity towards establishing any performance measurement system
Understand expectations from management Align the goals with Business goals and objectives Brainstorm with team

What are we trying to achieve?

G1 G2 G3

Achieve consistency in PPQA team performance Change perception of PPQA from inspectors to mentors Improve process compliance performance in projects
13

8 7 1

Define and Validate Performance Categories


| |

OPM
5 4 3

Use the Team goals to arrive at performance categories for monitoring The PPQA team used the Goal Question Metric (GQM) paradigm * as a method to identify the measurement categories or performance categories Identify Performance categories that indicate what the team does

Team Goals

G1

G2

G3

Questions derived from goals that must be answered in order to determine if the goals are achieved Measurement Categories that provide the most appropriate information for answering the questions

Q1

Q2

Q3

Q4

Q5

Q6

Q7

Performance Category1 (Metric1)

Performance Category 2 (Metric 2)

Performance Category 3 (Metric 3)

Performance Category 4 (Metric 4)

Performance Category 5 (Metric 5)

Performance Category 6 (Metric 6)

A family of measures thus created can reflect the work of the team in a way no single measure can.
*GQM method was originated by V.Basili and D.Weis

14

8 7 1

Define and Validate Performance Categories


|

OPM
5 4 3

Performance Category

Validate that these categories encompass the range of inputs used: labor, materials, capital and equipment, and the output factors of quantity, quality, timeliness and cost Validate that each performance category is positive driven and is measured in percentage
z

Metric Performance Categories*


Phase approvals OnTime CM Audits On-Time Number of approvals issued on time/Total number of approvals issued *100 Number of CM Audits conducted on-time/Total number of CM Audits Planned *100 % Compliance score based on Defects found from review of QA Reports in a month Overall Score Improvement Index: Total compliance questions from all projects/Total applicable questions in month for all projects expressed in %age Number of Meetings conducted in the month /Scheduled meetings*100

QA Report Quality

i.e. each metric is measured for compliance and not defects or non compliance

Improvement Effectiveness

Meeting Compliance

*Note: these are samples and do not cover all categories for a PPQA function

15

8 7 1

Set Goal Limits


|

OPM
5 4 3

1 2

Performance Category Goal Limits

Assess Current level of Performance


z z

Historical data to generate the initial OPM Conduct pilot

Goal Limits

Performance Categories
Phase approvals On-Time CM Audits On-Time PPQA verification Run Quality Improvement Effectiveness Meeting Compliance 94% 100% 98% 97% 100%

| |

Review and analyze with team Establish the goal limits for each performance category with team consensus

This step is useful in itself, as people and work groups should have targets for quality of service so that they know the standards of work that they are expected to achieve.

16

8 7 1

Prioritize the Categories


|

OPM
5 4 3

1 2 3

Performance Category Goal Limits Weights

Assigning importance weights (iw) for each Performance Category based on the value of the output summing up to 100 How to assign weights for a performance Category?
z

Performance Categories
Phase approvals On-Time CM Audits On-Time PPQA verification Run Quality Improvement Effectiveness Meeting Compliance

Weight
25 15 10 30 20

z z

Weighing by Expertise Weighing by Compromise Weighing by Consensus

Take expert judgment based on experiences and alignment to business objective Take an simple average based on a survey May be jointly established with the team and management representatives to ensure alignment to the Business value.

100
(Analysis of the following for each category provides the key to decide the weight ) o Who uses the services or deliverable ? o What is value of this delivery? o How is performance of each of the services/ deliverable affected? o What is impact of wrong service or deliverable?

Aligning the teams service more closely to business goals and objectives will promote significant productivity gains by eliminating unnecessary work.

17

8 7 1

Establish Performance Indicators


Point Score 10.00 100.0 9.00 99 98 99.5 99 99.5 8.00 98 96 99 98 99.0 7.00 97 94 98 98 98.5 6.00 96 92 97 97 98 5.00 95 90 96 96 97 4.00 93 88 94 95 96 3.00 91 86 92 94 95 2.00 87 84 90 93 94 1.00 83 82 88 92 92 100.0 100.0 100.0 100.0 | |

OPM
5 4 3

1 2 3 4

Performance Category Goal Limits

0.00 82 80 85 90 90

Weights Point Score Table

May be achieved by establishing a point score table Point score table as seen in the figure is a range of values fragmented horizontally in varying percentage fragments between two limits and each percentage indicating a certain point earned in a scale of 0 and 10 The lower limit of this target range is the minimum level of service that can be permitted or expected called as the minimum expected performance score (meps). The maximum range or the upper limit is fixed at 100% (Optionally, the maximum scale can be the goal score also) Intermediate values are determined for scores between these ranges based on past experiences and adjusted based on pilots

The targets goals levels may be set for each performance area either in terms of in terms of percentage or point score values.

18

8 7 1

Categories

Establish Performance Indicators


Point Score
6.00 96 92 97 95 98 95 95 95 97 5.00 94 90 96 93 97 94 94 94 96 10.00 9.00 99 98 99.5 99 99.5 98 98 98 99.5 8.00 98 96 99 97 99.0 97 97 97 99 7.00 97 94 98 96 98.5 96 96 96 98 4.00 93 88 94 91 96 93 93 93 95 3.00 91 86 92 89 95 91 91 91 94 2.00 87 84 90 86 94 89 89 89 93 1.00 83 82 88 83 92 87 87 87 92 0.00 82 80 85 80 90 85 85 85 90 meps
Goal Limits Point

OPM
5 4 3

Goal
Weight Total

Phase approvals On Time CM Audits On-Time PPQA Verification run Quality Improvement Effectiveness Meeting Compliance Performance Category Performance Category Performance Category Performance Category

100.0 100.0 100.0 100.0 100.0 100.0 100.0 100.0 100.0

94% 100% 98% 97% 98% 95% 95% 95% 97%

5 10 7 6 6 6 6 6 6

5 5 10 35 20 1 2 2 20 100 (iw)

25 50 70 210 120 6 12 12 120 625

GOAL INDEX

| |

Establish a grid with performance categories, point score table goal limits and assigned weights The goal limits of each performance category corresponds to a point score. The product of point score and weight gives the total goal scores.
E.g. : z in the first row: 94% goal limit corresponds to 5points z Point score (5) X assigned weight (5) = 25 ( Total goal limit score for category Phase approvals On time)

The sum of point scores based on the individual goal limits gives a single number that is an aggregate, weighted index called Goal Index 19

8 7 1

Record Actual Scores


OPM Measurement Report
Month1

OPM
5 4 3

Actual Scores
Goal

Ver 4 Performance Score

Categories
10.00 Phase approvals On Time CM Audits On-Time PPQA Verification run Quality Improvement Effectiveness Meeting Compliance Performance Category Performance Category Performance Category Performance Category 100.0 100.0 100.0 100.0 100.0 100.0 100.0 100.0 100.0 9.00 99 98 99.5 99 99.5 98 98 98 99.5 8.00 98 96 99 97 99.0 97 97 97 99 7.00 97 94 98 96 98.5 96 96 96 98

Point Score
6.00 96 92 97 95 98 95 95 95 97 5.00 94 90 96 93 97 94 94 94 96 4.00 93 88 94 91 96 93 93 93 95 3.00 91 86 92 89 95 91 91 91 94 2.00 87 84 90 86 94 89 89 89 93 1.00 83 82 88 83 92 87 87 87 92 0.00 82 80 85 80 90 85 85 85 90 meps
Goal Limits Point

Actual
Total

Weight

Actual 83% 98% 97% 95% 91% 97% 98% 95% 88%

Points 1 9 6 4 1 8 9 6 0

Perf Weight Score 5 5 10 35 20 1 2 2 20 100 5 45 60 140 20 8 18 12 0

Max Possible 50 50 100 350 200 10 20 20 200 1000

94% 100% 98% 97% 98% 95% 95% 95% 97%

5 10 7 6 6 6 6 6 6

5 5 10 35 20 1 2 2 20 100 (iw)

25 50 70 210 120 6 12 12 120 625

308

The actual measurement scores for each of the performance categories are recorded and translated as points earned from the point score table
z

OPM INDEX

This was done using a specific reporting and consolidation template in a spreadsheet designed to do the number crunching of all the measurement attributes and arrive this table.

| |

The Performance score for each performance category is then a weighted score of the point The sum of all performance category point scores gives a single number that is an aggregate, weighted index called OPM index
E.g. : z in the first row: 83% Actual score of Phase approvals On-Time corresponds to 1 point earned from the table z Point score (1) X assigned weight (5) = 5 (Total performance score for the category Phase approvals On time) z Sum of all performance scores = 308 = OPM Index for the Month 1

20

8 7 1

The Moment of Truth


PPQA OPM Dashboard
1000 900 800 700 600 500 400 300 200 100 0
STRETCH GOAL LINE
650 685

OPM
5 4 3

Calibrate
(yearly)

(Optional)

635 485 395 308 318 285 410 415 515 515

645

685

GOAL LINE

Calibrate
(initial)

Oct-04 Nov04

Dec- Jan-05 Feb-05 Mar- Apr-05 May- Jun-05 Jul-05 Aug04 05 05 05

Sep- Oct-05 Nov05 05

Dec05

| |

The OPM index trend indicates the Productivity line and Goal index gives the Goal Line. The OPM indices (productivity line) may be compared to the Goal line to measure performance and demonstrate improvement in productivity

21

8 7 1

Analyze OPM
|

OPM
5 4 3

Initial ( Pilot Analysis)


z z

Calibrate OPM based on pilot Adjust goal limits, point scores or weights Team analyses to measure productivity improvements Causal Analysis and discussion on improvement areas Elimination of assignable causes Root cause analysis to identify process improvements Calibration and Program corrections

Monthly
z z z

Yearly
z z

Questions/Symptoms
Change in scope of activities? Do we have the right measures? Performance goal is consistently not achieved Consistent 100-percent-of-goal performance on a performance category (or becomes the current level of performance)

Actions Recommended
Perform GQM Review performance categories Review Goal limits Review Goal Limits, Point Score, Weights Re evaluate performance category

22

Avenues to Improving OPM


|

OPM is a process completely driven by the team from selecting the performance measurement areas to actually improving them. Implementation successes may be replicated at other teams in the organization The point score table may be embellished with more objective sub- criteria of performance (0-10 levels) It is possible to combine all the team OPMs into a single OPM for the entire organization Once piloted and proven, build it as a part of your quality system

23

Lessons Learned
| | |

| | |

Increases team ownership for improvement (Process, Product, Productivity) Leave it to the practicing team to analyze and improve Initially coming to consensus with goals and getting acceptance from all team members may be slow, but its worth the time. The process accelerates upon team agreement and involvement Use it as a process to improve the performance areas and not as a end in itself Be open to continuous improvement based on root cause analysis Productivity is better measured by overall value delivered per unit effort/cost than by quantity delivered per unit effort/cost

24

CAUTION !
|

DO NOT use OPM to tie the results to any form of reward system (or worse penalize system)
z z

This does not yield good results in long term The team may not necessarily focus on the right performance categories for improvement

DO NOT use OPM to foster competition amongst teams


z

The objective is to improve not to prove

DO NOT use the tool to identify who is the problem


z

Let the team decide where is the problem and improve it

25

References
|

OSU Archives - Oregon Productivity Center Records (RG 219)


z

OMX Handbook

| |

Productivity by Objectives : By James L. Riggs, Glenn H. Felix The Use of Performance Metrics as a Strategic Management Tool Terry R. Collins, Manuel D. Rossetti and Julie A. Watson paper, University of Arkansas Rose, K. (1995) A Performance Measurement Model.. Quality Progress, V28, N2, 63-6 R.Solingen, E.Berghout: "The Goal/Question/Metric Method" McGrawHill Publishing Company, 1999

26

The primary objective of any process improvement activity is to improve and not just prove

Thank you
srinath.ramasesh@wipro.com 586-354-4813

Das könnte Ihnen auch gefallen