Capability Maturity Model (CMM)

for Software Development

Geir Amsjø
 What do a customer want?

Customers wants:
Predictable performance
• On time deliveries
• Costs according to budget
• All functionality
• Reliable promises
Visibility of progress and quality

SW-CMM 2
 Crisis-Driven Organizations

Processes:
• ad hoc - improvised and reinvented each time
• abandoned under stress
• reactionary
Schedules and budgets:
• unpredictable
• usually exceeded
Success factors:
• heroes
• overtime
• fire fighting
Blaming others: bad subcontractors, hardware problems...

SW-CMM 3
 Executive Problems in Software

Low visibility into progress and quality

Unpredictable performance
Constant surprises
Excessive costs

Difficult to plan for company strategies

SW-CMM 4
 Project Manager Problems

Hard to give accurate estimates

• under-scoped requirements
• schedule and resource shortfalls
Unmanaged work
• no corrective action
• unruly subcontractors
Volatile baselines
• changing requirements
• uncontrolled versions

SW-CMM 5
 What Frustrates Developers

Being given half the time they need to do the work

Continual overtime
Guessing what requirements means
Contending with uncontrolled changes in requirements
Finding they are not working with the current version
Redoing builds forever
Delivering products they are not proud of
Fixing old defects without design documentation
Constantly patching up old software that has no architecture

SW-CMM 6
 Sources of Crisis

Software development specifics:

Product complexity
Conformity – everything is possible
Continuous changes
Invisible product
Explosive code growth
Defect removal costs
Culture

Partly from Brooks

SW-CMM 7
 Cost (i.e Effort) to Remove Defects

Cost/Effort

1-2 orders of magnitude

Cost to repair

Time/phase

SW-CMM 8
Improvement Areas

Technology

Product

People Process

SW-CMM 9
 The Role of the Process

Management

Environment Disciplined Methods

Process

Staff
Technical assets
Reference: Bill Curtis, TeraQuest Metrics

SW-CMM 10
 Definition of Software Process

”…Software Process can be defined as a set of activities, methods,

practices, and transformations that people use to develop and maintain
software and the associated products (e.g., project plans, design
documents, code, test cases, and user manuals).”

Definition from SEI technical report SEI-93-TR-24, ”Capability Maturity Model for
Software, Version 1.1”

SW-CMM 11
 Potentials in Process-Based Development

Documents our common experience

• alive and updated
• learn from our mistakes
Promotes teamwork
• defines roles, interfaces and dependencies
Promotes individual training
• needs of the company and the individual
Defines activities and output
• nothing neglected nor superfluous

SW-CMM 12
 Misconceptions About Process

I do not need process if I have

• really good people
• advanced technology
• an experienced manager
Process interferes with creativity
Process = bureaucracy
Process hinders agility in fast-moving markets
Process is only useful on large projects

SW-CMM 13
 What is an Effective Process?

Defined

Documented

Used

Supported by management (enforced)

Trained

Measured/Known in quantitative terms

Supported by tools and technology

Tailored

Improved/maintained

SW-CMM 14
 The Process is a Part of the Competition
Process is a competitive weapon in almost every industry except software
development
Our software process decides quality, cost and cycle-time
Software Development
• Historically not considered to be engineering

“The quality of a software system is governed by the quality of the process used to
develop and evolve it.”
Watts Humphrey (Father of the CMM)

SW-CMM 15
 Internal Delivery Precision

Deviation between Max/Min of 12 Month Rolling Average

% 105 5
100
95 4

90

CMM Level
3
85
80
2
75
70 1
65
60 0
92/01

92/05

92/09

93/01

93/05
93/09

94/01

94/05
94/09

95/01

95/05

95/09

96/01

96/05

96/09

97/01

97/05

97/09
TG2 Rolling Average TG2 Min Roll Ave
TG2 Max Roll Ave CMM Level

SW-CMM 16
 Fault Density, First 6 Months
Fault Density, First 6 Months of Operations
(Plotted against PR-A Date)
Fault/KPlex
5
1.50
4

CMM Level
1.00 3

2
0.50
1

0.00 0
91-01
91-06

91-11
92-04

92-09
93-02

93-07
93-12

94-05
94-10
95-03

95-08
96-01

96-06
96-11

97-04
97-09
ETM/R Failure 12 mth Rolling CMM Level
Goal Density avarage

SW-CMM 17
 The Start of CMM

Millions of dollars wasted for DoD, Ada etc didn’t solve all problems.
Management problem, not technical!
Watts Humphrey, then at IBM, had an idea!
SEI and MITRE Corporation began the development late 1986
Brief description - September 1987
Managing the Software Process by W. Humphrey- 1989
CMM Version 1.0 - 1991
Version 1.1 - 1993
Integrated CMM (CMMI) released in 2001

SW-CMM 18
 Software-CMM by SEI:

“The Capability Maturity Model for Software (CMM) is a framework

that describes the elements of an effective software process.

The CMM describes an evolutionary path

from an ad hoc, chaotic process,
to a mature disciplined process”

SW-CMM 19
 Capability Maturity Model Objectives

CMM is TQM with a software view, is based on state-of-the-art SW

Engineering practices and helps organizations:

Characterize the maturity of their process

Establish goals for process improvement
Set priorities for immediate actions
Envision culture of software excellence

SW-CMM 20
 Improvement Challenges

The process cannot be continuously improved if:

• Sound engineering practices are sacrificed to schedule
• There is no feedback on process performance
• Each person does something different
• Wide variation occurs in performing identical tasks
• Commitment to improve is not organization-wide

The CMM
• Overcomes these hurdles one by one

SW-CMM 21
 CMM Design Rationale

Level 5 - Continuously improve process performance

Level 4 - Control process variation

Level 3 - Install a common process

Level 2 - Stabilize the environment

SW-CMM 22
 Maturity Framework

Continuous Level 5:
process
improvement Optimizing

Level 4: Change
Process Managed management
control

Process
Level 3:
Quantitative
definition Defined management

Process Level 2:
Engineering
discipline Repeatable management

Level 1:
Project
Initial management

SW-CMM 23
 CMM Architecture

Level Focus Key Process Areas

Optimizing Continous process Defect prevention
(5) improvement Technology change management
Process change management
Managed Product and process Quantitative process management
(4) quality Software quality management
Defined Defined engineering Organisational process focus
(3) process Organisational process definition
Integrated software management
Software product engineering
Intergroup coordination
Training program
Peer reviews
Repeatable Project management Requirements management
(2) and commitment Software project planning
process Software project tracking
Software subcontract management
Software quality assurance
Software configuration management
Initial Heroes
(1)

SW-CMM 24
 Key Process Area (KPA)

Key process area:

Important component of behavior at a maturity level

Collects a number of related activities
The the lower Key Process Areas will evolve as we reach higher
maturity levels.

There are 18 Key Process Areas in the CMM

SW-CMM 25
 Key Process Area Goals

Key process area goals:

Enhance the capability of the processes related to the KPA
Requirements for the maturity level

Are used to guide:

Process assessments
Alternative practices to fulfill a KPA

The most important description of a KPA!

SW-CMM 26
 Key Practices

Practices

Maturity Levels Key Process Area Goals Practices

Goals Practices
Key Process Area

Goals
Practices

Practices

SW-CMM 27
 Key Practices

Implements the goals of the KPA:

Describe what to accomplish

Does not describe how to accomplish it
Neither exclusive, nor exhaustive

There are 316 key practices in the CMM

The Key practices are organized in Common Features

SW-CMM 28
 Common Features

Goals

Key Practices:

Commitment Ability Activities Measurement Verification

Institutionalization Implementation

SW-CMM 29
 Common Features

Common features:
Organize the key practices
Supports institutionalization of process performance

It is important that the organization:

Commits to perform - Commitment
Supply the capabilities for performing - Ability
Performs in a repeatable way - Activities
Measure how well it performs - Measurement
Follow-up and ensures that it performs - Verification

SW-CMM 30
 Implementation vs. Institutionalization

Note that:

Institutionalization leads to implementation

Implementation does not have to lead to institutionalization

Do not forget the Institutionalization Key practices

SW-CMM 31
 Commitment to Perform

Actions to ensure that the organization is committed to the activities in the KPA:

Responsibilities are assigned

Policies for the organization supports the activities
Senior management sponsors the activities

SW-CMM 32
 Ability to Perform

Preconditions for performing the activities in the KPA:

Providing resources
Providing training
Establishing the necessary organizations (i.e. new roles and
responsibilities)

SW-CMM 33
 Activities

Tasks that implement the goals of a KPA:

Planning of activities
Performing activities
Tracking activities
Taking corrective actions
Informing about activities and results

SW-CMM 34
 Measurement and Analysis

Ensure that the activities are measured and analyzed:

Measure effectiveness and degree of implementation

Analyze to take corrective actions

SW-CMM 35
 Verifying Implementation

Ensure that the activities are performed according to:

Commitments
Policies
Processes
Documented procedures

SW-CMM 36
 The Structure of CMM - Summary

Commitment

Ability

Maturity Levels Key Process Areas Goals Activities

Measurement & Analysis

Verification

SW-CMM 37
 The Repeatable Process

Basic project management installed

Projects are planned and tracked
Changes to plans and baselines are managed
Good practices are observed

Commitments are made and approved

Successful projects can be repeated in a stable managed

environment

Process capability exists for meeting schedules

SW-CMM 38
 Responsibilities at Level 2

State expectations for project behavior

Motivate - set incentives
Organization Allow the process to be followed

Responsible for change

Establish disciplined management
Projects
Control baselines

Participate in making commitments

Keep to agreed upon standards and requirements
Individual

SW-CMM 39
 Moving to Level 2

Focus on projects - one by one

Train Project Managers - skill by skill
Implement measurements - do not complicate
Institute Quality Assurance - project support
Institute baseline control - all deliverables
Agreement

Manage Commitments

SW-CMM 40
 The Commitment Process

A pact freely assumed, visible and expected to be kept by all

parties.
• based on realistic plans
• includes all affected parties

A commitment process is
Participation increases commitment the core of level 2

making - reviewing - approving

commitments

SW-CMM 41
 The Repeatable Level

5. Optimizing

4. Managed

3. Defined

2. Repeatable Requirements management

Software project planning
Software project tracking and oversight
1. Initial Software subcontract management
Software quality assurance
Software configuration management

SW-CMM 42
 Requirement Management

customer software
project

SW-CMM 43
 Why Requirements Management?

Establish - between customer and SW project

a common understanding
agreement on requirements
maintain understanding and agreement

The agreement
covers both technical and non-technical requirements
is the basis for estimating, planning, performing and tracking

xxxxx
xxx
xxxxx

SW-CMM 44
 Requirements Management Goals

1. System requirements allocated to software are controlled to establish a

baseline for software engineering and management use.

2. Software plans, products, and activities are kept consistent with the system
requirements allocated to software.

SW-CMM 45
 Typical Problems

Requirements are volatile

• customer wants changes and additions
• customers don’t fully understand their needs/requirements
• required functionality originally under-scoped
• technology and competition evolve
Customers don’t want to document requirements
Software developers suffer from creeping elegance
• implement more functionality than required
• solution more elegant than required
Commitments made before scope is understood

SW-CMM 46
 What is Requirements Management

Software development

requirements design implementation

gather analyze/
specify
manage change

verify xxxx
xxx
xxxx.

understand - describe - agree

SW-CMM 47
 Maturity framework

Continuously improve Level 5:

process performance Optimizing

Level 4:
Control process variation Managed

Level 3:
Install a common process Defined

Stabilize the Level 2:

environment Repeatable

Level 1:
Initial

SW-CMM 48
 Level 3 - the Defined Process

Standard software processes are

• defined
• documented
• applied throughout the organization
Shared understanding of how the process works and each
person’s role is established
Software engineering process group created to focus on
process improvement
Process capability to meet schedule, cost, and functionality
targets exists within established product lines

SW-CMM 49
 Moving to Level 3

At Level 2 the focus is on projects

At level 3 the emphasis shifts to the organization:
• best practices are gathered from across the organization
• standard processes are developed for the organization by
integrating these best practices
• processes are tailored for use on projects
Organization process capability
• Meet schedule, cost and
functionality targets within
established product lines Level 3:
Process
definition Defined

Level 2:
Engineering
Repeatable management

SW-CMM 50
 Problems to avoid on Level 3

Bureaucratic - the same process everywhere

Overhead - tailoring is extra work
Few best practices - nothing collected
Too complicated - processes governing not supporting
Over-documentation - tries to document
everything in processes

SW-CMM 51
 What is Defined?

Organization's Library of software Organization's Guidelines and Specification of

Software process process related Approved criteria for organization's
database documentation software tailoring the standard
life cycles organization’s software process
standard software
Project1 1 process Software process
Project Architecture
Project 1
Size
Size
Size
$$$
$$$
$$$
Defects
Defects
Defects
Results
Results
Results
Lessons
Lessons
Lessons
Specifications of Software
process elements
•Developed from best practices
•Tailored for each project
•Consistent work products
•Comparable measurements
•Transfer of learning Paulk (1995)
•Coordination of improvement
SW-CMM 52
 Level 4 - the Managed Process

Statistical process control principles are used to address special

causes of process variation
Detailed process and product data are available - quality targets
are set
Process is capable of performing within narrowly defined
quantitative limits - targets are predictable
Detailed measures of the software process and product quality
are collected
Both the software process and products are quantitatively
understood and controlled using detailed measures

SW-CMM 53
 Moving to Level 4

At Level 3 systematic measures have been defined and collected

At Level 4 decisions are made based on data analyses

Projects are quantitatively managed and

process and quality variation are reduced

Level 4:
Process Managed
control

Level 3:
Quantitative
Defined management

SW-CMM 54
 What is Managed?

Control process variation

• statistical process control
• remove special causes of process variation
Quantitative software management
• establish quantitative goals
• manage process performance
Special cause of process
variation (e.g. defect
prone module)

Control Chart
Upper performance boundary

Average process performance

Process performance
Lower performance boundary

SW-CMM 55
 Level 5 - the Optimizing Process

Chronic causes of poor performance are identified and eliminated.

Software process is continually improved
New technologies are prototyped, piloted, and if successful, introduced into the
process
Process capability is continuously raised
Continuous process improvement is enabled by quantitative feedback from the
process and from testing innovative ideas and technologies

SW-CMM 56
 Moving to Level 5

The organization's focus shifts from special to common causes

• At Level 4 process and quality variation are reduced
• At Level 5 the average process and quality levels are continuously improved
Continuous improvement becomes a way of life
• At the lower maturity levels worker participation on continuous improvement
may be on the order of 20-30%
• World-class companies have 70-80% participation in improvement
programmes at any given time
The organization institutionalizes
change management

Continuous Level 5:
process
improvement Optimizing

Level 4: Change
Managed management

SW-CMM 57
 What is Optimizing

Continuous process improvement

• Eliminate chronic causes of poor performance
• Continuous evaluation of processes and technologies
• Individuals empowered to improve their processes
Chronic cause of
inefficiency
eliminated
(e.g. a new
Special req. process)
cause

Initial
capability Upper performance boundary

Improved
capability
Lower performance boundary

SW-CMM 58
 Management Visibility by Level

In Out
Level 5

Level 4 In Out

Level 3 In Out

Level 2 In Out

Level 1 In Out

SW-CMM 59
Process Capability by Level

Probability
target

Probability
target

4
Probability

target

3
Probability

target

2
target
Probability

1
SW-CMM 60
 Changing the Organization

Organization
establishes
Organization
process
framework
y

Tr
lit

us
i
ab

t
St

Management Project
establishes reduces
Project
process process
discipline variation

Ad hoc, Individual
Inconsistent improves
Individual
undisciplined personal
process process

SW-CMM 61
 Cultural Changes by CMM Level

Level 5 Culture of empowerment and innovation

Level 4 Culture of performance excellence

Level 3 Culture of engineering

Level 2 Culture of commitment

Level 1 Imported culture

SW-CMM 62
 Experiences - Raytheon’s
Equipment Division
Raytheon’s Equipment Division, USA;
• 400 software engineers
• real-time embedded radar and air traffic control
Started with process improvements 1988
Level 2 1990
Level 3 1992
Level 4 1995

Dion (1993), Haley (1995)

SW-CMM 63
 Raytheon’s Cost Analysis Method

Crosby’s cost of quality model:

Performance - cost of building it right the first time
Nonconformance - cost of rework
Appraisal - cost of testing
Prevention - cost of preventing nonconformance
Average% project time by cost type:

Perform Nonconf Appraisal Prevent

1988 37 41 15 7
1990 55 18 15 12
1992 66 11 23
1994 76 6 18

SW-CMM 64
Cost of Quality

SW-CMM 65
 Return on Investment

Invests $1.000.000 annually

Reports an ROI of 5,7 times per year
7.7 times in 1990

Received a $9.600.000 bonus for early delivery on one system (not included in ROI
numbers!)

SW-CMM 66
Better quality

SW-CMM 67
Higher productivity

SW-CMM 68
Predictability

SW-CMM 69
 SEI data and experiences

CBA IPIs and SPAs conducted since 1987 through June 2002 and
reported to the SEI by July 2002:

2325 assessments
• 1840 CBA IPI
• 485 SPA
1756 organisations
512 different companies
451 reassessed organisations
9632 projects

Kilde: http://www.sei.cmu.edu/sema/pdf/2002aug.pdf

SW-CMM 70
 SEI Industry Analysis results

Benefit Area Companies Annual result

Productivity growth
Pre-test defect detection
Time to market
Field error reports
Cost of improvements
Return on investment
4 37% increase
3 18% increase
2 19% decrease
5 45% reduction
5 $481.000
5 5.7 * investment

Herbsleb et al 1994

SW-CMM 71
Characteristics of successful organizations

Senior management actively monitors SPI progress

Clearly stated, well understood SPI goals
Staff time/resources dedicated to process improvement
Clear, compensated assignment of responsibility
SEPG staffed by highly respected people
Technical staff is involved in improvement

(Bill Peterson (SEI), E-SEPG, 1996)

SW-CMM 72
Characteristics of less successful organizations

Turf guarding
Organizational politics
Cynicism from previously unsuccessful improvement experiences
Beliefs that SPI gets in the way of real work
Need more guidance on how to improve, not just what

(Bill Peterson (SEI), E-SEPG, 1996)

SW-CMM 73
 More info

A collection of articles and papers mainy by Mark Paulk:

http://www.sei.cmu.edu/activities/cmm/cmm.articles.html

SW-CMM 74