Human Error in

Maintenance and
Reliability and What to Do
About It
By:
Jack R. Nicholas, Jr., P.E. CMRP
 Human Error in Maintenance and
Reliability and What to Do About It
Long term strategy & set of tactics for
error reduction
Simple analytical method for
determining where to take action to
eliminate errors
Learning to listen for error reduction
Polices & practices to get cooperation
Systematic root cause analysis
technique that focuses on human error
in M & R
 Human Error in Maintenance and
Reliability and What to Do About It
Fundamental truths learned over 50
years of involvement in leadership
positions in Maintenance & Reliability

Simple concepts for error reduction

explored and several key references
provided

Concepts are interrelated

 Concept #1 - Exercise and
practice leadership as well as
management.
Management functions
z Providing direction, objectives and goals through
effective (two-way) communications,
z Obtaining (and keeping) the resources needed for
people to do their jobs most effectively and
efficiently,
z Removing impediments to reaching the ultimate
objectives of the organization,
z Adding or removing constraints that are needed to
keep the organization focused,
z Thoroughly understanding, and constantly refining
the processes that the organization has to execute
to serve its customers.
z Providing leadership
 Concept #1 - Exercise and
practice leadership as well as
management.
Many managers arent leading

Much time spent on organization and re-

organization a waste of time

Should be concentrating on business

process development and refinement

Emphasis on customers (and other

stakeholders)
 Leadership & Processes
Peter Drucker
z Managements New Paradigms
z Drucker on Leadership
Michael Hammer
z The Agenda: What Every Business Must Do to
Dominate the Decade
Stephen J. Thomas
z Improving Maintenance & Reliability Through
Culture Change
Harvey & Lucia
z 144 Ways to Walk the Talk
 Concept #1 - Exercise and
practice leadership as well as
management.
Learning to listen
z Listen to those in positions above (who
must serve all below)

z Listen to co-equal persons along side you

z And (most important) listen to employees

in positions below yours (those you
serve).
 Bad parenting or
poor leadership?

Source: PeoplePC Online

 Concept # 2 Look first at
programmatic rather than
technical solutions to reliability
problems.
Programmatic root cause analysis
developed in aftermath of 1979 Three
Mile Island incident involving reactor
core meltdown
USNRC didnt have full legal authority
to regulate commercial nuclear plant
maintenance activities until
Maintenance Rule went into effect in
mid 1990s
Combining Leadership & Process
Analysis and Refinement
Engage key employees in technical and
first line supervisory positions of the
organization in developing processes
z Remove impediments,
z Add needed and remove unnecessary
constraints
z Obtain resources needed to make the
processes work
z Make processes most effective and efficient
(clean and clear) in actual practice.
z Obtain buy-in but also create a sense of
ownership which is more important
Programmatic Root Cause Analysis
Four Queries
z Training

z Procedures/Documentation

z Quality Control (QC)

z Management
 PRCA Queries
The queries focused on isolating such
root causes of inadequate maintenance
performance as:
z Line and upper management performance
(including their attitude towards their
responsibility for craftsperson performance)
z Procedures and documentation (both the
product and the process),
z Training (both delivery and the process),
z Managers of such program elements
z Quality control (always a co-cause, not a
primary cause)
 Results of Application of PRCA
Although technicians performing
maintenance were not eliminated as root
causes of defective maintenance, their
inadequate performance was found to
be most likely the effect rather than the
root cause of subsequent (infant or
premature) equipment failures

Management was found to be both a

primary as well as a co-cause of
inadequate maintenance by technicians.
 Results of Application of PRCA
Majority of root causes of infant or
premature equipment failures could be
eliminated or at least mitigated and
reliability improved less expensively and
more rapidly by programmatic solutions
than by technical (re-design) solutions

Finding concerning managements

direct involvement in equipment failures
is seldom, if ever applied outside the
commercial nuclear power industry
 Deciding What to Do Next

Technical or Programmatic Root

Cause?
Photo Source: PeoplePC OnLine
 Application of PRCA
Being considered for application (along with many
other initiatives) to the British Petroleum (BP)
Refinery at Texas City, TX
z Fatal accident on 23 March 2005
z 15 deaths and a much larger number of serious
injuries
z Cost not only to BP stakeholders but to consumers
nationwide in the form of higher fuel prices
In actual practice management and many other
programmatic causes of failure are considered co-
incidental, if not prohibited, areas of investigation
requiring corrective action
PRCA Has Wider Implications
The rash in 2006 and 2007 of childrens toys
and costume jewelry items containing Lead
having to be removed from store shelves
z Partner companies share responsibility
z Regulatory agencies, Congress share responsibility

Findings of contamination in food imports in a

large number of products from several
countries, that got heavy media attention
during 2007
z May be traced to the lack of U.S. Department of
Agriculture and/or customs inspectors at ports of
entry and in U.S. processing plants resulting from
political decisions
When No Root Cause Analysis
Works

Act of God
Photo Source: PeoplePC OnLine
Concept # 3 Look for indicators
of small, seemingly insignificant
but repetitious reliability problems
and act on the findings.
Cluster Analysis consists of the following steps:
z Sort the Data
z Identify clusters
z Determine which clusters are relevant
z Group the clusters into categories
z Determine the consequences of relevant clusters
z Determine technicians involved, when necessary
 Cluster Analysis Example
Problem/Category Cluster # in 1988* # in 1990
Damage 28 15
Tubing/Fitting 13
Packing 2
Seal/Gasket 4 2
Tightening 19
Wiring 1
Procedural 18 10
Diagnostics 2 1
Repair 2
Weld 2
Total 98 28**

* Actually an average of 1987 and 1988

**71% REDUCTION
Concept #4 Dont be afraid
of mistakes; learn from them.
Typically after an incident involving
substantial cost to recover, injury or death
the search is started to find the guilty
parties

This is the wrong (management) approach

in all but those cases where malicious
intent is apparent
 Learning from Mistakes
Adopt a No-fault Policy
z Need to learn and not suffer unnecessarily
from undesirable events

z Stopping any attempt to blame someone will

aid in more quickly getting to the truth of what
happened and the ultimate solution

z Learn from the mistakes; correct the

problems, and get on with business of serving
customers
 Learning from Mistakes

Boss, we do have a no fault policy, dont we?

Photo Source: PeoplePC Online
 Learning from Mistakes
Adopt a Compliance Policy
z Applies to the use of all operating and
maintenance procedures as written

z If procedure found deficient in some way, as

modified by competent personnel following
the approved procedures management
process

z Assumes the organization has adopted a goal

of becoming a Procedure Based
Organization (Concept #5)
 Learning from Mistakes
Practice Peer Review
z When a major equipment failure and/or personnel
injury/fatality incident occurs.

z Purpose of this practice is to fully identify what

happened and what should be done to eliminate or
mitigate the incident being repeated in the future.

z Producing an accurate picture of what happened and

coming to a conclusion as to what to do to prevent
such incidents in the future is greatly increased
when personnel involved are talking to their peers,
without managers or other outsiders present.

z It can be effective, however, only if the practice is

backed by the no-fault policy
 Peer Review of Incidents

Yeah, Ive something to tell my peer fork lift

operators ! Photo Source: PeoplePC OnLine
 Learning from Mistakes
Practice focusing on the incident at hand
while it is being investigated
z Those assigned to facilitate such analyses
must acquire and liberally apply the skill of
diverting such discussions and re-focusing
attention on the matter at hand

Start listing, by title only, Other Items of

Interest for presentation along with the
report concerning the incident at hand
 Concept # 5 Become a
Procedure Based Organization,
but dont overdo it.
A Procedure Based Organization
produces or receives and complies
with detailed written instructions for
conducting not only maintenance, but
also operations and routine checks

This seems so basic that it is

overlooked in most organizations and
for all the wrong reasons!
 Procedure Based Organizations

Procedure Based Organization

Process for Origination, Dissemination,
Feed Back and Follow-up

Operations Procedures Maintenance Procedures

and Check Lists and Check Lists

Compliance
Policy

Note the two way communications lines at the top

Procedure Based Organizations
Has to backed with a working process for
procedure and checklist origination,
dissemination, feedback and follow-up
Users must have on-going evidence that their
ideas for improvement are being received,
considered and acted upon promptly
Changes that are concurred in must be seen to
be incorporated in revised procedures and
checklists coming out of a process that functions
as well as is expected of all the maintenance
and operations processes it supports.
 Worlds Best Maintenance
Organization. Seminar for New
Organization in Old Plant
Seminar stressed, among other things, use of detailed
procedures and checklists for both operations and
maintenance

Management decided to apply the principles to startup of

their most complex manufacturing process

Check-off list for all systems needed to roll steel bars into
coils of wire prepared and applied to weekly startup

Startup went without any delay or incident, a first for that

plant under the new staff - Hallalujah!
Procedure Based Organizations
Gallatin Steel Company
z Procedure workshop held in mid-2005
z 500 detailed preventive and repair maintenance
procedures and checklists by mid- 2007
z Biggest benefit was the significant increase in
confidence that the work force had gained in
performing maintenance.
z Delays and frustration with not having the correct
tools or replacement parts was radically reduced.
z Rated by the Kentucky Chamber of Commerce
and the State Council of the Kentucky Society for
Human Resource Management as one of the best
to work for in the state in 2006. Rated 16th in
nation by Forbes Magazine as best large company
to work for
 Over Doing It
Contractor maintenance organization
resisting requirements of client
z Mandatory check-offs (by initialing) for each
step of every maintenance procedure they
were required to conduct
z A rigorous audit procedure with punitive
provisions for non-compliance by
maintenance personnel had been prepared
for implementation as part of the customers
compliance policy
z Craftspersons who were pushing back had,
in my opinion, a good case for doing so
Dont Over Do It! (Continued)
Need to trust in the client-contractor
partnership (or O & M relationships)

Recommended the audit requirement

be abandoned completely

Recommended procedures be
categorized per the definitions in the
table that follows with individual steps
required to be checked off only for
safety and critical maintenance tasks.
 Typical Procedure and Checklist Categories
Procedure/Checklist Where Used Manner of Using
Type
Safety or Complex evolutions where safety of Verbatim compliance
Critical Task personnel and/or hazards to Reader/worker team approach
(COP, CMP) equipment are principal concerns. Individual step sign-off
Standard Task Operating and maintenance Procedures available on file
(SOP, SMP, PM, procedures for common, often Used as training documents as
PdM) repeated tasks well
Can be taken on job site if needed
Captures experience
Utilizes skill of the craft

Special Task Procedures for major, complex Procedure is part of work

(SpOP, SpMP) and/or infrequent maintenance
and/or related operational
procedures (e.g., for post-
maintenance-testing.)
package
Maintained on file
Used on job site as reference,
especially if it contains check-off
lists or data collection
requirements
Includes post-repair tests
 Concept # 6 Eliminate as much
maintenance as possible and
increase emphasis on reliability.
In the past the traditional view was that the
two goals stated in the concept statement
above are contradictory and impossible to
achieve

More maintenance does not produce more

reliability pre se.

It can be a root cause of reduced reliability.

Optimum Maintenance Program
Has identified what maintenance to perform
(that which is (cost) effective and applicable
(i.e., it works) a result of a proper
application of Reliability Centered
Maintenance (RCM)

Has effectively determined exactly how

maintenance should be done (a result on
proper application of Total Productive
Maintenance (TPM) principles)

Neither RCM nor TPM is perfect nor are they

almost ever perfectly implemented
 Reliability Improvement -
The Neglected Function
Much can be done at the design stage to
eliminate, reduce or at least minimize the
hours spent maintaining equipment

World is faced with the equipment already in

place and in production, acquired on a
lowest purchase and installation cost basis

Challenge is to improve the reliability and

maintainability of the equipment we have, not
the equipment wed like to have
 Reliability Improvement -
The Neglected Function
Cost reduction from increased reliability and
decreased maintenance can be significant

Reduction in cost directly affects the profit

margin, and/or makes it possible for a company
to offer cost savings to customers

Management error that is often committed is to

mandate maintenance cost reduction without
compensating by providing a comparable
improvement in reliability or maintainability
 Who Suffers?
Easiest target for cost reduction is
most often maintenance personnel
(layoffs of excess personnel)

Results in a more costly approach as time

goes on, especially when lost opportunity
costs are considered

So, many other stakeholders lose, also

 What Happens?
Causes a pullback from proactive maintenance
and a fallback to reactive maintenance
Full impact may not be felt for many months,
and in some cases up to 2 years
When percentage of inoperative equipment
reaches an intolerable point, maintenance
personnel are again augmented
It takes about two more years to fully recover to
the high point of performance where the layoffs
began
Study performed at Massachusetts Institute of
Technology shows that these cyclic events do,
in fact, occur with alternating reactive &
proactive maintenance strategies
 Cycling Between Reactive &
Proactive Maintenance Strategies
Aluminum Plant Case Study
z 30 year old plant changed from foreign to U.S.
ownership
z New management team hired to improve profits
z Implemented PdM and Reliability Improvement
z Throughput increased from 50% theoretical capacity
to 75% in just 2 years
z Owners bought 3 more plants and promoted first
plant manager to VP to oversee all plants
z Replacement plant manager hired from another
company
z Downward cycle began his first week on the job
9 Dream Team all but evaporated
9 Year later the VP saw significant reduction in throughput
9 Plant manager fired and recovery has been in progress
since
 Whats the Correct Approach?
Nature of the jobs experienced maintenance
personnel are performing should be changed!

Emphasis should be placed on the following:

z Maintenance prevention and elimination
z Reliability improvement and sustainment
z Capacity enhancement

This is done by acquiring, putting in place and/or

using:
z Rules
z Tools
z Schools
 Concept # 7 - Dont forget the
roots of your M & R program
initiatives for improvement
Not uncommon, with so many new initiatives
being offered in M & R, to see earlier, even highly
successful principles and methodologies
abandoned, not learned or forgotten

Caused by promotion, retirement and transfer of

those who implemented them

Late 1990s U. S. Navy revitalization of ship

maintenance programs case in point
 Case Study
Revitalizing Maintenance
In 1970s and 1980s vigorous effort were undertaken to
change maintenance from more a costly, shipyard-
based strategy to one anchored in RCM and operating
base support

By the 1990s, most of those engaged in implementing

the new RCM-based approach had retired or moved
on to other jobs, due in part to the post-Cold War draw-
down

By the late 1990s the Navy found that its maintenance

programs were in need of overhaul and revitalization in
order to ensure reliability in the face of apparent return
of intrusive maintenance
 Case Study
Revitalizing Maintenance
New methods of contracting for ships often
resulted in efforts to provide RCM-based
programs being under-funded and inadequately
implemented.

The ship builders often simply implemented

original equipment manufacturer (OEM)
recommendations
z Heavily tilted towards regular overhaul, requiring
heavy life-cycle replacement parts costs

Some old-timers still remained in civil service

who had by this time achieved positions with
sufficient clout to rectify this problem.
 Case Study
Revitalizing Maintenance
Revitalization initiative devised to avoid inapplicable
and ineffective maintenance and reduce
maintenance costs without sacrificing reliability

Initiative was based on three parallel efforts:

z Rules Improving maintenance requirements and plans
(including reliability improvements)
z Tools Using computer and diagnostic technology (i.e.,
Condition-based maintenance)
z Schools Educating all levels of maintenance decision
makers in reliability and condition-based maintenance
principles

Commercial organizations suffer from the same

problems
 Revitalizing

Control please issue a work order to revitalize the

red lube truck. Source: PeoplePC OnLine
 Conclusions
Seven concepts for reducing human error in
maintenance and reliability are:
z Exercise and practice leadership as well as
management.
z Look first at programmatic rather than technical
solutions to reliability problems.
z Look for indicators of small, seemingly insignificant
but repetitious reliability problems and act on the
findings.
z Dont be afraid of mistakes; learn from them.
z Become a Procedure Based Organization, but dont
overdo it.
z Eliminate as much maintenance as possible and
increase emphasis on reliability
z Dont forget the roots of your M & R program initiatives
for improvement.
 Conclusions (Continued)
You can come up with many more ideas on
reducing the occurrence and impact of
human error in maintenance and reliability

Concentrating on these will make a big

difference in achieving the goals and
objective of your organizations

Manta for modern maintenance and reliability

programs everywhere could well be Rules,
Tools and Schools!
Questions?

Photo Source: PeoplePC OnLine