Welcome Back

Quality Leadership Academy

Session 3
“How Do You Know it Works?”
Anna Roth, RN, MS, MPH
 Report on Projects to Date

• Refine and refresh aim

statements

• Share results of our small tests

of change

• Will or did you revise your test

of change?

• If so, what would/did you

revise and why?
Theory
 Today’s Objectives
Review results of your project
 Review small tests of change

Review techniques for organizing and displaying data for

maximum impact

Your toolkit- Driver Diagram

Share examples of reports designed to get the attention of those

who need the information

Action Planning
How will we know?
 Why Else Should We Measure?

• You can’t manage what

you don’t measure
• How else would you
know that your steps are
making things better or
worse?
• It’s often cause for
reward, recognition and
celebration
Choosing appropriate statistics
 Median v Mean

• 10 people are on the bus

• The mean income of the
riders is $50,000/yr
• The median income of
the riders is $50,000/yr

• What does this tell

us?????
 Median v Mean

• The median income of

the riders remains
$50,000/year
• The mean income is
now approx $50 million
• So is the average
income of bus riders
now $50 million
because Bill Gates got
on the bus?
 Mean

Mean (average)
 Measures the center, or
middle, of a numerical data
set
 The sum of all the numbers
divided by the total number
of numbers
 May not be a fair
representation of the data
 Easily influenced by outliers
 Median
Median
 Also measures the center of a
numerical data set
 Much like the median of an
interstate highway
 The point at which there are an
equal number of data points
whose values lie above and
below the median value
 Is truly the middle of the data
set
 Better measure of CT than the
mean when there are outlying
values in the data set
 Percentage or Percentile?

• Suppose your score on the

GRE was reported to be
the 80th percentile

• Does this mean you

scored 80% of the
questions correctly?
 Honest Errors
• Arithmetic errors or
omissions
– Check to see if
everything adds up
– Double check even the
basic calculations
– Verify the total to put
results in proper
perspective; if sample
size really small you
may not want to use
Excercise
Report Out
Back in 15
minutes
Finding your way/Telling your story
 Data Display and Analysis

• How do you want to tell

your story??
• Who are you going to
tell your story to?
 Common types of data display

• Pie charts
• Bar graphs
• Tables
• Time charts
• Run charts
• Control charts
 Charts and Graphs
and Spiders Oh My

• Watch for pitfalls

• Size matters!
• Be aware of tick marks
on the y-axis
• 10s, 20s, 100s, 1000s?
• Check the scale to put
results in perspective
 Sizing up a pie chart
• Do the percentages add up
to 100
• Beware of slices that are
called ‘other’ if they are
larger than many other
slices of the pie
• Look for a reported total
number of units so you
can see how big the pie
was before it was divided
up
Elements of a Control Chart
An indication of a
special cause

UCL
Indicator

LCL

Time
Non-Random Rules for Run Charts
 Variation
Common Cause vs. Special Cause
Common cause Special cause
Always present Abnormal, unexpected
Inherent in process Due to causes not inherent
Is due to regular, natural, in process
ordinary causes Also known as non-random
Results in a stable process or assignable process
that is predictable
 Appropriate Actions to Take
Common cause Special cause
If undesirable need to • Identify and study special
change the process. cause
If only common cause • If negative, minimize or
variation and treat as prevent
special cause (tampering),
• If positive, build into
leads to greater variation,
process
mistakes, defects
First 24 Observations from Red Bead
Data
(without outlier employee)
 12 Runs
expect to find between 8 and 18
runs
On Death, Dying & Data
ACCEPTANCE

DEPRESSION

BARGAINING

ANGER

DENIAL
On Death, Dying & Data ACCEPTANCE
“I accept the burden
of improvement”

DEPRESSION
“This feels too hard to do”

BARGAINING
“The data are right; it is a problem; but it is
not my problem.”
ANGER
“The data are right, but it’s not a problem”

DENIAL
“The data are wrong”
 Stages of Facing Reality:
“To live divided no more”
• “The data are wrong”
• “The data are right, but it’s not a problem”
• “The data are right; it is a problem; but it is
not my problem.”

“I accept the burden of

improvement”
39
Crimson Bead Company
“Every system is perfectly designed to
achieve the results that it achieves”

Berwick: central law of improvement BMJ

1996 312:619-622
Discussion
Oversight
 Lesson #3
Execution

Oversight

Project-level e.g. System-level e.g.

• % AMI patients getting
evidence-based care
• % Pneumonia patients getting
evidence-based care
• Time to answer call light on 5
West
• Hospital mortality rate
• Cost per admission
• Adverse drug events per
1000 doses
• Patient satisfaction scores

46
 Projects Connected to Big Dots
* Surgical Site Infection Rate
* Percent of un-reconciled medications
* Staff reporting positive safety climate
* ER to bed placement time
* PACU to bed placement time
* Mortality Rate
* ICU to bed placement time
* Cost per Admission
* Bed to LTC placement time
* Adverse Events
* Functional Outcomes
* Patient Satisfaction * Percent of turnover
* 3rd Available Appointment in first year
* Voluntary Turnover * Employee loyalty

* Condition-specific,
clinical process indicators
* Preventive care measures
* ICU mortality
* Office visit cycle time
* Catheter related BSI
* Average ventilator days per patient
* Adverse events/ICU day
47
 A Senior Leader Perspective on
Projects
Spreading and Sustaining
Changing the These Design Concepts: “A
Organization: Place Where…”
•HR
•IT
•Finance
•Leadership Processes
•Business Strategy Spreading and
•Environmental Sustaining This
Strategy Improvement

The Project:
e.g., Ventilator-Acquired Pneumonia
 Issues at Each Tier (Examples)
 Aims of strategic importance to the system as a whole
 “Big Dot” measure of progress
Tier 1:  Executive, Board and Senior Leader engagement

Big Dot  Vision and the associated structural changes

 Strong linkage to finance
 Learning and mitigation of risks
 Managing the learning, the politics, and the risks

 Understanding “drivers” and causal linkages

 Outcomes of consequence tracked over time
Tier 2:  Middle Management key
“Connecting the Dots” – putting the learning
Portfolio together
 Continual readjustment of portfolio
 Strong linkage to finance
 Some structural changes (e.g., job roles)
 Team organization and capacity matter
 Process and outcome tracked over time
Tier 3:  Leaders remove obstacles
Projects  Change concepts help
 Ability to run PDSA cycles
 Temporary infrastructures facilitate progress
 Project Level Measure (Tier 3)
• Bundled orders with opt out • Family assistance
• May 05 to Oct 06: 17 months of NO VAP’s
• 30 degree head of bed elevation • IHI Mentor Hospital

marked on walls with tape

• Now spreading to floor beds post
extubation
Project Level Measure (Tier 3)

“One Patient,
One List”
 Project Level Measure (Tier 3)

• % meds unreconciled:admission 25%→ 3%

• % meds unreconciled:transfer 12%→ 4%
• % pre-admit meds unreconciled 19%→1%
• % of patients with ANY unreconciled
meds decreased from 36%→ 3%

• Discharge….still testing
Driver Diagrams
 What Changes Can We Make?
Understanding the System for Weight Loss

Primary Secondary Process

Outcome Drivers Drivers Changes

Track
drives Limit daily Calories
intake
Calories In
drives
Plan
drives Substitute Meals
low calorie
drives foods

AIM: Drink H2O

A New Not Soda
ME! Avoid
alcohol
drives
drives
Calories
Out
drives
Work out 5
days
“Every system is perfectly drives

designed to achieve the results

that it gets” Walk to
errands
 How Will We Know We Are Improving?
Understanding the System for Weight Loss with Measures

Primary Secondary Process

Outcome Drivers Drivers Changes

• Avg cal/day
Track
drives Limit daily Calories
intake • Running
Calories In calorie total

• Daily calorie drives • % of

count Plan
drives Substitute opportunities Mealsoff-
• Meals
used
low calorie plan/week
drives foods

AIM: Drink H2O

A New • Avg drinks/
Not Soda
ME! Avoid
alcohol week • Sodas/
• Weight week
• BMI drives
• Body Fat drives
• Waist size
Calories
Measures let us Out
• Monitor progress in improving the
• Exercise
drives
Work out 5 • Days between
system calorie count
days workouts
• Identify effective changes drives

Etc...
Walk to
AIM Primary Driver Secondary Driver
• At your tables write
down 4-6 primary
drivers for your project
• For each primary driver,
come up with 2-3
secondary drivers
• If you have time, write a
few small tests of
change for each
secondary driver
Report Out
Tying it together
 Transforming Care at the Bedside
(TCAB)
Total Joint Team Med-Psych Workgroup

VAP Prevention Clinical

Team Informatics

Perioperative ED Safety
Care

Medication Central Line

Reconciliation Infection
Team Team

Perinatal Impact Multidisciplinary

Team Rounds
Office Practice Rapid Response
Team Team
 OPERATIONS/
QUALITY DRIVERS
Primary Drivers Secondary Drivers

• Ownership of agreed upon set of outcomes

• Review of outcomes at each meeting
• Quality and safety comprises 25% of agenda
Leadership and
• Involve p atients in safety
Culture
• Visible on all senior leader agenda
• Culture of Safety/Fair and Just

• Mortality -RRT, Sepsis

• Medication safety
• Falls

Care that is; Deliver the Program

•
•
Pressure Ulcers
Re-admissions – Transitions

safe, effective, •
•
Harm/Adverse events
Infection-SSI,UTI,V AP,MRSA
patient - • Infrastructure supports improvement measurement
centered, • Clear, shared measurement set

timely, efficient Measurement • Inventory national programs and measurements

and equitable • Recov ery plans for unmet outcomes

• Strengthen IT infrastructure

• Shared meaningful vision from Board to the patient

Communication
• Expert at communication and marketing methods
coaching
• Program design and str ucture

• Staff satisfaction
• Involve Patients in all improvement teams
Capacity and • Involve ethics in all improvement and operations
Infrastructure
• Culture of contin uous quality improvement
• Build Innovation engine
System Level Aims
 System Level Aims
Planned System Level Aims to
begin by 2010
Additional System
Eliminate inequality in at least ten
Level Aims
improvement /operational areas by
Zero Hospital
25%
acquired infections
Primary Reduce Ambulatory Care Sensitive
Patient overall
System Aims Admissions (ACS) to CCRMC by
satisfaction to be
15%
>90%
Patient engagement on every
Readmission rate to
innovation and improvement team
decrease by 30%
by January 1, 2010

Develop a formal process for

engagement of ethics expertise in
operations and quality
improvement.
Prophylactic Antibiotics One Hour Prior to
Incision
Hours of Behavioral Restraint Use
Inpatient Psychiatry: Discharge Care
Planning
 VAP per 1000 Ventilator Days
14
Ventilator Days were 777 in
2006 and 645 in 2007
12
11.6
10.8
10

2
1.5 1
0
VAP per 1000 Ventilator Days
Number of VAPs and Ventilator Days
CCRMC 30 Day Readmission Rates
Heart Failure Discharge Instructions Given
Heart Failure Discharge Instructions Given
 Aiming for Perfect Care
•Discharge Instructions

•Evaluation of LVS Function

•ACEI or ARB for LVSD

•Adult Smoking Cessation

Advice/Counseling
Percent of Patients Who Received All Heart
Failure Interventions at CCRMC
Percent of Patients Who Received All Heart
Failure Interventions at CCRMC

All-or-Nothing Measurement
Why the time is now
Who will if not you?
What can you do by next
Tuesday?
 Thank you
Anna Roth, CEO
Contra Costa Regional Medical Center
aroth@hsd.cccounty.us
safetynethospital.blogspot.com