King Fahd Hospital

Jeddah –JCI Experience

Majdah A. Shugdar
MSc. CPHQ, MBA , , PGDRM - Bradford university –School Of Management
Academic associate – University of Cardiff institute – UWIC
UK

CBAHI – Director of Accreditation department

KFH Jeddah- Director of TQM department
King Fahd Hospital Jeddah –JCI
Experience

Agenda
 KFHJ indicators database

 Indicator operational definition

 Indicators Performance Challenges

 Validation example

 Validation queries
 INDICATORS

 Clinical
 Managerial
 International Patient Safety Goals
 International Library Of Measures
 Clinical Practice guide line
 KFHJ indicators Data Base

SN Area Indicator title Numerator Denominator

Type Frequency of
(structure, data collection Anticipated
Who (Owner) Target
process, (daily, weekly, reporting time
Outcome ) monthly

Improvement Rational for

noted as : selection/ Collaborative
Sample Size / Unit
increase or Department
decrease priority

How the data

will be CBAHI, JCI,
Data Source disseminated CBAHI
to staff
 INDICATORS:

Area : Radiology and diagnostic imaging services

Indicator title Numerator Denominator

Total number of OPD patients
Average waiting time for CT Total CT scan booking time for
waiting for CT scan in the
scan for OPD patients OPD patients in one month
same month
Frequency of data collection
Type (structure, process, Outcome) Who (Owner)
(daily, weekly, monthly)
Radiology
PROCESS MONTHLY
Anticipated reporting time Improvement noted as : increase or
Target
decrease
25th of Arabic month 3 days Decrease
Sample Size Collaborative Department
Rational for selection/priority
/ Unit
MOH requirement 100% of all cases IT
Data Source How the data will be disseminated to
CBAHI, JCI, CBAHI & JCI
staff
Newsletter, Quality Council,
Radiology JCIA
Monthly indicator report
 INDICATORS
Patient Identification
1. Operational Definitions/Clarifications Patient Identification – is the process of recognizing an individual thru sets of patient
identifier established by the organization to ensure that the right patient is indeed receiving the
right treatment / care.

Patient Identifier - is a piece of information specific to a patient while in the hospital setting, for
example the patient’s name, medical record number, date of birth, the purpose of which is to
reliably identify the individual as the person for whom the service or treatment is intended, and
to match the service or treatment to that individual.

2.Data Elements a. No of nurses compliant with Pt identification policy in specific month

b. Number of nurses observed in the same month

3. Inclusions Blood drawing, Medication administration, Any diagnostic procedure

4. Exclusion Procedures other than what is mentioned in inclusion

5. Formulas

No of nurses compliant with Pt identification policy in specific month / Total number of nurses observed in the same month * 100

6. Data Collection Plan

6.1 Who will collect the data? Head nurse

1. No of nurses compliant with the process of
identification
6.2 What data will be collected?
2. Total number of nurses observed in the same
period
6.3 Sample Size 144 nurse/month
6.4 When will the data be collected? Daily
6.5 Where will the data be collected? Data collection sheet (attached)
6.6 How will the data be collected? Observation of nurses
 INDICATORS

International Patient Safety Goals – IPSG

IPSG(1)- Patient Identification Data Collection Sheet

Modified
Clinical -Indicators
 Clinical Indicators
Area
Patient assessment
Laboratory services
Radiology and diagnostic imaging
services
Surgical procedures
Antibiotic and other mediation
use
Medication errors and near
misses
Anesthesia and sedation use
Use of blood and blood products
Availability, content, and use of
patient records
Infection prevention and control,
surveillance, and reporting
Clinical research
Indicator title
Pressure Ulcer Prevalence (Hospital-Acquired)
Average TAT for CBC
Average waiting time for CT scan for OPD
patients
Prophylactic antibiotic received within one hour
prior to surgical incision
ACEI or ARB for LVSD
Medication error rate
Rate of correct completion of anesthesia forms
Crossmatched /Transfused Ratio (CT Ratio)
Rate of correct completion of electronic
discharge summary
Hospital Acquired Infection (HAI) Incidence
Not applicable
 Indicators Performance Challenges
deficiency operational definition
ILOM selection criteria
100% compliance
select – validation method
validation – new memorandum Clinical
Deficiency operational definition
Select – deficiency operational definition

LAB TAT
TAT for CBC from time received till time
delivered in the lab for inpatient in one
month
----------------------------------------------- * 100
Total number of CBC requests for inpatients
in the same month
 Average TAT for CBC 1433
60
Two times computer system

INDICATORS
50
Average TAT for CBC
40 (Minutes)
Current Year Average
Minutes

30
Last Year Average

20
UCL (+2SD)

10 Target

0
1 2 3 4 5 6 7 8 9 10 11 12
Months

Current year average is higher than the last year average due to increased
the average in Muharram resulted form computer system failure at that
month
 Rate of medication errors

The total number of medication errors in

one month
------------------------------------------------ * 100
Total number of patient admissions in the
same month
 Rate of medication errors 1433
18.0%
16.0%
14.0%
12.0%
Medication Error Rate
10.0%
Rate

Current Year Average

8.0%
Last Year Average
6.0%
UCL
4.0%
Target
2.0%
0.0%
1 2 3 4 5 6 7 8 9 10 11 12
Months

Average improved from 9.1 to 6.1 in the current year and we

achieved our target in the last month, this is because medication
safety orientation lectures and starting the medication safety
rounds to ensure the implementation of the policies
Managerial
1. Le

Indicators
 Availability of emergency medications

Number of available emergency medications

in one month
------------------------------------------------- * 100
Total number of emergency medications in
the same month
 Availability of emergency medications 1433
120.0%

100.0%

80.0%
Percentage

Availability of emergency
60.0% medications
Current Year Average
40.0%
Target
20.0%

0.0%
1 2 3 4 5 6 7 8 9 10 11 12
Months

This rate represent only the availability of medication by MOH

supply, the hospital overcome this challenge by direct purchasing
and communication with other hospitals
 1. Le
IPSGs Indicators
 Nursing patient identification compliance

No of nurses comply with Pt identification

policy in one month
-------------------------------------------------- * 100
Total number of nurses observed in the
same month
 Nursing patient identification compliance rate 1433
100%
98%
96%
94%
Percentage

92% Nursing pt Identification

90% Compliance Rate
Current Year Average
88%
86% Last Year Average
84%
82%
80%
1 2 3 4 5 6 7 8 9 10 11 12
Months

We reached average compliance of 95.3% (last year was 86.1%) due to

comprehensive education and orientation lectures and on-job training regarding
patient identification
Verbal Order Nursing Compliance Rate
Frequency of Verbal Order Nursing
compliance in selected sample in one
month
-------------------------------------------------- * 100
Total number of verbal orders made in the
same month
 Verbal Order nursing compliance rate 1433
120%

100%
Verbal Order nursing
compliance rate
80%
Current Year Average
Rate

60% Last Year Average

UCL
40%
Target
20%

0%
1 2 3 4 5 6 7 8 9 10 11 12

The compliance average is slightly improved from 84.1 to 84.8,

still needs more education and monitoring
ILOM selection criteria
100% compliance
 Prophylactic antibiotic (hip arthroplasty) 1433

Number of surgical patients (hip arthroplasty) with

prophylactic antibiotics initiated within one hour prior to
surgical incision in one month
--------------------------------------------- * 100
All selected surgical patients (hip arthroplasty) with no
evidence of prior infection and who are > = 18 years in
the same month
 Prophylactic antibiotic (hip arthroplasty) 1433
120.0%

100.0%

80.0% Prophylactic antibiotic

Percentage

received within one hour

60.0% prior to surgical incision
(hip arthroplasty)
Target
40.0%

20.0%

0.0%
1 2 3 4 5 6 7 8 9 10 11 12
Months

Full compliance by orthopedic staff, needs

continuous monitoring ????
 Prophylactic Antibiotic (Knee Arthroplasty)

Number of surgical patients (knee arthroplasty)

with prophylactic antibiotics initiated within
one hour prior to surgical incision in one
month
------------------------------------------------- * 100
All selected surgical patients (knee
arthroplasty) with no evidence of prior
infection and who are > = 18 years in the
same month * 100
 Prophylactic antibiotic (knee arthroplasty) 1433
120%

100%

80% Prophylactic antibiotic

Percentage

received within one hour

60% prior to surgical incision
(knee arthroplasty)
Target
40%

20%

0%
1 2 3 4 5 6 7 8 9 10 11 12
Months

Full compliance by orthopedic staff,

needs continuous monitoring
Quality Improvement and Patient Safety (QPS):

Data Validation
 When to do data validation
Intent statement of QPS.5
 A new measure is implemented;
 Data will be made public on the organization’s
website or in other ways;
 A change has been made to an existing measure
 The data resulting from an existing measure has
changed in an unexplainable way;
 The data source has changed
 The subject of the data collection has changed, such
as changes in co-morbidities, CPGS, treatments,
 What measures need to be validated during the “phase-in?”

Intent identifies circumstances under which measure

should be validated
 Circumstances applicable to the Library include:
 A new measure is implemented (in particular, those
clinical measures that are intended to help an
organization evaluate and improve an important
clinical process or outcome);
 Measures selected from the library
 What measures need to be validated during the “phase-in?”

 Validate early in the process to assure measures are

collected
 data will be made public on the organization’s Web
site or in other ways;
 If measures from Library will be publicized
 research protocol alterations, new practice guidelines
implemented, or new technologies and treatment
methodologies introduced.
 If measure chosen from library addresses a new CPG
or treatment protocol
 QPS.5 Data Validation
• Internal validation includes:
– Re-collecting data by a second person
– Using statistically valid sample of records, cases
and data
– Comparing original data with re-collected data
– Calculating accuracy
– Noting unclear data and taking corrective actions
– Collecting new samples after corrective answers
How to do the validation?
 QPS.5 Data Validation

• Hospital integrates data validation into the

quality management and improvement
program

• Data validation includes the measures

selected as required in QPS.3.1

Continued on next slide

 Components of validation

Identify random selection of data source for re-abstraction

Perform re-abstraction of data (second person)
Compare original data with re-abstracted data
Calculate accuracy
Adjudicate discrepancies of data abstraction and re-
abstraction
Implement corrective actions
Collect new sample after corrective actions implemented
 Example:
measure implementation
I-AMI-1 – “Aspirin received within 24 hours of arrival to hospital in
patient with an acute myocardial infarction”
Identify data source
 ICD 9 or ICD 10 diagnosis codes
Implement measure
Review sources to identify:
ASA given within 24 hours
Reason for not giving ASA documented (i.e., asthma, GI bleed, etc.)
Calculate percent of measure compliance
Separate improvement actions are take to improve compliance
 Example:
measure implementation
100 Cases in a quarter meet the inclusion criteria for I-AMI-1, Aspirin
on Arrival
50 cases have ASA within 24 hours documented
35 cases have contraindication documented
15 cases have neither documented
Calculated % of compliance:
85 cases out of 100 show compliance with measure
 15 cases are not compliant
85/100 = .85 X 100% = 85% compliance with measure
 Example:
measure implementation

 Verify number of sources reviewed

 Determine statistically valid sample of records, cases, or other data
Total # of Data Sources Recommended Sample Size
1 to 16 All data sources
17 to 160 Minimum 16 or 10 %
161 to 480 10%
480 48

• Samples selected should be across a range of variables, such as:

– Dates
– Patient types
– Incidences
– Staff involved
 Example:
measure implementation
 Select and train a second person to perform
abstraction in same way as original abstractor
 Verify pool of cases (Denominator Verification)
 Re-abstract data from identified samples
 Re-abstraction utilizes the same data definitions
 Second person does not review previously
abstracted data prior to re-abstraction
 Example:
measure implementation
• Validation includes sample size of a minimum 16 cases
– Minimum 16 cases for data source of 16 to 160
• Re-abstraction of sample cases reveals
 8 cases have ASA within 24 hours documented
 2 cases have contraindication for ASA documented
 6 cases have neither documented
 10 cases out of 16 show compliance with measure
 6 cases are not compliant
 10/16 = .625 X 100% = 62.5% compliance with
measure
 Example:
measure implementation
Discrepancies
 Compliance from original data source is 85% while sample
reveals a compliance of 62.5%

 Compliance from sample size should be within 90% of original

compliance percent (i.e. 76.5%)

 Original and second abstractor review sample data source to

identify possible causes of discrepancy

 Review measures and data definitions to assure both have

same understanding
 Example:
measure implementation
Possible Causes of Data Discrepancies
 Abstraction Guidelines - not clear or misinterpreted
 Definition of data element not clear
 Training of abstractors may have been different
 Parameters of collection may not have been identified
 Data terms may not have the same meaning
 Missing elements – omission vs. “does not apply”
 Incorrect data entered
 Conflicting data in record
 Different data source used
 Example:
measure implementation
Corrective Actions
• Identify discrepancies
• Take corrective actions as appropriate – For example:
– Train abstractors in the same way
– Clearly define elements
– Identify specific parameters for data collection
• Source
• Time frame
 Re-validate after all corrective actions have been
implemented
 Compliance Questions

What is your method for data validation?

Do you publish data on a website?
Have you applied the method to all clinical measures including
clinical guidelines and LOM?
What is your re-abstraction variance?
What were the common reasons for variances?
What actions were implemented to decrease variances?
Validation Queries
 QPS.5 Data Validation

What are the criteria to decide for the

Sample size and frequency for
collection ?
 Data Validation: Sample size

Criteria Representative sample

<30 ----- all
30 - 100 -------50
10- 500 -------50
>500- ------70
 QPS.5 Data Validation
 Using a statistically valid sample of
records, cases, and other data.
 A 100% sample would only be needed
when the number of records, cases, or other
data is very small.
 QPS.5 Data Validation

For a large population number

Can I use the same sample extracted by the original data collector
or I have to withdraw another sample?

Can I use the other method of data collection, e.g.

Department list instead of electronic list of cases
 QPS.5 Data Validation

Internal validation includes:

Re-collecting data by a second person Not
involved in the original collection
What is the convenient cycle for
example,
How many cycles are enough for
validation ? For monthly indicator is it
enough to collect it for once per
one month or once every month
 QPS.5 Data Validation

What if there is improvement

happened due to intervention how I
am going to judge the validation
 “TIME OUT” compliance rate

Total number of procedures with “time out”

documented in one month

------------------------------------------------- * 100

Number of procedures done in the same

month
 “TIME OUT” compliance rate 1433
120.0%

100.0%

80.0%
“TIME OUT” compliance
Rate

60.0% rate
Starting QM Current Year Average
orientation
40.0% lectures Last Year Average

20.0%

0.0%
1 2 3 4 5 6 7 8 9 10 11 12
Months

Started QM orientation lectures to keep and

improve the average after month of 3, and
already achieved our target by month of 6
CPGs Indicators
 Septic shock mortality rate

Number of deaths with diagnosis of septic

shock in one month
--------------------------------------------------- *
100
Total number of septic shock cases in the
same month
 Septic shock mortality rate 1433
100.0%
90.0%
80.0%
70.0%
Percentage

60.0% septic shock mortality rate

50.0% Current Year Average
40.0% Last Year Average
UCL
30.0%
Target
20.0%
10.0%
0.0%
1 2 3 4 5 6 7 8 9 10 11 12
Months

Current year average is improved from 75% to

67% after strict monitoring of CPG
implementation
 Rate of correct completion of
anesthesia forms

Number of correct completed anesthesia

forms within selected sample in one
month
------------------------------------------------- * 100
Total number of anesthesia forms within the
selected sample in the same month
 Rate of correct completion of anesthesia forms 1433
120.0%

100.0%

80.0% Rate of correct

Percentage

completion of anesthesia
forms
60.0% Current Year Average

40.0%
Target
20.0%

0.0%
1 2 3 4 5 6 7 8 9 10 11 12
Months

We improved from 49 % to 68% completion rate by developing new anesthesia

form and training of staff for completion still we tray to achieve our target
 QPS.5 Data Validation

What if data is collected by more than

one person ?
 QPS.5 Data Validation

For the data collected by many staff members.

E.g. for pressure ulcer indicator we divide developed pressure

ulcer in the hospital by all high risk patients for developing
pressure ulcer,

All inpatient nursing staff are sharing in collection of this

indicator and send immediate report to the responsible unit, how
I recollect again? Or I have only to review the number reported?)
QPS.5 Data Validation

Calculating Accuracy
 QPS.5 Data Validation : Calculating Accuracy

Calculating the accuracy by dividing the number of data elements found to

be the same by the total number of data elements and multiplying the total by

100.

 A 90% accuracy level is a good benchmark.

 Not clear, What if the number collected during data validation exceeding the

original one, e.g. original is 10 and the validation are 11, so the percentage will

be 110%;

 I think they mean 10% variability is accepted, if so they have to rephrase it.
Thank You