Data Collection

Nermin Mahmoud Ghith

B.S., PGDip.TQM, C.P.H.Q
Healthcare Quality Consultant
Nermin_ghith@hotmail.com

Nermin Ghith, B.S., PGDip.TQM, C.P.H.Q

DEFINING THE
POPULATION

ENTIRE POPULATION
SAMPLING

Nermin Ghith, B.S., PGDip.TQM, C.P.H.Q

Sampling

Purposes:
To measure only a portion of a total group
or population, such as for high volume
aspects of care and service;
To achieve accurate representation of the
entire target population, such as all
ambulatory patients; a specific procedure,
diagnosis, or DRG; or all cardiologists;
To generalize the results to the larger
population based on sample findings.

Nermin Ghith, B.S., PGDip.TQM, C.P.H.Q

Sampling strategy

Process of selecting a portion of the population.

Factors to Consider:
The characteristics of the population that the
sample must represent;
The location and time period from which the
sample must be drawn;
The type of sampling technique that will
assure that the sample accurately represents
the population;
The selection of a sample that will not
introduce a bias

Nermin Ghith, B.S., PGDip.TQM, C.P.H.Q

 Basic Sampling Designs

Population
Sampling
Purposes of sampling
Types of sampling
Probability
Nonprobability

Nermin Ghith, B.S., PGDip.TQM, C.P.H.Q

Types of Sampling

Probability
Simple random sampling
Systematic sampling
Stratified random sampling
Multistage random
Cluster sampling

Non-probability
Convenience
Purpose or judgment sampling
Expert sampling
Quota sampling

Nermin Ghith, B.S., PGDip.TQM, C.P.H.Q

Examples

Probability:
Simple random sampling
Picking randomly from a phone book
Systematic sampling
Picking every 9th name from phone book
after randomly picking first
Stratified random sampling
Picking from <30 and >40 age groups
Cluster sampling
Picking from retirement communities

Nermin Ghith, B.S., PGDip.TQM, C.P.H.Q

 Nonprobability
Convenience
First 10 patients who walk in
Purpose or judgment sampling
Selection of Engineers to represent
college graduates
Expert sampling
Arizona physicians for skin cancer
Quota sampling
7% male & 93% female nurses

Nermin Ghith, B.S., PGDip.TQM, C.P.H.Q

Sampling

Nonprobability Sampling
An intentionally-biased way to sample,
involving qualitative judgment about an
issue that is suspected to be common or
widespread.
Examination of relatively few cases is
assumed to be enough to reveal the nature
of any problem and its probable causes.
This methodology does not include
techniques to estimate the probability that
each case will be included,
the results cannot be generalized to the
entire population without further study.

Nermin Ghith, B.S., PGDip.TQM, C.P.H.Q

Sampling

Types of nonprobability sampling techniques:

Convenience:
Using data most readily available, e.g., all patients
seen in the ED in a given week.
Quota:
Portions or percentages of persons/cases in a
stratified population (subset), e.g., 10% of male
patients with diabetes and heart disease over age
55.
Purposive:
Persons/cases/issues selected because they
demonstrate a desired characteristic and can be
measured against specific, predetermined criteria,
e.g., all patients over age 60 with total hip
replacements

Nermin Ghith, B.S., PGDip.TQM, C.P.H.Q

Sampling

Probability sampling:
Introducing statistical techniques into
the selection process, thus permitting
the reviewer to draw inferences
about a population. It assures that
each case in the population has an
equal and independent (random)
chance of being selected and is,
therefore, truly "representative" of the
entire population being sampled.
Nermin Ghith, B.S., PGDip.TQM, C.P.H.Q
 Sampling

Types of Probability Sampling Techniques:

Simple random:
Using a Table of Random Digits (available in all
statistical software) to select the persons/cases from
a list of every case in the defined population.
Stratified random:
Creating 2 or more homogeneous categories or
dimensions of a population and selecting an
appropriate number of persons/cases that are
representative of the whole. Patients with IVs in
home care might be sampled by diagnosis, type of
solution, or with and without complications.

Nermin Ghith, B.S., PGDip.TQM, C.P.H.Q

Sampling

Types of Probability Sampling Techniques:

Systematic random:
Randomly selecting the first case and then
selecting every nth case thereafter based on
standard/fixed intervals, e.g., every 5th referral to a
specialist by a primary care physician in an
HMO/hospital after random selection of the first
case.
Multistage random
In large studies, sampling could be done in stages,
a sample is drawn from each stage randomly.

Nermin Ghith, B.S., PGDip.TQM, C.P.H.Q

Sample Size/Analysis

Larger sample, more valid & accurate study

Statistical analysis
Reliability
Interrater reliability

Nermin Ghith, B.S., PGDip.TQM, C.P.H.Q

Sampling

Sample Size
Sampling error
Effect Size
Consequences

Nermin Ghith, B.S., PGDip.TQM, C.P.H.Q

Joint Commission general
guidelines for sampling:

These sample sizes for these populations

(total cases meeting criteria) are
considered statistically significant and can
be applied to measurement activities for the
specified time period, e.g., monthly,
quarterly:
Population Size Sample Size
< 30 cases 100% of available cases
30 to 100 cases 30 cases
101 to 500 cases 50 cases
> 500 cases 70 cases

Nermin Ghith, B.S., PGDip.TQM, C.P.H.Q

 Sampling error
Larger sample, more valid & accurate
study
Smaller samples: could cause sampling
error
Smaller differences require larger samples
Categorical outcomes need larger
samples

Nermin Ghith, B.S., PGDip.TQM, C.P.H.Q

 Effect size
Smaller differences/effects ; large sample
Large differences/effects; small sample

This is requiring clinical judgment

Nermin Ghith, B.S., PGDip.TQM, C.P.H.Q

 Consequences (if you are wrong;
regarding the sample size)
Patients
Practitioners
Systems
Staff
Management

If youd have catastrophic consequences;

choose large sample to be confident.

Nermin Ghith, B.S., PGDip.TQM, C.P.H.Q

DATA COLLECTION

DESIRED CHARACTERISTICS
Comprehensive:
Covering all aspects of QM/PI
activities;
Coordinated:
Systematic, reducing duplication of
effort;
Integrated:
Allowing the identification of patterns,
trends, and opportunities across
departments/services, settings, and
the healthcare network.
Nermin Ghith, B.S., PGDip.TQM, C.P.H.Q
DATA COLLECTION

PRINCIPLES AND CONCEPTS

Data collection is a discipline that begins
with assessing what data are really needed
and whether or not they are collectible.
Use and adhere to the epidemiological
principles of data definition and collection
Data collection that is based on patient
care and organization functions, related
performance measures/indicators, and
supportive criteria involves the entire
organization.

Nermin Ghith, B.S., PGDip.TQM, C.P.H.Q

DATA COLLECTION

PRINCIPLES AND CONCEPTS

Maximize the use of data already being
collected;
Minimize duplication of effort;
Maximize the organization's computer
capabilities;
Coordinate data collection efforts across
departments, services, and QI Team
activities.

Nermin Ghith, B.S., PGDip.TQM, C.P.H.Q

DATA COLLECTION

PRINCIPLES AND CONCEPTS

Specific criteria may be necessary, either
to obtain adequate data for a specific
measure/indicator or to provide objective
evaluation of a process or pattern
appropriate person(s) responsible for data
collection
frequency and duration

Nermin Ghith, B.S., PGDip.TQM, C.P.H.Q

DATA COLLECTION

Timing:
"On the fly":
Real-time entry by care provider
Prospective:
Prior to care being rendered
Concurrent:
While care is being rendered
Retrospective:
After care is rendered

Nermin Ghith, B.S., PGDip.TQM, C.P.H.Q

 DATA COLLECTION

Types:
Continuous or periodic measurement:
Routine, systematic collection of
information over time, either concurrently
or retrospectively, e.g., utilization review;
tracking of structure, process, or outcome
indicators (performance measures); blood
or medication use reviews, etc.;
Targeted studies
or audits are generally retrospective,
criteria-based assessment of care, using
document review (or a combination of
document and encounter/claims data) and
focusing on structure, process, and/or
outcome;
Nermin Ghith, B.S., PGDip.TQM, C.P.H.Q
DATA COLLECTION

Types
Service-specific studies,
based on the major services provided, functions
performed, and problems identified or suspected
(usually document review or check sheets/logs for
processes);
Generic screening (events/occurrences/incidents):
Reporting each occurrence of a predetermined list
of possible untoward events, typically used for
actual or potential adverse and sentinel events and
tracking over time for patterns/trends;

Nermin Ghith, B.S., PGDip.TQM, C.P.H.Q

 DATA COLLECTION

Types:
Medical records:
Inpatient, outpatient, emergency, home
health, etc.;
Summary reports of patient and staff
occurrences
(incidents or critical events), clinical
complications, infections, utilization,
services, committees, research,
organizational evaluations, credentialing,
special studies;

Nermin Ghith, B.S., PGDip.TQM, C.P.H.Q

DATA COLLECTION

Types:
Daily logs (surgical, neonatal, emergency,
cancer registry, urgent care, clinic, etc.);
Financial reports, including case mix,
claims, reimbursements, denials, costs per
case, etc.;
Direct observation and referral: Informal or
criteria-based surveillance of process of
care and compliance with established
procedures or standards;
Surveys of/interviews with patients, staff:
Written or face-to-face questions
concerning perception of care delivery,
outcomes, problems;
Nermin Ghith, B.S., PGDip.TQM, C.P.H.Q
DATA COLLECTION

Types:
Reports from external agencies or
reference databases (CBAHI, Joint
Commission, NCQA, CMS, etc)
Monthly data logs, check lists, etc. of
statistical data to be tracked over time, e.g.,
number of encounters/admissions, referrals
or transfers, C-Sections/ VBACs, Radiology
procedures, deaths, newborns <500 gms,
etc. Rates can also be calculated and
tracked, then summarized and displayed in
graph form.

Nermin Ghith, B.S., PGDip.TQM, C.P.H.Q

DATA COLLECTION
TOOLS

Considerations:
Keep the tool as short and simple as
possible;
Include all data elements necessary to
monitor the specified issue/indicator;
Consider computerizing whenever
feasible;
Provide appropriate definition of terms
and key for using the tool.

Nermin Ghith, B.S., PGDip.TQM, C.P.H.Q

DATA COLLECTION
TOOLS
Checklist for adequacy of the tool:
Does the tool really measure the process or
aspect of care and its indicator?
Will you get the information you really
need?
Will you get more than you need?
Will the data you get be interpretable? Will
it help to gather other data to facilitate
interpretation, e.g., age, weight, secondary
diagnosis, etc.?
Will data collection be a manageable task
with the tool? Too much data? Too much
time? Can it be cut down?

Nermin Ghith, B.S., PGDip.TQM, C.P.H.Q

DATA COLLECTION
TOOLS
Types of data collection tools:
Data Sheet or Work Sheet:
Form for recording data; requires subsequent
processing for analysis and interpretation;
Check sheet:
Form for recording data; designed to facilitate
interpretation directly from form;
Survey:
Form for getting input from a large group;
Interview or Focus Group:
Questionnaire format; can be open-ended
discussion to obtain input from people;
Download:
Automated retrieval from a computerized data
source.
Nermin Ghith, B.S., PGDip.TQM, C.P.H.Q
 ORGANIZATION OF
DATA
Establish a process for aggregation and
summarization of the collected information that is
consistent with its nature and relative impact on
patient care:
Groups/persons responsible
Time frames for tabulation, display of raw data,
statistical analysis, if any, and reporting
Aggregate and display data, summarizing and
trending over time to:
Determine the type, cause, or extent of a problem
Determine the type and cause of best practices
Summarize data in ways to permit meaningful
interpretation and formulation of accurate
conclusions regarding the quality of patient care and
services;
Use graphic display whenever reasonable to
enhance the analysis process.

Nermin Ghith, B.S., PGDip.TQM, C.P.H.Q