Practice Based Learning and Improvement

Stephen J. Kimatian MD
Assistant Professor of Anesthesiology and Pediatrics The Penn State, Milton S. Hershey Medical Center

PRACTICE-BASED LEARNING AND IMPROVEMENT

Residents must be able to investigate and evaluate their patient care practices, appraise and assimilate scientific evidence, and improve their patient care practices.

Residents are expected to:

analyze practice experience and perform practice-based improvement activities using a systematic methodology locate, appraise, and assimilate evidence from scientific studies related to their patients health problems obtain and use information about their own population of patients and the larger population from which their patients are drawn

Residents are expected to:

apply knowledge of study designs and statistical methods to the appraisal of clinical studies and other information on diagnostic and therapeutic effectiveness use information technology to manage information, access on-line medical information; and support their own education facilitate the learning of students and other health care professionals

The Mandate
Develop the skills of critical self examination and self improvement Become life long learners

The Question

How do you evaluate yourself if you do not know what the standard for competence is?

Must have an established standard.

How do you establish the standard for competence?

ACGME competency as applied to practice? Practice as applied to ACGME competency?

Perspective

What is a consultant?

Knowledge Skill Judgment Open Ended

Easy to use but low on detail

How do you evaluate?

Highly Specific
Complex and time intensive

Evaluation Tools
Easy to understand Intuitive to use Generate useful information

For self evaluation For evaluation of competence For focused remediation

Minimize Bias

The Evaluation Project

Goal to develop an evaluation tool that maintains a global perspective while allowing detailed analysis of specific areas of strength or weakness. A tool that is simple to use

Standardized language for the discussion of competency.

The Evaluation Project

Phase 1 Define Consultant using discrete statements. Phase 2 Characterize defining statements. Phase 3 - Create a Database (rubrics) that defines Competency Phase 4 - Manipulate the data base to create evaluation tools.

The Project - Phase 1

Define Consultant in discrete statements. Intravenous Access

Display knowledge of indications for IV access Develop plan for IV placement considering medical and surgical factors effecting gauge, position, and timing of IV placement. Communicate with patient throughout procedure showing respect and compassion Display knowledge of aseptic technique and universal precautions Display proper aseptic technique and universal precautions

The Project Phase 2

Characterize

defining statements

Subspecialty (general, peds, cardiac .) Phase of Care (preop, op, postop,clinical,..) Core Competency (pt care, pbli, comm..) Educational Domain (knowledge / skill / judgment) Training Level (PGY1, PGY2, PGY3. consultant)

The Project Phase 2 (cont)

Intravenous Access

Display knowledge of indications for IV access

General / Preop / Pt Care / Knowledge / PGY 1

Develop plan for IV placement considering medical and surgical factors effecting gauge, position, and timing of IV placement
General / Preop / Pt Care / Judgment / PGY 1

Communicate with patient throughout procedure showing respect and compassion

General / Preop / Pt Care / Skill / PGY 1

Display knowledge of aseptic technique and universal precautions

General / Preop / Pt Care / Knowledge / PGY 1

Display proper aseptic technique and universal precautions

General / Preop / Pt Care / Skill / PGY 1

The Project Phase 3

Create a Database (rubrics) that define Competency

IV Access
Patient Care: Residents must be able to provide patient care that is compassionate, appropriate, and effective for the treatment of health problems and the promotion of health.

Cognitive / Knowledge
PGY1/2

Skills / Behaviors
-Communicate with patient showing respect and compassion -Display proper aseptic technique and universal precautions
PGY1/2 CA2

Attitude / Judgment
PGY1/2

-Display Knowledge of indications for IV access -Display knowledge of aseptic technique and universal precautions
CA2

-Develop plan for IV placement considering medical and surgical factors effecting gauge, position, and timing of IV placement
CA2

- Display knowledge of advance IV access modalities

-Demonstrate skills needed to establish IV access in difficult patient - Effectively Place IV in uncomplicated patient with little or no assistance
CA3 CA3

CA3

The Project Phase 4

Develop Web based evaluation form

Rotating selection of 10 15 questions Substandard rating result in automatic expansion of question Resident performance tracked
System tracks which competencies have been evaluated Comparison of resident self evaluation vs. faculty evaluation Deficiencies characterized to create remediation plan focused on specific problems and with identifiable points for reevaluation

Faculty Clinical Teaching and Evaluation tracked

Puts evaluations in perspective with norms Identify Biases / Halo Effect Part of 360 degree evaluation

Project Time Line

Phase 1 / 2

Underway for General Anesthesia Working model by June 03 Subspecialties added by Fall 03 First meeting with program team 15 April Database parameters by June 03 Phase 3 Beta testing by Sept 03 Phase 4 Beta testing by Fall 03

Phase 3 / 4

Questions ?