Sie sind auf Seite 1von 19

Evaluating Training Programs

The Four Levels


Based on the work of
Dr. Donald L. Kirkpatrick
& Dr. Myron A. Eighmy

Muhammad Awab
M&E Specialist, TESP Secretariat
Higher Education Commission, Islamabad
1

Objectives
Upon completion of this presentation
you will be able to:
State

why evaluation of programs is


critical to you and your organization.
Apply Kirkpatricks four levels of
evaluation to your programs.

Why Evaluate?

Determine what instructional strategies


work

Determine the effectiveness of the


program design

Learn the needs for improvement

Four Levels of Evaluation


Kirkpatrick Model

During

program evaluation

Level

Level

Post

Reaction
Learning

program evaluation

Level

Behavior

Level

Results
4

Level
A

: Reaction

customer satisfaction measure

Were the participants pleased with the


program

Perception: if they learned anything

Likelihood of applying the content

Effectiveness of particular strategies

Effectiveness of the packaging of the course

Level 1: Examples
Quantitative:

The contents of this training were relevant and applicable to my work.


(Please circle a number)

Strongly Disagree Disagree


1 2

Dont Know Agree

Strongly Agree

Level 1: Examples
Qualitative:
The

best part of this program was

The

one thing that could be improved


the most ..

Level

: Learning

What did the participants learn in the


program?

What exactly did the participant learn and not


learn?

Instrument: Pretest & Posttest Forms

Requires developing specific learning


objectives to be evaluated.

Level 2: Examples

Develop a written exam based on the desired


learning objectives.
Use the exam as a pretest
Provide participants with a
worksheet/activity sheet that will allow for
tracking during the session.
Emphasize and repeat key learning points
during the session.
Use the pretest exam as a posttest exam.
Compute the posttest-pretest gain on the
exam.
9

What makes a good test?


The

only valid test questions emerge


from the learning objectives.
Consider writing main objectives and
supporting objectives.
Test questions usually test
supporting objectives.
Ask more than one question on each
objective.
10

Example

Match the following to the choices below

___ Reaction Level


___ Learning Level
A. Changes in performance at work
B. Participant satisfaction
C. Organizational Improvement
D. What the participant learned in class

11

Pre & Post Test Analysis

12

Level

How

: Behavior

the training affects performance?

The

extent to which change in behavior


occurred.

Was

the learning transferred from the


classroom to the real world?

13

Level 3: Guidelines for


Evaluating

Measure on a before/after basis


Allow time for behavior change (3-6
months)
Survey or interview one or more who are
in the best position to see change.

The participant/learner

The supervisor/mentor

Subordinates or peers

Others familiar with the participants actions.


14

Level 3: Instruments
Observation
Survey

or Interview

Participant

and/or others

Performance
Before

benchmarks

and after

Control

group

15

Level

: Results

Impact

of education and
training on the organization or
community.

The

final results that occurred


as a result of training.

The

ROI for training.


16

Level 4: Examples

How did the training save costs

Did work output increase

Was there a change in the quality of work

Did the social condition improve

Did the individual create an impact on the


community

Is there evidence that the organization or


community has changed.
17

Level 4: Guidelines for


Evaluating

Measure before and after

Allow time (6-12 months or even more) for


change to take place

Repeat at appropriate times

Use a control group if practical

Consider cost vs. benefits of doing Level Four

Remember, other factors can affect results

Be satisfied with Evidence if Proof is not


possible.
18

Any Questions?

Thank you

19