Kirkpatrick

The Four Levels

s Reaction s Learning s Behavior s Results

All about Kirkpatrick

In 1959, Kirkpatrick wrote four articles describing the four levels for evaluating training programs. He was working on his dissertation for a Ph.D. when he came up with the idea of defining evaluation. Evaluation, as according to Kirkpatrick, seems to have multiple meanings to training and developmental professionals. Some think evaluation is a change in behavior, or the determination of the final results.

All about Kirkpatrick (continued)

s Kirkpatrick

says they are all right, and yet all wrong. All four levels are important in understanding the basic concepts in training. There are exceptions, however.

Kirkpatrick: Evaluating Training Programs

s What s How s How

is quality training? do you measure it? do you improve it?

Evaluating
The reason for evaluating is to

determine the effectiveness of a training program. (Kirkpatrick, 1994, pg. 3)

The Ten Factors of Developing a Training Program

1. 2. 3. 4. 5. Determine needs Set objectives Determine subject content Select qualified applicants Determine the best schedule

The Ten Factors of Developing a Training Program

6. Select appropriate facilities 7. Select qualified instructors 8. Select and prepare audiovisual aids 9. Co-ordinate the program 10. Evaluate the program

Reasons for Evaluating

Kirkpatrick gives three reasons why there is a need to evaluate training: 1.To justify the existence of the training department by showing how it contributes to the organizations objectives and goals.

Reasons for Evaluating

2. To decide whether to continue or discontinue training programs. 3. To gain information on how to improve future training programs. (Kirkpatrick, 1994, pg. 18)

The Four Levels

s Reaction s Learning s Behavior s Results

The Four Levels represent a sequence of ways to evaluate (training) programs.As you move from one level to the next, the process becomes more difficult and time-consuming, but it also provides more valuable information. (Kirkpatrick, 1994, pg. 21)

Reaction:
is the measuring of the reaction of the participants in the training program. is a measure of customer satisfaction. (Kirkpatrick, 1994, pg. 21)

Learning:
is the change in the participants attitudes, or an increase in knowledge, or greater skills received, as a result of the participation of the program.

Learning
The measuring of learning in any training program is the determination of at least one of these measuring parameters: s Did the attitudes change positively? s Is the knowledge acquired related and helpful to the task? s Is the skill acquired related and helpful to the task?

Behavior
Level 3 attempts to evaluate how much transfer of knowledge, skills, and attitude occurs after the training.

The four conditions Kirkpatrick identifies for changes to occur:

s Desire

to change of what to do and

s Knowledge

how to do it
s Work

in the right climate for (positive) change

s Reward

When all conditions are met, the employee must:

s Realize

an opportunity to use the behavioral changes. s Make the decision to use the behavioral changes. s Decide whether or not to continue using the behavioral changes.

When evaluating change in behavior, decide:

s When s How s How

to evaluate

often to evaluate to evaluate

Guidelines for evaluating behavior:

s Use

a control group s Allow time for change to occur s Evaluate before and after s Survey/interview observers s Get 100% response or sampling s Repeat evaluation, as appropriate s Consider cost versus benefits

Results
Level 4 is the most important and difficult of all - determining final results after training.

Evaluation Questions:
Increased production? s Improved quality? s Decreased costs? s Improved safety numbers? s Increased sales? s Reduced turnover? s Higher profits?
s

Guidelines for evaluating results:

Use a control group. s Allow time for results to be achieved. s Measure before and after the program. s Repeat the measurements, as needed. s Consider cost versus benefits. s Be satisfied with evidence if proof is not possible.
s

Case Study #1

INTEL CORPORATION

Intels Compromise of the Kirkpatrick Model

Intel uses the four-level model as an analysis instrument to determine the initial training needs and design of its training program; as well as using the model for evaluations.

Intels Compromise of the Kirkpatrick Model

Their uniqueness of using the model is in the fact that the designers of the training program worked backwards in the analysis of the training, starting with Level Four.

The Model
This implementation of the Kirkpatrick Model stands as vivid testimony to the versatility of the model as a training tool, and in developing fledgling training programs.

The Model
It also reflects the open-mindedness of the senior executives at Intel for their infinite use of the model and the use of the genius and visions of Kirkpatrick.

How Intel applies the analysis to their training program

Level Four Determine the organizations structure and future needs. Level Three. Change the environmental conditions and employee conditions to improve business indicators.

How Intel applies the analysis to their training program

Level Two. Design a training program that would ensure a transfer of deficient skills and knowledge. Level One. Use a questionnaire, according to their skill level, that would instruct and inspire training participants.

How Intel applies evaluation to their training program

Level One - Questionnaire. Level Two - Demonstrate competency, create action plans through group simulations. Level Three - Follow-up to determine if action plans were met (specific steps to implement concepts of what was learned). Level Four - Ongoing process of tracking business indicators.

Case Study #2

ST. LUKES HOSPITAL

St. Lukes is unique s Evaluation

of outdoor-based training program, not classroom. s Results analyzed statistically to determine the significance of any change. s Evaluation led to recommendations for future programs.

The New Questionnaire

Used before attendance in the program. s Used 3 months after completion of the program. s Used again 6 months after completion of the program.
s (Communication showed statistically significant improvement, and Group Effectiveness showed statistically significant change.)

Kirkpatricks 4 Levels of Evaluation are:

Level 1 - Reaction: how participants reacted to the program. s Level 2 - Learning: what participants learned from the program. s Level 3 - Behavior: whether what was learned is being applied on the job. s Level 4 - Results: whether that application is achieving results.
s

Post-test Questions
(1) Name three ways evaluation results can be measured. (2) Do all 4 Levels have to be used? (3) Do they have to be used in 1,2,3,4 order? (4) Is Kirkpatricks method of evaluation summative or formative? (5) Which developmental view does Kirkpatrick use? (discrepancy,

IF YOU THINK TRAINING IS EXPENSIVE, TRY IGNORANCE. and, remember, the definition of ignorance is repeating the same behavior, over and over, and expecting different results!