Why Measure Training Effectiveness? Measuring the effectiveness of training programs consumes valuable time and resources.

As we know all too well, these things are in short supply in organizations today. Why should we bother? Many training programs fail to deliver the expected organizational benefits. Having a well-structured measuring system in place can help you determine where the problem lies. On a positive note, being able to demonstrate a real and significant benefit to your organization from the training you provide can help you gain more resources from important decision-makers. Consider also that the business environment is not standing still. Your competitors, technology, legislation and regulations are constantly changing. What was a successful training program yesterday may not be a cost-effective program tomorrow. Being able to measure results will help you adapt to such changing circumstances. The Kirkpatrick Model The most well-known and used model for measuring the effectiveness of training programs was developed by Donald Kirkpatrick in the late 1950s. It has since been adapted and modified by a number of writers, however, the basic structure has well stood the test of time. The basic structure of Kirkpatrick's four-level model is shown here. Figure 1 - Kirkpatrick Model for Evaluating Effectiveness of Training Programs

Level 4 - Results

What measurable organizational benefits resulted from the training in terms such as productivity, efficiency and sales revenue?

Level 3 - Behavior

To what extent did participants change their behavior back in the workplace as a result of the training?

Level 2 - Learning

To what extent did participants improve knowledge and skills and change attitudes as a result of the training?

Level 1 - Reaction

To what extent did the participants find the training useful, challenging, wellstructured, organized, and so on?

An evaluation at each level answers whether a fundamental requirement of the training program was met. It's not that conducting an evaluation at one level is more important that another. All levels of evaluation are important. In fact, the Kirkpatrick model explains the usefulness of performing training evaluations at each level. Each level provides a diagnostic checkpoint for problems at the succeeding level. So, if participants did not learn (Level 2), participant reactions gathered at Level 1 (Reaction) will reveal the barriers to learning. Now moving up to the next level, if participants did not use the skills once back in the workplace (Level 3), perhaps they did not learn the required skills in the first place (Level 2). The difficulty and cost of conducting an evaluation increases as you move up the levels. So, you will need to consider carefully what levels of evaluation you will conduct for which programs. You may decide to conduct Level 1 evaluations (Reaction) for all programs, Level 2 evaluations (Learning) for "hard-skills" programs only, Level 3 evaluations (Behavior) for strategic programs only and Level 4 evaluations (Results) for programs costing over $50,000. Above all else, before starting an evaluation, be crystal clear about your purpose in conducting the evaluation. Using the Kirkpatrick Model How do you conduct a training evaluation? Here is a quick guide on some appropriate information sources for each level. Level 1 (Reaction)

completed participant feedback questionnaire informal comments from participants focus group sessions with participants

Level 2 (Learning)

pre- and post-test scores on-the-job assessments supervisor reports

Level 3 (Behavior)

completed self-assessment questionnaire on-the-job observation reports from customers, peers and participant's manager

Level 4 (Results)

financial reports quality inspections interview with sales manager

When considering what sources of data you will use for your evaluation, think about the cost and time involved in collecting the data. Balance this against the accuracy of the source and the accuracy you actually need. Will existing sources suffice or will you need to collect new information? Think broadly about where you can get information. Sources include:

hardcopy and online quantitative reports production and job records interviews with participants, managers, peers, customers, suppliers and regulators checklists and tests direct observation questionnaires, self-rating and multi-rating Focus Group sessions

Once you have completed your evaluation, distribute it to the people who need to read it. In deciding on your distribution list, refer to your previously stated reasons for conducting the evaluation. And of course, if there were lessons learned from the evaluation on how to make your training more effective, act on them! You will be interested in

Our comprehensive toolkit can help you in all stages of your evaluation exercise. From initial planning, data collection and analysis to reporting results, our guide has over 20 customizable tools and templates to make your evaluation task as easy as possible. If you are not sure at which of the four Kirkpatrick Levels to conduct your evaluation, this resource will walk you through the decision process. Plus, you will learn the pros and cons of the various evaluation methods and how to isolate the impact of non-training factors on performance results. If you need to

convert training program benefits to financial results, such as Return on Investment (ROI), our guide contains worksheets for all of the common financial measures. Our toolkit contains everything you need to undertake a credible evaluation exercise in one volume. Training Evaluation Toolkit Training Transfer and Behavior Change Many trainers are faced with the challenge of motivating their training program participants to use the new skills they learned during the program back in their workplace. Whether it is using the new software system to enter customer interactions, acting in a more collaborative manner with other team members or delegating more often to direct reports, this is what the training program is meant to be all about. If the training program does not in the end change workplace behaviors, the money and time spent on training is simply wasted. All trainers have experienced at one time or another training program participants that are neither interested in the program nor motivated to apply the skills and knowledge in their jobs. Here are some tips that you as a trainer can use to help participants want to learn and to transfer that learning to their jobs. Working towards training transfer starts before the training course begins and continues on after the training completes. So, training transfer tasks have been separated into things you can do before, during and after the training is completed. Before Training Get the participants' managers to conduct a pre-course briefing with each participant. If they do not know how, show them. This briefing is the place for each manager to introduce discussion about how the principles, techniques and skills learned will be applied practically once the participant returns from the training event. Their manager is also in the best position to ensure that participants have completed any pre-requisite reading or exercises. Most important of all, the precourse briefing sends a powerful message that the organization cares about the employee's development and is serious about seeing the benefits of training. During Training For training to be effective, the fundamentals of training design will need to have been followed. These basics include selecting the right trainees, matching performance objectives to organizational outcomes, delivering at the right time and choosing the appropriate methods and delivery modes. In addition, the following four points need to be kept in mind during the conduct of the training sessions.

Goal Orientation

Participants actively engage the subject matter when they see a purpose in the learning. This could be reducing time to market for new products or minimizing the company's environmental impact. If there is a sense that the program is "going somewhere", that there is a significant point to the training beyond the training room, many trainees will latch onto that purpose so long as there is a "hook" to make that connection. That "hook" may be personal. It may be the social acceptance that will come from passing the course, or it may be earning the eligibility to join a respected professional association, for example. So, ensure that the organizational objectives of the program are clearly described to trainees at the start of the program and state the WIIFM ("What's In It For Me").

Real Work Relevance

Showing how the program relates directly to people's day-to-day work significantly lifts the level of participant interest in the program. Firstly, demonstrate your expertise in the knowledge and skills being taught, or at least rely on subject matter experts at the appropriate times. Next, use a host of real-life examples and scenarios from the participants' own workplaces. Make role-plays, simulations and examples as true to life as you can. In addition, demonstrate how models, theories and principles need to be contextualized for each workplace situation. Involve participants in making those connections by generating free and frank discussion about how the learning can be applied back on the job. Another fruitful strategy is getting the participants' supervisors and managers to introduce the program or each session. Doing this sends a strong message that the person to whom they report considers the program to be practical and relevant to their work. Even better, where possible, get the participants' supervisor or manager to deliver one or more components of the program.

Practice

Building in opportunities for practice during the training helps to spark participants' interest as they experience new aspects of the skill and builds their self-confidence as they gain success. Factoring in opportunities for practice also increases motivation to use the skills on the job by revealing to participants first hand how the new skills can improve their work on the job. Be sure to intersperse theory with practice sessions. The variety of physical movement and mental activity also helps to maintain participant interest.

Interpersonal Interaction

Learning in the workplace is largely a social activity, in which goals and aspirations are shared, experiences are discussed, different approaches are debated and ways of doing things are demonstrated. In some programs, participants will learn more from each other than from the trainer. And when the participants return to their

workplaces, shared learning between participants will be paramount. Interactions that encourage participation and collaboration will foster motivation and transfer. Things you can do here include asking plenty of questions that gain attention and generate discussion. Ask some questions of the whole group so that they can get to know something about their peers. Whole group questions start to dissolve the initial apprehension that people feel when faced with new people and surroundings. Next, plan for group work in your program design. Use groups consisting of two to six trainees to construct lists, discuss a scenario, role-play and solve problems. Relationships can quickly become fractured and learning blocked through the actions of one or more attention-seeking, disruptive or abusive participants. So, be sure to establish ground rules at the start of the program. Lastly, give trainees rewards to mark their achievements. Success that is recognized helps to develop team spirit, especially if all of the participants are striving toward a common goal. After Training Transferring skills to the workplace at the conclusion of the training program begins with a post-course debriefing. Continuing on from the pre-course briefing, get participants' managers to review with the participants the content of the training and the participants' experiences. The post-course debriefing is an ideal juncture at which to identify, plan and agree with the employee where the skills will be applied and to set specific goals for their application. You will be interested in

This Expert View introduces some of the key activities to include in a successful training transfer strategy. Our comprehensive training transfer guide and toolkit explains how you can implement each one of these tasks and shares other high impact activities that you can do to improve the effectiveness of your training programs. From Training to Enhanced Workplace Performance The Problem of Training Transfer "Transfer of training", as it relates to workplace training, refers to the use put by training participants of the skills and knowledge they learned to their actual work practices. (Note that it is sometimes referred to as "transfer of learning".) Why is this considered so important for managers and training professionals? Consider the following two workplace training scenarios.

John, the Executive Director, caught a number of employees smoking in the lunchroom in spite of the company's no-smoking policy. He instructed the Human Resources department to send all employees on a training session covering the nosmoking policy. Two weeks after the training session, John exploded talking with the Human Resources Manager, "I found a Production Team Leader smoking in the Foyer. That training cost us a packet. You were supposed to fix the problem!" In another company, a new inventory system was installed. Employees in the Purchasing Department were sent off to learn the new software. One month later, the Purchasing Manager finds that only two out of the twelve Purchasing Officers are using the new system. The expected cost savings have not materialized and the Purchasing Manager resolves to take issue with the Training Manager at the next weekly meeting. Does this sound familiar? Experts estimate that somewhat less that twenty percent of training investments lead to some organizational benefit. This anomaly is commonly referred to as the "problem of training transfer". Why is it that such a small proportion of training ends up being used back in the workplace? With increasing marketplace competition, leaner resources and a greater focus on tangible outcomes, more and more managers are asking this question. Transfer of Training Factors How can you increase the transfer of training in your organization? For any given training program, you will need to look into three areas: 1. training participant attributes (intelligence, attitudes) 2. training program design and delivery 3. workplace environment What can you do to enhance the positive impact of each of these factors? Training participant attributes may be influenced when introducing new employees to your organization through an effective recruitment, selection and induction process. Attributes can also be influenced before training begins through pre-qualifying nominees during the registration process. The second factor, training design and delivery, can be made more effective through ensuring that the training program objectives are clearly focused on your organization's priorities and goals. Tied in with this, participants' learning outcomes must be stated in terms of behavior required in the workplace and measurable performance standards. To help you with aligning learning outcomes with organizational objectives, review our practical guide, Writing Learning Outcomes. Along with effective design, to maximize training transfer to the workplace ensure that the training is delivered in accordance with what we know about how adults

learn best. Our Trainer Effectiveness Rating Form included within our Training Management Template Pack is a useful tool that can help you here. The most significant, yet most neglected, factor influencing the extent of training transfer is the third area mentioned above, the employee's workplace environment. What happens before employees attend the training event and what happens after they return to work are the most important variables determining workplace performance following training. If your organization is struggling to see tangible benefits from training, ask yourself these key questions of your last training program: 1. Did instructional designers, trainers and line managers work together in partnership or was work on the program done in isolation with little collaboration? 2. Were non-training solutions seriously considered or was a training request received and an off-the-shelf solution delivered? 3. Were training outcomes stated in behavior and performance terms or were outcomes unstated or stated in fuzzy terms? 4. Were training objectives tied to stated organizational objectives or were they left floating in the organizational ether? 5. Were managers and supervisors actively involved before, during and after the program or was the program divorced from the employee's day to day work? 6. Was post-training support provided back in the workplace, such as coaching and on-the-job aids, or were employees left to flounder with no opportunity to practice? 7. Were new procedures and role expectations clearly communicated to employees or were they left wondering why they were nominated for the program? 8. Were workplace performance expectations agreed with employees prior to the training, or was it back to "business as usual"? 9. Was the training integrated with a well thought-out and implemented change or improvement program, or was the training a single point "silver bullet" solution? 10.Did you measure the organizational impact of the program or rely solely on "happy sheets" for feedback? The PRACTICE Approach to Training Transfer

Business Performance Pty Ltd has consolidated the various attitudes and activities required for maximum transfer into an easy to remember and use model. We call our method the PRACTICE Approach. By focussing on each of the eight key elements, organizations can be assured of maximizing their training investments. These eight key elements of the PRACTICE Approach to improving the transfer of training can be summarized as follows: Procedures say how to perform and why

update relevant policies and procedures before training begins use actual policy and procedural documents during training

Roles & Responsibilities say what level of performance is required

clarify role responsibilities and update relevant role descriptions link learning outcomes to role descriptions

Aids on the job extend the training room into the workplace

replicate training aids on the job encourage employees to use on-the-job aids

Coaching overcomes individual barriers to skill application

plan for and dedicate on-the-job coaching resources train coaches in how to coach effectively

Targets and measurement proves people are performing

agree and set measurable organizational and individual goals link program learning outcomes to organizational and individual goals translate goals into required on-the-job behaviors

Incentives

give a personal reason to perform

modify incentives to reward goal achievement and expected behaviors provide employee feedback frequently and using a variety of methods

Communication informs and involves all stakeholders

communicate information to all appropriate levels in organization use a variety of communication mediums and styles

Engagement motivates participants to apply skills

brief employees before training on purpose and application of program managers and supervisors introduce training and attend sessions review learning after training and identify opportunities for skill application follow up regularly progress on skill application with employee

You will be interested in

Find out more about how to improve transfer of training and our PRACTICE Approach. Check out our comprehensive guide and workbook. It will lead you through applying each element of the PRACTICE Approach with a comprehensive set of guides, checklists and forms. And when you are ready to measure just how effective your training program was in achieving your organization's objectives, you will be able to use the tools provided to evaluate the resulting impact on employee behavior and bottom-line results. From Training to Enhanced Workplace Performance Workplace Environment and Employee Performance An employee's workplace environment is a key determinant of the quality of their work and their level of productivity. How well the workplace engages an employee impacts their desire to learn skills and their level of motivation to perform. Skills and motivation level then influences an employee's:

error rate output rate level of innovation collaboration with others absenteeism, and length of service.

(See the results of research by Towers Perrin, BlessingWhite and Gallup Consulting.) The most important of these workplace environment factors that either lead to engagement or disengagement are shown in the following diagram. A close consideration of each of these factors is also very useful in ensuring that employees apply the skills they learn during training programs once they return to their workplace. Tending to the structural and interpersonal aspects of each of these factors enables employees to apply the required skills in a consistent and habitual way. Figure 1 Workplace factors affecting employee performance

Workplace Performance Factors A Short Guide What does each of these nine key factors mean? The following is a brief explanation of each factor.

Goal-setting

Employees are involved in setting meaningful goals and performance measures for their work. This can be done informally between the employee and their immediate

supervisor or as part of an organization's formal performance management process. The key here is that each employee is actively engaged in the goal-setting process and takes ownership of the final agreed goals and measures.

Performance feedback

Information on how the employee is performing is fed back regularly to employees. This consists of both positive feedback on what the employee is doing right as well as feedback on what requires improvement. The feedback is objective and delivered with the appropriate interpersonal and conflict resolution skills and can be a mix of both informal feedback and feedback delivered as part of a formal performance management cycle.

Role congruity

The role that the employee is required to perform is consistent with their expectations on joining the organization and any subsequent training. The organization's role expectations are typically reflected in formal documents, such as Job Descriptions and Role Specifications. These expectations are consistent with tasks allocated by the employee's immediate supervisor.

Defined processes

The organization constrains the variability of how work is actually performed through documenting processes and communicating such expectations to employees. The organization verifies on a regular or random basis that the work is actually performed in the way required.

Workplace incentives

The organization has determined what motivates its employees and has set up formal and informal structures for rewarding employees that behave in the way required. Rewards may consist of a mix of internal rewards, such as challenging assignments, and external rewards, such as higher compensation and peer recognition.

Supervisor support

Immediate supervisors act as advocates for employees, gathering and distributing the resources needed by employees in order for them to be able to do a good job and providing positive encouragement for a job well done. Supervisors display the interpersonal skills required to engage employees and enhance their selfconfidence.

Mentoring/coaching

Skilled and respected people are available to employees to help them perform better in their current role and to assist them develop further into a future role. Mentors and coaches may be internal to an organization or external. Either way, they possess the necessary facilitation skills to assist employees develop and apply new sills.

Opportunity to apply

Time and material resources are available to employees, enabling them to perform to the best of their ability. Individual workloads and organizational systems and processes do not hinder employees from applying established skills or from practicing newly learned skills.

Job aids

The work environment is set up so that templates, guides, models, checklists and other such workplace aids are readily available to help minimize error rates and customer dissatisfaction. Each of these nine factors is significant in its own right. Taken together, they form a powerful coalition for maximizing the motivation and productivity of your employees. What actions are you taking now to capitalize on these crucial factors for optimum performance in your workplace? You will be interested in

Paying close attention to the above workplace environment factors will heighten employee motivation to apply their skills and hence improve your workplace productivity. To find out more about increasing the effectiveness of your training programs using these factors, check out our toolkit and guide. From Training to Enhanced Workplace Performance

The above factors are also important for getting the most out of your organizational change programs. For help in translating change initiatives into real organizational performance gains, check out our toolkit and guide.

Managing Change in the Workplace