You are on page 1of 7

EDAE 639 Instructional Design Alexandra Mockler

Formative Evaluation Plan


There are four stages of formative evaluation: design reviews, expert reviews, learner validation and ongoing evaluation (Smith & Ragan, 2005). Design reviews were continuously

conducted as the entire instructional design process was broken-down into each particular phase. Although primary and secondary objectives were identified earlier on in the design process, developing learning assessments and the associated content forced me to rethink if I had the correct written objectives. Since most of the design review was completed at the front-end of this process, the formative evaluation plan will focus on the last three stages. The last three stages will be completed in a 4 week period to ensure enough time has been allotted for review and amendments to existing content. Expert Review National Field Coach Director During an expert review, its essential to have the subject matter expert to examine the content of the instructional material for accuracy and completeness (Smith & Ragan, 2005). The ideas and content for the applied sales training course were referenced by a consultative selling model. As a result, it will be important to have our National Field Coach Director review the information since part of his role is to reinforce the main concepts of the selling model on a daily basis with other field coaches. This content will be sent to the Director ahead of time, prior to testing it out with any target learners. Ideally, we need to establish if the applied sales training course was designed with the same perspective as what is currently being taught in the field by other sales managers and field coaches. Although it is important that the content be up-to-date and accurate; the level of appropriateness must also be evaluated to ensure we are relaying the same message to all field agents from coast to coast. The

National Field Director will be given 2 weeks to review the content, while the course designers make

EDAE 639 Instructional Design Alexandra Mockler themselves available for any follow-up questions or comments. We have allocated time to change

course content and associated activities, particularly if they are not congruent with the sales positioning that is currently promoted in the field. Other design changes may need to be incorporated in a later stage once the material has been tested with a live audience. Learner Validation Small Group Evaluation and Field Trials Due to time requirements and resourcing constraints for the applied sales training course development, learner validation will be conducted in small group evaluation setting with some potential for field trials as time elapses. The small group evaluation will be conducted 2 weeks prior to launch, ensuring there is enough time for content and instruction adjustments. The small group will consist of approximately 5 new hires of Company XYZ. These new hires are a great target audience as they will have just experienced the orientation program without any applied sales training. That said, they have also been operating in the field for a few weeks and have a better sense of what is required of them to effectively interact with referral sources and direct clients. The small group will also include a few seasoned sales people who have been identified as high producers. From this perspective, these high producing sales agents will have a good understanding of the different skills sets taught in the applied sales training program and which are critical to master for continued success. Specific questions that will be asked during this evaluation phase include the following: 1) Do you feel that the learners overall success in this course depends on their former sales skills and background? 2) How did you feel about the course instruction? (Positive or Negative?) & Why? 3) How would you rate the course materials (slides and handouts) in terms of relevancy and structure? 4) Were the course instructions clear and concise? Explain

EDAE 639 Instructional Design Alexandra Mockler 5) How were the practice exercises (Too easy, too few, about right, too difficult, too many). 6) Rate the pace of the applied sales training course? (Too fast, too slow, just right?) Although some probing will be done with the aforementioned questions, most of the feedback from this particular evaluation will take place through a video-tape, by analyzing language, verbal and non-verbal cues from the active participants. Evaluating the interest level of the audience will be particularly important as this will impact the entire onboarding program. The analysis of the qualitative feedback will be most important to capture in this evaluation. Takeaways Regardless of what the feedback is during the small-group evaluation, there is merit to revisiting the content and delivery of the course to ensure that the most highly rated areas are accentuated and that areas that were identified as problematic or lacking in content or engagement are reviewed for additional changes. For example, although the Strategic Questioning course was timed at 45 minutes, I may uncover in the small-group evaluation that certain activities require more time in order to truly grasp the skill set being instructed. If this should be the case, more time would be allocated for this segment of the training program to ensure sales skills are appropriately ascertained. The idea of looking at possible pre-requisite courses will also be considered depending on what level the applied sales training is rated at, either beginner, intermediate or advanced. Since the learner audience will be made up of people at different levels of sales skills, I am confident that the feedback will be nicely balanced with suggestions on how to find a middle ground.

EDAE 639 Instructional Design Alexandra Mockler

Summative Evaluation Plan


Due to the stakeholders involved in this applied sales training program, the summative evaluation will be conducted at the end of the first session. Realizing that it is best to provide a summative evaluation once the training has been conducted for a second or third time, the training department simply does not have the luxury of additional time to evaluate this program. The specific questions that will be used in the summative evaluation are listed below (Appendix A). Other than the training department, the largest stakeholder will be the sales team, including management staff such as the VP of Sales, Regional Vice-Presidents and Directors of Sales. It will be critical to keep them included in the training process as much as the evaluation process. As it stands today, any member of the senior member of the sales team is invited to join in during the training, or join the new hires in evening events/dinners. This gives them an opportunity to speak to the new recruits and ask them candid questions about how they feel the training is going. Theyre also privy to any assessments and evaluations we conduct with the new hires. As originally identified, the recent sales production reports illustrate that newly hired sales agents are not meeting case quotas in the first few months of hire. Their inability to quickly reach sales targets is negatively impacting their ability to get a solid start in building their practice, impacting overall confidence and motivation, furthermore impacting retention and decelerating the growth rate of the overall company. The applied sales training program was introduced as part of the New Hire Training program to help new mobile sales agents reach sales targets sooner. The training courses aim to properly equip new agents with the sales tools they need to be successful during their initial meetings with various referral sources and direct clients.

EDAE 639 Instructional Design Alexandra Mockler As a result, senior management is expecting to see a major shift in the behavior of newly hired sales agents once they have attended the New Hire Training program. The summative evaluation will play a significant role in the future planning and delivery of this program. Depending on the feedback and the attitude of those who have gone through the New Hire Training, we may be able to receive additional funding to create other courses and even invest in external vendors that can bring other expertise into the business. Certainly, it will come down to understanding the bottom-line; assess

whether sales from new hires increased since the introduction of the applied sales training. That said, return on investment (ROI) will not be accurately measured until we have run the program numerous times and used productivity tools to track their success.

EDAE 639 Instructional Design Alexandra Mockler

Appendix A
Summative Evaluation Strategic Questioning Strongly Disagree 1 # 1 2 3 4 5 6 7 8 9 10 11 Disagree 2 Neutral 3 Agree 4 1 1 1 1 1 1 1 1 1 1 1 1 Strongly Agree 5 2 2 2 2 2 2 2 2 2 2 2 2 3 3 3 3 3 3 3 3 3 3 3 3 4 4 4 4 4 4 4 4 4 4 4 4 5 5 5 5 5 5 5 5 5 5 5 5

Strategic Questioning Applied Sales Training Course The course materials (handouts and resources) were useful The facilitator was professional, well-prepared, and presented the material clearly and effectively The facilitator responded effectively to questions regarding session content The pace of the workshop was appropriate The overall course content was relevant The workshop increased my knowledge and understanding of topics covered I am confident that I can use my new learning in my role I will be supported in using my new learning in my role I would describe this program as credible I would describe this program as practical I would describe this program as essential

23. How can we make this training more relevant and useful to future participants?

24. What were the strengths of this training session and how will you use the skills you learned in your new role?

EDAE 639 Instructional Design Alexandra Mockler

References
Smith, P. L., & Ragan, T. J. (2005). Instructional Design. John Wiley & Sons, Inc.