1

Program Evaluation Rubric

Laine Willkom

AET/570

June 19, 2017

Dr. Joseph Walters

 2

Program Evaluation Rubric

Academic Counselors recently completed a comprehensive training pertaining to coaching University students through issues

that impact their ability to be successful in college. The focus of the learning program was to teach Counselors developmental

coaching techniques with the added purpose of reducing transactional and purely instructional calls (InsideTrack, 2017). Academic

Counselors learned ways to appropriately assess student situations and needs by asking open-ended questions and by actively

listening. The training also included strategies to build student motivation and Counselor-student relationships as well as how to

support students to resolve issues through collaborative, intentional conversations (InsideTrack, 2017).

If applying the Kirkpatrick evaluation model to this program, the Reaction level would be evaluated through a post-training

survey to measure Academic Counselors feelings on the value of the training experience (Kirkpatrick, 2008). The second level of the

model is Learning (Kirkpatrick, 2008). The training and student services leadership staff designed goals and objectives as a starting

point to measure changes in the participants knowledge, skills, attitudes, and beliefs (InsideTrack, 2017). The rubric created for the

training program aids in identifying behavioral changes, which is the third level of Kirkpatricks model (Kirkpatrick, 2008). The

Behavior level encapsulates the purpose of recording and listening to student interaction calls. Trainers and leadership need to confirm

how well Academic Counselors are applying the new models and strategies. The final level; Results, is where educators and

management will holistically analyze the results of the training. This stage includes compiling and reviewing data from the post-

training survey in addition to the scored call interactions, which are based on a calibrated rubric. Analyzing results to ensure desired

outcomes are attained is vital to the ongoing success of the Universitys learning and development programs.
 3

Goals and Objectives

The call coaching training was extensive but focused on three main areas: assessing student concerns and issues effectively,

building motivation and relationships, and advancing a student forward to create his or her plan. The training department, in

cooperation with student services leadership, created goals and objectives, which they will use to determine the effectiveness of the

training program through authentic, performance-based evaluation. Training staff and student services leadership calibrated a rubric to

assess recorded Student-Counselor call interactions.

Goals

To ensure comprehensive student support is attained, training staff determined that the over-arching goals for this training are:

assessing student concerns and issues effectively, building student motivation and the relationship between the student and the

Counselor, and advancing students forward to form an achievable plan. These concepts were divided into four learning goals, which

include:

1) Academic Counselors will learn how to holistically assess student issues.

2) Academic Counselors will engage students to increase motivation.

3) Academic Counselors will build relationships and trust with students.

4) Academic Counselors will support student advancement.

Objectives
 4

Objectives include more specific requirements that can be measured. The first two objectives align to the first goal focused on

assessing student issues. The second goal and objectives 2a and 2b pertain to increasing student motivation by supporting student

reflection and by connecting student actions and attitudes to long-term success and goals. The third goal aligns with objectives 3a and

3b, which focus on building solid relationships by using instructional and developmental coaching models, when appropriate based on

the situation and student. The objectives also focus on building trust through providing accurate guidance and information and

maintaining a professional tone and demeanor during all interactions. The final goal and objective deals with collaborative student

advancement through partnering to build feasible plans to progress forward.

1a) Academic Counselors will assess student needs through call interactions at a proficient or higher rating, based on a

rubric.

1b) Academic Counselors will articulate the goal and reason for the call at a proficient rating, based on a rubric.

2a) Academic Counselors will uncover student motivation by asking probing questions at a proficient rating, based on a

rubric.

2b) Academic Counselors will use developmental coaching techniques to connect student motivation and actions to long-

term goals at a proficient rating, based on a rubric.

3a) Academic Counselors will build relationships and trust with students by correctly using instructional and

developmental coaching techniques and by providing accurate information at a proficient rating, based on a rubric.
 5

3b) Academic Counselors will maintain appropriate tone and professional demeanor throughout the entire call at a

proficient rating, based on a rubric.

4) Academic Counselors will support student advancement through building collaborative, achievable action plans at a

proficient rating, based on a rubric.

Call Interaction Rubric

The quality assurance department will listen to Academic Counselor calls and score them based on a rubric. The

training and development department, in partnership with student services senior leadership, formed and calibrated the rubric. Quality

assurance evaluators and managers experienced the same training Academic Counselors received as well as training on the new call

evaluation form and rubric by which to evaluate Counselor calls.

Criterion Needs Improvement Proficient Exceeds Expectations

Assesses Student Needs
Framing the Interaction
(1)
Did not assess student needs,
asked primarily close-ended
questions, or unable to correctly
identify student need(s)
Did not frame the call to ensure
both student and Counselor
understood the direction and goal
of the coaching session or
components of frame (context,
(2)
Asked both closed and open-
ended questions to identify the
students need(s), repeated
information to student to confirm
understanding, adequate listening
skills, and minimal interruptions
Components of a call frame
(context, history, and goal)
included and appropriately used
Score
(3)
Asked probing, closed and open-
ended questions in multiple focus
areas (student experience,
external factors, academics,
effectiveness, college processes,
and forward progress)
demonstrated excellent active
listening skills, no interruptions
Thorough explanation of the
purpose of the call and why that
purpose is important to the
student, including appropriate call
frame (context, history, and goal)

Engaging Students to Build Motivation
Building Relationship and Trust
Tone and Professionalism
history, and goal) missing
Unable to uncover existing
motivation, did not assist student
to develop a vision for the future,
did not effectively link immediate
actions to goals
Did not engage with student on a
personal level, did not normalize
student feelings or concerns, did
not or inappropriately shared
personal experiences, inaccurately
advised student
Improper or unprofessional tone,
pitch, volume, and pace, or tone is
argumentative, rude, or harsh,
repeated use of internal jargon or
overly-casual lingo or profanities
6
Asks questions to uncover
motivation for school and links
academic actions to motivation
and long-term goals
Expressed willingness to assist
student, partnered with student
on proposed resolution,
normalized feelings or concerns,
confirmed student buy-in to plan,
resolved student issue during the
initial call or appropriate student
and Counselor action items and
follow up set, provided accurate
information and resources
Proper tone, pitch, volume, and
pace used throughout the call,
courteous and professional words
and phrases used, refrained from
using internal or complicated
jargon or lingo
call frames and transparent role
articulation repeated throughout
the call to direct the coaching
session
Asks questions to uncover
motivation for school, links
academic actions to motivation
and long-term goals, recognizes
student progress, encourages the
student to connect current
situation to a vision for the future,
developmental techniques used
Expressed eager desire to assist
student, partnered with student
on proposed resolution,
normalized student feelings or
concerns while also supporting
student to reflect on behaviors
and attitudes through questioning
and personal relating, confirmed
student buy-in to plan, resolved
student issue during the initial call
or appropriate student and
Counselor action items and follow
up set, provided accurate
information and resources,
expressed confidence and support
to achieve goals
Proper tone, pitch, volume, and
pace used throughout the call,
friendly, polite, and professional
words and phrases used,
conversational style, refrained
from using internal or complicated

No student advancement plan was
developed, or Counselor used only
instructional strategies without
Collaborative Student Advancement
involving student on plan
formation, no accountability item
or follow up set
7
Counselor partnered with student
to confirm an issue with which to
move forward, mostly
instructional support given to
move student forward with
elements of developmental
coaching, Counselor sees the
situation through to resolution
jargon or lingo, adapted approach
to support based on the students
unique needs and personality
Counselor partnered with student
to confirm an issue with which to
move forward, support given to
allow student to realize next step
and best course of action,
Counselor supports student to
confirm accountability partner,
mostly developmental techniques
used with educational and
instructional guidance given,
when appropriate, Counselor sees
the situation through to resolution
TOTAL
 8

References

InsideTrack. (2017). Coaching. Retrieved from https://www.insidetrack.com/

Kirkpatrick, D. L. (2008). Evaluating training programs. Retrieved from ProQuest.