LOYOLA UNIVERSITY CHICAGO

GRADUATE ASSISTANT FORMATION COUNCIL

ORIENTATION PROGRAM
EVALUATION
Naseeb Bhangal | Kevin Krauskopf

INTRODUCTION
The Graduate Assistant Formation Council (GAFC) is a unique program that
serves Graduate Assistants working at Loyola University Chicago. Founded in
2009 by former Vice-President, Dr. Rob Kelly, GAFC was initially utilized to
reconcile outstanding concerns between Graduate Assistants and their
supervisors. Since 2009, GAFC has expanded its efforts towards helping Graduate
Assistants succeed in their assistantship role as well as achieve their professional
goals.
The following is a proposed evaluation to assess the Graduate
Assistant Formation Councils Orientation program for its shortterm impact on Graduate Assistants and their professional
development goals. This evaluation plan consists of:

Program Context
Qualitative Approach
Quantitative Approach
Limitations
Next Steps

INTEREST
The co-authors interest in the Graduate Assistant Formation Council stems from
their professional participation in the council as Graduate Assistant
representatives and participants.

Co-Author #1 served as the first-year representative within the council last

year, during the 2014-2015 academic year.
Co-Author #2 currently serves as one of the second-year representatives with
the council during this current academic year, 2015-2016.
Both authors took part in developing past and current professional
development workshops in their respective roles.
They also contributed to the planning, development, and implementation of
the 2015-2016 GAFC orientation, which is why they are invested in evaluating
the outcomes and impact of GAFC orientation.

Graduate Assistant Formation

Council Structure

Jane Neufeld, Vice President, Division of Student

Development
Rabia Khan Harvey, Assistant Dean and Title IX Deputy
Coordinator
GRADUATE ASSISTANT FORMATION COUNCIL:
Marci Walton, Assistant Director, Residence Life
Oliver Goodrich, Faith Formation Campus
Minister, Campus Ministry
Kevin Krauskopf, Assistant Resident Director,
GAFC 2nd Year Representative
Tina Cisarik, Campus Ministry Graduate
Assistant, GAFC 2nd Year Representative
Patrick Randolph, Assistant Resident Director,
1st Year Representative

GAFC PROGRAM CONTEXT

Background

Founded in 2009 by Dr. Robert Kelly and chaired originally by Jack McLean,
currently chaired by Rabia Khan Harvey and overseen by Jane Neufeld
Serves Graduate Assistants in the Division of Student Development and
Division of Academic Affairs
Developed to bring consistency to GA supervision and overall experience

Graduate Assistants in Program

Serve in a variety of offices and departments across campus, such as:

Residence Life, Campus Ministry, The Hub, SDMA, OSCCR, etc.
Attend 3 professional development sessions per semester in addition to
GAFC orientation
Evaluate assistantship performance via semesterly appraisals

PROFESSIONAL DEVELOPMENT GOALS

ORIENTATION, GAFC
While the Graduate Assistant Formation Council does
not have concrete learning outcomes established for
Orientation, specifically professional development, the
below goals were determined by stakeholders to
inform the evaluation plan:

Graduate Assistants will describe Ignatian values and Jesuit

philosophy and how it relates with the professional work
Graduate Assistants are asked to carry out at Loyola University.
Graduate Assistants will develop a draft of your professional
development plan for the year.
Graduate Assistants will identify how previous professional
experience will influence current professional role.
Graduate Assistants will identify and utilize Graduate Assistant
Formation Council professional development resources and
personnel.

EVALUATION JUSTIFICATION

Learning outcomes have not been established for professional development

even though professional development is one of the primary purposes of
GAFC, professional development begins with GAFC orientation
Evaluations are not in place to assess if GAs are identifying and achieving
professional development goals
Primary stakeholders requested more resources in order to sustain and build
upon the current foundation of GAFC
Provide a platform for GAs to influence their own professional development
and learning instead of being defined by the council
GAs may have other needs that are not being address by GAFC

EVALUATION QUESTION
How effective is the Graduate
Assistant Formation Councils
Orientation program in helping
Graduate Assistants identify their
short term professional
development needs and develop the
skills needed to fulfill their short
term goals?

FORMATIVE EVALUATION APPROACH

Determine whether or not orientation helps GAs identify and achieve

professional development goals
Identify areas of professional development that are overlooked by the council
Identify aspects of orientation that may need to be improved upon
Utilize GAs qualitative feedback to refine quantitative instrument, the primary
assessment tool

QUAL

quan

QUANTITATIVE APPROACH OVERVIEW

METHOD

Pre-experimental one
group pre-test post-test
design

SAMPLING &
RECRUITMENT

First-year Graduate
Assistants only

Survey is highly
encouraged to attain
close to 100%
completion but not
required

Two post-tests will be

used to evaluate
Orientation outcomes
amongst first-year
Graduate Assistants
Survey instruments in
Google Forms; sent out
in an email

Short survey to avoid

survey fatigue

Pre-test and post-test

(s) distrubuted via
email; post-test(s)
completion will be
encouraged during
Orientation and
November professional
development

INSTRUMENT

1. Pre-Test (10-12 Min. to

complete)
-

(20) 5-point Likert

Scale questions
Assess
Loyola/Jesuit
Knowledge,
Professional
development plan,
supervisor, and
Graduate Assistant
Formation Council
Knowledge,
Demographic
Information

2. Two Post-Tests (10-12

Min. to complete)
(20) 5-point Likert
Scale questions,
assessing same
topics
Tracking impact of
Orientation in firstyear

ANALYSIS

One-Way Repeated
Measures ANOVA
Help measure variances in
means across the three
tests (pre-tests; post-test
after Orientation; post-test
in November) to see what
change, if any, has
occurred among Graduate
Assistants as it relates to
Loyola/Jesuit Knowledge,
Professional development
plan, Graduate Assistant
Supervisor, and Graduate
Assistant Formation
Council knowledge.

QUALITATIVE APPROACH OVERVIEW

METHOD

Focus Groups
1. First-year Graduate
Assistants within GAFC
2. Second-year Graduate
Assistants within GAFC
Purpose
To refine the quantitative
instrument and discover
any aspects of GAFC
orientation that evaluators
are overlooking

SAMPLING &
RECRUITMENT
Sample Population:
- Current GAs in
Loyolas Division of
Student Development
and Division of
Academic Affairs
- Participants solicited
via email sent by
primary stakeholders

INSTRUMENT

CODING & ANALYSIS

- Focus group protocol

consists of 9 openended questions to
elicit responses from
GAs

- Identify micro and macro

themes
- Identify relationships
between themes
- Focus group transcripts
coded with Microsoft Word

LIMITATIONS
Quantitative
Running ANOVA for each of the 20 Likert questions will be time consuming
Potential for low response rate
Qualitative
No guarantee of participation
It may be difficult to improve survey instruments from focus group feedback
Individual feedback may be hindered due to group think
Evaluator bias and subjective analysis
Additional:
Confounding, external variables, i.e. previous work and/or professional development experiences, are
not controlled for in the quantitative instrument
Inconsistent understanding of professional development as a result of individual experiences and
missing professional development outcomes from the Graduate Assistant Formation Council
Inconsistencies among departments and graduate assistant supervisors as it relates to graduate
assistant professional development goals
Stakeholder commitment to evaluation plan and/or results may be a hinderance
Time constraints resulting from the Graduate Assistant Formation Council members volunteering their
time to distribute quantitative and qualitative portions of the evaluation plan

NEXT STEPS
The results from the evaluation will be used to:

Determine whether or not GAFC orientation helps GAs identify and achieve
professional development goals
Restructure GAFC orientation to achieve the intended goals, if orientation is
insufficiently meeting the intended goals
Continue utilizing surveys and focus groups for future years to come
Establish measurable learning outcomes for orientation and professional
development sessions
Gather feedback from GAFC alum to determine long term impact of GAFC
participation