Fall 2015 Ca DZan Docent Training

Evaluation Report

Laura Steefel-Moore & Victoria Eudy

Abstract
Each fall, the Ringling trains a
corps of docents to give tours
at the Ca DZan, the historic
home located on the museums
grounds. In the fall of 2015 this
corps consisted of 34 individuals
led in their training by the new
Museum Education & Training
Specialist over the course of
11 weeks. Over these weeks,
the evaluator worked alongside
the Museum Education and
Training Specialist to conduct
a formative, mixed-methods

evaluation aimed to emphasize

visitor-focused outcomes in
order to make the house more
accessible to the high volume
of visitors it receives each year.
What follows is an executive
summary of that training
guided by the initial goals
developed and administered by
the museums Education and
Training specialist.

Docent Training: A Quick Look

The Ringling hosts the
second largest docent corps
the country with over 110
volunteers providing tours of the
art museum, circus museums,
grounds, and its historic house,
the CaDZan (Annual Report,
2014). This evaluation focuses
specifically on the subset of
docents and their 4-month
training semester designed to
prepare these individuals for
docent responsibilities at the
Ringlings Ca DZan.
Before training begins, the
docents must pass a battery
of interviews over the six
months prior to the beginning
of the semester. Once he or
she successfully passes these
interviews, the future docent
then engages in a 10-week
course designed to provide
them with the tools to facilitate
40-minute tours designed to
bring the house to life for the
high volume of visitors it sees
each year.
The docents met regularly
on Fridays for half or full day
courses designed to develop
proficiency in not only the
history of the Ca DZan and
the Ringling legacy, but the art
of engaging a diverse and rich
array of visitors as docents.
The training sessions
took place each Friday from
early September through the
beginning of December. The
training specialist provided a
syllabus containing a list of
supplementary reading material
for each week, along with an
itinerary for the weeks session.
Classes consisted of a
variety of activities including but

not limited to various docent-led

projects, guest lectures, group
discussions, guided tours,
conversations with veteran
(mentor) docents as well
workshops on comedy improv
and voice projection. Docents
were also assigned mentors,
or veteran docents assigned to
assist and provide feedback to
the new class. These mentors
participated with the new
docent corps in-training by
attending lunches, shadowing
tours, providing feedback, and
answering questions on the
docent experience.
The course also involved an
online component through the
Blackboard platform. Through
Blackboard, docents were able
to access the weekly readings,
engage in online discussions
through the courses blog,
as well as post questions to
the instructor. Lectures were
also recorded and posted
to Blackboard through the
software Tegrity to allow for
absent docents to access these
lectures, and to differentiate
instruction as to allow docents
who wish to revisit the subject
audibly to do so.

Did the docent candidate interview

process establish accurate
expectations for what docent training
itself would be like?

I feel so proud to be part of an institution

where exacting standards are practiced
consistently and throughout the organization.
Ive definitely caught the Ca and Ringling
story/ legacy bug...

Evaluation
Goals:
Changes & Additions
to Training
As a condition of the Training Specialists position, efforts were
made to encourage more engaging, visitor-centered delivery of the
Ca DZan tours through addendums and alterations of the training
existing protocol including the following:

Allotting more time for docents in the Ca DZan

Increased structuring of mentor program (e.g. mentor
training session)
Allowing docents to give tours as docent candidates
(shared tour with mentor)
Extended training hours and number of sessions
Class designed for multiple types of learnersopportunities for leadership; optional sessions,
optional readings
Built in time for discussion and reflection of reading
Less initial focus on administration/logistics,
more on content
Including new guest lecturers and workshops on
engagement and storytelling
Heightened emphasis on museum education best
practices and dealing with diverse audiences

Due to the high number

of visitors to the house,
the Training Specialist
and evaluator explored
the challenge of providing
visitors a more engaging and
accessible experience without
compromising the integrity of
information through a mixed
methods formative evaluation
of the training semester. In
order to vet the aforementioned
augmentations to the standard
docent training, the Education
and Training Specialist outlined
the following initial goals
for the training semester
designed to assess the
augmentations to the existing
training protocol to improve:

Accuracy of docent
provided information
Docents ability to
engage with visitors
Level of comfort with Ca
DZan content and
ability to give tours
Confidence in tour giving
skills and knowledge
base
Docent utility of
Blackboard technology

Methods
Prior to the beginning of the
semester, the training specialist
and evaluator delineated the
forms of data collection through
the initiation of an evaluative
logic model, a working
internal mission to guide the
proceedings of the following
training sessions, as well as
updated assessment tools
geared to reflect a more visitorcentered philosophy (Appendix
C).
Data collection for the
evaluation consisted of the
following: weekly staff and
docent-reported evaluations for
7 of the 11 weeks of training, an
anonymous exit survey, and a
collection of summative docent
tour assessments. These data
sources were primarily viewed
through qualitative open-coded
measures, with the exception
being a portion of qualitative exit
survey results.
The training specialist
distributed the anonymous, 21
question exit survey (Appendix
B) immediately following the
final course of the semester.
The survey consisted of
a mix of open ended and
multiple choice questions
centering on perceived

effectiveness of training,
programming, preparation, as
well as questions specifically
targeting the initial goals of the
evaluation. The survey received
an 87% response rate, with
100% completion.
Modifications to the final
written assessment and tour
observation assessments were
made with respect to reflecting
an increased emphasis on
the promotion of audience
engagement and management.

Working Internal
Mission Statement

Docents are:
Stewards of the Ca DZan
Customer Service Associates
Facilitators of formative educational experiences

Tour of the Ringling librarys special

collections

Results

Observations
Alterations to the docent program occurred most
frequently through formal and informal discussion
and documentation of course observations.
Cultural competency, content assessments, and
differentiating instruction to meet docents specific
needs constituted the majority of discussion.
Docent-Reported Evaluations

Fig 1. Today I am Feeling...

Though the number of docents who chose
to participate in the self-reported evaluations
decreased over the weeks, these reports
reflected an increase in self-efficacy and
perceived knowledge base. Though docents
reported feelings negative and mixed emotions
(i.e. overwhelmed but excited) closer to the
midterms, these feelings appear to resolve
following the midterm assessment. Reported
feelings of excitement dominated the positive
responses for across the weeks. The primary
source of negative commentary centered
around feelings of fatigue and concerns over the
relevance of information provided.

Fig 2. I Want to Know...

Most notably, the responses to the open-ended
I want to know portion of the docent-reported
evaluation showed a positive trend towards
questions regarding tour technique and how
to best provide visitor engagement as the
semester progressed. Conversely, questions
regarding specific access and answers to
content related questions trended downward.
Only one response regarded how to use the
blackboard technology within the
first week of evaluations.

Fig 3. What I Need From You Is...

The question what I need from you [the
training specialist] received the least amount of
responses from the participating docents, and
contained an array of responses. Some saw
this as an opportunity to ask content, or course
related questions, while others saw it as room to
provide positive feedback such as keep up the
good work! The most frequent specific response
regarded incorporating a slower pace to the
presentation of information, seen within the first
and third evaluation distribution.

Exit Survey
Following the final exam, docents were sent an email asking to
participate in an anonymous survey containing multiple choice and
open-ended questions regarding details of their experience over
the course of the semester.
Preparedness
Of the survey participants, 77% felt that the docent interview
process established accurate expectations for docent training,
while the remaining 23% felt they were somewhat prepared
following the interview process. However, when asked What
part of being a docent do you feel most prepared for? docents
provided responses such as all of it and I am 100% prepared.
Others answered with specific details, stating faith in their
understanding of content and greeting guests constituted their
perceived strength. Crowd control and time management in giving
tours were often cited sources of needed personal improvement.
[I feel] Comfortable with the fact that I have many things to offer
in each room. This will allow me to choose discussion points that
seem relevant to each group and keep my presentation fresh.

Did the docent candidate interview

process establish accurate
expectations for what docent training
itself would be like?

[I feel] Very comfortable with the tours content as well as

navigating the space. I do feel like an expert on Ca[DZan],
the Ringlings and the 20s. This was borne out when I had to wait
before going upstairs on my shared tour so I asked my group for
questions. There were many, they ran the gamut and I
was able to answer.

Workshops, Projects, and Guest Speakers

Docents provided mixed reviews of the improv workshop,
designed to assist docents in expecting the unexpected as visitor
engagement often entails impromptu speaking. The majority of
written responses to whether or not the docent found the improve
training helpful contained remarks that the training helped them
to be spontaneous, work the crowd, relax, as well as positive
comments on the sessions team building components. However,
others failed to make the connection between the workshop and
the responsibilities as a docent. A less controversial workshop
was a session on voice projection technique, of which 81% of
docents found very helpful, with the remaining 19% considering it
helpful. Docents remarked on the individualized feedback of the
instructor, as well as the high level of relevance of the workshop to
their practice as docents.

Exit Survey [cont.]

Mentor program and veteran tours
The majority of docents (74%) found their mentors to be very
helpful with the remaining 26% commenting they found their
mentors somewhat helpful. Frequent descriptions of their
interactions with the mentors included references to helpfulness
of feedback, remarking at the generosity and abundance of time
allotted to answering questions, as well as the important role the
mentors played in providing support and encouragement to the
docents in training.
[my mentor] gave lots of time and suggestions.
She was always available via email. She specialized in
encouragement. She helped put some facts or rumors in context.
He was supportive and knowledgeable. I also gained a lot of
insight about docent life and history.
How helpful was your mentor?

Having someone who has been there is a confidence builder.

Likewise, 92% of the docents found the Ca DZan veteran tours to

be very helpful, reporting that participating on these tours inspired
and helped model forms of engagement, through seeing different
styles and techniques.
It is endlessly interesting to hear the different styles of
presentation.
It was good to see how each individual made the tour their own.
Listening to how other docents gave a tour reaffirmed that there
was no perfect way of giving the tour, that it would vary a little - or
a lot - and still be great.
[mentor program] weeded out who [I] wanted to
be as a docent.

Exit Survey [cont.]

Technology
The overwhelming majority of docents (87%) reported that
their ability to use blackboard improved over the course of the
semester. The most popular of the features used was the blog with
63% reporting using this feature the most. Docents used the blog
to pose questions, provide feedback, as well as engage in other
discussion and commentary. Aside from the blog, 13% report using
blackboard primarily for messaging other docents, while 25% used
the platform to access pre-recorded lectures. Participants were
split as to the frequency of their use of pre-recorded lectures, with
33% answering they accessed these recording very frequently,
another 33% reporting rarely, and still another 33% reporting they
never accessed the pre-recorded lectures.
Final Assessments
At the current time, a third of the docent training class completed
their summative evaluations as docents in the Ca DZan. These
qualitative assessments were overseen by both the Education
Training Specialist and a veteran docent. Of those assessments
received, both the Training Specialist and docent produced similar
evaluations of a single docents performance. Of those who
completed their tours, only one docent was required to retake the
exam. The majority of the assessments provided documented
excellent or good proficiencies in communication and quality of
information given.

Improv workshop

Over the course of your training,

do you feel as if your ability to use
Blackboard improved?

Exit Survey [cont.]

Additional Comments
The survey concluded with an open-ended question designed to
allow docents to leave any commentary that may not have been
addressed within the previous questions. Docents again suggested
the ability to engage with more tour experience prior to the final
tour assessment (some specified giving tours with their mentors),
adjusting class time to provide a more even schedule, as well
as providing details to enhance the Ca DZan experience for the
visitor such as a recording of the Aeolian organ in the courtyard.
However, the majority of reports contained praise for the quality of
instruction or citations of confidence and preparedness.
The training was impressive. I feel so proud to be part of an
institution where exacting standards are practiced consistently
and throughout the organization. Ive definitely caught the
Ca[DZan] and Ringling story/ legacy bug-- it was highly
contagious! As well, Laura did an outstanding job. She managed
content, timing and training with our class of high achievers (and
some very high maintenance!) with unflagging professionalism,
enthusiasm and grace. My only suggestion is to have a hand out
with class names, bios and contact info at the start of training so
we could get to know one another more easily/better. Also, I loved
my mentor when I saw her but she had a commitment at the Mote
every Friday.
I appreciated the thoroughness and thoughtfulness with which
our curriculum was fashioned. It is clear Laura and her team put a
great deal of effort into this docent training.
I think Laura did an excellent job of preparing us to be good
docents. If we did the readings and paid attention in class, we
should all be well prepared to lead interesting and informative
tours. For myself, I think only practice can increase my level
of confidence about dealing with unforeseen situations, etc.,
as I mentioned above. Kudos to Laura for doing a great job in
preparing us!

Discussion
The Training Specialists
emphasis on promoting
engaging, visitor-centered
tours as a core element of the
docent curriculum is apparent
throughout the evaluation.
Telling of this push is the
docent-reported evaluations,
in which the steady decline
throughout the semester of
concerns of content coincides
with an increased emphasis
on how to better perfect the
art of visitor engagement and
tour provision. However, this
is not at the expense of the
confidence in these docents to
provide accurate and in-depth
information about the Ca DZan.
Rather, docents used the exit
survey as well as the selfreports to often express their
increased grasp of Ca DZan
related content.
Likewise, their reported
increased confidence in using
Blackboard (89%) and Tegrity
may owe to the Training
Specialists strong emphasis
on electronic distribution of
information. The course relied
heavily upon the weekly use
of the Blackboard platform in
order for students to access
reading material, the syllabus,
ask/answer questions, and
were recurrently encouraged
to use the site as a means of
engaging with their peers. The
Training Specialists prompt
and successful response to
the needs of the docent corps

is perhaps most telling in the

docent-reported responses,
where docents noted the
Specialists prompt feedback
(via email) as very helpful.
Throughout the course,
efforts were consistently taken
to differentiate instruction to
meet the needs of the individual
docents, and these did not
go unnoticed by the docents
themselves. For example, the
Training Specialist responded
to feedback requesting a
slower pacing of material, and
the docents used the selfreports to appreciate those
efforts, Likewise, the docents
gave praise to the Specialists
use of blackboard to answer
content-related questions each
week, signaling that docents
found the evaluations as a
means to successfully voice
concerns. Docent mentors
also successfully provided
individualized attention to
the docents-in-training, with
the majority of exit survey
participants acknowledging
the helpfulness of the mentor
program.

Class session