Running Head: REDUCING DISRUPTIVE BEHAVIOR TRAINING MODULE 1

Reducing Disruptive Behavior Training Module

Eric Zachary

California State University Monterey Bay

IST 622

Dr. Bude Su

July 24, 2018

REDUCING DISRUPTIVE BEHAVIOR TRAINING MODULE 2

Table of Contents
Introduction ..............................................................................................................................................3
Methodology .............................................................................................................................................3
Learners................................................................................................................................................................ 5
Expected Outcomes .......................................................................................................................................... 6
Tryout Process ................................................................................................................................................... 7
Survey ........................................................................................................................................................................................................ 8
Pre-test ...................................................................................................................................................................................................... 8
Training..................................................................................................................................................................................................... 9
Post-test .................................................................................................................................................................................................... 9
Questionnaire ......................................................................................................................................................................................... 9

Observation ...................................................................................................................................................... 10
Tryout Conditions .......................................................................................................................................... 10
Results ..................................................................................................................................................... 10
Entry Conditions ............................................................................................................................................. 10
Instruction ........................................................................................................................................................ 11
Summary of Data ............................................................................................................................................ 11
Outcomes........................................................................................................................................................... 20
Recommendations ......................................................................................................................................... 21
Summary................................................................................................................................................. 22
References .............................................................................................................................................. 23
Appendix A.............................................................................................................................................. 24
Appendix B.............................................................................................................................................. 29
Appendix C .............................................................................................................................................. 32
Appendix D ............................................................................................................................................. 33
Appendix E .............................................................................................................................................. 36
REDUCING DISRUPTIVE BEHAVIOR TRAINING MODULE 3

Reducing Disruptive Behavior Training Module

Introduction
The Reducing Student Disruption in the Classroom training module was created to

provide beginning teachers with an introduction to effective ways of dealing with disruptive

behavior in the classroom. Due to state mandated class size reduction, as well as, an increasing

number of retiring teachers, the Apple Valley Unified School District (AVUSD) has been hiring

between fifty and seventy-five new teachers each school year. This shortage has led AVUSD to

hire many new teachers using intern credentials which allows these teachers to work in their own

classroom while they complete their credentialing programs. Although this fills the teacher

vacancy, these teachers are often inexperienced and have spent very little, if any, time in the

classroom. Prior to administration of this training module, five participants were selected to

evaluate the training module. This paper evaluates each participant’s pre and post assessments to

determine the potential significance that the training had on the those who took it.

Methodology
Prototype
The prototype used for this evaluation is one part of a three-part training. My capstone

project is a classroom management online learning module that will be designed for use initially

at my current school site, Sitting Bull Academy, but with a larger goal of it being implemented

district wide. The training content will focus on three areas of classroom management as

discussed by Dicke, Elling, Leutner and Schmeck, (1) minimizing verbal and physical classroom

disruption, (2) developing effective rules and procedures, and (3) building positive relationships

with students, (2015, p. 4). As beginning teachers, these skills are required from the first minute

of the first day of school even if they haven’t had classroom experience. The prototype will serve
REDUCING DISRUPTIVE BEHAVIOR TRAINING MODULE 4

as an introduction to the basic skills these teachers must possess to be successful beginning on

their first day.

The prototype was designed to be delivered asynchronously through an online portal. It

can be assigned through a learning management system or via the internet. To ensure

accessibility for the greatest number of participants, the training was designed to be adaptive,

meaning that it can be accessed using a smartphone, tablet, or a computer. The prototype was

created using Adobe Captivate. The training was designed to take between ten and fifteen

minutes to complete. At the introduction of the training module, participants are introduced to

Vicky, a new teacher whose ideal classroom environment is being destroyed by a disruptive

student. Vicky quickly transitions from a state if idealistic optimism to one of stress.

Throughout the training, Isaac, a veteran teacher gives Vicky guidance and advice on

handling a variety of disruptive behaviors in the classroom. At various points during the module,

participants complete knowledge checks to assess their learning. These knowledge checks allow

for multiple attempts until the participants chooses the correct answer. Additionally, at the end of

the module, participants are presented with two scenario-based problems and five accompanying

assessment questions. The participants are given two opportunities to answer each question and

are then given a pass or fail score.

The program was designed to provide participants with a variety of disruptive behavior

scenarios and some ideas of how to appropriately intervene to minimize the impact of said

behavior. The slide backgrounds on each slide rotate between images of different classrooms to

give a more customized feel to the training rather than just a generic background. The prototype

uses audio narration, animation, and summarized on-screen text to highlight key points. Most

slides provide forward and reverse navigation which allows the user to work at their own pace
REDUCING DISRUPTIVE BEHAVIOR TRAINING MODULE 5

and return to previous slides for review. Examples of the prototype slides can be seen in tables

1.1 and 1.2.

Table 1.1

Table 1.2

Learners
The learners selected for this usability test were all experienced teachers. Sixty percent of

the learners were high school teachers, teaching grades nine through twelve. Twenty percent of

the participants teach at the middle school level, and the remaining twenty percent teach at the

elementary level. Eighty percent of the participants have greater than ten years teaching

experience and twenty percent have between six and ten years of experience. One hundred

percent of the participants indicated that they were prepared to begin teaching on the first day of
REDUCING DISRUPTIVE BEHAVIOR TRAINING MODULE 6

school with sixty percent feeling somewhat prepared and forty percent feeling totally prepared.

One hundred percent of the participants feel that poor student behavior sometimes impacts the

effectiveness of their teaching. One hundred percent of the participants feel confident in their

classroom management abilities with twenty percent feeling fairly confident and eighty percent

feeling very confident. All participants indicted levels of either fairly competent or very

competent in all questioned areas of classroom management.

The target audience for this training are beginning teachers who either possess a

bachelor's degree or teaching credential. These learners will be in their first or second year of

teaching and have received little or no previous classroom management training. Learners will

ideally be technologically savvy and have a desire to further develop their classroom

management skills.

Expected Outcomes
Upon completion of the learning module, learners should be better prepared to react to

disruptive behavior in the classroom. This reaction should include the seven steps presented in

the training module. The first step is to recognize the disruptive behavior. This portion involves

attentiveness and teacher location in the classroom. Step two is to stay calm. This portion

requires the teacher to control their reaction. Step three is to stop and assess. This portion

involves considering the classroom environment. Step four is to consider the student’s typical

behavior. Is the behavior normal for the student or is the behavior unexpected? Step five

considers the possible drivers of the behavior. Step six is to intervene with an appropriate

response. The teacher must figure out how they will handle the situation. Step seven is to

document the behavior and if necessary, notify the proper authorities. This could include parents,

administrators, counselors etc.

REDUCING DISRUPTIVE BEHAVIOR TRAINING MODULE 7

Both the cognitive, as well as, affective domains are employed in this training module.

Cognitive Domain: Throughout the training module, the learners are expected to retain the

concepts provided in the instructional material. This would include memorization of the steps, as

well as, the ability to provide a brief explanation of what each step entails. Beyond just

memorization however, the terminal goal of the training module is for the learners to be able to

take the learning and apply it to new ill-structured situations. The training provides learners with

many scenarios that range from innocent horse-play to blatant defiance and disrespect. The

learner is shown how to dissect each situation and develop an effective response.

Affective Domain: Developing the necessary skills to handle disruptive behavior is vital to

overall teacher success Brouwers and Tomic (1999) explain that poor classroom management

self-efficacy often leads to teacher burnout. As a consequence, burnout leads to nearly twenty

percent of new teachers leaving the profession within five years (U.S. Department of Education,

National Center for Educational Statistics, 2015). This training module, if proven to be effective,

will lead to greater classroom management self-efficacy for the participants, and in turn reduce

burnout, which has the potential to increase new teacher retention rates.

Tryout Process
The tryout process consisted of six parts: survey, pre-test, training, post-test,

questionnaire, and observation. The participants in the survey were spread out geographically.

As a result of this geographical disconnect, it was determined that all materials needed to be

created in a way in which they could be easily distributed via the internet. The survey, pre-test,

post-test, and questionnaire were created using Google Forms. Once the training is implemented,

the pre and post-test will be taken as part of the training module. In order to maintain anonymity

through the tryout phase however, the training was administered via the internet rather than a
REDUCING DISRUPTIVE BEHAVIOR TRAINING MODULE 8

learning management system. Although this had the benefit of anonymity, pre and post test

scores could not be gathered from within the learning module, therefore, an alternative

assessment had to be created for the tryout process. The test questions however, were identical to

those used in the module. The participants were required to complete each of the five parts.

Survey

Participants completed the survey first. The survey was broken down into five sections:

demographics, self-efficacy, classroom environment, training experience, and training

opportunities. The demographics section provided information on job assignment, experience,

and education level. The self-efficacy sections asked a variety of questions regarding the

participants level of confidence in classroom management. Classroom environment asked

questions having to do with disruption and management in the classroom. The training section

inquired about any previous classroom management training. The training opportunities section

simply asked what types of classroom management training the participant would be interested

in if it was available. See appendix A

Pre-test

The pre-test was administered in an effort to assess the participants prior knowledge in

the area of dealing with classroom disruptions. Due to the terminal goal of the training being

application of the skill not just memorization of facts, the pre-test is written in the form of real-

world scenarios. The participants will be required to analyze the scenarios and answer multiple

choice questions regarding their understanding of the them. Gathering an understanding of how

participants interpret these situations is important as they directly relate to the instruction in the

training module. See appendix B

REDUCING DISRUPTIVE BEHAVIOR TRAINING MODULE 9

Training

Participants were provided with a hyperlink to the training module through email. The

program evaluation is intended to be a seamless process, as a result, the participant completed

the training immediately following the pre-test. As was previously stated, the training was

asynchronous and web-based. The link to the training can be found in appendix C

Post-test

Upon completion of the training module, participants were immediately prompted to

complete the post-test. In an effort to ensure a comparable measure, the post-test was identical to

the pre-test. Using an identical test allowed for a direct comparison to be made. Ideally, if

effective, participants score on the post-test will exceed their corresponding scores on the pre-

test. To ensure statistical significance in the results, a single tailed paired t-test will be conducted

to compare the pre and post scores.

Questionnaire

The questionnaire is intended to be completed at the conclusion of the post-test. The

questionnaire is designed to gather generic information about participants overall impression of

the training module, as well as, to gather specific information about participant’s classroom

management self-efficacy once the training was completed. The more generic portions of the

questionnaire are divided into questions regarding usability, engagement, and content. Usability,

focuses on ease of operation. Engagement, focuses on the presentation and intrigue of the

content. Content, focuses on the alignment of the content with the intended objectives. The self-

efficacy questions were pulled from the original survey and inquired into the participant’s

confidence in transferring the learning to the classroom. See appendix D

REDUCING DISRUPTIVE BEHAVIOR TRAINING MODULE 10

Observation
In all, five participants were individually observed. As stated previously, due to the

geographical disconnect of some participants, two of the participants were observed virtually

using a Zoom video conferencing session. Each participant was contacted during the first week

of July either in person or by text to solicit participation. Each participant was sent an email

which included directions for completing the evaluation, as well as, individual hyperlinks to the

survey, pre-test, training, post-test, and questionnaire. The two participants who were virtually

observed were members of my team and had experience using Zoom. All observed participants

actions were evaluated based on a simple observation checklist. The checklist can be viewed in

appendix E

Tryout Conditions
To represent the intended learning environment for the actual training, participants were

encouraged to work in a relatively quiet place where they were able to focus their attention on

the module. Participants were encouraged to use the provided email to access each component of

the study. With the exception of one participant, each learner used their own device to complete

each portion of the module. The participants were asked to complete each step of the evaluation

sequentially in a single sitting.

Results

Entry Conditions
In comparing the intended and observed conditions, all of the participants possessed the

necessary skills and resources to complete the evaluation as intended. All participants were able

to access each component. The two participants who were virtually observed using Zoom joined

the Zoom meeting without issue. I did face a challenge with one participant. In order to have the
REDUCING DISRUPTIVE BEHAVIOR TRAINING MODULE 11

Wi-Fi necessary to complete the evaluation, I met with one participant in a public place. This did

create some distraction; however, the participant did not indicate that it was a problem. All of the

participants were gracious and supportive in their willingness to sacrifice their time.

Instruction
Overall, all participants were able to successfully complete the instruction module. The

testing process did reveal some inconsistencies however. The training did not adapt to different

devices, as well as, expected. Four participants had issue with the play bar covering some of the

navigation buttons. Additionally, some participants had issue with the pedagogical agents being

incorrectly placed on the screen. These two issues will need to be remedied. All participants

indicated that the training was informative and simple to use. One participant suggested

removing some of the navigation buttons as she felt they were unnecessary and may interrupt the

natural flow of the training.

Summary of Data
As previously mentioned, in order to maintain anonymity, as well as, eliminate

accessibility issues, the data was gathered using Google Forms. To compare the pre and post

training scores, I organized the results into a simple table. Both the pre and post-tests had seven

questions. Four participants scored higher on the post-test, and one participant answered all

questions correctly on both tests. The average score on the pre-test was 6 compared to an average

on the posttest of 6.8. Table 2.1 shows the distribution of the of the raw scores. Table 2.2 is a

graph showing a visual analysis of the pre and post-test scores.

REDUCING DISRUPTIVE BEHAVIOR TRAINING MODULE 12

Pre-test Post-test
Scores Scores

6 7

7 7

6 7

5 6

6 7
Table 2.1

Table 2.1

Although scores were relatively high on both the pre and post-tests, question three showed the

greatest amount of improvement.

Question 3: Is Treyvor’s disruptive behavior intentional or unintentional? (intentional is the

correct answer) The pre-test showed that only one out of the five participants answered correctly

as where four out of five participants answered correctly on the post-test. Table 2.3 is a graph

showing the pre and post-test comparison for question 3.

REDUCING DISRUPTIVE BEHAVIOR TRAINING MODULE 13

Table 2.3

Once completing the post-test, participants were asked to complete a post-training questionnaire

providing their perspective on the training process.

100% of the participants felt the course was extremely easy to log in to. See table 2.4

Table 2.4

The questionnaire offered mixed views as to the use of navigation within the module. Forty

percent of participants felt the controls were somewhat difficult, twenty percent felt they were

somewhat easy, and forty percent felt they were extremely easy. See table 2.5
REDUCING DISRUPTIVE BEHAVIOR TRAINING MODULE 14

Table 2.5

One hundred percent of the participants felt that the amount of time needed to complete the

module was acceptable. See table 2.6

Table 2.6

One hundred percent of the participants felt that the content was well presented. See table 2.7
REDUCING DISRUPTIVE BEHAVIOR TRAINING MODULE 15

Table 2.7

One hundred percent of the participants agreed that the learning activities reinforced the content.

Twenty percent agreed, and eighty percent strongly agreed. See table 2.8

Table 2.8

One hundred percent of the participants felt that the e-learning was engaging and interesting. See

table 2.9
REDUCING DISRUPTIVE BEHAVIOR TRAINING MODULE 16

Table 2.9

Eighty percent of participants felt the e-learning met the stated objectives and twenty percent

were neutral. See Table 2.10

Table 2.10

One hundred percent of participants agreed that the content was logically organized. Twenty

percent agreed, and eighty percent strongly agreed. See table 2.11
REDUCING DISRUPTIVE BEHAVIOR TRAINING MODULE 17

Table 2.11

One hundred percent of participants agreed that the examples helped in understanding the

content. Twenty percent agreed, and eighty percent strongly agreed. See table 2.12

Table 2.12

One hundred percent of participants felt that overall, the training was an effective learning

experience. See table 2.13

REDUCING DISRUPTIVE BEHAVIOR TRAINING MODULE 18

Table 2.13

One hundred percent of the participants were confident they had the necessary skills to stop

disruptive behavior in the classroom. Twenty percent were fairly confident, and eighty percent

were very confident. See table 2.14

Table 2.14
REDUCING DISRUPTIVE BEHAVIOR TRAINING MODULE 19

One hundred percent of participants were confident in their classroom management abilities.

Twenty percent were fairly confident, and eighty percent were very confident. See table 2.15

Table 2.15

One hundred percent of the participants felt confident in their ability to determine the possible

drivers of disruptive behavior. Twenty percent felt fairly confident and eighty percent felt very

confident. See table 2.16

Table 2.16
REDUCING DISRUPTIVE BEHAVIOR TRAINING MODULE 20

Outcomes
The research hypothesis for this training was that the training would have a significant effect on

participants post-test scores. The null hypothesis would then be that the training would not have

a significant effect on post-test scores. Being that the research hypothesis predicted growth, it

was directional and therefore called for a one-tail test. Additionally, as the samples were

dependent, I ran a paired two sample t-test. With five participants in the sample, the degrees of

freedom were 4. Table 2.17 shows the t-test results. As the results show, the absolute value of the

t stat, 4, is greater the one-tail critical value of 2.13, The null hypothesis can be rejected as the

training has been shown to have had a significant effect on the post-test scores.

Table 2.17

Although the training was shown to have a significant statistical impact, I would have preferred

that the participants better represented the target audience for this training. All five participants

were experienced teachers. Unfortunately, due to the time of year, I was not in a position to have

beginning teachers test the prototype, which I feel would provide more accurate results.

However, even though the participants had previous experience, they still showed improvement.
REDUCING DISRUPTIVE BEHAVIOR TRAINING MODULE 21

Recommendations
1) Introduction slide. Several of the participants indicated that they were unaware that there

was a table of contents that would have allowed them to easily navigate to different parts

of the module. I feel that an introduction slide that described the available tools within the

module would be beneficial.

2) Improve navigation. Although, on my screen the navigation buttons were visible, four of

the participants had difficulty finding the previous and next buttons because they were

mostly covered by the play bar. I will need to adjust the formatting of the slides to allow

the buttons to be visible above the play bar. Additionally, one of the participants felt that

there were unnecessary automatic stopping points in the module where the users were

required to click next to continue. Although at some points in the training this is

necessary, in order to improve the overall flow of the training, there are places where

these buttons should be removed.

3) Slide layout. Due to varying screen sizes, the pedagogical agents did not appear in the

precise locations they were intended to. In order to remedy this, full body agents should

be used in place of the half body agents currently in place. This should allow for the

agents stay in place and be more aesthetically pleasing.

4) Reduce ambiguity in answer choices. One participant indicated that some of the answers

were hard to determine because the answer choices were similar. As I move forward with

this project, I need to ensure that answer choices are clearly different from each other.

5) Test product with target audience. Due to the time constraints of this assignment, I was

not able to test the module with participants from the target audience. I feel the collected

data would be more meaningful if it came from beginning teachers as opposed to the
REDUCING DISRUPTIVE BEHAVIOR TRAINING MODULE 22

more experienced teachers who ended up testing the module. I feel the feedback from

beginning teachers would provide a more accurate outlook on the quality of the training.

Summary
This training module is designed to support beginning teachers as they enter the teaching

profession. The objectives and content were derived from research, as well as, personal

experience. The goal of this module is that these teachers will have a better understanding of the

complex behavioral issues they should expect to encounter in their career. Based on the results of

the paired t-test, this training has shown to have a statistically significant impact on pre and post-

test scores. Testing this module has exposed several inconsistencies that will need to be

addressed as the development process moves forward. The recommendations of the participants

will guide modifications to the current module, as well as, the other two modules that will be

built to complete my capstone project.

REDUCING DISRUPTIVE BEHAVIOR TRAINING MODULE 23

References
Brouwers, A., & Tomic D. (2000). A Longitudinal study of teacher burnout and

perceived self-efficacy in classroom management. Teaching and Teacher Education, 16,

239-253.

U.S. Department of Education, National Center for Educational Statistics, (2015). Public School

Teacher Attrition and Mobility in the First Five Years: (NCES Publication No. 2015-

337). Retrieved from https://nces.ed.gov/pubs2015/2015337.pdf

Piskurich, G. M. (2015). Rapid Instructional Design, Learning ID Fast and Right (3rded.).

Hoboken, NJ: John Wiley & Sons, Inc.

REDUCING DISRUPTIVE BEHAVIOR TRAINING MODULE 24

Appendix A
Classroom Management Survey
REDUCING DISRUPTIVE BEHAVIOR TRAINING MODULE 25
REDUCING DISRUPTIVE BEHAVIOR TRAINING MODULE 26
REDUCING DISRUPTIVE BEHAVIOR TRAINING MODULE 27
REDUCING DISRUPTIVE BEHAVIOR TRAINING MODULE 28
REDUCING DISRUPTIVE BEHAVIOR TRAINING MODULE 29

Appendix B
Pre-test
REDUCING DISRUPTIVE BEHAVIOR TRAINING MODULE 30
REDUCING DISRUPTIVE BEHAVIOR TRAINING MODULE 31
REDUCING DISRUPTIVE BEHAVIOR TRAINING MODULE 32

Appendix C
Hyperlink to training module

http://itcdland.csumb.edu/~ezachary/mist/ist526/labs/Eric_Zachary_IST_522_Prototype/IST622

_Prototype.html
REDUCING DISRUPTIVE BEHAVIOR TRAINING MODULE 33

Appendix D
Post Training Questionnaire
REDUCING DISRUPTIVE BEHAVIOR TRAINING MODULE 34
REDUCING DISRUPTIVE BEHAVIOR TRAINING MODULE 35
REDUCING DISRUPTIVE BEHAVIOR TRAINING MODULE 36

Appendix E
Observation checklist participants 1-5

Checklist Yes No Comments

Learner was able to complete the X Open up scores on post test

entire learning module and answer
all questions.

Learner used navigation buttons X The navigation buttons were hidden

appropriately without issue behind the play bar, need to be moved

Did the learner encounter any X

usability issue, but was able to solve
the problem on their own without
assistance?

Did the learner encounter any X

usability issue that required
assistance before continuing with the
training module?

Training module adapted X Avatars didn’t adjust correctly use

appropriately to fit the screen size on full body images.
learner’s device.

If needed, learner used table of X Instruction on using table of contents

contents to return to previous slides remove option for scenarios
for review.
REDUCING DISRUPTIVE BEHAVIOR TRAINING MODULE 37

Checklist Yes No Comments

Learner was able to complete the X

entire learning module and answer
all questions.

Learner used navigation buttons X The navigation buttons were hidden

appropriately without issue behind the play bar, need to be moved
(send to front)

Automatically move to next slide

Did the learner encounter any X

usability issue, but was able to solve
the problem on their own without
assistance?

Did the learner encounter any X

usability issue that required
assistance before continuing with the
training module?

Training module adapted X Some elements did not adjust

appropriately to fit the screen size on appropriately to the screen.
learner’s device.

If needed, learner used table of X Instruction on using table of contents

contents to return to previous slides remove option for scenarios
for review.
REDUCING DISRUPTIVE BEHAVIOR TRAINING MODULE 38

Checklist Yes No Comments

Learner was able to complete the X

entire learning module and answer
all questions.

Learner used navigation buttons X Navigation buttons were hidden

appropriately without issue behind the play bar

Did the learner encounter any X

usability issue, but was able to solve
the problem on their own without
assistance?

Did the learner encounter any X Needed guidance to find navigation

usability issue that required buttons
assistance before continuing with the
training module?

Training module adapted X

appropriately to fit the screen size on
learner’s device.

If needed, learner used table of X Learner was unaware that a table of

contents to return to previous slides contents existed.
for review.
Learner should be provided with
more direction
REDUCING DISRUPTIVE BEHAVIOR TRAINING MODULE 39

Checklist Yes No Comments

Learner was able to complete the X

entire learning module and answer
all questions.

Learner used navigation buttons X Once learner the navigation buttons

appropriately without issue were discovered, the learner was able
to successfully navigate the training.

Did the learner encounter any X Learner commented that she would
usability issue, but was able to solve have liked to have seen her test
the problem on their own without results.
assistance?

Did the learner encounter any X

usability issue that required
assistance before continuing with the
training module?

Training module adapted Learner noticed some text size

appropriately to fit the screen size on inconsistencies on some slides
learner’s device.

If needed, learner used table of Learner was not aware of the table of
contents to return to previous slides contents. Indicated instructions may
for review. be beneficial.
REDUCING DISRUPTIVE BEHAVIOR TRAINING MODULE 40

Checklist Yes No Comments

Learner was able to complete the x

entire learning module and answer
all questions.

Learner used navigation buttons X

appropriately without issue

Did the learner encounter any X

usability issue, but was able to solve
the problem on their own without
assistance?

Did the learner encounter any X

usability issue that required
assistance before continuing with the
training module?

Training module adapted X

appropriately to fit the screen size on
learner’s device.

If needed, learner used table of X

contents to return to previous slides
for review.