Running Head: IMPACT ON LEARNING THROUGH PRE AND POST ASSESSMENT

Impact on Learning Through Pre and Post Assessment

ESRM Research Methods

Emily Cook

University of Arkansas
IMPACT ON LEARNING THROUGH PRE AND POST ASSESSMENT 2

Introduction

The goal for any teacher is to provide students the best opportunity and environment for

learning. Quantitatively measuring student learning and teacher effectiveness is a process that

takes time and thought. The purpose of this study is to measure personal teaching impact on

students in an engineering and technology education course. By administering a test before and

after the lesson it can be understood what knowledge gains can be attributed to the lesson itself.

The purpose of this research is to determine how my effectiveness as a teacher (independent

variable) would affect pre and post student quiz scores (dependent variable). The hypothesis is as

students are increasingly exposed to my teaching the scores on their quiz would increase. The

null hypothesis for this study is there is no significant difference in student quiz-scores when I

am presenting lessons.

Engineering and technology education courses or as they are better known STEM

(Science, Technology, Engineering and Math) courses are continually being implemented into a

broad range of curriculum. STEM education is considered a means to help individuals develop

different strategies in order to solve interdisciplinary problems and gain skills and knowledge in

order to sustain scientific leadership and economic growth in the United States (Lacey &

Wright, 2009). STEM education covers a broad range of topics and can be covered in specific

classes dedicated to the topic as a whole or integrated into core classes such as Science, Math,

English or History. Regardless of how it is applied STEM education is an integrated approach to

teaching and learning, where discipline specific content is not divided, but addressed and treated

as one dynamic, fluid study (Brown, 2012). Other research studies suggested that solving

engineering problems in science and math integrated curriculum-improved students learning in

science and mathematics (Bottoms & Uhn, 2007; Schaefer, Sullivan, & Yowell, 2003). As the
IMPACT ON LEARNING THROUGH PRE AND POST ASSESSMENT 3

trend in recent national reports call for increased attention on developing quality teachers in

STEM fields, there is a great need to understand teachers knowledge, perceptions, and

classroom practices of implementing STEM integration (Wang, 2012). As technology education

further evolves into what is considered STEM education, continued research needs to be done on

how teachers conceptualize and implement STEM in their classrooms. (Dugger, 2011; Williams,

2011).

Method

The student sample that was generated came from a junior high (8-9) in Fayetteville,

Arkansas. This school consists of a population with 590 students currently enrolled. 373 (60.2%)

students are White, 99 (16.7%) African American, (14.9%) 88 Hispanic, (3.0%) 18 Asian and

(1.5%) 9 American Indian. The majority of student demographics that make up the technology

education classes are White, Black and Hispanic, along with other students who are in Gifted and

Talented classes, ESL and Special Education.

For this study, participants were chosen from a Technology Education class. This class

contains a total of 17 students. 14 of these students are Caucasian, 2 are Hispanic and 1 African

American. 2 students are in Gifted and Talented, 3 with behavior modifications and 2 IEPs. The

socioeconomic status of this class ranges from very high to very low. There were 6 female

students and 11 male students, the most gender diverse of all classes available.

Participants were provided a pretest based on Arkansas Engineering and Technology

Education State Frameworks, Duty C: The Science of Technology and Gateway to Technology

standards 16 BM E BM N. The pretest was designed to include questions spanning the

weeklong lesson followed by an identical posttest. Participants scores were analyzed using the

Dependent t-test method in attempts to determine if participant pretest scores (Xpre) correlate to
 IMPACT ON LEARNING THROUGH PRE AND POST ASSESSMENT 4

participant posttest scores (Xpost). External validity threats such as pretest-treatment interaction

was determined not to be a threat due to the extended six-day period of time the students

participated.

The procedure of this study was planned to cover a six-day lesson over the topic of

Energy. Students were provided a pretest that spanned the entirety of subjects to be covered over

the six-day period. Each day was planned and carried out based on guidelines from state

frameworks and course curriculum. From there a series of lectures, presentations, student made

projects and assignments were performed as the student participants were exposed to the material

and teaching style. After the six-day lesson was completed, students were asked to take a posttest

over the covered material. Student scores were then calculated and compared to the previous

attempt.

Results
Pre-Test (out of 30) Post-Test (out of 30) Difference
Student 1 22 26 4
Student 2 14 26 12
Student 3 26 30 4
Student 4 24 30 6
Student 5 24 24 0
Student 6 26 30 4
Student 7 16 28 12
Student 8 24 26 2
Student 9 12 26 14
Student 10 26 30 4
Student 11 16 26 10
Student 12 22 30 8
Student 13 26 24 -2
Student 14 14 20 6
Student 15 12 24 12
Student 16 10 16 6
Student 17 30 30 0

Figure 1.
IMPACT ON LEARNING THROUGH PRE AND POST ASSESSMENT 5

Pretest Frequency

For this study, the data collected included the students pretest score and subsequent posttest

score (see Figure1). A dependent t-test was performed on the data collected from the 17 students

who participated in the collection of data from the pre and posttests. This method was used to

determine if the null hypothesis (Ho: pre = post) would be rejected or retained in this study.

The difference between student pre and posttest mean score was D = 6 and a standard deviation

of 4.74. The t-observed is 5.22 whereas the t-critical is 2.120 and alpha level for this study was

set at =.05. The frequency charts provided below (Figure 2, Figure 3) indicate a negatively

skewed data set whereas the posttest frequency chart is more negatively defined. The Pearsons r

correlation for this set of data was rxy=.66. According to this value there is a positive

relationship between the two data sets, however the relationship is not perfect (see Figure 4).
 IMPACT ON LEARNING THROUGH PRE AND POST ASSESSMENT 6

Figure 2.

Figure 3.
IMPACT ON LEARNING THROUGH PRE AND POST ASSESSMENT 7

Conclusion

The statistical conclusion of the t-test was the rejection of the null, or there was a

significant difference in participant scores when I present students a lesson. The students

average quiz scores significantly increased after participation in instruction. t(16) = 2.120,

p<.05. I have concluded that personal teaching effectiveness in this case was beneficial for the

participants in learning the lesson (see Figure 4).

Pre-Test (out of 30)

Linear (Pre-Test (out of 30))
Post-Test (out of 30)
Linear (Post-Test (out of 30))

In the future it would be beneficial to use a probability sampling method for the selection

of participants in this study. Convenience sampling was the method used. Probability sampling
 IMPACT ON LEARNING THROUGH PRE AND POST ASSESSMENT 8

is the ideal, but in practice, convenience sampling may be all that is available to the researcher

(Ary, Jacobs, Sorensen, & Walker, 2014). In my case convenience sampling was used. Also

threats to internal validity should be considered, such as diffusion. Experimental mortality was

avoided in this study as well as selection bias. The class chosen provided an accurate cross

sectional view of gender, ability, and social economic status at the school.

Works Cited

Ary, D., Jacobs, L. C., Sorensen, C., & Walker, D. A. (2014). Introduction to Research in

Education (9 ed.). Wadsworth Cengage Learning.

Bottoms, G., & Uhn, J. (2007). Project lead the way works: A new type of career and technical

program. Southern Educational Review Board. Retrieved July 2014 from

http://www.pltw.org/pdfs/SREB_Research_Brief.pdf

Brown, J. (2012). The current status of STEM education research. Journal of STEM Education:

Figure 4. Innovations and Research, 13(5), 7-11. Retrieved from

http://0-search.proquest.com.library.uark.edu/docview/1266493670?accountid=8361

Dugger, Jr.,W.E. (2011). Evolution of STEM in the United States. Retrieved July 2014 from

http://www.iteaconnect.org/Resources/PressRoom/AustraliaPaper.pdf.

Lacey, T. A., & Wright, B. (2009). Occupational employment projections to 2018.

Monthly Labor Review, November, 82-109.

Wang, H. (2012). A new era of science education: Science teachers' perceptions and classroom

practices of science, technology, engineering, and mathematics (STEM) integration.

(Order No. 3494678, University of Minnesota). ProQuest Dissertations and Theses,, 317.
IMPACT ON LEARNING THROUGH PRE AND POST ASSESSMENT 9

Retrieved from http://0search.proquest.com.library.uark.edu/docview/922637122?

accountid=8361. (922637122).

Williams, J. (2011). STEM education: Proceed with caution. Design and Technology Education:

An International Journal, 16(1), 2635.