"#$ %&&$'()*$+$,, -& (#$

./0"$,(12$34 5+'6
78(#$98()', 55 12-:289
)+ ;2)<:)+: (#$ 0'#)$*$9$+(
=83 )+ 78(#$98()', )+ (#2$$
=$-2:)8 >):# /'#--?,



LvaluaLors:

LvereLL Mcnalr
labrlan A. 8anklne
1arena 8uff

Ceorgla SouLhern unlverslLy
College of LducaLlon
l8l1 8433: rogram LvaluaLlon
lnsLrucLor: ur. 8. Carlson
lall 2012



1 | a g e

"0;@% AB CAD"%D"/

l LxecuLlve Summary age 2


ll. lnLroducLlon Lo Lhe 8eporL age 3


lll. locus of Lhe LvaluaLlon age 8


lv. LvaluaLlon lan and rocedures age 10


v. LvaluaLlon llndlngs age 10


vl. Concluslons and 8ecommendaLlons age 11


vll. 8eferences age 13


vlll. Appendlces age 14

2 | a g e

I. Executive Summary

This report presents the findings of the evaluation of USATestprep, Inc., an online
learning resource designed to improve students performance on standardized and End of Course
Tests. While USATestprep is comprised of several course modules, the Mathematics II
component of the program was the main focus. The evaluation of the USATestprep
Mathematics II module was conducted to assess the programs effectiveness in increasing test
scores in Mathematics II unit exams and overall performance mastery on Mathematics II End of
Course Tests.
The sample for this study consisted of high school students attending Berkmar High
School in Gwinnett County, Chattahoochee County High School and Tattnall County High
School. The data used in the evaluation consisted of pre- and post- test scores from the
Mathematics II Unit 3: Circles and Spheres, and Mathematics II End of Course Test results from
academic year 2010 2011 and 2011 2012, obtained from the Georgia Department of
Education website.
The aim of this evaluation was to investigate the following questions:
1. Does the USATESTPrep.com Mathematics II program help students master the
Mathematics II standards in their classes?
2. Does the USATESTPrep.com Mathematics II test preparation module help
students master the Mathematics II standards as evident by their EOCT scores?
Evidence of effectiveness in mastery of Mathematics II standards is based on at least a
10% score increase from the diagnostics assessment to the post-assessment and at least a 10%
3 | a g e

increase in the number of students at the selected schools meeting or exceeding mastery in
Mathematics II from academic year 2010 2011 and 2011 2012 on their End of Course Test.
According to the results of this evaluation, USATestprep, Inc. did help students increase
their unit test scores, but failed to provide an increase in the number of students who met or
exceeded performance mastery in Mathematics II from academic year 2010 2011 to 2011
2012 on their End of Course Tests. Based on these results, it is recommended that school
administrators and teachers reassess the value of USATestprep, Inc. and conduct an intensive
study on the variables that may have prevented students from achieving greater gains on End of
Course Tests.

II. Introduction to the Report

In his address to a Joint Session of Congress, President Barack Obama announced his
goal for the United States to raise the percentage of Americans earning college degrees by 2020.
(Nichols, 2011). The Obama Administration has implemented the Race to the Top education
reform initiative to increase the number of postsecondary graduates, and prepare the next
generation for careers in the science, technology, engineering and mathematics. As the economy
continues to evolve and become increasingly more complex, it is critical that our educational
system provides our youth with the skills, ingenuity, and critical thinking abilities that can
stimulate and maintain the economy as we advance in the 21st century (Nichols, 2011). But our
youth continue to lag behind, especially in the area of mathematics.
The College Board has conducted research to establish college readiness benchmarks on
the SAT and to provide schools, districts, and states with a view of their students college
4 | a g e

readiness. The results showed that students meeting the benchmark are more likely to enroll in
college; return for their second and third years of college; earn higher grades in both high school
and college; and are more likely to have taken a core curriculum as well as more rigorous
courses in high school than those not meeting the benchmark.

SAT Benchmarks and Attainment
Composite Critical
Reading
Mathematics Writing
Benchmark 1550 500 500 500
Number Meeting Benchmark 630,704 734,749 792,448 670,256
Percent Meeting Benchmark 43% 50% 54% 46%

Source: SAT Benchmarks: Development of a College Readiness Benchmark and its Relationship to
Secondary and Postsecondary School Performance (2011)

The SAT benchmark score for mathematics is 500. According to a report released by the
College Board, 55.92% of Georgias college-bound seniors for 2011 scored below the
benchmark in Mathematics.

Mathematics Benchmark for Georgia High School Seniors - 2011

Source: 2011 College-Bound Seniors State Profile Report Georgia, College Board
33.92
44.08
Score below 500
Score at 500 and above
Percent of Georgia High School Seniors
3 | a g e

The End-of-Course Tests (EOCT) are criterion-referenced tests administered to Georgia
public high school students. These tests are aligned with the state adopted curriculum. EOCTs
are designed to improve student achievement by assessing student performance on the standards
specific to each course tested. Student performance on each EOCT is provided to schools for
diagnostic and remedial use. EOCTs are administered three times a year in the winter, spring and
summer terms. The results of EOCTs are used to help make instruction more effective and to
ensure that all Georgia students have access to a rigorous curriculum that meets high academic
standards. These results are also used for student accountability and to gauge the quality of
education in the state. The EOCT are the final exams for each EOCT course. For students in
grade 10 or above beginning the 2011 2012 academic year the final grade in each course is
calculated by weighing the course grade 85% and the EOCT score 15%. For students in grade 9
beginning the 2011 2012 academic year and later, the final grade in each course is calculated
by weighing the course grade 80% and the EOCT score 20% (State Board Rule 160-4-2-.13).
The student must have a final grade of at least 70 to pass the course and to earn credit toward
graduation.
To provide reliable measures of student achievement and to give structure to the
assessment program, the content standards contained in the Georgia Performance Standards
(GPS) were grouped into content domains. Each domain was created by combining standards
that share similar content characteristics. Three domains were identified for Mathematics II:
Geometry/Algebra II/Statistics. (Georgia Department of Education, 2012).
The EOCT can help school administrators, teachers, parents and the community-at-large
determine if Georgia public high school students are mastering the skills needed to gain entry
6 | a g e

into postsecondary institutions and earn a college degree. Sadly, in the 2010 2011 academic
year, 43% of Georgia high schools failed their Mathematics II End-of-Course test.

2010 2011 State of Georgia End-of-Course Tests (All Schools)

Students End-of-Course Test Data for Mathematics II

2010 2011

Total number tested:
114,880

2009 2010

Total number tested:
95,979



In 2009, the Program for International Student Assessment (PISA) ranked the United
States of America 25th in the world in Mathematics (PISA, 2009). The 2009 assessment,
conducted by PISA, was administered to 5233 students randomly drawn from 165 high schools.
Fifteen-year-olds in the United States had an average score of 487 on the mathematics literacy
scale, which was lower than the Organization for Economic Co-operation and Development
(OECD) countries average score of 496. The mathematics literacy scale was established by
43 48 9
lall ass ass lus
46 43 10
lall ass ass lus
7 | a g e

PISA in 2003 to have a mean of 500 and a standard deviation of 100. Among the 65 other OECD
countries, non-OECD countries, and other education systems, 24 had higher average scores than
the United States, 29 had lower average scores, and 12 had average scores not measurably
different from the U.S. average score. Why should the United States of America continue to lag
behind other countries in mathematics and science?
Of significant note is the fact that we are living in the most technologically advanced
country, in the most technologically advanced age, with the greatest explosion of knowledge and
information this world has ever seen, through the use of computers, the World Wide Web, and
the Internet. Yet, more and more of our students are failing to graduate from high school. For
example, the United States of America has a graduation rate of about 76%
(http://www.pbs.org/newshour/rundown/2012/11/ latest- release -of-high-school-graduation-
rates-still-too-preliminary.html); while, Georgia high schools have a graduation rate of only
67%, which makes it among the lowest in the nation, as the intellectual gap continues to widen.
Other studies have shown that students in Georgia are performing equally as poor on state
standardized mathematics tests, such as Criterion-Referenced Competency Tests (CRCTs), End
of Course Tests (EOCTs), and Georgia High School Graduation Tests (GHSGTs). It is with this
concern in mind, that the evaluators decided to investigate the effectiveness of the
USATestPrep.com Mathematics II Program in three Georgia High Schools, namely, Berkmar
High School in Gwinnet County; Chattahoochee County High School; and, Tattnall County High
School, in this study.
In order for our youth to be able to compete within the ever-changing global economy, it
is imperative that we as educators assess the quality of education in Georgia public schools and
determine what measures are being used to increase students academic performance, especially
8 | a g e

in critical core subjects, such as mathematics. Some Georgia public high schools have adopted
the USATestprep, Inc. program to prepare its students for high-stakes testing and college
admission examinations. The purpose of this evaluation is to determine if USATestprep, Inc. is
effective in meeting its intended goals. The results of the evaluation will be shared with key
stakeholders within three Georgia public high schools, namely, Chattahoochee County High
School, in Cusseta, Georgia; Berkmar High School, in Gwinnett County, Georgia; and, Tattnall
County High School, in Reidsville, Georgia, where data was collected. The key stakeholders
include school principals, counselors, mathematics instructors, parents and students.

III. Focus of the Evaluation

Description: USATestprep, Inc. is an online test preparation resource designed to help
teachers and students prepare for End-of-Course assessments. USATestprep, Inc. services are
available 24 hours a day, seven days a week through the Internet. School districts must pay a
subscription fee in order for teachers and students to gain access to content.
USATestprep materials cover the fundamentals of the Georgia Performance Standards for
each subject. Some of the benefits claimed by USATestprep, Inc. are a user-friendly design, data
analysis features, sample test questions aligned to the Georgia Performance Standards (GPS),
and a format that keeps students engaged by using educational games and interactive quizzes.
Although USATestprep offers a variety of subjects, in this report, we will evaluate the GPS
Mathematics II module.
Evaluative Questions: USATestprep, Inc. claims to have 99% customer satisfaction with
their online teaching modules. The program provides practice exams, printable worksheets and
9 | a g e

interactive activities to help students perform better on state-specific End-of-Course Tests. Each
test subscription is supposedly aligned to the states performance standards, and contains
learning content that is consistent with the unit framework of each conceptual category in
Mathematics II. To determine the effectiveness of USATestprep, the evaluators were interested
in how well students performed on a particular unit in Mathematics II and how well students
were able to reach mastery in Mathematics II on the End of Course Test at the end of the 2012
Fall Term. The evaluators asked the following questions:
Q1: Does the USATESTPrep.com Mathematics II program help students master
the Mathematics II standards in their classes?
Evidence of effectiveness is based on at least a 10% score increase from the diagnostics
assessment to the post-assessment. The evaluators chose to analyze learning materials from the
USATestPrep Unit 3: Circles and Spheres module.
Q2: Does the USATESTPrep.com Mathematics II test preparation module help
students master the Mathematics II standards as evident by their EOCT scores?
Evidence of effectiveness would be assessed based on a 10% increase in the number of
students at the selected schools meeting or exceeding mastery in Mathematics II from academic
year 2010 2011 and 2011 2012. The three Georgia high schools that were evaluated for this
study have had active subscriptions to USATestPrep for at least the past two academic years.
Data Collected: For our evaluation, we collected End-of-Course Test data from academic
years 2010 2011 and 2011 2012. This information is available on the Georgia Department of
Education website. We also collected unit pre- and post- test scores of students participating in
the study.

10 | a g e

IV. Evaluation Plan and Procedures

Seventy-four students were administered a diagnostic assessment from the USATestprep
Mathematics II Unit 3: Circles and Spheres lesson in October 2012. After receiving supplemental
instruction from the USATestprep Mathematics II Unit 3 module, students were given the post
test in November 2012. Additionally, cross-comparisons were made with the results of
Mathematics II End of Course Tests from each school for academic years 2009 2010, 2010
2011 and 2011 2012.

V. Evaluation Findings

Q1: Does the USATESTPrep.com Mathematics 2 Program help students master the
Mathematics 2 Standards in their classes?
According to our results, there were significant gains in students performance on the
Mathematics II Unit 3 post test. This would indicate that the USATestPrep.com may have
benefited some students preparing for unit exams.

Test Results for Mathematics II Unit 3: Circles and Spheres

Diagnostic Test
(Prior to USATestprep)
Post Test
(After USATestprep)
Difference
Mean 36.39% 80.63% + 44.24%
STD 19.46% 19.94% + 0.48%
N 74 74 -
Min 0% 28% + 28%
11 | a g e

Max 70% 100% + 30%
Range 70% 72% + 2%

Q2: Does the USATESTPrep.com Mathematics II test preparation module help
students master the Mathematics II Standards as evident by their EOCT scores?
According to EOCT test scores from academic year 2011 2012, over half of the
students in our study who have taken the Mathematics II EOCT failed to meet satisfactory
performance standards (see charts below). These results would indicate that the USATestPrep,
Inc. is ineffective in preparing students for EOCT in Mathematics II.

EOCT Results for Mathematics II
2011 2012 Fall Semester

Performance Level
Does Not Meet Meets Exceeds
Berkmar High School 68% 32% 0%
Chattahoochee County High School 62% 37% 1.4%
Tattnall County High School 35% 45% 20%
AVERAGE
55% 38% 7.13%


VI. Conclusions and Recommendations

Based on the results of the evaluation of USATestprep.com, it appears that
USATestprep.com may help some students comprehend unit-specific content, but it fails to meet
expectations in terms of significant gains in mastery on the End of Course Tests. Further
investigation is needed to determine why there were gains in the unit assessment and not on the
12 | a g e

cumulative assessment. One explanation is that students may have difficulty retaining knowledge
of course material from the start of the semester to the end of the semester, and tend not to
perform well on cumulative assessments, such as the End of Course Test. Additionally, students
and teachers simply may not have enough time to cover the vast amount of material to
adequately address students weaknesses. While students may have proven mastery in one unit,
another unit gets little attention or ignored altogether. Although USATestprep resources are
available online, 24 hours a day, seven days a week, students may not be able to use them to the
fullest due to lack of technological capability and support.
Our recommendation is that teachers and school administrators conduct a more thorough
assessment of the USATestprep program. Perhaps a questionnaire regarding access to
technological devices and the Internet outside of class should be administered to students and
parents to analyze the practically of offering an online educational resource.
The USATestprep program does prove to be effective as a supplemental resource for
teachers and students, and can help students prepare for unit exams. However, it falls short in
assisting the vast majority of students in increasing performance mastery on End of Course
Tests. Perhaps further investigation will reveal other factors that contributed to some students
inability to meet or exceed standards in Mathematics II. With our youths future at stake, it is the
opinion of the evaluators that finding viable solutions to closing the mathematics and science
achievement gap in the United States should be top priority in the field of education.




13 | a g e

VII. References

Chen, K. (2012). Latest release of U.S. high school graduation rates still preliminary. PBS
Newshour. Retrieved December 1, 2012 from
http://www.pbs.org/newshour/rundown/2012/11/ latest- release -of-high-school-
graduation-rates-still-too-preliminary.html
Fleischman, H. L., Hopstock, P. J., Pelczar, M. P., & Shelly, B. E. (2010). Highlights from
PISA 2009: Performance of U.S. 15-year-old students in reading, mathematics, and
science literacy in an international context. NCES 2011-004 U.S. Department of
Education. Retrieved November 20, 2012 from http://nces.ed.gov/pubs2011/2011004.pdf
Georgia Department of Education (2012). End of Course Test (EOCT). Retrieved November 15,
2012 from http://www.doe.k12.ga.us/Curriculum-Instruction-and-
Assessment/Assessment/Pages/EOCT.aspx
Nichols, A.H. (2011). Developing 20/20 vision on the 2020 degree attainment goal: The threat of
income-based inequality in education. The Pell Institute. Retrieved October 9, 2012 from
http://www.pellinstitute.org/downloads/publicationsDeveloping_2020_Vision_May_201
1.pdf
Wyatt, J., Kobrin, J., Wiley, A., Camara, W.J. & Proestler, N. (2011). SAT Benchmarks:
Developing a college readiness benchmark and its relationship to secondary and
postsecondary school performance. The College Board. Retrieved October 31, 2012 from
http://research.collegeboard.org/sites/default/files/publications/2012/7/researchreport
2011-5-sat-college-readiness-benchmark-secondary-performance.pdf

14 | a g e

VIII. Appendices

Appendix A: Mathematics II EOCT Results for Berkmar High School
Berkmar High School
Gwinnett County
Grade Range: 09, 10, 11, 12
Enrollment: 3,005

School 2009 2010 2010 2011 2011 2012

N

C
o
u
n
t
s

M
e
a
n

S
c
a
l
e

S
c
o
r
e

Performance
Level
N

C
o
u
n
t
s

M
e
a
n

S
c
a
l
e

S
c
o
r
e

Performance
Level
N

C
o
u
n
t
s

M
e
a
n

S
c
a
l
e

S
c
o
r
e

Performance
Level
D
o
e
s

N
o
t

M
e
e
t

M
e
e
t
s

E
x
c
e
e
d
s

D
o
e
s

N
o
t

M
e
e
t

M
e
e
t
s

E
x
c
e
e
d
s

D
o
e
s

N
o
t

M
e
e
t

M
e
e
t
s

E
x
c
e
e
d
s

Berkmar
High
School
617 402 47% 49% 3% 645 394 61% 39% 0% 602 391 68% 32% 0%





* Percentages may not total to 100 due to rounding
* Students enrolled in Accelerated Mathematics II are administered the Mathematics II EOCT in December,
therefore this group of students results are not included in the Spring summary report.

Source: Gwinnett County Schools, Georgia Department of Education
http://archives.gadoe.org/ReportingFW.aspx?PageReq=102&SchoolId=35384&T=1&FY=2011







13 | a g e

Appendix B: Mathematics II EOCT Results for Chattahoochee County High School

Chattahoochee County High School
Chattahoochee County
Grade Range: 09, 10, 11, 12
Enrollment: 462

School 2009 2010 2010 2011 2011 2012

N

C
o
u
n
t
s

M
e
a
n

S
c
a
l
e

S
c
o
r
e

Performance
Level
N

C
o
u
n
t
s

M
e
a
n

S
c
a
l
e

S
c
o
r
e

Performance
Level
N

C
o
u
n
t
s

M
e
a
n

S
c
a
l
e

S
c
o
r
e

Performance
Level
D
o
e
s

N
o
t

M
e
e
t

M
e
e
t
s

E
x
c
e
e
d
s

D
o
e
s

N
o
t

M
e
e
t

M
e
e
t
s

E
x
c
e
e
d
s

D
o
e
s

N
o
t

M
e
e
t

M
e
e
t
s

E
x
c
e
e
d
s

Chattahoochee
County High
School
112 * 73% 25% 2% 115 * 65% 35% 0% 138 * 62% 37% 1.4%





Source: The Governors Office of Student Achievement
http://reportcard2011.gaosa.org/%28S%28x1naegxo3djtsn0ujyxf4pq4%29%29/k12/reports.aspx?TestType=QC
C&ID=626:206
















16 | a g e

Appendix C: Mathematics II EOCT Results for Tattnall County High School

Tattnall County High School
Tattnall County
Grade Range: 09, 10, 11, 12
Enrollment: 919


School 2009 2010 2010 2011 2011 2012

N

C
o
u
n
t
s

M
e
a
n

S
c
a
l
e

S
c
o
r
e

Performance
Level
N

C
o
u
n
t
s

M
e
a
n

S
c
a
l
e

S
c
o
r
e

Performance
Level
N

C
o
u
n
t
s

M
e
a
n

S
c
a
l
e

S
c
o
r
e

Performance
Level
D
o
e
s

N
o
t

M
e
e
t

M
e
e
t
s

E
x
c
e
e
d
s

D
o
e
s

N
o
t

M
e
e
t

M
e
e
t
s

E
x
c
e
e
d
s

D
o
e
s

N
o
t

M
e
e
t

M
e
e
t
s

E
x
c
e
e
d
s

Tattnall
County
High
School
204 * 50% 46% 4% 210 * 41% 50% 10% 215 * 35% 45% 20%





Source: The Governors Office of Student Achievement
http://archives.gadoe.org/ReportingFW.aspx?PageReq=102&SchoolId=36041&T=1&FY=2011

















17 | a g e

Appendix D: Item Analysis for Post Test Results Unit 3: Circles and Spheres





















18 | a g e

Appendix E: Post Test Results by Question for Mathematics 2: Unit 3: Circles and Spheres


# Question (Domain) (Element) % Correct
1) Find the area of the blue sector of the circle.
Round to the nearest hundredth.
70%
2) To the nearest tenth of a square inch, how much
surface area is added to a spherical balloon
whose radius grows from 10 to 20 inches long

52%
3) When the radius of a sphere is doubled, what is
the resulting change in volume?
56%
4) L is a point of tangency. Segments PL and PN
have a common point at P. Find the length KN.
69%
5) Determine the area of the green sector. 64%
6) Determine the measure of !DGF. 67%
7) A large spherical balloon is being deflated.
When the radius of the balloon decreases from
10 ft. to 5 ft., approximately how much volume
has it lost?
62%
8) Determine the length of arc JL. 64%
9) The diameter of the earth is approximately 7900
miles. The diameter of Jupiter is approximately
88,800 miles. Since the diameter of Jupiter is
more than 11 times that of the earth, which
comes closest to the volume of Jupiter?
56%
10) Calculate the area of a circle with a radius of 2
inches and a circle with a radius of 4 inches.
Leave your answers in terms of pi.

When the radius doubles does the area of the
circle double?
63%
19 | a g e

11) A state park is designed in a circular pattern as
shown. A park ranger drives along the circular
path from the ranger station to the nature
center and then from the nature center to the
petting zoo. How far does he drive to the nearest
tenth of a mile?
76%
12) Chords AB and CD intersect at point E, AE =
10, EB = 4, and CE = 8. Therefore, ED =
54%
13) Chords AB and CD intersect at point E.
According to the diagram, which pair of
numbers could represent lengths of segments
CE and DE?
57%
14) If you know the radius, r, of a circle, what do
you multiply r by to get the circle's
CIRCUMFERENCE?
81%
15) Leah has two congruent beach balls. The
diameter of one ball is 20 inches. Which choice
is the closest for the volume of the second ball?
68%
16) The DIAMETER of a sphere is doubled. What
is the new volume of the sphere?
56%
17) C is a point of tangency. Segments PC and PB
have a common point at P. Find the length PC.
65%
18) Determine the measure of !NOP. 54%