Extendedlearningtimeevaluationreportfinal 1

Running head: Reading Extended Learning Time Evaluation Report
Evaluation of Reading Extended Learning Time at Morgan County Middle School
Program Evaluation Report
University of West Georgia
Melissa Brown, Jason Garofalo, Thomas Kratowicz, Frederick Wright

Extended Learning Time Evaluation Plan
Background of the Program and Evaluation
Instructional technologies are continuously changing. From an iPad to a Chromebook,
the possibilities are endless as to how a teacher can use that piece of technology to differentiate,
remediate, hook, extend, or teach content standards to students. Nestled in the foothills of
Georgia, about 70 miles from the city of Atlanta, sits Morgan County Middle School. This
school serves a population of approximately 720 students comprised of 65% White, 25% African
American, 6% Hispanic, and 4% are mixed race. 47% of the entire student population receives
free lunch or discounts. Previous school data indicated that 21% of students were reading below
grade level. Also, special needs students and African American males are target groups in the
school and are identified as groups that have significant gaps in reading. According to Figure 1,
one of our intermediate term outcomes for the program would be to close the gap within their
instructional level and comprehension of vocabulary. Four years ago, Morgan County Middle
School decided to modify their academic schedule in an attempt to build in an extra 45 minutes a
day for remediation for reading instruction that would focus on the at risk students. At risk
students were identified as those students who fell below the 25th percentile on the Measures of
Annual Progress (MAP) test which is a nationally normed test. Additionally, once the Georgia
Milestones test was administered, any student who scored below proficient on the state test was
also labeled at risk.
This extra 45 minute class was called Extended Learning Time or ELT. During class
time, at risk students who were placed in the class would be working primarily on a reading
intervention program called Reading Plus, which would be monitored by one of the academic
reading teachers who (according to Figure 1) would be there to set up differentiated lessons
within the Reading Plus program. As shown in Figure 1, students would take a placement test
within this computer program which would then design a program specific for their learning
needs based on their comprehension level and their vocabulary level. Reading Intervention
Programs are a great resource to use to assist struggling readers. In a study conducted by
Cheung and Slavin (2013) technology led reading applications which specifically focused on
phonemic awareness, phonetic decoding, and accuracy of reading text in a small group setting
made a small impact on reading achievement of struggling readers. Researchers in this study
identified 20 high quality reading intervention programs which were used intensively with
struggling readers over a twelve week period. The study also found that in secondary schools,
the reading programs did not make an impact on reading outcomes for struggling readers. The
difference between the programs used at the two levels was that secondary levels used teacher
directed and whole group instruction. The programs that made the most difference were the ones
that were high intensity programs which included combined technology and non-technology
components in their reading interventions. The programs that were most successful were used as
daily activities and part of core curriculum, not just supplemental differentiation in activities. As
can be seen in Figure 1, these range from teacher differentiated lessons to reading assessments
every 9 weeks.
Another study conducted by Rasinski, Samuels, Herbert, Petscher, and Feller (2011)
focused on the Reading Plus Program. This study focused on the impact of the Reading Plus
Program and the achievement of students in various grades fourth through eighth in an urban
school. The study found that students who used the program made more gains on standardized
tests. Students who used the program for more than twenty hours of instruction made the largest
gains. The study also reported that students being served in special education and with learning
disabilities had gains using the program, as well.
The Reading ELT class, in addition to the time on the reading intervention program, also
provided opportunities for reading teachers to take that dedicated time working one-on-one,
(Figure 1), with specific students who were struggling with a particular reading skill or time to
re-emphasize standards that were being taught in the regular reading class. According to Figure
1, students will be working with technology and the reading instructor on various assignments to
not only improve their reading comprehension and ability but to also gain a sense of confidence
when reading. This will be reinforced through rewards that are set up to reward and encourage
the students who are meeting the challenges within the reading program on a weekly basis.
Figure 1:
Evaluation Purpose
The purpose of this program evaluation on the Reading Extended Learning Time is for
planning purposes for the 2017-2018 school year. This is the fourth year that this program has
been in place and like we stated before, the entire school academic schedule was moved around
and altered to accommodate this program. When planning for the new school year, our client,
Dr. Darrell Stephens, principal at Morgan County Middle school, needs to know if changing the
academic schedule and running this program has made a difference in the reading scores of the
students. Is the program and all of its components made up of extra one-on-one time,
remediation, and the reading program making a difference? Are the reading gaps of the students
enrolled in this program closing? Is the computer program, Reading Plus, which is used to attack
weak reading skills making a difference with the students in the class? Are reading levels
improving towards grade level? If the evaluation proves that the program is making a difference,
then Dr. Stephens and his leadership team can continue to justify the accommodations in the
academic schedule and plans for the new school year will include the program and teachers
identified to teach it. In order for a student to be placed in the program, they must score below
the 25th percentile on the MAP assessment (Measure of Academic Process) which is a nationally
normed test. Culturally, these students are in the low socioeconomic status and are some of the
lowest readers in the school. Putting them all together on one classroom has been questioned. Is
this the best way to reach these students for remediation?
The three evaluation questions that will be answered through this process are the following:
1. How effective is the Reading Extended Learning Time Program at helping students
increase their overall reading scores on standardized tests?
2. Is the Reading Extended Learning Time Program helping the students close the gap with
their reading levels towards grade level?
3. Is the Reading Extended Learning Time Program increasing students overall attitude and
self-confidence towards reading?
Data
The type of the research conducted for this study was a mixed method approach because
the research team used quantitative and qualitative data. We used quantitative research by
analyzing current 6th, 7th, and 8th grade Reading ELT students' MAP (Measure of Academic
Progress) scores and other reading data gathered this year. The MAP and Reading Plus test was
assessed three times this school year. Qualitative data was also gathered from surveys from the
students on their use and feelings of using technology to enhance reading instruction and how it
is affecting their overall feeling about reading. This survey was sent via Google Forms and was
collected anonymously. We used a correlational research design to exam the relationship
between technology and the reading progress for students in the 6th, 7th, and 8th grade Reading
ELT classes. We chose to use correlational research because we collected data on existing
variables and examined relationships between those variables (Anderman, 2009).
Sampling
The study took place in three 6th grade classrooms, two 7th grade classrooms, and two
8th grade classrooms that use Chromebooks for technology purposes. At the present time, the 6th
grade class had over a hundred more students enrolled in it which justifies the extra class.
Students in this study had various reading abilities and were between the ages of eleven and
thirteen years old. These classrooms were designed to utilize the Reading Plus Program along
with one-on-one instruction to focus on the skills that students were struggling with in reading.
All instruction from the teacher, work from students, and assessments were all completed via
technology. Students within the study had various technology skills, as well.
The evaluation team was comprised of four educators who had over 20 years experience
in education and expertise in instructional technologies. Melissa Brown, Jason Garofalo, Thomas
Kratowicz, and Frederick Wright were educators with strengths comprised of various teaching
responsibilities at all grade levels including one member who works as an adjunct professor at
the University of West Georgia. As a team, they had strong communication skills which aided in
the collecting and analyzing of data, as well as presenting their findings to the stakeholders. As
educators, they are experienced in data collection and analyzation which again benefited this
evaluation.
Analysis
The teachers who teach the ELT classes ran the reports the evaluation team needed in
order to have the data analyzed. The teachers all had access to the data so no further permissions
needed to be acquired. We used the cluster sampling strategy to collect data from all seven of the
Reading ELT classes in this study. Cluster sampling occurs when the sampling unit is not an
individual group that occurs naturally in the population such as a classroom (Teddlie and Yu,
2007). Our cluster has a total of 88 students which are representative of 40 sixth grade students,
28 seventh graders, and 20 eighth grade students. All of these students currently scored below
the 25th percentile on the nationally normed MAP assessment. None of these students currently
have an Individualized Education Plan.
Instrumentation
We collected our quantitative data via NWEA MAP reports and Reading Plus Reports.
NWEA MAP stands for Northwest Evaluation Association Measures of Academic Progress.
According to the Northwest Evaluation Association (n.d.), this assessment is a computer
adaptive interim assessment which is used to measure student progress and growth for
individuals in reading. According to the Institute of Education Sciences, Reading Plus is a
research based silent reading intervention that helps students gain proficiency by improving
student silent reading, fluency, comprehension, and vocabulary (2010). This program
differentiates for each child regardless of special needs, socioeconomic status, or gender. After a
benchmark is given, an individualized program is developed specifically for each child based on
their comprehension and vocabulary needs. We ran weekly reports from the Reading Plus
Program which identified growth in comprehension, reading fluency, and vocabulary levels of all
students and whether or not they have maintained or increased their levels. We also collected
data from various reading inventories used with students within the ELT classes. The SRI
reading inventory was given every three months and the CRI inventory was given every nine
weeks. The SRI stands for Scholastic Reading Inventory and is a K12 adaptive assessment
that measures both foundational reading fluency and reading comprehension. The CRI stands for
Comprehensive Reading Inventory which assess phonemic awareness, phonics, vocabulary, and
comprehension. It comes with nonfiction and fiction assessments. We assess on both of them.
We also gave a Cloze Reading Inventory to help identify vocabulary levels for the students. For
the qualitative data, a Google Form was created and emailed to the students. Students were
asked to answer several reflective questions regarding reading instruction and technology.
1. Discuss the use of technology in ELT class to teach, assess, and learn. Did you like the way
we used technology? Explain how it helped or hindered your improvement in reading.
2. Discuss at length your view on the Reading Extended Learning Program. How has being
enrolled in this program helped you improve your reading?
3. Discuss how the Reading Plus program helped you improve your reading. Even if you didn't
prefer it, how did it help you?
The evaluation team comprised of four educators will then analyze the data. After
analyzing all the data from the MAP scores, Reading Plus scores, and various reading
inventories, our hypothesis is that all of the reading scores will show significant growth. The data
that we retrieve from the MAP assessments will be measuring RIT Growth. According to the
Northwest Evaluation Association, RIT score identifies the level at which the student was
answering assessment questions correctly fifty percent of the time (2010). The ideal RIT growth
in reading for sixth and seventh grade students per year is 4 RIT. The ideal RIT growth in
reading for an eighth grade student per year is 3 RIT. The data we collected from Reading Plus
will be measuring growth in the comprehension and vocabulary in units of months and years.
The ideal growth in reading for sixth, seventh, and eighth grade students is one years growth.
The ideal growth in reading in regards to the various reading inventories for all students is also
one years growth. Our hypothesis was that the use of technology with the Reading Plus
Program will increase the reading progress of the students in our study. The dependent variable
is the reading progress and the independent variable is technology. These variables are
quantitative variables because the are measured on a numeric scale. They are also a discrete
variable because they all have a finite end. Reading Plus ends at grade level thirteen, the reading
inventories top out at level 12, and the MAP Lexile Levels top out at 2000.
For the qualitative study, we sent out the three questions in a Google Form and collected
data in Google Sheets. Student responses were open ended and anonymous. Students at Morgan
County Middle School are proficient in responding to open response surveys as the do this each
nine weeks in their academic classes. We, as a team, felt confident that students would give us
quality feedback, especially since they will be having a voice in this class for the next school
year. Morgan County Middle School is a Google school so students are very familiar with using
Google Forms. All students were emailed the link to complete the Google form. We analyzed
common themes from the Reading Plus Program, the ELT class, and the pros and cons of using
technology in the classroom according to the students who spent a year in the ELT class. As we
read the student responses, we used a color coding systems to help identify common themes
among the answers to the student responses. According to Belmont University, coding data is
where you read through open ended responses and organize them into three to seven major
themes (2016). Once major themes were identified, sub themes were utilized and summaries
were written.
Timeline For Data Collection
Cloze Assessment August 2016 and May 2017
CRI Assessment December 2016 and May 2017
SRI Assessment August 2016, October 2016, December 2016,

March 2017, May 2017
MAP Assessment August 2016, December 2016, May 2017
Student Surveys May 2017
Standards
Once all reports are finalized, the evaluation team will sit down with the stakeholders
comprised of the leadership team at Morgan County Middle School, the ELT teachers, as well as
the students and parents of students enrolled in the ELT classes and share the results of the
evaluation. The sources of the standards which will be used to draw conclusions originate from
Morgan County Middle Schools School Improvement Plan. The initial question of this
evaluation asks: How effective is the Reading Extended Learning Time Program at helping
students increase their overall reading scores on standardized tests? The source of the standard is
that 60% of participating students will meet growth projection on MAP testing. As described on
the plans Logic Model, (Figure 1), one of the long-term outcomes is that scores will increase on
standardized tests. Standardized tests, such as the MAP, provide students scoring at or below the
25th percentile, typically remain at this level without personalized intervention, such as the
Reading Plus Program. Thus, if reliable standardized MAP scores improve, the Reading Plus
Program is warranted; a causal relationship can be inferred by skills increases (growth measured)
in vocabulary, comprehension, and Lexile score improvements (Fournier, 1995).
The second question of the evaluation asks: Is the Reading Extended Learning Time
Program helping the students close the gap with their reading levels towards grade level? The
source of the standard is that 60% of participating students will close the gap between their
reading level and grade level benchmarks. As described on the plans Logic Model, (Figure 1),
this goal is an intermediate outcome. The data collected from the Reading Plus programs
embedded adaptive assessments, Scholastic Reading Inventory (SRI), CLOZE Deletion Test, and
Comprehensive Reading Inventory(CRI), provide reliable normed evidence to track growth,
fluency, vocabulary, and comprehension gains. The combined intersection of these instruments
warrants its use for accurate assessment of growth in relation to the Reading Plus intervention
(Fournier, 1995).
The third question of the evaluation asks: Is the Reading Extended Learning Time
Program increasing students overall attitude and self-confidence towards reading? The source of
the standard is that 80% of participating students indicated that their attitude towards reading has
increased as a result of the program. As described on the plans Logic Model (Figure 1), this
goal is a short-term outcome of the program. An intermediate outcome is indicated on the Logic
Model, (Figure 1), as well, that students are reading more for pleasure during class time than
they were previously. If the program is helping to improve skills related to reading, increasing
their ability to understand what they have read and improving test scores, an observed
improvement in overall attitude toward reading is likely to be observed. Thus, utilizing the
Reading Plus Program, which targets the deficits related to their ability, warrants observation of
positive changes in attitude towards reading (Fournier, 1995).

The budget for this study was minimal. The leadership team was able to run the reports
and send us the data that we as an evaluation team needed for analyzing. The only incurred cost
was a fee to pay each of the participating evaluators which was $50 per person for a grand total
of $200.
Findings
Table 1
Table 2
Table 3
Table 4
Table 5
One great thing about tracking student growth in reading is that there is a variety of different
reading assessments that can be given to assess reading growth. We collected data from a
variety of assessments to determine the validity of the Reading Plus ELT classes. According
Table 1, the Reading Plus data shows that 73% of the sixth graders in ELT made more than a
years growth, 81% of the seventh graders in ELT made more than a years growth, and 74% of
the eighth graders in ELT made a years growth while working on the program. According to
Table 2, of students enrolled in the ELT Classes 96% of the sixth graders made progress on the
CLOZE assessment, 90% of the 7th graders made progress on the CLOZE assessment, and 83%
of the 8th graders made progress on the CLOZE assessment. According to Table 3 over 78% of
each ELT grade levels showed progress on both their fiction and nonfiction CRI assessments.
According to Table 4, on the Georgia Milestone Reading Assessment, all grade levels of ELT
classes were 86% or above on grade level. According to Table 5, over 87% of students in each
grade level made progress on the SRI as well. The purpose of this analysis was to get an
overview of progress made in reading ELT this year. Significant progress has been made to grow
students in their reading abilities.
For the qualitative study, we sent out three questions in a Google Form and collected data in
Google Sheets. Student responses were open ended and anonymous. All students were emailed
the link to complete the google form. We as a team analyzed common themes from the Reading
Plus Program and the pros and cons of using technology in the ELT classroom according to the
students who spent a year in the class. As we read the student responses, we used a color coding
systems to help identify common themes among the answers to the student responses.
After analyzing the data from the student responses, some of the common themes that were
identified were that students like the fact that technology was used during the ELT classes, but
did become frustrated with the WiFi because it was not working properly or when the internet
made things too slow. Even though the majority of the students openly did not like the Reading
Plus Program that was used, they did admit that it helped them to become faster readers and help
them improve their comprehension levels. Students said that they enjoyed having the ELT time
to work on various computer programs so that they did not have to do them at home, and they
enjoyed playing the review games in class that helped them prepare for tests.
The first evaluation question that we attempted to answer through this evaluation was how
effective was the Reading Extended Learning Time Program at helping students increase their
overall reading scores on standardized tests? Through the data we collected, we can confirm that
the majority of the students made progress. The second question we attempted to answer was is
the Reading Extended Learning Time Program helping the students close the gap with their
reading levels towards grade level? Most of the students enrolled in these classes made at least
one years growth on all of their assessments. These students are significantly behind in reading,
but any improvement with gains is definitely worth noticing. The final question we attempted to
answer was is the Reading Extended Learning Time Program increasing students overall
attitude and self-confidence towards reading? Through the survey questions, students
consistently made reference that even though they didnt like to have to work on the Reading
Plus Program, they did realize that it helped them improve their reading. Several students even
mentioned that they liked having the smaller class sizes to ask the teacher for help and time to
finish tests or start tests early.
Scholarly Limitations
The impact of Reading Extended Learning Time on struggling readers depends on which
classroom you walk into and how that teacher has utilized the extra class time to work one-on-
one with students, utilize the reading intervention computer program provided for him or her to
guide instruction, and how they utilized their reading data to pinpoint critical reading skills areas
of improvement in their students. The Impact of technology within the Reading ELT class also
depends on the teachers enthusiasm and knowledge for using technology and the intervention
program to guide and assess instruction. Teachers who have confidence in technology are more
likely to utilize it to deliver instruction, prepare for class, and utilize it for student projects
(ODwyer, Russell, & Bebell, 2005). Through this evaluation, the evaluation team does see the
importance of the Reading Extended Learning Time Class which utilizes technology, reading
intervention programs, small group lessons, and individual practice to raise the reading levels of
at risk reading students.
The specific limitations of this study would be to further investigate to find the correlation
between amount of time spent on Reading Plus program and progress made with each student.
Did the amount of times spent on the Reading Plus Program within the Reading ELT class have
any influence on the amount of reading gains? This study also only used data collected for the
first year of the Reading Extended Learning Time Program. Perhaps the real impact of this class
could be tracked with the current Sixth Graders enrolled in the program as they continue through
their middle school years.
Conclusions
The evaluation team suggests that Dr. Darrell Stephens and his leadership team continue a
minimum of two more years with the Reading Extended Learning Time Program in order to
track the progress of these at risk reading students within their ELT classes. After a year in the
program, significant improvement in reading scores on nationally normed tests and individual
reading inventories has been noted as well as students have seen a notable improvement in their
reading abilities. We would suggest looking at the correlation between time spent on the reading
intervention program and progress made on tests to determine how many minutes a week the
ELT class should focus specifically on working on the reading intervention program. The
quantitative data collected all support growth and improvements in reading. The qualitative data
is not as confirming of the question, as the students attitudes did not really change toward
reading, though we have a better sense of what they like and don't like. As a team we suggest
more 'qualitative assessments' including teacher and student perspectives, so that in the next year
or more of the program, one could fine-tune aspects of self-confidence that are most important to
develop in the classroom. Also, some interview data might reveal key themes to aid in the overall
success of the program and potential to improve both growth and self-confidence in the students.
References
Anderman, E. (2009, December 23). Research methods: An overview. Retrieved April 7, 2017,
from http://www.education.com/reference/article/research-methods-an-overview/
Cheung, A. A., & Slavin, R. R. (2013). Effects of educational technology applications on reading
outcomes for struggling readers: A best-evidence synthesis. Reading Research Quarterly,
48(3), 277-299.
Fournier, D. M. (1995). Establishing evaluative conclusions: A distinction between general and
working logic. New Directions for Evaluation, 1995(68), 15-32.
Northwest Evaluation Association (n.d.). NWEA: Measures of Academic Progress (MAP).
Retrieved from https://www.nwea.org/
O'Dwyer, L. M., Russell, M., & Bebell, D. (2005). Identifying teacher, school, and district
characteristics associated with middle and high school teachers use of technology: A
multi level perspective.Journal of Educational Computing Research, 33(4), 369-393.
Qualitative Data: Coding for Organization | Belmont University | Nashville, TN. (n.d.). Retrieved
April 6, 2017, from http://www.belmont.edu/oair/assessment-support/pages/qualitative-
data-coding.html
Rasinski, T., Samuels, S. J., Hiebert, E., Petscher, Y., & Feller, K. (2011). The relationship
between a silent reading fluency instructional protocol on students reading
comprehension and achievement in an urban school setting. Reading Psychology, 32(1),
75-97. doi:10.1080/02702710903346873
Reading Plus. (2010, September). Retrieved April 6, 2017, from
http://ies.ed.gov/ncee/wwc/interventionreport.aspx?sid=419
Reading Plus - Online Silent Reading Intervention Program. (n.d.). Retrieved from
https://www.readingplus.com/
Teddlie, C., & Yu, F. (2007). Mixed methods sampling: A typology with examples. Journal of
Mixed Methods Research, 1(1), 77-100. doi:10.1177/2345678906292430

Appendix A
Survey Questions for Students

Appendix B
Instruments used in collecting Data
Name of Instrument How and Who Gives? Data Retrieved
CRI Reading ELT reading teacher gives assessment during Comprehension on fiction and
Assessment August, December, and May nonfiction levels
SRI Reading ELT reading teacher gives assessment August, Lexile score
Assessment October, December, March, May
Milestone State Test End of School Year 2017 Lexile score
CLOZE Assessment ELT reading teacher gives assessment during Vocabulary grade Level
August, December, and May
Reading Plus ELT reading teacher gives assessment during Vocabulary and
Benchmark Test August, December, and May comprehension grade level
Reading Plus ELT reading teacher runs report for August, Vocabulary and
Instructional Reports October, December, March, May comprehension grade level
MAP Normed Test ELT reading teacher gives assessment during Lexile
August, December, and May
Appendix C
EVALUATION CONTRACT
This is an agreement between Melissa Brown, Thomas Kratowicz, Jason Garofalo, and
Frederick Wright (hereinafter referred to as the Evaluator) and Dr. Darrell Stephens, Principal of
Morgan County Middle School (hereinafter referred to as the Evaluation Client).
GENERAL INFORMATION
Title of Project: Evaluation of Reading Extended Learning Time at Morgan County Middle
School
Scope of Work: The Evaluator will collect data (or use previously collected data) and
analyze this data to answer three evaluation questions regarding implementation or the
effectiveness of the Evaluation of Reading Extended Learning Time at Morgan County Middle
School as it relates to the mutually agreed upon components of the programs operation or
intended outcomes in a formative capacity.
WORK STEPS
Work steps include the following: a) develop a program logic model for the evaluation;
b) literature review as it relates to the function of the program and/or importance of the
program in the local setting; c) develop evaluation questions that are mutually agreeable
between the Evaluator and the Evaluation Client that will drive data collection and
analysis; d) identify and document data collection methods; e) define data collection
resources; f) define the sample of participants for which data collection and analysis will
be applicable; g) have evaluation plan peer- and self-reviewed; h) collect or gather
relevant data; i) analyze the data using valid and reliable analysis techniques (e.g.,
statistics) and tools; j) review initial findings with Evaluation Client and incorporate input
as necessary; k) prepare draft evaluation report and incorporate feedback from the
Evaluation Client as necessary, and l) submit a final evaluation report.
FIELD VISITS
All on-site fieldwork, data collection, interviews, and/or observations shall be

coordinated with the Evaluation Client. Off-site fieldwork is not expected, but if required
will be coordinated with the Evaluation Client.
PROJECT MANAGEMENT
Background: The purpose of this evaluation is to formatively investigate the

effectiveness of the Evaluation of Reading Extended Learning Time at Morgan County Middle
School program. Evaluation findings will be used to improve the program.
Performance Period: Evaluation planning will take place during January-April 2016, and data
collection and analysis will occur between August 2016 and May 2017.
Type of Contract: Time and materials. Any costs of the evaluation will be covered by the
Evaluator.
CONTRACT AWARD MEETING

The Evaluator shall not begin work on the evaluation until the Evaluator and the
Evaluation Client have met and approved the evaluation plan that is outlined within this
Evaluation Contract/Statement of Work.
GENERAL REQUIREMENTS
1. If desired, the Evaluation Client may designate another organizational representative
to serve as the Evaluation Client to act on his or her behalf. A signature is needed by
both the Evaluation Client (who represents organizational/building level authority)
and the designated Evaluation Client.
2. The Evaluation Client will approve and provide access to Milestones test scores, CRI
test scores, SRI test scores, Reading Plus Benchmark Scores, Reading Plus
Instructional Reports, CLOZE assessments
3. The Evaluation Client will authorize surveys, interviews, or observations of students
within the reading ELT classes.
4. Requests for additional data or research beyond the items listed above will require
written approval by the Evaluation Client and will be attached to the Evaluation
Contract as an Addendum.
5. All written deliverables shall be phrased in acceptable terminology of the field; words
shall be defined in layperson language. If necessary, statistical and other technical
terms shall be defined in a glossary of terms and referenced for validity and
usefulness.
6. Electronic copies of the draft deliverables will be submitted to the Evaluation Client
via email for review and feedback. When the Evaluator and Evaluation Client meet
to review a deliverable, a hard copy will also be provided by the Evaluator. If there is
not a response from the Evaluation Client within three business days from the data the
item was delivered, it shall be deemed approved. The Evaluator will have three
business days to deliver the final deliverables from the data of receipt of the
Evaluation Clients comments. All deliverables shall be delivered in software used
by the Evaluation Client.
7. The Evaluation Report will be written to adhere to the APA Style (6th Edition).
8. A formal presentation of the evaluation findings will be conducted with the
Evaluation Client.
SPECIFIC MANDATORY TASKS AND ASSOCIATED DELIVERABLES
Description of Tasks and Associated Deliverables:
The Evaluator will provide the specific deliverables described below.
Deliverable 1: An Evaluation Plan which describes a background of the Evaluation of

Reading Extended Learning Time at Morgan County Middle School program, a program
logic model, the agreed upon evaluation questions, a description of the evaluator, a
detailed description of the intended evaluation activities including the sample, data
collection methods, data analyses, relevant instruments including program
documentation, contract, and formative/summative metaevaluation forms.
Task 1: The Evaluator will develop the program logic model to be used to guide the
program evaluation. The program logic model will be shared with the Evaluation Client
for feedback.
Task 2: The Evaluator will use the program logic model to identify evaluation questions.
The questions will be mutually agreed upon by the Evaluation and Evaluation Client.
Task 3: A literature review will be conducted to defend the purpose of the program, its
function, and the need for evaluation work, useful instruments, as well as to identify
relevant standards for drawing an evaluative conclusion.
Task 4: The Evaluator will develop or find instruments for collecting data, or identify
relevant existing datasets.
Task 5: The Evaluator will write a data collection and analysis plan.
Task 6: The Evaluator will have a draft of the evaluation plan formatively metaevaluated
by peers in the MEDT 8480 course.
Task 7: The evaluator will conduct a summative metaevaluation of the evaluation plan.
Deliverable 2: A data collection and analysis report to inform the Evaluation Client on
the Evaluators ability to successfully collect data and complete the relevant data
analyses.
Task 1: The Evaluator will collect new or gather existing data.

Task 2: The Evaluator will use the methodology described in the evaluation plan to
analyze the data.
Task 3: The Evaluator will share the results of the data analysis with the Evaluation
Client for input, and update the Evaluation Client on emerging analyses that may be
relevant or if certain planned analyses are not feasible.
Deliverable 3: A draft evaluation report which contains all relevant components of the
evaluation report will be provided to the Evaluation Client for input and reaction.
Task 1: The Evaluator will write an evaluation report starting with the text of the
evaluation plan. First the evaluator will update this text to the past tense to reflect that
the data collection and analyses have been conducted. The specific methodologies, if
different than what was reported in the plan should be updated as well to reflect the
realities of the evaluation effort. The evaluation report will include a write up of the
evaluation findings in response to the evaluation questions, and include a description of
any limitations associated with the effort, and recommendations for improving the
program as well as next steps for further evaluation work.
Deliverable 4: A final evaluation report in APA format.
Task 1: The Evaluator should decide which feedback from the Evaluation Client to
include. In some cases, the Evaluator will choose not to incorporate specific feedback,
and will instead justify the decision not to change the findings.
Task 2: A formal presentation with the Evaluation Client and any parties designated by
the Evaluation Client will be conducted to discuss findings and important
recommendations.
SCHEDULE FOR DELIVERABLES
Deliverable Due Date

Evaluation Plan, which includes: November 13, 2016
Background Information on Program to be Evaluated
Program Logic Model
Evaluation Questions (previously agreed upon)
Sampling Plan (if necessary)
Data Collection and Analysis Plan
Instruments
Evaluation Contract
External Metaevaluation (peer review) forms
Internal Metaevaluation (self-reflection) form
Data Collection and Analysis Report May 1, 2017
Draft Report for Evaluation Client to Review May 19, 2017
Evaluation Report May 23, 2017
Presentation to Evaluation Client and Organization May 31, 2017
The Evaluator shall provide all deliverables to the Evaluation Client as agreed upon in the
schedule established at the initial meeting, and outlined in the table above. Unless
otherwise specified, the number of draft copies and the number of final copies shall be
the same. If for any reason any deliverable cannot be delivered within the scheduled time
frame, or the contents of the deliverable changes, the Evaluator is required to explain why
in writing to the Evaluation Client, including a firm commitment of when the work shall
be completed. This notice to the Evaluation Client shall cite the reasons for the delay and
the impact on the overall project. The Evaluation Client shall then review the facts and
issue a response in accordance within three business days.
CHANGES TO STATEMENT OF WORK

Any changes to this statement of work shall be agreed upon and approved by both the
Evaluator and the Evaluation Client. A copy of each change will be documented and
kept in a project folder along with all other products of the program evaluation project.
Any costs associated with the changes will be at the Evaluators expense.
REPORTING REQUIREMENTS
The Evaluator will provide the Evaluation Client with weekly progress reports via email.
The progress reports shall cover all work completed during the preceding week and shall
present the work to be accomplished during the subsequent week. This report shall also
identify any problems that arose with a statement explaining how the problem was
resolved. This report shall also identify any problems that have arisen but have not been
completely resolved with an explanation.
TRAVEL AND SITE VISITS

No travel is required or expected. Any unexpected travel must be pre-approved by the
Evaluation Client.
SCHOOL RESPONSIBILITIES
The school shall provide access to technical and procedural information regarding the
Evaluation of Reading Extended Learning Time at Morgan County Middle School
program. The schools shall provide a copy of a confidentiality statement (if required)
upon request by the Evaluator. If required, the Evaluation Client agrees to work with the
Evaluator to adhere to any district-level data access or data use agreements.
CONTRACTOR EXPERIENCE REQUIREMENTS
The Evaluator will perform this evaluation as an authentic learning experience to fulfill
requirements in the Ed.S. program at the University of West Georgia, College of
Education, Department of Instructional Technology. The professor for this course is Dr.
Carl Westine (Email: cwestine@westga.edu, Office Telephone: 678-839-6095).
CONFIDENTIALITY AND NONDISCLOSURE
This evaluation effort is considered to be an internal evaluation for the purposes of

program improvement, the results of which will not be released or disseminated beyond
Morgan County Board of Education and the school leadership team at Morgan County
Middle School. Only staff members as approved by the Evaluation Client will be able to
view the findings of the program evaluation. All data will be kept confidential, and no
student names or identifying characteristics/information will be used in the evaluation
report.
EVALUATION PLAN AND CONTRACT APPROVAL
The evaluation plan including the evaluation contract was reviewed and accepted by:
Evaluation Client: Evaluation Client Representative (if different):
Dr. Darrell Stephens

Print Name Print Name
Morgan County Middle School

Name of Organization Name of Organization
Darrell.Stephens @morgan.k12.ga.us
Email Address Email Address
<NOT NEEDED>
Signature Date Signature Date
Evaluator:
Melissa Brown
Print Name
Signature Date
Appendix D
Qualitative Metaevaluation Form Using The Program Evaluation Standards, 3rd Edition
MEDT 8480
Evaluator: Melissa Brown, Jason
Garofalo, Thomas
Kratowicz,
Frederick
Wright, Group 14
Metaevaluator: Leeann Denham, Sarita

Griggs, Michelle April 9,
Patrick, Jan Wilson, 2017
Group 12
NAME DATE
Overall Comments:
Group 14, your evaluation plan does a good job explaining the purpose and background of the program you are
evaluating as well as the purpose of the evaluation itself. Below are specific comments regarding each standard that
aim to strengthen your evaluation plan.
Utility Standards:
Since an evaluation is useless if stakeholders cannot find value in the process and products, ensuring utility standards
are solidly met is important to an evaluation. Developing credibility for yourselves as competent evaluators and
involving stakeholders, paying particular attention to any cultural sensitivities, in all steps of the evaluation are two
important tasks that will increase the value, and therefore, utility of the evaluation. Including information about
yourselves is the first required step toward developing credibility; however, developing relationships with
stakeholders and acting on the changing needs of the stakeholders throughout the process will further ensure a
successful evaluation. Adding statements to your plan that address these needs will better ensure that you intend to
meet the utility standards.
Feasibility Standards:
In order to meet the standards of feasibility, some important issues must be addressed. Providing clear roles for
those who are conducting testing, collecting data, and analyzing data is important for the evaluation process.
Defining such issues as providing data reports at the conclusions of each testing session so that the data can be used to
be responsive to each students needs is an issue that you need to address since the goal of your evaluation is to
determine the effectiveness of the program. This needs to be an ongoing process. Another ongoing process needs to
be communication with stakeholders. There is currently no plan as to who sees what report and what is being done
with each report. By doing this, your stakeholders will be more participative in the evaluation and may lead to more
success in relaying your final outcomes of the evaluation.
Propriety Standards:
Propriety standards were well defined in determining understandable and clear objectives with the stakeholders
along with evaluation agreements, expenditures, transparency and disclosures. Communication among stakeholders
and their considerations require a more thorough description to meet the needs of basic human and legal rights of
both participants and stakeholders. Further, definitions of what reports and data sets needs to be included as well as
how the evaluation report will be accessed by stakeholders and how their concerns will be addressed as they are
highly relevant to the evaluation.
Accuracy Standards:
As written, your conclusions about ELT may be justified. The information and data should serve its ultimate
purpose: to increase the reading scores of students. We believe parts of your data collection will yield reliable
information even though the majority of data is standardized testing. However, the qualitative data may present a
hazard to your evaluation. The student responses to open-ended questions may not provide high quality data. Relying
solely on student responses may not provide a clear indicator about their attitude and work ethic. It is unclear as to
why the correlation between technology and reading programs is being examined in the evaluation. The
communication and reporting standard is difficult to assess since it is not included in the evaluation draft, but will be
added to the final plan and report.
Evaluation Accountability Standards:

Your evaluation plan documents its purpose and provides a logic model for procedures and outcomes. An internal
metaevaluation of your plan will be beneficial in order to examine and clarify the standards that we documented as
clearly or highly relevant, yet not met in your plan. These standards hold programs accountable and should lead to
decisions about the program which should be included in your final report.
Additional Feedback:
Format:
headings for the various sections would make the draft easier to read
logic model is difficult to read
Consider putting references on new page
Four references are listed as being retrieved last summer (July 2016) - probably want to change this to
reflect 2017 date
NWest Evaluation Assoc and Institute of Ed Science are not listed as references although have
parenthetical citations
Consider bolding headings
Other:
In several places, your draft includes a hypothesis and mentions examining the relationship between
technology and reading programs, yet this isnt one of your evaluation questions.
Consider changing the wording to be more reflective of a program evaluation plan. Currently, it reads
more like a research project which could set a different tone for stakeholders than an evaluation.
Carefully read for grammatical accuracy. There are several small typo-type errors.
STANDARDS STANDARD RELEVANCE TO THE EVALUATION (CHECK QUALITATIVE

STATEMENTS ONE) FEEDBACK/COMMENTS
HARDL FAIRL CLEARL HIGHL

Y Y Y Y
UTILITY
U1 Evaluator Evaluations should This standard is not met.

Credibility be conducted by Evaluation team is not introduced
qualified people who or even mentioned in the draft
establish and plan. It is important to clearly
maintain credibility show the qualifications of the
in the evaluation evaluation team as a step in
context. building credibility with
stakeholders.
U2 Attention to Evaluations should This standard is not met. School

Stakeholders devote attention to administration, the teacher
the full range of leading each class, and the
individuals and students enrolled in the program
groups invested in are all stakeholders. In a situation
the program and where this evaluation became a
affected by its reality, parents of enrolled
evaluation. students may also be a legitimate
stakeholder group. It isnt clear
from the Draft Program
Evaluation Plan that involving
stakeholders in the evaluation
process is planned. Stakeholders
should be at least informed of, if
not involved in, all steps of the
evaluation, including following up
after the results of the evaluation
are complete. Questions for
students, a stakeholder group, are
included; however, it is concerning
that students who have been
identified as challenged readers
are expected to respond in writing
to open-ended questions. Consider
using a Likert scale questionnaire.
U3 Negotiated Evaluation purposes This standard is partially met.

Purposes should be identified While the purpose of the
and continually evaluation is stated and the logic
negotiated based on model showing needs and wants is
the needs of included, it is not apparent that
stakeholders. the evaluators are prepared to
make adjustments to the
evaluation based on the changing
needs of stakeholders. The
evaluation will be strengthened
with this need openly
communicated in the evaluation
plan.
U4 Explicit Evaluations should This standard is partially met.

Values clarify and specify Since the purpose of this
the individual and evaluation is to determine its
cultural values continuation, there are
underpinning stakeholders that will be affected
purposes, processes, by the conclusion. However, the
and judgments. evaluation plan does not
acknowledge any individual or
cultural values that may be
important in drawing conclusion.
The fact that students are given
the opportunity to have a voice
through the open-ended survey is
a good sentiment, however, as
mentioned in U2, its effectiveness
is questionable. Consider using a
Likert scale questionnaire. In
addition, there does not appear to
be any basis to the benchmarks
chosen for measuring responses to
the evaluation questions. Adding
justification for chosen
benchmarks will strengthen the
evaluation.
U5 Relevant Evaluation This standard is partially met. The

Information information should absence of justified benchmarks
serve the identified and the open-ended written
and emergent needs responses from challenged readers
of stakeholders. prevents this standard from being
met. Instruments chosen for
quantitative data collection seem
appropriate but would be moreso
if stakeholders were clearly
involved in their selection.
U6 Meaningful Evaluations should This standard is partially met.

Processes and construct activities, Using existing valid and reliable
Products descriptions, and data increases the accuracy and
judgments in ways usefulness of the evaluation
that encourage results. Therefore, stakeholders
participants to who understand the evaluation
rediscover, procedures will be likely to
reinterpret, or revise rediscover, reinterpret, or revise
their understandings their understandings and
and behaviors. behaviors based on the results.
Clearly involving stakeholders
would result in meeting this
standard.
U7 Timely and Evaluations should This standard is not met. There is

Appropriate attend to the no evidence that the surveys
Communicatin continuing presented to students will be
g and information needs written in a style that is
Reporting of their multiple appropriate to their specific
audiences. cultural or language needs or
reading levels. In addition, the
evaluation plan does not address
how or when stakeholders will
receive feedback and and in what
format that feedback will be (e.g.
full report, summaries, etc.).
Addressing communication and
reporting plans will strengthen the
evaluation plan.
U8 Concern for Evaluations should This standard is not met. There is

Consequences promote responsible no mention of stakeholder
and Influence and adaptive use involvement, reporting of results,
while guarding or any communication with
against unintended stakeholders (other than the
negative survey for students). Because
consequences and there could be individuals involved
misuse. who are simply against the 45-
minute extension of the school day,
it will be important to engage
school administration, the
teacher(s) overseeing the program,
and students in the evaluation
process to avoid possible
sabotage of this evaluation.


Y Y Y Y
FEASIBILITY
F1 Project Evaluations should The standard is partially met.

Management use effective project Roles need to be more clearly
management defined among those who are
strategies. collecting and analyzing data
within the ELT program.
Activity lists and timelines for

testing would be useful.
F2 Practical Evaluation The standard is met. You may

Procedures procedures should be want to consider an explanation of
practical and the data that you receive in
responsive to the way frequency periodically throughout
the program the year to be responsive to student
operates. needs.
F3 Contextual Evaluations should The standard is partially met. A

Viability recognize, monitor, more defined demographic
and balance the breakdown of your selected testing
cultural and political group may be needed to ensure
interests and needs of that a proportionate sample is
individuals and being tested. You seem to be heavy
groups. on 6th grade samples compared to
7th and 8th as it reads.
F4 Resource Use Evaluations should The standard is met. I would

use resources suggest clarifying your testing
effectively and acronyms and making sure that an
efficiently. explanation is included for the use
of each type of test and how its data
is meeting your intended goals.


Y Y Y Y
PROPRIETY
P1 Responsive Evaluations should The standard is not met. Define

and Inclusive be responsive to what reports will be generated with
Orientation stakeholders and sets of data. Explain how
their communities. stakeholders will have access to the
evaluation results and what their
role is once this evaluation is
complete. Define what questions or
concerns that stakeholders may
have with the evaluation results.
P2 Formal Evaluation This standard met. This

Agreements agreements should evaluations pertains to a specific
be negotiated to school within the district so as long
make obligations as communication is clear between
explicit and take into all members, then the standard is
account the needs, met.
expectations, and
cultural contexts of
clients and other
stakeholders.
P3 Human Evaluations should The standard is partially met.

Rights and be designed and Explain how stakeholders are a
Respect conducted to protect part of the communication
human and legal community and how their concerns
rights and maintain are being assessed and addressed.
the dignity of
participants and
other stakeholders.
P4 Clarity and Evaluations should This standard is met. Evaluation is

Fairness be understandable clear and fair in stating
and fair in stakeholder needs. At risk students
addressing identified as needing additional
stakeholder needs time reading as per Milestone test
and purposes. scores. Planning committee
addressed through needing
program data for future course
planning.
P5 Transparency Evaluations should This standard is met. Evaluation

and provide complete report states stakeholders involved,
Disclosure descriptions of previous data collected, and plans
findings, limitations, for future data collection. Logic
and conclusions to model is also clear in stating inputs
all stakeholders, and activities. New and existing
unless doing so data sources and collection
would violate legal methods listed.
and propriety
obligations.
P6 Conflicts of Evaluations should This standard is not met. No

Interest openly and honestly conflict of interest(s) stated.
identify and address Apparently, conflict of interest is
real or perceived not significant to this evaluation.
conflicts of interest
that may
compromise the
evaluation.
P7 Fiscal Evaluations should This standard is met. No expenses

Responsibility account for all listed. This evaluation may not
expended resources incur any expenses as it is a local
and comply with school however listing how your
sound fiscal program and evaluation will be
procedures and financed/not financed would be
processes. beneficial to the final evaluation
report.
STANDARD RELEVANCE TO THE EVALUATION (CHECK QUALITATIVE FEEDBACK/COMMENTS

STATEMENT ONE)
STANDARDS S
Y Y Y Y
ACCURACY
A1 Justified Evaluation This standard is met. Decision making is

Conclusions conclusions based on performance of at risk students in
and and decisions the reading program. Increased student
Decisions should be performance will result in program
explicitly continuation. Logic model states
justified in the progression stakeholders will take towards
cultures and long term outcomes. Hypothesis and
contexts where standards match those of the logic model.
they have Accuracy defined in data collection methods
consequences. of the three evaluation questions.
Collections methods appear to remain the
consistent throughout the study.
A2 Valid Evaluation This standard is met. Participants clearly

Information information stated. Literature validates evaluation
should serve questions. Multiple sources are also used for
the intended data collection including both new and
purposes and existing as well as mixed methods. Potential
support valid problematic areas are identified with
interpretations. supporting literature,and evaluation clearly
states benchmarks.
A3 Reliable Evaluation This standard is partially met. Outcome

Information procedures measures correlate with evaluation
should yield questions. Sufficient literature review, and
sufficiently data analysis to be completed. Evaluation
dependable appears to be replicable. There is room for
and consistent improvement in how consistent the program
information for is at serving subgroups- such as students
the intended with IEPs and ESOL students, currently no
uses. subgroup of the above nature is served. If
possible, consider adding a factor of
differentiation to your plan for students who
are at risk but also apply to another
subgroup it if becomes an option.
A4 Explicit Evaluations This standard is partially met. You state that

Program and should the evaluation process may affect the
Context document program and its continuance at school. You
Descriptions programs and described the program with sufficient scope
their contexts and provided appropriate context for
with evaluating the ELT program; however,
appropriate when describing the program and its
detail and subcomponents, a list of the acronyms,
scope for the descriptions of the tests and assessments in
evaluation an appendix would help clarify. In several
purposes. places, your draft includes a hypothesis and
mentions examining the relationship
between technology and reading programs,
yet this isnt one of your evaluation questions
so doesnt fit the scope of the evaluation.
Are you evaluating the extended learning
time or the technology?
A5 Information Evaluations This standard is met. Your group is using

Management should employ existing data and collecting new data from
systematic students. Your draft considers many sources
information of information and balances the use of old
collection, information (test scores) with new
review, information (up-to-date test scores each 9
verification, weeks). A Google Form and using Google
and storage Drive is a reliable storage method since it
methods. can be accessed in any location with Internet
access. You have many methods of
organizing the data within Google. The
single potential hazard is assuming that
students will provide high quality responses
to the open-ended questions. Consider using
a Likert scale for the questionnaire.
A6 Sound Evaluations This standard is partially met. Your

Designs and should employ evaluation uses mixed methods; and
Analyses technically provides qualitative and quantitative data;
adequate however, the methods are not mixed within
designs and the evaluation questions. It would be helpful
analyses that to have an analysis section that details the
are appropriate techniques and analysis you will use for
for the each evaluation question. Another suggestion
evaluation in regards to question 3, you will provide a
purposes. questionnaire to students, group their
responses into themes and then document in
a chart. Have you considered including the
teachers perspective on the student attitudes
and confidence with reading? You dont
mention an observation instrument, yet the
last paragraph of your draft discusses
observation of students reading more for
pleasure. This would provide additional data
for your analysis.
A7 Explicit Evaluation This standard is met. The reasoning for the

Evaluation reasoning evaluation is clear. Your reasoning that the
Reasoning leading from ELT class combined with Reading Plus and
information a reading teacher should result in an
and analyses to increase of student scores on standardized
findings, tests and close the gap are sound conclusions
interpretations, and judgements. A hazard for this standard
conclusions is evident for evaluation question 3 since you
and judgments are depending only on student surveys to
should be draw a conclusion.
clearly and
completely
documented.
A8 Communicati Evaluation This standard is met. Conversations between

on and communication evaluators and the reading teachers is
Reporting s should have assumed. Since this evaluation is for a local
adequate scope program and all stakeholders are local,
and guard detailed communication is not necessary. A
against summary of the testing results and outcomes
misconceptions are appropriate. You incorporated
, biases, references to other reports on Reading
distortions, and Intervention Programs and their impact.
errors. One suggestion is to provide information
about each of the evaluators to gain trust in
reporting.
STANDARD RELEVANCE TO THE EVALUATION (CHECK QUALITATIVE

STANDARDS
Y Y Y Y
EVALUATION ACCOUNTABILITY
E1 Evaluation Evaluations should This standard is met. You have

Documentatio fully document their described the evaluations
n negotiated purposes purpose, procedures, data and
and implemented outcomes. You have described
designs, procedures, adequate documentation of your
data, and outcomes. plan and will be able to show
reading results based on
standardized test scores. By
accurately documenting the
evaluation, others designing
similar programs can benefit from
your processes and outcomes.
E2 Internal Evaluators should This standard is not met at this

Metaevaluatio use these and other point but will be as part of
n applicable standards assignment 5. Before conducting
to examine the the evaluation, take time to review
accountability of the your plan to see if it adheres to the
evaluation design, professional standards.
procedures
employed,
information
collected, and
outcomes.
E3 External Program evaluation This standard is met. The external

Metaevaluatio sponsors, clients, metaevaluation was a week-long
n evaluators, and process conducted by a group of
other stakeholders evaluators, Group 12. The
should encourage evaluators created a Google Doc
the conduct of which was used to collaborate,
external understand the evaluation and
metaevaluations rate the relevance of the standards
using these and addressed in the draft.
other applicable
standards.
Appendix E - Self Metaevaluation
Qualitative Metaevaluation Form Using The Program Evaluation Standards, 3rd Edition
MEDT 8480
Evaluator: Group 14
Metaevaluator: Melissa Brown, Jason Garofalo, 4/26

Thomas Kratowicz, Frederick Wright
NAME DATE
Instructions: Rate the relevance of each standard as it currently applies to the present
evaluation effort. Then provide feedback to the evaluator on each standard by highlighting
where the evaluator addresses compliance with the standard statement in the evaluation plan,
and the extent that the standard is being met. If in your opinion the evaluator has not or has
insufficiently addressed the standard, indicate so and provide constructive feedback or a
suggestion as to how to improve the plan. Finally, in the Overall Comments section found
below, summarize your feedback as it pertains to the evaluations Utility, Feasibility, Propriety,
Accuracy, and Evaluation Accountability. Try to come up with an overall statement of the
evaluation plans merit taking into consideration the context of the evaluation, the relevance of
the evaluation standards to the evaluation effort, and the extent that the standards are adhered
to and met by the evaluator in the evaluation plan. NOTE: This particular assignment is to
provide students an experience with an important and often overlooked aspect of evaluation:
metaevaluation. Your metaevaluative conclusions and feedback will in no way negatively
impact the course grade of the particular evaluator; however, they should be a constructive, yet
fair assessment so as to help improve the overall evaluation effort.
When addressing each standard, consider the whole evaluation plan including appendices. This
is particularly true of the Accuracy Standards. Are the evaluation instruments (if any) and
methods sufficient to answer the evaluation questions? If you have additional feedback (e.g.,
formatting, editing, APA), please comment in the Additional Feedback section.
Overall Comments:
In the beginning of our plan, we gave good amount of researched information on the
assessments that are tied into the components of the Extended Learning Time Program like
technology and Reading Plus Program. We made a plan to establish a relationship with our
stakeholders, created our three evaluation questions, and made a plan to collect the necessary
data in order to evaluate the program.
Utility Standards - We developed credibility for ourselves as competent evaluators by giving our
background which gained trust of the stakeholders. We made sure we involved stakeholders in
all parts of the evaluation and communicated openly with them.
Feasibility Standards - We provided clear roles for those who were collecting and analyzing the
data. We gave a clear description of all the data reports that would be used as data and how
data would be analyzed from each report. We will be in constant contact with our stakeholders
to keep them updated throughout the evaluation process.
Proprietary Standards -We have clear and well defined objectives as well as evaluation
agreements. Communication is strong and legal rights are laid out for all parties involved. We
have labeled clear steps of how the evaluation report will be delivered to stakeholders and how
their concerns will be addressed.
Accuracy Standards - The data that we collected all centered around answering the ultimate
question: Is the program helping increase the students reading scores? We did decide to use
open ended student responses in order to get more quality feedback and we were successful.
We did make a suggestion for the next evaluation that next time we need to get teacher input
about the program.
Evaluation Accountability Standards - Our plan documented our purpose and we followed our
logic model for procedures and outcomes. We used an internal metaevaluation in order to
examine and clarify the standards that we documented as clearly or highly relevant and fixed the
areas of concern. All updates are added to the final report which we will share with all
stakeholders.
Additional Feedback:
We have proofed our report for grammatical errors, placed headings in to make it easier,
and formatted the report with the same style, size and font.
We are very proud of our final report.
StandardS STANDARD Relevance to the Qualitative Feedback/Comments

STATEMENTS Evaluation (Check
One)
H F C H
A A L I
R I E G
D R A H
L L R L
Y Y L Y
Y
UTILITY
U1 Evaluator Evaluations should U1 Evaluator Credibility- this standard is met. In the plan we
Credibility be conducted by x proved our credibility by describing the 4 educators serving on
qualified people who the evaluation team who hold a combined 20 years of
establish and experience in K-12 education and discussing how we work
maintain credibility together in a group and as a team.
in the evaluation
context.
U2 Attention Evaluations should U2 Attention to Stakeholders is met in the evaluation of the

to devote attention to x extended learning time class and the reading program used.
Stakeholde the full range of The evaluation includes input from teachers and students and
rs individuals and communication is present on a regular basis with the
groups invested in stakeholders.
the program and
affected by its
evaluation.
U3 Negotiated Evaluation purposes U3 Negotiated Purposes is met through negotiations with the
Purposes should be identified x principal Dr. Stephens and his leadership team at Morgan
and continually County Middle School.. The negotiated contract was added
negotiated based on component of the final Evaluation Plan.
the needs of
stakeholders.
U4 Explicit Evaluations should U4 Explicit Values is met. Our evaluation contained clear
Values clarify and specify x specifications of the individuals and cultural values within the
the individual and purpose of evaluating the Extended Learning Time ELT class.
cultural values The purpose of the evaluation was to determine if the class
underpinning was helping the at risk kids for reading.
purposes,
processes, and
judgments.
U5 Relevant Evaluation U5 Relevant Information is highly relevant in the Extended

Information information should x learning Class that is used to improving reading for at risk
serve the identified readers. . This standard is met; quantitative data will be
and emergent collected through the Reading Plus program and reports that
needs of program offers. Qualitative data will be collected from
stakeholders. students within the class.
U6 Meaningful Evaluations should U6 Meaningful Processes and Products is met in this

Processes construct activities, x evaluation; the formative evaluation will assist the client in
and descriptions, and determining if the first year of implementation of Extended
Products judgments in ways Learning Time ELT class is meeting the programs goals to
that encourage help students improve their reading.
participants to
rediscover,
reinterpret, or revise
their understandings
and behaviors.
U7 Timely and Evaluations should U7 Timely and Appropriate Communicating and Reporting is
Appropriat attend to the x met; reporting delays did not occur in this evaluation.
e continuing
Communic information needs of
ating and their multiple
Reporting audiences.
U8 Concern Evaluations should U8 Concern for Consequences and Influence is met; the final
for promote responsible x Evaluation Plan included a contract.
Conseque and adaptive use
nces and while guarding
Influence against unintended
negative
consequences and
misuse.

STATEMENTS Evaluation (Check One)
H F C H
A A L I
R I E G
D R A H
L L R L
Y Y L Y
Y
FEASIBILITY
F1 Project Evaluations F1 Project Management is met. In this evaluation plan, we have

Manage should use x clear timelines, time estimates on when we will collect data and
ment effective project analyze data, and how you we plan on communicating with our
management stakeholders. We have strong purposes, goals, and objectives..
strategies.
F2 Practical Evaluation F2 Practical Procedures is met. Our evaluation plan has strong
Procedur procedures x program goals and program reports. We have set into place
es should be procedures to make sure that the data we collected from
practical and students is permitted.We made sure that the stakeholders
responsive to the understanding of the programs purposes, goals and procedures
way the program matched our evaluation goals.
operates.
F3 Contextu Evaluations F3 Contextual Viability is met for this standard. By adding in the
al should recognize, x survey to teachers and getting their input on the program itself,
Viability monitor, and we have given them opportunities to have their opinions and
balance the concerns about the program told.
cultural and
political interests
and needs of
individuals and
groups.
F4 Resourc Evaluations F4 Resource. We have met this standard. In our written

e Use should use x evaluation, we mentioned the possible costs of running the
resources study.
effectively and
efficiently.

One)
H F C H
A A L I
R I E G
D R A H
L L R L
Y Y L Y
Y
PROPRIETY
P1 Responsive Evaluations This standard is met. Stakeholders have been clearly

and should be x identified and were involved in the development of questions
Inclusive responsive to and purposes of the the evaluation. The local setting, history
Orientation stakeholders and and culture are respectfully addressed.
their
communities.
P2 Formal Evaluation P2 Formal Agreements is met. It is clear that the team has worked
Agreement agreements x with stakeholders on what will indicate success of the program in
s should be regards to the evaluation questions. Formal documentation was
negotiated to provided. Specific responsibilities are clearly assigned. The
make obligations agreement formally displays the communication and tracking
explicit and take responsibilities.
into account the
needs,
expectations, and
cultural contexts
of clients and
other
stakeholders.
P3 Human Evaluations P3 Human Rights and Respect is met. There are no major privacy
Rights and should be x concerns or data sharing concerns are indicated. Data is already
Respect designed and shared between stakeholders and would be unlikely to require
conducted to special consideration. The formal agreement details the limits of
protect human shared information, ensuring confidentiality expectations are
and legal rights respected.
and maintain the
dignity of
participants and
other
stakeholders.
P4 Clarity and Evaluations P4 Clarity and Fairness is mostly met. Students experiences and
Fairness should be x opinions, in the form of qualitative survey data, were collected
understandable and considered as part of data collection and results were
and fair in provided in the final report. Major themes in the qualitative
addressing survey were shared and suggestions for how to utilize the results
stakeholder were addressed.
needs and
purposes.
P5 Transparen Evaluations P5 Transparency and Disclosure is met. Openness and

cy and should provide x communication with stakeholders are clearly indicated.
Disclosure complete Presentation of the logic model, evaluation questions, and data
descriptions of collection processes are clearly discussed. Rights of the
findings, stakeholders are presented in the formal agreement and weekly
limitations, and reports of progress are established to facilitate openness at each
conclusions to all stage of the evaluation process.
stakeholders,
unless doing so
would violate
legal and
propriety
obligations.
P6 Conflicts of Evaluations P6 Conflicts of Interest is met. There were no conflicts of interest

Interest should openly x while doing this evaluation.
and honestly
identify and
address real or
perceived
conflicts of
interest that may
compromise the
evaluation.
P7 Fiscal Evaluations P7 Fiscal Responsibility is met.

Responsibil should account x A statement of budget or funding has been indicated. Minimal
ity for all expended resources were allocated to the evaluation team.
resources and
comply with
sound fiscal
procedures and
processes.

One)
H F C H
A A L I
R I E G
D R A H
L L R L
Y Y L Y
Y
ACCURACY
A1 Justified Evaluation A1 Justified Conclusions and Decisions standard was met.

Conclusions conclusions and x
and decisions should It includes the students who score below the 25th percentile
Decisions be explicitly which are at risk students. The logic model shows that
justified in the teachers are a big part of this program as they are the ones
cultures and who are in charge of the class. Students will be assessed
contexts where throughout the year to determine if program is still needed.
they have
consequences.
A2 Valid Evaluation A2 Valid Information standard is met.

Information information x
should serve the The program outlines how data will be collected. Outside
intended research was used to help validate the program. Data will be
purposes and
support valid collected by one of the various programs which will be used
interpretations. to determine student scores.
A3 Reliable Evaluation A3 Reliable Information standard is met.

Information procedures x
should yield The tests you use to gather information should provide you
sufficiently with the reliable information you need to determine whether
dependable and or not the program works. One thing you can do to improve
consistent this standard is listing sources of error that could occur in
information for your evaluation.
the intended
uses.
A4 Explicit Evaluations A4 Explicit Program and Context Descriptions standard is

Program should document x mostly met.
and Context programs and
Descriptions their contexts The logic model provides a good understanding of the
with appropriate program that is being evaluated. The main program that is
detail and scope being used, Reading Plus, is identified and is explained in
for the evaluation the document. Is there any information from stakeholders
purposes. that would be relevant to the program being evaluated?
A5 Information Evaluations A5 Information Management standard was met.

Managemen should employ x
t systematic This is done through how the data was collected. The data is
information not just collected in the programs used in the class the data
collection, is also collected in various other methods such as google
review, forms/drive. Information is stored at safe and secure sites
verification, and and can be accessed if needed outside the building.
storage methods.
A6 Sound Evaluations A6 Sound Designs and Analyses standard is mostly met.

Designs and should employ x
Analyses technically The design and analysis of the program seems sound. A
adequate mixed method approach is used to address the different
designs and types of data. One thing to consider is the constraints of the
analyses that are program, are there any costs that are required by the school
appropriate for board? Are there any certain political or social aspects to
the evaluation consider?
purposes.
A7 Explicit Evaluation A7 Explicit Evaluation Reasoning standard is met.

Evaluation reasoning x
Reasoning leading from This is met through the transparency of the program. Which
information and means that the reasoning to evaluate the program is clear.
analyses to This is also met by explaining how the data will be used to
findings, assist the students in the program.
interpretations,
conclusions and
judgments should
be clearly and
completely
documented.
A8 Communicat Evaluation A8 Communication and Reporting standard is met.

ion and communications x
Reporting should have This is met by explaining the methods of how your data will
adequate scope be collected both quantitatively and qualitatively. It is
and guard assumed that this data will then be shared with stakeholders
against to determine the validity of the program and whether it is still
misconceptions, important to have this program. One thing to consider is
biases, whether or not multiple reports will be needed or if one full
distortions, and report will be sufficient for all stakeholders involved.
errors.

One)
H F C H
A A L I
R I E G
D R A H
L L R L
Y Y L Y
Y
EVALUATION ACCOUNTABILITY
E1 Evaluation Evaluations For E1 Evaluation Documentation- This standard was met. We

Documentat should fully x have listed the types of documentation in our writing that we
ion document their used in our and have placed it into chart form. . As stated in our
negotiated plan, there are types of reports that the system uses that we
purposes and utilized for our data. In the appendix we have included a chart
implemented with the explanation of all the reports which assisted us in
designs, answering our three evaluation questions.
procedures, data,
and outcomes.
E2 Internal Evaluators E2 Internal Metaevaluation- This standard was met. We utilized

Metaevaluat should use these x the correct resources at the school like program data, test
ion and other scores, and feedback for the data we were trying to collect. The
applicable activities were practical and complementary to the Reading
standards to Extended Learning Time program we were evaluating. The
examine the evaluation was conducted with integrity and answered the three
accountability of
questions that the stakeholders wanted and we provided ideas
the evaluation
on how to utilized the findings of our report.
design,
procedures
employed,
information
collected, and
outcomes.
E3 External Program E3 External Metaevaluation This standard has been met. Our
Metaevaluat evaluation x expectations on the needs and purposes of the evaluation were
ion sponsors, clients, clear and clearly relayed to the stakeholders and evaluation
evaluators, and team. We made sure that we identified and agreed on what a
other quality evaluation resembled. We listed resources that were
stakeholders required for the metaevaluation and we were in constant
should
contact and collaboration with the stakeholders during the
encourage the
evaluation process. Communication is key
conduct of
external
metaevaluations
using these and
other applicable
standards.
Appendix F- Student Survey Responses
Discuss the use of technology in Discuss how

my class to teach, assess, and the Reading
learn with. Did you like the way we Discuss how the Reading PLus ELT program
used technology? Explain how it program helped you improve your has helped you
helped or hurt your learning in my reading. Even if you didn't like improve your
class. it....how did it help you? reading Grade
It helped me improve my reading lexil more time to
I didn't like using technology because .Even when I didn't wanna do it, I still work on reading
the WiFi is very bad at this school. did it. plus 6
yes i liked it but instead of doing More time to
vobab and doc in one week do doc work on Reading
one week the vocab the next week. no i dont like it but it helped me. Plus 6
I personally don't really like
technology that much anyways, so I honestly didn't see any improvement
having school work on it wasn't all that with Reading Plus. I just simply didn't.
great in my opinion. It hurt my head These new fancy programs hurt my time to finish
and i just wish we could do just paper, head and after a while I'll just end up other class
like how it used to be. getting confused. assigments 6
Yes,I liked how we have did mostly
everything on technology .You should
next year let everything be on
technology its better than wasting a lot
of paper and pencils .But when you
use the technology is actually I like the reading plus program
sometimes goes down.The reasin that because it helps you alot to become a Time to start
is ........is because the technology box better reader and it helps you stay on tests early or
can only hold up too like 20 people. your grade letter . finish tests 6
I think that in this class using
technology helped a lot. I liked how
we used it every day no matter what. i I think that reading plus helped
think that it helped since most kids improve my reading because it made Time to work on
have bad handwriting. us read faster than what we could. Reading Plus 6
I'm fine with the technology used in The reading plus program did not help Time to work on
class it does not bother me. me improve what so ever. Reading Plus 6
I honestly did not like the technology.
It always was messing up somehow.
Also I despised Reading Plus. Either
the tech would not load or it was It did not. I say this because it was
broken in some way. For a few weeks always weird with me. It would say I time in class to
in Computer Sience we couldn't do have more assessments than I should. work on
anything because the computers And it just made me confused. THats computer
couldn't log in. why i despise Reading Plus. programs 6
The use of technology helped me in
your class because my handwriting is
horrible. I did like the way we used mini lessons
technology because we hardly ever helped make
used paper and pencil. It didn't. sense 6
Well for me, using the technology for The reading plus program not only
everything was a bit too much. We helped me read a lot faster than what I Kahoot warm
used technology a lot and I just do not used to but I also learned a lot more ups 6
think it was a good idea. Some people words. Although they could put
do not have internet at home and interesting stories on there for us to
something could happen with the read. That would probably get more
power. I have internet at home but I students to do their sea readers. On
rarely use my computer. the read around they could put more
complicated words also.
I liked the way we used technology
because it helped us learn even
better.I liked it because it also It helped me by making me increase
improved my way of learning.It helped my reading.It helped me read even
me learn in the class.It helped me faster then I normally do.It helped me
because I learned how to use by making me read harder books.It
technology better.I liked the also improve my eye coordination.The
technology because we got to fun last thing was that I got to learn Time to work on
activities on them. quicker. Reading Plus 6
I know that I did not do my reading
plus very much ,but I think that it
To me the use of technology hurt my helped me a little.It helped me I liked being able
learning in reading. I am not used to because to me reading plus was very to work on
using technology.I like paper and easy and I could understand this Reading plus in
pencil better that technology. program . class. 6
The use of technology in reading was
okay. I would have rather done paper
and pencil, but technology was easier.
I would have rather done paper and
pencil for assessments because if the
computer crashed or deleted our
work, we would have had to do
everything all over again. The
technology was good to learn with
because we could play games that
helped us with our work. Another
reason I didn't think that technology
was good because I feel like we didn't Reading plus wasn't that bad of a
get enough face-to-face teaching and program, because it helped a bout
learning. Another reason why 95% of us improve on our reading
computers and technology wasn't scores. However, I think it was a little
good was because we had too much considering we had to do
EVERYONE in the class on DOK, book reports, adn reflections
technology, and sometimes the AND had to do reading plus. Especially
internet was slow and we owuld be left those of us with really busy schedules
behind, or other classes had to use and sports. My opinion was that it
computers but they were running helped, but it wa sway too much work It motiviated me
really slow and we couldn't finish our combined with our other work, and to read. I like the
work in time. other classes work. candy. 6
It was good to use technology
because it is easier for me to type
than write I can fast but not very fast.
It was good to learn because using
paper and pencil makes everything The candy
feel dull. Yes I did like the way we It helped a little. It helped by have the competitions
used tech. for games, reading, or in questions after the passages help me made me try
classwork. It helped because I could comprehend better. harder. 6
get work done faster. It hurt because

sometimes I got side tracked.
The use of technology in class this Smaller class
year helped and hurt something's you it help to bring it up by making you a made it easier to
just can't do on technology. better reader ask questions 6
i think it's good that we used more
technology in class.most class don't
used that much tech in class.in i think that reading plus help me a lot I didn't feel like I
reading we use a lot this year.i like a like see reader because it is easy to was the dumb
lot this year.it helped me this year. do. one in class. 6
I liked it a little because it gave us an
experience to work by ourselves. It
helped me by learning the information Reading Plus helped me improve in
at my own pase. It hurt us by reading by boosting up how fast I can Reading Plus
understanding the information in read a word and by helping and time to work
different ways. comprehend what I am reading about. on it. 6
I honestly think it was a little much.
Now if you see a baby their playing
with a phone but when I was a baby
we played with barbies or dolls. So I I hated Reading plus it gave me stress
think this kind of learning is ahead of and I never had the time to do it
its time. It hurt me a little bit because I because of all my sports and
can finish things on paper way afterschool activities so after dinner I I liked being able
quicker. It takes me a while to get would get it done which is normally my to have you
used to new technology and anything family time. Also I would take all of my lower my speed
new for that matter. But I could see weekend finishing it. It would take all of when the
this working better in the next few me to say that it helped me so it didn't. program sped up
years. sorry! to fast. 6
It helped my reading skills because my
We use technology a lot in this class. mom makes me do it just because it's
We play games and did a lot of other for a grade. I get to adjust the speed of
fun stuff. It helped my learning in this the reading on the passage. Oh, and I
class because when I write I don't get get to choose the passage I want to
done on time. When I type I do get read. I liked it better when it wasn't a Flash cards and
done on time. grade because I didn't have to it. vocab activities 6
The technology use in this class was
okay. Although I did not like that some
people got to have a Chromebook and Although I didn't like Reading Plus, it
others had to get a desk computer, did help me improve. By challenging
everything was good. Everyone is myself to do better. When i put all my
able to get on and do there work. I effort in it my might just finish it in time.
would say the access to the internet is Although it didn't help me on a busy
okay besides that it is very slow and weekend having to either skip it or quieter room to
outdated. That's how I think of the finding a way that still isn't good for work on reading
internet access in this class. someone in my family. plus. 6
I like using technology. It helped me
sometimes because like when i need
help spelling a word it helps me. It
would also harm me. i would send it I liked having
but it would never go through. Also time to work on
because when i am typing and i click The reading plus helped me because my Reading Plus
my mouse on accident and i would the sea readers helped me read faster at school and not
start to type and it would eras every and more fluent. at home. 6
thing i had. Using technology helped

me but it also harmed me but i like
using it.
We use technology for work and when I liked having
we play game that involves time to work on
technology. I like the why we use my Reading Plus
technology in your class,it help me get Reading Plus help me improve my at school and not
work done. reading scores, Even if I didn't like it. at home. 6
I love the use of technology. I feel that
it is much better than just doing things
with paper and pencil. Only bad thing
about it is occasionally the connection
fails but when we get our new school
we won't have to worry about that. I
love how most everything you do
online can be saved, and with papers
they can get lost or misplaced. Also
the fact that you can share a
document with someone if you are
doing a group assignment instead of
everyone having to share one paper
they can do the same thing online but
all be working on a different part at couldnt answer not enough time, bad Time to work on
one time. I love the way we use it. computer Reading Plus 6
i think that we should not us
technology as much. we do i think that
we should be tout with book because
they want us to read more books and
that would be a good think to read
more in class it help me in prove reading skilles small class size 6
i liked using technology in class. it they gave use a grade on it so it gave
helped keep up with my things for someone people a boost in there Time to
class so i didn't have to caring it grade. it helping me reach my goals in complete
around everyday reading reading plus 6
I loved using technology in the
classroom because it is easy to turn in
assignments, you cant loose your I liked having
homework like you can with paper, time to work on
and I have bad hand writing and I am my Reading Plus
not a good speller but typing stuff is at school and not
way better for me. Reading plus just helped me a little bit. at home. 6
it helped because i would not lose
work that i have to send to the teacher
. using the computer would let me get
things done but i some time did not
turn it in or i tuned it i late. it helped me read faster reading plus 6
Yes i really liked how everything was
based on tech and not paper. And i
think this really helped me with my
grade because i was able to keep up it helped me by encoring me to do Time working on
with everything. good so i could get combos or medals. MyOn 6
We used technology to help us learn. It helped me by reading at a certain I liked having
You used PowerPoints to help us pass and remembering what I read. time to work on 6
better learn the material. You made us my Reading Plus

games to learn our vocabulary and at school and not
what we were learning about. You at home.
made us games to review before we
took tests. I liked the way we used
technology. We used technology a lot
and that helped me learn more and
better. It helped me learn better
because I understood more of what
we were learning.
I liked the way we used technology in
class because that way if we didn't Reading plus helped me get better at
have technology in class then the my reading because It took time to
school was able to provide for us. Also actually read the passages and answer
they where a lot easier to handle and the questions. You also can gain
they where a lot faster, so it made it knowledge and reading skills through
easier to get work done. They where this reading plus program, which i like Time to work on
also a good size so its better to carry. a lot. Reading Plus 6
Reading Plus really helped me this
I liked the technology this year. It year. I started out with a low lexile and
really helped me out when I didn't now I'm at a high lexile. Although, I
have any of my own technology. It didn't like Reading Plus from all of the
really helped me look things up fast. It reading and answering questions, it
actually helped my learning because it was worth it when getting a grade on it
would search whatever we needed to and getting candy. Reading Plus is Working on
search. I like the way we did really hard if you don't pay attention. reading plus
technology this year by signing the You earn rewards when you read a lot during school so
sheet to get technology. That way we or master a lot of words. You should I didn't have to
can tell who uses the last computer. It really get involved in it to boost you work on it at
helps us with responsibility. lexile. home. 6
I did not like it because everything we
do is on the internet. Some kids can
not access the internet at home. And
yeah we do have lunch and learn and
morning and after school tutoring. But
you still cant finish everything that the I liked having
teacher gives you. And some kids time to work on
may not be able to get to a place that Reading plus helped me one with my my Reading Plus
does have wifi or whatever to do their grades. But it also helped me with at school and not
work. helping me do be a better reader. at home. 6
The tech idea has actually helped me It kept my reading grade very well
because I am able to find what I need balanced and not out of control as it is
and all my other assets. But the one now which speaking of that, I may
problem I have with it is the WI-FI want reading plus to come back that
being so slow and because of that way my reading grade and come so Having time to
now I can barley get done with I need that the damage that has been caused work on
to get done because of the internet in the past can be fixed in the future vocabulary
having it's occasional heart attacks. today. games.. 6
I liked the technology usage in the Reading plus is a good program. I got I liked having
class. I liked it because it was mad at it sometimes, but I see how it time to work on
something new and I am the kind of helps you. It makes you a more fluent my Reading Plus
person who likes technology. It reader. It helped me expand my at school and not
sometimes hurt and helped me. Which vocabulary. It also helped me track the at home. 6
isn't bad. So, I would recommend words with a program they had.
using technology like this again. Overall, it is a good program.
When we use technology in the
classroom i like it better then pencil
and paper because you have to type Reading plus made me become a
and your hands want hurt so bad as better reader in reading and in all of
you do when you write. my classes. Kahoot games 6
I loved using technology in reading.
Instead of getting paper and writing I didn't like Reading Plus at first
like in all my other classes, we did it because it made me sleepy and i didn't
on the computer. This made it easier want to spend time doing it. But when I
to turn in work. I was also able to keep seen my lexile score a big happy smile Time to work on
up with my work and not have to dig in was on my face. This is all because weekly work in
my backpack for it. Reading Plus. ELT 6
Yes and no.I liked using technology
because it was easier all we had to do
was discuss the thing we were
working on in class for example.Doing
the vocab on the computer was easy
and the DOK. But if you didn't get
finished in class you . have to do it for
homework. The only reasons I don't
like this is because, some people
don't have there own internet or
device.So they can't do there work so It did help me improve with my vocab Time to work on
they won't get there work done. and my reading skill. Reading Plus 6
I liked being able to use technology for
a lot of things. I think its a good idea,
and that it helps out a lot. I don't like to
write most of the time. It helped a
good amount of my grades because
its easy to go to my mom's laptop and Reading Plus did help, even though I I liked having
start typing. It is not easy trying to go didn't like it. There was some really time to work on
through your folder looking for work boring stories with really hard my Reading Plus
that you could of sworn you had put in questions. Some of the questions had at school and not
there. words i didn't even know existed. at home. 6
I liked being able to use technology for
a lot of things. I think its a good idea,
and that it helps out a lot. I don't like to
write most of the time. It helped a
good amount of my grades because
its easy to go to my mom's laptop and Reading Plus did help, even though I
start typing. It is not easy trying to go didn't like it. There was some really
through your folder looking for work boring stories with really hard time to work on
that you could of sworn you had put in questions. Some of the questions had computer
there. words i didn't even know existed. programs. 6
I did not like to use technology, when Reading plus helped improve my
the technology did not work or was reading with the see readers and read Time on
slow it stoped me from doing my work. around. programs 6
To be totally honest I dislike the In my opinion I wanted no part in the
technology, because paper and pencil reading plus it didn't do me any good, it Having time for
is just a whole lot easier and u don't didn't help me or not help me, but I my teacher to
have to worry about messing up, or think that y'all should find a different motivate me. 7
network loss. So I think that the program for the kids, because I don't
technology hurt mrs Browns reading like reading plus and I don't think
class, but others might think different. anyone else does either.
This is what I think about the
technology of this school year in
reading.
I love the way we used the
technology.I believe it also helped my
grade.I believe that because anything
we did on technology that was a grade The reading plus program helped me
I aced it. increase my reading lexile. time 7
the reading plus program did not help
i did like the way we did technology me at all. it was easy at the begining
because in our other classes we have but began getting hard and hard. the
do pencil and paper work and the stories got worst and when i mean
technology we use gives us a break worst i mean hard. i failed three or four
off of pencil and paper work. i don't see readers when i first start and i try i
know if it helped or hurt my reading do i work so hard to get a good grade Time to work on
but i think it did both. on reading plus but i couldn't. missing work. 7
I personally like the ides of using
technology in class everyday. I believe Even though I disliked doing reading
this because most teens these days plus, it really did help my reading. The
use technology every single day and it see readers helped me on my
is a good way to connect with us by comprehension and the read arounds Time to work on
using technology everyday. helped my with my vocabulary. Reading Plus 7
the use of technology for me was
getting use to it because before this reading plus did help my reading level
school year i did not really kniw how to a lot i can tell that it helped me with
use a computer i was used to writing. reading and real world information.
but i had to get use to it and once i did though i did not like reading plus i
i could get all my work done. the use thought it was a good way to grade us.
of technology hasnt just helped me in one thing i would change would be the
reading but it has helped me with fact that is is 30% of our grade i would
typing and technology and things i make it a ten percent at first and if you
might use in the future. i didnt ever notice no one is getting it done raise it
have a problem with my technology to a thirty% but over all it helped me Time to work on
and it really helped me in this class. and thats really what matters in school. Reading Plus 7
I like using the technology in reading I
learn better own technology.Also I
think it was good because most
people don't even have paper and if
we have a technology class you don't
need paper. I like technology and It did help me do better in reading
don't have to hear can Ihave a piece class got me to read faster.I didn't like Time to ask
of paper.Can I have a piece of paper. it but i help me a lot. questions 7
I like using technology sometimes
because it's better than writing on
paper everyday. We have more Reading plus really isn't my cup of tea
access to the world than we do on because it's to much, and I know that it
paper. I also don't like it as much helps, but there always to much
because I don't know how to work lessons. The scanner on the passages
these high tech things. I got to get go to fast for anyone to read. I don't Time to work on
someone to help me most of the time like reading plus that much. missing work. 7
because I don't understand as well as

everyone else does.
i liked using technology in class but i
also disliked the use of technology in
class. i liked it because it was easier
to keep up with. i disliked it because i
did not always have technology to do
the assignments out of school. in it helped me read faster and it helped
conclusion technology both helped my grammar comprehension develop Time to work on
and hurt my learning in reading. more. Reading Plus 7
I liked the way we used all of the
technology but it really depends on
how you are using it like if you are
writing a story it is good to use the
computer to write your paper but if we
are taking a test i think it is better to
use pencil and paper because
sometimes the computers can crash IT has helped me by raising my levels
and it can lose all of your progress and becoming a stronger reader and
and make you take it again in another also helping me understand the stuff i Time to work on
time read alot better Reading Plus 7
The way that we used technology in
your class has both hurt me and
helped me. I liked it because you
would assign good websites and
games to help us learn our standards.
Sometimes I didn't like it because we Reading plus helped me to learn new
have a lot of technology problems. words. I didn't like it because it would
Also I sometimes couldn't access take some time. Also whenever I did
these things at home. I get my work good it would give me even harder one
done more often if it is done on paper. which caused me to fail. It was a grade Time to finish up
In my opinion I think that you should in class which brought it down tests or complete
do a bit of work on paper. sometimes. make up work. 7
Using technology all year long for
classwork was great. I loved it
because I never lost my work, and it
would always be where I could find it. I
disliked it because sometimes the The reading plus program helped me a
internet wouldn't work and we would lot, although I didn't like it. Reading
barely get any work done all period. I plus improved my reading overall, and
feel that using technology in this class it also helped me understand what I Time to work on
helped my learning. am reading better as well. Reading Plus 7
The use of the technology is very vast. Okay, I hate reading plus with a
We use technology everyday in the passion. But that doesn't mean that
class room, and I love it! It is a great reading plus grew me as a reader. It
way to turn in things, take tests, and did! It helped me read so much faster
receive feedback. I love taking tests than I used to. I'm able tto identify the
on the computer so I immediately main idea of a passage or two much
know my score, I don't have to wait 6 better. I am also able to notice vocab
years to find out. It saves teachers words that I have never seen before I liked having
time, because they don't have to go and figure out what they mean. time to work on
through all the stress of taking grades, Reading plus has really grown me as a my Reading Plus
and finding out who's paper is who's, reader, even though I absolutely hate at school and not
and they also don't have to grade it. at home. 7
them their selves. Technology makes

life so much easier, in class. When we
do things on technology I generally
work on it at home, so I can get it
done, and turn it in. I think using
technology is a great way to run a
class.
I liked the way we used technology
most of everything we have is on a the reading plus helps me understand how
computer and any or no paper was fast i read and what level i am on.I Time to practice
really wasted or we had to keep up think they should keep this for next my vocabulary
with. year. with the games. 7
I liked the way we used technology in
reading cause we can play candy
question games and we can get most The reading plus program helps me
of our homework done in this class with my reading in the futures and
and we don't have to worry about helps we with my reading and help and
doing it at home and we if people then it helps me with words that i need
don't have access to the computer at help and then i won't need help with
home Mrs.Brown gives us time in any other words in life but words i don't Time to work on
class to do our work. understands . Reading Plus 7
I think the use of technology was
awesome in this class. It helps us I liked having
keep book bag material low, and it time to work on
also helps us not lose anything. It I think, as much as I hate to say it, RP my Reading Plus
helped me because I didn't have to helped my reading. It helped me be a at school and not
keep up with papers or binders. faster, more comprehensive reader. at home. 7
I like the way we use technology. I like Even though I didn't like it it helped me
it because I think it's more engaging. I with my reading. At the end of the year
think this helped my grades. Because I noticed I went up more than 4 grade Time to work on
I like typing more than writing. levels. Reading Plus 7
Mini lessons
The usage of the tech was very good Reading plus is alright. help me. 7
I like the way we used technology. I
liked it because we never really had to
write anything. It is way easier to type
than to write. It helped my grade
because every day we used it and it It brought my reading level up 3.5 Time to work on
was easy to use. years. Reading Plus. 7
I like the use of technology in the
classroom. I think it helps some I didnt really like it. It hurt my head, the
people and doesnt help others. I think words moving fast, everytime i did it Time to
it helps me most of the time, but then took advil afterwords. But i guess it complete my
it is fun to use pencil also. I think I got helped me read a little faster. And i See Readers in
more work done using technology. learned some Vocab. class. 7
It helped me because if i didn't do my
I learned very well by using work i would have a bad grade in
technology because its helps me to reading but with reading plus if you
stay on task and to do my work, read carefully and answer the
because i know we are using it and questions right you will get a good Work on reading
tech. is a big part or things now. grade in this class and will be passing. plus programs. 7
I liked the use of technology because

it kept me awake and alert.
Sometimes I would have rather done
paper and pencil work because I like
to be able to touch and see things
physically. I understand things better The Reading Plus program has
when it right in front of me , not on a improved my reading skills. it has
computer screen. Other times I like improved my reading skill by making
having technology. I liked all the me a faster reader with all of those see Making sure my
games we got to play such as Plicker readers. It has also strengthen my weekly work is
and Rags to Riches. vocabulary. done. 7
I feel like the reading plus program did
not help me. I did the stories but none
of them made sense and I failed a ton
I loved using technology in your class. of them. I might have of learned new
It helps me because it saves all my vocab but that is about it. I would not
information and I don't have to worry suggest doing reading plus next year
about it if I do not have my book bag. I because it drops the kids grades. I am
can work on it from different locations just trying to help them so they have to
also. Using technology in your class go through another year of pain Reading Plus
really helps me a lot. (in a good way) (reading plus). and MyOn 7
I liked that we used technology this
school year. I liked that we used
technology for everything this year. I
liked it because it was easier to type
than write for most of the things we
did this school year. Using technology
helped me on my learning this school The reading plus program kind of
year by getting to type instead of helped me improve my reading. It
write. Technology really helped me helped me to read a little bit more fast working on my
because your hands don't get very and helped me understand different reading plus see
tired typing. words. readers 7
I like the use of technology in your
class, I think you should keep the
technology that you use. It helps me
and hurts me. It helps me by being
able to type faster plus it's easier to do It helped me improve by making me
the written questions so your hand read faster. I didn't like it, but it helped
does't hurt after you finish. It hurts my me a lot with my reading I went from
learning because sometimes you have an eighth grade reading level, at the
glitches and the computer will mess beginning of the school year, to a ninth candy rewards
up what you have worked on then you grade reading level, at the end of the for completing
have to restart. school year. my lessons 7
The use of the technology in class
was good.It made things a little
easier.Like the book reports we
usually write them.Also the use of Yes because I used to be at a sixth
technology in class helped me turn in grade reading lev l and now I am on a reading the plays
work earlier. eight grade reading level. aloud in class 7
The use of technology was great at Well to be honest I did not like it one
most of the time. But also the bit. But after all I showed a lot of
technology was really bad because of improvement at the beginning till now.
how long it would take to long on and The part that help me the most was Games with
pull up websites. But the use of doing the stories and answering candy rewards. 7
technology did help me in many ways question. It help me understand the

like when we had assignments for story more better because it had that
class that I didn't get done with I could line thing that would cover up the rest
easily access it at home. But of the story and leave one line open so
sometimes when I tried to get on that I could read it and go on. So that
some sites at home I wasn't able to part helped me not just skip around
because of something happening. I and just read some parts.
really did like the way we used
technology because we didn't have to
worry about having paper or pencils or
even worry about writing a lot. So the
use of technology this year was great
and I think using can be a lot more
easy and fun to learn what we need to
learn and which we did learn a lot.
Reading Plus helped me improve my
The use of technology was great. reading because it helped me maintain
None of my other teachers ever used my ability to read fluently, yet it also
technology this much in the taught me many new words, or at least
classroom. It was much easier using what some of the words I had heard
technology than having to keep up before meant. It helped me focus on
with tons of papers. Plus with important things in a story as well,
Edmodo, you could keep track of what since you have to answer questions
you had been assigned. I really about the stories you read. I think Repeating
enjoyed and benefited from the use of Reading Plus is a program the school lessons in the
technology in reading class. should keep. ELT class 7
I loved the way the way we used the
technology because it gave us the
relaxation from all the writing we had
to do.It helped us by that saying
because it was easy to assess and It didn't help me at all it caused me Being able to
was more understanding and less from well holding back from all my work one on one
complicated. other things and assignments. with Mrs. Brown 7
Technology help us and hurt us. It It didn't help me because I never did it
help us to look up information about when I should have. I should have
our vocabulary and other stuff. But I done my reading plus and I would
hurt us because we would play games have gotten a better grade then I do Time to work on
and not pay attention. have now. Reading Plus 8
I like using technology . It helps me
because we are always writing all day
. We finally get a breakin reading . It I'm one of those people that dont like
helped my learning in this period reading plus . It did help though . I kept
because I like technology . There is a on becoming a stronger reader . My
downfall because sometimes scores increased . I also became to Being able to
technology doesn't work and plan b not hate it so much because I saw how work one on one
comes into action . it helped . with Mrs. Brown 8
I use the technology good in reading
like i would help my teacher put the I would bring my grade up if i was
technology even if it wasn't asked for doing it, like sometimes i would stop
me to do so , and that's what i love doing it for a while, because it
about that.It helped me because i got sometimes make my vision hurt
most of my stuff done in this class , because i wear glasses, and so Time to work on
and i really like this class alot. without them i wouldn't be able to see. Reading Plus 8
I think the use of technology is good in Reading plus has improved my reading
the classroom because it helps us out because i can focus more on the main Repeating
with reading plus, sri, and even book idea and show me words i never knew lessons in the
reports. about. ELT class 8
I liked using technolgy because we
will use it in the future. I did and didn`t
like using technolgy, I think that we
should use paper and pencil more
often so we don`t lose our writing skill. It brought up my grade to 6th grade Being able to
The didn`t help me because i am not a reading level to a on grade reading work one on one
good typer. level...(LIKE YOU SAID:) with Mrs. Brown 8
I liked the way we used technology.
There is one part that I don't like and I am being totally honest I don't really
that is reading plus. Overall the use of think that reading plus helped me I
technology has helped my grade in didn't really see anything that helped in Time to work on
class. that program. Reading Plus 8
I love the way we use technology in
our class. I love using technology
because my had writing is bad. When
we use technology we use programs
that are sometimes fun to use and i
learn better when it is fun. We use The reading plus helped me to become
technology to help us learn and it help a faster reader and helped get me
us learn at our own rate. closed to grade level reading. Practice tests 8
Yes, I like the way we used
technology in our class. I think that I I personally don't like readingplus, but
do better on technology. I also think it it did help me a little bit. It helped me Smaller Class
helped my learning in reading class. with my comprehension a little bit. sizes 8
I hated it so much!! It guess it helped
(not really).I feel like if we were reading
something we like than we would want
to understand it more. People don't do
their reading plus. Therefore, they fail
this class. It is probably one of the
easiest classes as long as you do your
work. Many people would be passing if
we didn't have it or have it as a 30%.
I'm not saying it completely didn't work
because it did help me (a little) get my
reading lexile up. Last year in 6th
grade it was a 10% and everybody did
it (for the most part). I think we should
be doing FrontRow as our "Reading
Plus" because it is more like the tests
Yes. It helped me have a better we have because you can go back to
understanding of my reading the text anytime you want. Plus there
comprehension. It didn't hurt me are less questions, which makes the
because we did everything on Google reader more interested because they Time to work on
docs or slides so I could do it from can get it done faster. They also have Myon projects
home. a lot better stories. during class. 8
The use of technology was very good Candy
in my class. We used technology The reading plus program helped me a Challenges to
everyday. The technology helped lot but I didn't like it one bit. But it get my work
teach, asses, and learn because we helped me raise my lexile and help me done. 8
used different programs to help teach become a faster reader in the 7th
and learn with this year. Yes I liked grade.
the way we used technology this year
because we used it everyday of the
school year and I think the technology
helped us grow in our academic
success this year. The use of
technology helped my grade very
much because it is a lot simpler that
paper and pencil. And you can turn in
assignments online.
I liked that we never had to actually sit
down and write except maybe 5 times
this year if even that. I think it's easier
to turn in assignments and remember
them. I do think however the use of
technology can get overwhelming. It helped my eyes comprehend what i
The netbooks and chrome books don't was reading faster. I think that mine
autocorrect. They also don't have a went a little too fast but my eyes
right and left button on the mouse to adjusted to it and i can read faster
get the tool to correct it. (except now. I now know a lot more words Having a choice
netbooks) Overall i like using because of the vocab section than i did on Fridays of
technology because it's faster than before. Reading plus has helped a lot Myon or Reading
writing. even though it wasn't my favorite. plus 8
I like the way we used technology this
year. I liked how in class we would
use quizlet before we took a test. The
technology helped me this year in
reading. the way we learned with the The program helped me raised my
technology was good. This is how I reading level. I did not like it but I still Time to work on
feel about using the technology this liked how it helped my reading level go my computer
year. up. programs. 8
I like the technology. It gets us used to
doing most of your work on the
computer. The only thing I didn't like
was how I don't have internet and
reading plus is on the computer. I
didn't like how book reports where on
the computer because sometimes it is
easier to do it on paper. I like
everything else about the technology. I feel like it help me to be able to read reading plus
It helped me in most ways. faster. program 8
The technology was easier in some
views, but harder in others. It was very
frustrating when plans for the whole
class would get canceled because of
the technology. I did like it though
because it reached out to us as kids,
and met us where we were most
comfortable learning because most of Having time to
the time we would get different It helped me see how high of a reader I finish reading
assignments depending on where we am. It also taught me comprehension plus and not do it
were. skills. on weekend. 8
I loved the way we used the

technology in this class. It helped my
grade so much because I am a I liked the candy
technology person and I can get stuff It helped me learn new words and challenges to
don fast on technology. understand them. motivate me. 8
I think that the use of technology Even though I don't like Reading Plus,
helped us in this class. It allowed us to it did help improve my reading scores.
have more room to do more without I just didn't like doing it along with all of
buying a binder. Although, when some the other homework we get form other
people don't have technology, they teachers. I think it helped me though
have to use netbooks, and sometimes because my lexile went up a lot since A quiet place to
they didn't work. Technology helped the beginning of the school year. I work on my see
me in this class because I couldn't think on reading plus, I went up like 4 readers and not
forget any assignments. They were all grade levels since the beginning of the have to do them
online. year. at home. 8
I don't like reading plus. In my humble
Technology was helpful and harmful. opinion,mi don't believe it helped me at
On one hand, we didn't really have to all. My mother doesn't like it either, she
have paper the whole year and got to said she can't keep up with the story
be on technology all the time. We also and she even has a degree in
go to try new technology based children's education. If reading plus
educational games. On the other even did help me it would be for paying
hand, our wifi at MCMC is slow. attention to the stories. I tend to Reading plus in
Sometimes it's frustrating trying to wander when I read and this kept me class and not at
load a slow device. engaged. home. 8
I really loved the way we used Reading Plus helped me become a Working on
technology.I also think we used too faster reader.I once was a slow reader Reading Plus in
much technology. now I'm a very fast reader. class. 8
At the very beginning of the school
year, I did not like using technology.
The only reason, was because you
needed internet. I don't have internet
at my house, so it was difficult for me.
Until I began to get a hand of it. Soon
after a couple of months went by I find
it a lot easier. Than after so long I
realized that on google docs, there
was this thing called "Available offline"
where you can edit offline and you
don't need internet . Instead of paper Time to work on
and pencils I now use technology for computer
every assignments. Better Reader programs. 8
The use of technology is like a
everyday thing because we walk
in,read the board,and it says to secure
technology.I think this is a good thing
not to use paper a lot in this class to
save trees and have extra oxygen to
have more supplies for next year
because we use paper every day in
most of our classes it is a good thing I liked it a little because we learned a
not to have you writing and to be little because it gave us some
looking at some sort of technology information on the things we were Having computer
every once in a while. learning in class. time in class. 8

Extendedlearningtimeevaluationreportfinal 1

Hochgeladen von

Dokumentinformationen

Originaltitel

Copyright

Verfügbare Formate

Dieses Dokument teilen

Dokument teilen oder einbetten

Freigabeoptionen

Stufen Sie dieses Dokument als nützlich ein?

Sind diese Inhalte unangemessen?

Copyright:

Verfügbare Formate

Extendedlearningtimeevaluationreportfinal 1

Hochgeladen von

Copyright:

Verfügbare Formate

Running head: Reading Extended Learning Time Evaluation Report

Evaluation of Reading Extended Learning Time at Morgan County Middle School

Program Evaluation Report

University of West Georgia

Melissa Brown, Jason Garofalo, Thomas Kratowicz, Frederick Wright

Extended Learning Time Evaluation Plan

Background of the Program and Evaluation

Instructional technologies are continuously changing. From an iPad to a Chromebook,

also labeled at risk.

disabilities had gains using the program, as well.

this the best way to reach these students for remediation?

increase their overall reading scores on standardized tests?

their reading levels towards grade level?

self-confidence towards reading?

collected anonymously. We used a correlational research design to exam the relationship

variables and examined relationships between those variables (Anderman, 2009).

have an Individualized Education Plan.

According to the Northwest Evaluation Association (n.d.), this assessment is a computer

individuals in reading. According to the Institute of Education Sciences, Reading Plus is a

we used technology? Explain how it helped or hindered your improvement in reading.

enrolled in this program helped you improve your reading?

prefer it, how did it help you?

Timeline For Data Collection

Cloze Assessment August 2016 and May 2017

CRI Assessment December 2016 and May 2017

SRI Assessment August 2016, October 2016, December 2016,

MAP Assessment August 2016, December 2016, May 2017

Student Surveys May 2017

in vocabulary, comprehension, and Lexile score improvements (Fournier, 1995).

Comprehensive Reading Inventory(CRI), provide reliable normed evidence to track growth,

positive changes in attitude towards reading (Fournier, 1995).

students in their reading abilities.

finish tests or start tests early.

at risk reading students.

their middle school years.

outcomes for struggling readers: A best-evidence synthesis. Reading Research Quarterly,

Fournier, D. M. (1995). Establishing evaluative conclusions: A distinction between general and

working logic. New Directions for Evaluation, 1995(68), 15-32.

Northwest Evaluation Association (n.d.). NWEA: Measures of Academic Progress (MAP).

Retrieved from https://www.nwea.org/

multi level perspective.Journal of Educational Computing Research, 33(4), 369-393.

April 6, 2017, from http://www.belmont.edu/oair/assessment-support/pages/qualitative-

between a silent reading fluency instructional protocol on students reading

comprehension and achievement in an urban school setting. Reading Psychology, 32(1),

Reading Plus. (2010, September). Retrieved April 6, 2017, from

Mixed Methods Research, 1(1), 77-100. doi:10.1177/2345678906292430

Survey Questions for Students

Instruments used in collecting Data

Name of Instrument How and Who Gives? Data Retrieved

Milestone State Test End of School Year 2017 Lexile score

All on-site fieldwork, data collection, interviews, and/or observations shall be

Background: The purpose of this evaluation is to formatively investigate the

School program. Evaluation findings will be used to improve the program.

CONTRACT AWARD MEETING

SPECIFIC MANDATORY TASKS AND ASSOCIATED DELIVERABLES

Description of Tasks and Associated Deliverables:

The Evaluator will provide the specific deliverables described below.

Deliverable 1: An Evaluation Plan which describes a background of the Evaluation of

Task 1: The Evaluator will collect new or gather existing data.

Deliverable 4: A final evaluation report in APA format.

SCHEDULE FOR DELIVERABLES

Deliverable Due Date

CHANGES TO STATEMENT OF WORK