Ed Tech 505 DSIS Math Tutorial Evaluation

DSIS Math Tutorial Evaluation
Submitted to: Davis School for Independent Study

Completed by: Christine Simms
May 9, 2014

DSIS Math Tutorial Evaluation | Submitted to: Davis
School for Independent Study
Page 1 of 16

Summary
The Math Tutorial Program at Davis School for Independent Study (DSIS) is an intervention that
was put in place as a result of declining test scores and low course grades. It supplements the
direct instruction that occurs in weekly math classes to support improved academic achievement.
It offers four hours a week of tutorial time staffed by the sites credentialed math teacher. The
tutorial hours give students opportunities to get help with their homework and clarify content
understanding through one-on-one interaction with the teacher. Student attendance is voluntary
until a students grade falls below a C-; at that time, the student is mandated to attend at least one
hour of tutorial a week.
This formative evaluation was completed during the fourth year of the program to determine the
extent to which it is meeting its established objectives. It presents an analysis of data gathered
from interviews with administrators and staff, standardized test scores, course grades, the 2011
WASC Report, School Site Plans, program observations and a student survey as they relate to
the achievement of program goals to support students and increase academic achievement.
The evaluation found that the Math Tutorial Program is at least a contributing factor to student
success, but that academic achievement in mathematics remains a critical area of concern as data
does not reflect the desired student outcomes. CST scores have not risen by the 5% a year goals
outlined in the School Site Plans and many students are struggling to earn passing grades in their
math courses. The report identifies areas where historic data is missing and suggests
implementing tracking protocols that would support future evaluations.
Description of the program evaluated
DSIS is a K-12 site located in a small city flanked by a major university and surrounded by
farmland. Supervising teachers meet weekly with their students individually. Secondary math
courses meet three times weekly (except Junior High Math which meets twice) and are taught by
a credentialed math teacher. Because of the more limited contact time, math is a particularly
challenging subject to teach in an independent study setting.
The Math Tutorial Program was instituted in the 2010-2011 school year as one of the changes to
the structure of the math program after the schools Self-Study for their Western Association of
Schools and Colleges (WASC) accreditation renewal identified declining California Standards
Test (CST) scores in mathematics and low grades in math courses as critical deficits. Four hours
weekly FTE, divided across all five school days and distributed between morning and afternoon
hours in order to accommodate the diversity of student schedules, is allocated to staffing the
tutorial hours with the schools credentialed math teacher to better support student success in
mastering the course content delivered during math classes. (See Appendix A) This intervention
provides a less structured learning environment where students can get individualized help with
homework assignments, make up missed tests, and ask questions that support their developing
understanding of concepts and processes.
The program serves all eight levels of secondary math courses taught at the site: Junior High
Math, Transition to High School Math, Pre-Algebra, Algebra 1, Geometry, Algebra 2,
Trigonometry, and Business Math. It is open to all DSIS students on a drop-in basis and meets in
the school library. Once-weekly attendance is mandatory for students with Ds or Fs. The math
Page 2 of 16

teacher circulates among the students, checking in to see how things are going and responds to
individual questions as they arise.
Program Objectives
The Math Tutorial Program is in place to improve student achievement in mathematics at DSIS.
Based on the programs goals, the following program objectives were established.
Students with grades of D+ or below will receive the individual support they need to
increase their overall course grades to passing by attending a minimum of one tutorial per
week until their grades improve.
Students will have the individual help they need outside of class to have their questions
answered and content clarified to support content mastery as measured by increasing
overall course grades.
Students will receive the individual attention needed outside of class to have questions
answered and content clarified to improve content mastery to proficient or above as
measured by the CST.
Students will develop the content mastery necessary to increase confidence and
engagement to prevent them from dropping math courses without completing them.
Students will receive the individualized attention they need to increase content mastery
and confidence necessary to support them in continuing to progress on the math pathway
through high school graduation.
Program Components
The foundation for Math Tutorial Program is four hours of dedicated FTE for the credentialed
math teacher to provide individualized support in a casual group setting, on a drop-in basis, for
students taking all levels of math at DSIS. Students typically bring their own math books and
graphing calculators, but they are available to borrow as needed.
Evaluation Method
In order to determine the extent to which Math Tutorial is meeting its established objectives, a
goal-based model was used to design this evaluation.
Participants
Given the relatively small size of the school, all math students in grades 7-12 were surveyed
during their math classes and students attending tutorial were observed as a part of this
evaluation. Additionally, interviews were conducted with the DSIS math teacher and two former
principals who had been directly involved in establishing the program.
Procedures
The evaluator began by meeting with staff and administration to determine program history,
goals, and objectives, and identify existing data sources to be reviewed for the evaluation.
Existing data was gathered for review of program history and student achievement since the year
prior to the institution of the program to establish a baseline and examine progress toward
achieving program objectives.
A student survey was designed and administered as a follow up to the student survey on program
participation completed in the spring of 2013 as well as collecting additional data related to
student attitudes and future plans for continuing on the math pathway. Observations of Math
Page 3 of 16

Tutorial were conducted to see the program operating under normal, everyday conditions. It was
confirmed with the math teacher that the observations during the evaluation period were typical
of normal use.
Finally, survey and academic achievement data were converted to excel for analysis and the final
evaluation report was produced for the school.
Data Sources
Data was compiled from numerous sources to develop a comprehensive overview of the
evolution of the program from its inception to current practice. The evaluator conducted
interviews with the DSIS 2010-current math teacher, Jamie Wales, 2012 Math Committee Chair
and 2012-2013 principal, Laura Juanitas, and 2008-2012 principal, Kim Wallace. She reviewed
the 2011 WASC report, 2012 Math Committee meeting notes, School Site Plans for 2011-12,
2012-13, and 2013-14, and math department syllabi.
Student achievement data was gathered from the 2011 WASC Accreditation Self-Study Report,
School Site Plans, 2013-14 class marks recorded in Q, and 2013 California Standards Test (CST)
scores reported on Illuminate. Students also self-reported average grades on the student survey.
Data related to student understanding of and program use was gathered using a student survey
(See Appendix B), evaluator observations, and previously utilized attendance tracking
mechanisms/records. (See Appendix C) Existing data from a survey administered in the spring of
2013 was also reviewed.
Student attitudes about math and future plans to continue on the math pathway were also
gathered using the student survey.
Results
Academic Achievement
Prior to changes made to the structure of math delivery at DSIS and the institution of Math
Tutorial, student achievement data was indicating a steady decline in the percentage of students
who were performing at levels of proficient or above on the CST. (Table 1)
Though the state goal is for all learners to
be proficient or above, the 2010 results
reported that 43.38% of DSIS students
were in the proficient/advanced range.
This prompted changes to the frequency
of math classes and the inception of the
Math Tutorial Program. Moving forward
from this point, there was a noticeable
upturn in CST score data.
Table 1
Page 4 of 16

Scores from the spring of 2011 indicated that the percentage of students scoring
proficient or above had risen to 47% and
those from 2012 were fairly consistent with
that at 46%. There was a significant decline
in the 2013 results, with only 30% of students
achieving proficient or advanced. (Table 2)
Course grade data available was limited to
the current year, 2013-14. At the end of the
first semester, 49% of all math course grades
were in the A/B range, 33% were Cs, and
18% were Ds or Fs. Progress marks for the
second semester are recorded at the end of
the third quarter. At that time, almost the
same percentage (48%) of students were
performing with sufficient mastery to earn
As or Bs, 14% of students were in the C
range, and 38% of student marks were either
Ds or Fs. (Table 3)
Survey Data
The student survey provided data related to participation in the
Math Tutorial Program as well as information about student
attitudes about mathematics and their long-term math goals.
Of the students who completed surveys in 2014, 56% reported
attending Math Tutorial at least once during the current school
year. (Table 4) 14 of 52 students said that there are times when
attendance at Math Tutorial is mandatory. Of those, only 6
indicated that the condition that would mandate attendance is
poor grades. None of the students who identified failing grades
2009 42.00%
2010 43.38%
2011 47.00%
2012 46.00%
2013 30.00%
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
CST Mathematics 2009-2013
Percent of Students Scoring
Proficient or Advanced
Table 2
A/B
49%
C
33%
D/F
18%
2013-14 Semester 1
Math Grades for All Courses
A/B C D/F
Table 3
A/B
48%
C
14%
D/F
38%
2013-14 Quarter 3
Math Grades for All Courses
A/B C D/F
Yes
56%
No
44%
2013-14 Student Use
of Math Tutorial
Table 4
Page 5 of 16

as triggering an
attendance
requirement
reported averaging
grades lower than
Cs. The top three
reasons students
reported choosing to
attend Math Tutorial
were that they
needed help with
homework, they didnt understand something in class, or a parent required them to come. (Table
5) The three most frequently stated reasons for not attending were that they didnt need help, had
schedule conflicts, or that they get help elsewhere. (Table 6)
37% of students
surveyed reported
having sources
other than Math
Tutorial to support
their success in
math classes: 22 get
help from parents, 8
have UCD tutors, 7
have other tutors, 7
get help from
friends, and 4 have other sources of support. 76% of students reported that they always or
usually understand content the content clearly enough to be successful by coming to class and
completing homework/studying on their own.
A similar number of students were
surveyed in 2013. At that time, 63% of
responses indicated having attended
tutorial at least once during the school
year. Of the students who were not
electing to attend, 58% cited schedule
conflicts and 21% reported that they
get help elsewhere.
The 2014 student survey also
generated baseline data related to
student attitudes about math,
themselves as math students, and their
plans for taking future math courses. (Table 7) The majority of students surveyed reported
feeling pretty good at math and that they plan to continue taking math courses until high school
graduation. Students not only care about their math grades, but want to do even better.
1
0
4
6
5
13
20
3
11
0 5 10 15 20 25
Other
Help with Graphing
Like the Routine
Helps Me Feel Confident
Like One-on-One Support
Didn't Understand
Needed Help with
Ms. Wales Required
Parent Required
Why Students Report They Attend Tutorial
(could choose multiple responses)
Table 5
6
6
2
2
1
3
1
12
4
4
2
4
3
1
17
18
3
10
14
10
8
13
20
10
11
8
17
10
4
4
33
27
25
19
32
ENJOY MATH
GOOD AT MATH
CONTINUE MATH TO GRAD
CARE ABOUT GRADES
MATH IS IMPORTANT
TRY MY HARDEST AT MATH
WANT TO DO BETTER
Student Attitudes on a Scale of 1-5
1="no way" 5="definitely"
1 2 3 4 5
Table 7
3
9
3
9
24
19
23
0 5 10 15 20 25
Other
Might Help, but Not Motivated to Go
Don't Think it will Help
Satisfied with my Grade
Don't Need Help
Get Help Elsewhere
Schedule Conflict
Why Students Report They Do Not Attend Tutorial
(could choose multiple responses)
Table 6
Page 6 of 16

Attendance Data
Two attendance tracking mechanisms were included in the review of existing data. In the 2010-
2011 year, supervising teachers received an email listing all students who should be attending
tutorial (or working with a UCD tutor) due to low math grades. Students and their parents had
already been notified; the idea was that supervising teachers could remind their students when
they saw them at their individual meetings in order to improve student follow-through. In the
2012-2013 school year, the math teacher tracked attendance each week in a binder, writing down
the names of students present at each tutorial session and also collecting data related to their
degree of engagement during the hour.
Policy Related Data
Syllabi received by parents and students at the beginning of the course outline the tutorial policy
as being optional for students with grades of C- or better, but attendance is mandatory at least
once a week for those with current grades of D+ or below. The penalty for not attending when it
is required is a warning letter. (See Appendix D)
Both administrator interviews indicated that attendance is mandatory for students with Ds or
Fs. Interviews with the math teacher offered the same data, but revealed that there is currently
no tracking mechanism in place because the things she has tried have been cumbersome yet
unsuccessful. Responses on the student survey revealed that while 27% of students believe that
tutorial attendance is sometimes mandatory, only 12% are aware of low academic performance
being the condition that triggers mandatory attendance.
Discussion
The purpose of this evaluation was to determine the extent to which the Math Tutorial Program
is meeting its established objectives. After several interviews with stakeholders and a thorough
examination of all available data, it seems evident that the Math Tutorial Program is a
contributing factor in the overall success of DSIS math students. However, given that at least two
major adjustments were made at the same time, an increase in the number of hours each math
class meets weekly and the institution of Math Tutorial, along with incomplete historic data for
comparison, it is impossible to determine the precise impact of the program.
In looking at the CST scores, there is a significant increase (+3.62%) in the percentage of
students scoring proficient or above in the first test cycle (2010-2011) after the tutorial program
was established. There is a drop of 1% the following year, but given the small size of the school,
this is insignificant and both the 2010-11 and 2011-12 school years present as consistent. The
2012-13 scores offer a different picture, with only 30% of students scoring in the
proficient/advanced range.
A number of possibilities that have nothing to do with Math Tutorial, or the math program at
DSIS, may account for the sudden dip in scores. Scores are reported based on students currently
enrolled at DSIS rather than on students who took the test at DSIS last year. Because the school
started the year with fairly low enrollment, many of the current students were not at the site last
year, thus their CST scores do not reflect on the teaching or interventions available at DSIS.
Additionally, even students who take the CST at DSIS may not be receiving their instruction at
the site. Split site students may take math at a comprehensive site, but because they are
considered DSIS students, they are tested at and scores are attributed to DSIS. Another
Page 7 of 16

possibility is a changing student population at the site. These scores should be further analyzed to
determine whether the decrease in scores is related to an increase in the population of the
Socioeconomically Disadvantaged (SES) student subgroup. Historic data indicate that this group
has performed significantly lower than the general DSIS population. If a change in population is
responsible for the dip in scores, it may be necessary to reconsider the best interventions for
meeting student needs.
Undoubtedly, academic achievement in math continues to be an area of concern at DSIS. CST
scores have failed to reach the 5% annual gains established in the school site plans and many
students are receiving Ds or Fs in their math courses. Math Tutorial exists as an intervention to
support student success, but is not being fully utilized to reach its potential. This may be due to a
gap in student perception. 10% of students surveyed self-reported D/F grades three weeks after
Q3 Progress marks indicated 38% of students were earning Ds or Fs.
Student survey data also indicated a lack of student understanding regarding the Math Tutorial
attendance policy related to course grades. Though clearly stated in each course syllabus, only
12% of students reported being aware of D/F grades making tutorial attendance mandatory.
Student confusion may be attributed to a failure to attend to that detail in the syllabus or may be
related to the disparity between the stated policy and current practice which does not track
attendance or penalize for lack thereof. The math teacher reported in her interview that she feels
this is the biggest issue related to Math Tutorial. She has tried different methods, but has found
them to be unsuccessful. She would like support in identifying a system that would work for both
her and the students, 44% of whom cite schedule conflicts as preventing them from attending
tutorial hours.
The 56% of students were have attended tutorial this year were well-served by the program. Far
more (60%) responses indicated reasons for attending related to seeking one-on-one support than
indicated attendance because a parent or the teacher required it (22%). This demonstrates a
conscious recognition of the value students place on having individual attention from the teacher
outside of class time.
67% of students report that they get help with their math from sources other than Math Tutorial.
It may be appropriate to more closely examine the correlation between individual student
achievement and what sources they are using for support. Given the high percentage of students
with low course grades and the number of students stating schedule conflicts as their reasons for
not utilizing the tutorial hours, it may be possible to provide more consistent, reliable support if
tutorial hours can be configured differently or delivered in a different format, perhaps
synchronously online.
Several program objectives relate to student attitudes about math as they affect decisions to drop
courses without completing them or long-term plans to continue progressing along the math
pathway beyond requirements for graduation. As there was no historic data in this area, it is not
possible to determine how the individual attention students receive as a result of Math Tutorial
has influenced changes in attitudes or plans. Additionally, there are no mechanisms in place to
track either course retention or how many/when/why students elect to stop taking courses in
mathematics. If the objectives are to remain the same, it will be necessary to determine a plan for
gathering the necessary data. The data collected for this evaluation may serve as a baseline in the
area of student attitudes for future evaluations.
Page 8 of 16

Table 8
Objective Identified Strengths Identified Concerns
Students with grades of D+
or below will receive the
individual support they need
to increase their overall
course grades to passing by
attending a minimum of one
tutorial per week until their
grades improve.

Support in place to serve
student need

Attendance policy for D/F
grades
Support underutilized by this target
audience

No effective mechanism in place for
tracking or enforcing attendance
requirement

Communication of policy to students
not clear
Students will have the
individual help they need
outside of class to have their
questions answered and
content clarified to support
content mastery as measured
by increasing overall course
grades.
Students who attend are
getting individualized
support to clarify content
No tracking mechanism in place to
compare course grades historically

Students will receive the
individual attention needed
outside of class to have
questions answered and
content clarified to improve
content mastery to proficient
or above as measured by the
CST.
Support mechanism is in
place
Though growth was indicated, it never
reached 5% annually and declined
considerably in 2013

Further evaluation of the CST data is
necessary to address possible reasons
for the decline
Students will develop the
content mastery necessary to
increase confidence and
engagement to prevent them
from dropping math courses
without completing them.
Most students have generally
positive attitudes about math

Some students who attend
tutorial report that they do so
because it increases their
confidence

No historic data is available to
determine the role of program
participation in student attitudes. This
evaluation collected baseline data on
student attitudes that can be used going
forward.

There is no readily available data
related to course completion or the
reasons students have dropped math
courses
Students will receive the
individualized attention they
need to increase content
mastery and confidence
necessary to support them in
continuing to progress on the
math pathway through high
school graduation.
Students report feeing
supported by the individual
attention they receive at
tutorial

Most students have generally
positive attitudes about math

Most students are planning
to continue taking math
courses until graduation
No historic data has been kept related to
DSIS math students continuing to take
math until graduation. In order to
follow up on the achievement of this
objective, it would be necessary to
institute a mechanism for tracking this
information.

This evaluation collected baseline data
related to student confidence that can be
used for comparison in the future.

Page 9 of 16

The strength of the Math Tutorial Program is clearly in its ability to provide students with the
individual attention they need to address their own gaps in understanding. Because of the
structure of math instruction at DSIS, with the teacher having only two or three hours to deliver
all of the content students will need to understand for the week, classes tend to be packed with
direct teaching and offer very few opportunities for student questions. Math Tutorial definitely
fills the gap for students by offering personal support and encouragement on an as-needed basis.
The weakness seems to emerge from the inability or lack of desire for some students to recognize
their need for additional support, or from the schedule conflicts that prevent them from accessing
the resource. For this reason, it is necessary to establish a means of enforcing the attendance
requirement for those students who need help but are not coming on their own. It should be
manageable for both students and teacher. Rather than constant reevaluation, perhaps just use the
first of each month as the cut-off date to mandate weekly attendance until a reevaluation the
following month. This would put all courses on the same cycle. A tracking sheet could be created
as a simple check-off that is marked when the student attends once during the week, no need to
track attendance for those who are not mandated to attend or for multiple days for those who are.
In looking at each of the objectives established for the Math Tutorial Program, this evaluation
has identified both strengths and concerns that can be used to begin conversations between staff
and administration to chart a path for program improvement. (Table 8) It will be necessary to
revise program objectives in light of the move to Common Core Standards and the transition
from the CST to the Smarter Balanced Assessment (SBAC). Until baseline scores are available
after the 2014-2015 school year, achievement goals should be tied to course grades.
Project Cost
Principal Evaluator, Christine Simms, was on the project for a total of ten days of billable hours.
She spent two days on site at the school and district office conducting interviews with staff and
administration, conducting observations of the program, and gathering existing data. She created
a student survey to be administered by the math teacher, then compiled the results for analysis.
The balance of time was spent reviewing and analyzing data and producing the final evaluation
report.
Personnel Budget
Christine Simms [Principal Evaluator]: 10 days @ $500/day $5000
Total Personnel $5000
Miscellaneous
Supplies [paper, ink, copying, etc.]: $100
Total Miscellaneous $100

Total Budget $5100

Page 10 of 16

Appendices

Page 11 of 16

Appendix A
2013-2014 Secondary Class Schedule with Math Tutorial Indicated

Page 12 of 16

Appendix B
MATH STUDENT SURVEY (please do not include your name on this paper)
Math course ___________________________ My test grades are typically: [A/B] [ C ] [D/F]

Is attendance at Math Tutorial ever mandatory? [Yes] [No] If yes, when? _________________
Have you attended Math Tutorial during the current school year? [Yes] [No]

If yes, what did you find valuable? (select all that apply)
[Having a scheduled time to do my math] [Working with others from my class]
[Getting immediate feedback to my questions] [Other:]_____________________________

How frequently do you attend Math Tutorial in general? (select one)
[2+ times/week] [1x/week] [every other week] [1x/month] [<1x/month]
[only before tests] [only to make up tests]

Are you able to understand the content clearly enough to be successful by coming to class and completing
homework/studying on your own? [Always] [Usually] [Sometimes] [Rarely] [Never]

Do you have other support outside of math class besides Math Tutorial with Ms. Wales? [Yes] [No] If yes,
where does that support come from? (select all that apply)
[UCD tutor] [Parents]
[Other tutor] [Friends]
[Other:]__________________________________________________________________________

In weeks when you do attend Math Tutorial, what are your reasons for doing so? (select all that apply)
[Parent required it] [Ms. Wales required it]
[Needed help with homework] [It helps me feel more confident in my understanding]
[Didnt understand something in class] [I like the routine (its where I do my homework)]
[I like the one-on-one support] [Help using the graphing calculator]
[Other:]__________________________________________________________________________

In weeks when you do not attend Math Tutorial, what are your reasons? (select all that apply)
[Schedule conflict] [Im satisfied with my grade]
[I get help elsewhere] [I dont think it will help me]
[I dont need help] [I know it might help, but Im not motivated to attend]
[Other:]___________________________________________________________________________

Rate the following on a scale of 1 5 where 1 is no way and 5 is definitely

I enjoy math 1 2 3 4 5
I am good at math 1 2 3 4 5
I will continue taking math
classes until I graduate
1 2 3 4 5
I care a lot about my grades 1 2 3 4 5
Learning math is important 1 2 3 4 5
I try my hardest in math 1 2 3 4 5
I want to do better in math 1 2 3 4 5

Page 13 of 16

Appendix C
2010-2011 Mandatory Attendance Tracking

Page 14 of 16

2012-2013 Attendance Tracking Document

Page 15 of 16

Appendix D
Algebra 1 Syllabus/Course Policies 2013-2014
Page 16 of 16

Ed Tech 505 DSIS Math Tutorial Evaluation

Hochgeladen von

Dokumentinformationen

Copyright

Verfügbare Formate

Dieses Dokument teilen

Dokument teilen oder einbetten

Freigabeoptionen

Stufen Sie dieses Dokument als nützlich ein?

Sind diese Inhalte unangemessen?

Copyright:

Verfügbare Formate

Ed Tech 505 DSIS Math Tutorial Evaluation

Hochgeladen von

Copyright:

Verfügbare Formate

DSIS Math Tutorial Evaluation

Submitted to: Davis School for Independent Study

Das könnte Ihnen auch gefallen