f.

Selection of Goal-Based Model for New Evaluation

As I would truly get into trouble and my efforts would go completely unappreciated by evaluating the Freshman Academy, I have a new evaluation that will be welcomed with open arms. Our high school, as many has struggled with an appropriate standardized test. In Maine, the state had an 11th grade assessment which it found disfavor. Maine (and most of New England) adopted the SAT for its 11th graders. This would create several problems: now being the standard, more than 95% of 11th graders must take it and how to best study just for the SATs. The 95% problem proved huge; as a large percentage of students from this community have no post secondary plans. While a good year would see 75-80% of the students moving on, that was about the percentage taking the SAT test. To get 95% involved getting students with no future plans to take the test (the state picked up the cost). This often resulted in a room full of students the first school day after the test wishing they were not there for the makeup. These students would often only do just enough to have the test count, but received the lowest score possible. This brought the test average down considerably. While very different tactics would have to be employed with the students above, it was decided that a more concentrated SAT preparation time was required prior to testing to bring up the average for the other students. During student advisory for a month prior to the May test date, juniors spent their time in English rooms studying for the test (Math scores had seen a slow, small increase, but English scores had plummeted due to the above). (Now I can outline my evaluation, need to preface so the evaluation would make sense). I plan to evaluate data from previous years to this years test data. Additionally, if I am able to get and use PSAT data for the same students the previous year, I can even show hoped improvements on a student-by-student basis. This could really be considered a Formative Evaluation for next year. Based upon very preliminary data (and a very mechanical evaluation) from Educational Testing Services (ETS, the SAT people), this will not be a Decision Making Evaluation as the average test scores are up in English. Therefore I will be conducting a Goal-Based Evaluation to try to determine which portions of the preparation might be improved to increase scores next year. If this proves successful, perhaps the allotted time could be split with Math and as now Science has a Maine Only SAT achievement test. Results of this evaluation will be

beneficial to administrators that initiated this effort, guidance councilors and English teachers.

k. Data Sources for SAT Preparation Evaluation

As stated in f. above, I will be evaluating recent SAT data. The most recent group of students to take the test completed a directed study program once a day for three weeks prior to the testing. Test scores will be compared to past SAT scores (from other classes) and to this particular groups PSAT scores from last year. The primary source of the information will come from average scores of students from past years of SAT testing. There are at least two significant problems when comparing values from year to year; the exam changes and the curriculum for students change. A significant change in the scoring occurred four years ago when a third score, for writing was introduced. Even though the writing portion is always first and does not take much time, it allows less time for the other sections, therefore the test scores can only be compared for the past four years. Also, ETS does change the exam slightly every year to make it more current and in line with what colleges are looking for. Secondly, as the years go by, students taking even the same classes, are exposed to different curriculum (even the classes that students take before their junior year change). As my state has changed the required standards two times in the last four years, students have had a different exposure to the same core areas. Despite all that, math is math and english is english. Many studies have been done and if one does not try to do too much with the data, they are more or less similar for year to year. Perhaps the best data (while not absolute) will be to look at results for individual students between their sophomore and junior years or between the PSAT and the SAT results. Similarly, the two tests are not meant to be identical, so direct correlations cant be drawn. The PSAT (Preliminary SAT) is similar enough that students are never surprised with the SAT testing, format or even types of questions on the SAT. The tests are used for different purposes which does not support comparison. However, studies also support that correlations can be drawn. The year-to-year SAT comparison is the easiest to get data for as it is easily downloaded from several locations and is only available in an average score format. Some additional information is also available (class size, % of class taking test and a few others), it has little impact. I will be looking at the quantitative data on a student-by-student basis between the PSAT and SAT scores. Currently the data is only available in a .pdf format but it is hoped prior to the July deadline, a

huge spreadsheet will be transferred that documents student responses on a question-by-question basis. This would be very valuable for the English teacher but may be inappropriate for this evaluation (almost too much data) other than its useful format. Lastly, students were questioned after the testing and nearly all said they thought the additional sessions were beneficial. Preliminary results concur with that but I will evaluate that.