Designing and Implementing a Benchmark Assessment System in the Oakland

Unified School District

Amy Malen, TBR VI

Abstract and Introduction

This paper explains the process of designing and implementing a benchmark assessment

system in the Oakland Unified School District (OUSD). Specifically, we focused in

Oakland on teacher and principal buy-in, supporting the use of data to inform instruction

at the classroom and school-level, and overcoming the challenges associated with

designing a system intended to accomplish multiple goals for multiple stakeholders (ie,

instructional tools for teachers, monitoring tools for principals and central office

leadership, etc). The implementation of the benchmark system resulted in an increased

focus on state standards, an increased use of and transparency in data, and teacher and

principal dialogue regarding student strengths and deficiencies and the implications for

instruction.

Situation

The district had been administering district-wide assessments for several years, but they

were not standards-based. In most cases they were largely driven by textbook content.

The Chief Academic Officer wanted teachers, principals and central office leaders to be

regularly reviewing data that would inform instruction and provide evidence of progress

against state standards. Since the existing assessments did not provide adequate data and

stakeholders were not used to using data in this way, we needed to create a set of
standards-based meaningful assessments and provide significant levels of training to

principals and teachers on effective use of data.

Given this task, we designed the OUSD comprehensive assessment system, which is

designed:

o To provide teachers, principals, and other stakeholders with timely and relevant

information about students’ learning so that they may strengthen and enhance

specific areas of instruction.

o To expose students to grade level standards and high levels of academic rigor

and provide real-life test-taking experiences. In addition to providing teachers

with information about student learning, these assessments provide students with

information about their learning so that they feel prepared to succeed on the

California Standards Test.

o To provide district leadership with data to evaluate the effectiveness and impact

of instructional programs and to inform differentiated resource allocation to

schools.

Contribution

People: I started at OUSD in the Strategic Projects department working on Performance

Management. During my last 18 months, I served as the Director of the Research and

Assessment department. Once I was in the leadership role in Research and Assessment, I

felt as though I had ownership over the work and thus was in control of the project. I had

an incredibly strong assessment coordinator on my team who was critical to managing

the ground-level implementation. I had a very strong relationship with the Chief

Academic Officer, who was very clear with principals and Network Executive Officers
on the importance of the assessments. It was absolutely imperative that I worked closely

with the Executive Officer of Instructional Services despite our differing styles. I was a

member of our district’s Educational Leadership Team and had strong relationships with

the Network Executive Officers. It was EXTREMELY critical for me to develop and

maintain strong relationships with the Network Executive Officers because they manage

the principals and are the liaison between central office and schools.

We spent as much time communicating our message regarding use of assessments to

teachers, principals and Network Executive Officers as we did actually creating the tools.

It was always a challenge to appease multiple audiences who had multiple uses for the

data. For example, there was a concern that the assessments would be used as evidence

for teacher evaluation. We had to make the following message very clear and repeat it

over and over to multiple audiences:

‘As a district, our top priority is high-quality instruction that results in high levels of

learning for every student in every classroom every day. In support of this priority, the

benchmark assessments are designed first and foremost to be instructional tools for

teachers. To this end, teachers must have all data available to them immediately after

each assessment administration. Administrators should have access to the data to

SUPPORT teachers effectively utilize the results to improve classroom instruction. All

other purposes such as predicting state test performance and programmatic evaluation are

secondary. The assessments ARE NOT to be used for teacher and principal evaluation.

Network Executive Officers should be knowledgeable about the general performance

trends of their schools, but it is MUCH MORE IMPORTANT for them to be

knowledgeable about how their schools are using the assessment data in PLCs and how it
is informing professional development, strategic use of coaching services, etc. It is our

belief that if these structures are in place and the principal is effectively utilizing

resources to support teachers in targeted areas, that students will perform successfully on

end of year tests.’

The Oakland Educational Association (OEA) is the teachers’ union in Oakland, and they

vocalized many concerns regarding assessments. We met with them frequently and sent

out communications to principals and teachers when we knew they were voicing

concerns. In general, staying abreast of the union’s message and being open to dialogue

really minimized the impact that they could have on teachers. As an example, at one

point they tried to encourage members to boycott assessments, but our participation was

between 80 and 90%, which is on the high end of normal.

Building the Assessments:

We partnered with vendors to create the assessments, which is strongly recommended.

Previous efforts to create our own assessments in the district were extremely time

intensive and ineffective. We did feel as though it was essential to have teacher input,

though, so we established a teacher review process. This was critical to our success, both

because teachers do have insight regarding student understanding and curriculum and

because it was incredibly powerful to be able to respond to teacher concerns by letting

them know that the assessments were vetted by their peers.

We distributed assessment blueprints, which provided teachers with detailed information

regarding the numbers of questions per standard per test, for all assessments in the spring

before teachers went home for the summer. This was very much appreciated because

teachers felt that there was a level of transparency, which had not previously existed. We
also provided the assessment calendar at this time, and teachers could use all of these

tools to inform any planning they might do over the summer.

Using the Assessments-- Data Inquiry:

In general, OUSD feels that all assessments can be used for learning, meaning that

teachers can and should use the evidence and feedback assessments provide to identify

where students are in their learning, what they need to do next and how best to get

achieve this. Teachers discuss assessment results and share strategies and next steps with

their colleagues in their Professional Learning Communities.

We worked very closely with our principals each time assessments were administered to

help them understand the data and to provide them an opportunity to plan with their peers

regarding use of data at their school sites. This occurred at their network meetings,

which were led by their bosses, the Network Executive Officers. This model helped us

ensure that principals were looking at the data in a timely fashion and added an element

of accountability to the process.

Lessons Learned:

We faced many challenges along the way. These included:

•Teacher buy-in

•Many middle and high school students are performing several years below grade

level

•Lack of resources and support for intervention – Students are low performing, so

now what?

•Inconsistent level of capacity around data, technology, and instruction across

schools
 •Over-testing and burnout for students

These concerns that were often voiced by teachers were valid, and it was important to

acknowledge them. Communication and transparency was key. We sent regular updates

to teachers, principals, Network Executive Officers and others.

There were also issues that arose in the past that we realized were within our control. For

example, assessments would be sent out late and with errors. It was CRITICAL that our

materials were perfect, and that we fully admitted mistakes if errors were made.

We had a set of young principals who were deeply committed to use of standards-based

assessment data and they served as our voice to their peers and teachers, which was much

more effective than the message coming from central office.

Outcomes (and intended outcomes)

The Oakland Unified School District now has a comprehensive standards-based

assessment system. We administer quarterly ELA and math assessments, writing,

science, social studies and more.

Most impressively, teachers and principals now feel as though they have the data that

they need to more effectively do their jobs. In 2005-06, only 60% of teachers and 32% of

principals claimed they had access to the necessary data to do their work. In just two

years, those numbers rose to 80% and 95% respectively.