Improving Education

Outcomes

Mövenpick Hotel
Dubai
7th December 2017
Improving education and
assessment – principles
and practice in national
and international context

Roderic Gillespie
Director of Assessment 7 December 2017
Agenda

High performing jurisdictions

Educational reform
Standards
Effecting educational reform
Assessment as a force for change
Purposes of assessment
Assessment frameworks
 Characteristics of
high performing jurisdictions
 High performing jurisdictions have a good ‘ecosystem’, everything works
in balance and is very much dependent on context, form, history and
culture.
 ‘High performance’ is the result of complex interactions of elements which
take on a distinctive form that produces positive outcomes in a
jurisdiction.
 Interactions are very complex, but they are aligned, they are balanced
and they interact beneficially.
 Curriculum coherence is a key feature – the two senses of ‘coherence’
• System alignment – administrative/governance
focus/models/frameworks
• Curriculum construct, sequencing and balance – psychological-
educational focus
 Have ‘big ideas’ for educating the young people that drive ‘system
transformation’, have high expectations that are well supported
and are well understood and socialised in implementation.
 Evolution of Singapore’s education system
 In tandem with economic development & nation
building – how the examinations are being influenced by
curriculum changes and also how examinations have in turn
influenced teaching and learning.

Innovation
driven
Knowledge
Student-centric,
based
values-driven
Skills intensive Ability-driven from 2012
Efficiency 1997–2011
Industrialisation driven
1979–1996
Survival
driven
1959–1978 5
Discussion – educational reform

Consider an educational reform project that you are familiar

with.
What were the aims of the project?
Was there a ‘big idea’?
What were the factors
that influenced the success,
or otherwise, of the project?
Effecting educational reform

 Based on good international reference points – i.e. they were evidence-

based reforms and made changes only where justified.
 Unnecessary disruption to the education system was avoided, i.e. there
were clear, well thought through drivers, and ‘change’ was managed.
 Reforms had clear vision, which were well communicated.
 There was good governance and sensitive policy formation – timelines
were realistic, and there were development, implementation and
evaluation strategies.
 Change processes were managed – change absorbs resource in both
development and implementation.
 The relationships between control factors were examined – these are
complex; one factor was not adjusted without considering the effect on
the others.
 Teacher training, development and support was provided.
Case study – Scotland
Curriculum for Excellence
PRE-REFORMS
 Achievement in Scottish schools was above international averages – but
the gap was narrowing.
 OECD expressed concerns over attainment and progression.
 Particular concerns were raised about maths where performance had
fallen from high to average over the past decade.
 The primary school curriculum did not prepare students particularly well
for lower secondary school.
 The lower secondary curriculum did not prepare students particularly
well for 14 to 16 qualifications
 There were two different sets of 14 to 16 qualifications, and neither of
these provided appropriate progression to the 16 to 18 qualifications.
Curriculum for Excellence – aims

 Provide a coherent, flexible and enriched curriculum from 3 to 18.

 Equip learners with four key ‘capacities’ – successful learners, confident
individuals, responsible citizens and effective contributors.
 Provide a new 3 to 15 curriculum – to maximise progression through the
curriculum and onto qualifications and provide more personalisation and
choice.
 Give schools more autonomy – e.g. in curriculum structure, teaching
programmes (especially in secondary).
 Give teachers more flexibility over what and how they teach.
 Provide a set of 15 to 18 qualifications – build on strengths of old
qualifications and maintain high standards and credibility
 Assessments to have more focus on skills refreshed & relevant content.
Discussion – reforms and organisational
Discussion
alignment

Read the ‘Actions Required’ sections, look at how the scope,

roles and responsibilities of each organisation are described.
How useful do you think these sections are in
describing the roles and responsibilities of key
organisations and how they should interact?
 Evaluation – Curriculum for Excellence
OECD Report (2015) – found ‘much to be positive about’, including:
 Levels of academic achievement are above international and other UK jurisdictions,
averages in science and reading, but not maths.
 Achievement levels spread relatively equally.
 Scottish schools do very well on measures of social inclusion and mix.
 Immigrant children generally perform better than their non-immigrant peers.
 A large majority of students feel positive about their school and teachers.
 Nine in ten school leavers enter a ‘positive follow-up destination and two-thirds stay in
education.
But
 20% of schools are rated only ‘satisfactory’ and 10% are ‘weak or unsatisfactory’.
 There is still some confusion about the high level aspects of the reforms.
 Assessment results are often not used to improve pupils' learning progress – and a lack
of clarity over what should be assessed.
 Assessment arrangements do not provide sufficiently robust information, whether for
system-level policy-making, or for local authorities or an individual school.
Use evidence from international research and
best practice, for example …

• Curriculum coherence
 Schmidt and Prawat; Oates; Trends in International Mathematics and Science Study (TIMSS)

• Detailed scrutiny of arrangements

 Organisation for Economic Co-operation and Development (OECD); Reynolds and Farrell; Nisbet;
Sahlgren; Cambridge abroad

• Sound theory on the operation of education

‘arrangements’
 Bhaskhar; Sayers; Raffe

• International comparisons
 Using international comparisons to refine the National Curriculum (Oates): Ofqual; Schmidt &
Prawat, Raffe, Coles, Reynolds, OECD, Sahlgren

• Consideration of national and international datasets

 Programme for International Student Assessment (PISA); Trends in International Mathematics and
Science Study (TIMSS); Progress in International Reading Literacy Study (PIRLS); National Pupil
Database (NPD) assets
Discussion – standards

To be high performing means to have ‘high

standards’

What do we think is meant by

the term standards?
… standards internationally

The term is used differently

 USA – standards viewed as content and attainment and
aptitude measured by psychometric tests.
 China – standards viewed as ideas and incentives for
continuous improvement, but do not use year-to-year
standard.
 East Asia – clear learning outcomes and goals.
 Europe – the idea of teachers’ judgment of student
learning standards pervades.
 UK – standards (and accountability) of examinations
important (both between subjects and over time).
Curriculum curriculum
aligned examination standards
based exam standards

 Content standard Presentation title

over 2 lines
• What has to be learned (i.e. the curriculum)?
Space
- does not include concept of demand or difficulty for subtitle 20pt
 Demand standard
Presenter’s Name
• How demanding are the tasks (in termsJob
of the
Titleskills and knowledge
that are required)? Date
- does not include concept of the level of performance required

 Awarding standard
• The quality of performance required (e.g. for a grade/pass)
 Attainment standard
• How prepared for further study/employment?
- politicians say that they want to ‘raise standards’
I interpret the phrase ‘educational standards’, in the widest sense, to
mean the quality of educational provision, and I take that interest in
monitoring of such standards is motivated by a desire for valid information
upon which to base policies intended to improve that quality.

Despite being concerned only with some of the objectives of education,

public examinations play a major role in defining and monitoring
standards because their result are often used as output measures for
accountability.

Public examination standards therefore underpin much of the debate

about educational standards generally and, indeed, are themselves the
focus of controversy.
Mike Cresswell
(The British Academy, 2000)
Effecting educational change

 There were means of capturing and disseminating good practice.

 Textbooks were used as mediating instruments/steering elements.
 Construct precision, balance and secure age-related sequencing.
 Learning model – e.g. with a focus on ‘fewer things in greater depth’.
 Also, strategies were in place to ensure things were taught properly the first time
around.
 Assessment ‘wash-back’ was used effectively to drive educational change.
 An assessment framework was developed – with a sound construct, clear
purposes for each assessment and good alignment and optimisation.
 It was clear how assessment results would be reported, and used.
 Formal assessment methodologies were effectively used in teaching practice.
 Assessment complemented pedagogy and aligned with the curriculum.
Assessment as a positive force for
change

Assessment aligned to curriculum

 Assessment grounded on key principles – valid, reliable, fair
When considering optimal design characteristics for future
assessment systems it is necessary to bear in mind the
underlying purposes of those systems. The fact that a system
which is fit for one purpose will not necessarily be fit for all
purposes is a fundamental consideration when evaluating the
legitimacy of proposals. It is one of the most important
messages for policy-makers to understand.

Paul E. Newton (2007) Clarifying the purposes of educational assessment,

Assessment in Education: Principles, Policy & Practice, 14:2, 149-170
http://dx.doi.org/10.1080/09695940701478321
The intended purpose of assessment
What is assessment?

• Assessment is an essential tool in supporting and evaluating learning and

needs to be an integral part of the learning and development process.
• The insight gained from monitoring learners’ progress through classroom
assessments enables learners, teachers and parents to monitor and
recognise progress, while allowing teachers to plan teaching appropriately to
meet learners’ needs.
• Assessment models can range from low stakes, classroom-based tests or
question-and-answer sessions to high stakes, certificated examinations. Each
of these models has a different purpose and to be effective the assessment
design should match their intended purpose.
• Assessments should be linked to each other and have a clear relation to the
next step in a learning pathway in a subject. They should not be seen as
stand-alone experiences.
• The information about learning that is gained through an assessment should
have a direct impact on what happens next in the teaching and learning
process.
Discussion – purposes of assessment

Consider a set of assessments you are familiar with.

List the purposes of these assessments

Coherence of assessments

Secure
summative
assessments

Peer / self Teacher

assessments assessments

School-based
Homework
assessments

Class tests
What is an assessment framework?

• Frameworks are tools that provide a strategic overview to

underpin developments and help focus thinking and evaluation.
• Assessment frameworks describes all the different types of
assessment that learners will experience as they move through a
stage of learning in a subject.
• It is more than a list of tests and examinations. It presents clear
descriptions of the assessment approaches which apply
throughout the teaching and examination of a subject.
• The framework states clearly the aims and purposes of each type
of assessment and shows how the different assessments link
together and interact with each other.
• An assessment framework also shows how the results of
assessments and the data generated by them will be used to
support the next stage of learning.
Purpose of an assessment framework?

• An assessment framework is a key tool in communicating policy and practice to all stakeholders in
an education system, and can help ensure public confidence in the system.
• Producing an assessment framework is a consultative and iterative process, ideally involving
stakeholders at all stages.
• The assessment framework contains a clear statement of purpose for the assessment. This
describes what the tests and examinations are meant to achieve. Validity in any assessment
system depends on declaring a clear purpose, developing materials and processes that serve this
purpose, and ensuring, as far as possible, that users use it for the intended purpose.
• The assessment framework clarifies the principles and theory behind different approaches towards
effective assessment in each subject. Different subjects require different approaches to
assessment and this needs to be considered carefully when developing a framework.
• Assessment frameworks directly support the adoption and development of practical approaches in
schools. They set out expectations within a level in a subject and help schools to plan and deliver
learning programmes accordingly, and are useful tools in managing a period of curriculum change
or reform.
• Developing an assessment framework helps to identify where transitions within and between
subject programmes are well supported as well as showing where greater cohesion is needed.
• Assessment frameworks can be used to inform future development of programmes or qualifications
ensuring that they have a place in the wider provision for the subject, and that assessment has a
clear role and purpose in the learner’s progress through the subject.
• An assessment framework may also help inform innovations in assessment, such as the
development of digital or computer-based assessments.
What makes an effective assessment framework?

• An effective framework sets out the aims and purposes of

assessment, with the theory and principles behind the approach
clearly articulated. This facilitates the creation of fit-for-purpose
assessments in a subject.
• The framework should present a variety of assessment types,
ranging from periodic classroom tests, to end-of-stage tests or
examinations.
• An effective framework sets out clear expectations of learners and
aligns with curriculum goals, pedagogy, educational standards and
assessment outcomes. It supports teachers’ understanding of
effective assessment practice and promotes the sharing of
standards and expectations, leading to more consistent and
reliable assessment.
Discussion – features of assessment

Formal externally
Classroom tests End of year tests graded tests (e.g.
IGCSE)

Declared purpose

What they are

actually used for

How the tests are

administered

Frequency and
timing

Assessment
outcome

Learning outcome

Threats to validity
Example assessment landscape
A Cambridge approach to improving
education
 We use our considerable transnational expertise studying high
performing jurisdictions and experience in carrying out national
educational reforms.
 We do not offer readymade solutions – rather we offer observations
from international comparisons, and insights into how best to learn from
these.
 We do not borrow or ‘cherry-pick’ polices or initiatives – national
settings, strengths, drivers and challenges and cultural history and
development are all different.
 We first work first to understand local structures, forms and context, and
control factors and their interrelationships before suggesting/developing
solutions.
 We encourage ‘policy learning' – in context, addressing local challenges
and priorities (which need to be shaped and conditioned) and judgement
is required.
A Cambridge approach to improving
education
 We always work collaboratively and believe in skills transfer and close
working with partners. This is crucial to the way we work and to achieve
effective change.
 We understand the challenge of large-scale educational reform and
change – reform needs clear drivers and also needs to think about
appropriate incentives.
 System capacity for change has to be considered; change has to be
managed, change absorbs resource and requires resource.
 Preservation of curriculum coherence requires judgment, understanding
of local conditions and good governance (including monitoring and
evaluation).
Underpinned by theory, methodological
sources and antecedents

 Archer – change in educational systems

 Green – origins of the form of educational systems
 Raffe – country comparative theory
 Bhaskar – understanding social systems – critical realism
 Philips and Ochs – educational policy borrowing
 Alexander – deep understanding of national systems and role of culture
 Morris – motivations and practices in international comparisons
 Stigler and Stevenson – comparisons of teaching models
 Schmidt and Prawat – curriculum control and curriculum coherence
 Oates and Coles – function of qualifications and assessment
 Sahlgren and Oates – critique of previous work on Finland
 OECD, TIMSS, PIRLS – large transnational surveys
 Reynolds and Farrell – causal analysis of different systems
Thank you
Learnquestions?
Any more!
Getting in touch with Cambridge is easy

Email info@cambridgeinternational.org
or telephone +44 1223 553554
Panel discussion: Use of data in
improving outcomes in policy
The panel discussion will explore different perspectives on
the role and use of data in improving education outcomes.

Panel includes:
 Tim Oates CBE Group Director of Assessment
Research and Development, Cambridge Assessment
 Mounira Jamjoom CEO, Emkan Education
 Dr Raquel Warner Acting Director of Academic Affairs,
Mohammed Bin Rashid School of Government
Using Data to inform Decision
making in Dubai Schools
Dr. Racquel Warner
Mohammed Bin Rashid School of Government
 •PG page 23

Data-Driven School Leadership

• Standardizing expectations

• Standardizing tools and protocols

• Facilitating fast and accurate communication
• Enabling evaluation and accountability
Data Analytics in Education Sector

Analytics in education sector can be divided into two broader

categories:
1. Learning analytics (LA) -the measurement and analysis
of collections of data about learners and their contexts for
making learning more and more effective
2. Academic analytic- is concerned with the improvement
of resources, processes and workflows of the academic
institution through the use of learner, academic, and
institutional data (Anirban, 2014)
Develop a Data Team
 •PG page 27

Purposes and Functions

of the Data Team

To access and monitor:

• student success
• quality of instruction
• effectiveness of the team

Objective: to improve instructional practice and

performance
Data Strategies

Identify
Analyze Set Analyze
student Test Quantify
forces of learning school
learning hypotheses perceptions
change goals practices
needs
Strategies for Data-Driven Instruction

• Ask for and post public displays of data in schools.

• Ask data-driven questions of leadership teams.
• Ask students to quantify their learning progress.
• Ask teachers to cite data indicating how they gauge
student learning.
• Use charts and graphs to reduce data to useful
information by focusing on simple numbers, gap
comparisons, and trends.
• Ask frequent questions to ensure data accuracy.
Less Often
Summative Curricula
Assessments

Demographics & Perceptions

Benchmark Assessments
(MAP & CAT tests)

Formative Grade level

assessment

Formative classroom
assessments

More Often
The production and use of assessment in ‘outstanding’
schools has much in common with feedback loops (Burton,
2017)
Value of using data in schools

• Customize learning environment

• Personalized learning
• Targeted staff development
• Improve progression
• Reduce drop out
• Long term planning
References

• Anirban, S. (2014), Big Data Analytics in the Education

Sector: Needs, Opportunities and Challenges, International
Journal of Research in Computer and Communication
Technology, Vol 3, Issue11
• Burton, G. (2017), Improving Assessment in Dubai’s
Education System, Mohammed Bin Rashid School of
Government.
Understanding The Potential of School data
December 2017
 Agenda

1 State of Play- Data Driven Education System

2 Designing a Value- Add Schools Data Base and Extracting Value

3 Embedding a Data Driven Education Culture

Copyright © 2017, Emkan Education. All rights reserved 46

 State of Play

Copyright © 2017, Emkan Education. All rights reserved 47

 KSA allocated 25% of its budget to education

Total budget allocated to MoE in 2016

(SAR)

620 Bn

~125 Bn
825 Bn ~205 Bn (~60%)
(25% of
total KSA
budget)
~80 Bn
(~40%)

Total KSA Total Education

Other budgets K-12 budget Other budget1
budget budget

Copyright © 2017, Emkan Education. All rights reserved 48

 Furthermore, KSA’s education performance continues to decline
KSA TIMSS (math & science) scores declined by ~8%, and it did not improve in PIRLS
(reading)

KSA TIMMS Math KSA TIMMS Science KSA PIRLS

scores in grades 4 & 8 scores in grades 4 & 8 Reading scores
-8% -8% 0%

41 43
39 42 6 39 43 43
0 38 39 0 0
4 36 9 6
3 0
8

Gr 4 Gr 8 Gr 4 Gr 8 Gr 4 Gr 8 Gr 4 Gr 8 Gr 4 Gr 4
2011 2015 2011 2015 2011 2016

Copyright © 2017, Emkan Education. All rights reserved 49

…Although opex per student increased by ~13% in 2016
compared to 2011

KSA TIMMS Math KSA TIMMS Science KSA PIRLS

scores in grades 4 & 8 scores in grades 4 & 8 Reading scores
-8% -8% 0%

41 43
39 42 6 39 43 43
0 38 39 0 0
4 36 9 6
3 0
8

Gr 4 Gr 8 Gr 4 Gr 8 Gr 4 Gr 8 Gr 4 Gr 8 Gr 4 Gr 4
2011 2015 2011 2015 2011 2016
Opex per
# public students ~22K ~26K ~19K ~26K ~22K ~25K
(SAR)
+18% +18% +13%

Source: TIMSS Grade 4 & grade 8 Achievement (2011) and (2015), PIRLS 2011 & 2016

Copyright © 2017, Emkan Education. All rights reserved 50

 Breaking Down Data Barriers

Copyright © 2017, Emkan Education. All rights reserved 51

 Why

1 Monitoring and quality assurance of education provision

2 System responsiveness to needs

3 Targeted policy and program requirements

4 Transparency and accountability of school systems

5 Parent and citizen engagement

Copyright © 2017, Emkan Education. All rights reserved 52

 Designing a Value Add School Data Base

Copyright © 2017, Emkan Education. All rights reserved 53

 Madarisna is an online resource provides information about
schools targeting parents and schools

Schools Parents

• School Directory: Publishes detailed information of Saudi schools

• Schools Rating: Allows parent to rate & review schools
Main
Features
• School Portal: Enables school to update latest news/events to their page
• Madarisna Website: Helps schools to search and claim their pages

Beta Results + 20.7K + 35K + 3.6K

(Today) Schools Downloads Ratings

Source: Madarisna, Emkan Education

Copyright © 2017, Emkan Education. All rights reserved 54

 Data Structure Framework

School profile data

School evaluation data

Parents behavior data

(in searching for schools)

Students academic achievement data

(The average of student Asessments )

Copyright © 2017, Emkan Education. All rights reserved 55

 We published a report that identifies critical actions for our
education partners

Copyright © 2017, Emkan Education. All rights reserved 56

 Targeted Policy Making

Copyright © 2017, Emkan Education. All rights reserved 57

 Dashboard of schools ratings

Copyright © 2017, Emkan Education. All rights reserved 58

Copyright © 2017, Emkan Education. All rights reserved 59
 Embedding a Data Driven Culture

Copyright © 2017, Emkan Education. All rights reserved 60

 Policy Considerations

1 Establishing a National Education Data Set Initiative (Completeness and Accuracy)

2 Collecting deeper data with a longitudinal perspective

3 Opening access to data

Alignment and enhancements of complementary data sets ( classroom, school,

4
System)

5 Enhancing parent engagement and feedback to schools

Copyright © 2017, Emkan Education. All rights reserved 61

Use of data in
improving outcomes in
policy

Tim Oates,
Group Director, Assessment Development and Research

7 December 2017
Efficiency
http://www.gemseducation.com/organisation/education-solutions/impacting-education/the-efficiency-index/
http://www.gemseducation.com/organisation/education-solutions/impacting-education/the-efficiency-index/
Performance Ratios – school level

Contact Time
Outputs
Funding
Efficiency and quality
Data – what for?

Definition of ‘quality’
Goals and ‘public good’
Curriculum aims
Attainment, equity and engagement
Efficiency

Practical and pragmatic issues – Funding, Transport,

Facilities, Nutrition

Metrics
Target – complex criteria, thresholds

Contextual data
Outcomes data – domestic, international surveys
Two terms much misunderstood

Autonomy

Accountability
 High expectations

Low support High support

Low expectations
Which data – who for – and how is it analysed
and used?

External review
State

Intrinsic evidence Extrinsic evidence

Behaviour Test data
Pupil work Progression data

Internal review
Locality

Lesson study/observation
School improvement
Thank you
Learn more!
Getting in touch with Cambridge is easy

Email info@cambridgeinternational.org
or telephone +44 1223 553554
Embedding change in
education systems

Tristian Stobie, Director

Abigail Barnett, Deputy Director
Approaches to embedding change

Content and structure of this session:

 Understanding your system – its complexity and resilience
 Stakeholder management
 The role of the teacher, school leader and their professional
development
 The role of curriculum and teaching documentation
 Discussion, activity and feedback
Change and complex systems

Accountability
Allied social
Selection
measures

Professional Assessment
development Inspection

Institutional Funding
structures
Pedagogy Curriculum

Governance
National
Information Institutional Framework
and guidance development
Control and policy instruments

 Studies have shown

that control is necessary
to promote coherence Leadership

 Control need not come

from top-down
measures Organizational
learning

 Elements of the system

should interact in ways Organizational
performance and
that reinforce each innovation
other
Discussion

 Looking at the control factors cards, what are the priorities for
your context? Explain why you think they are priorities.
 How might changes to your priority area(s) impact other
areas of the system?

 Take 5 mins to discuss and then be prepared to feed back.

Effective stakeholder management

 Stakeholder management is the systematic identification,

analysis, planning and implementation of actions designed
to engage with stakeholders.
 It recognises the concept of stakeholders and stakeholder
salience, or the importance of various stakeholder groups
to a specific undertaking.
 It encompasses the process of forming, monitoring and
maintaining constructive relationships with stakeholders by
influencing their expectations.
 It is a way to sure that stakeholder engagement is efficient.
List of possible stakeholders

 Teachers  Employers
 Students  Media
 School Leaders  Exam boards
 Parents  Curriculum support bodies
 Ministry of Education  Publishers
 Ministry of Finance  Teacher training organisations
 Prime Minister  Inspectorate
 Universities
Key stakeholders: teachers
 What did I choose to do?
 How did I plan?
How can we encourage and  What activities did I use?

support teachers to change?

Concrete
experience

Active  What went well?

 What new ideas experimentation Reflection
should I try next?  What activities were
engaging?
 What feedback did I get
(both verbal and non-
verbal)?

Abstract
conceptualisation

 How can I explain what happened?

 Why did a particular activity work?
 Why was a particular activity less
successful?

Figure 2: Kolb’s cycle of experiential learning. Based on Kolb (1984)

Key stakeholders: school leaders

Effective school leaders:

 Build a consensus around, and clearly communicate, the
school vision and support this with a clear strategy.
 Use data wisely to evaluate student achievement, teaching
quality, impact of changes.
 Invest in good continuous professional development for
teachers and leaders, creating a culture of learning:
The leadership dimension that is most strongly associated with positive student
outcomes is that of promoting and participating in teacher learning and development.
[Viviane Robinson]

Effective schools can be defined as those that successfully progress the learning and
development of all of their students, regardless of intake characteristics, beyond the
normal development curve.
Discussion

 Refer to the control factors cards from the first activity and
the list of stakeholders from the second activity. Use both to
discuss what else and who else you will need to consider to
support your teachers and school leaders in making changes
and improving outcomes for your learners.

 Take 5 mins and be prepared to feed back.

What is important and influential in a
curriculum?

Number Algebra Geometry Measure Handling Data

 Values and aims
 Domains, constructs and
concepts
 Spiral curriculum
 Inspiring and relevant content Time

 Clarity around standards

In summary

 Use understanding of your context and expertise in system

analysis to plan and carefully manage change
 Plan communication and management of stakeholders
 Support teachers and school leaders to implement change
successfully
 Make targeted curriculum changes based on evidence and
your contextual requirements
 Plan new curriculum implementation according to the scale
of change being introduced
Thank you
Learnquestions?
Any more!
Getting in touch with Cambridge is easy

Email info@cambridgeinternational.org
or telephone +44 1223 553554
Working in partnership
The Bahrain National Examinations Project

Wafa Al-Yaqoobi,
Director, Directorate of National Examinations,
Education and Training Quality Authority, Bahrain

Karen Kester
Deputy Director, Education Services
Partnership journey

Background

The challenge

The approach

The benefit of phasing

Collaboration and capacity development

Lessons learned

Outcomes
Background

 Bahrain Vision 2030

“We aspire to shift from an

economy built on oil wealth to a
productive, globally competitive
economy, ….
Our society and government will
embrace the principles of
sustainability, competitiveness and
fairness to ensure that every Bahraini
has the means to live a secure and
fulfilling life and reach their full
potential.”
Education and Training Quality Authority
(BQA)
Board of Directors
Chief Executive
Directorate of Human
Directorate of
Resources & Financial
Communications
Affairs

General Directorate of National General Directorate of Educational

Qualifications Framework & Training Institutes Reviews

Directorate of Directorate of
Directorate of Directorate of Directorate of Private Schools Government
Academic Framework National & Kindergartens Schools
Cooperation Operations Examinations Reviews Reviews

Directorate of
Directorate of
Higher
Vocational
Education
Reviews
Reviews
Purposes of national assessment

Primary purpose:
 to drive improvements in the education system
by providing rich information about performance

Secondary purposes:
 to provide a beacon of good assessment practice
 to provide robust information for the monitoring of standards over time
and for research into value-added
Scope of the project

 to set up the Directorate

of National Examinations

 to develop and deliver

assessments at Grades
3, 6, 9 and 12

 to provide associated
capacity development
National Exams

Grade 6 Grade 9 Grade 12

Grade 3
Arabic Arabic Arabic Arabic
English English English English
Math Math Math Problem-
Science Science solving
The challenge

 In 2007 the Director

Directorate of
National
Examinations did
not exist Arabic and Science and Exam
English Maths Operations

 In 2008 we ran a
pilot of the Grade
3 and Grade 6
assessments
The challenge

 Trial implemented with a

limited number of DNE staff
which limited the experience
gained with knock-on effects
 Initial translations of the
Bahrain national curriculum
created some confusion
 Wider stakeholder group
needed support to understand
the new assessments
 Original capacity development
timescales were unrealistic
The approach

 Implementation and review cycle

 Open approach to issues

 Cambridge representative working in the DNE office

 Regular planning conference calls

 Face to face review meetings

 Joint project plans and risk register

 Recruitment support
The benefit of phasing

Grade 2008 2009 2010 2011 2012 2013 2014 2015

3 Pilot Cycle 1 Cycle 2 Cycle 3

6 Pilot Cycle 1 Cycle 2 Cycle 3

9 Cycle 1 Cycle 2 Cycle 3

12 Pilot Cycle 1 Cycle 2 Cycle 3

The benefit of phasing

 Series report each year

 Able to learn lessons and feed into
future planning
Cycle 1
 Spread of risk
 Possible to manage whilst team
Cycle 2 was growing
 Appropriate pace for schools and
teachers
Cycle 3  Supported embedding of capacity
development
 Willingness to learn and adjust
Collaboration and capacity development

Cambridge Joint DNE

led working led

 Assessments are complex

 Working together over several cycles reinforced learning

 Mix of formal theory training and on-the-job work

 Flexibility to run aspects at different paces

Lessons learned

Time taken to embed capacity development

Open communication

Willingness to learn and adjust

Form an effective implementation plan

Clear vision and scope

Outcomes

Local
Establishment
Capacity Stakeholders Best Practice
of the DNE
Building
Standard exams
Recruitment and
Training of item Spread a culture based on
training of DNE
writers of assessment international best
Staff
and national practice
exams
Established
Established Training of model for building
procedures markers national
examinations

Result reporting
Establishing an to stakeholders Issue log and
Logistics of
assessment continuous
national exams
system improvement
Thank you
Learn more!
Questions?
Getting in touch with Cambridge is easy

Email info@cambridgeinternational.org
or telephone +44 1223 553554
A Cambridge Approach
to Improving Education

Tim Oates
December 2017
Our approach

Our analysis avoids:

 Vacuous ‘parallel description’

 Naïve ‘policy borrowing’
 A sense of ‘inevitable trajectory’
 Empty guarantees
The current scene

 High levels of policy traffic

 Countries abandoning things just as other countries adopt

them

 Constructed anxiety – to justify domestic action

 Observation bias – the ease with which deeper structures

and causation can be missed
 Performance is more than ‘the shape’ of single aspects of
systems
…what’s the one thing you would change?
…what should be done about this?
 The temptation to use:
- Curriculum standards
- Qualifications
 Single, central instruments:
- Accountability
 Education policy is complex in its effects, with complex fine
tuning required to ensure desired outcomes
The history of ‘the autonomy argument’

 2000s OECD Autonomy allied to high performance

 2008 OECD Autonomy and a means of collaborating
 2011 OECD Autonomy and a means of sharing ‘best
practice’
 2015 CMRE defined autonomy plus collaboration with sound
ideas underpinning practice

Defining and recognising autonomy in different national settings

Defining high quality practice and sound educational theory
More on ‘autonomy’

Finland
Full system reform – pedagogic and curriculum content
First phase
From 1968, fundamental reform based on fully comprehensive model, highly
centralised, heavy State involvement. Revision of teacher training, grade tests,
State-approved textbooks, heavy school inspection
Second phase
Strategic move to higher institutional autonomy, office for textbook approval
closed in early 90s, inspection eased, data submission on school performance
continued – phase culminated in superlative performance in PISA 2000
Third phase
Decay in attainment, large programme of school closure, urban choice issues,
introduction of project-based cross-curriculum learning (20pc)
Throughout, to date, the final examinations (Abitur) fundamentally unchanged
Finland – avoiding superficial
understanding

From Real Finnish Lessons Heller Sahlgren 2015

Structural reform – the US case

USA
Full system reform – focus on attainment and equity
Evaluation analysis by Paul Peterson

1. No Child Left Behind – funding based

2. Federal Standards Initiative – standards based
3. School competition – structural focus

Variation in Charter school performance

Nations of interest
Sustained period of development and improvement
Data and evidence on the from of that system during improvement
PISA and YIMSS data
Limitation of translations – Russian Maths

Included Alberta
Mass USA
Hong Kong
Finland
Singapore

Of interest Japan
South Korea
Flemish Belgium
ED Hirsch
Sweden
Scotland
Australia
NZ
New curriculum in Singapore
Control factors

Explanatory factors
Control factors

1. curriculum content
2. pedagogy
3. assessment and qualifications
4. institutional development
5. institutional forms and structures
6. governance
7. professional development
8. accountability
9. inspection
10. funding
11. national framework
12. selection and gatekeeping
13. information and guidance about routes and choices
14. allied social measures
Control factors – unpacked

For the full set of ‘control factors’ see page 20 of the report
Explanatory factors
Alignment and coherence

‘Curriculum Control’

Relations and interactions

Explanatory factors: size

Size

Singapore 4,839,400 (world bank 2010)

Finland 5,313,399 (world bank)
Hong Kong 6,977,770 (world bank)
Mass 6,593,587 (uscensus)
Alberta 2,974,807 (finance and enterprise Alberta)
England 51,460,000 (office for national statistics)
South Korea 48,747,000 (world bank)
The ‘cycle of planned failure’
Breaking the cycle
Cambridge Approach

 Developed over a decade of analysis and action

 It has been used in a number of settings

 Case studies provide an interesting application:

Finland, Massachusetts, Singapore, England
Cambridge Approach

 It supports
• existing policy and action as well as development of new
initiatives and strategy
• small scale action as well as ‘grand’ reform

 It emphasises:
• complexity
• resilience
• culture as an object of policy
• the importance of ideas
Cambridge Approach

 Our approach is distinctive:

• Analysis should precede action; solutions cannot fall
‘ready-made’ onto systems
• It does not supply judgement-free solutions
• It is sensitive to national aims and context; it supports
detailed and sensitive analysis of specific national settings
• It is designed to lend structure and support to such
analysis
“…it is designed to support sound analysis of context
and circumstance, and to enable evidence-based policy
formation for educational improvement…”
Thank you
Learn more!
Getting in touch with Cambridge is easy

Email info@cambridgeinternational.org
or telephone +44 1223 553554
Question and Answer Session

Charles Clarke
Former British Secretary of State for
Education and Skills