This article was downloaded by: [University of Windsor]

On: 16 November 2014, At: 12:27

Publisher: Routledge
Informa Ltd Registered in England and Wales Registered Number: 1072954 Registered
office: Mortimer House, 37-41 Mortimer Street, London W1T 3JH, UK

The Educational Forum

Publication details, including instructions for authors and
subscription information:
http://www.tandfonline.com/loi/utef20

The Growth of the International

Baccalaureate Diploma Program:
Concerns About the Consistency and
Reliability of the Assessments
Tristan Bunnell

Copenhagen International School , Hellerup, Denmark

Published online: 16 Mar 2011.

To cite this article: Tristan Bunnell (2011) The Growth of the International Baccalaureate Diploma
Program: Concerns About the Consistency and Reliability of the Assessments, The Educational Forum,
75:2, 174-187, DOI: 10.1080/00131725.2011.552700
To link to this article: http://dx.doi.org/10.1080/00131725.2011.552700

PLEASE SCROLL DOWN FOR ARTICLE

Taylor & Francis makes every effort to ensure the accuracy of all the information (the
Content) contained in the publications on our platform. However, Taylor & Francis,
our agents, and our licensors make no representations or warranties whatsoever as to
the accuracy, completeness, or suitability for any purpose of the Content. Any opinions
and views expressed in this publication are the opinions and views of the authors,
and are not the views of or endorsed by Taylor & Francis. The accuracy of the Content
should not be relied upon and should be independently verified with primary sources
of information. Taylor and Francis shall not be liable for any losses, actions, claims,
proceedings, demands, costs, expenses, damages, and other liabilities whatsoever or
howsoever caused arising directly or indirectly in connection with, in relation to or arising
out of the use of the Content.
This article may be used for research, teaching, and private study purposes. Any
substantial or systematic reproduction, redistribution, reselling, loan, sub-licensing,
systematic supply, or distribution in any form to anyone is expressly forbidden. Terms &
Conditions of access and use can be found at http://www.tandfonline.com/page/termsand-conditions

Downloaded by [University of Windsor] at 12:27 16 November 2014

The Educational Forum, 75: 174187, 2011

Copyright Kappa Delta Pi
ISSN: 0013-1725 print/1938-8098 online
DOI: 10.1080/00131725.2011.552700

The Growth of
the International
Baccalaureate Diploma
Program: Concerns
About the Consistency
and Reliability of the
Assessments
Tristan Bunnell
Copenhagen International School, Hellerup, Denmark

Abstract
The International Baccalaureate (IB) world, known as the IB World, is
doubling in size every five years. The IB has become a complex educational
product, but offers high levels of consistency and reliability in terms of delivery
and assessment. However, since late 2008, a number of concerns have been
raised about the quality and manageability of the Diploma Program.
A major challenge for the future will be attracting new markets through new
initiatives, yet maintaining and managing a consistent and reliable assessment.
Key words: comparative education, curriculum and instruction, international education.
The Geneva-registered International Baccalaureate (IB) examination system, with
its three programs forming a continuum of international education, has emerged
over the past four decades as the worlds major form of international curriculum.
The initial Diploma Program (DP) first emerged within a social studies arena at the
International School of Geneva in 1962 and was joined in 1967 by Kurt Hahns Atlantic
College in South Wales. Fittingly, the IB assessment center is also in South Wales. The
DP has long been recognized as a challenging program, being once described as a
demanding core curriculum with rigorous examinations (Savage 1982, 602). It has
also long been seen as a consistent form of assessment; Renaud (1975) described it as
an innovative program, designed in laboratory conditions and aimed at producing standardization worldwide.
Address correspondence to Tristan Bunnell, Copenhagen International School,
Hellerupvej 22-26, Hellerup 2900, Denmark. E-mail: tbunnelluk@yahoo.co.uk

Consistency and Reliability of the International Baccalaureate Assessments

The DP was joined by the lesser-known Middle Years Program (MYP) in 1994 (Conn
2001). The Primary Years Program (PYP) appeared in 1997. In July 2010, there was a diverse
conglomeration of 3,000 schools offering a combination of 3,700 programs to 840,000 youth
in 139 countries. Because only 150 schools were offering the continuum as such, one can
readily see the extent to which the programs exist in practice as stand-alone ones.

Downloaded by [University of Windsor] at 12:27 16 November 2014

A Complex Educational System

As can be seen, the IB is actually a rather complex educational system. It is unique
among curricula in having at its heart a radical mission statement to create a better and
more peaceful world through intercultural understanding and respect. This system
provides a multidimensional set of areas for assessment. The IB is more than a creator of
curriculum and examiner of programs; it has an idealistic mission to create a well-informed
global citizen, as well as to provide the building blocks for intercultural competency
(Taylor 1994, 154). The IB is also unique in having two exam sessions. The May DP exam
session involves mainly schools in the Northern Hemisphere, whereas the much smaller
November session involves mainly Southern Hemisphere schools.
The IB has grown enormously over the past decade. Financially, the IB had an operating income in 2009 of $101.6 million, which was 12 percent higher than the year
(International Baccalaureate Organization [IBO] 2009b). The growth in infrastructure also
has been significant: The IB employed 382 full-time personnel in 2007 and 513 in 2009
(IBO 2009a, 18). The number of candidates examined for the DP increased from 1,217 in
1975 to 96,776 in 2009 (IBO 2009a, 19). At the same time, the average number of candidates
per school increased from 40.6 in 1975 to 61.2 in 2009 (IBO 2009a, 19). The number of DP
examiners and moderators has also grown: In September 2010, there were 10,500 people
involved (IBO 2010) as compared to 4,000 people six years prior (IBO 2004a). Accordingly,
the IB requires an extra 1,000 examiners and moderators each year.
A major challenge of such growth is maintaining a perceived high level of quality
assurance. There is a growing unease among insiders within the international school
movement regarding growth. Since late 2008, a complex mixture of issues, especially
concerning assessment, has emerged. A growth scepticism movement has arisen, openly
questioning the extent to which the IB can continue to grow without compromising its
qualityparticularly the consistency and reliability of assessment of the DP. As a result,
a moratorium on further growth has been suggested.
The Concerns of 2009
The IB authorized 438 programs in 2009 (Beard and Holloway 2010). Mott (2009, 19)
accused the IB of being intoxicated by quantitative growth. This issue, of a headlong
rush for growth of programs, surfaced at the meeting of 530 delegates to the World Conference for Heads of IB Schools (October 1316, 2009). The Academy of International School
Heads (AISH) held a special gathering in November 2009 to express a number of concerns,
including the overall integrity of the exam process (Betts 2009, 4), and a statement from
AISH was officially given to the IB. The concern with the exam process is a very important
issue, not only for the IB itself, but for the 3,000 schools that offer one or more of the three
programs; hence, the issue requires a wider audience than it has hitherto received. The
The Educational Forum Volume 75 2011 175

Bunnell
strategic planning journey undertaken by the IB from 1999 is worth reviewing because it
provides a historical context for understanding the concerns over assessment.

The Strategic Planning Journey

Downloaded by [University of Windsor] at 12:27 16 November 2014

The official IB Web site reveals the statistical growth (at http://www.ibo.org/facts/
schoolstats/). The IB hit the 100-school mark in 1982, the 25th anniversary of the first trial
examination. The 200 mark was hit in 1987, the 30th anniversary. By the end of the Cold
War, there were about 300 IB schools, and the period after saw rapid growth with the 500
mark hit in 1995. The IB reached the 1,000-school mark around 1999.
The IB instigated a strategic plan at that time which was mainly aimed at giving the
IB an online profile. In April 2004, a ten-year plan appeared. The number of IB schools
had, by this stage, reached 2,000; it was clear the IB had entered a process of compound
growth leading to a doubling in number of schools every five years. Planned growth
was placed at the heart of the new strategic plan (IBO 2004b), which projected that the IB
would be educating one million students by the year 2014. Another strategic document,
which appeared later, concentrated on access (IBO 2006a).
The rhetoric of the IB has changed substantially over the past decade. It has moved
away from mere survival and focus on growth per se toward a more directed and planned
strategy based on wider access and greater impact via a more sophisticated infrastructure and model of service delivery. In other words, the IB now has a huge set of growth
ambitions. The issue of wider access, for example, is six-dimensional (economic, cultural,
geographical, linguistic, political, and social). Thus, it is evident the IB is finding the task
of keeping ahead of growth a challenge.
The IB now has a goal of serving 2.5 million children in 10,000 schools by 2020. One
would expect at least 4,000 of these to be in the United States alone, whereas 63 percent
would be state-funded, and just 5 percent would be international schools (Beard and
Holloway 2010). The aim is, therefore, to expand by an additional 7,000 schools over the
coming decade. Within this context is the great challenge of providing wider and more
planned growth while guaranteeing to maintain high quality.

The Rapid Growth of the IB

With the IB now focused on wider access and greater impact, it is useful to view the
growth of the IB as a two-dimensional process, beyond the typical statistical analysis. On
the one hand, the IB has made enormous numerical inroads into national schooling, especially in Australia, Canada, and parts of the United States. This might crudely be termed
the level of access growthgrowth in terms of schools, candidates, and programs offered
as the IB has opened up avenues of access into current markets and new markets.
On the other hand, the IB has become an extremely complex product, especially since
2006. The three programs (DP, MYP, and PYP) combine to form a fourth programthe continuum. Also, the IB is keen to add online programs and create a vocational-type program.
This might crudely be termed the impact growthgrowth in diversity of the product as
the IB has innovated to make an impact in new markets. The main point is that the growth
176 The Educational Forum Volume 75 2011

Consistency and Reliability of the International Baccalaureate Assessments

Table 1. International Baccalaureate Growth Statistics
Variable
Number of schools offering the PYP
Number of schools offering the MYP

Downloaded by [University of Windsor] at 12:27 16 November 2014

Number of schools offering the DP

2005

2006

2007

2008

2009

247

315

422

490

612

336

374

445

510

749

1,346

1,487

1,646

1,856

2,065

Total number of programs

1,929

2,176

2,513

2,856

3,426

Number of exams taken (May)

215,31

245,030

273,019

303,125

339,010

Number of exams taken (November)

22,241

24,221

26,194

30,386

31,810

Number of exams (total)

237,554

269,251

299,213

333,511

370,820

Number of DP candidates (May)

27,971

32,126

35,903

39,914

45,130

Number of DP candidates (November)

3,735

4,186

4,483

5,477

5,823

Note. Source: International Baccalaureate Organization (2009a). PYP Primary Years Program; MYP Middle
Years Program; DP Diploma Program.

of the IB should not just be viewed as statistical phenomena, but should be viewed within
a wider infrastructure and diversity of product paradigm. Clearly, growing the IB is an
enormous task, while maintaining high levels of assessment consistency and reliability.
The Statistical Growth
The 3,000-school mark was reached in July 2010 when Brown Academy, Chattanooga
became the first PYP authorized school in the state of Tennessee. These schools were offering a combination of almost 3,500 programs in total (see Table 1; IBO 2009a). A wider
review of the past four decades reveals a larger, more complex picture. Focusing first on
the number of schools entered for the DP during the 35-year period between 1975 and
2009, there were 23 years that saw double-digit growth. The figures for the number of
candidates entered (again, during 19752009) are even more impressive. Only five years
during this period had less than double-digit growth (1983, 1984, 1992, 1993, and 1995). In
the AfricaEuropeMiddle East region, the number of schools following the DP rose from
388 in 2004 to 605 in 2009. In the Asia Pacific area, the number of schools nearly doubled
during the same periodfrom 133 schools to 247.
The Product Growth
Not only has the IB experienced statistical growth, but it also has grown enormously
as a product. Since 2006, in particular, there has been a whirlwind of technological and
pedagogical developments as the IB aims to consolidate its image and attract new markets.
In 2007, the IB began its Digital Space Initiative, intended to create an online, Web-based
virtual community. Furthermore, in April 2007, the IB launched its new visual identity with
a single logo and re-branded IB in place of the IBO moniker. Therefore, the IB became a
brand rather than an organization. The IB in 2007 also introduced its Learner Profilelisting
ten outcomes: risk-takers, balanced, inquirers, reflective, caring, open-minded, principled,
thinkers, communicators, and knowledgeable. This profile is intended to form a generic
IB vocabulary of humanistic behavior and to encourage internal discourse. In March 2008,
enrolment began for the first online courses, offered by the Virtual High School, as part
of the Diploma Program Online project. The IB in 2009, together with RM Education plc,

The Educational Forum Volume 75 2011 177

Bunnell
a leading provider of educational services based in Oxfordshire, England, launched an
electronic marking system called Scoris. In the November 2009 DP session, nearly one
fourth of scripts were marked using this electronic system (IBO 2009a, 5).

Downloaded by [University of Windsor] at 12:27 16 November 2014

Beginning in January 2010, the IB began the restructuring process to eventually operate out of three inter-competing global centers: The Hague, Singapore, and Maryland in
the United States. Future plans include a global alumni base within the IB Association.
In 2012, the IB intends to begin offering the IB Career-related Certificate. In addition to
coping with this rapid level of organizational, image, infrastructure, pedagogical, and
technological innovationall within a double-digit growth paradigmthe IB has to
maintain a high level of quality assurance. This is, in fact, an area of activity that the IB
has, at least until now, managed to control very effectively.

Perspectives on Reliability and Consistency

The Meaning of Quality
An entire edition of IB World magazine (Hayden 2005) was devoted to the topic of
quality in 2005. There, the Director General stated that quality should be judged in
terms of consistency and reliability (Walker 2005)a direction confirmed in 2007 when
a slogan titled Quality and Consistency, with Diversity appeared (IBO 2007). The IB
certainly prides itself on providing consistent and reliable assessment of its DP. Former
Director General George Walker (2007), in a speech at the Asia Pacific Annual Regional
Conference, boasted that the DP pass rate had consistently averaged 82.2 percent between
1998 and 2007, with a maximum spread around that figure of only 3.4 percent. Further
assessment of these standards follows.
Final and Predicted Grades. Many of the statistics for 2006 through 2009 reveal consistency. As shown in Table 2, the proportion of subject grades that exactly match the schools
predicted grades has been remarkably stable (averaging 43percent between 2006 and 2008).
The proportion of final subject grades that are within one grade of the school predictions
is equally impressive in terms of stability (averaging 90 percent between 2006 and 2008).
This data suggests that the criterion-based assessment model of the DP allows for schools
to accurately predict what their students should achievethat is, the grade awarded by
the external examiner almost exactly matches that predicted by the classroom teacher.
Mean and Maximum Grades. Other statistics also back up the remarkable consistency
and reliability of the IB (see the annual Statistical Bulletins published online at www.ibo.org/

Table 2. Consistency of Predicted and Final Subject Diploma Program

Grades
Proportion of Subject Grades
That Exactly Match the
Schools Predicted Grades
(%)

Proportion of Final Subject

Grades That Are Within
One Grade of the Schools
Predicted Grades (%)

May 2006

43.2

90.0

May 2007

42.9

89.7

May 2008

43.4

90.0

Year

178 The Educational Forum Volume 75 2011

Consistency and Reliability of the International Baccalaureate Assessments

Downloaded by [University of Windsor] at 12:27 16 November 2014

facts/statbulletin/). These have been summarized in Table 3. The number of DP students

who achieved the maximum 45 points (42 points in the six-subject section plus an extra
three points for the Extended Essay and Theory of Knowledge component) averaged
just 78 students between 2006 and 2009. The figure for May 2009 was 86 students out of
a total DP entry of 35,181. Interestingly, at least one half of these DP full-pointers can be
found in elite private schools in England, with schools such as Sevenoaks and Oakham
providing a considerable number.
In percentage terms, the number of full-point students has been very consistent, with
the years 2007, 2008, and 2009 producing almost the exact same figure (between 0.18 and
0.19 percent). This was despite a 40 percent increase in the total number of DP candidates during the 2006 through 2009 period. The proportion of DP candidates who pass
(i.e., obtain the necessary minimum of 24 points) has also remained very consistent,
with the maximum difference being just 2 percent (it was 80.34 percent in 2006 and 78.71
percent in 2009). The corresponding figures for the November examination sessions
during 2006 through 2009 are equally consistent.
The DP mean scores and mean grades worldwide further substantiate the level of
consistency (see Table 4). The DP mean score over five years deviated by just half of 1
percent (the highest was 30.09 in May 2005, and the lowest was 29.51 in May 2009). The
corresponding figures for the mean grade worldwide also were very close: the lowest at
4.66 in 2009 and the highest at 4.78 in May 2005. The MYP mean grade worldwide over five
years varied least of all. At the same time, the total number of MYP certificates awarded
increased from 4,821 in 2006 to 6,670 in 2009.

Table 3. The Consistency of Diploma Program (DP) Grading

Year

Percentage of
DP Candidates
Scoring 45 Points
(May)

Percentage of DP
Candidates Scoring 40 Points or
More (May)

Percentage of
DP Candidates
Scoring 24
Points or More
(May)

Percentage of DP
Candidates Scoring
24 Points or More
(November)

2006

0.28

1.76

80.34

81.40

2007

0.18

1.79

78.85

81.30

2008

0.19

1.90

79.17

79.47

2009

0.19

1.80

78.71

NA

Table 4. The Consistency of Mean Grades and Mean Scores

Year

Mean Subject Grade

(May DP Session)

Mean Score (May

DP Session)

Mean Subject Grade (MYP)

2006

4.74

29.89

4.73

2007

4.68

29.56

4.73

2008

4.69

29.57

4.79

2009

4.66

29.51

4.79

Note. DP Diploma Program; MYP Middle Years Program.

The Educational Forum Volume 75 2011 179

Downloaded by [University of Windsor] at 12:27 16 November 2014

Bunnell
World Average Subject Grades. However, a substantial caveat needs to be added here.
These average DP scores mask substantial differences among individual subject scores. Each
year a significant number of discrepancies between subjects occur, and this is one of the
major areas of assessment concerns being voiced by the growth sceptics. For example, in
May 2009, the proportion of candidates (out of 5,173) who obtained a grade of seven (the
maximum grade) in the Economics Standard Level was 15 percent, yet only one percent
of the 971 candidates who sat for the Social and Cultural Anthropology papers obtained
the highest possible grade. Of the 78 candidates who sat for the Russian B exam, 62
percent scored a grade of seven. The 4,029 Environmental Systems candidates scored a
mean grade of just 4.01 (exactly the same as for the Chemistry Standard Level), yet the
Economics Standard Level had a mean grade of 5.18 (out of 7.00). The implication of this
sort of discrepancy is that some subjects are easier than others, leadingin practiceto
students choosing certain subjects over others.
Comparison With Advanced Level
Overall, what these figures seem to prove is that the IB has not suffered from any
form of grade inflation over the past few years, despite the enormous growth in numbers
of schools and candidates. The level of marking and grading appears to have been consistent, providing a high degree of reliability and allowing teachers to accurately predict
results for their students. It is tempting to compare and contrast this situation with the
English General Certificate of Education Advanced Level system (A-Level), which has
undergone, with some justification, much criticism of grade inflation. In fact, one leading educator in England has openly stated in the popular press that the organisation
which runs the IB is free from the meddling hand of government and immune to grade
inflation (Seldon 2009, 11).
The figures for A-Level indeed are very different from the IB ones. The A-Level seems
to have become an exam that is impossible to fail. In 2009, a total of 310,000 students in
England, Wales, and Northern Ireland saw the pass rate of their A-Levels rise to
97.5 percent; this figure provided for an incredible 27 consecutive year-on-year rise in the
pass rate. This can be compared with the IB DP failure rate (scoring less than 24 points) of
21.29 percent in 2009. No less than 26.7 percent of A-Level results were at grade A, a rise of
0.8 percent over 2008, and the 12th consecutive year-on-year increase. At least 75 percent
of grades were at least a grade C. Again, this can be contrasted with the 1.80 percent of
students who obtained a perfect 45 points on the IB DP in 2009.
The May 2007 Issue
Despite the statistics provided here, both in the IB and in a comparison with A-Level,
the past couple of years have seen an open questioning of the consistency and reliability
of the IB DP. In April 2006, the IB undertook a School Satisfaction Survey. Interestingly,
96 percent of schools surveyed said they were very satisfied with the curriculum, yet only
85 percent were very satisfied with the assessment (IBO 2006b).
The next major sign of a problem came in the May 2007 examinations, when many
schools in northern Europe were sent the 2005 History paper by mistake (a fuller explanation has not seemingly appeared). It is worth commenting here that the DP has a very
180 The Educational Forum Volume 75 2011

Consistency and Reliability of the International Baccalaureate Assessments

Downloaded by [University of Windsor] at 12:27 16 November 2014

rigorous, yet arguably cumbersome, paper-setting process. The time taken for a paper
being commissioned and ultimately bagged and dispatched to schools is 86 weeks and
involves 17 stages in total, including translations from English into French and Spanish
(IBO 2004a, 29). The seemingly important issue of a previously set paper being reset did
not carry much attention at the time, though the British Times Educational Supplement (TES)
did comment on how 1,100 candidates in England had sat for the exam and had probably
seen the paper before (TES 2007).
The issue did not seem to do any long-lasting damage for the IB in terms of publicity, but it undeniably affected the confidence of examination centers that had hitherto
expected a reliable assessment process. It seems significant that in July 2008, the Director
General (by then Jeffrey Beard) sent out a two-page letter to all IB schools commenting
on how the enormous growth was causing much strain, but reassuring them that growth
was not a problem and that major infrastructure changes were about to be made quite
soon (e.g., creation of the three global centers). Interestingly, the ex-Director General had
stated, at the time of his appointment, I think the weaknesses have been associated with
rapid growth (Walker 1999, 8).
The letter in 2008 seemed to indicate that a degree of internal unrest existed at some
level or other, probably at head teacher level within the international school movement
because this was where the issue first came to the fore in the public domain. The highly
critical article (Toze 2008) that appeared in December 2008 in The International Educator
(TIE) proved to be the catalyst for three further attacks plus a couple of rebukes. Fuller
analysis of this vital issue is made next in this article; however, this development leads to
an assertion by this author that a growth scepticism movement has begun. Commentators,
and especially IB educators in international schools, are now openly questioning whether
the IB can continue growing at the current rate yet maintain the usual, and expected, standards of reliability and consistency in terms of assessment, service, and deliverythat is,
they are sceptical that the IB can maintain growth levels without compromising quality.

The Emergence of Growth Scepticism

There is a body of economics that questions the role of increasing economic growth
(measured as gross domestic product) as a model for economic development and the
overall happiness of society. A good example is The Limits to Growth (Meadows, Randers,
and Meadows 1972), a book that is not anti-growth as such, though such views do exist,
but questions whether the pursuit of economic growth is the best route to economic development. Such a viewbeing sceptical of growthmight question whether a high level
of growth is sustainable, or even desirable. Applying this notion to the IB is appropriate, yet
problematic. Growth scepticism in terms of the IB is obviously multidimensional. One
could, for instance, question the extent to which the IB can, or even wishes to, meet its
commitment to facilitate global peace through promoting intercultural understanding.
Thus, one can be sceptical of the idealistic mission of the IB.
Certain literature, expressing a sceptical view of the IB, is not anti-IB as such, but
critically questions the role and development of the IB. For instance, the role of the IB as a
force for positional competition has been suggested (Brown 2000). More recently, Lauder
The Educational Forum Volume 75 2011 181

Bunnell

Downloaded by [University of Windsor] at 12:27 16 November 2014

(2007) asserted that the initial idealism of the IB has been overtaken by social and class
interests associated with globalization. What also has been openly discussed (Doherty,
Mu, and Shield 2009) is how IB graduates in Australia are utilizing their capacity to cross
national borders (termed border artistry) and how the IB offers them a competitive edge
with which to strategically pursue economic and cultural capital.
The Toze (2008) Article
The emergence of another form of growth scepticismpublicly questioning the extent
to which the IB can keep growing yet maintain assessment quality assurancecan be traced
back to late 2008 when Concerns About IB Exam Results Undermining Confidence in
Program appeared in TIE (Toze 2008). In that article, which was a copy of an open letter
that was sent to members of AISH, David Toze, Superintendent of the International School
of Manila, began by saying, In recent years, we have become increasingly concerned
about the accuracy of IB grades (6), and went on to highlight two key issues.
First, Toze (2008, 6) questioned the standardization of marking of DP exams, stating,
the vast expansion of the IB has led to the appointment of some markers who are not
appropriately qualified or diligent; he also questioned the wide variations in average
grades between subjects and some stark anomalies in component results (6). He questioned the accuracy of the fact that, in May 2006, the world average for a Filipino was an
extraordinary 6.82 (out of a possible 7.00). Toze (6) stated, I am also sure that the drive
for expansion has created a number of other issues imbalanced demands on good IB
schools to provide trainers, markers, curriculum leaders and evaluation visitors. He then
went on to openly state that this increase in numbers has been achieved at the expense
of the integrity of the quality and consistency of the grading process (6).
Second, the Toze (2008, 6) letter highlighted a sentiment among international schools
that not only are they subsidising the expansion of the IB, through the imbalance just
mentioned, but their power is being withered away. He asserted that the IB was making assessment decisions without due discussion and feedback. In particular, he voiced concern
about the way the IB, in 2008, announced that from that point on, DP students who wish to
have a subject re-marked must face the chance that the grade can go down as well as up. The
obvious accusation being made here by Toze was that the IB was no longer able to deal with
the sheer volume of re-mark requests and so was adding a caveat to the decision to reduce the
demand. The outcome of this move by the IB is, of course, that schools will be more reluctant
to question a candidates grades and, thus, reduce the pressure on the assessment process.
The Matthews (2009a; 2009b; 2009c) Articles
The IB was quick to refute these claims. The IB communication manager (Al-Haddad
2009, 8) responded by pointing out that growth was nothing new to the IB and that the
organization remained strongly committed that the quality of our programmes and services do not suffer as a result of our growth. The message then stated that the overall
picture is one of stability (8). It was further stated that experienced DP schools in the
Asia Pacific region have even closer levels of agreement between teachers and examiners,
with 47.7 percent of predicted grades being matched in the May 2008 exam session and
92.5 percent being within one grade of prediction (Al-Haddad 2009).
182 The Educational Forum Volume 75 2011

Consistency and Reliability of the International Baccalaureate Assessments

Downloaded by [University of Windsor] at 12:27 16 November 2014

However, in the same edition of TIE, Michael Matthews (2009a, 28), head of the InterCommunity School in Zurich and Chair of the IB Heads Council, expressed his concerns
based on a close look at the inner workings of the IB. Matthews (2009a) noted that the
quality achieved by the IB is still high, yet the problem, in his mind, is that the IB has become a system; reliability is assessed from a statistical perspective, whereas schools have
individual concerns that are not aired. The general message from Matthews (2009a) was that
the schools have no coherent voice, and there is no broad dialogue about future growth.
The appearance of concerns by the two heads, Toze (2008) and Matthews (2009a),
led to a rebuke by an ex-head, now IB assistant examiner (Burke 2009, 8), who defended
the quality and integrity of the assessment side of the IB programmes and attacked the
public raising of concerns, calling it unwise. He concluded by remarking that as an
IBO examiner I contribute to the integrity of a curriculum that upholds an equitable, just,
transparent and fair system of assessment (Burke 2009, 8).
Six months later, Matthews (2009c) had an another article in TIE, citing six key myths
about the IB and giving evidence why there were real concerns. There, Matthews (2009c,
11) seemed to retract some of his earlier comments, and stated that the core activities of
the IBnamely, curriculum design and examination setting and markingare actually
done well by diligent and hardworking individuals.
In November 2009, there was a large gathering of heads in Seville, where the issue of
possible compromising in quality was discussed. The month after, Matthews (2009b, 19)
issued a third article in TIE, expressing his personal views post-Seville. This was useful
because it provided a framework for understanding the emerging concerns. Matthews
(2009b) presented the three key areas of concern that had arisen, from his perspective, in
Seville: quality assurance, organizational culture, and growth plus diversity. Within the
quality assurance chapter, he wrote, There is concern over whether the core activities of
the IBcurriculum development, assessment, and teacher trainingare as reliable, costeffective and responsive to the needs of schools and students as they should be (Matthews
2009b, 19). Interestingly, Matthews (2009b) claimed that these concerns also relate to the
MYP, and he asserted that they were concerns that were felt by other blocs of schools.
The Issue of Dualism
Basically, the IB is finding it difficult to satisfy the current customer base while innovating to attract new markets. In a business context, this is seen as the challenge of
dualism (Paap and Katz 2004). The IB, in this sense, is in danger of losing the support of
the current customer base. The international school movement, in particular, is liable to
lose patience with growth. Indeed, Matthews (2009b, 19) explicitly stated that the United
World Colleges are looking into the creation of a new non-IB diploma, and there are strong
voices among international schools calling for new beginnings at primary and secondary
level. This is an important development that will need further observation.

Conclusion
The IB has grown enormously in terms of numbers of schools and candidates for both
MYP and DP assessment. The number of countries involving an IB school globally has
The Educational Forum Volume 75 2011 183

Bunnell

Downloaded by [University of Windsor] at 12:27 16 November 2014

grown from 113 in 2005 to 139 in July 2010. The May 2009 DP exam session alone involved
52,500 youth in the United States; 9,100 in Canada; and another 4,000 in England (out of a
total of 96,800 candidates). These numbers are put into context when one considers that
the DP was limited in size until 1976 to a research-manageable 500 students each year.
Since 2004, the IB has attempted to forge further substantial growth (now aiming to be
catering for another 7,000 schools and 1.7 million children within the next decade), but
within a wider access and greater impact paradigm. This chosen direction has led to a
considerable amount of innovation and product development as the IB tries to break into
new markets (both geographically and socioeconomically).
This growth is problematic. By aiming to reach out for new markets, the IB has been
forced to innovate in new areas, such as online programs, and has embarked on a plethora of initiatives as it strives to maintain growth within a wider access paradigm. This
presents the IB with the difficult challenge of dualism (Paap and Katz 2004)fighting
on two fronts by ensuring efficiency for today while establishing new markets for the
future. In particular, by growing into new markets through innovation, the IB risks creating a product that is too expensive and complicated for many current constituents who
might seek a simpler and more pragmatic alternative. This is seen in a business context
as disruptive innovation (Paap and Katz 2004). Hence, there is talk of an alternative
non-IB diploma.
One of the major challenges facing the IB as it grows, apart from innovating and
maintaining a sense of community, is that of managing this growth (both of the numbers
and the product) without compromising quality. In terms of the IB, this means consistency
and reliability of assessment, as confirmed by Walker (2005). Major statistics related to
the DP May sessions (e.g., mean grade, mean score, proportion of candidates scoring
45 points, and proportion of candidates scoring 24 points) still show a remarkable degree
of consistency (certainly between 2005 and 2009) and stability, as argued by a leading examiner (Burke 2009). The figures relating to candidate scores and the schools predicted
scores also reveal a high degree of consistency and reliability. In other words, the IB DP
retains a strong reputation, certainly to the outside world, of being a quality-assured
product. The London-based group ACS International Schools conducted a survey among
51 university admissions officers in Britain and concluded that the IB is independent
of national boundaries and has a rigorous standard of assessment and grading which is
consistent over time (ACS 2008, 3).
Since late 2008, a new phenomenon has appeareda willingness by IB insiders (mainly
head teachers in international schools, such as Toze and Matthews) to openly voice concern about the growth and their perception of quality being compromised. There have
even been calls for a moratorium on further growth, allowing for a period of breathing
space to allow the infrastructure to catch up and even allow for a report of some sort to
be undertaken (Matthews 2009c). Overall, though, the IB community seems to remain
remarkably loyal, and the level of concern is still relatively low. Other commentators
(Burke 2009) have been quick to respond and rebuke the concerns. This articles purpose
is not to sensationalize the situation; the IB is not in crisis. However, there does seem to
be real concern about the assessment of the DP (and even the MYP, though that needs
184 The Educational Forum Volume 75 2011

Consistency and Reliability of the International Baccalaureate Assessments

Downloaded by [University of Windsor] at 12:27 16 November 2014

further investigation) among some IB adherents (at quite a high level of involvement),
and they deserve a wider audience. Interestingly, the 2006 School Satisfaction Survey had
highlighted assessment concerns also at the PYP level (IBO 2006b, 3).
The overall emerging concerns are seemingly threefold, and it is important to make
distinctions among them because assessment is just one dimension of the overall issue. First,
the IB has undergone much change in terms of its organizational culture. It has become a
more corporate, business-oriented purveyor of products. The international school movement,
the originators of the IB, apparently takes offence at being viewed as customers. Second, the
growth of the IB has led to a more diverse set of schools being involved. This in itself has led to
a growing set of constituents for the IB to juggle, each with its own concerns and educational
agendas. Maintaining the image of a coherent IB World is, thus, becoming a major challenge
for the IB, and it appears unlikely that the community can be permanently aligned.
Third, and most importantly for this article, there are concerns about the substantial
current and anticipated growth compromising the quality of assessment. At present, this
issue seems a little premature. The IB maintains a high level of consistency and reliability.
However, the event of 2007 (the History paper issue) was of real concern, and concerns
over DP subject world averages also are very real.
The degree to which the IB can continuously attract 1,000 DP examiners and moderators
annually (and a further 10,000 by 2020) is also questionable. It is significant to note that the IB
was advertising on its Web site in September 2010 for 18 chief and deputy examiners to start
a five-year term in September 2011. Furthermore, the Web site was calling for more examiners in 26 key subject areas. For May 2009, the IB trained an additional 1,000 Extended Essay
examiners (Al-Haddad 2009, 8). This becomes an even bigger issue when one considers the
implication made by Toze (2008, 6) that this assessment burden seems to fall disproportionately on the experienced (established) IB schools. It appears highly unlikely that the IB can
continue assessing its programs from only Cardiff in South Wales; and an assessment center
for the United States, or at least the Americas, seems a likely need in the near future.
Matthews (2009a, 28) asserted that growth continues to challenge the capacity of
the system. This issue really matters for the IB, which has never undergone any formal
advertising or marketing and relies totally on word-of-mouth comment. The IB takes
quality (in terms of reliability and consistency) very seriously; the 2009 Annual Review
(IBO 2009a) made seven references to the concept of high-quality services. The reputation, integrity, and general image of the IB are vital. In other words, the stakes of greater
growth and wider access are enormously high. The IB cannot afford to compromise on
quality; yet, at the same time, its idealistic mission statement and historical sense of mission arguably involve greater access and more impact. There seems no turning back from
further substantial growth; hence, the concerns about the level of diversity, the changing
organizational culture, and threats to assessment will probably also grow.

References
ACS International Schools. 2008. Research highlights benefits of IB study. ACS Market
Update. Available at: www.acs-alumni.co.uk/page.cfm?p=330.

The Educational Forum Volume 75 2011 185

Downloaded by [University of Windsor] at 12:27 16 November 2014

Bunnell
Al-Haddad, J. 2009. IB responds to concerns raised in Toze letter. The International Educator 23(3): 8, 28.
Beard, J., and S. Holloway. 2010. Head to head: A session for IB heads of school. Paper
presented at the IB Heads of Schools conference, March 25, Singapore. Available at:
www.ibo.org/ibap/conference/archive/2010/documents/JeffBeardandSallyHolloway.pdf.
Betts, B. 2009. AISH attempts advocacy over IB concerns. The International Educator 24(2): 4.
Brown, P. 2000. The globalisation of positional competition. Sociology 34(4): 63353.
Burke, L. 2009. IB has this vote of confidence. The International Educator 22(4): 8.
Conn, M. 2001. Assessment in the MYP. IB World 29: 1819.
Doherty, C., L. Mu, and P. Shield. 2009. Planning mobile futures: The border artistry of
International Baccalaureate diploma choosers. British Journal of Sociology of Education
30(6): 75771.
Hayden, M., ed. 2005. A letter from the editor. IB World 43, 1.
International Baccalaureate Organization. 2004a. Diploma Programme assessment: Principles
and practice. Geneva, Switzerland: Author. Available at: www.ibo.org/diploma/assessment/
documents/DPAssessmentPrinciplesandPractice.pdf.
International Baccalaureate Organization. 2004b. Strategic plan of the International Baccalaureate organisation. Geneva, Switzerland: Author. Available at: www.ibo.org/mission/
strategy/documents/sp2004.pdf.
International Baccalaureate Organization. 2006a. From growth to access: Developing an IBO
access strategy. Geneva, Switzerland: Author. Available at: www.ibo.org/mission/strategy/
documents/FromGrowthtoAccess.pdf.
International Baccalaureate Organization. 2006b. IB World school satisfaction report. Geneva,
Switzerland: Author. Available at: www.ibo.org/programmes/services/satisfaction/documents/CouncilandIHRCreportonschoolsatsurvey.pdf.
International Baccalaureate Organization. 2007. What can I be? Telling the IB story. Paper
presented at the IB Asia Pacific annual regional conference, October 1720, Beijing.
Available at: www.ibo.org/ibap/conference/archive/documents/SimoneEmmisonandJenanAlHadda-IntroducingIBNewIdentity.pdf.
International Baccalaureate Organization. 2009a. Annual review 2009. Geneva, Switzerland:
Author. Available at: www.ibo.org/facts/annualreview/documents/IBAnnualReview2009.
pdf.
International Baccalaureate Organization. 2009b. Financial review 2009. Geneva,
Switzerland: Author. Available at: www.ibo.org/facts/annualreview/documents/IBaccountsAW09singlepagePDF.pdf.
International Baccalaureate Organization. 2010. IB diploma examiners. Geneva, Switzerland:
Author. Available at: www.ibo.org/examiners.
Lauder, H. 2007. International schools, education and globalisation: Towards a research
agenda. In The Sage handbook of research in international education, eds. M. Hayden, J.
Levy, and J. Thompson, 44149. Thousand Oaks, CA: Sage.
Matthews, M. 2009a. Another head weighs in on the IB raised in Toze letter. The International Educator 23(3): 28.
Matthews, M. 2009b. Challenging the IB: A personal view, post-Seville. The International
Educator 24(2): 1920.

186 The Educational Forum Volume 75 2011

Downloaded by [University of Windsor] at 12:27 16 November 2014

Consistency and Reliability of the International Baccalaureate Assessments

Matthews, M. 2009c. The International Baccalaureate: Common myths, real concerns. The
International Educator 24(1): 11, 17.
Meadows, D. H., J. Randers, and D. L. Meadows. 1972. The limits to growth: A report for the
Club of Romes project on the predicament of mankind. New York: Universe Books.
Mott, P. 2009. IB Part 2. The International Educator 24(2): 19.
Paap, J., and R. Katz. 2004. Anticipating disruptive innovation. Research Technology Management 47(5): 1322.
Renaud, G. 1975. The International Baccalaureate. Prospects 5(1): 11119.
Savage, D. 1982. The IB challenges high school students. Educational Leadership 39(8):
60203.
Seldon, A. 2009, March 22. Think tank: New ideas for the 21st century: Exams are failing
our schools. The Sunday Times. Available at: www.timesonline.co.uk/tol/news/uk/education/article5950375.ece.
Taylor, E. 1994. Intercultural competency: A transformative learning process. Adult Education Quarterly 44(3): 15474.
Times Educational Supplement. 2007, August 10. IB dj vu. Times Educational Supplement.
Available at: www.tes.co.uk/article.aspx?storycode=2418981.
Toze, D. 2008. Concerns about IB exam results undermining confidence in program. The
International Educator 23(2): 6.
Walker, G. 1999. New directions. IB World 22: 810.
Walker, G. 2005. The concept of quality. IB World 43: 56.
Walker, G. 2007. International education: From contacts to connections. Speech delivered at
IB Asia Pacific annual regional conference, January 4, Singapore. Available at: www.
ibo.org/dg/emeritus/speeches/documents/george_walker040107.pdf.

The Educational Forum Volume 75 2011 187