FRAME - Baseline Study On Human Rights Indicators in The Context of The European Union

Fostering Human Rights among European
Policies
Large-Scale FP7 Collaborative Project

GA No. 320000
1 May 2013-30 April 2017
Baseline Study on Human Rights Indicators in the

Context of the European Union
Work Package No. 13 Deliverable No. 1
Due date
31 December 2014
Submission date
24 December 2014
Dissemination level
Lead Beneficiary
Authors
PU
European Training and Research Centre for Human Rights and Democracy
(ETC Graz)
Klaus Starl, Veronika Apostolovski, Isabella Meier, Markus Mstl,
Maddalena Vivona and Alexandra Kulmer, in collaboration with Hans-Otto
Sano and Erik Andr Andersen
http://www.fp7-frame.eu
FRAME
Deliverable No. 13.1
Acknowledgments
The research leading to the results presented in this report has received funding from the European
Community's Seventh Framework Programme under the project FRAME- Fostering Human Rights Among
European (external and internal) Policies (Grant Agreement Nr. 320000).
The authors are particularly grateful to Nicolas Hachez, Tamara Lewis, Karin Lukas, Axel Marx, Anja Mihr,
Helmut Sax and Alexandra Timmer and for their insightful comments on earlier versions of this report. All
errors of course remain the authors own. The authors are equally thankful to the EU officials and
Members of the European Parliament who agreed to share their expertise for this report.
The authors thankfully acknowledge the invaluable editorial assistance of Matthew Windham.
FRAME
Executive Summary
Human rights indicators are an essential instrument for planning, monitoring and evaluating the
effectiveness of human rights protection and promotion. On the international, regional, national and local
levels, numerous mechanisms for measuring human rights have been developed; some of them were
already applied in practice with varying degrees of success, while others remained theoretical attempts.
Such mechanisms usually resort to qualitative and/or quantitative indicators to measure the current state
and/or progress of particular human rights or assess the impacts of policies/measures in a defined
geographical area within a given time-frame.
The aim of this study is to critically assess and analyse existing human rights indicators systems, and
identify their objectives, target audience and methodology. This is done in order to formulate objectives
for human rights indicators to be used in the European Unions human rights (internal and external)
policies. Up until now, several EU institutions have taken initiatives to measure human rights or have at
least underlined the necessity to do so. Human rights measurements may indeed be utilised by the EU to
ground its policies on a solid base of evidence and further help backing EU actions with increased
legitimacy. When identifying meaningful and applicable mechanisms to measure human rights, the EU
may well build on the existing work on human rights indicators. However, a comparison of the EUs
attempts to develop human rights indicators reveals considerable discrepancies in the approaches taken.
These differences are often closely linked with the specific purposes and objectives of the producers and
users of human rights indicators.
This study starts with an analysis of the current application of human rights indicators in the European
Unions internal and external action, their legal and political framework, as well as their rationale. It
further assesses, through qualitative fieldwork research, the needs of key European stakeholders towards
a human rights indicator system.
An extensive mapping of various mechanisms for measuring human rights was done. The mapping results
encompass instruments produced by a variety of actors, based on different rationales, data sources and
with different areas of application. The focus has been put on those mechanisms which are developed
and applied by the organisations mentioned in respective EU documents, i.e. the UN institutions and the
Council of Europe.
In order to find out if those mechanisms are suitable for further adaption and use by the EU, their intrinsic
quality has been evaluated first. This was then finally matched with the requirements of EU bodies. The
selection criteria do primarily reflect relevance, appropriateness and reliability. Therefore, established
quality criteria for the identification of human rights indicators are key for the selection. Additionally,
pragmatic criteria such as being already used in practice, taking into account data availability and userfriendliness are considered. Due to the variety of purposes of human rights measurement, the range of
requirements that should be measured demand a comparative, but also differentiated analysis. The
methodology needs to be consistent and broadly accepted. Flexible frequency of application must be
ensured. The instruments need to enable the proof of causality between measure and impact.
The report concludes with the selection of the structure-process-outcome model by the OHCHR. This
model is designed to measure the extent to which human rights dimensions respect, protect, fulfil and
ii
FRAME
promote human rights standards in any given environment. As a human rights indicator model it does
fulfil all of the mentioned criteria. In order to give EU stakeholders a pragmatic tool at hand for their daily
work, it is proposed that an easy to access instant information tool, i.e. an information database on
compliance including a compilation of existing indicators and related data sets, should be developed.
iii
FRAME
List of abbreviations
ACP African, Caribbean and Pacific group of States
CEPEJ European Commission for the Efficiency of Justice of the Council of Europe
CFR Charter of Fundamental Rights
CJEU Court of Justice of the European Union
CoE Council of Europe
COREPER Committee of Permanent Representatives
CSP Country Strategy Paper
COHOM Human Rights Working Group of the Council of the European Union
COREPER Council of the European Union Permanent Representatives Committee
CSDP Common Security and Defence Policy

CVM Cooperation and Verification Mechanism
DCI Development Cooperation Instrument
DG DEVCO European Commissions Directorate General for Development and Cooperation
DG EMPL European Commissions Directorate General for Employment, Social Affairs and Inclusion
DG ENLARG European Commissions Directorate General for Enlargement
DG HOME European Commissions Directorate General for Home Affairs
DG JUST European Commissions Directorate General for Justice
DROI European Parliament Subcommittee on Human Rights
ECJ Court of Justice of the European Union
EDF European Development Fund
EEAS European External Action Service
EHRC Equality and Human Rights Commission
EIDHR European Instrument for Human Rights and Democracy
EIGE European Institute for Gender Equality
EMCO European Commissions Employment Committee
ENI European Neighbourhood Instrument
ENP European Neighbourhood Policy
EPM Employment Performance Monitor
iv
FRAME
EPSCO Council of the European Unions Employment, Social Policy, Health and Consumer Affairs
Council
ESN European Services Network
EU European Union
EU-LFS European Union Labour Force Survey
EUROSTAT Statistical Office of the Commission of the European Union
EU-SILC European Union Statistics on Income and Living Conditions
FEMME European Parliaments Committee on Womens Rights and Gender Equality
FRA European Union Agency for Fundamental Rights
FREMP Working Party on Fundamental Rights, Citizens Rights and Free Movement of Persons of the
Council of the European Union
FTA Free Trade Agreement
GGDC Good Governance and Development Contract
GRECO Group of States against Corruption
GSP Generalised Scheme of Preferences
HR Human Rights
IA Impact Assessment
IcSP Instrument contributing to Stability and Peace
IGO Intergovernmental Organisation
INTA European Parliament Committee on International Trade
IPA Instrument for Pre-Accession Assistance
ISPA Instrument for Structural Policies for Pre-Accession
JAF Joint Assessment Framework
LGBT(I) Lesbian, Gay, Bisexual, Transsexual (and Intersexual)
LIBE European Parliament Committee on Civil Liberties, Justice and Home Affairs
MAF European Union Agency for Fundamental Rights Multiannual Framework
MDG Millennium Development Goal
MFF Multiannual Financial Framework
MIP Multi-Annual Indicative Programme
MPG Migration Policy Group
MS Member State
v
FRAME
NEET Not in Education, Employment or Training

NSR National Strategy Report
NGO Non-governmental Organisation
NHRI National Human Rights Institution
NIP National Indicative Programme
NPM National Preventive Mechanism
OCT Overseas Countries and Territories
OECD Organisation for Economic Co-operation and Development
OHCHR United Nations Office of the High Commissioner for Human Rights
OMC Open Method of Coordination for Social Protection and Social Inclusion
PEFA Public Expenditure and Financial Accountability
PHARE Poland and Hungary: Assistance for Restructuring their Economies
PI Partnership Instrument
PSC Political and Security Committee
RIP Regional Indicative Programme
RBA Rights Based Approach
SAPARD Special Accession Programme for Agriculture and Rural Development
SHRC Scottish Human Rights Commission
SPC Social Protection Committee
SPPM Social Protection Performance Monitor
SSF Single Support Framework
TEU Treaty on European Union
TFEU Treaty on the Functioning of the European Union
UN United Nations
UNDP United Nations Development Programme
UPR Universal Periodic Review
vi
FRAME
Table of Contents
Acknowledgments.......................................................................................................................................... i
Executive Summary....................................................................................................................................... ii
List of abbreviations ..................................................................................................................................... iv
I.
Introduction .......................................................................................................................................... 9
A.
General introductory remarks .......................................................................................................... 9
B.
Guiding questions, goals and structure .......................................................................................... 10
C.
Methods .......................................................................................................................................... 12
D.
Brief introductory remarks on human rights indicators ................................................................. 13
II.
1.
What are human rights indicators? ............................................................................................ 13
2.
Why are human rights indicators important? ............................................................................ 16
3.
Human rights indicators related terminology ............................................................................. 17
Human rights measurements by the European Union ....................................................................... 20

A.
Legal bases and political commitments for measuring human rights ............................................ 20
1.
Legal bases .................................................................................................................................. 20
2.
Political commitments ................................................................................................................ 26
3.
Conclusions ................................................................................................................................. 38
B.
Purposes of human rights measurement ....................................................................................... 39

1.
Policy planning ............................................................................................................................ 39
2.
Implementing policies and assessing compliance with human rights standards ....................... 42
3.
Evaluation of human rights policies ............................................................................................ 45
4.
Conclusion ................................................................................................................................... 46
C.
Human rights measurements by EU bodies: current applications and future requirements ........ 48
1.
Mapping of existing applications of human rights measurement tools ..................................... 48
2.
Assessing current EU measuring instruments and identifying future requirements ................. 77
III.
Refining the European Unions approach to measuring human rights .......................................... 87
A.
B.
Review of existing approaches in measuring human rights ........................................................... 87

1.
Some measuring schemes stand out .......................................................................................... 90
2.
No consensus on appropriate methodology .............................................................................. 91
3.
Indicators are indispensable ....................................................................................................... 92
4.
Miscellaneous data sources are used ......................................................................................... 93

Definition and application of selection criteria .............................................................................. 96
vii
FRAME
1. Selection criterion 1: Established quality criteria for the identification of human rights
indicators ............................................................................................................................................ 97
2.
Selection criterion 2: Three pragmatic criteria ........................................................................... 98
3.
Selection criterion 3: Four criteria guided by requirements of the EU bodies ........................... 99
C.
Conclusion: Suggestion of a two-pronged approach .................................................................... 101
D.
The way forward ........................................................................................................................... 105
Bibliography .............................................................................................................................................. 106

Legal instruments.................................................................................................................................. 106
Case-law ................................................................................................................................................ 107
Policy instruments, reports and papers ................................................................................................ 107
Literature .............................................................................................................................................. 115
Books ................................................................................................................................................. 115
Books chapters .................................................................................................................................. 116
Journal articles and working papers ................................................................................................. 117
Other secondary sources: statements, newspapers articles, press releases and internet websites120
Annex I List of Indicator Schemes and Human Rights Measurement Instruments ........................... 122
Annex II - Case studies on the use of rule of law indicators in the EU enlargement process: Bulgaria,
Croatia and Montenegro ...................................................................................................................... 123
Annex III List of conducted interviews ............................................................................................... 184
viii
FRAME
I.
Introduction
A.
General introductory remarks
The FRAME work package on human rights indicators, which this report forms part of, is embedded and
closely interlinked with the other FRAME research topics. Human rights challenges, conceptions of human
rights in relation to the rule of law and democracy, the engagement and cooperation with international
governmental, as well as private actors, the issue of internal and external, vertical and horizontal
coherence, human rights in violent conflicts or human rights in European Unions (EU) democratisation
policies are all highly relevant to the topic of measuring human rights dimensions. The research findings
conceptually feed into the present report and indicate priorities and urgencies. Most importantly, they
give guidance on and clarification of the actual object which should be measured, why it should be
measured and at what levels.
This report starts from the assumption that the EU pursues a reliable human rights policy in all of its policy
fields. For different purposes though, the various EU bodies and actors need information on the human
rights situation and performance in their own sphere and about actors, countries and regions the EU is
cooperating with, no matter whether within explicit human rights policies or within any other field,
because human rights shall play a role in any case. Evidence on human rights can appear in different forms.
Within the policy context, decisions will be triggered by this evidence. Thus, models of information
provision which use indicators are deemed most appropriate. Therefore, the baseline study neither
questions, whether there are other forms of information which could lead to the same effects, nor does
it address the challenge of policies that tend to be designed to satisfy indicators rather than the rightsbearers.
The overall objective of this report is to map and critically assess existing human rights indicator schemes,
as well as attempts to develop human rights indicators in relation to EU actions and policies. The report
aims at analysing the state of the EUs application of respective instruments on the one hand and the state
of the availability of appropriate human rights indicator schemes which are deemed to give relevant
information to measure the respect, protection and fulfilment of human rights by referring to human
rights standards on the other hand.
The report is thought to go beyond a baseline study. Besides mapping existing techniques and mapping
the rationales and respective requirements of EU stakeholders concerning human rights information, the
report aims at providing arguments for a selection of appropriate indicator systems by matching the
requirements with the features of human rights information systems. Rather than reiterating the
academic discussion on human rights indicators and their underlying methodologies, the present report
should contribute to the usability of human rights information and therefore takes a users perspective.
Consequently, the overall conclusion of this report is an informed proposal of human rights-centred
methodologies under highly pragmatic conditions for use, gained by asking the current and potential
future users thereof. In this way, the report aims at providing a basis for deciding on appropriate
techniques for EU-specific purposes.
FRAME
The report is about measuring human rights dimensions. Hence, it neither critically discusses the content
of legal and policy documents dealing with human rights policies, nor does the report discuss exhaustively
academic literature on indicators, human rights indicators or their underlying methodologies. It
pragmatically goes only that far that a selection and a reliable proposal is possible concerning the right
instruments, based on the insight into the needs of the EU. These needs are derived from the legal
documents enshrining human rights as core values and from the other documents stating that human
rights measurement should be done with respective indicators and for what reason.
The report also reveals some possible gaps concerning the availability of instruments on the one hand and
the use of instruments on the other hand. Although this is beyond the scope of the report and therefore
not addressed explicitly, these gaps concern the usability of instruments, i.e. there are not many
methodologies or instruments focusing on human rights, being either too specific in scope or not
appropriate for the purposes due to their underlying methodologies. There are large-scale information
systems including indicators for major UN programmes in the field of development, health, education and
others. They are human rights-relevant, but are not human rights measurements in the narrow sense, as
they provide data on human rights-relevant facts, but do not measure human rights dimensions. Another
important finding is the lacking linkage between qualitative information provided by treaty-bodies,
monitoring bodies or other reports, which were declared as most relevant by the interviewed EU officials,
and quantitative information systems, mostly based on statistical data. The gaps on the user-side are
manifold and can be summarised as awareness gaps, knowledge gaps, resource gaps, efficiency gaps and
cooperation gaps.
B.
Guiding questions, goals and structure
The present report follows a pragmatic approach. It aims at coming to a conclusion of who needs what,
and for what reason. Human rights are a core value of the EU. The Union further aims at evidence-based
policy making. Thus, is there a legal basis for measuring human rights? Is there a political commitment to
do so? Is this the case in all policy areas or only in some? What can we find out about the implementation
of human rights measurement? Is the assessment of human rights an end in itself or a means to achieve
other goals? What human rights and in which of their dimensions should be measured? What
measurement systems are available and in use by EU bodies? Considering the users and their
requirements, what system can be recommended? And finally, how should an adequate set of measuring
mechanisms be elaborated and installed in order to provide the various actors with relevant, reliable,
accurate and timely information on human rights?
The report follows an inductive approach: Parts II.A and II.B scrutinise the legal obligations and political
commitments of measuring human rights. The report analyses the legal and policy background of existing
instruments, which are deemed to give information on the human rights situation in the EU, the objectives
and purposes of such policies and instruments and it discusses practical matters of measuring human
rights in the EU. In order to facilitate a more analytical approach, which further promotes the reaching of
meaningful and well-structured conclusions, rather than a mainly descriptive approach, we agreed upon
10
FRAME
an analysis structured along policy areas. These policy areas are the EU internal area, the EU enlargement
area and the area of external action. A subdivision was then conducted regarding the different stages of
policies, namely policy planning and formulation, implementation and evaluation in order to allow for a
meaningful differentiation between the requirements for those different stages of policy cycles. For
Chapter II.B, it will be shown that the structure of planning tasks, implementation of policies and
evaluation thereof is most appropriate in order to conclude what measurements will be appropriate for
what purpose.
Part II.C subsequently analyses what the requirements are in practice, what systems are already applied
and what should be done in the future in order to enable EU officials to fulfil their tasks on an evidencebase. This part relies on interviews with several EU officials.
Part III then analyses the overall issue of this report from a different angle. It looks at which systems are
available in the field. It starts with a discussion on the necessary features of a human rights measurement
system, satisfying human rights-relevant criteria. Part III.B matches the requirements of the EU bodies
with the quality criteria of human rights measurement schemes in order to select appropriate models for
recommendation. Finally, the conclusions and the proposed way forward are presented in parts III.C and
III.D.
The focus throughout the report is the measurement of human rights. The legal and policy basis as well
as the interviews and the mapping of the instruments were all conducted with the ulterior motive of
presenting the status quo and future perspectives on measuring human rights in all EU policy areas,
interior, enlargement procedures and external policies.
Annex I lists a large selection of human rights and human rights related indicator models including
aggregated human rights (-related) indices. The priority is given to models that are more comprehensive
in scope, which are in use and published by international organisations, such as the Council of Europe, the
World Bank and UN organisations.
The report is complemented by a case study in Annex II. In this case study, the researchers scrutinise the
application of human rights information in the enlargement procedures along three examples, namely
Bulgaria, Croatia and Montenegro. It is focused on the rule of law as the central interest of the EU in its
enlargement strategy. The case study analyses the development of the information requirements in the
Commissions assessment process. The case study gives important hints on the use of human rights
information in EUs practical use of such information and is therefore an important part of this report,
complementing the formal reasons, technical requirements and availability of information systems.
Annex III provides an overview of the interviews that have been conducted for the present report.
11
FRAME
C.
Methods
Differing methodological approaches were taken for the different parts of this report. The first two parts
on the legal basis and political commitments are based on desk research, attempting to cover the wide
picture of the EUs action and the status quo of measuring human rights.
Also based on desk research, a mapping of existing systems, concepts and mechanisms for human rights
or human rights related measurements has been carried out in parallel. About 130 such schemes were
identified and described along parameters which indicate their relevance, meaningfulness and
appropriateness to give information on human rights in a wider sense. The main mapping criteria are the
type of author, the thematic context, their rationales, their users, the types of indicators and the
underlying methodologies, data sources and quality of suppliers and processors, as well as the actual
application and frequency of publication. In Annex I of the report, 50 human rights information schemes
are listed and described in detail.
In section II.C.1 the research team looked into existing EU mechanisms and instruments, where human
rights were taken into account. The most relevant instruments for this research were chosen and
analysed. The attempts to monitor and/or measure human rights were especially underlined and
presented. The list of instruments is not exhaustive, but is framed according to the purposes of this paper
to come to valuable conclusions on the status quo. The findings of this mapping were then linked to the
findings of interviews conducted with EU officials.
19 structured interviews were conducted with EU officials by researchers of the ETC and the Danish
Institute for Human Rights. The qualitative interviews complement the desk research findings on
measuring human rights with the perspective of stakeholders.
The interview guidelines were already based on the mapping of existing human rights measurement
schemes. The guidelines encompass upstream questions on future mechanisms as well as a description
and an assessment of currently applied mechanisms. For both, future and currently applied mechanisms,
the following criteria had been addressed in the guidelines: the human rights topics covered, the
geographical area of application, the question on the provision and processing of available and reliable
data, the (non-)comparability of the results across various countries/regions, the frequency of application
and the strengths and weaknesses of currently applied mechanisms as perceived by the users.
The main criterion for the selection of interviewees was to find officials in key positions, who apply or
need human rights measurement. They were identified through organisation charts. Although the
readiness of officials to participate in the survey and to support the project was high, not all of them
specifically conduct human rights measurement. At the European Commission, interviews were
conducted with members of the following Directorates-General (DGs): Employment, Social Affairs and
Inclusion (EMPL); Enlargement (ENLARG); EuropeAid Development & Cooperation (DEVCO); Home Affairs
(HOME); Justice (JUST). Members of the Social Protection Committee (SPC) were interviewed too. At the
Council of the EU, interviews were conducted with members of the European External Action Service
12
FRAME
(EEAS), the Human Rights Working Group (COHOM) and the Working Party on Fundamental Rights,
Citizens Rights and Free Movement of Persons (FREMP). At the European Parliament, interviews were
conducted at the Committee on Civil Liberties, Justice and Home Affairs (LIBE), Committee on
International Trade (INTA), Women's Rights and Gender Equality (FEMME) and Subcommittee on Human
Rights (DROI) (mostly civil servants rather than members of Parliament). In each interview, one to six
persons participated; one person was interviewed twice.
The interviews were carried out either face-to-face or by telephone. Upon request, interviewees received
the guidelines in advance. The interviews followed the guidelines closely. In certain cases where the
interviewees time was lacking, the focus was set on the requirements and purposes of future
measurements. The interviews lasted between half an hour and an hour. The summaries of the interviews
were sent to the interviewees for approval where requested. Since the majority of the interviewees
wanted to remain anonymous, only the institutions will be cited in this report. However, it has to be
mentioned that the information provided by the interviewees represent their professional opinions and
not necessarily the official line of the EU bodies by which they are employed.
Again, for the case study presented in Annex II desk research has been applied and pieces of information
stemming from a few relevant interviews conducted for part II were used.
Part III brings together the findings of all previous parts and applies a conclusion based on inductive
reasoning. The requirements of EU bodies are identified from formal and normative documents, from
guidelines to be applied in the different fields of activities and the information gained from the mentioned
interviews. These findings are further corroborated by the case study in the area of enlargement. The EU
requirements on human rights information are then tested against the technical features of the
information schemes identified through the mapping. This is done with a view to define relevant selection
criteria. These criteria are applied to describe the appropriate model for measuring human rights
dimensions for EU use. Finally, an informed selection of an appropriate human rights information tool is
presented.
D.
Brief introductory remarks on human rights indicators
This report aims to identify what are the needs of the EU institutions, if any, regarding human rights
indicators and suggest appropriate human rights measurement tools, drawing on an extensive body of
literature and experience in developing and applying human rights indicators. This section offers brief
remarks on the concept and purposes of human rights indicators as well as a very brief description of main
human rights indicators related terms.
1.
What are human rights indicators?
Indicators can generally be defined as informational tools, or as Abbot and Gujit say pieces of information
that provide insight into matters of larger significance and make perceptible trends that are not
13
FRAME
immediately detectable.1 The most salient trait of indicators is thus that they simplify reality in order to
make assertions about (social) phenomena. Engle Merry observes that a key dimension of the power of
indicators is their capacity to convert complicated contextually variable phenomena into unambiguous,
clear, and impersonal measures. [] They depend on the construction of categories of measurement such
as ethnicity, gender, income, and more elaborated concepts such as national income.2 This process of
abstraction is extremely valuable for processing information and allows comparing situation across time
and space, but, as Kaufmann and Kraay observe, there is a good deal of subjective judgment involved3 in
developing indicators and in the process something is always left behind. Talking about statistical
indicators, for example, Engle Merry states that:
The essence of an indicator is that it is simple and easy to understand. Embedded theories,
decisions about measures, and interpretations of the data are replaced by the certainty
and lack of ambiguity of a number. [] But what information is lost? Does the number
bury the messiness of difference and allow equivalence?4
Problems arising by this process of simplification can be observed also in the field of human rights. One
of the most basic features of human rights is that they are universal in nature: they are rights that every
human being, irrespective of where one lives should be entitled to.5 Human rights could therefore be
understood as a topic that can be easily measured across societies and cultures. When transforming these
universal rights into measurable indicators, however, the conceptual complexity of human rights becomes
apparent.6 In a study on the development of childs rights indicators in Tanzania, Engle Merry for example
1
Joanne Abbot and Irene Gujit, Changing Views on Change: Participatory Approaches to monitoring the
Environment (1998) SARL Discussion Papers 254, p. 40.
In the same line the UNDP defines them as device for providing specific information on the state or condition of
something. United Nations Development Programme (UNDP), Indicators for Human Rights Based Approaches to
Development
in
UNDP
Programming:
A
Users
Guide
(UNDP
2006),
p.
21,
<http://www.undg.org/docs/11652/HRBA-Indicators-(2006).pdf> accessed 4 November 2014.
2
Sally Engle Merry, 'Measuring the World: Indicators, Human Rights, and Global Governance' (2011) 52 (Supplement
to Number 3) Current Anthropology 83, p. 84.
3
Daniel Kaufmann and Aart Kraay, 'Governance Indicators: Where are We? Where should We be going?' (2007)
World Bank Policy Research Working Paper 4370, p. 8.
4
Sally Engle Merry, 'Measuring the World: Indicators, Human Rights, and Global Governance' (2011) 52 (Supplement
to Number 3) Current Anthropology 83, p. 86.
In the same line see also: Russel L. Barsh, 'Measuring Human Rights: Problems of Methodology and Purpose' (1993)
15 Human Rights Quarterly 87.
5
On the issue of universal human rights see Alexandra Timmer, Report State of the Art Literature Review Human
Rights, Democracy and the Rule of Law (2014) Frame Deliverables, pp. 7-9, <http://www.fp7-frame.eu/reports>
accessed 6 October 2014.
6
Russel Lawrence Barsh, 'Measuring Human Rights: Problems of Methodology and Purpose' (1993) 15 Human Rights
Quarterly 87, pp. 92-93; Carr Center for Human Rights Policy at Harvard University, Measurement and Human
Rights: Tracking Progress, Assessing Impact Report on the Carr Centre Conference Measurement and Human Rights:
Tracking
Progress,
Assessing
Impact
of
5-7
May
2005,
p.
3,
http://www.hks.harvard.edu/cchrp/mhr/publications/documents/MeasurementReport2005.pdf accessed 13 April
2014.
14
FRAME
observes how even the wording rights becomes problematic, since it is commonly translated and
understood in Kiswahili as moral obligations instead of persons entitlements.7 Similarly, Coppedge and
Gerring discuss how defining democracy can lead to different measurement results on the example of the
Polity 2 index, that does not take into account the composition of the electorate in the USA (from which
women and African Americans were for a long time excluded) and therefore assesses the country as fully
democratic throughout the twentieth century and much of the nineteenth century.8
When discussing about human rights indicators, therefore, how should they best be defined? In line with
the definition offered by Abbott and Gujit, Maria Green defines a human rights indicator as a piece of
information used in measuring the extent to which a legal right is being fulfilled or enjoyed in a given
situation.9 This definition qualifies as human rights indicators those indicators that measure the
application of human rights norms. It further highlights that human rights indicators are not limited to
measuring the actions of the duty-bearers, but are also meant to evaluate the outcome of those actions,
namely how human rights are enjoyed by the right-holders.
Paul Hunt, in his 2003 Interim Report as UN Special Rapporteur on the right of everyone to enjoy the
highest attainable standard of physical and mental health, clarifies that in his opinion what tends to
distinguish a right to health indicator from a health indicator is less its substance than (i) its explicit
derivation from specific right to health norms; and (ii) the purpose to which it is put, namely right to health
monitoring with a view to holding duty-bearers to account.10 Hunt thus classifies as human rights
indicators only those indicators that explicitly, namely in a reasonably close and precise11 way, relate to
human rights norms. Furthermore, [j]ust as the right to health has to be seen in this broader normative
context, so do right to health indicators. Accordingly, right to health indicators should not only reflect
specific right to health norms, but also related human rights provisions, including non-discrimination and
equality.12 On the methodological side, for example, this implies disaggregating data in relation to as
7
Sally Engle Merry, Quantification and the Paradox of Measurement (2014 AHRI Conference, Copenhagen,
September 2014).
8
Michael Coppedge and others, Conceptualizing and measuring Democracy: A New Approach (2011) 9 Perspectives
on Politics 247
Similarly Spirer discusses how different definitions of the concept of political prisoner can affect the outcome of
human rights measurements. Herbert Spirer, 'Violations of Human Rights. How Many?' (1990) 49 American Journal
of Economics and Sociology 199.
9
Maria Green, What we talk about when we talk about Indicators: Current Approaches to Human Rights
Measurement (2001) 23 Human Rights Quarterly 1062, p. 1065.
10
United Nations Special Rapporteur of the Commission on Human Rights on the Right of Everyone to enjoy the
Highest Attainable Standard of Physical and Mental Health, Interim Report on the Right of Everyone to enjoy the
Highest Attainable Standard of Physical and Mental Health 28 August 2003, A/58/427, para 10.
11
Highest Attainable Standard of Physical and Mental Health 28 August 2003, A/58/427, para. 11.
12
15
FRAME
many of the internationally prohibited grounds of discrimination as possible13 in order to be able to detect
possible disparities in the enjoyment of human rights.
The Office of the High Commissioner for Human Rights (OHCHR) took up many of the definitional elements
proposed by Hunt and Green, by qualifying human rights indicators as specific information on the state
or condition of an object, event, activity or outcome that can be related to human rights norms and
standards; that addresses and reflects human rights principles and concerns; and that can be used to
assess and monitor the promotion and implementation of human rights.14 This definition qualifies as
human rights indicators those indicators that can be related to human rights norms and standards. The
OHCHR further specifies qualities of human rights indicators, namely that they also need to reflect the
principles of human rights (e.g. non-discrimination and equality, empowerment, participation and
effective measures) and their purpose, namely to assess and monitor the promotion and implementation
of human rights.15
2.
Why are human rights indicators important?
There are many ways in which indicators can be helpful as informational tools. Landman and Carvalho
highlight six purposes of human rights measurement: contextual description and documentation,
classification, monitoring and pattern recognition, secondary analysis and policy prescription, and
advocacy and political dialogue.16 Besides the more general scopes of indicators, such as providing
evidence to gauge the magnitude of a certain situation or provide evidence to understand the root causes
of a problem, in the human rights field two purposes are worth mentioning: strengthening accountability
and evidence-based policy making.
Human rights indicators can be used to hold duty-bearers accountable for their actions.17 Already in the
1990s the human rights monitoring bodies of the United Nations (UN) asked states to present more
evidence to their human rights reports in order to better judge the efforts of states in complying with and
promoting human rights.18 In 1995, for example, the Committee on the Rights of the Child stated that:
13
14
United Nations Office of the High Commissioner for Human Rights, Human Rights Indicators: A Guide to
Measurement and Implementation (United Nations 2012), p. 16.
15
16
Todd Landman and Edzia Carvalho, Measuring Human Rights (Routledge 2010), pp. 4-6.
17
United Nations Development Programme, Human Development Report 2000 (UNDP 2000), p. 89; AnnJanette
Rosga and Margaret L. Satterthwaie, The Trust in Indicators: Measuring Human Rights, (2009) 27 Berkeley Journal
of International Law 253.
18
Organisation for Economic Co-operation and Development (OECD), 'Measuring Human Rights and Democratic
Governance: Experiences and Lessons from Metagora' (2008) 9 OECD Journal on Development 2, pp. 23-24.
16
FRAME
[...] there is a need for a comprehensive network for the collection of data covering all
areas of the Convention and taking into account all groups of children within Italy, which
is essential for the implementation of targeted programmes on the rights of the child and
the evaluation of the effectiveness of legislative and administrative measures.19
The 2000 United Nations Development Programme (UNDP) Report, looking at how human rights can
contribute to human development, highlighted the importance of human rights indicators in identifying
and holding important actors accountable for their action. As was pointed out by Andersen and Sano the
report discusses accountability of actors of human rights advocacy (not just the State).20 Human rights
indicators are in fact an essential tool for auditing not only States, but also non-governmental
organisations (NGOs) and other actors, especially in the field of development aid as it will be discussed
also later in this report.21
If the issue of accountability is important for advocating and monitoring compliance with human rights,
policy makers are taking ownership of human rights indicators through what is called evidence-based
policy making: human rights indicators are used for identifying problems or issues in need of political
attention, for highlighting priorities or analysing root causes.22 Human rights indicators also support policy
makers in fine-tuning and evaluating policies.
3.
Human rights indicators related terminology

a)
Monitoring vs. measurement
Human rights monitoring and human rights measurement are often used interchangeably when
discussing issues related to human rights indicators. Indeed, human rights measurement can be used for
monitoring the implementation of human rights. However monitoring human rights means evaluating the
gap between the current and desired situation, often defined by human rights standards. Measuring
human rights is descriptive in nature: it aims to describe and analyse a certain situation and answer
questions such as How many elderly persons experience violence by a member of their family? or Are
lesbian, gay and bisexual persons afraid to hold hands in public?. While the outcome of monitoring is a
judgement, the outcome of measurement is not.
19
UN Committee on the Rights of the Child, Concluding observations of the Committee on the Rights of the Child:
Italy CRC/C/15/Add.41 of 27 November 1995, para.6; on this topic see also: Judith V. Welling, 'International
Indicators and Economic, Social, and Cultural Rights' (2008) 30 Human Rights Quarterly 933.
20
Bard Anders Andreassen and Hans-Otto Sano, What's the Goal? What's the Purpose? Observations on Human
Rights Impact Assessment, The Norwegian Centre for Human Rights Research Note 2004/2, p. 8.
21
Erik Andr Andersen and Hans-Otto Sano, Human Rights Indicators at Program and Project Level: Guidelines for
Defining Indicators, Monitoring and Evaluation (Danish Institute for Human Rights 2006); Siobhn McInerneyLankford and Hans-Otto Sano, Human Rights Indicators in Development (World Bank 2010).
22
See Chapter II.B of this report on purposes of human rights measurement.
17
FRAME

b)
Quantitative and qualitative indicators
There is not always clarity within the human rights community in understanding what human rights
indicators are. As a consequence, human rights indicators are often reductively understood as statistical
indicators.23 Human rights indicators, however, can be both: they can be quantitative and qualitative. It is
important in this respect to highlight that the distinction between qualitative and quantitative indicators
does not correspond to a distinction between judgment-based and fact-based indicators: both
quantitative and qualitative indicators can be judgement-based or fact-based.
A quantitative indicator is any kind of indicator that is expressed primarily in quantitative form, such as
numbers, percentages or indices.24 Thus, a quantitative indicator is suitable to measure everything that
is countable in a way. Qualitative indicators gather data that is best expressed and recorded in a nonnumerical manner25 and allow for contextual analysis and measurement (most commonly qualitative
indicators address questions of behaviour, views and attitudes). Human rights specific examples of
qualitative indicators are the status of ratification of a human rights treaty or if lesbian, gay, bisexual,
transexual and intersexual (LGBTI) associations feel obliged to conceal their true vocations.26
c)
Benchmarks
In a study for the European Parliament on the use of benchmarks for the EU external policy, Mihr defines
benchmarks as points of references against which the EUs external policy can be measured by means of
regular, timely and systematically applied human rights indicators.27 Benchmarks are targets to be
achieved28 and are extremely valuable for measuring performance. Classical benchmarks are for example
contained in the Millennium Development Goals (MDGs): within the general goal of reducing child
mortality, for example, the MDGs define the target of reducing under-five mortality by two thirds by
2015.29
23
Maria Green, What we talk about when we talk about Indicators: Current Approaches to Human Rights
Measurement (2001) 23 Human Rights Quarterly 1062; Organisation for Economic Co-operation and Development
(OECD), 'Measuring Human Rights and Democratic Governance: Experiences and Lessons from Metagora' (2008) 9
OECD Journal on Development 2, p. 33.
24
25
United Nations Development Programme (UNDP), Report to UNDP Philippines: Human Rights Programme Review,
HURIST, May 2004, p. 44.
26
Council of the European Union, Guidelines to Promote and Protect the Enjoyment of All Human Rights by Lesbian,
Gay, Bisexual, Transgender and Intersex (LGBTI) Persons of 24 June 2013.
27
Anja Mihr, Human Rights Benchmarks for EU's External Policy (Publications Office of the European Union 2011), p.
7.
28
29
United Nations General Assembly, United Nations Millennium Declaration of 18 September 2000, A/RES/55/2,
para. 19.
18
FRAME

d)
Indices
Indices (commonly referred to as composite indicators or aggregated indicators) are a weighted

aggregation of individual indicators that usually summarise different dimensions of a multidimensional
issue (e.g. education, health, etc.), which cannot be adequately represented through individual indicators.
To give an example, the Human Development Index offers a summary measure of average achievement
in key dimensions of human development for each country (a long and healthy life, being knowledgeable
and have a decent standard of living). Obviously, decisions about how to weight indicators in the index
can lead to different results.30
e)
Disaggregation
For the purpose of measuring the equal enjoyment of human rights within a given population,
disaggregation allows to break down the data collected for each indicator, for instance, into possible
grounds for discriminatory actions, such as sex, age, or ethnicity.
30
Michael Coppedge and others, Conceptualizing and measuring Democracy: A New Approach (2011) 9
Perspectives on Politics 247, p. 250-251.
19
FRAME
II.
Human rights measurements by the European Union

A.
Legal bases and political commitments for measuring human
rights
This study starts with an analysis of the level of formal commitment of the European Union to measure
human rights.
The first sub-section analyses whether there is any obligation to measure human rights in the EU primary
and secondary law. In order to do so the basic provisions on fundamental rights within the EU legal
framework will be presented and analysed, focusing in particular on their implications for human rights
measurement.
The legal analysis will be then complemented by the analysis of the political commitment of the EU to
measure human rights and in particular, if there are concrete policies in place to measure human rights
as well as what is being advocated in the various policy areas of the EU.
1.
Legal bases
Although the EU was created as a peace project after the Second World War, fundamental rights have not
always been a core value of the European Union. At the time when the Council of Europe (CoE) was
founded in 1949 and when the European Convention on Human Rights entered into force in 1953, the
ownership for the protection, promotion and the monitoring of human rights lay mainly with the CoE. The
three European Communities (the European Economic Community, the European Coal and Steel
Community and the European Atomic Energy Community) initially aimed at establishing a common
European economic landscape, where human rights did not play a prominent role and therefore were not
foreseen in the founding treaties (Treaty of Paris 1951 and Treaty of Rome 1957). However, some
provisions on equal pay for male and female workers, the prohibition of discrimination on grounds of
nationality, as well as the freedom of movement of workers were foreseen in these European treaties.
Back then, the European Court of Justice (ECJ) also refused to rule on human rights in the case 1/58, Stork.
Ten years later (case 29/69: Stauder), the ECJ changed its case law and started reviewing human rights
cases.31
The importance and value of fundamental rights further grew over the years with the transformation of
the European Communities into the EU. In 1977 the European Parliament (hereinafter the Parliament),
the Council of the European Union and the European Commission (hereinafter the Commission) issued a
joint declaration stressing
31
Case 1/58, Stork vs. ECSC High Authority [1959] ECR 1959; Case 29/69, Erich Stauder v City of Ulm [1969] ECR 419;
Jrgen Khling, Fundamental Rights in Armin von Bogdandy and Jrgen Bast (eds), Principles of European
Constitutional Law (Hart Publishing 2011).
20
FRAME
the prime importance they attach to the protection of fundamental rights, as derived in
particular from the constitutions of the Member States and the European Convention for the
Protection of Human Rights and Fundamental Freedoms. In the exercise of their powers and
in pursuance of the aims of the European Communities they respect and will continue to
respect these rights.32
In 1986 the preamble of the Single European Act contained a reference to democracy and human rights
and in 1989 the Parliament adopted the Declaration on the Fundamental Rights and Freedoms.33 In 1992
the Treaty of Maastricht contained a reference on human rights in its preamble, confirming [the Member
States] attachment to the principles of liberty, democracy and respect for human rights and fundamental
freedoms and of the rule of law.34 With the Treaty of Amsterdam of 1997 fundamental rights were finally
anchored in primary law, stating in the amended Art. F that the Union is founded on the principles of
liberty, democracy, respect for human rights and fundamental freedoms, and the rule of law, principles
which are common to the Member States. In December 2000 the EU Charter of Fundamental Rights (CFR)
was finally proclaimed by the Nice European Council. The entry into force of the Treaty of Lisbon further
strengthened the European Unions human rights protection: the Charter of Fundamental Rights acquired
the same legal value as the treaties establishing the EU35 and now constitutes a legally binding reference
point for the meaning and the content of fundamental rights in the European legal system as stated in
Art. 6 Treaty on European Union (TEU).36
Fundamental rights nowadays constitute a hard core of defining features in which every Union citizen
can recognize himself, irrespective of the political or cultural differences linked to national identity.37
They represent thus the very foundation on which the EU is built. Human rights, as founding values
enshrined in Art. 2 TEU, are not subject to limitations of competence between the EU and its Member
States.38 The Charter defines in closer detail those fundamental rights that are mentioned in Art. 2 TEU,
but the Charter is limited by the division of competence between the EU and the Member States. Art. 51
of the Charter of Fundamental Rights, in accordance with Art. 4 and 5 TEU, limits its scope of application
32
European Parliament, the Council and the Commission, The Protection of Fundamental Rights and the European
Convention for the Protection of Human Rights and Fundamental Freedoms (Joint Declaration) [1977] OJ C103/1.
33
European Parliament, Declaration on the Fundamental Rights and Freedoms [1989] OJ C120/51.
34
Treaty on European Union [1992] OJ C191.
35
Consolidated Version of the Treaty on European Union (2012) OJ C 326/13, Art. 6.
36
Sergio Carrera and others, The Triangular Relationship between Fundamental Rights, Democracy and the Rule of
Law in the EU: Towards an EU Copenhagen Mechanism (Centre for European Policy Studies 2013), p. 1.
37
European Commission, Article 7 of the Treaty on European Union. Respect for and Promotion of the Values on
which the Union is based (Communication) COM(2003) 606 final, p. 3.
38
European Union Agency for Fundamental Rights, The European Union as a Community of Values: Safeguarding
Fundamental Rights in Times of Crisis (Publications Office of the European Union 2013), p. 7; Armin von Bogdandy
and others, 'Reverse Solange - Protecting the Essence of Fundamental Rights against EU Member States' (2012) 49
Common Market Law Review 489.
21
FRAME
to EU institutions (irrespective of whether their action is internal or external) and the Member States
when applying EU law.
Given this ever-growing relevance of human rights in the legal foundations of the EU, the question arises
as to what extent the legal bases also envisage mechanisms or tools to measure human rights. This
question is relevant for Member States and third states alike. With regard to the Member States the
procedure foreseen in Art. 7 TEU is the only mechanism that can be regarded as a post-accession
supervisory tool which equips EU institutions with the means to guarantee that the principles of Art. 2
TEU are respected by the Member States.39 The introduction of this procedure by the Treaty of
Amsterdam was on the one hand a reaction to the then upcoming wave of EU enlargement and on the
other hand an attempt to tackle the discrepancy between the democratic model promoted by the EU in
its external relations and its modest capacity to intervene whenever democratic values are at risk of being
violated within one of its Member States.40 The procedure of Art. 7 can be triggered in case of a serious
and persistent breach41 of the core values of the EU by a Member State, but also when there is a clear
risk42 that this might happen. As clarified by the European Commission, Art. 7 TEU is not limited to areas
covered by EU law, but can be triggered also by breaches in areas where the Member States are
competent.43 As there is no threshold set to identify situations when a state is at risk of, or in breach of
Art. 2, monitoring of Member States compliance with fundamental rights is left to the political discretion
of the EU bodies entitled to initiate an Art. 7 procedure. The mechanism of Art. 7 TEU was described as a
nuclear option44 by the former President of the Commission Barroso and has in fact never been applied
so far. Recalling that respecting the rule of law is a prerequisite for the protection of fundamental rights,
the European Commission recently proposed a new framework to deal with systemic threats to the rule
of law. The new framework is supposed to take effect in cases where it is not considered opportune to
start an Art. 7 procedure.45 The new procedure thus precedes and complements the procedure of Art. 7
TEU.
Over the years, two bodies have been enshrined in the Treaty on the Functioning of the EU (TFEU), with
the explicit mandate to monitor employment and social protection, areas that have been of main interest
to the European Communities and now the Union since their establishment. Although this cannot be
considered human rights monitoring, it is still highly relevant to it. More specifically, the Employment
39
Olivier De Schutter, 'A New Direction for the Fundamental Rights Policy of the EU' (2010) Reflexive Governance in
the Public Interest Working Papers 33, p. 5, <http://refgov.cpdr.ucl.ac.be/?go=publications> accessed 8 April 2014.
40
European Parliamentary Research Service, Article 7 TEU: A Mechanism to Protect EU Values,
<http://epthinktank.eu/2013/10/07/article-7-teu-a-mechanism-to-protect-eu-values/> accessed 27 October 2014.
41
Consolidated Version of the Treaty on European Union (2012) OJ C 326/13, Art. 7 2.
42
Consolidated Version of the Treaty on European Union (2012) OJ C 326/13, Art. 7 1.
43
European Commission, Article 7 of the Treaty on European Union. Respect for and Promotion of the Values on
which the Union is based (Communication) COM(2003) 606 final, p. 5.
44
Jos Manuel Barroso, State of the Union 2012 Address (Press Release), SPEECH/12/596 of 12/09/2012
<http://europa.eu/rapid/press-release_SPEECH-12-596_en.htm> accessed 07.08.2014.
45
European Commission, A New EU Framework to strengthen the Rule of Law COM(2014) 158 final/2.
22
FRAME
Committee (EMCO), introduced by the Treaty of Amsterdam, has the mandate to monitor the
employment situation and employment policies in the Member States and the Union (Art. 150 TFEU).46
The Social Protection Committee, introduced by the Treaty of Nice, has the explicit mandate to monitor
the social situation and the development of social protection policies in the Member States and the Union
(Art. 160 TFEU).47 In the course of its mandate the SPC evaluates the status of poverty and social exclusion,
health care, and pensions in the EU Member States. Although both the EMCO and the SPC do not have
explicit mandates to measure human rights, but the mandate to monitor selected themes, they both
began developing indicators in those areas of economic and social rights included in their mandate soon
after their establishment.
In 2006 and 2007, the Council of the EU established two agencies with the mandate to collect objective,
reliable and comparable information.48 The European Union Agency for Fundamental Rights (FRA) was
established to provide information on the development of the fundamental rights in the Member States.
Regarding information on equality between women and men, the European Institute on Gender Equality
(EIGE) was set up. EIGE and FRA share similar mandates, as they have been entrusted with the task of
providing EU institutions and Member States with independent, evidence-based advice on fundamental
rights and gender equality.49 Both agencies also have the mandate to develop methods to improve the
objectivity, comparability and reliability of data at the European level.50 In practice, FRA and EIGE have
both started to develop indicators for this purpose. EIGE has developed the Gender Equality Index a
composite indicator51 on gender equality. FRA has so far developed indicators for the right to political
participation of persons with disabilities and on the rights of the child.52
46
Consolidated Version of the Treaty on the Functioning of the European Union [2012] OJ C326/49, Art. 150.
Consolidated Version of the Treaty on the Functioning of the European Union (2012) OJ C326/49, Art. 160; Council
Decision 2004/689/EC of 4 October 2004 establishing a Social Protection Committee and repealing Decision
2000/436/EC [2004] OJ L314/8.
48
Council Regulation (EC) No.168/2007 of 15 February 2007 establishing a European Union Agency for Fundamental
Rights [2007] OJ L 53/1, Art. 4 para. 1a; European Parliament and Council Regulation (EC) No. 1922/2006 of 20
December 2006 establishing a European Institute for Gender Equality [2006] OJ L403/9, Art. 3 para. 1a.
49
Rights [2007] OJ L 53/1, Art. 2; European Parliament and Council Regulation (EC) No. 1922/2006 of 20 December
2006 establishing a European Institute for Gender Equality [2006] OJ L403/9, Art. 2.
50
Rights [2007] OJ L 53/1, Art. 4 para. 1b; European Parliament and Council Regulation (EC) No. 1922/2006 of 20
December 2006 establishing a European Institute for Gender Equality [2006] OJ L403/9, Art. 3 para. 1b.
51
The OECD defines composite indicators as follows: A composite indicator is formed when individual indicators are
compiled into a single index on the basis of an underlying model. The composite indicator should ideally measure
multidimensional concepts which cannot be captured by a single indicator. OECD, Handbook on Constructing
Composite Indicators: Methodology and User Guide (OECD 2008), p. 13. See also: Joe Rovan, Composite Indicators,
in Miodrag Lovric (ed.), International Encyclopedia of Statistical Science (Springer 2011).
52
European Union Agency for Fundamental Rights, The Right to Political Participation for Persons with Disabilities:
Human Rights Indicators (Publications Office of the European Union 2014); European Union Agency for Fundamental
Rights, Developing Indicators for the Protection, Respect and Promotion of the Rights of the Child in the European
47
23
FRAME
While offering at the moment the most comprehensive analysis of fundamental rights developments in
the Member States, the FRA is somewhat limited by its consulting function. The Council of the EU defines
the thematic priorities for the FRA in the Multi-annual Framework (MAF).53 The FRA cannot address by its
own initiative matters that fall outside the MAF, as it is the case for issues relating to police and judicial
cooperation in criminal matters. EIGEs mandate shares similar limitations, as this institutes work
programme should be in line with the Unions priorities in the field of gender equality and the work
programme of the European Commission.54
Looking into another area of EUs competence, namely in the framework of the European Unions external
action, Art. 21 TEU extends the principles of Art. 2 TEU to its foreign policy by stating that:
The Unions action on the international scene shall be guided by the principles which have
inspired its own creation, development and enlargement, and which it seeks to advance in
the wider world: democracy, the rule of law, the universality and indivisibility of human rights
and fundamental freedoms,[...] and respect for the principles of the United Nations Charter
and international law.55
Since Art. 21 TEU extends the principles of Art. 2 TEU to the Unions external actions, the question about
the legal foundation of an instrument designed to measure human rights gains relevance also in countries
that are not member of the Union. The wording of Art. 21 TEU commits the EU to pursue the principles of
democracy, the rule of law and human rights in its external actions, and to develop relations, build
partnerships and define common policies and actions in order to support these principles. This provision
makes the EU one of the most committed state-based actors in promoting these principles abroad.
However there is no explicit obligation to measure human rights in the external area in the TEU or the
TFEU.
Beside the general provision of Art. 21 which is applicable to the entire external action area human
rights are explicitly mentioned in the field of trade policy and development and in the neighbourhood
policy. The TFEU requires the common commercial policy (Art. 207 1) and development cooperation (Art.
208 1) to be conducted in the context of the principles and objectives of the Unions external action,
among them, the principles of democracy, the rule of law, and the universality and indivisibility of human
rights and fundamental freedoms as laid down in Article 21 TEU. According to Art. 212 TFEU this also
applies for economic, financial and technical cooperation with third countries other than developing
Union Conference Edition (2010) <http://fra.europa.eu/sites/default/files/fra_uploads/1308-FRA-report-rightschild-conference2010_EN.pdf> accessed 11 November 2014.

53
Further information on content of the Multi-Annual Framework will be given in chapter on relevant EU policy
implementation instruments (II.C.1).
54
European Parliament and Council Regulation (EC) No. 1922/2006 of 20 December 2006 establishing a European
Institute for Gender Equality [2006] OJ L403/9, Art. 4 2.
55
24
FRAME
countries. None of these provisions foresee rules on monitoring, evaluating or measuring human rights.56
In the 1990s, the European Union has started introducing the so-called human rights clause in all
international agreements with third countries, including Free Trade Agreements (FTAs), stating that in
case of violations of human rights the EU might suspend the agreement or parts of it.57 There is however
no mechanism set to objectively measure when a serious violation occurs, leaving this determination to
the parties discretion.58
Art. 8 TEU foresees the development of special relations to neighbouring countries aiming to establish an
area of prosperity and good neighbourliness. According to Art. 8 TEU these relations have to be founded
on the values of the EU, including fundamental rights, still not foreseeing more specific provisions on how
those rights should be guaranteed.
In the enlargement policy, a particular focus is laid on human rights assessment and monitoring: according
to Art. 49 TEU, any European State which respects the values of Art. 2 and is committed to promoting
them, may apply to become a member of the Union. At the Copenhagen meeting in 1993, the European
Council agreed on objective criteria required for EU membership the so called Copenhagen Criteria.59
Besides the economic criteria and the criteria concerning the adoption of the acquis communautaire,
membership requires that the candidate country has achieved stability of institutions guaranteeing
democracy, the rule of law, human rights and respect for and protection of minorities.60 Although the
Copenhagen criteria are not explicitly mentioned, Art. 49 TEU refers to them as preconditions by
stipulating that [t]he conditions of eligibility agreed upon by the European Council shall be taken into
56
These third states can be distinguished according to their status and relation to the EU. Consequently, a
differentiation can be made between third states wishing to accede to the EU, states already in the process of
accession, states being neighbouring countries and further states the Union has relations to. Different types of
(financial) instruments are foreseen for those countries. A list of the currently applied restrictive measures is
regularly updated by the European Commission. The up to date list as of December 2014 is: European Commission,
Restrictive
Measures
in
Force
of
5
December
2014
<http://eeas.europa.eu/cfsp/sanctions/docs/measures_en.pdf>.
57
Christina Churruca Muguruza and others, Report mapping Legal and Policy Instruments of the EU for Human
Rights and Democracy Support (2014) Frame Deliverables, pp. 37-36, <http://www.fp7-frame.eu/wpcontent/materiale/reports/05-Deliverable-12.1.pdf> accessed on 14 October 2014; Lorand Bartels, The European
Parliament's Role in relation to Human Rights in Trade and Investment Agreements (Publications Office of the
European Union 2014); Laura Beke and others, Report on the integration of human rights in EU development and
trade policies (2014) Frame Deliverables, <http://www.fp7-frame.eu/reports> accessed on 11 November 2014.
58
Lorand Bartels, The European Parliament's Role in relation to Human Rights in Trade and Investment Agreements
(Publications Office of the European Union 2014), p. 10; Laura Beke L. and others, Report on the Integration of
Human Rights in EU Development and Trade Policies (2014) Frame Deliverables, pp. 58-71, <http://www.fp7frame.eu/reports> accessed 11 November 2014.
59
European Council, Presidency Conclusions on the European Council Meeting in Copenhagen of 21-22 June 1993
SN 18/1/93 REV. 1.
60
European Council, Presidency Conclusions on the European Council Meeting in Copenhagen of 21-22 June 1993,
SN 18/1/93 REV. 1, para. 7 (A) (iii).
25
FRAME
account. The fulfilment of the political criteria and the adoption, implementation and application of the
acquis are periodically assessed during the accession negotiations.
Fundamental rights have thus become core values of the European Union that need to be considered in
the European Unions internal as well as external action. Only recently however the issue of monitoring
and quantifying the level of protection and fulfilment of fundamental rights have been introduced in the
legal framework. Furthermore fundamental rights are one of the criteria to evaluate potential Member
States preparedness to join the European Union.
2.
Political commitments
The legal analysis showed a discrepancy between the high value attributed to human rights and a
corresponding system to measure whether and how far human rights are guaranteed and safeguarded by
the Member States. However, the issue of measurement is increasingly gaining attention in the European
Union and some important initiatives to strengthen human rights measurement have been undertaken in
the last years (e.g. the creation of FRA and EIGE). It appeared therefore necessary to examine a rapidly
growing body of policy documents in order to evaluate if, and how far the European Unions institutions
have committed themselves to assessing, measuring and evaluating human rights.
In the following section the analysis will focus on the need for concrete policies to measure human rights,
as well as political commitments towards strengthening already existing human rights measurement
tools. Because of the EUs different legal frameworks regarding its Member States, third countries and
accessing countries, the analysis will be structured according to countries status and relation to the EU
(internal, enlargement and external) and according to the different stages within the policy cycle. For the
purpose of this report the different stages of the policy cycle are defined as follows:61
1)
2)
3)
Policy planning: encompasses all the policy stages up to decision making. Policy
planning entails identifying and defining problems, agenda setting, defining policy
objectives, evaluating policy options and finally the adoption of the policy;
Policy implementation and monitoring of the implementation of the policy;
Policy evaluation, which measures the outcome of a policy relative to its initial
purposes.
However, as the term policy cycle already indicates, a distinction between these three stages is only
made for analytical purposes, as in practice these three stages are strictly interrelated. Focus is also kept
on the function that human rights measurement/human rights indicators have within the policy cycle.
Therefore, although some initiatives might fall into one or more stages within the policy cycle, the
distinction is kept up to allow for a more structured overview.
61
Patrice Muller and others, Performance-based Policy (Publications Office of the European Union 2013).
26
FRAME

a)
Internal
(1)
Planning
The idea to check for the compatibility of legislative proposals with the Charter62 and the possible impact
that these might have on fundamental rights was born already in 2001. This is despite the fact the Charter
was at the time not a legally binding instrument. Already in 2002 the Commission had started to evaluate
the possible impact of legislative proposals and those procedures have been strengthened in the years
since.63 Although impact assessment on fundamental rights is still incorporated in the analysis of the
economic, social and environmental impacts, the Commission has recently complemented the Impact
Assessment Guidelines.64 These guidelines provide a framework for assessing the impact of legislative
proposals on fundamental rights.65 The rationale behind this closer scrutiny of legislative proposals has
been explained by the Commission:
Respect for fundamental rights is a legal requirement, subject to the scrutiny of the European
Court of Justice. Respect for fundamental rights is a condition of the lawfulness of EU acts.
(...) The Court requires EU institutions to prove in the light of the fundamental rights
62
In 2001 the Commission stated that: Any proposal for legislation and any draft instrument to be adopted by the
Commission will therefore, as part of the normal decision-making procedures, first be scrutinised for compatibility
with the Charter. European Commission, Application of the Charter of Fundamental Rights of the European Union
(Memorandum) SEC(2001) 380/3, p. 4.
A draft methodology to assess the compatibility of legislative proposals with the Charter was issued in 2005 and
further refined in 2009: European Commission, Compliance with the Charter of Fundamental Rights in Commission
Legislative Proposals. Methodology for Systematic and Rigorous Monitoring (Communication) COM(2005) 172 final;
European Commission, Report on the Practical Operation of the Methodology for a Systematic and Rigorous
Monitoring of Compliance with the Charter of Fundamental Rights COM(2009) 205 final.
63
European Commission, Impact Assessment Guidelines SEC(2009) 92; European Commission, Strategy for the
Effective Implementation of the Charter of Fundamental Rights by the European Union (Communication)
COM(2010) 573 final; On this topic see also: Olivier De Schutter, The Implementation of the Charter by the
Institutions of the European Union in Steve Peers and others (eds), The EU Charter of Fundamental Rights (Hart
Publishing 2014).
64
European Commission, Impact Assessment (Communication) COM(2002) 276.
The Impact Assessment guidelines are currently under revision. In Particular the Impact Assessment Board suggests
that:
Greater efforts need to be made regarding the description of the problem, the assessment of the need
to act, the added value of EU action, and the development of clear alternative options to tackle the
identified problem(s). Efforts also need to be kept up as regards the analysis and where possible
the quantification of impacts, including on SMEs and competitiveness.
European Commission, Impact Assessment Board Report for 2013, p. 8, <http://ec.europa.eu/smartregulation/impact/key_docs/docs/iab_report_2013_en.pdf> accessed 10 November 2014. A study has also
commissioned to review different methods for estimating costs and benefits in its ex ante impact assessment: Renda
A. and others, 'Assessing the Costs and Benefits of Regulation' (CEPS 2013).
65
European Commission, Operational Guidance on taking Account of Fundamental Rights in Commission Impact
Assessment (Commission Staff Working Paper) SEC(2011) 567 final.
27
FRAME
protected by the Charter that they have carefully considered different policy options and
have chosen the most proportionate response to a given problem.66
In this statement, the Commission called for at the very least a qualitative assessment of the impact
of future legislation on fundamental rights, and suggested including in the policy planning stage, the
definition of which indicators should be used for monitoring and evaluating the fundamental rights impact
of policies, as well as what data should be collected for filling in those indicators.67
(2)
Monitoring
The same dynamic that saw the European Communities focusing initially on economic integration, and
only later taking ownership of fundamental rights, can be observed when analysing the political
commitments on monitoring human rights in the internal area: first the focus was drawn on those
fundamental rights more closely linked to the economic and social sphere, while the need to measure
fundamental rights as such only gained attention later, especially with the raising of the Charter of
Fundamental Rights into a full-fledged treaty.
The Lisbon Council in March 2000 defined modernizing the European Social model, investing in people
and combating social exclusion as a strategic goal towards becoming the most competitive and dynamic
knowledge-based economy in the world.68 To this end the Council adopted the Open Method of
Coordination (OMC) for social policies, a governance instrument primarily intended to orienting Member
States policies towards common strategic priorities and towards the adoption of shared action
frameworks for bringing about valuable outcomes.69 Part of this strategy consisted in the adoption of a
common set of indicators to evaluate progress towards commonly agreed objectives. The coordination
process started with the adoption of common objectives by the Member States at the Nice Council in
December 2000 (e.g. facilitating participation in employment and access for all to the resources, rights,
goods and services; preventing the risk of exclusion; helping the most vulnerable; and mobilizing all
relevant bodies).70 These objectives were subsequently transposed into national and regional policies.
Regularly Member States issue reports on the progress achieved, whose results are evaluated and jointly
published by the Member States and the Commission in the form of the Joint Report on Social Protection
and Social Inclusion. The first set of indicators for social inclusion, which had been developed over the
66
Assessment (Commission Staff Working Paper) SEC(2011) 567 final, p. 4.
67
Assessment (Commission Staff Working Paper) SEC(2011) 567 final, pp. 17 and 22.
68
European Council, Presidency Conclusions on the European Council Meeting in Lisbon of 23-24 March 2000 SN
100/1/00, para. 5.
69
On the Open Method of Coordination see: Maurizio Ferrera and Stefano Sacchi, 'The Open Method of Coordination
and National Institutional Capabilities: The Italian Experience' (2004) Urge Working Papers 2/2004, p. 1,
<http://www.urge.it/files/papers/2_1_wp_2_2004.pdf> accessed 17 October 2014; Egidijus Barcevicius and others
(eds.), Assessing the Open Method of Coordination (Palgrave Macmillan 2014).
70
European Council, Presidency Conclusions on the the European Council Meeting in Nice of 8 December 2000 SN
400/1/00.
28
FRAME
years by the Social Protection Committee and its Indicators Sub-group, was adopted by the Laeken Council
in December 2001: it included a provisional list of eighteen indicators on poverty, employment, health
and education, which has been fine-tuned and complemented over time by the Indicator Sub-Group of
the Social Protection Committee.71
In 2010 the Lisbon Strategy was replaced by the Europe 2020 strategy for smart, sustainable and inclusive
growth, also setting targets in the field of employment and social inclusion (e.g. employment rate of 75%
of the population aged 20-64; reduction of people at risk of poverty of 20 million).72 Progress towards the
targets is appraised in the framework of the European Semester where three instruments have already
been developed: the Joint Assessment Framework (JAF), the Employment Performance Monitor (EPM)
and the Social Protection Performance Monitor (SPPM).
In a recent communication, the Commission also suggested that social indicators be strengthened in the
Alert Mechanism Report in order to better reflect the social implications of macroeconomic
imbalances.73 In the same document the Commission suggested the creation of a Scoreboard of key
indicators requiring closer monitoring that should serve as an analytical tool allowing better and earlier
identification of major employment and social problems, especially any that risk generating effects
beyond national borders.74 The first scoreboard of five key employment and social indicators (e.g. the
unemployment rate (15-74 age group), the NEET rate (Not in Employment, Education or Training) in
conjunction with the youth unemployment rate (15-24 age group), real gross household disposable
income and the at-risk-of-poverty rate (15-64 age group) and income inequalities) has been published this
year by the Commission for the first time in the Joint Employment Report accompanying the
Communication from the Commission on Annual Growth Survey 2014.75 In a recent resolution, the
71
European Council, Presidency Conclusion on the European Council Meeting in Laeken of 14 and 15 December
2001 SN 300/1/REV 1, para. 22. On the history of the European Union efforts to measure social inclusion see:
Anthony B. Atkinson and others, Indicators and Targets for Social Inclusion in the European Union (2004) 1 Journal
of Common Market Studies 47; Isabelle Maquet Engsted, The European Context: Measuring Social Inclusion in the
European Union, in Ruggeri Laderchi C. and Savastano S. (eds.), Poverty and Exclusioni n the Western Balkans
(Springer 2013); European Union Agency for Fundamental Rights (FRA), Developing Indicators for the Protection,
Respect and Promotion of the Rights of the Child in the European Union (European Union Agency for Fundamental
Rights 2010), pp. 9197.
72
European Council, Presidency Conclusions on the European Council Meeting of 25-26 March 2010 EUCO 7/10;
European Commission, Europe 2020: A Strategy for Smart, Sustainable and Inclusive Growth (Communication)
COM(2010) 2020 final.
73
European Commission, Strengthening the Social Dimension of the Economic and Monetary Union COM(2013)
690 provisoire, p. 5.
74
European Commission, Strengthening the Social Dimension of the Economic and Monetary Union COM(2013)
690 provisoire, p. 6.
75
See: European Commission, Draft Joint Employment Report accompanying the Communication from the
Commission on Annual Growth Survey 2014 COM(2013) 801 final; European Parliament, Resolution on the
European Semester for Economic Policy Coordination: Employment and Social Aspects in the Annual Growth Survey
2014 of 25 February 2014 (2013/2158(INI)), P7_TA(2014)0129; Council of the European Union, Conclusions on the
2014 Annual Growth Survey and Joint Employment Report: political guidance on employment and social policies of
29
FRAME
Parliament has however demanded the social indicators be made binding for the Macroeconomic
Imbalance Procedure scoreboard, as well as to include additional indicators, in particular child poverty
levels, access to healthcare, homelessness, and a decent work index, in order to allow for proper
assessment of the social situation in the Member States.76
It can thus be said that in the field of employment and social protection there is a clear political
commitment towards the use of indicators: this political commitment has led to the development of a
methodology and a set of commonly agreed indicators for measuring employment and social protection
in the European Union.
In relation to fundamental rights, one of the first initiatives to monitor their status in the EU has been
promoted by the Parliament. The possibility to initiate an Art. 7 procedure and the proclamation of the
Charter, have motivated the Parliament to start regular monitoring of fundamental rights in the EU in
order to be able to identify whether real or potential breaches of Art. 2 occurred.77 The Parliament is
taking stock of the situation of fundamental rights within the European Union78 on a yearly basis and
voiced on many occasions the need to strengthen fundamental rights monitoring, as well as the prominent
role of the FRA in this regard. Although the report itself is not based on indicators, in the 2010-2011 Report
on the Situation of Fundamental Rights in the EU, the Parliament asked for a broader mandate of the FRA,
to cover also the former third pillar of the European Union (police and judicial cooperation in criminal
matters), as well as the establishment of an annual report on the situation of fundamental rights in the
European Union from the Commission.79
The European Parliament Resolution on the Situation of Fundamental Rights in Hungary80 reiterated the
need for a more comprehensive evaluation of EU Member States compliance with Art. 2 TEU, as well as
the discrepancy between a mechanism to evaluate the fundamental rights performances of accessing
countries and the absence of any comprehensive evaluation after accession. It further mentioned the
importance of the Commission as guardian of the Treaties; as the pre-destined body to be entrusted
with the establishment of such a mechanism. In its latest Resolution on the Situation of Fundamental
Rights in the European Union of February 2014, the Parliament urged the Commission to establish a new
10 March 2014; Marion Schmid-Drner, Employment and Social Affairs in the European Parliament (Publications
Office of the European Union 2014).
76
European Parliament, Resolution on the European Semester for Economic Policy Coordination: Employment and
Social Aspects in the Annual Growth Survey 2014 of 25 February 2014 (2013/2158(INI)), P7_TA(2014)0129, para. 4.
77
European Parliament, Resolution on the Situation as regards Fundamental Rights in the European Union (2000)
of 5 July 2001 (2000/2231(INI)) A5-0223/2001, para. 2.
78
European Parliament, Resolution on the Situation as regards Fundamental Rights in the European Union (2000) of
5 July 2001 (2000/2231(INI)) A5-0223/2001.
79
European Parliament, Resolution on the Situation of Fundamental Rights in the European Union (2010 - 2011) of
12 December 2012 (2011/2069(INI)) P7_TA(2012)0500.
80
European Parliament, Resolution on the Situation of Fundamental Rights: Standards and Practices in Hungary
(pursuant to the European Parliament resolution of 16 February 2012) of 3 July 2013 (2012/2130(INI)) P7_TAPROV(2013)0315.
30
FRAME
Copenhagen Mechanism to ensure that the fundamental rights and values of the Union referred to in
Art. 2 of the EU Treaty and in the Charter of Fundamental Rights of the European Union are respected,
protected and promoted in the Member States of the European Union.81 The Parliament suggests that
this mechanism could be activated immediately on the basis of a decision by the Commission. It defines
also the main traits that this new Copenhagen mechanism should possess and further identifies the
Commission and the FRA as main actors to be involved in this process. The Parliament suggests using a
set of indicators based on existing fundamental rights standards (i.e. developed by the UN, the CoE, etc.)
to be developed in consultation with non-governmental organisations, the FRA and the Commission. The
indicators should be based on objective and reliable data and information. This also includes strategies to
make relevant existing data and analysis more accessible and visible. Moreover the Report calls for
objective, comparative and regular assessments for each fundamental right and/or subject area and for
each institution and Member State individually. At the same time, all relevant parties should be striving
for maximum comparability.82
In the field of integration the Council of the European Union highlighted in 2004 that developing clear
goals, indicators and evaluation mechanisms are necessary to adjust policy, evaluate progress on
integration and to make the exchange of information more effective.83 Ten years later the Council, whilst
setting the priorities in the area of Freedom Security and Justice, asked the Commission to help Member
States develop core indicators in the field of integration in order to increase the comparability of national
experiences and reinforce the European learning process.84 Following this call, the ministers responsible
for integration agreed on a core list of indicators that was then approved by the Council.85 The Commission
then ordered a study on the development and use of EU immigrant integration indicators as well as a pilot
study on the availability and quality of data for those indicators that will be discussed later in more detail.86
In the field of justice and home affairs, the Commission has recently started to evaluate the efficiency of
Member States judicial systems. In 2012 the Commission published the first EU Justice Scoreboard,
anchoring it to the European Semester.87 Its findings, based on data collected on a voluntary basis by the
81
European Parliament, Resolution on the Situation of Fundamental Rights in the European Union (2012) of 27
February 2014 (2013/2078(INI)) P7_TA-PROV(2014)0173, para. 8 (o).
82
February 2014 (2013/2078(INI)) P7_TA-PROV(2014)0173, para. 9.
83
Council of the European Union, Council Conclusions on Immigrant Integration Policy in the European Union
14615/04 of 19 November 2004, p. 18.
84
European Council, The Stockholm Programme An Open and Secure Europe Serving and Protecting Citizens,
2010/C 115/01 of 4 April 2010, p. 30.
85
Council of the European Union, Conclusions of the Council and the Representatives of the Governments of the
Member States on Integration as a Driver for Development and Social Cohesion 9248/10 of 4 September 2010, p.
16.
86
Thomas Huddleston and others, Using EU Indicators of Immigrants Integration (Publications Office of the European
Union 2013); EUROSTAT, Indicators of Immigrant Integration: A Pilot Study (Publications Office of the European
Union 2011).
87
European Commission, The 2014 EU Justice Scoreboard (Communication) COM(2014) 155 final.
31
FRAME
European Commission for the Efficiency of Justice of the Council of Europe (CEPEJ) in the EU Member
States, only relates to the efficiency of civil, commercial and administrative proceedings (e.g. length,
number of cases pending, training of judges, perceived independence, etc.).
Concerning the fight against corruption, which is a fundamental part of the rule of law as well as directly
affecting the populations possibility of enjoying other human rights, the Stockholm Programme outlined
the reason why it was important to develop indicators, on the basis of existing systems and common
criteria, (and) to measure efforts in the fight against corruption.88 In response to this call, the Commission
has recently published the first EU Anti-Corruption report.89
As it was discussed earlier, in March 2014 the Commission also presented a New Framework for
Safeguarding the Rule of Law in the EU.90 However, according to the public information available on this
mechanism, no set of indicators on measuring the rule of law has been introduced in the assessment
stage.
In relation to the rights of the child, the Commission has stated that:
Experience with implementing the 2006 Communication has revealed a significant lack of
reliable, comparable and official data. This is a serious obstacle for the development and
implementation of genuine evidence-based policies. Improving the existing monitoring
systems, establishing child rights-related policy targets, and monitoring their impact are one
of the key challenges. Gaps in knowledge about the situation and needs of the most
vulnerable groups of children should be addressed as a matter of priority.91
(3)
Evaluation
On the specific topic of childrens rights the Commission, in the Staff Working Document accompanying
the communication Towards an EU Strategy on the Rights of the Child, has stated that all relevant EU
actions should review their impact on children and that:
The assessment would be made on the basis of a set of appropriate indicators. The indicators
would be both qualitative and quantitative and would cover the internal as well as the
external dimension. They would include, amongst others, the effect on childrens health,
economic situation, education, participation, living conditions and the enjoyment of civil
rights.92
88
European Council, The Stockholm Programme An Open and Secure Europe Serving and Protecting Citizens,
2010/C 115/01 of 4 April 2010, p. 23.
89
European Commission, Fighting Corruption in the EU (Communication) COM(2011) 308 final; European
Commission, Commission Decision establishing an EU Anti-Corruption Reporting Mechanism for Periodic
Assessment ("EU Anti-Corruption Report") C(2011) 3673 final.
90
European Commission, A New EU Framework to strengthen the Rule of Law COM(2014) 158 final/2.
91
European Commission, An EU Agenda for the Rights of the Child COM(2011) 60 final, p. 5.
92
European Commission, Commission Staff Working Document accompanying the Communication from the
32
FRAME
The FRA has taken up this initiative in 2010 by developing a set of indicators to evaluate the impact of
already adopted EU law and policy on childrens status and experience.93
b)
Accession
In order to follow closely progress in each associated country towards fulfilling the conditions of accession
to the Union,94 the Commission developed a methodology for assessing applications for membership that
should ensure that all candidate countries were treated on an equal basis. A composite paper on
enlargement was presented in the Agenda 2000,95 explaining the criteria and methodology used.
The methodology for the assessment of applications developed by the Commission was confirmed by the
Luxembourg European Council in 1997, where two significant elements were introduced in the
Enlargement Strategy. On the one hand, the European Council adopted a comprehensive enlargement
framework, under which the Commission was asked to submit regular reports to the Council assessing the
candidate countrys preparedness for accession in the light of the Copenhagen criteria. For these
assessments the Council decided to follow the method adopted by Agenda 2000 in evaluating applicant
states ability to meet the economic criteria and fulfil the obligations deriving from accession.96 The
progress towards meeting each criterion in terms of legislation and measures actually adopted or
implemented was to be assessed against a standardised checklist that ensured transparency and equal
treatment of all countries aspiring accession to the EU. On the other hand, the Luxembourg European
Council enhanced its pre-accession strategy, introducing the Accession Partnerships and increasing preaccession aid. The new Instrument for Pre-Accession Assistance (IPA) II regulation states that Progress
towards achievement of the specific objectives (...) shall be monitored and assessed on the basis of predefined, clear, transparent and, where appropriate, country-specific and measurable indicators.97
The Cooperation and Verification Mechanism (CVM) was created by the Commission in order to monitor
improvements in the areas of judicial reform and the fight against corruption for Romania and Bulgaria.
This was required because the 2006 Monitoring Report on the state of preparedness for EU membership98
showed that further progress was still necessary. The instrument is based on the Art. 37 and 38 of the Act
Commission - Towards an EU Strategy on the Rights of the Child - Impact assessment COM(2006) 367 final SEC(2006)
88.
93
European Union Agency for Fundamental Rights (FRA), Developing Indicators for the Protection, Respect and
Promotion of the Rights of the Child in the European Union (European Union Agency for Fundamental Rights 2010).
94
European Council, Presidency Conclusions on the European Council Meeting in Copenhagen of 21-22 June 1993
SN 18/1/93 REV. 1, p. 13.
95
European Commission, Agenda 2000: For a Stronger and Wider Union (Communication) COM(97) 2000 final.
96
European Council, Presidency Conclusions on the European Council Meeting in Luxembourg of 12-13 December
1997 SN 400/97, para. 29.
97
European Parliament and Council Regulation (EU) No. 231/2014 of 11 March 2014 establishing an Instrument for
Pre-accession Assistance (IPA II) [2014] OJ L 77/14, p. 14.
98
European Commission, Monitoring Report on the State of Preparedness for EU Membership of Bulgaria and
Romania (Communication) COM(2006) 549 final, pp. 9-10.
33
FRAME
of Accession.99 Under this mechanism, Bulgaria and Romania were requested to report regularly on the
progress made in addressing specific benchmarks in the field of judicial reform and the fight against
corruption.
c)
External100
(1)
Planning
In 2011 the Commission issued a series of communications redefining its policy towards a more tailored
and effective approach to country needs, highlighting among others the importance of evaluating the
human rights performances of third countries:
While the overall objectives of the EUs human rights and democracy policy remain valid
and unaltered, an approach that seeks to match objectives in a country with the realities
on the ground is more likely to deliver concrete results than a one size fits all approach.
Tailor made country strategies covering human rights and democracy should therefore be
an integral part of the EUs overall strategy towards that country. This will help to prioritise
and rationalise work, especially of EU Delegations and Member State Embassies, whilst
better drawing on the relevant mix of EU tools and instruments and working in the areas
most likely to deliver lasting improvements and change.101
In the communication An Agenda for Change, the Commission highlighted the importance of third
countries commitment to human rights, democracy and the rule of law, making the extent and modalities
of aid allocation dependent on third country necessities, capacities and, among others, human rights
performance.102 In this line, focus on human rights was further enhanced with the adoption of the human
rights country strategies, which are based on the EU Delegations quantitative and qualitative analysis of
human rights in third countries.103
For example, in relation to the issue of LGBT rights, the Council has highlighted the importance of
monitor(ing) the situation of the human rights enjoyed by LGBT people in the respective country to
99
Act concerning the Conditions of Accession of the Republic of Bulgaria and Romania and the Adjustments to the
Treaties on which the European Union is Founded [2005] OJ L157/203.
100
For a good overview of the EU policy instruments for human rights and democracy support see the Frame
Deliverable 12.1: Christina Churruca Muguruza and others, Report mapping Legal and Policy Instruments of the EU
for Human Rights and Democracy Support (2014) Frame Deliverables, <http://www.fp7-frame.eu/wpcontent/materiale/reports/05-Deliverable-12.1.pdf> accessed on 14 October 2014.
101
European Commission and High Representative of the European Union for Foreign Affairs and Security Policy,
Human Rights and Democracy at the Heart of EU External Action - Towards a More Effective Approach (Joint
Communication to the European Parliament and the Council) COM(2011) 886 final, p. 8.
102
European Commission, Increasing the Impact of EU Development Policy: an Agenda for Change (Communication)
COM(2011) 637 final, p. 5.
103
Rosa Balfour, The Role of EU Delegations in EU Human Rights Policy (Publications Office of the European Union
2013), p. 16.
34
FRAME
identify progress/setbacks.104 The Council produced a checklist that was further developed into a EU
Guideline a non-binding tool aimed at supporting EU representations to better frame and advance
specific topics in their relations with third countries. The LGBT guidelines suggest a series of indicators and
possible sources of information, such as the existence of systematic practice of torture or abuse by police
officers of LGBTI people or the provision within anti-discrimination law of grounds of sexual orientation
or gender identity.105
(2)
Monitoring
In 2000 the Council welcomed the Commissions proposal to develop common analytical tools as well as
sharing information for Country Strategy Papers (CSPs).106 These were further defined in the European
Commissions Guideline on the Use of Indicators of 2002, aiming to define principles and guidance for
the choice of indicators to be monitored in Country Strategy Papers intervention frameworks.107 In its
communication Human Rights and Democracy at the Heart of EU External Action the Commission even
suggested developing benchmarks to assess progress in human rights, democracy and the rule of law,108
while more recently the Parliament suggested the establishment of a common threshold for Member
States and for EU officials in terms of the human rights concerns that they have to raise, as a minimum,
with their strategic partner counterparts.109
The respect and promotion of human rights as a conditio sine qua non for the EUs cooperation with third
countries has been strengthened with the inclusion of the so-called human rights clause in the EU
agreements with third countries. This clause allows for the suspension of the agreement in case of serious
violations of human rights.110 At the moment there is no mechanism in place to evaluate when to make
use of the human rights clause.111 The Parliament has noted that the human rights clause necessarily
104
Council of the European Union, Toolkit to Promote and Protect the Enjoyment of all Human Rights by Lesbian,
Gay, Bisexual and Transgender (LGBT) People 11179/10 of 17 June 2010, p. 8.
105
Gay, Bisexual, Transgender and Intersex (LGBTI) Persons of 24 June 2013. In particular Annex II of the Guidelines
defines the indicators to be used for assessing the promotion and protection of human rights by LGBI.
106
Council of the European Union Development, Standard Framework for Country Strategy Papers: The Council
Conclusions 13357/00 of 17 November 2000.
107
European Commission, Guidelines for the Use of Indicators in Country Performance Assessment (December
2002), p. 1, <http://ec.europa.eu/development/icenter/repository/guidelines_indicators_cpa_en.pdf> accessed 3
October 2014.
108
109
European Parliament, Resolution on the Review of the EUs Human Rights Strategy of 13 December 2012
(2012/2062(INI)) P7_TA(2012)0504, para. 41.
110
For a good overview on the nexus between trade, development and human rights see: Laura Beke and others,
Report on the integration of human rights in EU development and trade policies (2014) Frame Deliverables,
<http://www.fp7-frame.eu/reports> accessed on 11 November 2014.
111
Lorand Bartels, The European Parliament's Role in relation to Human Rights in Trade and Investment Agreements
(Publications Office of the European Union 2014); Laura Beke and others, Report on the Integration of Human Rights
35
FRAME
requires a clear mechanism for its implementation at the institutional and political levels, and that it
therefore needs to be complemented by an operational enforcement mechanism.112 The Parliament has
also called for the introduction of benchmarks on the protection and promotion of human rights in these
agreements.113
Besides sanctioning mechanisms, the EU is using positive conditionality to strengthen human rights in
development policy.114 Human rights monitoring is used for example by the Commission in the
Neighbourhood Policy (ENP) where the introduction of the more for more mechanism made EU support
dependent on the progress towards the core values of the European Union, which is regularly appraised
through the ENP country reports.115 The ENP country reports build on a list of well established and
accredited human rights indicators systems (e.g. UNDP report, Gender Equality Index, Freedom House
Index, Transparency International Corruption Perception Index, etc.) as well as some structural
indicators116 (e.g. legality of homosexuality, ratification of core labour standards, etc.) to highlight
countries progress and setbacks.117 Similarly the Generalised Scheme of Preferences (GSP) and the GSP+
systems make their incentives dependent on third countries human rights commitment and evaluate
countries progress and setbacks mainly through analysis of the reports submitted to the main treatybased human rights bodies.118 From these few examples it can thus be said that although development
policy is working towards strengthened conditionality and clearer rules in its applicability, there is no
further need identified towards strengthening the use of human rights indicators and benchmarks. In the
recent conclusions on a rights-based approach to development cooperation however, the Council stressed
the importance of a context-specific assessment of the human rights situation, examining the capacity
gaps of both duty bearers to respect, protect and fulfil human rights and of rights-holders to know,
in EU Development and Trade Policies (2014) Frame Deliverables, pp. 59-71, <http://www.fp7-frame.eu/reports>
accessed 11 November 2014.
112
(2012/2062(INI)) P7_TA(2012)0504, para. 47.
113
See for example: European Parliament, Resolution containing the European Parliaments Recommendations to
the Council, the Commission and the European External Action Service on the Negotiations of the EU-Azerbaijan
Association Agreement of 18 April 2012 (2011/2316(INI)) P7_TA(2012)0127.
114
David DHollander and others, Integrating Human Rights in EU Development Cooperation Policy: Achievements
and Challenges in Wolfgang Benedek and others (eds), European Yearbook on Human Rights 14 (Intersentia 2014).
115
European Commission, A New Response to a Changing Neighbourhood (Communication) COM(2011) 303, p. 3.
116
Structural indicators measure the de jure protection of specific rights and thus, they give information on the
commitment of a state or government.
117
Implementation of the European Neighbourhood Policy: Statistical Annex (Joint Staff Working Document)
SWD(2014) 98 final.
118
European Parliament and Council Regulation (EU) No. 978/2012 of 25 October 2012 Applying a Scheme of
Generalised Tariff Preferences and repealing Council Regulation (EC) No 732/2008 [2012] OJ L 303/1; Laura Beke
and others, Report on the Integration of Human Rights in EU Development and Trade Policies (2014) Frame
Deliverables, pp. 33-35, <http://www.fp7-frame.eu/reports> accessed 11 November 2014
36
FRAME
exercise and claim their rights, with a view to identifying the root causes of poverty and social
exclusion.119
(3)
Evaluating
In its Resolution on the Annual Report on Human Rights and Democracy in the World 2008, the Parliament
called for a regular periodic assessment of the use and the results of European Union policies, instruments
and initiatives on human rights in third countries and to develop specific quantifiable indices and
benchmarks in order to measure the effectiveness of those policies.120 Following this call, the Strategic
Framework on Human Rights and Democracy and its annexed Action Plan were adopted in 2012 and
constitute key documents for the strengthening of the EU human rights policy in third countries.121 The
Action Plan transforms the policy goals of the Strategic Framework into concrete actions, identifying not
only the EU institutions responsible for its implementation, but also a time-frame in which the action
needs to be completed. Progress made by the European institutions in pursuing the objectives of the
Strategic Framework is appraised in the Annual Report on Human Rights and Democracy in the World. 122
The importance of a sound monitoring of human rights for policy reasons is highlighted in different parts
of the Action Plan, mainly as an instrument for making more justifiable decisions when delivering aid or
concluding trade and investment agreements.123
Notwithstanding improvements in the field of policy evaluation, the Parliament regrets, nevertheless,
that the country reports still seem to lack a systematic, clear and coherent framework that would allow
for more rigorous analysis on the impact and efficiency of EU action,124 and generally calls for
benchmarking policy for all instruments (including geographical policies and strategies) which should be
able to measure and monitor respect for human rights and democratic principles based on specific,
transparent, measurable, achievable and time-bound indicators.125
119
Council of the European Union, Conclusion on a Rights-Based Approach to Development Cooperation,

encompassing All Human Rights of 19 May 2014, para. 5.
120
European Parliament, Resolution on the Annual Report on Human Rights in the World 2008 and the European
Unions policy on the matter (2008) of 7 May 2009 (2008/2336(INI)) P6_TA(2009)0385, para. 6.
121
Council of the European Union, EU Strategic Framework and Action Plan on Human Rights and Democracy
11855/12 of 25 June 2012. For an overview of the genesis of the Strategic Framework and Action Plan see: Engelbert
Theuermann, The Review of the EU Human Rights Policy: A Commitment to Strengthened EU Action on Human
Rights (2013) European Yearbook on Human Rights 31.
122
Council of the European Union, EU Annual Report on Human Rights and Democracy in the World in 2013
11107/14 of 23 June 2014.
123
See for example: Council of the European Union, EU Strategic Framework and Action Plan on Human Rights and
Democracy 11855/12 of 25 June 2012, point IV, 10 and 11; Jarmo Oikarinen, Reform of the EU Human Rights Policy
and the Challenge of Implementation (2014) European Yearbook on Human Rights 115.
124
European Parliament, Resolution on the Annual Report on Human Rights and Democracy in the World 2012 and
the European Union's Policy on the Matter of 11 December 2013 (2013/2152(INI)) P7_TA(2013)0575, para 9.
125
(2012/2062(INI)) P7_TA(2012)0504, para. E.
37
FRAME
In relation to development policy, the European Commission suggested in an Agenda for Change the
introduction of a common EU results reporting framework:
for measuring and communicating the results of development policy, including for inclusive
and sustainable growth. In line with the Operational Framework on Aid Effectiveness, the EU
will work with partner countries and other donors on comprehensive approaches to domestic
and mutual accountability and transparency, including through the building of statistical
capacity.126
Indicators are also used in different programmes, like the European Instrument for Democracy and Human
Rights (EIDHR), to evaluate the impact of projects financed by the European Commission on human rights.
In the new Multiannual Financial Framework (MFF) for 2014-2020, when it comes to allocating funds to
partner countries, the EU aims for donor-recipient mutual accountability, stating that indicators will be
used to evaluate the efficiency of the financial instruments.127
3.
Conclusions
Looking at the historical development of the EU, it can be said that although the promotion of peace,
stability and human rights in the European continent was the reason behind the European project, the
European Communities only slowly took ownership of human rights; through the ECJs jurisprudence
human rights were taken up and brought forward within the Communities up to their inclusion in the
treaties.
Given the mandates of the EMCO and the SPC, one may conclude that the TFEU entails a certain obligation
to monitor although not to measure some issues related to economic and social rights in areas that
have been on the agenda of the European Communities since their establishment (e.g. right to work,
poverty, etc.). By way of secondary law, the EU has equipped itself with two institutions that have an
explicit mandate to collect objective, reliable and comparable data on fundamental rights and gender
equality within the EU Member States. This is certainly already a great achievement. However, while
fundamental rights today constitute core values of the EU and compliance with human rights is generally
required, there is no provision in the treaties foreseeing an instrument for measuring this compliance.
The policy analysis showed however that European Union institutions are increasingly calling for or
developing initiatives to strengthen human rights scrutiny within and outside the European Union. Thus,
a trend towards the creation or strengthening of human rights measurement systems can be identified.
This trend is in line with an effort called for by many human rights stakeholders and academia, i.e. to make
(the progress in the realisation of) human rights more quantifiable. Human rights measurements are
126
European Commission, Increasing the impact of EU Development Policy: an Agenda for Change (Communication)
COM(2011) 637 final, p. 11.
127
European Commission, The Multiannual Financial Framework: The External Action Financing Instruments
(Memo) MEMO/13/1134 of 11 December 2013; Council Regulation (EU, EURATOM) 1311/2013 of 2 December 2013
laying down the Multiannual Financial Framework for the Years 2014-2020 [2014] OJ L 347/884.
38
FRAME
required for both the EUs internal and external policies, as a matter of coherence and consistency. It
became also apparent that, at the very least, a strengthened monitoring of human rights is called for in a
wide variety of policies in the internal, enlargement and external spheres. Furthermore it became
apparent also that human rights measurement tools support/should support all phases of the policy cycle.
B.
Purposes of human rights measurement
This chapter discusses the purposes of current and possible future human rights measurements. The aim
is to find out who needs human rights measurement for what reason. The purposes of human rights
measurement will be analysed from the perspective of the users, namely the interviewed EU officials. At
the beginning of each interview, the following definition was shared with the interviewees:
The term mechanism to measure human rights denotes any kind of mechanism that resorts
to qualitative and/or quantitative indicators to measure the current state and/or progress of
particular human rights or impact in a defined geographical area within a given time-frame.
This includes, for instance, human rights benchmarks, indicators, data on progress or impact,
etc. Such a mechanism does not necessarily have to be a stand-alone initiative, but could also
be a (minor) part of a policy.
Despite this given definition, the perceptions of human rights measurement differ among the
interviewees. For instance, some interviewees understand human rights measurement as evaluation of
human rights policies, while others understand it as human rights-relevant data gathering. Thus, the
information provided by the interviewees cannot be regarded as exhaustive. Nevertheless the results
shed light on the importance of current and future measurements for EU officials from a practical
perspective.
The previous chapter on the political commitment to measure human rights shows that human rights
measurement tools support, and are needed in this capacity throughout all phases of the policy cycle.
Thus, the discussion of purposes of human rights measurements will be structured along these stages,
namely policy planning, policy implementation (which practically means assessing compliance of countries
with human rights standards and decision making in this regard) and policy evaluation. Whenever the
purposes differ, the analysis will be differentiated by policy area, i.e. internal, external and enlargement
policies.
1.
Policy planning
This section discusses the way in which human rights measurement is needed for the development and
assessment of policy proposals. At the stage of planning, human rights measurement has the overall
purpose of translating the fundamental values and principles of the EU into concrete measures.
Additionally it is needed to provide factual evidence for policy planning. As will be shown in more detail,
human rights measurement also serves the purpose of strengthening a coherent and consistent EU human
rights policy in both the EUs internal and external areas.
39
FRAME

a)
Provide evidence for policies
According to some interviewees, human rights measurement as defined in the introduction of this section,
as well as more general research on human rights, is required in order to help identify various issues as
human rights-related problems. This is true in different phases of policy planning, namely for the
identification of a problem and challenges, and the reasons for intervention and for the assessment of
policy options. Human rights measurement provides the empirical basis that is needed for evidence-based
policy planning and can serve as the basis for arguments on policy proposals or reforms.128 Some
interviewees mention that this is of particular relevance against the background of the broader trend
towards more evidence-based policies.129 Additionally, human rights measurement has the purpose of
discovering good practices to inform policy making.130 Human rights measurement thus initiates and
informs the process of developing policy proposals in various ways.
When it comes to already existing policy proposals, human rights measurement in the EUs internal area
has the purpose of assessing the impact of these proposals on fundamental rights. The Commission has
already provided guidance to its departments on how to assess the impacts of legislative proposals on
fundamental rights.131 It is a clear purpose of such impact assessments to help EU institutions designing
policies and laws in compliance with the Charter of Fundamental Rights. Additionally it facilitates better
informed decision making throughout the entire legislative process. Thus, at the stage of policy planning,
proposals are scrutinised for their compliance with the Charter of Fundamental Rights.132 The European
Commissions Impact Assessment Guidelines make clear that this approach helps ensure coherence of the
European Commissions policies and consistency with treaty objectives, such as the respect for
fundamental rights.133 The interviewed officials dealing with EU internal policies indeed confirmed the
necessity of measuring human rights to assess the impact of legislative proposals on human rights.134
As it was discussed earlier, the EU is currently turning to a country specific approach in external area that
seeks to better match objectives in a country with the realities on the ground. The underlying assumption
is that such an approach is more likely to deliver concrete results than a one size fits all top-down
approach.135 Human rights measurement can support this turn to more tailor-made country strategies
and help in achieving a maximised impact on the ground. More concretely, human rights measurement in
128
Interview conducted by the ETC with a member of the European Parliaments Committee FEMM (Brussels, June
2014).
129
Interview conducted by the ETC with an EU official from the DG Home (Brussels, June 2014).
130
Interview conducted by the ETC with a member of the FRA (Vienna, June 2014).
131
European Commission, Operational Guidance on taking account of Fundamental Rights in Commission Impact
Assessment (Commission Staff Working Paper) SEC(2011) 567 final, pp. 3-5.
132
European Commission, Report on the Practical Operation of the Methodology for a Systematic and Rigorous
Monitoring of Compliance with the Charter of Fundamental Rights COM(2009) 205 final.
133
European Commission, Impact Assessment Guidelines SEC(2009) 92, pp. 4-6.
134
Interview conducted by the ETC with an EU official from the DG Home (Brussels, June 2014).
135
Communication to the European Parliament and the Council) COM(2011) 886 final, pp. 5-8.
40
FRAME
external area has the purpose of informing decision-making on topics, as well as focussing target groups
and instruments of policies.136 Interviewees dealing with human rights in the EU external area point out
the need for accessible qualitative and quantitative information about the situation on the ground to
inform policy making.137 Here, previous research has shown that human rights measurement can serve
the function of contextual description and documentation.138 This includes gathering locally-based and
rich descriptive statistics, which cover human rights violations. In addition, data on the perceptions of
experts, as well as on the activities of state and non-state actors which have a bearing on human rights is
included.
Interviewees emphasised particularly, in regards to the EUs external action that one single set of
indicators is not reasonably applicable, since situations, programmes, topics and countries are too
different.139 Contextual indicators in country programming are relevant but they cannot be found in
practice, except for human rights indicators related to gender, children and LGBT persons. In addition,
given the new efforts and tools relating to a rights-based approach, there is a need to use (contextual)
human rights indicators. Also the stronger emphasis on ESCR necessitates thinking on how these are
measured.
b)
Strengthen coherence of EU human rights policies
Interviewees identify a lack of coherence related to the EU internal and external human rights strategy140
and some of them even worry about the credibility of the EU in this regard. Consequently, contributing to
the coherence and consistency of the EUs human rights policies was identified as an important purpose
of a mechanism to measure human rights. An objective, comparative and regularly applied human rights
measurement that assesses the compliance with fundamental rights by every Member State in an
effective and binding manner would make the EUs human rights policies in external action and
enlargement more coherent with the internal area. Externally it would make the EUs calls for action more
credible.141 In this respect also interviewees pointed out that the implementation of human rights
measurement at EU level would make the EUs human rights strategy in external action more coherent
and thus more credible.142 Interviewed EU officials, whose mandate explicitly deals with human rights,
mentioned that more coherence created by such a mechanism would make the Commissions call for
136
Interview conducted by the ETC with official from the European Parliament Subcommittee DROI (Graz, June 2014)
and from EEAS/COHOM (Brussels, June 2014).
137
Interview conducted by the ETC with EU officials from EEAS/COHOM and DG DEVCO (Brussels, June 2014).
138
Todd Landman and Edzia Carvalho, Measuring Human Rights (Routledge, 2010), pp. 4-5.
139
Interview conducted by the ETC with an EU official from the DG DEVCO (Brussels, June 2014).
140
Interviews conducted by the ETC with EU officials from the EEAS, DEVCO, FRA and LIBE (Brussels and Graz, June
2014).
141
European Parliament, Resolution on the Situation of Fundamental Rights in the European Union (2012) of
February 2014 (2013/2078(INI) P7:TA-PROV(2014)0173, pp. 15-16.
142
Interviews conducted by the ETC with EU officials from LIBE and DG HOME (Brussels, June 2014).
41
FRAME
action more assertive.143 Finally, the need for coherence and consistency between the EUs external and
internal human rights policies are addressed in the Action Plan on Human Rights and Democracy.144
2.
Implementing policies and assessing compliance with human rights
standards
The information provided by the interviewees clearly indicates that human rights measurement is needed
at the policies implementation stage and even more so when assessing countries compliance with
human rights standards. Human rights measurement is required by interviewees for the purpose of a
more consistent implementation of policies and a more standardised human rights assessment. In the
EUs internal area, human rights assessment concerns the compliance of Member States with the TEU and
the Fundamental Rights Charter. In the EU enlargement area, human rights measurement is needed to
support the assessment of compliance of candidate countries with the aquis communautaire and the
Copenhagen Criteria. In the EU external area, human rights measurement is used to assess the human
rights situation of those third countries benefiting from development aid, budgetary support or general
preferential EU market access schemes.145
The importance of human rights measurements for monitoring the degree to which states respect, protect
and fulfil the various rights over time is also discussed in the literature. The availability of human rights
measurement increases the possibility of making more profound analytical statements about the human
rights situation and about changes, progress or regress in this regard.146 Human rights measurement thus
has the function of contextual description and documentation. To achieve this, data on the ground needs
to be converted into systematic analysis.147
a)
Provide standardised tools for assessing compliance with
fundamental rights
Interviewees assessing Member States compliance with human rights standards and obligations point
out their need for human rights measurement including statistical data regarding the situation on the
ground.148 A set of standardised indicators to measure human rights has the purpose of improving the
quality of the assessment.149
143
Interview conducted by the ETC with officials of the European Parliament Subcommittee DROI (Brussels, June
2014).
144
11855/12 of 25 June 2012.
145
Interviews conducted by the ETC with EU officials from INTA (Brussels, June 2014).
146
Todd Landman and Edzia Carvalho, Measuring Human Rights (Routledge, 2010), p. 5.
147
Todd Landman and Edzia Carvalho, Measuring Human Rights (Routledge, 2010), pp. 4-5.
148
Interview conducted by the ETC with an EU official from DG Home (Brussels, June 2014).
149
Interviews conducted by the ETC with EU officials from LIBE and DG HOME (Brussels, June 2014).
42
FRAME
When preparing the Annual Report on the Situation of Fundamental Rights in the European Union, the
Parliament currently follows rather general guidelines.150 The interviewed officials of the European
Parliament Committee on Civil Liberties, Justice and Home Affairs thus emphasised the need to strengthen
human rights measurement in the EU internal area. They further asserted that by using human rights
measurement, monitoring fundamental rights in the EU and in the individual Member States can be
carried out more objectively and comparably.151 In its recent annual report, the Parliament called for the
immediate activation of a new Copenhagen mechanism.152 In the interview, officials of LIBE support this
claim but acknowledge that the comparability of Member States data is an important precondition for its
effective implementation. They believe that the implementation of human rights measurement in the EU
would make the Parliaments fundamental rights reports better informed and thus less vulnerable to
attacks at the political level.153 The importance of a shared set of comparable data and a common
methodology for assessing fundamental rights in the EU has also been emphasised by the interviewed
member of the Social Protection Committee.154
In addition, interviewees identified a risk of applying double standards in the current way of assessing
fundamental rights in the EU Member States: when it comes to assessing compliance with the EUs
fundamental rights and values, some Member States might feel to be under more scrutiny than others.
These interviewees believe that objective and comparable indicators on the compliance of Member States
with the Fundamental Rights Charter can help to avoid double standards in the internal application of
fundamental rights assessment tools.155
Currently, fundamental rights assessments of EU accession candidates are carried out in accordance with
the Enlargement Strategy and the Copenhagen criteria. The Copenhagen criteria have the purpose of
guaranteeing an adequate level of fundamental rights protection and to identify weak spots in the factual
enjoyment of fundamental rights in candidate countries. This purpose is clearly related to human rights
measurements. Interviewees dealing with EU enlargement point out that a mechanism to measure human
rights is needed to support the assessment of candidate countries regarding their capacity to implement
the acquis communautaire. They perceive this as particularly necessary since the tools to measure the
implementation of the acquis communautaire are not very elaborate in certain areas, such as the rule of
law. While the rule of law and the assessment of the candidate countries progress are core issues in the
150
Interview conducted by the ETC with officials of the LIBE Committee of the EP (Brussels, June 2014).
February 2014 (2013/2078(INI)) P7_TA-PROV(2014)0173, pp. 15-17.
152
February 2014 (2013/2078(INI)) P7_TA-PROV(2014)0173, pp. 15-16.
153
Interview conducted by the ETC with officials of the LIBE Subcommittee of the EP (Brussels, June 2014).
154
Interview conducted by the DIHR with members of the Indicators subgroup of the Social Protection Committee
(Copenhagen, June 2014).
155
Interview conducted by the ETC with EU officials from LIBE (Brussels, June 2014).
151
43
FRAME
enlargement strategy,156 truly measurable indicators are missing for both.157 Although benchmarks for
assessing the rule of law became available through the revision process of the accession tools, and
although the dialogue with candidate countries has been intensified, indicators are not available yet. Thus,
when assessing candidate countries performance, the European Commission still relies on the acquis, the
(good) practices of the Member States and standards set by international organisations like the CoE and
the UN. However, the Commission uses indicators occasionally. These indicators usually are structural
indicators measuring the progress in adapting the legal framework to the acquis. Although indicators are
lacking in the EU enlargement process, missions indeed receive detailed terms of reference from the
European Commission.
When assessing rule of law issues and providing candidate countries with related recommendations, DG
Enlargement intends to ensure a consistent and coherent approach.158 Thus, interviewees pointed out
that human rights measurement has the general purpose of ensuring a fair and strict negotiation policy
with candidate countries in order to create a coherent accession process. To achieve this purpose, a
common methodology needs to be applied and benchmarks based on best practices in Member States
need to be developed.159 Generally, it is found that a standardised human rights measurement supports
the implementation of the EU enlargement strategy.160
Human rights measurement has the purpose of creating more coherence and consistency when assessing
fundamental rights in different countries, irrespective of whether they are Member States or candidate
countries.161
b)
Provide evidence for incentives and sanctions in EU external action
The EU Strategic Framework and the corresponding Action Plan emphasise the joint responsibility of the
EU and its Members States in the promotion of human rights and democracy abroad. This encompasses
the promotion of human rights in all areas of external action, including strengthened efforts to assist
partner countries in implementing their human rights obligations162 and a differentiated approach to
development aid in order to reach maximum impact and value for money.163
Interviewees, who deal with the EUs external area, pointed out a need for benchmarks, and indicators
of progress and achievement in third countries human rights obligations. Human rights measurement is
156
European Commission Enlargement Strategy and Main Challenges 2013-2014 COM(2013) 700 final, pp. 2-6.
Interview conducted by the ETC with an EU official from the DG Enlargement (Brussels, June 2014).
158
Interview conducted by the ETC with an EU official from the DG Enlargement (Brussels, June 2014).
159
Interviews conducted by the ETC with EU officials from DG Enlargement (Brussels, June 2014).
160
Interviews conducted by the ETC with EU officials from DG Enlargement (Brussels, June 2014).
161
Interview conducted by the ETC and the DIHR with an EU official from the EEAS (Brussels, June 2014).
162
11855/12 of 25 June 2012.
163
European Commission, Increasing the impact of EU Development Policy: an Agenda for Change (Communication)
COM(2011) 637 final.
157
44
FRAME
also required to provide a tool for measuring the commitment of a government to human rights.164 Thus,
a mechanism to measure these dimensions (commitment, progress/regress and achievement) of human
rights in third countries has the purpose of facilitating the decision-making process of EU officials in
external action.
Further, human rights measurement in external action has the key purpose of supporting the application
of human rights related tools of EU policies (e.g., GSP+, Human Rights dialogues, Human Rights country
strategies) correctly and to inform decision-making regarding the provision of incentives, the (temporary)
withdrawal of incentives and the imposition of sanctions.165 This is also related to human rights clauses in
free trade agreements and other policies, which are based on incentives for the ratification and
implementation of fundamental rights obligations. In addition, human rights measurement is needed to
provide evidence for the EUs decision making on imposing sanctions.166
3.
Evaluation of human rights policies
Interviewees working in external area pointed out that the application of human rights policies needs to
be adjusted through the findings of human rights measurement. Human rights measurement therefore
needs to be applied in order to find out whether the EU policies are coherent with the EUs strategic
tools.167
Interviewees dealing specifically with human rights in external area additionally mentioned a need for
human rights measurement for the evaluation of certain human rights policies or projects. Thus, human
rights measurements serve to make human rights policies more (country-) specific. These interviewees
mention concrete measures to be evaluated, namely the human rights dialogues, the countries human
rights strategies and urgent actions on human rights.168 These issues are highlighted here because the
interviewed EU officials perceive these instruments as not sufficiently standardised and some (particularly
interviewed members of DROI and COHOM) even doubt their effectiveness.169 Interviewees noted that,
for instance, no overall checklist is publicly available for the human rights dialogues, while checklists are
available for example in the United States. Therefore, regarding the human rights dialogues, interviewed
EU officials call for an impact assessment in the sense of an evaluation.170 Additionally, according to the
interviewed members of DROI and COHOM, the human rights country strategies need to be standardised
164
Interview conducted by the ETC with an official of DG DEVCO (Brussels, June 2014).
Interviews conducted by the ETC with EU officials from the EEAS/COHOM and DG DEVCO (Brussels and Graz, June
2014).
166
For more information see: Laura Beke and others, Report on the integration of human rights in EU development
and trade policies (2014) Frame Deliverables, <http://www.fp7-frame.eu/reports> accessed on 11 November 2014.
167
Telephone interview conducted by the ETC with an EU official from the EEAS (Graz, June 2014).
168
Interview conducted by the ETC and DIHR with an EU official from EEAS (Brussels and Copenhagen, June 2014).
169
See also: Katrin Kinzelbach, The EUs Human Rights Dialogue with China: Quiet Diplomacy and its Limits,
(Routledge, 2014).
170
Telephone and face-to-face Interviews conducted by the ETC with a member and officials of the Parliaments
Committee DROI and with an EU official from COHOM (Graz and Brussels, June 2014).
165
45
FRAME
and improved concerning quality, scope and retraceability.171 Some EU officials furthermore believe that
an overall assessment of the implementation of the EU Strategic Framework and Action Plan on Human
Rights and Democracy through benchmarks and indicators has to be considered in the future.172
The interviewed officials of DG Enlargement call for an evaluation of the EU enlargement policies.
According to them, human rights measurement is a suitable mechanism for achieving such an evaluation.
The overall purpose of this evaluation would be to find out whether policies lead to changes in the
candidate countries human rights situation.173
Information on the situation, progress and impact of policies needs to be integrated into the analysis.174
The importance of information on the situation, progress and impact of policies was also highlighted by
the interviewed member of the FRA.175 Since the evaluation of policies and projects is required to allow
the identification of causality between measure and impact, it has to be discussed whether human rights
measurement is needed for project management in the sense of project monitoring. For example,
although benchmarks along core obligations of a specific right are indeed useful for project monitoring,
project management tools and scientific work provide valuable methods for project evaluation. In this
respect, secondary analysis can be used to examine the degree to which a policy intervention has had a
direct or contributory effect on a particular human rights situation. Literature on human rights
measurement suggests for example the application of social science methods, namely hypothesis-testing
and impact assessment for this purpose.176 The significance of relationships between and among human
rights and other variables (i.e. level of economic growth and inequality) needs to be tested in order to
achieve the purpose of project evaluation.
4.
Conclusion
The aim of this chapter was to find out who needs human rights measurement and for what reason(s).
The answer to that question is that the majority of the interviewees are convinced that human rights
measurement would be useful for their work. Regarding the question for what reason human rights
measurement is needed the overall conclusion is that human rights measurement is needed for
evidence-based policy making. Human rights measurement serves to create more consistency and
coherence between the internal and external human rights strategy of the EU and to make human rights
assessments more standardised and coherent in order to avoid double standards.
The findings of the interviews allow the identification of purposes for human rights measurement in all
stages of the policy cycle.
171
Interview conducted by the ETC with an EU official from COHOM (Brussels, June 2014).
Interview conducted by the ETC with an EU official from COHOM (Brussels, June 2014).
173
Telephone interviews conducted by the ETC with EU officials from DG Enlargement (Graz, June 2014).
174
Measurement and Implementation (United Nations 2012), pp. 33-70.
175
Interview conducted by the ETC with a member of the FRA (Vienna, June 2014).
176
Todd Landman and Edzia Carvalho, Measuring Human Rights (Routledge, 2010), p. 5.
172
46
FRAME
In policy planning human rights measurement is needed to inform the development of policy proposals
with evidence. Human rights measurement allows the identification of good practices as well as weak
spots and needs to be addressed in policy planning. Furthermore, human rights measurement in policy
planning supports the ex ante impact assessment of policy options.
At the stage of implementation, human rights measurement helps EU officials to apply human rights
policies and monitoring tools more effectively and coherently.
Additionally, human rights measurement supports decision-making in the course of monitoring countries
compliance with human rights standards and obligations, for instance decisions on incentives or sanctions
in external action. More generally, human rights measurement makes political decisions more evidencebased.
At the stage of evaluation, human rights measurement serves to tailor policies and assessing the human
rights impact of policies.
The parameters of meaningful and applicable human rights measurements have to be critically assessed
in light of the specific requirements of the diverse internal and external policies of the EU. In addition, the
experiences of EU officials who currently apply instruments to measure human rights, have to be taken
into account. Thus, the next section is dedicated to human rights measurements currently applied and to
the requirements for future measurements deriving from them.
47
FRAME
C.
Human rights measurements by EU bodies: current applications
and future requirements
1.
Mapping of existing applications of human rights measurement tools
This section analyses in more detail how instruments which are deemed to give information on human
rights are currently applied by the EU. To this end a broad mapping of instruments established by EU
institutions will be presented and a particular focus will be given to their methodology.
The mapping discusses instruments comprising elements of human rights protection. As introduced in the
previous chapters, the findings will be structured according to the EU (a) internal, (b) enlargement and (c)
external area.177 Furthermore, the analysis is structured according to the functions of these instruments
in the policy cycle as introduced in section II A. Therefore a closer look will be given to the stages of (1)
policy planning, (2) monitoring, as well as (3) policy evaluation. Some of the analysed instruments cannot
always be clearly attributed only to one stage of the policy cycle, as their results also inform other stages.
The policies and instruments will be presented and discussed regarding their scope, how they approach
human rights (i.e. which dimension is taken into account, respect for human rights (HRs), performance
regarding HRs, analysing the HRs situation on the ground, etc) and how they are analysed (i.e. which data
is used, who generates the information, etc). Furthermore it will be discussed whether there is already a
coherent approach of the EU regarding measuring human rights. Is there one underlying concept or are
the policies and instruments highly diversified?
a)
The internal sphere of the EU
This subchapter addresses instruments which are deemed to give information on fundamental rights in
the EU internal area. FRA and EIGE will be addressed twice, since they have instruments in place for policy
planning as well as for monitoring.
(1)
Policy Planning
With the FRA and EIGE, the EU has established two institutions to provide research, data and expertise on
the fundamental rights situation within the EU. As pointed out in section II B, such data is very useful and
is required for evidence-based policy planning. Additionally, in the preparatory stages of new legislation,
the European Commission foresees an ex ante impact assessment of policy options regarding fundamental
rights. The FRA, the EIGE and the European Commissions ex ante impact assessment will therefore be
described in more detail below.
The Fundamental Rights Agency
The FRA was established in 2007 with the specific task to
177
Please note that Annex II of this report presents three case studies on the use of rule of law indicators in the EU
enlargement process of Bulgaria, Croatia and Montenegro.
48
FRAME
provide the relevant institutions, bodies, offices and agencies of the Community and its
Member States when implementing Community law with assistance and expertise relating
to fundamental rights in order to support them when they take measures or formulate
courses of action within their respective spheres of competence to fully respect fundamental
rights.178
According to Art. 5 of the Regulation establishing the FRA, the Council of the EU shall adopt a Multi-annual
Framework for the Agency, which shall cover five years and determine the thematic areas of activities of
FRA. The fight against racism, xenophobia and related intolerance must always be included.
The FRA plans its work programme three years in advance. It is thereby supported by its Scientific
Committee and stakeholders through continuous dialogue and consultation. This allows the FRA to carry
out multi-annual research projects, following a cross-departmental approach. Research and analysis is
carried out by the departments Equality and Citizens Rights and Freedoms and Justice, as well as by
project stakeholders. Based on this research, the FRA develops opinions and expert advice. The
communication department then undertakes awareness raising activities and facilitates cooperation and
engagement with the FRAs key partners., such as the national Liaison Officers, EU institutions, the
Fundamental Rights Platform, National Human Rights Institutions (NHRIs), equality bodies and
ombudsperson institutions.179
In July 2011, the FRA installed a multidisciplinary research network, the FRANET network.180 It consists of
research institutes so called National Focal Points in all EU Member States, which are contracted by
the FRA to gather and present information and data on the fundamental rights situation in each Member
State. FRANET provides data for annual reports on the topics of the MAF. The current topics are access to
justice, victims of crime, information society, Roma integration, judicial cooperation (except in criminal
matters), rights of the child, discrimination, immigration and integration of migrants, visa and border
control and asylum and racism, xenophobia and related intolerance. The data provided by FRANET on
these topics is then collected and comparatively analysed by the FRA.
The methodology applied is a combination of legal analysis and data on the experiences of individuals on
the ground. Thus, desk research is complemented with social fieldwork research (surveys, interviews or
secondary analysis of existing data). The FRA publishes the results of this socio-legal research in reports
to the EU institutions and Member States. Additionally, the FRA prepares training materials and training
programmes, and collects and shares promising practices. Promising practices are those that have a
178
Council Regulation (EC) No. 168/2007 of 15 February 2007 establishing a European Union Agency for Fundamental
Rights [2007] OJ L 53/1.
179
European Union Agency for Fundamental Rights, How we do it < http://fra.europa.eu/en/about-fra/how-we-doit> accessed 25 September 2014. Prior to this network, FRA worked from 2007 until 2011 with a network of legal
experts called FRALEX.
180
European Union Agency for Fundamental Rights, FRANET <http://fra.europa.eu/en/research/franet> accessed
25 September 2014.
49
FRAME
beneficial effect on fundamental rights among the Member States. They aim at offering Member States
the opportunity to learn about successful solutions from others.181
In addition the FRA itself gathers data. This is usually quantitative research on one fundamental rights
topic, which is examined across the EU. Such fundamental rights topics are homophobia, transphobia, and
discrimination on grounds of sexual orientation and gender identity in the EU Member States (2010), or
violence against women (2014).182
The European Institute for Gender Equality
The EIGE was established in 2007, first in Brussels then moving to Vilnius. The official launch of EIGEs
activities however only took place in June 2010. EIGE was established as an autonomous body with the
tasks of contributing to, and strengthening the promotion of gender equality and gender mainstreaming
in all EU policies as well as the fight against discrimination based on sex, and raising awareness among
citizens. Similar to the FRA, EIGE has been entrusted with the explicit task of providing EU institutions and
Member States with independent, evidence-based advice on fundamental rights and gender equality,
therefore acting within the area of policy planning.183
European Commissions ex ante Impact Assessment
The ex ante Impact assessment of the European Commission has the purpose to assess policy options and
their possible impact on fundamental rights: it covers only legislative proposals made by the Commission,
while amendments to their texts are assessed in a later stage by the legislative bodies.184
The European Commission Impact Assessment Guidelines185 are meant to help find evidence on
advantages and disadvantages of policy options.186 The guidelines include different checklists
encompassing formal requirements of the Impact Assessment, such as procedural and analytical steps
and exemplary questions to ask. In 2011 the Commission issued a Staff Working Paper on how to identify,
compare and screen policy options impacts on fundamental rights.187 This operational guidance is one of
181
European Union Agency for Fundamental Rights, How we do it < http://fra.europa.eu/en/about-fra/how-we-doit> accessed 25 September 2014.
182
European Union Agency for Fundamental Rights, Using Indicators to measure Fundamental Rights in the EU:
Challenges
and
Solutions
(2011)
FRA
Symposium
Report,
<http://fra.europa.eu/sites/default/files/fra_uploads/1697-FRAsymp2011-outcome-report.pdf>
accessed
5
November 2014.
183
Council Regulation (EC) No. 168/2007 of 15 February 2007 establishing a European Union Agency for Fundamental
Rights [2007] OJ L 53/1, Art. 2; European Parliament and Council Regulation (EC) No. 1922/2006 of 20 December
2006 establishing a European Institute for Gender Equality [2006] OJ L 403/9, Art. 2.
184
European Parliament, Resolution on guaranteeing Independent Impact Assessments of 8 June 2011
(2010/2016(INI)) P7_TA(2011)0259.
185
European Commission, Impact Assessment Guidelines SEC(2009) 92, p. 6.
186
European Commission, Strategy for the Effective Implementation of the Charter of Fundamental Rights by the
European Union (Communication) COM(2010) 573 final, p. 6.
187
European Commission, Operational Guidance on taking account of Fundamental Rights in Commission Impact
Assessment (Commission Staff Working Paper) SEC(2011) 567 final.
50
FRAME
the initiatives planned in the Strategy for the effective implementation of the Charter of Fundamental
Rights of October 2010.
(2)
Monitoring
There are various EU instruments in place that allow for monitoring fundamental rights in the Member
States of the EU. On the one hand, these instruments address certain areas related to human rights such
as corruption, gender equality, independence of the justice systems, social protection, or employment.
These instruments are either mainly or at least partially based on indicators. On the other hand, the
European Commission and the European Parliament carry out regular reporting on the fundamental rights
situation in the Member States of the EU. These reports are primarily based on secondary analysis of
existing data and reports. The discussion of these instruments starts with those, which develop and use
indicator schemes for measuring certain human rights.
Promotion of the Rights of the Child
The FRA carries out a project specifically dealing with the development of human rights indicators. This
project also aims at gathering more comparable data and information across the EU Member States. As
early as 2007, the FRA had started to work on child rights indicators. Concrete structures, process and
outcome indicators were developed on child rights focusing especially in the issues of family and
migration, protection from violence and exploitation, adequate living standards, child poverty, and
education.188
The Right to Political Participation of Persons with Disabilities
A list of 28 indicators was developed by the FRA in close cooperation with the European Commission and
the Academic Network of European Disability Experts on the right to political participation of persons with
disabilities. The indicator set was then populated, wherever possible, with publicly available data. The
thematic areas for grouping data are: lifting legal and administrative barriers; increasing rights awareness;
making political participation more accessible; and expanding opportunities for participation. For each of
these areas, structure, process and outcome indicators have been developed. These indicators may be
considered independently but should be read in the context of the whole set of indicators. In May 2014,
the FRA presented its first findings:189 The study revealed the lack of systematic data collection in the
Member States and made clear that No reliable or accurate data were identified in the Member States
to populate many of the indicators. In many cases, the use of different methodologies and criteria for data
collection made comparative analysis challenging.190 The FRA generally aims at intensifying the
188
European Union Agency for Fundamental Rights, Conference Edition, Developing Indicators for the Protection,
Respect and Promotion of the Rights of the Child in the European Union of November 2010.
189
Human Rights Indicators (Publications Office of the European Union 2014).
190
Human Rights Indicators (Publications Office of the European Union 2014), p. 16.
51
FRAME
development of indicators in its work programme for the years 2013-2017.191 It is planned that indicators
and benchmarks will be developed and applied for the identification of trends in fundamental rights over
time192 as well as to evaluate the FRAs work and its impact.193
Gender Equality Index

EIGE was assigned by the European Commission with the task of constructing a composite indicator194 on
gender equality to reflect the multi-faceted reality of gender equality and tailored towards the policy
framework of the EU and its Member States.195 EIGE started working on this issue in 2010 and the results
were officially launched at an EU conference in 2013.196 The so-called Gender Equality Index197 was
developed following the Organisation for Economic Co-operation and Development (OECD) handbook on
constructing composite indicators198 and is formed by combining gender equality indicators of six core
domains (work, money, knowledge, time, power and health) and two satellite domains (intersecting
inequalities and violence). The indicators are based on outcome variables, which measure peoples rights
enjoyment instead of countries commitments and efforts to promote gender equality. The indicators
within the Gender Equality Index in the area of work for example deal with full-time equivalent
employment rates, duration of working life, training at work, flexibility of working time, etc. With regard
to money the indicators deal with earning, income, poverty and income distribution. With regards to the
domain of time issues of childcare activities, domestic activities, sport, culture and leisure activities as well
as volunteering and charitable activities are taken into account.199
191
European Union Agency for Fundamental

Rights, Annual Work Programme
2015
<http://fra.europa.eu/sites/default/files/annual_work_programme_2015.pdf> accessed 25 September 2014
192
Rights, Annual Work Programme
2015
<http://fra.europa.eu/sites/default/files/annual_work_programme_2015.pdf> accessed 25 September 2014
193
Rights, Annual
Work Programme
2015
<http://fra.europa.eu/sites/default/files/annual_work_programme_2015.pdf> accessed 25 September 2014, p. 7.
194
Composite indicators synthesise individual indicators relating to different areas into a single measurement.
195
European Commission, A Roadmap for Equality between Women and Men 2006-2010 (Publications Office of the
European Communities 2006).
196
European Institute for Gender Equality, Gender Equality Index - Launch Conference,
<http://eige.europa.eu/content/event/gender-equality-index-launch-conference> accessed 25 September 2014.
197
European Institute on Gender Equality, Gender Equality Index: Main Findings (Publications Office of the European
Union 2013).
198
Organisation for Economic Co-operation and Development (OECD) and European Commissions Joint Research
Centre, Handbook on Constructing Composite Indicators: Methodology and User Guide (OECD 2008).
199
European Institute for Gender Equality, Gender Equality Index Report (EIGE 2013).
52
FRAME
Apart from the Gender Equality Index, the EIGE provides for an overview of the key policy initiatives to
promote gender equality, as well as data on some structural indicators in its Country Profiles.200
European Justice Scoreboard
The EU Justice Scoreboard is a comparative tool seeking to provide data on the efficiency of non-criminal
justice systems in the Member States. This tool is not treaty-based, but anchored in the European
Semester, which is a yearly cycle of economic policy coordination set up by the European Union. In the
frame of the European Semester, the Commission undertakes a detailed analysis of EU Member States
plans of budgetary and structural reforms and provides them with recommendations for the next year.
An efficient and independent justice system contributes to trust and stability. Predictable, timely and
enforceable justice decisions are important structural components of an attractive business
environment.201 The EU Justice Scoreboard does not present an overall ranking of countries, but rather
provides an overview of the functioning of all justice systems based on various indicators. The EU Justice
Scoreboard brings together data from various sources. Most of the quantitative data is provided by the
Council of Europe Commission for the Evaluation of the Efficiency of Justice.202 The CEPEJ itself collects
relevant data from Member States.
The 2014 EU Justice Scoreboard focuses on litigious civil and commercial cases as well as administrative
cases. It looks at the same indicators as in 2013 (efficiency, quality and independence of justice systems)
while also drawing on some additional sources of information. Measuring the efficiency of justice systems
includes indicators on the length of proceedings, the clearance rate, and the number of pending cases.
Indicators on the quality of justice systems includes measuring the compulsory training of judges,
monitoring and evaluation of court activities, the budget and human resources allocated to courts, the
availability of Information and Communication Technology, and of alternative dispute resolution
methods. The independence of justice systems is measured through data and assessments rather than
indicators. For the purpose of measurement, independence is divided analytically into the dimensions of
perceived independence and structural independence. The 2014 EU Justice Scoreboard provides a first
general comparative overview of how national justice systems are organised to protect independence of
the juridical system in certain types of situations where it may be at risk. It looks at legal safeguards
against, for instance, the transfer or dismissal of judges.203 The findings of the Scoreboard indicators are
analysed more thoroughly through a country specific assessment carried out in the context of the
European Semester through a bilateral dialogue with the Member States authorities and stakeholders.
200
European Institute on Gender Equality, Gender Equality Index: Country Profiles (Publications Office of the
European Union 2013).
201
European Commission, The EU Justice Scoreboard: A Tool to Promote Effective Justice and Growth COM(2013)
160 final, p. 2; European Commission, The 2014 EU Justice Scoreboard COM(2014) 155 final.
202
Council of Europe Commission for the Evaluation and the Efficiency of Justice (CEPEJ), Study on the Functioning
of Judicial Systems in the EU Member States: Facts and Figures from the CEPEJ Evaluation 2012-2014 Exercise
CEPEJ(2014) 4 final.
203
European Commission, The 2014 EU Justice Scoreboard (Communication) COM(2014) 155 final, p. 7
53
FRAME
The aim of this assessment is to find out, for instance, the reasons for a poor performance and to take
into account the particularities of the countries legal systems. An outcome of this assessment may be
country-specific recommendations on the need to improve the justice system.204
Anti-Corruption Report
The European Commissions Anti-Corruption Report was established in 2011 and was first published in
February 2014.205 The report is produced by the European Commission every two years,206 and provides
an analysis of corruption in the Member States, and their efforts to prevent and fight it. The report adopts
a right-based approach, anchoring its understanding of corruption in international law and in the law of
the Member States. The report defines corruption as any abuse of power for private gain207 and thereby
covers two aspects. Firstly, the specific acts of corruption and measures of Member States to prevent and
punish it as defined by law. Secondly, it covers certain types of conduct and measures which influence the
risk of corruption and the capacity of a state to control it. The Anti-Corruption Report is methodologically
based on a qualitative assessment. Quantitative approaches play a minor role because it is difficult to
quantify corruption and even more to rank countries by results.208 Indicators were used, but not as a basis
of a new index on corruption, but rather to provide elements of analysis supplementing the qualitative
assessment that is at the core of the report.209 The qualitative assessment of corruption in each country
focuses on what works and what does not work in dealing with corruption. Information and data derive
from a different range of sources, such as monitoring mechanisms (e.g. Group of States against Corruption
(GRECO), OECD, World Bank, Transparency International), from other data sources (academia, national
authorities, independent experts, civil society organizations, etc.) and from various Commission
departments and relevant EU agencies (Europol and Eurojust).210
Joint Assessment Framework
The Employment Committee, the Social Protection Committee and the European Commission (DG EMPL)
developed a proposal for a Joint Assessment Framework for tracking progress and monitoring the
204
European Commission, The EU Justice Scoreboard: A Tool to Promote Effective Justice and Growth COM(2013)
160 final, p. 2; European Commission, The 2014 EU Justice Scoreboard COM(2014) 155 final.
205
European Commission, Commission Decision establishing an EU Anti-Corruption Reporting Mechanism for
Periodic Assessment ("EU Anti-Corruption Report") C(2011) 3673 final; European Commission, EU Anti-Corruption
Report COM(2014) 38 final.
206
European Commission, Commission fights Corruption: a Stronger Commitment for Greater Results (Press
Release) IP/11/678 of 6 June 2011.
207
European Commission, EU Anti-Corruption Report, COM(2014) 38 final, p. 2.
208
European Commission, EU Anti-Corruption Report COM(2014) 38 final, p. 37.
209
Release), IP/11/678 of 6 June 2011.
210
European Commission, EU Anti-Corruption Report COM(2014) 38 final, p. 37.
54
FRAME
Employment Guidelines211 under Europe 2020.212 It is an indicator-based monitoring system, which acts
primarily as an analytical tool with the aim to identify key employment challenges and to track progress
of Member States towards the Europe 2020 goals.
For this purpose, relatively homogenous policy areas are selected from the Employment Guidelines to be
assessed through indicators. In each of the selected policy areas, country specific progress in the
implementation of policies and towards the related objectives is assessed in three steps. Step one is a
quantitative assessment based on a limited number of indicators, building on existing monitoring
practices as developed under the Lisbon Strategy and the Open Method of Coordination.213 For each key
policy area a main, or key indicator is defined, which provides a representative summary of the objective
of that particular policy. This main indicator is then supplemented by policy specific sub-indicators, which
measure the outcome of the policy and explain the findings of the key indicators. For example, if the
country specific progress in the policy area increase of labour market participation is assessed, the main
indicator is the employment rate of a country. Explanatory sub-indicators then are the employment
gender gap or the employment rate of low-skilled persons.214 For the quantitative assessment, the key
indicator and the individual outcome-sub-indicators are standardized to allow for an easier comparison
and analysis (the EU 27 average is suggested as mean). In step two, the quantitative assessment of policy
specific progress and outcome is then complemented by a qualitative assessment. Hereby, contextual
information and country specific evidence, as well as expert knowledge, are taken into account. The
qualitative assessment involves, for instance, changes over time. This is important since the quantitative
part of the assessment is mainly based on the level from the latest available period. Additionally, the
quantitative assessment addresses data problems, recent changes or sub-areas, for which no indicators
are available. Building on the qualitative and the quantitative assessment, key challenges and best
practices are identified in step three.215
The Employment Performance Monitor216 reflects the results of the JAF for the employment policies area.
It assesses the employment situation in the Member States of the EU and the progress towards the Europe
2020 targets. Among others, labour market participation disaggregated regarding gender and age is
211
Council Decision 2010/707/EU of 21 October 2010 on Guidelines for the Employment Policies of the Member
States [2010] OJ L 308/46.
212
Council of the European Union, The Joint Assessment Framework and the Employment Performance Monitor
(Joint EMCOSPC Opinion) 16984/10 of 26 November 2010.
213
(Joint EMCOSPC Opinion) 16984/10 of 26 November 2010, p. 6.
214
(Joint EMCOSPC Opinion) 16984/10 of 26 November 2010, pp. 12-13.
215
(Joint EMCOSPC Opinion) 16984/10 of 26 November 2010, pp. 10-19.
216
Council of the European Union, Europe 2020 Employment Performance Monitor June 2014 10763/1/14 Rev 1
of 16 June 2014.
55
FRAME
presented for all EU 27 countries, as well as adequate and employment oriented social security systems,
and gender equality. It is published twice a year, in the framework of the European Semester.
Social Protection Performance Monitor
The Social Protection Performance Monitor aims to identify main and common social trends and reinforce
multilateral surveillance to the social dimension targets of Europe 2020. It is prepared by the Social
Protection Committee. The indicators used are based on those agreed in Laeken in 2001 and further
developed by the Indicator Subgroup of the Social Protection Committee in the field of social protection
and social inclusion. They aim to give a synthetic but comprehensive picture on the main changes in the
social situation in Europe and lead to the identification of trends.217 Data is predominantly provided by
EUROSTAT, through the EU Statistics on Income and Living Conditions (EU-SILC) and the EU Labour Force
Survey (EU-LFS) datasets.218 The Social Protection Performance Monitor contains an overview of the social
development trends in the EU, as well as more detailed country analysis, which looks at the progress of
each Member State towards the national targets. The list of indicators is not static, but is reviewed
regularly in order to ensure that policy and statistical developments are reflected appropriately. Trends
and data from the SPPM can be found in the Annual Report of the Social Protection Committee on the
social situation in the EU.219
Migrant Integration Indicators
The Zaragoza Declaration220, has been adopted in April 2010 by EU Ministers responsible for immigrant
integration issues and has been approved at the Justice and Home Affairs Council in June 2010. It called
upon the European Commission to undertake a pilot study to examine proposals for common integration
indicators and to report on the availability and quality of the data from agreed harmonised sources, which
are necessary for the calculation of these indicators.221 The report on this Pilot Survey has been prepared
by EUROSTAT within the frame of the Zaragoza pilot study on common integration indicators.222
The European Commission requested the further development and use of EU immigrant integration
indicators. This research based work was undertaken between 2012 and 2013 by the Migration Policy
Group (MPG) and the European Services Network (ESN) on behalf of the European Commission. It
reconfirms the relevance and usefulness of the Zaragoza Indicators and identifies three types of factors
217
Social Protection Committee Social Protection Performance Monitor (SPPM) Methodological Report by the
Indicators
Sub-group
of
the
Social
Protection
Committee
of
17
October
2012
<http://ec.europa.eu/social/main.jsp?catId=758> accessed 3 November 2014.
218
EUROSTAT, Measuring Material Deprivation in the EU: Indicators for the Whole Population and Child-Specific
Indicators (Publications Office of the European Union 2012).
219
Social Protection Committee, Social Europe Many ways, one objective (Publications Office of the European Union
2014).
220
Council of the European Union, Zaragoza Declaration of the European Ministerial Conference on Integration of
16 April 2010.
221
EUROSTAT, Indicators of Immigrant Integration: A Pilot Study (Publications Office of the European Union 2011).
222
EUROSTAT, Indicators of Immigrant Integration: A Pilot Study (Publications Office of the European Union 2011).
56
FRAME
that influence societal integration outcomes in four areas. These factors concern personal characteristics
of the immigrant population (such as time of residence, gender, origin country, socio-economic), the
general context in the country (welfare systems, education systems, housing or the labour market) and
its specific migration and integration policies. The four areas are employment, education, social inclusion
and active citizenship. In each of these areas, the EU selected an initial number of indicators (the Zaragoza
indicators), and complemented them with proposed new indicators. In the field of employment, for
instance, the Zaragoza indicators are: employment rate, unemployment rate, activity rate223, selfemployment and over-qualification. Proposed new indicators are: public sector employment, temporary
employment, part-time employment, long-term unemployment, share of foreign diplomas recognised
(data is not available every year) and the retention of international studies (data either needs to be
collected or migrant sample size boosted). In addition to these four areas already identified in the
Zaragoza Study, the researchers of the MPG and the ESN suggest a new category of indicators, namely
welcoming society. The new category includes the Zaragoza Indicators on perceived experience of
discrimination, trust in public institutions and sense of belonging (for all three, data needs to be collected).
Moreover, new indicators are proposed, such as public perception of racial/ethnic discrimination and
public attitudes toward political leaders having an ethnic minority background (for both, the use of
Eurobarometer data is suggested). Different options for the use of these indicators are mentioned, such
as to describe the situation in societies, to clarify the link between integration policies and societal
outcomes (monitoring). Additionally, the indicators can also be used for the evaluation of policies. Overall,
the report is designed to be explorative and descriptive in its nature.224
Regular reporting on Fundamental Rights in the EU
The European Commissions Annual Report on the Application of the Charter, first published in 2010,
contains an overview of how the Charter has been taken into consideration by the EU bodies when taking
decisions, promoting legislation or in the case law of the Court of Justice of the EU.225 It does not offer an
evaluation on the status of fundamental rights in the EU, but provides an overview of recent
developments on how the Charter was considered within the EU bodies.
The status of fundamental rights in the EU is also assessed by the European Parliament and published in
the Annual Report on the Situation of Fundamental Rights in the EU. This Report is drafted by a Rapporteur
within the LIBE Committee and then adopted in plenary by resolution. There is hardly any public
information available on the methodology used to draft the report. The Parliaments report bases its
evaluation on reports and findings of other European and International institutions, e.g. the European
Commission, the FRA, the Council of Europe, the human rights treaty bodies of the UN, etc.
223
The activity rate represents active persons as a percentage of the total population of the same age group.
Thomas Huddleston and others, Using EU Indicators of Immigrants Integration (Publications Office of the
225
European Commission, 2013 Report on the Application of the EU Charter of Fundamental Rights COM(2014) 228
final, p. 2.
224
57
FRAME
Also the FRA publishes an annual report on the fundamental rights situation in the EU 27, focusing on
those issues mentioned in its multiannual framework, covering for example the topics of asylum and
integration, equality and non-discrimination, racism and xenophobia, data protection, or rights of the
child.226
Rather recently, the EC and EUROSTAT started to gather data in the field of trafficking in human beings
EUROSTAT just published its second report with statistical data on trafficking227, and the EC published its
mid-term evaluation of the Member States implementation of the Anti-Trafficking Strategy.228 In the
EUROSTAT survey common indicators, such as information on victims by gender and age, number of
victims by citizenship, police data on suspected traffickers by age, gender and citizenship, data on
prosecuted traffickers by age and gender or on court data on judgments of traffickers by age and gender,
among others were selected.229 The EC report measures progress against the background of the AntiTrafficking Strategys key priorities, which are: identifying, protecting and assisting victims of trafficking;
stepping up the prevention of trafficking in human beings; increased prosecution of traffickers and
enhanced coordination, cooperation and policy coherence.230
(3)
Evaluation
As far as can be determined by looking into the mapping of instruments established by the EU institutions,
human rights indicators do not yet play a role in the evaluation of internal EU policies or projects. The
Indicators on Migrants Integration are suggested to be used also for evaluation. Other recent
developments regarding the use of fundamental rights indicators in this regard in the EU internal area will
be discussed in the next subchapter.
b)
Enlargement
This subchapter addresses instruments that measure fundamental rights in the context of the EU
enlargement process. As can be seen when looking into the historical development of pre-accession
assistance, the financing possibilities within this instrument were adjusted to ensure that pre-accession
assistance is more closely linked to the enlargement priorities and will favour a sectoral approach.231
226
European Union Agency for Fundamental Rights, Annual Report 2013 Fundamental Rights, Challenges and
Achievements in 2013 (Publications Office of the European Union 2013).
227
Eurostat, Trafficking in Human Beings 2014 edition (Publications Office of the European Union 2014).
228
European Commission, Commission Staff Working Document Mid-term Report on the Implementation of the
EU Strategy Towards the Eradication of Trafficking in Human Beings COM(2014) 635 final of 17 October 2014.
229
Eurostat, Trafficking in Human Beings 2014 Edition (Publications Office of the European Union 2014), p. 17. Note:
it proved difficult for some Member States to provide some of those common indicators, as was mentioned in the
report on p. 18.
230
European Commission, Commission Staff Working Document Mid-term Report on the Implementation of the
EU Strategy Towards the Eradication of Trafficking in Human Beings COM(2014) 635 final of 17 October 2014.
231
European Commission, 2012 Annual Report on Financial Assistance for Enlargement, Report from the Commission
to the European Parliament, the Council and the European Economic and Social Committee, (Publications Office of
the European Union 2013), p. 6.
58
FRAME
Enlargement is discussed separately from the internal and external issues, as it stands at point somewhere
in between those two areas and the measurement context is somewhat different.
The enlargement strategy and process changed and developed over the years. Similarly, the three case
studies on the use of rule of law indicators in the EU enlargement process of Bulgaria, Croatia and
Montenegro, which form Annex II of this report, also reveal that the tools and indicators applied for
measuring the rule of law in the EU enlargement processes have continuously improved over time.
Generally, any country seeking membership of the EU must confirm the prerequisites of Art. 40 and Art.
6 (1) TEU. The relevant criteria were established in 1993 in Copenhagen, and then reaffirmed and
strengthened in 1995. A composite paper on enlargement was presented as Part Two of the Agenda
2000232 and explained the methodology for an assessment of applications for membership through the
application of predefined accession criteria.
One of the criteria relevant for accession is the political criterion: guaranteeing democracy, the rule of
law, human rights and respect for and protection of minorities. To meet those criteria many of the
accession states had to adapt legislation, judiciary, etc. Progress is analysed within the accession
procedure. Once the criteria are met, accession to the Union is possible. Acting on the mandate of the
Council and the European Council, the Commission has a key position in the assessment of the progress
made by the candidate and potential candidate countries towards fulfilling the conditions of the
Copenhagen criteria. Each year the Commission presents a set of documents to the Council, explaining its
strategy in line with the EU enlargement policy and reporting on the progress achieved at country level.
The so-called Enlargement package adopted by the Commission includes the Progress Reports for each
candidate and potential candidate country as well as the annual Enlargement Strategy Paper. This paper
takes stock of the developments in the last twelve months and sets out the way forward for the coming
year.233 The Council and the European Council take their decisions to move on in the accession process of
a country based on the recommendations contained in the Commissions assessments. In this way, the
assessment of the progress towards the fulfilment of the membership criteria is one of the most important
and powerful tools in the accession process.234 Progress in the countries in question is partly monitored
by on-site visits by experts in the respective areas, but no common set of indicators is currently used.
The accession process is closely monitored by the Commission, taking into account various sources of
information, like international report, expert country visits, etc. However there is no generally applicable
indicator scheme developed for assessing progress in the accession procedures for the acceding countries.
In the next subsection, monitoring instruments linked to the enlargement process will be presented.
232
European Commission, Agenda 2000: For a Stronger and Wider Union (Communication) COM(97) 2000 final.
European
Commission,
DG
Enlargement,
Strategy
and
Progress
Reports
<http://ec.europa.eu/enlargement/countries/strategy-and-progress-report/index_en.htm> accessed 9 October
2014.
234
Dimitry Kochenov, EU Enlargement and the Failure of Conditionality: Pre-accession Conditionality in the Fields of
Democracy and the Rule of Law (Kluwer Law International 2008), p. 59.
233
59
FRAME
Instruments for Pre-Accession Assistance II

The Instrument for Pre Accession Assistance II will be presented here and not in the section on the EUs
external sphere, as it has a closer link to the enlargement area regarding its scope to bring countries
towards achievement of the aquis in the relevant sectors. As of 2007, the CARDS programme and the
other programmes providing assistance to candidate countries (namely PHARE, SAPARD and ISPA) were
replaced by one single instrument, the IPA.235 Assistance granted under the IPA was linked to the progress
and needs identified through the Commissions evaluations and was subject to a suspension clause in the
case of failure to make sufficient progress in the accession process.236 IPA was in place from 2007 until
2013.
The new IPA II was set up by a Regulation 231/2014 in March 2014 and is applicable retroactively as of 1st
January 2014.237 For this instrument monitoring, progress will be conducted by using appropriate countryspecific and measurable indicators.238 IPA II has a strategic focus and Indicative Strategy Papers are the
planning documents for each beneficiary for the seven years. For each country a country-specific strategy
paper is developed. Those papers foster a larger self-responsibility of the countries in question. IPA II
targets reforms in pre-defined sectors, which are closely linked to the enlargement strategy and also
include democracy and the rule of law, which should help bring the relevant sectors to EU standards.239
The IPA II Regulation foresees that progress towards the achievement of specific targets shall be
monitored by the use of clear, transparent, country-specific and measurable indicators. These indicators
shall cover inter alia the area of strengthening democracy, the rule of law, and an independent and
efficient justice system. The relevant performance indicators shall be pre-defined and shall enable
objective assessment of progress over time and across programmes. Indicators shall also be taken into
account for the contribution of performance rewards. The performance element laid down in Article 14
of the Regulation makes it possible to reward countries with good performance and to re-allocate funds
in case of underperformance. Therefore, the indicators shall act as a means of efficiency control. The
indicators are agreed with the beneficiaries and used to measure whether the goals are really achieved.
Each beneficiary shall set up sectorial monitoring committees no later than six months after the first
financing agreement enters into force. The sectorial monitoring committee shall then review the
effectiveness, efficiency, quality, coherence, coordination and compliance of the implementation of
actions in the policy areas. Progress shall be measured by means of indicators related to a baseline study
235
Council Regulation (EC) 1085/2006 of 17 July 2006 Establishing an Instrument for Pre-Accession Assistance (IPA)
[2006] OJ L210/82.
236
Pre-accession Assistance (IPA II) [2014] OJ L 77/14.
237
238
239
European
Commission
Overview
Instrument
for
Pre-Accession
Assistance
<http://ec.europa.eu/enlargement/instruments/overview/index_en.htm#ipa2> accessed 9 October 2014.
60
FRAME
as laid down in Art. 19 of the Implementing Regulation No 447/2014.240 Current beneficiaries are Albania,
Bosnia and Herzegovina, the Former Yugoslav Republic of Macedonia, Iceland, Kosovo, Montenegro,
Serbia and Turkey.
In June 2014 the Commission issued a multi-country indicative strategy paper for 2014-2020 on the IPA
II.241 It sets out the priorities for the period until 2020. The paper complements the Country Strategy
Papers and only foresees some more indicators, which are considered to be specific for regional
interventions.242 Additional to the indicators laid down in the CSPs, human rights indicators are predicted
to be relevant. Examples of such indicators include overall effectiveness of the governments or number
of Erasmus+ students of the region completing an EU Master programme. Annex 2 of the document
provides for the entire list of indicators mentioned in the paper. Data for those indicators is taken from
other institutions, like EUROSTAT or the World Bank.243
Looking for example into the indicative strategy paper for Kosovo (2014-2020) in the course of IPA II
several indicators are mentioned. There are three different types of indicators foreseen. Firstly context
indicators taken from EUROSTAT like public debt, unemployment rate or GDP per capita. Furthermore,
outcome and impact indicators are listed, and are closely linked to DG Enlargement Progress Reports,
such as progress made towards meeting the political criteria or progress made on implementation of the
acquis. The last set of indicators is the so-called sector indicators; the most relevant sectors to this
research being governance and democracy, and the rule of law and fundamental rights. In those sectors,
data from the World Bank, the World Economic Forum, Transparency International, as well as other actors
are used together with the DG Enlargement Progress reports to assess progress towards meeting the
Copenhagen criteria in areas like governance, judicial reform, fundamental rights or refugees and boarder
240
European Commission Implementing Regulation (EU) No 447/2014 of 2 May 2014 on the Specific Rules for
Implementing Regulation (EU) No 231/2014 of the European Parliament and of the Council establishing an
Instrument for Pre-accession Assistance (IPAII) [2014] OJ L 132/32.
241
European Commission, Instrument for Pre-Accession Assistance (IPA II) Multi-country indicative strategy paper
(2014-2020) of 30 June 2014.
242
(2014-2020) of 30 June 2014, p. 12.
243
(2014-2020) of 30 June 2014, p. 29.
61
FRAME
management.244 The same indicators are also foreseen in the IPA II Indicative Strategy Paper for Turkey245,
Albania246, the Former Yugoslav Republic of Macedonia247, Serbia248 and Montenegro249.
Cooperation and Verification Mechanism
Bulgaria and Romania acceded to the EU in 2007 but with the special condition that they must continue
reforms in the area of justice and rule of law. The Cooperation and Verification Mechanism was
implemented to monitor the implementation of reforms. A report is published by the Commission once a
year regarding improvements and challenges.250 Within the CVM structure, process and outcome
indicators are used, e.g. for measuring the independence and accountability of the judiciary.
Due to insufficient progress, the Commission decided in July 2012 to suspend the CVM for 18 months in
order to give Bulgaria the time to achieve visible results in the implementation of reforms. The
Commission underlined that the potential of the mechanism has not been used to the full and the lack of
consistent direction of reforms puts into question the sustainability and irreversibility of the progress.251
Due to a lack of political commitment to a long-term strategy for reform, and of concrete short-term
measures to bring the process forward also in January 2014 the Commission could identify no sufficient
progress.252 This leads to the conclusion that the post-accession incentives and the sanctions applied were
too weak to bring about the requested changes after accession.253
c)
External relations and foreign affairs
In this subchapter on external relations relevant EU instruments are presented regarding human rights
assessment and monitoring. Again the subdivision in different stages of the policy cycle is used in this
subchapter.
244
European Commission, Instrument for Pre-Accession Assistance (IPA II), Indicative Strategy Paper for Kosovo
(2014-2020), Annex 2, p. 38 seq.
245
European Commission, Instrument for Pre-Accession Assistance (IPA II), Indicative Strategy Paper for Turkey
(2014-2020), Annex 2, p. 47 seq.
246
European Commission, Instrument for Pre-Accession Assistance (IPA II), Indicative Strategy Paper for Albania
(2014-2020), Annex 2, p. 37 seq.
247
European Commission, Instrument for Pre-Accession Assistance (IPA II), Indicative Strategy Paper for the Former
Yugoslav Republic of Macedonia (2014-2020), p. 33 seq.
248
European Commission, Instrument for Pre-Accession Assistance (IPA II), Indicative Strategy Paper for Serbia
(2014-2020), p. 41 seq.
249
European Commission, Instrument for Pre-Accession Assistance (IPA II), Indicative Strategy Paper for
Montenegro (2014-2020), p. 39 seq.
250
European
Commission,
The
reports
on
progress
in
Romania
and
Bulgaria
<http://ec.europa.eu/cvm/progress_reports_en.htm> accessed 9 October 2014.
251
European Commission Progress in Bulgaria under the Cooperation and Verification Mechanism (Report)
COM(2012) 411 final, p. 20.
252
European Commission, Progress in Bulgaria under the Cooperation and Verification Mechanism (Report)
COM(2014) 36 final, p. 9.
253
Christophe Hillion, Enlarging the European Union and Deepening its Fundamental Rights Protection (2013) 11
European Policy Analysis <http://www.sieps.se/sites/default/files/2013_11epa.pdf> accessed 9 October 2014, p. 13.
62
FRAME

(1)
Policy Planning
Council Human Rights Guidelines

The Human Rights Guidelines play an important role in the formulation of the EUs policy on human rights
in its external actions. They were adopted in 1998 and cover human rights topics of particular importance
to the EU. They are pragmatic guidelines offering different EU actors a range of tools to carry out specific
actions in those areas of concern. So far 11 Guidelines have been issued covering a broad range of topics:
freedom of expression, promotion and protection of religion or belief, LGBTI, death penalty, torture,
children and armed conflict, protection of the rights of the child, violence against women and girls, human
rights defenders, promoting compliance with international humanitarian law and freedom of expression
online and offline.
These Guidelines are revised periodically. While the use of indicators has not been a common feature of
the Guidelines, it is worth noting that the Guidelines of the Council on the Enjoyment of Human Rights by
Lesbian, Gay, Bisexual, Transgender and Intersex Persons, adopted in 2013, set out a set of indicators with
a checklist to measure their situation.254 In these guidelines the EU states, that tailor made approaches
and human rights country strategies will be an important tool to identify the best way forward in different
contexts. The indicators are formulated along the lines of specific human rights, i.e. right to life,
prohibition of torture, non-discrimination, freedom of association, freedom of assembly, etc. The checklist
shall be used to track and monitor the situation of human rights of LBGTI persons in the country concerned
and to identify progress/setbacks.255 From the Guidelines it is not clear where, how and how often those
indicators are used in practice. The EU stresses the importance of identifying tools used in enabling data
to be collected and to help boost national capacities to collect and disseminate reliable, accurate data.256
Similarly, the Guidelines on freedom of expression online and offline,257 adopted in 2014, include a list of
resources related to freedom of opinion and expression that the EU may use in contacts with third
countries. This list includes a reference to the tools developed by the UNESCO for that purpose.
In the EU guidelines on violence against women and girls and combating all forms of discrimination against
them,258 the need for accurate, comparable, and quantitative data and relevant indicators is emphasised.
It is mentioned that there are still disparities in the types of data that are collected.
254
255
Gay, Bisexual, Transgender and Intersex (LGBTI) Persons of 24 June 2013, p. 8.
256
257
Council of the European Union, EU Human Rights Guidelines on Freedom of Expression Online and Offline of 12
May 2014.
258
Council of the European Union, EU Guidelines on Violence against Women and Girls and combating All Forms of
Discrimination against them of 8 December 2008.
63
FRAME
Country Human Rights Strategies

Country Human Rights Strategies were firstly established in 2011 to better assess and more
comprehensively analyse key human rights challenges in third countries. They shall also underline the fact
that human rights are at the very core of EU external action. These strategies include an analysis of the
human rights situation in the country and identification of priorities and objectives for EU action. As stated
in the Joint Communication, the Human Rights Strategies shall be taken into account when conducting
policy-making, and programming financial assistance with third countries, so they must be considered in
all further instruments mentioned below and should be mainstreamed effectively.259 There is no further
information available on those strategies regarding content or method, as they are kept confidential.
Tool-Box a rights-based approach encompassing all human rights for EU development cooperation260
In April 2014 a Commission Staff Working document on a rights-based approach encompassing all human
rights for EU development cooperation was released. The tool-box was foreseen in the Council
Conclusions on the EU Action Plan on Human Rights and Democracy in 2012.261 The tool-box describes
what a Rights Based Approach to development is by highlighting its core concepts and their rational,
clarifying common misunderstandings, introducing relevant legal references and other donors
commitments. Furthermore, it describes how to apply such an approach in development cooperation.
According to the document, all policies and activities of development cooperation, which were
implemented using a Rights Based Approach (RBA), are aimed at directly contributing to the realisation
of human rights.262 The rights based approach was again confirmed in the Council conclusions on a rightsbased approach to development cooperation, encompassing all human rights in May 2014.263
As an example, the new Development Cooperation instrument is framed in the line of rights based
approach and will be presented in further detail below. The working document provides a check-list for
those people working in the area, stating that the RBA has to be used in every step of the project cycle;
namely identification, formulation, implementation, monitoring and evaluation.264 The checklist suggests
259
260
A more extensive description and discussion of this tool-box is provided in FRAME Deliverable 9.1: Laura Beke
and others, Report on the Integration of Human Rights in EU Development and Trade Policies (2014) Frame
Deliverables, <http://www.fp7-frame.eu/reports> accessed 11 November 2014, section IV C, p. 135 seq.
261
European Commission, Tool-Box a Rights-based Approach, Encompassing All Human Rights for EU Development
Cooperation (Commission Staff Working Document) SWD(2014) 152 final. Also in 2014, a similar tool was released
by the EC and UNICEF in relation to childrens rights and development cooperation, see
http://www.unicef.org/eu/crtoolkit/. Module 5 specifically deals with child rights impact assessment.
262
Cooperation (Commission Staff Working Document) SWD(2014) 152 final, p. 6.
263
Council of the European Union, Council Conclusions on a Rights Based Approach to Development Cooperation,
encompassing All Human Rights of 19 May 2014.
264
Cooperation (Commission Staff Working Document) SWD(2014) 152 final, p. 20.
64
FRAME
considering whether a proposed intervention strategy identifies human rights indicators, without further
references to which type or content of indicators are to be used.
(2)
Assessment and Monitoring
Country Strategy Papers

In 2000 the Council asked the Commission to draft Country Strategy Papers to provide a common
framework for programming aid in developing countries.265 Those CSPs are set up with the purpose to
provide a strategic framework for EUs assistance programmes. The CSPs covered the period from 2007
to 2013, and more detailed indicative programmes are set up for shorter periods. The CSPs considered
available indicators to assess the situation in the country in question, e.g. World Bank Worldwide
Governance Indicators, Transparency International Corruption Perception Index, the Public Expenditure
and Financial Accountability (PEFA) Assessment, et. al. The improvements of the country in the given areas
are then assessed in the CSPs, analysing the developments over the last period using the existing
indicators. This is also done in human rights relevant areas such as education, health, etc. The indicators
used may vary from country to country. The CSPs of 2007-2013 were not set up in a standardised way,
which lead to criticism by many institutions such as DROI or COHOM, and doubts regarding their
effectiveness. Critics complained about a lacking retraceability as a result of the missing standardised
impact assessment and reporting. Connected with that, the confidentiality of the Country Strategy Papers,
as it was the case for the CSPs of 2007-2013, has been criticised by these organisations.266
According to instructions for planning the 11th Development Fund and the Development Cooperation
Instrument, CSPs should no longer be the rule, but the exception in the period from 2014 to 2020.267
Programming should, as a rule, mainly be based on the partner countries/regions own development
plans or other equivalent documents. The multi-annual indicative programmes of the EU should be
synchronised as much as possible with the countrys/regions plans. If the delegations come to the
conclusion that a plan of a country/region is not a sufficient basis for programming of EU assistance, then
a CSP should still be drafted.
However, the use of indicators for CSP has been debated for a long period of time before. As early as 2002,
the European Commission issued guidelines on the use of indicators with the aim to define principles and
guidance for the choice of indicators to be monitored in CSPs (i.e. Country Strategy Papers) intervention
framework.268 These guidelines propose that a minimum of ten indicators, drawn from the Millennium
265
Council of the European Union Development, Standard Framework for Country Strategy Papers: The Council
Conclusions 13357/00 of 17 November 2000.
266
See Chapter II.B on Purposes.
267
European External Action Service, Instructions for the Programming of the 11th European Development Fund
(EDF) and the Development Cooperation Instrument (DCI) 2014-2020 of 15 May 2012, p. 10.
268
2002), <http://ec.europa.eu/development/icenter/repository/guidelines_indicators_cpa_en.pdf> accessed 3
October 2014.
65
FRAME
Development Goals should be monitored across all countries.269 The Commission proposes as a typology:
input (i.e. process), output, outcome and impact indicators; indicating that according to a pilot project in
Burkina Faso, special focus shall be laid on outcome indicators.270 The Commission states that it is
preferable to restrict lists used to a number of essential indicators, which have to be defined clearly and
precisely. Also the necessity to keep in mind reliability and availability of data is clearly stated. It was
further proposed that the target values for the indicators will be set by the country in question and
discussed with the donors. The guidelines also emphasise that the evaluation of a countries performances
should never be a simple mechanical interpretation of indicators, but should rather be accompanied by a
policy dialogue with the responsible government.
Instruments for EU Development Cooperation271

The list of specific instruments for EU development cooperation presented is not exhaustive, but focuses
on instruments where there are hints to human rights measurements, and indicators as means to conduct
such measurement. The programmes by the EU in the area of development cooperation are divided along
two lines: on the one hand there are thematic instruments, like EIDHR, on the other hand there are
geographical programmes, like the European Neighbourhood Policy, the Development Cooperation
Instrument, etc. The instruments presented are those of relevance for further research regarding human
rights measurements. Through the Multi-annual Financial Framework 2014-2020 the instruments were
newly set-up. The instruments comprised in the MFF are:
-
Instrument for Pre-accession Assistance: 11,699 million
European Neighbourhood Instrument (ENI): 15,433 million
Development Cooperation Instrument (DCI): 19,662 million
Partnership Instrument (PI): 955 million
Instrument contributing to Stability and Peace (IcSP): 2,339 million
European Instrument for Democracy& Human Rights: 1,333 million
269
October 2014, p. 7 seq.
270
October 2014, p. 3.
271
Frame Report 9.1 presents development cooperation instruments in more detail: Laura Beke and others, Report
on the Integration of Human Rights in EU Development and Trade Policies (2014) Frame Deliverables,
<http://www.fp7-frame.eu/reports> accessed 11 November 2014.
66
FRAME
These instruments are complemented by Regulation 236/2014272 laying down common rules and
procedures for the implementation of the Unions instrument for financing external action. This regulation
talks about indicators generally, stating in the recitals that Whenever possible and appropriate, the
results of the Unions external action and the efficiency of the particular Instrument should be monitored
and assessed on the basis of pre-defined, clear, transparent and, where appropriate, country-specific and
measurable indicators, adapted to the specificities and objectives of the Instrument concerned. Also in
Art. 12 on monitoring and evaluation of actions, it is stated that evaluations shall be carried out on the
basis of such indicators. As of 2015 the Commission shall submit an annual report to the Council and the
Parliament on the achievement of the objectives of each regulation, measuring the results by means of
indicators according to Art. 13. No later than 31 December 2017 a mid-term review on the implementation
of each instrument is to be submitted by the Commission. This review shall focus on the achievement of
the objectives of each Instrument by means of indicators, measuring the results delivered and the
efficiency of the instruments as laid down in Art. 17.
Generally speaking for all instruments of the MFF, Multi-Annual Indicative Programmes (MIPs) are set up
for the thematic instruments and programmes. For national or regional programmes, national or regional
indicative programmes are established (NIPs and RIPs). Based on the strategies, annual action
programmes are developed, laying down plans for the respective year. Indicators are foreseen in those
programming documents in some areas also targeted directly at human rights, and in other areas with
no direct link to human rights. Examples of the areas where indicators are established will be presented
further below when discussing the instruments.
In relation to the geographic instruments, the Instrument for Pre-Accession Assistance II, dealing with
countries wishing to accede to the EU was already presented. Further instruments in this regard are the
European Neighbourhood Policy and the Development Cooperation Instrument. The European
Development Fund should also be mentioned in this regard, playing an important role as the financial arm
of the Cotonou Agreement. However, strictly speaking it is not an EU funded instrument, as it is financed
by the Member States rather than directly within the EU budget.
The Instrument contributing to Stability and Peace (IcSP)
This Instrument is the EUs main instrument supporting security initiatives and peace-building. It was
implemented through Regulation No 230/2014 and replaces the Instrument for Stability. According to Art.
8 of the Regulation, assistance through this instrument on the basis of conflict prevention, peace building
and crisis preparedness, as well as assistance in addressing global, trans-regional and emerging threats
shall be based on thematic strategy papers. These thematic strategy papers should also be accompanied
by a multi-annual indicative programme, pointing out priority areas selected for EU financing and outlining
objectives, expected results as well as performance indicators and the time frame of assistance.
272
European Parliament and Council Regulation (EU) No. 236/2014 of 11 March 2014 laying down Common Rules
and Procedures for the Implementation of the Unions Instruments for Financing External Action [2014] OJ L 77/95.
67
FRAME
Funding can be received by Member States, partner countries, partner regions as well as international
organisations.
Looking into the multi-annual indicative programme, performance indicators are presented among
others.273 In a lengthy annex to this programme, indicators are presented, focusing mainly on quantitative
data such as number of trainings, number of crisis response actions, etc. In the area of addressing global,
trans-regional and emerging threats, indicators regarding improvement of effectiveness of the criminal
justice system, further implementation of international standards in countering terrorist financing, etc.
are set.274 As has been shown, some of the indicators established are human rights-related, while others
are not human rights-related at all (e.g. increasing national and regional maritime information sharing
capacities).
Generalised Scheme of Preferences/ Generalised Scheme of Preferences +
Generally speaking the, Generalised Scheme of Preferences standard arrangement offers tariff reductions
to developing countries to export their products to the EU. It only looks into one single dimension, namely
tariffs and does not aim at tackling any other problems faced by developing countries.
The GSP+ scheme provides enhanced preferences and full removal of tariffs covered by the GSP, if
countries ratify and effectively implement international conventions relating to human and labour rights,
environment and good governance. Through this incentive human rights development shall be fostered.
Beneficiary countries are obligated to report about their progress in the implementation of the ratified
conventions in the field of human rights every second year. Additionally, the effective implementation is
monitored through a regular dialogue with beneficiary countries. The beneficiary countries are obliged to
accept and cooperate with the EU monitoring procedure. Furthermore, the burden of proof has been
reversed in the GSP+. When evidence points out towards problems in the implementation, it is up to the
beneficiary country to prove positive outcomes. Thus, the GSP+ enhances the monitoring mechanisms of
the EU compared to the GSP.275
Beneficiary countries reports are mandated more frequently now (every second year) and the European
Parliament has a scrutinising mandate in addition to the Council of the EU.276
European Neighbourhood Instrument
273
European Commission, Commission Implementing Decision of 11.8.2014 adopting the Thematic Strategy Paper
2014-2020 and accompanying Multiannual Indicative Programme 2014-2017 of the Instrument contributing to
Stability and Peace C(2014) 5607 final
274
European Commission, Commission Implementing Decision of 11.8.2014 adopting the Thematic Strategy Paper
2014-2020 and accompanying Multiannual Indicative Programme 2014-2017 of the Instrument contributing to
Stability and Peace C(2014) 5607 final, p. 43 seq.
275
European
Commission
The
EUs
Generalised
Scheme
of
Preferences
(GSP),
<http://trade.ec.europa.eu/doclib/docs/2014/may/tradoc_152502.pdf> accessed 15 October 2014.
276
European
Commission
The
EUs
Generalised
Scheme
of
Preferences
(GSP),
<http://trade.ec.europa.eu/doclib/docs/2014/may/tradoc_152502.pdf> accessed 15 October 2014, pp. 11-13.
68
FRAME
The European Neighbourhood Instrument specifically targets the southern and eastern neighbours of the
EU. In particular they are the South Mediterranean countries and Eastern neighbouring countries (apart
from Russia, which concluded a cooperation framework with the EU outside the ENI). The purpose of this
instrument is to reach close political association and a greater degree of economic integration. The
partner countries agree on an ENI action plan with the EU underlining their commitment to democracy,
human rights, and other core values. The action plans build on already existing agreements with the EU
and their implementation is monitored through committees. ENP progress reports are published once a
year. During the last review of the ENI in 2010-2011 the EU introduced the more-for-more principle, by
which the EU intends to develop stronger partnerships and provide greater incentives for those countries
making a quicker progress towards democratic reform. Progress towards democratic reform will come in
the form of free elections, freedom of expression, judicial independence, etc.277
Still within the period of the previous European Neighbourhood Policy, in the Statistical Annex to the
Implementation of the ENP in 2013, indicators which are deemed to give information about the human
rights situation in a country are listed.278
On 11 March 2014, the new European Neighbourhood Instrument was established through Regulation
232/2014.279 The ENI is a bilateral policy between the EU and each partner country. The objective of this
instrument is to advance further towards an area of shared prosperity and good neighbourliness [...] by
developing a special relationship founded on cooperation, peace and security, mutual accountability and
a shared commitment to the universal values of democracy, the rule of law and respect for human rights
in accordance with the TEU (Art. 1 (1)). An incentive-based approach is new to this Regulation and is laid
down in Art. 4 (1); thus, the EU will differentiate its support for the partners depending on their needs
and also their willingness to improve. This way, the EU will be able to strengthen its support to those
countries willing to improve their democracy and the implementation of human rights.
In Art. 2 of the ENI Regulation, dealing with the specific objectives of Union support, indicators are
mentioned:
The achievement of the specific objectives set out in paragraphs 1 and 2 shall be measured
using, in particular, the relevant Union periodic reports on the implementation of the ENP;
for points a, d and e (i.e. promoting human rights, development and confidence building) of
277
European
Union
External
Action
Service,
European
Neigbourhood
Policy
<http://eeas.europa.eu/enp/index_en.htm> accessed on 15 October 2014.
278
The document lists deep democracy indicators (e.g. Transparency International Corruption Perception Index,
Freedom House Freedom in the World, etc.) as well as other indicators (e.g. on the death penalty, LGBT rights,
Gender Inequality Index, etc.).
Implementation of the European Neighbourhood Policy: Statistical Annex (Joint Staff Working Document)
SWD(2014) 98 final.
279
European Parliament and Council Regulation (EU) No. 232/2014 of 11 March 2014 establishing a European
Neighbourhood Instrument [2014] OJ L 77/27.
69
FRAME
paragraph 2 the relevant indicators established by international organisations and other

relevant bodies [...]. The indicators to measure the achievement of the specific objectives
shall be predefined, clear, transparent and, where appropriate, country specific and
measurable and shall include, inter alia, adequately monitored democratic elections, respect
for human rights and fundamental freedoms, an independent judiciary, cooperation on
issues of justice, freedom and security, the level of corruption, trade flows, gender equality
and indicators enabling international economic disparities, including employment levels, to
be measured.
The progress of partner countries will be regularly assessed, in particular through the ENP progress reports
which include trends compared to previous years (Art. 4 (2)).
In the spirit of differentiation, the programming in the area of the ENI has been streamlined into a Single
Support Framework (SSF). Indicators can be identified by looking into the Sector Intervention Frameworks
that are annexed to the SSF. Indicators which are deemed to give information about the human rights
situation in a country are foreseen especially for the sector on justice reform, as reference is made to the
numbers of recommendations implemented by the National Preventive Mechanisms (NPMs) of the UN
Convention against Torture, as well as the number of human rights cases brought to the courts, etc. 280
Those Sector Intervention Frameworks foresee expected results, indicators, as well as means of
verification such as Council of Europe documents, UN reports, other reports.
For the programming period 2014-2020, strategic priorities and a multi-annual indicative programme for
2014-2017 were set for European Neighbourhood-wide measures.281 One such priority (Objective No 2)
targets higher education and as a result indicators for measuring the progress in the area of education
were established.282
European Development Fund
The European Development Fund (EDF) is the main instrument to provide development assistance to the
African, Caribbean and Pacific countries (ACP countries), and to the Overseas Countries and Territories
(OCTs). The EDF is the financial arm of the Cotonou agreement and strictly speaking not part of the EU
280
See e.g: European Commission, Programming of the European Neighbourhood Instrument (ENI) 2014-2020,
Single Support Framework for EU support to Armenia (2014-2017), <http://eeas.europa.eu/enp/pdf/financing-theenp/armenia_2014_2017_programming_document_en.pdf> accessed 16 October 2014.
281
European Commission, Programming of the European Neighbourhood Instrument (ENI) 2014-2020, Strategic
Priorities 2014-2020 and Multi-annual Indicative Programme 2014-2017, European Neighbourhood-wide Measures,
<http://eeas.europa.eu/enp/pdf/financing-theenp/enp_wide_strategic_priorities_2014_2020_and_multi_annual_indicative_programme_2014_2017_en.pdf>
accessed 16 October 2014
282
European Commission, Programming of the European Neighbourhood Instrument (ENI) 2014-2020, Strategic
Priorities 2014-2020 and Multi-annual Indicative Programme 2014-2017, European Neighbourhood-wide Measures,
<http://eeas.europa.eu/enp/pdf/financing-theenp/enp_wide_strategic_priorities_2014_2020_and_multi_annual_indicative_programme_2014_2017_en.pdf>
accessed 16 October 2014, p. 8 seq.
70
FRAME
budget itself, but financed by the Member States. The EDFs primary goal is the eradication of poverty in
the concerned countries. In the implementation of the cooperation, attention should be paid to the
themes mentioned in the Cotonou Agreement, including among others human rights and democracy.
Within the 10th EDF, action programmes are drafted by the European Commission and the respective
country or partner region in cooperation with the Member State present on the ground. These
programmes outline the general framework and provide for evaluation. They also include a chapter on
expected results formulated in qualitative and quantitative indicators.283
It was expected by experts, that the new, 11th EDF would establish a common set of indicators as opposed
to an ad hoc negotiation for each allocation.284 In May 2014 a Regulation was issued on Financial
Regulation applicable to the 10th EDF as regards the application of transition period until the entry into
force o the 11th EDF.285 In the Annex to the Regulation, replacement Articles are foreseen for Art. 1 to 159
of Regulation (EC) No 215/2008. In Art. 11 on the principle of sound financial management, indicators are
mentioned; specific, measurable, achievable, relevant and timed objectives shall be set and the
achievement of these objectives shall be monitored by performance indicators.286
Development Cooperation Instrument

The Development Cooperation Instrument shall increase the EUs development cooperation by merging
various thematic and geographical mechanisms into one single instrument. The DCI applies to all
developing countries, but not to those eligible for pre-accession assistance. A new regulation for the
Development Cooperation Instrument (Regulation 233/2014) entered into force on 11 March 2014 based
on Art. 209 TFEU and covers cooperation with partner countries and regions, namely Latin America, Asia,
Central Asia, the Middle- East and South Africa.287 The aim of cooperation in this instrument is to reduce,
and in the long term, eradicate poverty. However, consolidating and supporting democracy, the rule of
law, good governance, human rights and the relevant principles of international law shall also be points
this regulation should contribute to (Art 1 b ii). In Art. 1 the use of relevant indicators including human
development indicators, in particular Millennium Development Goal 1 for point a (i.e. poverty) and MDGs
283
European Commission, Budget Support (Publications Office of the European Union 2008), p. 38.
German Development Institute, Allocating the Next European Development Fund for ACP countries of 5
December
2013
<http://www.die-gdi.de/uploads/media/German-DevelopmentInstitut_Mario_Negre_05.12.2013.pdf> accessed 15 October 2014.
285
Council Regulation (EU) No. 567/2014 as of 26 May 2014 amending Regulation (EC) No 215/2008 on the Financial
Regulation Applicable to the 10th European Development Fund as regards the Application of the Transition Period
between the 10th European Development Fund and the 11th European Development Fund until the entry into force
of the 11th European Development Fund Internal Agreement (OJ L 157/52).
286
There is no further more specified information on those indicators to be found in the regulation.
287
European Parliament and Council Regulation (EU) No. 233/2014 of 11 March 2014 establishing a Financing
Instrument for Development Cooperation for the Period 2014-2020 [2014] OJ L 77/44.
284
71
FRAME
1 to 8 for point b (i.e. among others human rights) and, after 2015, other indicators agreed at the
international level by the Union and the Member States. Art. 11 on programming documents for
geographic programmes sets down that multiannual indicative programmes for geographic programmes
shall set out the priority areas selected for Union financing, the specific objectives, the expected results,
clear, specific and transparent performance indicators, the indicative financial allocations, both overall
and per priority area, and where applicable aid modalities. Similar provisions are foreseen for other
strands, such as Pan-African programmes. Those multiannual indicative programmes shall be the centre
of the programming process and EU specific strategy papers as CSPs should in the majority of cases no
longer be needed.288
The DCI is twofold and includes on the one hand, geographic programmes, supporting bilateral or regional
cooperation with developing countries. On the other hand the DCI comprises thematic programmes on
global public goods and challenges and civil society organisations and local authorities.
For the Pan-African programme within the Development Cooperation Instrument, the Multi-Annual
Indicative Programme foresees indicators for each of the priorities, namely peace and security,
democracy, good governance and human rights, human development, sustainable and inclusive
development and growth, and continental integration as well as global and cross-cutting issues.289 Specific
indicators have been formulated for each strategic area, together with priorities and objectives for those
areas. The joint priorities are peace and security; democracy, good governance and human rights; human
development; sustainable and inclusive development and growth and continental integration and global
and cross-cutting issues.290 Within the strategic area of democracy, good governance and human rights
four components were identified, namely African governance architecture; electoral observation and
support; CSOs contribution to good governance and human rights and public finance management. For
each of those components specific indicators were formulated.
EU Budget Support
Another instrument in the area of development aid is the EU Budget Support, which will be presented at
this point as another instrument with a link to human rights measurements.
General budget support from the European Commission is set up to promote development and reduce
poverty by improving living standards (this is measured using health and education indicators, which are
288
European External Action Service, Instructions for the Programming of the 11th European Development Fund
(EDF) and the Development Cooperation Instrument (DCI) 2014-2020 of 15 May 2012.
289
Pan-African Programme 2014-2020 Multiannual indicative Programme 2014-2017, <http://www.africa-eupartnership.org/sites/default/files/documents/c_2014_5375_1_annex_en_v1_p1_771842.pdf>
accessed
15
October 2014.
290
Pan-African Programme 2014-2020 Multiannual indicative Programme 2014-2017, <http://www.africa-eupartnership.org/sites/default/files/documents/c_2014_5375_1_annex_en_v1_p1_771842.pdf>
accessed
15
October 2014, p. 13.
72
FRAME
human rights indicators as such).291 The financing agreement, the formal fixation of the contract with the
country in question specifies the performance criteria and indicators for payments.
Budget support is an aid modality, as it is not an end in itself, but [...] means of delivering better aid and
achieving sustainable development results.292 There are three forms of budget support programmes:
Good Governance and Development Contracts to provide budget support to a national development or
reform policy and strategy; Sector Reform Contracts to provide budget support to address sector reforms;
and State Building Contracts to support in fragile and transition situations.293
The commitment to fundamental values is assessed as a pre-condition for signing the so-called Good
Governance and Development Contracts (GGDCs), and subsequently is monitored during the
implementation of budget support. Monitoring during implementation is based on a dynamic approach,
looking at past and recent policy performance benchmarked against reform commitments. This works on
the basis of process and outcome indicators, as well as extensive discussions with the Governments
through budget support dialogues. The indicators should be drawn from the respective countries national
and/or sectorial development policy and a mix of process, output and outcome indicators should be
foreseen. Indicators and targets are to be reviewed each year in the course of the annual review and
adapted according to lessons learned.
Partnership-Instrument for Cooperation with third Countries
The Partnership Instrument for cooperation with third Countries (Regulation 234/2014) was set up
through Regulation 234/2014 and adopted as of 11 March 2014, as the successor of the instrument for
cooperation with industrialised Countries (2007-2013).294 Its objective is to advance and promote EU
interests by supporting the external dimensions of EU internal policies and addressing major global
challenges. In the recitals of the regulation the following section on indicators was introduced:
The Union's external action under this Regulation should contribute to clear results (covering
outputs, outcomes and impacts) in countries benefiting from Union assistance. Whenever
appropriate and possible, the results of the Union's external action and the efficiency of the
instrument established by this Regulation should be monitored and assessed on the basis of
pre-defined, clear, transparent and, where appropriate, country-specific and measurable
indicators, adapted to the specificities and objectives of that instrument.
The material part of the regulation does not contain any further provision in this regard. As for other
instruments already mentioned, Multiannual Indicative Programmes shall be adopted for the Partnership
Instrument. The first MIP predicts within the programming principles that indicators of efficiency,
291
European Commission, Budget Support (Publications Office of the European Union 2008), p. 20.
European Commission, The Future Approach to EU budget Support to Third Countries COM(2011) 638 final.
293
European Commission, Tools and Methods Series Working Document Budget Support Guidelines,
Programming, Design and Management A Modern Approach to Budget Support of September 2012.
294
European Parliament and Council Regulation (EU) No. 234/2014 of 11 March 2014 establishing a Partnership
Instrument for Cooperation with Third Countries [2014] OJ L 77/77.
292
73
FRAME
effectiveness and impact should be provided.295 In the MIP, indicators are indeed mentioned for each of
the priorities (energy security, climate change, environment, trade, enhancing understanding and visibility
of the EU and its role on the world scene), but not formulated in detail. Some of the indicators roughly
listed in this MIP are closely linked to human rights (e.g. positive impact on the quality of higher education;
positive impact on the development of individuals).296
Indicators for the comprehensive approach to the EU implementation of the UN SC Res 1325 and 1820
on women, peace and security
By 2008 the Council had already adopted four indicators regarding women and armed conflict. In 2010
the Political and Security Committee (PSC) endorsed the indicators of the UN SC Res 1325 and 1820, and
transmitted the indicators to evaluate the EU implementation of Res 1325 and 1820 to the Permanent
Representatives Committee (COREPER).297 The EU Comprehensive Approach includes a commitment to
elaborate indicators, based on the Beijing +15 indicators regarding protection and empowerment of
women in conflict and post-conflict situations. The indicators were developed in a number of meetings of
the Informal Women, Peace and Security Task Force that commenced in April 2010. Meetings were open
to other participants, such as NGOs, UN agencies, etc. Also prior meetings took place with grassroots
organisations from conflict and post-conflict countries. The indicators are also based on a set of 26
indicators by the UN, with UNIFEM having the technical lead.298
The Task Force tried to compile indicators that were achievable, directly measurable (data available),
specific and relevant. Measurement concentrates on the implementation process and the steps taken,
without further specifications mentioned in the document.299 The UN indicators will be used to
complement the EU indicators. Seventeen indicators were developed, including topics such as number of
regional level dialogues that include specific attention to women, peace and security in outcome
documents, conclusions, and targets. Looking into the documents the indicators seem to be a mixture of
quantitative and qualitative indicators, e.g. asking for numbers of partner countries in specific areas or
modalities and tools. However, a focus is laid on quantitative data.
295
European Commission, Partnership Instrument - First Multi-annual Indicative Programme for the Period 20142017, <http://ec.europa.eu/dgs/fpi/documents/pi_mip_annex_en.pdf> accessed 14 October 2014.
296
European Commission, Partnership Instrument - First Multi-annual Indicative Programme for the Period 20142017, <http://ec.europa.eu/dgs/fpi/documents/pi_mip_annex_en.pdf> accessed 14 October 2014, p. 23.
297
Council of the European Union, Item Note Indicators for the Comprehensive approach to the EU
Implementation of the United Nations Security Council Resolutions 1325 and 1820 on Women, Peace and Security
11948/10 of 14 July 2010.
298
11948/10 of 14 July 2010, p. 6.
299
11948/10 of 14 July 2010, p. 7.
74
FRAME
On the basis of the indicators developed, questionnaires were sent out to all EU Member States 36 EU
delegations, as well as other actors such as EU Special Representatives, or Common Security and Defence
Policy (CSDP) missions. The response-rate differed, from 64% regarding EU missions to 89% of the
Member States. This first round of assessment showed, that the information provided did not always
allow easy compilation and further instructions: it was found that reviewing some of the indicators might
be necessary.300
(3)
Evaluation
Annual Report on Human Rights and Democracy in the World

As already described above in the policies section, the Strategic Framework and the Action Plan on Human
Rights and Democracy were adopted in 2012 and now constitute key documents for furthering the EUs
human rights policy in third countries. The Action Plan foresees concrete actions, identifying the
responsible EU institutions and a timeframe for completion. The EUs progress is assessed in the Annual
Report on Human Rights and Democracy in the World.301
Up to date the Annual Report is the main tool to evaluate the EUs human rights and democracy policies.
It has been published since 1999. Since the adoption of the Strategic Framework and the Action Plan the
Annual Report follows the structure of this document in order to allow for a thorough overview of
implementation of actions within the respective year. This report acts as a catalogue of the EUs action
and also allows the reader to see where further progress is needed. It does not use a coherent set of
monitoring tools or even a set of universal indicators, but presents the developments in a descriptive
manner on specific topics.
European Instrument for Democracy and Human Rights302
As already indicated by the name, the European Instrument for Democracy and Human Rights is a specific
instrument to promote democracy and human rights. EIDHR was first established in 2006 by way of
Regulation 1889/2006 with the main aim to provide financial assistance in order to contribute to
developing and consolidating democracy, the rule of law and respect for human rights. The current
Regulation 235/2014 covers the period from 1 January 2014 until 31 December 2020.303 The EUs
cooperation with civil society actors is especially high on the agenda of this instrument. Assistance by
300
Council of the European Union, Note Report on the EU Indicators for the Comprehensive Approach to the EU
Implementation of the UN Security Council UNSCRs 1325 and 1820 on Women, Peace and Security 9990/11 of 11
May 2011, p. 2.
301
11855/12 of 25 June 2012.
302
On this instrument see D 12.1: Christina Churruca Muguruza and others, Report mapping Legal and Policy
Instruments of the EU for Human Rights and Democracy Support (2014) Frame Deliverables, <http://www.fp7frame.eu/wp-content/materiale/reports/05-Deliverable-12.1.pdf> accessed on 14 October 2014.
303
European Parliament and Council Regulation (EU) No. 235/2014 of 11 March 2014 of the European Parliament
and of the Council establishing a Financing Instrument for Democracy and Human Rights Worldwide [2014] OJ L
77/85.
75
FRAME
EIDHR is not dependent on agreements with the respective countries, but can also be given to other
parties such as NGOs without the consent of the government. The new Regulation foresees certain priority
themes in Art. 2, such as:
-
Support to an enhancement of participatory and representative democracy;
Promotion and protection of human rights and fundamental freedoms;
Strengthening the international framework for the protection of human rights, justice, gender
equality, the rule of law and democracy, and for the promotion of international humanitarian law;
Building confidence in democratic electoral processes and institutions. Non-discrimination,

gender mainstreaming, participation, empowerment, accountability, openness and transparency
are introduced as cross-cutting themes for all measures of the Regulation.
The assistance shall be implemented through strategy papers setting out the priority areas selected for
financing, the specific objectives, the expected results, the performance indicators, and the indicative
financial allocations.304
In 2012 EIDHR presented a study on performance indicators for EU parliamentary support, providing a
guide on how to take performance indicators into account when implementing parliamentary support
projects.305
So far EIDHR has not developed a coherent set of indicators for measuring the output and/or impact of its
work. Still on its website EIDHR presents a couple of measurement schemes such as the Press Freedom
Index.306
d)
Conclusion
Looking into the considerable number of instruments presented above, it can be clearly concluded that
there is no systematic EU-approach to assess/monitor human rights situations or measure human rights
as such. However, it seems that in certain areas, such as development cooperation, a paradigm shift has
taken place from a particulate to a more comprehensive but flexible approach, that uses indicators for
assessing human rights in different countries.
Regarding the data used in the course of those instruments presented above it is noteworthy that quite a
few instruments make reference to already existing sources (for example Transparency International
304
European Parliament and Council Regulation (EU) No. 235/2014 of 11 March 2014 of the European Parliament
and of the Council establishing a Financing Instrument for Democracy and Human Rights Worldwide [2014] OJ L
77/85, Art. 4 and 5.
305
European Commission EIDHR, Strengthening Democracy Support to EU Delegations: from Performance
Indicators, Knowledge sharing to Expert Services Study on Performance Indicators for EU Parliamentary Support
of September 2012.
306
European Commission - EIDHR, Human Wrong Index, <http://www.eidhr.eu/human-wrongs-index> accessed 30
October 2014.
76
FRAME
Corruption Perception Index, Freedom House Freedom in the World and various others), which they also
take into account in their assessment.
While this compilation presents a variety of instruments established by the EU institutions for human
rights related assessment and measurement in the internal, enlargement and external area, little is known
about how they work in practice.307 In order to address these practical matters, interviews with relevant
EU officials have been carried out. The findings of these interviews provide insights into these matters and
will be presented in the next section.
2.
Assessing current EU measuring instruments and identifying future
requirements
This subchapter deals with the assessment of current EU instruments and the requirements identified by
interviewees for future instruments to measure human rights. It is based on the information gathered
through the interviews with EU officials. The aim of this subchapter is to get practical insights into human
rights measurement in the EU through the perspectives of those involved.
The interviewed EU officials were asked about the strengths and weaknesses, as well as about the lessons
learnt regarding currently applied human rights measurement tools. Interviewees were also asked about
data requirements. Is qualitative or quantitative data preferred and is comparability important? What are
the suggestions regarding the frequency of applying measuring instruments? The scope of human rights
measurement was discussed too.
Despite being confronted with a definition of human rights measurement (see chapter II B) the
perceptions on human rights measurement differ among the interviewees. Thus, the following remarks
are about measurements which EU bodies themselves perceive as such. They provide information on
which role human rights measurement plays in the daily work of human rights experts.
The information gathered through the interviews cannot be seen as exhaustive; neither for human rights
(related) measurement applied by EU bodies in general, nor for the interviewees respective departments.
Still the findings provide valuable information on how human rights measurement is carried out in
practice. The users perspective is an important complement to the mapping presented in the previous
section.
The discussion of findings is divided into two sections. Firstly the interviewees perceptions on currently
applied measurements are presented. Analysis is structured along the different approaches to human
rights measurement as identified through the interviews. Secondly the requirements stated by the
respondents on future human rights measurements will be discussed.
307
For further details regarding the use of rule of law indicators in the EU enlargement process of Bulgaria, Croatia
and Montenegro see Annex II of this report.
77
FRAME

a)
Evaluating the initiatives taken by EU bodies to measure human
rights
The findings from the interviewees perceptions on self-applied human rights measurement are strongly
related to their mandates. Generally, the findings indicate two approaches to human rights measurement,
which differ from each other regarding their purposes. On the one hand, there is human rights
measurement that pursues fundamental rights as the end in itself. On the other hand, there is human
rights measurement that is mainly policy-related. Within policy-related human rights assessment, two
types can be found: ex ante impact assessment and monitoring human rights. The latter measures
countries compliance with human rights standards (internal) and obligations (enlargement, external), as
well as countries progress/regress in this regard.
(1)
Research on fundamental rights
The FRA collects and analyses data on fundamental rights in the EU, basing its work on human rights
standards. For its work on indicators, FRA uses the conceptual framework on indicators developed by the
United Nations Office of the High Commissioner for Human rights. According to the interviewed FRA
member, the methodology of the OHCHR mechanism provides comparability and it is complex but not
over-complicated.308 Through the structure/process/output classification, this mechanism is perceived to
be suitable to translate international/EU human rights norms into everyday life, thus this division is
considered to be very useful by the FRA.309 Structural indicators are about states duties, they reflect the
ratification and adoption of legal instruments and the existence as well as the creation of basic
institutional mechanisms deemed necessary for the promotion and protection of human rights.310 They
are quite easy to populate with data, since data is accessible, and changes at the structural level do not
take place very often. Process indicators measure the actual activity of the duty bearers to transform
their human rights commitments into the desired results.311 Process indicators encompass strategies,
action plans and other activities of the duty bearers. The FRAs experiences show that the process
indicators are more complicated to apply. Outcome indicators look at the rights holders actual enjoyment
of human rights: an outcome indicator consolidates over time the impact of various underlying processes;
it is often a slow moving indicator, less sensitive to capturing momentary changes than a process
indicator.312 The FRA has already developed indicators on the right to political participation of persons
with disabilities.313 Examples for structure, process and outcome indicators in this regard can be found on
308
Interview conducted by the ETC with an EU official from the FRA (Vienna, June 2014).
310
311
312
313
Human Rights Indicators (Publications Office of the European Union 2014).
309
78
FRAME
their website.314 According to the interviewed member of the FRA, it is difficult to gather data on the
situation on the ground to populate outcome indicators and in practice often only proxies are available.
The FRA uses qualitative data for structure indicators, qualitative and quantitative data for process
indicators, and mostly quantitative data from independent sources for outcome indicators which are
usually individually gathered for the respective purpose.315
(2)
Ex ante impact assessment of policy options
Interviewed staff of EU bodies who deal with legislation, namely DG JUST and DG HOME, conduct ex ante
impact assessments of policy options. DG HOME coordinates infringement procedures and legislative
proposals. Regarding the latter, the DG assesses compliance with the Charter of Fundamental Rights in
the European Union. Therefore, they examine different policy options in order to assess the (negative or
positive) impacts on fundamental rights in the preparatory stages of new legislation. There are guidelines
and checklists available for ex ante impact assessment, but no indicators are used. According to the
interviewed member of DG HOME, the main drivers for legislative proposals are gaps in legislation and
eliminating barriers to the actual enjoyment of human rights. Usually, these gaps are identified through
NGO reports, or data from FRA or EUROSTAT. Furthermore, Parliament resolutions, initiatives of Member
States, and EU level political goals are drivers of legislative proposals. Although these EU officials do not
explicitly measure human rights (which is outsourced to other DGs), they mainstream human rights in
all stages of elaborating policy proposals. DG HOME assesses the human rights impact of legislation in the
preparation, presentation and transposition of policy proposals. When monitoring the implementation
and application of EU law, as well as in the context of infringement proceedings, the human rights
dimension is considered too. For both tasks, the DG resorts to data from other sources, such as the FRA,
NGO reports, international organisations, government or local authorities.316
The Parliaments Impact Assessment Unit was established in 2012 as part of the new Directorate for
Impact Assessment and European Added Value. This unit provides a detailed assessment of the
Commissions Impact Assessment (IA) (e.g. to check the quality and independence of this IA and its
relevance for parliamentary work). It additionally assesses the impact on substantive amendments being
considered by the Parliament (carried out by external experts).
(3)
Human rights assessment
The findings of desk research as pointed out above are confirmed by the findings of the interviews. The
largest share of human rights measurement is carried out within the framework of human rights
assessment, which usually means assessing the human rights situation in the EU internal, enlargement
and external area, and reporting on this. In the internal area this means assessing compliance with the
314
European Agency for Fundamental Rights, Indicators on the Right to Political Participation of Persons with
Disabilities,
<www.fra.europa.eu/en/publications-and-resources/data-and-maps/comparative-data/politicalparticipation> accessed 4 December 2014.
315
316
Interview conducted by the ETC with an EU official from DG HOME (Brussels, June 2014).
79
FRAME
Fundamental Rights Charter, while in the enlargement and external area this means assessing compliance
with international human rights standards and obligations.
This approach to human rights measurement is primarily adopted by the European Parliaments
Committees FEMM, DROI, LIBE and INTA (parliamentary reporting) and additionally by the Commissions
DGs ENLARG, DEVCO, EMPL and the SPC.
For human rights assessment, the interviewed EU bodies use different tools ranging from standardised
methods including coherent indicator systems, to unstructured analysis. Some of these bodies develop
indicators related to certain human rights, such as the indicator sub group of the SPC in the area of social
exclusion and poverty. When carrying out reporting on human rights, however these bodies usually resort
to data from established measurement tools, such as the FRA, EIGE, EUROSTAT, OSCD and EU-SILC data.
Others resort to information from the Bertelsmann Foundation, the NGO Freedom House, the World
Bank, and the Ibrahim Foundation for Africa. EMCO and SPC make use of a shared dataset and
methodology to have a common basis for decision making. This is the Open Method of Coordination
(OMC) for social policies, adopted by the Council (see section II A).317
When assessing human rights, the majority of the interviewed EU officials do not carry out their own
surveys/measurements. These interviewees do not perceive themselves as human rights measurers. This
is probably because the mandate of these bodies does not explicitly cover human rights measurement,
but is rather related to assessing and reporting.
DROI uses an informal set of indicators, and data from the UN and the Council of Europe but a system of
benchmarking based on indicators is not yet implemented. Interviewees perceive this secondary analysis
as problematic because the outcomes of human rights reports and data used differ and evaluation is
difficult. Thus, they are strongly in favour of a coherent set of indicators as they can also be used in their
statements, opinions and annual reports.318 For the Annual Report on the Situation of Fundamental Rights
in the EU, members of the Committee LIBE use data made available by the Commission, the Council of
Europe, Member States, the FRA and other human rights institutes. If legal opinions are needed, LIBE
consults the Parliaments legal department.319
In international trade relations, human rights assessments are carried out when applying GSP and the
GSP+. The Commission and the EEAS are required to assess whether or not benefiting countries meet the
requirements according to the GSP. They also monitor progress in this regard. Thereby they apply a
standardised set of indicators and a structured reporting mechanism. According to the interviewed civil
317
Interview conducted by DIHR with EU officials from DG EMPL and members from the Indicators Subgroup of the
Social Protection Committee (Copenhagen, June 2014).
318
Interview conducted by the ETC with civil servants from the Parliaments Subcommittee DROI (Brussels, June
2014).
319
Interview conducted by the ETC with civil servants from the Parliaments Subcommittee LIBE (Brussels, June
2014).
80
FRAME
servants at INTA, this assessment is working well and has been updated in 2014.320 An interviewed
member of EEAS also points out that the sanctions-regime itself is a nuclear option to strengthen human
rights.321
Regarding development aid and budget support, DG DEVCO uses a variety of tools including indicators on
human rights, democracy and the rule of law. These tools are used for the design, as well as for the impact
assessment of development aid policies. For the broad multi-annual programmatic level (which currently
covers the period 2014-2020), more standardised and general tools and indicators are applied. Therefore,
a focus lies on UN level standards as available, reliable and recognised information sources. DG DEVCO
sometimes commissions data collection to provide evidence for decision making.322
As the findings of desk research show, a more standardised system of indicators is applied in development
aid, since beneficiary countries are required to comply with certain criteria. An interviewed member of
DG DEVCO who is involved in the assessment of the beneficiary countries achievement, confirms this.
Additionally, he refers to fundamental values and risk assessments (e.g. corruption), which are carried out
by the Commission at country level.323
Particularly in the external and enlargement areas, assessing human rights is carried out through on-site
guided reporting performed by EU bodies (EU delegations, ambassadors) and by thematic experts. These
on-site assessments and reporting are more (e.g. enlargement) or less (e.g. human rights country
strategies) structured by internal guidelines or checklists, but do not necessarily build on indicators that
are rooted in human rights standards.
In the EU enlargement area, reporting on human rights and the rule of law is made by peer review missions
and experts. These reports are conducted for each candidate country. The focus lies on the capacity of
the candidate countries institutions to implement the acquis communautaire. Peer review reports are
available for most of the countries on a yearly basis.324 According to the interviewed members of DG
ENLARG, this is more closely linked with political reporting than the application of a human rights
measurement.325 Information gathered through this reports is supplemented by the use of existing
monitoring mechanisms and data from several international organisations.
Also in the external area, EU delegations perform regular on-site reporting on a weekly or monthly basis.
For countries where the human rights situation is of concern and a priority for the EU, additional ad hoc
320
Interview conducted by the ETC with civil servants from the Parliaments Subcommittee INTA (Brussels, June
2014).
321
322
Interview conducted by the ETC with an EU official from DG DEVCO (Brussels, June 2014).
323
324
Interview conducted by the ETC with an EU official from DG Enlargement (Brussels, June 2014).
325
Interview conducted by the ETC with an EU official from DG Enlargement (Brussels, June 2014).
81
FRAME
reports are prepared. These reports either focus on the HR situation in general or on specific human rights
issues.326
EU Delegations (ambassadors) and thematic experts who carry out guided reporting are located in most
countries around the world. An interviewed member of the EEAS perceives it as a good source of
information.327 The ad hoc assessments conducted by EU diplomatic representations (Head of Missions)
are the basis for the human rights priorities set by the EEAS.328
Human rights assessment through guided reporting by EU delegations also plays a role in aid funding and
development cooperation. DG DEVCO administers development funds together with partners and
stakeholders, ranging from Intergovernmental Organizations (IGOs) to small local NGOs. The interviewed
member of DG DEVCO mentions that EU delegations are advised to use indicators and benchmarks, which
allow for measuring each countrys achievements and progress in relation to the set objectives. In
practice, the EU delegations do not always apply all or constant indicators. Indicators and benchmarks are
applied flexibly according to country specific relevance, sector and topics at stake. Furthermore, the kind
of indicators used depends on the size of the development cooperation programme and the stakeholders
involved. Benchmarks, indicators and objectives are very carefully chosen in order to be relevant and
meaningful to country specific development aid projects. They also need to allow for measuring the
progress concerning the development aid measures taken and the achievements in relation to the set
objectives. DG DEVCO already tried to develop one set of indicators to be used in development
cooperation, but since situations, topics, programmes and countries are too different to be directly
addressed by it, they decided to use more country and topic specific indicators.329
As these few examples indicate, throughout the different DGs, different reporting mechanisms are
developed and applied in external action and enlargement.
b)
Requirements for future mechanisms
The interviewees were asked about requirements on future human rights measurements including
requirements on data (qualitative and/or quantitative). The findings from the interviews with EU officials
indicate the need for a system of indicators and benchmarks to allow standardised and comparative
human rights measurement in the internal and the external area. The findings generally indicate that there
is a strong need for reliable information and data on the human rights situation on the ground of each
country in focus. Apart from this, the following requirements for a mechanism to measure human rights
can be deduced.
326
328
329
Interview conducted by the ETC with an EU official from DG DEVCO (Brussels, June 2014) and ETC Graz, Minutes
Panel 3 Human Rights Indicators, Speech of Jean-Louis Ville (DG DEVCO) at the FRAME General Assembly, Leuven,
June 2014.
327
82
FRAME

(1)
Allow comparative and differentiated analysis
A mechanism to measure human rights is required to allow a comparative analysis of countries on the one
hand, and a differentiated analysis including contextual information about each country on the other.330
The requirement of comparability refers to the need for a common methodology and comparable country
data to be applied to all states.331 To achieve this requirement, indicators suitable to assess a diversity of
legal and political systems are needed.332 These indicators should not be too context specific in order to
remain comparable and applicable.333
Additionally, interviewees particularly point out the need for regularly and accurately gathered statistical
data, which can be subjected to comparative analysis. Quantitative data and indicators are better suited
to identify the magnitude of problems and to deduce suggestions for policies. Furthermore, only
quantitative data allows comparability of results.334 Identified gaps concerning quantitative data depend
on the interviewees respective body mandate and concern socio-economic rights, homophobia,
racism,335 and violence against women.336 Regardless of the issue, interviewees identify a lack of
comparative data.
On the other hand, quantitative indicators are perceived as artificially constructed, which give an
aggregated picture of subjects of qualitative nature. Thus, they are rough proxies and do not allow a
deeper look into more differentiated country situations.337 Some interviewees identified a focus on
quantitative data in current human rights measurements and perceived this as a common weakness of
the instruments in use. These interviewees criticised the lack of a possibility to assess the qualitative
dimension of human rights338 and to integrate different country standards into analysis.339 Some
interviewees even stress the lacking comparability of countries and suggest a general differentiation
between EU countries, candidate countries, and third countries when measuring human rights. However,
if a mechanism to measure human rights would also allow a differentiated (country specific) analysis, this
differentiation would not be necessary. Thus, a mechanism to measure human rights is also required in
330
This requirement is also pointed out in the 2012 annual report of the European Parliament in terms of the claim
for an immediate activation of the new Copenhagen mechanism: European Parliament, Resolution on the Situation
of Fundamental Rights in the European Union (2012) of 27 February 2014 (2013/2078(INI)) P7_TA-PROV(2014)0173,
pp. 15-16; Anja Mihr, Human Rights Benchmarks for EU's External Policy (Publications Office of the European Union
2011).
331
Interview conducted by the ETC with civil servants at the Parliaments Subcommittee LIBE (Brussels, June 2014).
332
333
Interview conducted by the ETC with civil servants at the Parliaments Subcommittee INTA (Brussels, June 2014).
334
Telephone interview conducted by the ETC with an EU official from the Parliaments Subcommittee FEMM (Graz,
May 2014).
335
Interview conducted by DIHR with an EU official from DG JUST (Brussels, June 2014).
336
Telephone interview conducted by the ETC with a member of the Parliaments Subcommittee FEMM (Graz, May
2014).
337
338
339
Telephone Interviews conducted by the ETC with EU officials from DG Enlargement (Graz, June 2014).
83
FRAME
order to facilitate the integration of contextual country information to enable treating different situations
with different tools.340
Therefore, the majority of the interviewees prefer a mix of quantitative and qualitative indicators. The
implementation of a standardised set of indicators and benchmarks (for comparability reasons)
supplemented by qualitative indicators (for contextual information) is suggested.341 An aggregated onesize-fits-all indicator system is believed to be insufficient to meet this double requirement of
comparability and differentiation.342
(2)
Consistent and broadly accepted
The issues of coherence and consistency were addressed by most of the interviewees. Particularly those
working in the external area identified double standards and a lack of coherence in the EUs internal and
external human rights strategy as a weakness of current instruments.343 Some of them believe that a
standardised system of human rights indicators and benchmarks, implemented at European level, could
help reduce incoherence. To address this issue, such a standardised tool to measure human rights is
required to follow a methodology and apply a terminology that is broadly accepted by both EU Member
States, and third countries. Meeting this requirement is perceived as a key challenge by some
interviewees.344
The interviewed FRA member additionally stressed challenges in accessing information pertaining to the
situation on the ground, and making concepts operational. According to the interviewees, a consistent
human rights measurement requires an acknowledgement of the interrelated character of human rights.
Therefore, a mechanism to measure human rights should not prioritise certain human rights. However, a
particular focus might be chosen according to the specific topic under analysis. One interviewee pointed
out that such a mechanism is also required in order to help identify the drivers for improving human rights
and to provide the empirical proof for arguments related to these drivers. Additionally, human rights
measurement must not be limited in its applicability to a particular geographical area. It rather needs to
be generally applicable to all countries worldwide. To meet the requirement of consistency and
coherence, human rights measurements must be applicable for external and internal policies.
A consistent human rights measurement is also needed to mainstream human rights within the EU.
According to an interviewed member of DROI, awareness on the cross-cutting character of human rights
is still low among EU bodies. In particular, EU bodies dealing with international trade relations are not
aware that their work is also human rights related. As a consequence, DROI has no access to information
340
Interview conducted by the ETC with civil servants at the Parliaments Subcommittee LIBE (Brussels, June 2014).
342
Interview conducted by the ETC with an EU official from DG HA (Brussels, June 2014) and Interview conducted by
the ETC with an EU official from the FRA (Vienna, June 2014).
343
344
Interviews conducted by the ETC and the DIHR with an EU official from EEAS (Graz and Copenhagen, June 2014)
and interview conducted by the ETC with an EU official from the FRA (Vienna, June 2014).
341
84
FRAME
about the human rights related outcomes of their policies (e.g. human rights clauses, trade of torture
equipment).345
(3)
Flexible frequency of application
The interviewees suggestions regarding the frequency of application range between more than once a
year to every five years. Some interviewees believe an annual application would be ideal, but have
admitted that it would be too ambitious, and so have agreed that every second year would be more
realistic.
According to the FRA member, the frequency of application depends on the spectrum of rights, on the
issue, and on the part (structural, process, outcome) of human rights measurement. As some of these
aspects change more often than others, the frequency of applying human rights measurements is required
to be flexible. This interviewee also considers survey fatigue and costs as arguments against an application
that is too frequent. Generally frequency of application depends on the resources available.
(4)
Proving causality between measure and impact in project
evaluation
Particularly interviewees dealing with human rights in the external area pointed out the need for an
evaluation of human rights policies or projects. For some human rights (related) tools, such as the Human
Rights Dialogues, evaluation is unscheduled (further information in section II B). Human rights
measurement in policy evaluation is required in order to prove causality between measure and impact.346
Thus a solid method for project monitoring and benchmarking is needed. It is doubted whether policy
evaluations are human rights measurements in a narrow sense, but human rights indicators could feed
into such assessments as performance indicators.
c)
Conclusion
The aim of this subchapter was to find out how human rights measurement is carried out or used by EU
officials in practice and to identify requirements and needs for future human rights measurements.
The mapping shows that the majority of instruments in place support human rights related monitoring
and assessing compliance with human rights standards in both the internal, and the external area. Some
instruments are in place for policy planning, and only a limited number of instruments focus explicitly on
policy evaluation. The newly implemented instruments within the 2014-2020 Multiannual Financial
Framework indicate a trend towards a strengthened role of policy evaluation.
Also the findings of the interviews indicate that human rights measurement is mainly carried out in the
course of monitoring policy implementation.
345
Interview conducted by the ETC with EU officials from the Parliaments Subcommittee DROI.
Interview conducted by the ETC with EU officials from the Parliaments Subcommittee DROI and with an EU official
from the Councils Human rights Working Group COHOM (Brussels, June 2014).
346
85
FRAME
Apart from the FRA, the interviewed EU bodies do not explicitly measure human rights in the sense of
developing and applying human rights indicators, which are then populated with current and comparable
country data. The methodology applied by the majority of the interviewed EU officials is secondary
analysis of reports, and existing data and political reporting. While within the former, indicators are indeed
applied, the latter sector uses mainly guidelines and checklists.
Furthermore, human rights measurement is strongly related to the interviewed bodies mandate. Apart
from FRA, EEAS, COHOM and DROI, the bodies mandates do not deal with the entire human rights system,
but rather with certain areas of or with specific human rights. These are for example social protection,
gender equality, child protection, or the rule of law. The measurements currently applied (Gender Equality
Index, Joint Assessment Framework) reflect this specialisation.
The understanding of what measuring human rights can actually denote, is quite diverse among the
interviewed officials and the bodies they represent. Although they repeatedly stress that human rights
are a core value of the EU, which must be followed coherently and consistently, it can be assumed that
the initiatives for measuring human rights have not all been developed to their full potential. Some
interviewees realised only in the course of the interview that human rights measurement could be
important for their work.
Thus, it can be concluded that there is no systematic approach by the EU to measure human rights; there
is not even a systematic cooperation or collaboration of EU bodies when measuring or assessing human
rights. The findings of the interviews indicate a need for human rights measurement among EU bodies
and officials.
86
FRAME
III. Refining the European Unions approach to measuring human

rights
Section II of this report revealed that human rights measurements are beneficial for EU bodies at all stages
of the policy cycle, namely policy planning, implementation and monitoring, as well as evaluation. It was
further found that three types of human rights measurements are currently applied by EU bodies during
these stages, as in practice these measurements take the form of human rights research, human rights
assessments, as well as impact assessments of legislative proposals. Although these measurement
activities are done by different stakeholders and despite the fact that they are very different in nature,
they all share a simple, albeit essential common denominator: they necessitate accurate and reliable
information on human rights situations.
The strife for information on human rights situations is shared by many stakeholders beyond the EU
institutions and bodies. Over the recent years, human rights measurements have increasingly attracted
the attention of academic experts, international, regional and local organisations, states, human rights
monitoring bodies, human rights activists, NGOs and further civil society actors. Depending on the
conception and purpose of the measurement, different methodologies and tools were used to analyse
either the overall situation in terms of general trends within a country or to determine the performance
of a specific government, or the practical situation on the ground.347 As a matter of fact, the target groups
and purposes, the substantive human rights measured, as well as the geographical area of interest vary
considerably from one measurement model to another.
The development of a tool to measure human rights, therefore, is everything but a new endeavour for
many human rights stakeholders. As numerous approaches to measuring human rights have been
proposed on the international, regional, national and local levels, the EU bodies do not necessarily have
to reinvent the wheel, but are well advised to take account of this existing work on human rights
measurements when refining their tools for human rights measurements. This section therefore aims to
elucidate key findings of an analysis of the diverse existing approaches to measure human rights. This is
done with a view to propose applicable measurement tools that meet both, the quality standards of the
state of human rights measurement and the requirements identified by EU officials.
A.
Review of existing approaches in measuring human rights
A review of existing attempts to develop indicator-based human rights measurements is an indispensable

starting point for the selection of appropriate human rights measurement tools. Hitherto prominent
347
Todd Landman and Julia Husermann, Map-Making and Analysis of the Main International Initiatives on
Developing Indicators on Democracy and Good Governance (Human Rights Centre of the University of Essex 2003),
< http://www.humanrightsimpact.org/resource-database/publications/resources/view/84/user_hria_publications/
> accessed 4 March 2015, pp. 3-4.
87
FRAME
stock-takings of human rights measurements have been made by Malhotra and Fasel348 in the course of
the development of indicators for the United Nations Office of the High Commissioner for Human Rights,
by Payne and Schaefer349 in the course of the development of a human rights measurement framework
for the Equality and Human Rights Commission (EHRC) and the Scottish Human Rights Commission (SHRC),
and by Landman350 in a broader study commissioned by EUROSTAT.351
However, a mapping that takes into account the particular requirements of the EU bodies has not been
conducted so far. Therefore, an up-to-date, yet non-exhaustive mapping of human rights measurement
tools and related initiatives (partly) resorting to (human rights related) indicators was completed for this
report in the course of the FRAME project. Classification criteria that are of particular relevance for human
rights measurement systems for the EU have been identified. The actual mapping exercise was then
conducted by way of desk research. In the following, an overview of the mapping criteria applied is given
with a short explanation of the respective guiding questions.
Mapping criterion
Guiding questions
Type of actor
For and by which type of actor has the indicator system been
developed? Was it, for instance, an international or regional
organisation, a state, an NGO, or a monitoring body?
Thematic context
What is the thematic context in which the measurement

system has been developed? Was it, for instance, development
aid, rule of law, or discrimination?
Rationale and purpose
What is the rationale and purpose of the indicator scheme? Is

it, for instance, monitoring, situational analysis, informing
public policy, measuring progress or implementation,
identifying violations of state obligations, early warning, etc.?
Users and target group
Who are the users and the target group of the indicators? Are
these, for instance, researchers, the civil society, politicians,
monitoring bodies, media, etc.?
348
Rajeev Malhotra and Nicolas Fasel, Quantitative human rights indicators: A survey of major initiatives (Nordic
Network Seminar in Human Rights Research, bo, 1013 March 2005).
349
Anne Maree Payne and Laura Schaefer, Human Rights Measurement Framework: EHRC/SHRC Specialist
Consultation on Selection of Indicators, Long-list of Indicators and Measures (2010),
<http://personal.lse.ac.uk/prechr/hrmf/HRMF_longlist_of_indicators_and_measures.pdf> accessed 4 March 2015.
350
> accessed 4 March 2015.
351
See also the study on human rights benchmarks for the European Parliament's Subcommittee on Human Rights:
Anja Mihr, Human Rights Benchmarks for EU's External Policy (Publications Office of the European Union 2011).
88
FRAME
Reference to human rights
Do the indicators explicitly refer to human rights? In case

human rights are addressed only implicitly: Is the indicator
scheme expected to yield results that could be used for
measuring human rights?
Human rights covered
What (areas of) human rights are covered by the measurement

system?
Number of indicators
Are indicators used for the measuring system? How many

indicators form the measurement system?
Types of indicators
Does the measurement system comprise quantitative (i.e.,

numbers, statistics and tables) or qualitative (i.e. based on
narratives, estimates, assessments) indicators, or both?
Geographical area covered
What geographical area is covered by the indicators? Are they

measuring human rights at the local, regional, national, or
international level? Are they restricted to a number of specific
countries, etc.?
Methodology applied
What methodology has been applied for the development of

indicators? Have there been, for instance, participatory
approaches, expert consultations, etc.?
Data sources
What data sources are to be used for populating the indicators?

Do the indicators require data from official statistics,
questionnaires, or interviews with officials, professionals,
activists, etc.?
Quality of data providers and What information is available on the (requirements for the)
processors
quality of data providers and processors for the measurement
system?
Presentation of results
How are the indicator results presented? Are the results, for
instance, scores on a numerical scale or colours on a traffic light
system?
Comparability
Is a comparison of the indicator results possible and/or

intended across different countries or regions, as well as across
different timeframes?
Frequency of application
How often is the measurement system supposed to be applied?

When was the measurement system applied for the first time
and how often has it been applied since?
89
FRAME
Status of application
What is the current status of the measurement system? Has,

for instance, only the theory or methodology been developed
so far, has it been (regularly) applied already, or has it even
been abandoned?
As it can be seen from these classification criteria, a broad understanding of measuring human rights
was chosen for this mapping exercise. This intentional choice was made in order to grasp the wide range
human rights measurement systems may have. In total, more than 130 human rights measurement
systems, tools, or related initiatives have been identified during the mapping. Each measurement scheme
identified was examined and described with regard to each of these sixteen criteria in factsheets. A
description of 50 relevant schemes identified in the mapping is attached to this report in Annex I.
When looking at the mapping results, a number of key findings emerge, which are presented below.
1.
Some measuring schemes stand out
During the desk research it became apparent that some instruments were more prominent in the human
rights indicators discourse than others. The analysis of the interviews conducted with EU officials came
to the same results: the Office of the High Commissioner for Human Rights system, the UN and CoE treaty
bodys reports, the UN Universal Periodic Review (UPR), World Bank Indicators (on development and
governance), Indicators on Gender Equality developed by the OECD, OECD Social Expenditure Index,
Bertelsmann Stiftungs Transformation Index, Freedom House Index and the Millennium Development
Goals were mentioned as valuable tools for assessing the human rights situation in any given country.
The analysis of these instruments showed that their importance was mainly to be attributed to their
practical application as well as to the frequency of their application, which makes them useful tools in the
hand of policy makers. The process for the development of human rights indicators is complex: as stated
by Engle Merry, indicators should be able to convert complicated, contextually variable phenomena into
unambiguous, clear, and impersonal measures.352 In the development of indicators, the search for
appropriate data to populate those indicators is also challenging.353 Thus, some of the instruments
mapped did not even reach the stage of practical application.354 Moreover, many initiatives that were
applied in practice are either one-off activities or their application was ended soon after a few attempts.355
While the reasons for non-continuation are not always clear, lack of funding and difficulty in gathering the
necessary data seems to play a crucial role in this regard.
352
Sally Engle Merry, 'Measuring the World: Indicators, Human Rights, and Global Governance' (2011) 52 Current
Anthropology S3 (Supplement to Number 3), p. 84.
353
United Nations Development Programme, Human Development Report 2000 (UNDP 2000), p. 90.
354
See for example the UN Housing Rights Programme, which tried to develop indicators for the right to housing.
355
See for example, the Torture Scale Index developed by Hathaway to measure with a large-scale quantitative
analysis how human rights treaties affect countries' human rights practices. The index was published in 2002 and
never used again. Oona Hathaway, 'Do Human Rights Treaties make a Difference?' (2002) 111 Yale Law Journal.
90
FRAME
A further key element of those instruments is the nature of the actors involved in developing the tool. It
became apparent that human rights measurement tools developed by international organizations are
more prone to be used compared to those developed in the academic community or by NGOs. Indeed,
measurement tools developed by NGOs may be perceived as being less objective and more biased
compared to those developed by international organizations. However, this is not to argue that well
known or more prominent indicator models automatically have a better quality or are more useful. It
certainly has to be acknowledged in this context, that potent actors might also have more budgets for
promotion of their work. However, it should also be borne in mind in this context that the Freedom House
Index has long been criticised for reflecting the foreign policy interests of the United States, and offering
a source for their legitimacy and validation.356 Yet, irrespective of the criticisms raised, the index is widely
recognised and used to assess and compare political rights and civil liberties worldwide.
Other issues that seem to play a role in these cases are the quality of the methodology developed, and
the user-friendliness of the results, which often synthesise very complex issues in a very immediate result.
2.
No consensus on appropriate methodology
A comparison of these attempts to develop human rights measurement models reveals considerable
discrepancies in the overall approaches. Generally, a distinction is to be made between ready-made
indicator systems and methods to develop indicator models.
The broad range of human rights measurement tools indicates that there is no consensus on the most
appropriate tool for measuring human rights among the different stakeholders. This holds even true for
the measurement models that stand out. Instead, a variety of methodological approaches is suggested
and sometimes applied in practice with the aim to grasp the complex nature of the progressive realisation
of human rights in different settings. Consequently, understanding the process leading to measurement
initiatives, and reflecting on, for instance, why they are needed, what to expect from them, and how to
use what data, is key.
Moreover, numerous measurement systems have been identified that do not cover human rights directly,
but assess factual situations that are of indirect relevance for human rights. In this vein, assessments of
the economic situation or the demographic development deserve mentioning, which may provide
country-specific information that may well be relevant for a deeper understanding of a countrys efforts
in the realisation of human rights. It is important to note, however, that it is not the prime purpose of the
tool to measure human rights.
356
On this issue see for example Edward S. Herman and Noam Chomsky, Manufacturing Consent: The Political
Economy of the Mass Media (Pantheon Books 1988); Scott Mainwaring and others, Classifying Political Regimes in
Latin America, 1945-1999' (2001), Kellogg Institute for International Studies Working Paper 280,
<https://www3.nd.edu/~kellogg/publications/workingpapers/WPS/280.pdf> accessed 4 March 2015; Philip Alston,
'Richard Lillich Memorial Lecture: Promoting Accountability of Members of the New UN Human Rights Council'
(2005) 15 Journal of Transnational Law and Policy 49; Diego Giannone, 'Political and Ideological Aspects in the
Measurement of Democracy: The Freedom House Case' (2010) 17 Democratization 68.
91
FRAME

3.
Indicators are indispensable
Although the existing initiatives to measure human rights are very different in nature, it is widely accepted
among human rights researchers and practitioners developing such systems that human rights indicators
are an indispensable mean to gain meaningful information. Generally, indicators describe and compare
situations that exist. Indicators can be used to monitor compliance with obligations and objectives to be
achieved. Furthermore, indicators are a planning tool and can be used to assess progress towards specific
goals, to track changes over time and to draw comparisons between places. Yet, depending on the policy
area addressed or the field of application, the definition of the term indicator may vary. As discussed by
McInerney-Lankford and Sano, human rights indicators link the conceptual discussion about human rights
compliance to implementation practices. They link the normative level of international legal obligation
with the practical level of empirical data.357 In the glossary of key terms the OECD Development
Assistance Committee defines an indicator as a [q]uantitative or qualitative factor or variable that
provides a simple and reliable means to measure achievement, to reflect the changes connected to an
intervention, or to help assess the performance of a development actor.358
According to the definition of the United Nations Development Programme, indicators are a device for
providing specific information on the state or condition of something.359 In the guidelines for defining
indicators for human rights of the Danish Institute for Human Rights the authors opt for a broader
definition of the term. According to their definition, indicators are seen to capture more than a particular
trait of social reality:
Indicators are data used by analysts or institutions and organizations to describe situations
that exist or to measure changes or trends over a period of time. They are communicative
descriptions of conditions or of performance that may provide insights into matters of larger
significance beyond that which is actually measured.360
Also, the Office of the United Nations High Commissioner for Human Rights applies a similar broad
definition of the term in its Guide to Measurement and Implementation of Human Rights Indicators.361
357
Siobhn McInerney-Lankford and Hans-Otto Sano, Human Rights Indicators in Development (World Bank 2010),
p. 14.
358
Development Assistance Committee (DAC) Working Party on Aid Evaluation, Glossary of Key Terms in Evaluation
and Results Based Management (OECD 2002), <http://www.oecd.org/dac/evaluation/2754804.pdf> accessed 4
March 2015, p. 25.
359
United Nations Development Programme (UNDP), Indicators for Human Rights Based Approaches to
Development
in
UNDP
Programming:
A
Users
Guide
(UNDP
2006),
<
http://gaportal.org/sites/default/files/HRBA%20indicators%20guide.pdf > accessed 4 March 2015, p. 21.
360
Erik Andr Andersen and Hans Otto Sano, Human Rights Indicators at Program and Project Level: Guidelines for
Defining Indicators, Monitoring and Evaluation (Danish Institute for Human Rights 2006), p. 11.
361
92
FRAME
The OHCHR further stresses that indicators need to be explicitly and precisely defined and should be based
on an acceptable methodology of data collection and presentation.362
Generally, indicators can either be quantitative or qualitative.363 Quantitative indicators are equivalent to
statistics and can be expressed in numbers, percentages or proportions; while qualitative indicators cover
any narrative or categorical form of information.364 Compared to quantitative data, qualitative indicators
are often criticised for a lack of reliability and comparability. Still, they can be applied to measure complex
concepts where purely quantitative indicators often miss the broader context.365 In the broader fields of
human rights, democracy and the rule of law the information covered by indicators often goes beyond
pure statistics. Qualitative information coming from checklists or interview guidelines can be used to
complement the interpretation of quantitative indicators. Questionnaires are mostly used for quantitative
research; interviews are typical for qualitative methods. In this context, specific target groups
(traumatised refugees, women etc.) may bring specific challenges (including costs), e.g. interviewing
children may raise issues of ethics, parental consent, and appropriate interview settings. Reciprocally,
quantitative indicators can elaborate qualitative evaluations by providing statistical information.366
4.
Miscellaneous data sources are used
There is a variety of data sources used for human rights measurement tools. The most commonly used
data sources and data collecting methodologies range from reviews of legislation and other documents,
via administrative data and public or expert surveys, to event-based data. Depending on the scope of the
human rights to be measured, it may be appropriate to rely on greater or smaller numbers of data sources.
However, drawing wide-reaching conclusions based on data from a single source might lead to systematic
misinterpretations and the underlying concepts may be misrepresented. The use of diverse data sources
usually makes indicators more robust and able to yield to a more nuanced and complete picture, as the
measures or a variety of sources are compiled.367 On a more general level, it has to be noted that the timelapse between when data on indicators is gathered and when the final results can be presented might
constitute a challenge to the usability of indicators. In practice there might be a time-lapse of more than
362
363
On the use of quantitative methods in human rights measurement see: Malcolm Langford and Sakiko FukudaParr, The Turn to Metrics (2012) 30 Nordic Journal of Human Rights 222.
364
365
United Nations Development Programme, Human Development Report 2000 (UNDP 2000), p. 90; Hague
Institute for the Internationalisation of Law (HiiL) in collaboration with the European Union Agency for Fundamental
Rights,
Measuring
the
Rule
of
Law,
Justice
and
Fundamental
Rights
(HiiL
2013)
<http://fra.europa.eu/sites/default/files/concept-paper-on-the-rule-of-law-hiil-fra.pdf> accessed 4 March 2015, p.
17.
366
Network Seminar in Human Rights Research, bo, 1013 March 2005), p. 2.
367
See also Jim Parsons, Developing Clusters of Indicators: An Alternative Approach to Measuring the Provision of
Justice (2011) 3 Hague Journal on the Rule of Law 170, p. 172.
93
FRAME
one or two years which makes it difficult to use the indicator results directly in a policy context. In the
following sections the advantages and drawbacks of the most commonly used data sources are described.
a)
Document review
One method of data generation is the review of legislation and other documents, including rulings and
decisions of judicial institutions, such as courts or prosecutors, customary justice rules, administrative
acts, and reports from international organisations. By conducting document review, the legal and
legislative commitment of the state and its authorities with regard to human rights are assessed. In this
way, document reviews typically focus on the existence of laws, treaties and legislative agreements, rather
than the practical implementation of them.368 The gap between the law on paper and the law in practice
can only be detected by reviewing such documents, describing the use of resources and public spending,
as well as (NGO) reports of problems or abuses.369
b)
Administrative data
Administrative data can provide an important source of quantitative information. These datasets are
routinely collected and compiled by the state through government authorities and may be stored as hard
copies or computerised. They refer to national registers and administrative records, using standardised
methodologies to collect information, usually with reasonable reliability and validity. Administrative data
captures a large amount of information related to administrative action and, therefore, is of prime
importance to the measurement of the fulfilment of obligations by the state. They can be applied for
tracking institutional progress over time and for holding the state accountable.370 Analysing the
administrative data generated by state institutions can often lead to straightforward conclusions, if the
records are complete and accurate. The accuracy of administrative data depends on the resources
provided to the agencies for maintaining this data, and other factors that could lead to fluctuations in the
generated data over time. Another factor leading to possible bias in reporting is that administrative
records do not measure the actual prevalence rate but only capture the officially registered cases.
Furthermore, there might be incentives to underreport certain activities or events that would negatively
influence the performance record.
c)
Survey-based data and census
Survey-based data can be generated by population surveys or by expert surveys. Population surveys are
means of data generation that identify perceptions and experiences, and can be used to check the
credibility of administrative data. While population surveys collect direct quantitative or qualitative
368
369
Jim Parsons, Developing Clusters of Indicators: An Alternative Approach to Measuring the Provision of Justice
(2011) 3 Hague Journal on the Rule of Law 170, pp. 174-75.
370
Measurement and Implementation (United Nations 2012), p. 56 seq.
94
FRAME
information from a selected subset of the population, a census is a complete record of all members of the
population. A census provides baseline data on key characteristics of the population and allows for a
highly disaggregated statistic. Therefore, census data can also be used for generating well-structured
samples for public surveys, where complete enumeration is impracticable.371 Population surveys are
usually designed to produce findings that can be generalised. Large samples of the population are
necessary to gather representative data, and this is what makes the implementation of population surveys
expensive.372 Expert surveys typically gather information from key informants that have specific
experience or hold a professional position in the specific field of interest. Therefore, these surveys are
suitable data sources for topics where specialised knowledge is required and shall reflect the expert
understanding of a situation, rather than the personal experience of the respondents. The advantage of
expert surveys is that conclusions can be drawn based on relatively small samples, meaning expert surveys
are often less resource intensive than other survey methods.373 The results of population and expert
surveys are very much dependent on the sample characteristics. The selection criteria for key informants
in expert surveys strongly influence validity and reliability. The data gathered through expert surveys
might not sufficiently represent experiences and priorities of marginalised or vulnerable groups, who are
most likely to, for example, experience problems in accessing justice, and lacking equality and fairness in
the system as a whole. With regard to this, population surveys are particularly relevant as they directly
capture the respondents views and provide an assessment of public opinion and perception. Through a
systematic combination of data from expert evaluations and population surveys, a cross-examination of
all relevant witnesses can be achieved. Each group has a specific perspective (insider versus outsider) with
a varying degree of technical knowledge. By combining the data, conflicting opinions of experts and the
general public can be revealed and more reliable results can be achieved.374
d)
Event-based data
Event-based data refers to qualitative and quantitative data resulting from counting specific events that
promote or impede specific rights. The events used to collect such data are usually divided between
positive and negative occurrences. These events reflect improvements and achievements in the fulfilment
of a right, and individual or collective violations of a right. There is a variety of sources for events-based
data, including information provided by the media, reports from states, civil society organisations and
NGO information networks, business monitoring of government performance, and testimonies of victims
371
372
Jim Parsons, Developing Clusters of Indicators: An Alternative Approach to Measuring the Provision of Justice
(2011) 3 Hague Journal on the Rule of Law 170, pp. 176-78.
373
374
Jos Juan Toharia, Evaluating Systems of Justice through Public Opinion: Why, What, Who, How and What For?
in Erik G. Jensen and Thomas C. Heller (eds), Beyond Common Knowledge: Empirical Approaches to the Rule of Law
(Stanford University Press 2003), p. 35.
95
FRAME
or witnesses.375 Compared to other categories of data, indicators derived from event-based data are more
concrete in terms of demonstrating compliance or non-compliance with specific rights. The use of eventsbased data has initially been confined to monitoring civil and political rights. The usefulness of the
methodology for gathering evidence in support of the administration of justice has also been
demonstrated. However, the indicators derived from this kind of data suffer from shortcomings in terms
of reliability and validity, as the data collection methodology is not statistically representative and may
underestimate the incidence of violations. Still, the information compiled through event-based data can
complement information captured by other data.376
B.
Definition and application of selection criteria
While building to a large extent on the information provided by the dedicated bodies of the CoE and UN,
most EU bodies appreciate the further development or refinement of the EU tools to measure human
rights. Yet, developing a measuring system that meets the EU bodies diverse requirements is certainly an
enormous, and maybe even an impossible task. It even may be argued that this is not necessary, as long
as the responsibilities are clear, structures are in place, the results are transparent and the results are also
utilised. In fact, the review of existing approaches in measuring human rights revealed that there is
currently no suitable single human rights measurement tool in application which could be used as a
measurement model for the diverse purposes of the EU bodies. Taking a close look at the mapping results,
it seems unlikely that a ready-made one-size-fits-all model is capable of producing satisfactory and
appropriate results. Even those monitoring schemes that stand out (as discussed in subchapter III.A.1.)
are not necessarily suitable or sufficient as a ready-made tool. Given the manifold endeavours to measure
human rights and the huge amount of human rights related data available, the refinement of the EUs
approach to measure human rights necessitates meaningful criteria that can guide the revision or the
enhancement of human rights measurement tools for the EU.
For this reason, important selection criteria are suggested below, in a three-step approach: Firstly, a
number of quality criteria for human rights indicators that have emerged in what can be called the state
of the art in human rights measurement are presented. Secondly, three pragmatic criteria are discussed,
which are required to safeguard the feasibility of human rights measurements. Finally, the key
requirements of the diverse bodies of the EU that have emerged in the course of the desk research and
the interviews with key officials of the EU are presented.
375
> accessed 4 March 2015, p. 5.
376
96
FRAME

1.
Selection criterion 1: Established quality criteria for the identification
of human rights indicators
From an analysis of the mapping results it can be concluded that indicators are the most appropriate
instrument for meaningful human rights measurements. Therefore, it may be very well argued that human
rights measurements by EU bodies, likewise, have to be indicator-based. Yet, for the identification of
appropriate human rights indicators, a sharp distinction has to be made between human rights
measurements and human rights-related measurements. Practically, this means that any indicator used
for the measurement of a human rights situation in the EU or beyond its borders must: (1) be rooted in a
clearly identified human rights standard, and (2) indicate the extent to which a human right is respected,
protected and/or fulfilled. Human rights-related measurements measurements drawing arbitrarily on
data that is merely related to human rights are not appropriate tools for informing evidence-based
decision making in the area of human rights. The use of proxy indicators can lead to less validity, since
there might be a gap between the concept being measured and the indicators used to measure it.377
Moreover, the selection of human rights indicators must be based on established quality criteria.
Depending on the information content of the source of data, indicators can be categorised either as factbased, objective indicators, or as judgement-based, subjective indicators. Objects, facts or events that can
be directly observed or easily verified are categorised as objective indicators. Subjective indicators are
based on opinions, perceptions, assessments or judgements expressed by individuals.378 They are
therefore often considered key for qualitative assessments. However, also in the category of objective
indicators, elements of subjectivity cannot be fully excluded, as the selection of the indicators and
consequently the indicator results are already based on subjective assumptions and choices. Therefore,
the use of clear, specific and universally recognised definitions is of great importance for the identification
and the design of any type of indicator. Moreover, the methodology and criteria applied for the
identification of indicators must be transparent and well documented.379 Subjective indicators applied for
the purpose of measuring human rights are sometimes considered to be prone to concerns of validity and
bias, as they do not necessarily capture the underlying reality but are representing the perception of a
specific group of respondents.380 However, this certainly depends on the quality of the empirical research,
e.g. the number of interviews, the selection criteria, cross checking with other data etc.
When developing indicators, it is widely agreed that certain criteria must be taken into consideration to
ensure quality in measurement. Based on the SMART criteria, indicators must be specific, measurable,
377
Linn Hammergren, Indices, Indicators and Statistics: A View from the Project Side as to Their Utility and Pitfalls
(2011) 3 Hague Journal on the Rule of Law 205, p. 307.
378
379
Markus Mstl and others, Monitoring Racism and Discrimination at the Local Level (ETC Graz 2013),
<http://www.etc-graz.at/typo3/fileadmin/user_upload/ETC-Hauptseite/themen/ECAR/ECAR_Monitoring_Manual_ETC.pdf> accessed 4 March 2015, p. 9.
380
This argument was made for rule of law indicators by Tom Ginsburg, Pitfalls of Measuring the Rule of Law (2011)
3 Hague Journal on the Rule of Law 269, p. 275.
97
FRAME
attainable, relevant and time-framed.381 For indicators to be specific and measurable, it is necessary that
they are precisely defined and verifiable. The collection and measurement of the required data must be
attainable, and the indicators must provide information relevant to the specific goal of the measurement.
For an indicator to be time-framed, it must be clearly indicated, when and how often the indicator shall
be reported, and when change is expected. Not all of these requirements are always respected in practice
and depending on the type of indicator, certain criteria can be neglected. The use of qualitative indicators
in some cases implies less restrictive requirements in terms of measurement and specificity.382 However,
an overall good standard of indicator determination shall be respected, when developing indicators. The
quality of measures with regard to statistical adequacy is evaluated in terms of validity, reliability and bias.
Reliability refers to whether the estimate or the value of an indicator is consistent, if the employed datagenerating mechanism is repeated. The reliability is affected by biases in the data-generating mechanism,
resulting from misspecification of definitions or questions, apprehensions of the respondents or nonrepresentativeness of the sample.383 Validity refers to the truthfulness of information provided by the
value or estimate of an indicator. In other words, the criterion of validity requires that the indicator
effectively measures the concept, which it is supposed to measure. The notion of validity includes among
other dimensions the content validity, constructive validity and predictive validity.384
However, there are also further approaches suggested to ensure the quality of human rights indicators.
The OHCHR, for instance, suggests the RIGHTS criteria according to which an indicator has to be relevant
and reliable, independent in its data-collection from the subject monitored, global and universally
meaningful but also amendable to contextualisation and disaggregation by prohibited grounds of
discrimination, human rights standards-centric anchored in the normative framework of rights
transparent in its methods, timely and time-bound, simple and specific.385
2.
Selection criterion 2: Three pragmatic criteria

a)
Already used in practice
Human rights measuring systems have a starting point, then they can either be abandoned or continue
running. The mapping of existing measurement tools revealed that for some schemes, only the theory or
methodology has been developed so far, while others have already been applied regularly. Those
381
For more details on the application of the SMART criteria in the context of human rights indicators see Andersen
E. A. and Sano H.-O., Human Rights Indicators at Program and Project Level: Guidelines for Defining Indicators,
Monitoring and Evaluation (Danish Institute for Human Rights 2006), p. 12.
382
Erik Andr Andersen and Hans Otto Sano, Human Rights Indicators at Program and Project Level: Guidelines for
Defining Indicators, Monitoring and Evaluation (Danish Institute for Human Rights 2006), pp. 12-14.
383
Network Seminar in Human Rights Research, bo, 1013 March 2005), p. 5.
384
Juan Carlos Botero and others, Indices and Indicators of Justice Governance and the Rule of Law: An Overview
(2011) 3 Hague Journal on the Rule of Law 153, p. 156.
385
98
FRAME
measuring schemes that are in use over time can be refined based on the practical experience gathered
during the application. This way potentially weak spots of the model can be identified and overcome.
Therefore, the refinement of EU human rights measurements should pay particular attention to indicator
models or methods to develop a system of indicators that has been tested in practice and is still followed.
b)
Data availability
Data availability is of utmost importance for all human rights measurement models. The mapping revealed
that for some measuring models, the data might be easy to collect, while for other indicators the
information required is not available and cannot realistically be generated. In case indicators cannot be
populated with adequate data, empty spots remain which render the measurement model less
meaningful. Therefore, any refinement of the EU human rights measurements should be careful not to be
too demanding with regard to data. It should be realistically possible to populate the indicators with
appropriate data. If data is not yet available, the question whether it may realistically be collected in future
must be addressed, for instance in a feasibility study. There are good reasons not to limit oneself to
available data. It is one of the intrinsic features of human rights that it concerns issues which are often
neglected or ignored. The lack of political will could result in a situation where there is no data available
on purpose. However, a rights-based measurement must not be restricted by the mainstream political
will and therefore there are good reasons to find ways to generate new data. Moreover, the intricate issue
of disaggregation must be taken into consideration for the refinement in a practical and feasible manner.
c)
User-friendliness
Any refinement of the EUs approach to measure human rights should consider how easy or difficult it is
to apply the system in practice. Human rights measurements may be highly complex endeavours requiring
knowledge and expertise in diverse fields, such as law or statistics. Yet, any refinement of existing EU
human rights measurements must ensure that the measurement tools remain applicable in practice by
the EU stakeholders involved. The imperative of user-friendliness, however, might necessitate a certain
balancing act between over-simplification and the negligence of due complexity. Linked to this is the
question of training. Any serious measurement effort needs a training strategy (and budget), irrespective
of the indicators used.
3.
Selection criterion 3: Four criteria guided by requirements of the EU
bodies
a)
Allow for a comparative and differentiated analysis
The issue areas relevant for human rights are manifold, often complex and usually cannot be
comprehensively measured with one indicator alone. In order to provide a comprehensive picture of a
human rights situation (including the progress made in respecting, protecting and fulfilling human rights),
various components have to be considered by cross-checking several indicators in order to reduce
ambiguity. Evidence-based decision making necessitates a differentiated measurement of qualitative and
99
FRAME
quantitative aspects. This requirement is of direct relevance for the presentation of the results of the
human right measurement system. Any refinement of the EUs human rights measurement tools should
first and foremost aim at the provision of specific and relevant evidence for the EU policy at stake.
Consequently, the indicator result should not necessarily be presented in an aggregated score, but should
provide for a differentiated analysis of the respective human rights situation that can be used for
evidence-based decision making. This holds especially true for different target groups and disaggregation
according to age or gender.
However, some EU policies might require or benefit from a comparative analysis of the human rights
situation across different countries. Despite the prime effort to provide a differentiated analysis of the
human rights situation in a given country, the refinement of the EU bodies human rights measurements
should thus be flexible enough to allow for the assessment of different legal and political systems. While
it might be easier to compare statistical data, the widely welcomed comparison of qualitative data is more
demanding and requires at least a commonly agreed methodology that is applicable for the different
states under scrutiny.
b)
Consistent and broadly accepted approach
It was repeatedly stressed in the interviews with key EU officials that human rights are a core value of the
EU, which have to be followed consistently. This firm conviction has important implications for the
development or identification of appropriate human rights measurement systems for the EU.
Firstly, understanding human rights as a core value of the EU implies that a systematic approach that takes
into account all human rights on an equal footing is required. It has to recognise that human rights are
indivisible, interdependent, and inter-related. Therefore, a mechanism to measure the dimensions of
human rights should not prioritise certain human rights over others. However, a particular focus might be
chosen for measurements according to the specific EU policy or decision to be made. Thus, a mechanism
requires enough flexibility to allow for such focusing.
Secondly, consistency implies that measurements of the protection, promotion and fulfilment of human
rights are not limited to a particular geographical area, but are generally applicable to all countries
worldwide. More specifically, the requirement of consistency also implies that a human rights
measurement system must be applicable for all external and internal policies of the EU alike, or at least
the principles applied must be consistent.
Thirdly, the methodology for providing evidence for decisions has to be authoritative and ideally, broadly
accepted by both, EU Member States and third countries alike. This requirement is, to a large extent based
on the fact that the decisions taken by EU bodies on the basis of the results of the human rights
measurement tool might have serious consequences and implications for Member States and/or third
countries. Only broadly accepted measurement tools enjoy the authority that is required to back the EU
bodies decisions with the legitimacy required. This further implied that such tools must be well
communicated and transparent.
100
FRAME

c)
Flexible frequency of application
Human rights measurements may generally provide information on the current state of human rights or
trace trends across a certain period of time. Depending on the EU policy at stake, the required frequency
of the assessments may vary between one to five years. Any refinement of the EU bodies human rights
measurement model should consider how to best ensure such a flexible application. Still it has to be kept
in mind that human rights measurements using indicators can never act as an early warning tool capable
of continuously providing information at short or even instant notice.
d)
Proving causality between measure and impact in project
evaluation
Some representatives of EU bodies claimed that a measurement tool that is able to establish the causality
between an EU measure taken and the progress or impact of that measure is required. If human rights
related policies are to be assessed, a human rights measurement tool would be indispensable for that
purpose. However, if the measurement of human rights achievements is merely a means for another
purpose, then providing this causality is not obviously achievable by way of a human rights measurement.
Instead, recognized project management tools, like the Logical Framework Matrix used by the World Bank,
could be used to establish this causality. However, the results of human rights indicators might feed into
such project assessments. Therefore, the refinement of the EU bodies approach to measure human rights
should include indicators that may serve as performance indicators for established project assessment
tools.
C.
Conclusion: Suggestion of a two-pronged approach
The desk research and the interview findings revealed that there is no systematic approach followed by
the EU bodies to measure the state of, or progress made in respecting, protecting and fulfilling human
rights. Currently, there is also an insufficient systematic cooperation and information exchange among
the EU bodies on human rights data. The interviews with key EU officials revealed that their initiatives for
gaining information on the relevant human rights situations are not always developed to their full
potential yet.
While building to a large extent on the information provided by the dedicated bodies of the CoE and the
UN, most EU bodies appreciate the refinement of the tools currently applied by EU bodies to measure
human rights, and name highly specific requirements for that purpose. When asked about possible
reforms and the possible future mechanisms for measuring human rights, numerous specific
requirements have been identified.
In addition to these practical requirements, there are subject-specific requirements that a reliable and
meaningful measurement system must fulfil. Most prominently, the normative framework of human
rights requires that only those indicators are used, which are rooted in a clearly identified (international)
101
FRAME
human rights standard and measure the extent to which a human right is respected, promoted, protected
and/or fulfilled.
A review of more than 130 existing human rights measurement schemes revealed that there is no single,
or readymade measuring system that meets all the requirements identified in the course of this paper by
the interviewed EU officials and the researchers. In practice, the existing measuring systems often
produce aggregated results, end after the completion of the conceptual work or are limited in their
practical application. While numerous specific human rights measurement instruments using indicators
and manifold human rights-related indicators have been identified, there are only two indicator systems
(OHCHR and CIRI) that may be regarded as general human rights measurement instruments using
indicators.
Thus, there is a lack of a commonly agreed measurement tool for the provision of relevant, reliable and
meaningful information on the state of human rights. Consequently, a two-pronged approach appears to
be the most meaningful proposal that could guide the refinement of the EUs approach to measure the
dimensions of human rights.
On the one hand, a common methodology for developing meaningful human rights indicators addressing
the need for human rights measurements can be proposed. The desk research and the interviews, as well
as an overview of academic literature give strong indications that the approach developed by the OHCHR
provides the most appropriate framework for further developing indicators for human rights
measurements. More specifically, the mapping revealed that the OHCHR approach is currently the only
general human rights measurement instrument that uses indicators and at the same time addresses the
human rights dimensions by measuring the respect, protection, fulfilment and promotion of human rights
in a given context. Indeed, a closer review of the approach suggested by the OHCHR reveals that this
system fulfils all the selection criteria identified above.
-
The OHCHR approach and selection criterion 1: The OHCHR-system suggests quality criteria
that put an explicit focus on a rights-centric approach. It therefore acknowledges the key
requirement that human right indicators have to be rooted in a clearly identified human rights
standard and have to indicate the extent to which a human right is respected, protected
and/or fulfilled. More specifically, the approach suggests a procedure guiding the
identification of relevant attributes of a human right, which aims to ensure that indicators are
anchored in human rights.
The OHCHR approach and selection criterion 2: The conceptual and methodological approach
suggested by the OHCHR is already used in practice by states like Brazil, Bolivia, Ecuador,
Mexico, Paraguay, Philippines, Kenya, Portugal, and the United Kingdom,386 and most
386
United Nations Office of the High Commissioner for Human Rights, A best-seller: the users manual for
implementation
of
human
rights
indicators
See
more
at:
102
FRAME

importantly by the FRA. The framework suggests the use of qualitative and quantitative
indicators, and further addresses the issue of data availability. Moreover, the OHCHR
addresses concerns of user-friendliness, as it published a comprehensive Guide to
Measurement and Implementation describing the conceptual and methodological
framework to identifying indicators for monitoring.
-
The OHCHR approach and selection criterion 3: The OHCHR-system allows for a differentiated
measurement and suggests the use of structure, process and outcome indicators to measure
the human rights commitments and efforts in implementing relevant administrative,
regulatory and judicial policies, and results. Structural indicators reflect the ratification and
adoption of legal instruments in national law, as well as the allocation of necessary resources
and existence of institutional mechanisms to assist the realisation of the right concerned. In
this way structural indicators measure the de jure protection of specific rights and thus, they
give information on the commitment of a state or government. Process indicators measure
how policies contribute to a specific objective and can be defined in terms of concrete causeand-effect relationships. Process indicators relate policy instruments with milestones that
cumulate into outcome indicators, measuring the attainment of an objective and de facto
realisation of a right. Outcome indicators are used to measure the results and, therefore, are
determined by a combination of structural and process indicators. The outcome indicators
might feed into project management tools that aim to establish the causality between EU
measures and their impact. Albeit the methodology proposed by the OHCHR does not (yet)
provide for indicators for all human rights, it provides a common approach to develop
indicators for civil and political rights, and economic, social and cultural rights that may be
applied and contextualised at national level allowing for a temporal and spatial comparison.387
The fact that it is a methodology to develop concrete indicators provides for the opportunity
to take into account all human rights on an equal footing and is in principle applicable to all
countries worldwide. For the same reason, the approach suggested by the OHCHR provides
for the flexibility in the frequency of application and a particular focus might be given for
human rights measurements in a specific area of interest.
The OHCHR system is therefore proposed as common methodology for refining the EU institutions
approach to collecting relevant information on human rights, as it is deemed to fulfil the main requests
identified during desk research and the interviews with officials.
<http://www.ohchr.org/EN/NewsEvents/Pages/HumanRightsIndicators.aspx#sthash.J7TJN0iZ.WR9iZHOH.dpuf>,
accessed 4 March 2015.
387
The FRA child rights indicator study also used the OHCHR model; see European Union Agency for Fundamental
Rights, Conference Edition, Developing indicators for the protection, respect and promotion of the rights of the
child in the European Union of November 2010.
103
FRAME
On the other hand, an instant information tool, i.e. an information database on compliance including a
compilation of existing indicators and related data sets could be established. This would take account of
the manifold efforts made by various (international) organisations, NGOs and academia to make the
realisation of human rights more transparent and accessible. Although many of these existing tools do not
necessarily build on human rights indicators, they are able to provide insightful data that is at least related
to the human rights situations in countries all over the world.
A key benefit of such an instant information tool is that it could provide useful information on the basis
of already existing data and could be quickly accessed by interested EU bodies. In that sense such a tool
addresses the gap between qualitative information provided by treaty-bodies, monitoring bodies or other
reports, which were declared as most relevant by the interviewed EU officials, and quantitative
information systems, mostly based on statistical data. Indeed, an instant information tool could pool
existing data, and information structured according to specific issue areas that are relevant for human
rights. Particular measurement schemes meeting certain selection criteria, such as relevance, reliability,
availability, accuracy, or being up-to-date, could therefore be compiled and made available in an adequate
manner in the instant information tool. Certainly, the data included in such a tool would also have to pass
a reliability test as well, for instance by way of a thorough assessment of the underlying methodology, the
data producers and the data sources.
104
FRAME
D.
The way forward
As a result of the findings presented in this report, two complementary tracks may be suggested for the
future refinement of the EUs approach to measure human rights.
On the one hand, the OHCHR-system could be contextualised for the requirements of the EU bodies in
order to develop and test appropriate indicators for selected policy areas of the EU. The OHCHR proposed
a sound conceptual and methodological framework for measuring human rights, which does, however,
not provide for a ready-made system of human rights indicators that may be applied by the EU bodies for
their specific purposes. Therefore, it is suggested that a pilot application of the OHCHR-system is made
for selected policies in the internal, external and the enlargement area of the EU. This step would also act
as a feasibility check of the application of a measurement system building on the OHCHR-system in the
context of the EU.
On the other hand, preparatory work for an instant information tool would be required in order to
establish an information database, including a compilation of existing indicators and related data sets. For
this endeavour, the detailed specifications and quality requirements would have to be defined and agreed
in cooperation with EU bodies.
It is suggested to follow both tracks of this two-pronged approach in the course of Work Package 13 of
the FRAME project. Overall, the work on the refinement of the human rights measurement tools of the
EU bodies also aims to sensitise EU stakeholders to the potential and the limits of human rights
measurements and should be undertaken in close cooperation with key EU stakeholders.
105
FRAME
Bibliography
Legal instruments
Act concerning the Conditions of Accession of the Republic of Bulgaria and Romania and the Adjustments
to the Treaties on which the European Union is Founded [2005] OJ L157/203.
Consolidated Version of the Treaty on European Union [2012] OJ C326/13.
Consolidated Version of the Treaty on the Functioning of the European Union [2012] OJ C326/49.
Consolidated Version of the Charter of Fundamental Rights of the European Union [2012] OJ C326/391.
Council Decision 2000/436/EC of 29 June 2000 setting up a Social Protection Committee [2000] OJ
L172/26.
Council Decision 2004/689/EC of 4 October 2004 establishing a Social Protection Committee and repealing
Decision 2000/436/EC [2004] OJ L314/8.
Council Decision 2010/707/EU of 21 October 2010 on Guidelines for the Employment Policies of the
Member States [2010] OJ L 308/46.
Council Regulation (EC) 1085/2006 of 17 July 2006 Establishing an Instrument for Pre-Accession Assistance
(IPA) [2006] OJ L210/82.
Council Regulation (EC) No. 168/2007 of 15 February 2007 establishing a European Union Agency for
Fundamental Rights [2007] OJ L 53/1.
Council Regulation (EU, EURATOM) 1311/2013 of 2 December 2013 laying down the Multiannual Financial
Framework for the Years 2014-2020 [2014] OJ L 347/884.
Council Regulation (EU) No. 567/2014 as of 26 May 2014 amending Regulation (EC) No 215/2008 on the
Financial Regulation Applicable to the 10th European Development Fund as regards the Application of the
Transition Period between the 10th European Development Fund and the 11th European Development
Fund until the entry into force of the 11th European Development Fund Internal Agreement (OJ L 157/52).
European Commission Implementing Regulation (EU) No 447/2014 of 2 May 2014 on the Specific Rules
for Implementing Regulation (EU) No 231/2014 of the European Parliament and of the Council establishing
an Instrument for Pre-accession Assistance (IPAII) [2014] OJ L 132/32.
European Parliament and Council Regulation (EC) No. 1922/2006 of 20 December 2006 establishing a
European Institute for Gender Equality [2006] OJ L 403/9.
106
FRAME
European Parliament and Council Regulation (EU) No. 978/2012 of 25 October 2012 Applying a Scheme
of Generalised Tariff Preferences and repealing Council Regulation (EC) No 732/2008 [2012] OJ L 303/1
European Parliament and Council Regulation (EU) No. 231/2014 of 11 March 2014 establishing an
Instrument for Pre-accession Assistance (IPA II) [2014] OJ L 77/14.
European Parliament and Council Regulation (EU) No. 232/2014 of 11 March 2014 establishing a European
Neighbourhood Instrument [2014] OJ L 77/27.
European Parliament and Council Regulation (EU) No. 233/2014 of 11 March 2014 establishing a Financing
Instrument for Development Cooperation for the Period 2014-2020 [2014] OJ L 77/44.
European Parliament and Council Regulation (EU) No. 234/2014 of 11 March 2014 establishing a
Partnership Instrument for Cooperation with Third Countries [2014] OJ L 77/77.
European Parliament and Council Regulation (EU) No. 235/2014 of 11 March 2014 of the European
Parliament and of the Council establishing a Financing Instrument for Democracy and Human Rights
Worldwide [2014] OJ L 77/85.
European Parliament and Council Regulation (EU) No. 236/2014 of 11 March 2014 laying down Common
Rules and Procedures for the Implementation of the Unions Instruments for Financing External Action
[2014] OJ L 77/95.
Treaty on European Union [1992] OJ C191.
Case-law
Court of Justice of the European Union:
Case 1/58, Stork vs. ECSC High Authority [1959] ECR 1959.
Case 29/69, Erich Stauder v City of Ulm [1969] ECR 419.
Case C-617/10 klagaren v. Fransson [2013] ECR I-0000.
Policy instruments, reports and papers

Council of Europe Commission for the Evaluation and the Efficiency of Justice (CEPEJ), Study on the
Functioning of Judicial Systems in the EU Member States: Facts and Figures from the CEPEJ Evaluation
2012-2014 Exercise CEPEJ(2014) 4 final.
107
FRAME
Council of the European Union:
Council of the European Union Development, Standard Framework for Country Strategy Papers:
The Council Conclusions 13357/00 of 17 November 2000.
Council of the European Union, Council Conclusions on Immigrant Integration Policy in the
European Union 14615/04 of 19 November 2004.
Council of the European Union, Zaragoza Declaration of the European Ministerial Conference on
Integration of 16 April 2010.
Council of the European Union, Toolkit to Promote and Protect the Enjoyment of all Human
Rights by Lesbian, Gay, Bisexual and Transgender (LGBT) People 11179/10 of 17 June 2010.
Council of the European Union, Item Note Indicators for the Comprehensive approach to the
EU Implementation of the United Nations Security Council Resolutions 1325 and 1820 on Women,
Peace and Security 11948/10 of 14 July 2010.
Council of the European Union, Conclusions of the Council and the Representatives of the
Governments of the Member States on Integration as a Driver for Development and Social
Cohesion 9248/10 of 4 September 2010.
Council of the European Union, The Joint Assessment Framework and the Employment
Performance Monitor (Joint EMCOSPC Opinion) 16984/10 of 26 November 2010.
Council of the European Union, Note Report on the EU Indicators for the Comprehensive
Approach to the EU Implementation of the UN Security Council UNSCRs 1325 and 1820 on
Women, Peace and Security 9990/11 of 11 May 2011.
Council of the European Union, EU Strategic Framework and Action Plan on Human Rights and
Democracy 11855/12 of 25 June 2012.
Council of the European Union, Guidelines to Promote and Protect the Enjoyment of All Human
Rights by Lesbian, Gay, Bisexual, Transgender and Intersex (LGBTI) Persons of 24 June 2013.
Council of the European Union, Conclusions on the 2014 Annual Growth Survey and Joint
Employment Report: political guidance on employment and social policies of 10 March 2014.
Council of the European Union, EU Human Rights Guidelines on Freedom of Expression Online
and Offline of 12 May 2014.
Council of the European Union, Conclusion on a Rights-Based Approach to Development

Cooperation, encompassing All Human Rights of 19 May 2014.
Council of the European Union, Europe 2020 Employment Performance Monitor June 2014
10763/1/14 Rev 1 of 16 June 2014.
108
FRAME

Council of the European Union, EU Annual Report on Human Rights and Democracy in the World
in 2013 11107/14 of 23 June 2014.
European Commission:
European Commission, Agenda 2000: For a Stronger and Wider Union (Communication)
COM(97) 2000 final.
European Commission, Application of the Charter of Fundamental Rights of the European Union
(Memorandum) SEC(2001) 380/3.
European Commission, Impact Assessment (Communication) COM(2002) 276.
European Commission, Guidelines for the Use of Indicators in Country Performance Assessment
(December
2002),
<http://ec.europa.eu/development/icenter/repository/guidelines_indicators_cpa_en.pdf>
European Commission, Article 7 of the Treaty on European Union. Respect for and Promotion of
the Values on which the Union is based COM(2003) 606 final.
European Commission, Compliance with the Charter of Fundamental Rights in Commission

Legislative Proposals. Methodology for Systematic and Rigorous Monitoring COM(2005) 172
final.
European Commission, Monitoring Report on the State of Preparedness for EU Membership of

Bulgaria and Romania COM(2006) 549 final.
European Commission, Commission Staff Working Document accompanying the Communication

from the Commission - Towards an EU Strategy on the Rights of the Child - Impact Assessment
COM(2006) 367 final SEC(2006) 88.
European Commission, Impact Assessment Guidelines SEC(2009) 92.
European Commission, Report on the Practical Operation of the Methodology for a Systematic
and Rigorous Monitoring of Compliance with the Charter of Fundamental Rights COM(2009) 205
final.
European Commission, Strategy for the Effective Implementation of the Charter of Fundamental
Rights by the European Union COM(2010) 573 final.
European Commission, Europe 2020: A Strategy for Smart, Sustainable and Inclusive Growth
(Communication) COM(2010) 2020 final.
European Commission, An EU Agenda for the Rights of the Child COM(2011) 60 final.
European Commission, Fighting Corruption in the EU COM(2011) 308 final.

109
FRAME
European Commission, A New Response to a Changing Neighbourhood COM(2011) 303.
European Commission, Operational Guidance on taking Account of Fundamental Rights in

Commission Impact Assessment (Commission Staff Working Paper) SEC(2011) 567 final.
European Commission, Increasing the Impact of EU Development Policy: an Agenda for Change
European Commission, The Future Approach to EU budget Support to Third Countries

European Commission and High Representative of the European Union for Foreign Affairs and
Security Policy, Human Rights and Democracy at the Heart of EU External Action - Towards a
More Effective Approach (Joint Communication to the European Parliament and the Council)
European Commission, Commission Decision establishing an EU Anti-Corruption Reporting

Mechanism for Periodic Assessment ("EU Anti-Corruption Report") C(2011) 3673 final.
European Commission EIDHR, Strengthening Democracy Support to EU Delegations: from

Performance Indicators, Knowledge sharing to Expert Services Study on Performance Indicators
for EU Parliamentary Support of September 2012.
European Commission, Tools and Methods Series Working Document Budget Support
Guidelines, Programming, Design and Management A Modern Approach to Budget Support of
September 2012.
European Commission Progress in Bulgaria under the Cooperation and Verification Mechanism
(Report) COM(2012) 411 final.
European Commission, The EU Justice Scoreboard: A Tool to Promote Effective Justice and
Growth COM(2013) 160 final.
European Commission, Strengthening the Social Dimension of the Economic and Monetary
Union COM(2013) 690 provisoire.
European Commission, Draft Joint Employment Report accompanying the Communication from
the Commission on Annual Growth Survey 2014 COM(2013) 801 final.
European Commission, Impact Assessment Board Report for 2013, <http://ec.europa.eu/smartregulation/impact/key_docs/docs/iab_report_2013_en.pdf> accessed 10 November 2014.
European Commission Enlargement Strategy and Main Challenges 2013-2014 COM(2013) 700
final.
European Commission Progress in Bulgaria under the Cooperation and Verification Mechanism
(Report) COM(2014) 36 final.
110
FRAME
European Commission, EU Anti-Corruption Report COM(2014) 38 final.
European Commission and High Representative of the European Union for Foreign Affairs and
Security Policy, Implementation of the European Neighbourhood Policy: Statistical Annex (Joint
Staff Working Document) SWD(2014) 98 final.
European Commission, Tool-Box a Rights-based Approach, encompassing All Human Rights for
EU Development Cooperation (Commission Staff Working Document) SWD(2014) 152 final.
European Commission, The 2014 EU Justice Scoreboard COM(2014) 155 final.
European Commission, A New EU Framework to strengthen the Rule of Law COM(2014) 158
final/2.
European Commission, 2013 Report on the Application of the EU Charter of Fundamental Rights
European Commission, Instrument for Pre-Accession Assistance (IPA II) Multi-country Indicative
Strategy Paper (2014-2020) of 30 June 2014.
European Commission, Commission Implementing Decision of 11.8.2014 adopting the Thematic

Strategy Paper 2014-2020 and accompanying Multiannual Indicative Programme 2014-2017 of
the Instrument contributing to Stability and Peace C(2014) 5607 final.
European Commission, Programming of the European Neighbourhood Instrument (ENI) 20142020, Single Support Framework for EU support to Armenia (2014-2017),
<http://eeas.europa.eu/enp/pdf/financing-theenp/armenia_2014_2017_programming_document_en.pdf> accessed 16 October 2014.
European Commission, Programming of the European Neighbourhood Instrument (ENI) 20142020, Strategic Priorities 2014-2020 and Multi-annual Indicative Programme 2014-2017,
European Neighbourhood-wide Measures, <http://eeas.europa.eu/enp/pdf/financing-theenp/enp_wide_strategic_priorities_2014_2020_and_multi_annual_indicative_programme_201
4_2017_en.pdf> accessed 16 October 2014.
European Commission, Restrictive Measures in Force of 2 September 2014

<http://eeas.europa.eu/cfsp/sanctions/docs/measures_en.pdf> accessed 16 October 2014.
European Commission, The Reports on Progress in Romania and

<http://ec.europa.eu/cvm/progress_reports_en.htm> accessed 9 October 2014.
European
Commission,
DG
Enlargement,
Strategy
and
Progress
Reports
<http://ec.europa.eu/enlargement/countries/strategy-and-progress-report/index_en.htm>
Bulgaria
111
FRAME
European Commission, Instrument for Pre-Accession Assistance (IPA II), Indicative Strategy Paper
for Albania (2014-2020).
for the Former Yugoslav Republic of Macedonia (2014-2020).
for Kosovo (2014-2020).
for Serbia (2014-2020).
for Turkey (2014-2020).
European Commission, Partnership Instrument - First Multi-annual Indicative Programme for the
Period 2014-2017, <http://ec.europa.eu/dgs/fpi/documents/pi_mip_annex_en.pdf> accessed
14 October 2014.
European Commission, Mid-term Report on the Implementation of the EU Strategy Towards the
Eradication of Trafficking in Human Beings (Commission staff working document) COM(2014) 635
final of 17 October 2014.
European Council:
European Council, Presidency Conclusions on the European Council Meeting in Copenhagen of

21-22 June 1993 SN 18/1/93 REV. 1.
European Council, Presidency Conclusions on the European Council Meeting in Lisbon of 23-24
March 2000 SN 100/1/00.
European Council, Presidency Conclusions on the European Council Meeting in Nice of 8

December 2000 SN 400/1/00.
European Council, Presidency Conclusion on the European Council Meeting in Laeken of 14 and
15 December 2001 SN 300/1/REV 1.
European Council, Presidency Conclusions on the European Council Meeting of 25-26 March
2010 EUCO 7/10.
European Council, The Stockholm Programme An Open and Secure Europe Serving and
Protecting Citizens, 2010/C 115/01 of 4 April 2010.
European External Action Service, Instructions for the Programming of the 11th European Development
Fund (EDF) and the Development Cooperation Instrument (DCI) 2014-2020 of 15 May 2012.
European Parliament:
112
FRAME
European Parliament, the Council and the Commission, The Protection of Fundamental Rights
and the European Convention for the Protection of Human Rights and Fundamental Freedoms
(Joint Declaration) [1977] OJ C103/1.
European Parliament, Declaration on the Fundamental Rights and Freedoms [1989] OJ C120/51.
European Parliament, Resolution on the Situation as regards Fundamental Rights in the European
Union (2000) of 5 July 2001 (2000/2231(INI)) A5-0223/2001.
Council of the European Union, EU guidelines on Violence against Women and Girls and
combating All Forms of Discrimination against them of 8 December 2008.
European Parliament, Resolution on the Annual Report on Human Rights in the World 2008 and
the European Unions Policy on the Matter (2008) of 7 May 2009 (2008/2336(INI))
P6_TA(2009)0385.
European Parliament, Resolution on guaranteeing Independent Impact Assessments of 8 June

2011 (2010/2016(INI)) P7_TA(2011)0259.
European Parliament, Resolution containing the European Parliaments Recommendations to the

Council, the Commission and the European External Action Service on the Negotiations of the EUAzerbaijan Association Agreement of 18 April 2012 (2011/2316(INI)) P7_TA(2012)0127.
European Parliament, Resolution on the Situation of Fundamental Rights in the European Union
(2010 - 2011) of 12 December 2012 (2011/2069(INI)) P7_TA(2012)0500.
European Parliament, Resolution on the Review of the EUs Human Rights Strategy of 13
December 2012 (2012/2062(INI)) P7_TA(2012)0504.
European Parliament, Resolution on the Situation of Fundamental Rights: Standards and

Practices in Hungary (pursuant to the European Parliament Resolution of 16 February 2012) of 3
July 2013 (2012/2130(INI)) P7_TA-PROV(2013)0315.
European Parliament, Resolution on the Annual Report on Human Rights and Democracy in the
World 2012 and the European Union's Policy on the Matter of 11 December 2013
(2013/2152(INI)) P7_TA(2013)0575.
European Parliament, Resolution on the European Semester for Economic Policy Coordination:
Employment and Social Aspects in the Annual Growth Survey 2014 of 25 February 2014
(2013/2158(INI)), P7_TA(2014)0129.
European Parliament, Resolution on the Situation of Fundamental Rights in the European Union
(2012) of 27 February 2014 (2013/2078(INI)) P7_TA-PROV(2014)0173.
113
FRAME
European Parliament and Council, Regulation (EU) No 235/2014 of the European Parliament and
of the Council establishing a Financing Instrument for Democracy and Human Rights Worldwide
of 11 March 2014.
European Parliamentary Research Service, Article 7 TEU: A Mechanism to Protect EU Values,

<http://epthinktank.eu/2013/10/07/article-7-teu-a-mechanism-to-protect-eu-values/> accessed
27 October 2014.
Social Protection Committee Social Protection Performance Monitor (SPPM) Methodological Report by
the Indicators Sub-group of the Social Protection Committee of 17 October 2012
<http://ec.europa.eu/social/main.jsp?catId=758> accessed 3 November 2014.
United Nations Development Programme (UNDP), Indicators for Human Rights Based Approaches to
Development
in
UNDP
Programming:
A
Users
Guide
(UNDP
2006),
<http://www.undg.org/docs/11652/HRBA-Indicators-(2006).pdf> accessed 4 November 2014.
United Nations Office of the High Commissioner for Human Rights, Human Rights Indicators,
<http://www.ohchr.org/EN/NewsEvents/Pages/HumanRightsIndicators.aspx#sthash.J7TJN0iZ.dpuf>
accessed 4 November 2014.
United Nations Special Rapporteur of the Commission on Human Rights on the Right of Everyone to enjoy
the Highest Attainable Standard of Physical and Mental Health, Interim Report on the Right of Everyone
to enjoy the Highest Attainable Standard of Physical and Mental Health 28 August 2003, A/58/427.
114
FRAME
Literature
Books
Andersen E. A. and Sano H.-O., Human Rights Indicators at Program and Project Level: Guidelines for
Defining Indicators, Monitoring and Evaluation (Danish Institute for Human Rights 2006).
Balfour R., The Role of EU Delegations in EU Human Rights Policy (Publications Office of the European
Union 2013).
Barcevicius E. and others (eds.), Assessing the Open Method of Coordination (Palgrave Macmillan 2014).
Bartels L., The European Parliament's Role in relation to Human Rights in Trade and Investment
Agreements (Publications Office of the European Union 2014).
Carrera S. and others, The Triangular Relationship between Fundamental Rights, Democracy and the Rule
of Law in the EU: Towards an EU Copenhagen Mechanism (Centre for European Policy Studies 2013).
European Commission, A Roadmap for Equality between Women and Men 2006-2010 (Publications Office
of the European Communities 2006).
European Commission, 2012 Annual Report on Financial Assistance for Enlargement, Report from the
Commission to the European Parliament, the Council and the European Economic and Social Committee
(Publications Office of the European Union 2013).
European Commission, Budget Support (Publications Office of the European Union 2008).
European Institute on Gender Equality, Gender Equality Index: Country Profiles (Publications Office of the
European Institute on Gender Equality, Gender Equality Index: Main Findings (Publications Office of the
European Institute for Gender Equality, Gender Equality Index Report (EIGE 2013).
European Ombudsman, Annual Report 2012 (Publications Office of the European Union 2013).
European Union Agency for Fundamental Rights (FRA), Developing Indicators for the Protection, Respect
and Promotion of the Rights of the Child in the European Union (European Union Agency for Fundamental
Rights 2010).
European Union Agency for Fundamental Rights, The European Union as a Community of Values:
Safeguarding Fundamental Rights in Times of Crisis (Publications Office of the European Union 2013).
European Union Agency for Fundamental Rights, Annual Report 2013 Fundamental Rights, Challenges
and Achievements in 2013 (Publications Office of the European Union 2013) .
European Union Agency for Fundamental Rights, The Right to Political Participation for Persons with
Disabilities: Human Rights Indicators (Publications Office of the European Union 2014).
115
FRAME
EUROSTAT, Indicators of Immigrant Integration: A Pilot Study (Publications Office of the European Union
2011).
EUROSTAT, Measuring Material Deprivation in the EU: Indicators for the Whole Population and ChildSpecific Indicators (Publications Office of the European Union 2012).
EUROSTAT, Trafficking in Human Beings 2014 Edition (Publications Office of the European Union 2014).
Herman E. S. and Chomsky N., Manufacturing Consent: The Political Economy of the Mass Media
(Pantheon Books 1988).
Huddleston T. and others, Using EU Indicators of Immigrants Integration (Publications Office of the
Kinzelbach K., The EUs Human Rights Dialogue with China: Quiet Diplomacy and its Limits, (Routledge,
2014).
Kochenov D., EU Enlargement and the Failure of Conditionality: Pre-accession Conditionality in the Fields
of Democracy and the Rule of Law (Kluwer Law International 2008).
Landman T. and Carvalho E., Measuring Human Rights (Routledge 2010).
McInerney-Lankford S. and Sano H.-O., Human Rights Indicators in Development (World Bank 2010).
Mihr A., Human Rights Benchmarks for EU's External Policy (Publications Office of the European Union
2011).
Muller P. and others, Performance-based Policy (Publications Office of the European Union 2013).
Organisation for Economic Co-operation and Development (OECD) and European Commissions Joint
Research Centre, Handbook on Constructing Composite Indicators: Methodology and User Guide (OECD
2008).
Schmid-Drner M., Employment and Social Affairs in the European Parliament (Publications Office of the
Social Protection Committee, Social Europe Many ways, one objective (Publications Office of the
United Nations Development Programme, Human Development Report 2000 (UNDP 2000).
Measurement and Implementation (United Nations 2012).
Books chapters
DHollander D. and others, Integrating Human Rights in EU Development Cooperation Policy:
Achivements and Challenges in Benedek W. and others (eds), European Yearbook on Human Rights 14
(Intersentia 2014).
116
FRAME
De Schutter O., The Implementation of the Charter by the Institutions of the European Union in Peers S.
and others (eds), The EU Charter of Fundamental Rights (Hart Publishing 2014).
Khling J., Fundamental Rights in von Bogdandy A. and Bast J. (eds), Principles of European Constitutional
Law (Hart Publishing 2011).
Maquet Engsted I., The European Context: Measuring Social Inclusion in the European Union, in Ruggeri
Laderchi C. and Savastano S. (eds), Poverty and Exclusion in the Western Balkans (Springer 2013).
Rovan J., Composite Indicators, in Lovric M. (ed), International Encyclopedia of Statistical Science
(Springer 2011).
Toharia J. J., Evaluating Systems of Justice through Public Opinion: Why, What, Who, How and What For?
in Jensen E. G. and Heller T. C. (eds), Beyond Common Knowledge: Empirical Approaches to the Rule of
Law (Stanford University Press 2003).
Journal articles and working papers
Abbot J. and Gujit I., Changing Views on Change: Participatory Approaches to monitoring the
Environment (1998) SARL Discussion Papers 254.
Alston P., 'Richard Lillich Memorial Lecture: Promoting Accountability of Members of the New UN Human
Rights Council' (2005) 15 Journal of Transnational Law and Policy 49.
Atkinson A. B. and others, Indicators and Targets for Social Inclusion in the European Union (2004) 1
Journal of Common Market Studies 47.
Barsh R. L., 'Measuring Human Rights: Problems of Methodology and Purpose' (1993) 15 Human Rights
Quarterly 87.
Beke L. and others, Report on the Integration of Human Rights in EU Development and Trade Policies
(2014) Frame Deliverables, <http://www.fp7-frame.eu/reports> accessed 11 November 2014.
von Bogdandy A. and others, Reverse Solange - Protecting the Essence of Fundamental Rights against EU
Member States (2012) 49 Common Market Law Review 489.
Botero J. C. and others, Indices and Indicators of Justice Governance and the Rule of Law: An Overview
(2011) 3 Hague Journal on the Rule of Law 153.
Carr Center for Human Rights Policy at Harvard University, Measurement and Human Rights: Tracking
Progress, Assessing Impact, Report on the Carr Centre Conference Measurement and Human Rights:
Tracking
Progress,
Assessing
Impact
of
5-7
May
2005,
p.
3,
<http://www.hks.harvard.edu/cchrp/mhr/publications/documents/MeasurementReport2005.pdf>
accessed 13 April 2014.
117
FRAME
Churruca Muguruza C. and others, Report mapping Legal and Policy Instruments of the EU for Human
Rights and Democracy Support (2014) Frame Deliverables, <http://www.fp7-frame.eu/wpcontent/materiale/reports/05-Deliverable-12.1.pdf> accessed 14 October 2014.
Coppedge M. and others, Conceptualizing and measuring Democracy: A New Approach (2011) 9
Perspectives on Politics 247.
De Schutter O., 'A New Direction for the Fundamental Rights Policy of the EU' (2010) Reflexive Governance
in the Public Interest Working Papers 33 <http://refgov.cpdr.ucl.ac.be/?go=publications> accessed 8 April
2014.
Engle Merry S., 'Measuring the World: Indicators, Human Rights, and Global Governance' (2011) 52
(Supplement to Number 3) Current Anthropology 83.
Ferrera M. and Sacchi S., 'The Open Method of Coordination and National Institutional Capabilities: The
Italian
Experience'
(2004)
Urge
Working
Papers
2,
<http://www.urge.it/files/papers/2_1_wp_2_2004.pdf> accessed 4 August 2014.
German Development Institute, Allocating the Next European Development Fund for ACP countries of 5
December
2013
<http://www.die-gdi.de/uploads/media/German-DevelopmentInstitut_Mario_Negre_05.12.2013.pdf> accessed 15 October 2014.
Giannone D., 'Political and Ideological Aspects in the Measurement of Democracy: The Freedom House
Case' (2010) 17 Democratization 68.
Ginsburg T., Pitfalls of Measuring the Rule of Law (2011) 3 Hague Journal on the Rule of Law 269
Green M., 'What we talk about when we talk about Indicators: Current Approaches to Human Rights
Measurement' (2001) 23 Human Rights Quarterly 1062.
Hague Institute for the Internationalisation of Law in collaboration with the European Union Agency for
Fundamental Rights, Measuring the Rule of Law, Justice and Fundamental Rights (HiiL 2013)
<http://fra.europa.eu/sites/default/files/concept-paper-on-the-rule-of-law-hiil-fra.pdf> accessed 4
November 2014.
Hammergren L., Indices, Indicators and Statistics: A View from the Project Side as to Their Utility and
Pitfalls (2011) 3 Hague Journal on the Rule of Law 205.
Hathaway O., 'Do Human Rights Treaties make a Difference?' (2002) 111 Yale Law Journal 67.
Hillion C., Enlarging the European Union and Deepening its Fundamental Rights Protection (2013) 11
European Policy Analysis <http://www.sieps.se/sites/default/files/2013_11epa.pdf> accessed 9 October
2014.
Kaufmann D. and Kraay A., 'Governance Indicators: Where are We? Where should We be going?' (2007)
World Bank Policy Research Working Paper 4370.
118
FRAME
Landman T. and Husermann J., Map-Making and Analysis of the Main International Initiatives on
Developing Indicators on Democracy and Good Governance (Human Rights Centre of the University of
Essex 2003), <http://www.oecd.org/dac/governance-development/20755719.pdf> accessed 4 November
2014.
Langford M. and Fukuda-Parr S., The Turn to Metrics (2012) 30 Nordic Journal of Human Rights 222.
Mainwaring S. and others, Classifying Political Regimes in Latin America, 1945-1999' (2001), Kellogg
Institute
for
International
Studies
Working
Paper
280,
<https://www3.nd.edu/~kellogg/publications/workingpapers/WPS/280.pdf> accessed 23 October 2014.
Malhotra R. and Fasel N., Quantitative Human Rights Indicators: A Survey of Major Initiatives (Nordic
Network Seminar in Human Rights Research, bo, 1013 March 2005).
Mstl M. and others, Monitoring Racism and Discrimination at the Local Level (ETC Graz 2013),
<http://www.etc-graz.at/typo3/fileadmin/user_upload/ETCHauptseite/themen/ECAR/ECAR_Monitoring_Manual-_ETC.pdf> accessed 4 November 2014.
Oikarinen J., Reform of the EU Human Rights Policy and the Challenge of Implementation (2014)
European Yearbook on Human Rights 115.
Organisation for Economic Co-operation and Development (OECD), 'Measuring Human Rights and
Democratic Governance: Experiences and Lessons from Metagora' (2008) 9 OECD Journal on
Development 2.
Parsons J., Developing Clusters of Indicators: An Alternative Approach to Measuring the Provision of
Justice (2011) 3 Hague Journal on the Rule of Law 170.
Payne A. M. and Schaefer L., Human Rights Measurement Framework: EHRC/SHRC Specialist Consultation
on
Selection
of
Indicators,
Long-list
of
Indicators
and
Measures
(2010),
<http://personal.lse.ac.uk/prechr/hrmf/HRMF_longlist_of_indicators_and_measures.pdf> accessed 4
November 2014.
Renda A. and others, 'Assessing the Costs and Benefits of Regulation' (CEPS 2013),
<http://ec.europa.eu/smartregulation/impact/commission_guidelines/docs/131210_cba_study_sg_final.pdf>
accessed
10
November 2014.
Rosga A. J. and Satterthwaie M. L., The Trust in Indicators: Measuring Human Rights, (2009) 27 Berkeley
Journal of International Law 253.
Spirer H., 'Violations of Human Rights. How Many?' (1990) 49 American Journal of Economics and
Sociology 199.
Theuermann E., The Review of the EU Human Rights Policy: A Commitment to Strengthened EU Action
on Human Rights (2013) European Yearbook on Human Rights 31.
119
FRAME
Timmer A., Report State of the Art Literature Review Human Rights, Democracy and the Rule of Law
(2014) Frame Deliverables, <http://www.fp7-frame.eu/reports> accessed 6 October 2014.
Welling J. V., 'International Indicators and Economic, Social, and Cultural Rights' (2008) 30 Human Rights
Quarterly 933.
Other secondary sources: statements, newspapers articles, press releases and internet
websites
Barroso J., State of the Union 2012 Address (Press Release), SPEECH/12/596 of 12/09/2012
<http://europa.eu/rapid/press-release_SPEECH-12-596_en.htm> accessed 7 August 2014.
Development Assistance Committee (DAC) Working Party on Aid Evaluation, Glossary of Key Terms in
Evaluation
and
Results
Based
Management
(OECD
2002),
<http://www.oecd.org/dac/evaluation/2754804.pdf> accessed 4 November 2014.
Engle Merry S., Quantification and the Paradox of Measurement (2014 AHRI Conference, Copenhagen,
September 2014).
Release) IP/11/678 of 6 June 2011.
European
Commission
The
EUs
Generalised
Scheme
of
Preferences
(GSP),
<http://trade.ec.europa.eu/doclib/docs/2014/may/tradoc_152502.pdf> accessed 15 October 2014.
European Institute for Gender Equality, Gender Equality Index - Launch Conference
<http://eige.europa.eu/content/event/gender-equality-index-launch-conference>
accessed
25
September 2014.
European Commission - EIDHR, Human Wrong Index, <http://www.eidhr.eu/human-wrongs-index>
European Commission, The Multiannual Financial Framework: The External Action Financing Instruments
(Memo) MEMO/13/1134 of 11 December 2013.
European
Commission,
Overview
Instrument
for
Pre-Accession
Assistance,
<http://ec.europa.eu/enlargement/instruments/overview/index_en.htm#ipa2> accessed 9 October
2014.
European Union Agency for Fundamental Rights, How we do it < http://fra.europa.eu/en/aboutfra/how-we-do-it> accessed 25 September 2014.
European Union Agency for Fundamental Rights, FRANET <http://fra.europa.eu/en/research/franet>
accessed 25 September 2014.
120
FRAME
European Union Agency for Fundamental Rights, Annual Work Programme 2015
<http://fra.europa.eu/sites/default/files/annual_work_programme_2015.pdf> accessed 25 September
2014.
European
Union
External
Action
Service,
European
<http://eeas.europa.eu/enp/index_en.htm> accessed on 15 October 2014.
Neighbourhood
Policy
Pan-African Programme 2014-2020 Multiannual indicative Programme 2014-2017, <http://www.africaeu-partnership.org/sites/default/files/documents/c_2014_5375_1_annex_en_v1_p1_771842.pdf>

121
FRAME
Annex I List of Indicator Schemes and Human Rights Measurement

Instruments
See end of document.
122
FRAME
Annex II - Case studies on the use of rule of law indicators in the EU

enlargement process: Bulgaria, Croatia and Montenegro388
Table of Contents
Table of figures ......................................................................................................................................... 125
List of abbreviations .................................................................................................................................. 126
I.
Introduction ...................................................................................................................................... 127
II.
Rule of law as a principle in the EU ................................................................................................... 128

A.
The rule of law as an internal principle of the EU......................................................................... 129
B.
The rule of law as criterion for the accession of new Member States ......................................... 131
III.
Mechanisms for measuring the rule of law .................................................................................. 134
A.
Major initiatives for measuring the rule of law by international organisations ........................... 135
B.
Mechanisms for measuring the rule of law in the EU and its Member States ............................. 137
C.
Basic principles for the use of indicators to measure the rule of law .......................................... 139
IV.
Measuring the rule of law in the EU enlargement policy ............................................................. 141
A.
The case of Bulgaria ...................................................................................................................... 142

1.
The new strategy for EU enlargement ...................................................................................... 142
2.
The monitoring mechanism applied to Bulgaria ....................................................................... 143
3.
The indicators applied to measure the rule of law in Bulgaria ................................................. 146
B.
The case of Croatia........................................................................................................................ 152

1.
The framework for negotiations ............................................................................................... 152
2.
The monitoring mechanism applied to Croatia ........................................................................ 154
3.
The indicators applied to measure the rule of law in Croatia .................................................. 158
C.
D.
V.
388
The case of Montenegro ............................................................................................................... 162

1.
The new approach to negotiations ........................................................................................... 163
2.
The monitoring mechanism applied to Montenegro................................................................ 164
3.
The indicators applied to measure the rule of law in Montenegro .......................................... 166
Implications for the use of indicators in the EU enlargement policy ........................................... 170
Conclusions and outlook ................................................................................................................... 172
Authored by Alexandra Kulmer, Wolfgang Benedek and Klaus Starl, European Training and Research Centre for
Human Rights and Democracy (ETC Graz).
FRAME
Bibliography .............................................................................................................................................. 175

Legal and policy instruments ................................................................................................................ 175
Case Law................................................................................................................................................ 176
Literature .............................................................................................................................................. 176
Policy and other reports ....................................................................................................................... 178
Websites ............................................................................................................................................... 182
124
FRAME
Table of figures
Figure 1: The monitoring mechanism applied to Bulgaria ........................................................................ 143
Figure 2: Indicators to measure the independence and accountability of the judiciary .......................... 149
Figure 3: Indicators to measure access to justice ..................................................................................... 152
Figure 4: The monitoring mechanism applied to Croatia ......................................................................... 154
Figure 7: The monitoring mechanism applied to Montenegro................................................................. 164
125
FRAME
List of abbreviations
AG
Art.
CARDS
CEPEJ
CoE
CVM
DG
DPKO
EC
ECHR
ETC
EU
FRA
GDP
HiiL
ICTY
IPA
NGO
OHCHR
OJ
para.
PHARE
SAA
SAP
TEC
TEU
TFEU
UN
UNDP
WGI
Advocate General
Article
Community Assistance for Reconstruction, Development and Stabilisation
European Commission for the Efficiency of Justice
Council of Europe
Mechanism for Cooperation and Verification
Directorate General
Department of Peacekeeping Operations
European Community
European Convention for Human Rights
European Training and Research Centre for Human Rights and Democracy
European Union
European Union Agency for Fundamental Rights
Gross Domestic Product
Hague Institute for the Internationalisation of Law
International Criminal Tribunal for the former Yugoslavia
Instrument for Pre-Accession Assistance
Non-governmental Organisation
Office of the High Commissioner for Human Rights
Official Journal
Paragraph
Poland and Hungary Aid for Restructuring of the Economies
Stabilisation and Association Agreements
Stabilisation and Accession Process
Treaty establishing the European Community
Treaty on European Union
Treaty on the Functioning of the European Union
United Nations
United Nations Development Programme
Worldwide Governance Indicators
126
FRAME
IV.
Introduction
The rule of law is seen as a precondition and as a basis for respecting human rights, promoting democracy
and also for a smooth economic development in states around the world. Therefore, promoting the rule
of law has become a main subject in peace building, transitional justice and development programmes of
many international organisations. It is regarded as a core mission by the United Nations (UN), the Council
of Europe (CoE) and the European Union (EU). The quality of law making, access to justice and the
independence and accountability of the judiciary are considered as key elements of the rule of law.
The rule of law is laid down as one of the foundational principles of the EU in the Treaty on European
Union (TEU) and according to Art. 21 TEU it is one of the main priorities for the Unions external relations.
As one pillar of the Copenhagen political criteria, the rule of law is now at the heart of the EU enlargement
process and is referred to as a key principle in the European Commissions Enlargement Strategy. The EU
has set strict requirements and has established a monitoring system for measuring the rule of law in
enlargement countries. Countries aspiring to join the EU need to secure the rule of law by establishing
core institutions and promoting judicial reforms, ensuring the independence, impartiality, efficiency and
accountability of judicial systems. In many enlargement countries, the requested reforms also include the
fight against corruption and organised crime. International organisations, like the UN, the CoE and the
World Bank, have developed monitoring mechanisms for measuring the impact of their rule of law policies
and actions. With regard to measuring the rule of law in the candidate and potential candidate countries,
the EU has made use of the monitoring instruments of these organisations and has continuously improved
its own assessment tools.
This paper deals with the question of how to measure the rule of law in the EU enlargement processes.
To answer this question, the assessments of the rule of law in the three EU enlargement countries
Bulgaria, Croatia and Montenegro are analysed. This analysis is conducted on the basis of the theoretical
discourse on the rule of law as a principle in the EU internal and external policy in Chapter V. Furthermore,
existing monitoring mechanisms developed by international organisations like the World Bank, the CoE,
the UN and the EU are investigated in order to identify basic principles for the use of indicators in rule of
law measurement in Chapter VI. Based on this theoretical background, the mechanisms and indicators
applied by the Commission to measure the rule of law in the EU accession process are examined in three
case studies of the enlargement countries Bulgaria, Croatia and Montenegro in Chapter VII. These three
cases were selected for examination, as they reflect three different periods in the EU enlargement policy
and they show the consistent further development of the Enlargement Strategy that is guiding the
accession negotiations. The progress assessments conducted by the Commission are investigated with
regard to the application of indicators for measuring the independence and accountability of the judiciary
and access to justice in the accession process of these three countries. Based on the findings of the case
studies, general implications are derived for the use of indicators in the EU enlargement policy. In the final
Chapter V, overall conclusions are drawn and an outlook is given on the application of rule of law
indicators in the EU enlargement processes, as well as for the promotion of the rule of law in the EU
Member States.
127
FRAME
V.
Rule of law as a principle in the EU
The EU is based on the adherence to fundamental rights, democracy and the rule of law. These principles
are inherent to many of the legal texts and documents of the Union. The rule of law is a complex concept.
A variety of theoretical formulations can be identified in academia and among practitioners of different
legal traditions. In scholarly literature, the notion of rule of law is explained basically through formal and
thin conceptions or through substantial and thick conceptions.389 In fact, these two categories are
broad differentiations and there is a spectrum of alternative formulations, with fewer or more conditions
and requirements.390 The formal conceptions of rule of law impose requirements only with regard to the
form of legality the law must take and in terms of procedural law. However, they say nothing about the
actual content of the law itself.391 Substantive theories include various specifications on the content of
the laws that are added to the formal elements and include versions with different amounts of substantive
elements.392
Various formulations of the rule of law can not only be found in academia, but also in the legal texts of
international organisations active in promoting the rule of law. These conceptions reflect the different
mandates or purposes of the organisations at the international level.393 The EU has determined the rule
of law as an essential principle and foundational value, nevertheless, no uniform definition of the notion
is given in its legal documents. Similarly, the UN and the CoE have issued different rule of law conceptions
within their documents and legal texts that range from formal to substantive notions.394
The European Commission for Democracy through Law (Venice Commission) at the CoE adopted a
consensual definition of the rule of law that should allow a practical application and expanded the formal
conception of the rule of law by several other elements. The substantial definition of the rule of law
provided by the Venice Commission includes (1) Legality, including a transparent, accountable and
democratic process for enacting law (2) Legal certainty (3) Prohibition of arbitrariness (4) Access to justice
before independent and impartial courts, including judicial review of administrative acts (5) Respect for
human rights (6) Non-discrimination and equality before the law.395 In its Communication on a new EU
Framework to strengthen the Rule of Law, the European Commission makes reference to this definition
that reflects the important but not exhaustive common and generally shared traits of the rule of law in
the Union.396
389
Ronald Janse, Maria D Sanchez Galera and Rain Liivoja, Rule of Law Inventory Report: Academic Part (HiiL, 2007)
9-11.
390
Brian Z Tamanaha, On the Rule of Law: History, Politics, Theory (Cambridge University Press 2004) 91.
391
Paul Craig, Formal and Substantive Conceptions of the Rule of Law: An Analytical Framework [1997] Public Law
466, 467.
392
Tamanaha (n 390) 91.
393
Janse, Sanchez Galera and Liivoja (n 389) 28.
394
Laurent Pech, Rule of Law as a Guiding Principle of the European Unions External Action (2012) 3 CLEER Working
Papers <http://www.asser.nl/upload/documents/2102012_33322cleer2012-3web.pdf> accessed 13 January 2014,
29-31.
395
European Commission for Democracy Through Law, Report on the Rule of Law CDL-AD (2011) 003rev, para. 41.
396
European Commission, A new EU Framework to Strengthen the Rule of Law (Communication) COM (2014) 158
final, Annexes, 1-2.
128
FRAME
This paper follows this definition of the rule of law, comprising the elements as described above. However,
the limited scope of the paper does not allow for a comprehensive examination of the use of indicators
to measure all the rule of law elements inherent to this substantial conception. Therefore, the analysis is
rather focused on the measurement of the judicial system as a core element of the rule of law and, in
particular, the independence and accountability of the judiciary and access to justice.
A.
The rule of law as an internal principle of the EU
The rule of law as an internal principle of the EU derives from the law of the EU, from the jurisdiction of
the Court of Justice of the EU and from national doctrines. Based on these sources, there appears an EU
supranational concept of the rule of law that can be described as a constitutional principle of the
European Union.397 Initially, the EU primary law did not contain any explicit reference to the rule of law.398
The concept was first mentioned by the Court of Justice in the judgment Les Verts, where the rule of law
was determined as one of the fundamental principles of the constitutional framework of the European
Community (EC).399
The treaty amendments following this judgement the Maastricht Treaty of 1992,400 the Amsterdam
Treaty of 1997401 and the Nice Treaty of 2001402 have led to the inclusion of multiple references to the
rule of law in the EUs primary law. The rule of law was referred to in the Preamble of the TEU as one of
the common principle of the contracting parties. With the Treaty of Amsterdam, an important provision
was inserted into the new Art. 6 TEU, which provided that the Union is founded on the rule of law, a
principle that is common to the Member States. According to the European Court, this principle is,
therefore, a criterion for assessing the legality of the actions of its institutions and of the Member States
in matters for which the Union has jurisdiction.403 The Treaty of Amsterdam has also introduced the
development of the Union as an area of freedom, security and justice, where mutual recognition and
mutual trust evolved as the main principles in order to guarantee the protection of individual rights and
to enhance judicial cooperation. EU policies in the area of the rule of law, democracy and fundamental
rights were aimed to maintain and further develop the Union as an area of freedom, security and justice.
This objective became the framework for the EU action in the field of justice and home affairs.404
397
Laurent Pech, The Rule of Law as a Constitutional Principle of the European Union, (2009) 4 Jean Monnet
Working Paper <http://www.jeanmonnetprogram.org/papers/09/090401.pdf> accessed 15 January 2014.
398
The Treaty establishing the European Economic Community, signed in Rome, 25 March 1957 did not mention the
rule of law as an underlying principle.
399
see Case 294/83 Les Verts v Parliament [1986] ECR 1339, para. 23.
400
Treaty on European Union [1992] OJ C191/01 (TEU).
401
Treaty of Amsterdam Amending the Treaty on European Union, the Treaties Establishing the European
Communities and Certain Related Acts [1997] OJ C340/01 (Treaty of Amsterdam).
402
Treaty of Nice Amending the Treaty on European Union, the Treaties Establishing the European Communities and
Certain Related Acts [2001] OJ C 80/01 (Treaty of Nice).
403
see Case C-345/04 P Gestoras Pro Amnistia and Others v Council [2007] ECR I 1631, Opinion of AG Mengozzi,
para. 77.
404
Marloes Spreeuw, Double Standards in the Application of the EU Values and Principles: Member States versus
Potential Candidate and Candidate Countries (2012) <http://www.pravo.unizg.hr/_download/repository/
Paper_M._Spreeuw.pdf> accessed 6 April 2014, 2.
129
FRAME
Furthermore, the Amsterdam Treaty has led to the inclusion of a provision that allows for sanctions
against a Member State in case of a serious and persistent breach of the values laid down in former Art.
6 TEU. The entry into force of the Nice Treaty also gave the Union the capacity for preventive action in
the event of a clear threat of a serious breach. Art. 7 TEU still constitutes one of the core legal instruments
for such situations. Not only member states actions when implementing EU law are covered by Art. 7,
but also areas, where member states act autonomously. The European Commission, the European
Parliament and the Member States are conferred the power to monitor the rule of law in the EU and
identify potential risks.405 The EU institutions are supported in their duty to monitor fundamental rights
and the rule of law by the European Union Agency for Fundamental Rights (FRA), founded in 2007.406 The
mechanism of Art. 7 is based upon political persuasion, but also has a punitive dimension. Possible
sanctions foreseen within the Art. 7 mechanism involve the suspension of certain rights, such as the voting
rights of the representatives in the Council.407 In its Communication of 2003, the European Commission
as the guardian of the Treaties stated its intention to exercise its new rights in full and with a clear
awareness of its responsibility.408 However, as the Commission did not establish any centralised
monitoring tools to evaluate respect of human rights or rule of law, there was no clarity, on how to make
the Art. 7 mechanisms operational. So far, Art. 7 has never been used in practice and the mechanism is
considered as a last resort solution409 by EU policy-makers.
With the entering into force of the Lisbon Treaty on 1 December 2009,410 the reference to the rule of law
as foundational principle of the Union was included in Art. 2 TEU, which provides that
[t]he Union is founded on the values of respect for human dignity, freedom, democracy,
equality, the rule of law and respect for human rights, including the rights of persons
belonging to minorities. These values are common to the Member States in a society in which
pluralism, non-discrimination, tolerance, justice, solidarity and equality between women and
men prevail.
Even though multiple references to the rule of law can be found in the Treaties, the term is nowhere
defined in the EU primary law. Art. 2 TEU includes an explicit linkage of the EUs constitutional system
with the traditional values of the Member States. In this system, the rule of law is presented as a value
that is common to the EU Member States. Despite the constitutional traditions and the persistence of
405
Sergio Carrera, Elspeth Guild and Nicholas Hernanz, The Triangular Relationship between Fundamental Rights,
Democracy and the Rule of Law in the EU: Towards an EU Copenhagen Mechanism (CEPS Paperbacks 2013) 5-6.
406
European Union, The European Union Agency for Fundamental Rights (16 July 2013)
<http://ec.europa.eu/justice/fundamental-rights/agency/index_en.htm> accessed 2 May 2014.
407
Carrera, Guild and Hernanz (n 405) 7.
408
European Commission, On Article 7 of the Treaty on European Union: Respect for and Promotion of the Values
on which the Union is Based (Communication) COM (2003) 606 final, 3.
409
In his speech on the State of the Union on 11 September 2013 Commission President Jos Manuel Barroso called
the Article 7 mechanism a nuclear option.
410
Treaty of Lisbon Amending the Treaty on European Union and the Treaty Establishing the European Community,
signed at Lisbon, 13 December 2007 [2007] OJ C306/01 (Treaty of Lisbon).
130
FRAME
some significant differences, these national interpretations can be a useful guidance to identify a
European meaning of the rule of law.411
The EU assumes a rather substantive conception of the rule of law, as in the EUs constitutional framework
the rule of law is always mentioned in conjunction with the principles of democracy and respect for
fundamental rights. The renewed EU fundamental rights framework412 that has been introduced with the
entering into force of the Treaty of Lisbon constitutes a new reference point for a substantive meaning
and the content of the rule of law in the EU. The new Art. 6 TEU gives the Charter of Fundamental Rights
of the European Union413 the same legal value as the Treaties establishing the European Union and makes
reference to the European Convention for the Protection of Human Rights and Fundamental Freedoms
(ECHR). The rule of law and the right to a fair trial are enshrined in Art. 6 of the ECHR and in Art. 47 of the
Charter of Fundamental Rights of the European Union. Based on the provisions of these Articles, the
judiciary must be independent and impartial and everyone shall be entitled to fair and public hearings
within a reasonable time. The guarantee of legal aid shall ensure effective access to justice. Art. 6 in
connection with Art. 14 of the ECHR and Art. 47 in connection with Art. 21 of the Charter of Fundamental
Rights of the European Union also provide for equal treatment and fair trials free from discrimination,
applicable to both, civil and criminal procedures. The equal treatment imperative requires formal equality
in the access to justice, in front of the law and through the law and, therefore, is binding on the jurisdiction
and on the legislation.414
B.
The rule of law as criterion for the accession of new Member States
The principles of democracy and respect for fundamental rights have always been decisive elements for
accession to the EC and later to the EU.415 Together with the willingness to accept the fundamental
objectives of the Union and the capacity to adopt the entire body of legislation (acquis communautaire),
these principles were at the heart of the conditions imposed on new Member States in the first
enlargement rounds. The end of the Cold War opened up new perspectives for enlargement to the postcommunist countries of Central and Eastern Europe. This was the first time that immediate accession was
denied to applicant countries due to an insufficient level of development and capacity to implement the
acquis.416 There were heavy concerns, as the functioning of the internal market and mutual trust among
Member States rely on the compliance with the rules and credible commitments of all members. These
concerns led to imposing strict conditions for EU accession. Therefore, the EU policy towards the Central
and Eastern European countries is commonly described as a policy of conditionality. The desire of these
countries to join the EU in combination with a need for effective solutions to domestic policy challenges
411
Pech, The Rule of Law as a Constitutional Principle of the European Union (n 397) 41.
413
Charter of Fundamental Rights of the European Union [2010] OJ C83/389.
414
Anke Sembacher and Klaus Starl, Anti-Discrimination for Judges and Prosecutors: General Part, (ETC Graz 2006)
<http://antidiscrimination.etc-graz.at/cms/fileadmin/user_upload/Projekte/laufend/ADTJ/English/
manual_english_090606.pdf> accessed 24 June 2014, 1-7.
415
See e.g. European Commission, Solemn Declaration on European Union (1983) 6 Bulletin of the European
Communities.
416
Viljar Veebel, European Unions Positive Conditionality Model in Pre-accession Process (2009) 13/3 TRAMES 207,
212 and 220-21.
412
131
FRAME
have provided strong incentives for the target governments to comply with the imposed obligations.417
Contrary to the former accessions, the candidate countries of the fifth enlargement round were not
granted the right to opt-out from any EU policy.418
With the establishment of the EU by the Treaty of Maastricht, it became necessary to identify general
criteria that have to be fulfilled by countries in order to be eligible for EU accession.419 At the Copenhagen
meeting in 1993, the European Council has agreed on objective criteria required for EU membership the
so called Copenhagen Criteria. The guarantee of the rule of law is one of the key elements identified as
the political criterion for EU accession. Besides the economic criteria and the criteria concerning the
adoption of the acquis communautaire, membership requires that the candidate country has achieved
stability of institutions guaranteeing democracy, the rule of law, human rights and respect for and
protection of minorities.420 The formal use of the rule of law as a principle in EU accession was brought
about by the Treaty of Amsterdam. Art. 49 TEU constitutes the legal basis for any accession of new
Member States. According to Art. 49 TEU, all European countries can apply for membership of the EU.
However, the applicant country must adhere to the values referred to in Art. 2 TEU (former Art. 6 TEU) in
order to join the EU. Even though the Copenhagen criteria are not explicitly mentioned, Art. 49 TEU refers
to these conditions by stipulating that [t]he conditions of eligibility agreed upon by the European Council
shall be taken into account.
While the association of East and Central European countries had proven to be a successful step towards
integration, the EUs relations towards Balkan countries experienced several difficulties.421 The countries
in the post-conflict situation experienced a range of very specific problems and were unable to develop
good governance practices on their own behalf. Eliminating corruption, promoting human rights,
encouraging the exercise of citizen rights and the protection of minorities are, therefore, fundamental
tasks of the reform agenda in these countries. Very specific efforts are needed to establish and uphold
rule of law in such settings.422 The EU has become the prime state builder in the Western Balkans. The
transfer of the mechanism of conditionality to state-building projects has led to tensions between the
accession conditionality and the efforts to end political conflicts.423 To deal with the history of ethnic
antagonism, unstable economies and widespread corruption, the EU integration policy emphasized
stabilisation and regional cooperation on the Balkans. The willingness to re-establish good neighbourly
417
Frank Schimmelfennig and Ulrich Sedelmeier, Governance by Conditionality: EU Rule Transfer to the Candidate
Countries of Central and Eastern Europe (2004) 11/4 Journal of European Public Policy 669, 670.
418
Veebel (n 416) 223.
419
Faton Bislimi, EU Foreign Policy towards Balkans: An Opportunity or a Challenge? (2010) 1 The Western Balkans
Policy Review 29, 40.
420
European Council, Conclusions of the Presidency (Copenhagen, June 1993) para. 7 (A) (iii).
421
Bislimi (n 419) 40-41.
422
Markus Mstl, Core Tasks of Good Governance and Their Impact on Peacebuilding and Human Security, in
Wolfgang Benedek and others (eds), MultiPart Thematic Paper on Multi-Stakeholder Partnerships Active in the Field
of Good Governance, Democracy and Rule of Law (ETC Graz 2010) 53.
423
Florian Bieber, Building Impossible States? State-Building Strategies and EU Membership in the Western Balkans
(2011) 63/10 Europe-Asia-Studies 1783, 1791.
132
FRAME
relations and economic cooperation with one another was a precondition for the countries to proceed in
the accession progress and to access financial assistance.424
In 1999, the EU launched the Stabilisation and Association Process (SAP) as a more comprehensive and
individualized framework for Balkan countries to make further progress on their way to EU membership.
The instruments covered by the SAP continued to depend on compliance of each country with the general
and specific conditions set out by the EU.425 The importance of the Copenhagen criteria in the accession
of new Member States was emphasized in the Thessaloniki Summit in 2003, where all Western Balkan
countries were granted the perspective of EU membership subject to fulfilment of the necessary
conditions. The use of conditionality is a very powerful tool to export the values on which the EU is
founded.426
In the context of EU enlargement it is critically important for the EU institutions to have a clear idea about
the status of the rule of law, justice and fundamental rights in the applicant countries.427 Therefore, the
fulfilment of the political criteria and the adoption, implementation and application of the acquis are
continuously assessed during the accession negotiations, and the Commission issues regular monitoring
reports on each candidate country. These continuous assessments are intended to make sure that the
countries are prepared to meet their obligations as Member States, once they join the EU. The acquis is
divided into negotiation chapters, each of which corresponds to a different area of EU legislation and
policy.428 EU policies aimed to maintain and further develop the Union in the area of the rule of law and
democratic governance are now covered by chapter 23 (judiciary and fundamental rights), and chapter
24 (justice, freedom and security). These chapters are now at the centre of the negotiation process.
Member States must ensure respect for fundamental rights and a solid legal framework, providing for
reliable institutions and an independent and efficient judiciary, guaranteeing fair trial procedures. Equally,
the fight against corruption and organised crime must be ensured. Together with the Copenhagen political
criteria, chapters 23 and 24 are essential for safeguarding and developing the rule of law in enlargement
countries.429
Most of the enlargement countries are facing a number of key challenges and need to go through major
reforms, as necessary legislation and institutions have to be put in place to fully assume the obligations
of EU membership. The EU has designed several instruments and programmes to support the enlargement
countries in their reforms. From 2007 to 2013, the Instrument for Pre-Accession Assistance (IPA) granted
funds for both, candidate countries and potential candidates to support reforms with financial and
424
Arolda Elbasani, EU Enlargement in the Western Balkans: Strategies of Borrowing and Inventing (2008) 10/3
Journal of Southern Europe and the Balkans 293, 300.
425
Florence Benoit-Rohmer and others, Human Rights Mainstreaming in EUs External Relations (Study, European
Parliament 2009) 81.
426
Spreeuw (n 404) 8.
427
Martin Gramatikov and Morly Frishman, Measuring the Rule of Law, Justice and Fundamental Rights (Concept
Paper, HiiL 2013) 9.
428
European Union, The Accession Process for a new Member State (28 February 2007) <http://europa.eu/
legislation_summaries/enlargement/ongoing_enlargement/l14536_en.htm> accessed 12 April 2014.
429
European Union, Chapters of the Acquis (27 June 2013) <http://ec.europa.eu/enlargement/policy/conditionsmembership/chapters-of-the-acquis/index_en.htm> accessed 12 April 2014.
133
FRAME
technical means, building up the capacities of the enlargement countries. IPA has replaced the former
programmes as one single instrument pre-accession assistance.430 Through IPA over 800 million preaccession assistance has been provided in the period from 2007 to 2013 to improve the justice sector,
independence of the judiciary, fight against corruption and organised crime as well as border
management and security. Furthermore, the functioning of institutions guaranteeing democracy has been
supported by over 30 million pre-accession assistance, as well as 190 million to support civil society
organisations.431
After the expiry of IPA in 2013, IPA II was launched as the second Instrument for Pre-Accession
Assistance.432 The EU Multiannual Financial Framework for 2014-2020 allocates the budget that is agreed
for seven years to the EUs political priorities of the External Action Financing Instruments.433 One of the
main underpinning principles of the external instruments for the period is a greater focus on human rights,
democracy and good governance. When it comes to allocating and disbursing funds to partner countries,
the EU will take account of these values and the greater emphasis on the rule of law is represented also
in the IPA II. With IPA II the EU continues to provide substantial support for the preparation of accession
in order to support for political reforms in the enlargement countries, inter alia through strengthening
democracy and the rule of law.434
VI.
Mechanisms for measuring the rule of law
The measurement of the rule of law has a shorter history than the measurement of democracy and human
rights. It started in the 1980s with first attempts by academics to measure good governance in order to
classify countries according to the results obtained in this respect. These measures of good governance
included civil and political liberties or political freedoms as proxy measures for the rule of law. One of the
first works was Freedom Houses Freedom in the World that used so-called proxy measures of the
liberties and freedoms of political and legal institutions to approximate rule of law and governance.435
In the early 1990s the debate moved from scientific journals to international discussion as international
organisations, like the World Bank and the International Monetary Fund, started to consider the ways, in
which good governance influenced the economic performance of countries in their development
assistance.436 World Bank researchers emphasized that only in well-governed countries foreign aid could
430
European Union, IPA (7 September 2012) <http://ec.europa.eu/enlargement/policy/glossary/terms/ipa_

en.htm> accessed 12 April 2014.
431
European Commission, Enlargement Strategy and Main Challenges 2013-2014 (Communication) COM (2013)
700 final, 7-9.
432
Regulation (EU) 231/2014 of the European Parliament and of the Council of 11 March 2014 Establishing an
Instrument for Pre-Accession Assistance (IPA II) [2014] OJ L77/11.
433
European Union, Memo: The Multiannual Financial Framework: The External Action Financing Instruments (11
December 2013) <http://europa.eu/rapid/press-release_MEMO-13-1134_en.htm> accessed 14 April 2014.
434
Reg 231/2014, Art. 2 (1) (a) (i).
435
Todd Landman, Map-Making and Analysis of the Main International Initiatives on Developing Indicators on
Democracy and Good Governance (Human Rights Centre, University of Essex 2003) 28.
436
Robert Ccile and Jurine Magalie, Human Rights and Democracy Indicators: Definition and Implications (Study,
European Parliament 2006) 6.
134
FRAME
be used effectively and called attention to the institutional prerequisites for economic growth. The rule of
law has been regarded as a basis to provide a secure environment for contracts, investments and market
transactions and for a functioning market economy. In that way, good governance became both, a means
of achieving development and a development objective in itself.437
Also other international organisations, like the UN and the EU, have imposed the protection of the rule of
law as a condition for the provision of financial assistance on recipient countries.438 In response to the
growing demand, international organisations started to work on quantifiable indicators and benchmarks.
A number of aggregate governance and rule of law indicators have been produced. However, these first
attempts of international organisations to measure a countrys adherence to the rule of law did not escape
criticism as methodological problems were debated among the scientific community.439
A.
Major initiatives for measuring the rule of law by international
organisations
There have been many attempts to develop mechanisms for measuring the rule of law by academia,
states, intergovernmental organisations andNGOs. Three major initiatives by the World Bank, the CoE and
the UN have been selected to show different approaches for measuring the rule of law that are of specific
relevance also in the EU. These three initiatives have to be regarded as a non-exhaustive selection of
relevant mechanisms that exist on the international level.440 Based on the analysis of these three
initiatives, basic principles for the development and use of indicators to measure the rule of law are
identified in the last section of this Chapter.
With the initiation of the Worldwide Governance Indicators (WGI) project, the World Bank has been at
the forefront in developing measurement tools to monitor the performance of national politico-legal
systems over time. The rule of law is defined as one of the six dimensions of governance by the WGI
project. The data for the WGI are gathered from 35 data sources provided by 33 different organizations,
including commercial survey institutes, think tanks, non-governmental organizations, international
organizations and private sector firms. 25 of these different data sources are used by the WGI project to
produce the rule of law index. These data sources capture a mixture of rule of law elements, like strength
and impartiality of the legal system, independence of the judiciary, trust in judicial accountability and law
enforcement but also other elements, like confidence in police force, which are usually not included in the
rule of law conceptions illustrated in Chapter V. Further elements usually assigned to the rule of law can
be found in other dimensions defined by the WGI project.441 The indicators combine the views provided
by a large number of expert survey respondents from the public and private sector, NGOs, enterprises and
citizens in industrial and developing countries. As a result of this sort of data collection, the indicators do
not reflect an objective measurement but rather the perceived level of the rule of law based on the
437
Melissa Thomas, What Do the Worldwide Governance Indicators Measure (Johns Hopkins University 2006) 2.
Landman Map-Making and Analysis (n 435) 2.
439
Ccile and Magalie (n 436) 6.
440
A mapping of main international initiatives on developing indicators for rule of law can be found in Landman
Map-Making and Analysis (n 435).
441
Daniel Kaufmann, Aart Kraay and Massimo Mastruzzi, Governance Matters VIII: Aggregate and Individual
Governance Indicators, 1996-2008 (2009) World Bank Policy Research Working Paper No. 4978, 77-78.
438
135
FRAME
subjective opinions of survey respondents. Therefore, the WGI project is representing the demand side
of the rule of law and does neither measure the rule of law from an institutional point of view, nor pays
attention to what is thought of as the law in the books.442
The European Commission for the Efficiency of Justice (CEPEJ), established by the CoE, conducts regular
evaluations of the judicial systems of the CoE member states to measure the rule of law from an
institutional perspective.443 These evaluations are based on the Pilot Scheme for Evaluating Judicial
Systems,444 an instrument developed by the CEPEJ in 2003. The scheme is meant to gather qualitative and
quantitative information on the daily functioning of judicial systems based on reports by member states
and national correspondents, appointed by the states.445 The pilot scheme is a set of originally 108
questions, categorized by ten topics related to access to justice, functioning and efficiency of justice, fair
trial as well as enforcement of court decisions in both criminal and civil cases. The CEPEJ study is based on
data gathered from judicial institutions, governmental agencies in the justice sector and legal
professionals that represent the supply side of judicial systems. This way, the evaluation scheme is based
on an institutional conception of the rule of law by assuming that the availability of certain guarantees,
proceedings and legal institutions assure adherence to the rule of law. The correspondents are also
delivering replies to questions concerning the rights and position of the users of the justice systems.
However, the evaluation scheme does neither include questions regarding the quality standards of law
practices, nor the level of satisfaction in the society and public trust in judicial systems, as these areas are
falling in the responsibility of other institutions within the CoE.446
The UN launched a Rule of Law Indicators Project in 2008 as a joint initiative of the Department of
Peacekeeping Operations and the Office of the High Commissioner for Human Rights. The UN Rule of Law
Indicators were prepared by experts from the Vera Institute of Justice in collaboration with other
members of the Altus Global Alliance and consultants from academia. The primary objective of the
initiative is to provide an empirically based approach for measuring the strengths and challenges of the
rule of law sector in a given country in order to assist both, international donors and national authorities
in their rule of law reform efforts. It has to be noted, however, that the UN Rule of Law Indicators form a
tool for assessing the rule of law in post-conflict situations. The set of indicators is designed to obtain
information on successes and shortcomings among the law enforcement agencies, the judicial system and
the prison system and to monitor performance of institutions within a country over time. The UN Rule of
Law Indicators Project is not designed to produce a single rating for a country or to rank and compare
countries in terms of their rule of law performance.447 The indicators shall be rated or measured on the
442
Pim Albers, How to measure the rule of law: a comparison of three studies (2007) <http://www.coe.int/t/
dghl/cooperation/cepej/events/onenparle/Albers251007.pdf> accessed on 20 April 2014, 2-3.
443
CEPEJ, Evaluation of European
Judicial Systems <http://www.coe.int/t/dghl/cooperation/
cepej/evaluation/default_en.asp> accessed on 20 April 2014.
444
CEPEJ, Pilot-Scheme for Evaluating Judicial Systems <http://www.coe.int/t/dghl/cooperation/cepej/
Source/GrillePilote_en.doc> accessed on 20 April 2014.
445
CEPEJ, European Judicial Systems (Council of Europe Publishing 2012) 5-6.
446
Albers (n 442) 10.
447
UN, The United Nations Rule of Law Indicators Project: Frequently Asked Questions (07 October 2008)
<http://en.unrol.org/files/FAQ%20-%207%20Oct%202008.doc> accessed on 11 May 2014.
136
FRAME
basis of four main sources of data, including surveys of experts and public surveys, to a smaller extent
administrative and field data, as well as the review of legal and/or administrative documents.448 Similar
to the WGI project, the UN Rule of Law Indicators Project focuses more on the rule of law in practice,
rather than on the law in the books. In this way, the instruments measure the rule of law by performance
and outcome indicators, rather than measuring in detail the institutional means in terms of legal and
regulatory frameworks.449 However, the UN Rule of Law Indicators try to combine multiple data sources,
which shall yield a more nuanced and complete picture. Through the combination of a public survey and
a survey of experts, the UN Rule of Law Indicators describe the operation of justice institutions from
multiple perspectives. This shall also compensate for the weaknesses that often exist in the administrative
data of states in conflict and post-conflict situations.450 The UN Rule of Law Indicators and the guide for
implementation follow an empirically based approach to measure the strengths and effectiveness of law
enforcement, judicial and correctional institutions. So far the UN Rule of Law Indicators have only been
applied in pilot implementations in Haiti and Liberia, with no publicly available results.451 However, the
guide for implementation provides general guidelines for developing and applying indicators to measure
the rule of law.
B.
Mechanisms for measuring the rule of law in the EU and its Member States
As the EU applies a substantive conception of the rule of law by connecting it to democracy and
fundamental rights, the monitoring of the rule of law in the EU and its Member States must equally focus
on the law in the books and on the realisation of the rule of law in practice. Accordingly, the EU needs
instruments to measure legal and other relevant institutional inputs as well as compliance of public
institutions with the relevant norms and standards. As described in Chapter V.A Art. 7 TEU places the EU
institutions under the obligation to maintain surveillance in order to detect possible risks of breaches of
the rule of law in the EU Member States or by the EU itself. In monitoring the rule of law in the EU, the
European Commission, so far, mainly relied on sources of information from the EU bodies, from
international organisations like the CoE and civil society organisations.452
In 2013, the Commission presented the EU Justice Scoreboard as a new initiative, for evaluating the
functioning of national justice systems in terms of quality, independence and efficiency.453 The Scoreboard
is a comparative tool, covering all Member States and is used to monitor the functioning of national justice
systems over time. It focuses on civil and commercial justice, encompassing non-criminal cases, litigious
civil and commercial cases as well as administrative cases. The findings are related to the efficiency and
448
DPKO and OHCHR, Rule of Law Indicators Implementation Guide and Project Tools (UN 2011) 24.
Pech, Rule of Law as a Guiding Principle of the European Unions External Action (n 394) 40-41.
450
DPKO and OHCHR (n 448) 1-2.
451
UN, United Nations Rule of Law Indicators Project: Newsletter Number 2 (June 2009)
<http://www.unrol.org/files/Newsletter%20for%20United%20Nations%20Rule%20of%20Law%20Indicators%20Pr
oject%20June%202009.pdf> accessed on 11 May 2014.
452
For a detailed description of the main EU mechanisms for measuring the rule of law see Carrera, Guild and
Hernanz (n 405) 4 et seqq.
453
European Union Press Release: EU Justice Scoreboard: European Commission broadens the scope of its analysis
of Member States' justice systems (27 March 2013) <http://europa.eu/rapid/press-release_IP-13-285_en.htm>
accessed 2 May 2014.
449
137
FRAME
quality of justice systems and the independence of the judiciary. The indicators used to measure the
efficiency of proceedings are the length of the proceedings, the clearance rate and the number of pending
cases. Further factors, like the availability of regular monitoring systems for the courts, the use of ICT
systems, the availability of Alternative Dispute Resolution methods, the training of judges and resources
available for the courts, are part of the assessment.454 The EU Justice Scoreboard uses different data
sources. Most quantitative data were collected in accordance with the CEPEJ methodology for Evaluation
of European Judicial Systems and presented in a study on the functioning of judicial systems prepared in
2012 by the CEPEJ on behalf of the Commission.455 Additional sources of information were data from
Eurostat, the World Bank, the World Economic Forum, the World Justice Project and the European judicial
networks. The EU Justice Scoreboard makes use of benchmarking to promote a high level of the rule of
law in the EU Member States and provides reliable and comparable data on the functioning of national
justice systems. Furthermore, the EU Justice Scoreboard was designed to support the country specific
analysis of the European Semester and to give recommendations to the Member States. However, [i]t is
not a mechanism for guaranteeing the rule of law across the EU.456
A new debate on how to monitor and safeguard the rule of law in the EU and its Member States has
started among the European institutions, when the EU has been confronted with Member States
deficiencies in protecting the rule of law on several occasions. In its Resolution of 3 July 2013, the
European Parliament called on the Commission as the guardian of the Treaty to create an independent
monitoring mechanism and an early-warning system.457 These instruments should make sure that any
potential risks of serious breach of Art. 2 TEU values are addressed at an early stage through a structured
political dialogue with the relevant Member State. The need for a comprehensive monitoring tool for
measuring the rule of law and human rights in the EU was repeatedly expressed by the European
Parliament and the Council, making reference to the major initiatives of international organisations, such
as the CoE and the UN.458 Based on these requests for a collaborative and systematic method to tackle
rule of law issues, the Commission has set out a new framework to ensure an effective and coherent
protection of the rule of law in all Member States459 in 2014. Within this new framework the Commission
seeks to contribute to reach the objective of safeguarding the rule of law in the EU, based on the expertise
of the Venice Commission of the CoE and the FRA. This new framework shall address future threats to the
454
European Commission, The EU Justice Scoreboard: A Tool to Promote Effective Justice and Growth
(Communication) COM (2013) 160 final, 2-5.
455
The full study prepared by the CEPEJ for the European Commission in 2012, is available on the website
<http://ec.europa.eu/justice/effective-justice/files/cepej_study_justice_scoreboard_en.pdf>.
456
European Union Memo: Questions & Answers: EU Justice Scoreboard (27 March 2013)
<http://europa.eu/rapid/press-release_MEMO-13-288_en.htm> accessed 2 May 2014.
457
European Parliament, Situation of Fundamental Rights: Standards and Practices in Hungary (Resolution) P7_TAPROV (2013) 0315.
458
The Council of the European Union underlined the importance to make full use of existing mechanisms for the
assessment of the rule of law in its Conclusions of the Justice and Home Affairs Council Meeting (Luxembourg, June
2013), para. 9. The European Parliament reiterated in its Resolution Evaluation of Justice in Relation to Criminal
Justice and the Rule of Law P7_TA-PROV (2014) 0231, para. 6 that a mechanism for the assessment of the rule of
law must seek complementarity with the work of other international institutions.
459
COM (2014) 158 final, 3.
138
FRAME
rule of law in Member States, before the conditions for activating the mechanisms foreseen in Art. 7 TEU
are met and is an addition to the infringement procedures foreseen under Art. 258 TFEU.460
C.
Basic principles for the use of indicators to measure the rule of law
Based on the foregone description of existing initiatives, basic principles can be identified that need to be
considered for the use of indicators to measure the rule of law in the EU accession process. A meaningful,
comprehensive and acceptable approach for measuring the rule of law requires certain aspects as
summarised below.
(1) Combining structural, process and outcome indicators
Recent research on measuring the rule of law461 suggests using a mix of structural, process and outcome
indicators, such as it is proposed by the UN Rule of Law Indicators Project.462 As the law in the books is
not necessarily always translated into practice, the rule of law cannot be measured through structural
indicators only. Following a substantive conception of the rule of law, indicators should also evaluate the
impact of governments policies and the perceived level of the rule of law by individuals.463 To measure
the results at the level of beneficiaries and to identify any inequalities or discrimination, it makes good
sense to further disaggregate data by individual characteristics, like age, gender or ethnic groups.464 By
combining structural, process and outcome indicators, the commitments of states can be assessed at the
same time as their performance in terms of delivering justice. In this way, the indicators are
complementary and can only properly reflect the realisation of rule of law standards in terms of
commitment, effort and results, when taken together.465
(2) Combining quantitative and qualitative indicators
Creating a meaningful set of indicators requires a combination of both, quantitative and qualitative
indicators. Quantitative indicators can be used, where it is possible to identify statistical information, and
their numerical precision often allows for a more objective interpretation of results. In this way,
quantitative indicators are usually preferable. However, qualitative indicators can supplement the
numerical data with a richness of information that cannot be displayed by pure statistics.466 As indicated
in the previous sections, there is a series of existing mechanisms from international organisations that
provide for quantitative indicators to measure the rule of law. These quantitative measures provide
460
European Parliament The Situation of Fundamental Rights in the European Union (Resolution) P7_TA-PROV
(2014) 0173, para. 11.
461
see Berenschot and Imagos, Evaluation of Governance, Rule of Law, Judicial Reform and Fight Against Corruption
and Organised Crime in the Western Balkans: Lot 2 Main Report (2012) 33.
462
DPKO and OHCHR (n 448).
463
Todd Landman, The Scope of Human Rights: From Background Concepts to Indicator (Background paper for the
UN Expert Meeting on Human Rights Indicators, Turku 2005) 13-26.
464
Klaus Starl and Jennifer Pinno, Study on Challenges in the Development of Local Equality Indicators: A HumenRights-Centred Model (Study carried out by ETC Graz in the framework of the Working Group on Indicators of the
European Coalition of Cities against Racism, UNESCO 2010) 28.
465
Gauthier de Beco, Human Rights Indicators for Assessing State Compliance with International Human Rights
(2008) 77 Nordic Journal of International Law 23, 45.
466
USAID Center for Development Information and Evaluation, Performance Monitoring and Evaluation Tips:
Selecting Performance Indicators (1996) <http://pdf.usaid.gov/pdf_docs/PNABY214.pdf> accessed 20 June 2014, 4.
139
FRAME
valuable sources of information that can be applied also to underpin qualitative assessments of the rule
of law in a specific country.467
(3) Combining universal and country-specific indicators
Like for any other set of indicators, a balance between universally relevant indicators and country-specific
indicators needs to be found. Certain elements of the rule of law, relevant across all countries, shall be
measured by universal indicators. Such universal indicators allow for making comparisons of particular
elements across countries. However, universal indicators do not take into account the different capacities
of each individual state.468 Through the creation of country specific indicators, the contextual relevance
of the assessment can be increased and differences in the nature of institutions, policies and in the state
priorities can be better reflected.469
(4) Combining multiple data sources
By using multiple sources of data, the institutional perspective of the supply side of the rule of law can
be combined with the demand side, reflecting the rule of law in practice. A set of indicators combining
multiple data sources must be flexible enough to cope with the different types of data but at the same
time should be standardized enough to be meaningful concerning the indicators results.470 Indicators
need to be clearly defined and carefully selected in order to facilitate the identification of relevant data
sources. The required level of reliability of the data has to be adjusted to the purpose of measurement to
allow for reasonably confident decisions. While quantitative data can be gathered by reviewing
administrative documents and legislation or from statistical surveys, qualitative information can mainly
be obtained from media review, public or expert surveys. The involvement of responsible authorities has
proven to be crucial in selecting the most relevant indicators and accessing sensitive data sources.471
(5) Tracking changes over time and setting benchmarks
Indicators are most useful, when the same measure can be tracked over time, as this allows identifying
improvements or deteriorations in the realisation of the different rule of law elements.472 A baseline
assessment drawing on clear indicators is a helpful starting point for developing specific reforms. The
progress achieved can then be measured against these baseline measurements. In this way, the
monitoring schemes are capable of revealing continuous trends and dynamics, which can be the basis to
identify feasible benchmarks.473 Setting benchmarks allows for monitoring the progress towards reaching
a specific target or goal. By using intermediate benchmarks, progress can be monitored on a regular basis
to help states reach the obligations.474 Effective benchmarking requires a clear understanding of the rule
467
see Berenschot and Imagos (n 461) 33 et seqq.

OHCHR, Report on Indicators for Promoting and Monitoring the Implementation of Human Rights (UN 2008)
para. 16.
469
DPKO and OHCHR (n 448) 2.
470
Starl and Pinno (n 464) 27.
471
Berenschot and Imagos (n 461) 16 et seqq.
472
DPKO and OHCHR (n 448) 4.
473
Martin Gramatikov and Ronald Janse, Monitoring and Evaluation of the Rule of Law and Justice in the EU: Status
Quo and the Way Ahead? (Concept Paper, HiiL 2012) 2.
474
de Beco (n 465) 47.
468
140
FRAME
of law and of the key features it compromises. For the development of meaningful benchmarks
measureable indicators, adequate measurement methods and reliable data sources are necessary.475
VII.
Measuring the rule of law in the EU enlargement policy
Based on the presidency conclusions from the Luxembourg European Council in 1997, the EU established
an evolutionary and inclusive accession process, under which all candidate countries are destined to join
the Union on the basis of the same criteria and [] on equal footing.476 The rule of law forms a basic
condition for EU accession as one component of the political criteria agreed in the Copenhagen European
Council in 1993, namely the stability of institutions guaranteeing democracy, the rule of law, human rights
and the respect for and protection of minorities. Furthermore, several elements of the rule of law are
also part of the acquis criteria that is the ability to take on the obligations of membership, including
adherence to the aims of political, economic and monetary union.477
Acting on the mandate of the Council and the European Council, the European Commission has a key
position in the assessment of the progress made by the candidate and potential candidate countries
towards fulfilling the conditions of the Copenhagen criteria. Each year the Commission presents a set of
documents to the Council, explaining its strategy in line with the EU enlargement policy and reporting on
the progress achieved at country level. The so-called Enlargement package adopted by the Commission
includes the Progress Reports for each candidate and potential candidate country as well as the annual
Enlargement Strategy Paper, taking stock of the developments in the last twelve months and setting out
the way forward for the coming year.478 The Council and the European Council take their decisions to
move on in the accession process of a country based on the recommendations contained in the
Commissions assessments. In this way, the assessment of the progress towards the fulfilment of the
membership criteria is one of the most important and powerful tools in the accession process.479
In this Chapter, the use of indicators for measuring the rule of law in the EU enlargement policy is analysed
with a specific focus on the transformation of the judicial system in three case studies of the countries
Bulgaria, Croatia and Montenegro. Along the three case studies, the adaptations of the EU monitoring
mechanism for measuring the rule of law in terms of the independence and accountability of the judiciary
and access to justice are analysed over time. The three case studies reflect three different periods in the
EU enlargement policy. Three major steps, that were aimed to improve the measurement of the rule of
law in the enlargement process, can be identified. The first step was the introduction of a monitoring
system in the accession process of Bulgaria to overlook the progress achieved in fulfilling the Copenhagen
political criteria. The second step was the establishment of a specific negotiation chapter on judiciary and
475
Raffaella Del Sarto and others, Benchmarking Democratic Development in the Euro-Mediterranean Area:
Conceptualising Ends, Means, and Strategies (EuroMeSCo Report 2006) 14-15.
476
European Council, Presidency Conclusions (Luxembourg, December 1997).
477
European Council (n 60) para. 7 (A) (iii).
478
European Union, Strategy and Progress Reports, (23 October 2013) <http://ec.europa.eu/enlargement/
countries/strategy-and-progress-report/index_en.htm>, accessed 10 June 2014.
479
Dimitry Kochenov, EU Enlargement and the Failure of Conditionality: Pre-accession Conditionality in the Fields of
Democracy and the Rule of Law (Kluwer Law International 2008) 59.
141
FRAME
fundamental rights as an integral part of the EU acquis and the identification of specific benchmarks in
the accession process of Croatia. Finally, the new approach to rule of law in the EU enlargement policy is
guiding the accession negotiations with Montenegro and has put the rule of law at the heart of the
accession process.
The case studies conducted in this Chapter are based on an analysis of the Progress and Monitoring
Reports issued by the European Commission and other relevant documents related to the EU enlargement
policy. Furthermore, recent research and studies conducted by different authors are considered.
Background information gained through an interview conducted by the European Training and Research
Centre for Human Rights and Democracy (ETC Graz) with an official from DG enlargement480 is used to
back the findings of the case studies and to derive general implications for the use of indicators in EU
enlargement policy in the last part of this Chapter.
A.
The case of Bulgaria
The Europe Agreement481 has entered into force on 1 February 1995 and provided the legal basis for
relations between Bulgaria and the EU (the EC at that time). Based on this framework for political dialogue
and the gradual integration into the Union, Bulgaria has submitted the application for membership of the
EU on 14 December 1995. At that time all together eleven applications for membership were submitted
to the EU, which led to the adoption of a new strategy for enlargement by the European Council.
1.
The new strategy for EU enlargement
With regard to the criteria adopted by the Copenhagen European Council in 1993 and on request of the
Madrid European Council in 1995,482 the Commission had developed a methodology for the objective
assessment of applications for membership that should ensure that all candidate countries were treated
on an equal basis. A composite paper on enlargement was presented as Part Two of the Agenda 2000483
and explained the methodology for an assessment of applications for membership through the application
of predefined accession criteria.
The methodology for the assessment of applications developed by the Commission was affirmed by the
Luxembourg European Council in 1997, where two significant elements were introduced into the
Enlargement Strategy. On the one hand, the European Council adopted a comprehensive enlargement
framework, under which the Commission was asked to submit Regular Reports to the Council, assessing
the candidate countries preparedness for accession in the light of the Copenhagen criteria. For these
assessments the Council decided to follow the method adopted by Agenda 2000 in evaluating applicant
States ability to meet the economic criteria and fulfil the obligations deriving from accession.484 The
480
ETC Graz, Interview with an official from DG Enlargement (Brussels and Graz, 17 June 2014). The interview was
conducted in the context of the FRAME research project. The project FRAME - Fostering Human Rights Among
European Policies is a large-scale, collaborative research project funded under the EUs Seventh Framework
Programme. The interview report is not publicly accessible but was available for the author.
481
Europe Agreement establishing an Association between the European Communities and their Member States, of
the one part, and the Republic of Bulgaria, of the other part, Brussels, 8 March 1993 (Europe Agreement).
482
European Council, Presidency Conclusions (Madrid, December 1995).
483
European Commission, Agenda 2000: For a Stronger and Wider Union (Communication) COM (97) 2000 final.
484
European Council (n 476) para. 29.
142
FRAME
progress towards meeting each criterion was to be assessed against a standardised checklist that ensured
transparency and equal treatment of all countries aspiring accession to the EU. On the other hand, the
Luxembourg European Council introduced an enhanced pre-accession strategy, consisting of increased
pre-accession aid and the introduction of the Accession Partnerships.
The Accession Partnerships were concluded to support the applicant countries in their preparations for
membership by providing a single framework for the countries actions, to address the priority areas
identified in the Regular Reports and to target the financial assistance available from the EU. A number of
financial instruments were designed for candidate and potential candidate countries to carry out the
demanded reforms. The PHARE programme485 was remodelled into an accession driven instrument for
institution building and investment in the regulatory infrastructure to ensure compliance with the
acquis.486 The measures implemented through the EU financial assistance as well as the progress achieved
through the Accession Partnerships and related National Programmes or Action Plans were to be assessed
within the Regular Reports.
2.
The monitoring mechanism applied to Bulgaria
In line with the new strategy for enlargement and the adoption of the comprehensive enlargement
framework by the Luxembourg European Council in 1997, the Commission had developed a monitoring
system to effectively underpin the accession negotiations of the Council and the Member States. This
monitoring system is made up of several monitoring tools (as presented in Figure 1), designed to assess
the priorities identified in the policy documents and reform plans.
Figure 1: The monitoring mechanism applied to Bulgaria
a)
The Opinion on the Application for Membership
On request of the Madrid European Council in 1995, the Commission conducted a baseline assessment of
Bulgarias preparedness for membership and presented an Opinion on Bulgarias Application for
Membership of the European Union in 1997.487 In the light of the Copenhagen criteria, this assessment
followed the threefold division into political criteria, economic criteria and the ability to assume the Union
acquis. The extent to which democracy and the rule of law were actually operating in line with the EU was
assessed under the political criteria for membership. In conducting the assessments, the Commission
utilised a wide range of information coming from the Bulgarian authorities, the Member States and
485
The Poland and Hungary Aid for Restructuring of the Economies (PHARE) was initially designed to support
transition to democracy and the market economy in Poland and Hungary.
486
European Commission, Composite Paper: Reports on progress towards accession by each of the candidate
countries COM (99) 500 final, 8.
487
European Commission, Opinion on Bulgarias Application for Membership of the European Union DOC/97/11.
143
FRAME
numerous international organisations. Based on the Commissions opinion and on the Presidency
Conclusions from the Luxembourg European Council in 1997, the accession process with Bulgaria was
formally launched by the Ministers of Foreign Affairs Council on 30 March 1998.
b)
The Regular Reports on Bulgarias Progress towards Accession
The process of accession and negotiation followed the structure as introduced by the decision of the
Luxembourg European Council in 1997. In April 1998, the Commission launched an analytical examination
of the acquis, the so-called screening, with the aim to identify any possible problem areas for the
adoption of the acquis. Starting in 1998, the Commission submitted annual Regular Reports on Bulgarias
progress towards accession.488 The structure of these Regular Reports followed the structure of the
Commission Opinion on Bulgarias application for membership from 1997 and was based on the
Copenhagen criteria. The Regular Reports presented the results of the monitoring and provided both,
criticism and support for the applicant country. Furthermore, the reports were of high significance for the
decisions on the opening and closing of negotiating chapters.489 Starting with the Regular Report of 2000,
the assessment of the acquis criteria has been restructured and the ability to assume the obligations of
membership was divided into 31 negotiating chapters.490 The structure of the reports was continuously
improved with every reporting year, leading to higher consistency and transparency. Furthermore, the
Regular Reports were gradually becoming more and more detailed, also broadening the scope of the
issues assessed related to the rule of law.491
To address the deficiencies identified by the Commission in its assessments, concrete, achievable goals
were formulated in the Accession Partnerships, Action Plans and in a Roadmap for Bulgaria. The Accession
Partnerships with Bulgaria set out a single framework for the reforms in the priority areas identified by
the Commission and for the financial assistance supporting the implementation of these reforms. The
priorities were divided into short and medium term, depending on whether substantial improvement was
expected within one year or more than one year to complete.492 The Accession Partnerships were under
regular revision to take account of the progress made and to allow for new priorities to be set.
Carrying out the assessment for the Regular Reports, the Commission collected information from a
number of data sources. A main source of information was the review of legislation adopted,
administrative documents as well as official statistics. The Bulgarian authorities were invited to provide
information on the progress made since the publication of the last report. Various peer reviews conducted
by country experts have taken place in order to assess Bulgarias administrative capacity in a number of
areas. Reports from the Commissions delegations and the Member States embassies as well as Council
deliberations and European Parliament resolutions have been taken into account for the preparation of
the reports. The European Commission also used the assessments provided by the Member States with
488
European Commission, Regular Report on Bulgarias Progress Towards Accession COM (98) 707 final, 6.
Kochenov (n 479) 245.
490
European Commission, Enlargement Strategy Paper: Report on Progress Towards Accession by each of the
Candidate Countries COM (2000) 700 final, 25-30.
491
Kochenov (n 479) 246.
492
Council Decision (EC) 1999/857 of 6 December 1999 on the Principles, Priorities, Intermediate Objectives and
Conditions Contained in the Accession Partnership with the Republic of Bulgaria [1999] OJ L335/49, Annex.
489
144
FRAME
regard to the political criteria for membership. Furthermore, the rule of law measurements conducted by
international organisations were important sources of information for the Commission.493
c)
Continued monitoring and the Cooperation and Verification Mechanism
After the closing of the accession negotiations in December 2004, the Commission, as guardian of the
Treaties, had the duty to monitor Bulgarias preparation for accession in the fields, where Bulgaria had
committed itself to completing specific measures by the time of accession.494 The 2006 Monitoring Report
on the state of preparedness for EU membership495 showed that further progress was still necessary in
the area of judicial reform and the fight against organised crime and corruption. The Commission
established a Cooperation and Verification Mechanism (CVM) to monitor improvements in these areas
after accession. Under this mechanism, Bulgaria was requested to report regularly on the progress made
in addressing specific benchmarks in the field of judicial reform and the fight against organised crime and
corruption. For judicial reform three benchmarks were included in the CVM.496
The monitoring method of the CVM consisted of periodic reports sent from the Bulgarian government to
the Commission, giving details about the measures taken to respect the EU acquis and reforms
implemented. Based on these reports and on other available sources of information, a regular evaluation
of Bulgarias progress was carried out by the Commission.497 The results of the CVM mechanism were
presented by the Commission in six-monthly Progress Reports that were accompanied by the Technical
Reports, providing the basis for the analysis. At the end of each Progress Report, specific
recommendations were given to bring forward the progress in achieving the benchmarks set.498
In drawing up these reports, the Commission relied on information received from the Commissions
representation, the Member States diplomatic missions, specialised international organisations, EU
agencies and NGOs. Furthermore, the Commission was sending country experts from other Member
States to on-field visits and missions in Bulgaria.499 Where available, the Commission used the standards
as defined by international organisations, like the CoE and the UN agencies, as points of reference for the
progress made by Bulgaria. Also, the situation of other Member States was taken into account for a
comparison with the situation in Bulgaria.500 In 2012, the Commission requested a Flash Eurobarometer
493
COM (98) 707 final, 5 and later European Commission Regular Reports on Bulgaria.
European Commission, Opinion of 22 February 2005 on the applications for accession to the European Union by
the Republic of Bulgaria and Romania COM (2005) 55 final/2, para. 9.
495
European Commission, Monitoring Report on the State of Preparedness for EU Membership of Bulgaria and
Romania (Communication) COM (2006) 549 final, 9-10.
496
European Commission, Establishing a Mechanism for Cooperation and Verification of Progress in Bulgaria to
Address Specific Benchmarks in the Areas of Judicial Reform and the Fight Against Corruption and Organised Crime
(Decision) C (2006) 6550 final, Annex.
497
498
cf European Commission, Progress in Bulgaria under the Cooperation and Verification Mechanism (Report) COM
(2012) 411 final and Commission Progress in Bulgaria under the Cooperation and Verification Mechanism (Report)
COM (2014) 36 final.
499
European Commission, Bulgaria: Technical Report (Staff Working Document) SWD (2012) 232 final, 4 and
European Commission, Bulgaria: Technical Report (Staff Working Document) SWD (2014) 36 final, 3.
500
494
145
FRAME
survey to be conducted under the CVM that addressed public attitudes towards the state of the judicial
system and the perception of corruption in Bulgaria and Romania.501
3.
The indicators applied to measure the rule of law in Bulgaria
The judicial system was considered by the Commission as one of the main challenges for realising and
upholding the rule of law in the enlargement countries. As there was no acquis in terms of substantive
provisions for this area of the rule of law, the Commission assessed the progress of the enlargement
countries against the standards set by international organisations, like the CoE and the UN, the case law
of international courts and tribunals, and also considered the practice of the Member States.502 However,
in the field of the judiciary, the Western European legal systems provide for a plurality of models. Due to
the lack of such a common European standard, the Commission relied on a loosely-structured set of
elements for the assessment.503 In its Guide to the Main Administrative Structures Required for
Implementing the Acquis, the Commission identified the guarantee of sufficient human resources and
qualified staff, adequate and modern equipment, acceleration of court proceedings, reduction of the
number of pending cases so as to avoid unreasonable delays, measures to ensure the enforcement of
judgements, and procedures to ensure ethical conduct by the judiciary and the effective access to
justice504 as requirements for the establishment of an independent and efficient judicial system.
In order to measure the progress in the field of the judicial system, the Commission has used
predominately structural indicators, reflecting decisions taken, legislation adopted and international
conventions ratified. Process and outcome indicators were used to assess the implementation of
legislation and its application in practice. The measures were mostly qualitative in nature, while
quantitative indicators were used in several areas to underpin the qualitative assessments. With the CVM,
benchmarks were introduced as a new tool to measure the progress in six specific areas.
a)
The assessment of the independence and accountability of the judiciary
Strengthening judicial independence was included among the short-term and medium-term priorities of
the Bulgarian Accession Partnerships. The assessment of the independence of judiciary was divided into
two main components, namely the institutional independence and the personal independence of the
individual judges. Judicial self-governance was viewed as one basic element to ensure institutional
independence of the courts. The Commission promoted the establishment of functioning self-governance
organs for the judiciary, which should at least have the competences to recommend judges for
501
TNS Political & Social, Flash Eurobarometer 351: The Cooperation and Verification Mechanism for Bulgaria and
Romania (Survey Conducted on the Request of the European Commission, July 2012).
502
see Kochenov (n 479) 249 et seqq.
503
Daniel Smilov, EU Enlargement and the Constitutional Principle of Judicial Independence in: Sadurski W,
Czarnota A and Krygier M (eds), Spreading Democracy and the Rule of Law? (Springer 2006) 314.
504
European Commission, Guide to the Main Administrative Structures Required for Implementing the Acquis (Mai
2005)
<http://ec.europa.eu/enlargement/pdf/enlargement_process/accession_process/how_does_a_country_join_
the_eu/negotiations_croatia_turkey/adminstructures_version_may05_35_ch_public_en.pdf> accessed 25 June
2014, 71.
146
FRAME
appointment, to supervise the functioning of the judiciary as well as to overlook the general
administration and the budget of the courts.505 Therefore, the Commission welcomed the reforms of the
structures of the judicial system as progress in the Regular Reports.506
Another factor to increase the institutional independence of the Bulgarian judiciary, considered as crucial
by the Commission, was to ensure budgetary independence. In the Regular Reports, a clear connection
was drawn between the self-governance of the judiciary and the ability to control its own budget. This
included the requirement that budgeting decisions on the financing of the judiciary had to be respected
by other branches of power and that the usual practice of considerable budget cuts effectuated by the
Bulgarian parliament had to be abandoned.507 The efforts of Bulgaria in this respect were consistently
monitored throughout the Regular Reports. Furthermore, the Commission tried to quantify the progress
in the institutional independence in terms of the increase of the budget for the judiciary and by comparing
it to the average rate in the EU-15.508
The independence and impartiality of individual judges was measured by the Commission based on
several aspects, including the procedures for appointment of judges and prosecutors, probation terms,
the remuneration and allocation of cases. The standards applied by the Commission in terms of individual
independence of judges were laid down in the recommendations issued by the CoE and by the UN,509
however, no direct reference was made to these standards in the Regular Reports. The Commission
repeatedly requested Bulgaria to assure the full independence of magistrates and judges and to apply
transparent criteria and competitions for the recruitment and promotion of judges. The progress with
regard to these indicators was quite consistently monitored throughout the Regular Reports.510 Also the
use of uniform criteria for the competitive selection of judges or to monitor the performance before
granting tenure and promotion were assessed by the Commission.511 In the 2003 and 2004 Regular
Reports, the Commission reported on the improvement of working conditions of judges, prosecutors,
investigators and judicial staff.512
Concerning the impartiality of judges, the Commission welcomed the increase of magistrates salaries and
also the system in place to ensure random allocation of cases was continuously assessed by the Regular
Reports. The analysis of the Commission did not only focus on the legal framework adopted, but also on
the improvement of the practical situation. In terms of accountability, the Commission assessed the
505
Kochenov (n 479) 257-59; European Commission, 1999 Regular Report on Bulgarias Progress Towards Accession
COM (99) 501 final, 12 and 59; and European Commission, 2001 Regular Report on Bulgarias Progress Towards
Accession SEC (2001) 1744, 18.
506
European Commission, 2002 Regular Report on Bulgarias Progress Towards Accession SEC (2002) 1400, 23 and
European Commission, 2003 Regular Report on Bulgarias Progress Towards Accession SEC (2003) 1210 final, 17.
507
Kochenov (n 479) 275-76.
508
SEC (2002) 1400, 24 and SEC (2003) 1210 final, 17.
509
cf CoE Committee of Ministers, Recommendation on the Independence, Efficiency and Role of Judges R (94) 12;
CoE, European Charter on the Statute for Judges DAJ/DOC (98) 23 and UN, Basic Principles of the Independence of
the Judiciary (1985).
510
COM (2000) 701 final, 17; SEC (2001) 1744, 18 and SEC (2002) 1400, 25.
511
SEC (2002) 1400, 25.
512
SEC (2003) 1210 final, 19 and European Commission, 2004 Regular Report on Bulgarias Progress Towards
Accession SEC (2004) 1199, 18.
147
FRAME
provisions regulating the immunity of judicial bodies as well as the systems for monitoring in place. Several
measures adopted in 2002 were aimed to establish a system of accountability of courts, prosecution
offices and investigation services. Anti-corruption measures for the judiciary included mandatory property
and income declarations. Also the adoption of a Code of Ethics for magistrates and administrative staff of
the judiciary was regarded as a positive contribution to establish a system of accountability.513
After Bulgarias accession to the EU, the assessment of independence and accountability of the judicial
system was continued under the CVM through several benchmarks.514 For each benchmark a list of
qualitative indicators mostly focusing on structural reforms was designed to measure progress.515 When
assessing the constitutional amendments and reforms adopted to strengthen the independence,
impartiality and accountability of the judiciary, as addressed by benchmark 1 and 3 of the CVM, the
Commission focused on the legal framework in place to restrict the immunity of magistrates, the election
process for members of the Supreme Judicial Council and the introduction of provisions on the
inspectorate. The number of irregularities reported by the inspectorate and a track record of disciplinary
proceedings and sanctions were applied as outcome indicators. Furthermore, the system in place to
guarantee the application of objective criteria for the appointment and promotion of judges and the
random allocation of cases in courts were continuously monitored under the CVM.516 Through the Flash
Eurobarometer survey conducted under the CVM in 2012, it was shown that there is a strong common
interest in rule of law issues by the Bulgarian population. The data gathered through this survey allowed
to measure the perceived level of improvements of the judicial system, the fight against corruption and
organised crime on the ground, which mirrors the public confidence in the judicial system.517
Figure 2 gives an overview of structural, process and outcome indicators used in the Regular Reports and
under the CVM to measure the independence and accountability of the judiciary.
513
SEC (2002) 1400, 25.

see C (2006) 6550 final, Annex.
515
European Commission, Report on Bulgarias Progress on Accompanying Measures following Accession COM
(2007) 337 final.
516
European Commission, Bulgaria: Technical Report (Staff Working Document) SWD (2012) 232 final, 4-19 and
European Commission, Bulgaria: Technical Report (Staff Working Document) SWD (2014) 36 final, 3-18.
517
COM (2012) 411 final, 2-3.
514
148
FRAME

Figure 2: Indicators to measure the independence and accountability of the judiciary
Independence and Accountability of the Judiciary

Structural indicators
Process indicators
Outcome Indicators
Legal
framework
in
place
guaranteeing self-administration
and organisation of the judiciary
Legal
framework
in
place
guaranteeing and restricting the
immunity of magistrates
Provisions in place for the
recruitment of judicial staff and the
appointment of judges
Provisions in place for assigning
cases to individual judges
Codes of Ethics in place
Provisions in place for inspections
and performance monitoring of
judicial institutions
Budget available for the judiciary

Salaries of magistrates and judges
Mandatory property and income
declarations
Objective and transparent criteria
established for the recruitment of
judicial staff and for the appointment
of judges
Application of a system for random
allocation of cases to individual judges
Enforcement of the Code of Ethics
Number of inspections carried out and
monitoring reports prepared on the
judiciary
Amendments to the budget

proposal of the judiciary by
executive bodies
Number
of
disciplinary
proceedings and sanctions
against judicial staff
Number of appointments
based on transparent and
objective criteria
Number
of
irregularities
reported
through
the
inspections
Public confidence in the
judicial system
b)
The assessment of access to justice
The Accession Partnership contained several demands with regard to judiciary reform and the
Commission demonstrated quite clearly the importance of the restructuring and improvement of the
efficiency of Bulgarias judiciary.518 Access to justice was measured by the functioning of the judiciary in
terms of the structure of the judicial system, the qualification and training of judges, human and material
resources, work load in courts and length of court proceedings. With regard to the structure of a threetier judicial system, the Commission relied on the recommendation of the CoE and the rights granted by
the ECHR.519
With regard to the efficiency of the judiciary, the Commission concentrated its assessments on issues, like
the reduction of backlogs and the length of judicial proceedings and pre-trial detention as well as the
improvement of quality of investigation. Emphasis was also put on the improvement of the physical
infrastructure and human resources of the judiciary. Progress was measured in terms of new court
buildings opened and the computerisation of the courts and equipment in terms of a unified information
system. Furthermore, the Commission assessed the number of magistrates and the new administrative
positions created to relieve the work of judges and to accelerate court proceedings. 520 The training of
judges, prosecutors and administrative staff was an issue consistently monitored in the Regular Reports
and was addressed as a priority in the Accession Partnerships, with the goal to ensure the proper
functioning of the judicial system. The Commissions reports were very detailed in assessing the
availability of training programmes for new and existing staff within the judiciary and funding of the
518
Kochenov (n 479) 249-51.

cf CoE Committee of Ministers, Recommendation Concerning the Introduction and Improvement of the
Functioning of Appeal Systems and Procedures in Civil and Commercial Cases R (95) 5 and Article 2 of the Protocol
No. 7 to the ECHR concerning the right of appeal in criminal matters.
520
SEC (2002) 1400, 23-25 and SEC (2003) 1210 final, 19.
519
149
FRAME
training institution.521 The Commission particularly recognised the special trainings in EU law provided to
the judicial staff as an indicator to increase the professionalism and competence of the judiciary.522
Also, the complexity of legal procedures and the availability of legal aids were considered as contributing
factors for access to justice. While the legal framework for legal aid was found to be adequate, the
information gathered through surveys indicated that a large part of defendants did not actually have legal
representation at all stages of judicial proceedings.523 This was seen as an indicator for insufficient legal
aid and, therefore, had to be addressed in the priorities of the Accession Partnership of 2003.524
The assessment of access to justice and efficiency of the judiciary was continued under the CVM after
Bulgarias accession to the EU. As requested under benchmark 2 and 3, Bulgaria had to ensure a more
transparent and efficient judicial process and to enhance professionalism, accountability and efficiency of
the judiciary. The indicators used to measure the progress in meeting these benchmarks addressed the
procedural framework in place and the institutional set-up to speed up civil and criminal court
proceedings and to ensure effective investigation in criminal matters. The consistency of human resource
planning and the strategies to reduce the workload of judges were analysed by the Commission through
process and outcome indicators. Also the quality of the trainings provided for judicial staff was assessed
with regard to the embedment in the overall judicial structure.525 The perceptions of judicial shortcomings
at the level of the Bulgarian population could be measured based on the Eurobarometer survey conducted
in 2012 and was assessed by the Commission as an outcome indicator for the judicial reforms in the CVM
Progress Report of July 2012.526
521
SEC (2003) 1210 final, 19.

SEC (2004) 1199, 18.
523
SEC (2002) 1400, 25 and 30.
524
Council Decision (EC) 2003/396 of 19 May 2003 on the Principles, Priorities, Intermediate Objectives and
Conditions Contained in the Accession Partnership with Bulgaria [2003] OJ L145/1, Annex.
525
SWD (2012) 232 final, 8-19 and SWD (2014) 36 final, 4-18.
526
COM (2012) 411 final, 2-3.
522
150
FRAME
Figure 3 gives an overview of the structural, process and outcome indicators that were used in the Regular
Reports and under the CVM to measure access to justice and efficiency of the judiciary in Bulgaria.
151
FRAME

Figure 3: Indicators to measure access to justice
Access to Justice
Process indicators
Outcome Indicators
Legal framework in place

guaranteeing
a
three-tier
jurisdiction
facilitating quicker proceedings
Legal provisions in place to
provide human resources to
courts
ensuring the qualification and
training of judicial staff
Legal framework in place to
guarantee legal aid

Speed of execution of rulings
Quality of investigation
Human resources available in courts
Material conditions in courts (court
buildings, courts equipped with
computer
systems,
unified
information systems in place etc.)
Availability of trainings for the
judicial staff
Budget available to provide free legal
aid
Length of judicial proceedings

Length of pre-trial detention
Backlog of cases
Transparent and efficient handling
of cases
Number of cases sent back to
investigation
Number of trainings attended by
judicial staff
Number of defendants without
legal representation in judicial
proceedings
Perceptions
of
judicial
shortcomings
Judicial reform in terms of the independence and accountability of the judiciary and access to justice has
proven to be one of the most difficult issues in Bulgarias accession process.527 The Commission
continuously addressed these issues in the Regular Reports. Over the years, the scope of the reports was
more and more broadened and the issues selected for monitoring were described in more detail.
However, there was no clear list of indicators that would have allowed for consistent monitoring of these
areas. Only rarely quantifiable measures were used to document progress or setbacks. The assessments
conducted to measure the progress in addressing the benchmarks under the CVM did follow a
standardised set of mostly structural and process indicators. The reports also made reference to the
results of previous assessments. This ensured the comparability of the progress over time. In this way, the
methodology applied to conduct the assessments under the CVM was more transparent and consistent
than the methodology applied in the Regular Reports that were issued before accession.
B.
The case of Croatia
The policy of conditionality, introduced in the fifth enlargement round, was also applied and gained even
more significance in the enlargement policy towards the Western Balkan countries. In the accession
process with Croatia, the EU pursued a broader agenda of conditionality reflecting also the legacy of ethnic
conflicts in the region.
1.
The framework for negotiations
The Luxembourg General Affairs Council in April 1997 adopted conclusions on the application of
conditionality, with a view to developing a coherent EU strategy for its relations with the countries in the
region of Former Yugoslavia.528 The EU strategy was intended to serve as an incentive and not as an
obstacle to fulfil certain general conditions that were applied to all countries in South-Eastern Europe. The
527
Daniel Smilov, Bulgaria: The Discontents and Frustrations of a Newly Consolidated Democracy in Morlino L and
Sadurski W (eds), Democratization and the European Union: Comparing Central and Eastern European PostCommunist Countries (Routledge 2010) 115.
528
Council of the European Union, Conclusions from the General Affairs Council Meeting (Luxembourg, April 1997)
Annex III.
152
FRAME
conditionality defined by the Council included the requirements of co-operation with the International
Criminal Tribunal for the former Yugoslavia (ICTY) and regional co-operation. Furthermore, specific
conditions were set as obligations for certain countries only. The Commission guaranteed an operational
cooperation between the EU and the Western Balkans through the Stabilisation and Association Process
(SAP) for Countries of South-Eastern Europe to make further progress on their way to EU membership.529
The SAP was aimed to improve the situation in the region by assisting the creation of effective and
accountable law enforcement institutions and improving the border control and migration management.
These reforms were seen as a pre-condition to prepare for membership. A core element of the SAP were
the Stabilization and Association Agreements (SAAs), which provided the formal contractual relationship
between the EU and the potential candidate countries to support the adoption of the EU standards and
necessary rules.530 The Feira European Council in June 2000 reiterated its support for the reforms in the
Western Balkans and confirmed the objective of the fullest possible integration of the countries
participating in the SAP by acknowledging them as potential candidates for EU membership.531 To
support the Western Balkan countries in the implementation of the relevant reforms in the SAP, the EU
provided assistance through the Community Assistance for Reconstruction, Development and
Stabilisation (CARDS) programme in the period 2000 to 2006.532
Taking account of the experiences of the fifth enlargement round, the Brussels European Council in
2004533 requested that a general framework for negotiation had to be agreed in advance of the
negotiations. Within the Enlargement Strategy 2004,534 the Commission proposed the negotiating
framework for Croatia and in this context introduced benchmarks as a new tool to monitor the progress
in the accession process. The purpose of benchmarks was to improve the quality of negotiations, as they
are providing quantifiable measures referred to the legislative alignment with the acquis or to a track
record in the implementation. Upon proposal from the Commission, benchmarks were set to be met by
the candidate country to open negotiations as well as to provisionally close the chapters. While opening
benchmarks concerned key preparatory steps for the future alignment with the acquis requirements,
closing benchmarks reflected the need for legislative measures, administrative or judicial reforms to
implement the acquis. The benchmarks were regarded as incentives for the candidate countries to
stimulate the necessary reforms at an early stage of the accession negotiations. The provisions of the
negotiating framework also provided for the suspension of accession negotiation in case of serious and
529
European Commission, On the Stabilisation and Association Process for Countries of South-Eastern Europe
(Communication) COM (1999) 235 final.
530
Florian Trauner, From Membership Conditionality to Policy Conditionality: EU External Governance in South
Eastern Europe (2009) 16/5 Journal of European Public Policy 774, 779.
531
European Council, Conclusions of the Presidency (Santa Maria da Feira, June 2000) para. 67.
532
Council Regulation (EC) 2666/2000 of 5 December 2000 on Assistance for Albania, Bosnia and Herzegovina,
Croatia, the Federal Republic of Yugoslavia and the Former Yugoslav Republic of Macedonia repealing Regulation
(EC) No 1628/96 and amending Regulations (EEC) No 3906/89 and (EEC) No 1360/90 and Decisions 97/256/EC and
1999/311/EC [2000] OJ L306/1, Art. 2.
533
European Council Presidency Conclusions (Brussels, June 2004) para. 34.
534
European Commission, Strategy Paper of the European Commission on Progress in the Enlargement Process
(Communication) COM (2004) 657 final, 7.
153
FRAME
persistent breach of the principles of liberty, democracy, respect for human rights and fundamental
freedoms and the rule of law. 535
The European Partnerships and Accession Partnerships remained a central element of the accession
strategy. The Partnerships were setting out the priorities identified for each country in the Commissions
Progress Reports and provided the framework for the EU assistance.536 As of 2007, the CARDS programme
and the other programmes providing assistance to candidate countries were replaced by one single
instrument, namely the IPA.537 Assistance granted under the IPA was linked to the progress and needs
identified through the Commissions evaluations and was subject to a suspension clause for the case of
failure to make sufficient progress in the accession process.538
2.
The monitoring mechanism applied to Croatia
Croatia signed the SAA with the EU in 2001 and started the preparations for membership. The institutional
framework of the SAA and the negotiating framework adopted by the Council provided for the continuous
monitoring of Croatias progress in fulfilling the obligations. The Commission made use of the monitoring
system that had been developed in the previous enlargement rounds and added new monitoring tools
like the Screening Reports or Interim Reports to measure the level of realisation of the rule of law in
Croatia. An overview of the different monitoring tools used by the Commission during the accession
process of Croatia is presented in Figure 4. The monitoring tools were used throughout the different
stages of the accession process to assess the progress towards fulfilling the obligations and implementing
the political priorities of the SAA, the European Partnership and the Accession Partnerships between the
EU and Croatia.
Figure 4: The monitoring mechanism applied to Croatia
a)
From 2002 on, the Commission was evaluating the SAP for South-Eastern Europe in annual reports.539
Croatia presented its application for membership in 2003. Already in 2004, the Commission recommended
535
649, 6.
536
ibid, 7.
537
Council Regulation (EC) 1085/2006 of 17 July 2006 Establishing an Instrument for Pre-Accession Assistance (IPA)
[2006] OJ L210/82.
538
European Union, Instrument for Pre-Accession Assistance (IPA) (20 February 2012)
<http://europa.eu/legislation_summaries/agriculture/enlargement/e50020_en.htm> accessed 3 July 2014.
539
European Commission, The Stabilisation and Association Process for South East Europe: First Annual Report
COM (2002) 163 final, 19 and European Commission, The Stabilisation and Association Process for South East
Europe: Second Annual Report COM (2003) 139 final, 30.
154
FRAME
to open accession negotiations and submitted a favourable Opinion on Croatias Application for
Membership.540 The methodology adopted by the Commission in drawing up this Opinion was based on
the experience from earlier enlargement rounds and the structure of the Opinion followed the
Copenhagen criteria and the conditionality established by the Luxembourg General Affairs Council in April
1997. The judiciary mainly was assessed as part of the political criteria. Other rule of law elements were
covered by the acquis criteria in chapter 24, dealing with Co-operation in the field of justice and home
affairs.541
Croatia was recognised as a candidate country at the Brussels European Council in June 2004 with the
prospective of launching the accession negotiations in March 2005. Also the SAA finally entered into force
on 1 February 2005, after an enlargement protocol had been signed in December 2004.542
b)
The screening and the benchmarking process
At the same time with the formal opening of the accession negotiations, the Luxembourg General Affairs
Council in October 2005 adopted the negotiating framework for Croatia, setting out the principles
governing the negotiations and providing a preliminary indicative list of negotiation chapters. The acquis
was organised in a list of 33 negotiation chapters plus two chapters concerning institutions and other
issues. The acquis in the field of justice, freedom and security was negotiated in chapter 24 that covered
issues like border control, visas, external migration, asylum, police cooperation, the fight against
organised crime and judicial cooperation in criminal and civil matters. The negotiating framework
provided for a new chapter 23, dealing with judiciary and fundamental rights, under which the political
issues of the rule of law were addressed. As a result of the adoption of the new negotiating framework,
the rule of law was no longer regarded solely as part of the political criteria but was also conceived as an
integral part of the acquis, which should allow the Commission to keep the crucial areas of the rule of law,
notably judiciary and fundamental rights, under close scrutiny.543
Immediately after the formal opening of accession negotiations in 2005, the Commission introduced the
process of screening to obtain preliminary indications of the accession criteria for each negotiation
chapter. The process of screening was completed in 2006. Separate Screening Reports were issued for
each negotiation chapter. After a chapter had been screened, the Council decided on whether to open a
chapter for negotiations or on the benchmarks to be met by the candidate country before opening it.544
540
European Commission, Opinion on Croatia's Application for Membership of the European Union
(Communication) COM (2004) 257 final, 119.
541
ibid, 105-109.
542
European Union, Press Release: The Stabilisation and Association Agreement with Croatia enters into force today
(1 February 2005) <http://europa.eu/rapid/press-release_IP-05-122_en.htm> accessed 9 August 2014.
543
Christophe Hillion, Enlarging the European Union and Deepening its Fundamental Rights Protection (2013) 11
European Policy Analysis <http://www.sieps.se/sites/default/files/2013_11epa.pdf> accessed 2 July 2014, 4.
544
COM (2006) 649, 6.
155
FRAME
After Croatia had made sufficient progress in fulfilling the opening benchmarks, the EUs common position
for the opening of the negotiating chapter was adopted by the Council on 25 June 2010.545
Together with two other chapters, the chapter on judiciary and fundamental rights was among the last
three negotiating chapters. In the EU common position the closing benchmarks that had to be met to
provisionally close the chapter, were tabled. The Council set 10 closing benchmarks to be fulfilled by
Croatia in the field of the judiciary and fundamental rights. Several closing benchmarks of chapter 23 were
addressing the judicial system, very vaguely requesting Croatia to strengthen the independence,
accountability, impartiality and professionalism of the judiciary and to improve the efficiency of the
judiciary.546 In this way, the benchmarks were not reflecting specific standards or targets, but were rather
based on broad concepts that had to be further defined through specific indicators in the Commissions
reports. However, from the available sources it does not appear that the development of the closing
benchmarks was based on clear concepts, measurement methods, indicators, collection methods or time
frames.547
c)
The Progress Reports on Croatia
A mechanism for the implementation, management and monitoring of all areas of relations between the
EU and Croatia was established through the institutional framework of the SAA.548 The implementation of
the obligations under the SAA was monitored in the annual Progress Reports together with the criteria
for membership as defined by the Copenhagen European Council. The Commission presented annual
Progress Reports on Croatia from 2005 to 2011. In drawing up the Progress Reports, the Commission
followed the methodology applied in the previous enlargement rounds. Rule of law was covered under
the political criteria for membership and in the chapters 23 and 24 of the acquis. In this way, the results
of the assessment of the judicial system were presented in two sections of the report and cross-reference
was made between these two sections. The opening and closing benchmarks that were introduced in the
accession negotiations for the specific negotiation chapters would have provided an instrument to
measure progress against. However, the Commission did not make reference to the closing benchmarks
in the annual Progress Reports. This made the overall process of progress assessment quite
untransparent.549 In 2011, the Commission presented an Interim Report on the negotiation chapter 23 to
describe the progress made in meeting the ten closing benchmarks.550
Based on the main priorities identified in the Progress Reports, the Commission proposed Accession
Partnerships for Croatia. Two Accession Partnerships with Croatia were adopted by the Council in 2006
545
European Parliament, Progress in EU-Croatia accession negotiations at a glance (5 November 2014)

<http://www.europarl.europa.eu/meetdocs/2009_2014/documents/d-hr/dv/1129_07_/1129_07_en.pdf>
accessed 2 July 2014.
546
European Commission, Interim Report on Reforms in Croatia in the Field of Judiciary and Fundamental Rights
(Negotiation Chapter 23) COM (2011) 110, 2-4.
547
Wolfgang Benedek and others Mainstreaming Human and Minority Rights in the EU Enlargement with the
Western Balkans (Study, European Parliament 2012) 69.
548
European Commission, Croatia 2005 Progress Report SEC (2005) 1424, 4.
549
Benedek and others (n 547) 68.
550
COM (2011) 110, 2-4.
156
FRAME
and 2008.551 The financial assistance under the PHARE and CARDS programs was well focused on the
Croatian accession priorities as defined in the partnerships and addressed the specific negotiation
requirements in terms of opening and closing benchmarks.552 As from 2007, Croatia has benefited from
financial assistance under the IPA that was intended to support the preparations for accession. The
Accession Partnerships provided a framework to determine the different areas to which funds were to be
allocated and at the same time served as a checklist against which to measure progress in the annual
reports.553
In preparing the Progress Reports, the Commission utilised numerous sources of information. Croatia was
given the possibility to provide information on the progress made since the previous report and also the
information provided by Croatia within the framework of the SAA was considered by the Commission.
Various peer reviews and technical consultations have taken place to assess Croatias administrative
capacity. Furthermore, the deliberations and reports from the EU institutions, various international
organisations and NGOs were considered.554
d)
Continued monitoring
The accession negotiations were closed at the thirteenth meeting of the Accession Conference with
Croatia on 30 June 2011 by closing the last four negotiating chapters.555 The Council pointed out the
considerable progress made with regard to fulfilling the commitments under chapter 23, however, the
importance of the continuation to develop a track record of implementation and to further demonstrate
concrete results was emphasised. Continuous effort was necessary for Croatia to meet the closing
benchmarks of chapter 23, covering different areas related to the judiciary and fundamental rights. The
progress in the respective fields was assessed by the Commission and presented in Monitoring Reports
every six months up until the time of accession.
The assessment of the Commission was based on information gathered through own analysis, the input
provided by Croatia and by Member States as well as information from international and civil society
organisations. Furthermore, the Commission used specific Monitoring Tables to update its findings for the
reports. These Monitoring Tables were working tools that were used to following up the details on all of
Croatias commitments from the negotiations.556 Also in the last Monitoring Report presented in March
2013, the Commission identified several areas with regard to the judiciary and fundamental rights, for
551
Council Decision (EC) 2006/145 of 20 February 2006 on the Principles, Priorities and Conditions contained in the
Accession Partnership with Croatia and repealing Decision 2004/648/EC [2006] OJ L55/30 and Council Decision (EC)
2008/119 of 12 February 2008 on the Principles, Priorities and Conditions contained in the Accession Partnership
with Croatia and repealing Decision 2006/145/EC [2008] OJ L42/51.
552
European Commission, Country Sections & Additional Information (Staff Working Paper) SEC (2010) 1604, 22.
553
Benoit-Rohmer and others (n 425) 86.
554
SEC (2005) 1424, 4 and Croatia Progress Reports of the following years.
555
Council of the European Union, Thirteenth Meeting of the Accession Conference with Croatia at Ministerial level
(Press Release, June 2011).
556
European Commission, Comprehensive Monitoring Report on Croatia (Staff Working Document) SWD (2012)
338 final, 5.
157
FRAME
which further efforts were necessary in order to fulfil the commitments.557 However, unlike in the case of
Bulgaria, the accession of Croatia was not made subject to the monitoring of the CVM, as benchmarks
were to be reached before accession.558
3.
The indicators applied to measure the rule of law in Croatia
Even though the Commission made use of new monitoring tools, like the Interim Report, to measure the
rule of law in the accession process of Croatia, the indicators applied closely resembled those of previous
enlargement rounds. With the introduction of benchmarks in the accession negotiations, the Commission
made use of a new tool to assess the progress made towards achieving a specific standard that had to be
fulfilled prior to accession. To define the standards against which the independence, impartiality and
efficiency of the judiciary were measured, the Commission made reference to the definition of the content
of these notions in the jurisprudence of the European Court of Human Rights, as this is an accepted
reference for the EU acquis under Art. 2. In particular this definition included the following criteria:
Courts must be established by law; there shall be no discrimination in the appointment
procedures of judges; the judiciary must not be influenced in its decision-making by either the
executive or the legislature; judges must act impartially and be seen to do so; their conditions of
tenure must be adequately ensured by law; the grounds for disciplinary action or removal from
the post must be limited and laid down in the law.559
Furthermore, the Commission used the recommendation from international organisations as guidelines
to identify a common European standard in the field of independence and accountability of the judiciary,
efficiency of the judicial system and access to justice.560 The progress made by Croatia in fulfilling the
membership conditionality was assessed by the Commission against these standards in the annual
Progress Reports. The assessments of the judiciary provided in chapter 23 of the reports was quite
consistently structured along the areas independence of the judiciary, impartiality, professionalism
and competence, efficiency of the judiciary and judicial reform strategy, which allowed to track
changes over time in these areas.561
a)
The cooperation in the field of justice and home affaires established under the SAA specifically focused
on the reinforcement of institutions and the consolidation of the rule of law. The independence of the
judiciary was one of the priority areas identified in the SAA.562 Also in the Accession Partnerships of 2006
557
European Commission, Monitoring Report on Croatia's Accession Preparations (Communication) COM (2013)
171 final, 4-10.
558
559
European Commission, Screening Report Croatia: Chapter 23 Judiciary and fundamental rights (2007), 2.
560
In the Screening Report of Croatia: Chapter 23 (n 559) the European Commission refers to the CoE Committee
of Ministers, Recommendation on the Independence, Efficiency and Role of Judges R (94) 12 and to the CoE
Conference of Prosecutors General of Europe, European Guidelines on Ethics and Conduct for Public Prosecutors
(the Budapest Guidelines) CPGE (2005) 05.
561
European Commission, Croatia 2006 Progress Report (Staff Working Document) SEC (2006) 1385, 49-51 and
Croatia Progress Reports of the following years.
562
Stabilisation and Association Agreement between the European Communities and their Member States, of the
one part, and the Republic of Croatia, of the other part [2005] OJ L26/3 (SAA Croatia), Art. 75.
158
FRAME
and 2008, judicial reform and the establishment of an open, fair and transparent system of recruitment,
evaluation, promotion and disciplinary measures in the judiciary were identified as priorities to improve
the independence, impartiality and accountability of the judiciary.563
In its assessment of the independence and accountability of the judiciary, the Commission has used very
much the same indicators that had already been applied to Bulgaria. The institutional independence of
the judiciary was measured through structural indicators evaluating the legal framework in place for
judicial self-administration and the budgetary independence of the judiciary. In terms of accountability,
the regulations on the appointment and dismissal of judges and the responsibility for disciplinary
proceedings were examined. These parameters were measured through process indicators, assessing the
structure and functioning of the State Judicial Council.564 As of 2006, the Commission measured the
number of disciplinary proceedings conducted by the State Judicial Council and the sanctions applied
against judges, like reprimands, fines and dismissals or acquittals.565 However, in 2009, the Commission
criticised the lack of transparency in disciplinary proceedings and the absence of legal means for the
Ministry of Justice to verify the declarations of assets by judges and prosecutors.566
The introduction of a Code of Ethics was assessed as a structural indicator to improve the accountability
and impartiality of the judiciary.567 The impartiality and independence of the individual judges was
measured through the regulations in place for recruitment of judicial staff and the appointment to
permanent positions, the regulations on the immunity of judges and the salaries of magistrates and
judges. Shortcomings in the application of transparent criteria for the appointment of judges and
prosecutors were criticised throughout numerous reports by the Commission.568 Only in 2010, the
Commission recognised the improvements made in this regard through the revision of the selection
procedure for new judges, the application of transparent and objective selection criteria and the abolition
of probation periods for judges. However, the Commission requested Croatia to establish a track record
of the appointments based on the revised procedure and the new selection criteria in practice. Also the
improvements in terms of judicial inspections were recognised in the Progress Report.569
The impartiality was a specific issue of the judicial system in Croatia, as there was a persisting ethnic bias
against Serbs in local courts. In order to ensure a random allocation of cases, the implementation of an
integrated case management system in all courts was requested and continuously assessed by the
Commission.570
Benchmarks imposed for the opening and closing of specific negotiation chapters were a new instrument
against which the progress of the implemented reforms could be measured. In the Monitoring Reports,
563
Decision 2006/145, Annex; Decision 2008/119, Annex.

SEC (2005) 1424, 83.
565
SEC (2006) 1385, 49, European Commission, Croatia 2007 Progress Report (Staff Working Document) SEC (2007)
1431, 48 and European Commission, Croatia 2008 Progress Report (Staff Working Document) SEC (2008) 2694, 51.
566
European Commission, Croatia 2009 Progress Report (Staff Working Document) SEC (2009) 1333, 51.
567
SEC (2007) 1431, 48 and SEC (2008) 2694, 51.
568
SEC (2007) 1431, 48; SEC (2008) 2694, 51 and SEC (2009) 1333, 51.
569
European Commission, Croatia 2010 Progress Report (Staff Working Document) SEC (2010) 1326, 47.
570
SEC (2005) 1424, 16 and 83-84; SEC (2006) 1385, 49-50 and SEC (2010) 1326, 48.
564
159
FRAME
the Commission assessed the continued effort to meet these closing benchmarks by applying similar
indicators as in the annual Progress Reports. The legal framework guaranteeing judicial selfadministration, the systems for recruitment and appointment of judges and the limits to the immunity of
judges were evaluated. To monitor the progress in addressing the specific benchmarks, the Commission
relied very much on process and outcome indicators, focusing on the implementation of measures and
the establishment of solid track records.571 The Commission requested Croatia to establish a track record
of the application of uniform, transparent and objective criteria in recruiting and appointing judges and
prosecutors as well as of the conduct of disciplinary procedures against errant judicial officials.
Furthermore, the implementation of the system for asset declarations for judges and the application of
the code of conduct were monitored in the reports.572
Figure 5 gives an overview of structural, process and outcome indicators applied in the Progress Reports
and Monitoring Reports on Croatia to measure the independence and accountability of the judiciary.

Process indicators
Outcome Indicators
Legal
framework
in
place
guaranteeing self-administration
Legal
framework
in
place
Provisions in place for the
recruitment of judicial staff and for
the appointment of judges
Provisions in place for evaluating
the impartiality of the judiciary
Provisions in place for judicial
inspections

Salaries of magistrates and judges
established for the recruitment and
appointment
of
judges
and
prosecutors
Implementation of an integrated case
management system in courts
Enforcement of the Code of Ethics
Methodology adopted to evaluate
judges for the purposes of promotion
Mandatory declarations of assets
Legal means to verify the declarations
of assets by judges and prosecutors
Number of inspections carried out
Reduction of risk of
political interference in the
judiciary
objective criteria
Number of disciplinary
Number of irregularities
reported through the
inspections
b)
The improvement of the effectiveness of the judiciary and the training of the legal professions were
areas of particular focus under the SAA.573 The Accession Partnerships of 2006 and 2008 contained as
priorities for the judicial system the reduction of backlog of cases, the rationalisation of court organisation
and development of modern information technology systems, as well as the enhancement of
professionalism through training and the proper execution of court rulings.574 The progress in addressing
these priorities was assessed by the Commission in the annual Progress Reports in chapter 23. Similar to
the indicators applied in the accession process of Bulgaria, the efficiency of the judiciary was measured in
571
COM (2011) 110, 2.

COM (2011) 110, 3; SWD (2012) 338 final, 33 and COM (2013) 171 final, 4.
573
SAA Croatia, Art. 75.
574
Decision 2008/119, Annex and Decision 2006/145, Annex.
572
160
FRAME
terms of the total budget provided for the judicial system, the infrastructure and equipment of courts, the
number of judges and other judicial staff and the average duration of proceedings and backlog of
unresolved cases. The Commission provided quantitative measures for these indicators in the Progress
Reports and monitored the changes in the figures over time. Also the availability of Alternative Dispute
Resolution methods and the reduction of the complexity of legal procedures were assessed as structural
indicators to increase the efficiency of the judicial system.575 The training programmes provided for
judicial staff were assessed as a contribution to enhance professionalism and quality of the judiciary. Also
the budget allocation for the Judicial Academy was considered as a quantitative indicator in the
assessment of the Commission.576 The Commission especially emphasised the importance of initial
training and trainings covering matters of EU law in its reports.577 The access to justice in terms of the
legal framework in place to guarantee legal aid was not only assessed in chapter 23 of the Progress
Reports, but also under the aspect of Civil and Political Rights and in other chapters of the acquis.578
One of the closing benchmarks of chapter 23 specifically requested Croatia to improve the efficiency of
the judiciary. Under this benchmark, the reduction of the backlog of cases and of delay in court cases, the
introduction of new methods of enforcement as well as the improvement of human resources and of the
physical infrastructure and computerisation of courts were important parameters to measure against the
progress of reform implementation.579 The Commission continuously monitored the roll-out of the
integrated case management system and the rules governing the mobility and transfer of judges. Also the
performance of the Judicial Academy in terms of professional training programmes provided was assessed
in the biannual Monitoring Reports. However, the successful completion of the priority areas identified
under this closing benchmark could only be reported in the 2013 Monitoring Report.580
Figure 6 gives an overview of the structural, process and outcome indicators that were used in the
Progress Reports and Monitoring Reports to measure access to justice and efficiency of the judiciary in
Croatia.
575
SEC (2005) 1424, 84-86; SEC (2006) 1385, 50-51; SEC (2008) 2694, 52-53 and SEC (2010) 1326, 48.
SEC (2005) 1424, 84; SEC (2006) 1385, 50 and SEC (2007) 1431, 49.
577
SEC (2009) 1333, 51 and SEC (2010) 1326, 47.
578
SEC (2006) 1385, 54 and SEC (2007) 1431, 11 and 60.
579
COM (2011) 110, 3.
580
SWD (2012) 338 final, 33-34 and COM (2013) 171 final, 4-5.
576
161
FRAME

Access to Justice
Process indicators
Outcome Indicators

Availability of Alternative Dispute
Resolution methods
Legal provisions in place to
provide human resources to
courts and for the mobility of
judges
guarantee legal aid

Human resources available in courts
Material conditions in courts (physical
infrastructure, computerisation of
courts, integrated case management
system in place etc.)
Availability and quality of professional
trainings provided to the judicial staff
Availability of budget for the Judicial
Academy
Transparent and efficient

handling of cases
Backlog of cases
Length of investigation phase
in criminal procedures
Length
of
judicial
proceedings
Number
of
trainings
attended by judicial staff
Overall, the independence, accountability and efficiency of the judiciary and access to justice were
important issues tackled in the accession negotiations with Croatia. The prospect of EU membership was
a powerful incentive for Croatia to bring about the requested reforms under the SAA and the Accession
Partnerships.581 The Commission continuously monitored the progress in fulfilling the required standards
in its Progress Reports and Monitoring Reports. The transparency in measuring progress over time has
been increased by following a more structured approach and by consistently applying the same indicators.
However, the assessment of the progress towards fulfilling the closing benchmarks, that had been
introduced as a new monitoring tool by the Commission, lacked consistent measurement methods. The
underlying concepts and objectives were not well-defined and no clear reference to the benchmarks was
made in the Progress Reports.582 Only in the Monitoring Reports, the progress towards fulfilling the closing
benchmarks was evaluated in a more consistent and transparent manner by applying a set of
comprehensive indicators directly related to the benchmarks defined.
C.
The case of Montenegro
As one of the Western Balkan countries, Montenegros perspective for EU membership is based on the
decision of the Feira European Council in 2000 and on the Thessaloniki Agenda for the Western Balkans
that had been endorsed by the Thessaloniki European Council in 2003.583 Even before declaring its
independence on 3 June 2006, Montenegro maintained relations with the EU as part of the state union
Serbia and Montenegro. The European Partnership584 and the SAA585 with the new state Montenegro were
581
Trauner (n 530) 782-83.

583
European Council, Presidency Conclusions (Thessaloniki, June 2003).
584
Council Decision (EC) 2007/49 of 22 January 2007 on the Principles, Priorities and Conditions contained in the
European Partnership with Montenegro [2007] OJ L20/16.
585
Stabilisation and Association Agreement between the European Communities and their Member States of the
one part, and the Republic of Montenegro, of the other part, signed in Luxembourg, 15 October 2007 (SAA
Montenegro).
582
162
FRAME
signed in 2007, laying down the formal framework for relations and for Montenegros way towards EU
membership.
1.
The new approach to negotiations
To take account of the experience acquired from previous accession negotiations, the Commission
proposed a new Enlargement Strategy based on consolidation, conditionality and communication in its
Enlargement Strategy Paper of 2006. This strategy formed the basis for a renewed consensus on
enlargement that called to address issues related to the rule of law at an early stage of the accession
process.586 The renewed consensus on enlargement was confirmed by the Brussels European Council in
2006, where also the EUs commitment towards the Western Balkan states was renewed. At the same
time, the European Council reiterated that each country's progress towards the European Union depends
on its individual efforts to comply with the Copenhagen criteria and the conditionality of the Stabilisation
and Association Process.587 In the light of this, the Commission intensified the dialogue on the rule of law
with the candidate and potential candidate countries and extended the use of peer missions. The use of
benchmarking in the accession process served as an important catalyst for reforms and gave the clear
obligation, to establish a convincing and credible track record.588
To acknowledge the importance attached to the area of the rule of law and fundamental rights in the
accession process with the Western Balkan countries, the chapters 23 and 24 were put in the forefront of
the accession negotiations. This new approach should allow maximum time for establishing the necessary
legislation, institutions and a solid track record of implementation.589 With the Councils endorsement of
the Commissions proposed new approach, Montenegro was the first enlargement country, where both
chapters were to be tackled early in the negotiations.590 In the new approach to negotiations, the rule of
law and democratic governance are firmly anchored at the heart of the accession process.591 This new
approach has increased the credibility of the enlargement policy through fair conditionality, established
criteria and the principle of own merit.592
The new approach to negotiations was also reflected in the Commissions proposal for the new IPA II
instrument for the multi-annual financial framework 2014 to 2020. Stronger linkages with the priorities
identified in the Enlargement Strategy should guarantee an improved strategic focus of the pre-accession
financial assistance provided to enlargement countries. Through country and multi-country strategy
586
COM (2006) 649.

European Council Presidency Conclusions (Brussels, December 2006), para. 8.
588
European Commission, Enlargement Strategy and Main Challenges 2010 2011 (Communication) COM (2010)
600 final, 7.
589
666 final, 5.
590
Council of the European Union, Conclusions on Montenegro of the General Affairs Council Meeting
(Luxembourg, June 2012).
591
Council of the European Union Conclusions on Enlargement and Stabilisation and Association Process of the
General Affairs Council Meeting (Brussels, December 2012), 2 and European Commission, Enlargement Strategy
and Main Challenges 2012-2013 (Communication) COM (2012) 600 final, 2-4.
592
European Commission, Enlargement Strategy and Main Challenges 2014-15 (Communication) COM (2014) 700
final, 1.
587
163
FRAME
papers clear targets and a performance element were introduced.593 The IPA II Regulation594 also foresees
that progress towards achievement of these specific targets shall be monitored by using clear,
transparent, country-specific and measurable indicators.
2.
The monitoring mechanism applied to Montenegro
After the declaration of independence in 2006, Montenegro took part in the reporting arrangements
applying to potential candidate countries. The monitoring mechanism applied by the Commission
consisted of the tools that had been used in the previous enlargement rounds (as presented in Figure 7).
Figure 7: The monitoring mechanism applied to Montenegro
a)
The SAA and the Progress Reports on Montenegro
Even though contractual relations between the EU and Montenegro were only established in 2007, the
Commission presented the first progress report under the SAP on Montenegro already in November
2006.595 Progress Reports were issued on an annual basis from 2006 to 2009. In 2010 the Commission
presented its Opinion on the Application for Membership and from 2011 on, the annual Progress Reports
followed the structure of the Copenhagen criteria, assessing the ability to assume the acquis in 33
negotiating chapters. The conditionality established by the Luxembourg General Affairs Council in April
1997, namely the co-operation with the ICTY and regional co-operation, were included in the political
criteria for membership. The 2011, 2012 and 2013 Progress Reports provided a detailed analysis of the
judicial system under the political criteria for membership and in chapter 23 of the acquis criteria. To draw
up the annual reports, the Commission gathered information from different sources. The data provided
by the government of Montenegro, the EU Member States and the European Parliament as well as
information from various international organisations and NGOs contributed to the reports.596
b)
Montenegro presented its application for membership of the EU in 2008 and the Commission issued an
Opinion on Montenegros application in 2010.597 In preparing the Opinion, the Commission followed a
methodology similar to that used in previous Opinions and included the elements of the renewed
consensus on enlargement agreed in 2006. A detailed analysis of the criteria for membership was
593
COM (2012) 600 final, 19.

Regulation 231/2014.
595
European Commission, Montenegro 2006 Progress Report (Staff Working Document) SEC (2006) 1388, 4-5.
596
cf European Commission, Montenegro 2011 Progress Report (Staff Working Paper) SEC (2011) 1204 final;
European Commission, Montenegro 2012 Progress Report (Staff Working Document) SWD (2012) 331 final and
European Commission, Montenegro 2013 Progress Report (Staff Working Document) SWD (2013) 411 final.
597
European Commission, Opinion on Montenegros Application for Membership to the European Union
(Communication) COM (2010) 670.
594
164
FRAME
presented in a separate Analytical Report accompanying the Opinion.598 Based on the results of the
Analytical Report, the Commission recommended granting the status of candidate country to Montenegro
but at the same time identified seven key priorities to be addressed as a prerequisite to opening accession
negotiations. Strengthening the rule of law through judicial reform was one of these key priorities.599 In
this way, the Opinion reiterated the need for reforms that had already been identified under the SAA and
as a short-term priority in the European Partnership in 2007.600 Furthermore, chapters 23 and 24 were
identified as two of those chapters, where considerable and sustained efforts are necessary to align with
the EU acquis.601 The Commission supported Montenegro in addressing these issues by providing financial
and technical support through IPA. The funds provided through the IPA have increased the capacity in the
areas of the rule of law, justice and home affairs and have contributed to the opening of negotiations in
June 2012.602
c)
The screening and the opening of accession negotiations
With a view to opening the accession negotiations, the Brussels European Council in 2011603 requested
the Commission to examine in particular Montenegros progress in the area of the rule of law and
fundamental rights and to prepare a proposal for a framework for negotiations, incorporating the new
approach to negotiations. Following this request, the Commission presented a report on Montenegros
Progress in the Implementation of Reforms in May 2012. This report assessed the efforts made by
Montenegro to address the seven key priorities as set out in the Opinion on the Application for
Membership. In its assessment, the Commission confirmed that Montenegro had achieved the necessary
degree of compliance with the membership criteria and in particular the Copenhagen political criteria, to
start accession negotiations.604 Based on the Commissions report, the Luxembourg General Affairs
Council in June 2012605 adopted the general EU position and the negotiating framework to open accession
negotiations with Montenegro. The negotiations were formally opened on 29 June 2012. 606
Following the request of the Brussels European Council in 2011 and in line with the new approach, the
Commission initiated the screening of chapter 23 and chapter 24 in spring 2012, as these chapters were
to be tackled early in the accession process. This should allow Montenegro to develop the required solid
track record of reform implementation and ensure sustainable and lasting progress in the realisation of
the rule of law.607 The Screening Report of chapter 23608 and chapter 24609 presented the results from the
systematic examination of the acquis and also identified specific issues that needed to be addressed
598
European Commission, Analytical Report (Staff Working Document) SEC (2010) 1334, 4-5.
COM (2010) 670, 11.
600
Decision (EC) 2007/49, Annex.
601
COM (2010) 670, 9-10.
602
COM (2012) 600 final 19.
603
European Council, Conclusions (Brussels, December 2011) para. 12.
604
European Commission, Report on Montenegros Progress in the Implementation of Reforms COM (2012) 222
final, 12.
605
Council of the European Union (n 590).
606
European Council, Conclusions (Brussels, June 2012).
607
COM (2012) 600 final.
608
European Commission, Screening Report Montenegro: Chapter 23 Judiciary and Fundamental Rights (2012).
609
European Commission, Screening Report Montenegro: Chapter 24 Justice, Freedom and Security (2012).
599
165
FRAME
before accession negotiations on these chapters could be opened. In its recommendations, the
Commission requested the adoption of one or more Action Plan(s) that should guide the reform process
on the rule of law. The recommendations in the field of judicial reform specifically addressed the
shortcomings in terms of independence, impartiality, accountability as well as the competence and
efficiency of the judiciary.610 The analytical examination of all chapters of the acquis was completed in
May 2014.611
Based on the recommendations given in the Screening Report on the Chapter 23 and bearing in mind the
conclusions from the 2012 Montenegro Progress Report, the Government of Montenegro adopted an
Action Plan for Chapter 23 Judiciary and Fundamental Rights612 on 27 June 2013. This Action Plan outlines
a comprehensive and detailed reform agenda for the area of rule of law and covers three sub-areas,
namely the judiciary, the fight against corruption and fundamental rights. The Action Plan gives an
overview of all commitments and deadlines impsed during the screening process and should enable the
fulfilment of the accession conditionality through clearly defined objectives and by determining indicators
of results and impacts. With the adoption of the Action Plan, Montenegro fulfilled the sole benchmark for
the opening of accession negotiations in chapter 23.613
After the opening benchmarks had been fulfilled by Montenegro, chapters 23 and 24 were opened for
accession negotiations in the third meeting of the Accession Conference with Montenegro at Ministerial
level on 18 December 2013. In the EU common position on chapter 23 and chapter 24, the Accession
Conference adopted a list of interim benchmarks that have to be met before the negotiation process can
proceed to the next steps. The Accession Conference emphasised that particular attention shall be
devoted to monitoring these issues throughout the negotiations.614 83 interim benchmarks were set for
chapters 23 and 24 that mirror the ambitious Action Plans and cover issues reflecting the reality on the
ground.615 The interim benchmarks for chapter 23 especially address the independence, impartiality and
accountability and professionalism, competence and efficiency of the judiciary as well as the handling
of domestic war crimes cases.616
3.
The indicators applied to measure the rule of law in Montenegro
The monitoring tools and indicators applied to measure the rule of law in Montenegro, quite closely
resemble those applied in the case of Croatia. The Enlargement Strategy and the conduct of accession
negotiations that had already been introduced in the previous enlargement rounds were further defined
and became more comprehensive. Besides the opening and closing benchmarks that were already applied
610
European Commission (n 608) 21-22.

European Commission, Montenegro 2014 Progress Report (Staff Working Document) SWD (2014) 301 final, 1.
612
Government of Montenegro, Action Plan for Chapter 23 Judiciary and Fundamental Rights adopted in Podgorica
on 27 June 2013.
613
ibid, 4-9.
614
Council of the European Union, Third Meeting of the Accession Conference with Montenegro at Ministerial level
key Rule of Law Chapters opened among Others (Press Release, December 2013).
615
European Union, Memo: EU-Montenegro: Inter-Governmental Conference (18 December 2013)
<http://europa.eu/rapid/press-release_MEMO-13-1182_de.htm> accessed 2 November 2014.
616
Council of the European Union, European Union Common Position: Chapter 23: Judiciary and fundamental rights
(Accession Document AD 17/1, December 2013)
611
166
FRAME
as monitoring tools to Croatia and Bulgaria within the CVM, the Commission made use of interim
benchmarks, allowing for a closer monitoring of the continuous progress towards fulfilling the obligations.
The interim benchmarks set by the Commission are based on the obligations and measures determined
in the Action Plans.617
In its assessments the Commission referred to the standards established by international organisations
and conventions in the field of the judiciary and fundamental rights that had already been applied to
Croatia.618 The specific requirements identified with regard to independence, accountability and efficiency
of the judiciary in chapter 23 were a firm commitment to eliminating external influences over the
judiciary, adequate financial resources and training, [l]egal guarantees for fair trial procedures and to
fight corruption effectively.619 These requirements are to be assessed by the Commission in the annual
reports. The following analysis is conducted on the basis of the reports prepared by the Commission
between 2010 and 2013.
a)
The monitoring tools applied by the Commission in the accession process of Montenegro made use of the
indicators adopted in previous enlargement rounds to measure the independence, impartiality and
accountability of the judiciary. In the Opinion on Montenegros application for membership, the
Commission identified the de-politicisation of appointment procedures and the reinforcement of
independence, autonomy and accountability of judges and prosecutors as key priorities to strengthen the
rule of law.620 The Analytic Report of 2010 and the Progress Reports of the following years assessed the
improvements made in addressing these priorities. The indicators applied measured the de-politicisation
of the Judicial and Prosecutorial Council, the system applied for the recruitment and appointment of
judges and prosecutors, the budget available to the judiciary and the salaries of magistrates, prosecutors
and judges. The existence of a Code of Ethics was welcomed by the Commission as a structural indicator
for improving the accountability in the judiciary and also the number of proceedings initiated for its breach
was continuously monitored. The system in place to ensure a random allocation of cases to individual
judges was assessed by the Commission as a process indicator to guarantee the impartiality of judges.621
The interim benchmarks introduced with the opening of negotiations on chapter 23 by the Council in
December 2013 also addressed the independence, impartiality and accountability of the judiciary. Based
on these benchmarks, Croatia is requested to establish (among others) initial track records of
appointments of high-level judges and high level prosecutors based on transparent and merit-based
procedures, recruiting judges and prosecutors on the basis of a single, nationwide, transparent and merit
based system and []obligatory initial training in the Judicial Training Centre, implementing a fair and
transparent system of promoting judges and prosecutors and regular inspections of the work of judges
617
Government of Montenegro (n 612) 5-6.

cf European Commission, Screening Report Montenegro: Chapter 23 (n 608) 2 and European Commission,
Screening Report Croatia: Chapter 23 (n 559) 2.
619
SEC (2010) 1334, 96 and European Commission, Screening Report Montenegro: Chapter 23 (n 608) 2.
620
COM (2010) 670, 11.
621
SEC (2010) 1334, 97; SEC (2011) 1204 final, 10-11 and 56; SWD (2012) 331 final, 44-45 and SWD (2013) 411 final,
36-37.
618
167
FRAME
and prosecutors.622 These benchmarks were specifically addressed in the Progress Report of the year
2014 under chapter 23, however, no direct reference is made to the measures and deadlines determined
in the Action Plan.623
Progress Reports to measure access to justice and efficiency of the judiciary in Montenegro.

Process indicators
Outcome Indicators
Legal
framework
in
place
guaranteeing
self-administration
Legal
framework
in
place
Provisions in place to apply objective
and transparent criteria for
recruitment
Provisions in place to guarantee depoliticised
and
merit-based
appointments and promotion of
judges and prosecutors
Provisions in place for regular
inspections

Salaries
of
magistrates,
prosecutors and judges
established for the recruitment of
judicial staff and for the
appointment of judges
Methodology adopted to evaluate
judges for the purposes of
promotion
Application of a system for random
allocation of cases to individual
judges
Establishment of a commission for
monitoring the enforcement of the
Code of Ethics
Mandatory declarations of assets
Publicity of court rulings
No of inspections carried out
Reduction of risk of political

interference in the judiciary
De-politicisation
of
the
Judicial and Prosecutorial
Councils
Number
of
disciplinary
Establishment of a countrywide single recruitment
system
objective criteria
Number of irregularities
reported
through
the
inspections and sanctions
applied
b)
To reinforce the efficiency of the judiciary was one of the key priorities identified by the Commission for
the field of rule of law in its Opinion on the application of membership.624 The Commission consistently
monitored the efficiency of the judiciary and access to justice in the annual Progress Reports of
Montenegro. The Analytical Report of 2010 and the following Progress Reports provided a comprehensive
overview of the judicial system and identified several improvements and achievements as well as
shortcomings. The three-tier jurisdiction was assessed with regard to the structure of the court system
and of the public prosecution. Furthermore, the legal framework in place facilitating quicker proceedings,
like the new Criminal Procedure Code with simplified pre-trail proceedings, was investigated. Also the
budget allocations for the courts and the administrative capacity in terms of infrastructure and equipment
were indicators to measure the efficiency of the judiciary. With regard to the professionalism of the
judiciary, the Commission assessed the training for judges and prosecutors provided by the Judicial
Training Centre and criticised the lack of permanent mandatory courses and of a set curricula. Also the
improvements with regard to the guarantee of free legal aid were monitored and the Commission
622
Council of the European Union, European Union Common Position: Chapter 23 (n 616) 19-20.
SWD (2014) 301 final, 36-37.
624
COM (2010) 670, 11.
623
168
FRAME
welcomed the implementing acts adopted on the Law of Free Legal Aid in 2012 and the opening of free
legal aid offices.625
The interim benchmarks introduced for chapter 23 by the Council in December 2013 addressed the
professionalism, competence and efficiency of the judiciary and required in particular to develop an initial
track record of backlog reduction, to implement the Judicial Information System that allows monitoring
the workload and performance of judges and courts as well as the adoption of a law on judicial training,
securing the necessary financial and human resources for the Judicial Training Centre.626 In this way, the
interim benchmarks adopted by the Council reflect the priorities established in the SAA and European
Partnership and are based on the objectives set in the Action Plan. These objectives were already
monitored by the Commission in the reports from 2010 to 2013 and specifically addressed by the 2014
Progress Report. In the field of professionalism, competence and efficiency of the judiciary, all interim
benchmarks set in 2013 were followed up in the Progress Report of the year 2014, however, no direct
reference was made to the measures and deadlines determined in the Action Plan.627
Progress Reports to measure access to justice and efficiency of the judiciary in Montenegro.
Access to Justice
Process indicators
Outcome Indicators

guaranteeing
a
three-tier
jurisdiction
Availability of Alternative Dispute
Resolution methods
Legal framework in place with
regard to the right to legal
remedy
guarantee free legal aid

Implementation of the law on the
right to trail within a reasonable
time
Human resources available in
courts
Material conditions in courts
(infrastructure, equipment and IT
system)
Mandatory
courses
and
systematic training for judges and
prosecutors
Availability of budget for the
Judicial Training Centre
Length of judicial proceedings

Length of pre-trial detention
Backlog of cases
Rejections of complaints on
procedural grounds
Transparent
and
efficient
handling of cases
Number of trainings attended by
judicial staff
Number of free legal aid offices
In the current EU enlargement policy towards Western Balkan countries, the accession conditionality has
been expanded and specific criteria to be fulfilled by the candidate countries have been identified.
Montenegro is one of the countries, for which the new approach to negotiations should allow establishing
the necessary legislation, institutions and solid track record of implementation before the negotiations
are closed.628 The Commission is promoting and monitoring the reforms to be implemented in the field
625
SEC (2010) 1334,16-17, 19 and 98; SEC (2011) 1204 final, 11-12 and 57; SWD (2012) 331 final, 45-46 and SWD
(2013) 411 final 37-38.
626
Council of the European Union, European Union Common Position: Chapter 23 (n 616) 21-22.
627
SWD (2014) 301 final, 38-39.
628
Hillion (n 543) 6.
169
FRAME
of judiciary and fundamental rights in its annual reports. In the Analytical Report from 2010, the
Commission stressed the paramount importance of an independent and efficient judiciary to safeguard
the rule of law in the EU and its Member States.629
In its assessments, the Commission has very consistently addressed the key priorities identified in the
Screening Report, the SAA and the European Partnership in terms of independence, accountability,
efficiency and professionalism of the judiciary. With the opening of accession negotiations for chapter 23
in December 2013, very specific interim benchmarks have been established, requesting Montenegro to
develop convincing track records in several areas of judiciary. In what way these specific benchmarks will
be addressed in the assessments conducted for the annual progress through indicators remains to be seen
in the future.
D.
Implications for the use of indicators in the EU enlargement policy
The Commission has set up a comprehensive monitoring mechanism for the EU enlargement policy in
order to measure the progress of candidate and potential candidate countries in realising the rule of law
and fulfilling the membership criteria. Areas, where only little progress was made, were repeatedly
addressed as priorities in the subsequent Accession Partnerships with the countries, and the progress in
implementing these priorities was monitored in the annual reports. Based on the lessons learned from
previous enlargement rounds, the EU continuously improved its Enlargement Strategy. As one can see
along the three case studies, the consistency between the different monitoring tools used by the
Commission has improved over time and more transparency was achieved. Based on the findings of the
case studies, the following general implications for the use of indicators for measuring the rule of law in
the EU enlargement policy can be derived.
(1) Combining structural, process and outcome indicators
The case studies have shown that in the fifth and sixth enlargement round, the monitoring of the
Commission primarily focused on structural and process indicators. While the progress in terms of
legislation and policies adopted for the promotion of the rule of law was assessed in great detail through
the various monitoring instruments, there was an apparent deficiency in measuring the results achieved
by the requested reforms through outcome indicators. Following a human rights centred approach, the
Commission has recognised that also the realisation of rule of law perceived by the population is an
important element to be assessed in the rule of law measurement. As a first step in this direction, the
current Enlargement Strategy specifically focuses on the development of solid track records of reform
implementation in the enlargement countries.
(2) Combining quantitative and qualitative indicators
The measurement of the rule of law in the accession processes initially was performed mainly through
qualitative assessments and the results were presented in a descriptive manner in the annual reports. As
pointed out by an official from DG Enlargement in the interview,630 the rule of law is a field, where
appropriate quantitative indicators are hard to define. Therefore, the assessments will always be mostly
629
630
SEC (2010) 1334, 96.

ETC Graz (n 480).
170
FRAME
based on qualitative indicators. However, the case studies have shown that the Commission makes more
and more use of quantitative measures provided by international organisations to conduct the
assessments. To enhance the consistency and comparability of the reports, the Commission should
indicate the data sources more explicitly and continuously follow up on the measures provided in previous
years.
(3) Combining universal and country-specific indicators
As the decisions on accession shall be based on the same criteria for all the enlargement countries, the
Commission tries to unify the indicators used for measuring the rule of law in the candidate countries as
much as possible. The use of a reasonable core set of universal indicators is meant to ensure objectivity
and consistency in the measurements and to allow for comparisons among countries.631 However, the
enlargement countries strongly vary in terms of the initial level of realisation of the rule of law, as some
of the current candidate and potential candidate countries in the Western Balkans are weakly established
states suffering grave political problems in terms of national identity, ethnic conflicts and corruption. 632
These particular challenges have to be taken into account, when assessing progress in the field of the rule
of law in these countries. To address these particular challenges and to provide meaningful measures, the
indicators applied to measure the rule of law require a thorough contextualisation for the specific
situation in the country and should be adapted to the local specificities in cooperation with the respective
state authorities and actors from civil society.633
(4) Combining multiple data sources
As the case studies showed, the Commission draws upon multiple data sources to prepare the annual
reports. These sources include data provided directly by the enlargement countries themselves, data
collected by the Commissions delegations and through expert reviews as well as the data stemming from
international organisations and NGOs. As it was pointed out in an interview,634 the Commission is in
continuous contact with international institutions that provide data for the measurement of the rule of
law. To increase transparency and to allow for greater comparability between enlargement countries, the
different data sources used and the measures applied to conduct the assessments should be more clearly
indicated in the reports.
(5) Tracking changes over time and setting benchmarks
As one can see along the three case studies, the Commission has become more consistent in tracking
changes over time throughout the latest enlargement rounds. According to the current Enlargement
Strategy, an analytical examination of the acquis is conducted by the Commission in the beginning of the
accession negotiations with each country. This assessment provides the basis for the definition of
appropriate opening and closing benchmarks for all negotiation chapters. As emphasized in an
interview,635 the Commission mainly relies on the acquis, the practices from the Member States and the
standards set by international organisations to decide on appropriate benchmarks in the field of the rule
631
ibid.
cf Bieber (n 423).
633
634
ETC Graz (n 480).
635
ibid.
632
171
FRAME
of law. The Commission is following-up on the progress of the enlargement countries in addressing the
benchmarks in its annual reports. The issues monitored correspond to the priorities identified in European
and Accession Partnerships and reflect the improvements achieved through the financial assistance
provided by the EU. However, for the sake of guaranteeing consistency in the overall monitoring process,
the various monitoring tools should be linked in a more transparent manner and the progress made
against reaching the specific benchmarks should be clearly indicated in all reports. Benedek and others
suggest:
to spell out in the Progress Reports more explicitly which priorities have been identified in
the European and Accession Partnerships and make a concrete statement as to the degree
in which these priorities are met or work is taking place in order to reach them, and as to the
measures to be taken to reach the goal.636
As stressed by Bieber, it often remains unclear, what changes are essential for further progress towards
EU membership.637 Clear guidance and concrete recommendations, on the one hand, would help the
enlargement countries in their efforts to reaching the set targets and, on the other hand, would also make
it easier for the Commission to assess the progress or decline in the reports.638
VIII. Conclusions and outlook

Over the last decade, the EU enlargement policy has been constantly linked to the prerequisite of the
realisation of the rule of law in the enlargement countries. As non-compliance with this requirement could
undermine the functioning of the internal market and mutual trust among Member States, the EU has
imposed strict accession conditionality on the candidate countries. This conditionality in conjunction with
the continuous monitoring efforts of the Commission has proven to be a successful tool to bring about
the demanded reforms for the realisation of the rule of law. The three case studies have shown that rule
of law issues always had a prominent position in the EU enlargement processes and this position was
further strengthened in the current EU Enlargement Strategy.
The tools and indicators applied for measuring the rule of law in the EU enlargement processes have been
continuously improved over time. In conducting the country assessments for the annual Progress Reports,
the Commission has been relying on qualitative and quantitative indicators from diverse sources of data.
Through the close cooperation with national authorities, NGOs, country experts and international
organisations, multiple data sources could be accessed. The performance of the enlargement countries
was measured against the standards set through the acquis communautaire and by international
organisations, like the CoE or the UN. By setting common standards in the accession negotiations, the
comparability and transparency of the conditions to be fulfilled by the enlargement countries could be
increased. For several areas of rule of law, the Commission has set specific benchmarks, reflecting the
required standards to be achieved prior to accession. These benchmarks are adapted to the specific
636

Bieber (n 423) 1795
638
Benedek and others (n 547) 92-93.
637
172
FRAME
situation of the respective country and shall allow for tracking changes over time through the annual
reports. However, not all elements of the rule of law are covered by the acquis, and for some areas no
common standards could be identified at international level. In these areas, it is harder to ensure
comparability, and the realisation of the rule of law can only be measured through qualitative
assessments. Also the benchmarks established for these areas of rule of law often lack a clear definition
of the underlying concepts and of the objectives to be achieved, which hampers a consistent and
comprehensive monitoring process.
By analysing the quality of the Progress Reports over time and over countries, it becomes clear that there
is a serious effort by the EU to enhance consistency and transparency in the enlargement process. Through
the incorporation of the lessons learned from previous enlargement rounds, a comprehensive negotiation
policy has been established that shall ensure equal treatment of all enlargement countries. Throughout
the latest enlargement rounds, the Commission has shifted its focus from a structure-driven approach to
a more result-oriented approach. The promotion and realisation of the rule of law is now measured by
simultaneously applying structural, process and outcome indicators. In this way, the progress in terms of
legislation adopted can be matched to the policies applied and the respective results achieved. Under the
current Enlargement Strategy, enlargement countries are asked to develop a solid track record of reform
implementation, demonstrating the results achieved with regard to the realisation of rule of law in
practice. In the long term, this should not only assure that enlargement countries fulfil the membership
conditionality in terms of legislation adopted and policies implemented, but should also improve the
situation for the population of these countries.
The membership conditionality is considered to be the fundament of the successful EU enlargement policy
in Central and Eastern Europe. The case of Bulgaria, however, has shown that it is crucial to meet specific
standards already during the pre-accession phase. At the time of accession, Bulgaria had not met all the
conditions for membership. With the establishment of the CVM, the Commission intended to support
Bulgaria in achieving the same standards as other Member States. However, also five years after accession
no sufficient progress could be observed, what leads to the conclusion that the post-accession incentives
were too weak to bring about the requested changes after accession. As pointed out by Hillion,639 postaccession monitoring may be useful to spell out the required reforms, but the credibility and potential of
such a mechanism is hampered by the lack of serious consequences in case of non-compliance. This
underlines the importance of the accession conditionality that shall lead to the realisation of the required
standards already in the pre-accession and negotiation phase. To tackle the problems experienced in
previous accession processes, the EU policy of conditionality has been expanding from one enlargement
round to the other. The case studies of Croatia and Montenegro have shown the wider scope of the
accession conditionality on the one hand and the growing number of monitoring tools applied by the
Commission to measure the rule of law on the other hand. In its enlargement strategy, the Commission
emphasises the transformative effect of the EUs enlargement policy by ensuring a stronger focus on
addressing fundamental reforms early in the enlargement process.640 Also for the future enlargement
towards the countries of Eastern and South-Eastern Europe, the EU has imposed strict accession
639
640
Hillion (n 543) 12-13.

COM (2014) 700 final, 1.
173
FRAME
conditionality. The credibility of the conditionality applied to the candidate and potential candidate
countries can only be ensured, if the indicators applied for measuring the countries alignment with the
membership criteria are concrete and specific enough and if benchmarks are set in a systematic way and
based on clearly defined concepts and objectives.
The EU was often criticised for applying double standards, with regard to the conditionality applied in the
field of the rule of law and fundamental rights. On the one side, the EU requires strict adherence to those
values by the enlargement countries as a prerequisite for joining the EU. On the other side, the EU is aware
that there are current Member States that do not fully comply with those values.641 Even though there is
a long-standing experience in measuring the rule of law in enlargement countries, the information
available about the rule of law in the Member States is often intermittent, hardly comparable and based
on weak data sources and data collection methods. When exploring better ways, on how to monitor and
safeguard the rule of law in the EU, the vast institutional experience in assessing candidate countries could
be used as a basis.642 This goes in line with the recommendation that the EU shall apply the same indicators
and benchmarks for its internal as well as for the external policy to increase the credibility of the EUs
policy and to legitimate the export of its values towards enlargement and third countries.643
A consistent set of indicators and data collection methodology for the EUs internal and external policy
would make the progress in promoting the rule of law and justice more transparent, comparable and
manageable and would assist the enlargement countries as well as Member States in realising and
upholding the rule of law. This would also respond to the request from the European Parliament for
establishing an effective mechanism for a regular assessment of the Member States compliance with the
fundamental values of the EU.644 Such a new Copenhagen mechanism645 could ensure the respect and
protection of the EUs foundational values and can be applied as a tool of the EUs internal and external
policy to promote the rule of law.
641
Spreeuw (n 404) 12 et seqq.

FRA, Promoting the Rule of Law in the European Union (FRA Symposium Report 4 th Annual FRA Symposium,
Vienna, June 2013) <http://fra.europa.eu/sites/default/files/fra-2013-4th-annual-symposium-report.pdf> accessed
20 January 2014.
643
Anja Mihr, Human Rights Benchmarks for EUs External Policy (Briefing Paper, European Parliament 2011) 5.
644
Resolution P7_TA-PROV (2014) 0231, para. 6.
645
Carrera, Guild and Hernanz (n 405).
642
174
FRAME
Bibliography
Legal and policy instruments
European Convention for the Protection of Human Rights and Fundamental Freedoms, signed in Rome, 4
November 1950.
Treaty establishing the European Economic Community, signed in Rome, 25 March 1957.
Treaty on European Union [1992] OJ C191/01 (TEU).
Europe Agreement establishing an Association between the European Communities and their Member
States, of the one part, and the Republic of Bulgaria, of the other part, signed in Brussels, 8 March
1993.
Treaty of Amsterdam Amending the Treaty on European Union, the Treaties Establishing the European
Communities and Certain Related Acts [1997] OJ C340/01(Treaty of Amsterdam).
Council Decision (EC) 1999/857 of 6 December 1999 on the Principles, Priorities, Intermediate Objectives
and Conditions Contained in the Accession Partnership with the Republic of Bulgaria [1999] OJ
L335/49.
Council Regulation (EC) 2666/2000 of 5 December 2000 on Assistance for Albania, Bosnia and
Herzegovina, Croatia, the Federal Republic of Yugoslavia and the Former Yugoslav Republic of
Macedonia repealing Regulation (EC) No 1628/96 and amending Regulations (EEC) No 3906/89 and
(EEC) No 1360/90 and Decisions 97/256/EC and 1999/311/EC [2000] OJ L306/1.
Treaty of Nice Amending the Treaty on European Union, the Treaties Establishing the European
Communities and Certain Related Acts [2001] OJ C80/01 (Treaty of Nice).
Council Decision (EC) 2003/396 of 19 May 2003 on the Principles, Priorities, Intermediate Objectives and
Conditions Contained in the Accession Partnership with Bulgaria [2003] OJ L145/1.
Stabilisation and Association Agreement between the European Communities and their Member States,
of the one part, and the Republic of Croatia, of the other part [2005] OJ L26/3 (SAA Croatia).
Council Decision (EC) 2006/145 of 20 February 2006 on the Principles, Priorities and Conditions contained
in the Accession Partnership with Croatia and repealing Decision 2004/648/EC [2006] OJ L55/30.
Council Regulation (EC) 1085/2006 of 17 July 2006 Establishing an Instrument for Pre-Accession Assistance
(IPA) [2006] OJ L210/82.
Council Decision (EC) 2007/49 of 22 January 2007 on the Principles, Priorities and Conditions contained in
the European Partnership with Montenegro [2007] OJ L20/16.
Stabilisation and Association Agreement between the European Communities and their Member States
of the one part, and the Republic of Montenegro, of the other part, signed in Luxembourg, 15 October
2007 (SAA Montenegro).
Treaty of Lisbon Amending the Treaty on European Union and the Treaty Establishing the European
Community, signed at Lisbon, 13 December 2007 [2007] OJ C306/01 (Treaty of Lisbon).
Council Decision (EC) 2008/119 of 12 February 2008 on the Principles, Priorities and Conditions contained
in the Accession Partnership with Croatia and repealing Decision 2006/145/EC [2008] OJ L42/51.
Charter of Fundamental Rights of the European Union [2010] OJ C83/389.
175
FRAME
Government of Montenegro, Action Plan for Chapter 23 Judiciary and Fundamental Rights adopted in
Podgorica on 27 June 2013.
Regulation (EU) 231/2014 of the European Parliament and of the Council of 11 March 2014 Establishing
an Instrument for Pre-Accession Assistance (IPA II) [2014] OJ L77/11.
Case Law
Case 294/83 Les Verts v Parliament [1986] ECR 1339.
Case C-345/04 P Gestoras Pro Amnistia and Others v Council [2007] ECR I 1631.
Literature
Albers P, How to measure the rule of law: a comparison of three studies (2007)
<http://www.coe.int/t/dghl/cooperation/cepej/events/onenparle/Albers251007.pdf> accessed on 20
April 2014.
Benedek W and others Mainstreaming Human and Minority Rights in the EU Enlargement with the
Western Balkans (Study, European Parliament 2012).
Benoit-Rohmer F and others, Human Rights Mainstreaming in EUs External Relations (Study, European
Parliament 2009).
Berenschot and Imagos, Evaluation of Governance, Rule of Law, Judicial Reform and Fight Against
Corruption and Organised Crime in the Western Balkans: Lot 2 Main Report (2012).
Bieber F, Introduction (2011) 63/10 Europe Asia Studies 1775.
Bieber F, Building Impossible States? State-Building Strategies and EU Membership in the Western
Balkans (2011) 63/10 Europe-Asia-Studies 1783.
Bislimi F, EU Foreign Policy towards Balkans: An Opportunity or a Challenge? (2010) 1 The Western
Balkans Policy Review 29.
Carrera S, Guild E and Hernanz N, The Triangular Relationship between Fundamental Rights, Democracy
and the Rule of Law in the EU: Towards an EU Copenhagen Mechanism (CEPS Paperbacks 2013).
Ccile R and Magalie J, Human Rights and Democracy Indicators: Definition and Implications (Study,
European Parliament 2006).
CEPEJ, European Judicial Systems (Council of Europe Publishing 2012).
Craig P, Formal and Substantive Conceptions of the Rule of Law: An Analytical Framework [1997] Public
Law 466.
de Beco G, Human Rights Indicators for Assessing State Compliance with International Human Rights
(2008) 77 Nordic Journal of International Law 23.
Del Sarto R and others, Benchmarking Democratic Development in the Euro-Mediterranean Area:
Conceptualising Ends, Means, and Strategies (EuroMeSCo Report 2006).
DPKO and OHCHR, Rule of Law Indicators Implementation Guide and Project Tools (UN 2011).
Elbasani A, EU Enlargement in the Western Balkans: Strategies of Borrowing and Inventing (2008) 10/3
Journal of Southern Europe and the Balkans 293.
ETC Graz, Interview with an official from DG Enlargement (Brussels and Graz, 17 June 2014).
176
FRAME
FRA, Promoting the Rule of Law in the European Union (FRA Symposium Report 4th Annual FRA
Symposium, Vienna, June 2013) <http://fra.europa.eu/sites/default/files/fra-2013-4th-annualsymposium-report.pdf> accessed 20 January 2014.
Gramatikov M and Janse R, Monitoring and Evaluation of the Rule of Law and Justice in the EU: Status
Quo and the Way Ahead? (Concept Paper, HiiL 2012).
Gramatikov M and Frishman M, Measuring the Rule of Law, Justice and Fundamental Rights (Concept
Paper, HiiL 2013).
Hillion C, Enlarging the European Union and Deepening its Fundamental Rights Protection (2013) 11
European Policy Analysis <http://www.sieps.se/sites/default/files/2013_11epa.pdf> accessed 2 July
2014.
Janse R, Sanchez Galera M D and Liivoja R, Rule of Law Inventory Report: Academic Part (HiiL, 2007).
Kaufmann D, Kraay A and Mastruzzi M, Governance Matters VIII: Aggregate and Individual Governance
Indicators, 1996-2008 (2009) World Bank Policy Research Working Paper No. 4978.
Kochenov D, EU Enlargement and the Failure of Conditionality: Pre-accession Conditionality in the Fields
of Democracy and the Rule of Law (Kluwer Law International 2008).
Landman T, Map-Making and Analysis of the Main International Initiatives on Developing Indicators on
Democracy and Good Governance (Human Rights Centre, University of Essex 2003).
Landman T. The Scope of Human Rights: From Background Concepts to Indicator (Background paper for
the UN Expert Meeting on Human Rights Indicators, Turku 2005).
Mihr A, Human Rights Benchmarks for EUs External Policy (Briefing Paper, European Parliament 2011).
Mstl M, Core Tasks of Good Governance and Their Impact on Peacebuilding and Human Security, in
Benedek W and others (eds), MultiPart Thematic Paper on Multi-Stakeholder Partnerships Active in
the Field of Good Governance, Democracy and Rule of Law (ETC Graz 2010).
OHCHR, Report on Indicators for Promoting and Monitoring the Implementation of Human Rights (UN
2008).
Pech L, The Rule of Law as a Constitutional Principle of the European Union, (2009) 4 Jean Monnet
Working Paper <http://www.jeanmonnetprogram.org/papers/09/090401.pdf> accessed 15 January
2014.
Pech L, Rule of Law as a Guiding Principle of the European Unions External Action (2012) 3 CLEER
Working Papers <http://www.asser.nl/upload/documents/2102012_33322cleer 2012-3web.pdf>
accessed 13 January 2014.
Sembacher A and Starl K, Anti-Discrimination for Judges and Prosecutors: General Part, (ETC Graz 2006)
<http://antidiscrimination.etc-graz.at/cms/fileadmin/user_upload/Projekte/
laufend/ADTJ/English/manual_english_090606.pdf> accessed 24 June 2014.
Schimmelfenning F and Sedelmeier U, Governance by Conditionality: EU Rule Transfer to the Candidate
Countries of Central and Eastern Europe (2004) 11/4 Journal of European Public Policy 669.
Smilov D, EU Enlargement and the Constitutional Principle of Judicial Independence in: Sadurski W,
Czarnota A and Krygier M (eds), Spreading Democracy and the Rule of Law? (Springer 2006).
177
FRAME
Spreeuw M, Double Standards in the Application of the EU Values and Principles: Member States versus
Potential Candidate and Candidate Countries (2012) <http://www.pravo.unizg.hr/_download/
repository/Paper_M._Spreeuw.pdf> accessed 6 April 2014.
Starl K and Pinno J, Study on Challenges in the Development of Local Equality Indicators: A Humen-RightsCentred Model (Study carried out by ETC Graz in the framework of the Working Group on Indicators
of the European Coalition of Cities against Racism, UNESCO 2010).
Tamanaha B Z, On the Rule of Law: History, Politics, Theory (Cambridge University Press 2004).
TNS Political & Social, Flash Eurobarometer 351: The Cooperation and Verification Mechanism for
Bulgaria and Romania (Survey Conducted on the Request of the European Commission, July 2012).
Thomas M, What Do the Worldwide Governance Indicators Measure (Johns Hopkins University 2006).
Trauner F, From Membership Conditionality to Policy Conditionality: EU External Governance in South
Eastern Europe (2009) 16/5 Journal of European Public Policy 774.
Veebel V, European Unions Positive Conditionality Model in Pre-accession Process (2009) 13/3 TRAMES
207.
Policy and other reports

CoE Committee of Ministers, Recommendation on the Independence, Efficiency and Role of Judges R
(94) 12.
CoE Committee of Ministers, Recommendation Concerning the Introduction and Improvement of the
Functioning of Appeal Systems and Procedures in Civil and Commercial Cases R (95) 5.
CoE, European Charter on the Statute for Judges DAJ/DOC (98) 23.
CoE Conference of Prosecutors General of Europe, European Guidelines on Ethics and Conduct for Public
Prosecutors (the Budapest Guidelines) CPGE (2005) 05.
Council of the European Union, Conclusions from the General Affairs Council Meeting (Luxembourg, April
1997).
Council of the European Union Tenth Meeting of the Accession Conference at Ministerial Level with
Croatia (Press Release, June 2010).
Council of the European Union, Conclusions on Montenegro of the General Affairs Council Meeting
Council of the European Union Conclusions on Enlargement and Stabilisation and Association Process of
the General Affairs Council Meeting (Brussels, December 2012).
Council of the European Union, Conclusions of the Justice and Home Affairs Council Meeting
Council of the European Union, Third Meeting of the Accession Conference with Montenegro at
Ministerial level key Rule of Law Chapters opened among Others (Press Release, December 2013).
Council of the European Union, European Union Common Position: Chapter 23: Judiciary and
fundamental rights (Accession Document AD 17/1, December 2013).
European Commission, Solemn Declaration on European Union (1983) 6 Bulletin of the European
Communities.
178
FRAME
European Commission, Opinion on Bulgarias Application for Membership of the European Union
DOC/97/11.
European Commission, Agenda 2000: For a stronger and wider Union (Communication) COM (97) 2000
final.
European Commission, Regular Report on Bulgarias Progress Towards Accession COM (98) 707 final.
European Commission, On the Stabilisation and Association Process for Countries of South-Eastern
Europe, (Communication) COM (1999) 235 final.
European Commission, Composite Paper: Reports on progress towards accession by each of the
candidate countries COM (99) 500 final.
European Commission, 1999 Regular Report on Bulgarias Progress Towards Accession COM (99) 501
final.
European Commission, Enlargement Strategy Paper: Report on Progress Towards Accession by each of
the Candidate Countries COM (2000) 700 final.
European Commission, 2001 Regular Report on Bulgarias Progress Towards Accession SEC (2001) 1744.
European Commission, The Stabilisation and Association Process for South East Europe: First Annual
Report COM (2002) 163 final.
European Commission, The Stabilisation and Association Process for South East Europe: Second Annual
Report COM (2003) 139 final.
European Commission, On Article 7 of the Treaty on European Union: Respect for and Promotion of the
Values on which the Union is Based (Communication) COM (2003) 606 final.
European Commission, Governance and Development (Communication) COM (2003) 615 final.
European Commission, 2003 Regular Report on Bulgarias Progress Towards Accession SEC (2003) 1210
final.
European Commission, Opinion on Croatia's Application for Membership of the European Union
European Commission, Strategy Paper of the European Commission on Progress in the Enlargement
Process (Communication) COM (2004) 657 final.
European Commission, Opinion of 22 February 2005 on the applications for accession to the European
Union by the Republic of Bulgaria and Romania COM (2005) 55 final/2.
European Commission, Croatia 2005 Progress Report SEC (2005) 1424.
European Commission, Monitoring Report on the State of Preparedness for EU Membership of Bulgaria
and Romania (Communication) COM (2006) 549 final.
European Commission, Establishing a Mechanism for Cooperation and Verification of Progress in Bulgaria
to Address Specific Benchmarks in the Areas of Judicial Reform and the Fight Against Corruption and
Organised Crime (Decision) C (2006) 6550 final.
179
FRAME
European Commission, Enlargement Strategy and Main Challenges 2006 2007 (Communication) COM
(2006) 649.
European Commission, Croatia 2006 Progress Report (Staff Working Document) SEC (2006) 1385.
European Commission, Montenegro 2006 Progress Report (Staff Working Document) SEC (2006) 1388.
European Commission, Report on Bulgarias Progress on Accompanying Measures following Accession
COM (2007) 337 final.
European Commission, Screening Report Croatia: Chapter 23 Judiciary and fundamental rights (2007).
European Commission, Country Sections & Additional Information (Staff Working Paper) SEC (2010)
1604.
European Commission, Enlargement Strategy and Main Challenges 2010 2011 (Communication) COM
(2010) 600 final.
European Commission, Opinion on Montenegros Application for Membership to the European Union
(Communication) COM (2010) 670.
European Commission, Analytical Report (Staff Working Document) SEC (2010) 1334.
European Commission, Interim Report on Reforms in Croatia in the Field of Judiciary and Fundamental
Rights (Negotiation Chapter 23) COM (2011) 110.
European Commission, Enlargement Strategy and Main Challenges 2011-2012 (Communication) COM
(2011) 666 final.
European Commission, Montenegro 2011 Progress Report (Staff Working Paper) SEC (2011) 1204 final.
(2012) 600 final.
European Commission, Report on Montenegros Progress in the Implementation of Reforms COM (2012)
222 final.
European Commission, Montenegro 2012 Progress Report (Staff Working Document) SWD (2012) 331
final.
European Commission, Progress in Bulgaria under the Cooperation and Verification Mechanism (Report)
COM (2012) 411 final.
European Commission, Bulgaria: Technical Report (Staff Working Document) SWD (2012) 232 final.
European Commission, Comprehensive Monitoring Report on Croatia (Staff Working Document) SWD
(2012) 338 final.
European Commission, Screening Report Montenegro: Chapter 23 Judiciary and Fundamental Rights
(2012).
180
FRAME
European Commission, Screening Report Montenegro: Chapter 24 Justice, Freedom and Security
(2012).
European Commission, The EU Justice Scoreboard: A Tool to Promote Effective Justice and Growth
European Commission, Monitoring Report on Croatia's Accession Preparations (Communication) COM
(2013) 171 final.
final.
(2013) 700 final.
European Commission Progress in Bulgaria under the Cooperation and Verification Mechanism (Report)
COM (2014) 36 final.
European Commission, Bulgaria: Technical Report (Staff Working Document) SWD (2014) 36 final.
European Commission, A new EU Framework to Strengthen the Rule of Law (Communication) COM
(2014) 158 final.
(2014) 700 final.
final.
European Council, Conclusions of the Presidency (Copenhagen, June 1993).
European Council, Presidency Conclusions (Luxembourg, December 1997).
European Council, Presidency Conclusions (Madrid, December 1995).
European Council, Conclusions of the Presidency (Santa Maria da Feira, June 2000).
European Council, Presidency Conclusions (Thessaloniki, June 2003).
European Council Presidency Conclusions (Brussels, June 2004).
European Council Presidency Conclusions (Brussels, December 2006).
European Council, Conclusions (Brussels, December 2011).
European Council, Conclusions (Brussels, June 2012).
European Commission for Democracy Through Law, Report on the Rule of Law CDL-AD (2011) 003rev.
European Parliament, Situation of Fundamental Rights: Standards and Practices in Hungary (Resolution)
P7_TA-PROV (2013) 0315.
European Parliament The Situation of Fundamental Rights in the European Union (Resolution) P7_TAPROV (2014) 0173.
European Parliament, Evaluation of Justice in Relation to Criminal Justice and the Rule of Law
(Resolution) P7_TA-PROV (2014) 0231.
UN, Basic Principles of the Independence of the Judiciary (1985).
181
FRAME
Websites
CEPEJ, Evaluation of European Judicial Systems <http://www.coe.int/t/dghl/cooperation/
cepej/evaluation/default_en.asp> accessed on 20 April 2014.
CEPEJ,
Pilot-Scheme
for
Evaluating
Judicial
Systems
<http://www.coe.int/t/dghl/
cooperation/cepej/Source/GrillePilote_en.doc>accessed on 20 April 2014.
Commission, Guide to the Main Administrative Structures Required for Implementing the Acquis (Mai
2005)
<http://ec.europa.eu/enlargement/pdf/enlargement_process/accession_
process/how_does_a_country_join_the_eu/negotiations_croatia_turkey/adminstructures_version_
may05_35_ch_public_en.pdf> accessed 25 June 2014, 71.
European Parliament, Progress in EU-Croatia accession negotiations at a glance (5 November 2014)
<http://www.europarl.europa.eu/meetdocs/2009_2014/documents/dhr/dv/1129_07_/1129_07_en.pdf> accessed 2 July 2014.
European Union, Press Release: The Stabilisation and Association Agreement with Croatia enters into
force today (1 February 2005) <http://europa.eu/rapid/press-release_IP-05-122_en.htm> accessed 9
August 2014.
European Union, The Accession Process for a new Member State (28 February 2007) <http://europa.eu/
legislation_summaries/enlargement/ongoing_enlargement/l14536_en.htm> accessed 12 April 2014.
European
Union,
IPA
(7
September
2012)
glossary/terms/ipa_en.htm> accessed 12 April 2014.
<http://ec.europa.eu/enlargement/policy/
European Union, Instrument for Pre-Accession Assistance (IPA) (20 February 2012)
<http://europa.eu/legislation_summaries/agriculture/enlargement/e50020_en.htm> accessed 3 July
2014.
European Union Press Release: EU Justice Scoreboard: European Commission broadens the scope of its
analysis of Member States' justice systems (27 March 2013) <http://europa.eu/rapid/pressrelease_IP-13-285_en.htm> accessed 2 May 2014.
European Union Memo: Questions & Answers: EU Justice Scoreboard (27 March 2013)
<http://europa.eu/rapid/press-release_MEMO-13-288_en.htm> accessed 2 May 2014.
European Union, Chapters of the Acquis (27 June 2013) <http://ec.europa.eu/enlargement/
policy/conditions-membership/chapters-of-the-acquis/index_en.htm> accessed 12 April 2014.
European Union, Memo: The Multiannual Financial Framework: The External Action Financing
Instruments (11 December 2013) <http://europa.eu/rapid/press-release_MEMO-13-1134_en.htm>
accessed 14 April 2014.
European Union, Strategy and Progress Reports, (23 October 2013) <http://ec.europa.eu
/enlargement/countries/strategy-and-progress-report/index_en.htm>, accessed 10 June 2014.
European Union, Memo: EU-Montenegro: Inter-Governmental Conference (18 December 2013)
<http://europa.eu/rapid/press-release_MEMO-13-1182_de.htm> accessed 2 November 2014.
UN, The United Nations Rule of Law Indicators Project: Frequently Asked Questions (07 October 2008)
<http://en.unrol.org/files/FAQ%20-%207%20Oct%202008.doc>, accessed on 11 May 2014.
UN, United Nations Rule of Law Indicators Project: Newsletter Number 2 (June 2009)
<http://www.unrol.org/files/Newsletter%20for%20United%20Nations%20Rule%20of%20Law%20Ind
icators%20Project%20June%202009.pdf> accessed on 11 May 2014.
182
FRAME
USAID Center for Development Information and Evaluation, Performance Monitoring and Evaluation Tips:
Selecting Performance Indicators (1996) <http://pdf.usaid.gov/pdf_docs/PNABY214.pdf> accessed 20
June 2014.
183
FRAME
Annex III List of conducted interviews
Nr.
1
2
Date
May 2014
June 2014
Location/Mode
Telephone
Copenhagen
Identification of Interviewee(s)
EP - FEMM
EC - SPC
June 2014
Telephone
4
5
6
7
8
9
10
11
12
13
June 2014
June 2014
June 2014
June 2014
June 2014
June 2014
June 2014
June 2014
June 2014
June 2014
Brussels
Brussels
Brussels
Brussels
Brussels
Brussels
Telephone
Brussels
Brussels
Brussels
14
15
June 2014
June 2014
Telephone
Brussels
EC Indicators Subgroup of the

SPC
EC - DG DEVCO
EC - DG Home
EP - INTA
EP - LIBE
EP - DG JUST
Council - FREMP
EC - DG Enlargement
Council - COHOM
EP - DROI
EC - DG Strategy, Research and
International Affairs
EEAS
EEAS
16
17
18
June 2014
June 2014
July 2014
Telephone
Telephone
Vienna
EP - DROI
EC - DG Enlargement
FRA
Interviewer
Maddalena Vivona
Erik Andr Andersen
and Hans Otto Sano
Erik Andr Andersen
and Hans Otto Sano
Klaus Starl
Klaus Starl
Maddalena Vivona
Maddalena Vivona
Erik Andr Andersen
Hans Otto Sano
Markus Mstl
Klaus Starl
Maddalena Vivona
Hans Otto Sano
Markus Mstl
Erik Andr Andersen
and Hans Otto Sano
Markus Mstl
Markus Mstl
Klaus Starl
184
FRAME
Deliverable No. 13.1 Annex I
Annex I, List of Indicator Schemes and Human Rights Measurement Instruments
The following lists and tables are the outcome of an intensive mapping of datasets, indicators, aggregated indices and models for measuring human rights. The
mapping was open to all kinds of instruments giving information on human rights or human rights related issues. Out of more than 200 such initiatives, 130
were described in detail. The description encompasses the authors, whether they are intergovernmental, governmental, academic or civil-society based, the
topics, their relation to human rights, their target group and audience, the purpose and goals, data used, their methodology, an estimation of the data sources,
their status of elaboration and use and the frequency of application. The following list is a selection of 50 information tools. The selection already takes into
account the information requirements of EU bodies: they are applied, in use by EU bodies, are uncontested to a large extent and/or recommended as
information source by official documents and guidelines.
The list is structured into three chapters: Instruments dealing with human rights, instruments dealing with specific human rights including the rule of law and
instruments which are human rights related, i.e. give information on human rights in some aspects, even though not designed for this purpose (e.g. governance
indicators). Within the chapters, subchapters are devided into groups of authors: (inter)governmental and non-state actors. The order of listed instruments is
from global to local.
The mapping is up to date as at Dec 2014.
FRAME
Table of contents
A.
General Human Rights Measurement Instruments Using Indicators ................................................................................................................................................... 4

Intergovernmental Organisations ......................................................................................................................................................................................................... 4
OHCHR Human Rights Indicators ..................................................................................................................................................................................................................... 4
2.
Non-State and Academic Initiatives ....................................................................................................................................................................................................... 9
Cingranelli-Richards (CIRI) Human Rights Dataset .......................................................................................................................................................................................... 9
B. Specific Human Rights Measurement Instruments Using Indicators ................................................................................................................................................. 13
1.
Intergovernmental and Governmental Organisations ..................................................................................................................................................................... 13
FAO - The State of Food Insecurity in the World ............................................................................................................................................................................................ 13
Measuring Education as a Human Right........................................................................................................................................................................................................... 18
UNESCO EFA Education for All Development Index ..................................................................................................................................................................................... 21
Housing Indicators Program ............................................................................................................................................................................................................................. 23
WHO World Health Statistics ........................................................................................................................................................................................................................... 26
UNICEF The State of the World's Children (2013: Children with Disabilities) ............................................................................................................................................... 31
UNICEF Childinfo - Multiple Indicator Cluster Survey (MICS) programme .................................................................................................................................................. 34
UNICEF Indicators for Children in Formal Care ............................................................................................................................................................................................. 37
Measurement of Juvenile Justice Indicators ..................................................................................................................................................................................................... 41
WISTAT Women's Indicators and Statistics Database ..................................................................................................................................................................................... 44
Camp Indicator Report (refugees) .................................................................................................................................................................................................................... 50
UNESCO MDI Media Development Indicators: A framework for assessing media development .................................................................................................................. 53
Joint United Nations Programme on HIV/Aids ................................................................................................................................................................................................ 58
UN OHCHR Rule of Law Indicators ................................................................................................................................................................................................................ 62
European Commission for the Efficiency of Justice (CEPEJ), Evaluating Judicial Systems ........................................................................................................................... 66
Social Cohesion Indicators ............................................................................................................................................................................................................................... 70
Laeken Indicators - Open Method of Coordination for social protection and social inclusion (Social OMC) ................................................................................................. 73
Indicators for the Protection, Respect and Promotion of the Rights of the Child in the European Union ........................................................................................................ 76
The FRA indicators on the right to political participation of persons with disabilities..................................................................................................................................... 79
EIGE Gender Equality Index ............................................................................................................................................................................................................................ 82
Equality Measurement Framework and How fair is Britain .......................................................................................................................................................................... 85
How Fair is Britain ........................................................................................................................................................................................................................................... 89
2.
Non-State and Academic Initiatives ..................................................................................................................................................................................................... 92
Freedom in the World ....................................................................................................................................................................................................................................... 92
Freedom of the Press (Freedom House) ............................................................................................................................................................................................................ 96
Press Freedom Index (Reporters sans frontires) ............................................................................................................................................................................................ 100
CIVICUS Civil Society Index (CSI) ............................................................................................................................................................................................................... 104
1.
FRAME
Torture Scale ................................................................................................................................................................................................................................................... 110

WJP Rule of Law Index, VERA-Altus Indicators for the Rule of Law .......................................................................................................................................................... 113
VERA-Altus Indicators for the Rule of Law .................................................................................................................................................................................................. 118
RED ................................................................................................................................................................................................................................................................ 122
C. Human Rights-related Indicators ............................................................................................................................................................................................................ 128
1.
Intergovernmental and Governmental Organisations ................................................................................................................................................................... 128
Worldwide Governance Indicators (World Bank) .......................................................................................................................................................................................... 128
Millenium Development Goals Indicators ...................................................................................................................................................................................................... 135
International Human Development Indicators ................................................................................................................................................................................................ 140
World Bank Indicators, World Development Indicators ................................................................................................................................................................................ 147
KILM - Key indicators of the labour market .................................................................................................................................................................................................. 151
ILOSTAT Database ........................................................................................................................................................................................................................................ 155
Paris Declaration Framework for Aid Effectiveness ...................................................................................................................................................................................... 158
EU Justice Scoreboard .................................................................................................................................................................................................................................... 161
EU Enlargement Package: Strategy and Progress Reports ............................................................................................................................................................................. 164
EU Employment and Social Policy Indicators ................................................................................................................................................................................................ 167
Europe 2020 Strategy Indicators ..................................................................................................................................................................................................................... 172
Political Instability Task Force Report & Dataset .......................................................................................................................................................................................... 176
2.
Non-State and Academic Initiatives ................................................................................................................................................................................................... 179
Quality of Government Datasets ..................................................................................................................................................................................................................... 179
The Failed State Index .................................................................................................................................................................................................................................... 183
Minorities at Risk (MAR) & Minorities at Risk Organizational Behaviour (MAROB) ................................................................................................................................. 187
Global Peace Index (GPI) ............................................................................................................................................................................................................................... 190
Ibrahim Index of African Governance (IIAG) ................................................................................................................................................................................................ 194
Bertelsmann Transformation Index (BTI) ...................................................................................................................................................................................................... 198
D.
List of Human Rights Specific or Related Indicators ...................................................................................................................................................................... 201
FRAME
A.
General Human Rights Measurement Instruments Using Indicators
A human rights measurement system is an instrument which gives information on a dimension of human rights in respect to any given human rights standard.
Concretely, a human rights indicator scheme must be able to give information on respect, protection, fulfilment and promotion of the formal and substantive
content and scope of a human right. In order to know about human rights in any given context it is necessary that such measurement system provides
information on the commitment of a government and the factual enjoyment of the rights-bearers.
This chapters lists models which take a holistic approach and are valid for all human rights or the overall human rights situation. Only two instruments fulfil
these criteria.
1.
Intergovernmental Organisations
OHCHR Human Rights Indicators

Author
OHCHR
Type of author
IO
Thematic context
Human Rights
Give an overview of the thematic context, for which the indicator scheme is developed. (e.g. development aid, etc.)
Rational/purpose
- Monitoring human rights

- Inform policymakers
- Inform civil society,
FRAME
Describe the main rational/purpose of the indicator scheme. E.g. monitoring/ situational analysis/ inform public policy/ measuring progress or implementation/
increase accountability/ basis for allocation of funds/identify violations of state obligations/ early warning, etc.
Users/Target Group
- UN treaty bodies
- States (for reporting to treaty bodies)
but also:
- policymakers
- human rights practitioners
- development practitioners
Describe the target group of the indicators. Who is supposed to use the results of the indicators scheme? Specify if they are policy makers/ CSO/ monitoring
bodies/ media/ EU, etc.
Reference to human
rights
- explicit hr indicator scheme

- normative base: UDHR, ICCPR, ICESCR, ICERD, CEDAW, CAT, CRC ICRMW, CRPD, + comments and recommendations on them
- Right to life (UDHR, Art 3)

- Right to liberty and security of person (UDHR, Art. 3)
- Right to adequate food (UDHR, Art. 25)
- Right to enjoyment of the highest attainable standard of physical and mental health (UDHR, Art. 25)
- Right not to be subjected to torture or to cruel, inhuman or degrading treatment or punishment (UDHR, Art. 5)
- Right to participate in public affairs (UDHR, Art. 21)
- Right to education (UDHR, Art. 26)
- Right to adequate housing (UDHR, Art. 25)
- Right to social security (UDHR, Art. 22)
- Right to work (UDHR, Art. 23)
- Right to freedom of opinion and expression (UDHR, Art. 19)
- Right to a fair trial (UDHR, Art. 10-11)
List the human rights that are covered by the indicators. Sort by importance/main focus.
FRAME
13 rights, average of four attributes/right, 10 - 20 measures/indicator categorized into structural, process and outcome
measures
additional common indicators: population and general demographic trends, social and economic situation,
Types of indicators
- Average of four attributes/human right

- All indicators are categorized into structural, process and outcome indicators
- List of illustrative indicators on various rights can be found in the report
Geographical area
covered
International
Methodology
- Normative base (UDHR, UN hr treaties)

- narrative on the legal standard of the right was translated into a limited number of characteristic attributes that facilitate a
structured identification of appropriate indicators for monitoring the implementation
- structural-process-outcome indicators in order to assess States in meeting their obligations
Measures:
Respect, protection, fulfilment and promotion of human rights in a given context.
Briefly describe the methodology used for the development of the indicators. E.g. expert consultations, etc.
Data sources
General distinction between

- socio-economic and other administrative statistics
- events-based data on human rights violation
i.e.
- administrative records at UN Office of Legal affairs (for structural indicators)
- administrative records at the national and sub-national level
- administrative records maintained by local authorities, registration system for population data, records of health ministries
and household surveys, including Demographic and Health Surveys
- national human rights institutions,
- non-governmental organisations
- administrative agencies responsible for monitoring rehabilitation
- court records and reports of the office fo the prosecutor
FRAME
Describe the data sources that are supposed to be used for the indicator scheme. If possible, name them.
- socio-economic statistics (quantitative):
> quantitative information compiled and disseminated by the State, through its administrative records and statistical surveys,
usually in collaboration with national statistical agencies and under the guidance of international and specialized organizations;
> States' obligation to report
> importance of standardized methodology for transparency, credibility and accountability of monitoring
Type of data used
- events-based data (quantitative or qualitative):

> data on alleged or reported cases of human rights violations, identified victims and perpetrators
> usually reported by NGOs and are or can also be processed in a standardized manner by, for instance, national human rights
institutions and special procedures of the United Nations
> In general: underestimate incidence of violations and may prevent valid comparisons over time or across regions, yet it could
provide some indication
> most effectively used for monitoring the violation of civil and political rights only
Describe the type of data that are used for the indicator scheme. Particularly explain, if the data is qualitative or quantitative in nature.
Quality of data
providers/processors
- socio-economic statistics: states are supposed to provide data, depends on respective state whether data is reliable or not
- additional use of other sources improves overall quality
Describe who is supposed to provide data for the indicator scheme. Refer to the ECOI list to assess their quality.
Presentation of indicator
results
No special way of presentation: ratio, percentages, etc
Describe how the indicator results are presented. E.g., scores on a numerical scale; traffic light system etc.
Comparability
- Comparability is intended, comparability is criterium for selection of indicators

- Actual comparability depends on reliability of data, no checks and safeguards intended (as far as I can see): training,
workshops, cross-checks, reviews would be necessary to ensure similar methodology in generating the data
- events-based data usually underestimates the incidence of violations and, therefore, negatively affects comparability
FRAME
Status
Methodology, guides for practical use provided, incl. list of indicators for about 20 hr; practically applied by several institutions
and government, such as the FRA, Brazil etc.
Describe the status of implementation of the indicator scheme. E.g. theory developed only/ first applications made/ regular application/ failed. If the scheme
failed, please give reasons for failure.
- normatively based in UN HR framework + sophisticated methodology for developing indicators: translation of human rights
into attributes + search for appropriate indicators for attributes at all three levels (structural, process and outcome)
Comments
-> assessement/measurement of entire spectrum of respective human right
- emphasizes also structural and process indicators
Please add any further important remarks/comments regarding the indicator scheme here.
Link to website
http://www2.ohchr.org/english/bodies/icm-mc/docs/HRI.MC.2008.3EN.pdf; 28.08.2013
FRAME

2.
Non-State and Academic Initiatives
Cingranelli-Richards (CIRI) Human Rights Dataset

Author
Cingranelli-Richards (CIRI) Human Rights Data Project
Give name of the author of the indicators, i.e. name of organisation/ institution etc.
Type of author
NGO, Keele University School of Law
Describe the type of author, e.g. international or regional organisation/ state/ NGO/ monitoring body/ EU institution, etc.
Thematic context
HR measurement
Rational/purpose
- Measuring/analysis of human rights practices

- Provide database for researchers
- Inform policymakers & civil society
- Scholars and students who seek to test theories about the casues and consequences of hr violations
- Policy makers and analysts who seek to estimate the hr effects of a wide variety of institutions changes and public policies
Users/Target Group
Reference to human
rights
HR indicators
FRAME
Explain, whether the indicators refer to human rights explicity or only implicitly. Or, does the indicator scheme yield results that could be used for measuring
human rights?
Emphasis on civil and political rights (but also economic and gender rights)
- Physical integrity rights: rights not to be tortured, summarily executed, disappeared, or imprisoned for political beliefs
- Civil liberties: freedom of speech, freedom of association and assembly, freedom of movement, freedom of religion, and the
right to participate in the selection of government leaders.
- Workers' rights
- Rights of women to equal treatment (politically, economically, and socially)
- 17 government human rights practices
Give the number of indicators used in this indicator scheme.
Types of indicators
- Quantitative indicators
- Outcome indicators
Describe the type of indicators used in this indicator scheme. If possible, clarify whether the scheme makes use of structural, procedural and/or outcome
indicators as suggested by the work of the OHCHR.
Geographical area
covered
- 195 countries covered (33 added in 2004)
Exlpain, whether the indicator scheme is supposed to be applied on the local, regional, national, international, EU- level, number of countries, etc.
Methodology
- Expert coding (3 level score)

- code book (coding is explained, guidance for coders, transparency)
- evaluation of implementation of rights by coders, supposed to be objective, clear guidelines: See Coding Manual
- reliability depends on particular right: i.e. instances of torture are at least quantifiable, whereas it is very subjective to say that
women's representation is described as "few", "low"... Furthermore, weighing of different elements is subjective.
-1 Country/ 1 Year: Amount of violations of right: Value of 0 if more than 50, value of 1 if more than 0 but less than 50, value of
2 if no violation reported. (Mokken Scale Analysis)
- For physical integrity rights see Cingranelli and Richards, Measuring the Level, Pattern, and Sequence of Government Respect
for Physical Integrity Rights, 43 International Studies Quarterly 1999, 407, 409 et seq.
10
FRAME
Data sources
- Annual US Department of State's Country Reports on Human Rights Practices (see http://www.state.gov/j/drl/rls/hrrpt/)
- Amnesty International's Annual Report: only used for "Physical Integrity Rights" (rights to freedom from extrajudicial killing,
disappearance, torture, and political imprisonment) (see http://www.amnesty.org/en/library/aireport)
Type of data used
Quantitative data
Quality of data
- limited sources
- biased towards US and their "allies"? (US Country Reports are main data source)
- disaggregated measures of specific hr practices

- two aggregated indices
- absolute values
results
- percentages, rates,
- structured by country, by topic, aggregates for geographic regions, income groups,
Comparability
Comparison is possible
Annual application since 1981 (updated until 2011)

Give the year of the first application of the indicator scheme and explain how often the scheme was applied since then.
Regular application
Status
criticism by Wood, Reed M., Gibney, Mark, "The Political Terror Scale (PTS): A Re-Introduction and a Comparison to CIRI":
- "[] the CIRI claims a level of precision that is not possible given the source data from which both datasets are coded"
Comments
- "[] the CIRI's method of summing across abuse types leads to some inappropriate categorizations. For instance, the absence
11
FRAME

of one type of abuse prevents a state from being coded into a more repressive overall category regardless of the levels of other
types of abuse.
- CIRI doesn't account for "the "range" of violence committed by the state"
- three level scale cannot capture nuances
- some indicators are composed by many different components and evalued on scale from 0-3
(example from SIDA indicators report: In terms of credibility, an important problem is the broad indicator categories which
pertain to a few rights as illustrated by the case below: Womens Economic Rights provides an interesting example of the
problems of measurement. It encompasses:
- Equal pay for equal work
- Free choice of profession or employment without the need to obtain a husband or male relative's consent
- The right to gainful employment without the need to obtain a husband or male relative's consent
- Equality in hiring and promotion practices
- Job security (maternity leave, unemployment benefits, no arbitrary firing or Layoffs, etc...)
- Non-discrimination by employers
- The right to be free from sexual harassment in the workplace
- The right to work at night
- The right to work in occupations classified as dangerous
- The right to work in the military and the police force
Inherent in the in this case four level score from 0-3 are all of the sub-questions quoted above. These questions are all
interesting and relevant, but to collapse them into one score seems a challenge to credibility)
Link to website
http://www.humanrightsdata.com, 10.03.2015
12
FRAME
B.
Specific Human Rights Measurement Instruments Using Indicators
Specific human rights instruments and indicators are designed to measure one or more dimensions of a specific right.
1.
Intergovernmental and Governmental Organisations
FAO - The State of Food Insecurity in the World

Author
FAO Committee on World Food Security
Users/Target Group
Type of author
IO
Thematic context
- FAO monitoring
Rational/purpose
- inform policymakers (FAO and others) to make informed decisions concerning food and agricultural threats and emergencies
- inform civil society etc about state of malnutrition and hunger in the world
Users/Target Group
- policymakers
- civil society,
13
FRAME
Reference to human
rights
- initially based on and developed for 1996 World Food Summit goals, used by FAO World Food Insecurity reports
- recommendation of experts gathered in the Committee on World Food Security (CFS) Round Table on hunger measurement,
hosted at FAO headquarters in September 2011
human rights?
Right to Food
8 core indicators, 26 in total

INDICATORS:
- DETERMINANTS OF (INPUTS TO) FOOD INSECURITY (= structural conditions that are likely to worsen food insecurity in the
absence of adequate policy interventions):
1) Availability
> Average dietary supply adequacy (Core Indicator)
> Food production index (Core Indicator)
> Share of energy supply derived from cereals, roots and tubers
> Average protein supply
Types of indicators
> Average supply of protein of animal origin
2) Physical access
> Percentage of paved roads over total roads
> Rail lines density
> Road density
3) Economic access
> Food price level index (Core Indicator)
4) Utilization
14
FRAME

> Access to improved water sources
> Access to improved sanitation facilities
- OUTCOMES
1) Inadequate access to food
> Prevalence of undernourishment (Core Indicator)
> Share of food expenditure of the poor (Core Indicator)
> Depth of the food deficit
> Prevalence of food inadequacy
2) Utilization
> Percentage of children under 5 years of age who are stunted (Core Indicator)
> Percentage of children under 5 years of age who are wasted (Core Indicator)
> Percentage of children under 5 years of age who are underweight
> Percentage of adults who are underweight
VULNERABILITY/STABILITY (= vulnerability to possible future food insecurity)
> Domestic food price volatility (Core Indicator)
> Per capita food production variability
> Per capita food supply variability
> Political stability and absence of violence/terrorism
> Value of food imports over total merchandise exports
> Percentage of arable land equipped for irrigation
> Cereal import dependency ratio
Geographical area
covered
Methodology
- Most of the indicators are already being produced and published by FAO and other IO, other indicators have been introduced
to fill gaps in food security information systems (especially social-economic dimensions of food insecurity)
15
FRAME
Data sources
- FAO,
- International Road Federation,
- World Bank,
- WHO,
- UNICEF,
- World Governance Indicators
Type of data used
indicators are gathered from other authors, publications,

no comprehensive information on data
Quality of data
indicators are used from other authors (WB, UNICEF,), publications (FAO)
quality of data depends on their sources, methodology, etc
results
Excel Sheet, numerical values
Comparability
Comparability across regions and over the years is intended

no further information
Is a comparision of the indicator results possible/intended across different countries/regions etc.? Is such a comparision the purposes of the indicator scheme?
16
FRAME
First application in 2012
- Proposed in report 2012 (following the recommendation of experts gathered in the Committe on World Food Security (CFS)
Round Table on hunger measurement, hosted at FAO headquarters in September 2011),
- most of the indicators are used already, but not in a systematic manner (not as indicator system)
Status
Comments
two sets: 8 core indicators, which are supposed to show coherent picture, + 18 additional indicators for more comprehensive
analysis
Link to website
http://www.fao.org/economic/ess/ess-fs/fs-data/en/; 27.08.2013
17
FRAME
Measuring Education as a Human Right

Author
Right to Education Project (RTE)
Type of author
NGOs
Thematic context
right to education
Rational/purpose
Rational: The right to education recognised under human rights law has to be implemented at national level. States have the
obligation to protect, respect and fulfil the right to education. Purpose: Indicator scheme serves as a tool to evaluate States progress
towards the full realisation of the right to education, to identify violations of the right to education, and to enable civil society to hold
governments to account for their obligations regarding education.
Users/Target Group
The Committee on the Rights of the Child as well as other Treaty Bodies, States and civil society organizations
Reference to human
rights
Main international instruments on the right to education, such as the UNESCO Convention against Discrimination in Education
(CADE), International Covenant on Economic, Social and Cultural Rights (ICESCR), International Covenant on Civil and Political Rights,
Convention on the Rights of the Child, etc.
Human rights
covered
Right to Education, non discrimination, rights of the child
Number of indicators over 200
Types of indicators
Qualitative and quantitative indicators based on the 4A framework (Availability, Accessibility, Acceptability, Adaptability)
supplemented by governance indicators.
Qualitative governance indicators reflect the international legal framework.
Availability indicators examine whether education is generally available (different schooltypes are clustered, starting with early
childhood up to tertiary education).
18
FRAME

Accessibility indicators address obstacles in accessing education, such as economic, administrative or socio-cultural obstacles.
Acceptability indicators address the quality of teaching in terms of skills, tolerance and disciplining.
Adaptability indicators measure how the educational system is adapted to social problems outside the school, such as child labour,
child solders or armed conflicts.
In addition, three transversal issues are addressed alongside these indicators, namely: discrimination, participation and
accountability.
Geographical area
covered
worldwide
Methodology
Expert consultation for the development of an initial set of indicators; testing phase with bilateral and multilateral donor agencies,
UN bodies and agencies; a civil society phase to adapt the prototype indicators package for their use and also to identify case studies
to test and refine their use by civil society.
Data sources
UNESCO Institute for Statistics (UIS) Education Data, EFA Global Monitoring Report Data Search Tool, World Bank,
UNESCO/OECD/EUROSTAT (UOE) Database on Education, Education at a Glance 2014: OECD Indicators
Type of data used
quantiative data (disaggregated by vulnerable groups) for the 4A indicators and qualitative data for the governance indicators.
Quality of data
Secondary data should be used, collected by UNESCO Institute for Statistics (UIS), World Bank, OECD, EUROSTAT. Data should be
providers/processors collected by NHRIs and other actors at the national level.
Presentation of
indicator results
Online publication of indicators scheme and guidelines for use.

Depending on the kind of data required, responses
can take the form of percentages, ratios, yes or no answers, a choice in a list or open responses.
19
FRAME
Comparability
not addressed, since it is meant to be a flexible instrument for application. An "Indicators Selection Tool" is featured to support users
in the selection process
Frequency of
application
The scheme has been tested, but it is not clear if and how often the scheme was applied since then.
Status
An indicator scheme has been developed and tested. No further application so far is known.
Comments
The indicator scheme is a comphrehensive and userfriendly tool.

The RTE supports users in adapting these indicators at the national level and provides them with various guides and tools for usage,
e.g. on data collection and interpretation.
Link to website
http://www.right-to-education.org/resource/measuring-education-human-rights-list-indicators; 12.12.2014
20
FRAME
UNESCO EFA Education for All Development Index

Author
UNESCO
Type of author
IO
Thematic context
right to education
Rational/purpose
Assess global progress towards achieving the six 'Dakar' EFA goals, adopted by the World Education Forum in April 2000.
Users/Target Group
The results are targeted to policy-makers at the national and international level, and more broadly, at all those engaged in
promoting the right to quality education teachers, civil society groups, NGOs, researchers and the international community.
Reference to human
rights
Explicit reference to the right of education contained in the United Nations Convention on the Rights of the Child.
right to education; non-discrimination
4 main indicators, subdivided in over 30 indicators
Types of indicators
structural, process and outcome indicators are used
Geographical area
covered
205 countries around the world
21
FRAME
Methodology

The EFA Development Index is a composite index that provides a snapshot of overall progress towards Education for All. Due to
data constraints, the standard index captures only four of the six goals. The value of the standard EDI for a given country is the
arithmetic mean of four components:
- universal primary education (goal 2), measured by the primary adjusted net enrolment ratio;
- adult literacy (goal 4), measured by the literacy rate for those aged 15 and above;
- gender parity and equality (goal 5), measured by the gender-specific EFA index (GEI), an average of the gender parity indices
(GPIs) of the primary and secondary gross enrolment ratios and the adult literacy rate;
- quality of education (goal 6), measured by the survival rate to grade 5
Data sources
Type of data used
Data are provided by states through survey results reported to and processed by the UNESCO Institute for Statistics. For some
countries education data are collected by the UIS via surveys carried out under the auspices of the World Education Indicators
programme, or jointly by the UIS, OECD and the Statistical Office of the European Union (Eurostat) through the
UIS/OECD/Eurostat (UOE) questionnaires.
mainly quantitative data
Quality of data
The Index relies on strict statistical criteria combined with a solid theoretical framework. The data can be considered as reliable.
results
Comparability
Status
The overall percentage of countries projected to achieve a benchmark for five EFA goals by 2015 is presented in a table. More
detailed country data on each indicator are presented in tables at the end of the report.
Comparison across countries is intended and made possible.
published yearly since 2002
regularly applied
Link to website
http://www.unesco.org/new/en/education/themes/leading-the-international-agenda/efareport/; 11.12.2014
22
FRAME
Housing Indicators Program

Author
UN-HABITAT and World Bank (joint program)
Users/Target Group
Type of author
IO
Thematic context
- HR monitoring (mainly in developing countries)

- managing the housing sector
Rational/purpose
- Develop conceptual, analytical and institutional framework for managing the housing sector as a whole
Users/Target Group
- Policy makers
- Researchers
Reference to human
rights
- No information provided
Explain, whether the indicators refer to human rights explicitly or only implicitly. Or, does the indicator scheme yield results that could be used for measuring
human rights?
Right to housing
23
FRAME
25 key indicators, + alternative indicators:

25 key indicators are assigned to 11 modules
- Coordinator Module (The House-Price-to-Income Ratio, The Rent-to-Income Ratio)
- Demographic Module (Households Per Dwelling Unit OR Urban population growth + City population)
- Financial Module (The Housing Credit Portfolio, The Credit-to-Value Ratio,...)
- Government Budget Module (Housing Subsidies, Targeted Subsidies)
- Housing Quality Module (Homelessness, Floor Area Per Person, Permanent Structures, Water Connection, Unauthorized
Housing, Residential Mobility, The Vacancy Rate, Owner-Occupancy)
- Local Government Module (Rent controls, Property Tax Receipts,..)
Types of indicators
- Building Regulations Module (Permits Delay, Land Subidvision Controls,...)
- Transportation Module (Jouney to Work)
- Real Estate Development Module (Housing Production, Housing Investment, The Housing Price Index, The land Development
Multiplier, Infrastructure Expenditures Per Capita, Industrial Concentration)
- Construction Industry Module (Construction Cost, Import Share in Construcion, The Skill Ratio)
- Economic Statistics Module (socio-economic indicators)
- Regulatory Audit
Geographical area
covered
53 cities in 53 countries
Methodology
- No special information provided

- quantitative indicators based on quantitative data: calculation
24
FRAME
- Existing data: household survey and associated socio-economic data already existed for 10-12 countries that had been studied
by the World Bank
- Intensive Surveys of the Housing Sectors: few countries, values for a comprehensive set of approximately 150 indicators
Data sources
- Extensive Surveys of the Housing Sectors: in one major city in each of 35-40 countries, 25 key indicators, 25 alternate
indicators, 10 indicators of broad socio-economic performance related to housing sector outcomes
- Quantitative data
Type of data used
- Survey
Quality of data
- Extensive Surveys: depends upon quality/skills of respective in-country expert
- Intensive Surveys: performed by UN-Habitat experts: high quality expected
results
- no special presentation: rates, percentages, numerical values,

- tables with indicators
- monograph "Housing Indicators for Policy making", research papers,
Comparability
intended, regional workshops are implemented, inter alia to ensure comparability
1990, few years, for 53 cities in 53 countries
Status
Failed due to the difficulty and cost in obtaining data
25
FRAME
WHO World Health Statistics

Author
World Health Organization
Type of author
UN Agency
Thematic context
WHO is the directing and coordinating authority for health within the United Nations system. It is responsible for providing
leadership on global health matters, shaping the health research agenda, setting norms and standards, articulating evidence-based
policy options, providing technical support to countries and monitoring and assessing health trends. WHO is financed by member
states and other contributers.
Rational/purpose
this statistics covers 194 states and aims to to track progress related to achieve the health-realted Millenium Development Goals and
associated targets
Users/Target Group
the indicator scheme is first and foremost meant to be used by the WHO itself, furthermore it can also be used by member states to
shape their policy in the direction of achieving the MDGs
Reference to human
Right to health
rights
human rights?
26
FRAME
Human rights
covered
Human rights covered are i.e. right to life, right to health,
Number of
indicators
there are nine major groups of indicators

the nine main groups of indicators are the following:
I. life expectancy and mortality
(life expectancy at birth; life expectancy at age 60; stillbirth rate; neonatal mortality rate; infant mortality rate; under-five mortality
rate; adult mortality rate)
II. cause-specific mortality and morbidity
(age-standardized mortality rates by cause; number of deaths among children aged <5 years; distribution of causes of death among
children <5 years, age-standardized adult mortality rate by cause; maternal mortality ratio; cause-specific mortality rate; incidence
rate; revalence)
III. selected infectious diseases
(cholera, diphteria, H5N1 influenza, Japanese encephalitis, leprosy, malaria, measles, meningitis, mumps, pertussis, plague,
polimyelitis, congential rubella syndrome, rubella, neonatal tetanus, total tetanus, tuberculosis, yellow fever)
IV. health service coverage
Types of indicators
(unmet need for family planning; contraceptive prevalence, antenatal care coverage, births attended by skilled health personnel,
births by caesarean section, postnatal care visit within two days of childbirth, neonates protected at birth against neonatal tetanus,
immunization coverage among 1-year-olds; children aged 6-59 months who received vitamin A supplemention; children aged <5
years with ARI symptoms taken to a health facility; children aged <5 with ARI symptoms receiving antibiotics; children aged <5years
with diarrhoea reveing ORT; children aged <5 years sleeping under insecticide-treated nets; children aged <5 years with fever who
received treatment with any antimalarial; pregnant women with HIV receiving antiretrovirals to prevent MTCT; antiretroviral therapy
coverge among people with advanced HIV infection; case-detection rate for all forms of tuberculosis; treatment-success rate for
smear-positive tuberculosis)
V. risk factors
(population using improved drinken water sources; population using improved sanitation; population using solid fuels; preterm birth
rate; infancts exclusively breastfed for the first 6 months of life; children aged <5 years who are wasted; children aged <5 years who
are stunted; children aged < 5 years who are underwight; children aged < 5 years who are overweight; prevalence of raised fasting
27
FRAME

blood glucose among adults aged > 25 years; revalence of raised blood pressure among adults aged > 25 years; adults aged > 20
years who are obese; alcohol consumption among adults aged > 15 years; prevelance of smoking any tobacco product among adults
aged >15 years; prevalence of current tobacco use among adulescents aged 13-15 years; prevalece of condom use by adults aged
15-49 years during higher-risk sex; population aged 15-24 years with comprehensive correct knowledge of HIV/AIDS)
VI. health systems
(physicians per 10000 population; nurins and midwifery personnel, dentists per 10000 populaiton, pharmacists, environment and
public halth professionsl; community health workers; psychiatrists per 10000 population; hospitals per 100,000 population; hospital
bed per 10,000 populaiton; psychiatric beds per 10,000 population; computed tomography units per million population; radiotherapy
units per million population; media availability of selected generic medicines in public and private sectors; median consumer price
ratio of selected generic medicines in public and private sectors)
VII. health expenditure
(total expenditure on health as a percentage of gross domestic product; general government expenditure on health as a percentage
of total expendirute on health; private expenditure on health as a percentage of total expenditure on health; general government
expenditure on health as a percentage of total government expenditure; external resources for health as a percentage of total
expenditure on health; social security expenditure on health as a percentage of general government expenditure on health; out-ofpocket expenditure as a percentage of private expenditure on health, privat prepaid plans as a percentage of private expenditure on
health; per capita total expenditure on health at average exchange rate; per capita total expenditure on health, per capita
government expenditure on health at average exchange rate; per capita government expenditure on health)
VIII. health inequities
(contraceptice prevalence: modern methods; antenatal care coverage: at least four visits; births attended by skilled health personnel;
DTP3 immunization coverage amont 1-year-olds; children aged <5 years who are stunted; under-five mortality rate)
IX. demographic and socioeconomic statistics
(total population, median age of population, population aged <15 years, population aged > 60 years; annual population growth rate;
population living in urban areas; civil registration coverage of births and causes of death; crude birth rate; brude death rate; total
fertility rate; adolescent fertilty rate; literacy rate among adults aged >15 years; net primary school enrolement rate; gross national
income per capita; population living on < 1$ a day; cellular phone subscribers per 100 population)
The type of indicators varies according to the indicator at stake; some are outcome indicators, some are process indicators
Geographical area
covered
this statistics covers 194 states, this means it is applied globally
Methodology
the indicator scheme was developed through expert meetings at the WHO
28
FRAME
Data sources
data is based on statistical and household surveys, adminstrative data as well as socioeconomic data and census, death-registration
records, special studies on deaths due to HIV and to conflict, immunization coverage rates for vaccine-preventable diseases;
questionnaires on health service use (like the UNICEF Multiple Indicator Cluster Survey, the Demographic and Health Survey, country
health and economic surveys), labour-force and employment surveys, health-facility assessments, routine administrative information
systems, surveys conducted using WHO/Health Action INternational standard methods, the National Health Accounts (NHAs).
Type of data used
the data is of a quantitative nature, but the quality of data provided by each data source has to be taken into account too when
comparing the information of the indicator scheme
Among the many data providers is the Inter-agency Group for Child Mortality Estimation (IGME), as well as WHO programmes and
Quality of data
the WHO itself, estimates conducted by other UN agencies (like UNICEF, the United Nations International Telecommunication Union,
providers/processors the United Naitons Departement of Economic and Social Affairs, the United Nations Educational, Scientific and Cultural Organization,
the World Bank) or the countries themselves,
Presentation of
indicator results
the results of the indicators are presented in a table which makes a cross-national comparison easier
29
FRAME
Comparability
it is intended to be applied cross-national, therefore a comparabilty is considered, although one has to be careful with this, since not
all the 194 states observed in this statistics have the means to provide sufficient data
Frequency of
application
first year of application was in 2005, since then it was applied every year
Status
this indicator system is applied annually
it has to be taken into account that not all the data is very high in quality, since some states may have an unsufficient way of
gathering this certain type of information, i.e. due to a lack of registration of birth/mortality rates etc. or states are too weak to
Comments
produce reliable data on this issues
Link to website
http://www.who.int/gho/publications/world_health_statistics/en/index.html; 10.10.2013
30
FRAME
UNICEF The State of the World's Children (2013: Children with Disabilities)
Author
UNICEF
Users/Target Group
Type of author
UN Programme
Thematic context
Children as the most vulnerable group in our society; especially facing exclusion and discrimination when being disabled
Rational/purpose
Measuring progress concerning rights of disabled children, as well as disclosing still existing gaps between laws and reality
Users/Target Group
UNICEF itself, other UN agencies, policy makers, media
Reference to human
The Convention on the Right's of a Child, The Convention on the Rights of Persons with Disabilities and Optional Protocols
rights
human rights?
Human rights
covered
Human Rights covered by the before mentioned conventions
31
FRAME
This survey has its focus on 13 main indicators (basic indicators, nutrition, health, HIV/Aids, Education, Demographic Indicators,
Number of
Economic Indicators, Women, Child Protection, Adolescents, Disparities by Residence, and The Rate of Progress and Equity)
i.) These main indicators are subdivided into more concrete indicators, which aim to make clear the main categories of indicators.
indicators
ii.) The number of indicators varies from 4 - 18 specific indicators for main categories of indicators
Types of indicators
according to the OHCHR Report outcome indicators are used (e.g. exclusive breasfeeding <6 months, overweight, antimalarial
treatment among febrile children, youth literacy rate, etc.)
Geographical area
covered
applied internationally
Methodology
Consultations with experts groups, co-operation with the Washington Grop on Disability Statistics, national statistical offices and data
collection agencies, academics, practitioners, disabled people's organizations, and other stakeholders
Data sources
depends on the indicator

typical data is gathered by UNICEF itself, the World Bank, ODA, UNESCO Institute for Statistics UIS, WHO, UNFPA, and other agencies
of the United Nations, MICS, DHS and other national surveys.
Type of data used
Quantitive Data
32
FRAME
Quality of data
data is drawn on inter-agency estiamtes, nationally representative houshold survey (e.g. MICS, DHS), as well as from other UN
providers/processors organizations
Presentation of
indicator results
data is presented through charts, respectively through lists
Comparability
Yes, a comparison is intended
Frequency of
application
data is conducted annually, first applied 1996
Status
since 1996 it is applied regulary
Comments
This analysis only refers to the State of the World's Children 2013. Children with Disabilites, but it has to be taken into account that
each year another main focus is laid out by this survey
Please consider following website for further information http://www.unicef.org/sowc/index_sowc.html
Link to website
http://www.unicef.org/sowc2013/files/SWCR2013_ENG_Lo_res_24_Apr_2013.pdf; 30.09.2013
33
FRAME
UNICEF Childinfo - Multiple Indicator Cluster Survey (MICS) programme

Author
UNICEF
Type of author
UN Programme
Thematic context
Monitoring the Situation of Children and Women
Rational/purpose
- Indicators for Children's and Women's Rights

- support of countries to collect data
Users/Target Group
- used as support for national reporting for i.e. MDG

- policy makers, NGOs,
- data for UNICEF reports
Reference to human
rights
based on UNCRC
human rights?
Children's Rights, health, education,

~ 100
- Indicators are categorized by area: Child Survival and Health, Child Nutrition, Maternal Health, Newborn Care, Water and
Types of indicators
Sanitation, Education, Child Protection, Child Disability,
34
FRAME

(See http://www.childinfo.org/statsbyarea.html)
- MICS4 indicators:
MORTALITY (5 indicators): i.e. Under-five mortality rate, infant mortality
NUTRITION (19 indicators): i.e. Vitamin A supplementation, low-birthweight infants
CHILD HEALTH (21 indicators): i.e. Malaria diagnostics usage, child at increased risk of disability
WATER AND SANITATION (6 indicators): i.e. Place of handwashing, water treatment
REPRODUCTIVE HEALTH (13 indicators): i.e. Content of antenatal care, institutional deliveries
CHILD DEVELOPMENT (7 indicators): i.e. Father`s support for learning, Early child development index
LITERACY AND EDUCATION (10 indicators): i.e. school readiness, transition rate to secondary school
CHILD PROTECTION (14 indicators): i.e. child labour, polygyny
HIV/AIDS, SEXUAL BEHAVIOUR AND ORPHANS (21 indicators): i.e. Comprehensive knowledge about HIV prevention, HIV testing
during antenatal care
ACCESS TO MASS MEDIA AND USE OF INFORMATION/COMMNICATION TECHNOLOGY (3 indicators): i.e. use of internet
SUBJECTIVE WELL-BEING (3 indicators): i.e. life satisfaction
TOBACCO AND ALCOHOL USE (4 indicators): i.e. alcohol use befor age 15
Geographical area
covered
International (nearly 200 MICS surveys in approximately 100 countries )
Methodology
- MICS (survey) rounds, last round: MICS 4 2009-2010

- Core Questionnaires: Household Questionnaire, Questionnaire for Individual Women, Men, Children Under Five,
- Child Disability Questionnaire,
- Vaccinations at Health Facility Questionnaire
- UNICEF provides technical support and training through a series of regional workshops covering questionnaire content,
sampling and survey implementation, data processing, data quality and data analysis, report writing, data archiving, and
dissemination
Data sources
UNICEF maintains a series of global databases for tracking the situation of children and women globally. The databases include
only statistically sound and nationally representative data from household surveys and other sources. They are updated
35
FRAME
Type of data used

annually through a process that draws on the wealth of data maintained by UNICEFs wide network of 140 field offices
(http://www.childinfo.org/statistical_tables.html)
- Quantitative data
results
- Indicators are used in flagship publications "Teh State of the World`s Children" and "Progress for Children"
- Main source for MDG goals concerned with children's rights
- No special mode of presentation: percentages, numbers,
- intended to be comparable
- safeguards to ensure comparability: UNICEF provides technical support and training through a series of regional workshops
Comparability
covering questionnaire content, sampling and survey implementation, data processing, data quality and data analysis, report
writing, data archiving, and dissemination
4 waves completed, 5th survey in progress
Status
Regular application
Link to website
http://www.childinfo.org/index.html; 11.09.2013
36
FRAME
UNICEF Indicators for Children in Formal Care

Author
UNICEF
UN Programme
Type of author
Thematic context
Monitoring of Formal Care Systems for Children
Rational/purpose
Generating data for monitoring of progress, preventing separation, promoting re-unification and ensuring provision of
appropriate alternative care of children in formal care
Monitor policy and practice improvements at the level of individual care services and at the
national level;
Help governments, child welfare agencies and child advocates to identify the needs of
children in formal care;
Provide policy makers and managers with information to guide programme development and
budgeting;
Support advocacy to improve systems and services for children at risk or in alternative care;
Increase the visibility and status of those engaged in the provision of formal care; and
Demonstrate national commitment to globally accepted measures of formal care.
Users/Target Group
- individual childcare agency: to help analyse and

improve their childcare practice,
- district government oversight office: to monitor and improve
the childcare system in a specific area
- national government bodies
- NGOs, childcare agencies,...
37
FRAME
Reference to human
rights
- UNCRC
- UN Guidelines for the Appropriate Use and Conditions of Alternative Care for Children
human rights?
Children's Rights
15 indicators (4 core indicators)
- Quantitative Indicators: Indicators 1-12 (numerical information about children in formal care)
- Policy/Implementation Indicators: Indicators 1315 (provide descriptive information about laws, policies and practice relevant
to children in formal care)
- Quantitative Indicators:
1 (Core) - Children entering formal care Number of children entering formal care during a 12-month period per 100,000 child
population
2 (Core) - Children living in formal care Number of children living in formal care on a given date per 100,000 child population
3 (Core) - Children leaving residential care for a family placement
Proportion of all children < 15 years leaving residential care for a
family placement, including reunification, in a 12-month period
4 (Core) - Ratio of children in residential versus family-based care
Proportion of all children in formal care who are currently
Types of indicators
accommodated in non-family-based care settings
5 - Number of child deaths in formal care
Number of child deaths in formal care during a 12-month period
per 100,000 children in formal care
6 - Contact with parents and family Percentage of children in formal care who have been visited by or visited their parents, a
guardian or an adult family member within the last 3 months
7 - Existence of individual care plans Percentage of children in formal care who have an individual care plan
8 - Use of assessment on entry to formal care (gatekeeping)
Percentage of children placed in formal care through an established assessment system
9 - Review of placement Percentage of children in formal care whose placement has been reviewed within the last 3 months
10 - Children in residential care attending local school
38
FRAME

Percentage of children of school age in residential care who are
attending school within the local community with other children
who are not in residential care
11 - Staff qualifications Percentage of senior management and staff/carers working with children in formal care with minimum
qualifications in childcare and development
12 - Adoption rate Rate of adoptions per 100,000 child population
- Policy/implementation indicators - Qualitative Indicators
13 - Existence of legal and policy framework for formal care
14 - Existence of complaints mechanisms for children in
formal care
15 - Existence of system for registration and regulation
Geographical area
covered
Methodology
Data sources
International
- Calculation of indicators based on quantitative data
- All indicators are to be disaggregated along the following categories: Sex, Age, Ethnicity, Disability status, Type of formal care
setting, Family placements, Country of origin, Category of staff, Category of adoption
- individuals or institutions responsible for initially placing children in formal care settings: social work departments, courts of
law, police, military forces, religious institutions, heads of formal care services
- census data,
- residential care institutions (orphanages, children's home, transit centres,...)
Type of data used
- quantitative data for Indicators 1-12 generated by national authorities, (see Data sources)
- qualitative data for indicators 13 - 15 (information about legal framework, policy framework, complaints mechanism,)
Quality of data
- main sources are national authorities and national institutions --> quality differs highly
39
FRAME
results
- No special mode of presentation

- Percentage, numbers, qualitative information
Comparability
- aims to develop comparable indicators, but no special safeguards to ensure this comparability
Not regularly applied, framework for application by national authorities etc.
Status
theory developed, no information about applications
In most countries around the world, the majority of children not living with their own parents are being cared for by extended
family members, relatives or others through informal arrangements.
A much smaller number of children live outside all forms of care, on the streets or in situations of economic exploitation. These
Comments
indicators do not cover these more complex situations. However, it
should be noted that the situation of children in informal care arrangements can be monitored through other means.
Link to website
http://www.crin.org/BCN/FormalCareIndicators.asp
http://www.unicef.org/protection/Formal_Care20Guide20FINAL.pdf; 11.09.2013
40
FRAME
Measurement of Juvenile Justice Indicators

Author
United Nations Office on Drugs and Crime, UNICEF
Type of author
United Nations Agency
Thematic context
UNODC helps Member States in their struggle against illicit drugs, crime and terrorism, as well as to fight transnational crime.
As a heads-up: UNODC relies on voluntary contributions, mainly from Governments, for 90 per cent of ist budget
Rational/purpose
The purpose of this indicator scheme is the following: When government officials and the institutions making up the juvenile
jusitce system do not have information either about the functioning of the system or the children who are in contact with it,
abuse, violence and exploitation can occur with impunityk, and the experience of the child is unlikely to be in his or her best
interests. therefore the rationl lays in guaranteeing the welfare and protection of the interests of children in conflict with the
law
Users/Target Group
The results of the indicators are supposed to be used by local institutions and national policy makers, to identify certain areas
for improvement or reform, as well as helping States in adhering to international standards
Reference to human
rights
yes, there is a connection to human rights
human rights?
41
FRAME
human rights covered in this indicator scheme refer to the CRC
there are fifteen indicators, dealing with juvenile justice
Types of indicators
I. QUANTATIVE INDICATORS:
1. Children in conflict with the law (Number of children arrested during a 12 month period per 100,000 child population)
2. Children in detention (Number of children in detention per 100,00 child population)
3. Children in pre-sentence detention (Number of children in pre-sentence detention per 100,000 child population)
4. Duration of pre-sentence detention (Time spent in detention by children before sentencing)
5. Duation of sentence detention (Time spent in detention by children after sentencing)
6. Child deaths in detention (Number of child deaths in detention during a 12 month period, per 1,00 children detained)
7. Separation from adults (Percentage of children in detention not wholly separated from adults)
8. Contact with parents and family (Percentage of children in detention who have been visited by, or visited, parents, guardian
or an adult family member in the last 3 months)
9. Custodial sentencing (Percentage of children sentenced receiving a custodial sentence)
10. Pre-sentence diversion (Percentage of children diverted or sentenced who enter a pre-sentence diversion scheme)
11. Aftercare (Percentage of children released from detention receiving aftercare)
II. POLICY (Structural) INDICATORS
12. Regular independent inspections (Existence of a system guaranteeing regular independent inspection of places of detention;
Percentage of places of detention that have received an independent inspections visit the last 12 months
13. Complaints mechanism (Existence of a complaints systems for chilren in detention; Percentage of places of detention
operating a complaints system)
14. Specialised juvenile justice system (Existence of a specialised juvenile justice system)
15. Prevention (Existence of a national plan for the prevention of child involvement in crime)
Geographical area
covered
indicator system is supposed to be applied internationally
Methodology
work was initiated at UNICEF, an expert group worked more that six months to select the final indicators; the indicators scheme
was tested in the Phillippines by the Bureau of Jail Management and Penology and the Departement of Social Welfare and
42
FRAME
Type of data used

Quality of data

Development
quantitative data is used for the indicator system
Data can be provided by detention centres, such as police stations, prisons, remand homes and educational or rehabiliation
institutions, as well as local magistrates or local district offices of the public prosecutor. Moreover data shall be gathered from
governmental ministires (e.g. ministries of justice, interior, home affairs or penal management)
results
the results of the indicators are supposed to be presented in a very simple way; graphs and figures shall be used in doing so
Comparability
yes, the indicator results are intended to be compared betwen the countries
there is no information that this certain indicator scheme is applied regulary, but there have been reports on the juvenile justice
system in certian countries (e.g. Afghanistan, South-East Europe, Lebanon)
Status
a field-test was conducted in the Phillippiens in 2006 to prove the scheme working
Comments
i. indicator system is supposed to be applied internationally, but a comparison can also be made on a more regional or local
level
ii. for the usefulness of the indicators it is important to keep the
particular context of each country in mind
iii. some data has to be gathered over a period of 12 months
Link to website
http://www.unodc.org/pdf/criminal_justice/06-55616_ebook.pdf; 07.10.2013
43
FRAME
WISTAT Women's Indicators and Statistics Database

Author
United Nations Statistics Divsion
Type of author
UN Programme
Thematic context
The United Nations Statistics Division is committed tothe advancement of the global statistical system; it develops standards
and norms and supports countries' efforts to strengthen their national statistical systems
Rational/purpose
WISTAT is a global database of statistics and indicators on gender, population and social development; its aim is to monitor the
actual situation for women all over the world
Users/Target Group
Reference to human
rights
Yes, there is a direct reference to human rights (e.g. right to education, right to health, right to private life etc.)
human rights?
Human rights covered in this indicator system are first and foremost reflecting the rights und the CEDAW
44
FRAME
there are nine main subjects, all of them contain sub-categories

all in all there are about 60 indicators

1. POPULATION COMPOSITION AND DISTRIBUTION
- Estimated and projected population by age and sex
- Total, urban and rural population by sex
- Foreign-born population by sex
- Average annual rate of population growth
2. LEARNING AND EDUCATION
- Illiteracy by sex and age group, for total, urban and rural areas
- Estimates and projections of adult illiteracy, by sex
- Enrolment at first level by sex
- Enrolment at second level by sex
- Enrolment at third level by sex
- Net enrolment ratio by level of education and sex
- Gross enrolment ratio by level of education and sex
- Third level enrolment by field of study and sex
- Third level students per 100,000 population by sex
Types of indicators
- Teachers at first level by sex
- Teachers at second level by sex
- Teacher at third level by sex
- School life expectancy, by sex
- Educational attainment of population 25+ by sex, for total, urban and rural areas
- Total public expenditure on education as percentage of GNP and all public expenditure
- Number of circulation of daily newspapers
- Number of radio receivers per 1000 population
- Number of television receivers per 1000 population
3. ECONOMIC ACTIVITY
- Estimated and projected economic activity rate by sex and age group
- Population, economically active population and economic activity rate, by sex and age group
- Total, economically active and not economically active population by sex, for total, urban and rural
- Female economically active population by marital status and age group
45
FRAME

- Economically active population by status in emplyment and sex
- Economically active population by sex and major occupation group
- Economically active population by sex, status in employment and major industry group
- Estimated economically active population by sex and branch of economic activity
- Unemployment rate by sex
- Women's wages in agriculture, in non-agricultural acitvities and in manufacturing as a percentage of men's wages
- Maternity benefits
4. HOUSEHOLDS; MARITAL STATUS AND FERTILITY
- Population by sex, age group and marital status
- Singulate mean age at marriage by sex
- Percentage ever married among persons ages 15-19, by sex
- Estimated and projected total fertility rate
- Estimated age-specific fertility rates
- Age-specific fertility rate for total, urban and rural areas
- Houshold headship by sex and age group
- Number of households and average household size, total, urban and rural areas
- Femal population by age and number of children born alive
5. HEALTH AND HEALTH SERVICES
- Estimated and projected life expectancy at birth, by sex
- Life table survival values at selected ages, by sex (100,000)
- Estimated and projected infant mortality rate (per 1000 live births)
- Child mortality rates by sex
- Deaths by cause and sex
- Percentage of population with safe drinking water, for total, urban and rural areas
- Percentage of population with adequate sanitation facilites, for total, urban and rural areas
- Child malnutritiion indicators by sex
- Smoking prevalence among persons 15 years of age and over, by sex
- Estimated number of people living with HIV/Aids, numbers of Aids cases and Aids deaths
- Reported cumulative Aids cases and their % distribution by sex and by mode of transmission
6. REPRODUCTIVE HEALTH AND REPRODUCTIVE RIGHTS
- Estimated maternal mortality ratio
- Contraceptive prevalence among currently married women reproductive age, by method used
- Contraceptive prevalence among currently married women of reproductive age, by method used and urban/rural residence
46
FRAME

- Percentage distribution of births by person assisting mother at the time of delivery, for total, urban and rural areas
- Percentage of infants breastfed
- Percentage of births delivered by C-section
- Percentage of women currently married or in union with unmet need for family planning, for total, urban and rural areas
- Percentage of women who had given birth by age 20
- Percentage of women aged 40-44 who have not had a live birth
- Percentage of women requiring more than an hour or two hours travel time to the nearest facility providing family planning
services or supplies
- Circumstances under which abortion is permitted
- Women and men's expressed ideal number of children, for total, urban and rural areas
- Estimated percentage of pregnant women with antenatal care, deliveries in health facilities and deliveries attended by skilled
personnel
7. PUBLIC AFFAIRS AND POLITICAL PARTICIPATION
- Years in which women acquired the right to vote and to stand for election
- Distribution of seats in parliamentary assemblies by sex
- United Naitons staff in posts subject to geographical distribution, by nationality and grade,total and female
- Professionsl staff of the organization in the UN common system by grade, total and female
- Distribution of ministerial posts by sex and ministry
- Date of ratification and entry into force of CEDAW
8. VIOLENCE
- Prevalence of violence against women by an intimate partner
- Deaths from external causes, by sex, ages and cause of death
9. NATIONAL PRODUCT AND EXPENDITURE
- Total and per capita gross domestic product (GDP) in US dollars
- Average annual growth rate of GDP
- Percentage distribtution of government social security benefit expenditure by branch of social security
data is on the one hand of a structural nature (i.e. Date of ratification and entry into force) on the other hand the gross of
indicators is of an outcome nature
47
FRAME
Geographical area
covered
the indicator system is intended to be applied globally (more concrete it is applicable for 206 countries)
Data sources
administrative data, which is gathered by i.e. civil registration offices, ministries, local authorities, local schools etc.) as well as
statistical surveys, sometimes even censuses
Type of data used
the data in that case is of a quantitative nature
Quality of data
since most of the data relies on the government or government-near institutions it might be difficult to access this data;
moreover one has to be careful about the objectivity respectively subjectivity of the data providers themselves
results
the results of the indicators are presented as excel charts and tables, which are published in annual reports
statistical tables should be accompanied by a clear explanatory text and, if possible, analysis
to highlight the differences between countries it is desirable to use figures, maps and graphs
Comparability
this indicator scheme is intended to be compared between states
many indicators were applied in 1970 for the first time
48
FRAME
Status
this indicator scheme is applied regularly; especially for the period 1970 - 1997
Comments
Fee to access
Link to website
http://unstats.un.org/unsd/demographic/gender/wistat/; 07.10.2013
49
FRAME
Camp Indicator Report (refugees)

Author
UNHCR
Type of author
UN Agency
Thematic context
Monitoring the well-being of refugees and persons of concern
Rational/purpose
to ensure effective protection and quality assistance to refugees, person of concern and the implementation of durable solution
via monitoring
Users/Target Group
UNHCR, policy maker and other organizations (e.g. NGOs)

Reference to human
rights
1951 Refugee Convention, ICESCR, CRC, CEDW
human rights?
Rights under the Geneva Convention, Right to Shelter etc.
50
FRAME
71

MAIN CATEGORIES OF INDICATORS:
I. Protection
II. Community Services/ Development
III. Food, Nutrition, Health and HIV and Aids
IV. Water, Sanitation, Shelter&Physical Planning, Environment
Types of indicators
V.Education
VI. Economic Security
VII. Durable Solutions
an overall use of outcome indicators
Geographical area
covered
International
Methodology
consultation among different UNHCR sections, as well as the Regional Bureaus and Field Offices, and other organizations
Data sources
i. Administrative registers (e.g. OFFICIAL CAMP REGISTRATION for refugees)

ii. Other administrative sources
iii. Surveillance
iv. Houshold surveys
v. Census
Type of data used
quantitative indicators complemented with additional information from qualitative indicators
51
FRAME
Quality of data
UNHCR, implementing partner, host government authorities, UN agencies and NGOs
results
Presenting data by using TRENDS, TABULATIONS, MAPS, EXCEL-CHARTS

Presentation through REPORTS
i. Situation Report (Sitreps)
ii. Annual Programme Interim Report
iii. Country Report
Comparability
indented to be compared, comparisons between countries
varies from annually to monthly

depends on the type of indicator
Status
Regular Application
UNHCR established further indicators, especially for measuring the implemention of human rights for the Country Level, Urban
Programme, Returnee Area
Comments
Please consider therefore the following website http://www.unhcr.org/40eaa9804.html for further information
Link to website
http://www.unhcr.org/40eaa9804.html; 27.09.2013
52
FRAME
UNESCO MDI Media Development Indicators: A framework for assessing media development
Author
UNESCO United Nations Educations, Scientific and Cultural Organization
Type of author
UN Agency
Thematic context
Free, independent and pluralistic media empower citizens with information that enables them to make informed choices and
actively participate in democratic processes; media can also help to strenghten and enhance transparency and accountability; it
also plays a crucial role in improving the public's understanding of current or emerging events, priorities, issues and policy
pronouncements and options
Rational/purpose
The United Nations are using the Media Development Indicators (MDIs) as a basis for the elaboration of a common approach in
the fields of communication development and good governance
the United Nations but first and foremost this indicator scheme is intended to be a living docuement and shall be tested and
Users/Target Group
adjusted by ist ultimate intended users -JOURNALISTS and NEWS ORGANIZATIONS and CITIZENS'GROUPS
Reference to human
rights
Human rights are implicitly covered
human rights?
This indicator systems fist and foremost covers implicitly the freedom of expression, moreover references are drawn to good
governance and human development
53
FRAME
The MDIs consist of five major categories of indicators; these are broken down into a series of broad indicators - this leads to
the sum of 50 sub-indicators

The MDIs makes use of structural as well as process indicators:
'A. A system of regulation conducive to freedom of expression, pluralism and diversity of the media
- Freedom of expression is guranteed in law and ersprected in practice
- The right to information is guaranteed in law and respected in practice
- Editorial independence is guaranteed in law and respected in practice
- Journalist's right to protect their sources is guranteed in law and respected
- The public and civil society organisations participate in shaping public policy towards the media
- Independence of the regulatory system is guaranteed by law and respected in pracitce
- Regulatory system works to ensure media pluralism and freedom of expression and information
- The state does not place unwarranted legal restrictions on the media
- Defamation laws impose the narrowest restricitons necessary to protect the reputation of individuals
- Other restrictions upon freedom of expression, whether based on national security, hate speech, privacy, contempt of courts
laws and obscentiy should be clear and narrowly defined in law and justifialbe as necessary in a democratic society,j in
accordance with international law
Types of indicators
- The emdia is not subject to prior censorship as a matter of both law and practice
- The state does not seek to block or filter Internet content deemed sensitive or detrimental
B. Plurality and diversity of media, a level economic playing field and transparency of ownership
- State takes positve measures to promote pluralist media
- State ensures compliance with measures to promote pluralist media
- State actively promotes a diverse mix of public, private and community media
- Independent and transparent regulatory system
- State and CSOs actively promot development of community media
- State plan for spectrum allocation promotes diversity of ownership and content
- Independent and transparent regulatory system
- State uses taxation and business regulation to encourage media development in a non-disciminatory manner
- State does not discriminate through advertising policy
- Effective regulation governing advertising in the media
54
FRAME

C. Media as a platform for democratic discourse
- The media - public, private and community-based - serve the needs of all groups in society
- Media organisations refect social diversity through their employment practices
- The goals of public services broadcasting are legally defined and guaranteed
- The operations of public services broadcasters do not experience discrimination in any field
- Independent and transparent systems of governance
- PSBs engage with public and CSOs
- Print and broadcast media have effective mechansisms of self-regulation
- Effective broadcasting code setting out requirements for fairness and impartiality
- Effective enforcement of broadcasting code
- The public displays high levels of trust and confidence in the media
- Media organisations are reponsive to public perceptions of their work
- Journalists, assosciated media personnel and media organisations can practice their profession in saftey
- Media practice is not harmed by a climate of insecurity
D. Professional capacity building and supporting institutions that underpins freedom of expression, pluralism and diversity
- Media professionals can access training appropriate to their needs
- Media managers, including business managers can access training appropriate to their needs
- Training equips media professionals to understand democracy and development
- Academic courses accessible to wide range of students
- Academic courses equip students with skills and knowledge related to democratic development
- Media workers have the right to join independent trade unions and exercise this right
- Trade unions and professional associations provide advocacy on behalf of the profession
- CSOs monitor the media systematically
- CSOs provide direct advocacy on issues of freedom of expression
- CSOs help communities access information and get their voices heard
E. Infrustractural capacity is sufficient to support independent and pluralistic media
- Media organisations have access to modern technical facilities for news gatehring, production and distribution
- Marginalised groups have access to forms of communication they can use
- The country has a coherent ICT policy which aims to meet the information needs of marginalised communities
55
FRAME
Geographical area
covered
international
Methodology
indicators were developed following a two-year consultation process involving experts and organizations all over the world
Data sources
administrative data, as well as expert judgments and surveys
Type of data used
Whenever possible it is preferd to use quanitative data
Quality of data
'Data is provided by local NGOs, national surveys, treaty ratifications, various international and regional reports of the special
rapporteurs on freedom of expression etc.
results
The results shall be presented in a special report on those certain indicators
Comparability
the results, gained by the Media Development Indicators, are not intended to be compared among different states, but rather
are used as an analytic tool designed to help stakeholders assess the state of the media and measure the impact of media
development programmes
56
FRAME
since 2008
The MDIs were endorsed by the IPDC Intergovernamental Council at ist 26th session in 2008
up to now such studies have been launched or completed in Bhutan, Brazil, Croatia, Ecuador, the Maldives, Mauritania,
Comments
Mozambique, Nepal, Egypt
Link to website
http://www.unesco.org/new/en/communication-and-information/resources/publications-and-communicationmaterials/publications/full-list/media-development-indicators-a-framework-for-assessing-media-development/; 02.10.2013
57
FRAME
Joint United Nations Programme on HIV/Aids

Author
UNGASS United Nations General Assembly Special Session
Type of author
United Nations Programme on HIV/Aids
Thematic context
to achieve universal access to HIV prevention, treatment, care and support, uniting, speaking out, mobilizing, empowering and
supporting
Rational/purpose
the rational is acknowledging the importance of an effective response to HIV/Aids, and taking action to halt and begin to reverse the
AIDS epidemic
Users/Target Group
Reference to human
yes, there is a direct reference to human rights
rights
human rights?
Human rights
covered
right to the enjoyment of the highest attainable standard of mental and physical health
58
FRAME
Number of
indicators
~ 40 indicators

main type of indicators are outcome ones
Types of indicators
Concrete type of indicator:

- Estimated HIV prevalence - adult(ages 15-49)
- People living with HIV (all ages)
- People living with HIV (ages 15+)
- Estimated new HIV infections (all ages)
- Estimated new HIV infections (ages 15+)
- Percentage of young people aged 15 to 24 who are living with HIV
- Estimated AIDS deaths
- HIV testing, multiple sexual partnerships and condom use
- Sex workers - population size estimation
- Percentage of sex workers reporting the use of a condom with their most recent client
- Percentage of sex workers who are living with HIV
- Men who have sex with men - populationn size estimation
- Percentage of men reporting the use of a condom the last time they had anal sex with a male partner
- Percentage of men who have sex with men who are living with HIV
- People who inject drugs - population size estimation
- Number of syringes distributed per person who injects drugs per year by needle and syringe programmes
- Percentage of people who inject drugs who report the use of a condom at last sexual intercourse
- Percentage of people who inject drugs who reported using sterile injecting equipment the last thime they injected
- Percentage of people who inject drugs who are living with HIV
- Percentage of infants born to HIV-positive women receiving a virological test for HIV within 2 months of birth
- New HIV infections among children
- Preventing mother-to-chil transmission of HIV
- Twelve-month retention on aniretroviral therapy
- Estimated number of adults receiving and needing antiretroviral therapy and coverage
59
FRAME

- Estimated children receiving and needing antiretroviral therapy and coverage
- Projected number of people in need of antiretroviral therapy according to 2013 WHO antiretroviral criteria guidelines
- Percentage of estimated HIv-positive incident TB cases that received treatment for both TB and HIV
- HIV-positive tuberculosis patients on antiretroviral therapy
- Estimated number of TB-realted deaths among people living with HIV in Africa, in TB/HIV high burden countries
- Domestic HIV spending from domestic public sources
- Domestic HIV spending from international sources
- Total domestic HIV spending from domestic public and international sources
- Young people's knowledge about HIV prevention
- Proportion of ever-married or partnered women aged 15-49 who experienced physical or sexual violance from a male intimate
partner in the past 12 months
- Number of HIV infected femal adults
- HIV-specific restrictions on entry, stay, or residence
- Health facilities provide HIV services integrated with other health services
- Current school attendance among young people aged 10-14
Geographical area
covered
indicator scheme is supposed to be applied internationally; sometimes regional comparisons are established
Methodology
the indicator system was established through a series of high - level meetings
Data sources
socioeconomic and administrative data
60
FRAME
Type of data used

household surveys
quantitative data
data is provided through two sources:
Quality of data
- Global AIDS Response Progress Reporting (GARPR)
providers/processors - Modelled HIV estimates by national epidemiological teams
Presentation of
indicator results
the results gained by the indicators are presented in tables
Comparability
yes, there is an intend that the data shall be compared internationally and also between regional countries (e.g. African countries)
Frequency of
application
i.)the indicator system is applied periodically

ii.)up to now the indicator system was applied twice
for the first time it was applied in 2001, the latest application was in 2012
Status
periodic application
Link to website
http://www.aidsdatahub.org/dmdocuments/UNAIDS_Global_Report_2013_en.pdf; 04.10.2013
61
FRAME
UN OHCHR Rule of Law Indicators

Author
United Nations Office of the High Commissioner for Human Rights and the Department of Peacekeeping Operations
Type of author
International Organisation
Thematic context
Rule of Law analysis, monitoring tool for assessing the rule of law in post-conflict situations
Rational/purpose
The United Nations Rule of Law Indicators are part of an emerging body of empirically based approaches to measuring the strengths
and effectiveness of law enforcement, judicial and correctional institutions.
The instrument is to be implemented in collaboration with national Governments and potentially adopted by them as an ongoing
monitoring mechanism. Participating countries will find this instrument very useful for monitoring their own progress in developing
Users/Target Group their criminal justice institutions and strengthening the rule of law. When applied over time, the rule of law indicators is not only
helping governments to monitor their own performance, but also creating awareness among civil society organisations and the
media about the state of the rule of law in the country.
Rule of law refers to a principle of governance in which all persons, institutions and entities, public and private, including the State
Reference to human
itself, are accountable to laws that are publicly promulgated, equally enforced and independently adjudicated, and which are
rights
consistent with international human rights norms and standards. It requires, as well, measures to ensure adherence to the principles
62
FRAME
of supremacy of law, equality before the law, accountability to the law, fairness in the application of the law, separation of powers,
participation in decision-making, legal certainty, avoidance of arbitrariness and procedural and legal transparency.
human rights?
The indicators assess the transparency, integrity and accountability of the police, judiciary and prisons. They also consider how these
Human rights
institutions treat vulnerable groups such as minorities and refugees; and their capacity to fulfil their duties. The indicators are rooted
covered
in international human rights and criminal justice norms and standards and can be applied to both civil and common law systems.
135 indicators are grouped under three institutions: the police (41 indicators); the judicial system (51 indicators); and prisons (43
Number of
indicators). For each institution, indicators are grouped into several baskets which make up all together 25 baskets of indicators for
indicators
all three institutions.
Structural, Procedural and Outcome Indicators are used.
Types of indicators
Geographical area
covered
post-conflict countries - not applied so far, pilot implementation was foreseen to be conducted in Haiti and Liberia
Methodology
1. Phase one. Introduction and assessment

1.1. Building relationships
1.2. Understanding the local context
1.3. Identifying sources of data
1.4. Addressing potential challenges and obstacles
2. Phase two. Collecting and assessing the data
2.1. Sources of data
2.2. Accessing existing data
2.3. Collecting your own data
2.4. Ethical considerations in data collection
2.5. Assessing the data
2.6. Data entry and management
63
FRAME

3. Phase three. Analysing the data and presenting results
3.1. Developing indicator rating
3.3. Producing initial ratings
3.4. Producing narrative descriptions
3.5. Reporting results
Data sources
Data sources are surveys of public and expert perceptions, administrative data and document review.
Type of data used
Qualitative and quantitative data from surveys of public and expert perceptions, administrative data and document review.
Data collection partners typically include staff from:
National criminal justice institutions
National and international NGOs
Quality of data
United Nations peacekeeping missions
providers/processors Other United Nations agencies
The national statistical office
The United Nations peacekeeping mission and other United Nations agencies also should be able to share whatever information they
collect as part of their regular monitoring and reporting functions and may be able to facilitate access to Government data.
Presentation of
indicator results
Each one of the rated indicators is eventually expressed as a numerical value ranging from 1.0 to 4.0. This value is the result of
averaging all the individual responses on a four-point scale ranging from 1 to 4, where 4 is the highest positive score possible and 1
the lowest and most negative score. A number of indicators are not rated but are expressed instead in the form of a percentage. In a
few other instances, indicators require the calculation of a rate or ratio. The individual indicators are designed to be rated in isolation
and also to be combined to provide aggregate measures at the level of the baskets and major dimensions for each institution.
64
FRAME
Comparability
The instrument, in contrast to some other measurement

tools, is designed to highlight apparent successes and
shortcomings within institutions and to monitor changes
over time within countries. It is not meant to support direct
comparisons between countries or rank them.
Frequency of
application
not applied yet
According to the information of the OHCHR from 8 July 2011 they are presently being applied in Haiti and Liberia in an pilot
implementation.
Comments
http://www.ohchr.org/EN/NewsEvents/Pages/JusticeAndRuleOfLaw.aspx
Link to website
http://www.un.org/en/events/peacekeepersday/2011/publications/un_rule_of_law_indicators.pdf; 27.05.2014
65
FRAME
European Commission for the Efficiency of Justice (CEPEJ), Evaluating Judicial Systems
Author
European Commission for the Efficiency of Justice (CEPEJ)
Type of author
Council of Europe commission, established on 18.9.2002 by way of Resolution Res (2002)12
Thematic context
The aim of the CEPEJ is the improvement of the efficiency and functioning of justice in the member states, and the development of
the implementation of the instruments adopted by the Counci of Europe to this end; its tasks are to analyse the results of the judicial
systems, to identify the difficulties they meet, to define concrete ways to improve, on the one hand, the evalouaiton of their results,
and, on the other hand, the functioning of these systems; to provide assistance to member states, at their request; to propose to the
competent instances of the CoE the fields where it would be desirable to elaborate a new legal instrument
Rational/purpose
The aim of the study is to compare the functioning of judicial systems in their various aspects, to have a better knowledge of the
trends of the judicial organisations and to suggest reforms to improve the efficiency of justice. The evaluation Scheme and the
analysis of the outcoming results should become a genuine tool in favour of public policies on justice, for the sake of the European
citizens
Users/Target Group
the main user group is the CEPEJ itself
Reference to human
there is one direct reference to human rights
rights
human rights?
66
FRAME
Human rights
covered
there is a reference to Article 6 ECHR - right to a fair trail; limited to civil law procedures
Number of
indicators
there are 16 key indicators

1. Demographic and economic data
- Inhabitants, DGP and average gross annual salary
- Total annual budget allocated to all courts (euro)
- Does the budget of the courts include the following items (salaries, computerisation, legal costs, court building, training and
education, legal aid, other
- Annual public budget allocated to the prosecution services
- Is the budget allocated to the public prosectuion included in the court budget
2. Legal Aid (Access to Justice)
- Annual number of legal aid (provided by public authorities) cases and annual public budget allocated to legal aid
3. Organisation of the court system and the public prosectuion
- Judges, non-judge staff and Rechtspfleger
- Public prosecuters and non-prosecutor staff of the prosectuion services
Types of indicators
- Level of computer facilites used within the courts
4. The performance and workload of the courts and the public prosecution
- Number of cases regarding Article 6 ECHR (on duration and non-execution), for the year of reference
- Total number of civil cases in the courts
- Litigious civil cases and administrative law cases in the courts
- number of cases received and treated by the public prosecutor
- Criminal cases in the courts
5. Execution of court decisions
- As regards a decision on debts collection, can you estimate the average timeframe to notify the decision to the parties which live in
the city where the court seats
6. Legal and judicial reforms
- Please describe the main activities of legal and or judicial reform in your country which contributes to the improvement of the
67
FRAME

quality and efficiency of your judicial system
in this case there are structural as well as outcome indicators used to measure the judicial system in the European Region
Geographical area
covered
the survey covers the area of the CoE member states
Methodology
it was developed and designed by the CEPEJ, they were consulted by experts
Data sources
the figures in this report are from 2010

the data is based on reports by member states and entities, which were to appoint national correspondents, entrusted with the coordination of the replies to the scheme for their respective states or entities
Type of data used
quantitative as well as qualitative data is gathered from the member states on the information of the daily functioning of judicial
systems
68
FRAME
the data is provided by the states and their ministries or other officals dealing with that issue at stake
Quality of data
it has to be noted that the quality of the figures depends very much on the type of questions asked in the data collection instrument,
providers/processors the definitions used by the countries, the system of registration in the countries the efforts supplied by national correspondents, the
national figures available to them and the manner in which the figures have been processed and analysed.
Presentation of
indicator results
The results are presented in a report conducted by the CEPEJ, but for a more clear presentations charts and tables are used in the
report, too.
Comparability
yes, it aims to be comparable
Frequency of
application
Periodically planned
Link to website
http://www.coe.int/t/dghl/cooperation/cepej/evaluation/2012/Rapport_en.pdf; 18.10.2013
69
FRAME
Social Cohesion Indicators

Author
Council of Europe
European Committee for Social Cohesion (CDCS)
Type of author
Council of Europe
Thematic context
The governments of member states are entitled to appoint representatives with the following qualifications for the Committee:
senior national civil servants having responsiblity for the planning, development and implementation of policies relevatn to the work
of the Committee and appointed by their governments to co-ordinate at national level all elements of government policy relevant to
the work of the Committee.
The CDCS has been a pioneer in defining social cohesion and developing tools in this field. The current and future challenges are
analysed by the CDCS and recommendations and guidelines are drawn up to assist member states in adjust their social policies
accordingly. The CDCS has developed and maintains a pan-European approach to co-operation in social matters.
Rational/purpose
The Social Cohesion Development Division of the council of Europe wanted, with the support of the member states' government, to
design and produce a guide for the analysis of social cohesion in line with the Council of Europe's Revised Strategy for Social
Cohesion. The guide has three objectives: 1. to serve as a reference framework that can become a common instrument for coordinating and steering the political choices of the various private and public players and social bodies at different territorial levels; 2.
to help devise ways of accumulating knowledge that can be adopted to different contexts and facilitate the implementation,
monitoring and assessment of social cohesion action plans; 3. to facilitate the social cohesion learning process. The guide
incorporates numerous questions that can help its users search for information and select the most appropriate indicators for
responding to the specific needs of the players and bodies concerned.
Users/Target Group
the indicators scheme is ought to be an inspiration to all those involved in building what some now call the "welfare society"
70
FRAME
Reference to human
there is only an indirect reference to human rights
rights
human rights?
the Council of Europe sees social cohesion as an integral part of human rights; individuals are viewed as key players in a social
process to enhance the substance of these rights through collective participation, consideration of the distribution of social benefits
Human rights
and recognition of the need for diversity. Social cohesion thus takes account of how the various social players interact and the degree
covered
to which they succeed in ensuring the well-being of eyerone. the aim is to achieve social cohesion through human rights, of which
the constituent elements are freedom, equality and solidarity
the number of indicators is not clearly given, but depends on the issue at stake
Number of
there is a reference tot he European Commission National Action Plans on Social Inclusion, where nine indicators were drawn up, six
indicators
of them related to the indicator as such and three to the portfolio of indicators chosen
the indicators are operational indicators exploited in monitoring and evaluating action conducted in the territory and its contribution
to social cohesion
- the MAIN INDICATORS ARE:
- Inequality of income distributions
- Long-term unemployment rate
- Life expectancy at birth
- Proportion of homeless in the population/Population without access to quality housing
- Assumption of senior responsibilities for women
Types of indicators
- Ethnic or religious ghettos
- Elderly people who receive a minimum old age allowance
- Proportion of overind-ebted households/Proportion of households below the poverty threshold in spite of both parents working
- Dropout rate at the minimum school-leaving age withoug qualifications
- Ability of children from disadvantaged social background to succeed at school
- Participation in elections by 18-34 year-olds
- Proportion f the budget reserved for social issues
- Workers with disabilites iin the public and private sector
71
FRAME

- Proportion of jobs in the voluntary sector
- Elderly people living with their families
- Confidence in public institutions
- Suicide rate
- Awareness of human rights and of the right to justice
- Subjective perception of health
- Murder rate
Geographical area
covered
the statistics cover the region of the CoE member states' territory
Methodology
the Council of Europe already adopted recommendations and resolutions dealing with social cohesion, and from that starting point
the methodology guide was developed
Under the co-ordination of the Social Cohesion Development Divison, various departements have contributed their knowledge to the
guide. Moreover case studies carried out in different countires with intergovernmental committees have helped to confirm the value
of the method suggested in this guide.
Data sources
national respectively international statistical surveys, as well as administrative data, specific studies, opinion polls
Type of data used
there are three types of data used in this methodology guide: 1. quantitative and objective data/indicators; 2. qualitative and
objective data/indicators, 3. qualitative and subjective data/indicators
Status
Link to website
the indicator scheme is only developed in theory, except of the fact that the Methodology Guide is currently (last update in 2012)
implemented in the French city of Mulhouse (Alsace Region), which conducts this process at the local level. The innovative
implementation has enabled not only to test the approach and ascertain its feasibilty, but also to specify and clarify the methods of
consultation and construction of joint indicators.
http://www.coe.int/t/dg3/socialpolicies/socialcohesiondev/indicators_en.asp; 28.10.2013
72
FRAME
Laeken Indicators - Open Method of Coordination for social protection and social inclusion (Social OMC)
Author
European Council; Indicators Sub-Group of the Social Protection Committee (SPC), European Commission
Type of author
EU institution
Thematic context
The European Union strategic objective by 2010 of becoming the most competitive and dynamic knowledge-based economy in
the world, capable of sustainable economic growth with more and better jobs and greater social cohesion. At the Nice European
Council in December 2000, Heads of State and Government reconfirmed and implemented their March 2000 (Lisbon) decision
that the fight against poverty and social exclusion would be best achieved by means of the open method of co-ordination.
Rational/purpose
Key elements of the open method of co-ordination are the definition of commonly-agreed objectives for the European Union
(EU) as a whole, the development of appropriate national action plans to meet these objectives, and the periodic reporting and
monitoring of progress made. In this context the Laeken European Council in December 2001 endorsed a first set of statistical
indicators for social inclusion, which will allow monitoring in a comparable way of Member States progress towards the agreed
EU objectives.
Users/Target Group
the EU and the Member States
Reference to human
rights
The indicators cover four important dimensions

of social inclusion: financial poverty, employment, health and education
human rights?
73
FRAME
Poverty and social exclusion, Child poverty, Education, Employment, Health status,
They cover four important dimensions of social inclusion and social protection, namely financial poverty, employment, health
and education, which are measured by a two-tier structure of 10 primary indicators and 8 secondary and context indicators pus
one additional indicator. (see European Commission, 5 August 2005)
The indicator scheme uses mainly outcome indicators that measure different breakdowns of poverty rate, income distribution
and self defined health status. The structural indicators used by the scheme are different breakdowns of distribution of
Types of indicators
population.
Geographical area
covered
EU Member States
Explain, whether the indicator scheme is supposed to be applied on the local, regional, national, international, EU- level, number of countries, etc.
Methodology
Member States were requested to make use of the commonly agreed indicators in the NAPs; they were also invited to use
third-level indicators defined at the national level to highlight specificities in particular areas not adequately covered by them,
and to help interpret the primary and secondary indicators.
Data sources
European Community Household Panel (ECHP) / Community Statistics on Income and Living (EU-SILC)
EU Labour Force Survey (EU LFS)
PISA Survey - OECD
Eurostat demographic statistics
74
FRAME
Type of data used
The scheme uses mainly quantitative data from different EU statistics and sets benchmarks.
Quality of data
The data provided through European and national statistics are processed by the member states in their National Action Plans
on Social Inclusion.
results
The European Commission carried out an assessment of the National Action Plans for social inclusion (NAPs/inclusion) and
adopted a Joint Report on Social Inclusion.
Comparability
Indicators are further developed and there is no continuous application of the same indicators.
Status
Link to website
As part of the open method of co-ordination, Member States submitted their first National Action Plans on Social Inclusion in
2001. Member States were obliged to use the common indicators in preparing their 2003 round of NAPs. However in the Joint
Report on Social Inclusion 2003 it is reported, that not all Member States have applied the indicators for their NAPs.
The indicator scheme agreed and developed within the open method of coordination for social protection and social inclusion
(Social OMC) have been incorporated to monitor the progress within the target of reduction of poverty and exclusion of the
"Europe 2020" strategy for jobs and smart, sustainable and inclusive growth.
http://www.insee.fr/en/insee-statistique-publique/colloques/pauvrete/pdf/Eurostat_social_indicators_EN.pdf
and
http://www.cso.ie/en/media/csoie/eusilc/documents/Laeken,Indicators,-,calculation,algorithm.pdf
15.05.2014
75
FRAME
Indicators for the Protection, Respect and Promotion of the Rights of the Child in the European Union
Author
FRA
Type of author
EU Agency
Thematic context
Fundamental rights indicators for policy evaluation
Rational/purpose
Measuring implementation of child rights in Europe in order to improve data collection and research on the impact of EU
activities on children
Users/Target Group
Targed at FRA and EU and Member States

Reference to human
rights
- UN child rights framework: 1989 UN Convention on the Rights of the Child

- EU relevance of right (normative basis) is explicitly mentioned for each indicator
human rights? Which treaty, convention or document do they refer to?
Rights of the Child:
- Family Environment and Alternative Care (3 indicator groups)
- Protection from Exploitation and Violence (3 indicator groups)
76
FRAME
- Education, Citizenship and Cultural Activities (3 indicator groups)

- Adequate standard of living (4 indicator groups)
> 80

- Balanced approach: Structural, procedural and outcome indicators
- But: Adequate Standard of Living and Education, Citizenship and Cultural Activities: focused on outcome indicators
Types of indicators
Family Environment and Alternative Card and Childr Protection: focused on process indicators
Geographical area
covered
EU member States
Methodology
Development of indicators:
> mapping and analysis of conceptual framework: legal, sociological, methodological, iddeological and ethical issues
- emphasis on children's rights indicator research
- emphasis on EU child law and policy
- indicators are grounded in CRC
- Features of Children's rights indicators: normative base (CRC and EU), evaluative function, views of the children
> Expert consultation: online discussion forum, online survey, consultation meeting, one-to-one key actor interviews,
Data sources
- Attempt to fully exploit existing diverse range of data sources and to use most reliable data
- Eurostat
- Member States statistical data, reports to monitoring bodies,...
- UN agencies
- NGO "shadow" reports
77
FRAME

- Other international organisations (ILO, )
- Other sources: i.e. (for online safety) Reviews of CSR Statements by the private sector, Policy statements of internet service
providers, telecommunication companies,...
Type of data used
Qualitative and Quantitative

Subjective data (children's perspective) is included
Quality of data
data is gathered from EU member states and all possible other sources: quality varies, but is in general quite good
results
No specific information, probably no special presentation?
Comparability
comparability across countries: comparability is intended and should be possible because the use of a huge variety of data
sources
comparability over time: no statement
Once, planned regularly
Status
Link to website
Indicator scheme developed, but has not been applied yet

http://fra.europa.eu; 30.07.2013
78
FRAME
The FRA indicators on the right to political participation of persons with disabilities
Author
Fundamental Rights Agency
Type of author
EU Institution
Thematic context
as mentioned by the title these indicators were developed for the right to political participation of persons with disabilities
Rational/purpose
monitoring, measuring progress, identification of violation of state obligations
Users/Target Group
The EU institutions
Reference to human
rights
explicite mentioning of human rights
human rights?
right to participation, right to vote
79
FRAME
28

Structure, Process, Outcome
Types of indicators
Geographical area
covered
EU Member States
Methodology
Fundamental Rights Agency developed the indicators together with the European Commission and ANED
Data sources
Data is collected through des reserach by FRANET and ANED. They gather publically available information in the member states
and draw on EU statistics. Also a number of data requests is made to public authorities.
Type of data used
Various data - statistics, specific information requests to public authorities
Quality of data
Public bodies, EU bodies, NGOs,
80
FRAME
results
Presented in a report by the Fundamental Rights Agency.
Comparability
Comparison possible within EU countries
2014, only once
Status
first application made
Link to website
http://fra.europa.eu/sites/default/files/fra-2014-right-political-participation-persons-disabilities_en.pdf; 05.06.2014
81
FRAME
EIGE Gender Equality Index

Author
European Institute for Gender Equality (EIGE)
Type of author
EU Agency
Thematic context
Gender equality in the EU
Rational/purpose
Users/Target Group
Analysis of the multi-faceted reality of gender equality in specific domains and improving gender equality in the EU and in the
Member States.
EU institutions, Member States,
Reference to human
rights
explicit reference to the Charter of Fundamental Rights of the European Union
human rights?
Equality between women and men
It consists of eight domains, the first six (work, money, knowledge, time, power and health) being combined into a core index,
and an additional two satellite indices (intersecting inequalities and violence). The domains are devided into several subNumber of indicators
domains. The sub-domains are measured by alltogether 30 indicators
Types of indicators
The focus is on outcome indicators, which measure a current status as opposed to process or input variables.
82
FRAME
Geographical area
covered
EU-level and Member States
Methodology
The Gender Equality Index is a unique measurement tool that synthesises the complexity of gender equality as a multidimensional concept into a user-friendly and easily interpretable measure. As with any other composite indicator, the Gender
Equality Index is a measure obtained by compiling individual indicators on the basis of an underlying model of the multidimensional concept that is being measured. In other words, it is a mathematical combination of a set of individual indicators,
which aims to provide a summary of a complex reality. The Index is based on the widespread and internationally accepted
procedure developed by the OECD and the European Commissions Joint Research Centre
Data sources
Eurostat, Eurofound, UNESCO, OECD, European Commission
Type of data used
quantitative data
Quality of data
The development of the Gender Equality Index relies on strict statistical criteria combined with a solid theoretical framework.
The data sources are reliable.
results
The scores, overall and by domains are presented in a table per country. The overall scores per country and in the EU are
obtained by aggregating the first six domains, the so-called core domains of work, money, knowledge, time, power and health.
Furthermore, the relationships between different domains are presented in the report.
83
FRAME
Comparability
Comparision of indicator results across EU countries is possible as results are presented in scores and also the EU-average is
given.
not clear
Status
first application, first index published in 2013
Link to website
http://eige.europa.eu/content/gender-equality-index; 23.07.2014
84
FRAME
Equality Measurement Framework and How fair is Britain

Author
UK domestic authorities:
- Equality and Human Rights Commission (EHRC)
- Government Equalities Office (GEO)
Type of author
UK national authority
Thematic context
- Domestic monitoring of social outcomes - Human rights and equality monitoring
Rational/purpose
- Equality Act (2006): legal duty to monitor and evaluate progress towards equality and human rights, taking account of gender,
ethnicity, disability, age, sexual orientation, transgender status, and religion or belief... duty to monitor social outcomes by
developing indicators and assessing progress in a triennial report.
- to provide a baseline of evidence to inform policy priorities
- to help to identify inequalities that need further investigation
Users/Target Group
General, policy-makers
- based on Equality Act (2006)
Reference to human
- international human rights framework and HRA with ECtHR jurisprudence: crucial importance, foundation for EMF
- based on capability approach (Amartya Sen)
rights
- based on consultation with the general public, and individuals and groups at risk of discrimination and disadvantage
85
FRAME
human rights?
- Equality: based on idea of "substantive freedom" or equality in the central and valuable things in life that people can actually
do and be (outcome, process, autonomy)
- central and valuable freedoms (10, with 3-5 indicators each):
> life,
> health,
> physical security,
> legal security,
> education and learning,
> standard of living,
> productive and valued activities,
> individual, family and social life,
> identity, expression and self-respect,
> participation, influence and voice
- Importance of disaggregation on basis of discrimination charcteristics such as age, sex, religion etc.
48 indicators: 3-5 indicators for each of the 10 domains
Types of indicators
Outcome, process and autonomy (empowerment, degree of choice and control)
Geographical area
covered
UK
86
FRAME
Data sources
- Mainly offical data from government authorities
Type of data used
- Quantitative data (survey, census, )
Quality of data
- Official data from government authorities: overall high quality, but shortcomings when it comes to comparability among
England, Wales and Scotland, datagaps are still present, availability of data disaggregated along specific characteristics,..
results
- No special presentation: rates, percentages,

- no online data tool
- used for triennial review
Comparability
- Comparability across countries: not intended, comparability among England, Wales and Scotland limited
- Comparability over time: no information,
2009, 5-annually
87
FRAME
Status
First applications made
Link to website
http://www.equalityhumanrights.com/key-projects/equality-measurement-framework/index.html; 16.08.2013
88
FRAME
How Fair is Britain

Author
Equality and Human Rights Commission
Thematic context
Human Rights Monitoring (domestic)
Rational/purpose
Monitoring: monitor the progress that society makes towards becoming one that is more equal, where every individual has the
opportunity to achieve their potential, and where people treat each other with dignity and respect. This progress must be
reported on every 3 year's.
Users/Target Group
Reference to human
rights
- Equality Act in 2006, Section 12 of the Act: responsibility to monitor the progress that society makes towards becoming one
that is more equal, where every individual has the opportunity to achieve their potential, and where people treat each other
with dignity and respect.
- Utilising the Equality Measurement Framework (EMF) indicators
human rights?
89
FRAME
The key areas that these research projects focused on & institution conducting analysis:
- Life and Health: Centre for Health and Social Care Research, Sheffield Hallam University
- Access to Housing and Standard of Living: Centre for Housing Policy, University of York
- Financial Exclusion: Institute for Public Policy Research
- Civic Life: Institute for Social Change, University of Manchester
- Employment: Policy Studies Institute, University of Westminster
- Education and Learning: School of Education, University of Southampton
- Legal and Criminal Justice: UNESCO Chair in Gender Research Group, Lancaster University
- Wellbeing: The Young Foundation
40

Outcome indicators, also subjective indicators such as "Fear of crime", "Dignity and respect in health treatment",...
9 areas (40 indicators)
- Life: i.e. Life expectancy, Moral illness, Suicide, ...
- Legal security: i.e. Equal treatment by the criminal justice system, Offences reported and brought to justice
- Physical security: i.e. Crimes against the person, Targeted violence, Fear of crime
- Health
Types of indicators
- Education: i.e. Level of development at age 5, Permanent exclusions from school, Bullying, respect and support at school,...
- Employment
- Standard of living
- Care and support
- Power and voice
Methodology
Quantitative data --> quantitative indicators: calculation or rates, percentages, etc
Data sources
- Mainly official domestic sources (i.e. Annual Population Survey, British Crime Survey, Department of Education,)
- Additional literature, studies,
90
FRAME
Type of data used
Quantitative data
Quality of data
Official domestic data --> data sources with high quality, but still limited
results
Review, percentages, rates, diagrams
Comparability
-Comparability across countries: not intended, domestic report; but comparability among England, Scotland and Wales is
intended, sometimes problematic due to difference in sources and technical specification of measures.
- Comparability over time: no information, first review
2010, triennial (so far only one review)
91
FRAME
2.
Freedom in the World

Author
Freedom House
Type of author
NGO
Thematic context
Measuring democracy and political freedom
Rational/purpose
Situation analysis
Users/Target Group
- Policy makers
- Media, civil society
- International corporations
Reference to human
- The methodology of the survey is grounded in basic standards of political rights and civil liberties, derived in large measure
92
FRAME
from relevant portions of the Universal Declaration of Human Rights.

- Indicators are primarily aimed at measuring democracy. This is done by assessing implementation of political rights and civil
liberties.
Grounded in basic standards of political rights and civil liberties, derived in large measure from relevant portions of the
Universal Declaration of Human Rights.
- Political rights:
> Electoral process,
> Political pluralism and participation
> Functioning of government
- Civil liberties:
> Freedom of expression and belief,
> Associational and organizatioal rights,
> Rule of law
> Personal autonomy and individual rights
> 23 indicators based on 23 checklist questions (See Freedom of the World 2013, 33- 34)
> Rating 1-7 (known as "freedom rating" or "Freedom House Index" based on average of:
- index of level of political rights (12 indicators/questions)
- index of civil liberties (11 indicators/questions)
rights
Types of indicators
Quantitative indicators
Geographical area
covered
International level (195 countries + 14 related and disputed territories covered)
93
FRAME
Methodology
- Each country is assigned a numerical rating from 1 to 7 for both political rights and civil liberties, with 1 representing the most
free and 7 the least free. The ratings are determined by the total number of points (up to 100) each country receives on 10
political rights questions and 15 civil liberties questions; countries receive 0 to 4 points on each question, with 0 representing
the smallest degree and 4 the greatest degree of freedom. The average of the political rights and civil liberties ratings, known as
the freedom rating, determines the overall status: Free (1.0 to 2.5), Partly Free (3.0 to 5.0), or Not Free (5.5 to 7.0). Freedom
House also assigns upward or downward trend arrows to countries which saw general positive or negative trends during the
year that were not significant enough to result in a ratings change.
- Expert coding: multilayered process of analysis and evaluation by a team of in-house and consultant regional experts and
scholars.
- Political rights ratings are based on an evaluation of three subcategories: electoral process, political pluralism and
participation, and functioning of government.
- Civil liberties ratings are based on an evaluation of four subcategories: freedom of expression and belief, associational and
organizational rights, rule of law, and personal autonomy and individual rights.
Data sources
Survey conducted by Freedom House

other sources (difficult to figure out which sources for which indicator,...)
Type of data used
- Survey, including both analytical reports and numerical ratings

- Qualitative Data:
- Checklist Questions (See Freedom House, Freedom in the World 2013, 33), such as
"A. ELECTORAL PROCESS
1. Is the head of government or other chief national authority elected through free and fair elections?
2. Are the national legislative representatives elected through free and fair elections?
3. Are the electoral laws and framework fair?"
94
FRAME
Quality of data

uses broad range of information:
- foreign and domestic news reports,
- academic analyses,
- NGOs
- think tanks
- individual professional contacts
- visits to the regions,
results
> Published in Freedom in the World Reports

> Classification of rating (1-7):
- 1.0-2.5: Free
- 2.51-5.5: Partly Free
- 5.51-7.0: Not Free
Comparability
- comparability over time: changes regarding methodological issues: not revised retroactively, limited comparability over time
- comparability across countries: intended, but limited, due to aggregated value
1972, annually
Status
Regular application
Link to website
http://www.freedomhouse.org/report-types/freedom-world, 26.07.2013
95
FRAME
Freedom of the Press (Freedom House)

Author
Freedom House
Type of author
NGO
Thematic context
Measuring media independence
Rational/purpose
Situational analysis
- scholars,
- policymakers,
- international institutions,
Users/Target Group
- media, and
- activists
Reference to human
rights
Article 19 UDHR
- Freedom of the press

96
FRAME
109 indicators aggregated to 1 index (Freedom of the Press index)
Types of indicators
Quantitative indicators: points are allotted to quantify indicator

i.e. 1 indicator-1 question: 1. Do the constitution or other basic laws contain provisions designed to protect freedom of the press
and of expression, and are they enforced? (06 points)
Geographical area
covered
International level (197 countries and territories)
Is the indicator scheme supposed to be applied on the local, regional, national, international, EU- level, number of countries, etc.
Countries are given a total score from 0 (best) to 100 (worst) on the basis of a set of 23 methodology questions divided into
three subcategories. Assigning numerical points allows for comparative analysis among the countries surveyed and facilitates an
examination of trends over time. The degree to which each country permits the free flow of news and information determines
the classification of its media as Free, Partly Free, or Not Free. Countries scoring 0 to 30 are regarded as having Free
media; 31 to 60, Partly Free media; and 61 to 100, Not Free media.
Methodology
Expert coding (See Freedom of the Press 2013, 35-36):

- multilayered process of analysis and evaluation by a team of regional experts and scholars (several dozen analysts including
members of the core research team headquartered in New York, along with outside consultantswho prepared the draft
ratings and country reports) .
- element of subjectivity inherent in the index findings, the ratings process emphasizes intellectual rigor and balanced and
unbiased judgments.
- conclusions are reached after gathering information from professional contacts in a variety of countries, staff and consultant
travel, international visitors, the findings of human rights and press freedom organizations, specialists in geographic and
97
FRAME
geopolitical areas, the reports of governments and multilateral bodies, and a variety of domestic and international news media.
- 23 methodology questions and 109 indicators divided into three broad categories: the legal environment, the political
environment, and the economic environment.
- For each methodology question, a lower number of points is allotted for a more free situation, while a higher number of
points is allotted for a less free environment: i.e. for legal environment (0-30 points)
1. Do the constitution or other basic laws contain provisions designed to protect freedom of the
press and of expression, and are they enforced? (06 points)
2. Do the penal code, security laws, or any other laws restrict reporting and are journalists or
bloggers punished under these laws? (06 points).
information from correspondents, staff and consultant travel, international visitors, the findings of human rights and press
freedom organisations, specialists in geographic and geopolitical areas, the reports of governments and multilateral bodies, and
Data sources
a variety of domestic and international news media
Type of data used
information from
- professional contacts in a variety of countries,
- staff and consultant travel,
- international visitors,
- the findings of human rights and press freedom organizations,
- specialists in geographic and geopolitical areas,
- the reports of governments and multilateral bodies, and
- a variety of domestic and international news media.
Quality of data
- varies due to variety of data used,

- experts evaluate objective and subjective data
- they include their own subjective information
98
FRAME
results

> Published in Freedom of the Press Reports
> Classification of rating (0-100):
- 0-30: Free
- 31-60: Partly Free
- 61-100: Not Free
Comparability
- comparability over time: changes regarding methodological issues: not revised retroactively, limited comparability over time
- comparability across countries: intended and possible
1980, annually
Regular application
Status
http://www.freedomhouse.org/report-types/freedom-press; 26.07.2013
Link to website
99
FRAME
Press Freedom Index (Reporters sans frontires)

Author
Reporters sans frontires
Users/Target Group
Type of author
NGO
Thematic context
Freedom of media/press
Rational/purpose
- Analysis of freedom of press in countries all over the world

- Awareness raising, inform civil society, etc.
Users/Target Group
- Civil society
- Measures freedom of media, which is part of freedom of expression
Reference to human
- Also refers to human rights violations against journalists and media organizations
rights
- No explicit reference to any normative base
human rights?
Freedom of expression
100
FRAME
Index, composed of 6 areas, > 80 questions

A. Accounting for human rights violations (by RwB staff, not included in questionnaire)
B. Media legal status: (10 questions) i.e.
B.1 - In your country, do any of the following exist?
B.2 - What are the factors apparently preventing the creation of independent, privately owned media? (Political, religious,
ethnic, linguistic)
C. Legal status of journalists: (14 questions) i.e.
C.1 - Is journalism training available at a professional level, with emphasis on developing the capacity for critical judgement in
journalism students?
D. Pluralism and editorial independence: (18 questions) i.e.
Types of indicators
D.1 - Do completely independent media exist - that is, media whose staff may take positions of any kind on public issues with no
limits of any kind from owners or the government?
D.1.1 - If yes, how many of these media companies exist?
D.1.2 - Name them:
D.2 - Do media reflect the range of opinions among members of the public?
E. Legal doctrine and practice (21 questions)
F. The internet and technical resources (11 questions)
(See http://www.rsf.org/index/qEN.html)
Geographical area
covered
international: 150 countries (for other countries no reliable information available)
Methodology
- Questionnaire
- Quantitative questions about number of violations of different kinds are handled by Reporters without borders staff (i.e.
number of journalists, media assistants and netizens who were jailed or killed in the connection with their activities, number of
journalists abducted,...)
- rest of questionnaire is sent to outside experts and members of the RWB network (18 freedom of expression NGOs and 150
101
FRAME

experts around the world,...): issues that are hard to quantify (i.e. degree to which news providers censor themselves,
government interference in editorial content, or transparency of government decision making)
- score (0-100) and position are assigned to each country in final ranking
Six general criteria: system of weighting for each possible response --> score between 0-100 for each of the six overall criteria
1) Pluralism (Plu)
2) Media Independence (Ind)
3) Environment and self-censorship (EnA)
4) Legislative framework (CaL)
5) Transparency (Tra)
6) Infrastructure (Inf)
The overall ascore, the one that determines a country's ranking, is calculated on the basis of these seven scores in a three-step
process.
final score = max(scoA, scoB)
ScoA is calculated on the basis of the questionnaire alone, using a special weighting:
scoA = 1/3*Plu + 1/6*(Ind + EnA + CaL) + 1/12*(Tra + Inf)
- imprisonment of reporters is penalized with a further algorithm (Exa) and included in scoB
scoB: 1/5 * Exa + 4/15*Plu + 2/15*(Ind+Ena*CaL) + 1/15*(Tra + Inf)
Data sources
- Questionnaire, filled by RwB staff and in-country experts
Type of data used
mainly qualitative data
102
FRAME
Quality of data
- depends on respective individual that
results
- score on numerical scale (index), ranking, annual report
Comparability
- intended but limited because of highly subjective judgments
annually
Status
Regular application
Link to website
http://en.rsf.org/; 21.08.2013
103
FRAME
CIVICUS Civil Society Index (CSI)

Author
CIVICUS Alliance
Type of author
CIVICUS is an international alliance consisting of civil society organisations, either citizens organisations or foundations, and of
individual committed citizens united around the common values of citizen action and participation.
CIVICUS convenes many different sectors of civil society in its various projects and activities, and it has more than 300 members
(at the 31st October 2011) from over 80 countries
(See http://www.civicus.org/about-us-125/civicus-alliance/members)
Thematic context
Action research project that aims to assess the state of civil society in countries around the world
Enhance the strength and sustainability of civil society and strengthen civil society's contribution to positive social change by
generating and sharing useful and relevant knowledge on the state of civil society and increasing the capacity and commitment
of civil society stakeholders to strengthen civil society
Rational/purpose
Users/Target Group
designed to assess and score four different dimensions of civil society:

(1) the structure of civil society,
(2) the external environment in which civil society exists and
functions,
(3) the values held and advocated in the civil society arena, and
(4) the impact of activities pursued by civil society actors.
- Civil society organizations (CIVICUS alliance members)
- Civil society,
- policymakers
104
FRAME
Reference to human
rights
Explicit normative stance: CSI necessarily makes normative judgments as to what the defining features of civil society are, what
functions civil society should serve, what values it should embrace, etc. In all of this, the CSI took guidance from universal
standards (e.g. UN Declaration of Human Rights), CIVICUS' own values and the broad academic and practitioner's literature on
civil society's characteristics, roles and enabling factors.
- Civil and political liberties

- touches upon governance, social and economic rights,
73 indicators, grouped into 25 sub-dimensions and 4 dimensions
- Dimension 1 - Structure (covers e.g. the breadth, diversity, spread, and resources of civil society):
> Breadth of citizen participation (undertake political actons, donate to charity, belong to a CSO, do volunteer work, participate
in community activities)
> Depth of citizen participation (how much people give to charity, how much volunteer work,..)
> Diversity within civil society (participation fo women, minorities and other social groups in CSO leadership and
membership,...)
> Level of organisation (existence and effectiveness fo CSO umbrella bodies, efforts to self-regulate,...)
> Inter-relations (information-sharing, alliance-building)
> Resources (financial, human and technological resources to achieve goals)
Types of indicators
- Dimension 2 - Environment (relates to the political and socio-economic and cultural context, the legal environment, the basic
freedoms and rights, the state-civil society relations, and the private sector-civil society relations)
- Dimension 3 - Values (democracy, transparency, tolerance, non-violence, gender equity, poverty eradication, and
environmental sustainability)
- Dimension 4 - Impact (capacity to influence public policy, by the ability to hold state and private corporations accountable, by
the response to social interests, and by the empowering of citizens, and meeting societal needs)
--> aggregated to diamond!
105
FRAME
Geographical area
covered
International
designed to assess and score four different dimensions of civil society:
(1) the structure of civil society,
(2) the external environment in which civil society exists and
functions,
(3) the values held and advocated in the civil society arena, and
(4) the impact of activities pursued by civil society actors.
Each dimension is composed of several subdimensions and, in turn, each sub-dimension is composed of an even larger
number of
individual indicators. The process of implementing the CSI centres on carrying out research and analysis with regard to each
of these indicators. The process of carrying out this research and analysis is considered important in its own right, as an
opportunity for civil society networking, awareness-raising, collective reflection and capacity-building.
Methodology
Process:
1. Identify in-country 3person National Index Team (NIT) made up of:
> National Coordinating Organisation (NCO) - Responsible for the overall coordination and management of the project.
Responsible for undertaking the secondary data review and media review and for preparing the preliminary overview report.
> Civil society expert - Responsible for drafting the country report (potentially in collaboration with other members of
coordinating team).
> Participatory researcher - Responsible for conducting/facilitating regional stakeholder consultations, community- level
research and national workshop.
2. The NIT carries out a preliminary stakeholder analysis and identifies in-country 12- person National Advisory Group (NAG),
representing a diverse set of civil society stakeholders.
3. A review of secondary data is conducted by the NIT and a draft overview report is prepared and distributed to the NAG and
CIVICUS for comment/input.
4. The NAG meets to: (i) review the overview report; (ii) discuss (and adapt as necessary) the proposed project methodology;
(iii) discuss the concept and definition of civil society in the country, (iv) conduct a social forces analysis (an analysis of key
actors and power relations in society at large to help situate/contextualise civil society); (v) create a map of civil society
(charting key forces/actors within civil society and relations between them), and; (vi) assist in identifying conveners and
participants for regional stakeholder consultations.
106
FRAME

5. Primary research is carried out: Regional stakeholder consultations (with up to 20 participants each) are conducted in
different locations in the country. Participants respond to individual questionnaires and subsequently participate in a day- long
group discussion. Community sample research is conducted to investigate, among other things, the value dispositions of
community members, their activities within civil society and attitudes towards and engagement with community-level CSOs. A
review of appropriate media is conducted to gather information on civil society activities, attitudes and values expressed by civil
society and other public actors as well as to establish the media image of civil society. Additionally, fact finding is carried out to
assemble information/data about civil society that already exists but that is not necessarily published or publicly disseminated.
6. All findings are submitted to the civil society expert (and/or drafting team) who prepares a draft country report.
7. The NAG meets to assign scores for indicators based on the draft country report and according to scoring guidelines. These
scores are aggregated into sub-dimension and dimension scores. The scoring results for the four identified dimensions of civil
society (Structure, Environment, Values and Impact) are graphically represented in
the form of a Civil Society Diamond.
8. The draft country report is updated to include the results of the NAG scoring meeting.
9. A national workshop (composed of civil society actors and external stakeholders from government, media, academic
institutions and the business sector) is convened. Participants receive the draft country report prior to the workshop. The goals
of the workshop are to review and validate CSI research findings, to analyse principal
strengths and weaknesses of civil society and to identify potential civil society strengthening activities.
10. Final scores and national workshop results are incorporated into a final country report.
11. CSI Evaluation is conducted. This evaluation is based on the findings from the continuous process documentation as well as
internal monitoring of the CSI undertaken by the NCOs.
SCORING METHODOLOGY:
- NAG scoring exercise by using "citizen jury" approach: group of citizens comes together to deliberate, and makes decisions on
a public issue, based on presented facts. NAG's role is to give a score on each indicator based on the evidence presented by the
NIT.
Data sources
a) Review existing information

- comprehensive review, seek to cover as wide a range of
different data sources as possible
- result: overview report on the state of civil society (basis of the final CSI country report)
- serves to identify data gaps and to determine the nature and extent of primary research
b) Regional stakeholder consultations
107
FRAME

- several different locations in the country
- two steps: 1) select number of informed stakeholders each respond to a questionnaire 2) day-long stakeholder consultation
c) Community sample research
face-to face survey interviews where ordinary members of the community are asked about their involvement in civil society
and their experience with CSOs in their community
d) Media review
(i) gather information about civil society activities reported in the media
(ii) provide insights on how the media perceives and portrays civil society
print media as well as TV/radio
e) Fact finding studies
The fact- finding research consists of several different research methods and studies, including desk reviews, key informant
interviews and two specifically designed studies to gauge the extent of corporate social responsibility and civil societys policy
impact in a
number of selected policy fields.
- Secondary data + various forms of primary data
- Qualitative data + quantitative data
Type of data used
> Draw on all available sources of information Given the lack of secondary data on civil society in many countries, the CSI
makes use of all forms of existing relevant information from all reliable sources and undertakes its own primary (quantitative
and qualitative) research as necessary.
> Appropriately select sources As the CSI seeks to gather information on different aspects of the state of civil society, it is
crucial to select and design appropriate datagathering instruments. There is no single source which can offer all the information
the CSI is looking for. (This is an important lesson learned from the CSIs pilot phase, where the project relied too much on a
single research method, namely a stakeholder survey.) As a result, the CSI proposes a relatively large number of research
methods and a rather resource-intensive research design. This mix of different methods is considered essential to ensure
accurate and useful research outputs.
>30:30Use participatory methods of research a core source of knowledge about civil society is civil society stakeholders
themselves. The CSI uses participatory methods of research to consult with a large number of civil society stakeholders,
soliciting both individual and group responses to a mix of closed and open-ended questions.
Quality of data
Secondary and primary data is used. Broad variety --> generally high quality can be expected
108
FRAME
results
- Indicator scores from 0 to 3

- indicators are aggregated into sub-dimention and dimension scores --> Civil Society Diamond
Comparability
Cross-country: The CSI seeks to generate information about civil society that can be compared across countries. While there is
strong interest at the international level to have access to such cross-country data, the CSI's decision to seek cross-country
comparability is, in fact, based upon demand from national civil society partners. Participants in the pilot phase of the CSI
clearly stated the importance of comparable information to leanr lessons across countries and to identify best (as well as less
successful) practices. There si a tension, however, between seeking "standardised" information that can be compared across
countries and maintaining adequate flexibility to ensure that country-specific factors can be taken into account. The CSI is
specifically designed to achieve an appropriate balance between these two opposing demands.
1997, regular application
Status
Regular application
Comments
- Very comprehensive paper on methodology
Link to website
https://civicus.org/csi/; 02.09.2013
109
FRAME
Torture Scale
Author
Hathaway, Oona A.
Type of author
Researcher
Thematic context
Implementation/effectiveness of human rights treaties
Rational/purpose
Measuring effectiveness of human rights treaties, comparative cross-country analysis of ratification of CAT and instances of torture
Users/Target Group
Researchers: developed and intended only for article "Do Human Rights Treaties Make a Difference" (2002), Hathaway, to analyse
effectiveness of human rights treaties
Reference to human
Normatively based on CAT
rights
Human rights
covered
Freedom from Torture and Degrading Treatment
110
FRAME
Number of
indicators
Types of indicators
Outcome Indicator
Quantitative Indicator
Geographical area
covered
International (166 nations excl US)
Methodology
Coding of US Department of State reports, scale from 1-5 (no/low instances of torture to widespread torture)
Data sources
US Department of State reports
Type of data used
Qualitative data (state reports)
111
FRAME
Quality of data
- Reliance on limited sources (namely only one) is major flaw
providers/processors - US Department State reports are accused of being biased
Presentation of
indicator results
Scale: 1 (no/low instances of torture) - 5
Comparability
- comparability across countries: intended, possible

- comparability over time: not intended
Frequency of
application
- 2002, for 1985-1999 (scores were coded), since 1999 coding of country reports
- applied only once by Hathaway in 2002
Status
Link to website
No website, but file:Hathaway_DoHumanRightsTreatiesMakeaDifference2002_Torture_Scale; 08.08.2013
112
FRAME
WJP Rule of Law Index, VERA-Altus Indicators for the Rule of Law
Author
World Justice Project (WJP)
Type of author
independent, non-profi organization
Thematic context
Rule of Law analysis, monitoring
Rational/purpose
Comprehensive analysis, monitoring and assessment of states' adherence to the rule of law in practice throughout a variety of
dimensions, identify changes and compare countries among each other
Users/Target Group
- policymakers
- businesses
- NGOs
- other constitutiencies
Reference to human
rights
Based on four universal principles of the Rule of Law:

> The government and its officials and agents as well
as individuals and private entities are accountable
under the law.
> The laws are clear, publicized, stable, and just, are
applied evenly, and protect fundamental rights,
including the security of persons and property.
> The process by which the laws are enacted,
administered, and enforced is accessible, fair, and
efficient.
> Justice is delivered timely by competent, ethical,
and independent representatives and neutrals who
are of sufficient number, have adequate resources,
113
FRAME
and reflect the makeup of the communities they

serve.
The principles address the extent to which a country protects fundamental human rights. But given the impossibility of
assessing
adherence to the full panoply of civil, political, economic, social, cultural, and environmental rights recognized in the Universal
Declaration, the principles treat a more modest menu of rights, primarily civil and political, that are firmly established under
international law and bear the most immediate relationship to rule
of law concerns.
Rule of Law
> 400 variables (48 sub-factors, 9 dimensions)
9 dimensions:
- Limited government powers
(i.e. sub-factors: 1.1 Government powers are defined in the fundamental law
1.2 Government powers are effectively limited by the legislature
1.3 Government powers are effectively limited by the judiciary
1.4 Government powers are effectively limited by independent auditing and review
1.5 Government officials are sanctioned for misconduct
1.6 Government powers are subject to non-governmental checks
Types of indicators
1.7 Transition of power is subject to the law)
- Absence of corruption
- Order and security
- Fundamental rights
- Open government
- Regulatory enforcement
- Civil justice
- Informal justice
114
FRAME
Geographical area
covered
2012-2013 Index assessed 97 countries (more than 90 % of world's population)
Methodology
ten steps:
1. conceptual framework (9 factors and 48 sub-factors) in consultation with academics, practitioners, and community leaders
2. Index team developed five questionnaires to be administered to experts general public. adapted to reflect commonly used
terms and
expressions
3. more than 300 potential local experts per country to
respond to questionnaires, engaged services of leading
local polling companies
4. Polling companies conducted pre-test pilot surveys of the general public in consultation with the Index team, and launched
the final survey
5. team sent questionnaires to local experts + continual interaction
6. Index team collected and mapped data onto the 48 sub-factors.
7. Index team constructed final scores using five-step process:
a. Codified the questionnaire items as numeric values.
b. Produced raw country scores by aggregating the responses from several individuals (experts or general public).
c. Normalized the raw scores.
d. Aggregated the normalized scores into sub-factors and factors using simple averages.
e. Produced the final rankings using the normalized scores.
8. data were subject to a series of tests to identify possible biases and errors. (cross-check of all sub-factors against more than
60 third-party sources, including quantitative data and qualitative assessments drawn from local and international
organizations.)
9. sensitivity analysis by the Econometrics and Applied Statistics Unit of the European Commissions Joint Research Centre, in
collaboration with the Index team, to assess the statistical reliability of the results.
10. data were organized into country reports, tables, and figures to facilitate their presentation and interpretation.
Data sources
almost entirely based on new data collected by the WJP from independent sources
- variables (indicators) are drawn from two novel data sources collected by the WJP in each country:
(1) general poplation poll (GPP) conducted by leading local polling companies using a representative sample fo 1,000
115
FRAME

respondest in the three largest cities in each country; and
(2) qualified respondents' questionnaires (QRQ) consisting of closed-ended questions completed by in-country practitioners
and academics with expertise in civil and commercial law, criminal justice, labor law, and public health.
Type of data used
quantitative data (see data sources)
Quality of data
- data is generated by WJP itself (poll and questionnaire is conducted by partner institutions (market research institutes)
- data cross-checks are performed (triangulation)
results
Scores, ranking, report with vivid graphs
Comparability
- Comparability over time: not possible, data is normalized across entire sample --> when new countries are included,
comparison is not possible
- comparability across countries: intended, possible (data is generated by independent institute, and not country itself)
2009, annually
116
FRAME
Comments
Link to website
http://worldjusticeproject.org/; 03.09.2013
117
FRAME
VERA-Altus Indicators for the Rule of Law

Author
Vera Institute of Justice (in collaboration with American Bar Association's World Justic Project, and 27 researchers from Altus Global
Alliance member organizations in Chile, India, Nigeria and the US)
Type of author
Independent, nonpartisan, nonprofit center for justice policy and practice in the US
Thematic context
Assessment of justice systems (developed for US, applied to other countries as well)
Rational/purpose
- Inform civil society, policymakers, experts about shortcomings of justice systems (courts, law enforcement, immigration, social
services)
- using strong individual indicators, that, when used in clusters, can reliably measure crucial aspects of the rule of law such as
transparency, participation, and equal access to justice
Users/Target Group
- (local) policymakers
- justice system professionals and experts
- civil society
- complements World Justice Project's Rule of Law Index (WJP Index)
Reference to human
- translates de jure statements of WJP Index to de facto measures
rights
- rooted in UN covenants
Human rights
covered
Rule of Law, civil and political rights
Number of
indicators
60 (13 core sub-sectors)

Indicators are divided into 13 sub-sectors:
Types of indicators
118
FRAME

- Transparency
- Participation
- Bias in Public Administration
- Bias in Franchises and Public Contracts
- Bias in Enforcement
- Police (Indicators: i.e. Public perceptions of police fairness, Police promotions are based on competence/merit, Salary of entry level
police as percentage of area median income of households and individuals)
- Judiciary
- Non-State or Informal Justice Mechanisms (Indicators: i.e. Consistency of outcome, disaggregated by SES, Public perceptions of the
fairness of non-state or informal justice mechanism, NGO reports of human rights abuses by nonstate or informal justice
mechanisms)
- Prisons
- Information/Awareness
- Accessibility
- Legal Representation
- Engagement (Indicators: i.e. Proportion of public trials involving poor victims, Proportion of crime complaints that are investigated
by police, disaggregated by complainant SES/gender/complaint type)
Geographical area
covered
Methodology
intended to be global, so far 4 cities (Chandigar, Lagos, Santiago, New York)

The Indicators Project focuses only on two of the four bands in the WJP Index. These bands describe the process for enacting,
administrating, and enforcing laws, equality of access to justice, and the operation of justice institutions. As a first step, we distilled
each of the bands and their factors and subfactors into 13 core principles informed by United Nations covenants. Second, the team
developed
between three and seven indicators for each of the principles, resulting in a total of 60 indicators. This was an iterative process and
involved internal discussions at Vera, consultations with the projects expert advisors, and several rounds of written feedback from
the project teams in each of the pilot sites. Finally, the team created operational definitions for each of the indicators to guide
selection of measures and defined minimum standards for data collection.
119
FRAME
Data sources

- Public surveys
- expert surveys
- administrative data
- documents and legislation
- third-party reports
- case study/observation
Type of data used
Qualitative and quantitative data
Quality of data
Variety of data: quality differs, no information about i.e. which third-party reports are used...
Presentation of
indicator results
Report with interpretation of findings, many indicators are quantitative in nature: yes/no questions
Comparability
intended, only applied to four cities
Frequency of
application
developed in 2003, applied to four cities: Chandigahr (India), Lagos (Nigeria), Santiago (Chile), and New York (USA)
Status
Theory developed and first applications made (according to Report from 2008: six month pilot project; no follow up information
available)
Comments
- No scores or other unified measurement, findings are presented as combined quantitative-qualitative statement (i.e. 35% of
respondents said they do and 53% that they do not believe that non-state or informal justice mechanisms are fair in their decisions.
32% said that these mechanisms treat men and women equally and 58% said that they do not.)
- Raw data not accessible, methodology not entirely clear
120
FRAME
Link to website
http://www.altus.org/index.php?option=com_content&view=article&id=9&Itemid=21&lang=en#; 03.09.2013
121
FRAME
RED
Author
RED Network (independent research network building the RED early warning system composed of 17 Research and Civil Society
Organisations in EU Member States, i.e. Institute for Rights Equality & Diversity i-RED (leader, Greece), Ludwig Boltzmann
Institute, European Forum for Migration Studies - University of Bamberg,...)
Users/Target Group
Type of author
independent research organizations
Thematic context
Discrimination/Racism/Xenophobia Monitoring
- European Early Warning System for Discrimination, Racism, Xenophobia, Hate Crimes,
- Main purpose: awareness raising
Rational/purpose
The RED Early Warning System aims at covering the lacunae in monitoring hate crime and discrimination by:
- timely reporting and response to racism, hate crime and discrimination phenomena in contrast to traditional
reporting which is always important but not responding to the need to timely monitor the real situation and inform
the public and policy makers. Traditional generic reports are finalized and published many months after the
occurrence of the phenomena.
- Focusing at national level and by identifying and analyzing in-depth, confronting and countering racist violence and
attitudes-stereotypes. These gaps, in timely or real-time reporting and in national-regional focus of analysis, persist in the very
moment that most EU governments do not provide for nation-wide monitoring and research centres on racism
and discrimination phenomena. The available information and knowledge about what is really happening in the
Member States and why, remains very much anecdotal, coming almost exclusively from activist or media sources
and is not further interpreted, contextualized, EU compared and therefore, efficiently comprehended by
122
FRAME

policy makers either on a national or EU level. Not each and every incident is reported on the map. The
RED System reports on the most important ones, as well as on the significant hate crime alerts and policy
responses or positive initiatives. These are selected by RED Network national partners.
Nevertheless, national partner organisations reserve the option to report all national level single incidents if they
wish to. In this way the RED system offers the technical infrastructure for national level reporting/monitoring
mechanism for all EU Member States. The RED National Experts select hate crime alerts when
one or more of the following criteria to be assessed by the national partner - are met:
n Major public or media attention - NGO highlight
n Duration sustained impact-significance chronic pattern
n Targeting-discriminating-victimising specific groups
n Serious violence - murder
n High intercommunity tensions protests - clashes
n Group incidents of same type(s)-victim group(s)s
n State reaction-response (police-judicial-government) or lack of it despite public critique
Users/Target Group
- Mainly: civil society (awareness raising)

- specialized publics,
- media,
- policymakers,
- stakeholders,
- scholars,...
Reference to human
rights
Discrimination, no explicit normative base, but limited to Europe, so some kind of consensus can be expected what
discrimination is (ECHR Art 14, )
human rights?
123
FRAME
Discrimination, Equality, Minority rights
125 indicators (based on 125 questions)

and 38 key statistics and demographic figures

Structure, Process and Outcome Indicators
Most indicators are binary due to their qualitative nature
Types of indicators
Indicators (areas):
- Anti- discrimination Legislation & Implementation:
i.e. RED1 Is racial discrimination defined in national law?
RED2 Is there a definition of discrimination on the grounds of race, ethnic origin and/or religion in national law in conformity
with the EU Directives?
RED3 Does the national law cover all grounds of discrimination as in the International Conventions and EU law or additional
discrimination grounds?
RED4 Does nationality, citizenship serve as a ground for discrimination in access to public goods and services, employment
and/or deprivation of economic and social rights?
- Anti-racist Crime Legislation & Implementation: i.e.
RED20 Is there legislation against racist and hate crime?
RED25 Is there an independent assessment of the impact of anti-racist legislation and its application in practice?
RED26 Is there an estimate or evidence that hate crime cases/incidents are under-reported disproportionately in relation to
other crimes?
- Political Parties organisations- Racist & Xenophobic Discourse
- Anti-racist Policies & Organisations
- Policing-Law Enforcement-Justice
- Housing & Segregation
- Education
- Health And Social Protection
- Political & Civic Participation
124
FRAME

- Policies On Integration - Cohesion
- Public Life, Culture, Sport & Media
- Policies On Integration - Cohesion
- Public Life, Culture, Sport & Media
Statistics and Demographics:
- Migration & Minority Demographics/Statistics
- Migration & Minority Economics
- Racist violence - Hate Speech Statistics
- Discrimination Statistics
Geographical area
covered
EU MS
Methodology
- No in-depth information on methodology

- issues with RED as found by Red network annual report 2012, p 12:
> Varying definitions
> Data availability/quality depends on quality of national statistical agencies, other national authorities and varies substantially
> Differences in legal systems (even within EU, and EU Directives on discrimination implementation varies, and comparability is
difficult)
> Cultural and political contexts: different views on minorities,...
Data sources
- classic reporting and evaluation by independing authoritative sources

- probably only national authorities?
125
FRAME
Type of data used
Qualitative Data (provided mainly by national authorities)
Quality of data
Depends on the quality of national authorities and, thus, varies
results
Annual reports, Yes/no for each question, or respective number (only very few quantitative indicators)
Comparability
Limited, due to above mentioned limitations (Methodology):

> Varying definitions
> Data availability/quality depends on quality of national statistical agencies, other national authorities and varies substantially
> Differences in legal systems (even within EU, and EU Directives on discrimination implementation varies, and comparability is
difficult)
> Cultural and political contexts: different views on minorities,...
2012, supposed to be annually
Status
126
FRAME
- Too many indicators,
- no conclusions possible,
- no aggregation to index
Comments
- mainly qualitative questions, therefore mainly yes/no answers (binary)
- no in-depth information about methodology, sources,
Link to website
http://www.red-network.eu/?i=red-network.en.home; 27.08.2013
127
FRAME
C.
Human Rights-related Indicators
Human rights related information systems are datasets, indicators or information models which have certain relevance for human rights or take human rights
aspects into account for their information purpose. The topics range from governance and development indicators over labour market indices to peace- and
conflict data.
1.
Intergovernmental and Governmental Organisations
Worldwide Governance Indicators (World Bank)

Author
World Bank
Users/Target Group
Type of author
IO
Thematic context
- Measuring governance
- research dataset summarizing the views on the quality of governance
- data are provided by a large number of enterprise, citizen and expert survey respondents in industrial and developing countries
- data are gathered from a number of survey institutes, think tanks, non-governmental organizations, international organizations,
and private sector firms.
- not used by the World Bank Group to allocate resources
Rational/purpose
Measuring governance:
- traditions and institutions by which authority in a country is exercised
- process by which governments are selected, monitored and replaced
- capacity of the government to effectively formulate and implement sound policies
- respect of citizens and the state for the institutions that govern economic and social interactions among them
128
FRAME
Users/Target Group
- Policy makers (decision making process)

- World Bank
- Private companies? (investment decision-making)
- Researchers
- Civil society
Reference to human
rights
Implicitly refers to human rights
human rights?
Human rights covered Civil Liberties and Political Freedoms

6 indicators
129
FRAME
Types of indicators

Structural, Procedural and Outcome Indicators
6 dimensions:
- Voice and Accountability
- Political Stability and Absence of Violence
- Government Effectiveness
- Regulatory Quality
- Rule of Law
- Control of Corruption
Geographical area
covered
international (215 economies)

- 30 individual data souces
- data is rescaled and combined to create six aggregate indicators using a statistical methodology (unobserved components model)
- generates margin of error: has to be taken into account when making comparisons across countries and over time
Methodology
3 STEPS:
STEP 1: Assigning data from individual sources to the six aggregate indicators. Individual questions from the underlying data
sources are assigned to each of the six aggregate indicators. For example, a firm survey question on the regulatory environment
would be assigned to Regulatory Quality, or a measure of press freedom would be assigned to Voice and Accountability. (See i.e.
http://info.worldbank.org/governance/wgi/pdf/va.pdf)
STEP 2: Preliminary rescaling of the individual source data to run from 0 to 1. The questions from the individual data sources are
first rescaled to range from 0 to 1, with higher values corresponding to better outcomes. If, for example, a survey question asks for
responses on a scale from a minimum of 1 to a maximum of 4, we rescale a score of 2 as (2-min)/(max-min)=(2-1)/3=0.33. When an
individual data source provides more than one question relating to a particular dimension of governance, we average together the
rescaled scores.
The 0-1 rescaled data from the individual sources are available interactively through the WGI website here, in the country data
130
FRAME

sheets, and in the data files for each individual source. Although nominally in the same 0-1 units, this rescaled data is not necessarily
comparable across sources. For example, one data source might use a 0-10 scale but in practice most scores are clustered between
6 and 10, while another data source might also use a 0-10 scale but have responses spread out over the entire range. While the
max-min rescaling above does not correct for this source of non-comparability, the procedure used to construct the aggregate
indicators does.
STEP 3: Using an Unobserved Components Model (UCM) to construct a weighted average of the individual indicators for each
source. A statistical tool known as an Unobserved Components Model (UCM) is used to make the 0-1 rescaled data comparable
across sources, and then to construct a weighted average of the data from each source for each country. The UCM assumes that
the observed data from each source are a linear function of the unobserved level of governance, plus an error term. This linear
function is different for different data sources, and so corrects for the remaining non-comparability of units of the rescaled data
noted above.
The resulting estimates of governance are a weighted average of the data from each source, with weights reflecting the pattern of
correlation among data sources.
The UCM assigns greater weight to data sources that tend to be more strongly correlated with each other. While this weighting
improves the statistical precision of the aggregate indicators, it typically does not affect very much the ranking of countries on the
aggregate indicators. The composite measures of governance generated by the UCM are in units of a standard normal distribution,
with mean zero, standard deviation of one, and running from approximately -2.5 to 2.5, with higher values corresponding to better
governance. We also report the data in percentile rank term, ranging from 0 (lowest rank) to 100 (highest rank).
(See http://info.worldbank.org/governance/wgi/resources.htm#cross)
Data sources
30 individual data sources: surveys and expert assessments generated by NGOs, companies,
- Mainly quantitative data from secondary sources, additional qualitative data is used
Type of data used
- Surveys of households and firms (9 data sources including the Afrobarometer surveys, Gallup World Poll, and Global
Quality of data
Competitiveness Report survey),
providers/processors - Commercial business information providers (4 data sources including the Economist Intelligence Unit, Global Insight, Political Risk
Services),
131
FRAME

- Non-governmental organizations (9 data sources including Global Integrity, Freedom House, Reporters Without Borders), and
- Public sector organizations (8 data sources including the CPIA assessments of World Bank and regional development banks, the
EBRD Transition Report, French Ministry of Finance Institutional Profiles Database)
List of sources:
> African Development Bank Country Policy and Institutional Assessments (ADB)
> African Electoral Index (IRP)
> Afrobarometer (AFR)
> Asian Development Bank Country Policy and Institutional Assessments (ASD)
> Business Enterprise Environment Survey (BPS)
> Bertelsmann Transformation Index (BTI)
> Cingranelli Richards Human Rights Database (HUM)
> European Bank for Reconstruction and Development Transition Report (EBR)
> Economist Intelligence Unit (EIU)
> Freedom House (FRH, CCR)
> Global Corruption Barometer Survey (GCB)
> Global Competitiveness Report (GCS)
> Global Insight Business Condition and Risk Indicators (WMO)
> Global Integrity Index (GII)
> Gallup World Poll (GWP)
> Heritage Foundation Index of Economic Freedom (HER)
> IFAD Rural Sector Performance Assessments (IFD)
> iJET Country Security Risk Ratings (IJT)
> Institute for Management &Development World Competitiveness Yearbook (WCY)
> Institutional Profiles Database (IPD)
> International Research & Exchanges Board Media Sustainability Index (MSI)
> International Budget Project Open Budget Index (OBI)
> Latinobarometro (LBO)
> Political Economic Risk Consultancy (PRC)
> Political Risk Services International Country Risk Guide (PRS)
> Political Terror Scale (PTS)
> Reporters Without Borders Press Freedom Index (RSF)
> US State Department Trafficking in People report (TPR)
132
FRAME

> Vanderbilt University's AmericasBarometer (VAB)
> World Bank Country Policy and Institutional Assessments (PIA)
Presentation of
indicator results
- Numerical scores from -2.5 to 2.5 (corresponding to 0.005 and 0.995 percentiles of standard normal distribution)
- Excel, stata and interactive data tool
Comparability across countries:
The six aggregate indicators are reported in two ways: (1) in their standard normal units, ranging from approximately -2.5 to 2.5,
and (2) in percentile rank terms from 0 to 100, with higher values corresponding to better outcomes.
- all country scores are accompanied by standard errors: reflect the number of sources available for a country and the extent to
which these sources agree with each other (with more sources and more agreement leading to smaller standard errors).
- reflect the reality that governance is difficult to measure using any kind of data
Comparability
- graphical presentation of the data: 90 percent confidence intervals or "margins of error".

- indicate the statistically-likely range for governance based on the available data for a country
- If when comparing two countries these margins of error overlap, the difference between the two countries should not be
interpreted as signaling a statistically significant difference
(See http://info.worldbank.org/governance/wgi/resources.htm#cross)
Comparability over time:
- short year-to-year periods: difficult to measure with any kind of data, typically quite small
- do not focus on short-run year-to-year changes but rather in trends over longer periods
- margins of error: if confidence interval overlap, data should not be interpreted as signaling meaningful changes
- majority of year-to-year changes in the WGI are too small relative to margins of error to be viewed as statistically, or practically,
significant, and so should not be over-interpreted as indicating a significant change in governance performance
- over longer periods of time such as a decade, the WGI data do show significant trends in governance in a number of countries.
133
FRAME

- Changes over time reflect combination of three factors
(i) changes in underlying source data,
(ii) addition of new data sources
(iii) changes in the weights used to aggregate the individual sources - large changes: changes in the underlying source data are most
often the most important of these three factors
- smaller and often insignificant changes over shorter periods: a combination of all three factors contributes to changes in country
scores.
Frequency of
application
1996, every two years between 1996 and 2002, annually since 2002
used for ad hoc purposes
Status
Regular application
Link to website
http://info.worldbank.org/governance/wgi/resources.htm#sources; 23.08.2013
134
FRAME
Millenium Development Goals Indicators

UN Secretary-General;
Based on UN GA Resolution A/RES/60/1, World Summit 2005
Author
Type of author
Thematic context
Development Indicators
Rational/purpose
Monitoring of Millenium Development Goals. Main purpose is to measure progress towards MDGs and to inform community of
states (UN GA) and civil society about it.
Users/Target Group
Community of States (UN GA), Civil Society;

Indicators are published in annual reports of UN SG to UN GA and periodic country reports.
135
FRAME
Reference to human
rights
- Indicators are primarily aimed at measuring development with regard to Millenium Development Goals, which are defined as
targets.
- Normatively based on UN Millenium Declaration
- Human Rights are incorporated in this concept of development and, therefore, these indicators can be seen as aiming to
measure human rights as well.
- Right to Development
- Right to Education
- Right to Equality and Non-Discrimination
- Right to Health
- Right to a Safe, Healthy and Ecologically-Balanced Environment
48 indicators (21 targets)
Types of indicators
Quantitative indicators
Geographical area
covered
International level
136
FRAME
Methodology
- Indicators have been established by UN SG under advice from experts from IMF, OECD, World Bank, DAC (OECD)
- Indicators have been developed relative to target
- Quantitative indicators based on quantitative data: computation
Data sources
1) Country data (where available and reasonable quality)

- Data source & quantitative value: decided by consensus among key stakeholders in respective countries (National surveys,
country statistics, censuses, estimates from sample surveys, direct and indirect estimation techniques
- national statistical system: should own data and related indicators)
- Existing data sources & reporting systems: should be used where possible, particularly where line ministries have their own
statistical systems.
2) International data
- For global reporting, use is made of indicators compiled by international organizations. Internationally compiled indicators,
based on standard concepts, definitions and methodologies, more readily facilitate crosscountry comparisons.
- World Bank, ILO, FAO, UNICEF; UNHCR; UNESCO, ILO, IPU, UN WOMEN, WHO, UNFPA, UN Population Division, UNAIDS, CDIAC,
UNEP, UNFCCC, IUCN, UNEP-WCMC, UN-Habitat, ITC, UNCTAD, OECD, ITU (See The Millenium Development Goals Report 2013,
60.)
- The Millenium Development Goals Report 2013, 58: weighted average of country data; individual agencies are desigated as
official providers of data and as leaders in developing methodologies for data collection and analysis; data typically drawn from
official statistics provided by governments; additionally, surveys sponsored and carried out by international agencies
Type of data used
Quantitative data
137
FRAME
Quality of data
Providers: governments, international agencies.

Processors: UN Inter-Agency and Expert Group on MDG Indicators (IAEG)
Presentation of indicator - Percentage change of values compared to baseline (baseline year: 1990)
- annual publication as reports with interpretation of data
results
Comparability
International comparison among UN member states has been explicitly mentioned as selection criterion for indicators.
Is a comparison of the indicator results possible/intended across different countries/regions etc.? Is such a comparison the purposes of the indicator scheme?
Frequency of application Annual reports, periodicity of measurement varies from indicator to indicator (from annually to every 10 yrs)
Status
Annually since 1990 (a few indicators have been added in 1999, 2005, 2009, 2010)
138
FRAME
Comments
- Indicators have been developed relative to goals/targets:

i.e. Goal 1: Eradicate extreme poverty and hunger.
Target 1.A: Halve, between 1990 and 2015, the proportion of people whose income is less than one dollar a day
Indicators for Target 1.A:
> 1.1 Proportion of population below $1 (PPP) per daya
> 1.2 Poverty gap ratio
> 1.3 Share of poorest quintile in national consumption
Target 1.B: Achieve full and productive employment and decent work for all, including women and young people
Indicators for Target 1.A:
> 1.4 Growth rate of GDP per person employed
> 1.5 Employment-to-population ratio
> 1.6 Proportion of employed people living below $1 (PPP) per day
> 1.7 Proportion of own-account and contributing family workers in total employment
Target 1.C: Halve, between 1990 and 2015, the proportion of people who suffer from hunger
> 1.8 Prevalence of underweight children under-five years of age
> 1.9 Proportion of population below minimum level of dietary energy consumption
- First set of indicators was developed in 2002 (following Millenium Declaration) and used until 2007. Now revised set with two
new indicators is in place (following World Summit).
Link to website
http://mdgs.un.org/unsd/mdg/Default.aspx, 18.07.2013
139
FRAME
International Human Development Indicators

Author
UNDP, Human Development Report Office
Type of author
Thematic context
Rational/purpose
Information on human development status
Users/Target Group
NGOs, governments, researchers, civil society;

Annual publication of HDI, Human Development Report
140
FRAME
Reference to human
rights

> Development Indicators
> Refer to HR explicitly in some cases: child labour,
> Human rights related indicators, such as
- literacy rate
- school enrolment
- adult mortality rate
- gender inequality index (shares in parliament,..)
- internet users,...
human rights?
- Life, Health
- Education
- Equality and Non-Discrimination
- Environment, Natural Resources
- Property
- Economic Rights: Employment, Access to technology (internet, electricity,...)
- Children's rights (child labour)
141
FRAME
11 main areas with 2 to 20 indicators:

- Demography
- Education
- Gender
- Health
- Human Security
- Income
- Inequality
- Innovation and technology
- Poverty
- Sustainability
- Trade economy and income
Types of indicators
- Mainly objective data, limited use of subjective data (overall life satisfaction, satisfaction with freedom of choice, satisfaction
with community,)
- Quantitative data
Geographical area
covered
- (International) Human Development Report

- regional, national and local HDRs (over 600 reports in over 140 countries)
142
FRAME
Methodology

HDI has been established by Amartya Sen and Mahbub ul Haq: weighting of income, health and education;
- Quantitative Indicators based on quantitative data: computation
-
Data sources
Human Development Report 2013: Statistical Annex, 199.

- International agencies (i.e. UN agencies, CDIAC, CRED,...)
- Independent studies (i.e. Alkire, S., A. Conconi, and J.M. Roche. "Multidimensional Poverty Index 2012: Brief Methodological
Note and Results." University of Oxford, Department of International Development, Oxford Poverty and Human Development
Initiative, Oxford, UK.)
Type of data used
Quantitative data
The Human Development Report Office (HDRO) is a data user. It does not collect data directly from national statistical systems
but uses indicators produced by United Nations Agencies and affiliates with data collection, compilation and dissemination
mandates.
Quality of data
Major sources for HDR 2013: http://hdr.undp.org/en/statistics/understanding/sources/

- Barro-Lee Dataset
- Carbon Dioxide Information Analysis Center (CDIAC)
- Centre for Research on the Epidemiology of Disasters (CRED)
- Demographic and Household Surveys (DHS)
- Development Research Centre on migration, Globalisation and Poverty
- Food and Agriculture Organization (FAO)
- Gallup World Poll
143
FRAME

- Global Footprint Network
- ICF Macro Demographic an dHealth Surveys
- Inter-Paliamentary Union (IPU)
- Internal Displacement Monitoring Centre (IDMC)
- International Energy Agency (IEA)
- International Institute for Strategic Studies (IISS)
- ILO
- IMF
- International Telecommunication Union (ITU)
- International Union for Conservation fo Nature and Natural Resources (IUCN)
- UNAIDS
- Luxembourg Income Study (LIS)
- National Bureau of Economic Research
- Organisation for Economic Co-operation and Development (OECD)
- Penn World Table (PWT)
- Stockholm International Peace Research institute (SIPRI)
- UNICEF
- UNCTAD
- UNESCO
- UNHCR
- UN Treaties Deposited with the SG
- UNODC
- UNPOP
- UNSD
- World Bank
- WHO
- WIPO
- Yale Center for Environmental Law and Policy
144
FRAME
results

- Indicatorspresented as absolute values, rates or percentages
- included in weighted indices:
> Human Development Index
> Inequality-adjusted HDI
> Gender Inequality Index
> Multidimensional Poverty Index
Comparability
- Comparability among countries is a major goal. UNDP presents annual country ranking.
- However, because national and international data agencies continually improve their data series, the dataincluding
the HDI values and rankspresented in annual Reports are not comparable to those published in earlier editions. For the HDI,
trends using consistent data calculated at five-year intervals for 19802012.
- National and international data estimates can vary because
international agencies harmonize national data for comparability
across countries, produce an estimate of missing data or do
not incorporate the most recent national data. When HDRO
becomes aware of discrepancies, these are brought to the attention
of national and international data authorities.
Annuallysince 1990
Status
Regular application
145
FRAME
Comments
Link to website
http://hdr.undp.org, 22.07.2013
146
FRAME
World Bank Indicators, World Development Indicators

Author
World Bank, World Bank Development Data Group
Type of author
Thematic context
Rational/purpose
Measure effectiveness of financial and technical assistance for developing countries
Users/Target Group
- World Bank
- Millenium Development Goals
- Government agencies for development cooperation
- research
- civil society (i.e. as World Development Report on various topics)
147
FRAME
Reference to human
rights
- prevalence of economic indicators

- hr related indicators, such as education/health/gender related indicators
human rights?
No normative base for hr indicators explicitly mentioned

- Economic Rights: Employment, Internet, Electricity
- Equality and Non-Discrimination
- Education
- Health
> more than 8000 indicators (http://data.worldbank.org/)
> divided into following areas:
- Agriculture & Rural Development
- Aid Effectiveness
- Climate Change
- Economic Policy & External Debt
- Education
- Energy & Mining
- Environment
- Financial Sector
- Gender
- Health
- Infrastructure
- Labor & Social Protection
- Poverty
- Private Sector
- Public Sector
- Science & Technology
- Social Development
- Urban Development
148
FRAME
Types of indicators
- Quantitative indicators
Geographical area
covered
- international, regional and national (214 economies)
Methodology
-Quantitative indicators based on quantitative data: computation

(detailed information on methodology: See http://data.worldbank.org/about/data-overview/methodologies)
Data sources
primarily official sources + surveys:

- Country data
- International and government agencies: i.e. CDIAC, GIZ, FAO,
- Private and nongovernmental organisations: i.e. Conainerisation International, DHL, IIS, IRF, PWC,
See http://data.worldbank.org/about/wdi-partners
Type of data used
- Quantitative data
- Mainly objective data, limited use of subjective data (overall life satisfaction, satisfaction with freedom of choice, satisfaction
with community,)
149
FRAME
Quality of data
providors/processors
- member states: data for about 210 economies mainly provided by national statistical agencies
- supported by international agencies and NGOs
-absolute values
- percentages, rates,
- structured by country, by topic, aggregates for geographic regions, income groups,
-comparability across countries: intended, but depends on quality of data provided by countries and their and national
statistical agencies
Comparability
- comparability over time: for most countries times series from 1970 onwards
Frequency of application Annual application since 1960, updated quarterly
Regular application
Status
results
Link to website
http://data.worldbank.org, 24.07.2013
150
FRAME
KILM - Key indicators of the labour market

Author
ILO
Type of author
IO
Thematic context
Labour market analyses
Rational/purpose
- Monitoring progress towards MDG,

- monitoring equity in labour market,
- assessing employment,
- identifying "best practices",
- policymakers
- researchers
- civil society
Users/Target Group
Reference to human
rights
no explicit reference to human, but some indicators are relevant for employment, equality, discrimination etc.
- Employment
- Equality
151
FRAME
18 indicators
Quantitative indicators, mostly outcome indicators
(KILM 1. Labour force participation rate
KILM 2. Employment-to-population ratio
KILM 3. Status in employment
KILM 4. Employment by sector
KILM 5. Employment by occupation
KILM 6. Part-time workers
KILM 7. Hours of work
KILM 8. Employment in the informal economy
KILM 9. Unemployment
Types of indicators
KILM 10. Youth unemployment
KILM 11. Long-term unemployment
KILM 12. Time-related underemployment
KILM 13. Inactivity
KILM 14. Educational attainment and illiteracy
KILM 15. Average monthly wages
KILM 16 Hourly compensation costs
KILM 17. Labour productivity
KILM 18. Poverty, income distribution and the working poor)
Geographical area
covered
International
Methodology
quantitative data is used for quantitative indicator: mathematical calculation, explicit information for each indicator
152
FRAME
Data sources
- ILO concentrates on bringing together information from international repositories: rarely collects information directly from
national sources, but these organizations rely heavily on national sources and/or official national publications:
> ILO Department of Statistics
> OECD
> Eurostat
> WB
> The Conference Board
> UNESCO Institue of Statistics
> US Bureau of Labor Statistics
Type of data used
Quantitative data
Quality of data
Varies, depends on the quality of the national statistical agency providing the data (230 countries and territories)
results
- annual publication of KILM (report)
Comparability
-Comparability across countries: intended; comparability across countries is criterium for selection of KILM indicators, section
about comparability for each KILM indicator
- Comparability over time: intended (ongoing time-series is intended, but cannot be guaranteed)
1999, 7th edition in 2011
153
FRAME
Status
Regular application
Link to website
http://www.ilo.org/empelm/what/WCMS_114240/lang--en/index.htm; 02.08.2013
154
FRAME
ILOSTAT Database
Author
ILO
Type of author
IO
Thematic context
Labour market analyses
Rational/purpose
Situational analysis and monitoring of labour market
- researchers
- policymakers
- civil society,
Users/Target Group
Reference to human
rights
no explicit reference to human rights, but some indicators are relevant for employment, equality, discrimination etc.
- Employment
- Equality
> 100
Types of indicators
155
FRAME
Geographical area
covered
International
- statistical data provided by countries is used for quantitative indicator: mathematical calculation, explicit information for each
indicator
Methodology
- "metadata driven" -> "labour statistics: sources and methods" (still for Laborsta, but will be provided for ILOSTAT as well, See
http://laborsta.ilo.org/applv8/data/SSMe.html) provides detailed information about surveys, methods, etc. used by countries
Data sources
- draws heavily on official data

> Official data from national authorities (population census, household surveys, child labour surveys,)
> Insurance records
> Records of employers' and workers' organizations
> Other
Type of data used
Quantitative data
Quality of data
- Varies, depends on the quality of the national statistical agency providing the data (230 countries and territories)
- various Conventions, Recommendations, Resolutions and Guidelines in order to ensure standardized approach of national
agencies (See http://www.ilo.org/global/statistics-and-databases/standards-and-guidelines/lang--en/index.htm)
results
- Indicators presented as absolute values, rates or percentages

- Interactive map (colorcoded)
156
FRAME
Comparability
-Comparability across countries: intended, but limited due to reliance on national statistical agencies and their different
methodologies; various Conventions, Recommendations, Resolutions and Guidelines in order to ensure standardized approach
of national agencies (See http://www.ilo.org/global/statistics-and-databases/standards-and-guidelines/lang--en/index.htm)
- Comparability over time: intended (also prior to 2008, ILOSTAT data is included, ongoing time-series is intended, but cannot be
guaranteed),
2008, successor of Laborsta, annually, short term indicators dataset with monthly, quarterly and semi-annual data (updated on
monthly basis)
Status
Annual application
Link to website
http://www.ilo.org/ilostat/faces/home/statisticaldata?_adf.ctrl-state=jq2b941zp_4&clean=true&_afrLoop=870058169613902;
02.08.2013
157
FRAME
Paris Declaration Framework for Aid Effectiveness

Author
OECD
Type of author
IO
Thematic context
development aid
Rational/purpose
- Measuring progress towards year 2010 targets of development aid within OECD
- "The Indicators of Progress provide a framework in which to make operational the responsibilities and accountabilities that are
framed in the Paris Declaration on Aid Effectiveness. They measure principally collective behaviour at the country level."
- providing a benchmark against which individual
donor agencies or partner countries can measure their performance at the country, regional, or global level.
- OECD, member states
Users/Target Group
Reference to human
implicitly touches upon human rights
rights
Human rights
covered
Not aimed at measuring human rights, but covers right to development, and to some degree good governance
Number of
indicators
13 indicators (9 targets)

i.e.
- goal: Partners have operational development strategies; target: At least 75% of partner countries have operational development
Types of indicators
strategies; Indicator: Number of countries with national development strategies (including PRSs) that have clear strategic priorities
158
FRAME

linked to a medium-term expenditure framework and reflected in annual budgets.
- goal: Strengthen capacity by co-ordinated support; target: 50% of technical co-operation flows are implemented through
co-ordinated programmes consistent with national development
strategies; indicator: Percent of donor capacity-development support provided through co-ordinated programmes consistent with
partners national development strategies.
Geographical area
covered
international (See www.oecd.org/dac/effectiveness/parisdeclaration/members)
Methodology
- Baseline year 2006, target year 2010

- Measurement of indicators at country level
- Aggregation at regional and/or global level for statistical comparability
Data sources
Type of data used
- No explicit information
- States
- Participating organizations (such as African Development Bank Arab Bank for Economic Development in Africa
Asian Development Bank Commonwealth Secretariat
Consultative Group to Assist the Poorest (CGAP) Council of Europe Development Bank (CEB)
Economic Commission for Africa (ECA) Education for All Fast Track Initiative (EFA-FTI)
European Bank for Reconstruction and Development (EBRD) European Investment Bank (EIB)
Global Fund to Fight Aids, Tuberculosis and Malaria,...)
- country data
> government and donor questionnaires (mainly)
> WB Government Questionnaire,
> World Banks Country Policy and Institutional Assessment (CPIA),
> OECD-DAC Methodology for Assessing Procurement Systems,
- donor data
> government and donor questionnaires (mainly)
> OECD-DAC Creditor Reporting System (CRS). Data is only available for DAC members reporting the tying status of aid.
- Data are supposed to be measured in close collaboration between partner countires and donors.
Quality of data
- In practice: information is mainly collected through government and donor questionnaires --> problematic: Developing countries
providers/processors often lack reliable statistical agencies
159
FRAME
Presentation of
indicator results
- Reports (2008 + 2011)

- Percentage of extent to what targets have been met
Comparability
not intended
Frequency of
application
2006 (baseline study)

subsequent surveys (in 2007 and 2010) + reports in 2008 and 2011
target year 2010
Status
target year 2010, already completed
- only 1 goal out of 10 has been reached
- recommendations in report 2011: While the indicators of progress agreed in Paris aimed to offer a range of proxies through which
progress against the five principles could be assessed, some offer more relevant insights into the sorts of behaviour that matter for
aid effectiveness than others. While the Accra Agenda for Action reflected a deepening of the Paris Declaration, monitoring some of
Comments
the commitments set out in it has been challenging in the absence of agreed norms and indicators. The optional modules on inclusive
ownership and gender equality piloted as part of the 2011 Survey could be drawn on as examples to inform the development of
indicators and assessment methods covering a wider range of commitments.
Link to website
http://www.oecd.org/dac/effectiveness/parisdeclarationandaccraagendaforactionfullrelateddocumentation.htm; 29.08.2013
160
FRAME
EU Justice Scoreboard
Author
Euorpean Commission
Type of author
EU institution
Thematic context
The EU Justice Scoreboard is an information tool within the European Semester, the EU's annual economic policy coordination
process that aims to to support reforms in national justice systems required to render justice systems more effectie for citizens
and businesses.
Rational/purpose
The EU Justice scoreboard is an information tool aiming to assist Member States in promoting the quality, independence and
efficiency of justice systems in the European Union, and hence contribute to fostering economic growth in the Union.
Users/Target Group
EU institutions and EU Member States
Reference to human
rights
not explicitly, implicitly the rule of law and efficiency of judiciary shall guarantee the realisation of human rights
human rights?
The scope of the Scoreboard focuses on the parameters of a justice system which contribute to the improvement of the
business and investment climate: Efficiency of the justice systems, quality of the justice system and independence of the
judiciary.
161
FRAME
4 indicators for efficiency of justice systems

5 indicators for quality of justice systems
1 indicator for independence of the judiciary plus 5 indicators for structural judicial independence
Geographical area
covered
EU Member States
Methodology
The methodology was based on CEPEJ biannual exercises, which consist of a questionnaire that the national correspondent
of the CEPEJ (usually from countries Ministries of Justice) completes and the results of which are then statistically processed
and analysed.
Data sources
For preparing the 2013 Scoreboard, the Council of Europe Commission for the Evaluation of the Efficiency of Justice (CEPEJ) was
asked by the European Commission to collect data and conduct an analysis. The Scoreboard also uses data from other sources,
such as from the World Bank, World Economic Forum and World Justice Project.
Type of data used
quantitative indicators
Quality of data
It has to be noted by CEPEJ that the quality of the figures depends very much on the type of questions asked in the data
collection instrument, the definitions used by the countries, the system of registration in the countries the efforts supplied by
national correspondents, the national figures available to them and the manner in which the figures have been processed and
analysed.
results
Results are presented in the yearly reports in terms of percentages or through scores or ranks or other numerical statistics.
162
FRAME
Comparability
It is a comparative tool which allows for comparison over time and between Member States.
The Justice Scoreboard is published on an annual basis.
Status
The first Justice Scoreboard was published in March 2013.
Link to website
http://ec.europa.eu/justice/effective-justice/scoreboard/index_en.htm; 16.07.2014
163
FRAME
EU Enlargement Package: Strategy and Progress Reports

Author
European Commission
Type of author
EU institution
Thematic context
Each year the Commission adopts its "Enlargement package" - a set of documents explaining its policy on EU enlargement and
reporting on progress achieved in each country.
Rational/purpose
The Enlargement Strategy Paper and the Country Reports take stock of the progress made over the last twelve month by each
candidate country and potential candidate and sets out the way forward for the coming year. It is the basis for the Councils
proceedings in the accession negotiations with the enlargement countries.
Users/Target Group
EU institutions, Member States, candidate and potential candidate countries
Reference to human
rights
The stability of institutions guaranteeing human rights is one pillar of the political criteria for membership, as defined by the
Copenhagen European Council in 1993.
human rights?
Human rights norms stemming from international conventions and the acquis communautaire: Human dignity; Right to life and
to the integrity of the person; Prohibition of torture and inhuman or degrading treatment or punishment ; Prohibition of
slavery, servitude, and forced or compulsory labour; Respect for private and family life and communications; Right to marry and
right to found a family; Freedom of thought, conscience and religion; Freedom of expression including freedom of the media;
164
FRAME
Freedom of assembly and association; Treatment of socially vulnerable and disabled persons and principle of
nondiscrimination; Right to education; Right to property; Gender equality and woman's right; Rights of the child; Liberty and
security; Right to a fair trail; Respect for and protection of minorities and cultural rights; Measures against racism and
xenophobia;
No fixed number of indicators; individual country benchmarks are set

Structural indicators that are used to assess the legal framework in place; process indicators are used to assess the
Types of indicators
implementation of reforms; outcome indicators are used to measure actual results based on track-records of implementation
Geographical area
covered
EU candidate and potential candidate countries
Methodology
The Commission assesses the progress made by the enlargement countries on a yearly basis. This assessment follows the
structure of the Copenhagen criteria and assesses the 35 negotiation chapters. The assessments are based on the policy
priorities identified in the European Partnerships and Accession Partnerships and the Action Plans adopted by the enlargement
countries.
Data sources
Carrying out the assessment, the Commission collected information from a number of data sources: data from document and
legislation review, official statistics, information provided by government authorities. Furthermore, the assessments conducted
by third-parties like the CoE, the OSCE, the international financial institutions and NGOs are an important source of information.
The information is gathered through various peer reviews that are conducted by country experts.
Type of data used
Mainly qualitative data are used. Quantitative data are used where data are available.
165
FRAME
Quality of data
A variety of data is used, the quality differs, no information about i.e. which data from third-party reports exactly are used.
results
The results of the assessments are presented in qualitative statements in the annual strategy and country progress reports.
Where available the qualitative statements are supported through quantitative measures. No scores or other unified
measurements are used.
Comparability
The country reports mostly follow the same structure. However, the comparision of the results of the country assessments is
not possible as there is no standardised set of indicators or unified measurements. The topics addressed in the reports vary
according to the priorities set for the countries in the European Partnerships and Accession Partnerships.
Status
regular application on an annual basis; continuous adaption and improvement of the methodology and the measurement tools.
Link to website
http://ec.europa.eu/enlargement/countries/strategy-and-progress-report/index_en.htm; 17.07.2014
166
FRAME
EU Employment and Social Policy Indicators

Author
Eurostat
Thematic context
Eurostat was established 1953 to meet the requirements of the Coal and Steel Community. Over the years its task has broadened and
when the EC was found in 1958 it became a Directorate-General of the EC. Eurostat's key role is to supply statistics to other DGs and
supply the Commission and other European Institutions with data sot they can define, implement and analyse Community policies.
Eurostat offers a whole range
Rational/purpose
the employment and social policy indicators provide various sets of data which track the following EU policies: employment
guidelines, social inclusion and social protection, education and training, information society, youth, equality (age and gender),
migrant integration
Users/Target Group
the main user of this statistics is the EU itself, to measure the implementation of its policies in the member states
Reference to human
there is no direct reference to human rights
rights
167
FRAME
human rights?
Human rights
covered
as above stated there is no direct reference to human rights; but taking into account the indicators it is likely to assume that this
statistics measure the implementation of economic, social and cultural rights in general, although human rights are not directely
mentioned
Number of
indicators
there are seven key groups of indicators, each divided into sub-groups
these sub-groups are also divided into more concrete indicators
therefore there are about 80 indicators in total

the indicators can be used to assess progress towards the policy goals
- overarching indicators
- social-inclusion indicators
- pensions indicators
- health and long-term care indicators
- Adult participation in life-long learning
- Low achieves in basic skills
- Tertiary level attainment
- Early leavers from education and training
Types of indicators
- Early childhood education and training
- Learning mobility in higher education
- Learning mobility in initial vocational education and training
- Employment rate of recent graduates
- Digital Single Market - Promoting e-commerce
- Digital inclusion
- public services
- Broadband and Connectivity - households
- Broadband and Connectivity - Individuals
168
FRAME

- ICT usage by individuals
- e-Public Services
- child population/ youth population/ ratio of young people in total population/ mean age of young people leaving the parental
household
- Education and training
- Employment and netrepreneurship
- health and well-being
- social inclusion
- youth participation: use of Internet
- Age equality indicators
- Gender equality indicators
- activity rate
- unemployment rate
- employment rate
- overqualification rate
- self-employment rate
- highest educational attainment
- Share of 30-34-year-olds with tertirary educational attainment
- Share of early leavers (aged 18-24) from education and training
- median net income
- persons at risk of povery or social exlcusion
- at risk of poverty rate
- the share of population perceiving their health status as good or poor
- Ratio of property owners to non-property
- The share of foreigners resident in EU MS who have aquired citizenship
- The share of third country nationals who have aquired citizenship
- The share of long term residens among all third country nationals having a valid permission to stay
169
FRAME
Geographical area
covered
the European Region is covered by this survey (EU member states)
Data sources
surveys, administrative data, ICT surveys, civil registration systems, questionnaires, telephone interviews, face-to-face interviews
Type of data used
the data used is of a qualitative as well as quantitative nature, depending on the indicator
data providers are among others the EU-SILC (Community Statistics on Income and Living Conditions), the Labour Force Survey (LFS),
Quality of data
UNESCO-UIS (Institute for Statististics), OECD, Eurostat, National statistical Institutes, ECHP
providers/processors the data is processed by the EUROSTAT
Presentation of
indicator results
there is no information available about the presentation of the results of the indicators; but it is very likely - when considering the
practice of Eurostat related to presenting results - that the results are presented in a report, where charts and tables are used to
highlight certain progresses or change
Comparability
yes, the statistics are meant to be compared among EU member states
Frequency of
application
data was first collected in 2004 and then annually, for some indicators there not yet a statistics availiable but it isunder preparation
Status
for some indicators (e.g. "Mean age of young people leaving the parental household") there is no data available
170
FRAME
- The European employment strategy provides a framework for EU countries to share information, discuss and coordinate their
employment policies. The guidelines for the employment policies of the Member States are intrinsically linked with the guidelines fo
rthe economic policies of the member states and of the EU. Together, they form the intregrated guidlines that underpin form 2020 th
Europe 2020 strategy. To monitor progress in implementing the employment guidelines within the context of the European 20202
Comments
strategy a joint assessment framework has been developed by the European Commission, the Employment Committee (EMCO), and
the Social Protection Committee (SPC). TheJap is an indicator based assessment system. It uses a series of indicators to measure the
current situation and trends throught ime in member states.
Link to website
http://epp.eurostat.ec.europa.eu/portal/page/portal/employment_social_policy_equality/introduction; 25.10.2013
171
FRAME
Europe 2020 Strategy Indicators

Author
EU, Eurostat
Thematic context
The EU is working hatd to move decisively beyond the crisis and create the conditions for a more competitive economy with higher
employment. The Europe 2020 strategy is about delivering growth that is: smart, through more effective investments in education,
research and innovation; sustainable, thanks to a decisice move towards a low-carbon economy; and inclusive, with a strong
emphasis on job creation and poverty reduction. The strategy is focused on five ambitious goals in the areas of employment,
innovatin, education, poverty reduction and climate/energy. To ensure that the Europe 2020 strategy delivers, a strong and effective
system of economic governance has been set up to coordinate policy actions between the EU and national levels.
Rational/purpose
The Europe 2020 strategy designed as the successor to the Lisbon strategy, adopted by the European Counil in June 2010 is the EU's
common agenda for the next decade. It puts emphasis on smart, sustainable and inclusive growth that can overcome the structural
weaknesses in Europe's economy, imporove its competitveness and productivitiy and underpin a sustainable social market economy.
Users/Target Group
first and foremost the Europe 2020 statistics ought to be used by the EU and all its institutions
Reference to human
rights
human rights?
Human rights
covered
the Europe 2020 statistics focus more on the economic growth etc with promoting new
172
FRAME
The Europe 2020 strategy consists of 5 main key indicators (a. Employment, b. Research and Development, c. Climate change and
Number of
energy, d. Education, e. Poverty and social exclusion), which are then sub-divided in more concrete indicators
the Europe 2020 strategy is a set of eight main indicators and three sub-indicators has been developed - this means that there are in
indicators
total 11 indicators
the indicators are the following:
- Employment rate by sex, age group 20-64
- Gross domestic expenditure on Research and Development
- Greenhouse gas emissions, base year 1990
- Share of renewable energy in gross final energy consumption
- Primary energy consumption
- Final energy consumption
Types of indicators
- Early leavers from education and training by sex
- Tertiary educational attainment by sex, age group 30-34
- People at risk of poverty or social exclusion
- People living in households with very low work intensity
- People at risk of poverty after social transfers
- Severely materally deprived people
the indicators are structural indicators, sine they measure the progress in policies
Geographical area
covered
the Europe 2020 focuses on the area of the 27 (now 28) member states
Methodology
Eurostat has been invovled in the process of defining the indicators to support the strategy
Data sources
data is collected through surveys, questionnaires, administratice registers, EU-LFS (a rotating random sample survey of persons in
173
FRAME

private households), ECHP (European Community Household Panel 20014-2005), EU-SILC
Type of data used
the indicators are based on quantitative as well as qualitative information (depends on the indicators)
data is provided by the EU Labour Force Survey, the U.S. Bureau of Labour Statistics, the Ministry of Internal Affiars and
Communications Government of Japan (Eurostat obtains those data via OECD and disseminates them without any processing), ESS,
OECD, Oil bulletin of DG TREN
Quality of data
the data processor is Eurostat, which promise to offer a high quality standards by Reg 223/2009 on European Statistics; the European
providers/processors Statistics code of practice and Communcition (2011) 211 Towards robust quality management for European Statistics
Data include the EU aggregates, data for indivdual Memeber States, EFTA, and candidate countries, for some indicators also data for
USA and Japan
Presentation of
indicator results
the results are presented in a report, published by Eurostat, which contains tables and charts for a better understanding of the
results and explanations are added
Comparability
yes, the Europe 2020 statistics are supposed to make a comparison between the member states possible
Frequency of
application
there was a publication in 2012 deailng with the Europe 2020 Strategy - towards a smarter, greener and more inclusive EU economy
174
FRAME
Status
In Oct 2012 Eurostat issued a short publication, from the series Statistics in Focus: Europe 2020 Strategy - twoards a smarter, greener
and more inclusive EU economy; this presents trends and latest data for each of the headline indicators, in the principle areas of the
strategy, at EU and MS level
The Commission adopted seven flagship initiatives in addition tot eh headline targets, in order to drive progress towards the Europe
2020; these are 1. Smart growth (through a digital agenda for Europe, an Innovation Union, the Youth on the move), 2. Sustainable
growth (through a resource effecient Europe, and an industrial policy for the globalisation era), 3. Inclusive growth (through an
agenda for new skills and jobs and an European platform against poverty) further actions undertaken are EU-climate action and new
skills for new jobs
Comments
They have also set 5 targets for the EU 2020: 1. 75% of the 20-64 year-olds to be employed; 2. 3% of the EU's GDP to be invested in
R&D; 3. greenhouse gas emissions 20% lower than 1990, 20% of energy from renewables, 20% increase in energy efficiency; 4.
Reducing the rates of early school leaving below 10%, at least 40% of 30-34-year-olds completing third level education; .5 at least 20
million fewer people in or at risk of poverty and social exclusion
http://epp.eurostat.ec.europa.eu/portal/page/portal/europe_2020_indicators/headline_indicators/context
Link to website
http://ec.europa.eu/europe2020/europe-2020-in-a-nutshell/flagship-initiatives/index_de.htm
25.10.2013
175
FRAME
Political Instability Task Force Report & Dataset

Author
Political instability Task Force (PITF)

Type of author
US panel of scholars and methodologists (funded by CIA, hosted by Center for Global Policy at George Mason
University)
Thematic context
Early Warning for State Failure
Rational/purpose
Inform policymakers about states vulnerable to political instability and state failure, identify key risk factors
Describe the main rational/purpose of the indicator scheme. E.g. monitoring/ situational analysis/ inform public policy/ measuring progress or
implementation/ increase accountability/ basis for allocation of funds/identify violations of state obligations/ early warning, etc.
Users/Target Group
US policy-makers
Reference to human
rights
Refers implicitly to human rights: some of the indicators can be used to measure hr (i.e. with regard to genocides, democracy,
minorities,)
- minorities, genocide
- democracy
- indirectly political freedom: violence & fatalities (?)
176
FRAME
> 10
Types of indicators
Geographical area
covered
International
Methodology
- Expert Coding: quantitative and qualitative data is coded using values from 0-4
Data sources
Open-source data
Type of data used
"Open-source data": No further information available
Quality of data
No information available
results
- 4 Excel spreadsheets with quantitative indicators: adverse regime changes, ethnic wars, revolutionary wars, genocides and
politicides
- Consiolidated Problem Set (PDF) without quantitative indicators, only brief summary
177
FRAME
Comparability

No information, but obviously not intended
1994, annually
Status
Annual application
Link to website
http://globalpolicy.gmu.edu/political-instability-task-force-home/; 01.08.2013
178
FRAME

2.
Quality of Government Datasets

Author
QoG Institute
Type of author
Independent research institute (within Department of Political Science at the University of Gothenburg)
Thematic context
Indicators to test scientific hypotheses in the field of political science and good governance
Rational/purpose
Situational analysis
Users/Target Group
Mainly targed at researchers

Reference to human
rights
Indicators are not generated by Quality of Government Institute. It simply gathers indicators from other indicator schemes.
Therefore, indicators are rather diverse, but most of them directly touch upon human rights.
179
FRAME
Wide variety, focus on civil and political rights.

around 900 (derived from other indicator schemes)

WII (What It Is) variables, that is, variables pertaining to the core features of QoG (such as corruption, bureaucratic quality,
and democracy)
HTG (How To Get it) variables, that is, variables posited to promote the development of QoG (such as electoral rules, forms of
government, federalism, legal & colonial origin, religion and social fractionalization); and
Types of indicators
WYG (What You Get) variables, that is, variables pertaining to some of the posited con-sequences of QoG (such as economic
and human development, international and domes-tic peace, environmental sustainability, gender equality, and satisfied,
trusting and confi-dent citizens).
Geographical area
covered
192 countries (UN members in 2002 + Taiwan)

- both datasets cover same countries, with 13 additional (historical) countries for time-series
- specific variables are often limited to less countries
Methodology
Indicators are not compiled by Quality of Government Institute. It simply gathers indicators from other indicator schemes.
Data sources
Numerous freely available cross-sectional data sources are used
Type of data used
- wide range of different types of data: expert-coded indicators and classifications, various demographic measures, national
180
FRAME
accounts data and aggregated individual-level survey data.

- compiled from numerous freely available and well-known data sources, including datasets produced by independent research
projects, international research initiatives, NGOs and IGOs.
-QoG draw on other original or secondary sources and do not generate their own data or indicators.
- Quality depends on primary source and, therefore, varies.
- indicator schemes used: i.e.
> Bernhard, Nordstrom & Reenock
> Bertelsmann Transformation Index
Quality of data
> Bueno de Mesquita, Smith, Siverson & Morrow
> Coppedge, Alvarez & Maldonado
> Cheibub, Ghandi & Vreeland
> Cingranelli & Richards
> Economist Intelligence Unit,...
5 sets of indicators: Basic, Standard, Social Policy, Expert Survey, EU Regional.
- Indicators are available as .sav, .dta and .csv files, and are to be used in statistical softwares and/or Excel
results
- online Data Visualization Map
- comparability over time: cross-sectional time-series
Comparability
- comparability across countries:
- QoG Cross-Section Dataset: data for 2002 (or the closest year available),
Frequency of application - QoG Cross-Section Time-Series Dataset: data from 1946 to 2009.
- updated continuously (last update: 8 June 2012)
Status
Regular application
181
FRAME
Link to website
http://www.qog.pol.gu.se/data/, 29.07.2013
182
FRAME
The Failed State Index

Author
The Fund for Peace

Type of author
NGO: Nonprofit research and educational institution
Thematic context
Failed state index
Rational/purpose
Early warning for state failure

-Researchers/Educators
- Governments
- Military
- Private sector
- Multinational organizations
- No human rights indicators, but failed state indicators
- touch on human rights, and incorporate them, but do not focus on them
- one indicator explicitly focuses on: Suspension or Arbitrary Application of the Rule fo Law and Widespread Human Rights
Abuse
> Press Freedom
> Civil Liberties
> Political Freedoms
> Human Trafficking
> Political Prisoners
> Incarceration
> Religious Persecution
> Torture
> Executions
Users/Target Group
Reference to human
rights
183
FRAME
- Civil and Political Rights

- Discrimination
- Minority Rights
- Economic Rights (Equality)
- 12 primary social, economic and political indicators, over 100 subindicators:
> Social Indicators: Demographic Pressures, Refugees and IDPs, Group Grievance, Human Flight and Brain Drain
> Economic Indicators: Uneven Economic Development, Poverty and Economic Decline
> Political and Military Indicators: State Legitimacy, Public Services, Human Rights and Rule of Law, Security Apparatus,
Factionalized Elites, External Intervention
(http://ffp.statesindex.org/indicators)
Types of indicators
Geographical area
covered
International (178 recognized states, other countries: lack of data)
IS the indicator scheme supposed to be applied on the local, regional, national, international, EU- level, number of countries, etc.
Methodology
First, we download millions of documents, including a variety of digitized news articles, essays, magazine pieces, speeches, and
government and non-government reports (we do not use blogs, twitter, or other social media.) Then, we apply our content
analysis software to scan the documents using Boolean phrases on indicators within our CAST framework. The data used in each
index are collected from the preceding year and stored on our servers so that we can go back to them when needed. Our search
landscape has expanded from 90,000 to 115,000 online English-language publications worldwide, giving us a wide variety of
data sources upon which to base our findings. Filters built into the software extract irrelevant or erroneous documents so the
search can zero in on the specific subject matter defined in the Boolean phrases, and correct for false positives, pack journalism,
and media drift.
184
FRAME

Second, we incorporate quantitative data from reputable institutions, such as the UNHCR, WHO, UNDP, Transparency
International, World Factbook, Freedom House, World Bank, and other reliable sources.
Third, the results are compared with insights from a separate qualitative review of each indicator for each country.
Taken together, the three methods serve as internal checks. Aggregated data are normalized and scaled from 0-10 to obtain
final scores for 12 social, economic and political/military indicators for 177 countries. These results are then critically reviewed
by analysts different from those who conducted the original research.
This multi-stage process has several layers of scrutiny to ensure the highest standards of methodological rigor, the broadest
possible information base including both quantitative and qualitative expertise, and the greatest accuracy.
Data sources
Huge variety of data, three primary sources:

- content analysis (electronic scanning),
- quantitative data,
- qualitative data. (reports and information from all over the world)
(i.e. news articles, essays, magazine pieces, speeches, and government and non-government reports - we do not use blogs,
twitter, or other social media.)
Type of data used
Huge variety of data

qualitative and quantitative data
Quality of data
- quality of data providers varies, but due to diversification and number of sources overall quality of data seems to be reliable
results
Color-coded maps, table & ranking system ("Alert", "Warning", "Moderate", "Sustainable")
185
FRAME
Comparability
-Comparability across countries: intended and possible

- Comparability over time: intended and possible
2005, annually
Link to website
http://ffp.statesindex.org/; 30.07.2013
186
FRAME
Minorities at Risk (MAR) & Minorities at Risk Organizational Behaviour (MAROB)

Author
Center for International Development and Conflict Management, University of Maryland
Type of author
Interdisciplinary research institution, affiliated with University of Maryland
Thematic context
Analysis/Understanding of minority conflicts
Rational/purpose
- MAR: Monitoring and analysis of status and conflict of politically-active communal groups
- MAROB: Idenification of factors that motivate members of ethinc minorities to become radicalized, to form activist
organizations, and to mvoe from conventional means of plitics and protest into violence and terrorism
- Primarily: Researchers
- also activists, policy makers,
> No explicit reference to human rights
> Variables with regard to
Reference to human
- Group status (autonomy, displacement,)
- External support (from members in diaspora, foreign states,)
rights
- Group conflict behaviour (inter-, intracommunal conflicts, reppression by state)
can be used in the measurement of minority rights implementation
Users/Target Group
187
FRAME
Minority Rights
- MAR: 4 areas - group characteristics, group status, external support & group conflict behaviour with > 70 variables/indicators
- MAROB: 8 areas - organizational identity, type of organization, grievances/motiavations/ideology of organization,
organizational ideology, organization-state relations, external support, organizational behaviour, nonviolent behaviour, violent
behaviour, criminal activity initiated by organization, official participation in crime, domestic crime networks, transnational
crime networks,
- MAR: mainly outcome indicators, such as group autonomy status, group representation,
- MAROB: mainly procedural indicators relating to non-state actor, such as: Does the organization advocate democratic forms of
Types of indicators
government? Does the state use violence against the organization?
Geographical area
covered
MAR: international
MAROB: 26 countries of the Middle East and North Africa
Methodology
coding of qualitative and quantitative data:

values (0-1, up to 0-7) are assigned on the basis of a Codebook to quantitative data and qualitative statements
Data sources
Type of data used
Quantitative and qualitative data
188
FRAME
Quality of data
results
Qualitative - report-style - "Minority Group Assessments": assessment of each minority group (Risk assessment and analytic
summary, based on indicators)
Quantitative indicators: available as .csv, .dta, or .sav file
Comparability
Comparability across countries: possible

Comparability over time: changes of variables in each phase --> limited comparability
MAR: founded 1986, 5 phases

MAROB: released in 2005: MAROB data from 1980 to 2004
Status
Link to website
MAR: Phase I covered 227 communal groups which met the criteria for classification as a minority at risk for the years 19451990;
Phase II covered 275 groups from 1990-1996;
Phase III covered 275 groups from 1996-1999; and
Phase IV covered 283 groups from 1998-2003.
MAROB: released in 2005: data from 1980 to 2004
http://www.cidcm.umd.edu/mar/; 31.07.2013
189
FRAME
Global Peace Index (GPI)

Author
Institute for Economics and Peace (IEP)
Type of author
Non-profit research organization
Thematic context
- Measuring peacefulness (internal and external levels of peace)

- Measuring development of peacefulness of states over time
- Ranking countries according to it
- Initiated by Steve Killelea (IT entrepreneur, philanthropist), huge media response, endorsed by Dalai Lama, Kofi Annan, etc.
Rational/purpose
"GPI aims to make a valuable contribution to

- be able to analyse impacts of peace on sustainability,
- estimate the value of peace to the world economy
- uncover the social structures and social attitued that are at the core of peaceful societies
(See http://www.visionofhumanity.org/#/page/about-gpi)
- Civil society,
- researchers,
Users/Target Group
- policymakers,
- government
Reference to human
rights
Some indicators refer implicitly to human rights: i.e.

- Number of refugees and displaced people as a percentage of the total population
190
FRAME
- Level of violent crime

- Number of homicides per 100,000 people,
(See http://www.visionofhumanity.org/#/page/about-gpi)
human rights?
- Touches upon civil and political rights
22
Types of indicators
Qualitative (7) and quantitative (15) indicators

- Ongoing Domestic and International Conflict (5 indicators): i.e.
Number of external and internal conflicts fought
Level of organised conflict
Relations with neighbouring countries
- Societal Safety and Security (10 indicators)
- Militarisation (7 indicators)
Geographical area
covered
International (162 countries)
Methodology
- indicators were selected by an international expert panel and are reviewed annually
- scores from 1-5 are assigned to indicators by EIU Country Analysis team (> 100 full-time country experts and economists +
191
FRAME

support from 650 in-country contributors)
- qualitative indicators:
> individual country analyst scores indicatos,
> meets with regional teams to assess indicators and ensure consistency and comparability,
> scores are checked by EIU's Custom Research team to ensure global comparability,
> discussion about questionable scores among the actors,
> scores are assessed by external advisory panel,
> discussion among actors
- two sub-component indices (internal peace and external peace)
- weight: 60% internal peace, 40% external peace
- quantitative indicators: data gaps are closed with alternative data sources and/or estimates
Data sources
i.e.
- EIU,
- UNODC,
- UN Survey of Crime Trends and Operations of Criminal Justice Systems,
- International Centr for Prison Studies, University of Essex, World Prison Brief,
- Political Terror Scale (Gibney, Cornett & Wood),
- SIPRI Arms Transfers Database,
- IEP, Global Terrorism Index, GTI,...
Type of data used
Qualitative and quantitative data
192
FRAME
Quality of data
- various sources: international agencies, surveys etc.

- mainly internationally recognized agencies etc. --> relatively reliable
- in order to fill gaps: estimates by EIU
results
- GPI Map (colour-coded)

- annual report
Comparability
comparability across countries: intended, measures to ensure comparability - individual analyst meets with regional teams to
assess indicators and ensure consistency and comparability, scores are checked by EIU's Custom Research team to ensure global
comparability
comparability over time: intended, only 5 yrs. since launch --> no further information available
2007, annually
Status
Regular application
Link to website
http://www.visionofhumanity.org/#/page/about-gpi; 08.08.2013
193
FRAME
Ibrahim Index of African Governance (IIAG)

Author
Mo Ibrahim Foundation
Type of author
the foundation is a non-grant making organisation
Thematic context
the foundation was established in 2006 with a focus on the critical importance of leadership and governance in Africa. By
providing tools to support advancements in leadership and governance, the Foundation aims to bring about meaningful change
on the continent
Rational/purpose
the IIAG provides a statistical measure of governance performance in African countries
Users/Target Group
the IIAG provides a framework for citizens, governments, institutions and business to assess the delivery of public goods and
services, and policy outcomes, across Africa
Reference to human
rights
there is no direct link to human rights; but in general it can be suggested that since the whole political and economic situation
of a country is measured, this also includes the human rights compliance of that certain country
The IIAG governance framwork comprises four dimensions (categories): A. Saftey and Rule of Law; B. Participaiton and Human
Rights; C. Sustainable Economic Opportunity; D. Human Development
these categories are made up of 14 sub-categories, consisting of 94 indicators.
194
FRAME
most of the indicators are structural indicators because they measure the implementation of certain convention in the national
law
A. Saftey&Rule of Law
(Judical Process, Judical Independence, Sanctions, Transfer of Power, Property Rights; Accountability, Transparency&
Corruption in the Public Sector, Accountability and Transparency in rural areas, Corruption& Bureaucracy, Accountability of
Public Officals, Corruption in Government&Pubilc Officials, Prosecution of Abuse of Office, Diversion of Public Funds)
B. Participation and Human Rights
(Fre&Fair Elections, Free&Fair Executive Elections, Political Participation, Electoral Self-Determination, Effective Power to
Govern; Core IHR Conventions, HR, Political Rights, Workers' Rights, Freedom of Expression, Freedom of Assembly&Association,
Civil Liberties, Gender Equality, Gender Balance in Primary&Secondary Education, Women's Participation in Labour Force, Equal
Representaion in Rural Areas, Women in Parliament, Women's Rights, Legislation on Violence Against Women)
C. Sustainable Economic Opportunity
(Statistical Capacity, Public Administration, Inflation, Diversificaiton, Reserves, Budget Management, Ratio of Total Revenue to
Total Expenditure, Fiscal Policy, Ratio of External Debt Service to Exports, Revenue Collection, Soundness of Banks; Competitive
Types of indicators
Environment, Investment Climate, Investment Climate for Rural Businesses, Rural Financial Services Development,
Bureaucracy&REd Tape, Customs Procedures; Electricity, Roads, Rail Network, Air Transport, Telephone& IT Infrastructure,
Digital Connectivity; Public Resources for Rural Development; Land&Water for low income rural population, agricultural
research&extension services, Agricultural input&Produce markets, dialogue between Government& Rural Organisations,
Agricultural Policy Costs)
D. Human Development
(Welfare Regime, Social Protection & Labour, Social Exclusion, Welfare Services, Equity of Public Resource Use, Access to Water,
Access to Sanitation, Environmental policy, Environmental Sustainability; Education Provision & Quality, Educational Sysem
Quality, Ratio of Pupil to Teachers in Primary school, Primary School completion, Progression to Secondary School, Tertiary
Enrolment, Literacy; Maternal Mortality, Child Mortality, Immunisation, Antiretroviral Treatment Provision, Disease,
Undernourishment)
Geographical area
covered
the African Region is covered by the Index
195
FRAME
Methodology
the index is developed by expert consulations; and it is revised retrospectively every year, in accordance with best practices.
Data sources
in 2013 the IIAG was calculated using data from 32 different independent sources. Among those were: African Econmic
Outlooks, African Development Bank/ OECD, UNDP, UN Economic Commission for Africa, AIDSinfo UNAIDS, African Statistical
Yearbook, AU, Bulleting Board on Statistical Capacity, World Bank, Bertelsmann Transformation Index, Bertelsmann Stiftung,
Cingranelli-Richards Human right dataset, Child Mortality Esimates Info, Commd CDD, Commd EIU, Commd IREP, Country Date
EIU, Country Performance Assessment, Democracy Index, Freedom in the Wolrd Survey, Freedom House, Global
Competitiveness Report, WHO GHO, Social Institutions and Gender Index under Gender, Global Overview of Trends IDMC, ICT
Database ITU, Index Econ Freedom HER-WSJ, UNHCR, PBAS IFAD, PTS, Trafficin in Persons Report, UCDP/PRIO Armed Conflict
Dataset, UIS UNESCO, World Development Indicators, UNICEF
Type of data used
the data is of a qualtitative and quantitative nature - depending on the indicator
Quality of data
the data is provided by international organizations respectively my UN agencies/ programmes (but also the World Bank is a data
provider) the quality of the data providers has to be analysed in a case by case study
the data processor is the IIAG itself
results
the results of the indicators are presented through online interactive tolls and country profiles, as well as through an interactive
excel file
196
FRAME
Comparability
yes, it ought to be compared between the African countries
it was established in 2007, and has been applied ever since
Governance is defined by the Mo Ibrahim Fondation as the provision of the political, seocial and economic public goods and
services that a citizen has the right to expect from his or her state, and that a state has the responsibilty to deliver to its citizens.
For the comparison the region is splitted into Central Africa, East Africa, North Africa, Southern Africa, West Africa, Island
Countries, Landlocked Countries, Coastal countries
Comments
FYI: Paucity of data in Africa remains a core concern for the Foundation. Many crucial indicators of governance, such as poverty,
do not yet meet the Foundation's inclusion criteria, specifically with regards to time series and country coverage. The
Foundation funds two major African initiatives: it is working with Afrobarometer, and with the Global Integrity Trust
Link to website
http://www.moibrahimfoundation.org/interact/; 24.10.2013
197
FRAME
Bertelsmann Transformation Index (BTI)

Author
Bertelsmann Stiftung, Gtersloh, Germany
Type of author
NGO
Thematic context
In-depth country reports provide the basis for assessing the state of transformation and persistent challenges, and to evaluate
the ability of policymakers to carry out consistent and targeted reforms.
Rational/purpose
Users/Target Group
Analysis and evaluation of the quality of democracy, market economy and political management in 129 developing and
transition countries to measure successes and setbacks on the path toward a democracy based on the rule of law and a socially
responsible market economy.
Policy Makers
Reference to human
rights
Implicit reference to human rights
human rights?
Right for Political Participation, Rule of Law, Quality of Democratic Institutions, Political and Social Integration,
198
FRAME
52 indicators (49 questions, 3 quantitative indicators)

Mainly outcome indicators
Types of indicators
Geographical area
covered
129 countries, excluding countries that might be considered long-consolidated democratic systems and in which economic
development can be regarded as well-advanced (all members of OECD before 1990 are excluded)
Methodology
Guided by a standardized codebook, country experts assess the extent to which a total of 17 criteria have been met for each
country. A second country expert then reviews these assessments and scores. The BTI aggregates the results of this
comprehensive study of transformation processes and political management into two indices: the Status Index and the
Management Index. The Status Index has two analytic dimensions one assessing the state of political transformation, the
other the state of economic transformation.
Data sources
The Transformation Index is based on a qualitative expert survey in which written assessments are translated into numerical
ratings and examined in a multi-stage review process. The country experts are required to draw upon a set of quantitative
indicators (ranging from inflation rates to education spending). Facts such as constitutional provisions and official economic
data can be interpreted and weighed in context.
Type of data used
qualitative data based on the subjective assessment of country experts and quantitative, statistical data
199
FRAME
Quality of data
In order to ensure the validity, reliability and comparability of the assessment, each individual score undergoes a multistep
process of review by the country experts, the regional coordinators, the project team and the BTI board.
results
written assessments are translated into numerical ratings: the two country experts translate the assessment into a numerical
rating on a scale of one (the lowest value) to 10 (highest value), structured by four levels of score-based categories contained in
the codebook.
Comparability
Comparability over time and across different countries is possible and intended: Standardization of the analytical process makes
targeted comparisons of reform policies possible
Published every two years.
Status
First application in 2003; Available for 2003, 2006, 2008, 2010,2012 and 2014
Link to website
http://www.bti-project.org/index/; 12.06.2014
200
FRAME
D.
List of Human Rights Specific or Related Indicators
Name
How fair is Britain? (Triennial Review Project)
Life and Health, Civic Life, Education, Justice
2013 Index of Economic Freedom
Economic freedom
Afrobarometer 2011
political right
Backman, Gunilla et. al: Health systems and the right to

health: an assessent of 194 countries
right to health
Childinfo Multiple Indicator cluster Survey programme
Childrens Rights, Health, Education
CIRI Human Rights Dataset
Civil and Political Rights, physical integrity, civil liberty
CIVICUS Civil Society Index
Civil and Political Liberties (Governance and Socio-Economic Rights implicitly)
CoE - Enhancing the Impact of the Framework Convention

Indicators for Assessing the Impact of the Framework
Convention for the Protection of National Minorities in its
State Parties
Non-discrimination, Minority Rights (implicitly)
CoE - Family Policy Database
Right to Non-discrimination, Right to Private and Family Life, Right to Work etc.
CoE - Indicators on Cultural Participation and Access to

Culture
Right to Freedom of Expression, Cultural Participation, Non-discrimination etc. (implicitly)
CoE - Social Cohesion Indicators
Freedom, Equality and Solidarity as basic principles related to all human rights
CoE - The Intercultural City Index
Principle of Non-discrimination (implicitly)
201
FRAME
CoE CEPEJ - Evaluating Judicial Systems
Right to a fair trial
CoE initiative: Developing Indicators to raise awareness of

human rights at local and regional level
1. Right to political participation and freedom of expression, assembly and association

2. Right to access to justice and the rule of law
3. The right to education
4. The right to work
5.right to health
6. The right to social welfare
7. Right to housing
DESA and DSPD: Promoting Empowerment of People in

achieving poverty eradication, social integration and full
employment integration and full employment and decent
work for all.
direct link to human rights
Environmental Performance Index (EPI) 2012 & Trend EPI
rights are covered by the Millenium Development Goals, all the climate convention, Rio
Declaration etc
Equality Measurement Framework
Equality (life, health, physical security, education etc.)
European Protest and Coercion Data

EUROSTAT - Employment and Social Policy Indicators
economic, social and cultural rights (indirect link)
EUROSTAT - Europe 2020 Strategy Indicators
economic, social and cultural rights (indirect link)
EUROSTAT - Sustainable Development Indicators
Right to Health, governance, safe environment, education etc. (indirect link)
202
FRAME
Failed State Index
Freedom of the Press, Civil Liberties, Torture
FAO - Food Security Indicators
Right to Food
FAO The State of Food Insecurity in the World
Right to Food
FRA - Indicators for the Protection, Respect and Promotion of

the Rights of the Child in the EU
Rights of the Child (Family, Protection from violence, education, standard of living)
Freedom in the World
Political rights and civil liberties
Freedom of the Press
Freedom of the Press (Art 19 UDHR)
Global Housing Indicators
Right to Housing
Global Peace Index
Civil and Political Rights
Heinrich Bll Stiftung - Monitoring Implementation of the

Right to Water: A Framework for Developing Indicators
Right to Water, indirect connection to Right to Health, Life etc.
Housing Indicators Program (UN and World Bank)
Right to Housing
Human Rights Data Analysis
None
Humanities Indicators
Education (implicitly)
IbrahimFundation - Index of African Governance (IIAG)
Governance, Rule of Law, indirect link to general human rights compliance of a country
ILO - ILOSTAT Database
Employment, Equality
Inter- Parliamentary Union - Women in National Parliaments
Discrimination against women (indirectly)
International Crisis Behaviour Project
none
203
FRAME
International Human Development Indicators
Life, Health, Education, Equality, Environment etc.
Key Indicators of the Labour Market (KILM)
Employment, Equality, Non-discrimination
Millennium Development Goals Indicators
Development, Education, Equality, Health
Minorities at Risk (MAR) & Minorities at Risk Organizational

Behaviour (MAROB)
Minority Rights
Nutrition Indicators for Development
human rights in general and their implementation in national law (World Food Summit and
MDGs at country level for all issues relating to nutrition of populations)
OECD - Health Data 2013
Right to Health, link to other human rights
OECD - Paris Declaration Framework for Aid Effectiveness
Development, Governance (implicitly)
OECD - State Fragility Index
none explicitly mentioned
OHCHR Human Rights Indicators
UDHR Art 3, 5, 10-11, 19, 21-23, 25-26 (life, liberty & security, food, health, education
among others)
Performance indicators and evaluation for judges and courts
Political Instability Task Force Report & Dataset
Minority Rights, Democracy, Political Freedom
Political Instability Task Force Worldwide Atrocities Dataset
right to life is indirectly covered
Political Terror Scale
Civil Liberties & Political Freedoms, Torture, Disappearances
Polity IV Project: Political Regime, Characteristics and

Transitions, 1800-2010
No significant reference to human rights; indicators on "elections held" included, which

relates to the right to take part in political participation
Press Freedom Index
Freedom of Expression
204
FRAME
Quality of Government Datasets
Civil and political rights
RED
Discrimination, Equality, Minority Rights
Right to Water Indicators
Right to Water
SPACE I
although there is no direct link to human rights, this statsitics actually disclose the
fulfillment of human rights obligations relating to prisoners,
Council of Europe Annual Penal Statistics

SPACE II (Statistiques Pnales Annuelles du Conseil de
l'Europe)
Council of Europe Annual Penal Statistics

Non-Custodial Sanctions and Measures Served in 2007
Torture Scale
Freedom from Torture and Degrading Treatment
Total Economy Database
no direct link to human rights
TransMONEE Database (UNICEF)
Rights of the Child
Transparency International - Corruption Perceptions Index

2012
none
UN - Joint United Nations Programme on HIV/AIDS
Right to Health
UN - Manual for the Development of a System of Criminal

Justice Statistics
Right to a fair trial, prohibition of torture, ne bis in idem, nulla poena sine lege etc are
implicitly covered
UN - The Worlds Women 2010: Trends and Statistics
Womens rights, gender equality
205
FRAME
UN Housing Rights Programme - Monitoring Housing Rights
Right to Housing
UN Statistics - WISTAT Womens Indicators and Statistics

Database
Discrimination against women
UNDP - Governance Assessment Portal

UNEP - Key Environmental Indicators
Right to Water, Health, Development
UNESCO - Indicators of ICT Application in Secondary

Education of South-East European Countries
Education
UNESCO - Media Development Indicators
Freedom of Expression, Governance, Development
UNFPA - Reproductive Health Indicators
Health, Education, Private Life & Family, Non-discrimination etc
UNHCR - Camp Indicator Report
Rights under the Geneva Convention, Right to Shelter etc.
UNICEF - Indicators for Children in Formal Care
Childrens Rights
UNICEF - Indicators of Violence Against Children
Right to Life, Health, Education etc.
UNICEF - The State of the Worlds Children 2013: Children

with Disabilities
Childrens Rights
Unified Database (UID)
direct link to human rights
United Nations Disability Statistics Database (DISTAT)
indirect coverage of the right to non-discrimination of persons with disabilities, right to life,
health etc
UNODC - United Nations Surveys on Crime Trend and the

Operations of Criminal Justice Systems (CTS)
Right to a fair trial, prohibition of torture (indirect link)
206
FRAME
UNODC and UNICEF - Measurement of Juvenile Justice

Indicators
Rights of the Child
VERA Altus Indicators for the Rule of Law
WHO - Global Health Observatory (GHO)
Right to Health, Prohibition of Torture, non-discrimination, private life
WHO - The European Health for All Database
Right to Health
WHO - World Health Statistics 2013
Right to Life, Right to Health
WJP Rule of Law Index
World Bank - Worldwide Governance Indicators
Civil Liberties and Political Freedoms
World Bank Indicators, World Development Indicators
Economic Rights, Equality, Education, Health
World Income Inequality Database
there is no reference to human rights
207

FRAME - Baseline Study On Human Rights Indicators in The Context of The European Union

Hochgeladen von

Dokumentinformationen

Originaltitel

Copyright

Verfügbare Formate

Dieses Dokument teilen

Dokument teilen oder einbetten

Freigabeoptionen

Stufen Sie dieses Dokument als nützlich ein?

Sind diese Inhalte unangemessen?

Copyright:

Verfügbare Formate

FRAME - Baseline Study On Human Rights Indicators in The Context of The European Union

Hochgeladen von

Copyright:

Verfügbare Formate

Fostering Human Rights among European

Large-Scale FP7 Collaborative Project

Baseline Study on Human Rights Indicators in the

Deliverable No. 13.1

Deliverable No. 13.1

Deliverable No. 13.1

Deliverable No. 13.1

CSDP Common Security and Defence Policy

Deliverable No. 13.1

Deliverable No. 13.1

NEET Not in Education, Employment or Training

Deliverable No. 13.1

General introductory remarks .......................................................................................................... 9

Guiding questions, goals and structure .......................................................................................... 10

Brief introductory remarks on human rights indicators ................................................................. 13

What are human rights indicators? ............................................................................................ 13

Why are human rights indicators important? ............................................................................ 16

Human rights indicators related terminology ............................................................................. 17

Human rights measurements by the European Union ....................................................................... 20

Legal bases .................................................................................................................................. 20

Political commitments ................................................................................................................ 26

Purposes of human rights measurement ....................................................................................... 39

Policy planning ............................................................................................................................ 39

Evaluation of human rights policies ............................................................................................ 45

Mapping of existing applications of human rights measurement tools ..................................... 48

Assessing current EU measuring instruments and identifying future requirements ................. 77

Refining the European Unions approach to measuring human rights .......................................... 87

Review of existing approaches in measuring human rights ........................................................... 87

Some measuring schemes stand out .......................................................................................... 90

No consensus on appropriate methodology .............................................................................. 91

Indicators are indispensable ....................................................................................................... 92

Miscellaneous data sources are used ......................................................................................... 93

Deliverable No. 13.1

Selection criterion 2: Three pragmatic criteria ........................................................................... 98

Selection criterion 3: Four criteria guided by requirements of the EU bodies ........................... 99

Conclusion: Suggestion of a two-pronged approach .................................................................... 101

The way forward ........................................................................................................................... 105

Bibliography .............................................................................................................................................. 106

Deliverable No. 13.1

General introductory remarks

Deliverable No. 13.1

Guiding questions, goals and structure

Deliverable No. 13.1

Deliverable No. 13.1

Deliverable No. 13.1

Brief introductory remarks on human rights indicators

What are human rights indicators?

Deliverable No. 13.1

Deliverable No. 13.1

Deliverable No. 13.1

Why are human rights indicators important?

Deliverable No. 13.1

Human rights indicators related terminology

Monitoring vs. measurement

Deliverable No. 13.1

Quantitative and qualitative indicators

Deliverable No. 13.1

Indices (commonly referred to as composite indicators or aggregated indicators) are a weighted

Deliverable No. 13.1

Human rights measurements by the European Union

Deliverable No. 13.1

Deliverable No. 13.1

Deliverable No. 13.1