The Devil You Know

but Are Afraid to Face

UNDERREPORTING BIAS AND ITS DISTORTING
EFFECTS ON THE STUDY OF TERRORISM

KONSTANTINOS DRAKOS
Department of Economics
University of Patras, Greece, and Hellenic Open University, Greece
ANDREAS GOFAS
Department of Politics and International Studies
University of Warwick, United Kingdom
Institute of International Economic Relations, Athens, Greece

Despite substantial progress in the applied study of terrorism, one important methodological issue has
remained underdeveloped. Multiple warnings have urged for caution as the validity of extant findings may
have been distorted from the well-known devil of underreporting bias. Yet, extant research has fallen
short from addressing the issue in a systematic fashion. This article discusses a way for assessing whether
underreporting is present by using the widely studied relationship between terrorism and regime type as
its laboratory. After formally presenting a setup for the accommodation of underreporting bias, the
authors discuss how it relates to press freedom. According to their results, underreporting is indeed pre-
sent, implying that the used databases for terrorism represent an understatement of the true number of ter-
rorist incidents.

Keywords: transnational terrorism; democracy; underreporting bias; press freedom

In recent years, a small but growing group of scholars has engaged in rigorous
applied analysis of terrorism that supplements case studies, with quantitative models
that control for the generality of their ad hoc claims, and offers new insights on a
host of issues associated with this complex phenomenon. One of the principal aims
of this burgeoning literature is to model observed terrorist activity by investigating
its relationship with covariates that span a wide variety of country characteristics,

AUTHORS NOTE: We are indebted to two anonymous referees for their insightful comments. We
also thank William Eubank, Quan Li, Bruce Russett, and Leonard Weinberg for their encouragement and
useful comments on previous versions of the article. Any remaining errors and/or ambiguities remain our
responsibility.
JOURNAL OF CONFLICT RESOLUTION, Vol. 50 No. 5, October 2006 714-735
DOI: 10.1177/0022002706291051
2006 Sage Publications

714
 Drakos, Gofas / DEVIL YOU KNOW BUT ARE AFRAID TO FACE 715

such as regime type, economic conditions, social factors, and so on (see, e.g.,
Eubank and Weinberg 1994, 2001; Weinberg and Eubank 1998; Eyerman 1998; Lai
2003; Li and Schaub 2004; Li 2005; Drakos and Gofas 2006). Despite progress,
some methodological issues still remain unresolved. In this article, we concentrate
on one important problemnamely, underreporting bias. The essence of underre-
porting is the suspicion that observed terrorist events might well not correspond to
the actual number of attacks, as only the events that found their way into open
sources, such as the media, have actually been reported (see, e.g., Schmid 1992;
Eubank and Weinberg 1994; Sandler 1995; Li 2005). We submit that although this
problem has been bedevilling the applied study of terrorism for some time now,
we still have not found a way to test for its suspected presence and hypothesized dis-
torting effects.
Addressing this challenge is not an easy task. However, given the centrality of
the issue to the study of terrorism, sweeping it under the carpet will simply not do
the trick. The aim of this article is ambitious and aspires to grasp the nettle of
underreporting firmly in the middle. It does so by suggesting a way that can test for
the hypothesized distorting effects of underreporting as a means of investigating its
presence.
Before briefly explaining how, let us preface the discussion with some brief gen-
eral remarks. Even a cursory look at the actual distribution of terrorist activity
across polity levels immediately reveals two characteristics. The first is the obser-
vance of excessive zeros in the data. That is, a considerable number of (nonde-
mocratic, we should stress) countries, for a substantial length of time, seem to have
experienced no terrorism at all. The second, which has been well established in the
literature, is the monotonic increase of terrorist activity as we move to higher levels
of polity.
These two data characteristics can be rationalized as being the outcome of two
possible factors: the so-called encouragement effect and/or underreporting bias. The
former suggests that as the level of democracy in a given country increases, the prob-
ability of experiencing terrorism will also increase because of a set of fundamental
regime properties idiosyncratic to democracies that may encourage acts of terrorism.
The latter suggests that excessive zeros may be present in the data due to under-
reporting bias, a process that does not allow the complete number of terrorist inci-
dents to find their way to publicly available sources.
A discrepancy between actual and reported terrorist activity would be relatively
harmless, provided that it was random and uncorrelated with country characteris-
tics. However, the sampling of terrorist events is based on public sources, which pre-
dominantly correspond to the media. Clearly, after excluding the possibility that the
media are inherently biased, one may conclude that the reporting propensity depends
on the level of press freedom, which, in turn, is highly correlated to regime type.
Indeed, it is only natural to think that if any underreporting were present, it would
be more pronounced under political regimes that have a tendency to either censor or
in some way control the press. In contrast, as the level of democracy increases, this
716 JOURNAL OF CONFLICT RESOLUTION

effect dies out, and we expect at sufficiently high levels of democracy underreport-
ing bias to be minimized. Thus, polity emerges as the fundamental determinant of
underreporting bias, although latent, exerting its impact via press freedom.
So, the first challenge for the student of underreporting is to find a way to address
the high correlation between polity and press freedom.1 Yet this is only part of the
difficulty. As we discuss in the following sections, press freedom also carries another
effectnamely, publicity seeking. Considering the fact that terrorists seek publicity
for their goals, one expects the migration of terrorism from counties that will not
provide coverage to those that will accommodate extensive coverage of
terrorist incidents. Then, it is only natural to think that publicity would tend to be
higher in countries with higher press freedom and thereby higher reporting proba-
bility. So, the terrorist seeking publicity would take into account that countries with
low press freedom, to the extent they have a tendency to underreport terrorist inci-
dents, would definitely not be a suitable location for carrying out attacks. This under-
lines a two-stage process related to maximizing publicity: (1) maximize the
probability of the attack being reported and (2) select the location that will provide
the highest coverage. From this observation, it becomes clear that an important com-
ponent of the terrorists decision process reflects underreporting. In other words,
press freedom carries a dual effect (i.e., a pure underreporting bias and an exacer-
bating effect via publicity seeking). Thus, the second challenge, complicating things
further, is to find a way to account for these two effects of press freedom.
In view of the above, we argue that simply including press freedom in our
models will not take us far. It will just bring us firmly back to the questions and
dilemmas we were initially trying to address. That is, including press freedom in
the model is a methodological loop for two reasons: (1) it brings into the model the
adverse impact of the high correlation between polity and press freedom, and (2) the
parameter of press freedom would be a mixture of both the pure underreporting
effect and the intensification of publicity seeking.
Our strategy in developing an answer to the above dual challenge will be based
on addressing the adverse impact of the high correlation between polity and press
freedom. We argue that a comparison between the subsample consisting of countries
with a partly free press and the whole sample allows us to assess the presence (or
not) of underreporting bias. The main properties of this subsample are the maxi-
mization of regime types represented and the minimization of the correlation
between polity and press freedom. The first property controls for the encouragement
effect, thus making the comparison meaningful, while the second exploits differ-
ences in press freedom and polity correlation as indicative of the presence (or not)
of underreporting. The main conclusion from the proposed analysis is that underre-
porting is indeed present, implying that the used databases for terrorism represent an
understatement of the true number of terrorist incidents.

1. This high correlation is not just conceptual. As we demonstrate in subsequent sections, it is, most
important, a statistical one.
 Drakos, Gofas / DEVIL YOU KNOW BUT ARE AFRAID TO FACE 717

The argument unfolds in the following stages. Considering that terrorism, regime
type, and press freedom form a conceptual trinity, we begin with a brief review
of competing arguments over the correlation between democracy and terrorism.
Our purpose is not just to provide yet another literature review of the polity-
terrorism nexus. Rather, we use this literature as a springboard for pointing to the
unsatisfactory treatment of the problem of underreporting. In the following section,
we examine issues concerning the nuanced relationship between press freedom and
terrorism. After these theoretical preliminaries, we move to a more formal treatment
of underreporting, seeking to establish the distinctiveness of our approach and the
value added of the proposed focus on the group of countries with a partly free press.
We proceed with a discussion of data issues, an analysis of the statistical strategy
adopted, and a presentation of the estimation results. We conclude with a brief out-
line for future research.

TERRORISM AND DEMOCRACY

The canonical distinction of terrorism-generating factors, going back to

Crenshaws seminal 1981 article The Causes of Terrorism, is between direct and
permissive or enabling. Direct factors refer to grievances generating political, social,
and economic conditions that directly inspire and motivate terrorist activity
(Crenshaw 1981, 381), while enabling or permissive factors refer to conditions that
provide opportunities for terrorism to happen (Crenshaw 1981, 381). The distin-
guishing feature of polity is that, contrary to other variables, it resists compartmen-
talization in either of the two, as some regime attributes of polity correspond to the
direct factors while others correspond to the permissive factors (Drakos and Gofas
2006). Out of this idiosyncrasy, two competing explanations of the correlation
between democracy and terrorism have emerged.
Under the assumption that terrorism is a strategic choice, where the leaders of ter-
rorist groups will try to maximize the return while minimizing the cost of engaging
in terrorist activity, the so-called political access school argues that democratic
societies are expected to be associated with less terrorism. That is because the insti-
tutions and procedures of democratic polities provide effective conflict-reducing
mechanisms (Schmid 1992, 17). These mechanisms can help citizens address and
channel their grievances through established institutional venues and, eventually,
resolve them in a nonviolent fashion (Schmid 1992; Ross 1993; Eubank and
Weinberg 1994; Eyerman 1998; Li 2005). In effect, democracies eliminate the
suspected direct factors that generate terrorism by increase[ing] the expected return
of legal activity and offer[ing] multiple channels of non-violent expression without
the threat of government retaliation (Eyerman 1998, 154). The increasing currency
of this line of argumentation among policy circles is reflected in the prominent place
that democratization has gained on the agenda of the war on terror, especially in the
aftermath of the attacks September 11, 2001 (see, e.g., Windsor 2003).
718 JOURNAL OF CONFLICT RESOLUTION

Yet a different reading of how democratic regime attributes (the permissive/enabling

this time) enter the decision calculus of terrorists points to the opposite direction, lead-
ing to the second argument in the terrorism-democracy literature. According to the so-
called strategic school, democracies encourage terrorism by decreasing the price of
and risk associated with engaging in terrorist activities (Eyerman 1998). Democracies
lower the price of illegal activities because of their commitment to civil liberties. These
civil liberties, such as the freedom of movement and association, could provide fertile
ground for the creation and function of terrorist groups. Moreover, the prerequisite in
democratic legal systems of rigorous proof for conviction makes democracies more vul-
nerable to terrorist operations (Schmid 1992).
Most authors seem to agree that democracy is positively correlated with terror-
ism. Ross (1993, 321) argues that the lions share of terrorism takes place in democ-
racies, and most of the applied studies on the subject seem to support this view
(Eubank and Weinberg 1994, 2001; Weinberg and Eubank 1998; Li and Schaub
2004; Li 2005). Indeed, recent high-profile events, such as the attack on the Twin
Towers in New York, the train bombing in Madrid, and the coordinated attack on the
London transport network, lend a degree of prima facie credence to the argument, as
they highlight the vulnerability of democracies to acts of terrorism.
As already stressed, however, democracies may erroneously appear as experienc-
ing more terrorist activity, compared to nondemocracies, due to the high propensity
of the latter to underreport terrorist attacks. In effect, despite the important insights
that extant literature has generated, it may have not fully specified the manner in
which terrorism and polity are linked since the hypothesized presence of an under-
reporting bias has not been adequately addressed.
At this point, let us provide a simple stylized presentation of the discussion thus
far. The following table summarizes the expectations from the three factors identi-
fied in the literature to be affecting the observed correlation between regime type
(polity) and terrorism.

The Polity-Terrorism-Reporting Nexus

Encouragement/ Grievances/Direct Reporting

Enabling Effect Effect Probability

Democracies Max Min Max

Autocracies Min Max Max

In the first column are the two extreme opposing poles of the polity spectrum,
while the first row shows the three factors identified in the literature to be affecting
the observed correlation between polity and terrorism. An even cursory look at the
table reveals that the so-far established positive relation in the literature between
democracy and terrorism may just be a function of reporting probability. Without
making any claim over the relative magnitude of these factors, we see that the two
 Drakos, Gofas / DEVIL YOU KNOW BUT ARE AFRAID TO FACE 719

opposing effects of regime attributes (encouragement and grievances) are expected

to attain their maximum magnitude at the two extremes of the polity spectrum. Thus,
reporting probability emerges as a determinant factor of the observed (i.e., reported
but not necessarily actual) terrorist activity and, hence, relation between democracy
and terrorism. These important issues need to be empirically addressed.
However, despite the apparent importance of the terrorism-polity-reporting
nexus, research has fallen short of addressing it in a systematic fashion. More specif-
ically, Eubank and Weinberg (1994), in their pioneering study, provide evidence that
democracies are more prone to terrorism. By applying methods of statistical inference,
they demonstrate that the likelihood of finding terrorist groups in democracies is sig-
nificantly greater than the likelihood of finding them in autocracies.
Their research did not go unchallenged. Miller (1994) criticized their research for
failing to resolve the very problem the authors themselves addressnamely, the
underreporting bias of nondemocracies. Sandler (1995) questioned the validity of
using the presence of terrorist groups, rather than events data, as a measure of ter-
rorist activity. In a follow-up article, Weinberg and Eubank (1998) try to address
their critics but only partially. By employing events data, from the RANDSt.
Andrews Chronology of International Terrorism for 1994 and the U.S. State
Departments chronology of international terrorist acts for 1995, they confirm their
earlier finding that democracies are more prone to acts of international terrorism.
Moreover, they find that regime change itself, irrespective of the direction, was a sig-
nificant factor as countries undergoing political transition were particularly volatile.
However, Weinberg and Eubank do nothing to control for the hypothesized distort-
ing effect of underreporting bias because they assume that [t]here is no obvious way
by which such a hypothesized effect might be measured (Eubank and Weinberg
1994, 418).
The same year, Eyerman (1998) evaluated the two competing explanations of the
way in which democratic regime attributes enter into the decision calculus of the ter-
rorist organization. By using events data from the ITERATE (International
Terrorism: Attributes of Terrorist Events) data set and employing a multivariate neg-
ative binomial regression, he found the correlation between established democracies
and terrorism to be negative while that between new democracies and terrorism to
be positive. Eyerman not only acknowledges the potential distorting effect of under-
reporting bias but is also trying to address the issue in a progressive way. First, he
argues that the data collection techniques used in the ITERATE project somehow
minimize the potential bias by relying on both national and international media.
However, as he himself concedes, that can address the problem only partially. After
all, international media do depend to a large extent on domestic resources, and they
also tend to report only the major and most lethal incidents. Second, and most impor-
tant, Eyerman is trying to evaluate the relationship between the type of a states
media and the level of reporting based on the correlation between the number of
events per state and the size of the media, where the latter is measured as the aver-
age number of daily newspaper subscriptions. The proposed methodology seems to
hold some promise. However, it quickly falls back from the kind of precision that
720 JOURNAL OF CONFLICT RESOLUTION

otherwise characterizes his study, as his chosen indicator of media type is inappro-
priate to address the problem at hand. Obviously, newspaper subscriptions are a dis-
tant and, at best, rather tangential indicator of underreporting, given the availability
of a superior indicator in the form of press freedom.
More recently, Lai (2003) and Drakos and Gofas (2006)in a study of the
factors that affect the level of terrorism experienced in a state and a study that
provides a sketch of the average terrorist attack venue, respectivelysuggest the adop-
tion of a zero-inflated negative binomial regression as a way of addressing the issue of
underreporting. Although their suggestion addresses the issue at hand in a method-
ologically progressive way, it does not fully resolve it. As already noted, excessive
zeros are attributable to (1) structural reasons that generate zeros (a country not expe-
riencing terrorism, irrespective of the presence of underreporting) and (2) underre-
porting bias. The problem is that a zero-inflated model cannot disentangle between the
two zeros-generating factors.

TERRORISM AND PRESS FREEDOM

As noted earlier, in addition to the pure effect of press freedom on underreport-

ing bias, press freedom is also related to publicity seeking. Publicity is an important
factor in the terrorists decision-making calculus, as it increases the expected return
of carrying out an attack in a country whose media will provide extensive coverage
(see, e.g., Li 2005; Sawyer 2005). This is for a number of reasons. First, publicity
enhances the ability of terrorists to propagandize their goals and, thus, to recruit new
members. Beside providing the venue for the spread of terrorist ideology, media cov-
erage can also provide strategic advantages to terrorists. For example, it can supply
terrorists with critical real-time tactical information, such as particular political pres-
sures that can be exploited during negotiations (Sawyer 2005). Even more, in the
event of hostage taking, it will increase the political value of those taken captive,
thus increasing the negotiating power of terrorists (Sawyer 2005).
At the same time, in modern societies, violence, both as information and enter-
tainment, has become a commercial product for media (Schmid 1992, 22). In effect,
media and terrorism seem to be embroiled in a symbiotic relationship (Wilkinson
1997, 54) that creates an upward spiral of violence (Kelly and Williams 1981,
quoted in Scott 2001, 216).
However, despite the seeming truism of the upward spiral of violence, there is
a certain degree of caution with which we should interpret the media-terrorism com-
plex. Scott (2001) hits the nail on the head by pointing out that there is a natural limit
on the media-based returns to terrorist activities. The departure point of his analysis
is that there are actually three, rather than just two, actors involved in this complex.
While terrorists and the media might be embroiled in a mutually beneficial symbi-
otic relationship, the media must consider the publics preferences for various kinds
of news coverage (Scott 2001, 216). By analyzing the utility-maximizing mix of
various types of news, he demonstrates a media congestion effect, according to
 Drakos, Gofas / DEVIL YOU KNOW BUT ARE AFRAID TO FACE 721

which terrorists will attract less media attention as more terrorists compete for the
public eye. Using column inches in the New York Times as a measure of media cov-
erage, he finds that a terrorist groups coverage falls by 0.124 inches when another
terrorist incidents coverage increases by one inch (Scott 2001, 225). Thus, the most
appropriate indicator for the publicity effect would actually be media coverage,
rather than press freedom, as it would allow us to estimate the threshold after
which the media-based returns to terrorism start to diminish. Moreover, nothing
guarantees that for a given level of press freedom, media coverage will be equal
among different countries. That is, press freedom cannot account for coverage
variations within a given level of press freedom, thus excluding potentially impor-
tant information. Nonetheless, this need not concern us here, as our purpose is not to
evaluate publicity seeking but to propose a test for the existence (or not) of the pure
effect of press freedom (i.e., underreporting).
However, before we turn on these matters explicitly, we need to discuss two
recent attempts at examining the complex relationship between terrorism, democ-
racy, and press freedom. In a path-breaking study, Li (2005) identifies and investi-
gates new mechanisms by which democracy affects transnational terrorism. He also
presents the first study that is trying to examine underreporting bias by controlling
for the effect of press freedom.
Lis (2005) overall aimindeed, the very point of departure for his analysisis
to tackle the disconnection in the terrorism-democracy literature between theoretical
arguments and empirical analyses. As already mentioned, polity is a rather idio-
syncratic variable as different regime attributes are simultaneously affecting terror-
ism in opposing directions. Li lucidly points out that existing empirical studies
fail to distinguish between the negative and positive effect of democratic regime
attributes because they are based on an aggregate indicator of polity. By drawing on
a range of theoretical literatures, he makes a convincing case for taking into account
the heterogeneity of democratic systems both in terms of variations in institutional
checks and balances and variations in electoral systems. This allows him to intro-
duce government constraints and democratic participation as alternative and disag-
gregated indicators of polity. Because of the new theoretical mechanisms identified
and disaggregated indicators introduced, polity is no longer as much of an idiosyn-
cratic variable. Second, Li is trying to control for the effect of press freedom, which
in turn he considers as part and parcel of civil liberties. The thrust of his argument is
that civil liberties not only epitomize the grievances-alleviating nature of democra-
tic regime attributes but also create an additional incentive for terrorism via the pub-
licity provided by the media coverage of free press.
After assessing in a multivariate analysis the significance of the new mechanisms
identified, Li (2005) demonstrates that democratic participation and government
constraints play a significant role. Democratic participation reduces transnational
terrorism, while the institutional constraints over government influence significantly
the positive relationship between democracy and terrorism. Li concludes that
the effect of civil liberties (and of press freedom as an indispensable part of them) is
driven and encompassed by the impact of government constraints.
722 JOURNAL OF CONFLICT RESOLUTION

In the context of such a major contribution, it may seem somewhat churlish to

point to the limitations of Lis (2005) account. However, when it comes to the
very issue of underreporting bias, his analysis is not without problems. Although we
welcome the joint inclusion of the disaggregate measures of polity and press freedom,
we still believe that this model cannot fully address the issue of underreporting. Put
simply, the simple inclusion of press freedom as an additional regressor cannot dis-
tinguish empirically between the encouragement effect and underreporting.
Sawyer (2005) presents another engaging and thoughtful study of the relationship
between democracy, terrorism, and press freedom. He rightly acknowledges that the
main challenge in the analysis of underreporting is to account for the high correla-
tion between press freedom and polity. Sawyers starting point is that media public-
ity is an important variable in the decision-making calculus of terrorists.
Consequently, a testable implication of such behavior would be the internationaliza-
tion of terrorism (i.e., the migration of terrorist activity from countries with censored
media toward countries with higher potential for publicity). To test this hypothesis,
Sawyer uses a subsample of internationalized terrorism, expecting a positive effect
of differences in press freedom as a direct result of the publicity effect. His proposal
advances the debate but suffers from the following problems. His chosen subsample
definitely includes the maximum of the effect he is trying to capture, but also con-
tains the maximum of the encouragement effect and the minimum of the reporting
probability (or the maximum of underreporting bias). That is, his analysis does not
distinguish between these sources of variations as explanatory of the migration of
terrorist activity. A further disadvantage of this strategy is the need to focus on a sub-
sample corresponding only to a small fraction of the population, which definitely is
not representative.
To take stock of the discussion thus far, we have argued that terrorism, polity, and
press freedom constitute an inseparable conceptual trinity that poses to the student
of underreporting a number of major challenges. First, one has to isolate the part of
terrorism and polity correlation attributed solely to underreporting by effectively
controlling for the encouragement effect. Second, one must take into account the
high correlation between press freedom and polity. In the following section, we
focus directly on this challenge, hoping to provide a more comprehensive analysis
of the effect of underreporting bias.

UNDERREPORTING BIAS: A MORE FORMAL TREATMENT

The potential underreporting bias of terrorist activity can be classified under the
well-known statistical phenomenon of incomplete count data, where only some of
the events of interest are reported. The incompleteness of recording (and hence
reporting) results in so-called thinning, where the recorded process is an understate-
ment of the true underlying process (Solow 1993; Yannaros 1993). Thinning high-
lights that the inclusion probability (i.e., the probability that an event is reported) is
less than 1.
 Drakos, Gofas / DEVIL YOU KNOW BUT ARE AFRAID TO FACE 723

Consequently, the observed process is the outcome of two mechanisms: (1) the
fundamental generating mechanism of the true process and (2) the thinning process,
which is clearly dependent on the inclusion probability. Given that the inclusion
probability is bounded below by zero and upwards by unity, one can easily deter-
mine the two extreme casesa zero inclusion probability (in such a case, there
would no reporting of the process) and an inclusion probability of unity (where
underreporting would be absent; i.e., full reporting).
Let us offer a rather simple setup that will accommodate a better understanding
of the problem at hand. Define Yi,t as the true number of (terrorist) events in time
period t in country i. Now define Yi,t as the reported number of (terrorist) events in
time period t in country i. In addition, define Pini,t as the inclusion probability associ-
ated with country i, which accounts for any discrepancies between Yi,t and Yi,t and sat-
isfies the standard condition 0 Pini,t 1. Then, the probability of the observed
(reported) number of terrorist incidents is the product of the actual (true) probability
of occurrence and the reporting probability:
Pr(Yi,t = k) = [Pr(Yi,t = k)] [Pini,t ]. (1)

From equation (1), one can easily see the two extreme and degenerate cases:

the case of zero reporting; [Pini,t ] = 0, which implies that Pr(Yi,t= k) = 0, and
the case of full reporting; [Pini,t ] = 1, which implies that Pr(Yi,tt= k) = [Pr(Yi,t = k)].

For intermediate values of [Pini,t ], some nontrivial degree of underreporting will

take placethat is, when 0 < [Pini,t ] < 1 Pr(Yi,t= k) < [Pr(Yi,t = k)], i, t. The interested
reader has surely already realized the prominent role played by the inclusion proba-
bility, Pini,t . It is needless to say that the assumption of a known inclusion probability
would be totally unrealistic, if not methodologically hazardous. Thus, there is an
inescapable need for a statistically sound method that will allow one (1) to establish
the presence of underreporting (i.e., that the inclusion probability is indeed lower
than unity) and (2) to provide an estimate of its value (i.e., the degree of underre-
porting). In the present study, we are dealing with the former. That is, we deal with
the issue of establishing whether any underreporting takes place and, if it does, what
its effects will be. In what follows, we discuss a few points, which we feel will assist
us in shedding light on these issues.
Establishing underreporting using equation (1) would not be a problem provided
that the set of state variables determining [Pr (Yi,t= k)] shared no common elements
with the set that determines Pini,t . Problems arise insofar there is some commonality
in these sets and, furthermore, as we will discuss later, when the impacts of these
common elements move to the same direction.
Let us start with the determinants of Pini,t . We argue that the most influential variable
for the reporting probability is freedom of press, FP. However, FP depends on the level
of democracy. The suspected gap between the actual occurrence of terrorist events and
the data sets may have detrimental effects on the subsequent estimation and inference
conducted by applied researchers. A common feature of all available terrorism events
databases is that they are primarily compiled on the basis of information appearing in
724 JOURNAL OF CONFLICT RESOLUTION

Overall Positive Impact of Polity on Terrorist Activity

Indirect I: Reporting
Direct: Encouragement Effect Probability
Indirect II: Publicity Seeking

Figure 1: Impact of Polity on Terrorist Activity

publicly available sources, which most of the time coincide with the press. Excluding
the possibility that the press is inherently biased leads us to believe that the inclusion
probability (and, of course, underreporting bias) will depend on the level of press free-
dom. It is only natural to think that if any underreporting were present, it would be
more pronounced under political regimes that have a tendency to either censor or in
some way control the press. So, polity emerges as the fundamental determinant of
underreporting bias, although latent, exerting its impact via press freedom. Then, it
follows that the positive impact of polity on terrorist activity, which has been estab-
lished so far in the literature, consists of a direct effect (encouragement effect) and
two indirect effects (reporting probability and its intensification of publicity seeking
via press freedom), as shown in Figure 1.
To show the interrelation between polity and press freedom, we used data from
Freedom House, which classifies each country, in terms of level of press freedom, as
not free, partly free, or free. Based on this classification, we constructed the follow-
ing set of dummy variables to capture the level of press freedom:

Di,tF = 1, if press in country i and year t is classified as free and 0 otherwise;

i,t = 1, if press in country i and year t is classified as partly free and 0 otherwise; and
DPF
Di,tNF = 1, if press in country i and year t is classified as not free and 0 otherwise.

Then we calculated the sample correlations between each dummy and polity, which
are reported in Table 1.
Inspection of the sample correlations reveals that press freedom is highly corre-
lated with the level of polity, especially as we move away from the partly free press.
In particular, press freedom is increasing (decreasing) as the level of democracy
increases (decreases). The positive association between the two variables is evident,
suggesting that press freedom is increasing with the level of democracy. Moreover,
if polity and press freedom were independent, then for any given level of polity, the
probability that a countrys press was free, partly free, or not free would be equal. In
 Drakos, Gofas / DEVIL YOU KNOW BUT ARE AFRAID TO FACE 725

TABLE 1
Sample Correlations of Polity and Freedom of Press Dummies

Corr[Polity, Freedom of Press] DFi,t DPF

i,t DNF
i,t

(Polityi,t) 0.67 0.11 0.74

other words, for each point on the polity scale, the distribution mix of countries
would be roughly equal across the three possible statuses of press freedom (more
formally, the distribution of counts across press freedom would be uniform). To gain
more insight regarding the dependence (or not) of these two fundamental character-
istics, we calculate, for each polity level, the percentage of countries that are classi-
fied as having a press that is free, partly free, or not free.2 Figure 2 depicts the
percentages of a free, not free, and partly free press for each polity level.
One immediately observes that the distribution of press freedom strongly depends
on polity, especially for extreme levels of the latter. In other words, for the vast
majority of strongly autocratic states, press is not free, while for strongly democra-
tic states, it is essentially free. This strong conditionality implies that as we consider
purer types of polity, the distribution of countries across press freedom statuses
becomes unimodal; in fact, for extreme levels of polity, these distributions become
almost degenerate. The distribution of a partly free press against polity depicts a
totally different picture showing considerably higher dispersion and does not seem
to be related to a specific level of polity. The same conclusions are reached if we
observe the reverse distributions (i.e., the mix of press freedom across polity levels).3
Thus, the actual data indicate that even though the monotonicity of g() is defi-
nitely apparent, there is another more subtle, but rather crucial, characteristic of the
actual data. This is the fact that the concentration of a not free press increases rapidly
for polity levels below a certain threshold a, while the concentration of a free press
increases rapidly for polity levels above a certain threshold b. In contrast, for levels
of polity between a and b, the mix of press freedom is relatively dispersed, with all
three types (of polity) being represented. In other words, between these thresholds,
the correlation between polity and press freedom is minimized.
Hence, since there is not perfect correlation between the level of democracy and
the level of press freedom, one may statistically isolate the encouragement and
underreporting effects. This identification will be based on the following:

1. Under the validity of the encouragement effect, for a given level of press freedom (FP),
terrorism activity increases with the level of polity; lim [Pr(Yi,t > 0)] 1.
Polity ax|FP

2. In the presence of underreporting bias, for a given level of polity, the propensity to
report terrorist attacks increases with the level of press freedom; lim (Pini,t) 1.
FP ax|Polity

2. It is obvious that these three cases are mutually exclusive (in any given year, a countrys press will be
classified as free, partly free, or not free) and exhaustive (the sum of these percentages will add up to 100).
3. We do not show the relevant graphs for space conservation reasons. The graphs are available from
the authors upon request.
 100

726
80

60

%
40

20

0
Polity Scale: (-10, +10)

F PF NF

Figure 2: Distribution Mix of Countries by Press Freedom across Polity

NOTE: Vertical axis denotes the percentage (0-100) of free (F), partly free (PF), and not free (NF) countries for each polity level. The horizontal axis measures the
polity level (10, +10). For each polity level, the sum of F, PF, and NF adds up to unity (100 percent).
 Drakos, Gofas / DEVIL YOU KNOW BUT ARE AFRAID TO FACE 727

Press Freedom

Polity Not Free Partly free Free

Probability of -10
attack occurrence
increases -9
downward
(under the validity -8
of encouragement
effect)

+9

+10
Probability of reporting increases rightward
(under the presence of bias)

Figure 3: Terrorism Activity by Polity and Freedom of Press

Let us provide some intuition regarding the statistical strategy that will be
adopted later. Essentially, we will exploit (1) the joint probability distribution of
terrorist activity and polity, conditional on press freedom, and (2) the joint probabil-
ity distribution of terrorist activity and press freedom, conditional on polity.
Consider Figure 3, which describes the space spanned from actual data (a state will
be characterized by a pair of values for its polity level and press freedom) in the pres-
ence of the encouragement effect and underreporting bias.

DATA ISSUES

We use a sample of cross-sectional time series, where the cross-sectional dimension

consists of 153 counties and the time dimension covers the period from 1985 to 1998
(annual intervals). The variable of interest is transnational terrorist activity (TERRi,t),
which measures the number of transnational terrorist incidents that occurred (that is
reported) in country i (i = 1, . . . , 153) in period t (t = 1985, . . . , 1998). Terrorism is
defined by the nature of the act, not by the identity of the perpetrators or the nature of
the cause. Terrorism is violence, or the threat of violence, calculated to create an atmos-
phere of fear and alarm. These acts are designed to coerce others into actions they would
not otherwise undertake or refrain from actions they desire to take. In transnational
728 JOURNAL OF CONFLICT RESOLUTION

TABLE 2
Sample Distribution of Counts (Based on 4,567 Attacks for
the Period 1985-1998, across 153 Countries)

Panel A: Descriptive Statistics

Standard
Mean Deviation Skewness Kurtosis

TERR 2.13 7.19 8.86 116.08

Panel B: Observed Frequencies

Actual Number of
Distribution of Observed Percentage of Attacks (Count *
Counts (TERR) Frequency Frequency Observed Frequency)

0 1,357 63.35 0
1 286 13.35 286
2 114 5.32 228
3 101 4.73 303
4 59 2.75 236
5 33 1.54 165
6 21 0.98 126
7 20 0.93 140
8 19 0.89 152
9 11 0.51 99
10 13 0.61 130
11 12 0.56 132
12 11 0.51 132
13 5 0.23 65
14 4 0.19 56
15 6 0.28 90
16 7 0.33 112
17 4 0.19 68
18 2 0.09 36
19 6 0.28 114
20 5 0.23 100
21 3 0.14 63
22 2 0.09 44
23 3 0.14 69
24 7 0.33 168
25 4 0.19 100
26 1 0.05 26
27 2 0.09 54
28 1 0.05 28
29 1 0.05 29
31 1 0.05 31
33 2 0.09 66

(continued)
 Drakos, Gofas / DEVIL YOU KNOW BUT ARE AFRAID TO FACE 729

TABLE 2 (continued)

Panel B: Observed Frequencies

Actual Number of
Distribution of Observed Percentage of Attacks (Count *
Counts (TERR) Frequency Frequency Observed Frequency)

34 1 0.05 34
35 1 0.05 35
36 2 0.09 72
37 2 0.09 74
38 1 0.05 38
42 2 0.09 84
43 1 0.05 43
52 1 0.05 52
57 1 0.05 57
66 2 0.09 132
81 1 0.05 81
86 2 0.09 172
109 1 0.05 109
136 1 0.05 136
Total 2142 100 4567

NOTE: In panel B, the information provided in the four columns is as follows. The first column shows
the counts of attacks that have occurred in the sample. The second column refers to the number of times
each count has been observed (frequency) and is also equal to the size of the sample (number of country-
years = 14 153). The third column shows the same information as column 2, but in a percentage form.
Finally, the fourth column reports the actual number of terrorist attacks considered in the sample.

terrorist incidents, terrorists go abroad to strike their targets, select domestic targets
associated with a foreign state, or create an international incident.
Data on transnational terrorism were collected from the National Memorial
Institute for the Prevention of Terrorism (MIPT) Terrorism Knowledge Base and are
publicly available free of charge. The MIPT Terrorism Knowledge Base integrates
data from the RAND Terrorism Chronology, the Terrorism Indictment database
(University of Arkansas and University of Oklahoma), and DFI Internationals
research on terrorist organizations. In Table 2, we provide a summary of the sample
statistics of terrorist activity.
The empirical distribution of terrorism shows some interesting characteristics. We
observe that the sample standard deviation value is almost four times larger than the
mean. Valuable information is also encapsulated in the skewness and kurtosis mea-
sures, which attain values of 8.88 and 116.67, respectively. Recall that skewness is
a measure of possible asymmetry of a density function around the mean. While dis-
persion indicates the degree of variation around the mean, skewness reveals the
direction of variation. In our case, the value indicates that the sample distribution
is positively skewed. The observed distribution has many zero values (no terrorist
730 JOURNAL OF CONFLICT RESOLUTION

TABLE 3
Counts of Terrorist Incidents across Countries

Number of Incidents Countries

0 Belize, Bhutan, Botswana, Burkina Faso, Cameroon, Cape Verde,

Comoros, Cte dIvoire, Eritrea, Gabon, Gambia, Guinea, Guinea-Bissau,
Iceland, Kazakhstan, Lesotho, Madagascar, Malawi, Maldives, Mauritius,
Mongolia, Oman, Qatar, Sao Tome and Principe, Seychelles, Slovenia,
Turkmenistan, Uzbekistan, Viet Nam
1-10 Armenia, Azerbaijan, Bangladesh, Belarus, Benin, Bulgaria, Burundi,
Canada, Central African Republic, Chad, Congo (Kinshasa), Croatia,
Czech Republic, Djibouti, Estonia, Finland, FYROM, Ghana, Guyana,
Hong Kong, Hungary, Indonesia, Kenya, Kyrgyzstan, Laos, Latvia,
Liberia, Libya, Lithuania, Luxembourg, Malaysia, Mali, Malta, Mauritania,
Moldova, Namibia, Nepal, New Zealand, Niger, Nigeria, Norway,
Paraguay, Portugal, Romania, Rwanda, Senegal, Singapore, Slovakia,
Suriname, Swaziland, Syria, Tanzania, Togo, Tunisia, United Arab
Emirates, Uruguay, Zambia, Zimbabwe
11-20 Australia, Bahrain, Brazil, Costa Rica, Denmark, Ethiopia, Kuwait,
Mexico, Morocco, Mozambique, Nicaragua, Panama, Poland, Saudi
Arabia, Sierra Leone, South Africa, Sweden, Tajikistan, Thailand,
Uganda, Ukraine
21-40 Albania, Angola, Argentina, Austria, Belgium, Bolivia, Cambodia,
China, Cyprus, Ecuador, El Salvador, Georgia, Guatemala, Honduras,
Iran, Japan, Jordan, Netherlands, Sri Lanka, Sudan, Switzerland,
United States
41-60 Somalia, South Korea, Venezuela
61-100 Egypt, India, Ireland, Italy, Yemen
100-150 Algeria, Chile, Greece, Philippines, Russia
151-200 France, Pakistan, United Kingdom
201-250 Germany, Israel, Peru, Spain, Turkey
251-350 Colombia, Lebanon

incident) and also high frequencies of low terrorist activity (values ranging from 1
to 4). Kurtosis measures peakedness in relation to tails. Overall, these measures sug-
gest that the terrorism distribution is highly positively skewed and leptokurtic with a
long right tail. The number of terrorist incidents (over our sample) on a country basis
is provided in Table 3.
Colombia, as it is well known, is by far the country with the most terrorist attacks,
followed by Lebanon, Spain, and Germany. What is remarkable is that just twenty-
three countries (15 percent of our sample) account for 74 percent of all terrorist
attacks that took place between 1985 and 1998.
We also collected data from POLITY IV on the Overall Polity Index; (POLITYi,t)
takes values from +10 (strongly democratic) to 10 (strongly autocratic). Finally,
based on data reported by Freedom House, each country has been classified in terms
of its press freedom level.
 Drakos, Gofas / DEVIL YOU KNOW BUT ARE AFRAID TO FACE 731

Autocracies Semi-democracies Democracies

Not Free
A B C G1=A+B+C
Press

Partly Free
D E F G2=D+E+F
Press

Free Press G H I G3=G+H+I

G4=A+D+G G5=B+E+H G6=C+F+I

Figure 4: Sample Decomposition by Level of Polity and Press Freedom

ECONOMETRIC METHODOLOGY AND ESTIMATION RESULTS

Figure 4 offers a decomposition of the sample by level of polity and press free-
dom (it essentially replicates the previous table in a more compact way). Essentially,
we decompose the sample in terms of the joint distribution of polity and press free-
dom. Clearly, the sum of components is exhaustive, and components share no com-
mon elements (i.e., they are mutually exclusive).
Each row depicts the distribution of polity for a given level of press freedom.
Similarly, each column shows the distribution of press freedom for a given polity
level. If one constructs these components based on data, it turns out that C and G are
effectively empty sets (i.e., there are no democratic states whose press is not free,
and there are no autocracies with fully free press). As outlined, the student of under-
reporting bias is faced with a dual challenge: (1) to find a way of disentangling the
encouragement from the underreporting effect, both of which move in the same
direction, and (2) to address the high correlation between polity and press freedom.
It is clear that using the whole sample space (i.e., all segments) would suffer from
the above-mentioned problems, which would lead to erroneous econometric results.
Therefore, one needs to focus on a subsample that ultimately will be compared to the
whole sample to assess the presence of underreporting. Groups G1, G3, G4, and G6
are disqualified on the following grounds: they either contain segments A, I, which
introduce the highest impact of the encouragement effect, or segments C, G, which
are empty and therefore offer a lopsided view of the polity spectrum. Similarly, G5
contains a rather restricted range of polity. Hence, group G2 emerges as the only
candidate, offering a fairly representative picture of the polity spectrum (controlling
for the encouragement effect) and the lowest correlation between polity and press
freedom.
732 JOURNAL OF CONFLICT RESOLUTION

TABLE 4
Terrorist Activity on Polity

Conditional on Partly
Whole Sample Free Press

Dependent Variable: Coefficient Estimate

(TERRi,t) (t-Statistic)/(p-Value)

(POLITYi,t) 0.065** 0.102*

(2.95)/(.00) (1.74)/(.08)
(TERRi,t-1) 0.424** 0.277**
(8.74) (2.80)
(TERRi,t-2) 0.330** 0.571**
(7.37) (6.17)
Intercept 0.775** 0.543
(4.40) (1.37)
Wald statistic 507.62** 165.57**
Observations 1,671 489

NOTE: The Wald statistic stands for the overall significance of the regression.

Based on these, we propose a way of assessing whether any underreporting bias

is present by using the following two regression models:
(TERRi,t) = 0,i+ 1 (POLITYi,t) + 2 (TERRi,t1) + 3 (TERRi,t2) + i,t (2)

and

i,t ) = 0,i + 1 (POLITY i,t ) + 2 (TERR i,t-1) + 3

(TERRPF i,t-2) + i,t
PF PF PF PF PF PF
(TERRPF PF
(3)

where i and t denote the country (cross section) and year (time), respectively, while
s are unknown parameters to be estimated, s are spherical disturbance terms,
PF
and (TERRPF i,t-1) and (TERR i,t-2) are past levels of terrorism activity used as control
4
variables.
Equations (2) and (3) have similar structure, but they fundamentally differ in
terms of the sample they employ in the estimation stage. Equation (2) is estimated
over the whole sample, while equation (3) is based only on a subsample that
emerged from the previously discussed elimination process (partly free press).
How does this help us in assessing the presence or not of underreporting?
Essentially, any difference between (2) and (3) can be solely attributed to underreport-
ing, given that all polity levels are present in both (2) and (3) (i.e., the encouragement
effect is affecting both equations in a similar manner). Thus, underreporting is
qualified as the only source of variation explaining differences between (2) and (3).

4. The choice of lag length was based on a formal Wald test that rejected the significance of the
fourth lag, the third lag, and jointly the third and fourth lags.
 Drakos, Gofas / DEVIL YOU KNOW BUT ARE AFRAID TO FACE 733

Suppose that underreporting is absent.5 Conditioning the sample on the basis of

press freedom would be irrelevant; that is, variations in press freedom would not
contain any relevant information for the probability of reporting and, hence, the
parameters 1 and 1PF should be roughly the same in terms of absolute value, as
well as significance. Now assume that underreporting is present. How would it
manifest itself in terms of the estimated parameters? To the extent that the sample
used in equation (3) is adequately purified, while the (whole) sample used for
equation (2) still suffers from the above-mentioned problems, we expect the
following.
Given the strong correlation between polity and press freedom, extreme levels of
democracy are associated with artificially inflated terrorism, when compared to
extreme levels of autocracy due to underreporting in the latter. This results in a sta-
tistically stronger relationship between terrorism and polity, which is directly
inflated because of underreporting bias. Furthermore, the magnitude of the point
estimate of 1 will be lower than the true value since regression will result in an aver-
aging of coefficients across countries with low underreporting bias (democratic
states) and countries with high underreporting bias (autocratic states). Thus, the
strong correlation of press freedom and polity will cause a double bias in the esti-
mation of 1an upward bias of its significance and a downward bias of its magni-
tude. To investigate these issues, we estimate the parameters of equations (2) and (3)
using random effects and report the estimation results in Table 4.
Based on the whole sample, the partial derivative of terrorism with respect to
polity is 0.065, with a z-score of 2.95 (probability value .00). When we regress ter-
rorism on polity for the subsample of countries satisfying the criterion that their
press is partly free (i.e., exclude the two extreme cases of free and not free press),
the results we obtain are consistent with the patterns of bias discussed earlier.
The point estimate of the partial derivative of terrorism with respect to polity is
0.102. In addition, the significance of this partial derivative decreases substantially
between the two regressions. The z-score drops from 2.95 in the whole sample to
1.74 in the subsample of the partly free press group. The apparent differences
between the magnitude and significance are attributed to the averaging in the whole
sample across heterogeneous levels of reporting associated with different levels of
press freedom.
Hence, the substantially different results obtained from the two regressions high-
light the presence of underreporting bias. As a by-product of our analysis, one may
easily appreciate the adverse impact of the correlation between democracy and press
freedom on the statistical properties of the 1 estimate. As discussed earlier, econo-
metric applications based on the whole spectrum of press freedom will produce esti-
mates that suffer from a double bias. First, the obtained derivative of terrorism with
respect to polity will be downward biased due to the averaging effect and, second,
the statistical significance of this derivative will be upward biased.

5. A similar argument would be in place if the probability of reporting was less than unity but equal
across polity levels.
734 JOURNAL OF CONFLICT RESOLUTION

CONCLUSIONS

In this article, we have discussed the issue of underreporting bias in the study of
terrorism and how one may assess the presence of this bias. Our analysis has
exploited the correlation between press freedom and polity as a means of investigat-
ing whether the observed correlation between terrorist activity and polity can be
attributed to underreporting.
We discussed in what ways the potential underreporting bias may express itself
and have also attempted to illustrate these effects. The main conclusion of our study
is that underreporting is indeed present, implying that the databases used by applied
researchers represent an understatement of true terrorist activity worldwide.
Moreover, this understatement is not simply an overall scaling-down effect ran-
domly distributed across countries. In fact, it is highly concentrated in countries
whose press is not free, which typically correspond to countries that lie on low levels
of the polity scale (nondemocracies). Clearly, this has significant implications for
issues such as constructing indices of terrorism risk on a country level, as well as the
efficient deployment of counterterrorism measures.
In our opinion, the next major challenge the literature faces is to come up with a
quantitative assessment of this bias, which is the nuts and bolts of underreporting.

REFERENCES

Crenshaw, Martha. 1981. The causes of terrorism. Comparative Politics 13 (4): 379-99.
Drakos, Konstantinos, and Andreas Gofas. 2006. In search of the average transnational terrorist attack
venue. Defence and Peace Economics 17 (2): 73-93.
Eubank, William, and Leonard Weinberg. 1994. Does democracy encourage terrorism? Terrorism and
Political Violence 6 (4): 417-43.
. 2001. Terrorism and democracy: Perpetrators and victims. Terrorism and Political Violence 13
(1): 155-64.
Eyerman, Joe. 1998. Terrorism and democratic states: Soft targets or accessible systems? International
Interactions 24 (2): 151-70.
Lai, Brain. 2003. Examining the number of incidents of terrorism within states, 1968-1977. Paper pre-
sented at the annual meeting of the American Political Science Association, September, Philadelphia.
Li, Quan. 2005. Does democracy promote or reduce transnational terrorist incidents? Journal of Conflict
Resolution 49 (2): 278-97.
Li, Quan, and Drew Schaub. 2004. Economic globalization and transnational terrorist incidents. Journal
of Conflict Resolution 48 (2): 230-58.
Miller, Abraham H. 1994. Comment on terrorism and democracy. Terrorism and Political Violence 6 (4):
435-9.
Ross, Jeffrey I. 1993. Structural causes of oppositional political terrorism: Towards a causal model.
Journal of Peace Research 30 (3): 317-29.
Sandler, Todd. 1995. On the relationship between democracy and terrorism. Terrorism and Political
Violence 7 (4): 1-9.
Sawyer, John. 2005. Terrorism, press freedom, and democracy: A nuance relationship. Paper presented at
the annual meeting of the International Studies Association, March, Honolulu, HI.
 Drakos, Gofas / DEVIL YOU KNOW BUT ARE AFRAID TO FACE 735

Schmid, Alex P. 1992. Terrorism and democracy. Terrorism and Political Violence 4 (4): 14-25.
Scott, John L. 2001. Media congestion limits media terrorism. Defence and Peace Economics 12:215-27.
Solow, Andrew R. 1993. Estimating record inclusion probabilities. The American Statistician 47 (3):
206-8.
Weinberg, Leonard, and William Eubank. 1998. Terrorism and democracy: What recent events disclose.
Terrorism and Political Violence 10 (1): 108-18.
Wilkinson, Paul. 1997. The media and terrorism: A reassessment. Terrorism and Political Violence 9 (2):
51-64.
Windsor, Jennifer L. 2003. Promoting democratization can combat terrorism. The Washington Quarterly
26 (3): 43-58.
Yannaros, Nikos. 1993. Analyzing incomplete count data. The Statistician 42 (2): 181-7.