Qualitative Research Skills Readings

Tutorial 1
The Practice of Qualitative Research Sharlene Nagy Hesse-Biber and Patricia Leavy
Chapter 1 An Invitation to Qualitative Research
asks questions such as why, how and what
focus on meaning the social meaning people attribute to their experiences, circumstances,
and situations, as well as the meanings people embed into texts and other objects, are the focus
of qualitative research
not only the results, but all processes of qualitative research matter

Dimensions of qualitative research

ontology a philosophical belief system about the nature of social reality; the researcher's
ontological starting point determines how he sees the world, i.e. as consisting of different
patterns that are predictable, or as socially constructed
epistemology philosophical belief system about who can be a knower
those two dimensions influence the researcher's work and determine how he approaches
research
methodology is an account of social reality that extends further than what has been
empirically investigated; there are three methodological approaches (post-positivist,
interpretive and critical)
post-positivism stipulates that the world is patterned and that causal relationships can be discovered
and tested

intrepretive view assumes that the world is socially constructed and that reality can only be
understood through observing the perspectives of different actors involved

critical perspective also sees reality as socially constructed and furthermore assumes that discourses
created in shifting fields of social power shape social reality

two approaches to using theory inductive (generation of the theory out of the data) and
deductive (testing of a theory against data)
methods: the tools that researchers use to collect data: through listening to informants,
observing behaviour and examining historical traces and records

A Holistic Approach to Research

researchers should be open to change their methodology throughout the process, as unforeseen
issues might arise, the methodology might need revision
methodology as a bridge between philosophical framework and methods design (ontology and
epistemology determine the philosophical perspective the methodology then determines the
theoretical perspective which influences the methods design)

Quantitative Research and Positivism

positivism assumes that there is a knowable reality which exists independent from research
and can be observed
 there exist causal relationships between different variables, which can be observed and
explained - thus, patterned social reality is predictable and can potentially be controlled
positivism sets a clear hierarchical relationship between researcher (subject) and the
researched process (object); if we do research on human beings, they thus become objects to
the researcher

What kinds of questions and problems can be addressed with qualitative methods?
usually use of inductive methods start of with a big amount of data, which is then being cut
down and specialized to specific data, the analysis of which leads to a more general
understanding of the topic
open research questions that allow for broad findings
three research purposes:
exploratory: explore an area that has been under researched, help shape a foundation for future
research

descriptive: describe the aspect of social reality under investigation

explanatory: explain social phenomena and the relationship between different components of a
topic

all three research purposes have a commonality: they investigate how people assign meaning
to their experiences as well as social events and topics; furthermore, they examine how the
meanings we assign to our experiences, situations and social events shape our attitudes,
experiences and social realities

Chapter 3 Designing Qualitative Approaches to Research

How do we know what we know?
reliance on members of authority for knowledge-building within societies
the rational mode of knowledge-building: based on an objective logic of inquiry, and unbiased
systematic empirical observation and verification
the scientific method, based on positivism, as a method that relies on unbiased inquiry,
accuracy and objectivity
reliance on our common sense in every-day life as a source of knowledge often misleading
and biased assumptions

What is a qualitative approach to research? Qualitative Research inquiry: a dynamic dance

research paradigm is a set of values and ideas that every researcher has and that impact his
process of research
it is important to know your research paradigm as it will heavily influence your work
our methodological viewpoint is the lens we look through and it determines how and what we
study
reflexivity: the awareness that all knowledge is affected by the social conditions under which
it is produced

Formulating Research Questions: What do you want to ask?

derive inspiration from existing literature and add to potential shortcomings
Research Methods: How will you answer your research questions?
Sampling working with small samples often use of purposive or judgment samples
(samples chose not by chance but on purpose), selection of the samples based on several
criteria)
opportunistic sampling sampling by chance or simply by an opportunity that oppens up,
such as an unplanned meeting with someone who then becomes a studies interviewee
convenience sample: often researchers find the selection of informants boils down to who is
available, who has some specialized knowledge of the setting, and who is willing to serve in
that role
theoretical sampling- also a type of purposive sampling the researcher collects and selects
the data he wants to analyse based on his ground theory
the researcher will begin with one sample and then choose another set of samples afterwards to test
the previous findings. If these findings are confirmed, theoretical saturation occurred

snowball sampling: sampling from a known network, it is used to identify participants when
appropriate candidates for a study are difficult to locate
if a group (for example abused women) is difficult to approach, you will first interview other
informants to get some information and tips on where and how to approach your target sample group
creation of a network of relations

Validity
validity is a process whereby the researcher earns the confidence of the reader
subject your own findings to contra-arguments to proof that it is trustworthy and stable
negative case analysis means looking for negative instances where your theory or findings to
not hold up
communicative validity implies opening your findings for scrutiny by other experts or also the
community where you conducted your research (do the observed people agree with the
researcher's interpretation?)
pragmatic validity observes how the findings impact those who were studied. Which impact to
the findings have on the wider community?
Participatory action research is a form of research that aims on going back to the initial
findings and then observe their implications on the observed community. This requires the
active collaboration of client and researcher and stresses the need for co-learning
include the research participants on all or many of the research phases

triangulation as a validity tool mixed methods approach, two different theoretical

perspectives, use different investigators studying the same phenomenon, use different data
sources

Reliability
is the data reasonable? Is there consistency over time and in different social contexts?
External consistency refers to verifying observations with other divergent sources of data
asks questions such as: is the researcher's relationship to the group and setting fully described?
Is the documentation comprehensive and fully referenced? Are the key informants fully
described? Are the sampling techniques fully documented?

How do you generalize research results in a qualitative study?

analytic generalizability detect underlying social forms and patterns
A Tale of Two Cultures: Contrasting Quantitative and Qualitative Research
James Mahoney and Gary Goertz
1) Approaches to Explanation
- core goal of qualitative research is to explain the outcomes in individual cases
- a causes of effects approach that tries to determine the causes of a specific outcome that is observed
- quantitative research follows the effect of causes approach, identifying the average effect of one or
more causes across a population of cases
- qualitative researchers will try to identify the cause of an observed event in a particular case, while
quantitative researchers will look for the average causal effect of one or more independent variables on
the given phenomenon
- both approaches should thus complement one another, with the qualitative one giving insight into a
specific case and the quantitative one then attempting to generalize this to other cases
2) Conceptions of causation
- qualitative researchers assume a necessary or sufficient causation, i.e. saying that x causes y,
claiming that y would not have occurred without x
- quantitative assumptions are denser, assuming that there are other variables playing a role
3) Multivariate Explanations
- in qualitative research, one focuses on the impact of combinations of variables and only focuses on
effects of individual variables occasionally; a variable is only being observed when it might be a
necessary cause or individually sufficient
- the quantitative researcher is more likely to observe the effect of individual causes
4) Equifinality
- multiple, conjunctural causation
- the idea that there are multiple causal paths to the same outcome
- key role in qualitative thinking; here, the focus is in finding the relevant causational paths that lead to
the observed phenomenon. Each of those causational paths is defined by particular combination of
variables, and the qualitative researcher wants to determine which case (for example which country)
follows which path (for example the development to a welfare state which factors lead to a welfare
state?)
- absent in quantitative research
5) Scope and Causal Generalization
- qualitative: narrow scope of theories so that inferences can only be generalized to a small set of cases
they fear that a too broad generalization will lead to missing out some variables or causal
relationships
- contrary, the scope defined by quantitative researchers is broad as they attempt to be able to
generalize their findings on a large base; they might focus on specific variables only and then
determine their effect on the dependent variable but generate generalizable findings
6) Case Selection Practices
- qualitative researchers start by selecting the cases where the observed outcome occurs since they
want to explain this outcome and find the causes; they choose positive (confirming) as well as
negative cases to test their theory
- quantitative researchers use random selection to overcome possible bias
7) Weighting Observations
- for qualitative researchers, the theory is always just one critical observation away from being
falsified; a finding that contradicts their initial theory will lead to the researcher claiming his theory to
be erroneous
- for quantitative researchers, all observations weigh equally and a single observation can thus not lead
to a rejection of the overall theory
- causal- process observation (qualitative) is helpful when one wants to explain a specific outcome in a
particular case
- data-set observation (quantitative) can be used when one wants to generalize findings for a large
population
8) Substantively Important Cases
- for the quantitative researcher, all cases are assumed being equal
- for the qualitative researcher, some cases are more important than others; some cases are seen as
more important due to some past findings or due to the variables role in domestic or international
politics (when a theory, for example, did not hold in some cases in the past, the researcher is less likely
to employ this theory, although it might work for other cases)
9) Lack of Fit
- as the qualitative researcher knows much about the different cases under investigation, he will be
explicitly concerned with cases where the causal model does not work and will seek to explain why
this case does not fit into the model
- for the quantitative researcher, a case that does not fit into the model can be ignored, as long as
enough other cases confirm his theory; he assumes that the lack of fit might be due to randomness or a
measure error
10) Concepts and Measurement
- Qualitative researchers spend much time on clearly defining their applied concepts to ensure validity
- Quantitative researcher focuses rather on the operationalization and use of indicators instead of
defining their concepts

Tutorial 2 Readings
Berg The Nature of Case Studies
- Case study as a detailed examination of one setting, or a single subject, a single depository of
documents, or one particular event
- Systematic gathering of sufficient information on a particular person, social setting, event or
group to allow the researcher to understand how the subject operates or functions
- Can be focused or broad
- Embedded case study: looking at one case study but including several levels or units of
analysis
Theory and Case Studies
- Case studies can be helpful in formulating a theory, as well as testing it
- Theory formulation prior to the case study can be helpful, since it allows the researcher to
thoughtfully choose his cases to be studied, help stimulate rival theories and support
generalizations
 - Case studies can also be used to generate theory; the case study can lead to the formulation of
a new theory and opens the researchers eyes to new perspectives; there will be a close
connection between theory and data; the theory will thus closely mirror reality
The individual case study
- Using interview data consider interviewing not only the person subject to the case study, but
also people present in his life
The researcher should have an inquiring mind, constantly challenge his findings and be
opened; be able to listen and include observation and sensing; adaptability and flexibility
are also crucial; have an understanding of the issues being studied; be able to interpret the
data unbiased
- The use of personal documents any written record created by the subject that concerned his
or her experiences (diaries, journals, letters, memos, autobiographies); also use of photos or
video material
Intrinsic, instrumental and collective case studies
- Intrinsic case studies undertaken when the researcher wants to understand a particular case;
the researcher does not want to generalize his findings but understand a case that stands out
due to its uniqueness
- Instrumental case studies provide insight into an issue or refine a theoretical explanation, to
make it more generalizable. The case becomes secondary, what is more important is a
theoretical issue that is observed on the case
- Collective case studies extensive study of several instrumental cases to allow better
understanding of an issue in a broader context
Case study design types
- Exploratory the data collection may be undertaken before defining the research question.
Discover theory through observation; helpful as a pilot before beginning a broader field study
- Explanatory useful when conducting causal studies; use of multivariate cases to examine
patterns
- Descriptive the researcher formulates a descriptive theory as the basis for his study. The
analytical framework and ways of interpreting the findings are defined prior to the case study
Kinds of case studies
- Snapshot case studies detailed, objective case studies at one point of time
- Longitudinal case studies quantitative or qualitative studies of one research entity at
multiple time points
- Pre-post case studies study of a puzzle at two time points which are separated by a decisive
event
- Patchwork case studies set of multiple case studies of the same research aspect, using
snapshot, longitudinal and pre-post designs
- Comparative case studies a set of multiple case studies of multiple research entities for the
purpose of comparison

Intelligent Research Design Bob Hancke

Constructing Case Studies and Comparisons
- Data, cases and arguments are closely related
- Case studies offer detailed insight into mechanisms, motives of actors, and constraints they
 face at particular moments
- One single case study is not enough to generalize a theory, as you need to proof that your
theory holds in other cases as well
- Case studies are thus not made for generalizations
A Case of What?
- a single instance of a decision, policy, institution, event, process, in which the case is
differentiated from other cases by having a single value in terms of either the outcome that
you are interested in or of the explanation that you are exploring
- It must be limited in time and space and allow you to say something meaningful beyond the
case in question
- The case needs meaningful boundaries
- Point out why and how your case studied is different and similar to other potential cases
- Process-tracing case studies start off with a minimalist causal argument (A causes B) and
then develop an understanding of the deeper mechanisms
Need to weigh the different theories in light of the case evidence
Critical Case studies: challenging a theory
- Powerful case study
- Need for a theory or argument at a high level of specification, which says that under a set of
reasonably well-defined conditions, A causes B and, secondly, need for a case that conforms in
every relevant instance to these conditions that the theory has specified, but has a different
outcome than predicted
- A critical case study forces you to think through carefully what the strongest version of the
opposite argument might be, built that into the way you engage empirical material and then
show it is problematic
- Main function is to disprove a theory; need to re-specify the theory and explain what your
findings add and change
Comparative research
- Requires the researcher to choose two cases that will allow for a conclusive answer to the
question he is asking
- Best to start with two cases which represent a counter-intuitive outcome, i.e. developed
differently despite same starting point
- Method of difference you select two cases that are similar in every relevant characteristic
except for the outcome and for the aspect that you think led to the differentiated outcome
- Method of agreement everything between the cases is different, except for the explanation
and the outcome
- As cases often do not differ or are the same in only those two aspects, you can take another
case and then make smaller assumptions
A shadow case offers you the variation that you are interested in only on the dimension
that is unclear from the initial two cases

Jack S. Levy Case Studies: Types, Designs, and Logics of Inference

What is a case study?
- A case is defined as an instance of a class of events and the case study then is the detailed
 examination of an aspect of a historical episode to develop or test historical explanations that
may be generalizable to other events
- The aim of a case study is then to generate as many testable implications of ones hypothesis
as possible in a given case
Typology of Case Studies
- Idiographic case studies, which aim to describe, explain or interpret a particular case
Inductive case studies: highly descriptive and lacking a theoretical framework; the
researcher assumes that everything is somehow connected and thus wants to observe all
aspects of the case
Theory-guided case studies: well-structured by an analytical framework that guides the
research, thus focusing on some aspects while ignoring others
- Hypothesis generating case studies
Aim to generalize beyond the data; observation of the case for the purpose of developing
theoretical propositions that will then be tested through other methods
This form of case study can seldom lead to the formulation of a whole new theory, but
rather aims on explaining why a particular theory does not hold for a case and thus refine
the theory
It helps detecting causal mechanisms; the researcher learns more about intentions and
individual decision making and can thus detect the causal mechanisms that link different
developments
- Hypothesis testing case studies
- Plausibility probes, an intermediary between hypothesis generating and testing, involving
illustrative case studies
Comparable to a pilot study that allows the researcher to sharpen his hypothesis or to test
his theory before engaging in costly research study
Varieties of case study research designs
- Question how many cases to select for the case study; if the population we want to test the
hypothesis on is small, we should not choose too many cases for the initial case study as not
too many cases will be left to test our hypothesis
- Selection bias in small N-research, random case selection will often generate serious
biases, wherefore theory-guided selection of non-random cases is required
This again can lead to selection bias
Problem of selecting on the dependent variable; researchers thus also incorporate negative
cases that do not fit into the theory
Selection bias might also occur when the researcher relies on secondary sources from
other historians, who do not produce theoretically neutral analyses but employ their own
theories thus the need to test the sources against other interpretations
- Comparable-case research designs
Employ the method of agreement (use cases that are similar on the DV and different on all
but one IV) and method of difference (similar on most IV but different on the DV)
- Process tracing - causal process observation to provide insight into cause and effect
- Crucial case designs based on most-likely and least likely designs can be useful for testing
certain types of theoretical arguments
Assume that some cases are more important than others for testing the theory
 A small number or a single case can be important for testing our hypothesis
- Deviant case studies explain why the case deviates from the theoretical expectations

Qualitative Research Strategies Readings

Tutorial 3

Daniel N. Posner The Political Salience of cultural Difference: Why Chewas and Tumbukas are
Allies in Zambia and Adversaries in Malawi

- A researcher can study many different aspects among people, such as the food they eat,
clothes they wear but only a tiny fraction of those aspects are deemed crucial by the people
for defining themselves
- For every cultural cleavage that serves as a basis of political division there are numerous
others that have no political import at all
- The political importance of a cleavage depends on the size of the groups that it defines relative
to the size of the arena in which political competition is taking place
- Mere focus of the author on cultural demography, i.e. the sizes of the groups that the cleavage
defines in relation to the political and social arenas in which they operate
- The author observes how the relations between the two African groups, divided by a border,
varies as a consequence of the different environments in which those relations take place
- The two African groups can be found on each side of the border dividing Malawi and Zambia
and their relations are studied by the author who detects that the different settings impact if the
cleavages between the two groups are politically salient

Chewa Tumbuka Relations in Zambia and Malawi

- The author chose villages representing both groups that were similar in both countries, the
only difference being that the one villages were in Zambia and the others in Malawia
- Open-ended survey questions were used in all the villages to ensure that the cleavage was not
a product of objective differences between the groups on each side of the border
- The divisions between the two groups run much deeper in Malawi than in Zambia;
respondents to the survey there were less likely to vote for a president from the other group or
intermarry with the other group

Why Chewas and Tumbukas are allies in Zambia and adversaries in Malawi

- One explanation is that both countries had different missionary groups which put a different
emphasis on education and overcoming differences disproved
- Another explanation does not focus on the amount of missionary education that the Chewas or
Tumbukas received but stipulates that the notion of Tumbunaness was a creation of a small
group of intellectuals and missionaries who wanted to unify the Tumbuka under a common
ethnic banner
This only explains the cleavage between the two groups in one country, but not on the
other side of the border
- Third explanation is that ethnic relations per se are different in the two countries
Presidents in the two countries chose to divide the countries along different ethnic lines
and to different degrees
Group size and cleavage salience

- Zambias land area is almost eight times larger than Malawis and the size of the Chewa and
Tumbuka communities relative to their country as a whole differs
Chewa and Tumbuka in Malawi account for 28% and 12% of the national population, but
only for 7% and 4% in Zambia
- In Malawi, both communities are large enough to constitute political coalitions in the
competition over national power; politicians seek access to one of the groups for support and
thus deepen the cleavage between the two groups
- In Zambia, both groups are too small to be used as political vehicles and their cleavage thus
remains politically unexploited
- President Banda in Malawi mobilized the Chewa as Chewa themselves and exploited the
cultural cleavage to win them as a part of a coalition
- President Kaunda in Zambia won the votes of both Chewa and Tumbuka, as he did not stress
the cultural cleavage; his political counterpart in Malawi, on the other hand, only won the
votes of the Chewa and split the voters into two parties

The Power of Administrative Boundaries

- Another explanation might be that the administrative boundaries prevail over cultural ones and
only coincide with one another in Malawi

Conclusion

- A cultural cleavage itself does not necessarily lead to political or social salience
- Some cultural cleavages matter a lot for political competition and others do not
- Administrative boundaries with no cultural meaning can have the power to create salient
cleavages
- The political and social salience of the cleavage depends on the sizes of the groups that the
cleavage defines relative to the sizes of the political and social arenas in which the groups are
located

The American Political Science Review Graham T. Allison

Conceptual Models and the Cuban Missile Crisis

- A researcher has to single out the relevant and important determinants with which he wants to
explain the event; a researcher cannot take all the potential causes and relations into account
- The Rational Policy Model attempts to understand the event as the more or less purposive act
of a unified national government; it examines how we can explain why the government acted
in the way it did and why its action was reasonable
- The author employs three different models to explain the US blockade during the Cuba
Missile Crisis

Model 1 Rational Policy

- Important to analyse why neither the US nor the Soviet Union actually made use of their
nuclear weapons the question of balance: when there is a clear and stable balance between
the two opponents, striking first by one party will lead to striking back by the other party
 - The author clearly lists the different aspects of her study:
1) Basic unit of analysis: policy as a national choice actions of national governments in
foreign affairs are assumed to be chosen by the actors; governments want to maximize the
strategic goals and objectives
2) Organizing Concepts national actor (government as the agent); the problem (threats
and opportunities make government act); static selection (action a choice between
alternative outcomes); Action as Rational Choice (rational evaluation of the goals and
objectives, options and consequences and choice); dominant inference pattern (the nation
had a reason to choose a specific outcome); general propositions (nations evaluate the
costs and the likelihood of a possible outcome)
- Variants of the rational policy model scholars apply different variations of the basic mode,
for instance observing personal character traits of a leader that might influence decision-
making

Answers to the Research Questions

1) What phenomena are the authors of these papers analysing?

What are the research questions and main arguments?

Allison, P. Conceptual Models and the Cuban Missile Crisis

- Analysis of the US blockade and US decision-making during the Cuba Missile Crisis
- Important to study why neither the US nor the Soviet Union actually made use of their nuclear
weapons
- the principal purpose of this essay is to explore some of the fundamental assumptions and
categories employed by analysists in thinking about problems of governmental behaviour,
especially in foreign and military affairs (2)
- Although studying the US reaction during the Cuba Missile Crisis, the real focus of the study
is to scrutinize the three different models and to evaluate how they can explain the reaction of
the US

Daniel N. Posner The Political Salience of cultural Difference: Why Chewas and Tumbukas are
Allies in Zambia and Adversaries in Malawi

- The author observes how the relations between the two African groups, divided by a border,
varies as a consequence of the different environments in which those relations take place
- The two African groups can be found on each side of the border dividing Malawi and Zambia
and their relations are studied by the author who detects that the different settings impact if the
cleavages between the two groups are politically salient

2) What are the dependent and independent variables?

Allison, P. Conceptual Models and the Cuban Missile Crisis

- Dependent variable: the action of the US in the Cuba Missile Crisis and the reaction of
national governments to foreign crisis in general
- Independent variables: national actors (nation or government following a specific set of
goals); the problem (threats and opportunities have an impact on the nations decision to act);
 the static selection (action as a choice among alternative outcomes); action as rational choice
(goals and objectives, options, consequences, choices)

Daniel N. Posner The Political Salience of cultural Difference: Why Chewas and Tumbukas are
Allies in Zambia and Adversaries in Malawi

- Dependent variable: political cleavage between the Chewas and Tumbukas

- Independent variable: influence/presence and extent of missionaries, ethnic relations in
general in the country, group size, administrative boundaries

3) Which cases are they looking into? Was the rationale for the case selection explained?

Allison, P. Conceptual Models and the Cuban Missile Crisis

- The author focuses on the Cuba Missile Crisis since this constitutes a crisis that could have
ended in a great disaster with many deaths
- That such consequences could follow from the choices and actions of national governments
obliges students of government as well as participants in governance to think hard about these
problems (2)

Daniel N. Posner The Political Salience of cultural Difference: Why Chewas and Tumbukas are
Allies in Zambia and Adversaries in Malawi

- focus on the Chewa and Tumbuka communities in Zambia and Malawi, since they are only
divided by a national border and despite that equal

4) Are the authors using a theoretical framework in the analysis? Be concrete provide
examples from the text.

Allison, P. Conceptual Models and the Cuban Missile Crisis

- The Rational Policy Model attempts to understand the event as the more or less purposive act
of a unified national government; it examines how we can explain why the government acted
in the way it did and why its action was reasonable
- The author employs three different models to explain the US blockade during the Cuba
Missile Crisis
- Use of three different models to explain the action of the US:
1) Rational Policy Model
2) Organizational Process governments are divided into big organizations, each functioning
according to social patterns of behaviour
3) Bureaucratic Politics Paradigm

5) What analytical techniques are the authors using to collect and assess data in the two papers?
a. Is the methodology used clear?

Allison, P. Conceptual Models and the Cuban Missile Crisis

- The approach and methodology used is very unclear. The introduction introduces way too
many concepts and different ideas, wherefore it is not clear what the paper is about and how
 the author goes about her research. She structures the paper according to the three different
models, but never explains this in the beginning. In the course of the paper, it remains unclear
at which point of analysis or model the reader finds himself.

Posner

- Interviews, voting behaviour

b. Is the rationale for the method used discussed?

Allison, P. Conceptual Models and the Cuban Missile Crisis

- The author defines in the beginning that it is important to improve understanding and analysis
of government behaviour

Daniel N. Posner The Political Salience of cultural Difference: Why Chewas and Tumbukas are
Allies in Zambia and Adversaries in Malaw

- the author clearly defines which sources he uses, how and why
- he explicitly defines how he conducted the interviews and which problems he experienced

6) What did you think about the authors argument and analysis? Did you find them persuasive?
If not please list concrete shortcomings of the papers and how you would advise the authors
to correct them.

Allison, P. Conceptual Models and the Cuban Missile Crisis

- The introduction should be rephrased and clearly determine the aim and research question of
the study
- The author remains very vague regarding her own stance and only refers to other scholars

Daniel N. Posner The Political Salience of cultural Difference: Why Chewas and Tumbukas are
Allies in Zambia and Adversaries in Malawi

- The analysis and the argumentation are very interesting

- It is interesting to see the political cleavage from varying perspectives

Qualitative Research Designs Readings Week 4

Process tracing From philosophical roots to best practices

Andrew Bennett and Jeffrey T. Checkel

- Contemporary puzzles require answers that combine social and institutional structure and
context with individual agency and decision-making hypothesized causal mechanisms shall
explain the underlying causal relations
- Process tracing as a solution and good way to detect those underlying causal relations and
mechanisms
 - Process tracing has only recently been elaborated on and scholars began to define the different
features of process tracing
- Originated in cognitive psychology in the 1960s techniques for examining the intermediate
steps in cognitive mental processes to understand better the heuristics through which humans
make decisions
- Can be applied not only at the individual, but also the collective level of decision-making
- Examines different steps in a process to make inferences about hypotheses on how that
process took place and whether and how it generated the outcome observed
- Process-tracing identifies the intervening causal process the causal chain and causal
mechanism between an independent variable and the outcome of the dependent variable
- An intervening variable does not alter or add to the dependent variable but simply helps to
better explain the outcome
- Process tracing examines if the theorized causal mechanisms can explain the specific case
- Case: defined as an instance of a class of events class of events such as democracies,
revolutions, capitalist economies, wars
- Within-case is defined as evidence from within the temporal, spatial, or topical domain
defined as a case
- Process tracing uses causal-process observations (instead of data-set observations) that focus
on processes, mechanisms or context
- Macro-correlation explain historical cases at a high level of generality through
universalistic theories
- Micro-correlation (propagated by Roberts, who sees this as the important approach in process-
tracing) minute tracing of the explanatory narrative to the point where the events to be
explained are microscopic

Philosophy of social science and process tracing

- Causal mechanisms are viewed as unobservable, as we can only make inferences about their
existence
- Authors think that process tracing can also be conducted on the macro level
- Do causal mechanisms incorporate relations of sufficiency, i.e. when do I end my research?
- Causal mechanisms are defined by the authors as: ultimately unobservable physical, social,
or psychological processes through which agents with causal capacities operate, but only in
specific contexts or conditions, to transfer energy, information, or matter to other entities
- Challenge is to develop theories about causal mechanisms in which we can confide

Generalizability and process tracing

- Researchers might test their theory on other cases, or look for until then unobserved and
independent evidence from the case they have been working on

Interpretivism and process tracing

- Interpretivism or constructivism
- Constructivists argue that there are some underlying mechanisms and structures in social life
that are superior to others
- Observe where agents challenge existing social structures or where structures prevent agents
from acting upon or even conceiving of courses of action that are taboo bracketing the
existing structures and thus observing agents reactions to different elements
 - Another view is that language arguably one of the most central of social structures is
inherently ambiguous and open to many interpretations textual analysis as process tracing

Bayesianism and process tracing

- Use evidence to update ones beliefs in the likelihood that alternative explanations are true
- Central idea that some pieces of evidence provide higher inferential powers than others
- Closely related to process tracing
- General agreement with process tracing in key aspects, but Bayesianism does not employ an
inductive theory-generating approach
- Both approaches use evidence to affirm some explanations and reject others
- Both use evidence to probe existing explanations and to challenge competing theories and
hypotheses
- Both warn against being too assure of ones theory or explanation as there are always limits to
observational evidence
- Central to both approaches is the idea that some pieces of evidence are more important and
have higher inferential power than others
- Four tests to observe the value of alternative explanations the probative value of evidence
depends on the degree to which a hypothesis uniquely predicts that evidence, and the degree to
which it is certain in doing so
- Hoop tests involve evidence that is certain, but not unique failing the hoop test disqualifies
an explanation, thus helpful to exclude hypothesis
- Smoking-gun test unique but not certain passing the test affirms an explanation, but
passing a test is not necessary to build confidence in an explanation
- Doubly decisive test use evidence that is unique and certain, but it can miss alternative
explanations
- Straw- in the wind tests provide weak or circumstantial evidence that is neither unique nor
certain. One test is not helpful, but a number of them can increase confidence

Techniques and best practices of process tracing

- Best practice: three aspects that make best practice

- Meta-theoretically research based on a philosophical base that leaves room for related
approaches
- Contextually process tracing uses the pluralism detected and reconstructs causal processes
- Methodologically take alternative causal pathways serious
- Ten steps that are crucial for best practice:
1) Cast the net widely for alternative explanations
- Consider explanations of specialists and experts and question if any important aspects of
social explanations (institutional constraints, material power, social norms) were omitted
2) Be equally tough on the alternative explanations
- Consider evidence that fails to fit the explanation that we are interested in or that fits the
explanation that we are interested in least
3) Consider the potential biases of evidentiary sources
- How to argue against someone who has an instrumental motive to convince observers of his
explanation?
- Consider the context of the evidence why does someone argue something?
- Consider evidence selection bias
4) Take into account whether the case is most or least likely for alternative explanations
 5) Make a justifiable decision on when to start
- A reasonable starting point might be a juncture at which an institution or practice was open to
alternative paths and exogenous influences or actors determined the path to take
- Choose the point where a key actor enters the scene or gains some material, or capacity
6) Be relentless in gathering diverse and relevant evidence, but make a justifiable decision on
when to stop
- Use triangulation
- Think of your financial and time limits and thus limit the scope of your process tracing
- One should stop when repetition occurs
7) Combine process tracing with case comparisons when useful for the research goal and
feasible
8) Be open to inductive insights
9) Use deduction to ask if my explanation is true, what will be the specific process leading
to the outcome?
- Apply different theories to specific aspects of the causal steps you analyse
10) Remember that conclusive process tracing is good, but not all good process tracing is
conclusive

Process tracing on general categories of theories

- Rational choice theories actors make rational decisions to maximize their preferences
Problem: how to identify and set the actors preferences?
How to decide if rational choice led to maximization of preferences
- Cognitive models how to infer revealed beliefs? How do we know if publicly espoused ideas
are same as the private and real ideas of an actor?
- Norm approach needs to show how norms have influenced actors choices and behaviour

Process Tracing Jeffrey T. Checkel

Causal mechanisms and process tracing

- mechanisms operate at an analytical level below that of a more encompassing theory; they
increase the theorys credibility by rendering more fine-grained explanations
- A mechanism is described as a set of hypotheses that could be the explanation for some
social phenomenon, the explanation being in terms of interactions between individuals and
other individuals, or between individuals and some social aggregate
- Process tracing is then to trace the operation of the causal mechanisms at work in a given
situation
- Compatible with other approaches, such as statistical techniques, case studies, content analysis

Process tracing in action: the case of European institutions

- To which extent do European institutions socialize, i.e. promote preference and lead to
identity shifts?

Assessing process tracing: the good, the bad and the ugly

1) Coming to grips with first mover advantages

 Researchers usually have a favoured theory they will apply first ; the process tracing
approach urges them to consider alternatives and constantly make the researcher think
about the different causal relations and the connection between expected result and data
2) Answers: how much data is enough?
3) Helping to bring mechanisms back in and explain the world as it really is and not how it ought
to be
4) Promoting bridge building connects different tool kits, such as rational choice and social
constructivism
5) Negative lesson: proxies are a pain how to capture causal mechanisms of socialization,
such as persuasion or change of mind?
6) Negative: takes a lot of time
7) Bad: just how micro to go? Where is the stopping point?
8) Negative: non-parsimonious theories not generalizable
9) Bad: missing causal complexity simplification and selection or focus on few mechanisms,
while ignoring others
10) Negative: losing the big picture lose sight of the broader context
11) Losing the ethics process tracer is prone to overlook normatively ethical context

Process Tracing and Causal Inference Andrew Bennett

- Process tracing can reveal the direction of a correlation: did x cause y or did y cause x
- If x and y are correlated, is it because x caused y or is there a third variable correlated to both?
- Hoop test: can eliminate alternative hypotheses - it is limited, since it only requires evidence
to jump through a hoop without being further scrutinized was the accused in the state on the
day of the murder?
- Smoking gun test support a given hypothesis but failure to pass such a test does not
eliminate the explanation - absence of a smoking gun in a suspects hands does not mean he
cannot be the murderer
- Straw in the wind test provide neither a sufficient nor a necessary criterion for establishing
or rejecting a hypothesis
- Doubly decisive tests confirm one theory and reject the others
People filmed by a camera during a robbery are seen as the suspects, while all other
people that are not on film are considered un-guilty