http://www.cs.tut.fi/ihte http://www.cs.tut.

fi/ihte

Outline of presentation
• Chapter 5: Collecting the evidence
- 6 sources of evidence
R.K. Yin: Case Study Research - 3 principles of data collection
Design and Methods • Chapter 6: Analyzing case study evidence
- chapters 4&5 - Analytic strategy
- Analytic techniques
- High quality analysis

Heli Väätäjä 25.3.2007

http://www.cs.tut.fi/ihte http://www.cs.tut.fi/ihte

Six sources of evidence Some other sources of evidence

• Most commonly (Yin) used sources • Films
1. Documentation • Photographs
2. Archival records • Videotapes
3. Interviews
• Street ethnography
4. Direct observations
• Diaries, life histories
5. Participant-observation
6. Physical artifacts • Logs
• etc

http://www.cs.tut.fi/ihte http://www.cs.tut.fi/ihte

Documentation as a source of evidence Reasons for use of documents

• Relevant to every case study topic • To corroborate and augment evidence from
• Many types of documentation like other sources
- Letters, memoranda, and other communication - Helpful in verifying the correct spellings and titles or
- Agendas, announcements and meeting minutes & other written names of organizations mentioned
reports of events - Provide other details to confirm info from other
- Administrative docs – proposals, progress reports & other sources
internal records • If contradictory, need to pursue the problem by inquiring
- Formal studies or evaluations of the same ”site” under study further into the topic
- Newspaper clippings & other articles appearing in the mass - Inferences from docs, but only as clues for further
media or in community newsletters investigation
- Web sites, discussion forums, emails etc (by Heli) • Overall value – systematic search for relevant
docs important
 http://www.cs.tut.fi/ihte http://www.cs.tut.fi/ihte

On document usage Archival records

• Criticism on potential overreliance on docs
- Documents do not contain the whole truth
- Written for specific purpose and audience
- Reflection of communication to achieve some
other objectives
• Often computer files and records
- Service records (nr of clients served over period of time)
- Organizational records (charts, budget over period of time)
- Maps and charts (layout)
- Lists of names and other relevant items
- Survey data (census records or data about a ”site”)
- Personal records (diaries, calendars, telephone listings)

- Need to try to identify objectives, so that less • Find out the conditions under which produced and
likely to be misled by documentary evidence accuracy (specific purpose & audience…) for
interpretation
- be critical in interpretation

http://www.cs.tut.fi/ihte http://www.cs.tut.fi/ihte

Interviews Interviews cont.

• Important for case study • Focused interview
• Guided conversations
- Follow your own line of inquiry - Respondent interviewed for a short period of time
- Ask ”conversational”, friendly and non-threatening questions in (e.g. 1 hour)
an unbiased manner
- Why questions to HOW to reduce defensiveness
- Conversational, but follows certain set of questions
derived from a case study protocol
• Ask about facts and opinions
• Also ask insights into certain occurences, and let - Purpose e.g. to confirm certain facts
suggest other persons for interview and other sources of - Carefully worded questions to be able to appear naive
evidence. and allow respondent to provide fresh commentary.
• Respondent -> informant by assisting
• key informants important, avoid overly dependency
- Report contradictory views

http://www.cs.tut.fi/ihte http://www.cs.tut.fi/ihte

Interviews cont. Direct observation

• Structured interview (resempling survey) • Field visit to the site
- Produces quantitative data • Behaviors or environmental conditions
- Follows sampling procedures and instruments available for observation
used in regular surveys and analyzed in • Formal & casual data collection
similar manner • Provides additional information
- Difference is the survey’s role in relation to • To increase reliability, common to have
other evidence – result used for evaluation more than one observer making
• Record if needed, but remember ethics observation
 http://www.cs.tut.fi/ihte http://www.cs.tut.fi/ihte

Physical or cultural artifacts Participant-observation

• Technological device, tool or instrument, • Researcher may assume a role and participate in the
events being studied
work of art, some other physical evidence • Often in anthropological studies
• Collected or observed as part of field visit • Gain access to inaccessible events or groups
• Perceive reality from the viewpoint of someone inside
• Ability to manipulate minor events
• Problems with
- being external observer,
- becoming supporter of group,
- taking too much attention from observing,
- being at the right place at the right time

http://www.cs.tut.fi/ihte http://www.cs.tut.fi/ihte

Source of evidence Strengths Weaknesses Source of evidence Strengths Weaknesses

Documentation - stable – can be -Retrievability – can Archival Records -Same as for -Same as for
reviewed repeatedly be low documentation documentation
- unobtrusive – not - biased selectivity, if - precise and - accessibility due to
created as a result of collection is quantitave privacy reasons
the case study incomplete
-Exact – contains -Reporting bias – Interviews -Targeted – focuses -Bias due to poorly
exact names, reflects (unknown) directly on case study constructed questions
references, and details bias of author topic - response bias
of an event -Access – may be - insightful – provides -Inaccurate due to
- broad coverage – deliberately blocked perceived causal poor recall
long span of time, inferences - reflexivity –
many events, and interviewee gives what
many settings interviewer wants to
hear

http://www.cs.tut.fi/ihte http://www.cs.tut.fi/ihte

Source of evidence Strengths Weaknesses

Direct observations -Reality – covers events
in real time
-Time-consuming
- selectivity – unless
Summary of sources of evidence
- contextual – covers broad coverage
context of event -Reflexivity – event may • Procedures for collecting each type of
proceed differently
because it is being evidence need to be developed and
observed
- cost - hours needed by
mastered independently
human observers
• Choose relevant sources
Parcipant observation -Same as above -Same as above
- insightful into - bias due to • Need to be able to use all types or have
interpersonal behaviour investigator’s
and motives manipulation of events colleagues that master ones you don’t
Physical artefacts -Insightful into cultural -Selectivity
features - availability
-Insightful into technical
operations
 http://www.cs.tut.fi/ihte http://www.cs.tut.fi/ihte

Convergence and nonconvergence of

Three principles of data collection multiple sources of evidence
Archical records
• Principle 1: Use multiple sources of evidence
Documents Open-ended
- Strength interviews
Fact
- Triangulation in doing evaluations Observations Focus interviews
1. Of data sources (data triangulation) (direct and participant)
2. Among different evaluators (investigator triang.) Convergence of evidence
Structured interviews
3. Of perpectives to the same data set (theory triangulation) (single study) & surveys
4. Of methods (methodological triangulation)
Site visits findings conclusions
- Construct validity (multiple sources – multiple
measures of same phenomenon) survey findings conclusions
Nonconvergence of
Documents findings conclusions Evidence (separate substudies)
Analysis

http://www.cs.tut.fi/ihte http://www.cs.tut.fi/ihte

Principle 2: Create a case study database Principle 3: Maintain a chain of evidence

• Data or evidentiary base • To increase reliability of information
• Report of the investigator, whether article, report • External observer (reader of case study)
or book form allowed to follow the derivation of any
• Formal, presentable database for other evidence (from initial research questions
investigators to review (increases reliability)
to conclusions)
- Case study notes
- Case study documents • Presented evidence should be the same
- Tabular materials as the collected
- Narratives – open-ended answers to the questions in • No original evidence should be lost
the case study protocol

http://www.cs.tut.fi/ihte http://www.cs.tut.fi/ihte

Summary on three principles of data

Maintaining a chain of evidence collection
Case study report
• Data collection for case study more
Case study database complex than for other types of study
• To ensure quality control during collection,
Citations to specific evidentiary sources
in the case study database formal procedures need to be followed
-> 3 principles
Case study protocol • To ensure that final results reflect a
(linking questions to protocol topics)
concern for construct validity and for
reliability to be worthy of further analysis
Case study questions
 http://www.cs.tut.fi/ihte http://www.cs.tut.fi/ihte

Analyzing case study evidence (ch. 5) Three general analytic strategies

• Analysis consists of 1. Relying on theoretical propositions
- Examining
- Categorizing 2. Thinking about rival explanations
- Tabulating
3. Developing a case description
- Testing
- Otherwise recombining
Of quantitative and qualitative evidence to address
initial propositions
• Strategies and techniques have not been well
defined for case study
• Define priorities for what to analyze and why

http://www.cs.tut.fi/ihte http://www.cs.tut.fi/ihte

1. Relying on theoretical propositions 2. Thinking about rival explanations

• Original objectives and design of case
study based on propositions which reflect
• Research questions
• Reviews of literature
• New hypotheses or propositions
• Focus attention on certain data
• Helps to organize the study and define
alternative explanations
• Define and test rival explanations
• Craft rivals
- The null hypothesis, threats to validity,
investigator bias
• Real life rivals – think in advance and
during study
• The more rivals analysis addresses and
rejects, more confidence on findings

http://www.cs.tut.fi/ihte http://www.cs.tut.fi/ihte

3. Developing a case description Analytic techniques

• Develop descriptive framework 1. Pattern Matching
• Use when difficulty making either of 2. Explanation building
previous work 3. Time-series analysis
• Especially for descriptive studies 4. Logic models
• May help to identify causal links to be 5. Cross-case synthesis
analyzed
 http://www.cs.tut.fi/ihte http://www.cs.tut.fi/ihte

1. Pattern matching 2. Explanation building

• Compare an empirically based pattern with • Special type of pattern matching
a predicted one (or several) • Goal to analyze data by building an explanation about
the case
• If matches – strengthens internal validity • For explanatory studies
• If explanatory study, patterns may be • Stipulate a presumed set of causal links of phenomenon
related to dependent or independent • Iterative process
variables of the study • To avoid drifting away
- refer constantly to original purpose of inquiry
• If descriptive study, relevant if predicted - Use case study protocol (what data to be collected)
pattern defined prior to data collection - Establisha case study database for each case
- Follow a chain of evidence

http://www.cs.tut.fi/ihte http://www.cs.tut.fi/ihte

3. Time-series analysis 4. Logic models

• Lays foundation for conclusions • Stiputates a complex chain of events over time
• Simple time series • Events staged in cause-effect-cause-effect patterns
- single dependent/undependent variable
- Match between trend of points compared to
• Match empirically observed events to theoretically
• Theoretically significant trend specified before onset of predicted events
investigations • Four types of logic models:
• Some rival trend, specified earlier
• Any other trend based on some artifact or threat to internal validity
- Individual level logic model (individual person)
• Complex time series - Firm or organizational-level logic model
- Trends more complex or multiple set of variables - Alternative configuration for an organizational-level logic model
- Strong evidence when predicted matches actual time-series - Program-level logic model
• Chronologies – descriptive and analytic technique • Define logic model prior to collecting data and ”test” how
• How and why questions about the relationship of events well collected data supports it

http://www.cs.tut.fi/ihte http://www.cs.tut.fi/ihte

5. Cross-case synthesis Four principles of high-quality analysis

• Specifically for multiple cases (at least 2) 1. Analysis shows that you attended to all the
evidence
• Each case study treated as separate study
- Exhaustive set of analytic strategies including rival
• Aggregate findings across a series of hypotheses
individual studies - As much relevant evidence as possible, no loose
ends
• Analogous to cross-experiment - Otherwise vulnerable to alternative interpretations
interpretations
2. Address all major rival interpretations
- Make alternative explanations into rivals, evidence?
 http://www.cs.tut.fi/ihte http://www.cs.tut.fi/ihte

Four principles of high-quality analysis cont. Summary of analyzing case study evidence

3. Address most significant aspect of your
case study
- demonstrates analytic skill
4. Use your own prior, expert knowledge
- demonstrate awareness of current
thinking and discourse about the case
study topic
• Potential analytic difficulties can be reduced if
you have a general strategy for analyzing results
• Within general strategy, use analytic techniques
• No cookbooks availabe for analysis – most
difficult stage of case study
• Begin with simple and straightforward case
studies to gain experience (two-case design)

http://www.cs.tut.fi/ihte

More info
• Eisenhardt, K.M. (1989) Building Theories from
Case Study Research, Academy of
Management Review, Vol. 14, No. 4, 532-550
• Järvinen, P. (2004) On Research Methods.
Tampere: Opinpajan kirja
• Layman, L. et al. (2004) Motivations and
Measurements in an Agile Case Study, Workhop
QUTE-SWAP, Nov. 5, 2004, Newport Beach,
CA, USA.
• Yin, R.K (2003) Case Study Research, Design
and Methods, Newburu Park, Sage.