/$zrtc hnr ftwrvr1 nrrv nfl!?r, hr FI ~ r n ~ nw r r!r o n h nlt,rltodr Irr ~ h o r ~ k r r / ,, cr,*ot , r ~ rh~nrt ~ t t ~ r~ ~ t r~ z i i i - r q < z r { t 3 f tI!! t t $ ~ c \ ~, n i / t ~ d t r ~ g unwrt(i . T/w ~ ~ ~ r !

~ ~ ~ /,r to t r < / t n : th,,07ff,q. t . f , i l ~ o t i o tlnr<l ~!, O ~ ~ ~ I ~ ,, % U. F F III\I,. ~ I ~ I ~ I~ I I -~ o ~ l ~ l ( J 4 . rt n nrti7 ff ~ ~rf ~ rq l~ t t. d r f t - . r ~l~~I~r r >~ ~v-rr~ H/ ~ . r f~ i !~,di nnd v ~tJ!r~r t ~ rn~trrrrrl>r~ ~t~tlJ lrrrne l / l p I / ! # ~ T ~ , I ~

nnru;ddc

0
Vr
Y

-find tlrrr rrn P I rrr,mr.ly ~rnttmhle. IPPII~ ~ ' r r r f i -borlk. rt

P R ~ ) F F C , ~A ~t R ks
RR\HNt.

I . N l v F R ~ l T Y <)I L r l l l C 1 t f f O R O i ' I ; H

n ~ r ~ n rD d~ ?r ; g r ~n ~ S~~rraI Rr~~rflrrJr IS onr-stop rritlral guide that d~monstratrs the impc~rtanw r,F attend in^ to design issr~csIn sorial r ~ ~ e a r r h .
CI

&
H

Yesearch Design i n Social Research

A

thlr Fll'ld p r i c > r ~ t clnfa ~ ~ rrollrctinn strat rm and statistical W l ~ ~ rn e tal l ~~ t~ooks r analysis, de Vaus arjiur4 that, a- s ~ g n l f i r a n t as t h r w Isslirs arv, rnost p p r t l n ~ n nf t all ro r ~ ~ r ' n r rtl l~ r h l i l q 15 ~ tthe ~ ~ S I T U C ~ ~ Iand T ~ drsign of thr rewarrh.

David de Vaus

1'111~ bnnk:

* Prvirles t h t~~ a d sit ~ r11 an ur~rl~r~rarirI~~i~ of the irn1)orranrr nf' rcsparrh dr+i am1 it< place in tire researcll pmww
Drscrtbes. thp maln r ) ~ of s r~s~arr rlrstgt15 h 1 1 1 socinl rrwarrh
Erplarns thp Ioglr a n d
IIurpSeF

o f d e ~ to ~ canatde p students to rvatuatr

equip^ studrnu

wil h

t h clr~ipzi ~ sk ~ l tol rjprrdtr ~ in rral wnrlrl

Divided into 5 partc. part I rvplorrs iqstrrq ;lro~lndr ~ s ~ a r tvpps, c l ~ ronwpts;. qurst inns, t , a u s ~ l ~ t arld y tlas~c rssirps ln t ht' ~ I P S I K I I ~ ~ O C T S S T. h v terna~nina 4 perL.i focus on diff~rrnttypes of rrsearch design. Exprrmental, lon+tr~dinal,rroas. m t o n a l and rw.stutly rnetllodz arr clearly artd qstrrnat~callu r~amtnprl, at~d thrir strengths ant! wraknm.~snre descsibrd.
'This bonk will b~ invaluablr to hoth sturlerl~s arid r c s ~ a r c l l ~ r s il~forming , and p i l l : Ing t hrm tawarrlq s ~ ~ r ~ ~ swt-1~1 s f u l rc~r*;lrc11.

Cnrrr d ~ < r c and n rl!r~rmrrronhv.4, rrl1,nr Rtrr

I S B N 0 - 7 6 1 9-5346-9

frE Pulblications
r,onnon
J

troricatrd Oak$

~ P I I L ~

T i n t puhlr.;hril ?t?(!l

Apart h r n a n v fair dralrng for lhr purpowq of research or yrr\ ate ~tn<l\., o r crihcrzrn or rc*vit.trv. a5 pernii tted under the Cvpvrigh'ht, I\-~psand rnientc Act, 19W. 1ht5 publicatlvn mav be reproduced, slnrc~1 or tr~~nsniltted In any fom, or try anv means. cmlv rvlth t11(* prior pcsrrnlrs!r)n in wrltln): r ~ the t yuhllqherc. or In the l a s t k 1 1 f reproqraphlc rrpnxluctbnn, in arctlrdilncr w ~ t h the fchrmsuf I l c e r ~ c e L~ S ~ L I C L ! bv the Copk right I ~ct+~lsing A ~ r n c y Inquirres . cmrernlng r c ~ ~ ~ r n d u c otltclde t ~ c n ~ thew tprrnz 4inuld ht, cent to the> pul7lr~hers
SAGI' I'ublicati[>ns Ltd h nnnhill S t r r ~ t

CONTENTS

List
List

uf F ~ , ~ I I T P P

xi xiv xvi

trf Totdrs

PWfnct'

h(>ndon FCZ.1 .FI1U

PART I : WHAT IS RESEARCI-E DESIGN?

1

T h e Context of Design
Description and ekplanation Uescripfive research Explanatory resen rch Thcory tsting and theoq constructinn Theory bullding Theory testing What is rrsearch d r z i p ? Desigm versus method Adopting a sceptical approach to cxplana tions Summary

SACX I'ublications l n d i , ~ IXvt I td 32, C1-Rlnck Market Greatc~r IGilfazh - 1 New Tlelhr 1 1 0 048

Eritirih Library Cataloguing in Publication data

A catillopir r c ~ o r d for I ! i ~ s bonk from the Brltlql~ Library
1%

availahlr

TSRN O 7619 5?46 9 ISHN 11 7619 53.17 7 (pbkj

Lihran, o f Congress catalog record available
Tvprrr3t by M;avht=wTvyrscthnq, Rhavades. I'owr,.i Pr~ntc~tl In C;t*j~t Rrrtaln I,\ rltddl~s Z td, GtrLl,ltilrd, Surrr-l

Tools for Research Design Before design Focusing and clarifying the research question Identifying plausible rival hypothcqes Operationalization Concepts for research design Internal validity External validity Measurement error Summary
Cawsation and the Logic of Reqearch Design Inferring causal relrlt;ionshipq criteria For infprring cause
T,.v~v ,,fF
,,.-I

=3

7+*,.l-n7

Cornyar~ng tirnc points hlaLrn7 m c ; l n i n ~ f rcomparisons ~l Int~rvcntinnl: a n d indepcntlent variables Dirnrn.;iclns crf a rrc;c.lrch ~ i c - 1 ~ ~ A wngc of rrsearch designs
ExpcfirnrntaI cle.;ip I,~>nqih!d inaE dec;r~n

Summary

Notes

Crc~s-~ l(mal c t dv\ien

C w t stud~rs Summary Vrltr,c

P A R T 11: EXPERIMENTAL DESIGNS
4

Tvppg of Experimental Design TIw classic c ~ \ p ~ r i m c n t dt*sign al 3:upcrirnr~tlal ctlntcuts Simplct c\prr~rncnt~iE de~igns I ' o ~ t - t ~ t only rt w12hcontrol grnLlp
I<t~lrusyci-l ilme <>ip~rirn l,ll i ~design ~i \.lnrc, crmlpll-. expt>rrmcnt,~l Jei;iqns

Analysing Experimental Data 5clcctin~ the mcthod of ~ r ~ a l v s i s \Vhat kind of annlvsis is rcqlti rcd: dcscriptinn o r infcrrnc~? W h a t w r t of .;ample do we hL3vc7 \'\'hot Icvel of data d o w e ha\.c' Typc of cornp,lrisons rt7quired' Mrhnt type of displav i .: reyt~irtd? f-lorv m a n y )!,mItFS art? to bc cornpasrd7 Arc thc c ~ m p ~ i r i s o groups n indrpmdcnt samples? 1 s thc dcpendcnt vnr~~lblc nnrmalIy distributtgd? Is thc vnrianct nn thc rtependcnt variable similar b~tween each comparison grc~up? How lnanv indtpenctmt vari,lbles? Summ,lry Froce\s for selecting the right rneastlre
(%her
~ S S L S ~ ~ S

Changr scorc3.:

hluLt1plc p,st-tt.5t.. MuItipEc g r o u p 50lorn4m tour-croup d e ~ i c n Factnrial ~lcsigns

Trunc,.ilim eff~*cts (flrwr ~ n i ! ci*ilinq cfftctl;) Trend analysis Summary

Nntc

Summary
Volr\ 5

PART 111: LONGITUDINAL D E S I G N S

Issues in Experimental Design Methndological i.;<~t*.: The prnI7lem ot rxplanatorv narrownesq l'mblems with i~itrmmlvalidity I'n)bIclnq 12-lth r%ulcrnal validi?
I'mcticaI i ~ s u c ? I-low rnz~cll dn yi~u tell participnnts? I-lori. mnnv past icipantq' l - l c l w 5holtl~i partic~pantqI-r r~cruitvd? i ;aps b r t ~ r e c n tcyst.: and intcrvcntir~ns Il'llicll nirthc~d(tf Jat,? ccnllcthun7 I'rthlems x\-ith r,~ndorn imcl a s q ~ r i r n e n t Unrveneic~~.; of I ntcrvent ions T'ht wlf-fulEiIlinr: prnphec~. Ethical i s s i ~ w Vol untarv pdrticip,~ tion I n f n m r d ronstlnt Nn harm t n pnrtlcipant5
,+inon\,~~iitx~ ,In1f r - ~ ) l ~ f i ( i r ~ ~ ~ f i l l i t ~ *

Types of Longitudinal Design I'urp~~ oft lon~itudinnl ~ rlesi~n Dwcrihing patterns of change and stabilitl: EstnLlli\hing trmpow8 order Estatll ishing develr~pmcntal( ~ g ecffccts ) Est;tbli s h i n ~ historical (period ) effcct.: I , i k course 'career' analysis Tvp.p.~s crf longitl~dinal rl~qip
Procptbsti~~ pane! dtwqn.; Retro.;ppcti\ c designs Qunki-longitt~dina I designs S~trnn1.1~1 knte

Iswes in Longidutinal D e s i p

Mctho~iolog~cal Iqsueq Is.;ur\ n t ~ntrrnalvalidih/ 1$4111'< of eutt>rnal vnlrdie Practical Issue.; <I ,,,, I ,,.,4;.,I;,,., , , I ; + * ,

b>..,n>lm>#7!,

I'n~elattrition R~spondc.11 t Zlurden Rcspunrlpn t rr.cal l Cnst Method uf data collection hTumhernf iravcs Gap betwtcn waves S.irnple error Sample s i m In~trumenb design
5taifin~

Practical is.;uc.; Method clf collecting data Samplc si?es Sufficient variation in the sample on kc! variables Infnrmation for statistical controls Length Types of data Ethical imurs Summa? Y otes
32

I;thzcal issue< Voluntary participation Informed conscnt Harm to participants Cnnfidentialih. and anonvmity Summary
9

Data Analysis in Longitudinal Design Misqinq data hwtrccs of missing data ~ ~ 'a 1 ~TX&Il'fil' 1 ~ Iiicntifying miwing data bias Ikaling with miwing data M~asuring ch~n~c A ~ ~ r e g a Icvcl te versus inctividual level change Qual~tativr v c n u s quantitative change Measuring change in panel d c s i p s Dclscrihjng c h a n g ~ Tablcs Graphical Summary
Nrlti.

Cross-Sectinnal Analysis Descriptive nnaly sis HOW ~ J R V ? Lei~el nf iictnil Form of data Who? Factor structures and scale structures Haw ~ t . n r r a l STON l CIOSC~ Explanatory a n a I y i s The locic ot stahstical cimtrels Multiplc statistical cnntmk The elabnration techniquc Basic approach
Interpsrtation Problems with the elaboration model Multivariablr analysis Cohort analysis Constructing and reading cohort tablr.: Problms with cohort analysis
Summary

PART V: CASE S m Y DESIGNS

PART IV: CROSS-SECTIONAL DESIGNS

10 Cross-Sectional D e q i p \ ( r time dimcn\ir>n Rrliancc on existing diffewnces Thl. nature nf ' ~ r n ~ r pin s ' thc cross-sectional i i ~ s i g n Obtaining a timc cl imcnsion: rc*pcnted csnsq-sclct~onal s h d ips
II

23 Case Study Design \*at is a caw' Units of ilna!v~is Moliqtic ar~d embedded units of analvcis Case studies and theory
Explanatory case studirh Descriptirtc case 5tud1c.; Other elrments of case qtudv design< Singlc nr multiple case\' Partlllcl nr sequential? Rctrosprctive or prospcctivc? Types of caw study d e s i p s

Issues E n Cmss-Sectional Design h.lrtlit~dologica l wues Intrrnal validity External valtcli t i )

CONTENTS

W i a t a case study is not Not the one-shot case shtd y Not a data collection n~ethod Summary Note
14

Issues in Case Study Design Methodological issues Internal validity External validity Practical issues Sampling

LIST OF FIGURES

Number of investigators: getting consistency When to go in to the field Prescnhng case studies Ethical issues

Summary
Notes
15

Case Study Analysis Statrstical analysis Meaning and context Analysis in descriptive case studies Theoretical dimension of descriptive analysis Explanatory case studies Theory testing analysis Literal and thcnreticaI replication Analysis for theory building: a n a l y ~ c induction Summary Note

Three types of causal relationships Theory building and theory testing approaches to research The relationship between divorce and parental conflict The logic of the research process Kelaticmship between research dcsign and particular data collection methods Causal and non-causal explanations of the relationship betwt.cn school type and acadcnlic achievement An intervening variable An extraneous variable Starching for causes Searching for effects A specific causal proposition A more complex causaI model for increases in divnrce Dimensionalizing the concept of chiId's wellbeing A spurious relationship An indirect causal relationship More complex indirect causal relationships Types of relationships between three variables Propositions and observations without a comparative frame of reference Using propositions with a n explicit comparative frame of reference Multiplc comparison groups m a t e d by combining two independent variables CIassic experimental design Simple longitudinal design Simple cross-sectional design Simple causal proposition Structure of the classic experimental design Classic experimental design: effect of difficulty of joining a group by perceived desirability of group membership Post-test only experimental design with a control group ~ e t r o s ~ e c t i experimental r~b design

1.157 0 1 ' FIGURES

~ 1 1 1

Expc~.imcntal dcsign wrtf? multiple grtnlps and one p r r - t ~ sand t one post-ti'st Snlnmon four-group drsign I I\,pnthptical e x a m ~ l c o f thc %I@rnnn four-group d e s i p Dtt~ning groups in a 1 2 factorial dc-ip Elrmcnts nf a 2 x 2 f,rrtnrial design UFrm-t and indirect i ~ f k ~ cof t s gender on inctmrc Pri,blt~mswith comparing averages Graphical teprewnt.itionq nf productivitv changes bv

Cau.;at model of indirect causa t rela tionship bet-ween ~cndcr and hcomr MocEcl indicating indirect rrla tionshp h e t w c ~ n school t p t * ~ n student d re!igiousness Thcorcticallv derivd typology nf deviants Variatinns of case 5tudy dcsrqs h c - s h o t case study d e s i ~ n ( m e group, pmt-test onlv

design}
Pattcrn matching for two variables each with two categories Pa ttcrnq for two variables each with 'three categories Pa ttcrn matchine for parental sty Ie and child b~haviour Logical patterns with two independent variables and one

Summ~ry ~tatistics:productivity chanac bv bonus sclicrnc Srlccting statistical methods of analysis, part 1 Scli.cting statisticai mr~hods of analysis, part 2 Sclrcting statistical nlcthcds OF analysis, part 3 Sc,lccting statistical rncthods of analysi~, part 4 k r n c nutcomes of an c~perirnentalvnrinblc on a time
writ,+

dependvnt variable L O R I Cppatterns ~ with one independent variable and five dcpcndcnt variables Prrd ictcd patterns d ou tcomcs on five dependent varlablrs by level of worker autonomy

MultipIc prospcctrve p i e l de5ih.n Sin~lrnpanel dtsign withnut replacemtant Single panel desikn with replacement Rotating pnnel d e s i p C'r>liort s~quentialpancl design Sirnplc retrospective pant4 design Multiple point retrwpcctiue panel design Simtrlnled before-afttsr rlccip Crudc divorce rate, At~stral~o, 1970-95 Rmplo~mentrate of pcwple aged 55-59 by gender, .Ai~qtmlia,1975-201 0 act~laland predicted Emplovmcnt rate i y f people aged 55-59 by gender, huqtrafia, 11975-2010 actual and predicted: (a) flattened S C J ~ P .( h ) c x p a n d ~ d scalc Mali* su~cide deaths and unemployment rate ratios, 20-24 vclas old^, Australia, l Vhh-sO Fu-nuptial births and scrious crime rate, Australia, 1Y l ) i - L ) l
Cro--.-.cctional design: mnr~ tai happinc5r; bv parenthood status G T T H Iin ~S cross-scct~nnnldesigns Rcpcntcd cross-sectional design 7 hrce causal models ot child's academic acliiev~plent A \~.~riabIe bv case matrix

LIST OF TABLES

mfinin~ groups in a 3 x 2 factorial design Dctvrnlin~ng the IcvcE of measurement of varia hlcs Pr~n111ctiv~ty changv according tn incentive schernc Mcan percentage c h a n ~ in c productivity by bonus scheme Age cohorts over t i m ~ Development effects: changes in p d i tical conserva tkrn from 1 L W l to 2000 by aRe cohort P e r i d r~ftcctq: changr.: In politic~l cnn.;en.atisrn from 1980 to c,;l(l[ril aKc cohort Creattng groups from panel varintinn Method.; of analysis I'cw panel data Wav.; o f examining aggregate change over twr, waves Adlusting for inflaticrn Chanqr~ I n :-score< of individrtals Chan~c.:measured bv changes in rank Mean :-score for mcn and women, 1490-2000 Femalr carnings a s a proportion of male earnings Changrs in proportions: bottom income quartilc as a proportion of thc top income quartilc Evamplrs of t n b u l ~ r wavs of prcsmting data about aggrqylc chang~ ExarnpErs of tabular wayq of pres~nting data about individual change ( l i v i n ~ standards) Advanta~es and disadvantages of tlircc methods sf qucrtrt~tit~n~re adrniniqtratim Sarnpli*S ~ J requirrd ~ S f o r ~rariouqsampling error.; at 4 ; per ccnl cnnfidence Ic*\fel{simple ra~idom sampling) .4n il lustratiiln of recorl ing masking a relationship 1lIu~tr;ltionof dix-idinl: a ~ a r i a b l e rntn three ervlrp.; rt5ing substant~vc a n d dirfrit~~ttional recorlrn~: method.. Zcrwnrdrr and cond~tlc~nat tabTcq ind~cating speCific;lt !on Zero-ordcr and condi tic~naltablcq illir.;trating replication Zero-order and conc! itiilnal tables il tuqtrating an ipd irect
causaf rt~lntionship

%ern-order and conditinnal tnblcs illustrating a spurious rcla tionship 1 . 7 Interpretshon of conditional rela tionship+c~rmparcdto ~ e r m r c i rdatinnships ~r 12.R Cnhort table of percentaE.c attending church a t least monthly hy agr cohnrt and ycar, 197U-20011: ageing effect.; church g at least 12.9 Cohort table 17f pcrccntagc , ~ t t ~ n d i n mnnthlv by agc cohort. a n d ycar, TY70-2030: period effects 12.10 Cohort table rrf percentaRe attending church at leaqt mnnthly bv a c r cilhnrt and year, I9TD-?I]I10: cnhor! cff~cts 12.F 1 Cclhnrt table, lor lrreplarlv <paced surveys: ageing anid cohort effects fnr percentagr regular church ,ittendanc~ (at least monthly) 12.1 2 Cohort table for irreplarlv spaced survevs: period effect? For percentage regular churcll . I ttendance (at Icast monthly) 12.6

209
110

213
714

214

216

217

PREFACE

put and often d o not supyurt tlrc c o n c l ~ ~ < i r that ~ ~ l,ire s Frcqi~crltly drawn from i t . Technology has assisted greatly in the conduct r ~ qocial f rrqearch. Put some c h the basic thinking about the lr)<v~r o j rr~mrt.li is <till missing in research, and no arnor~nt of technology will rcplac~ the nec~lfor a clitarsigl~trdtlnder~tandinji of the princ~plcs of research design. Tlie aims of thr book arc.:
1 to prm-ide social scicnce students with a clear understanding of the importance of research design and its place in the res~arch prwcss 2 to dcscribcn thr m a i n tvpas nC rcsearch J~rigns in s w b l r*rscarc!n 40 that 5tudents a n d 3 ti) rliqcuss thc.;c dcqigns in terms of thtarr I o ~ t c resmrchers will be ablc to evaluate partlrular ~ l r s i g n strategies d n d to sefcct stratcgics apprrlpriate to the problem a t h a n d 4 to providc students a n d rcwarchen with the principlcu; of d c s i p such that thvy will L-rr. ahle to adapt dcsignr. efifrctix~elvtrl meet the contingencicq o f real world rescnrcli s r t ~ ~ ~ ~ t i o n s 5 tn providc <ti~dents .>nil rcscarchers rvith the tmd.; by which they can c ~ ; l l ~ ~ 5hp n t esltrcn~ths and ~vcakneweq[if vannllc design <trateciis.i tor s an,!lysing d ~ t a cnfllccted i l h ~ t ~ ) : h t<o highlight the mat11 t e c h n i q i ~ ~ the various drbqip str.ltc~ics a n & t ~ provide 1 an undcrstan~din of ~ how to sclcct the moqt appropriatt- rnethnil n l anal\-+i<.

In 1"63 Campbell and Ttanlev q u ~ r r ~ McCall's l BQ??! statemrnt that 'Thiw ari. t*xcelleni books mrl courstBsof insiruction dealing w i t h thc statistical manipulation of c x p e r i m ~ ~ i t data, al but there is little help to be Found i>n the m r t h d l ; of securing adequate and proper data on wliicl~ to apply statistical prtlccdure.' -1 here art* now cvcellent books on snmc of t h r technical aspccts of collecting quality dnta (e.g, sampling, qutwticm dt-s~gn,ob.;rn7ational technrqlres). However. it remain5 true that there is a lack t>t attention ~ i v e n to 'securing adrquate and proper ds t , ~ ' in tlic. spnsc III the lop(. of the type of ~ l . l tthrlt ~ a r r collt.drd ancl thr cnpaci tv to m;l kc useft11 and cnnvinrinl: comparisons b c k e e n crt)~! Tfi~< ~ ~ may 2 - r ~ 1 ~ 5 5+I\ in psvchologv, and t c l scme c~tcnfin cducatinnnY r~search, hut in the other cncial sciences it remains true that studpnts arc. typlc.-~llv poorly prrparcd to think about thts most hasic of se.;rarch I~.;UFS. C O M ~ U ~ P and P t;ophfrticaM statistical analvsis programr hatrr llclped In some of the dnalv.;is questions, but analysis without first designing the structure clF the data coPIection will he flawed and Iimited. Thew flaws arc all too nften c\+ident in a considerable arnnunt of pmtgr;radu.~terescarch and in the work of too many estab1ishr.d rescarchcrs. Many texts and averview coursm teach different data collmticm strate~eq ~ ~ t as c hi ~ i t ~ r ~ i e questionnaires ws, and nbsemation, This book is designed around tllc belief that these are subsidiary issues and that the fcrcuq on dnta cnllcction strategv and r n e t h d s of statistical analvsis neglects thc core i s s u ~ qnamely , Fhe smcture and d m i p of the wsearch. T h t l focus on data cnllcction techniqul-s leads to the n e ~ I e c t of approprialc comparison grciups anci pays too little nttcntion t o the notion of rt'a lua tin^, a [t~~snatil-t. c ~ p l a n a t ~ c >;.on f sa phenorncnon. rlic n e d tn attend t i , Issues of research design - thr structure nf the rtasrarch - a p p l i ~ s rqi~nlly to 'cl~ralitativc'resen rchers a s ti1 'q~~antitntive' rescarcher~. [ t npplte~ r~+eardtcs.; of tht. rarticular methtdq that art1 used to collect d a t a . Tl~c book i.; desim*lr~l to show social wirnce students thr impcrrtance of artcnding tn rdcsiy i s w r s rvhl-n undcrtaklng scrcial rescarch. Ont=0 6 the hrnr.lan1cnt~12 fl;l!c.; nf ; l grPat deal nf wcial stlrnce sewarch is that its d~si~ and n structurt. arc inapprnpriatr f<lrthp u.;rs to which i t IS being

The b(wk is divided intrl five parts. Part I consider5 types of seWc~rch, research concepts, research que.;tionr;, cawality and some crf the cnnccp ts that arf l-rasic to under5tanding rcwarch destgn. Each of th~.other four parts focuses on n broad typc of research design: cxperimcntal, I o n ~ i tudinal, crr~ss-wctionaland case .intdv. Within each part the bmk:
m

de~critres thc. hasic dcqign and its variants and provldcs cxamplrs of studies using these designs identifies thc strengths and \rcnknesr;cr uf i l i ~ design and I t r a y of maximizing the strpn~tlisand minimizing the weaknesses, centre~i around the three cow x e a s of methodological, practical and ethical cnn~idrrationq ouhines thc main ways of a n a l y s i n ~the bpr of dain that arc collected uqin): the particular dcsign.

In preparing a book vich ar: thi%them art, man!. proplc to tvhorn one owes tll~nks.1 w , ~ n tto , ~ c k n o w l i ~ r lrnv g ~ hpecial appreci~tionto Chi.; Roiek, Kcn Dernp~cl;,[an Winter .ind lunc r j e Vauc for thtir rnccrumqcrnent, advice a n d timc.

PART I

WHAT IS RESEARCH DESIGN?

THE CONTEXT OF DESIGN

Before examining types of research designs it is important to be clear about the role and purpose of research design. We need to understand what research design is and what it is not. Wc nccd to h o w whcrc design fits into the whole research process from framing a question to finally anaIysing and reporting data. This is the purpose of this chapter.

Description and explanation

Social researchers ask two fundamental types of research questions:

1 What is going on (descriptive research}? 2 Why is it going on (explanatory research)?

D~scrfptive research
Although some people dismiss descriptive research as 'mere description', good description is fundamental to the research enterpristr and it has added immeasurably to our knowledg~ of the shape and nature of our society. Descriptive research encompasses much government spunsorcd research including thc populahon census, the coElection of a wide range of social indicators and economic informatinn such a s household expenditure patterns, time use studies, employment and crime statistics and the like. Descriptions can be concrete or abstract. A relatively concrctc. description might describe the ethnic mix of a community, the changing age profile of a population or the gender mix of a workplace. Alternatively

tFlr dc.cri~~trc,nrnrght a& morr ahshacf qiicsshon\ quih a5 'I.; the 1evt~I of sr,~.~,il inequality incrtx,i+il~g or ~ l ~ c l i n i n ~'Hnw ", sccular i5 societv?' or 'Hcv\*tnz~clip o t . ~ r h 'i~ there in th!\ cun?mttnih7' o r potet'rty have Icc~lratp dtlqcripti~rns thtb l ~ v e lot ~lncnaployrnt~nt hi\tnrically pl,~vcda ktlv rnlr in sr~rialpol~cy reforms (Mar\li, 1487). Ey rlc*rt~rlmstratin the er~*tcncenf scrc~alproblems, competent description can c h a f l e n ~ accep'tt~~i t~ ass~lrnptrons atlout the tvav t h ~ n p arc and c,ln prc,vtrke actit~n. Good d e w r ~ p t i o n pmvokcs tlic 'wlly' quel;tions of c ~ p l a n a t o r v rrst.arch. I f we1 deter! p e a t c r S O C I prrlar17ation ~ ~ trvcr the last 20 vmrs (I.(*the rlcti are sett~nt:rtcher and the pnnr arc. grtSinq pnrrnlr) rvr arc ic~rcrtdto ;ltL '\Yhv i~ th15 liaplx~n~iig?' Rut hcfnre asklng 'why"we muqt bt* sure about thc fact a n ~ dlnirn.;ions i 01 the phenomenon of tncreasing pcll,~ri/ntirln.I t 13 all vrry \t.cll to develop etabnratc tlimrics as to n h \ sc~cicbty rnighl m e n b polari7cd nniv than in the rccent past, but i f the hisic prcrni.;c is w r o n c ( I c. socltlty is not bccnming nlore p7nl,lrizcd) thcn atlthnip&tc, criplain a non-eu~stcnt phcnclrnenon arc vllv. Of cnursc i[csmipticm can tIcll;rnerattt to rnindrrqs Fact g'lthering i.rr what C.tV Mills ( l 0 5 U ) callcd 'ab.stractrd p~npiricrsn~' fhrrcl arc plrnty 13f twniplt>s o t ~ ~ n f c ~ c t si~l.v<~vs ~ ~ c ~ c l and castn s t u d ~ c sthat rciport trivial inf~rrnai1r3n ,~nci fail tn pmvrrktl anv '12.ht ' ytrestic>n*. or pro\,lcli, an!. b ~ r ~ s for gt.rreral~rat~on. F h)nfcvcr, 2 h ~ s1s a tunction nt ~ n c r ~ i i s e y ~ r c n t 1 ~ 1 dc.;cripbinns rclther tlian an incl~ciment of descriptive rcqearch itqelf.

a] Dtrect causal retationshb

bj Indirect causal refationsh~p, a musat chain

cl A more complex causal r,?od~l s f drect and rndrrecr causal hnks

Part flme or full

Ch~ldcare

Eupl,l~~atorv re~~earch Cncuses on i l ~ qi~cqtions. l ~ ~ Fnr rxarnplr, it is one th~nc: to dcwrrbc the cr~nlr rate in n cntlntry, to e ~ a r n i n e trend5 over time nr tcr cnrnpsrr thc ratrs in diffcrcnt countrrc.;. It is quite a ditfrrent thin^ to devplnp r\planabon.; about why the crime rate E l i S I; high ~1s ~tis, why sornt. type.: of crime arr increasing or why the rate is htght-r in some cn~orit~ics than in ntIapr5. 'tht. way in which rpsearclicr~devcl.clp rescarch designs is fundamen t~ lly afftbctrd bv wllether the research questton iq descriptive o r euplanatory. I t affect.; ~ \ ~ h ,information it 1s cclIlwted. For example, if tvtX want tcr c ~ p l ~ ~ r n v some h ~penplc are mortbl i krlv to b~ a p p r r h ~ n d p d anif convoctrd i d crimes rtzr nccci to hnve hunchus about why this is so. lVr mas havc m~inv pnwihlv ~ncurnlp~~tible htint-l~cs and will nivd to collrct infrrrmatic~n ! h ~ rnablc< t 115 tn \cvt. s.r'l-\ich!iiznches ~r.c~rk best t.rnpiricall\l. ~2n\1vcrin,: !litb 'M-111' qilestion., 111~01v1>s de~clop~n snrrzr?! g cxrcplanntiotls. Causal c.xplancition\ argur that plitnoinenc~nY ( e . ~incume . l~vcl) is atfrctcd b! tactnr S ( t . . ~ .~t.nc4t.r). %rnc causal t.kplanatlc7ns will 21r. siinpl~ rcliilt~ o1ltt.r~ 13 i l l be rnc,rts cr3mplcx. For c.iamplc, 1z.p might argur that lllcrc a rforerf rrflrct of pbndrr on incomc (i.r. siniplc genJr*r drscriminatinnl (Figurt- I la). Wr*might arb:ur for a c,llrsal ch- in, such as th;lt ccnder ,ittt~ik c h t ~ r i rof held of t r a i n ~ nxv~trc~; ~ in hlrn affrcl5

occuyntic.nn1 opEinn4, \.r.hich are linkcct to npporbunihcs fnr promotion, 1-lb).O r u7eco~rld pmit a more which in turn affrct income Icvcl [Flg~rrt. cornplcv model i n v o l v i n ~ a number of interrcl;rtcd caulinl chains (Figure 1.1~).

PREDICTION, CORRELATION

AND CAUSATION

People o f t m confuse c o r r e f ~ h o nwith carrqntic>n. Simp!\. because one event f{~llaws another, or two factor< cn-vary, does ncrt mean that one causes thc other. The link bctween t w o cvents may be cnlncidental rather than can.;aP. Thew 1.; a cctrrclntion k ~ t t v c r n thc number of fire engincs at a fire and the arntn~nt of r i a m n ~ c cnuwd by tlic fire (the more fire cngines thr more ciamap*\. Is it thcrerore rcacc3nable to cr~nrlitdtthat thc numbcr ot fire en~inescauses thc arnnrtnt of damage7 CTcnrlv thc number of hrc rnpj;mr< d n ~ the j amount c,t damage will boilr 1,tb due tcl some thlrrt factor such a s the serio~rsness (if the flrc Similarly, a s t h dtvorcv ~ rate changed over t h e twcnticth century the crime rate ~ncreasr.il R ~ C M ' 1~ear5later. But this dws not rntBnn that divorctl causes crime. Rathcr than divorcc calrsing crrnlc, divc~rcrand crime riltrs m ~ g h t both thr clur to cjIl~r*r social yrocessc.; such as srculari ~ a t i o n greater , i n ~iv~dt~alisrn l or pot'rrtv.

Sti~dents af fer paving pri\.atc5schonlq kprcallv perform better in thvir final rrar nf schnolin~ than tlic~sc a t govcrnrnpnt fundtd u~cst~cwls. R u t th15 nced not be bcral~se private scliooEs prndlrt-1, better pcrformancc. I t mav be t h ~ attending t n private school and bc~ttc*r final-yrar prrformancc arc I v f l r tht- ouutcomc of wme othcr cause (sec later discuwon). Confu<in~ caus;l tion with corseFation akn confuses p r ~ d i c t i o n n0i th causation and prrdichon with rxplnnation. U'hcre hn.o ~ \ ~ c nor t schsracterist~csare corrc1atc.d we can predict o n r from the oth1.r. Knowing the b y e elf schwl nttrnded improves our capacity to prcdict acadcmic a c h i ~ v ~ r n e nRut t . this does nnt mean that the whod t ~ affects ~ c ;ICJdemic nchievrmmt predict in^ pcriorntancc on the ba+ cchor~lt ~ p o doc.; not tell uq n71ry private school studcnts do better. Gnori pr.t~dirtinrrrilrt.5 not ilr7pr3rrd on rnt1~17I rrlntirvzslti/~sNor doc's (hr, nbllrt!~to prrdict accrrmr"rl!l
rirnrt>ri~trrrle nn!rtlrity n l x ~ i r fm l ~ ~ l t l i f ! ! . Rclcognizing thnt causation iq more than correlation highlights a ~ r o l d r m IVhzle . wr can ohsewc cnrr~lahon irlr mnlmf r>flsr+rzgtv Cflll5t'. Wc h a w tn irrfrr cause. These infcrmces howtvcr are 'neccqs,~ rily falliblr . . . [tliryj nrc onlv indirectly linkrd to o b w r v a h l ~ ~ (Cook ' and Camyhr>ll, 1 W u : t i ) ) . Becnuw n t ~ rinfcrtlntcs are fallibl~we r n m t rninimi~ctht> chancr~ of incclrrcctlv 5;lrT1n);fIr.lt ~3 rcIntion.;l~lpis cawal t\ hen In tn<t i t is not Ortc @! fhr ftl~?,fi7~rren!#/ ~ I I ~ ~ W C d W r 1 ~ ~ 1 ~ 1t 1ir~ ch q~qn ~sp!~~r~/it~rt/
~~'SI,RI.C\I

is to I~zJ(I;TJ

I H I J R infirr~wt:s. ~ ~ ~

D~FRMIKISA TN ID C I'RORABT1.15TIC

COYCTII.\ OF CAu5?\'l l0\i

'Illere arc two ways of thinking about causes: deterministical/v and prot-rabilistically. Thc smoker who denies that tobaccr, causes cnncrr becausc he srnok~sheavily h11t has nut ccantracted c~incerillrlqtra tcs deterministic cawatinn. Prob~bilisticcausation is i l l ~ i ~ t m t c bv r l hral th authorities t ~ h pnin o t tn the incrttased chances. of cancer nmong s m o k c r ~ D ~ t ~ r m i n i s tcauwtion ic is whcrc variable X is said tn cnuw Y it, and onlv if, X rnvnrinhly p r d u c c s Y. That is, wlicn X is prvstlnt then Y will 'neccwarilv, inevital-4~ and infnl lihly' occur (Cook and Campbell, I Y79: 14). This approach to e<itat,fi<hcausal In;llf quch as: r\*hrnevcrw n t t r is hratcd to 100 O C ~t always boils. In reality Iaws arc never this simple. Tliry will always specify p;Erticular rtlr?dific??~s uncdcr which that law i>pcmtcs. Indt~t~cl a grPdt dral of scientific investigation ~ ~ V O I Y P S < ~ C C I ~ $ R F ; the condl t1c3ns m d e s which particitlar Paws crprrabc. Th11s. tvc. might 5av that nf G r n Irr+r.lheatme tlrrrtj zrattlr t o 100 *C wiIl illwavs C ~ I I + C water tc, hc+iE. Altcrna tively, t l ~ clarv might he stated in tht. form ot 'othcr thing-, being equal' then ?( u~11 always p m ~ - l ~Y ~.c Aed~terminrstic version o f the relnt~onship betwrcn race and inclrme let-el 1v0l1ld5 . 3 ~thnt other thin$:.: being [,qua1 {age, c=rluca tion, pcrsonali t y , c\pc*rience c"tu.1 thrn ; I xvhr t c perscm tvill [alwayql earn a li~ghcrincomc tlian a black prrstm. Thai 15, I racc (X) causes income level (li ).

Starcd like thi5 the notion of det~rnministiccausation tn the < t l c ~ n l scienc~ssound< o d d . I t is hard to c o n c ~ i v r nf a charact~risticor w P n t that will invart;lhly result in a given outcornc even i f n fairly ti#[ sct uf conditions is spccificd. Thc rnnr(?/r7x~h[ of human social bchaviour a n d thc sr~bjt*<t~i'c, n~m)rin~qfrrl ntzr! ~ r r d l ~ ~ l l n r l s components hc of human behaviour mcan that it will never be po.;qiblc to arri1.r at causal statements of the h ~ 'If c 31, and A and R, thrn Y will alw,~yqf~llow.' Most causal thinking in the social scicnccs is pral~nI~ilis~ic rather than fitrrrttiuistic (Suppcs, 1970). 'That is, wc work at the level that a Riven factor Incseawq Iur drcreascsl the probabil i tv o i a partictl far crutcnmr*,for cxam ylc: bcing tcmnle incrmws the prt>b.~ t i l i p of 1%-orking past time; racc affects thc p r t h a b i l i ~ of having a high status jnh. W r can impmvr probabilistic explana t ~ o n s by spwi fying condi t ~ o n s undtar which ?( i.; less Iikelv and more likely .to affect Y. Rut we will ncvrr a c h i ~ v ecomplete or detcrrninistic esyl.>nsticms.Hun~nnhcha\+iot~r is both ; r 7 i ! l P d and mrrcril: there i s a double-qi Jrd character to human social behaviour, Pet~plt.co~zstrri~t ilieir social world and tlirr~are creative aspccts to hurn'ln action but t h i ~ freedurn ,qncl agcncv will alwavs be constrnincd bv !Fir qtructurc.; within which people l i ~ c Reca~rse . bc1hhr7vicl~~ I < r not sirnpl! dcttrrn~ncd rre cannot achlcvc dctrrnl~nlstic explarlaticlnq Hmvevt~r, because hrhavinur is ctlnstrainccl w c can achirvc probah~listicexplanations. Wc can say that a given Factor will inrrcase thc liklihond ot n given c l ~ ~ t c o m but r tlicri*will n w e r b r certainty about otttcnrnc%. Dcspite thc ~robabilisticnature of c~u.;al statements in the cocial science';, much popular, ider~lo&al and pr~liticaliliscnurse translates these into deterministic statcrnmts. Findings about thr causal effccts of c I ~ s s gender , or rthnicity, for cxnmplc, arr nften read as i f these factors invarinbly and ccrrnpleteIy produce particufnr outuornrs. One could be forriven for think~ngh a t social science has demonsttatcd that gender completely and invariably dctrrmincs positinn in ~ c r i c t y , roles in famili~s, values and ways nf relating to crthcr pcoplc.

Theory testing and theory construction

Attt~rnpkto a n s w l r the ' w l i ~ 'clue st ion^ in social scirncc arc. theorirs. Thesc tl~cnrieq I nr\- in their c - , l t r r y l l r - . ~ ~ e(hcmr r mnnv var~ahle< and I rnks), i l f ~ ~ f r , ~ t and .l~~~ .;t-opr. ~r To understand t h r role of thnlrv E n empirical 1.csenrc11 it i~ u.;cful to dlbtingt~iqh bctwc.ran twn diCft-rent sh)lc.; of r ~ s r a r c h theory : tcsting and theory building (Figure 1.2)

Thenry build in^ is a process in which rrwnrcll begins xvith i>bser\-iltinnq and uccs irrdrrrfrrv' rcnsoning tn derive a tlirnry from thrw obscrvatirrns.

T H E COYTEXT 0 1 : I3ESIGN

Empirical level

Stan
here

Conceptual-abstract level

Low
Parental conflict

(a) (cl

Ib)
Id)

H!qh

Theory testing approach Conceptual-abstract

~PVPI

Start
here

Empirical
level

Thcrta t l ~ c u r ~ .lttcrnpt es to mnkra .censc of otlrcrvations. l<rc.~use thc thcsorv iq ~ T C K ~ I d I tC i-~ r i>I-t*tnration':are madc i t is o f t ~ n cnflcd p o ~ t,(;7(*1141?1

'

theory (Merton, TWK) or ex / ~ o fnrlo ~ f theori~ing. W i i z form of thcory kruildini: r n t a i l s askiny: whether tl~c observation is a pnr.ficrrlnr cnsr, of'n Iwrr g~~rr,mlfni7ctor, or t ~ n w the r~bsrrvation-fits rrtto a pttr.rtr or n stcrnr. For e ~ a m p l c Uurkhcim , obscrvecl that tht. suicide rate was higher arnonfi Prfrtcstaxltr thnn Catholics. Rut is rvligious affiliation a partict~lnr case c)f something niorc general? Of what more gcncrcll phenomenon might it be an indicator? Arc there other ohscrvatinns that shed light on this? He also ob.;cn~cdthat men were more suicidal than ivornt3rt,urban dwrllers morc than mral drvellers and thc wciaE11~ mobile mow than the sot-ialiv stable. Hc argued that the comrntm factor behind all t h ~ s e observations was that those groups who wcrp mast suicidal wcre also less wcll socially integrated and crpcrienced ~ r c n t c a r rnbi~t~itv about how to behave and what i q right and wrong,. He tlrrorized that one of the c*xplanation.; for suicidal behazrinur w a s a wnw of norrnlesr;ncsq a d isconncctedncw rrf indi\o~rlt~als frnm tht*ir w c i a l ~vnryd.O f course, thcrt>may Iiavc b c c ~ i ir~ther ways nf accounting for thcsr observations but a t Ica.;t D ~ ~ r k l i ~ i rrrplanatinn n'+ was cunsi.;tt*nt with tlic f.lct5.

~f Elri* Ihuo~!, is Irttr t h t r crrfnrn Ilri)~<qs _slri~til~i f;>f/oii* in t h ~ real worFd. MIp thcn a s s ~ ~whctlicr s: these yrc*dictions arc correct. I f tF~cyare correct thc theory i s supported. rf they do nut hold up then the thcory nccds to be eithtr rejected or modified. Fnr c r ~ m p l e we , may wish to tcqt the tt~ro?'that it 1102 divorce itself that nffccts the ~ f c l l b e i n g of children but the le~~el of conflict trctw~rrt parrnts. To test this idea we can make predictions abnu t the u~tllbcit~)= of childr~n under different familv conditions. For the simple throrv that it is p~rcntalccintltct rather t h ~ n divurcc that affects a child's ~vrtl'tlring thert are four hn.;rc 'conditrons' (see Fiq~trt* 1.1). For ~ c 'ccmditinn' h the thcwry would make different prediction.; about thc Irt~elof childrcn's wellbeing t h t wc can examinc. I f the theory that it is p a r ~ n t a l conflict rather than pnrcnbl divorce is correct the following propoqitions should be qupporttd:

off
rn

rn

In contrast, a thcory testing apprnach 1~r;yi~ts with a theorv and u w s t h e u n to ~ u i d c rvliich o b f ; e r v a ~ ~ nto n sm a k r i t move< from the fit'ntral tcs t l ~ c p>nrtrcuFar. Tht. nbstanatiun5 ~ h c r ~ ~ l provide cf a tmt of the worth of thr theow. Usrnx tk?drllrcfir-v rrnsoning to cFcrivc a syt of propoqitinns from thtl theory dolb.; t h i s . We nr.rd to dcvrlnp ihcse propositions st1 that

Prupositio?r 7: children in srlrvttions (n) n t ~ d(b) ujnrrlri 1~ cqtml[v 717t711 That iq, where parental conflict law, chi1drt.n with divnrcrd parents will do just a s weIl as those whose par~nt..;are married. I'rclprrsifirrn 2: cf;rldren In ~rftmfion.: ( r )~ r r i l( d ) sltn~rl~l be eqrrtrlllt rworfy vff That is, children in conflictual couple 6amiEies will do just as badly as children in post-divorce familirr where parcnks sustain high conflict. Proposrticrrl 3: rlrrfdrnz if; sift~ntt(lrl C ~ rvill J do ;~vfit' tltnrt t l m v rn s r t t m t i ~ ) n (nl That is, those with married pnrcxnts in high crmflict will dcr worse than those who h ~ v c married . pnrtvts who n r r not in conflict. Srr~position4: 1.llrldrrbnirr .srtrrrlhon l d ) nivll i l r ~ 7rrorstut l t l r r l H F D 171 ~ sit~rnftnli (b) T h a t i.;, those with divorced p r c n t s in h r ~ h conflict r v i l l d o MFrrr<cB l a n thore who have divorccd parrtits who .~rtnot in conflict Pri?osilion 5: rhrlriml 111 sihrntro~(bJ ~ i r r l lrln brffer f l i n ~ Hiosr ~ in t t r ~ n l r o n (c) That is, children with divorced parunts who art. not in conflict will do better thnn those rvith married parents rvhn are in conflict. > ~rf11(8l1t)rt ~'roppsitiot ti~ c!t~!~ir iu m< f / t m t l o r (~ r) ~ ~(*i!! bP!frr ~TI~III / h ~ ti11 (iil That is, children t v i t l ~ married parcnts who art. not in ronflict will d o betlcr than thmc with divorcrd parent< rvlln are in cnnflict.
is;

THE CTJVTt-X 1 O F DESK;\

r~

J
Inference

Theoy

,1
Deducrion

Similarlv, ~ ) ~ i ra ~l s e a r c hntvdk design or n <tructure before dGitn culEection or tlnL~lysis can conlrni.ricc. A rewarch d c ~ i ~ is n rrnt just a wnrk plan. A ~vcri-kplnn rlctails what ha< tr, be done to cumplctc the project but tire wnrk plan will flow, E r ~ m thc proiirt'5 rcwarch d r s i ~ nT.T l r p firrtrtrorr of
R S C ~ ~ * R T CrEr*s:,cl~ !? 14 t(7

rf:siIrt7 !ffr?t I ~ R ~~~ii-inrct* ' IFIF? IF?('^^

1'1111!7!~'.;

?If f~ Rl15illt'l' / I l l '

l ~ l r t r n lql~rsfjtwr os

Develop measures. samplp etc.

1 Collect data I
Start~ng point of

theory burfd~nq

Yo ';in+> ~-rrnposition 1~1-nuIt1prtlvide a comprlling test of thv original theory. In~lccd, taken crn it.; ozvi~proposition 3, tor cxamplc, would reve,~lnothing bout t h impart ~ of d i v n r c ~ .F-lowrver, taken a package, thc ~ r - foi proposihon.; prtlvidcs a stronscr tcst of the theory than anv single prclpnsition. r'ilth~~ theory ~ ~ h testing ntld theory building arc often yrest.nted A S alternativr, rnodcs of r ~ s ~ ~ i l r tlic'y c l i sllould be part of one ongoin!: process (Figure 1.31. Typicallv, tlieorv I7uilding will prcjduce a p l a u s i h l ~ account or explanntfon uf a set of ohsrrvations. Hnwtqvcr, w c h explanations are fr~quently illst one of a number of possible cxplanntions that fit khc datn. Whilt plausible they arp not necessarilv compelling. Tkev r c q ~ ~ i r c svsternntic ttlr;tln:: 5%-hertdafa arc coIlwtcd kn spcrifically evaluatc tlnw \\.ell the crplanntion holds wllrn slrbjectrd to a r<lnKcof crucial tfit.;t.;.

rrrrn~~zbi~rw 17.~ yws.;~\~/r. r~h Obtaining r ~ l c v n n te17idt-ncc entails syccifying thc type cif ct'idct~ccnwded to a n s w e r the wscarcli question, to l r s t a thevry, tc) c~nlunte a propammc o r to accurntcly describe somc phmomenon. fn other words, when designing researcl~ w e need tn a&: gtvrn this reqtarch q u e ~ t i o n(nr thtclr~), what hpc of evidence ntlt.rltd to anqlvcr tht. qllc~tion( c r tc5\t n h r theov) r l f 17 crvrr~itrri~;~ rr>rlr/' Resrarch d~,.;ign 'deals witli A Ioqit-nl problem and no2 a Ioxis!~mE problem' (Yin, 1iIH9: 24). Bcf~>rc a httilclcr nr arcli~tect call deveIop a work plan nr ordcr materials they mtist First establisli thp tvpe of building required, it.; II+PS and the nrrds u f the occupant.; The* wnrk plan flow.; From thiq. Sin1il,~ rlv, in sucial rtl\tnrch the isques o f qnrnpling, method of data collcctir\n It..$, qurstitmnnlirtx,nbscrvat~on, dtwl~rnent analvsiq), d ~ s i of p quc.;tit~n-, are all s~rb.;~drarv to the rnattcr of ' l i ' h a t evidenre do 1 need t r ~ cr~llcrt" Too oftcn n~\cnrchcr~ design ( ~ i ~ ~ ~ \ t i i i n n a or i rbrl~ill cs int~rviewing far ton early - brfurc thinking Lhrou~h what informn I ion hey r e q ~ ~ i rto r answer t11t.i r ri1c;tlnrch queshclns. \Y ithou 1 attendi nx ti] these research design rnattcr.; ;it tlic beginning, tlic ctmilusions drawn will norma2ly hc weak and unconvincin~ and f;l I l to answer the rficbarch question.

What is research design?

ic thtx tcrm 'research rIrn.;iqm' to bc rtqerl i t 1 th~?; book' A n analnqv rn1cl7t h i p ii-hcn cnn5trircfri1!: a b u ~ l d in< !Iic~rv1s no pn~nt orilcrunr: matcrj~lq o r +ctting crit~cat date.; for c o m y l ~ t i u n rli project s t a ~ c l~n!il s mrt' know {what sort of building 1s bring c o n s t n ~ c t r d YIP , first decision i.: whethcr w r nccd a high risr, cltficc building, a factory Ior manufactt~ring machintry, 3 ' school, a r ~ s i d r ni;ll t home or an ,~p;lrttnmt block. Until this 1s dolit* rrttl r-annot sketch n plnn, c>l>tain pr,rmits, wurk o r ~ t; 1 ~vc~rk

How

Research dcsign is different fr~jnithe method hy which data arc collected. Mnnv r~searchrnttl-lorlc tcvts confuse rrwa rch designs with methods. 11 15not nncomrntm to scc research design t r c a t d as a rnoclc of data cullerti~~n rather than as a Ic7ctcal ~ t r u c t u r e ot thc i n q u i v . But therc 1s n o t h ~ n Entrinwc ~ about any rr5tlarch design that rc.qllrres a particular method of data ctlliection. Al thougIi cross-section~l st1n7eys arc frvquently equattd with questioiinai r r s ,111d case ~ t u ic.i d arc often eqirattd with p;lrtrcilrant olwcrvatinn ( c . ~ W . h yte's Sfrr*r'! Cor'rrr'r Sncicfi/, 1443, datn for anv dt,.;icn can S r t ct7llrrtc.J with any datn ccrllcction methot! (Figure I .sl. t l r l \ v the data arc crlllrc.td is irrclel-ant t o the l t l ~ ~ nfc thc

JP~F~T
Failing trl cFl<t~ngulshtret~r-ern r l c s i p and mcrhtx-l leads to prnw e ~ ~ a l u a t i oof n dt%.;qns. Equating crr~rs-sectionald ~ s i h ~with .; questinnmires, vr cnsc s t u d ~ ~ with s parlicipant obsc.rv,~trtlr~, mcans that thc designs arp clftcn ~ ~ ' r l l ~ again<! ~ a t ~ dt h strengths ~ ~ and wi~aknesscsof thtl methot3 rathcr lhan their ability tc, d r a w r c l a h v c l ~ i~n,lmbi~tlou conclits won< o r to <t.lcct I-rct\r,een t i v ~ l plnusil-rlc h v ~ ~ t l i ~ ~ t , * .

5cheJtilr~ cjr clrdcr rnatcri;ilq.

tvpe D@slgn

777
Lonq~tvdrnaF Cross-sectronal Ousst~unnaire Quest~onna~re Questionnaire
lntew~ew

: 1at.n the c a w a p i n t of c n n t ~ r s ~ . ~. m I ~ a sbeen thr t ~ n f o r t u n a l rI l n h i n ~1-rt.t~ study m e t l i o ~and l ccrtnln typw 01 date> r t ~ l l e c t i o n - hlr i~xamplclhose t ( x u s ~ l ~ ~ o n qua11tativv methods, ettrriograyhv, o r p a r t i c i p n t nbsen-At Inn. I'cnplc 11av~t k ~ o ~ that ~ h t the case ~ t ~ t d rn v c t h d re~llrited thcm embracr t l ~ r s edata collcchon rnethcds . . . (31 the contrArv, the rnrthcxl dm:: not 1rnp1~. an\' ~ r l i c u l a form r of data cid!echon - which can hr q i ~ a l t t a t r v rnr quantitatrvr. (lcW3: 32)

Questtonnaire
Interview (structured or loosely structured)

Mefhod data
COlleCT10n

lntphr~ew
(structured or
loosely
I

Interview

(structured or
loosely

IstrucTuredor loosely
structured)

5imilarly, Marsh (1982) argue5 that quantitative 5unreys can prwidc information and explanations that arr 'adequate at thc l t v d o f mean in^'. Whilc recognizing that rurvey rt.sc;lrch has not ,llwavs hrcn goorl at tapping the suhicctive dirnt~nsionof brhaviour, *hc argue< that:
Making sensr o( social action . . . is . , . hard and sur\rcys have not traditionallv hc~n w r y good a t ~ t TIw . cnrlicst sllrvrbv rfiearchvr.: started n tradition . . . n F b r i n g i n g thr rneanmg frtrm nutside, ~ i t l ~ bv c r makin): u<e of thv researcher's ctt~lr nt p l . ~ t ~ ~ i l cxplan.~ttons rIc . . . or hv htinging ~t trom w b < i d l a n - rn-deptl~ intrnriews <prinkling quts~m . . . Pibernllv nn thr r a w cnrrelahon%drrived frcrnl the survey, Silrvey res~nrchhrcamc much mirre ixxciling. . , when it h c ~ n n
~nsludinq mr-anincful d~mlw.;ic,ns vn l l ~ c qtr~dv dcki\:ri. [Thi? ha< Ilc,rn donr~1n1 hvn rvavs, f ~ r < t l v IbyJ ashinl;. the actor cither for kit-r reastin% dirPctlv. or 111 <upply ~nfnmatuon about ~ h tc - ~ n t r av l a l u r s in ht-r I ~ t t .around ~ t l u i h rye m a r a w l m e she i s ortenting her life. IThl.;l involvrp collect~n): a .;uffici~ntlv complet~ ptclure of the c o l ~ t c x in t which an actor fiilds h e r w l l Ilint n team ol ou biders mnv mad off the mcaninpful dimensinn\. (1 Q82: 223-4)

structured]

structur~d)

Observat~on

Observation

Obse~atlon

Observanon

Analys~s 01
Clocuments
I

Analys~s of
docvm+nts

Analysis of documents

Analysis of documents

I
Unobtrustve
methods

Unobtr~slv~ methods

Unobtrusive methods

Unobtrusive methods

The need for rcsearch design stems from a sccpticnl approach ti) research and a view that scientific knoruled~r.must alrvnvs be prwisional, 7 h c purpose of rcsrarch dedm is to r~dzicc the ambiguity o f much research
QUANTTTA'I.EVF

AND QuAl.tl.AITVt

KPSCARCH

Similarly, d c s i ~ arc ~ s oftcn equated with qualitative and cluantitativta rcqcnrch methods. Social rurveyq and pupcrimcnt.; are fr~clucntly viewcd a.; piirne cuamplcs nf c l l l a n h t a t i ~ rrst.;lrch ~r and ~ r trvaluatt*d . a ~ a i n s tht> t strmgthq and tt-eaknf-cc of statiqtlcal, q t ~ a n t i t a t ~ \ . reccarch e meth0134 ~ n anal\-<I%. d Case ?turltt9.;,on the ~ ~ t h hand, cr arc oftcn qrcn a s prime rauample:: of y i ~ nita l tivc rrsr.nrch - w l i r c h acfopt5 .~n i n t c r p r r - t ~ v approach e to data, st~ldres 'things' \rf~thin tlicir ctjntext and considers thr subjective mcnnings that pmplc bring to their situatinn. I t is ermncclus to equate a p t r t i c u l a r r e s r a r c h deqign with eithpr qunntitahvc o r qualitativca methods. Kin /199?), s respected autllority nn castL slud y drvsign, has shcssed thr irrelevance nf the quantitativt*/ q t l n l i t a t i v e distinction for case studic.; He point< out that.

evidcncc. We can always find sornc cvidenct* consistent with alrnort any theorv. Howeves, we should bc sceptical nf the evidcmcc, and rather than seeking evidence that ic ron.ci<tnlf with our theory w e should seek evidence that provides a m r r i p t * I I i ~ tcst ~~~ of the tlirory. There arc two related rtrategie.; tor doing thi.;: elirnina ting rival euplanationc of the er.idcnce and dcliberatrly sccking evidcnce h i t cnu l d rJisymi~. thc theory.

PLALSIRLE RIVAL

IIYPOTIT~SI:$

A fundamental . ; t r a t e ~ nf social research involve.; cvnlun'tin~ 'plausibtc rwal h v p o t h r < c ~ ' .We ncrd to cxaminc and cvaluatr alternat~\~e wFa!.q tut cxpl~ining a particular phenomenon. This applic..: rcgardEcsc ot whetlirr thc data arc- quantitative or qualitntivr; rcgardl~ss of thtl particular r e s r a r c h d e ~ i q n (experirnmtat, crvz\-wcfional, lono,ituc!inal nr casta

IVI TAT I 5 RESFARCI I DFSIGY'

Causal relationsh~p
Academic ach~evemenf

Alternatrve explanallon*s~lect~vity on child's ~ n ~ tab~l~ty ~al

1 achievement 1
Alternative explanation lamily resources

resources

ach~evernent

Atternaf!ve explanatron educational values

I,

Child's

Figure 1 h Ctrrrsnl ond n t r t ~ - r i ~ l t s at~xplnnnlirrtl.: f of Ill' sr!:ou! hflrrnn ~ i j iur7t1rfer?rrr na'lrtt.r'rv)rtv!/

r~lntin~r~lrip lvt7~wn

study); and regardless crt the method of data collectinn (p.g, observation, questionna irc). Out rn indsct needs to anticipate altcrnativc ways of in terprcting findings and trr regard anv intcrpreta tion of these findings as prori.iion~! - sutriect t o further tcst~ng. Ihe i d r ~ nf evaluatlnf; plau3ible rrvn l h y p n t h e ~ can ~ ~be illu.;kratrii using tlic crample of tht. cnrrclation b r l w ~ e n FFC crt >ch~>ol attenci~d nnd academic acliirucrnent. Many parents a c c ~ p the t causal propos~tion that attendance a t f ~ e payin? private scl~nc>ls tmprovcs a child's acadtsmic pcrfnrrnanc~~ {Flhwrc 1 . f ~ ) .%-hnuls thrmcc.l\-es pmrnc~tt~ thc same nolion h v prclrnincntly a d \ . e ~ i < i n g their P I < ' : rates and comparing them with those of otfirr qchools or with natimnl averages. Pv irnphcat~nnthey prnposc a cat~.;al connrcl ion: 'Send your child to our sclinnl and thcy will pass (or K F ~grades to gain cntry intc~prt%qti*ous institutions, courses).' The data t hpv provide artb consistent rvitli their pmpp~lsi tion that tht=sc qchonls producr trcttpr rc.;i11tq

Rut t h w e data art* not cornpclliiig. There arc a t lrnst three o ~ h r ways r of accoi~ntingfor this correlation rtrithout accrpting thc. caliral link bttn.t.u.n SCIICWI kpr*and achievt*mr.nt (Figure 1.6). There is thc ~r,lt,rtrritu ex~lanat~rm khe : more able student< may bc srnt to fee paving private school< In t h firs1 ~ place. Thew is t h e j l ~ n l f rrsnr~ri-r7s y explanation: parents who can afford to send their childrrn to fee pa, i ~ r g private schnnIs can also afford other hclp (c.g. books, private tutoring, quiet studv space, computers}. It is this hclp rather than the type of school that p r d ~ r c e the s bcttcr pcrformanc~ nF psivatc. schml sh~detlts. Finally, there is lhc family irnllrc.~ cuplanatinn: parcnts who value ~ducaticmmoqt ate prtlpared to send t h c ~ rchildrcn ti) fee paying private schouls and I E is this family emphasis on ed ucntion, not t h schools ~ themsclvcs, that produces the trcttvr academic prrfr~rrnancr,All these explanat ions are equally con- I sistent th the obwrv.~tionthat prt va te school .;tudents do bctter than go\.ctnment schoor ~rudcnts.Wi tl~cju t colIecHng Curther ev~dcncewc cannot choose betwccn these explanations and therefore must remain open niinded abnut which one ma k t ~ smost empirical sense. Thrrr- m i ~ h also t bi>rnetlrodologrcal euplanation.; for the Fitiding that pricatcl ~ c h o n l5hrdt.nt.; pcrfclrrn t,t~~lt.r acadcrnical lv. Thew rnrthosfologicn l 1\+1rt=3 might ilntltvnine A n \ a r ~ w m e n tt h ~ t a causal conncrlt~nn ~ ~ 1 5 1t \. lr~ the rc.;ultq due to q~r~hlirm.~ble w,~y< of rncnsurin): arhlcvemerit? From what mngc aird numhcr uf schools were the data obtained? On I ? c w many c a w %,I re the conclusions based? Could the pattern simpIy be a functirm of chance"ese arc*all possible altcrnati\~e evplnnntions for thc finding that prii-ate schwl students perfrjrm L~tfer. Good re5carcl-r dwign will anticipate cornpetin# explana ticlnq Ivfnre collecting data so that relevant in forma tion for evaluating the rclalive merit4 u Z thew cumprting explanntions is obtained. In this example of schot,lc and academic nchievement, thinking about alternative plau~ible h p c l t h w e s beforehand would I ~ a dus to find out about thta parcnts' financial resource.;, the study rr.;r>urces available in the home, the parent<' and child's attitude? about ducati ion and the child's acndcmic abilities b ~ f o r cnt~ring e the schnnl. Thr fidlirrit of nftirtttr~tv!lttb ior?crq~tt+rrf Althnuqh evid~nccma\* be consistcnt rzmlth an i n i t ~ proposihr~n ~l ~t might he rqunlly consktcnt r\.ith a rangin of alternative yrnpositionq. Too oftcn p~uplt' in not even think of thc allernrttive hhvpcltlicses and simply conclude that since the eviclcncc is con~iqtvnt with tlirir ihcon. thcn the thcorv 1 ;. tnw. Thik form of r9lsnning commit5 t l ~ c I(yical fi7!brcl, of iifirrrrrriy t\rra c-(vrs~t71irrrt. This form nf rc;lqoning has thr follrnving lo~icalsbucturr:
a a

If A i.; true thcn R should follow. Wc i3bwn.e R. 'T'l~rrriore A i< truc

!\'I1A.S I5 RESEARCH VTS1CY3

If wr apply this logic to ~ I I Pl y e of school and achievement prt~pcwitinn, tlic logicLil struchlrc of the scl~ool tvpc and achieveme11t arEurncnt becnrnt..; clearer.
Ini ti,! prnpos~tion:
a

Privntc schools pruducp better students thnn do government schools.

ilclthing !+ill chnnge the mind n f thc true bclic\'cr. Fxchangp ~llcnrv (Homans, 1Yh1; f3lnii. 1964) ir hrgt-lv non-falsifinblr. I t assurntas t h d i wc alwavs r n a x i r n i ~o~ u r gains a n d a\vid costs. RII! MY can see a l r n r ~ s t nnythin~ ;r* .I gain. Great sacrifict-s to care for a disabled r~latirce can bc interpreted as a p i n (satisfactinn of helping) rathrr ihan a lvss (incomtl, time for self rtc.). We need to fr;lmr our prc~~cwitrrrns and define (rur terms in s ~ r c h s way that thcy art* capable of bcint; diqprnven.

The trst:
tjvtt R If privntc wlrnols p r o d i ~ c e bcttcr students ( A ) tl~critheir ~titdcntqshould get bcttcr final mark.; than thoqc from gnviv-nrnen t funded schools (B). F 1s: Irrrrl Private school qtudcnts dc, achic3ur better final marks than gcwrrnment school st11dr11ts(observe. H). 'I'lrr'vt~fi~ur A is t r l t ~ 'l'hc.rcforc private, schools do p r o d ~ i c cbcttcr str~dcnts(A is truc).

THEPRDVTGlONAL

NATURE C)F SUPI'ORT FOR TI 1 E O l t l I.\

~ L I

ICA

Ev~n whtrr thr. theory is cclrrt>hc+mtcdand ha< sur\,irtd attcrnpt.; disprcr~ it, ~ thc thcnry remains prt~visiona!:

Rut I hat.? ~lwactyar~lrctl,bhc b e t t ~ r pt~r(cjrrnnnceof privi1fr ~ c l i o a l shldcnts might aIso r ~ f l c ~ th tr effect of othcr f.ictors. The problcrn hcrc is that ,iny number of cxplanntions may br ccrrrrct and tlir cvidcncr does not hclp rule out many nf these. For thr sncinl scientist thiq Itwrl of indctcr'minacy is quite ~tn.;ntisf~~ctory. T n rCfcact we are only in a position
tn say:

fnlsific.lhnni.;rn stresses the arnbi1;uity of conhrmatrt~l~ , . . i.orrnhoration ~ i v t n . : only Ihr c~nitortthat the tllerwy has hrrn testcd allrl <iir\ ived the test, t h a t even altrr t l r t * mo4 irnprcq<~vv ct>rrnbt~ratinn< o f prr~dictrunsit ha'; rlnlv nchicvcr! !lrt, ctattz< crf 'not vrt ~Iisct~nhrrncd'. 7 hic . . . 1 s t,>r trnm thr <tatus of nnd Csm(-trr-ll, I L ~ ~ ?OF ~ l 'b~inq trrrr, (C~ujk

* If
v

A lor C, or D, or F, or F, o r . . .I then R. crbsenle B. Theriifcrrc A [ox C, or 13, or E, o r F, or . . .] is tnre.

C l ' t

These always may be a n unthcru~lit-ofcxplanatinn, Wi* cannot anticipatr or evaluatc r1\?cry p n s ~ i b l c~ j > l . ~ n a t i ~ The > n rnorr, dltiwiative explcinaticrns that havr b ~ e n elirninatcd and the more wc h t i v t ~ t r i d to disprin~t* our thwrv, thr. more confidence wtl will have in it, but rue should avr7ic-I thinking that i t is /7roi7cli. Howt~ r r ivt3can dlsprclrlr ; l throrv. The logic of this ic:

Although explanation (A) i q still in the running because it i s conststcnt with thc nbsenlations, we cannot say that i t is the moqt plausible explanation. We need to t r ~ to u r prupositinn more thororrghly bv eva lunting the worth of the ~ l t c r n a t i v e propo<itinn+. FALSII:IC'ATION: LOOKING
FOIZ F V l n E N C E TO IIISI3RC)VE THE TllFORY

If t h ~ n r v A is true then f l sliould filllow R docs r ~ r ~follow. l

Ther~forc A is not true.

As wcll as c v a l u a r i n ~ and ?liminatin~ nltrrnative explanations we shot1 lcl rigorc~uslveva luatc o u r vwn theol-iw. Rather %ban asking 'Wh'lt evrdcncc M ' I I L I I ~ coni;titlitt' Supp~Jrt for t h tli~~r!'?', ~ ask 'What evidence W ~ I I L cr>n\.ince ~ m e that thc t h c n y ic ; t v m r ~ I~ t iq npt difhcitlt tc, hnd evidrncv consistent with a tlieoy. It ir; nli~cl?tougher for a Ihrnry to suruivc thc f c ~ of t pcnplr trving tci disprove it. Unforl u n n t ~ l y some thctirlcs nre closcd svstrmh in which a n y cvidence call bc intwprcted as support fur the fheorv. Such theories a r c <aid to be non-lnlsifinhle. Many religions or belief ~vstcrnsgan brcarntl c1ust.d wst~rn.;tv her+ all e\birlc*nci* can be accnrnrncid,~ tcd bv the tlimrl- and

So long ar; D is A valid test of A tlit- ahsence of B should make tls rcicct cw rel'isc ihc Ilicnry In rcality, wt) w i ~ u l d not reject <I tli~.nr!! simply becau.;tl a single facl o r observation cFoc.5 nett fit. Befort. rt=jr*ctinga plausihl(b t h e n 6 wc w n i ~ l d requirc rnultiplc disconfirrnnt~~~n.; ~iqingcliff~miit mcasurr.;, dittcrcnt samples and d~ffcrent method.: ot data cc,licction and
an~lys~\.

In surnrn,lry, \vc ~ l i o u l dadopt a ~cepticalapprcrach to euplanat~t~n.;. We should nnticipabe rival intrrprctatir~ns and c o l l t ~ t data to ennblt thr winnowing t r t t t o f the weakcr 11xplnnntion< and L h r ~drr~tificatinii of w hicli altcrna t ivc theories nla kc* 11lnstempirical st*nsc3. Wc also nivd to ask what data wro~lld challenge tfic explanation and cdlect data tc, evalrlati* thra thrr>rv from this nirlrr d ~ m n n d i n f prr.;pcctiif~. i

WHA I ' IS RESEARCIH DESIGN?

Summary

This chapter has outlined the purpose of rcscarch design in both dcscriplive and explanatory rcscarch. In explanatorv research the purpose is to develop and evaluate causal theories. The probabilistic nature of causation in socia1 scicnccs, as opposed to dctcrrninistic causation, was J i scussrd. Rcsearcli design is not related tr) any particular method of collecting data or any particular tvpe of data. Any rcsearch design ran, in principle, U ~ any P type of data collection method and can use either quantitative ur quaIitativc data. Research design refers to the strucflarr of an enquiry: it is a logical matter- rather tl-rdn a logistical one. It has been argued that She central role of research design is to minimize t11c chancc nf drawing incorrect c a u ~ a linferences from data. Design is a logical task undertaken to ensure that the ~vfdencc collected enables us to answer questions or to test theories as unambiguously as po~siblr. When designing re.;earch it is essential that we identify the type of evidence required to answcr the research qucslion in a co~lvincing way. This means that we must not simply collect evidence that is conststunt with a tmrticuldr thcory or expianaticln. Research n c d s to be qtructured in such a way that the evidence also bears on alternative rival explanations a n d enables us to identify which of the competing explanations is must compelling cmpiricallv. It also means that we must not simply look for cvtdence that supports our favourite theory: we should also look for e~ridencc that has the potential to disprove our preferred explanations.

TOOLS FOR RESEARCH DESIGN

Tn achicvc a rcasonabl~ research deqign we nerd to attend to a number uf matters before wc arrivc a t Lhc final design. The first section of this chapter outlines these preliminary steps that prccede design. I L then expands on the idea of alternative rival hypotheses that was introduced in Chapter 1. Thc second section introduces a number nf concepts that are f~~ntlamental to designing good rcscarch - internal validity, external validity and measurement error.

Before design

In the same way that an architect needs to know the purpose of tllp building before designing it (is it an office building, a factory clr a home?) social researchers must be clear about their research question before develop~ng a research design.

Fncr~sil?~ at7d clurrfih~~q the rt"~earch questi071

The first questinn to ask is, 'Wl~ntquestion am 1 trying to answer?' Specifying a question is more than identifying a topic. It's not enough to say, 'I'm interested in gctting some answers about family breakdown.' What answers to what questions? Do you want to know the extent of family breakdown? Who is most vulnerable to family brcrlkdown? Changing rakes of breakdown? Over what period? Where? Or are you really looking at the causes of brcakduwn? The effects of family breakdown? AIF tht cffectq or just particular ones?

'1'0 1iarrow thp t o c ~ t s of descriptive rcscarch W P need to specify the scope of what is to bc described. The following guidelines, using farnilv

breakdown as an example, help narrow down a descriptive ~esearch topic into a researchable question.
1 What is the .scopr5nf th(5 rorr roncryts? What is to be inclucled in the concept f n r ~ ~ i l r ,llrcnkdo~c~n? Do wc mean divorce? What about

5
6

.t.pxabion1 d4r~a wc r e t v r r i l ~on!! ~ to ttic b r c , ~ k t I o ~ rIn n rn;lr~tal r c ~ l , ~ t ~ t , n ~ I>o h t pw ~e ' mran thc tolnF brt-nLdnwn nr \ I I I I F E ~ FOI.,~ ttaTdt ~ o n ~ l l i p \Yha! s? ,iboult relatinnships bcttrivn p r e n t 5 and chifdrm o r brtwcen children? Until wc specify what we mean by our corc concrpts rt is RurnR to he imynssiblc to t r r ~ i n thc de~cription, What is the tttrrr* f r i r n ~ for ~ the descript~nn? Is our 1ntt1re5t in c l i a n ~ c nvcr timc or jukt about ro!rh,trrpc~m?flevrl< of famiti7 breakdown? If ~t 3 s ~ h n u chanp*, t over what pcrind? What is the 8,-cymphiatl location for the descripticm? Is the intercst in Family brcakdnwn in a particular community, in d i f f t w n t region5 or tllr suhole n a t ~ o n ? I< i t rrm-tparativc, Zooking at breakdown in Jifftarent hpcs ot co~lntrirs (t>.g. lu~hlv 1ndu5trialirrJrcrws rnpi~Ftl* industrraltzin~ lFrrsusirnp(>vrrished cnunkries)? I low 'yenenrl 1s thc de.;criptirJn to bc? 1jo you want tn be ablc to Jcscribe pattrrnq for aprr.iFic subgroups (e.g. arncmg those who married as tecnngers, amcmg those who arc in i1v,ht-!o relationqhipc, \tu-tjnd nrarr1aKes ctc.)? What n.qpect o l I he topic a r c vou inter~stcd in? Is thc inicrcst in ra lcs of I~rc,ikdnwn? 'Tlrc cxperirncr? Laws? Attitiidcs and hc*lleFs? I l o ~ \~dr.;trnt.l . i\ vcmr intcrll5t' 15 your inlt,rta.;t in f,~rn~Ev 1~roakdc)it'ti or irr fnrn~Ey brcnLclnwn as a rr>flrrhonnf sljmethinq morr abstract (c.g. 'it~.ly?l frilgrn~ntat~~ SL n~ , C I ~ .~(lnflrct, I~ ~ n d ~ i i d u a l ~ s{hc r n , role (11 the stntc in the private lives of citizens)? What is the ~rnittifa~mlws~s? Thc unit of .~nalysisis Iht~ 'thing' about which we collcct ~nforma tion and from which we drnrv cnnclusion~. Often this is a person (e.g divorced pcrqcm) bwt ~t may h~ ' t h ~ n ~ . ; ' such as organi7ations (divorcc courts), a family ax a whole, cvcnts (c.~ d.i v o ~ c ~ s periods ), (divorce in diffcrcnt years), places (cnmmt~nities, countries). Thtl questic~nsrvc can a n w e e r will dcpcnd on the unit nf analysis. Wc co~zldcompare divorcrd individuak with nnn-rlri~orced ~nrlividlir7t~(indiviii~~al as the unit of analysis). We could studv a serirls of ~1ivorct.sand ctxarnine whai the process nf b e c o m i n ~divnrced was (rvcnt as unit of analysis). Wc might itqr vear as the unit of anajvsiq and track changt.s in d i t - o r c ~ rates sincr*. Iq45. Uking cnuntric5 urc might examine the different divorce rates in differ~rlltvpcs nf muntricl.: w i t h rl vivw to comparing the p a t t r r n s in d i f h w n t typts of colrntrics. Allcrnatively familics might br tlic unit5 of analvsis and wc may w a n t to Inok a1 the char.~ctrristrcq of d i v n r c ~ n FantiIleq ~ ( e . ~~. 1 7 1 ~ . fC~milv incomt>, farnil!. ~ p c , ndhllri* (71 sclatir,n\hrp in tarnill) compmed wbtli those of nun-di ~ ~ o r c i n fami g lies. Thinking bcvclnd individuals a s UII~ t sof anal y s i ~broadens thc ranKc nf resuarch question.; we ask ~ n broaden.. d the rangc atld crrllrccs OF d ~ nvailablr. t ~ l'or r.rarnpl~*, ti YeaF wtatrnthe untt of analvsis we ~%'ould obtain st.ltistlcs f~clmthe rrr[e\.ant nntirrtial collt*r.tann agencicq ri=l:arcIing d i ~ u r c c for each yc~lr.We t z r t ) ~ Id ~ also col lcct

other infurmaticm about t ~ a ~ * pear I i l e . ~z ,l n ~ r n p l ~ ~ y ICVP!, ~nt~ int1.1 ~ t t~on

r,~tc, chance+ t r r Inws) that \%a<rel~v.mtt v thc h\*pothews.

F L J ~ V G I EXIV N C ;A N A

TORY RFSFAIZCH QUESTIONS

ttr furthcr spcc~fy o u r f~jrus. Explanatory rcrt~.~rch explvri.5 cause< and /or constc-prnces of a phpnilmt*iirrli, s o t l i ~ ' researcli q u r s t ~ o nnittst hp clear n b o l ~ tthe stvlc c-rt explan,ltory rcsrarch and icltvltify which causes nr consrquences ~t will in\-{=st igate. Pt4tlrc ilutlln~ncsomr difftlrrnt t y p ~ + o f c\~lan.ltirrv research i t i.; ~ I C C ~ to I I intrrwdclunb ~ wrne tprni.;.

In frnrnin~ cxplanntory qnccti(>ns tve need

This i.: the vanahltb that is trc,itr.d ns the r*ffi,r.t In i t 1 thc* causal rnt~dcl:it is dt-pendent o n tfrc inflticncc r l l somv othcr t ,~ctor. The {lcprndent \.nr~a blc is a1c.c~referred t13 a4 the clutcornc \rnriablp and ~ncausal d f ~ g m r n ~t s is mnvontionall\>d c s i p a t e d ns the Y variable. Irr~!cb~rrrrrl~,tll ;nrttdl!r ni\ i* Ihc vCiri;lt~tc th;tt 75 tht, yrta<urncd c'r7/hcL. i t l < a l w callid tt~clpredictor vdriablc, t l ~ c \ p ~ n m c n t a l \ ~ ~ ~ nor ab the l~ c,xplar~at~x-y ~ ~ n r r n hand l c I, { l p s i p ~ ~ l tin i d cauwl ~ i i a g r a m s as thr> 1 vnri~hle (as In tsducntinn (X') 4 incnrnc level (Y)). I~rh~t-rwzi~r~ r v r k i l d r ~ I w r c vnriablcs come between Il~c independrnt i.,iriahle anti tlrr dependent variable i n a causal cliain. Tliey arc. the nln7n.;by n.hic11 carise S prt~duces eff~cl 1'. lntrrtominr:\.ariablcs are rcbpwsentcd in causal d l a ~ r a r n sby the sl~mhoI%, as illustratcd In Figure 2.1. F~fmr~mrr.; rwrr(1llEcs Two vnriables can bc. corrrlatr*d withor1 t bclng csusall~ rdtited. The corrc!;ltion r n d p Irt. d u e to tht- two F~ctr~rx bcin): r~utcomesof a third variahlc (see C h a p t ~ r I). This third variable i s called an exttancous variable and is also symbolirud as Z in C ~ I ~ S ~ I I dingrams, and the form o f this reIatinnship is illustratcd in Figi~rc. 2.2.
is the Irast focused typt*r-rf cxy7an;ltnrv Smrr I r r ~ r for , ~ T I F I ~ W or r f f r c f ~ T l l i ~ r~si*rlrcIi. I t ilivoIv~*< idcnti F y ing the corc pht~nomcntln(P R. c l i a n ~ c E n~ ~ divorce rate dncc World War II) and then searcliinl: for caizhc.; or constaquc.ncw r l f thiq. Searcliin~for catl.;r.: would invol~.ridentif~jing

p c l s ~ i l ~causal l ~ ~ f,ictors ( c . ~ ihanging \'alue5, rlc*clinc~in rcl~gron, chanpnji population mix, crrsnnmic chang~..;, I c y 1 rt%forms, changr.; In welfar~.suppnrt for lonc parc=nts). Wc would then drsign research tn ~vnl~l.itc> which ol tFie~ecau\is< hc*lpq e ~ p l a i n changrs in divorce mtc.;. nil<form of r ~ ~ r a r cqhu e ~ t i o n i k ~llustratcd In F i q i r r 7.7 rVlrrnafivt.lv normight focn~, nn the rcmqilqucncw rn tlirr than cauwq of cli,in\:cs in dii.nrcc sate (Fl~ui'c 2.4).

f ducatjon

Occupat~on

Income 1eve14~)

Changes In l @ v ~ of l welfare orovisron ( x )

Rise ot

I
Workforce
particlpabon

hgllre 2

Jf1 ~ ~ l f ~ 1 7 1 1 r ~ I~ ~ t I tI 5 di/t. ~ I

Economic

independence ChalFengo of
Increase !n
<,*lr,

IXP
Fi+.urc 2

lr~rr=,, 6~

nlr<r..
Rrlinio~~s

marrlaqe

conll~cl ~n

divorce rate

Changes In d~vorce rate slnce WWll ( X )

?(YJ
absolutes

Rejection of
wrongness

E x I I i A mnrc fncir~cdresearch q~tcstion will 5pr.cify a povficr4lnr cntawl proposition lo cvaluatc (Figurc 2.5). [t might prop(Iw an impact of particular factor o r examine n spccific con~equc*nt~r. Fnr example, we might propow that changes in grwcrnment benefit.: t r p lone parvnts have led to an incrcase in divorcr ratp< since \Yorld \Tar IT.
Milri,
,ire qimpIistic in that thcy do no1 .sl>call out the rncch;rni~mshy w h ~ c htlit* two factors m i ~ l i t bc related. Wt) might d e ~ p l n p nmrc cornpIclr nindrk that spell out qclrnt. of t I ~ ernech,lrli~rnsin thr causal chain. Th15 t u l l t ~rntldcl then hcurn~.; thy facuq t ~ the t research nnrl prnl life5 the frarnc\v{>rk ~ \ ' i t h ~ wn h ~ c hthe r e w ~ r ~rltk\lcn lt 2t.111 lv frarnryl (Fikwre 2.6). 12 ht- clL>r~tvinq a r~scarch rllrcqticln i t i 3 h d y f ~ 1tt) 1 draw- diagrams like thaw in F l s ~ i r c - 1.1-2.5 I t I < ,i/<o hrlyfull to four key qurstions:

4 W ~ a lpn.;sihlr. mechanism.: connect the prcsumcd causes to IIIP presumcd r'fCr*ctq (what arc* thr i n t ~ r v e n i n v~ ari~bl~s)?
I V A ~ OF framing rc~cv~wh qtjestions is to formulate d i f f ~ r r n ! undprstancling a phenomenon and then crrrnprp which of thew cnmpetin~ apprrrnchcs b e ~ f~ts t t l r r facb. For examplrl, wc might cnmparr threc diCtcrtwt w.iys of ~indcrsl,inding change< i r l r j i v n r c ~ rat^^: nn rconomlc appnmcl~, A social valu~as approach a n d a lr~i<latir,e appro;lcli. The econrrnit~.,~ppn>ach might ;lrprc that chanecq in ~ltvnrcc rate.; .;Irhrn from econrvnlr t;lslor< such ; I . ; tht- I r a \ t . 1 of afiltrenct., '3t.cr.5~ of rromi,n t r l y o r k and Icl L~I. o f welfare . ; ~ ~ p p r for t lone parent5 T l ~ c wcinl VJ 1111%~. cxplrrnalion rnifilit .;et-k tn expl,>~n thta illcreased dtvorcr, rate in tcrrn* r b i incrcasinfi Icvrl.; nf inctivrclu,ili~rn a n d greater qo~'ia1 acccptancr Of Jivorcc. A I c y i s l a trvc approach rn~gl>t focus on thr iinpnc! of I c g i ~ l ~ ~ t i v r changes srarli ,IS no-fault rli\*orcr fir ne141 rnlc.: rr*l:ariling propcrtv

Another

rolliplr,r i-nirsrll mo~it~l< 5ttcIi propositions

ways of

1 Wh;tl atn T trving to cuyl,~iii (I.c. what is Ihv dcrendent vari;rblr)7 2 nrh,7t arc the pnssitrlr caust..: trt-hat arr t l ~ c Fnclrp~ndentvari~tllt..;)? 7 IVhich o111cr.c will 1 ~,\plon.'

divlrion.

rrl trilrlrtraq ot thi, ucmplc

T17c a b o ~ , tr.t~rnples , tocu.; on pnrticu lar causcs nnd consrqu t-nccq nnd att~.nlpt t c l provi~fc onlv a ~lrrrtttrlcvplanatirln c j t c l ~ ~ ~ n in j i rd s~ v n r c rak't. t~ Tlic.;~ ~ - \ p l ~ n " t ' o n a ~r e ~ . . 1 1 1 t ~ l !~ot?!i?fI~~tli cxpIanations - partial eupIanattc\n< of a class of CJSCS rathcr than a 'full' explanation 01 a y a r t ~ c t ~ l ,caw. lr The\- in\-nh.r nn examinatinn of r~lativel?;ICM. c n ~ i ~ r l l factor.. and a larger n u m b r r nf cnses. In contrast, ~ i i i u ~ ~ ~ ncuplanatiuns yd~ic focus 1711 p ~ t i c u I a cases r and develop as cnmplrtr a n explanation of rnch casc as pc~rqihle. They invnlvc cxnrnining as many factors a s possihlc that c o n t r i t ~ ~ r~l rI~ thc I casc i n c l u r l ~ n unique ~ f~ctnrc. :\ nt7mothrtic explanatirrn o i divorce might fc>cirson the impcwtancc n l a 5i11ylcitlntributing factor s~rch ,75 the intcr~cnt~rational tran5nit\<ion of

Iikrlv to di\'orcc I( onc oi t h e ~ r pnrc*nt<liave dil-nrcrd7 Tlit- study might in\vcll\rc comparin~ pcnrrple who had divnrcud with tliow wIi(l h a d not 1 0 stira if khe divorcvrs wcrc more likrly to have a faniilv histcrrv of divorcc. III contrast, an idinglnpliic a p p r o a c h might focr~r on ; I pnrticular cJivt>rcrd couple and hullil .I t ~ ~explanation lt o f w11y ffr~s c o ~ t p l L~I\-urccit. t~ Tli~ rkplar~ahnn wcruld ce~n<idcr the family histvrv of this ccrttple a n d cxarnlnt* t h i s alonq wrth nianv other cnntrih~tirl< factc~r+ 1 liv ~dlngrapliicc.\plLlnatinn ~ v c r r ~ l ip lrn~~~ u3 de rvitl~ ,I g t ~ r d uncIc.r.;l.~n~li~i~ of thc c.,lcr* tvli~lct11c nom~lth~tici~xplanatiim wrwld prtrvide an ilndcrstanding o f thc influence of a f n u ~ n r .
divorce, Arc couplcs mnrc

- thrir pcar*oiiality, tl-re~r \,,I l ~ r c ,ind ~ belirts a n ~ l thcir ~ntcrpt~r~cliinl sh~lls. I t c o u l ~ Iocir5 i on ccclrlrvrrrt. t.ictoi-s arid intcrprrt marital ha~pinr*q.; in terms c d tlrr ~-ostsand I-cntlSlt< tcl each pnrtntqr. r\lternati\tllv, thc rrplanation ctruld adopt a !if(+fL\;r?ir' !l~'r5{lcrfl~'r* th~t i n t c ~ r cd t~ ~ i i r r m c in c ~n~arital harpiness accorrling to $\-herethe c(~t~pIci q in the lifc coirrw (e.g. nelvlv r n a r s ~ ~beiore d, clrilrlrcn, young rhililri~n, adolescent chilifrt%n, ernptv nest, I ~ t c r Tifr). 1 1trnllnist approach rneglit trl, to ewplnir~ ~ n ~ ~ r iliappiiic~>.; tal in tcrms of gctldt-r rnlcs, division t r f labour and pr~wi-rwithin the rcla t i n ~ ~ s h i A p . d~rnc~grapher migli t focu.: on tlie birth cclhnrts ( r . ~ dcprcqsion . marriages, poqtwilr marriages, 1Qullq rnarriagr~r4c.j Other resrarclit~rsmight wek tcl iaupl.i~n marital happiness i n ~tru~-tirr;l! tvrnrs ~ u c l ,T< i thc c~~itple's IC,YL'F e l f c c ~ 1 ; 1 1d i s a d ~ i l n t ~ ~ ~ . Social n~tl\.clrktheorists might concentrate m thta futent to ~ \ ~ h i cah couple is intcgmtucl ~ n t o the widr*r fnmilv and cornni~~nlt? a n d is n b l ~ it3 rcccivc .;tippnrl from these nrtw nrL.;. Tliis list i b not rxhdl~stivc. l'hc point is t11~itdiffcrcnt approaches ( c . ~ . p~ycIx~logi~ life ; ~ IC(NI~SP, , t.wlinn~~ structural, , lrrnillist) will have a particuldr 'angle' which alert.; 21.i to diffcrcnt w x . : r,f 11lt3kjn~ a t an i w l r . When thinkrnq ahout a yrcrbl~%n~, a 4 yourself q~rc\titrn.;quch as 'Flon' might a h.in~n~\f ,Iirnunl for t h ~ ~ ' 'I-I{v\. ', rni~ht ,I 1 r t a i r r l ian p s v c ! ~ n l r ~ ~ ~ ~ t cxylain thiq", 'l-luw rvnuld a n rxcli,~lige theorist ;Iccrrunl inr his?', 'M hat might a conl1lc.t t l ~ t ~ n rsay?' ~st

In Chsptcr 1 F stressed ttic rmvc*rt;lnccof identifying plauqible rival rvays of accounfilig for the phtzntrmt-non being studied. ObviorrsIy i t maker; sensc tn anticipate thesc altcrnativcs before cnrrving nut the rc.;carch. Rut

Oilfcr restnt.r.irl'r.: Prcvious rt~sr-<~rcti on the topic can I)c a rich source o T cornpct~nc v\planations. Rrad tFrr literature in jnzlmals and starch electronic ~latnb.wcs.Kevie~s article..; a n d intrnclt;ctnry c~\rervit.~j.s can tit. c~fremelvlicrl7tirl. r t ,- i s , , i t 'Insiders' with practicaI knowltdgc of a field can hr invalurlblc. T n n ~ti~d on y divorce, marriage co~~n.;rllors, married c t ~ ~ p l c divorcPd q, p c ~ ~ p and l c advocscy g r o u p can provide valuablc i n s ~ ~ h tL s ~ t c r a t u rcucli ~ as novclt; and plays can a l w prt~videkeen idc,~.; rhat r a n be Z C S ~ C .;yqternaticalFy ~ Itx.g. Tolstoy's nnvd Alr~inKRrr~rzrrnp r o ~ ~ i d e one s way c.rf intcrprcting n ~ a r i t ~ ~ l un happincqs).
J

how d o wtl identify tliesc nInrrnative explanatii~nsl There a r e two marn types of rival hypcrthrses a n d these suggest w;lys of anticipating altcrnative cuplan.itit>ns.

1 5 nrr magical wav of prcxiucing a s r t of alternative substaniivc or thct~rcl i r ~ larplanations. l 111 t hc end the re<~arclicr murt f o r r n ~ ~ lt~ ht~ cm . Thc niclria familiar thc rtqcbarciicv. is with thc particular substantivr l o p i c and w~fll~ C ~ C ' I J Jsciencr rnoJt~I5 and thcoricr, thc nlore likely thcat. .lrtl to antlstp,?tr9diltermt way.; o t interpretin< a givtan ~t ot result\. The folI~v5 In!: prt)\*rc!t. .;ousrcs o i ~ l t c m a t i c \ c r ~ l , ~ tticin<. ~,l

There

I ) i r l l ~ t7Tl)r'tt t h ~ f ~ . lIrrrrrt-!IPS, b, arrd i i r t ~ i r l i ~ ) ~ DO h not igiir)ri*yoiir own pkpt~ri enct, VOLI r 11\1~11 intutt~ons and Iiun~lic*~. In thc cntl a l l t~cplanations <tart lt-ith hrBnc!%rwthat q ~ r i front r ~ ~~ n d i r i d u ~w l sh o liar11~~lc.al; and n17starrr t hlncs atr)utl~lt h t w V3c thrcc ~n.ighh, rxpcricnct.* and r)bscn.at~cjn. and t t ~ n them \\..-.tcmaticall? .

7 1 z r ~ S ~ t , o l r l r r - nEttrrrrtrircb l Rrrwrl approaches In a rli5clpline can yrta.;mt diffcrcnl ways rrt viewinr: L I I ~ question. Supptv;cb, lor examy lc, t h a ~ we wantrd t r l tlnderstancf wliv sornc propIc seem have happicr marriagrs than ntht.1. lx=oplcdo. A n c\plnn,~ ticln coulcl conc.tln trd te on the /lrvr.;crr~nl

T1ic.r~ is nn r i ~ h way t of dc\,cfop~iig ideas. Do nclt linrit y~111rsclf to fornml social scipnct- rrscarch or to rc*.icarch t - m the very ~p~ncific topic you arc working nn tl.;r. c f ~ v c r ~ sn e u r r w ot ideas. Brainstrlrrn w ~ t h ; 1 flotlp tltld

24

WHAT IS RESEAKCH DESIGN?

dcbate the topic. Think latcrallv: if your topic is marriage breakdown, look b~yclndthe marriage and d ~ v a r litcraturt.. c~

Befnre developing indicators of concepts we must first clarify the conccpts. This involves developing both nominal a n d opcmtional defini tiuns of the concept.

Tf findings are likely to bc due to poor measurement (see below) then any theoretical interpretation of these results will be unconvincing. Throughout this book 1 will identify many techilical and methodological factors that can undermine the cnnclusiomq we draw from our research.

Good rtsca~ch design will minimize the threat from these sources. 1 will iiot a t this point go into these methodological issues in detail. I-lowever, it will be helpful to highlight the types of methodological rivals that will bc cxnmincd. Thcw arc outlined in Goldenberg (1992). Derwurrd rlrlz~ar/eristir.s of the sihration, Tronsirnf prrstvlnl c!iarnct~~i~frcs such as the respondent's mood, attention span, health etc. Si~rmtiunn!ffiifctars such as anonymity, people present when data were [as5 of i nrestipitor and respnndcnt. being collected, gender, dg-e ~ n cd S n n ~ p l i q ynf 1i~nl5 Arc lhc conccpts w ~ l measured? l Nnt~re [iJ IIzr sflntple Can we geiicralize froin the sample? I , ~ c ko f rlnrity r$ flw inqfrurnmt Are the questions clear and unambiguv~is? Forrnnf of dotn uoll~ction Are the results an artifact d the data collection method? Would different patterns be found if a different method was used (e.g. cibservabon rather than questionnaire)? Varratiun in the ndrnrnistwatiorr of the instrum~ti!in studies tracking change over time. Prnc~ssir~~/n~zalysis crrrws.

I I

13pcmfion~lizntii1n
Most social science research invol~~es making observations that we presume tap concepts. I f w7e were cunducting a study on the effect of marital breakdown 011 thc wcllbeing of children wc would need first to ~wi-k out what is mcant by marriage breakdc>wn, wellbeing and children. This invoIves defining these conccp ts, which in turn requircs developing a rrortlii~nldffinifin~l and an o p r ~ a f i o ~ l dcfirritiolr al of each concept. Concepts arc, by their nature, nnt dircctly obsmablr. We cannot see social class, marital happiness, int~lligence etc. To use concepts in research we need to translate concepts into something observable something we can measure. This involves defining and clarifjling abstract coiicgts and developing indicators of them. This process of clarifying abstract concepts and transrating then1 into specific, observable measures is called nperatii~nali7ationand involves drsccr~dir~g flw ladder of ubstmcfiun.

I
I

Noirzirrnl ~!c,iirzitio~zs Concepts d o not have a fixed or correct meaning. Marriagc Freakdmvn could be defined in terms of the law (such as whether the decree nisi has been granted), thc quality of thc relationship or practical arrangements (living apart). Similarly, wc need to define children. Is a child defined by a blood relationship, a legal relationship (incIudes adopted ), d social rcla tionship (& f i ~ d o parents), chrt~nological age, level nf dependency or some other criterion? By deciding c > n the type of definition we provide a nominnl definition: it sptcifies the meaning of thc coi-rcept but remains abstract. Different dcfinitions produce different findings. Consequently, definrilg concepts is a crucial stage of research. It needs to bc done deliberately and to he sy c,tcmnticall y and carch~ll y ju5tified. There are three steps in developing and narrcwving d i w n a nominal definition. Firqt, obtrrin 17 vorr,p3 c!f t r ! ~ f i w i t i n ~ Louk ~ < . a t review articles, d isciplinc dictionaries ( e . ~ .R dictionary of sociology), encyclopaedia5 ( e . ~an . ellcyclopaedia of swial =,rieruces) and journal articles. Look for both explicit and implicit definitions. Second, d~ciiIr un rr defirrition. Fmm your list either select one definitirm or. create a better definition from the common element? nf several dcfinitions. Explain and justify your approach. Third, drlmede She dimensions of thp concept. Many concepts have a number of dimmsions and it is hclpful to spell these out as they can help tc> furlher refine your definition. This can be illustrated using tl-re concept of the child'< tvrllheing. W e can think of a nurnbtr of dimensions of wellbeing: emotional, psychological, physical, educational, financial, social, environmental, legal etc. 16 ure are arguing that marriage breakdown affects a child's wellbeing, what sort of wellbeing are we talking about? All of these? Just one or two? Having delineated various dirnensions you will need to decide which are of interest in the present study. You map examine all aspects or limit yourself to one or two. A concept may have s;~bdfm~'17sil~n5. Suppose MT focusrd m qnci~7E wellbeing. This broad concept could incorporate subdimensinns such a s the level of qafety in the neighbourhnod, the nature of the child's rela hvnships ancl hcr expel.ience5 of social diqcrimination . T l ~ csubd.irnension of 'relationships with others' could be hrthcr divided intu relationships with particular people such as mother, father, pecrs, siblings and grandparents. Having settlcd on the particular relationships with which we are going to deal, w e wotild ncud tn identify what raspf7rfs 05 the relahonships to measurc nnd decide how to measure them (see below). In Figl~rc 2.7 relationships are measured according tn the lcvel of

WHAT 15 RESEARCH DESIGN!

TOOLS FOR RESEARCH DESIGN

contact, cnnflict, closeness, helping and type of shared activities. These measures or indicdtors would then provide the core around which to frame specific questions or to fncus an observatic~nal study. In this example I have focused on one dimension at each level but I could have been exhaustive and developed measures for each conceivable dimension and subdimension. The decision of whether to adopt a focused or an exhaustive approach will depend on what you are interested in. Thc advantage of adopting a systematic approach as outlined in Figure 2.7 is that it helps focus and refine the research queqtion a n d forces you to make deliberate decisions about how to measure a concept.
concepts you will develop Opevotionr~ldrfin~fions Having defined t h ~ an nperafinnal defftzitiov~ - the obsen~ations to measure the concept. What irtdicnftlrs o f marriage breakdown will you use? We might operationally dcfine marriage breakdown according to the quality of the rclationship. Using this definition wc might muasurc breakdown according ro level of conflict, type of communication, s i p s of lack of affection and level of cooperation or lack of cooperation. Idow well s ~ t c h indicators tap thc cr~nccpt as dcfintld will have a criticdl bcaring on the value of the cc>nnclusions drawn from the study. If the indicators tap something other than what we claim they do then our cmclusinns can be easily challenged. For example, if our measures of marital breakdown simply tap social class differences in marital style rather than breakdown then our conclusions about marital breakdown will bc suspect. Once the operational definition of the concept has been developed we come to the final stage of operationalization. This entails the precise way in which thc indicators will be measured. This might involve developing queqtions for a questionnaire or identifying what and how observations will be made. Articulating the mechanics of these strategies is beyond the scope of this bnok.

Concepts for research design

Twr) concepts, internal and external validity, arc. fundamental to developing researu1.r d e s i p s . Ideally research designs shrjuld be hnth intcrnally and externally valid.

Infprnal zlnIidity
We need to be confident that the research design can sustain the causal cunclusions that we claim for it, The capacity 0 5 a research design to do this reflect? its internal validity.

TO01,S FOli RESEARCH DESIGN

29

Imagine a rcscarch project that compares the emotional adjustment of children from divorced families and intact families. It finds Hiat children from divorrrd families are less lvcll adjusted than children from intact families. Can we conclude that divorce caused emotional maladjustment? Not on the basis of these rewlts. The design does not enable us to eliminate alternative explanations. The poorer adjustment of children with divorced parents might be due t o adjustment differences that predated parental divorce. A different research shdy may deal with this problem by tracking children before parents divorce and for some years afterwards. It may find that these children do show a significant decIine in emotional adjustment after thcir parents' divorce. Docs the research design s l ~ o w that divorcc is producing this decline in adjustment? No. The decline in adjustment may simply rcflcct a general decline in emotional adjustment as children get older. The same decline may be cvident among children from intact kamilies. Yet another shldy might try to overcome this problem by tracking changes in adjustment c ~ chikdren f bufure and after thcir parents divorcc and changes among children tram lntact families as well. If children troni inlact families show less drteriuration in adjustment than children from divorcing famflies, this must surely demonstrate the effect of divorce. No We would need to bc sure that the two groups of children were romparahle in othcr relevant respects ( e . ~age). . The different rates of change in adjustmerlt could be because those from divorcing families wcrc younger on average, Maybe younger childrrn show greater changes in adjustment over a particular period than older children. It may be age differences rather than having divorced parents that account for the adjustment changes of the two groups of children. Internal validity is thc cxtent to which the structure of a rcsearch design enables us to draw unambiguous conclusions from our results. The way i n which the study is set up (e.g. traclung changes over time, ma king comparisons between cornparabIe groups) can eliminate dlternative explanations for our findings. The more the structure of a study eliminates these alternative interprets tions, the stronger the internal validity of the s h d v . A central task of research design is to shch~re the study so that the ambiguities in the abovf examples are minimi7ed. It is impossible to climinate all ambiguities in social rcsearch h u t we can certainly reduce them.

our capacity tr, gcncrnlizc m t m widely from a rcscarch study is tl-rc use of unrepresentative samples. This, and other threats tcl rxtcrnat valid~ty, will be discussed more fully in Chapters 5, 8, 11 and 14.

A further threat to the concIusions that can be drawn from any study is measurcmcnt error. This occurs when we use flawed indicators to tap concepts (see Chapter 1).

Indicators must meet two fundamental criteria. They must be both valid and reliable. A valid indicator in this context means that the indicator measures the cnncept we snv it does. For example an 1Q test is used to measure intelligence. If it really measures intelligence the test would be valid. If the lQ test measured son~ething tlsc instcad, such as cducatitm level or cultural background, thcn it would be an invalid measure of intclligc~~cc. Reliabilitv means that the indicatnr ~'o)rsistcnll~t comes lip tvlt1-r thc same measurement. For examplc, if pctlylc consistently obtain the same [Q score on repeated IQ tcsts, then the test would he reliable. If their results fluctuate wildly depending on when they take the test, then it would bt. unreliable.
of internal validity related to the validity V~liclrf!/ The earlier discu~sion . addressed the question: is t h research ~ design of the research d ~ s i p i It delivering the cnnclusions that we claim it delivers? In addition we need to cxamine the validity of the nzemures uscd in any piece of research. The validity of a measure depends both on the use to which it is put and on t h sample ~ for which it is used. For example, the validity of using frcgucncy of arguments hetween partners to measure marital happiness turns on what we mean by marital happiness. The validity of this measure may vary for different cultural groups and for the same cultural group in different histnricaI periods. Measures of children's emotional adjustment will vary according to thcir age and thcir cultural group. There arc three basic ways of assessing validity. Criferrt~rlvnlidity is best suited to situations where there are well-c.;tahlished measures of a cnncept that nctd adapting, shortening or updating. It ~nvolvcscomparing the way people rate on the new measure with how they rate on yell-established rncasurcs of the concept. If ratings on the new measure match those of an established measure we can be confident of i b validiiy. Criterion validity has two limitations. First, it requires that the established benchmark is valid. If the benchmark is invalid then i t is of littlc valuc in assessing the new measure. Second, there arc no estabfished nleastlrrs for many social science concepts.

External validity refers to the extent to whicli results from a study can be generdlized beyond the particular shtdy. h study may I-rave good irlteinal validity but its value is limited i f the findings unly apply to the peoplc in that particular investipltion. The critical question is whether I the results are likely to apply rntrrc widely. The mosE cnmmnn threat to

hrrnt,t~tncscrrtcriun Krtwys c i n 1.~2 u.rvcl t o a>-*c\s ct~tr.rirvn <'oli~lit!. In<tcnd (IF cnmp.lrmg n rnt.a<urt. a ~ a ~ nan c t e\i.;t ~ r 1)runchmark l ~ mcaqurr, thc. nrw rneaqtlrt can hi1 tri~lfed. T.or eramplc, ; l nrw rncawrc ot manta1 happinew could be tnaPIctl nn couples w h n seek rnarit.11 counseTlin~. We would cxpcct that t h i ~ g r t ~ u pu E cottplt.~~ , o u I dn n r n ~ a l l vobhin low scores on a I alitl mt-asurt. of marital happiness. I f thc5e couple\ ~ c t u ; l l l ~ ~ obtain high %cormon tl>r.mar-tal happiness mensure rue ~l-rrlrld prohabl y \ran1 to rj~,eqtion whether thc rnrasllrv w6w rc~FIy tapring tnari~al happincr;s. i c l ~ trbnf r i~lirlit!(cvalua trls h(jw well tht* mrasrircs tap fhc differcxnt aspt*cth 01 thc. ctlnccpt a\ sve haw* drfinc-d t t . h Lest of arithnlc~tic skills thEitonly Ierted st~htractinn ?kill$ would cli~arlv not ht- a valnl rncasurr~)f arithmrtic skills. sirnil.arl\~n meakurc o f marital happiness [hat only nskcd about t h r frequency of arguments hetwccn partner< would pmbahlv lach cont~nt valrdity unlesq we had deflncd marital h a p p i n ~ ~ q simp34 as lthc abscnce tlf arg~irnents.Fvleasi~r~s of marital happiness cottld a l w incltrdr tlic nahlrc o f t l ~ c areumcnts, Ieisurt, ;lctivities shared by partners, cnmrnunicat~cm, method': cd reqr,l\-ing crmfl~ct, the quality crf thv <c.u~~al rclation~hip etc. C11.thn ~ l i ~ a ~ t ~ ~ t tabr3r1i ~ m r . the n 'ctmltbt~ t' r l t manv snil;l l tcicmctb cnnr-rpts i t C J I I bc. diffirul~ to drvclnp rncaqurr15 that havr ncr~w-lvaliditv. r-:vu.n il i r e can agnx*iin the ronrrpi ~ n rnpn<urc. d tt ti sin^ a whn!t- bnttrry vf qucstinnq, w t then face t h c prrrtrlcm o f thc ~r'!nt!-r*r ~rtlplrtnnr-co f t h r vnsitru.; cornyrlncnts of tlw rneaqurc. TJilr ruarnple, s l i t ~ l Mmtxapilws of thc lrequcncv of argument< bc as important a.: Lhe nature crl t h r arKum m l r , the method of conflict rcsolut~on,the qtvlp n T cornmunicnt~on or statumcnts about level uf subicctivt. marital satisfnctmn? Corish.rrct zanlirfify rclirs on seeing how WCII thc rmlilts rvr n h t ~ i n when ~ ~ q i nthe g rncasurch f t t with theorcticai mpecfatinns. To validatr n men\urc of manta1 happinpm we might nntic~yatt., nn the hasi-=of kheo?, that happiness will vary i n prcdictnhle wavs accnrJing to sbgc in thc 11fe cvclc. I t the result5 of a studv p r n v i ~ i ctlnhrmatir~n, t thib coulcl rimflect the validih. of o u r rnpasurc of marital happine~q. HonPevcr,Ihiq approach to a~wssing \ralidiv rrfies on the correctness nf n u r cxpectakions. I f o u r thcrvy i.: not supportrd this could be for nnc of two reasnns: tlie measure of marital happinew cnuld bc wrong nr thta t h e o v a p i n \ t which thcb rnc.isurc i< h r i n ~ btanchrnarked mav bc ivrnng. TIicrta is n o ideal way of asstsss1nK v a l i ~ l i t ~ 1f. a rncnsurr passi,.; ~ 1 1 threr, tt3stl; i t f > more Eikclv to bc. \-,hid hut \vtn cannot bv c r r t ~ i n .In thc hnal ;~n;llvqr~ lrc, ~vill nrr-rl In ! ~ t . y r rfur ~ t h r r.,~ltditr ~f our rnc;lsurc.:
R t n l m l l i l i f ~ t A rrl iahle rnca.;lirr 15 nnc that wfit-s the wrnc 'rending' whrn u.;ed nn rtlpca tmi occ-astons. Tor cxamplt%, r7<< t ~ r l l i rtl~r.rt~ ~ r r r r r c !TO ( 7 ~ - fti01 t,lmrr,v(-, r~liat.rlt> rnr.1wrc3 of rnarit~3happine.;.: sl>nuld !*ic.lrl tlic 5anw . . . , . -- -*-.-' 111fiermt c x c a r i ~ t w . A thrrmometcbr that rnpas'c bhp n v x t wrr~ald
I"=

~ J L ' ~ t ~ t t ' ~\ V \ . I l I C r l III,..-changr c,r just rncasurr~nmt 'nt>ise'? Unrelinh~litv can stern from manv svtrrccs. Prior question rvt,rrl~ngmnv cause a r e ~ p o n d e n tto unrlerstand k h ~ questicln drtfcrrntlv nn diitervnt occasions. IXffcrcnt inttrviewers can clicit ~ l i f f e r ~ n answers t from a person: thc match nf agc, ~ e n d r r class , and rfhnrcitl; of an interv~crr+rr a n d interviciver can intlucncc. rctsprmses. A q k i n ~ cl~ir~tion.; about which prople have nt? opinion, 1m.e inrufficicn t ~nforrna tin11 or requirt. too precise an answer can lcad tn unreliable data. Thc answers to somr qur~tion* can be affpcted hv mood and bv tlie particlilar context in which tliclj nrts askcd. Mca.iurt-s ncvd tn bc both t,alicI and rcli;il-If. . I \ E t l ~ n r ~the50 ~h two concepts are related they arc not the qarnc. A mcasurr can bt- rc*lbable withotit being v a l ~ d Thnf . is, a measure can be cnns~stt-ntly rurnng For cxarnpic, people cnnsistentlv undcrcstinlate thcir Icvel nl ,~lcnhoI conwmpticln in q ~ ~ e s t ~ o n nsuwevs. a ~ r c IIEcnhol consumption rnvasures arp rt.liable but do not accuratclv tell us al-rnut thr~ t r u e Ic.\.tl n! nlcolrnl

ctlnsurned.

Mea.;lirc< rvitl never be p~rfcctly rclial-rlc anti pcrft~rtl~f valid. ?IIc'~c ilre nut ,ill o r ntrth~liy concepts ;lnJ thr g-{r.ll iq tcl ~ I ~ I X ~th~b ~ I r~'Ii,ll~ilitv J C and valid th.If !hew aspects of measurtnmilnta r c then tltr rcsu Its nf the s t u ~vi that u ~ e s thrm might plnitsibly 2 7 ;i!trillr~tcr! ~ to poor mra>tlrernent rathrr than tellinfi u s a n y t h i n ~ abortt social real~ty.

Error can take different forms and the conspquences of error will vary depencllng nn i t s form. These forms of crror art, rnlrdrwr, r-on4nllt a n d

ru rrrlnted.
R R I I ~ J ~ I I~'rrnr II i~ that which har no s y t e m a h c form I t meam that in some case5 a rncawrcrncnt for a variable might be too l o w whiIc in others it is too high. The mcasurcmcnt nf somennta's w e i ~ h tmight display random error. %metimes peoplc~undcresf matr thc,i r w c i ~ h t while others may overestimate it, but il: thme errors are randnm thew will b~ thc samc number ol ovcr- and undcrestim,~tesand the s i ~ of e tlw o~crcstimat~ will s he thc same on avcrngr as the ~indcrwtirn~lte.:. Whcn the avcrAgp (me,ln) ih calculated Fclr the ~ ' h o ql c~ ~llp it wtll b ~ ,~cc~tratt. . bccaure tl~c ut-crcstirnntfi , ~ n d the undcrestirnatrs c,incel each other ottt. Furthermnrc, tliesta mis-estimatet arp not cr>rrclatrd i v ~ t hany other characterirtic t e . ~ ~ . c n d r r 'l~c) , but ,isc i r u l ~ random. . I3rcauw random emur doe< nnt di+tort mcanq and is ~rncorrelatrd 1z71tf1other Factr>rs, i t 1s !u.;s sersotis than other fnrms of crror. Cnn.;tnl~t rrrnr occurs rvhcrc t h ~ r c is the sdmC error for cvtlry caw. For rxamplr, i f Pveryonc ~~ndcrrcprrrted their wcighl bv ;ki l ~ ~ r ~ l l m s would have constant errnr. Such error is unccx-n.latcd zvith ibther ~-lisrtictcristic.;. Although purely constant rrrrbr will he rarc* thtv'c wiil he

WHAT IS RFSFARCJ I I3ESlGN7

TCK)LS FOI3 RESEARCH DFSIGN

33

variables for which thrre will bpcally be n crwiponenl of constant et.ror (c.g.ovcrstaicmcnls of frequencv id sexual intercourse and undcr5ta ternrnts about the amount nf alcohol ct~nsumcd).Because such error is constant it Jws not canccl out but has an effect on sample estimates. Thus the averagc weight of the sample would be a n underestimate to thc extent of the constant error. Corret.lotcd error takcs place when the amount and direction of error vary systematically according tn other characteristics of respondents. For example, if women tend to c~vercstirnatc their weight while men underestimate thcirs then this error wouId bc correlated with gender. If the format or lang~iage in a questionnaire is ditficult then m i s t a k ~ sin answering rl~iestinnsmay well hc correlated wit11 education. This wou Id produce results that make it appcar tliai people with diffrrenl levels of education hehdve or think differently wliilc in fact i t is only their capacity to undcrstand the question that differs. A crucial gcml of the design and administration of survcy instruments is the minirn~zationof the various forms c~frnc.asurempnt error. Achicving this cn tails paying carcf 111 attcn lion to question wording, indicator quality, interviewer and observcr training, and to ways of identifying social dcs~rability reqponses ancl ~ t h t ' forms r of dclibcrate misrrpresentatinn bv respondents. rn many cases it is difficmlt to identify the extent to which such errors actuclIl!! occur. However, this does nut reduce the need to d o all that cmc can to minimim their likclihoc>d and to have built-in checks to identify soinc sources of error. Such checks includc looking for inconsistenc~csin answers, tising multiple questions rather than single question? to tap concepts, identifying social desirability response S P ~ S , ma king intcr-interviewer chccks and careful field work sapcrvision.

Finally, thrw corc concept< that are a t the heart of guod design wcre discilssed . These were the concepts of internal validity, external validity and measurement error. Later chapters will evaluate the various designs
using

time concepts.

Summary

This chapter ha? empIiasized the importance of clarifying rcscarcll qutstions and concepts beforc developing a research design. Lack of clarity regarding the research question and the central research concepts wiIl severely compromise an); research design. Guidelines were provided tn help focus hot11 descriptive and explana tosv research questions and to clarify the concepts tliey cmploy. It i q also unwise to devclop a rescarch d ~ s i b v unless alternative wav? of underqtandit~gthc matter a t the heart of thc sc<earch qucslir~nhave been identified. Sincr one nf ttre PtirposeS of resr,lrch design is to help identify w~hichof a rangy of alternative explanatinns work best it is desirable that thcse al t&;t ti1.e explanatinns be identifled before the research design is developed. Thc design can then be structured in such a wag that relevant d a b arc collected to cnable u s to chonse between these alternative< G~lidelines were providccf in asqist in identifying these altcrna l ~ v e explanatiuns. #

Sector (publ~c
or prlvate)

A Q ~

CAUSATION AND THE LOGIC OF RESEARCH DESIGN

Achrevemenr

onentatton
(low or h q h )

Establi~h~n caitsal g relation%htpq is at the heart of cuplanatory rcw-arch d e s i p . I lowrrver, it is not a 5imple mattcr trr c5tahlish that one cvcnt causes another (Blalock, 1904; Hage and Meeker, IYHS). The ma in rc3sson why it xh diftic~1Etto establisl~causal relationship.; is becausc wr. cannnl actually crlr<rnn~r, one phent~rncnnn producing changc it1 anothtr. Even though rlnc cvcnt might always follow anoth?r rvc rlrl not know that this is licc~tr~t' rlnp went c a n w . thc clthcr. Ca~t.;aI r ~ ~ l a t i n n s l ~muGt i p s thcrrnfore b c x inft~rrcd rather than vbwrved. TJlp Irir ryshr* a! rrst,n.rrrr-ilt dt+*iyr~ Irl r r { ~ l a ~ ; i ~ trt ( ~b r i+ tdr v ~is ~~ fo ~ I F T J ~ f h ~ t lI q ) ~ ~ mlify of iw r i flrt<t~l ?~;fir~fk-[~>.

It3nst, necd to demonstrate t h ~ tprivate sector wr lrkrrs had a higlirr achievement nrirntation than public sector w o r k ~ r q .I f two factors did

Inferring causal relationships

I n Chaptt-r 11 1 distingui.;l.rd 'trchvwn probabilistic and deterministir concepts o f sausa tion. Prc7babil istic approaches tcr causation a r r those that argue Illat a given factor incrcases (or d e c ~ c ~ ~ s the e s ) probability o f a particular c>i~tcorne. For cxamplr we may a s p w that Ithere is a causal relationship between gender and ~ ~ c r r k i n part g time - that g~ncndcr afft-cts the probability OF working part timc. T n ortlr r to infer that a prohabili5tic causal rcla tianship .exist< between trr-o variabrr'q, m e baqic critcr~nmust be met. F I ~ sthere ~ , mu5t h r cm variatinn o f causal and outconip variahlcs ( c . ~ ,between genclcr and hcing a l'i111 timc or part time workct); and second, thc asscrtlon tha l one variabltl nffrbcts the othcr m u ~ make t sense.

not co-vary - that is public and private sector workers had identical levels of wnrk achievement - thcn we would bc harti pressed to arguc that they arc causally related.' Howevcr, while co-variation i< a precondition it i s not cnuugh for 11.; to assert t h ~ thc t variables are c a i ~ s ~ lrelated. lv Cortcla t ions can also reflcct ncm-rauwr n-latic*n<h~ps. 12'lit~ntwci variables rlr t5\,vn ts ;Ire scirrrl ,I tcit hut not cawal ly rclated the rclaticm\hip between thc two variables is s a d to bc spurtoll< (rcc. Chapter 1 ). I or rxnmplc, tlir corrt.lation bctrz-ccn sector of crnploymcnt and e m p l n p m ~ tnnrienta tion m i ~ hbe t due entirr-tv to a third variable such as agr (Figure 3.1). Youngrv. peoplc might bc more likelv than older people t c ~ work in the privatr srctor and vounErr people rniqht a l m have a higher dchicvement orrcnt~tinnthan olclcr people. Thtls~-two patterns will mean h a t sector and achievement orientation arc c~?rrrlated(rnnre young people in thc* p s i ~ ~ a t stytor, e so therefore t h private ~ sector is asscrciated with a c h i t ~ v ~ m c orientation). nt Howevcr thc link between stctnr and achicvrmi*nt uricntatinn is coincidental rather than causal.

Any assertinn that c ~ v a r i a t i o nrc.fIc.ctq a causal retatinnship mrrs! be plausibye I t rntw make sense a t three levels.

factnm are causall\~r r l a t r d thev muqt ni I r . ~ c ! br corrrlatctl: tF1r.v must co-v;rsy. IfS causes 1' ihcn pcople ~ v h o r l i f f c r Erom onc anr~tlit*r on X sl~ouldtcnd to differ frvm onc another on Y . I:or pxamyle, if w r rvrre to a r p c that working in tht* private sector ratlici. t h a n the public qrrtnr make5 prnplc morc achi~igt*rnmt oricnted at wurL w r drould, nl t l ~ r 'r 1 ( , t ~ t

Ff hvn

Timr or&r I f two variables arc. cr~rr~lnted the causc m 11qt come hr:fi,n-fhc cff~~ Caitsal t. rc.irclnirig has no t1mt1 for the assertion that a fuh~re rvcnt can h a w a prcscnt effect (ti=lci~Iogicalexplanatinn). Our calisnl propnsition muct be such that rhr causal variablis t m u r s trtforc t h r Twsurncd effect. The fime gap trctwcm cause and rff'~3ct can be mrnnt~s tlr may be yrvrs ( e . ~the . ~tft'tt of education nn InctmP can take many years to show i t r ~ 1 f ) . Even though two variabler might bc causally rclatrntl i t can sornetirncr Ilc difficult t o work nut which v , t r i n b l ~comes first ,ind thercforc t o

36

WHAT IS RLSEAKCI-I I )FSIGN7

CAUSATION AND T H E I.OCIC OF RESEARCH DESIGN

establish which variable i s thc c;ll~st. and M'IIICII i> t h ~rftect. ' For example, docs .;cctor vf emplnymcnt ,?ffPct achievement orientation or is i t the other way around? E v m wherc we assert that vne v ~ r ~ a b l comes t. first f employment the causal relationship may be two-way. That is, sector o may affect achievement orientation which in turn influences Future decisions about the sector of ernpfoyment in which one works. Causal relationships can be reciprocal (two-rvay) rather than one-way.
Drp~~idcw rmrinblc i ~ l ~ u bc. s t ctapnhlf of ~ ~ l l o n If ~ we t ~ say that a corrrlation hehvccn two ~ariables i s because one is causing the other, we must make sure that the dependent \ nriable (the effrct) iq cnptrblc o f being changcd. I f it cannot bc changed then a cauqal accrmnt c ~ tht. f r ~ l tionship a makes no sense. For example, any causal relationship between sex and income coutd only be in thc. direction of sex affecting income. The opposite prr)position (income-sex) makes nn sense.
Figure 3.2 An indirect calrsnl relnfionship

a) Long causat chain X--tZ--tW+V--tP--tY

b) Multiple indirect paths

Tl;~orctiinl plnuqihilify Thc causal assertion must make scwso. We should b r able to tell a story of how X affects Y if we wish to infer a causal relationship behveen X a n d Y. Even i f we cannot cmpiricallv demonstrate ! ; D T ~X F nffecls Y we nccd to p r o v ~ d e d piat~sible account of the connection (plausible in terms of other research, cu r r ~ n throrv t rtc.). Fnr cxan~plr, to support the assertion that sector o f crnploymcnt affects achievement oricntat~onwe might arguc that the private sector fosters the dcvcl(~pment of an achierement orientatinn by strategi~s such as paying performance bonuses, developing a culture of Fligl-rcr expectations, providing better resources and creating less job security. When backed up by this typ~ of rcasrlning, any cnrreIatinn between crnployment sector a n d achievement oricntatinn can bc plausibly interpreted in causal terms.

TYPES OF RELATIONSHIPS
Types qf cnrrsaf pdterns
~ I R E C TA N D INDlRECT C h U S A I REl.ATlONSHJPS

I N A THREF-VARIARLE MODEL

can be cithcr direct or indirect. A direct relationship is one wherc we assert that the cause affects the outcnme directly rather than tFiaother variables. An indirect causal relationship is one whcrc tlie cause has its ~ f i e c by t operating via its lnflucncc on another variable that, in turn, produces the cffcct. 'P'l~evariable through which the hvo val-iables arc. rulatcd is called the i~lterrtrrllripvariable: it comes in time a n d in n causal sequence bt.tcuvcn t11c inikini cause and Il~c. rffcct. For examplc, we might argue that the way the priv,lte sector produces higher achievement orientation is by making e J t l ~ ! o y fear ~ ~ for their jobs (the intcrvcning variable) (Figure 3.2). Tndtrect causal relationships may be simple (as in Figure 3.2) nr consist of an extcndcd causal chain or a number of diffyrent causal paths ( F i p r e
Cauqal relationships

Anv relationship between trvn variables will consist of two components a cdusal component and a non-causal {spurious) component, The causal component can consist of a direct component, an indirect component or both. I t follovvs then that any relationship between two variables can be int~rpretedas:

a direct causal relationship an indirect causal relationship n spurious relationship any combination of these.

3.3).

Figure 3.4 il lustratcs the possibiIities where we have three variables which, for the purposc o f the example, T will call X, Y and Z. The relationship between X and Y could be any of the following:

{,I 1

11-1 ti tnt,, I I\ c a p d l ~ l t ot ~ Iltl~ng c h . l n ~ c dand , i r jk plau~ibte t l ~ a tN c i l ~ ~ t yd rtdr~cc ch;lnqt.s in I In lIip ab.;i*nse t v ( finrttng anv nthrr \.ariablr that is ri5ponsibI~ for this rrlation41ip ~ v t . may cmtinue to arguc that thr vbwrvPd ctjrrclGlt~on IS dircct n i i d is causal ( F i ~ u r 3 e 421. '1 s cLip~blr nf being changetl, (%) firrJlrrr! r f i ~ ~ s n lY fcrllo\v<X in time, 1 and i t is pla~~sible that X cnuld producr shan~t-sIn Y. I-lclwt.\.cr, In this case we .Irtl syt*lliiiqout the n ~ e c h n n ~ v bv n which X affcctq 1'. Wc m;ly think of Z a< a 5inclc mcctianisnr, or a wt~nlclot of i n t e r v m i n ~

LJ~nlcl ~ r r r r c r r l \ 6r1llr)zv.;

/-CLJ
I

I l d i Direct and nblrecl causal r@lat~onrn~p]

varlablcr; (Figure 3.4L-r). Y arc not causallv r r l a t ~ dto nrlr annther. Even (c) G ~ I I I T I ~ I ,\C I I ~~ n d thm~~ Y'h might follc~ir.Y in t i m ~ nncl bib sapn'tllc of l l c ~ n ch ~~ m y c d , both t' and I a r e joint c f f t ~ t s of somp third variable Z. X ;lnJ ) (-0vnrv purr*ly hecauw Z Iias a simultnneorrq c*ffeck on both S artd S
(ct)
Eta)

(Figure 3 4c). E,)fI, rf!r,rl oitcl i t ; d i r c ~ - t The cffrctof X may bt. partly vi.1 its cffect cm an i n t e r v r n i n ~ variable and pnrtlv d~rcct( F i ~ u r .3.+Ld). c Prrt-rt nr;d zlidrvt7c.f ciirlwl nrlri sljrrrrtltr.: TIIV rvlationship betwervn S and I'coutd cnnsiqt of threc mrnpclncnts:,I dircct 1-at13al p;rrt ( S - . Y ) , -111 intl~rcctc-ausal pJrt -1 - Y ) . ~ r l < l > , \purriril< part (Y. Z .: )
~ / ~ ~ lL ~ . O i~ ( Iv ~ ~ ~ ~(F:iqt~re [ ] I I ~ ~ / I !

te) Dlrect and lnd~rect causal relahon5h1pand a sp~lrlous component

[Frgtrrc 3 . 4 ~ ) . ( f ) L)lrcLt (t11(.<01 ri-hti(v14ti~) t*orrtl>!ttf~ :it1111 ~' {r

( 1 ) Splrnous

(nnn-causal) and r11r~cT

sal relal1on5hlp of X w ~ t hY

't.4f). { g ) Imlirr,cf crltr,wl ridntw~nshrp rorr;brrit+rl'ilr~fii n

~p~irlo~i':

c t ~ ~ p ~ i r r ' r (Figure li

3.4~)
T h i s s r t of pos.;~bilitics rc~sultsfrom situations In which rzrc ha\,e only t1irt.t. vnrial.rlcs. The more variabI~5wc takt. intc~. nccnunt, tlic more ccrrnp1t.y matter< l ~ c u m e . k17hcn wr collect and nnalysc data i t w n hp hclpfu! to d r a w rfitngr~rn.; t o ~ p r l out l the ways in wh~ch we ptc~poye variable< arc inttarrclatrd. W P nred to 5pecify:

1 ~ gS~urtous ) (non-causal)and rndlrect causal relat~onshtpof x W I ! ~v J

1 whethrr selatinnships arp presumed to he causal or spurious 2 whether cauwl ri*laSicmshipsart- expecttd to be direct or indircct ? the mcchanr~ms (inten-rning \-,lr~;rble.;) unrlcrl!.fng anv indirect ca~rsalrc1ationship.i.

Providing a frame

o f reference

WI-l A T I5 RESEARCH DESIGN?

CAUSAl'IC3N AND TFTE I,OGIC 0 1 ; RESEARCH DESIGN

Propositron Dlvorce leads to emot~onal problems In young children.

[divorce

emotional problems of children]

Propostiton Dlvorce leads lo emot~onal problems m young ch~ldren.

Restated pruposr!ion Children whose parents have dlvorced are more likely to d~splay emotronal problems than are children whose parents

Observation We find that many ch~ldren with divorced parents have

emotional problems

have not d~vorced.

[parental rnar~tal status -level of emotional problems]

Propusrtion Private schools produce high academic performance among therr students. [pnvate schools . high performance]
Observal~onThere are many instances of studems from private schools achlevlng h~ghly. Praposrtron Youth unemployment IS responsible for youth suicide [unemployment hrgh suic~de rate]

Proposrbon Private schools produce h~gh acadern~c performance among their students.

Restated proposition Private schools will have hrgher levels of acadern~c performance among their students than wrlfgovernment schools. [school type level of academ~c performance]

Proposdion Youth unemployment is responsible for youth sutcide.

Obsen/airon 50% of young people who commit suicide are unemployed Figurt* 3.5 Prop>sittnrzs nnd otisrwwt~ons n-itl~orrt rr n ) ~ ) r p r n t r rf~ rv r i ~ ~of r
~~y~~rt'it~>r

Reatatedproposrtion Unemployed youth are more likely to comrnft su~c~de than employed youth [employment status -su~crde rate]

research study is providing a compamtivc frame of reference. Dfffcrcnt research designs go about the task of providing compnrisnns in different ways.

By making comparisons we provide a frame of referencc within which to try to make sense of particular findings. 'me importance of this can be seen in khe iIlustratio~~s in Figure 3.5. In these examples the obsesvatinn in each case hardly provides convincing support of the propositii~~. One 'reason for this is that there is no frame of reference within which to make sense o f the observations. Thcre are n o groups wit11 which to cornparc and contrast the observations. It is only hy making rnvrpnrisor!s that o u r observations take on much meaning and we are able to clirninnte al tcrnative explanations. The propclqition that rii-rlorce leads to e ~ n u f i t ~ r?lnlhl~rns r~! in ! / n r i 1 7 ccl~il~~rrrr ~ wcluld encourage us to expcct lo f ~ n d emrjtional problcms among children whose parents a r c divorced. But finding such evirlence will not get us very far. To go further down the path Jt explatlatian we must make compasisnns. Arr emotional problems greater than, the same as or less than those among chilcfren from intact farnilics? Do the obscrvahorls among children of divorced parcnk simply rcflcct that ~ ~ l i i c we h would find among any group of cliildr~n? 1

I
I

I t will be recallcd that when we say that two variables are related it means that variation or difference on one variable is linked to differences on the other variable. In this itlstancc we have two variables: (I) parents' marital status (married versus divorced) and (2) child's emotional a d j u s t m ~ n t(low vcrsus high). Tf our research design incIuded only divorced parents we could say nothing about the impact of divorce on children. Even if 100 per cent of the children from divorced parents had a low level of emotional adjustment we could draw no conclusions abont the impact of parental divorce on childrrm's emotional adjustment. We would need to compare the adjustment of the children with divorced parents with the adjustment of children From intact families. The d$krmcr in adjustment levels in the two groups will provide strong evidence regarding the alleged effect of divorce on children's emotional adjushnent. 'The propositions in Figure 3.5 only mtlntirln one particular group (children of divorced parents; private school students; suicidal youth). It can be helpfu! to restate thc propositions to highlight the implicit cornparisnns. Figure 3.6 restates these propositions using explicit comparisons.

MULTIPT~E COMPARISON

GROUP5

The independent variables above (parcnts' marital status, school type, youth crnyloyment stahls) have all been treated as two-category variables (dicIintomies). However, we are not restricted to comparisons of

hnt~r\.~ r h ~ c uth the ~ i i l t c r ~ ~ itr..l\ ~ c t ~rt,~pon+il,!r < I t v an!, cliltclrct~sc\ In ,~c,~~lt.rni,a ~ - l i i t - ~ c t' ~nc~~i C'on~pari\cm\ 11f chi Id rt*n frcrnl ci i\-orcc*ii ,lticl 1nt.7~tfiimilrcs ,Ire. cnmplicnte~lbv tllc lac! t l l ~ t t~picilllvc F i i l ~ t ~ (1I .~n d ~\~orccil yarcmtk arc olcler than chlldrcn from intacl f n r n ~ l ~ eAny s difference< bckwren the lcvcl of cmtrtiunal a d l u . ; t m ~ i ~ ol t thc two s ~ t . ; of c h ~ l d r e n col~!dbtl diic to .igta diffVrcncc<rathcr thdn t l ~ c mar~tal status uf t h e ~ r pnrrant.;. rhr s;lmt, prohlrrn can arisc. 1vht.n lor kin^, st tl~rstimc grcwp or cntrgny of peopIc o\.er time. Icicnl l y rvr th comparisons over hmc thc c~nly d i i l ~ r t ~ n cbetst-vcn ~.; thc pre-tc5t 2nd the p o q t - ~ t ~ \houlcl t b r thc ci7cntwe artb pr~jpurinqa* thr r;iu.;c nt ;InV c11an):c. M o r v t - ~ ~ man!. r, crt~ntsc,in ci<Ur rvliit-t~ <.In aiiillint frlr Ihc changc ~ n d tI1c.r ma). conf uqc oilr Cilnl~TlWtlq2nd ntir m~tirib~ttir)n of 1% hcit lies behind Ihc chnngr i r i lie rr~~tcrvnt. v;in,~blb.[wc Chnptt-r -1 for Furt hcr di>c~rs<iuri) Since corn!>ari.;rlnh arc. ccili rl71 t(l g(,nEi r c s ~ a r ~dckicn I1 si0cmuqt cn511re th,it tfir1v art1 mraningful r l ~ t y rnorr- MT car) rt'rnoIrv u t i ~ n t u n ~ l rand d unknown clilfcrtbnctaqtlet~v\.ct*n groupr, tlits n1tx-t. n f c rcdllcta tho rtqk ot m i s t , i h i n ~. ; y u r r t ~ urt~lation.;liips ~ for cnus;ll rrlaticliishi~.;- tli,it i?, thc mrjrc wrc3t'linl~nnte~ l t t m , ~ t i pIiiti~iblc \~e cal>lat~ationr;. l-kit3rt-C I ~ L * tt~urT I I ~ I ~\ I IrLitt3~it*\ for T T I ~ ~ \ I ~ ~ I / t ~h Ic >~ K ~~mip~ir~~ tjl bil~t\

Ii'e could ihcn ctmtp,lrc t h t ~ s r group.; and sec i f t11c.r~rt3r,rr differr n i v s in their I ~ v c lof acadrrnfc ncliirv~rncnt. T!icre .lrc many prilblcrns with thiq approach The trbviotts clnv is Lhnt rr7c must have thr r r l e ~ ant information rln nrhich to niatch. 5incte thiq will alwavf; btl Itmitrd, t h t groilpq arc. likclv to remain i~nrnatchedon important t w t u n k n t i ~ ~taclor.;. n Ant~tl~ prol-rl~m ~r i s that many casrs Crnm both group\ s~rnplvwill not have matches frrrrn thr% other Rroup and will have tn I*c ~liscarded. 1f WL- match nn more than a smnll numbcr of factor\ wc r ~ i loften l end with only a very small number of people in thr s t u r l ~ slncta the nurntrcr nf cilse.: from both group.; that match c.,in Lit, cluitt. small.'
.I+*.

S"""3.

W1~t.nrwruitinq thc xrt)tlp< w v c.ln dcIil-rt.ratvl\. match then1 on rclel'ant c h a r a c t e r r ~ t i c ~TIIIIY . wlicn cc*mp;lsing sludrnt.; a t t ~ n d r n gpj\.ernnlml qchool.: on lllr onc hand ,ind fvc paying prir.,itc ~ ~ 1 1 1 7 0 (in 1 ~ thta ~ ~ t l i r r 11rl11ij we s h o u l ~ l enwtre Ilint the two st44 ~ l .;luclPnls f are similar in tl.1.m.; IBF intvlligcnl*c, ; ~ s p + a t i r ~ n s p,lrcntaI , rcquurctbs,t-d ucntinn vnl~res,fatnil v Ilistr~ry, gentler and agc. R v comparlnf; like with I ~ k e wr. stiuuld be ablo to ~'inlatc thr ~ I f e c t nf tlie t v t * o f ssl-rt,clf on acndc.m~cachirvrmcnt. 7'he ptntrltm in i o m p a r i r ~ g likr with lihtl is tcl e<tahfisl~ the idtantit]; of all thc* t*asiahli** on whE1.11 M'C nvecl b match the grnttps. \Ye can match lor fhc charactt.ri>ticc ttmt w r k f ~ o i c nirght ~ rcmtattlinat~~ o u r restillc btlt hc o t h t ~ I S C ~ C J T ' ~ that 1t.e have 1101 t h t l u ~ h of. t Ihcrc FI / ~ o % Ctc!,i f !!ij!s: icdtafi!!~ j:rot!p > ! ~ ( ~ T1~~ Ir L -nI~ t c h ~ c ~ti !li<, b t a y ~ q thtb - 1 1 1 ~ 1 i lwfdlrt* r-ritical P \ t>nI< (1b.g dii.orit', iiffr+r~~fin?: 11ie q ~ h t ~ r >:Intrthi.r l) ,~ppr('*lch rt-ferred t13 bt. Spcrtcw (l"81: 4 4 ) a\ ,T 'p~it-17\\-rd, p-t,cctl~~~ i nr . t 'r)!\.rtc r t q t i n ~ ~r~aicllctf r r i w 17s frotn a ~ c h o h~. r > t \ of l .;tud\ p.~rticip,int.:nl/t7r. all thr. datn hn\mP Ivt-n r r ~ l l ~ ~ c tFor ~ . dc\arnylc, . we rn~clit ha\^ a l , ~ r <niunb(~r r nl41irclrnls frnnl qcr~w-nmcnt and fr(b p,>vin:: privalt' rc111101~.1 r i m pcrril rut*c~ruld r ~ t t r ~ la c tjirotlp of p w r r n m t ~ n t rcIrotr[ +furlcnl< ant1 a crrrup nt fcr plyirl): pr!\.ntta .;ch~,ol st~~clt*nt< ~vlrt~ arta c.ornpar,~bIe111 ttltnls of ~ntc!lif:ence, a+piration$, p,?r~ntnl rcsrntrrcs, . . .I . . - -'.+-*-* Eh.3 imntrrt,in~tn r i l ~ ~ I u c , i t ~f o, n ~ ,t n ~ Il v ~ I ~ I V, ~ o gcn11~r, I ;1ni1
T I I , ~ ? ~

A simpler dnd m o w ~>fft-ctivc wav ~ ) ma f kin^ ~roup:: comparrzblt1 i s to randornlv nllocntp ppnplc to diftenwt ~ r ~ u at ps the h e g ~ n n i n of ~ n study. Rv randomly assignjng inditidtrals t r ~each g s ~ i u pany d ~ f f ~ r c n c c s ht*t~t~u Kroup.; n s l ~ n t ~ be l d random rather than svstrmatic. 5 3 Ion)= a s grotlps ;lrchI.?rgc r t i o u ~ h random , assrgnmvnt s h ~ u f d atit~>mntically pror!~c.c' g r { ~ ~ ~ pc riih crvny;lrrlbTt~p+ufilc.c: clil Lvth Irnor\.n mil 11nLnown t,ritcrn. Fron~ a qtn!ist i c ~ ~ ! ~ ~ r ; p ~ ~ t~3nCjo1n i 1 * t , , a<.;ignment of ycrrplt, hr qrotl p%12111 111~1kv iht' grt,t1~,1;lilcl~t~cill for rll E intents anti pt~r\'oqt.s .ln~J yrol idc*s 13,hat Dnvi.i call< '7hc all purpow kpuricwitv in5uranrc of rCinrlnlni7atic~n' (IQK4: 95). \Wt> cotitrrd tor .In ~ n f i ~ i i n t~ uq m b e r of plsnciblr~ rll-al h v p o t h r s r ~ rr*ithour S J I P T E S I ~7l*lrai ~ P ~ C R ) Iw of t l i i 7 1 1 l nrib(Camphell, 1'W. Thiq 15 thi* npproach takcn in d r i ~ e t r i ~ l s .Individual4 are randomlv n s s i ~ ~ cto ~ ionc i of severaI Eroups and cach g r t l r ~ p is thcn R ~ V P R ; 1 d~ffert>n! drug. Sincc the g r o u p shoi~lcih,lvc v l r h ~ n l l yidentical profi1r.s tu hegin WI th, any difftlrencps in outcnmes bcttvecn the groups should be Jut1 trr the d~ftcrcntt r ~ ~ i h n e n ndrninistcred ts to rach group. kkI(>~'t'vt'r, this approach rs r ~ f t c n not n p p l i r a b ! ~in social res~nrch hefause practical and cthicrrl considerations prr-cludc 11.; a r s i ~ m i n gprr3plc !I> groups and thcn dc~ing sontetlring tn one p o u p to sec what effect i t ha< (wc Chaptcr 5). 4~lrantagc.;and dra1vback5 of thiq appronch tn social rtsearch nre d i q c u ~ s e di n C'haptrrs 4 and . '

nInq of

I f ' l ~ i Irandrmiz,itit,n t nlinirnileq thr chancesuf initial rli tfercncw Ilettct.cn Errwtys t h ~ r e rc stlF1 t h r clinnc'~ that thrrc u-ill he d~ffcrtnnct*~ ~~~~~~~~~n the qnluys - t.+pt,ci~lly in .;rn;lllcr RrrlrlFq 12:h~.re tt-nnt to Ilr certarn thnt tlw ~ r o i ~ art. p s cc~rnpar,?ble nn .I partjcular chnractrrrstic ~ c can c take an additional \ t ~ p to plnranlcc t l i ~ the t gn,ups arc ctlrnpamblt* a t least on q-rccific \7L~rialllc*. 'I his i< achtrvprl zirllh u har i< called the randcllniscrl block dt,5tKn - ,I ctunbinatirm t>f h(lt11 rnr~ijomi;r,~iicrii and rnatchitlj: 511pposcwr- wanted to c ~ ~ , l I i ~ thr ate ~ F f c ' t011 a r a d ~ m i c perfornianctnof thrrta nirtiro~-!'; r + F t l r l r v e r ~ n ~rnix*rrsitv ~ courstxs: (I ) d(~livcr~.

46

WHAT IS RESEARCH DESIGN?

CAUSATION AND THE LOGIC OF RESEARCI F DESIGN

47

of lecturcs and tutorials, (2) correspondence using written materials and (3) electronically using t h intcmet ~ and e-mail. Further, suppose that we could assign individuals to one of these three modcs. Random allcratinn would be the obvious way. But suppose we want to be absolutely certain that the students in each of the three modes are of comparable academic

1
I

abiIity. To achieve this we war~ld do the following:

1 Obtain a measure of academic ability at the pretest. Rank all the students from highest to lowest ability. 3 Select the three students with the highest abilitv (because we have three learning modcs). This group of tl-rrcc students is block 1. 4 Randomly allocate each of the three students to a group (one to face-to-face; one to written correspondence; and one to elcckonic
2

Throughout this chapter I llarc used the terms 'intervcniion', 'treatment', 'independent variable' and 'groups' more or tess interchangeably. Research designs vary in terms of the type of independent variable employed and in the number of independent variables built into the study.

learning}. 5 Select the next three most able students (block 2) and repeat the random allocation to groups. h Repeat this until all students have been allocated to ont? of the three groups.

I
I

An alternative way of making groups comparable is to do so at thc data analysis stage after data have been collecfed. It involves multivariate analysis that, in one respect, matches woups on specified variables (see Chapter 12 for a discussion of the logic of some nf these techniques). Although the procedures of multivariate analysis can be very complex, the essential logic is simple. Suppose we want to compare the emotional adjustmcnt of childrcn from divorced and intact families but we believc that any comparison would be confounded by the fact that, on average, children whose parents divorce are older than those from intact families. h y greater maladjustment amnng children xvhnse parents have ' divorced cnuld occur because they arc older, a s older children display ' > greater maladjustment. Multivariate analysis rcmoves any effects that might be due to age by comparing like with like - by selecting, say, preschoolers and comparing those from divorced dnd intact farnilics tn see if, despite similarity of age, the children show different levels nf adjustmcnt. The same comparisons could be repeated among say 5-8 year olds, 9-12 year olds, 13-15 year olcls and so on. The obvious shortcomings of this approach are similar to that of marching. We can remove the influence of variables t h a t we have thought of and on which we have data but we cannot remcwe the cffccb of unknown variables, or thwe for which we have no data. Since we can never know what factors we have missed thcre is always the danger that factors we have not thought of may be contributing to group differences.

Whcn thinlung i n causal terms (X+Y) X is thc independent variable. When we conduct the research and do the analysis we compare the outcomes (Y) for different groups. The groups are defined according to which category of the independent variable they belong to. In some studies the investigator introducing an active inferv~ntiondefines the independent variable (e.g. allocate different people to one of three modes of course delivery - face-to-face, correspondellce or electrnnic). In other situaticms the indepcndent variable is d e f i e d by s nnfurnlly c~cc;rrrin~y ,. ~~~ferzlenfin (e.8. n ' a person retires or remains employed). In other situa tiuns the indepcndcn t variable is determined by the rclativcly j x e d nitrib~ites of participants (e.g. sex, education, race etc.) rather than by intttrvc-tions o f any snrt.

R~search designs can also differ in terms of the number of interventions that are madc. Where the independent variable involves either active or a passive intenrention we can examine the effect of single versus repeated interventions. We can see whether a singlc intervention has a different impact than the cumulative impact of repeated interventions. For example, we might be evaluating the impact of teachers giving students ncgative feedback on their work. Initial criticism of the work may boost a student's effort and performance but repeated negative feedback may lower performance as the student loses confidence. A multiple intervention design helps refine our understanding of t h way ~ in which interventions affect participants and enables us to rule out alternative rva ys of interpreting results.

Qimensions of

research design

The abovc discussion has ida~tifiedsix core elerncnts uf a research design. The particular mix of the elements in any study will yield different designs. Thcsc six main elements in producing a research design are:

48

WHAT IS RESEARCH DESIGN?

CAUSATION hVD THE LLKIC Of: RESEARCI I UFSIGN

49

T k r rr11tr~br7r of pn~rrpsi l r tlrr desi,y~r Designs will vary from those with no curnparisons ( r . ~single . case d c s i p oi casc study) to those wit11 many different cornparison groups. T/rrb nunrbf>r of 'prr>-tr,5t' r n i ~ a ~ r r ~plrnsrs ~ ? ~ ~ r Designs )~/ vary from those with no 'prr-test' (e.g. cross-scctlonal designs some experimental dcsi,ps) to thuse with a series of 'pre-tests' which cstablish prcexisting trends before an event. Tlrr ~17117rber. qT 'post-t~st' mcastirewrmt phnses All designs s q u i r e at least one 'post-test' - the measurement of an outcome variable. In some designs (e.g. cross-vctional) there will be onf 'post-test', while otIl~r designs can have many post-test? to help distinguish between short and long term outcomes n r e ~~~rih if onllncfltiun d of c n s ~ 5fn grnlrfls In multiple group designs groups call bc made comparable by allocating people to diffe~ent groups by random allocation, 'pre-tcst' matching, post hoc matching or block matching or by sing statistical controls in the analysis phase. T l i p nat~lrPof t l z ~ rr~frr:~rrttiunStudies that rely on existing variation variables) (cross-sectional designs and tlwse wit11 'fixed' ~ndependent I~ave 1-r~ in tc.r\-cntion5. Other designs rely un intcrventions between a prc-test and a post-test. Thcsc 'interventions' may be either active or ~iah~ral.' T!rc t ~ l ~ r r l h tI$ ~ t '~~ ~ ~ ~ ~ ~ ~ ~ vDesigns L ~ I I o with ~ T s an intervention can have t.it11c.r a single ~ntrrvention os multiple interventions. Multiple interventions can bc used to identify thc effect of cumulative 'treatments'.

Method of allocation to
groups

Random

outcome varrable
(experimental

outcome vanable

outcorns varrabie

Measure an outcome var~able

(control group)

3 Random allocation to the groups brforc thc prc-tcst. 4 One inter\ ention (test/ treatment). 5 One post-intervcntir~n(pust-test) measure on the oufcomc variable.

A range of research designs

The carnbinatiim o f the possibilities created by these six elements of a research design yields a large number of possible designs. A useful summary discussion of many of these designs is provided by Spector (1981).To impose some order on this range of possibilities it is helpful to think in terms of four broad types of design. For each design type, decisions taken by the investigator will produce variations within the type. These four types providp the structure for the rest of t11c book. The four brorlJ lvpes of design are expcriment~l, longitudinal, cross-sectional ,jnd cast qtudy. 1 zvill examine each uf thcsc in turn.

This d e s i p is illustrated in Figurc 3.8. The analysis of any effect of the intervention focuses on changes in the experimental group before and after the interventio~~ and a comparison with the rate of change in the control group. If the change is greater in thc cxperimcntal group than in tlie control pinup the researcher will attribute this to the impact of the independent variabIe (the treatment).

The basic form of this design involves:

One group. One pre-'intrrven tion' moasurrment on the nutcornc variable. 3 Onc 'intervention' wl~ere everynnc ruceivcs the 'heatmen t'. 4 One post-'intervention' mensurement on the outcome variable.
2

The class~c version c ~ the f t.xperimenta1 dcsjgn has the following elements:

1 Onc pre-inter'! ention (prc-test) measure (In the outcome variable. 2 Two groups: one gmup that i s rxposcd to the intervention (the expenmcntal group) ~ 1 1 nne d group that is not e x p s e d to the intervcnlion (the control group).

I n uffcct, this dcsign is similar to the experimental design cxccpt that there is no control grnup and typically only one 'experimental' group. The design is ~llustrated in Figure 3.9. The analysis in this design cornparcs the pre-intervention measures with tlie post-intervention measures. Change in these scores ran reflect the influence of X on Y. Mowe~rer,the nhscnce of a randomized control

50

WHAT IS RESEARCH DESIGN?

CAUSATION A N D THE LCK;TC OF RFSEARCH DESTGN

51

Method of allocation to groups

1
11

I
11

Pre-test

Infervention

Post-test

Method ol allocation to groups

----:

Pre-test

Intervention

Post-test

(XI

Random

'Treatment'

Measure on Outcorns variable

(one group)

--

"on-random

None

No 'Ireatmen

Meaon outcome varlable

group makes it difficult to know whethcr the intervention or some other factor prcxluces any change.

-. . . . . --

TIlc basic elements of thc cross-sectional d e s i p arc as lallows:

1 Instead of interventions the cross-sectic?nal design relies on existing variations in the independent variable(s) in the sample. 2 At ledst one independent variable with at least two categories is

present. 3 Data are collected at une point of time, 4 There is n o random allocation to 'groups'.
This design mirrors the post-intervention phase of the classic cxperi-

mental design but without any random allocation to 'groups' being made. The data for this design are collected at one point of time and are analysed by examining the extent to which variation in the outcome variabIe is linked with group differences. That is, to what extent do those in diffcrcnt categories of the independent variable differ in reIation to the outcome variable? Causal relationships are established by utilizing statistical controls rather than by random aIlncatinn of people to groups. This design is illustrated in Figure 3.10. T n this casc thc 'inten~cntion'is simply being in a different category of the independent variable.

Case study designs might consist of n single cd5e study (e.g. a community study, a study of an organization) or a series of c a w 5hrdit.s with perhaps cach cnsc testing a theory from a difftxnt angle. It is useful to think of a case study in a similar way to an experiment. We do not finally reject or accept a theory on the basis of a single experiment; we try tw replicate an experiment and conduct it under a varicty of conditions. Similarly, a case strldy project that entails a single case study is analogo~oustn a single experiment. If similar results are found in repented case studies, or predictable differences in results are found for particular cases in thc study, then we develop greater confidence in the findings of the cases in the same way that we gain confidence in experimental rcsults that are found in repeated experiments.

Summary

Case study designs rely less on comparing cases than on exhaustive analysis of individual cases and then on comparing cases. A distinguishing characteristic nf case studies is h a t contcxtual information is cnllected ahout a case so that we have a context withiq which to understand causal processes.

This chapter has ~ x a m i n r d ways of strircturing rcscnrch designs to help draw convincing causal inferences from the research. Since causes cannot be 01~t)rx ed they must bc infcrred from obw-vations. However, rncorrtct Inferences can casilv he made. The chapter has considered ways of structuring research to improve the quality of these infercnccs. A rangc of ways of interpreting correlations between variables was orrtlined and criteria required for inferring that a correIation reflects i l causal connection were provided. Tl~c chapter emphasized the importance of making meaningful cornpnrisons behveen groups as a core clcrn~nt of drawing causal inferences.

W H A T tS RESEARCH DESTGhr?

Onc of the tasks of research design is to structure the research so that meaningfu! cnmparisons of outcomes between groups can bc made. Finally, tlie chapter outlined s i x core elcrncnts of research designs: nurnbcr of gror~ps, number of pre-tests, number of post-tcsb, nature nf allocation to groups, type of interventions and number of interventions. vary in the way these eIerncnts are dealt with. Four Research dts~gns main types of research design - experimental, longitudinal, crosssectional and case study - were then briefly described. These broad categories of design provide the framework for the remainder of this book.

PART I T
I

EXPERIMENTAL DESIGNS

Notes

TYPES OF EXPERIMENTAL DESIGN

I

1 R c r c can be exceptions to thi5 where a suppressor variable may he operating to mask a causal relatinnship- See I I o s ~ i ~ b ~ (1968: r g Chaptcr 4 ) . 2 We sl~all see later that this pmhtem of Ioss uf cases can be overcome with cprtarn fcorrns of multivarlatc analysis. 3 A nalurnIly o c c u ~ ring ~nlervcntion1 s an event. llint takes place b~twccn pretust and pocl-ic.;t w ~ t l ~ to u tl-te invcsiigd tor initialing a n y intervention. An example mtght be a study of fnrn~l~es over tilnc In which wrne fam~lies cxperience parental divorce. Divorvc is the 'natural intr wcntlon'.

1
I
I This ch(1ptc.r builds on the elements uf rcscarch dcsign outlined in Chapter 3 to intrt~duccn variety of types of experimental designs. It.
I

outhncs thc diffwen t environm~ntsin which experiments can be conducted and describes both simple and more complex variations of experimental designs. This chapter does not seek to evaluate experimental designs: that is the task of Chaptcr 5. The discussion also providcc; a useful framework within which to undcrstand some of the strengths and weaknesses of thc research designs discussed later in this kmok.

The classic experimental design

The classic experimental design focuses on two variables: the independent variable (the cause/intervention) and the dependent variable (outcome). 'll-rcpurpose of the design is tn remove the influence of other ~~ariablcs 50 that the effect of the intervention can be clearly seen. Since the cIassic cxperimmtal dcsign was outlincd in the previous chaptcr thew is n o need tn repeat i t here. I fowever, an example will help cIarify its elements. Suppose we wanted to test the proposition in Figure 4.1, narnply thr' I H I I T P diffic~llt if is to joitt n gro~rptlrr more clcsirable tlre gronp will serrn.' In order tn test this proposition it is necessarv to devclop measures of desirability of membership and to dcfine what is meant by 'difficuTties nf joining'. The expression could refer to things such as restricted memberhip, long waiting periods, high fees, harsh initiation, Iligll time commitments. For the purpose of this exercise we will use 'severity of

Dilticultv of rornlng group

Perretvd

belong~ng to

Random atlocallon

(expenmental

rre-rest a v ~ r a g e desrrab0l.f~ .SC-P

'Severe

rnrt~atlon'

'Po5T-tr>r

Erhr~nqs= 2

- El

~vareuc

des~raol:,'y s:o?e

,
initi~tion'inkc) tlit, grnnp as thc indicator of diflicultv o F join in^. Tlie cla%%icexp~rimmtal design for such an invttstigation e n t a ~ l s ti111 i r j l low in^ qteps:
1

group)

Random
P ~ t e saVerRgP I

NOnttaton

C ,
Post-test averaop (f~~rr; I

Cchalrqe

'

-~

2 3
4

RandornEy n l l n c . ~ t i npeople ~ to t n w group: a n ctpertmental group and a contri~lgrnup, Ubta~ning rnen<tll.cs of prrccivrJ ijrsirability of gmup rncmbi.r<liip. Subjecting i h rrp~rimenta ~ l groir p to severp inittnlinn reqi~i rt*nicnts and the control group 5 0 ntl initi.~tic>n rcquircmtwts. Obtaining rntncurl- nf desirabtlitv of group nlcrnbcrqhip from both ?:mtrp aftcr thtb t n i t i a t ~ u n t * r tlic erpcri~ncnbal grouy ha5 I ~ < ~ ~ ~ I I ~ornplcted C'nEculrltin~the i.lrrr~;~r in dc~irnhilityof mcmbvrsh ip for thc r.~ijr,rir ~ r t , tnl r ~ gmtrp bcfort- a n d aftcr tlir initiation and nt the control gmup at T I and TZ. Comparing t t ~ Ic1.t.l r of c h a n in ~~ the experimcnta! group with that in Z ~ P control Ernup.

Experimental
inlt~atron'
I

Mean
I

I f chatigcs in the prrccived desirability o f rnembersliip arc diffi*rt*nf bctwc~li the ~ Y L firoltps I (MOTC' t11aii wot~ld be duc to chance) this should b r 1 bccnuw o n e grnilp had ecpcbricnced sevrrt. initiation. 'This is rrprtwnted in Figurt. 4.2. Here thc qvmbol TI reprtwnts time 1 or the yrp-tmt. E, reprvwnt.; the expctirn~ntalgroup at timp 1 and C, r ~ p rrqcnts the contrnl Krirup at time 1. Thr amount of cf~angc m the cxperirnpntal group rs symhol~zed as C,.h,lnF,y and 1s thc d ~ f f r r e n c e between Ihe grinip'~desirability score a t TI compared with T ,. 'l'he comparable change for the ctmtml group can bc ~ymbnlizeda < C, h,lnp,.. The ~ ' f f of ~ ~ t thc intervention ic the* difference 171,ttv~en the arntrunt 13t change In t l ~ e c*\yr*r~mc.nta! ernlzp bcfort. and aftclr the inter\.rn!tcli~( Cihanrc I A ~ C I~ I IC ~rno~ln rrtt c h a n p , In i he control qrotlp ( C h ,k , bc7t1vt.m 7, and T- l .P. T ' , I ~ . , ~~~ C,-l,Anl:, , ). I f t h i s i~ jirratcbr Ihnn zcro t11t'~t M'I, wnuld p r < d ~ ~ h l y cnnclucle that thc ~niti~ltion had a n i m p c - t on rlcb~rahtlitv. If the rncasurc of desirability was mcasureci qr~antitativcly on a scnlc vt, sav, 0-100 with a t ~ i ~ score h ~ n d i c a t ~ nhigh g c l c q ~ m b i l i w~ ~ , ctlulJ cj~tantif~. the e f f c c ~nf the intcn,rnt~on(st!-cr:tir o t initiation). Thi.; ir ~1Itwtr~itrd wlfli hvpothctical data In r - i p r e -2.3 Vking these fip1rr.k w r J ctln <t,rA Illat:

1
I

i
I

Control group

No initlation
MPT

Rcrth ~ r t + u phad < thi* .;arnta luvcl n i d t r ~ r , > l - ~ l ibefort% t\ thr initiation (tht>rr3cull o f ranclclnl allcr,~tion). 2 TIic dcqira bility sctlrcb nl the experrrncntal F O L I ~ incrca;lsrs (hy 25 point'). 3 Tht. rlt~\iwbilitv _scnrc of thc control srirtip a l w incrr,.iws (by 10 p"linIsl. 4 Thcrrftrrt, wme cllnn~t. in the c.upt-rrrnt.nt;ll grtirlp I < not due t r ~ initiahon ( I n points tztrrtli ~ i n c e thiq Ic-\.rl crcc~arq u,itllr~t~ initiatinnl. l

56

EXPFRIh4ENTAL DESIGNS

TYPES (IF EXPERIMENTAL DESIGh'

57

5 The change in thc cxpcrin~entalgroup is greater d ~ a n in the control group thy 1.5 points). 6 'l'herefilre scvcritv o f inihation leads on average to a 15-point increasc in percei\,ed desirability nf group membership.
The logic of this experimental design is based on two conditions:

Labnratorv cxpcriments have been used to great effect in tl-tc physical a n d biological sciences. I -lnwever, for a variety of methodological, practical and ethical rcastms their use in social research has been limited (scc

Chapter 5).

1 Thc groups are the same in all relevant respects before any intervention (TI). 2 The groups cxpcricncc the same conditions between TI and Tz except for the specific intervention/heatment.

I f either of these conditions is not met then there are a l f ~ r n a t i v plnusiblp ~ h,ypothcses that cnuld account for the different changes between the experimental and conlrol groups. The different 1~vels of change could he due to initial differences bctwcen the two groups rather than the severity of initiation. If the two groups differ in other experiences between TI and Tz ( p . g . mixing with diffcrcnt people in thc group) these differences rather tI1n11 sev~rity of initiatiun co111d account for the different rates of change in thc h ~ groups. u
Experilr~en tnl r-ontex!~

Experimental designs can be implemented in three different ways: in a laboratorv, in the field a11d by utilizing natural clccurrences.
LABORATORY EXPERIMENTS

The ldboratory experiment is designed to ensure that experimental and control groups are exposed to exactly the same environment except for the experimental intervention. This is achieved by standardizing and controlling the environment and all the events between TI and T2.Maximum control over the environment is designed to ensure that the ~ n l y feasible reas011 for group differences at Tz is the differential treatment. An example of a laboratory experiment is a study designed to test the effect of extremely graphic images of alcohol caused road accidents on vicws abnut drink-driving. Participants could be randomly nllncated to experimental and control groups and asked abnut their views regarding appropriate penalties for d ~ ~ n kriving. -d The experirncntal group could be shown a graphic film of alcohol causecl accidents. Members of the other g-ruup would nnt be shown thc film. At a specified time aftcr showing the fiIm both groups would again bc asked about their views regarding drink-driving penal ties. If the views of the experimental group changed more than the views of the control group i t wc~uuId bc reasonable to assume that viewing the film caused the cliangc. The laboratory setting mabltd the rescnrcher to control everything, so the, nn!y difference bctween the gi-oups was h e exposure or lack o f exposure to the film.

Because of the artificiality and impracticality of lalwra torv experiments, social scientists have established experimental designs in real world environments. In a field cxpcsiment the investigator creates cxpcrimental and control grnups through randomized allocatirm of participants. The experiment also involves a n interventir~n.The intervention is implcmented in the 'field' where a rcal life environment is trcated as the laboratory. This approach has become especially pnpular among those evaluating policy initiatives and pilot p r ~ g r a m m c s . ~ In order to illustrate the nature of ficld cxpcriments 1 will recount t11c major features of an American shdy of income maintenance among families of Inw socio-cconornic standing. The study is known as the %a ttlc /Denver Tncornr Maintenance Experiment (Hakim, 1986). Almnst 5000 low income families partic1patc.d in the experiment in thc 1970s. The purpow of t l ~ c study was to establiqh the effect r ~ thc. f introd~iction oi a guaranteed minimum income on the work effort and maritaE stability ilf participants. Families were randomly alltlca ted to diflcrcn t versions of the scheme (no income support, short ttrm support, throt~ghto long term guaranteed income). Among the many variables measurcd were work effort and marital stability. They were measured at the beginning of the scheme and then at various points throughout the experiment. While participating in the experiment these families went about their normal lives and were subject to a11 the varying experiences and additional changes (e.g, political dnd social changeq) that took place over the period of the study. I participated in a field experiment conducted by the Australian Department of Socia! Security (DSS). ?t was designed to see i f the participation of welfare recipients {DSS clients) in self-help groups increased their living standards. Various self-help groups were esta blished in 80 locations. Client5 of the BSS living in those locations could choose whether or not to participate in the groups. Extensive measures of living ytandards were obtained before participation and 12 months later. In each of the 80 locations a random sample of DSS rEients who did not about their living participate in the self-help groups were also qu~stioncd standards. Thcsc people acted as a control or comparison group. Clearly this was an attempt tc) implement thc l o p c of the experimrntal dcsign in a ficld situation. Ilowever, the design had a serious flaw. Participants were not randomly allocated to groups, so wc d o not know whether those who participated in the self-help groups wvre initially similar to the non-participants. In addition u7e d o not know whether

EXPERlMENTAL DESIGNS

TYPES OF EXPERIMEN'TAL DESIGN

participants and tlnn-participants wcre exposed to comparabfe influcnces over thc 12-month period. We observed that self-help group participants dici b e t t ~ over r the 12 months compared to mcrnbcrs of the cnntrol Emup. However, wc could nnt be sure whcther this was due to participation in the self-help groups nr whckher thc sorts of people whu volunteered to participate were more self-help oriented prior to the cornrnr.ncement of I ~ cxperimen F t. These initial differences might have accounted for h e i r greater improvement in living standards. While laboratory experiments suffer from problems of artificiality, field experiments encounter problems horn inadequatc control of events between T , and T2. Thurr is always the danger that uncontrrrlled events rather than the experimental intervention are producing any obscrr.cd
changes.

Howevcr, other factors (i.e, uncontrolled events) should nnt be a major prob!em if hofll thp exprrimentnl a ~ llzc d cntllrol groups ore eq~ially ~xpusedEO tlrrsc ollrrr influenres. Total change in the experimental group between TI and TZ consists of two efements: that produced by the intervention and that produced by other intervening events.' In the control p u p , total change n d l be due entirety to the othcr intervening events. I f the same intervening events apply equally to both groups N e can shll jdcnlity experimental change by lr~okingat the diffcre?~cc~ in the total change between the two groups. L n the carlier examplc (severity of initiation) the total change in the experimental group was 25 points compared to 10 points in the control group. The difference in total change between the two groups was 15 points - the experimental effect. We do run into problems, however, when any of these external factors influenctl one group more than the other. This can occur in at least two ways:

I One group may be selectively exposed to additional factors more than the other group. 2 The external factors may infmgct with [he intervention. That. is, when a person is cxposed to both the experimental intervention lrnd the external factnr this ro~nhi~nztion has a unique effect that neither the experimental nor the ext~rnal factur has on i t s own. An example of an interaction effect is evident in the treatrncnt nf depression. Drug treatment on its nwn mav have some limited effect on dcprcssion, and psychotherapy on ~ t own s may have snme effcct But when the two apprt~achcs are used together they can have a dramatic impact on depression. The conibincd impact is much greater than the simple cumulativ~ impact of each approach: it is the mir that has the most effect.

I f WP can antic~pate that an c-r~cnt will occur we can collect information before and after to monitor its impact. To help intcrprct thr rcsults we ct~uldtrv to identify a comparison group that will not be exposed to thc event. A natural experiment is well suited to s t t ~ d y i nthe ~ impact of Ic~alizing pokcr machines on family wellbeing. For such an experiment we would obtain mcasurcs of family wellbeing before and after the legalization of poker machines. We w t ~ u l d obtain a comparison group by measuring family wellbeing in a comparable regon, state or country in wh~ch poker machines remained illegal. Similarly, a nahlral experiment is well suited to examining tlic impact of the introduction of no-fault divorce on divorce rates. We would examine the divorce mtes in a country before and after the introduction of no-fault divorce and compare the rate of change with a cnmparable country without no-fault divorce. Annther form of natural experiment can be achieved by constructing experimcntal and control groups (Per the 'intervention' has occurred. Let us assume, for examplc, that wc want tr) establiqh how women's return to the workforce impacts on the division of housework and child-care between marriagc ppartners. We would collect baseline information about the d ~ ~ r n e s tdivisicm ic of lnbnur from couples where khe wife is not in the paid workforce. We could rehlrn to the same sample sc'17cral years lntcr anticipating that some wives will have returned to the workforce. These could be treated as thc exp~rimcntal group while tllose couples where the wifc remained out of the labour force would constitute the cnntrol group. Changes in who did what around the home would be compiired in the two groups to assess the impact of the return to the workforce. The difficulty here is that the groups are self-forming rather than being created by random allocation. We might be able to match repons, countries or farnilics to some extent but matching has its obvious limits (see Chapter 3). These limits mean that the groups we are comparing may be dissimilar in important ways. This necessarily compromises om ability to s a y exactly what is producing any change or any differences between the groups. In the gambling example the problem will be finding a truly comparable region in which gambling remains illegal; with divorce the problem will be finding truly comparable countries. The housezvork examplc would encounter the problem that the women returning to work may bc different from those staying at home.

Simpler experimental designs

In m a n y circumstances wc cannot obtain all the measurements required for thc classic cxpcrimental design (pre-test and post-tcst fur both the control and the experimental grnups). In some circumstances valid inferences can be drawn without a 7 1 these measurement po~nts.

The natural experiment relies on naturally occilrring cvcnts as interI ventions rather than on controlled cxperimenter-induccd in tervm tinns.

TYPES C1F LXPERIMENTAL DESIGN

-

61

- -

Method of allocation to groups

I
1

Tlrnel(i,) Pre-test

Inletventton

Time 2 (T,)
Post-test allacat~on to groups

T~me 1 (TI) Pre-test

(XI
-

I ntervenfion (X)
- .-

T~me 2 (T,)

Post-test

Random allocatton

None

'Treatment'

1,

outcome vanable

Retrospect~ve matching (experimental group?

MeasurB on outcome var~able

'Treatment'

Measure on outcome variable

(experimental

Random allocation (control group)

None

No 'treatment'

Measure on outcome var~able

Retrospective matching (control group)
--

Measure on outcome v anabls

.

No 'treatment'

..

I I

Measr~re on outcome variable

Figurc 4 4

Post-t~stonly rsprvrrnrntnl tf~sign uuth n corrtrol Xmup

This J c s ~ g n is illustrated in Figure 4.4. So long as groups are large enough, random allocation to cxpcrimental and control groups means that any diffcrcnct.~ between expesimontal and control groups a r e random and will not account for group ciiffesences in outcr~mes. Therefore, where we fi~rm groups using randon-1 allocation, the pre-test measurement p h a s e can be ornittcd. Camphell and Stanlcy argue that:
While the pre-test is a concept deeply cinbcdded in the thinking of research workers . it is not actualiy essential trr true experim~ntaldesigns. For psychological reasons it is difficult to give up 'knuwing for sure' that the . . ~ ~ o u were p s 'equal' brfore the differential cxpcrimental treatment. Nonetheless, the most adequate all-purpose assurance of lack of initial biases behuceti groups is randomization. (1 963: 25)

groups are not Formed through random alli~ation any differences hetween thp groups cou Id simply reflect 'pre-intervention' differences.
This design can be improved in two ways using retrospcctivc data (Figure 4.5). 7he two groups could be nmtch~dso that t h e y are comparable in a t least some rclcvant ways. Some of the information for matching wnuld relate to past events. This mfomation would have to be collected by asking about these past events. The design can also be improved hy refining the measurement of c/ia~z~ye on thc outcome variable. If we simply have 'after' measures we cannot see whether, following the 'intervention', one group has changed more than t h e other. To measurc change we could obtain 'before' mcasures retrospectively and then assess the amount of change. So even if the two groups were different to start with we can see whether the amount of change in one group IS grrater than in the other. For example, we might want to see whether having children encourages p c o p l ~ so be more politically conscnrative."To achieve this md we could compare the level of political conservatism nf a group o f familits w ~ t hchildren and a group without children. Since any differences between groups could be d u e to factors other than parenthood w e would cc~lfcct ntlier infornmt~cm abnrlt the familits and match the two groups so that we compare like with like. However, we still cannot be surc that having children has produced changes in political views. To test this prnposi tion we could ask participants to describe their political vicws a t some earlier point before they had children. This stratcgy provide.; us with a pseudo 'before* measure. We could use this measure to assess whether parents had changed any differently from thc nun-parents.

When this design is utilized analysis cannot be undertaken by examining t s r f l n l i p between ~ the groups. Rather, analysis focuses different a m r ~ r ~ nof on post-tcst differences between group?. Because participants have bccn randomly alloca tcd to their groups, post-test di ffcrcnccs (In the outcome measure should bc thc same a s thc difference in change scores of ehpcrimental and control groups in the classic experimental group.

Thiq design entails constructing expcrimen tal and contr?I groups @el* an ' i n t e r v e n h o n ' (see earlier dacussion of natural expcrimcnts). Since such

qroups

"'
1
I

2 li , l

Time 3 (T,i Tine 4 / T , /

'

Post-IF.,

Pos!-tesf 2
I

POSI-test J

Random allmatron
(expenmentaf

Mrsstrr~

Mealuro le on ou~cone

group)
I

I
.. . .,. , ,.. . - , .<. , L : u , r , v r , ~

vnr~abla
iYj

, Random

I:

allocat~on
icontro

. >... . , .;

'Treatment'

(!
11

, QrouP)

nlultiple variation.;. Thcrc may L ~ P qc\cr'~l* ~ l t r r n a t i ~ intcrvrntlon.: t. to t%rallld tc. For euarnplc., t\.r may be intrres'led in t h cffcct ~ ot compuI<r)rv votinc on electoral turnout 1Yc might con<idi*r tivc tcmditions (1) c o r n ~ l r t ~ b v voluntary vnbin~,(2) compulsion without sanctions for non-compliancc, (3) compulsion with a warning only for t 1 1 r first offence, (4) compulsion with a fine of $50 for non-compliance, ( 5 ) compulsion with a fine of $250 for non-complrancc. hftcr randomlv assigning individuals to one o f f i ~ c grnupswe could ask about their intention t o vote in the next election. i\sr;uming that voluntary voting is the qtatlrs quo, four of thc fivc ~rt,upq will be told that voting is gokg to be made compulsor!+ a t tlrc n r x t rlrction. Each Rroup is to be told a diffprcn1 khing about the ~anctiuns that will he i r n p t ~ ~ for d non-voting. The control gsorip will tlclicvi, !hat voting will remain voluntary. After bcing ~ i v e n this ~nformationprtlpIc in each group will hc nskcd about thcir voting intention. Diffprcnc~s in thc T? voting intrntions of the five groups s h o ~ ~ reflect ld thr c f f t ~ of t ct~rnpulsion kin thc .;tatid intention to vote. T h i s & s i p 1 s d i a p a m m r d in F1jiure 4.7'.

More complex cxperi mental designs Wc can add to thr classical experimental cfc5ign by i n b o d u c i n ~more than two grnilp<,c l d ~ l i additional n~ prc-te~tand post-test rneasrr rcs a n d tncflzding morc than one independent variable.

Since the effects of an intervention might not show u p imrnediatcly or might only last for a short time, multiplr p < ~ ~ t - t e can s t s help idrntify the short and longcr trrm outcomes. Fnr ruarnplc, divorce law rrform designed to simp1i fy divorce may prod~tcc a sharp increase in divorces initially (pent-up dc~rnand,formalizatinn of farnil? breakdown) hut over the longer term thr divorce rate may d r ~ l i n and r skabili7e t o prc-reform lev~ls. . 1 , multiple pt+xt-tcqtcrperimenial dc\ign i* represented in F~!:urt*4.6. 'The qame dcqign could be modificd tn l>ro\'idc rnulti~lc~ ~ n x - t r ~ .to ~t.: rstimatc the evt~lnt20 which c h a n ~ c wmir 1~1king place brfc~rc.f l l r ~ntcrvention to 5cc In whnl rvtrnt the ~ntcrvt~ntion arcclcrat~sor alclws the 1.a tc of change.

Sn far 1 have onlv d t w r i b c d des~cns ~vhrrr the independent i~ariahlr nr

intcmcntion ha.; twib ra t~eorirs. Hoivc\.~lr, in tcnmen tiunk frc=qum t lv Iiavc=

One problem whcn d ~ s i g n i n g e~perim~nt:: which have befnrc and a f t ~ r rncnsurement phases i~ that making rn~aaurtments can producp chnngc on its own, regard lrss of anv int~mention. Thiq is the problem elf r!rstrrrrrrrnt rt.nctiz~ier.To thc clxtent that change rnnv bc attributed to thr ctfcct nf measuring rather than the cxpcr~rncntnl 'tr~atrnent', the rt..;ultq will rcmain arnbiguou~. The Solomon four-gmup d e i p helps rvalrla tc whcther changc in the outcome variablr is due to instrument rcai.tivit?; {Figure 4.8). In this cxarnple I describe this dcs~g-n for an i n d c p n d r n t variable with only two categories, b u t t l ~ logic r of the d ~ s i g n could casiky be extended to indeor conditions. pendent variable$ with more than two cntrgor~es Using this d r . s i v the experimental gTOLtF is split into W o scrbgrnups, as is the control group. &e experimental Rroilp im.o!ves a yrc-tcst ~%'hlie the other docs not. VIP same conditions npply to the ccmtrtd group. I f thcre is no instnlni~n t rcach.rrlty then t l i ~hvn cupenmental Ertrup:: 41uuld scnrc identically on the outct,mc mtwasurrs a t TZ,a5 5lloul~d~FIc' two control group.;. Any difference brtwcrn the T2 scorcs of thtl 1 ~ 1 1 ~ \ p ~ r ~ m e ngro~ips taI or of the two C O I I ~ ~ O I groups S ~ O U ~ he C n ~ ttrib~~t,~blc to instrument rcrlctivtty. The example In FEcurr 4.9 illustrates thiq dp.;ign 11singf i p ~ r r s For . tlie cake of the cxarnple thr outcome vari.tblt* 7.; measured on a ~ c ~r l f~ 11l r 1L10. In the ewperirncntal group3 the pw-tt~t ~ri~n obtained p ; I wnrr i j t compared wlth n sccrrc ot 7 1 1 for t h e e x p e r i m m t ~gmup l n-ithout tl~c prctcst. lf we r a n assump Ithat tlww werv n o i3tIi~~r d~ffcrcnccs bchvccn the

(14

IIXPERIMEN'l'Al, DESIGNS

TY13E5 OF' EXPFRTMFN'l'Al DESIGN

rM~vhod of allocation lo groups

I

I -

T lme 1 l l . )

Intervenllon

'

Tlme 2 ( f,l Post-tesl

Intervent~on

Time 2 (Tzh

1X)
Pre-test

Random aflocat~on
twperlmentaf group 1 I

O.

~nt~ndmnn

Compuls~on wrthout sanctrons

-- " ; lntendim

Post-test

Random atlocat~on (e~penmenlal gfoupl

Mertqurn oo

o u t ~ n w v;ir * :bin
I Y I

'Treatment'

MPRSU~P on
nil!

Fe

I
I

Sb inter
I0 VC

Random allocat~on
I

Nnn

'Treatment'

(exper~mental qroup 21

I
"'a inter

1
1 1

Measure on outclme vsrbabk

(Yi

(exoeivnanfal group)

to
VC

Random atlocatron {expsrlmental group 3)

*-I

1
I

Cornpuls~on wllh modest frne as sanctron

(control group)

) out~~nin
I

IYI

- -

Ranrlnm ~llocahon

". ntclr

CornpvTsion wlth

", ~nt~n&ntl
I

--

Pan-lo~ n!lr,r7tlon

, AS,

, I

Nr! 'fr~atrnent'

b . l n x l ~ r - 02

(con110qmup)
--

- -

Random allocatron
(control group)

Figure 4.X
I
-

Solr)~trclrrhrrr-qro~q' ,lr.;rtv~

-Figure* 4.7
1 7 ~ -l13\t 4

I.
t?lf~ll~~ 6)r l lr' o ~ da11d p ~ nric' !rrrb-lest

E,Y~IPY~II rEr:<i(p~ IPII/I with I!

nnd

ottr

behvccn

measurp

clliityc.

thrl two groups and we may wish to cond~rct prc-tests to In such c a s w nrc may ntx=rl this more cnrnplcx design.

twn ~rtwtpqthen thc rfiscrcpancy of 10 points bctxwn tlrc hvo ~ 0 1 1 pi s q attril~utalde~ C Itlitl c f f 4 ~ ~of t t h ~ onlv nther difference hetween thc. two groups - being cxposrd tn a prc-tmt. The same logic can be applied to

the hvn control grniips. Thtl rtvl effect c ~ thr f intcrven tion would be the d i f f ~ r ~ n behvc.cn cc th~. c\pllri nlental qrntlp , ~ n d the contrrd ~ m u p tvith thc ~ f t t - c of t inshumi3nt rcacti\.~h rcmtl~'ctl. I hi< ~\.nuld ~ T V II - Ii l ~ Score of 70 for ~ I I C e ~ p e r ~ m r n t , a ! gro~ip a n d a sctjrc t j ( 513 for the c o n t r ~ dgroup. W r v,'r~zllJ conclurlc that thc ~nt{\i-\,e~~tirm h . ~ t Ian cttect of 20 poillt~. Yiw might ask, whv htrthcr using \11ch n complicatt~~i d e q i p kc) remove thc rfft,ct of pre-tt-\trn~' Could wc3not hnvc achievnd the same outcomrl b ~5, i n ~ pd I~ ropp~n tlw ~ pre-test altogctllcr and ct~mparingthe T: Gcnrih.; nt J --ir~cEt. ~ x r c r i m t - r ~ tg nr l ~)~z ip i~~ CId single ctrntrnl > ; r n ~ i p t\'e ~ ~vnutd 1ia1.r arl-It.cd at e u r i ly the snmr anslztrr Flvrvcvrr, w h w c ha^^ .;mall .~ . tln .~ f f ~ r c ~ i i v r ~ Erolly.: mndomi~atirmmay be inruffir-itwt to rernnvr 5 ~n I ll

AFI the r~p~v-imental deqigns so far have dralt with just one independent variable. Elnwever, wc will nftpn want to cvnluate the effects of a number of independent variahlps. W r could, i f wc had the timp and r~sourcps, contluct a separate evpcrin~rnt for tach independent variable and !-ropefulT~ rt.rrrk out rvhic11 inrl~pendenti ,~r~nl.rlc had t h t ~rcatcst effect But +cpar<~tc' cuperirncnl.; ~ v u u l dbe both ~nt-fiicienta n d impractical. Vnre irnycwtantlv, separ,>tp ~ ~ u p r r i m e would nt~ not allow 11% t u w e hoxt' thc vatlorl5 inclcpendpnt varinhlcs t\.cv-k tr~):kbilicrIn particular minbinalions to p ~ o d ~ un1quc1 s c ~ cffractq. An i n d ~ p c n d e n t variable can have an cffect on it< own (called marn ct'fects or d~rtacteffects) or in ctrmbrnation ~vitfo i thrr r~;lri,ihfs (~ntcr~ctin effects). n :\ ni~lrn effect !~,IIc~I, r e ~ a r d Icss u t an\-!!jinq else, tlw \,nrj;lblr afttact\ the irutcomc. An tntrtaction effect crccurq when the effrct of nnc inclcprnrlrni variable i s at Icast partl!* c o n h g r n t on c~therhctorx also being prrsrant,

i
J

Method of ,o groups

allocation

J
l l ; ; ;-; ; t T

- -

Control (XI )

1ntey;rlo
Post-fes~

High la\

l o w lbl

Random allocation # I Mwn-5rl (experimental

-.
II

Treatmen

Male (a) exper~m~ntal qroups 1 s 10 IRO mlnuc 7 0 ) Thrs IS ~nnllut7hle lo lnstrumenr rrnrtlvtty Gender

Males

Mates

I
I

I
Randsm allaal~on

(Xzl

(expenmental !3rou?J)

II
I
I

*Inm

Treafrnenl'

Mean=iO

Random allocatron (control group)

Moan-50

No 'treatment'

Mean=60

D~fference b~!wc.en
control group< IS 10 (60 mmus 50) J h ~ s
1

rs attr~hulahla lo
tnslrument r~ac11vlb

Random allccation
[control group)

No Treatmen1

M~an=50 I

i
I

For cxarnplc, J lctd~ol. on its own, nffc-crk mood. On their cr\trn, .myhetarnines afsi) affect mnnd. The effect.; arc ninln ~ffcts: Ithey cnch Iiavc an effect a n tllcrr own lxut the rnixturp (>f ;~lcc>hol and arnphctarninc.; can have unique efkcts .ibovc and b e ~ o n d cithi-r alcnhol or ampht~tamines independentty. This will be more than juqt a n additive effect: it will hc a uniqu~ effect that i.; only produced by tnixiny: tile twn drugs. Sim~larlv, giving workers nlortb contrcbl over t h c ~ r wtirk might improve wnrk ati islaction for cvcrvnnc (main effect) h t it rni~l11have rmlrr. cffcct nn particular type.; of p w p t e such as pmlr.;siclnslq or older pcr~plr{lntcraction effcc!). VIE'~ U T L W t ~ tf.lctt~rr;h euperinit*ntal drstgns is simultant~rru\lt. to tl\.~rnine t h i ~ lirv~~ k'ffect5 t trnaln citvcts) ot a nl~rnherut indcptlntlt1nt vnriahles and tcr b t r * how various c.c>mb~n,ition< OF charactcrrrtit:< wclrk togcthcr to product. ,In effect Thc sinlplrst vrbrsiun of thiq design I.; calIt~d tht. 2 x 2 factor1.1l dzlsi~n, thdt is one b s ~ c on i twn indepcni4tlnt \.arlablcs cnch with iuqt twrn cat(-xclrics. Thiq st~nplr.? Y 2 d e s l ~ can t ~ . i ~ i be l\~ cttcnded tn mrirta than t ~ v o variablcc. cnch (?I which might I l n v v trvr, t,r marc cate~oricz rir conditions. The simyle 2 x 2 d c s i ~ n can be ilt~r.;tratrd bcqt with an cu,lmplr* in whlch ~ v c htir.c htrn i n d ~ ~ e n d c v na t r~~l,lc atill < nne deikndcnt r.;lri;lhlc

(Figtrrc 4.10). TIIPdcytlndt.nt variable (1')i < jnh snti<facbi*n r n ~ a * l ~ r1311 t~~l uvith 100 indicahnq vrrIr hrzh sati<f,~clion. 13nc i n d ~ p e n d m wriabl~k t 1.; h ~ r e o cf control o~.t?s (me'<~ t w r kv hh>l itin t 2't) rl<~t\rfi~ <imply d ac. I u ~ c ~ n high ~ l crintrol Illr* {ither indtbytbn~lr.ntk , l r r nblr i5 gender (?i7). A factorial dc*sign entails forming cupcr~niental grcrupq f(*r c-ach possible conrIli~;~ltio)~ nf thc two indcpr>ntlc.n! variables. ln this casr the lour R ~ O U Q S art' ~IIIKC* defined in Figurr -1,lO.Fvrnbolically thc tnro indcpcndent v a r i a b l c ~c a n he rcpresentpd a s )ib [contrcll) and X ? (grncicr). The categories of vacf-r can be repreqcnicd ns 'a' and 'b'. Thus ma 1t.c tvith high cnntrol can ht> .;\mboli7ed a5 .Y,,Y:, while lonu control I t m a l ~ s 5vol1ld be symhnli7cci a s xlbS?h. In t h i ~ part~culnrcAsr, ~ i n c e gender i~ an independent \.arialllt. hlil is Iinrdly a n 'intervcntinn' we would start with a randomly st~lcctciigroilp of males and a randomly selected group of fcma les (Figure 4.1 F 1. '1 ( 1 rach c-rf these gmups wp would tnboduc~ tht* 'trpatrnent' of high and l i l ioh ~ ~ control. Half the r n a l ~ s would bc randomlv assigned tn a low cnr~lrnl work environment and half to a high crrntrr~l ~ v o r kenviriinmcnt. The same would apply to fcmaleq. 1%'~ w o ~ i l dtlit~shave fmrr g r o u p fvs tz-hich we would thvn obtain before and aftcr rncawres. The number of grotlpc grnrvs i f tit1it.r vLir~al7!c. has motc lli,in !\to categories. It ts ptarl1,~ps mure diific~tlt to v i s l ~ ~ l l w ~h rc a t \v~)ulcihappan with three or mrlrtxvariables. We c a n c l o thi.; ~vrnhnlicnllvwhprt>wi, h n ~ r three indeprndcnt variahlcs with n v ; l ~ . v ! nnitnlher ~ oi cati*jirlrEcs: (two categories c,~llcda ,ind b), X 2 (two i a i t ~ ~ o r c~lIect i t ~ ~ a ,Inti b) nnrl N1 (t1i.n catcgoric-s called a and b). Tkc firqt catcaxor\- (if the first vsri~l,I(ac~1n bc cvmbolr7ed a s S,", and the second cat(-!:(try nq Sib: the h-I s.ittni:t,n. of tiit' C . C ' T C ~ rndey~nrlt*nt ~ variable i s rcprrwlntt~rl.I\ s7,,; ~ n so d on 'f,lblt. 3.1 i1lur;trates thr n~lrnbcr of groups that cnuld bc proclliccd by c-nch n f Zlip p o ~ l ; i b rorn2~inaticln~ l~ nf these tl1rt.c- inrll-ycnrlcnt v r l r ~ a h l r ~
a ycale of 0-100,

M~~ho 0' d ?llncatinn 70 groups

I

fbme f IT,)
Pre-fesi

i
I

'Interv~nfrons'
( X , and X,)

Summary
T~mp 2 (T?I

Posl-lest

Random allocat~on
I

job sat~sfact~on
4Y
I

Males w~th hroh control

X I , x?,

i
I

Measure on
tron
I

Random allocation

I M~asure on lob s;r:rc!actron

(Y)

~ e a s

XI., X1 h Malcs with

ICIW

I
Random allocation
I

control

Measure t

1'

Females w~th
h~qh control

job satisfac
,Y\

Thrq chapter has c1ull1nt.dA v ~ r i e h . of c*\pc.rlm~ntnldesign< af v a v i n g ci,rnplcrity. Ric 10pc o i drawing c~~i.;nl ~ntt~rcncc from s data obtalnccf wi tli experimental rlr.;igr~swas out1it ~ ~ d Tllc . various es ycri rntantal designs vary in term< clt their use uf prc-ti>.;lr,the number of prc-twt and p n ~ t - t e s trncn.;t~rcrncntpoints and the nu mbcr of groups cc~mpnred rvitl~lnthe Jcs~qn. 1\11 cxperirnental r l t ~ s i ~ n wilI s ha1.e ccrmp,lrl<on Rroupr: and all , ~ t t ~ r n p tot make h c grnrips sq similar as pos~iblr, cvcept t r l relation tn t l ~ c t=\p,rimrntal inter\ cntion. r\ltlinugh random ~ l l t r c ~ ~ t i o n to groups is fhra t;lvclt~rr*d method, tht* rnc,In.; bj. n.hic11 Rrt-lLryq tviFE bc malie romp-rrahlc w i l l vary dcpcnd in^ t ~ n tht. circumstancr< of tlir rt%search(see Cliapirr 3). The riguur of r x p r r i m ~ n t a designs l will vary dcpcndin~ o n t hc ctmlr1uts within which thc cxpcriment 1s conducted. Lnborntosv setting.: provirlc maximum cnntrol of confound in^ cxti>rnal influences whilt* n,ltural crpcrimcnts prtn'idc. the least ci~nkol.

s
yli, X,I,

, Ranlnn? ~1loc:ttron

I I

MPRFUV on job satwtac

Notes
1 7111s is <I tncnlifit-,?tion 0 1 an arpr~nt~nt t ~ a < l t . ( E 111 ,In experiment bv Arttnson ,~nd Mi115 (1959) a n ~ l rl~\ruh\c~l 111 Kerlingc*r (E'17')' ' ) I ) 2 Untorhindtriv Ihr-sr. p>nFs arr not nlways spi'cifi~da t the hrcinning of thta ~mplrmcntationr l f tht* pilot prngr;lrnmr.; snd t~vaEuations t u d ~ r qarta all too frt.quently cornmi+ tr>n~*tl rr!tr.r tlitl p~totprngrarnmt. has been de\,f,lc)ptd and
~mplrmcntcd. 7 There i~ also r n t . n ~ i r r 4change that 1 s dr~tnto various type5 of rrror (see

(vl

C'liapter 1). 4 T l ~ targument h c ~ t i lhaf ~ ns pwplc havt. Oi~ldr~n their ccmc1.m It)r thp wellbeing nt thtair tarnil\ ii~,~L.rc them brccmir~morr ronc~rnedahnut sr>rialand pcll~ticalstability.

Ft y l ~ l c k l vbcir~rnt%\ .Jplwrcnt that tht>n~tmhcr ot groups rrquirtld can t y l o d e ~ v l f l it l i t a ~ d i l ~ t l onf n cach atlclitir>nal rranabIc and a* ruch of thcmt. 1.arinble5 hri* ;l( ltl trnnnl catcqr)r~tn\.I l f c-o~r r.;c we dn nnt nvrd to dra~~ the ~ nr ~ c c a r t - h{vrth ctcn. ycls.;~l,lca ctrmb~natinn ~ n c l u J r * ~F l .r ~ r thec~rctical and pr;lt-tlcal rtusonG tvc niight juqt wlect ~ a r t ~ c u l . <ct.; j r nl crmbiriations ~ n l t..raminc. l tlip e f f ~of ' ~ tht*<c* ~ ~t'l~cc t i~ )r dn l i r ~ . ~ t ~ o n ~ .

~ ~ D C ~ V I / A TM I 4 CW I F ~ ! T D I F F I C UIT TO

r< I A A L I S H

TWF KO1 1 OF

OTHrR FAC'TOItS

ISSUES IN EXPERIMENTAL DESIGN

Euyerirnental J e s t ~ n k have bevn u*~drIy used in the n.~turalsclenctls and in ysvchnlodcal ;~nilleducational rtlwsrclr and have prc>vidcd ; I ~ r c n t dt-nl OF vaIuablc. rr.~t*arch data. W(lwever, eupcrimt=ntal destgn* Ilnvt. br.cn IPSS zvidrlv IISC'LI in the soc~a! sc~rncrs. This lack nf 11scin tht>wlcial scicnccs sterns irnm rncthodolo~~cal, practical and ctliicnl considcratic>ns. Thi.: is not to say, howevcr, that cxpcr~mcr~tal drsi~nr cannot be zrscd in onc fnrm or anothrlr in social qcir~nct* research or that the logic of tlw cvpcrinlental d c ~ ~ cannot m bc invaluable in dcl-clop~n.: other rck~aarc.h cl~,slcn\that ~7t-I, I ~ ~ ~ p l ~ in i t tht* ~ L \1wi111 > l ~ sclcnceq. T h r h chapter r a y l ~ ~ r t - s w n ~ nf c the s t r m ~ f h 5 and xveaknt~.;.-.r~~ nf e.cpcrimrntnl d e q i p and ihc issuvs that muht Ilr ilenlt with in J r v r l ~ l p i n g a n y rrwarch that uses thts
rlt*\i~n.

The <trcn>:ili of randon11;ratizm is also it.; wcakne-: in rtlnlorlng thta effect of numerous var1,1bfi1.; i t also rrmoves too much inf(>i+mation. While tht. control that mndi~mitatinn prt~vtdrs helps isrrlato the impact of the ~ . r p ~ r i m e n tvariablt., al i t prevents u~ tcqting morp cr~rnplc~ causal rnndcls. nnrking lilt ~ ~ ;lnhintrrvent~on r has an impnct. I h i , dr~~fbrlc can k bc illusttated witli the earlier pxamplc of the impact uf school type cln student acadrmic nchievemrnt. lrnaginc an experi nicnt where wc ctrulcl randomly 1' loc;l!ra stude~lls tn ~ o v e s n m c n t fundrcl schon19 and other.; to fee p a y i n ~ privtitc school\ qtlch n random a l l u c tion ~ sl~ould rcrno\*r1 nnv differenctac t r r t w ~ r n thr* twil sets of students in key variablw such a< initial qtudent abilih. and home backgrnund. A t the end of the stud!! p * r ! r d w e caulrf compare the acadcrnlc achicvemcn t of siudcnts in t11r Kovernmen t fundcd schools with that nf students in the private schools. Any difft.rcnctlx should be
attributablt>to tht school. Hnrv~vxsr, what we Icnm sbnut actual lift, qituatFc~nsL n . i n v o k i n ~ thiq ~ ~ L ' C C ~ I I I Ii< * limited. In t h t ~ r t v l tj.urlcf rn;lnv ~rthcr factors ,iyart from t l ~ c tvpe nf .;chtrc~litself mav contribute ti) tht- ,~ctuaEdiffcrcncrs u7t7 see. Yet we hc~\?i* eliminated such rt~lcvant i n f l ~ ~ ~ , l ic~ e.g, . s homth backgrt)und by randoniization and ccm\ctluently wc carinnt identify t l i r ~ r impact. Ever1 if wv dcrnonstrntt>that, despitr qtnrting off thr. same, private schcwll s t u d ~ n k perform bcttcr than govt-mmen t schovl students, ruc d o not know ir?lru the former perform butkr. lt'hat i q i t abcl~rtthe private school< t h,lt produces bctter perfnrmancc7 1% i t trctter tcaclicrs, superior resources, an environment that emp1insi;lrl; academic yrrh~rrnsnceor what? A11 that we know is lhat privatc qrhnol students prrfc~rmbetter acad~mic~illy. Ln summarv, the expcrimcntal design can be effective in identifying w h a t i n t c * r ~ ~ c n will t ~ m be ~ cffectice but i t doe5 not help L I understand ~ rrhy an intervention is c f l t ~ t l v e .
RAN~~MIZATIO CAM N UNI)FlZFSTIMATE
TIIT: T O T A L C A U S A L r f F F C T

7'h~ qkcngth of cxpr*l+imcntalrescarcli i .; the ability it prtwides us with to sav how much dirrct causal impact a rarinble has. Fxpcrirnenk allow irq to iudate the impact o f the e.rpcnmcnta1 variatllc. 7 hi< is achlci.ctl tw randomly allrwating ~ ~ c n p l to e cxpr:rimental and control groups - a process that rnakcs the h~oups ccrmmpiirable in aII rc3spccts except with scp't-d to the expcrimcntat v a r i a b l ~ Hakim , argues that:
f'uptlnmcnb a r r appmpriate whm kfit,rr iq wrnc ar~:trmpnt as 20 thr. t.\r-tcnce nf n i a u c , ~ Irtal;i!ir>n5htnb~.t~vt,r.n h' and I tnr.! o t clther canfi.\tt~,~l trr rt,l.lttlcl h c t o r ~ l tir . lvl~rn the F T P C I L ~ ~ nlt4rlct!rcof tht. rl/r . 111t1 ij~rectv(\n ot l f i ~ ~.tta=t= I \~ n w d v d I lor(, l O F 1

Randomization can rps~lltIn the undcrcstimasion of thc tnhd uarrsnl rplntln?rrl~r~l bphvefn thc c\ptrimental i~ r i a b l c ~ n tht d vi~lctlmrvariable. h the rra l world the rota! c n ~ ~ ctiect ~ n l bctlvcaen hvo \.aria14 t- consists of two rrrm~oncnts:
m
0

'I hc. kitrcngth o f cxptlrin~cntsgaincd through randnmizntinn, I~owc*vt,r, cornrs ,at a cnnsidcr,~blc cost. Exprsirncmtnl resvarch 1.; not urell witcad t c l ~ r o \ * i r l ian n ~i , . ~ ~ l r ~ r r r ~ l of ~ r l the rl r~<ult.i - to identltv~nr, the mechani\nw 171~ \ r l > ~ i one h rarr;rt~lt~ ,?ffects anr)tt~er. Yor dcw5 i t allow us to b u t l ~ la rict~rrilof the crrmplcx ~ r t of factorc t h ~ F t ~ ~ U C ,given P outcnrnr. E\pvrlrncnts fixu.; ltn the imyart nl' ji~rtone or two fnctrlrs.'

dirwr causal effect i n d i r t ~ causal t effect.

Ranclomii.aticrn ensure.; that 111e g r o u p I ~ V compare ,Jrcm alikc in a V 1 relevant rcqprcts. \ T r can only look at thc direct eftcct r l f t h r c F r r l rntmtal variable. We cannot lorlk a t the rt3nl world cFfrr1.; crf a v ~ r ~ ain bl~

var~able nvr ~ r ,lnt , to I?(, \ I I ~ C = t h ~ ilt 1 % d ~ i t t -o tvhat \\.I. t h ~ n k i t is r,ithcr thin < o r n e t l i ~ elw. i ~ ~ 1Vlit.r~the I ~ ~ and I c Gtnlcture (I( a de<ign art. f ~ i ~ t t y anti f;lil tcr climinnte c-ampcting t'\plnnatiuns nf reqult'i then tkir d c q i ~ n lack.; ftrkprrinl ?vilrrlit~~. At1 desiqns !ace thrr,~t< to infvrnal \ratf~i~tv. Carnphcll and StanIcv ( 14t~7) h a w ul~mtifiellthr. thrwt.; disrusscii Iycknw.

thoqe circum.;tancc+ in 1vhic.11 'in intewt+iition or t~.cprs~mciit,~l \rnriatllc. works inii ~rcactly This st~~lrtcarnrng <,In be ilfustratcd bv rnnsidcring the rr>l,itic>nshrp ~I-~M.E~ j;tanilcr II , l r ~ l l rncornt1. lllc ~~~~~~~~t ~ f f c c tt r f gcnilcr rln in corn^. c~rnsists 111 tht~ inzcmit- ~ftffercnct, ! ~ r h z . ~mrn l ~ n and Ivtlrncn ~vhcl arc alikt. rtl, , 1 1 1 rchbj .ir>trekpri I \ r.\icrt 1ht.r t. 5cn~lr.r I t i.; tht- ct 1,-rt t l ~ gc-nde~r ~ t l~.i* 111 , I ~ L ] t ~ i it-tnl f - not l ~ + c ~ i t d it < l ~~ t r A ! 111 <vliicl~ ~t*ndt-r rnich! I-v 11nLtd M ' I I ~~ I ) t l i eInriors ~' s~isli , ~ stEic I , V I N ~ of job, Iravcl clt wcrrkforoe pirticipatiim t>r cd L P T , I ~1evc.E. I L > ~ I \ y c r ~ n i { ~ n tclcsi~n.; ,~l ,lrc gclnd at iclr-nt~t \ring L l i ~ ~ c-xlcnt c h a direct t - f ' t t ~ t The pn4>FV~rr, T~oi~tb\.c~r., i.; t l ~ , ~ ~ tr n d e <-an r a l w l1.11.r. a n inclirtt-2 effect via o t l ~ c r~~1rinblt.s. l'or c.tnrnl,It>, gendcr n f l r c k t h t ~ tvpc n C crl7~1patfon.: people c h t l t or ~ WIIICII t h t J ~i-1' atloweli to 4.11 tcr. Thr tvpe of cu-cupation In t ~ l m r ~ l t ~their ' ~ t 11,vt'l ~ inct~rnc.50, .is ~melln5 nien e n r n i n ~ rnnrc. pl~rclyI,rt-ailr<t n t thtbir gendtlr (kirnplt- ~liscriniin~itirm), thcy also cam more bccau!-t* of isiclors such n.: the tvpr7s of jobs ilirv I~;lvc.,tlic greater likclihoo~l o l wnrLbnq FuFf t~tnc.and n ~ r c a t e rnppc~rtunitv t c l accert nr-rrtirnc I f l i p r e 5 1 ) li we v~l~rnatecl the* impact of p . n d ~ r tin incomt>Iy imly Inc>kingat its direct cflr3r-t wc M T ~ ~wriii~~sl! IM undcrc+!irnnte tlic clv~rallinfl~it-ncpof gthnder o n iniomt.. Iqnilrlng indirect r f t r a r t < has ~ r ~ ~ i t i imylrsations. cal I,cndcr d~f!tbrt.ncr. r r i Incornto i5.111 not t-r Irrercnrntn .;lrnplv h i ~ a ! , i n ~ [lien an41 itrc>rnent111- qamc mrlntrv fns t h r , .;amp I L ~ Hccnuqc J. plnclcr I + litikcd to ~nr-onit,~t~~ljrr>ctl!l, arlIr policv rlt><igntg~l ~ L I incrr~cta genclrr r.~l~tr ~~ ty ~ l ~ alkrl ~ l l i tltv~rl l trl ;lildrl-.s tht- z ~ i r * c h ~ n i ~ that rn. I ~ n h zrnrier to Incrrrnl, I + , t h t ~ IlnL I,c!u.~r-n !:cnrf~.r ,\nd C I C ~ E Itiunrll ~ I ~ 1 1 ~ ~n t ~l 1 bctxvccn t:c,n~lrr ant1 hours t j t t\rt+rk).
.1 :

I t , in additton to tht* cxperirnontnl intcrvrntion, ~ ~ v c n ttakta s place 1 7 ~W 1 C'C~ prtb- t t * 4 and t>n\t-trst thcr~ Ihcsc ckrcnts can thrt*atm the inl(*rnat i,,ll~drty171 thtb reseal-ill. 'I F~csr. 1.1cnts, r.at11cr tllnn tlw erpc.rrrncntal inttmrent Ion, could prod ~rccany cliange r\.r. ub~cn.e.The prrlblrrn of ~utr~~neo vvcnts us i s ~rta~~k Ins t l i ~ l danrl natural cuprrim~nt.; in a n ~ i z ~ k n c ~ wav. w n In such expcrirntbnt.; wr l ~ n v clesq r-oiitrol ovvr what h a y p n ' i to ptv~ple.T I I I lack ~ vt r-c~tilrolis a ):rcat dancpr to the rta.;rarch n q thc timr- I,t-tr~~tcn tht* pfc-te-t .~titf po<t-lt3+tIncrt';lsl'\. f',\prr~nlc,nl;llJesi!:n> deal wr 1 h such p r o b l e n ~ s17)' using cnntrnl qrvups 50 l<>tig t l ~ 1-t?i7ti-~?I , croiip ,IIIL~ t hi> e-xp~~rinit~n!al X ~ ( > L I arp ~ cLp~,!ll~ c\pir<o'rl t v f t ~ t . k r , ~rnctrntrc~l[t-r3 c\ tc.tn;ll c\.tan!x thcn tht. cxprrir n t v t , ~ l in ttv-1.1mtion rtbrn,iiti<t ha* trnlv cfiflvrcnce lit-lnvt~rn the. crilups. Altl~olighi l i ~ boccurrLsIlc.tb of su1.11 c*ttcrnal t,vk>nts mean.; thal wtnc a n n i ~ t attril~uk ~rlr'I lit, chnii\:t~ivt.. c>t?<r-rvr>to thc ~ntrrvcntir~ri luc can still attnl-~ttc. ~lii+i.r~,rrtIn thc , ~ n l o ~ol ~n i ltl ~ n q r rn ~ h c\pcrirn~*iital c and controI grotilw to tht- ~ntcr\.trntit~n. For caiample cvca rnigI~tc,t-;eme -50 unit4 i l f change rn t l r t a trperimcntal Emup ; ~ n d 30 1111itu; vf cl~.inpl in kt111 cnntrnl grotip. If ~ ~ s t c ~ r n Cn \ rlc a r ~ dre l ~ clp-ratinq n i ~ dproduct* the 30 unit< nf i h o n ~ in c twtli e \ y r s i r n t v ~ t ~ and l control grtwps the rt%rnainin~ amount ot cllsnge in the c ~ ~ ~ t ~ r i m cgroup n t a l (211 u n ~ t s )is ;lttritrritahlr, t o the c~xprr~rncnt~ll intervention. I:rcn ~ v h c ~ tcontrol i Ernup5 nrru w e d tl~eqe ui~ctmtrvllcd rxtcmal inIl1itnct7~ c;rn cause prnl~lcms Iclr c ~ p c r i m e n t sIVe . cannnt be cert.itn that both thc t.rp>rimen~.il. ~ n duontrtll Kroups are equnll y cxpnsrd tn thc 11n know n nrld ~ ~ n tm r~ n lrd ~ l cxtc~rnalinflucl~ces.If, tor qomr rcnqon, the t > \ p ~ i m ctal n crc~upr\.nc mare tlkpoxcd this could accrwnt for ant. pnqttckt diFfcl.rcwct.+bct~x.r.i.n the ctlti tscrl and r \ y e r i r n c n ! ~ 1 srnup.; I:urthermistsktm for a nlorlT, thcrtz al+o ma1 I-ta ;rn ~rltr~rnt-tr~l~r 11ff~t-t [hat .;~mpledircct cffect L I tlic ~ ~nttn~ vcv~l~im. That iq, tlic tntpr~enticrnmight h,i\.t. 1t5 t ~ f ! c - only ~t 2~rb~,1uw ~ n ntwr t unknr,\\,n or unintended tavctit n l ~ n
( 3 ~

totqA r ! ~ l i ~ .

l'hr p u r p v w of cltasi):n I < tc- qtrurtllre t h cnllcctinn ~ nnd ~ n , ~ l y OF \ ~ data > f . tr, rcacli ~ ~ n ~ ~ r n b i ~ co in ~c oltur~ \ ~ c lI n t ~ \vt3 ob<er\.ctyh,lnqr> In ,in ~ > ~ r t c o n l c

In a 5hrdr. r \ f chance wrme of t l ~ c ~hangtb ut)til~l bc ~lurnto tht* pa-lng o i tirnr rathtar tliC\n thc c~\p~*rirncnt;ll ~ n t e r v ~ n l i oFor n , c*unmple, I 1 wc ~visli t o t w t thc ~ ~ m p ~nF a c~1 t It..1ching nitltf~nclr,n m;lthem.ltrr*.il performance of

74

F.XPERI34FINTA I . DESIGNS

ISStiFS IN EXPBRTMVIFN 1 AI. DESIGN

cliil~lrcn w e wotll~i ~LIYC'tn be surv tlrat an!' i r n p r t r ~ c n ~ rin n l perf(7rmancc is J u r to tlic 1nlcs1cntic~n( t e a c h l n ~ method) r;ltl~rrthan b ~ a u s ctl\c chlldrt-n are firoi\.rng rrldcr. \Ve nctd to c'nsurt8 that w c do not confu~r thc impact of all intcr\,pnSion for dcvcFnprn~nta[ or maturational c h a n ~ c s that tn k e place in thc coursc of a n cxpcrl ment. Ma turd tional effccts rnigl.12 occur in a study of orwnizational change. In such studies we n w d to diqtinguish behwwn thc Impact OF the inkolluctinn of nc.w ways of do in^ thin^, t ~ u r n tlic ih;lncrc that arcruc clrnply becauw, vvCr time, peopltb are in the organization longer a n d bccnmc. more experienced. In a Idboraiory cxpcrimcnt hungrr, tiredness and t-rorcdom providc further cxamplrs nf maturation effccis. S~ncc both the ~*\pcrimentaland control groups should be erluatlv \lulneratrFc to matur,ition, these c f f c ~ t s can he largcttr removed by using rL~ndnmi7ed control groups. The l o ~ i c for this is thc same as nutlir~rdin t l i t drscussicln of thi, influence of liist~ry. The usc of multiple prr-ttlsts and pst-tests can also lielp detcct maturation as tht-v allr~w u s to srta i f c l i a n g ~in the exprrrmrntal pericld ih simply part of an ongoing tiatand that befrrrp thr intcnrention.

contarnina tc the proced urc, as can thr s,lmc pcrwn bt-cnm tng morc experi~nctdand lamiliar with collecttng thv data. Providing careful training and s p t t a r n n t ~ c instructic~n.; for t h n v collecting data can reduce this problcm. The introduction o f conhol groups and the use of the same instruments fnr botIr expcsimcntal and control grnups can su k t a ntiall y reduce the prLihlcm (31 instrument decav. Fvtn rf there is instrument decav, its ~ f f e c t can bta rcrnoved by compartnK rilfiri7nc~s hetwrcn lthc exycrimrntnl and ctrntrtd groups in the amount ot change nn the outcornc variable.

'I'csting participants s ~ v e r a ll i r n c ~ccln contarnina tc rilsults. A pt3rson may remember prcwious answcr.; ,and subseq~ientl y answer clucastinns I n ~ c ah way as 10 appear con5istcnt. CompEcting qlrc.;tinns can srnsiti/c people to iqwt-s nhout which thcv had previou.;lv ~ i v e n little thnr~ght and thi.;, rather than any ~xperimcnralintcn-ention, cJn producc change. Similnrly, darnillaritv with a keqt ( c ~ g . an IQ t ~ s t )can lead to bcttrr pcrformnnce. Assuming t h a t tcqting will havc Ihp same effects for both experimental and control group5 the effect of tcqting can be rcmt>vcd by usrnR a control group, in thc same manner a< described ahnve. The d a n ~ c r nf interactron effect< (trsting and the internention combined have a unlquc effect) is a danger t h a t the Solnmnn four-group desim (Chapter 4) can identify and enahlc us tn removc statisticalSg. The trsting effects in n randomized expfrimcn t can also bc clirnfnated by dropping thc prc-tttst pharr altogeth~r. Th~r approach Is cvidcnt rn the randnmized post-twt onlv Jeq~gn (Chaptt.r $1.

Grnup5 or people who register c.;tremc scortbson a n c7utcnmtbmpasure at TI ( e . ~ high . o r IOWIF)artx likely to cchngc to a more rnrrdt~rati=score by T7. T'IS ~t,~tisticaI tcndcncy r ~ t h e trh a n an cxpcrirnental intcrvcntion can account for some changc. Statistical regression occrirs hecause ,111 xcorrts are LIP i l l part 2 0 chance variation.;. An t*xtrernelvh i ~ or h low TQ hcorc is morc likrlv to contain an element of r.~ndomcrrcrr than a moder,itr ~ c r r r The . odd< art. against a pcrwn aillrc\ rnq a n rutrt~mt.5ctjrc ~ > I !v,rr I clccasipn.; dut, 20 r,~ndi)m error, Th(-rcfrrrc,the odd< ,trt- tlint a fair prnporhon of tho.;r people who initially nchii.ued a h i ~ h sil>rr due to wnltx ran~lurncrror will nnt be subjcct to thr~ same randonr crror a scccrncl timrb. Accnrdin~ly in a second test t h ~ i scnrc r 1 s likeIv to bc less extremr th;ln it initially was. Stat~st~cal regression cc,ln alsn he d u e to the fact that rneasurtm~ent errrrr can only change extreme .;cores in one dircclit~n.On a r;calt. of I to 10 a score of 10 can onEv get In;r+*r due to measurement crror. A qccrre of I can onIy gct Iti,ylrt,r. The bia.;, thrrriore, of rneast~sement error bnwd change wlll br to ,I more moderatu wore. By u t i l i ~ i n control ~ groups the researcher can deal with this problem. So long a< people with cutreme score< arc randomIy allncatt-d to the control and crperimental ~ r t l u pthe ~ , Eevcl of statistics t re~rc.;sion should be t h e samc tor both grntlps

Mi.as~~rt.d changc c o ~ ~ he l d due to changes in the way prc-te4t a n d p t ~ l teqt nlcasures are mnsle. C h a n ~ i n t l~ ~c questinn wt~rding,the rcr;pon.;tt alttlrnatiucs (e.g. add in^ a 'dm't k n t w ' uption t t l attitude qir~~tir,n.;), thc qurqtion sequt3ncr o r the data c+oIFcction mc~rir.( e . facetrl-fnce ~ inti*n-1t.w at 7, and telephone tnt~rx-icw a t T,) ctruld all account fot I mcawrcbdchange. A different person collecting thr infijrmntion a l ~ u c.ln

rf hvfr grrllrps are initialIv di ff~rent, pmt-trqt differcncc.: bctwern groups could hc clur to this in~tfnldifft=rcncc r;ltllrr than to thc rewarcher's i n t e n c n t i n n . Rtnsonatrlc 4 r c d randcrmi;lcJ cnntrol ~ o u p k sllcluld eliminate -clrdinn effects. \Vhcrc randomlzatinn is not Seasihlc, rnalclling can clirninatc ~pccifiedselection effects but thr' pcfisibili t!, ol ~ i n n ~ a t c h r d criffercnccq influencing rcsult.; rcrnains (scc Chapter 3).

If an rxprriment is cclnductcd over bimc a low of certain t~pr.qnf participant<,miller than an c x ~ r r i m c n t a l int~rvrntinn, may br r~3qponqihte for

m~;l*urt-~ cl h a r ~ ~ Tt>r c . elarnrlt., w c mislit mc.;lsurcl thc acntftmic qradrs nt n z r o u p ot qttldcnt~a t TI <ind c a l c ~ i l , ~ thr. tt~ ~ r t ~ u ~y vcr,~~c. \Ifc t h t v ~ n ~ y l r m t m an t cuyr~rfrnrntalintcr\.t.ntion nvcr a pcr~od c?f vrar ~ n d t t ~ n rtrneasurt. ~ ~ r h m n a n lrl '~ e . m i ~ th clb.;erv~ a n i n c r ~ a s pIn ,3Vcr,ljip ~rarfe5. 11'011ld thi\ Iw due tn Zhc in ttbnornttr~n? Pt~sqrbly.But j t cc~ulda l w ht>dur* In thtb Icss atlltn people dropping nut nf thc prclqramme. li Iht. It.;< abh* ~lrclyptndoirt then thtl i ~ i s r t > ~rn, ~ saver,?cc r mark< c o u l ~ l sim~lv b r b c c , ~ ~ lhe ~ s r! ~ q hc3tllc ~ c c r rnot be in^ included in the calcuInt~rbns;it T*. 7hca lv\t t\,<~v of Jt'illlnq w i t h the impact c p f tlir low of participant< i c to minimif? ~;t,ltact~r.~c d~t)pr>~jI. 'lrhr n n t bts! t l i i n ~ is tcr rtnnlrri-t.11, toffprt , ~ t the n n a l ~ q i s.;tap-. In the alx)vt~ ~.xnrnplc* wc colrld rcralcul,itr tF1c~ 7 , niftlr;l~c's I L W ~ for tliv s t u J ~ * n t s t h t tcmained t h r o ~ ~ g l i o ~ th lt r sh~dv. ?\lternnti~*t~l!~, ~ v h c r c~hi.rc*i~ t.ibidcncc o f ~elrcti~ dmpclut, c~ w e cnit!d t v e i ~ h t h ~nrnpEc5 ~ to alljird for qclrctivr dropout (C'haptrr 'j), intrtd~lce qtnh.;tic.al conhnT+(Chal>ttrr 123 ur cnndtict an.~lvsiswparateEv for v,iric~us ql~hqrcrup.;
any

1
I

~vorld.li thc crpcrin~cntalp < ~ ~ t i c i p , ~ are nts difft.sr~*ntin irnpr~rtmltways tram the 'rral ~ v ~ ~ r prriplt. lil' fiw wt~rmitllc intcnentjtm i$ i ~ ~ t ~ m w(# ~lcd cannot l7c surc n'hat thc outcome 1rr11l be. Fnr e\arnplr, w t l rniyht usr f~rst-yesr u n i ~ ~ r s l qtudents ty to twt tht* eSfcct of s t r i n ~ a film portraying ~rapllrc road accrdcnb nn d r i v i l r ~ bc~haviour. Can w r be ct~ntldcntthat anv effect on thi.: voung and prcdominantlv middle clasr ~ r t will ~ ~ p apply to other typpr; of prlrplr7 rliere are two approachtls to d e ~ l i n gwith this prrrhlerrl. Onc i 5 tu ct~nductexperiments ii.;ing large rrprcstlntative sarnplrs and to analvsp rt.wlt.; tu wtL if thcv ll~lld P ~ ~ I for E ~ al E ~ type< ~ ~ V nf ~ c c ~ p lIn e . the ah5tlnc~ of r;lnrlnm l y selected rvprrsrn t,iti\'c <Girnplcs t hr* Iv.; t htrategy is to rr+jlt9n! the crpvriment manv times with diftcrcnt sarnplt>s anrl in diHercnt crlnteutq T h i ~ i~ cnllcd rr~dit.ntrn11.[f we koep g c t t i r l ~tht. same ri*sult.; with d i f f ~ r c n tvyrs t elf ss rnplc*~ w e can be n w r c ionfidr~nt that thc rcqtillq apply ntclrt. ivirlrly. If, however, wc fail ta rt*plicatr. tl-rc rcsuits with p a r t i u i a r lvpe\ nf pc.opIi- or under particular conditlonq I! rlnr.; not rntm;ln that the prm-erlurc is fi~ndarncnfallyf l a ~ , c d . R,lthcr, i t ~ n ~ ~ b l uq e ' to : spccif! t!ic l ~ m i t s nt our ~mc~rali/atirin and the i!'pes nf p13ilyl~ to whom thib FCCIL(I!+ do an<! ~ l 1n 10f , 7 p p ! ~

The notion 01 cittarnnl i m a l ~ d i tr~tt'i-1; v f ~ the ) e\tc,nt to ~ v l i i c lr~ t r L-JII p ~ n c . r n Ethe i~~ re<ult<oi t h t,\perm~cntnl ~ .;ample to t l i r p c ~ p ~ ~ f ~ ii l t ii~ on mean1 tn rcprcbstltlt.Wr~uldt h e cxpcrirntnntal ink-neent~on work In a rcwl world qituatlon nr zs ~ t l ;etfect only p i n < to occur i l l a n erpcrimcntal svtting t ~ i t h a p~Ttic111.1r SC'~ of participant<?Cnmptirll and S t ~ n l c y (1W3: 5 4 ) rdcntih a nuniht~rof factors that can u n d p r m ~ n rour ab~litvto gcner,~liz experimental ~ find i n ~ bc.vnnd s thc spccihc c~xpcrirnrnt.

C
I

In ~ n ~ i e a ~ o u r to i nachieve g intr.maf ~.,ilirlittand trl attcnd to etl~ical and practical m a t t t w we can encl up ~ 7 1 tan h crpcrirnihnt h a ! i.; cleganl ar~cl ethical hut is .;o artificial thnt w~ learn n o l l ~ i n of ~ any u.;e ahout real PCRPIC in n*al srxial contcuts. T;ljft,l (11W3) r~mrnii* 115 that the ~ . p i c a l s r r ~ a psycholog!l l expcrimcnt is hnscd on
t ~ m p o r x ycollt-ct~onof late , ~ d o l ~ w stmnp=rs ~nt glrrcn a puz7le to wlj8e ~tntierbi7nrrr cnnJi!rnn< in r7 I t r n i t r d frmr <luring their first rncrting while being p t ~ e r at ~ l from trvhii~d a rnlrrnr (In Hnhirn, IqSh. 11 1 )
a

I f pa-tcstiiig itqclf clr,~nj:esbehaviol~r or nttiti~dcq or ~;cn.iiti~es pct3plc so that ~ h r y arc mc>rra \.ulnclr;lble to the eff~cts of the intcwentinn, h r n any crperimtmtal finding.; may trc. .in artitact rrf the r~prrimental~itlration and not apply to p c n p l ~ who do not participntc in ilir c~xperlmwit. If the prr-test ra111t.r than the intrn~rtitionprrldr~crsthe cl'Ccct then . l h ~ rffrrt \TITI nclt rlcr-~lr111 spa1 w j r l d irnplc~meli~~itic~n of a n intewtan til>n\lnct5 lhc ~ P J Ir\8r~rld cotltrvt trill not includc t>upr,rimc,ntal prc-tvst.; In s~tit~ltlcms cvhrrr, tht.rl, 15 07 rhancc ui prp-ttst l~ffwts, i t i~ hcst t c b ~tit~p a t~ 7 w 1 c n wifh(?utthta prt7-f~%si rdias~-, t~ 1 1 s ~ t l ~O ~ ~ ~ I " I P !"~i~r!I gro~~ tIr<lcn p 5n t h d t thc tnipact t ~ !th!.; tl pc l r f ~nt~r;lctlnt? can Ilr, p ~ t i r n c ~ hand m d tnh-n Into account.

Simplzr he in^ s c l t ~ t e dInr a < f ~ i dcan i ~ makc propltl fvel special and product* , ~ r t i f i c i ~ rcsnfts. iI : I f~rnotiq ~ t ~ r cof i y the \Ve<tern E l r ~ t r i c i-la~\ethor~~r \\'arks. rn U h i c a ~ o in the t.,>rll; lC'.30< Hrst d~scovercrltht' irnpnrtancc of what I1;15 L-r~.crhrncknown ,I< the Hnwtliornc effect (r\nt.thl~sht-r~er and Dickscln, 1970). TIM. rc.;rarcherl: rtllecttd a erc7up o f worhtr.: and cha1icr.d the16 working rrn irc~nrnr~nt in an cffr>rt t r ~irnpro\*ra~sr,ductiiit\.. T!WY f 6 ~ 1 n that ~ l r t y n I l e s s of which rntc~r\cnt~or~s tht=v made, thr c r t ~ ~ r p becrlmc more pro(!L I C ~ I V TIICV ( ~ . cone!uded that it was not the intcrvcntioils that rtJerC cf!cctir.e - it W A G being ';ycciall\. selvcted tlint produrt3d tht* cFt\>ct.f iacl tire sarntbc h a n ~ r s bwn nl,lrle r v i ~ ! r o r ~ glr t Ing people (he tctaling that they w e r e qpecial we cantlot bc ronfidtlnf that sat~sfartion and , , ..A ....&...;I.. ..,.... 1.4 L . . . , . .. , . , . , , . +

Practical issues

account the IY,IV in which w e \rill arl.llyse thc dnla. difft,rt*nt n ~ e t h n r l q rt~~1uii.e diffcrtwt ran~plp si/.cs.

Thcrc arc fCc.rv simple ans\trers to thc many practical qucstiom h a t .iri.;c ~ v h c ndesigninq sclci~lcrperiments In most cases the answer tn thc rlwstion i<, 'It depend$.' Extcrnal validitv is best nchicved by choosing a snrnple thnt i.; rcpresentatijbrof thc iv~clcr popliiation 10 which we wiqh ko extrapolate. A rcprr.;mtati~esnmplc is bcqt gained by u.irng random s ~ r n p l i n trchniqut-=-. ~ T h i s requires u . ; to select p,?rticipantr. in such a w;ly that each persnn In the wider pnpi~lationhas ,In equal or a known prnbabilitv of bcinfi .;tllt=ctcd. Prnl?;thilitv thcnrv provldrs IIG rt~ithn wnv of estimating how Ilkrlv our rehlil!~are t o I~clkJfor the* \\.idrr pclct~l;it~irn fnjm \\.htch thtb 4,lmpIk' MIRG \t'lt'Ct~d. Unlortundtrly, even i f wc use random sampling we will not necest;a l+ilv end up wltll a reprcsc~ntatir,c srrmpI~. It can bc biased btlcaus~ w~ go ,~bor~ our t rcqt~archin a n ethical i v , ~ \ ~ kt. is uncthicnl ~ i t l i c r tu prcswrrl potential sub~~r-tq 2 1 1 par!icipatc or to try to prcvrnt them withdrawing from the study nt any stage during thi. researrh. TIIF ~ x t r n t of the bia.; thnt rcsults will depend on how manv potential particip,lnt\ rcfusc to participate, 01. cirbsequctifli \\,~thdra\v.

Sht~uld we tdl participants they arc in the control group? HOIVm ~ ~ c l i sltotdd wc tcll them about the experimental hcatment7 Should thpy know whnt thc research hypnthcsi:. is7 W l ~ i l ctllc ethical principll- c;f ~nforrncdct>nstlnt (scc Eattcr in thiq chapter) i;ug~ests that wc should trll pnrtrcipant\ r v e r y t h l n ~clning so could makr the studv worthEes5. I f prticipants know w l ~ a t w c are lorlking for they might bchavt. in such n way as 10 cither confirm c11. sabotage our exprctations. T~lIing pcoplr tht~ are in a control group or crplaining tllr range of c~perimcntal intt.rvmtinn+ ( c . ~ the . rLingeof income support 1 ~ 1 ~ ~ in7 thr s parantrcd income prr~jcct)may make them ftubl that thcy a r e missin): out on t l ~ r rlcsirable in tc*rv~ntrons and creak rtlwntments ~ n non-c<,opera d tion. 'T1~t.r~ a r r n o hard a n d fast rulr.: about h n ~ much . inlormation t o prtn.ide part icipmts, 1711 t within thc rcquiremcn tr of informcrl conspnt i t 1.; hafest to nlit11mi7et h r ;ln~onnt of dctailed information prrJi.ided bta!r>rc llic experiment is complctrd. This minimize5 the chances of con tam in^ ti n the ~ exprrimmt. Snmctfrnes an involves cxplici t deception to avoid sabotage. fn such cases ethical guidefine.; and cart-tul debriefing nt~t-d to f i ~cr~~pu10ut;lv ~ adhered trr.

Thr importance rve attach t ~ ~ i c x t e ~ validity nnl nnil t h t apprnnch we takt* to external validity will have a Iargc bearing on thc* si?e of fhc snmpte w c chot~se.If wc want to gencrali7e staiiqticalfy to A wider populahon we n t r d to use a s;lrnple that is both randomlv srlccted and b i enough ~ fmm rvhich tn gtrncralize confidently. Many bonk.; provide Formulae for wnrking out tlir sample qi7c r e q u i r ~ d fur statistical generdi7a tion (MRHT and Kalton, 1W1; Kaltnn, 19R3b). If, however, wc use a replication (sce above) to achieve generalization the critical factor is whcklier we h.lvr rt*plrcnted t h t ~rvperirncnt a sufiiclrnt nurnllcr of timw ,~nrf under n qlrfficient range n i conditlc~nc;.Thrwrctical cnn.iidtrationh a n ~ i cornmcln scnsc a s to tlitl rangc rrt conditions under whicli we attempt rrpl~cations

muqt guidcl 11s. i t our main c m c c r n 15 iteith intrrnal ~ a l i d i t vnrc can manage :e\.ith much 5rnalIer .;nrnpIeq than are rcqt~trcdfor ~tnt~.;tical gcncsalization but thcy must st111hi1 large rnnufih to dctrrt an cff~ct. That 15, thr sample s i ~ c musl be largr rnough to ~ i v e thP cxper~mentstatistical 'pnwer'. Procrd irrcq for rvnrking out tht* 5arnpTc q i w requircd tu defect t*ff~cts can he lr)trnd ~ p J ~ r \ v ) ~(Cohrn, rre IW7). Thv <ample .;l?r must a l w take into

Hnrv long st7ould the ~ l bc p hehvccn ~ntrodt~cin,: an intrrvr~itinnand the post-test? T ~ lP en~th of the gap muqt depend on \\+hat our thenrip4 and previous rttsrarch 1c.d uq to expcct about the time it ~vnuld takr for an cffcct to hr dVtcctab~e. The length r d the gap bctwcen intcrvcntion and poq-test will "st' depend c m whether we arp looking for shnrt term nr Ennqrr term c f f t ~ t s . The gap h ~ t w c c n a prc-trst nnd an ~ntrrvention should bc as shorl 3 . ; pohsiblc. A hhillrt gap is dmirable bccai~scit minimizes the c l ~ a n c (~ ~f cvcnts taking place befnrr the intewrnlion that might makc the cxperirncntnl and control groups different in some rvav betore tltc intervention. Horcever, wc may sornetinlcs need to I ~ a v e a g;lp ~f n7es u s p t ~ t that the p r c - t ~ s t might interact with the intcrvention (c.g the pr~=-test ha.; sen<il~zcd pnrticfpants to iqsues). In t-qtablishing the 5 i 7 t b rrf thc tcnip\ral gap hct\rven the prc-tc~t anvf thc pnqt-teqt \ Y V must bc mindful c7C thc impact r3f thc Eoss nf participants and its thrcnt ti3 both intcrnnl and c,rtrrnal ~ a l i d i t y .Sincr drnpnut.; incrt~osethe Iongcr a studv I,ists. wc ntscd to sct thc gap will1 , i view t o n~~ nmizing i th r ~ r occurrcXncc. I-lcr~\,ever,tcx) short n time gap can prodt~ctbn h e r mct h(rdnloqical prot>ltms, IThrrra the gay Eiiltween prca-test anrl prlst-trst I < ~ h c ~ ~t r liq li kt-lv that participant.; will remcmb~artheir prr%-trstrcsponscs. Thcir rntlrnt,ry mav ~nfluence thth wav thrv respond in thc pn+t-tc+.;t phaw nf thy (.uperimt.n!.

ISSUES IN FXI'ERIMENTAL DESIGN

Such rncrnory effects can he dealt with in a number of ways. Pilot twting might cstablrsh the amount of time that has elapsed before parhcipants forget their pre-test responses. Alterna tivcly, if we have allncatcd participants randomly and have a sample of sufficient size thc pre-test can be eliminated altogethct (Chapter 4). A third alternakive is to use the Solomon four-group desipjw. This design overcomes the problem a s it enables us to assess and statistically remove pre-testing effects. Practical matters will also determine time gaps between stages of the study. The amount of time availabIe to complete the study and the availability of funds at our disposal to keep track of p e o p l ~ will influence the way we finally design the experiment.
Which mr!hod of data cnllectic~n?

LIMITS OF

RANDOM AI.I.OCATION

Choosing a small sample can impede achieving comparability of expcrimental and controI group^. For example, randomly assigning a sample of 10 people into two gruups of five people each may easily result in dissimilar groups. As the size of the total sample increases the averagc charactcristics of the experimental and controI groups should converge. Wl-tere the sample size cannot be increased, the ramdnmized block design can be used to ensure equivalence on specified charactcristics. However, with large groups there can still he important differences between the groups on variables with low variabiIity despite randomized assignment (Cook and Campbell, 1979: 342).

GATEKEEPERS AND

THRFATS TO R A N D O M ALT.OCATION

The particular research design we adopt does not dictate a particular method of data collectinn (e.g, observation, interview or questionnaire). The data collrct~onmethod we adopt will depend on the nature of the ubservations wc need to make (Chapter 1). What is critical is that the method of data collection produces reliable, valid and meaningful data. Since the expcrimen tal design relics on detecting differcnccs over time and dlffercnces between groups it is critical that we have measures that accurately and reliably reflect change. We must be confident that measured change is real rather than just an artifact. of poor methods of informabion collection. The need to achieve high reliability does not point us to a single data collection method. The reliability of a method will d e p ~ n d on what data are being cnllcctcd and on the skill of the person collecting them. In some cases we will achieve much better information by watching people than by asking them highly structured questions. Sometimes a less structured questionnaire that enables an interviewer to probe and follow up responses might be more effective than a highly stnicturcd one. AS well a s yielding reliable data our data collection method must produce valid information: we must be measuring what we think we are. There is little use in collecting mliablc data tl-rat are wrong. There is little merit in cnllecting data using very reliable instruments on, say, level of racial prejudice i f thc method of data collcction is getting people to express socially desirable views rather than their real views.

Random assignment can be difficult to achieve in field scttinp where practical realities may require negotiating with authorities and other 'gatekeepers' who control access to potential participants. Among the gatekeepers the researcher is likely to come across are school princ~pals, manngcrs of organizations, and professional welfare workcrs. Gatekeepers can cause problems by insisting on allocating people to groups ti~cmsclvpsnr by undermining the research by poor implementation. They might, for examplc, allocate khe 'most deserving' to desirable treatment groups. PdeaFly the researcher should allocate people to groups this is not possible, close scrutiny of those doing the allocabut, w h ~ r c tion is required.

Tht. practice of randomly assigning participants is one of the distinguishing characteristics of experimental design. For a number of reasons, however, random assibpnent docs not necessariIy achieve its goal of ensuring that the exprrimentrtl and control groups ,are initially comparable.

If people believe that they are in a control group they can fccl badIy done by. Tile resentment that results can lead to a high dropout and produce a control group that differs in important ways from the experimental group (e.g. more negative). These perceptions, rather than the effect of any interventions, could be responsible for any final outcome differences between the p u p s . One solution is to disguise the fact of being in the control group by using a 'placebo' - i.e, an irreievant treatment. In dnig research the placebo might be a sugar pill. In social research i t might bc a treatment that is known not to affect the imtcorne variables and is thus safe to use. Alternativelv, wc can assure control group participants that they can have the deqirab3e treatments, sl~ould they wish, at she cnnclusinn of thc expcriment.

A comparable problem can arise when people allocated to a given tr~atrnent refuse to participate because the treatment is seen to bc

82

E X P E R I M E V A t DESIGNS

ISSUES IN GXPERI MENTAL DESIGN

83

undesirable. Drnpaut a t tl~lspoint would mean that despite initial mnJom allocation the groups could be different 17efurf the in tervcntion. This prrjblcm can bc avoided by poqtponing random allocation to groups (Cook and Campbell, 1979: 358). Having recruited participants we can ascertain their willingness tn participate in the control ar any one of a ~iurnber of different experimental trcahnent~. We could then eliminate from the study participants who were unwilling to participate in any particuiar group. Then we could randomly allocate the remaining willing participants to the various groups.

confident what interxention has actually taken piace and we may thus have difficulties making sense of the data.

Since experimental designs examine change over time they face the danger of differential dropout from diffcrcnt groUFS ( c . g the less desirable interventions). This differentidl dropout could be responsible For any post-test group differences (see earlier in this chapter). This call be a serious problem in long term, field based social research. Furthermore, even where the dropout rate i s much the same across all groups, the drc~pc~uts may differ significantly from tl-rost whn remain in the experimcnt, thuq tlrrcatening the external validity of the studv A kev thing to check for is whcther dropouts are dist~nctivc in relation to their initial scores on the outcome variable. For example, are those with low initial s c r r c s on thc outcnme variable more pranc than others to before, during or after the experimental intervention? rf drop out e~ther they are mnre like1y to withdraw then differential dropout rate between groups could be rcsponsiblc for post-test differences. If participants with Inw scores drop out this will automatically boost the average score of the group, even i f the intervention itself did not produce the boost in scores. Apart from minimizing dropout the next best thing is to identify the characteristics of thosc who drop out and then adjust for these characteristics a t the data analysis stage. We can use the information collected at the ptc-test stagc to help identify these dropout characteristics and then inctude these variables in a multivariate analysis to see if group diffcrcnccs would have emerged if there was no differential dropout.

In anv rcscarch there is always the danger that the researchcr's expectations and values will inadverkcntty distort the way he or she collccts and interprets information. For example, i f we cxpcct that a partictllar teaching method might improve tlie level of children's classroom cooperation we might kste' more cooperation than if we did not hold this bellef. A researcher's expectations can affect what they see, how they ~nterpret it and h o i ~ l~rrrdthey look for cvidcncc contrary to their exprctations. In drug cxperin~entsthis problem is minimized hy using a 'doubleblind' approach. Nci thcr thc parfirfpan t nor t h ~ investigator knows which drug or placebo is being administcr~dto which participants. But this solution is rarely availabie in social research. Because of the operation of thc ethical principle nf informed consent (see later in this chapter) it can be difficult to usc placcbo trcatmtntq. Similarly, it1 many social cxpcri~nents it iq vrrv difficult to hide i ~ h i c h group5 arc suhicct to which intervention. About tlie closest cvr can get to thc 'dr~ublt-bImd'approach in social research 1s to cnsure that the person collecting the data is unaware of the researcl~ hypothesils. But cvcn if this condition is achieved the rcsearchcr's values and beliefs can distort what they see and how they report what they sce. We can take two approaches that can reduce the danger of only seeing what wc want to see. One is to develop clear operational definitions and research protocols that reduce distortion of ubscrvations and interpretat~ons. 'The secol~dis to use multiple 'judges' and observers so that the different nbservers can act as a check on one another. By using multiple r~bservers we can identify the extent to which the patterns are observer dependent.

Ethical issues
In field based expcrimcntal sncial rcscarch both political and practical considerations reduce our degree of contrid over the intervention. M1hcn we rely on a variety nf people to implement a n intervention (c.g, a di ffcscni person might be responsible for carrying out intervention in different orcanizations) we may not be able to ensure that h e interventinn 1s consiqtently or properly imptemented. When the intervention is uncvcnly carrlcrd out we dn no! necessarily have a good test of the impact of the intended intervention. As far as possible the investigator should maintain Sight control of the wav In which intertren tions arc implcmcnted. Without t h ~ we s cannot be

t~egarcllcss o f the restarch dcsign, social research shnrrld conform to four broad ethical principles (Kirnrnel, 1988; Hornan, 1991).

A well-established princ!pIe of SIK~RI r~scarchis that people should not be requircd nr led L o believe that the!! are requircd tr, partic~patcin a study. Furthermore, participanks should know they can withdraw from the study at any p i n t . We shuuld avoid people gaining the impression

ISSUES

rN EXPERIMENTAL DESIGN

from the c o n f ~ din which the request is made that participation is required. In situation.; where there is a 'captive audience' and where there is a power differential hehvcen the researcher and the potential cuperi~nental participants (e-g. first-year psychology class, welfare recipients a t a welfare agencv) the researcher needs to be especially careful to stress that participation is truly voluntary. The surest way nf ensuring that people understand that paltiripatinn is voluntary is to explicitly tell them. For example, at thc beginning of a study participants can be told
',2ltho~~gh your pdrticipat~onin this study will be greatly valued, you ,>re not required to p,trhctpate Ynu can stop a t any point or choose not to answcr any particular questitm. lust let the intcrr~icwer ki~ow.'

3 2

the purposc of the study and its basic procedures thc identity of the researcher and the sponsor 3 the use to which the data might be put.

They should also be provided with the following:

4

5 15
7

an outline of reasonably foreseeable r i s k , ~mbarrassmentor discomfort a description of the likely benefits of the study a description of how they were selected an offer to answcr any questions a statement that participation is voluntary, that each participant is free to withdraw a t any time or to decline to answer any particular queshon.

Voluntary participatioil can produce a number of problems. It can thrcatcn the external validity crf the experiment. Since certain types of people (e.g. those with lower levels of education, those from nonEnglish-speaking backgrounds, oldcr pcople) arc more likely than others to dt.cline ti) participate in studies, vol~lntaryparticipatio~~ can p r n d ~ ~ c e biased samplcs. It can also threaten the internal validity if we get differeiitia l dropout in the experimental and contrt~l groups (see eaxl~er section on problems with randomization). In addition, if voluntary participation leads to high dropout rates it can lead to too few cases for meaningful analysis. Howcvtr, compulsion is not the answer. Although cornpul.;ion might reduce sample bias, unwilling participants can undermine the quality of data in a study. Since we can statistically adjust for known sample biases it is best to maximize khc quality of responses and encourage voIuntary participation. T t is better to cncuurage rather than require people to participate in studies. There are two main ways of doing this. One is to appeal to people's altruism by pointing to the study's possibIe benefits. PeopIe are remarkably generous and willing [n participate in studies where they believe it will d o some good. However, in our enthusiasm to recruit participants we must avoid 'false advertising' which overstates the benefits. Second, we can appeal to pec>ple's self-interest - hnlv participation can benefit them. We might offer rewards fnr participating or at least compensate for their time and anv costs they rnEght incur by pnrlicipating. Somc studies provide token gilts and offer to provide participants with a siirnrnary uf the main findings.

Infnrmcd consent is a close cousin of v d u n tary participation. Typically participants should be informed about:

While the issue of informed consent seems entirely reasonable and desirable it is not always straightforward. How much infnrma tion should we provide before a participant can be considered informed? How firlly informed should participants he? What does it mean to be informed? Providing more information need not mean that people wilI bc better informed. Simply providi~lgdetailed descriptions of the study does not mean that respondents will be a n y mom enlightened as a result. indeed, detailed technical information may confuse, distract and overwhelm rather than inform. Furhermc>re, providing details about the study - especially detailed information about the study d c s i p , about the hypotheses and thcories we are testing - can distort the way people answer questions and undermine the validity of the findings. One solution is to provide baqic information and to offer to answer further questions. If this information could distort the way people b~have we can tactfully explain that this is a concern and offer to deal with the questions nftw the experiment is over. h some psychological research participants are deliberately deceived since accurate knoudedge would invalidate the study (Homan, 1991: 9tiff). Where this is warranted it is critical that participants be fuIly debriefed after the study. 'This raises the question of when consent should bc obtained. Most ethics ctmunittccsz require that consent be obtained hPfnre participation. However, a case can he made that participants do not really l u ~ o w what they are agreeing to until they have participated, and that they can only provide irtfi?~wrrrlconsent affu they have participated. - w!10 should give consent? Who should give permission in research of young children, the intellwh~ally disabled and others who may not be in a pnsitinn to fully understand the implications of participating in a survey? Participation still ought to be voluntarv but consent may need to be obtained from other pcop,plc such as parents and guardians as well a s the particip~n t. In hierarchical organizations gatekeepers may grant

pcrmiq.;i(m tcrr illnior p r t ~ p l e to yarticipatr and t11115,111 w m c srn<c, t h l i g ~ rtlnior p i ~ o p l rkc7 ~ n r t i i i p a t c In . .;~tch si!uation.; b i . ~~ h r w l t l rln.;ilrrn thal ~.lin ct,n*ctit frtm both rhc qatck(,eper.: anci thc ;lctu;ll pnrticip,~nt\.

:'in o h v r o u ~w n l 117 r\.l~icl~ participanl< onn Ilc- hnrnic.cl i.; h i faitlrrr t,

In sc>rnc t.xpcrimcntal studie., particirpants arc pcltcntiallv evp~lscd tn Iiarm. !'or k>karnpic, in medical rlipl*rrmcnt.; in which a new d r u ~ ic; tria1lr.d. plrticipants at'c pi~tcntlally~ n d ~ l n j i c r r [n d . ~syrhrdirrical ruprrir n t m t s parlicip,inl+ r n i ~ h !be given stimuli (11. bc iniillcrd 10 L-rc=I~n\re in I \ ~ ; I ~ Gtfial thy? later rt*grr%t ;~ndfind di+trr>ssing.' 14'here thvrr- ic ati! d,injict' uf harm trl pnrtic~pantqilic p r i n c i ~ l ca C inklrmt~c!cc,nstxnl rrqtrlrc>s that partiripan15 \-retol~l ( r f thzb dancer< hefort, p,3rticiyatin?; in [hi. etpclrimrnt. 12'htnrtn thc.rt5 is ~@teI?ti~l ior h.rmi , ~ n d tl-11. p ~ ~ i - ~ t st111 l n agrLbtJ:. t o particiy.itr tll~re is the nccll f(3r c;rrrt~ll drbr~chn ,lftcr ~ thc c\pt,rimcnt. D ~ l p ~ n r l i n on g thr- typc o f pntt.ntlal Ii,irm thrrc. m.iv hr the n t w l for Irlnl; trVrrn ( C I I I O W - L I ~ t o t*nqtirtA~ h , T ~ RV t p ~ t c ~lallv n t li,~r'n~iul ctfrac-15a r c n~on~trrrcrl. Lnlikt> o t f ~ r dp.!gns, r nianv t~~pertmtlntal dc-s~~n.: ~ n ~ . n l \tht~ . c invt>stigGitt>r act~\~vl dclin): v w n i t > t h ~ n potcnl~allv q hnrrnlul to thtl p r t ~cipantq. IZ~li~lr ~t <hrlulrl IT ob\.irru.: t h a ~ n1ci n l ~ rriol t s u b l t ~ pcvplc t to p ~ ~ f r n t i a l l ~ ~ 11.1rrnl~rl inter1 entions. i t <ecrn.; rrsnw~rableto ~ c up t rl\ytr.inicnts ivhcre thc inlcrvc~itinns~ ' 11 i lwnefi t the participants. F L t~rc~~nlp!t> ~ t ht. g11~rantrxetl income c ~ p e r ~ m e n tcli~scril-red r; in Chapter 1 ct>uld bc just i f i d 1>ec'd11r~ Ihtx intrr~~cnticm of prrlr-tding a ~ u a r a n t w dincr~rncworllrl rmlv benefit parti(ip,~nt<(Fl.lktrn, Icr8h: 102). t lolvc~rr,thc a r p ~ m e n tt h ~ ,in t inten.cntion w ~ l lbr lrtnpf'icial pre*umes that we know thc outcomes of the intenfention. We rn,y trrltrrrr that rnconlc maintenancp w ~ l l at , wnrst, bc harm!css, but ra7do not A ~ T L J ~ I J t h i s rrl~t-~rd clt tlwrcn. Even wiih the t,r*st o l intentions thcrc c a n be irn~ntendrrl harmful r.ffi>ct.;. 'Therr it; albo thc isst~eof 'beneficial to whom?'. l-lakitn ( 3986) ncltcs thnt thr. income maintenance e r p c r i m c ~ i t s led to an Incrcaqc i t 1 t 4 i $ r n ~ ~ a< e ,SI)MC' wclmrn bt.cam~.financialt!~ morp secure. 1-hi+ ) r r n t ~ 1ia1.c I~t>nt.fittd thr wclrncn but thc husbantls (.jnd ti IIL! r ~ n rn;r\. ) 11n1.ehtrld i.cbrl.il iffr~rmt I irrvc \2%i1n thtare artv perrri\.cad bt-r~cfit\ t i ) parhcipatiriq in a n t.uprirnc.nt I P . ~ Jn . incr\mrnmaintcnnnce ru~vrimcn!, rnvdical trial\) t v r m;ly FRCC t l ~ t b I r V n - i t c I : I>t'~~Cfici,il trtbdtnlcbnt frrrni t l l ~ C O I I Z T ~ ~ ~ EToup. IS it tdif 10 dr'nv w>nie ptlcryle Zhrn polcntkal brnrfit u f ttic trr-atmr~nt' Ran~lurn . ~ l I t ~ c a t ~can o n rrducc sum<, 11f tht* ~athrcal: prot~lrm.;in thls rt*~3rrl. Wc can ~ l w undrrtaki~t o prnvi~lethta crln~tbl pruup M.I'~II ~ C C ' P < Sto th{' tre~trncnttmcr wtnknc~wit h a s brbcn succesqli'r~l: (or ; l mo~lificat~u of n tht\ trtvlmtnnton the h,~qi.:d wlint wnc: lrarnetl from lhr c ~ v c r i r n e r ~ t )

hunclttr prrlrniwc of C-~niidcntialilv PC-oplr p r t i c i p ~ t i n111 ~ ,r ~ p t ~ r i r n ~ n t s ,ire r r ~ t i t l c tti ~ l c u p t ~ t h ~ they t cannot bth idcntifred a< the ~ r u r c t of . any yarticu1nr infflrrnation. Frcqucntlv, rm-inl \cicncv tbkl-rurimmts 2,ntalF thp collecting of personal informatinn which, t f rnLidr rublic, souId be ~ . m b a r r ~ i < sor i nhurnilintin~ ~ o r carz<e liarm to p;lst~crpant+ In one way o r another. It 1% r<s~-ntial tlicrtafore t h , ~ infnrmnl~nn t bc collrcte~l in ruch ,1 w a y that ct~rlf~~lrnt~ coin ~ ~ lw lifv g~~arant~.c I! d .i.: ,~l+tr tasqcntial l l ~ l tthc pcbol-rlc invt3l r c d ui ctdlrnctin~, an;llvsin~ and rrbportitig tllc find in^.; r w p c t s u r u p u l ~ u s l vtlic c i ~ n f ~ d t . n t ~ nrcq~~ircmtantq. l~tv In clrdrr In pn>tcct partlcipat~tc, data muqt be '.tortd in *i!ch 1 ' way d 4 t13 Frcclutlc ;rnv (rnatlthiwi7ed ni+cc.;s. This may rncxan tori in^ data in I r ~ k e d c . ~ b ~ t i c(lnd t s rrvnovin): ,111 ~ d c n t i f v i n infomatinn. ~ If thc data an., <torell elcctrt3nic'dllv thrn ~~a>st\.rrrds and r t ~ s r ~ , p t i can c ~ nhcIp t.nl;ilrr p r i ~ a c ! . I. t is h ~ g h dt-siralllt* l~ that prrsonal ~dt-ntikyingtl~tai3qGinill h ~ rtlspnsr\ rt*l;lt~ns 10 p s r h i i p ~ n l s hc kcpt s ~ p a r a t i , l etlforrnat~on ~. about s pt8rs(1n's natnc ;lnd .~iliirr~.;\ shoul~l 1-c 3torctl in i r t i c lltt, ,inti 6i'11 Ih(brr ,jtv-wer. la cl~~t'ctir>n> <lioultl I-re .;tot.~tlin n ~ i i i t i ~ n ~ tilt,. n t Idcnllficaticrr~nurnb(*r\ ftbr crn\~-rt5tcr~cnc m ~un s~ t bo ~ t n r e t li l l quch a WJ\J thtir onlv ar~tharizt\d pcrlplc ran rn,lkc tht* m , ~ f . r h i n ~ . I'rtn'iilinq .w.;urnncc\ c ~ fcnnfiduntialitv ir irnpc>ttar~t tor mett~ndolcrgic-al a5 ~vtlEas ctllical rt-as~~nk. Ii p r t i c i p ' ~ n t s arc. confident that t h r ~ r respnnws are* truly corifrdenti,~l (or et0en better i f thcy arc a n o n v m o i ! ~ ) \vt, cnn expect that pcoplc art- m o w likrly tr) p r l i c l p a t c in tlrc + t t ~ d y ilspcc-ially 11 i t IS al>otrt private matter.. 14't. car1 atso p*pt~ctthal ~f a pt3rstv-i fei*ls that thtir answer.; ;ire trulv confiil~ntia!thrv xvi ll be mtrrc* I~hctv to provide k ~ n and k honrst answer?. one .;hould not milkc prclrni<lbs nf ct>nfldcntialitv un1i.s~they can bt* kvpt. In a stud7r nf 5clioc1l chi2iirc.n v i l r l might Icarn ot thc idpntitv o f a drug puslrrr nr a trachcr tvhn i\ 5cw11allv rnolcstiny: s t ~ d m t s In . a qtudv nf pri~cmcrs yc>u might Irarn of plan.; fur ,ln c s c a ~ c nr of thr identitv OF ;I F ~ S Wwho I ~ murdered an inmatc. In such castt.; vnu fL)c{. thp. pr~>blcm nf ~vc~iqh~ Imctur n g prorniw OF confidrntialitv n~;lin<tthc harm olu<ctl to c3tl1i~rst-r! r c < p c r t i n ~ that From1.t:. 'il;lLr. qurc !ll,it \ - c ~ td<r ~ ntlt rn,lhr ~~nrtvlrsli prttrni~t.+ c of sr~nhrEc.ntiali~ In promi\ing cnnfirlrntialil\ bc ,1\varr o f the I c p l rami t i c .rticlns r vr>trs yrornlw. Drytwding nn tt14. ~ r ~ n ~i lc l 9% \ lh ~ l claf.1 i are rirllectr~l, r l ~ t mar. - ~ tw w b i ~ ~ t (t ~ fret.~lorn ~ l ii n ' t o r n ~ ~ ~ !t?fii~l,it~i)ti. t ~ i l ~ ~ In nianv ci1untr1t.i rc<~>arch d,~ta c t ~ l l t ~ c twith t ~ I tht. g u a r a n t c ~ o f confi~let~tti.~lity dn not r n p y Icg,~lprlrilt.~t,,inti crrufcl bv st~l~piii=n,icd cimrts, C;IVPR*.~tch ~ i t u a tlon 11 15 ~mpc>rfant to clLlritvvnur r i ~ h t > to rnaint,iin thc rontidc~ntialitv of d ~ t awhrn wiIrk~ngIn Rnvrrnmctnt agcnclcs or under n ctlntmct Contracts Frr-q~lcntlvmakc cup!~ c i trtnt13rncntq nticnrt tho owner.;hrp ot
jf
L I

d ~ t anil , ~ it 14 irnpnrt~tlt tc, encure tIi,?t tlirasr- prtlr Fsintl- dl3 not cornpromise ~ O I H .~111dvrt;lking'i re~atdiru sonfidrntialitv. ~

6
ANALYSTNG EXPERIMENTAL DATA

F.rpctimc.nfat tlrsigns arc valiiablic for isolating the cau5al impact of qlv*cific x . l r i ~ h l ~1-f~rs r . 1s rkpcclally thr case n-hcn the!' are cot~~luctcd in hl~V11vcnntrollcd environinent~.The structure. of the dr.;ign, ~specinlly tht. t ~ s c of ~ n t d cc~ntrcllgroup--, help< nlaxirni~e the int(=rnalt.alrditv of crpuriinr>nt~l rr,.;enrrh. Hnwc\~ci., the nrttficinlity of s i ~ c h rcsearch and the il,lrrorvnr~+r l t t f ~ dtlsigrlr r often rne,ln that, fnr w c i a l w-tencr researil~, ~ u p p r i r n ~ ~ ndo51gn\ tal y1ul~donly Timitc.il informatinn th,~tfrequently has poor c~tilrnalvalid~tv. T h i s chapter ha.; disrussed come of thc threat< tn ~ntcrnalJ ~ ruternal J validiv 111 rcscarch and ha5 rlutlincd thc w a y s in which c~xperimentaldesims wcrcclrnc t l i c ~ cthreats. In ~ w i scirnce ~l scwarch p r n c t ~ ~ a .~ l n d rlfhical rnattvrq nl-ck mtaan l h , ~ tcuperimcr~tal rlclqi~ns (,Iten ranntbt be used. T l i ~ s chnptcr lids crttt1inl.d t h c s ~ practical , ~ n detl~icalis\tres.

Notes

1 Tlit+c irr.n;Cnt--~c~:,d r a n ~ l r ~ n ~ i t , ~ t.lrtT r n n rnt~sl evitlrnt with s ~ m p l e tauperinuhnltnl t l r s l ~ n srn whrr-13 Ihtbrr I\ onlzo nne rndcpendtbnt varlablr 'Tlivv can br rr.~luctl~l .;ornPr\.hat In marc cnmplrv twp~rimrii tnl dr+im< that 1nt.1ude \twcral indrpt.nrlcnt variables (sets d~scus~irm nn factrbrrill d p r ~ ~111 is Chapfrr 4)
mrnt dt.partnwnts) I ~ . ! v c ~ri-t~lal~on ctlrics at rornm~ttrrsM ll~cfimuqt appr0rre pmject~lrnm a11 etY71r~: per*pvcti~,~~ b ~ f i > rtunding *> iq rtblcawd for thr str~tty ? n l r \l~lgr,~rn t,xper~rnt.nbcln r~bcdrmcearta c l a ~ ~ r example< c of thr\. Fee TXn~rrnrrnd(1 Oh3) and Milgram (lW14)

~lurnb~ t i tr ~ t a t i q t i c a l tcchn~qurkthat a n 111. used for T11t.r~.lrr6 I n r ~ e analvsin~ expr~r~rnciital dnh. Thc purpose of this chap1i.r i s to p r o v ~ d c some g~liilancrin stalcctinq from amcmg klir.;e mnnv diflertxnt M-a\.< of ,~n,~lys~ expt-rimet~tal nx dntn. -1-he c-rvre r r f euprlrimcntal arr;llvsi< is thc c i 7 t r r j ) t ; r r q r l r i of grrl;rp<. Dn grtwy.; that rllffcr In tcrm- t,f t h r indrpendi~nt varl,~hle also dl t t ~ or1 r the ~ l t ~ y c n ~v;lr1dtrlr.7 l ~ ~ n t no grcruyk that hnve diffcrcnt ?~sc,-ltrncnth'prrltrrnr J~itcrrntlt nn the rrfitcomil varlnE1!c7 In a rrrll-cltls~cnerlriuprr1rnpnt the rinly t i c r t . n ~ bc!n,tacn .~~ Illc g r i n ~ p s ~ I I u ~ I ~\lt> L ~ thtb diFt~'ri'~lt *~~t'~tl7ll'llt5' 4(, rvhzt-h Ihtby a r e t~\yost~tI Tht. morr IVP can be hi1 rc t F i ~ I nrtr !;roup ?re similar cxcrpl in tcbrmq of thc indcpt*ndcllt variilble {tn'tervt*ntion)thc mnre Lr't, can be crlnfidrnt that thy grtwp d i f f ~ r t - n c e 4 m ~ thrl nutrtlmc variablr are duc tcl the infcrvent~on. In m.lkrn~comynrii;on~thrcrx thine.; nwO krr hr + - ~ c w c .

nn ,yrLvrp - nut indi~'idual.;. 1 Thc comp.~risonfcq-u<~.l; nc~ tobc 3 M7t*nt)edurayrpf workingri~zthow r n ~ s u l i d i f f t * r ~thercnceds hctwpr-n c~sperimcntal and control grtwps bt*forc i v e attrtbu It, a n v irnprrrtanctl to thc cliff(-rencc* For instance, ~ 2 . rm i ~ h ohsr~rvr t qrnall bul inlportant dilfcrc%nccf; ur we might rrhscrvc substantial differc~iice< that might be q~rnplv tFue to sampling and rnr<asrarcrnentrrror. 3 Wr compare grnirps in terms of thc outcome variable.

s~letfing the method

of

analysis

'Tlicrc ,Irk1str itrnnv t ~ . c h n ~ q u e lor s ancllv~illq r,spt>ri rnrntal data. Rir rcingra nt tt>cl~n~q~itb< is bt%cau*rl of thta ran!:t. of type< nf \.arialde< .~ticlrli\'crw tt>rni< t h t ~h t a lliaf ,arr1 cc.lltbctctI To ;llialy.c e\pCrirnvntal C I ~ In'ra ~ ~ ~ nc~d t o match tht= mcthral uf ;l.nalysic to tfic ch,>ractcr~.;tic~ c l t thc tlata

Lj~l

FXI'ERllwPI'N'I'rl I . DESIGNS

rt,ttibtic%cstablt~li11 t'tic tind~ncc lor n sample can bv cxtrapolaterl t o thc i t i d i /iol!~ilnf/oclli ~ i r r m n.Ii~ch the s,irnple is drarrn. For c*uample,1r.c might f i n ~ l;Irnt>ngrnernl>cr\ of the samplr that peclplt n . l ~ r arc ~ unenipln\.~d h n \ ~ nrnrse i~ hvalth than employed pi*oplc=.How liktnly is it that thr < a m p link L-lotween t~ncmpluymcntand hralth would hc fnund in thv widcr ~~opl~lati that o n tllr <.~mple is meant ti) rcpresent? Inferential statiqtics prcn'ide a ivay of t-.;t~mnting whcthtxr %amplepattcrns will hoSd in tlie r\.lrfcr population. Idrally, anal?+ s h u ~ ~ lin~.oF\~c d both descripti\fe and inferential aiinly-;is. We shollld carefully cicscribr thc c x l ~ n to f any diff~rcnrcs bratwccn ~ o u and p ~ tl~l>n work out w h ~ t l i e r any di tfcrcnceq arc Ei k r l y to Ioi~ndin the widtbr population.

nrdinal arid nurninal (a'tscl called catc.grv+lcalrrr rlunlitatii.~). t 1it.w can he best expta ~ n r d tzfith thrctl c+nr~rples.

Infrrcntial statistics can imIy be uwc.d to generalize finilings if a ~ ~ o l r n h l i ~ ~ ~ used. A probability sarnplr is one nf two hl+oad types of saniplr-s. 'Tlic ~jthcris a nun-probability sarnplr. A probabrlity sample is nnr in which t ~ pcrscln h h,lr an cqual or at Icaqt known c l ~ ~ i n (probabi!it\.) ce (>f I>r.inq \clrxctrd Jhc- % ~ r r c . ittv a ~ ot . nbt.11nlngsuch J -,1rnp1e 15 to r ~ ~ i i l t ~ ~ n l ~ , ~ r a l t dsnmpIc rntmhcrk from a knt)wll and dciincd pcrpl~lation. 1Ylit-n w r hnvc probability snrnplt* we can U R Z ~ Ir*gitirnatelyKcncrilli7i' I11 the spt~ciCicpopulation I r t m which the sample is d r a w n . A pmhability s a m p l ~ o f a country which systematics llv cxcIuded prnplc .~gt-cl under 18. thaw withnut telepl~nn~q or thosc living in institutinns rt,n\uld yietd findrngs that couId onlv bta gvneral17cd t c ~ the populofic~n of nt>n-tn$tih1tirmali7cdadults rvith a tcycphnne. Tf an cvperiment I < bawd cln n prohabilip samplc of students in a f i ~ t - y e a r psyclinlogy class In ,3 particular irniversitv then we can only confidcntyy ge~~eralize to this pop" la tion (unless wc can demonstrn tc that t h ~ s r f~rsi-yearpsychology studcnts at the particular unir~rsity are r~presentntivcof some ividrr pop11 Inti~~n). 11 12.t. d o no! hn111- n probability .;ample thew iq little point in w i n g inlcrrntial statistlcs t r l generalize to snmc wider populntic>n. We <lio~~lcl limit n ~ ~ r s e l v c to s iicscriptive statistics that sumrnari~c. patterns in that sarnpl~. Many rcsrarcliei.~ are unablc t o recruit ra~irlornsample%. Npvcrthrlcv- they continirc 10 irse infcrmtial statistic7 inappropriatelv in thrir . I ~ , It I < i t . IZ'herr prt~'t~al.rl ~tp sample=< arr not fea5iblt~thc m w t apyroyrl;~tv rsrnl 117 test for ~cn~'r~ili7ab111tv is to rcpcat the tl\pt~riment in diftrrc~nt rcrntibut% a n d wit11 rl'rftcrcnt sarnplrks.
.WIII/I/~, 2s

An interval/ratio \rariablo is one in which thc categories can be ranked tmm IF;(* far Irryl~i n some* meaningful npay.In add~tiontcr ordering the cntcgorit\ Isom Iaw to I~i~ch it i.; p s i b l c to sprcliy the ot~lotrrrf~irtl~fiert-rrrr. hhzqeenthc cntcgyvirs. Age, when mea4urc.d in ycars, is, an r r a m p l t of an inkerval variable. Wr. cclla rank order the catc~orim of agc ( y ~ a r sfrom ) voung~qt(loweat) to nldeqt (highest). Purthrbrmorr, we c.ln specify the arno~~n zrf t d iffcrence brtwccn the catcgoric..;. 'rhe d ifftwncc bchz7cc.n the ciltcbgory '20 years old' and the cat~gor!. '25 yens< old' 1s 5 years. i'ariablt.~ \uch a s the wceklv number caf litwrs of paid rvork, I@, height, weight, Income [in dollam) Arc. all variahlr.: where the c n t q n r i ~ sare numcric and ~ I i r r thp e ~ntcrva Is bvhvrcn thc cntegories can bc \pccificd psecis~l!~.

An ordlnsl \..iriaI,le is c~ni, n.hcrc eve can rank order cateqrrrr~..;irnin low tn high. I-lnwcvcs, WP cminnt specify in numcric terms Irtliv ) l r r i l - l t diffcrcncc thrrr i'i l~c'kwecn t'nc c,itr);nries. For crcrl~npfc, when thc variable age has catcgorirs quch a s 'child', ',~dolcsccnt', 'young adult', 'middle aged', and 'elrlt~rlv7tht. variable i . ; m~asrrred at thc* 0rcl inal level. Thc catcgclrics can be nrdcrt=d from \-ounpa.;t to eldest but n v ccannot sp~cEl\~ prt.ciseI!* the age ~ a hptween p proplr in different cate~tjries. If we mva~tlredl e \ ~ I of workforctl participation as full hmc, part tirne and not in tlrr labour force this vnriablc would bc a n ordinal variahlc. If we aqkcif I l l r precisf numbcr of hot~rs workd tach wcck the varial3le (hours of wnrk) woufrl he m e a q ~ ~ r c at d the interval level.

A nominal variable is onv whcrc the different categories hnvc no set rank order. With a nnminal ~ariallle i t makes no setwe to sav tIi.lt the categorieq cnn t3c ordcrcd tronl tcnv ti1 high 117 .;clmc qenw. I<vligiu~ts affrliation is a non~innl v a r i a b k wFit=r~ i v e can d i\tin~uish t r t t ~ u v ~ cnt~goric.; *~i oi afhlintir~n (c.~ Ic~visli, . I<ninan Catholic, /Irthrxln\, Prt)t~,.;tn~it, I4arnic. n o rrliginn). TI1c.s~ catc~trrii.* ha\^ n o nnttjr,~lrank rrr~frr. When sclr\cting a mdhod c>F data analysi.: we must first j~ltbntifythe level of mcaruwment of c,lch of the variablrns in our analpis 'TO work out the Icl L'! 05 mcasurtlmpnt rrf an!. variill,lta, ask the c~utstirrri..in Tablr

I t 1 \tatiqE~calanaf!,.;ih thr. l~-i,cl of ? t r r , , r ~ ~ / r r ? ? of lr~l vilri,~!llt'~ I IS crltical in wlc.chng particul;lr .;tatisticar analiw~q.There are thrW main Ic1r.1.; a t n'h lcll ~ ~ a r i a b l arc w rncnq~~red: int{.rvnl /ratio (alkn c a f t ~ dcontinuc~u.;),

6.1.

In sel~ctirlg approyriatt>mt*thoclsof a n a l v i < the level of mt*nsurcmfnt nf IatJr tht. grotipmg v;lria'l.!r ( ~ n d c p c n d ( ~ o nr t inten~cnt~~m and > ) the

92

EXIIEKTMENTAI, DESIGNS

Aw tlirre ditfesenl calegnrres' Can I rank the catt.gortc\ from Iow t c ~h i ~ h ' Can I spectfy the amount nf difference between

Ye$

Yrs
Yes

Yes

Nn

Ye.;

t h cat~gories' ~

No

No

Yes

nutcornc variable are relevant. Thc wlcction nf particuIar methods u~ill depend (in part) on the combination of levels of measurement across the two variables and on which variable (dependent or independent) is a t which Ievel of rncasurcment. Figures ti.5 to 6.8 demonstrate how this affects the choice of statistical methods.

I~im2 e

Score on outcome variable

Grnups are compared 111 tcrms of the outcon~evrlriilbIc. These cornparisuns might examine the amount uf prc-test and post-test a31angt in L h r experimental and control groups or the absolute differences between the groups at the post-test stage. There are different aspects of the outcome variable that can form the focus of the cnmparisons.
~ F F E K E N C F SIN CENTRAL TENDENCY

Gmups can be compared in terms of their averages on the outcome variablc. Diffcrcnt typcs of avcrages (mean, median or mode) can be used depending on the Ievel of measurcmcnt of the outcome variable. The most widely used comparison technique is to compare the means of groups, but this rcquircs that the outcnmc variablp is measured at the interval level {see below). A t the descriptive level this simply requires comparing the means. However, thew differences in the sample may not hold in the population: thcy may simplv bc thv rcsult o f sampling error. We could use an inferential statistical technique to work out whcther thc difference between the means is like1y tco reflect a real difference in the popuFa tion or is simply due to chance (sampling error). To do this we couId use a technique such as a /-test, an F-test or analysis of variance. Unforhnatelv, avcragcs can hide ~mportant in fcr m a tion. For ~xarnple, we can get idenhcal averages between twn groups yet the two groups can differ sharply. The example in Figure 6.1 ~llustrates this point. In this example the experimental and control group5 are identical a t time 1. Each person in both groups ~chicved an identical score on the outcolne variable ( 5 ) and the average for each group was 5. At /ime 2 the control group remains unchanged and maintained an average score of 5 . How-

ever, thc experimental group has changed a great deal. A11 individuals liavc chdnged: half havc much lower scores and half have higher scores. However, the average score for this experimental group remains at 5. The averagcs of the two groups at kime 2 remain the same. Consequently, a simple comparison nf the two averages without tooking at other elements of tht. distributions wnuld lead to a false conclusion that thc intcrvention had n o effect, since the averages reflected neither change nor differences hetwccn experimental and control groups.

DIFFERENCES IN V A R I A B I L I T Y
An alternative way of comparing groups is to measure thc degree c ~ f diversity within cac11 of the grnups. Arc some groups morc uniform than others? Do stlrnt. gr~mps change to become morc uniform nr less unifc>rm than others7 The impact of an intervention may be not to increase the scores overall but tn m a k ~ a grnup more uniform nr more Jivcrse (a5 in Fig~irt'6.1). Wicrc arc statistical measures such as sangc, variance and ~tandard deviation that prnvidc sumrnarics of the degree of uniformity Ihnrnogencity) in groups and therc are teqts which measure whcther the lcvcls of homo~eneitvin groups differ. Graphical techniques such a s histograms and box and whiskcr plots can also be used tn examine whether groups arc equally diverse, Which trsts nf variation arp used in part dcpends on thc lcvcl of measurement nl the variables concerned.

EY I'EIIIMEN'I'AL DESIGNS

ModoIify

[a)

Modal~ty

( b l Symmetry

Distr~buhonscan bc unimodnl, bimodal, nr multrrnsdal (Figure 6.2a). An experimental intervention might change a unimodal distribution to a multirnodal distribution. Symmefry Distributions of ordinal or interval variables can be symmetrical or skewed ( F i p r e 6.2b). Skewed distributions can be positive (casts clustering around the low end of the outcome variable} or negative {cases clustering around the higher end of the outcome variable). An intervention might change the symmetrv of the distribution. Kurtosis Distributions can vary in terms of how clustered thcy are around thc ccntrc of thc distribution. This is called the kurtosis The distribution can be very clustered or peaked (Figure 6 . 2 ~ ) . (leptnkurtic), bell-shaped (mesokurtic) or fairly flat (platyk~rrtic).

High
High

Thcsc cliffwent ways of thinking about distributions all provide different ways of examining the impact of an intervention on a n outcome variable. Rather than Focusing narrowly on the impact nn the average score of a group we can look at other dimensions of the outcome variable to detect an effect.

Platykun~c

,;.:':

,,-..,
1

',

*,

'..~esokurt~c

..
LOW

Hlgh

Low

High

(c) Kurtosis

Id 1 Spread

The i m p a d of an intervention may be seen in its cffect on thc slrnpc of the scares on thc niltcomc variable. Comparisons of group homogeneity arc

one form of comparing shapes ul distributions. Otlicr aspects of the shape nf a distr~bution can also form thc basis of gmup comparisons and reveal thc impact of an intervention. Apart from comparing thc. degree of sameness or difference within a group we can also compare the distributions using other dinlensions of the shape of a distribution. These different dimensions arc illustrated in
Figure h.2.
i

Another approach to comparing groups, best used for categorical (nominal) outcome variables or variables with few categories, is to compare thc proportion or the percentage from each group giving a particular response. For example, imagine that we wanted to examine the effect on university course complcsion r;l tes of different methods of delivery of a course. We examine completion ratcs of those who are taught in face-toface situations with those who take their course over the internet. We might find that in the face-to-face group 75 per cent successfully completed compared to 60 per cent of the mcmbtrs o f the intemct goup. These percentages provide a description of the different outcomes in the two groups (descriptive statistics). We could then undertake an analysis that would tell us whether these differences behvecn the groups (75 per cent and 60 per cent) are likely to be due to chance or whether thev are likely to reflect real differences in the pnpulation from which the sample was drawn (inferential statistics). In the example just provided, the twusamples test for difference bchveen proportinny u,ould Fulfil this function (Loether and McTavish, 19741: 189ff).

Groups also can be compared usefully by examining thc association or correlation between variables. StatisticaP measures of association test the

42

WHAT IS RESEAIKI-F VFSIGNT

CAUSA I ION AND THE LOGIC OF KE5EARCI I IIESIGN

43

just two groups. Multiple ccrn~pariso~~ groups are possible and will arise from at least two circumstances.

Male
Young

Female Group 2:

Ir~rft.pi,ndrn! varinbles with moviJ than tu7u cate~urics Where independ~nt variables have more khan two categories we can compare more groups and provide stronger and subtIer tests of theories. For example, in examining the effect of divorce on children we could simply classify rnarriagcs into divorced or intact, or we couId classify them as single (never married), Intact, separated, widowed, divorced (repartnered) and divorced (not repartnered). If we took the latter approach we could make multiple comparisons. [n doing so we can get a clearer picture of what is going on. lf u7e Find that it is the children of divorced parents who have repartnered who havc the most problems we achieve a more focuscd understanding of the impact of parental mari taI status on children. We have learned that it is not. divorce per sr but the arrangements that follorv divorce that are critical. Had we simply compared divorced and intact marriages we might ncver have identified this.

r"; 1

Group 1 :

Group 3: old males

1 1
Group 4:

old females

CnwrhEwni effech of clcffeewnf i n d ~ p ~ n d e vnrinbles nt It is possible thdt it is onIy when people havc a particular combination nf characteristics that an effect is produced. For example, it may be the joillt effects of gender and age rather than cach independently that is linked to suictde. Wc may hypothesize that when a person is both male and young, the l j keli hood of suicide is at its highest. If, for h e purpose of this example, we think of age as a dichotomaus variabPe (15-35 = y o q ; 35+ = oId), then we can think in terms of four groups based on the combination of these two independent variah'les. These are illustrated in Figure 3.7.
Comparing tirr?r puinrs

A finding that children from divorced families are less well adjusted emotionally than those from intact families prestimes that divorce is resyonsibfe for this difference. The problem with a simple comparison between groups (divorced and intact) is that it does not tell us whethcr divorce has actually produced any chnnge in the emotional adlustmcnt of children. Any conrlusions wouId be much more convincing if we could track the emotional adjustment of children both before and after divorce to see if thcrc was any change in the emotional adjustment of children following divorce. By tracking children over time we could see if thcrc wa5 a change in emotianal adjustment of the children and evaluate whether h i s was attributable to their parents' divorce.
MULTIPLE PRE-TESTS
A N D POST-TESTS

remeasurement at the end is called t h ~ 'post-test'. Between the pre-test and the post-test certain critical events occur - in this case parental divorce (or non-divorce as the case may be). These events scrvc as th.c independent variables in the study - i.e. the presumed causal variable behind the observed change, In some designs thcsc intervening events are called the intervention or the heatment (see Chapter 4). We can collect data about more than fwo time points rather than being restricted to simple 'before' and 'after' data collections. We might have repeated measurements over an extended period to track 'ups' and 'downs' and to track /rends before and after any critical event. Multiple 'pre-tests' and multiple post-tests can help distinguish between short term and long term trends. They also help identify the effect of the 'intervention' or independent variable. For example, a simpfe measure of emntional adjustment before and after divorce might show a decline in adjustment. But what if there was strong evidence of the declinc having commenced well before the divorce? Only multiple measures before divorce would show that a trend had already begun. Similarly, a simple post-test might show that adjustment is lower after divorce but mulliplc post-tests might show poor adjustment immediately after divorce but a steady improvement over time (see Chapter 6).

T n the above example the measurement of emotional adjustment a t the beginning of the study is often referred to as the 'pke-tt.st' and the

Idcall!! fhe Xrotrps 7uc are conrpnring sr~nulif hp the sarnr in ull rfler~nritrespt7cts except in regard to the indcpendcnt variable. For cxamplc, il wc want to test the idea that non-government schools produce students that achieve better academically than government schools we would need to be confident thal both types of schools contained comparable studcnts. We would need this so that we could bc confident that the only relevant difference between the two sets of students is in the type of school attended. If the studcnts differed in additional ways, how would we

extent to wli~chtwo v,~rinblr~ I-0-vary. Wllct~we say that gcndvr and income nrc a ~ ~ o c i a t wc ~ r l nlcarl that hcnowinq someone's g ~ n ~ i hclps cr prcdzct f l ~ e income ~r bcttcr than i t we did ntit know their ~ c n d r r . A <in~plr experimental d ~ w i ~ has n hr-o v i l r ~ a b l c ~ the intcr\.vnt~nn variabltb {independent \.arlabIrl 2nd the outcome variable. If the interrcntilm hnq an effect on thc outcome variablc thcn the two t'sr~nblesthe intrr\,cntinn and thc otitcornc - will be nsqociatcd statisticnllv. That is, i f tht~\r, prople in the Rrotrp that recrivcnd rhc intervention (i*xpt*l.rmental grollp) scored dif fcrtan tlv on the .oittcornr v a r i a b l ~ than thc15r.who did not rt.st*ive the intcrvcntir>n, the two variables - inten.cntir,n and outcrrmts - rvtwld be qtatistic~l lv associat~d. 5i1cli an a s w c ~ a t i n ~ can i b~ di<pl~vrd tr.;lng cro.;r;-t;lbulali(~n~, graph.; o r +tahqtical summarit*.: quch as ~ o r r ~ l a t r n coefficient. n In atl~litlonto rlescrihiny: ~ h extent r to which t h e is an arsocintion bc-twccn Ihc cxperirnenta l intervention and tl~r outcome, wc shou Id use a n infrrcntiat statistic dcicrmi~wwhether thc as4ociation is likcly to hold in t l ~ c&\rider poy~rlation.Different rnfrrpnt~al tests havc L3ccn developrd fnr different sr~rm*lntic~n coeificient~.

"b changr i n
~~LIL~!I~-II\*I~V

Fixcd

ratr

~roup In11 l

C;n>~rp bonu.; ~rtlup (no \

Cntl~vidualbonus ~:rr\up(no.)

When deciding 10 display data wi tli a graph we must also decide which b ~ nfe graph lo use. rn Figurt* 6.7 just three of many possible graphical displays arc pfiw~dcd:histogram?, stcm and I ~ a f pl~lt.; and ho\ . ~ n d wliisker plnb 4II three grayhq arc based on tht. 5amr d ~ t and a yrocrdr d~ffcsentl v t ~ y q 01 thinking abou I and ~ ~ i s u a l i z ~ thr. n gdistrib~rtinns.

Since the* purposc of analysis in cxptrimcnts is to compare groups, we must ~lrcirlc how to s S l i ~ i inny ~ group differencrs. O u r choicc is l?r>twc*cn usinq ta blrq, graphs or summary statistics. Which nne(s) we choow tvill

I
I

deprntd partly on the ntrdlrncc and the cornpleuity of the data. Thc t h r c ~forms of dt.;pI~l;ing resultr; cnn h illustratd w ~ t han mampIc. Suppose we had an cxamin~ng the trnpacl c l t diCf~rent forms ot rvwnrds on wclrlipl,~ceproductivity. The three exy~rirxr~ntaf condition< would be: (1) standard fixed ratcs of pay rcgardlcqs of productivitv, (2) a productivi ty bonus for the w h r d ~ group a t thccntl of the year drpcnding on g r o u p nutput, (3) indir~idl~al productivitv trnnuseq dependin): rrn the ind~~idual'.; productivitl;, Thc nu tcome vnriablv rrcjulcl be the pcrscntrtge change in thc prductivi& rrf tbach individual. Ftarnplrs c d tabular, graphical and statistical methods of rllsplaving result%a rtl illustrated belowr.

Alternativ~ly, wc* could avoid the z~sc nf tables and cr,~phs altogether and simply Jernonqtr~te group differonce%by using avcraqcs, percentages or some othcr form of statistical summary. ~ i & w (3.4 provides a set o f surnrnarv stntiqtlcq In relalion 10 protduchvity c h a n ~ in e the three group';. This is a con~prrhensic-e statisticar description of thc changes in thc tl1rf.c ~roups, Htiowcvcr, much of tht* same plrture can bc gleaned from thc graphs and table.

I
I
I I

Ifuzr? rt?nr;v prrvrp.; nrr to bc cnnrpr~rcil~

Another defcrminan! of the rvay w e analvse our data i q the nrr~nlrrvof grorrps we cvnipare. Some sta tistical techniques arc limited to compari~nnq nf h r o t;mirps (c.g. f-test) wh11eothcrs are a p p r ~ p r i ~ ifor t c multiple groups (e.g F-ratio.; nncl analysis of varinncr). Some mrrvlntion coefficients a r r dcsigned For ti<m;lTl number nf ~ r u t t p s (Kendall's tau, rta) while other.; are apprclprintr for i n d e p e n d ~ n t vnriablrs with a Isrcta number of catrqorics. Sirnil.~rlr., the appropriatenr~sof tabular a n d graphical diqpla~ tt~hniquel, 1\+11Tt l ~ p e n d nn YIIY ntlmhpr OF Ernufs in t h t - analyrs.
@YF-GROUI' COW PAIIISONS W111.I A KNOWN VALUE

T a b k h 2 pro\ ides a n exarnptc n i hnxv rr<trlt. f r t m a stud\' nlz!:lrt bc prewnttbil rn tabular fc~rrn.In thiq exarnplc thi. clegrw clf p r n d ~ l ~ t i r i t v chan~c I S rcported for cazh of the three bnnz~s sch~arn~ Thy s ctnllh in the table iniltcatr t h e numbcr of p~cjple from each bonus s c h e m ~ cxhihitlng varinmr lc-velq O F productivity change. To Frwk for differences in tht* r * f f ~ ~ - t of thtt t h r ~ v bonus schcmc.: we need to look at thi*pattern and syrturl uf I nurnbrrs for each of thc thstr* \chemcs.

The simpli*st lorm of comparison i q where we h ~ v cwly c one group ancl we wish to conlparc. Zht. reqults in thi.: xruup with ,i known value. Thest* methods OF anal,v.;is are referrc-d tn a< c~rlr-~nr?rplt' drqiqn.;.

EXPERIMENTAI. DESIGNS

(a) Histograms
Bonus system % productiv~ty change Flxed rate pay Statlst~c Std. Error

Mean
Median

Var~ance Std Devlat~on Minimum

Maximum Range
lnterqwartile Range Skewness
(b) Stem and leaf plots
Kurtosis

-.5455 .OOOO 186.443 13.6544 -30.00 30.00 60 00 7 5000

2 3769

-.I61
1

409

078

798
5 9012

k groductiviry change Stem and L e a f Plot f o r : GROUP= Fixad rate pay Frnqucncy S t e m & Leaf (=<-IT) 5 . U O Fxtremes 1.00 -0 . 9
-90 2.00 3.00 1. 0 0 -0 -0 -0 -0 3
I)

Group bonus

13.00

1. 0 0
3.00 2.00

3
L'

. iio

. . 45 . 223 . 1 . 0000030001 . 23 . 445

.

Mean Median Varlance Std. Deviation Min~mum Maxlmurn Range lntwrquartlle Range Sltewness KU~~OSIS

45.4848 39.0000 1149.195 33.8998 -5.00 130 00 135 00 53 0000

,567

409
,798

-.261
31.1 51 5 14.0000 27 1 5.445 52.1099 -45.00

n .

8s

4. n 0 E x t rPves (>=17) GROUP- B r o u g bonus Frequency S t e m & Leaf 1-00 -0 . 0

29.00 11.00
0 S

2.30
GROUP-

. . .

0000111222233333444 55667778899
03

lnd~vidual bonus Mean Median Var~ance Std. Deviat~on M~nimum Maxlmum Range

9.071 2

125 00
170.00

Individual boaus Frequency S t e m & Ledf

7 . CO 16.00 3.00
7.00

-C 0 0

. . . 1 .

Skewness Kurtos~s

0123334 DOClUl100111112222

569

Figure 6.4 Summary staliqtirs: productivity ch~npe Iry hnnws scheme

0111122

(c) Box and whisker plots

Say for example we wanted to estabIish what connectinn there was, if any, between having a child with a disability and the likelihood of the child's parents divorcing. Let us suppose that we know that the national divorce rate in anv onc ycar for couples with a child in the household is 12 per 1000 married couples. We then find from our research that in a sample o f families in which here was a child with a serious disability the divorce rate was 14 per 1000. The rate of 14 for the disability sample is certainly higher than the national overalI figure, but is the difference likely to reflect a real difference or simply bc due to sampling error? An inferential statistic can answer tIis questinn. The same logic o f comparing a single sample group with a known comparison h p r e can be applied to analysis in which we focits on the mean, proportion or variance of a single gruup (see Loether and McTavish, 1 9 7 4 ~ Chapter 7).

EXPERIMENTAL DESIGNS

ANALYSING FXPERIMEN'TAI, DATA

TWOGROUPS
The classic experimental design cornpsiscs two groups - the experimental group and the control group. In experimental dcsiws with an interval level outcome variable thc /-test is used to establish the stahstical significance of intergroup differences. It indicates whcther the difference behvepn the means of the two groups is likely to be due to chance or wh~ther it rcnccts true differences in the population. Thc use of this test depends somewhat on the distribution of the variable (see below). Othtr tests, such as the critical ratio, can be used to establish the significance of group diffcrcnces for large sarnptcs whcre the assumption uf normaliiy is not satisfied (Mueller et al., 1977: 422). There is nothing about the logic of the experimental design that restricts u s to interval level outcome variables. Where therc arc only t-cvo groups and the outcome variable is either nomind or ordinal we might use a measure of association as a summary statistic to describe the l ~ n k bctwwn group membership and outcome. Alternatively, we might use cross-tabulations nr graphs to display the differences between hz70groups. 1f we needed to generalize about wliethcr any such rrli~tiunship shown in tahlcs or graph< was likely to hold in the wider population we coulcl use an appropriate inferential statistic. Chi qqrmvf is one widely used statistic that is cspccially useful when outcome variables arc measured at the nominal level. This tells us whethcr the hnro variables are independent of one another - that is, whether group membership makes any difference to the oi~tcorne variable.
THREE OR
MORE GROUPS

Table 6.3 Maru pcrct+uhl~c ~'hnlrxe I?? ;im~Juctiriil~/ Iry ilntlus SCJICHPI,
Honus group
Fixed rate of pap Gruup bnnus lndiridual bonus

Mean "A change in productivity

-0.55
45.48
31-15

An analysis of variance may show that, overalI, productivity change varies according to group membership. However, the problem with this analysiq is that we do not know zuhiclr group differences are important. Fairly clearly both the group bonus and individual banus groups do better than the fixed pay group, but is the difference between the voup and individual bnnus systems statistically significant! Could the diffcrencc between the 45.48 per cent improvement and 31.15 per cent improvement be simply due tn chance? To h n i d out which groups have means that are statistically different from one anothcr we need a multiple comparison procedure. In such a procedure all possible pairs of groups are cornpnrcd and i t can be estab, have means that are significantly lished which pairs of ~ ~ o u pifs any, different from one another. One widely used multiple comparison technique is the Scheffe test. Thc Schcffi. test gives us an estimate. It does so by looking at the differences in means between the following pairs of groups:

1 fixed pay versus group bonus 2 fixed pay versus individual bonus 3 Group bonus versus individual bonus.
It can be shown that the differences in the first two pairs (fixed pay versus group bonus and fixed pav versus individual bonus) wcre unlikely to be due t o chance. However, the difference between the preductivity increases in the group and individual bonus schemes was probably due ta chance. This means that if the observed differences in productivity imprnvernent of those with individual bonuses and those with group bonuses is due to chance then essentially the two schemes are equally effective. It does not matter which bonus scheme is used.
Are the conlparistln gmtrp75

Some experimtntal designs require three or more comparison groups. For example, we may want to compare a range of interventions a s well as having a control group. When the outcome variable is nominal nr nrdinal we will largely be restricted to tables, graphs, measures of association and other ways of comparing di5tributions. These methods are satisfactory so long as there are sufficient numbers of people in cach Ernup and there are nnt ton many groups to compare. When there are more than six or seven groups to compare the comparisons can become very d~fficult. If the outcome variablc is an interval level variable t h analysis ~ is most effect~vely made by cnmparing the means of the different groups. A nue7lay nlanlysrs cf variflncr is frequently used for this purpose. A one-way anaTysis of variance will tell us whether there are differences within a set of means. For cxample, in the earlier illustration of the impact of bonus schemes on productivity there were thrcc groups: fixed rate of pay, group bonus and individual bonus. Let, u s suppose the resuIts outlined in Table 6.3 wpre obtained in this study.

i n d q e n d ~ n sanryles? t

~ D F P F N D E N TSAMPLES DESrGNS

The simplest and most effective way of making groups initiatly comparable i s to randomly assign individuals to expertmental and control groups. This is called an ~ndcprndellf sarnples dmign. Random allwation to

IU?

EXPERIMENT.4L DESIGNS

A N A LYSING EXPERTMFNI AL I I A T A

1 U3

cxrcrimental and control groups should ensurt hidcpcndcncc betawen the groups. There should also be independence between cases rvfflrin groups. Srnce the sel~ction of one sample element should be independrnt of the scIcction of Pverv other dement, a gruup consisting of, say, husband and wife pairs docs not produce an independent samples design.'

histogram can give a visual idea of whckhcr Ihc distribution is a bellshaped, unirnodal distribution. A normal curve will also have a skewness of O (i.c.symmetrical) and a kurtosis of 3 (i.e. mesokurtic) (Loether and McTavish, 1974b: 162). When the sampIc is small (e.g. less than 50) and is not randomly drawn from the population, and ~ t s population distribution is unknown, the qafest strategy is to use either nonparametric statistics or a robust parametric statistic.
1s the vnriunue an the d~pe~tdent varinblr similar betz~wn rack

FAIREDSAMPLES

DLSrCNS

In a prlirtld snmp1t.s design groups are made comparable by pairing sample elernenis rather than sin~plcrandom allocation. Here the selection of people in one group is not independent of selecting pcoplc for other gruups. For examplc, wc might have a group of identical twins and assign one twin to the experimental group and the other to thc control group. Alternatively, we might simply administer different experimental treatments to the s u m group at different times. By using the same group wc theoretically remove any other differences between the groups (except of course later treatmcn ts might be contaminated by earlier treatments to the same gmup). In effect people in one experimental group arc paired wit11 themselves for the next trcatmcnt. Some ir~ferential statistic5 need to take into account the way in which peoplt. were allocated to the different experimental condi tiuns. Werc thev pniwd in somc wav or wcrc they randurnly (independently) selected?
I,q t h d ~~ ~ n d e rvarznblp zf normally dislribzjfed?
Some inferential statistics, caIled parametric statistics, require in tt.riral

cornpn~ifo ~~ ro ~l r y ?
Group comparisons can be usefully executed by comparing the typical response for each group. However, comparisons ot what is typical can bc invalidated if the groups have very different variances. Comparing the means nf groups with very different variances make5 it difficult tc> establish whether any differences in means are due to differences in central tendency or not (Loether and McTavish, 1974a: 176). Some statistical techniques require that wc establish that the variance in thc (mu)grtwps is similar. This is particularlv important for t-tests of differences bctween the means. In the case of /-test5 there are diffcrcnt forms of the test d~pending O n whether there is cqualiv of variances between the groups. Most statistical packages include statistical techniques to test for this condition.

How many irrdqlrndent vnriablfs?

The simplest experimental designs have just nnf independent variable. However, many designs include several independent variables. Factorial and Solomon four-group designs examine thc impact of each of the experimental variables on the outcome variable. They also identify the impact of interactions between these independent variables on the nutcome variable. In field experiments where random allocation is not possible we need to estimate and statistically remove the effect of known group differences so that we can examinc the 'pure' effect of the cxpcrimental variable. When mu1tiple independent variables are used the analysis strategy will depend on:
a

level ou tcnme variables that are normally distributed in the population. Unfortunately often either nutcome variables are not. normally distributed in the population (in the statistical sense) or their population distribution is unknown. When the outcome variable is not normally distributed or we cannot reasonably assume a normal population distribution we have three main options (Loether a n d McTavish, 1Y74a: 2hXff).
1 We can 'normalize' the distribution by transforming i t (Marsh, 1988:

!i
I
I

Chaptcr 11) 2 M.Tt ran violate t h a~sumption ~ of normality dncl use the paramctr~c statisbc anyway. rt is argued that some of these statistics are robrrst and vield very similar results regardless of whether their assumptions arc met. 3 Wc can u?e a notr-pnmr~ictrrcstatistic that docs nnt assume a normal distribution.
How can wc tcll whether tl-rc outcome variable iq normqlIy distributed In the pnpuIatinn? Given a sample of sufficient size a line graph or a

I e

the level of measurement of all the variables (both independent and dependent) whether we want to examine the rrlrrfilte inlpact of each independent variable (including interaction effects) or whether we want to look a t the effcct ot a s i ~ ~ gindepcndent le variable with h e effect of other variables controlled.

1D4

EXPERIMENTAL UESIGNS

ANRLYSINC: EXPEIIIME!+JTAL DATA

When a design uses multiple independent variables and an intcrrlal level outcome variable, two-way or fac/ori~la~lalyqis?f rwrinrruc is a common method of analysis This method enables the investigator to look at the impact of each independent variable w ~ t h the effects of the other independent variables removed. That is, the analysis can isolate the 'pure' effect. nf each ind~pendent variable and each possible interaction effect. This approach can be illustrated by employing a chemical analogy in which there are three chemicals A, R and C. The question is, 'What is the corrosive effect of these three chcrnicals on steel?' We could look a t the effect of chemical A alone, chemical B alone and chemical C alone. We might also want to know what happens w h ~ n we combine the three chemicals in different ways: A with R; A with C; R with C; and A, I 3 and C combined. Factorial anaIysis o f variancc cnablcs us to answer these sorts of questions. Multiple regression is another approach employed for analysis involving a n interval level outcome variable and multiple independent variables. Multiple regression provides a way of examining the effect af cacl-r independent variable minus the effect of every other indepencient variable, and of the interactions between the independent variables. W~ert. the tlzd~prtdenf variables are not measured at the intcrval Icvcl (normal multiple regression analysis assumes that all the variables are measured at the interval level) we can use a modified form of regression analysis called dlrmmy zwriahl~s(see de Vaus, 19%). If thc drrpendent variable is nominal or ordinal in character, analysis of variance or standard regression analysis cannot be used. There are, however, other forms of regression analysis such as logistic regression that can bc used far these situations,

If we are interested in the effect of just one independent variable we may need to statistically remove the effects of okher group differences on the outcome variable. Analysis of variance and regression achieve this. However, with nominal and ordinal outcome variables we can also use cross-tabulatiuns to rcrnove the effect of 'contarnina ting' variables. To do this we would uqe ehboration analysis (see pp. 203-2101. There is insufficient space to describe this approach hcrc, buk clear explanahons are readily available (Rosenberg, 1968; Johnson, 1977). Another wav of removing thc confounding effects of othcr variables is to use an average partial table (Loether and McTavish, 1974b: 290).
Sunr~irn~ process !~ for sele~-fil~g fllc right Inemure

F i p r r s 6.5 through 6.8 are designed tn help you select, an appropriate form of slatrstical analysis given different types of rxperimcntal situations

ourwme vanable

.........................
N u m k r of groups

. . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . . .
. , -

. . . . . . . . . . .

many ' Indcpmdofll (urou~~nq) variabl* are there7

HOW

. . . . . . . . . .

I'

Is the nutcoma

,
'

. . L W 8 ~ 1 :

.
' '

, -

'Skewness ' Kuflcs~s ' Ccnlml IOndenv Knlmogolo u--Sm,rnw ' H~sloq'am ' Stem and leal g o t

,
,

.
I

Graphs 1e.g. rlns grspr.3. stwn and l m f plms for each group, hlstcgrams for Each qroup erc ) ' Crws-tabulatons (only w h o ~ o u a r l a bhave l ~ law catcqonos) ' Comp~nson orihe groups rn termsol cerltral tendenv, v a l h i on, shape ' Mcasufe of assoclatlon- Pnarson's r, eta

',
hsmlprlon
Gmphs 1p.g llne graphs, sfem and leal p,ms 40, 8 8 m group. hxlugrams for ~ a c group h ptc ) ' Crops-tabvlairona ( ~ n l y where varlablos h a w few categories] Commnson ol Iho groups In rsrmf ol cer>trillfendeqw. varstlofl. shape
'

Descdpltve
' Multlp'elpartlat

cormlm~on

ln'e-lbl

Two-way tiacforlnll analys~s 0 1 variamje

.

I
'.

~nfererrce
' 07s-way analysis of vaflanoe (oomwnrqmearsl

I
.-.

. . . . . .

. -

' Kwskal-Wallis K t e s t
. . . . . . . .

Figure 6.6 Sclecfing statlsfical methods of analysis, part 2

0 0Level of measuromant of ttw outcome varlabla Palmd or ~ndepend@nl groups

.

. . . . . . . . .

. . . .

Palred groups

'. ! Cornman arnuos uslna:

Descriplbn ' Graphs ' Cross-tabulauom (11 s m l l vumkrolcategorles @I mlt var~ables) ' Summav s t n t ~ s t l a such it5 centra'tendmry unrlal nn qhane 01
C~cinbvt~on ' Propr: ~ n s * Moasufes cf assoclat'on (whm companmn groups can be 'ank areredl gamma Kendell's rau, 5++ermal'4 rho Measofesol a s s a ~ a l ~ n (when n comparlscn qmdpscannol be rank ordered) lamoaa Cramer's V

Compfe groups using:

1
1

h rnpreg, o;p9"31"g
fescnphwr Grap?~ * Crws iabuaettloqs(n m a l l n u m w r 0I
categome<01 l l a n varlablss) Surrrray slaI8sl cssuch as aentral Iendcrcy vorla! on s h a w ol d~s~rlbutlon

Descrlpilon
' Glapvs

Cross.rabulations [II small numbar d categoves or boib variables)

' S u n ~ r n a ~ s t r t t ~ ssuch t ~ c s as centml tendency,

varlatlun, rtinpe uf dl>tr:@ullon
' Pmuo't~o~s

'Odds rntlw ' C?, squdre based measures of a s m ~ a t l o n . ph~ Crarners V , contingency mefhcleni ' P n burell measures dassw~aumlambda. Go@dmal a i d Kruslal s mu. Yule's 0

Inbmncs Any oumber of grwps " Cbl w v o r e twf

' approy>r,ats lest

Prnwn8ons ' Measu-es ot assoc~al~oh

'

of s ~ g n ~ l l c a nIm c r gamma Kenailll s Iau or

Inference
* Ch1 muare 1851
' Marg~rlnl hornog?ne~t) ,'lest (only lor

Spes'man's rh3

Twu Groups

- Tcslsof slqnmf~wr;ce f o r a c h prc bassd

meqsure

mn'par8son of 2 g r o u p )

:
--

Kntmogoro&!3m~r,lov2-rest (2 rndependent srnples, ron.earamoInc) Watt-Wollow~tz tuns t a s l ( 2 Indnmndont somples. non-parametric) * Mann-Whltney N e s t (2 independent wrnrles, nnrrparamelv~c) ' Two-Sawple rnedlan t e ~ i
'

I :$ :;
I
I

- McNe-nar lest (dchomous varlaWes

IPSI

W.lmxon marcbed palrs tesi

Thrse
' Cocoran's

gmvps

--

Three or more gmups

' KwskeI-WBIIIS

Hr~qde~enden sarnpte.. t 3+ QICI~US. 110n-parametric) k-sample msdlan tssi -

ststistical methods of analysis, part 3 Figure 6.7 Selecti~l~q

EXPERIMEWAL DESIGNS

ANALYSING EXPERIMENTAI, DATA

109

and different types ot data. These figurcs cannot coves all pnssibilitics but provide guidance for a fair range of typical situations. Most of these statistrcs are easily computed using packages such as S f i S or SAS.

Selecting the appropriate method of analysis essentially involves asking a series of questions: What sort of sample do you have? Probability. a Non-probability. What IS the level of measurement of the outcome variable! a Nominal. w Ordinal. w Interval. How many group4 are you comparing? m One (agninst a known standard).

Two. Tnrce or mo-re. How were pcnplc allocated to each group? Are samples (groups) independen t7 Are the samples (groups) paired or matched in some way? How is the outcome variablc distributed in the population?
Normal. Not normal or cannot assume normality. Do t l ~ e groups have similar variance on the o~itcome variable? r Equal variance. m Unequal variance. W h a t group comparisons are required? Ccntral tendency. Variability /shape. Proportions. m Association. What type of way do you want to display your comparisons between groups? Tabular. Graphical. 0 Use r ~ summarizing f statistics. Mixture. How manv independent variablps do vou have' One. e Two nr more. Are you mainly concerned with description or inference? m Describe patterns in the sample? Generalize from the sample? m Both.
a

Ntit all t h t qutu.;tir\nr ~ ,Ire . ~ p p m p r i a t In ~ ,111 circt~mstnncrs. 5ornt9 arta npprnprh~ted c p t - d i n g cln h i ~ wt3al.l itxr quv5tion%arr sn<t.vi%rt,~l ( c . ~ I.I y o u r t r ~ ntlt ha1.p n yrirh~lilltvsample then VOII c a n n o t usc qtatiqttcal. inftlrcrlcv . ~ n d ~ v i l lthuq bt- rtwtricted to drscriptirm)
I

Other issues

Tht. lo):lc t d thl* cI,l<sic rxpr~simentnlJ t . s l ~ nM.~'IPIIL~1~,1d115 to ~151-chang~. sclrrt'k. I\-licmn 17 c hart, II pre-tt,ht \\.c (-,Inra\amlne hot79 milch c a c l ~ jirt?lip chanec. on tht. nutsonlt. nl'asure l,rfore and ;~Ctcr thc inttan'cr~ticm, 5ncc WI, rccoRnlze thal thr ci>ntrrrl group mtIv at50 cxllibit change dt11. to t ~ c t ~ j qwch rs a< matur;lSion ~ v c tvoulr.l s~rnpll;h ~ * p o t l i e ~ itli;lt l t . thangc neill it.lrn vcatcr in tllc cxprrimt~ntalgroup than in thr conkol grt~lry. rclr exarngilv'

? or 4 I t t l ~ L , ~ n a l v +n ~lq r a ~ i l r t tJtagr~ata b~ ol chtit~qc,r<~thcr Ih,~n\lrnyIv t.hangc / ) r h r ' . i t ' t11t~ 'iltrur' (11 tl~c= w a l l , c a n Itaarl u> 1 1 7 ~ r n d t a r c s t c m ~ ih ~tt, clt.qrot* c + l ch,tnp. Thrq ik ,I prcrlllem with many rnrtarures fhat have ilpprr and Irwt-r limii\. I t ~ n t r t d ~ r c c n s bias in tc1.m~ot the tvpc nt c h ~ n p Illat , WP C J ~ InPa+1trc . ~ t the indlvidual Itnvcl (w.r Chaytrr qj and t l i c ~ a~ n undere<tirn;ite the cxtcnt of c h a n ~ c and w r t j n ~ l vt..;tirnatc tl-tc dircctinn (16 d~ancc o w r a I!. It i < diffic~tlito dpnl with this prot7lcm of dnt,i dm6i-n front scales wrth qtric't tiprer and lo\vVr llrnits at t!ie analvsis <t~iqr. Thtn pn)blem of truncatlclrr rfftbch 1 s best ~1aii~I1ecl at the 11eqi~n <t;lqcawlirrt> w r nctbd tc* en+urtn tli,~t mlr mt5;ls11rcs;ITC Je\~gnt,iI in s ~ t c h way t h ~ pt*oylr. l artnntlt cfu<tc*rrcl, ~ \>lie t t-nd c d ,I ~ c ~ l\2'e c . 5hni1Fd tamplux rnt.xrlrch.; [hat har r s h i ~ l rccilirl~c,imJ l ~ u floorr ant1 n h m , ~ t w p l (do ' xit3t cluqtpr tcl~vdr~l PI 1 her mJ.
CI

In rrrrI1.r t r t icl(-rltli\. .;Irt\rt tt>rnlanti b t y tl.rm cmffimctc ~ * t . C'ha~tt.r t~ 4 1wmrv\\leri~?~r~n cle.si<n\ tdl rtlclrlir(, r n ~ ~ l t i p btafnri> l ~ < anrf a f t r b r mcr7curt3+.TIle\th
I\'c can thcn ttndrstakt. ;In ~n,ilyqic cot vntiancc t o i f thr. I ~ P ~ T E( 3 ~i d iff&-renceIn chanp. score.; 17-0 compared tn 5 ) could h a v r oc~-ilrrr*d 14 i ~ e to chancta (wmpling crror) If llic ~lifft*rcnce c-o~ilrltrc, dirr to s a n ~ p l i n g rrror w t ~zlijulclns\llrntS t h , ~ t thc.rP rv,.crliId be nir differcnc~.in thib pnplllatirm from r\.hich the qnlrnplewas drawn. 71erc arc seibcral protllcms wilh using simple c l i a n ~ c scor~.:to detect change , ~ n dthe<(> will bc d~scu.;seJ in Chapter 9.
I
In dc.;ip.; zcheri*scores for a gstrwp cluqter around the top rlr bottom end of a scalr ( i t , t h r mcnasureof the outcome varlablr) wc m ~ t s tremrrnbrr that changc can PR~! bp detc.cted in one dirpction. This tact can intrrrducc a bia.; in I he nature o f the nt.rserved changt'. For ertarnpI~, we may use a 0 in 10 scalp. 10 rnrasure happiness wlierc a si-nrt- of I F ~ndicstcqirnhnppine.;s. 5llppo<e that wc had p r o ~ l e 111 nur croup< that qccrrvrt 11, i.r. \*.ere ~t thv ekt rprnr. PI unhappint.<s 0.. r r r i , , ~ < r r r l . l ! I~I rI v r t ~ i r r f r .I i n w ~ v t r if , thew pcoplt- bcbca rnr rwen ~nh~7pplc.r nvcr hmr thcrrl is no way of dcatccting t h ~ ,rvrth clus Incasurc.. Such pcoplc wt~ulil bt. itlcntrticd as nr3t c h . ~ n ~ i n g \Irk* t o ~ i l i l n n l ~ .idenill\. peclplt. n ~ h n btbcamr Ii,ippicr 'Thlr- .;Arne IIiinq ~vt,uId a ~ l y l r ti? thost, wllr, .;r.(lrtvl TI) cil~~l vnlv d dr*tecl thnsr rl;ho hcc;lmi% 1e.s happv. Fvtm t h l ~ ~vhc) r art1 not a t tht. ta\trcrnt- encj of a +calt. !,lit art3 nrdr thi* end prcq~an t .I prr+bIcln T r v t-\;lrnylt~, nltl1ou\:li thr ptlrsr\n \\rho obtain< ; 1 score of 1 in thc prtX-tesl can lower (thc~vcan cc~t 0 ) thcy cannrrl ~ l ~ a tc3 n the ~ r sdmc r + j t r , l i f as. \omctmia whir ol7ta1nt.d ari initial w r r e u i qav

di,$rgns c.ln yrc<tbnt p c i ~ ~ t ~ c t i prtdilcrn\ lar ul ,3ii;lly51~ atrd can ~ n \ ~ t ) l ~ i b ~ l l ~ i t~ i lo n i p I ~ ,in;ll!.\t-. \ if ~ t c CIIV try in;: !t r c.ctirnp,~rc t1z.o trrndk (a< t ~ p p ~ h t 'tn d S<T\ tw11 171edn\) Thc~rta t~ a r ~ l n e e c r t r!*l?at are callcd tin~r* .;crlcs .lnalvsi< mcthorls tlrtit arc I ~ ~ , y n n tl~c d scoycn (11 thls bwlk 1 IOM'CI PT,a I I S P ~ L I I i i i r t i . ~ l apprnnch to a n ~ l p i thekc n ~ sort< I,! d,ita IS ti3 dr~w 5impEr line ~ r a y l ~T sh . ( w may cir m a y not rcvral ,I clear Innq or short t r a mcff~ct or intEicatc th,~t thc. intcrv~hntit~n 31~1s n o rlistit~clive eflcct (Fi~un> 6,s). The patttsrn 111 tht- e r y e r i ~ n ~ n t a ~$(711p(h) l can tl1r.n h r cr)mpartd rrith that ol a cr,ntri>! crolrp. The ciifficlll tv, hl~tvcvcr, i.: th,it L V I th r n ~ltlpltl olx~r\,nrions [he patt~rr~:. ,ire frequently no1 a4 ~ l e ~ ~ r - a5 c u tlinse l ~n Fih~lrc.h.'). We can ~ r l ~ s c r v c lit hvrcnct5 ~t di t f ~ v = ~ time n t ptbint- s~rnplydill= to ch.>ncr n n ~ lmcasltrcn~ent ermr. The prrlbltrn ' t h ~ n b t ~ c o n one ~ ~ l i~ r f what con<titute% a rral diffr-rence I ~ c t w ~ c ditlcrent n time ct.ric.; Iinrs. 'l*lii< isqtlc is ton I n r ~ c to hc cfeart with ~lrfinitii cry hcrc. I f you 17la11 to d o this lypc of .inaIy<i5 cnsure that !.oil c>bt.vn r a t d s t ~ i t i ~ t ~aJ\bicr c a l beftlsc. undt~taLincthv analy31.;.

Summary

'Tbi.. chsytcr hac cx;lnlintd

the I r r p of ~ tllt~ ,~rlaIv~ of i +~b\ptbrrrncntalcl.it.1. Iwr fhc most pnrt t l l r ~h a w logic I\ thdt of c.omparinK ,:roItps In rralatlon t o h t m . t>ncli FOIIF ~lcrtnrnm t c t ~t h ~ o . ~rtc~m L nri;ll~It* c lllic b a ~ 11f ~ s the cq~)rnl-ranw~n can frc averazrc. ~.r.lit~lc. ~liqtrib~i tion.;, v~lrrance r v m,Tnv oththr rc)~amctrarl?tic.; (11 di<frihi~tir>ns rhrrc i + a h ~ * ~ i ~ l l c f ~ ;Irr;lv r ~ n )crl : st,itistics t l ~ ,Ire ~ t .iv;l~l.~hlit(1r t h t ~ an;lI\.+l+01 e x p c ' r ~ r n e n t ll,lt,~ ~ l In tbrdrbr ti) ,ls.;r.;t

112

EXPERIMENTAL DESIGNS

14

PART PI1

12 -

LONGITUDINAL DESIGNS
-

7
--

TYPES OF LONGITUDINAL DESIGN

+F

-I -G
You will find a wide vaiiety of IongitudinaI designs in the literature. However, they a l l share the common goals of measuring change over time and of doing so by collecting data concerning at least two time points. In both respects they are similar to the classic experimcntal design that has a prr-test and a post-test. However, longitudinal designs do not typicauy have a randomized control group.' Developing longihdinal designs involves a number of critical decisions. The decisions taken will determine just what type of longitudinal design evolves.

01

02

03

04

05

06

Observation time

Figure C7 9 Sonle otrtcornes of

nil

rx;rrini~ntnl vnriablr on a

filnp

sfrim

with thc seIection of the appropriate statistics this chapter h a s outlined the main considerations that need to be taken into account when selecting the particular method of comparison. Finally, since experirncntal designs normal1y involve the analysis of change the chapter discussed a number of issues about the measurement of change: the usc of change scores, truncation effects and trend analysis. Further problems witli measuring change will be considered ill Chapter 9.

1 h7iIlthe same cases be followed over time? Here the choice is between trend studies and panel studies. Trend studies (or repeated crosssectional sbdies) entail collecting information from comparable samples over time but not from the same people. Annual surveys such as the General Social Survey in the US, the British Social Attitudes Survey, the Eurobarorneter surveys and the International SociaI Science Program Surveys are examples of trend studies in which similar questions are asked of a new sample each year. This
design allows the tracking of changes in attitudes and behaviour over time. A panel design involvcs repeated surveys of the samc people. This design allows the tracking of change at both the aggregate level - and the individual level (see Chapter 9). 2 Will dafa be collpcted at onr poznt of time? Although panel designs involve collecting data concerning at least two time points, they d o nut necessarily q u i r e that these data be collected on two different nccasions. The prospective longitudinal design entaiIs collecting data on several different occasions. In such a design a group of people will

Note
1 This ts t r u e if ~ndlvldualsare the units nf analysis in h e ~xperiment. Ef the cnuple M'3S the unit of analysis then having the couple would not violate Ule reqt~isrnrrri ts for indcpendencr.

11y I , , t k I 1 i ~ t v t . ~ l lrr1L , ht.13

, I I I ~~!L ~ I I ~ I ~ ~~ >C i \I ~ t b s ti ! ! L * , I < ~ wi~.t-,il sonit, Itrh>r!? tinic. 1 !I<, rr7frrl-:'h.il r;?. yrlnt.! ~lc,<i~m =nL6ir-r c.nt,li 1 caT!c.r-ting i1~t.i un r7nI~ r>tlla tlccxiot~. l i ~ t i and ~ * changia ~linietw~c>n.: to tlic 5 t u ~ l y

rti~

,wL~nl: pt~ty!~ to. recall w h ~ tt h ~ n wrm ~ s likt* a t sonic ~vrllttrp0111t "'I h m r ,IF, IVCII <I< af the prficnt. In !hi\ \\'a\. rve can nl-rtnin cclmc ~ndicabon of thc cvtent iu which the pLarson ha\ c h a n j i d t n ~ Irime
;Irt> rdlt,~ined171

I+ongittirlitla1cttl\i~ns wri vnrv ~ n c l u ~ it t! ,i ~ lf t b l l t l t t ~ n g *

OII

manv other ~iirnenstons as wrll. Thcqth

5rtti11,: Thc, logic t h ~ \lonh.itu~li~ial rlt'.;i~n c.ln trr ,~pplii,d in qc!t~ncs r,~ri::inc from t l h ~1 5 h l ~ controllctl lal,trsntor\. crtztnticm. sm,~llt,r .;cntc grnL1p.i (z*.g.srhocr! l ' f . i ~ + ~p > ; ,l r t ~nf an rrrg~ni;.atron)t o 1ari:t~ wale r i , ~ I ~ o nsurvevh. ,~l T i - 7 The g a p wrer~vhti-h cti,~nxe (art be m t , ~ q r t r ~ can *d \'an' from a \.er\' hhclrt p~,riod( r , a ~buton- and a f t r r mtnasurt.r t l nt2ihldta.; aftrbr sccinq .> f~lrn) to nlanv \,pnr\. rlrt>h'nrqonal Child Dclrtdrrp~nt~nt C;tuJ\ 1% a n t.\,3n1pIt- r r t a l t ~ n s krrn panel stln elr. I-lit# ~ t u r l v llegan In IqLH ,\nrI int Iride~t ,111 chilrirran born in R r i t ~ ~ %jehrcrt.n n 7 .lncl Mart-h. Thrstt, c h ~Jrrbn l l~avrb Iven trnc ked n n r l in1nl\*rrlin +uliqcqut-ntfnIIo\t,L I PL I A ~ c(jll~-a ~ I titon\ i r i TQfi;, lL't79,It374, I L + 5 i , leu1 , I ~ I L ? ~ II(~~I i l ' r r t r r l ~ r ~ 11f r . f / r l r r q p c l r t l t - L o n ~ i ~ u d i n5tctdic.s al will irnrvin trxrmsof hnw nian! trmw ~nforrn,tl~tm i< C ~ I I I L T ~ P L ~ Irom tlic *;an>tb cct nf p c t ~ p l e in n pantrl > t u J ~ crr , ilitftart~nt set>~ r p f ~ o p l ns t in n trend + t t ~ dT~ h. r t;7mtrtrl; '7-t~p' TV prt~grnmnit~, cv11ic.t~ h a s ftdloweif ..I p u p OC cllildrrn ovi3ra loiig prlr~otfi t(r .;ec \\-hat changcs ha\.c takVn place in their I T I ~ P ~ , i.; an euc+~llpnt c x . ~ r n ~136 l e a pancll tlrstgt~.I t ccmn~mc~nced i\.~th chrlclrcn ivho W ' ~ I . V\t3ven vcnrs L ~ L I,~ncl ha. rcvisi 1r.d t I ~ c chi ldrcn cvery scvcn ycarq tinct. Tht*scb rhll~lrr,narc nt7w in !heir 40.; The I3ritit;Ii 1 lrrusehr~ld Panrl 5urvt.v pruvidcq an c ~ a m p l r n F a pane1 sturlv with more frcqut.tik data c n l l w t i c ~ n It ~ .r t ~ ' i 4 i t its ~ ~ i a t i n ~.;amplv ~ ~ i l e w r v yrar. ' * I Lrrnqitudinn I survey* rvilF atco 1 arv in term< I r f whvthcr th1.y inx~c)lvt. plannrd 'fntt-n cnticw,' or whrther tht-v reFv on ~bsrr\~ilr tllc g resul Is elf nat11mHv ocrurriny: rrrent<.W h t w thwc nr13 pTsnilccl ~nkt~rl.enticm.; the lon~itu~fin,rl dt-ilen is qimilar f r l rhe c!a*cic c ~ p v r i r n e n t de5iqn ~~l rlurcpt ~ l i r 7 ttt~tnrtawill l,tl nu r,i1lCjt~rni7t*CI contrnl

"

C,tu.;al anntvqit; ~ n t ~ i e ls ql ; r b l ~ ~ h ~ thc n r :trmpor~l order of rvents T'llis i< nrt-i-+wr\. \ltbiau>c3 a llasii ttmet ot i , l u s a l rr;lson!nq I < t h t ~ a cali+r r n w t prcc(.de ~t.;cltf~ctin time In crtlss-scct~nnal r e s ~ ~ i r cwh1.i-t. h all d a t a artcolltcted at c m c .p i n t of time it can be Jiftict~ltto rvctablisli the i~rilcr in r~-Iiichr t c n t s occur (we C hapirr !(I) f lt~it.rvcr,It>neit~rtlrnal dta.;i~m. ~ariir.ulnrlv prn.;pt~c~~vt~ ~ltqsignz,i~nablr. trackiilg the rwJcr i t ) which cvrb~it.;take* place I : Iex,~mplc, ~ (I ~-rtrhs-ztbrt~tmaf \trtdy miqht c ~ t a l l l ~ s ah c o r r t ~ l ~ ~ t I3ctwtvn ian tinemplovn~c~rt ,~nd mental hralth hut wo~rlcf have di 1 Ficultv t,.;lnhli<hinj: whtathcr unt~mpln!~nient <~flt>rted mcntal h t . ~ 1111 or vict. vt3rw .\ Fongr ~ l r d i n ~ l F e ~ i g Irmvri n, rr. could m e a s ~ ~ r thc r rnent~l Iit~althot ,T \nrnpltb cli cmplr>);e~l pcnplc and thew qec ~ v t l ch.~ngcs ~~t to mvnlal lit~.llth,1 1 nnv, nccllr among thna. that cubwqucntlv hccr~mc u n t ~ n i ~ l t ~ v r~ ~Irllt.t ~ m a t i \ ' . t~ nee I t cl%uldc t u d ~ .untmplor tad ~ m p r t and * ol>+~arve i.h.incrs 117 rnrntal l~cnlth,irni\nfi t l i ~ ~ c tli,lt ' get .I lob.

'gi-CjLI

12.

In ctutain ,ilid~c<of c h , l n ~ i , i t l.;l~ibe c I l i f t i ( - ~ ~ lto t t=itatrli~~I~ ~ ~ hw~rt t ~ of t change rlccrrrs. Fc~r~unrnyllc,age and pilltical c-tmserv.l!icm art, rorrl*Enttd: p t > l i t r ~ a c l n ~ s ~ r v n t ~ is < nnlore i p r r j n ( > ~ l nantrlnj: ~ ~ ~ I otdrr people tlr,~narnonK vounscr pctlrlr. Hrriz.r\.et, tlljk rorrvlntinn in itrelf d o r ~ nclt e\plt3in thr. link. 14 it, tor tx%nrnylc, bcu-ausc ~eOl7kbtncc?rne rn(brt5consrarv,itivtl ,I< they grow c~ldrr,rtr il/lcs tht> corrcl,it~onr~4l~pct surnethinfi abritit thv rra In \v111ch Ihc ol~1t.r petlyl~.yrriv lip 311~1 formi~il t h c ~ r pc>ltttcaI\ . I ~ ~ wFor s ? exarnplc, tlir t>ctmomicinscc~rritv c~prricnctlrl bv tbr jirricra tian growing u p d i ~ r i n g tht, Great Ilcpresqion of Ih(1 1 9 3 0 ~ nlav hc rcspnnsiblr inr Iht. political cnn.;crvn tism nf thc c,lcler gtw~ratirrn. I hen again, thr. c c n w n atism nt older prople mnv be d u e to ,I t ! ( ~ ~ E ( > l l ~ l r r - l ~ t cfitbct: nl pcoplc mav brcnmc more c t ~ r s e r v a t ~ as \~c thrv grow nldcr. Tht-v ma\. hecornc~It-ss a Jt.pntun%us,nerd grca t ~t ccrt.a~nt\.or b ~ c n m c ~ niclrc cct*pfical ot thost. w h r ~grrtlrnlst. t l l , ~ t rh.ltik:t~ will brin): ;r btatr~ar ~'11~l TINd . r>nlirrr.1 i,rblrh IY,IV tcl t ~ ~ t ; l b l ~ ivlictllr~r +li pc>lit~c;r t cnii~i.rt.nti~n1 i- ?tit. n.cr~lt a1 ; l tlc,\.c.I~rpn~c.ntd t h i i ~ c t to t r ~ i k thc s a m t intli\,idtr;ll.ovcbr I~rnt* ,ind s c ~ . li t h c ~~ l i a n c t a< , thcv ai;tl.

Vurpoqek of loneitudinal design

Tlit'rc
Tht3FLtr1'PW ot [ ~ v ' k ~ i t ~ ~ rdtb+renc f i n a l c , ~ tt arv ~ frt>m tht, drscripbiv~ to thc ~ , ~ p l d n ; l l t ~r\< r y .; I dcwripti\.r> tclul the. I alue 111 the I ( , ~ { ~ i h ~ ~dcklgn l~n;ll
01

iq a prc>blc~ii with ;~ttc,ptirll:.i trackin< Gtralrcv. IF wt. track -1 Crtwp ifnrrng p t * o p l ~ C ~ P T 11nw ant3 trbct)rvt. that 1 1 i t - i . btac-l~rnt. rntlrc conF C ~ N J ~ ~ V C w c s CLInllothe stlrc th,it lhv ct~.l~i!:c in . ~ t t l t l l t l t * c .1 5 thc' i-ri<tllt tli

~ f ~ i ~ z ~older. i i i g Fveqcinc in the ~ ~ ) c i c t y rep>rc?less , of J ~ P ,m ~ g h tbe bccon1111g more c o n s ~ r vtivv. a M'l~at ~t is nccc-wary 5 0 do tu a n w c r our quc+~tion i s to track diffcrlant agcmgroups nvtlr tinlc. Dr~ing this ~.n~ilrE enable us to Fnak at ageing or dcvc~lt~prnenkaI rffccts. T~;ltllc 7.1 reprcscntq thrh w a y in rvliich w7c could track a mngc of age cohort.: cwer timc. For each n p = cohort (those born in a .;pcciflcd peritid) M-c c0~11d Imk a t any changes in pr~litical conwr\,ati<rn CIVET tht, ?/)-year ptlriod ( 1 YHO-2000). If r511t-hcohort ~ T P I \ . more cnnwi.vahve crvrr the 70-year per~odand if thev did qo at thc same rate we roi~Fdconcluclc that thc c h n n g ~ tn more conwrvativc posihon is '3 rlr.;~c,l~~rrr?rr~t~tn/ pff~bct.That iq, srorzing older re.;llltq in pci~plcbecnrning mvrc pc~litjcalfvcrlnservat~vi.. In t cxampl~,, l'rtbl~. 7.2 show\ Iliat each cc>liort tl{$corne> morr cons~rvntive by 10 poin tr rach dccade. F~~rthrrmort, re~ardlcss (if the cl~cndc.,pec>pTr , l ~ e d SF-60 have a conwn,ati.;m <core of 50. In othcr \t+ort&,rrgardlrss ot the pibrind nr timc in history, ptwplr of a given age obtan a particu lnr level OF consewn tisrn. Furth~rrnnr~, all coliurt 5 display A similar Ipvel of change that is indcptlndent of thc particular histortcal periru-i. Quite <imply, timr, not historical puriod, prod~lce.; chan~c, k l n ~ only i onc cohort hctln backed (e.g, tliohr Imrn In thc 1950s) we would h a w observed that as tlwy p e w older they bc-came more con3rn.ative. Howe\.cr, 5r.c M-ouZd nut know whcthcr this \\.as beca~~sc* 06 agcing nr becausc this parkict~larcaliort was sc>rnehow di.;finctive and bccainr lnorc conqrrvakivr brcause of some pecttliar effect of growing u p in thr* 14hC)s and 19705. To d istinpiqh behveen dcvelopmrntal (ageing) effect5 and hrr;tnrical (cohort /pcrrtrd 1 effect5 we need to examine rnnllipfi= cohorts over timr In some ~ v n ha\,ing ~ ~ s mu1 tiplt cohort.; iq cornp~ rahlc. to ucing an rxprariment,~ l design w hcrc thc control jiroup prm idraq a point of ccrrnpari~or~ to help interpret change$ in the cxprrlmental S r U U F I f I l l r a ~ . u h ~ rdid t < nnt bt~rtlnicrnr)rthconser\.,~! lvca as tlic,\ 1:rew oldtar t h i s w o l ~ l dindicatc ihe at7hc~nct. of n dt~veloprnrntnleffect. Ilower~ci; In Tablr 7.3 cnnscrv,ati.;rn 5cnrc\ deptln~l cln when pccrpir wenBborn. Thnsc born tn IQY-q 02ltained a cnnwn.atrqm scc>reo! 70 in lQW and rt~rtr~rr~rt-if that conservat~vc~ tl~rnugh 1Q40 a n ~ l 2000. T l ~ c ~ born e tn the 10(10.; obtainr-d a con.w-vntisrn s c r w of 1 0 Irr 1980 and rctnn!~lr,ilat that lrvcl

2
...

5 5 2

5:
ir

rr

,1

;>

f
fA

. ;
7

.- , z -

-.

--

. d
L

:J

zg
C

--.
'" 'I.

kE s.5

2
I

"2

<: :

:. : -

;g z

:,. G -.
LL

L z < s2 z 7 ..:
h.,

+
.

<

&,C

ar,

.-.
?

3;
I 5

Pl

k.

LONGITUDINAL DESlCNS

TYPES OF LONGTWDINAL DESIGN

tE~rough1930 and 2000. Differences in the cnnsesvatisrn o f thc agc groups in, say, Zfl[)O is becairse nf period effects rather than because some people are older. The nldcr group in 2000 has 'always' becn more consewative.

T~me 1

Intervention

Time 2

Measure variables

1
Measure variables

Longitudinal strzdies can hcIp researchers study 'carc~rs'(Menard, 1991: 15). At a purely dcscripti~le level we frequently do not know how various 'careers' develop. Obtaining an accurate sequence of the way 'careers' unfold can bc crucial in developing sensible explanations of these carccrs. For example, in the 'career' of a drug user what is tlw sequencc of events? Do people start off using soft drugs and go on to hard drugs, or is there no progression? Do people mix with drug users and then hcgin to use drugs, or does drug use lead to participation in a drug subculture? Understanding the order of events is critical to understanding the pathways in to drug use and developing intervention strategies. Work histories can bc tracked with longitudinal studies. We might look at the agc a t which peopEe move in and oul of the workforce, and the types o f p b changes they make, and plot this history onto other cvcnts such a s life stase events including marriage, having childr~n, divorce.

Figure 7.1

Sjlllplf

~ ) m ~ p " t i t l{el n t f ~ ldCs;gr~

Types of longitudinal design

of childless couples and observe the extent to which men and women share the domestic work. Wc could track changes over a period of time during which some couples would have children while others would not. Wc could then group the sample into those who became parents and those who remained childless and see whether those who became parents showed a greater move towards a gendered dividon of labour. We may find that the parents did adopt a more gcndcred division nf labour. The problem with this approach, however, is that it does not take into account the fact that the two groups may bc different in manv ways - not just parentl~ood(e.g. age, ethnicity, level of workforce participation). Beci~use M~P I I ~ V C not controlled for thcse possible differences w e cannot be sure what is respnnsible for the final d~ffcrences in t h division ~ of labour. Howcvcr, if our sample 1s sufficiently largc and diverse and wc collect the relevant Information we can statistically remove the impact of known differences betwwn the groups and at [east eliminate some of the alternative cxplanations for the group differences.

MULTIPLE POINT
This simple design (Figure 7.1) requires the collection of data at two poinls of time from the same sample. We may make an active intervention or we might simply observe the effect of naturally occurring events between time 1 and time 2. The simple prospective panel design is very much like the top row of the classic experimental design (Figure 4.2). That is, the design has an 'cxpcrimcntal group' in which change can be measured. However, the design lack5 a randomized control group, which produces the problem of not knowing whether any change i s due to any 'intervention' by the rcscarcher, thc elapse of time or somc other influence. The prnblcrn af not having a conlrol group can be alleviated somewhat where a wffiriently large and d~vcrse sample is used in the study. Such a sample rtllorvs us to drtt~rmincif, over timc, somc pcopIc experience 'interventions' while others do not. Adopting this approach can produce comparison gro~lps that provide a useful, but limited, way of examining thc effvct of specified 'interventions'. For example, we may be interested in establishing whether or not having children leads to an increase in thc gendered division o f labour in thr home. We wnuld begin with a sample

P R U S F E C I ~ I V EP A N E L DESIGN

This design is similar to the simple two-point d e s i p but simply involves more data collection points (Figure 7.2). We might h a w spccific interventions between any set nf data collection points or simply collect information about relevant intervening evcnts between each measurement point. Theoretical considerations and previous research would affect what are considered 'relevant' intervening events. Like the simplc p a n d design this multiple point design reqlifses sufficient variation in the initial group and wfficicnt diver5ity in intervening experiences to enable ur to construct comparison groups in the dnalvsis stage. 'nit purpose of multiple data collection points is to:

examine long and short term effects track when changes occur (e.g. in 'career' analysis) plot the 'shape' of any change tdentify factors that precede any change ((or nun-change)

I
For cuample, f i t * rnrtv wish 10 (rack the wnv in which marriages change nvvr time. Wc could lonk rlt these ch~ngesalong any nurnbrr uf rl in~cncinnqsuch sr level.: o f intimacy, t r h o eucrci.;es ptnvcr, the wav in r v h i r h ilomeqtic Ial7r~rrr is .;h,lrt.d, happin~*5s, Eer el< n t ctmflict and st, forth. \Vc cn~ilrlIrask chnr~gt,sIn t l ~ e s c asp-ct.; of thr rc.l;ltionship trlrm cvhrn n ccluple b r ~ l n s to livr t o s t h c r a n d at regular it~trrvaIs thrrralter - SJV t v e r v two ycar5. WP W O L I I ~ a l find ~ out about otl~ersliai-rgcs nccu;rinR <ctr<cm o u r tracking \.int5 I r c. Icvel of ivnrktorce partrcipatlcln, birth of i h ~ l d r ~ nPmV n , pcnple ~ u t -az l~ a parvnl nr p a r c n t - ~ n - l n l v lolnlrig the h ( ~ ~ l ~ l i o or l dcllnngrd , econnrnic circurn.;t,incc.;). II w c wantct-l ti) cxplnrt. !tic effect rrl having n child on marital l i . ~ p p ~ n\vtn ~ wC O ~ I ~ Lby ~ , hnckin~ couflcs rt8gularly rwcr a long prrtod, rl~\t~n::rri*hIwttvt-rnn <hurt, rntl~li~irn and Eunq term t ~ l f ~ ~ c IZfe t \ c ~ l t l r .;PC i thth t t i 7 5 , ~ n ddii\rnk that n two-point dr\rt.,n simplv ~ r u r ~ not l d pcrmit
Inlt~at sample size

* Dropouts
I

I + Dropouts

1
TI
J2

Final
sample size

T?

T4

T 5

+ Dropouts

--I \.l-ht.thcr o r not ;r simpIe trzt)-point (prc-tcqt, pclqt-ttl.tl cjr a rnztltiplr. p i n t design i q trscx1, a deci<ion will nrcd to be madc about what fo tin cunccrning dropcmts. The morr points a t w l ~ i c hdata arc* collected and thr longcr the ~ l ~ a rion a t (3F a \tuCIv, the mnrrl dropnuts hrcr~rnc a pro1,ltm. Fignrc 7.3 illurtrates a panrl c l ~ s i p rvithout replacement. T1.terr. .Ire h z ' r l kc? prc~ldcm. jrvth thi5 rir*5ih~. C h e iq that in a nllil~rpoint qt~rrlvt h ~ curnulativc drnpnut nf rcspondcntq can rrksult in an unacceptabl!! +malt f ~ n la sample. Thv other is that the final s a r n p l ~ can become unreprcwntativr <ind Slit].: (hrvnten ~utc*rnaIvaliditv. Unrepresentntiveness qtcms frnm t ~ + . i ,.;c>urc.t3\:h r s t thc pctlpte who drop out arc normally diffrrcnb I n prtrcuTar ~ Y ; I I - . from thmc I ~ * I I O remain in the stud^,; .;tcond, in mnnv pnp~~l.lt~o RC'I*' n s rntlmbcrq inin and old members dcpart. E11t.n if thc ni-i~:in;rl panel rrrnnin:, r c p r ~ ~ l ~ n t ~ of ~ttl i l tv ~ pnpulation c a< i t was ~ti11t,11 t 1 1 ~ j ~ , ? t 1 1 ' 1i ~ ' r rfrr;t ~ 1 ri r!rrhTri,a < t h ~ , populati~ln~ t \ r lcIlan~t3. f it ma17 \?r,cr)n~r rtnrr7prta.;mtati\ t b .
ln~~ral

Dropouts
lcDrOpouts

Dropouls-

..........
Recruits ,

Aecrurts + New --

-----

Recruits

,
Recruits +

u !
Re~wils +

Ovt-r time, nntl depending on the i r i . ~ Iof dropout, the panrl t t r l l becrwnc. cornpcrqrd increaslnqlv i7f d i f f ~ r c n tindir.~dnnl.;to h t h o q f lwhir wt.rta ~ n ~ t ~ a l srlected. lv IY h t t h p r the pnrlcl maintains thp initial

Onc. I\*ar of nrlrli-t-<sing thtw. prnblern. I.; tt, rrnylov s o m r tclrrn a i rrplaccrnent for sample dropolrt.; and rcl rnclttdc net\. tvpc.; ot rn~mbcrs o t thrf prjpula111tn.In this t r a y s,?rnple <17t1 and, h n p r f i ~ l l ~ repreccnta~, t~\~t*nc.\:. ,Irc rna~ntaincd. TIII. lypc nf dtbsi~ni h illus!ra!t.rl In FigurP 7 4.

rcprcsentativencss- depends on the metl~odof replacement. Given that peoplt. do not drup o u t randomlv, r e c n ~ i t sshould bc rn,~tchcd,wlwre pnss~bFe. with drrlpnuts. T h i s can be done a t a broad It.rcl by ensuring that rcpIaccment5 are of the Fame age, sex, race and cia.;<. However, $11i'Il matching will nlrvays be Ips.; than perfect and will Imd to a Ins? of

r.~n~loiiinr.;<, tIlu5 niaking it dlihcult tn cxtr~polatr with conf~rlrlnccfrom thr, \,irnplc. I<c~pl.ic~.nimt alqn intrt,d~lctww r n e difticzlltic.5 with analysis. Sincc, in Fiq~ln, 7.4, many r.rf tlic wst+ a t T,, arr difftwnt rases from those it1 the k.~nlplt.at T I , rvc r l c i ncl! h.77 c measures nrt,r lime for m a w 1ntlivi~lunl5. -r]tl\ .~listnce p i data wC7r t ~ n w for individun!~ rnmak~s analysik o f clianqe nf X I I I ~r?rtfrivrltrirl It.i'r*l difficult. \%'emay be rt*strictrcl to l o n k i n ~ at change a t an ;lt;jircgatc rathcr th.ln J I a n indiiriclu;ll It\ el (see Cliaptcr 'I). 1 1 1 s can br t,rrrr.nme 111 p irt rtdlecting sornt- rctrosp~ctivc inftrmn tinn for snmc \ ari,~blcs(see br*low). We can nlso I I W certain prnrrrlurr~s tt) rst~ni.itt* wllnt we arc3 Irhc'i,~tci 11nt.e bccn Itkr at earlier stage*. To chiev eve th~s rvt9 r\.auE~l ilsc lhc inh~rnlatinn wc hart. nbout similar caqrs that I ~ ac l l ~ t b t , prcsent ~ in tht. *ample from thc btyinning.
c7rlcl

This is ,I further d r s i ~ n th,lt provides ,I mnrr systcrnatic and ranrlomizcd w a y ( 1 1 tlc.aling with %anipltl dropout and In so doing rnaximi/es the ~ytt-rn,>l \.aljdity of thtl d c . ~ ~ nI.t 15 called a rot;ltinc panel rTc\ign. -1 171. <l(.<rqti I - 1% 1 c k ~ 1 1 ~ I + I > In<qhxdiC< ! t h G i trtaqillrp f r e q ~ ~ ~ n ,n ~It ~ a ~ k ~ r ~ = s . Lal~trtll* ttlr-4-r ~ L I T Y C L7rcll ~ . . ;iilc such an ru.implt~.I'anel data nf this kilrd rn,ibltb lh<. ~.cs~;lrcl~t.r tcr r t - t nwrli3t happens I t ) t cn~ploymenl patt~*rns of nn i/rrtrr~rrllrol(e.g. how long i t takcs to find work, efforts tn find work, r n o v t ~ ~ l ~ i in ~ nand t s o u t trl ~ub';ctc.) at frrtlitcnt interval< ( e . ~rnt~nthlv). . Tlica r r r h t i n ~ panel clr~ign <ilsominimize< the problcrn of d r n p o l ~ (and t tlie conqrq~tr~n problems t ~\.ith rpprcsentntivenczs) and burnout (drrnsnding tno ml1sF1of the -me yroplc). \.t:rtli ,I rotahng pant4 t l ~ ~ q i ga n serieq of pnneys is establi3hc*d. Fach p ~ n r l Irta~ins : at a diffr~rrnttime but ovcrlapr t~mporarilywit11 at Tcnst somc tvt Ihc other pnncl.;. t3sst~ntiallya sct nf pnncls is created !ha1 have sknggi'ribd starting and f~nisliing times. Rut Ihp same information is ~oll(~c.tc~d from each ~ r l n c lThc . logic of this d r q i ~ n is i l l ~ ~ t r a t t In d f'igurc 7.5 In thv. ~'\arnpIf the F ~ b pilncl t (mrv 1 ) i.; rcttcqted each rnontl? fvr 5lu mnnthc. niter this panrl lin.: conrmenced anoth~xrpanel begins a rnrrnth I ~ t t ntiA ~ r 1 5 followrd !hrt)ugh for six month<. Tcqting of the stkcclnri panel wrll 171, cc%mplcted a mot1111 after Ihe first pnn(1l. A numbcr ol ht,lgp,ert.d parli,l.; <*,Inbe run s~rn [IItanctn~sly. F'lch 1 j l 1 t t t ~ 1 ha? a Irrnitvrl Irtr, ru h u p c f ~ ~ l lpartioiyanl.; v arc, not c.\IiL~~~--lt.rI petlyte. Horvta\ r r , thc ~frrrlt/ can rvtrnd indcfinitr*lv. HI tral.R~n): , i p , ~ r t ~ v u l p3nt.l ar 1ncl1~1dual uIi;lnr;r~ r,ln trc studrrd o\ tar t h ~ t ptar~oil. Ct~ii~equentlv. wtJ t~litalna <erir\ c ~ \llurt f tcrm panel qtudics. Rv marntajning the <hid\. c x 4 L . r a \'cry long ~ r r ~ o t ,~ il , b ~ a~ it -ith cii f f t ~ t ~ t cascs, it aggnagntc% change ovpr tirn13 can be shcllcrl. I lic nclvnntage 0 o C 11,1vrn): cwcrlappin~: panrls i q tliat at an): point of tinlr t h y ~ n l n r m a t i o frtlni ~ ~ nil thc panels car, he1 pcrclledFforcrow-bcct~onnl
h t b

,ln~I!,~i( W Y , 1'.irt 1'1') The p,~rtti.ul,~r ;I& antayc r,f tt~i.; ,~pproach is th,lt it cnablcs ihcb rt~\rarcherto I7icrcaJ<rthe samplc ~ i ~ a c n. d thus rninrmi~t* s a m r l c t3rl-t,r I:\! providing ~1 I,arqrg poolc~l S I I I I ~ I ~ 111~' d e ~ i g n~ l s r ~ provides rnr>rta cascs of hard t c ~ find Kroups (t.g,lonc mothers). In 1~ho11r iorcc 511rv~~v.; lhc. precision id sample estimatt.5 can be partic~rlarl y important. h).the advantagr* of pooling t h r pGantll. for cross-sectional analysis cnnnot be o v e r v m p h a ~ r ~ c i l .

Rr ~TTlplm'i"~ a wries nt ~>rctl,~pping pane\<the rr~+cl.~rchcr can smooth out-thc impact nf factors quch a* tlropout and parli.1 conditioning (.;t~ Chapter 8). I:t.en though p;lnc*l 1 ~ ~ 1 1 suffer 1 snmc dropout and crltlditiontiig LIFtrlrIhc first data cnllrctinn, the impact L > ( thcsc tnctnrs can 1 c . a s s e s s ~ dbv comparing estimakcs from h e scconil panel that will not be sublrvt t o thr cffccts ot d r t y w l ~ tor conditicrntng. Thc same logic a~plit.> at T 7 ~ n thrcru~linrit. d Srncc fresh .;arnplrs ,ire being obtainid continu~~Tlv the rffycts o f burntrtlt and d r o p r ~In ~ tthc oldcr panr.~s can b~ adju<trd (crr. lwrthermore, in any cross-sectional analysiq of the prmlcd saml.7Tc< thr 111clusion of n c ~ . ~ r pnncls r can rnoderntc the hiasing efterl.; o f dropout

r could bc u*i*ii. In all tht, p ~ n Clt.5ihm5 ~ l d i s i ~ ~ c \ ; t ~ iar l a s ~ n n l ccrllurt Using a sinl:ltk cuhori inx.ol\.c< st*lrctinga grouk3 oi .;~tnilarlyaged pct,plr. at one pwnt of tme and h ~ l l n w ~ r it~ over g a n ib~trv~~.ltbd period. l-hihy might ct?~i.;islo C people b o r l ~in a p7rticular ytur, tlip ciass n F 2000 o r Show wlln t ~ i tlwir l first child in 1001.

'Thp pn*blt%rn w~th ii'-lnq a .;inqltn cthort, ~ 1 % ~ d c , nt tl~ t ~ r lrbar!~cr,i5 th,lt ~t I\ irnyt~sc~hle to trill ~ v h ~ a l l ~ th c~ rI sh a n ~ c In the qrrlzl p i.: bccall.;~, the K r n u p i< c c t t ~ n c rlldcr or I ~ i a l l w clrcxrvcmr I < changing. S ~ r p p that ~ e a <tud\# on thta \cxual bcliavici~rrand krrnwlt3d~i= nf ,r Erouy 111 ?[I vcar old* cclmmrncecl in I Lrq(F The .;tudv rllllcctcd information about r n ~ t t t ~ SL s I C ~a% thc It1vt.l of SCY LMI,~ctivitv, numhcr o f partnrrc, a n d knowledgr and pmcttcr of safe qck t-lad thcu>20 vcar rrlds bwn t r a c k ~ d over r hc npxt 15 vt8;Ir\ the itr~d!. probabIv wclul J I1a1.e found tha! thcqc ynltnq pcople bccnmc mrjrc nindernte anif 'cnnscrvativc' 5~;rr uallv itit11 lhc rJl;snge nnf time. For 1-~amy-rlc, the study tvould prohablv find tFrnt over tinw thcxse \-c~unr:people had Itb\< .;CY wilh Iexvcr pcnllle and adopted .;.ifpr \c\
/T~c~~C'C<

Years o l surveys

'1'0 lvhat is this cl~ancttcl btt attrit-utcd7 I.; i t .;irnpTv a matter of ~ c t t i n ~ rdJt.r7 I t might prrtfy Ire that: ;ls penpl11grow r1ldc.r fhcv c~hktlctlorvn rvitl? onta partner. Ht3t 11 1 l o ~ 1 I 1 - i also br ifuty to histc~ric~l effects Zl~iringt h ~ Iiiqtnrisal pc*rioil in ~ . l i i t + I the\. i are growin,: IIIJPF, AEDS cli+c(tvt~rt~~E 'rliih ~iissc3~erv gin>.; riw Z n nurncmuG rncdin cnrnp.lign5 d c . ; ~ ~ n c d to cI7angta qlx\i~al pr.lcticc.;. It i.; in~pr~*itrle to t ~ l horv l much the cll;tni:e tr rluc to a q e i n ~ eifects a n d Iluw mucll i* r j u ~ .ti1 pcrlcxi e!lec7tr u n l e ~ q cli~nct" tlrnonq .I ranee of ace g o u p < ,Trr ~ i r n ~ ~ I t , ~ n t c ~u onu m ~~ln ve i i Multrylc colmrt rlcsiqv, ovtlrcornc this prnhlenl

The main feature t r f thr. multiple cohort ~ L ' S I ~ is T t h >prt>,~rl ~ rif cuhortq T l ~ i c cprcad permit5 cornpatiqons hutt\.~cn cohurts and t h u ~hrllp.; d i ~ t i r ~ q ~ t-rct~zw-n li~h ag~ing and p c r i ~ d cffccts. It i. not nccessi? tn est;lbll~h qeynmte cohorts at thc beg inn in^ of a s t ~ ~ d 50 v . lotlg as the sample h ~ . ;a suffrclent divrr.;~tvnf aKc grclupq the colrort~ rat1 bc c..;tt~bli~ht~rl sutrst~q~ientl!. at t l ~ e data a n a l y + ~+:aKc <
the year 2001, thow who were 50 vear old< i n 2000 are now q5 years old. T/ic sample can tell us nothing about the new 50 year old.;: thaw who h~rn 511 i n 2005. The distinctive aspect cjf the cohort ~equentialiief;iq is that twcr limp cohort< drop out of thy studv. This 1s similar in way3 tcr the p a n c l ~ in the rcitnting panel desim cxcept that it iq coho* that arr rr%placrd rathcr than whole panels. In tht* etarnple in Figure 7.6 tve drop a rof~orta f c r they rpach the agt. of 75. ttre oltlcr;t rthurt drops out oT the 5tudv thcv arc rcplncc*d with A nrr\., vclunc (70 y a r old) cohnrr. This newb cphnrt I n t'ffpct \lrllst~tute> f ~ r ~ y h at ~ t the f~rqtd n g r w a s the y o ~ ~ n cohort. ~e~t This design rnablrs 11s tn do three things:
1 It enable< ur, in each tire-war sur\*ey,tu hak'r J. nerv crocs-qectlonal sarnpl~ ni ppople aged 50,55,hO. 6 . 7 0 and 7 5 . Fach column in Fleurt1 I h reprewntq n .;cp~ra te cms+scftional rample. Wc could a n a l v ~ c

A Ic5* common but prltc.nti,~ll\pn\verful drsign i s onc that usta.; nlulsiple cohort4 and ha< a form of panel rcncrval elver hmcn(t~gurr, 7 6 ) .One of th? sllortccrminp I ~ F thc p n r l r l e ~ i p i~ that i f the panet laqh for a n ~.utt>rntlc(f pt.rirlt? tlie 1vIic4t~~ a n t l l ajir.: ,ind the pnt.1 tviF1 ntlt ~nclude \'ounccr pt-ode. Con.;cr(zic~t-ttl~,, ah thc. panel ilxe. nt+thinq I. Ivarncif nb(wt czmt11rnporL3r~ vcwnc pt~opltb For euarnplc~,r\.e r n l ~ h t cnrld~lct.I I ( m ~ i t u d r ~ i .ai ll u d ~ (71 ;1gt.ini:. I\'ta w a n t tr, knr3~rw h a t i s 11 tikc, t t r Crutv rdd H~IW do old*-rp c ~ p l r think ~ al-rortt ,rqrlnt: and rn~>rt,~lit\.' 13rl pcorlc'k V I ~ \ V exycriencc,\, ~ , ptrcrptitin* and [car>ch;rnge as the\ gro\\ older' 1%'~. 11avtl plenty sf time Z n track c h a n j p and decidc tn incIi~rlt= cs>hr>rt< rrmho, tti thc \,e,$r20fl13, arc (isrd 51). 5 5 . 1d1. h i , 70 and 75. ~ V can P at trrrtl to h , ~ c k p . c 7 l > l t l throl~gh to t hc year 207;. 5 n c e n c have- d ~lFt*rent cr~hortc rse can ctrrnpart. chang~nc attitudrs Iltt<vrcntht. coh(~r!c. .l.htl r!r>tllcm 1 st h a ~ by

TYPES OF LONGITUDINAL DESlGN

tl~is in the same way a s w e wor~ld analvsc any cross-sectional survey (see Cl~apter 12). 2 Ol,er timc w e would build rip panels for each cohort. We could track through those who were 50 in the yeas 2000. By thc ycar 2025 we wor~ldhavc trackcd thiq cohort right through. We can do the samc thing for those who were older except that the panel for that cohort would run for a shorter time. These panels are rcprcsented by each row in Figure 7.4. These pancEs allow for the standard analysis of individual change. The multiple ~ n e l also s prnvide multiple cohorts to explore thc c f f w t of ageing as distinct from wider societal changes (see carlier d iqcussion on the advantages of multiple cohort studies). ?I It enables u s to plot thc changing experience of an age group over timu. Because wc have rolling cohorts (i.e. we havc information on 5n year olds in tlie year 2000, 2005, 2010, 2015, 2020 and 2025) we can track how the cxpcrience of being a 50 year old changes over time. Sirn~larlv,we can track what it is like to be 75 years old over time. Does the expcritance of this age gmup change over historical time?

TI

'Intewentbontavent'

Tz

Recollection of marital happiness then

Birth of first child

y l
Level of marital happiness

Figure 7.7 Slr~tpler t ~ t r o s p e ~ + prtlel t i ~ ~drszgrr ~

I of opttrnlsm j at age 50

Recollect~onj Recollection j Recollection : Recollect~on : Recollect~on : of optim~srn : of optlm~sm j of optlmlsrn j of optlmlsm : at age 55 at age 60 : at age 65 : at age 70

Opt~mism now at age 75

So far all the designs have involved following a sample forward ovcr a period of time. This approach has many advantages as will be shown in Chaptcr 8. But it also I ~ a s disadvantages to do with matters such as cost, sample attrition and the delay in obtaining useful data abnut changes in the sample. The primary advantage of longitudi~~al designs is that they enable 11s to establish the sequence of events which, in turn, enables us to make strnnger asscrtic~ns about causal order and causation. In some studies it is possible to reconstruct data over time by collecting all the information a t one point of time. It is possible to reconstruct the degree of change and the sequence of ~ v c n t s by asking people to rccnll what happened, and when it happened. We might, for example, ask people about their current situation or attitudes and then ask them tn provide similar material about an earlier point in their lives. We might, for argument's sake, want to know how
the birth of a first child affects marital happiness. ,l\ retrospertive design would ask a samplr of people who have Jiad their first child about their current level nf marital happinew and ask them to rccall how happy they were beforehand. This d w i p is reyresented diasrama tically in F~hqire7.7. Thc same approach could bc applied in the study of the experience of growing oldcr described earlier. We might take a group of 75 year nlds and ask thcm to recall how optimistic thcv felt about the future when they were 50, 55, 60, 65, 70 and how thev currently feel. This design is r~prescntcd diagramatically in Figure 7.8.

The problems tvith this sort of deqign for thesr sorts ( ~ mntturs f arc obvit>us.There will be a g r ~ a dcal t of distortion (deliberate and otherwise) in those recollections. Not only wiIl pcoplc not bc ablc to rccall accurately how they felt, but we know that recollections of past experiences are interpreted in the light of sr~bscqucnt cvcnts and cxpericnces. All that these sorts of designs wo11Id tell us is how people construct their

past But this problem is not grounds to condemn all retroqpeckive studies. When dealing with certain events, quite good and reliable information
can be nbtahcd abnut the scqucnce of past ~ v c n t s - espccialIy if they are significant events in the person's life. From these accounts nf the sequence of past rvents wc can r~construct'careers' and in so doing solve some of the problems of temporal order so necessary to iievclopin~ causal explanations, For cxample, we might want to examine the influence of family life stage on the workforce participation of women. We can ask womcn retrospectively about the timing of particular rvents. We would ask abnut the timing of their marriage, birth of first child, birth of subsequent children, divorce etc. We could also ask about any changes in lr=velsof workforce participation such as when they entered the workfnrce, whcthcr and \+-hen they lelt thc workforce, any changes in workforce status (part t i m ~ / f ~ t l time/casual) l or the hrning of work (weekend, evening etc.). We couId then plot changes in family life stage and changes in wnrkforcc. participation to help identify the sequencc of cvmts. Given that such tvcnts arc significant ~ ~ e nfor t smost pcoplc wc can reconstruct the temporal order reasonably wcll and build up a picture of work and family careers for women. The study could also ask these women to

1YI'I-S OF LONGlTUT)IU hl. L31:SIGN

rcrulllcct WIIY thtlir ~vt)rkfnrc~ p~rtlcrpntioi~ st~nnzed at ~ i \ . e n pt~nt*. This rcould pro1 rrlc furthtar insight inti3 a n y c , ~ ~ l s n links l betrvccn I t k r qtagr and workforcr p r t i c i p t i n n amnnK wnrncn.

TI

Interventlon7

T?

This retrospcctivc approach is m w t rt.llab!e when we arc dcaling with rnemorablc cvcntq. Various devices can Ile used to hclp peoplc rpcons t n r c t thc ordcr in which things tcwk plncc. Wc can work hack from the nrnund n present to the pact o r focus on 'anchor cvtwtq' and thcn g u ~ s t i c ~ khrsc. For ~ u a r n p l crvc might aqk thc qliwtions: '?\'hen I~TIS vcm~rf i r s t child born?', 'k'crt. vou working at thc hrnr", 'HOWrnt~ch belnre having the baby did vou sthr wnrk?', 'Did v o ~ go i hack to work after !hc bnbtr? , 'Hnw long a f k r w a it?' ~

universtty course)

n g ~ r e g a klevcl. Howt*\*rr,since the d ~ t Artx , ~ collected from diifrrent individuals this tvpr of design cannnt b~ used to track changr* at the

ind i ~ i d u a l levpi.

Although a r c m r ~ llln kage design i.; ncrt ncccssarily a rc*trr~rppctive design it usct~llyi.;. It is a design ~ v h c r r data are collntcd from official rwords. W h ~ r r thcsc* records have been ctlllcu-tcd over time ( ~ ~ ~ l ~ for ally official purpmrs) tvta can construct .I tr-rnpnral qequencc of t.\.r.li t.;. In order to implen?r>ntthir type of de5irn \r.ra rvtwld need to hnvt. prr+clnnI identifiers an t hc rrcords so that we c,ln makc, sure that llnk ri*cord< for thc same yi>r<nnover time. FrL~mpFcs n l rucli rcctrrdh might hr income tau rrstilrnx, social security rr,rords, medical records a l i i l yoli1.e rccnrds. ny linking thcse records wp can build up a life history ilr kcrcer' for that person. The main shortcoming with a rect>rd linking design is the difficult!, of obtaining stlch, d ~ t a .In many countricc confldentialih provisions preclude accessing sltch dntn. Thc other main difficult\. is rhnt <itch rtrorclq tl..picaUy cmlv cnllt.ct limited infnrmntinn and m 3 v not contain key information nlhcc<l;arvfor develoyin~ a n c.rplnriation for the acVqurncc of events. Ncv~r! hclrass record link~ngcan htdp us accurately map n~rta bcmporal seguencp elf c ~ r t a i n cvcnts and thc nature and extent 171 changes over time. I11 this respect these rrcord.; can help teqt whcthcr the scquence i q at I c ~ s consistent ~ with our t htwry.

TIic simple prnspcctivc panel design dcrrrihcd ~ a r l i e rent,~ilrd men+ uring a group hctr~rtban intervention and :i~;lin afterwards. Thi.: design is himilar to thc classic t-xperimental design hilt withnut a cn~itmlRrnup. In the _si~?rrrlntr-tl hctnrc-after cteqlgn, nlrasurt..; art. obtainrd fnrm clnc group a t Ti and thcr-1 frrlrn a cnrnparablr grrltrp nt T? atter w m c rvmt or ~ntrrk mtion ha5 tnkrn place. cctrnprint: the twn grtwtpq ,?ftpr tlic '.entmcntic~n' thc impact of the intrn.i,nticm i.; cstimatrrl. Ior examplt*, nsv il-c,~y bc interested in tlrt. inlpact on mlig~nuk bral1i4sof attending univi>rsitv. WP cnuld estahli\h Ihc impact bv rn~a.;l~rin!:tlic religious bclicfs crf a sarnplc of first-ycnr anrl final-year studrlith a t the onc point n l timr (i.t*. n cross-sectional s f i ~ d v )Rut we would nni kni~w whether any dif fcrcnccs between fir5t-yc~rand final-year shdcn t.; were clue to initid1 d ~ t f r r ~ n c ' c (i.c. s the final-yra r s h ~ d e n t smav havp been different even i v h r r ~ they were first-!car ~tudcnt5). The sirnulatcd h c f o r ~ and after d r - ~ g n would entail r n c a . ; ~ ~ r i the n~ religious belicfs of sample of first-year .;t~~dcnts when they cnmmr~nccd un11-ersity.Tl~m, say thwe or four v c a v latrr, the religrn1;ui;bc*lic1fqnf a sample of linal-ymr studcnts woulcl br nicasurcd. The t w t ~samples would i n c l u d ~ ri iIfcrpnt people. Vic palfern of beliefs of the firsl-vcar students cou!d bc compared wit11 t l i ~ bclirf patterns of the samplim of final-year students takcn four years later. The simulated b~fnrt--after design i s rc1pr~<entedin Fipirc 7 9.

AH the IongitttrFinnl desims d e r r i b r d nbclve are panel dmicns and invnlve fi-arhlng the 5ame p ~ o ~over I e timr. R V so doing, c h ~ n q r 5 ; ~ both t an aggregatr atid an individsral Irvcl c.in bc plotted to ru5olvr iswe5 about sequcncc r l f cvcnts. However, panrl based longitudinnl drsign~arp not the only w n v of obtaining data o l c r t i n e - a t least a t tlic a~jiregrlte lerel. 111rw <iItemative types of Inngitudinal d c s i p s enfail cr~Tlr.ctin~, data a l difft~rr*nt points nf time 5(1 that change.; can be plotted, but thew data are crdlvctrd from different indivtd~~al.: at cach time point. \l'licn, at each timc po~nt, the wnlples artmrcprt-.;tantntir.c of thp pclyi~ laIicr11 the ap roach 01 uqirrg drlicrent sarnplri a l cvch lime pclint c.111 i * i b ~ t i ~ ~ br alry 1 changrs ak fllc

Anuther wav i3f r~bt,7i1iin~ data over timi>that ;~IIc>ws us tn tmck ~ll;ln):e ,?t an aggrrgat~ Ievcl is the repeated crow-~cctinnald e s f ~ n .'Thi.: 1s discussed fully in Clinptpr 10.

Summary
Longitudinal rlchignf prtiv~dr a mean.; of tracking c h a n g ~ nvcr tlmr and of p.;tatrlishing the* sctlucnce in w h ~ c h rf.rntc t.lhr placr - an 1rnportm"t

mattcr t o r ,lrlr ca~rcal;lnalv<ic. Ihlf cti;lpt~-rhas rwtl~nr.d a r . i n ~ t (it ~lilfcrcrrt w;tiJ< 1.11 cr~t-rstrttcting .I I t ~ i ~ p t u ~ l i ~tud!' n a l ,I< n rncdns i d [racki11~ ch,?nge. 7 h r w irlrtudr, yrosprctik e and rctro%pwt~ve Jt.~rqns. tlrsi~s that Fallow t h ~samr . peclplr ovrr t i m ~ (panel ~ t u i l i c and ~ ) llrtrsc that Intjk nt difttbrcnt prloylr , ~ dilfcrent t time prlrnt5 (trend ~tudicq), \Vithin each rvpe of rtudv thrrc are a number crf d ~ r l + t o nthnt < ncl*d tu bc madr about tlir struclurc o C the ctudv. Tf~esc includl>matttxrs such as thc sethng, t l ~ c n~trnbt-r of t1rnt.z , I c t o ~ ~ i k pf o l l t ~ r v ~try, d the- ovt*r.~lld~rraticrnu i tliy qttidy, thc gap bchvrrm WLII~CS, tht* ustl of active interventions, tht. way in which ~ttritirln is d t > ~ l n.ith. t anrl the L I . ; ~ OF mtrltiplc panels. Thr rvav r n t\fh~zh Illrsc Irtlrlous,r n a t t ~ r * art> r r s ~ l \ ~ c m ~ c'inv l particwlar shtdv rcsutk in ,-I i,aricty ot diffc*rr*nt fornis at longitt~~linal d e i q n Tht- main type5 rrt Irmgituclinal design\ ~vcrc.outlined. Chaptrr F cc)rv.ide~ comv o f th(. h t r c n ~ t l ancl i ~ weaknrqse\ of thcst. dcsignt ant3 tlic is+ueq [lint nr.1v-I to br itrmq~dcr~d when imylcrnt-nting Irmg-ituilinal ~lc.;ip~.;.

8
ISSUES IN LONGTDUTINAL DESIGN

4 nur~itvr t ~ the f ~n~~thvrlolo~i pr.~vtical caI, .rnd ctl~lcali-ut~q thnt appli t o rxyc.rimcntal dr7si~nscan dl.;ij appl! to Ion~ittidin~il ~ l c > ! ~ nTh15 <.

clisplcr fuci~sias rln r n a t t t ~thnt arlcl* m w t shnrplv i t 1 lon~iiudinar dtbc~gn\,alth1111~l1 Inan\ 111 thekc rnattr,r.; arc not t ~ n r y u to ~ thi< J e s i p .

Methodological issi~es

In the prcviou. c h a r t c r 1 itirl~cati-rlthat p,~nt.l designs art. rnLlrrrtl bv khrir failurc tn include rnncf(\nlizeiI control gmups. Illis slinrtcnming I* nianact~J,howrver, IIV ~ ~ t ~ l ivariatitln l i n ~ ~uitliinlhc pant4 to ctmstrr~ct gr~wp\. , ~ thr t anal! .;I+ s t a ~ cfor , ctlrnparlson purprltes. Fnr ~.uamplr,thi* p u ~ c ~ of * ta ~ Inn~itudin~i <tudy l might hr lo exelmine Ihr i ~ n p n c tof i ~ n c m p l u i m m nn t llic mental hcalth of \.ourlc p t o p l r IGravt7, 19Q7). Tht3 rnitial Irra\,e (h the +tilcly could ~ n s l ~ rtmth ii~~ ~~nrplnvc\el anrl ~lnrrnployi~d vtrllne people. Ry the h m r subst>quent ivalrcs occur n number clf cle\,clr1prne1it.i 12711l Iinl-P rilken place: .;rime of ihose rrhn wen. cmplo!lrd r n tlic CIrqt wave will Iiavc brcc~rnr uncmployrd; Porn<. unrmplrryr*~l will rcrnain iincrnylovcd; some crnpl(~\,ed rv i l l rcrnain cniplovc~d;and w m r of the ~ r i i e r n p l o v ~will d h,we b P c o r n ~ cmplnvcd. 7 - l ~ c s fnur ~ pclrsibilititlq arrb ill~rstratr=d in T,~blc 5 1. Tht* man* ~ \ . a i . t ~ tree s hnvt), tht* mnrt. \ ,~ricd r\?ilT bt. khc ratigc uf ~ v l < s ~ b ~ l t t i u s . i ; o Ionc a\ mcaFtlscs of mcntal hcaltli .IVC ,iv;lilabli~for b ( ~ t h w.~\.~ IVC s can +c'lrntl idra I,! thc ~ m p s i f nf t-tr~plovnicntiunc.rnpltrvmr~nt and rli,~ngt~.; tl-tcretjf nn mental 11er1Ftli. T h o rhht ic~ir\prot~lcrnI + that L ~ ~ ~ C R U S I * t h v(+up.; ~ ,?rtnnot forrnetf bv r , ~ r r t l c ma..;ign,nmcnt thi*v rn,i\. be iirtlcren? In criticnil tvay< oth1.r than kheir cmplt~ymcnt htatus. Thest>o t h ~ ditferr cbncesnrav bc W.;~<*n~!blp lttr a n t di ffert-nceq in rnentaF hca Ith c ~ i tCilmP'.. r Thr.; prt'blcrn i;ln tic rcd uced ~ o r n c w h n twith mu lti\.nria tc , i n a l v s i ~ bcacnus~.it ;ll2rwv\ the rt~scarcher t o TC>TI trol for kliow~i ~li t fert~r~ces t~tatwcrn tllc grotIpS.

LONGITUDINAL DESIGNS

ISSUES IN LOl\rCIDUTINAl, DESIGN

rablu 8.1 Cn,rrtfry X irnr~t?tiol~

'Gmupq'
t 2 3
4
Waxcl

~ O Ij I r~ ~

i~ pn?ieI r~
Wave

Zrnp1uyt.d Unemplc>yt.d

I-'rnplnyed

Un~npluyed
Un~rnplnved

Ernploved Unemployed

Employed

In any study that extencis over time, many events may take place in the world in which people live, and it can be difficult in panel studies to isolate which events are having an impact. For example, we might want to examine the impact of introducing a scheme where unt-rnployrn~nl benefits arp only available to unemployed people who engage in rlnpaid community projects for two days a week. W e could measure the work seeking behaviour before the introduction of the scheme and again, say, 22 months later. The problem with this design is that any changcs in work seeking behaviour could bc d u e to other events taking place in the 12-month pcriod. For example, the economy might improve arid incrcastd job prospects couid increase work secking behaviour. T h e weather might be better. Unlcss t h impact ~ of all these sorts of factors can be isoIated it is difficult to tell how much change in work seeking behaviour is due t o the introducticm of community work requirements.

,Changes can also be d u e to people simply growing older (developmental effects). For example, a study o f the effect of unemployment on the mental health of ynung people will encounter the problem that late adolescence is a period when mcntal health problems such as depression and schizophrenia arc more likely to develop anyway. I f we simply track the incidence nf mental health problcrns among a group of young people who become unemployed we risk overestimating the effect of unernployment on mental health. It is prubable that some of the incidence in
mcntal health problems would be due to a common increase in problems in this age group regardlesq of employmmt status. In order to identify anv distinctive contribution that becoming unemployed madc to mental health we would nccd to track several groups of young penple: thosc who remained employed; those w h o bccame I employed; and those who reniained unernploycd.

Panel conditioning occurs when panel participants respond to questions in ways that are influenced by their previous exposure to the same questions. When people remember how they previo~rsly answcrcd questions it may influence the way they respond on s~ihsequentoccasions. Participants may respond so that they rtppcar consistent over time. Alternatively, they may respond in a particular way because thcy have Learned !o avnid an extended series of follow-up questions. Panel conditioning can contaminate the responses o f panel participants and either undere5timate or overestimate the degree of real change over kime. Experimental shiciies can minimize testing problems by avoiding the prc-tcst stage altogether nr by using designs such as the Solomon fourgroup design (Chaper 4). Unfortunately i t is difficult to eliminate this problcm whcn using a prospective panc.1 design since, by dchi~ition,at least two tests are required. However, there arc some ways of eqtirnating the effect of testing. One way is to employ either the rotating panel design or the cohort sequential design or other designs that have replacement stratepl;. Thcsc al tcnlative designs provide a means of estimating testing effects. Using these alternative approaches we can crlmyare the responses of new participants or panels with responses of panel membrrs who arc similar in all other relevant respects. With thcn;c comparisons we can evaluate whether the 'experienced' members are answering questinns in a distinctive way. h some studies that have many waves wc also can experiment to estimate the effect of testing. For example, we can vary the frequency with which particular q ~ ~ c s t i o are n s askvd. For example, we could ask a particular set nf questions of some people in t~er!! wave but ask the same questions less frequently of other people. By comparing the responses of people who have answcred the questions in each wave with those who have been asked the questions irregularly we can estimate the extcnt of testing effects.

h a panet survey changes over simc can also be d u e to changes in the

way data are c~lllectedin each wave (Chaptcr 5). Varying question wording, ordcr, interview length, interviewer, methods of collecting data (e.g. face-to-face a t wave 1 and telephone a t wave 2) can produce apparent but not actual changes in attitudes. -The more the data coIlection method i~ open to the srrbjective interpretations of the people collecting the data, the grcatcr the danger that apparent changes could reflect observer differences rather than changes in that which is being observed. Even where the same obscrvc.r is involved that person is subject to changes and can be affected by historical and maturational factors. Cher time observers may see the

1 2 0 h r G I ~ D I N A DESIGNS L

ISSUES IK LLONGlDUTlNAL DESIGU

same things differently becnusc thcjl, a s obqervers, have changed in some way. Thtrc arc famous examples in anthropology where diffcrcnt anthmpnlogists have 'seen' the same community in very different ways. Red field ( 7 930) and Lewis (1451) both conducted ethnographic studies of a Mexican village 20 years apart and 'saw' the community totally differently. Similarly, Mead (1943) and Frteman (1984)both studied Samoans some years apart but nbscrved the culture totaIly differently. Givcn the unstructurcd iiature of the anthrupolugist's methods in these cases it is difficuSt to be certain to what extent thc different obscn~atinnsreflect changes in the communities or simply reflect the different ways the observers interpreted their nbscrvations.

Since a key reason for conducting longitudinal strrdies is to measure change it i s crucial that we observe real change rather than simply measurement error. Using unreliable measurement instruments will cause us to detect 'change' whew h e r e really is no change. For example, we might measure a person's level of prejudice at TI and again at TI. If, l~owevrr, the measure of prejudice is unskabIe, how can we tell whether any change is due to the unreliable instrument or real change? Equally, i f thc measure .of prejudice is insensitive it may fail to dctect change evcn h u g h there has been changc. Measurement error can lead to serious overestimates of the amount of change at the individual level (i.e. identifying individuals that change). Using unreliable instruments cause many individuals to appear to change at least marginally. In descriptive analysis the problem will not be as pronounced a t the aggregate level. IS we look at the mean scores of a panel ur of subgroups of the panel then the effect of mndnwr measurem e n t error will be muted. Those who appear to change in one direction will cancel out error caused by penple who appear to change in the other direction. For example, in a panel of 1000 people the average score on n measure of prejudice that ranges from O to PO might be 5 . If, due to ,measurement error, 200 people show u p as becoming less prejudiccd and 200 show up as becoming more prejudiced (and the degree of change in one direction is the same as the other) then the average prejudice score will remain unaffected. B I ! g~aphing the change scores one can see if the ubserved change i s due to random measurcmcnt error. Tf thc observed d ~ a n g c is due to random measurement error the change scores should form a norma1 distribution each side of no change.

experimental designs. In other words individuals with extreme scores on wavp I will tend, a t wave 2, to 'regress' to the mean. I-Iigh scores will tend to get lower and low scores will tend to get higher. When analvsis of change is conducted at the irrdividual level rather than a t the aggrrp~televel wc will mistakenly attribute some individual changes to a n intervention or some other factor when it is simply an artifact of shtistical regression. At the aggregate level we should not make this mistake since movement From low to moderate scorcs will balance movement from high to moderate scores (see discussion of truncation effects in Chapter 6). i n experimental designs it is relatively easy to distinguish change due to statistical regression from change due to otl~er factors. Since both the experimental and the control groups will be equally liable to statistical regression, it will he the drfrrence In thc 1cvc.l of change (rather than the absnlute amount) that will reflect the influence of other factors. In panel studies the absence af strictly comparable control groups makes it more difficult to measure the extent of regression.

The concept of mortality - thc differential drc3ppout in the experimental and control groups - was discuqsed in Chapter 5. Different outcomes For the two groups can be due to diffcrenkial dropout at some stage betwee11 pre-test and post-test rather than the intmn,ention. The same problem arises in panel studies since dropout will not be consistent for all categories of people. The importance of the prnblem depends on the particular analysis (comparisons) undertaken. Identifying the biases introduced by the dropout and then statistically removing the effcct of such biases can minimize the impact of differential dropout.

PANEL ATTRITION
A major potential threat to external validity in panel studies is the loss of cases over time (attrition). Thc longer a panel study exists and the more waves it has, the greater the danger of attrition. Attrition is a threat to external validity because it is not random. Some types of people are more likely to drop out tltan arp others. As dropout is compounded across waves the sample becomes increasingly unrepresentative. There are several ways of responding to attrition and its threat to external validity. One way is to replace dropouts with people with similar characteristics so that the known biases are redressed. This is more easily said than done. We may not know what biases are intr* duced by dropout. Also it may not be possible to recruit the appropriate

The concept of statistical regression of extreme scores to lcss extreme scores over several measurement waves was discussed in Chapter 5. Kcgression is no less a problem with lnngitudinaI dt!signs than with

<nrnpItx n.tmll,rr~. I f wta 115c rcal~l~icc-mcnts thcn the data for thew nr*w rtacruils r \ . ~ l l bc ~ncnrnylt~tc, thu\ Ilrnit~nl: ~ i n . ~ ~ to v szg~rt'gdte r~ ~vI'I
ncw

tht- .;tl~dy7ni):ht nnlt bc* cornparabl~ tn Ihosc nrnorlji ~.Fli ti1 rcn n t tnarri,lgt* brca kdown more gcntbr;ilIy

anal! ~ 1 . i ( C ~ I Chaptcr 4). I'his probltm m n he a d d r c s s ~ db v w t - ~ g h t i nthc ~ samplt* to take acct~t~n of t .rttrittm. Wt.i,shting mtailc ~tati~ticstlly boo.;ting tht. proportion ot ntndtxr-reprr~riitc~f SrrvlpF In thta sarnplt* so that t h e ~ r prt~pnrtion reflt>ctq thpir pmpnrtirm in tlrr p n p u i ~ i l ~ o(C'hnpter n 'j). I f tlilc srlmplr i.; \vt.i<htpd ~ r c ~ i r a t r lthcn v i t i n n still prn~~iiit* a uscfuI bask tor gentlraIi~,?tion. Thc probl(*nis cauwd bv snrnpl~n t tritiun n l w can bc addrcsqed hy acknt31vledgin~thr I71awq crist and nctqepting that tlici* will aftect s~t;n i t ~ c , i n t lu~ u t i~paClt!' tn genrralizc ~t rhc ~ l ~ ~ c ca I ~ CT i C~ ~I. l ~o w h ~rv e rn .i hhc I * X ~ ] R I I R ~ O ~ Irl7rl V b ~ n sr n ~ ~ not h t rnattcr. 11, for cxamplr, a panrl studv is bin.;cd bec(1i ~ w of ttii. diqpr~jportionatedrnpout of voungcar peoplc (the! m o r e ;lrr)i~ncF marc nn~t art. thcrdorc harctcr to ktytbp track t r f nIocrtinic), thi5 age bznq only mnttcrs i f R E P is linked ~ G the J variables t h a t are Ilcing tnu;lrninrd. If' X affracts Y, rtynrillc.;s rlt agp, tlwn the age bins L ~ U P I(> attritlnn doe\ not mLittt3r. 1 % "can ~ cclnildentlr ekpcct t h a t thtl rcyal Ionship br~trvemX nnd Y izrill PX i.;! in thta w11lc.rpopul,~iinn, clespitr. 2 1 ciiflc~r~ncc,~. ~ in the* ~ i g c pro'filc~ of t h t , ramplc ,~nclp c l p ~ i l ~ ~ l i l ~ t ~ . In thwe circrrmstan<cq in { v h ~ c h a \,irnpIe Ilia< ictr~~-ti is proriuced trv attrition ma\ attect t h t h v a t t r r r ~ s WP &jrl> pxarnlnlng, \z,tA nccd tu rcmc.rr.c strttiql.ic.~llythe effcct of thiq b i x and wta if llic rclatinnqhip btatween X and \ rchrnalns.Iree Ch,?ptefi Lp and 12). I t the ri*lation4wptwtwcrn 4: a~lci 1' rcm;lltls aftt+r removing thc cffect nt any variable cln w h i c t ~there I;. snmplr hia.;, wc can w k l y grnerali7r to tht. wider pnpulatinn despi tr sample bia.;.

t-lcnrr\-tr, tlrere I* no teaqtln rzqhr rrinditiorlin~ shnulrf aFfcct the r(*ltltic?n.;lfrl~ b~,l.tir't+c.~ t ;r,rr~nl*lt*.;. M'r m ~ f i h w t ,~nt tn .;tu>if h i ~ h con t l i d tl~vorci.:. produce pcwrrr adimtrnent nnionc chi tdrm Illan Itir\, conflict el ivorcc.;. WhiZr prticiyatinn in the \ t ~ l d y rnw rnoilrrntc tt>c IrvcF nf c r m f l ~ c t k h u c m di\*orcrdparrnts and may r 7 f f i ~tlie t ~~flustmcn ofchilrlrrn t thcrt* is nil rc.;lrm why y a r t ~ c i p a t ~ o n ill ,iltcr tlri, l l r t b b c t w c ~ nconllict aird adl~~strntmt. F r m thr>~igh thr absnltrtc levclf; of conflict and ndlustrncnt may not r~~fll>ct thi> Icavcl.; in the ~ r i i i e rpupulatlrm of d i v t r r c ~ sthc relnt~onsliiptlt*tw~cn 1-onflict and adlustrnmt +fioul~l remain t~~iaftcctcil Cot~.;cqi~entl\, fIt15 n,l,rttcvrd~lrj cnn br ct-ncralii.cvl ttl the, ~ d ept)p~tlatinn. r tn tt~ose c i r c u m s t ~ t ~ cwlicrr cs panc.1 cnndjtioning 15likely to I7ecomr a serlou< probltm for thtx rxtrrnnl vafirlitv u F a .;luJy i t m,Iv be appropriate tc) E)IW diffctcnt dr.;i~m.A rc*pt.attrf cmqs-stnct~cmal d i , ~ l ~mav w nfkr a suit,~lllca3tcrnntivc. rotating pnnrml cle5i~n ~ l s o prrr~~de as way n i estimating thcb cvtent to which ccrndltit~ri~ng might bc sffrctins r c s u l t ~I t dm=c this by comparing r c ~ ~ l l in t . ncn. panel.; w r h tt~o.;~, of 'c~pt-rlenccd'
pat~t~I.;

test in^ ~ n it< d irnpIir;ltinn~for internal validitv ( a t t r i h u t i n ~ chnngc to a n inlcrrpentitin or othrr facttjr whpn I I is ~ ~ a l drte l v to s i m i l v kin^ qtlrdiedl nl~rrha.: implicatinns irlr ekterr~al \,af~dlty. J i j~nrticr;mttr.rt in a l o n g i t u ~ l m ~ study l p r i > d u c ~change. * (because i t alrarts participants to ntattcr%the!, w o ~ r l d nthrrwise nnt think about) i t is diffici~ltto < ; 1 r that c l i a n ~ t w c?b~er\,cd in thth .;tudv trill take p l a c r In the xvldcr pnpl~lation. H o w c \ r r , th~.; is mnrc ol a problrm ,it th~.descriptive thin nt thta P\~!NLI~L) Icvcl. T~ TF ~ ; l r t i i i p n t i t ~ in r:; I pnnsl *tudv afftact.: the brahavicr~ls < ~ n ;~ttjt~rdcd r l t p a r t ~ c i ~ n n ~t t s i \ r i l l n1,rI.e it d,fi~cnlltto j:clncrall/r. from thr. p i n ~ tc, ~ l thta pcjpu [ ~ F arm. I H r ? ~ c \ ~ ccrn, n d i t i n n i n ~ sh{bil not affect thr tvav lb~ctclrs ,>rt> linked to nnc .~nnther.For euamyle, in c1rdt.t to rxplvrc the rnlpact ot thr 1~rcaLclnu.n id nlarriaj:r irn thc wrllbclng ~ i c h i l r l r c n not* miglir t r x k fnmilies w h r ~ are t1\pcrienclng marriagv Lbr~akijown.li being pnrt elf the sturtv makr.: parcntk and ~-1i1ldr~n morp nlsrt to mnttcrs r l t adiuqtrnc~ntat711 ~ ' ~ l l t l c , ~ th n tq~ nthe lc>v~I< of atilrrsht~n! we c)bwnreIn
The csrlier discc~s<ion cin the i.ff~cts rd

Panclq can br.cclmc un rrprcwntative i ( thr population From which tllc pancl i.; clr,~wnchances. Ccrt'lin h-pth$ of pt-uplc nj,>\ Ieavc the arcv wlicrc the r;tucly i.s k i n g cnncluctcd. As a n*<uttthc pancl mnv orFcrreprcscn t thrw leaves.. (outmigmnts). Simi larlv, n r w tvprs of prt~ple mnv movta r n to tlic area and tht. pnnel wilE con~c*i[~rcntlv r~ndcr-rtbpremt thew inmipatlts. Whrrc. pcyjtilation changes arc c r e a t i n ~ a substantial problcnl for tlir prcjtbctthe srdtrtion lc to 113t* rj~namic carnplt-. Dvnarnir sarnp1t.r can bc achrcu~d in ~ r ~ w - ruavs. al Yhcy are: rotating ppant.1 c J ~ ~ r ~ decign< ns; thnt aIlrrw for r~placemenlcrr sarnplc si~pplcmcn tLi t ion; ancl repca It'd crowsectional ~ ~ r n - c ~ ~ ~ s . I f w r use dvnarnic panels wc* need to hu clrar about the scmrcc of nnv changc in crtrr depenili1nt vnriahles. Changes nn the a g ~ r c p i t e lcvel in ; l C I I R J I ~ ~ C pant1 mat- rrfltyt rinr of ~ \ Y t Il~inp c-lianc(, In trrdrivri~rnls,tor c h a n ~ c qto the, c.clrnpo~iti.nnof thr panct. l-tjr t~x;lmplr~, ~vc r n ~ g h thitrach~n levcnl.; ~ of tife hnilrfacticin in a p a r t i c ~ ~ l local a r rcgion nncl ohscrvr that, 01 rLr tirntg,thy i l ~ ~ t ~ d IVVC~I q p of 11 tv +,I tast,~~-titln dc*cliiltw. m h ~ c ilcd inc ma!' br Jut, t o i n d i ~ i t l t ~ , i l <bt#comin,gIr-\ satiqtlt=tlrlr tv ~lt'uplc\v110 h~c1 high I t v e l 5 c1I life sati\tlcticln Ii%.i\'ing[lit> ;lrrn;l( ~ n d t h ~ ~ r c ' 1t I~l v ~ h panrll At thtl same time ptt>ple lvitli low I t ~ 1 7 r 3 5 r r f < a t i ~ f ~ ~ z t i rn~l!, o n rno1.c into ~ h area c (and thu. ;Ire rikcruited into o u r d v r ~ a r n ~ yc n n ~ l ) Ij~.clinint: . aggrcnKate Ic\.c,lq nf li fr .;atisftiction ~ ( 3 1Id 1 bc 11111' entnribl v to changcs tn thc crmrnyo~tirt>n of thc panrl , ~ n dnirt tu c h a n ~ t ~ in \ 1I1c lettbl.: r ~ Iilr i sati~lactl~ cn x p ~ r i ~ n c!.I,~ ~icnlr l r ~ ~ t d ~ t a i ~

15SUES IN LONGIDU I'I N AI. I3ESIGN

139

Practical issues.

In both the clrsicn and rutuuhon stage4 of R longitudinal study thcrc arc m a n s practical rnattcrs to scrl~re. Somc of these arr to d o with ma\irnizin~ internal and cxt<*rnalvalidity. Othrr.; h a v r more to do with tlir cost OF implerncntin~ ~iiff~rci designs it and thr practicalities of gettiill: tlic siudy completed.

S~nce change i< thi. core focus of Inn~itudinal studies rve n w d trr be sure that obscncrl c h a n p ~ reflects real change rstlrcr than chanqc- in tht* data arc c n k l ~ c t ~ Id t .iq eqsential thcrcfnrc to ~tnndardizc thc way data are cchlccted for thi. ~ ~ i r i o u wave5 s of thc study. I t is necessary to standardiw tlic rncthod hv which data a r cnllccted, ~ the way qitcstic~ns are asked, the order in wiiich thcy are asktad etc. Wc need to clcvclop clear and cnns~stcntmlc.; n h u t what to d o in particular s i b a t i o n s and rcligiously f~llorvthcm. The time nf vear at which data are crjllcctcd may nccc! tn tv ct)n.;r~h=nt fc3r each wave. Bus\- times, glmmv t1nrt.s n i the \'ear, tirncc (?I .;!rtxsr { r . ~ Christmac, ; ~ n rd j t financ~al year), \.ac,ltilrr~timp Inay all atfect thc k ~ n d s of rcf;pon<r< wr PI. Jt is not nlwavs pos5ible to a c h i r v t ~ t ~ i n d ~ l r d i z a t iThe o n . Ivss .;tructurcci the data collcciioii method the more difficult i t is to stan~iardizrprocedures. Whl*rt* more rluaIitahve and subjective techntqucr; S L I C I ~ ds ohscn~aticmand ~rnstntcturedintervlcws are called for, thc ~ r c n t c rthe ~ossibilitvthat changcs may be dup to thc FPOFIC collect in^ thr. data and the rnrtlmdq thcm~r~11.e~. Compl~te qtandardi7ahon is not alwavs dmirablc. For examylc, it can interfere with our r p ~ t a r c h goals in lifc crrurse studie5 in tzthich pt~ople arc tracker) as Ihry grow older. In longit~ldinalstudies peuplc's clrcumstances chanjir 417 we may necd to aciilpt our indacato~s to reflt=ctsuch changes. In thtb same way that tlic indicatnrs of i n t e l l i g e t i ~nrc ~ age specific, sc) our ind~catorsof change may also need to be l i f r stage specific. Mcaqi~rc.;elf family w e f l b e i n ~ and cnhesion mav well nccd to he adapted according to the srape in ithp farnilv life cycle. In o n ~ n i n qv n v 1 studies there rq nftt3n a cnre oi standard rncazllrtSs cr~llectedin t ~ n c rz7,;x.c l~ ot the 5hdv. In ,~clditittnmodules ~vhichd l ~ n g e frcorn time t r ~ timr rErts.IISO f+equrntlv i n c l i ~ d ~ d .

thrldren o r wc~rkplnctw that it thc rt-qpondcnt mores rvr can ctwtact tlicw clthcr backlip cuntacts to find the* rcbcpnndent. Public rcctlrd.; can also be uwd T ~ l c p h ~ directvrie~, ~ne e l t ~ t o r a rolls l and ~ u b l i c l v a\ratlabl~ records from IIK.~II cr~uncils can help IOC'R~C r ~ s p o n d ~ n \vhn t ~ Ii;1ve moved. Of course, i t is easier if panrl participants tell 11s t11c.v hnvc rnovcd. A t each intrrview and any othcr cnntnct i t is helpful to provide change of address notification cards to participants to mail back tcl the .;tudi manasrr.; i f thcy move. Keeping u p c o n t ~ c hetwtvn t suntcv wJves is anither helpfrll strategy. A useful tracking device is to send n birthda!, card to participants. I f tl-tis card i> 'rcturncd to sender' we arc atcrtcd to the need to st,lrt tracking them down ~ n ~ m r d t a t e l Thc y . card dl.;(l pro\.ides a n opprlrt~~ntty 'to scnd a c h a n ~ t of address card and ;tct.; n s nn acknou,ledgrmmt of appreciation fnr part~cipnting m the httrdy. To srducc n ttrilticln i t is essential to maintain the respondent.;' motivation to continuc to participate in vrdcr to rcduce attriticln. !'mvid~ng incentives such CISrnnney for the t i m ~ pcople qpend on the s t i d y, .;malt tcrkens of thank< or summaries of key wrvcy rcsdts can hclp achieve thi~ goal. WhiEr rnnt~rlnl incentivw can hrlp maintain participation, Frobdb!v thy rnrrqt citcchrre ~ncentivc I < for pcuple to f w l t h ~ t their ~nrtisiya!ion r< both important and cnn<bttclivc. The morc pvoplt. !PC! lhev are participant.; ~ n p da r h e r s in rc*sc;lrcli ra tlier than 'sublcct.;' the better. T ~ nvcrridillg E ~ ( 1 should ~ 1 bc ta nmkc parhc~pationin t l ~ t study , a positive exp~ritlncc~ rt>t1-trrthan an unwrlcnme burden and intrusion (Dillman, 1YTH). Attrition is a t - i i ~ g ~ problem r in somt- kinds o f st~tdiesthan othtars. Sh~dicsof peoplc w-hcr are highlv mobile, marginal or membcr.: o f 'extrerne"roup5 can cxpcricnce high attrition m tes. For e~arnplc. stutlieq tlf parenk aaftcr clivcirccc inrariablv crp;lte the probPern of m a ~ n h ~ n i n g contact rvith non-rcqiclent parent< - especially non-resideni f ~ t h c r < . Longitudinal s t l ~ d i c s of potip': such as criminals, drug USPrF arid some S O ~ ~ ofSuncrnptcrycd people havc very high attrition rates induccd by high levels of niubility, illness and dcath. Panel studics of prnple e~periencing lifr transitions can also havc highcr khan average attrition rates, Researchc*rs investigating young, pcnple becoming ad (11 t, ynung pec~ple entering Ihe labour market, per~plc after divorce, or Itho<e p r o p l ~ who leal-P the IsElnur force all face thr prrrblrm that people expcrtcnctng theqe transition.; a r c hiqhly likely tn c h n n ~ nddrt-5. c As a r c ~ u l t ihcv s;ln be lost to tlic panel studv.

Sample attritioi-r prod uccs problems with both sample size and rcprcwntatir,cness and tf~tlrt*fnre must be rninirni f e d . To rnin~rnirr attrihcm we muqt kecp track of panel r n e m b ~ r sTrackrng dernnnds considcrablc resources. A rangr of t c r h i q u e s is ttwd tn a s ~ i q t w t h track in^ prclp!c. Additional infnmmntion is collect$d a b u t parents,

Panel surveys can impose a considerable h u r d ~ n on participants. Thi.: is especially so i f rach wnve uf the shrdy i s closta togcthpr or i t inttkrvitaws/ qir~5ticmnatreqarc- Icnqthv or mtnis~vt*. I-1~a1.y respondent Ptlrdrn r sn problem b~caustli t can prtduce hirh a r t r ~ t ~ o and n a Ims of data quality as the burden increases.

140

LONGITUDINAL DESIGNS

ISSUES IN LONGIDUTINAL DESIGN

141

Therc are a number nf ways of reducing rcspondent burden. Some questions d o not need to be asked at every wave: every second ur third wave may L 7 c sufficient for some questiims. Alt~matively,a set of qucs%ionsmight be asked o f only half the panel and a different set of questions put to the other half. Although we have f ~ w e people r answcring the questions we map stiIl havc sufficient nrtrnbers for meaningful analysis.

Rchuspective panel design? rely on respondent recall. This has two problems. The first i s that pcople will recimskruct past evcnts in the light of subsequent cvcnts. This can result in distortion and reinterpretation by which recalled events bear little resemblance to actual events. For example, a study that asks people who 11ave divorced to describe aspccts of their marriage will be affected by the divorce - by the nature of the divorce, who initiated it ctc. The second problem with studies requiring respondent recall is telescoping and reverse telescoping (Menard, 1992). People can rnisremembcr and think that events l-tappenrd morc sccen tly than thcy actually did (telescoping) or longer ago than they really did (rcvcrse telescoping). A number of techniques can be used to reducc this problem. Getting people to Focus on certain rn~mclra'tllecvcnts and constructing a calendar of thcse key evcnts can providc anchor points to help remember the sequence o f events more accurately. Fur txarnple, we might be interested ~ between family life stage and the work patterns of mothers. To in t h link provide anchor points we cor~ldfirst ask mothers to provide key dates such as the year of their first job, the date of birth of each child and so forth. Since thcse are likely to be dates that can be accurately remembcred we can then, piecc by piece, reconstruct how long after these memorable events other, perhaps leqq memorable, events took place.

information needs only to be collected once (in thc first wave} thus enabling us eithcr to collect extra informatinn in suhsequcnt waves or to reduce the length (and cost) of subseqtrent data collections. UltimateIv the cost efficiency o f the panel design will depend on the difficulty of maintaining the panel - how much time is needed for tracking and the cffnrt rcquircd for 'call conversions' (convincing panel membcrs who initially refuse to continue). The cost will also depend on the method of collecting the data. Of the three main methods of collecting data in panel studies - tace-to-face interviews, telephone interviews and self-administered postal questionnaircs - face-t(>-faceinterviews are the most expensive. However, they tend to yield the best response rates. National panel surveys typically rely on some form of cluster sampling (Moser and Kaltun, 1971; de Vaus, 19%) where scts of interviews are concentrated in a number of geographical areas. By this mean5 it is financially feasible to conduct a number of face-to-face interviews in the one area at the same time as maximizing response rates and minimizing attrition. Chrcr the course of a panel study face-to-face interview5 m a y became less financiall~~ feasible. Despitc cIustcring inter~iews initially, in particular areas high rdes of geographical mobility can lead to 'dccIustcri11g' or scattering of the sample, thus increasing interviewing costs considerabIy. Fortunately, deveIopmcnls in tclcpllonc based interviewing (see next section) can alleviate decIurtering problems (Buck ct al., 1995). However, changing from tacc-to-face to telephone interviews can introduce instrumentation problems as d i s c ~ ~ s s c earlier. d

Panel studieq are typically rnt~chmore expensive than a single crosssectional study. 'Thr expense of panel surveys sterns from the use of multiplc data collections and the cost of tracking and maintaining the panel. However, if the choice is between say a five-wavc panel study and f i v ~ repeated cross-sccrinnal surveys thc cost disadvantag~of the panel design is less obvious. Tncre is some evidence (Buck et aS., 1995) that the cost of panel maintenance may he no more than the cost nf recruiting new samples. Furthermore, the cost of collecting the same background data in each cross-sectional surrey will limit the amount of other I information that can bc collerted. L n a panel design most background

There is nothing intrinsic in any research design that dictates that the researcher use a particular method of data collection. T n principle, data can be collected using structured qucstionnaircs, observation, structured or unstructured interviews or any other methcd. However, when it is proposed to utilize a panel design there are real advantages in using computer assisted interviewing techniques. Here thew are two a1ternatives available: computer assisted telephone i n t r viewing (CATI} and computcr assisted personal interviewing (CAPI). Therc are tWn advantages to using computer assisted methods in panel designs. First, these designs can involve extremely complex sets of questions, with many qucstions being relevant to only small subsets of respondents (depending on their charactcrisbcs or the patterns of change we have observed). For exampl~,wc may require a special module of questions for families that had experienced divorce between waves and require diffcrcnt questions for r e s i d ~ n t and nor?-resident parents. With cornputcr assistcd in t ~ riewing t m e t l i ~ d sall the possible qucstions arc programmed inlo the interviewing package. On the basis of answcrs to earlier questions ihc cornputcr selects which questions

1.UNGITUDINAL DESIGNS

ISSUES IN LONGlDUTINAL DESIGN

the interv!c\vt.r necds to ask. This simplifies the task of administering con~pIcv questionnaires and reduces recording and interviewing errors. A further advantage of computer assisted inknriewing i5 that in the secmd and subsequent waves the answers from previous waves for that respondent can be fed into the intenricw. These can serve as excellent prompts that provide respondents with reminders as to how they have previously rcsyonded. For example the CAT[ or CAPT interviewer can say, 'Last time you said . . .' or 'Last time you indicated that vou were planning to retire. Have you actually retired yet?' Not only does this provide respondents with a reminder of how h e y were feeling East time, but it personalizes the interview and makes the respondent feel that their previous answrrs ~ R V C bccn taken seriously.

In establishing the gap between waves, respondent burdcn and panel must bc considered. Short gaps can exacerbate these problems. On the other hand long gaps can create tracking problems and increase attrihon.

Nu nr bur nJ wavcs The number nf waves utilized will partly dcpend on the purpose of the study. [f we are interested simply in 'before' and 'after' a particular cvent, two waves should be sufficient. If we are trying to track a process of change, multiple waves may be needed. For example, if we simply wanted to knnw how well people have adapted to retirement, say two years a Ftcr retirement, h ~ waves o may be all that is required. If wc wan tetl to track stages o f adaptation to retirement we might require multiple waves to detcrt the ups and downs of the process (see Chapter 7). The number of waves will also depend on funding since each wave adds comideralsly to the cost of the study. Respondent burden must also be c~msidered when planning t h number ~ of waves. Many large scalc prospective panel surveys adopt an open ended appro ad^. Funding is available for an initial set of waves and decisions about how Ic>ngto continue with the pane! will depend on the capacity to sccure hrther funding for subsequent waves.

Sample error is, in part, a function of sarnpIe size. The smaller the sample the greater the chance of sample error. In prospective panel surveys (without replacemcnt) sampIe error will occur when the initial sample is selected. Subsequent attrition may produce bias and will increaw sample error as a rcsult of reduccd samplc size. If the sample is replenished sample error due to overall sample si7e will be controlled. I-tow~ver, ~eplacemcnt will be based on new mini samples and these replacement mini samples will be subject to sampling error and can therefore aifecr the precision with which we can confidently gmeralize our result beyond the panel.

Setting the gap hetween waves will bc affected by 'political' considerations such as the pressure to come up with some results. The nature and content of the study will also be crucral in setting the gap between waves. Studies of high frequency events or processes of reIatively rapid change may requit-c only small gaps between waves. The theoretical model being tested will also dictate the gaps. For example, in a study of adaptation to retirement that anticipated different phases in the retirement transition le.g honeymoon in first 3 months; distress by 6 months; rebuilding new- identity by 72 months; stabilization by 18 months) thc model would dictate the gaps between waves. Where changes are expected to be gradual there i s IittIe point in f'reqiient revisits. The famous '7-UP' scries, which tracked a group of people every seven years from the age of seven, is an example of a long I term study with large gaps between waves.

Sample sizc creates a problem with any type of desip. 5mall samples encounter problems with sampling error and with insufficient cases in special subgroups t c ~allow meaningful analysis. In studies of change and transitions, for which lon@tudinal shdics arp cspwially useful, the actual number of people in a panel who experience a particular change or transition can be quite small. For example, we might want to use a panel design to examine the effect of divorce on children. If we used a general population panel of households the number of households that actually cxperiencc divorce in a given period will be yuite small. For example, on the basis of the Australian annual divorce rate of 1 2 per 1000 couples (i.e. in any given year 12 out of cvery 1000 married couples will divorce) we would require a very large panel indeed to have enough couples divorcing in a year to allow meaningfu1 analysis. Even if we had a panel of 5000 couples nnly hO couples would divorce in a year. I f we followed the panel over a five-year period this wouId increase tn something approaching 300 couples. But even this number may ccmstltute too small a sample if we wanted to look at divorcing couples with young children. The numbers would be even smaller if we wanted to look at fathers o f such children whu had primary reqponsibili~ for caring for the children. -Where uur interest is in small groups we may need to refine the initial panel more (e.g. just couples with young children rather than a random san~ple of hnuseholds). Alternatively it mav be possible to oversample certain tyres of people (e.g. couples with young children) to maximize our chances of obtaining sufficient numbers rrf pcoplc who subsequentll~ experience the changes in which .rut arc interestcd.

Thc qi/c r,f thc iniiiol G J ~ Fir~ill I P ,11.;t> ilct,d to taktb t ~ r c o n nof t tht. likclv attrition %in(-rattrition tv11I Jepcnd tn p , ~ r t on the dumtion of thtn .;tudy and tF~cnt~mhtlr and irt,<ltrvniv at IV.~\I..;. the i n ~ t ~ p a l3 ~ r slze l uv~llntrd to takv ,iccciunt of cqflRT,TI(YI iittriti~n r.lttT.;.

implcmi~ntcd.Sincr th1.s~ principlthq wcrc discu.;scrl in Chapttsr 5 this section simply hiyhlrglits matter< th.~t arise particulntlv with Iunqitudinal dt,.;igns.

On tht' fiwt ~ ~ e 2 L I > ,~% ~>r(~c;pvctil-c. pinrE <tin-ey c ~ c rvill typically qpmd quit? ,I t ~ of t t i r n t abtaining inir7rrnatlon such a s ~ t ~ r c w i e r n n q r a ~ h ~ c , housr!lnTd , ~ n d hi<tc+rrc,llin tormation. Thik M-auld incli~de such thing:. yc;lr 01 nl~lrrlagc, act3 J I ivl'tainin~, l~rstj ~ r ' t letc. Qn .;uh.;cq~icnt wavcn\ we wiPI nyt*~lto c l ~ c c kwhthth1.r romp things l-tave cl1,ingcd ( c . g . nl,~rilnl stattr*, anv ~-h,lngc< to hrrusc~lioldcornpositinn, ~h~1ngc.i In inh etc.). The ~ l r q ~ qot n in\trumi3nts \\-ill {,at-1. rfcpending rln the mcthorl of cnllectiti~ ~l;lta.Ccrrnpufrri7ed methrrdk alloi\. for rt~rnplr,: queqtlonnaire structurr,< n ith s u b ~ n l u ymodulrs ( w c rfiscuqsinn ot CAW a n d CA 1'1 nbovc) r ~ h i l t . wlS-adm~nrstered qt~c*rtionnairesnt>rcf to Ec ~ i r n p l c . Instrumrnt dcsign will cal.;o bc afkcctc~ltry dccislnnq about the us(* of core clur-~tirm<, rotatin$ rnnijt~lr< ant! .t*t.; r>i yuezticln* K1t.t.n tci tnrllv haFf thc ~;lrnl71il h t ' t ~edrl t ~ < r )I,cII+~c\~)

G i w n the need to minimize panel attritir~n there is thr danger that ~ r n d u e p r c 3 ~ i 1 ror t ~ unrth~c,ilnleans arc crnpltlyed to crmvjncc ptwprl~l to rontinuc to participatr in each wave t l q the 'inveqtment' in cach pewon grnw~ 3' they participate in mtlrl. warmes, She ' c c ? ~ nf ~ 'penplc d r ~ r p p i n g out incrr;lse~.N ~ v r r t l i c l c ~ s rnlunt,irv , participn~icln must rrmaln the bar;is on which pcrrplc continue to pariicrpatc. Tht clumtinn also ariscs as to ~ v h ~to l t do with ptwple whrr rcblu~cto participate In clnc walre. Should t h a t p r ~ c l u d tr?.inq ~ tn seurr~i t thcm again ~n ,I sub~equcntw i l ~ c ? At li.hr1t point doi.s r p t u m i n ~t o prior re!uwr.; bccomc. Iiara~sment?

A I o ~ ~ ~ i ~ u i studv l i t ~ n ilt ill h ~ n r f i gtCLlt t lv trom thc i~mployrncntof rirlltrained st,itf who ha\(, ;rn ongoin< involvement w i t h the shturlv. II' a person I < rr~tcr\,rc~i,cd the qamc r n t m l e w e r ovrr t~rnc* there* I < niore chance of 1111% respondt.nt c t m h n u l n ~ in tlic study h.causc nt a f e ~ F l n of ~ cnrnm~tmt>nt tn the pcrsorl as much a.; to the study (IZose et al., lL)YI). A wcll-coiiducttd p,~ncbl study will rrcl~rirr people w h o arc particulnl'l~ 45lled at w n \ . ~ n c i n g p'1nt.l members ~ v h o m a v be inclincd to drop cvut of the study t ~ continue. l I'crst~adingpanel rnembcfi ~ I i ~ on ~ t i a l I refuse y to continur, in !hr study I < c~IIJPCI ' c t l n ~ c r t i n ~ refuwlq. ' I t take5 p a r t ~ c ~ ~ l a r skills tcl i~ndcrtnke h 1 . s task ~ l ~ c c e s s f ~ wrthout illv contravening thc cthic;ll principle.; (h voluntary and informed conqent. Data rnnllagrrncnt i~ nlqo crihcn! to the succeqs clF r7 large r;ralr. pancl st~~dr SF;~Ilr.li .. data rnantlCt8rs r4,ho .tort* d a t a , p r o p c r l ~ match data f r t m kucrrs.l\ I, ;1\.c5 and rtt*l.~lup tran~p.lr~n data t manaKrrnent yrr1tnct~Ts r et i Finall\-, ~ l ~ .innl!,sis c= 171 Ir rnqrti~rfinald a t , ~ c~ln bwnnit* \.t>rtJ romplcx and project mt,mllcrs who mrl prnpprly h a n ~ j l r t h e analvsis w ~ l l h e inipcrrt,~nt.

r\t rvh,lt pe*int >hc~tllrl participant.; 1.1. ,ickcd for lita air ct+nccnt7 In a <tutFy that c,in crtcnd iclr rn,ln\ years, d r ~ *strnbent c at thr Z>t*ginnrng 1,151 lor the rtmainrltv of the 5t ud\ ri~gnsdlrs:. cvt hot\. many u v . l \ ra.; ~t c v c n t ~ ~ ahas? ll~ O r is informed consent rctluitcd a t ravh wave? A n ~ntcrc<tlng qiicsticvn ariqes in ticsigns ~nvr~ltring rectrrd litlkngp. Althougli privacr. law.; In many countrres prevrnt quch Ilnk,>cr, thew devcn5 nFso patentiallrr contrarrrnc the principlra ot infomeil crrnwnt. Even ~ l i n u ~ a hprrson may havr- givcn informcd conscni ~ v h they r ~ ~ providrd tbach srparatr. set of ~nforrnation,this 1s not tlrc snmc a s agre-wng that the dL1tnbe linked srncc thiq is a lrcrv different u~ of the data ant! is unlikclv h) be what the inslividuat rt,n.;cnted to.

Ethical issues

Longitudinal d ~ ~ i pc{)~it m r(mt tlic G ; l r n t X ctliical i \ c ~ t l \ s A< a n y t1tI1t.r deqi~w. T h a t 16, the matttar<o f ~ ~ o l u n t a l,.~rticiyation, n, thformed cclnsi>nt, nn h a r m 2 1 1 p a r t i c ~ p a n t s and c o n f 1 t l t ~ n t i a l i ~ / i l 1 1 ~ v n v n 1 irntt\t t1' be

In txpcrimcntal d ~ s i v in s which thrlrc is an activtb interventrun t h t w is an ~ t h i c a l sequirernct~tthat bhis intcn~t7ntiond(w<not evpose participant.; to h a m . T h e same cnnccrn can arj.;r in panel dc'i~qnsthat ~ n t . ~ d v an c act1r.r ~ntcrventicv~. I-l~~tvcver, panc.1 \ t u d m mt>ri> typ1c~T1\rrl\*on the e f i ~ r of t 'naturallv crcci~rring'event.: #tnd thu5 d o I I r n ~t i f r ~ n t tht, pr1.bIcm (11 Flarrnful intcrvcntions in t l i r * samc waV Ilia1 t l ~ r cupcr~nlc~ilcr does. Flnrvvirr, the rc<rarr.htbr ma\ xvt,lt <t>ntrc>nt \ltrl.il~onc in ~ v h i r hthrv Ie&-n of individuals M,IIO nrc c~pclqt-rllto harm l'rw tam-npl~. ttll. c t ~ ~ d v ma\. i d m t i h a pt.mnn w h o iq ~ u i r i d ~ l l lrleprewcd v or fhey nItl\, uornt. acros3 caws of cl~ilclal7usc. The rr.;ranrchcr must dccidt~whnt I t ) i l o in such c;r.;c.;. There mav hr the temptation to do nothin): qince nnv onward referral for other hclp mav jmpartli~r-thp ongo1nj: part~cipationnf the p e n o n rn flrc pancl. l'urthcrrnorc, \ ~ t i - l 1 intervcntrrjr~mav 'cnn~.~rninate'

146

LONGTTUDW AL DESTGNS

the research by changing L-rehavicr~tr. My own view i q that the wcI1being of participants shrr~~ld take priority and that, by inviting people to participate in a study, the researcher takes some sesponsibility for the safety and wellbeing of the participants - even where it is people other than the investigator causing the harm.

DATA ANALYSIS IN LONGITUDINAL DESIGN

Since wc necd to be able to match cases between waves and to track cases it is impossible to assure participants of anonymity. However, it is necessary tu cnsurc confidentiality. Apart from normal guarantees not to divulge information that would compromise any person's privacy, there is a need to ensure that data are securely stored. Since we need to be able to match rcspnnses with information that can identify respondents, it is important that data arc stored in such way that no unauthorized person can match this identifying information with other responses. This can be achieved by storing the information in separate files with only a very limited numhcr of people who have the information necessary to match thc hun files.

Many of thc matters discussed in Chapter 6 in relation to the analysis of experimental data also apply to Inngitudinal analysis. Rather than repeat that discussion, this chapter focuses on a number nf analysis issues that are particularly pertinent to longitudinal analysis.

Missing data
Summary

Wlulc experimental designs are ErequentIy not Feasible in social science research, Iongitudinal designs provide an important way of tracking and understanding change a t the individual and societal level. Whilc longitudinal designs can encounter some problems with internal validity because of the absence of randomized control groups, there are ways of minimizing these effects. These have been discussed briefly and will l-re deveEoped more fully in Chapter 9, Other potential problems with internal validity have been discussed and ways of structuring longitudinal designs to overcome thesc problems have been outlined in this chapter. i, As i l general rule, longitudinal designs tend to have bettcr external validity than experimental designs but the threats to externaI validity remain and must be efirninated. The chapter outlined these threats and indicated strategies for minimizing them. I,ougiludinal designs can be quite compIex to implement and involve a widc variety of practical deciqions. The practical issues and ethical issues that arc particularly pertinent to panel designs wcrc outlined.

Missing data arF a problem for any s t ~ r r of analysis but are particularly problematic in panel analysis. There are two types of missing data: item non-response and unit non-response.

Sourcijs of missinx data

I T ~ M NON-RESPONSE
Ttem non-response occurs where individuals do not respond 5 0 a particular question. In somc cases this is because they are not required to respond to thc. item (e.g. questions about a spouse for people who are not married) while in otl~er cases some people may refuse to respond to a question (e.g, income).

This is a problem that is peculiar to panel studis.' Since panel designs jnvolve collecting data from the same individuals on several occasions =md measuring change over that time, we encounter difficulties if wc fail to collect data frvm the case on the second or any subsequent occasion. There are turn korms of unit nnn-response. One is where people drop out of the study altogether. The other occurs in multiwave panel studies where some people miss some waves but are not permanently lost to the panel. This is called wavr non-rpstxl~~se (Rose et al., 1991).

LONGITUDl l\lA L DESIGNS

DATA ANALYSIS T N LONGlTUDlNAL DESIGK

1dt.n tifying nrissil~gh t n bias

Missing data represent a prol-hcm for two main reasons.
E n dealing with the problem of item non-response we can check whether non-responders arc different from responders simply by comparing them on other variables. For example, we could chwk whether nonresponders to a particular question were any different from responders in terms of age, income, education, sex, ethnicity or their attitudes on particular issues. We can also check fur bias due to unit non-response. Une of the advantages of panel studies is t h a t when a person does not respond in a particular wave or drops out altogether we still know a great deal about that person from previous waves. We can use this information to see if those who drop out are different Frnm those w h o continue. If they arc we can ernpluy various strategies during the data analysis to make adjustments. Even if wc ldentify bias it is nor necessarily a problem. Bias is a probIem i f we wish to makc generalizations and arrivc a t populabon estimates for the variables on which there is bias. However, in cxplanatory research the bias wiIl nnt always matter. If thc characterishc on which there is sample bias is not related to the variables that makc up the ~xplmatclrymodcl then the bias will not affect the patterns in the explanatory model (Chapter 8).

Item non-response can lead to a substantial loss of avaiIabIe sample size for data analysis. This is especialIy a problem when constructing scales or doing multivariate analysis where rnlssing data on any one of a set of variables leads to the loss of that case for the analysis. This problem may quickly compound to produce an unacceptablc loss of cascs. For example, suppose that we arc conducting analysis u5ing only three variables and that for cach of the t h r c ~ variables 10 per cent of people had not snswcrcd the question. Suppose that for each variable a difruent 10 per cent of people had not answered. We would then have a loss of 30 per cent of cases. The problem of item nun-response can compound over waves. This can he illustrated with an extreme cxarnpIe. Tmagine a five-wave panel study in which an individual participated in each wave. We may have a set of 10 question? that form a prejudice wale administered in each wave. I f R person tailed to answer just onc item in one wave we map be unable to construct a scale scvrc for that person in that wave. We ~ ~ o u l d therefore have incomplete data for that person which could force us to eliminate that case from the analysis. Because of one piece of missing data a great deal of data about a particular person woulrF need to be discarded. Attrition of this extent can lead to a sc.vpre loss of cases and daka. As a panel matures the problem created by missing data can worsen sincc the chancc nf an individual having missing data on a relevant variable a t some point increases as thc number of waves increases.
,

Missing data create a problem because nf their effect on sample size and on sample hias. The strat~gies for dealing with missing data will depend partly on which problem we are trying to minimize. Of course, we should try to minimize the problem a t the data collection stage, but there are three ways of addressing the problem during data analysis: imputation, weighting and statistical controls (Rose et al., 1991: 19).

Missing data can also introduce bias into the sarnplc. Where iten1 nonresponse is systematic rather than random (i.c. certain sorts of pcople are more likely than others to refuse to answer questions) unrepresentative samples can bc. the outcome. For example, i f higher income earners are more likely than Iow'r income earners to rcfusc to reveal their income we a r c left with inaccurate income eqtimates. ConsequenMy, any analysis in which inctlme is a variable will be affected by [he loss of cases due to item nvrn-rcspnnsc. Unit non-response will introduce other sampIe biases. Those peopIe who drop out of a panel are unlikely to be the same as those who remain. Loss of panel members may be due to death, gcograpliical mobility, changed family status, or some other type of dtanqe in a person's circumstances.

Essentially imputation involves constructing responses for the questions which people do not answer. There are a number of forms of imputation, some of which are discussed by H e r t ~ l(1976) and by KaEton (1983a; 1986). These include the following.

Samplr nzwi? flpjlroach Tf we do not know the value of a person on any pivcn variable, a reasonable guess for that person is the average respclnse by other sample members for that variable. With interval levcl variables we could replace missing values with the mean for the variablc. We could use the median for ordinal variables and the mode for nominal \,ariables. Although we will make errors in our guesses this method produces fewer errors than guessing any other value. Where the sample is relatively homogeneous bhis approach may result in few bad 'guesses'.

7 hv prtjhlcm t ~ 1 1 1 1 ~rlc~pttl tlliq ~ c apprrwch, hrliccavrr, i < t h , ~ ~t~ redl~iths sample \ .~ria't-rTtt1 ern !hta \,+r~ablt. rtlr tvhrch t h t ~ mlc.;ing ~ l a t a art. I-+t*ltig tlrtirn.~h+rl. Thl.;, ~n turn, rr~illccr;tht* ctlrr'lntion t%ctwccnfhfs v a r ~ n b l r and r ) t l ~ \,ar~alllta.;. r
I I I : Ch7~ ivav of cnt-rcnrning fhi.; probll-rn IS tc, u w group mcLinsmth1.r than t-ht*overall <ample mcan, In du !hi+ we would ~liv~~ 1171l o S ~ I I T I ~ ~ I ~into . croup.; on thr basis 11f a harkgr~jilnd varinlllr* Ic E. ~;I.~Y. aze ant1 cthnic~tt.) that corrclntes rvrll rvitl-t thc nli<l;ins c a l u t t.ar1.2bTv \\e wu~tlilthen obtain the mtvn tor tlir- missinq d , ~ t a \~asi;rfilr for c.i<.h u t thp.;~, sclhgn)llp+. l-nr c x n m p l ~ , if nrv ~ v a n t c dto cactinratt, the inctr~il~b Inr p~%rjl,lt' w h o cit>clrnrd to ntiswcr t l ~ ( r jncornta qllp~tion i\*l> micht rilvidt. tl11. <ample itlttl vcrup\ according tn their chtlt~cationIrrcl and t h l . ~ ,within taach cduca hon !r~,t*l, Furthcr 14 ivide t h t ~5;lrnplc lntn mallas ,?nd fen~;tlt*s. For cach cclucat~cinletpelatid ~ e n d c r .;trhfir0~1pw v cnul~i 1ll7t;lintht. rnt-an incomr Ievel for Illnqc ruho did dnwvC'r 1 1 7 ~income ' qtu~~tirw W . c r n i ~ l i tfind thaf thy Incnme Irvcl I i l r tc.rtjnrt edt~salr\iI fcrnti It-. E q VO,O131) F a. rvhilr trrtiarv c.ti~tcatedrn,~les enrncd. on avcw>:t-. <75,0111);lnd 30 tc?rtl~. U.;ing thy ~ 0 ~ 1 1 ~ 7i ~ a approach n s w r n'111lld slrnplv ~ ~ ~ b \ l i thew* ( ~ l t 111~~111 t ~ ~.11111-. for rni.;.;ili~value#\ Ior pct,pl{. ,ir-cordln< the Crrrrrp inhi tvhich t l i t * ~ tcT1. Rich glr-ad\~;lnt,~yt~ of th[\ approach 1% that it t3\.E~gerntr.. t i ~ Jcgrt>r ~ a r,t hnrnop~t~rlity wlfhin grt>up-- and t l i ~ ~ ccan ; ~1vt~t~'~tir thc n ; l vananre t~~ b~t13~rbr~n g r o u p and tor ihr *arnplc nvt-rail. Thi.; c5rror,in turtr, can 1nHntt1 the cc~rnal,ltion.;ir*hcn using tfic r-ariablr t ~ which r t h e mi- in^ data 11.ir.tbccn cstimatrd

This i s a rnori* cnrnpl~~x metllnd rtf impirtntir~nthat intnlvcs the 115e nf rc!:rr-ion t o yrdicb thc i alue of tht, F e r w n (In thc nliw,lnc 3 al~lcq karial7lth. C7n thc l,n.;i~ ni a \\.hole set r l t charactt~rr~tirs we c,tn cshrnatr how, on ;IirrraRc, a person with those c't~nrarteri~tic.; nrc~ulcP a r l r n r e t t h r question nli rvhicli tlicv have rnisqing data. In this rrqpccl i t ia c o r n p x ~ b l tt> c thc ~ grrwp means approach. Thc prnhlt*m with thc mcthod i\ that 5111ccr e q , ~ ~ < sI <~ u bnsctf t ~ o n a ctim~lationnlatri\ in rvl~lrhthp p.i~rwisrarnibtlind of I i ~ n d l i n g r n w i n g data (st117 below) I.; ~1st.d. thl.ri. v\,1I1 nt)rmnllv I-rtl home lo%% of caqcs.
I < i - , q r t ~ ~ ~rrrrrl~,.;is: ~~ou

For <om? stnblr*charaztcr~stics rvca can 'carry t t i i i ci~~cr\nbut%s trnm ~ l n rarlier w.ivr (Kmc rt dl., 1YQl) or check [lack from k~11,~t.qucnt w.~vcs. 'Hti~i~cvc.r, flrr variable\ that a r c Irkcl y to t l c n subject t o c h a n c ~ ,lnd . for nunrrric \ . ~ r ~ ~ ~ in [ d p;rrt!ck~lat c=< t t can be incnrrccl to 'c;trrb nrcgr' rerponwdc from ; I prer-io~~ w \ a r e . Ovt-r time i t may be pnwlble ttl fill in sonlrl gaps CAI~SI.YI by M ~ ~ V non-rilqp{rnse I* l-rv Inrjking at 1111, r ~ r c v i r rrvavca i ~ ~ a n ~ t!he q ~ r h r - q u c nw t alme and r r n y i r ~ ~ nrn~tlpoinl ~ r,,ltae fnr the- rnisqinq uvar-c.

f I i t This approach i s rimilar 11) the f i n j i ~ p npproaclr 111 that i t rc-lit*~ on ~livlding tht, +arnpl~ i r ~ t t rL T O U ~ cln the I,J\I< n t o t l ~ c characti.r~\trcs ~r that a r c Itlel! tr, bc corrtll~tcrlwit11 t t ~ c r n i s s i n ~data variable. But ~t d ~ f f r r sin that i t does not involve sutl.;tih ~ t i n gI E W g r ~ l ~ nlcan p for any rniqsirlfi clata. In<tr*nd,wF~l.nr\,e Incattb a casc iz.rth m i w v n ~data a n ,I particul,w vasiablc we ~vntaldrook .lt thc vnlur on the qamc variablc of the noarcBhtprpr-ding casc In [hat gat7up and give that snmr ~ ~ I L to I C thy caqe w ~ f h the m i w n g valur, l'his nlcan.: that thr* missint: lire5 ;Irt7 rcplacrrl w t h n t.3rit-k of d ifh-mnt ~ r , ? l t r t a < rantlonllv chrv.*r~from tt.lt7~1n th~t +irtl~rl'r\ip t > t c ~ s cl ~ Firc ayprc>acl~ doc..; 1 1 c ~ t atrcvt ,,~rnplror gri)uT ~ . ~ i r ~ a t , ~if t ~has t v . n o t n f f ~ r ton thc stret~gtli of rol.rt~lativns ,~ncl avoid.; nnv Inss of c a s w I h p i t c btxing ~crrnt~it'l~at morenrr,mylcx to taut-cutr, t h ~ > 1 s a h ~ q h l vd r ~ l r ; h ~rnctlrt>rl lt~ r>t hantllltlg rni-lni; d a t a
I

rncarl.;

c h proIg~rol~i~ /lrru r r : f~J.ss.inl:~ i t l t i l Thi5 is not a n imputation L ~ p v r o ~hut ~ ~ d ac W.IV s nf rnatmgin): with miwing data when ct>n.tt~rctinc scales. I t rntailh c.alculatin): q c n l ~ ScorcG bv calculatin< ,I perwn'5 mean <illre h r tjtd- ;-l~ril~/l? h cr- wItiL.11 !lft*t/ I i ~ i ~ /?~,r,i > : v d r i ftn17Ti7i-r.q.5 ~ 1 p p 0 \vta ~ i v t w conqtrurting a scale from 10 qucsticms cacll h,iving pn\<lble scorc.: ranging trl>~ii 0 to 1. : I p e r w n who c-rbt~inrd a ~c4,rt. nf 2 oil all I0 itcrnq tvoul J achre~ t* a total qcnrc of 7n ovcs ,711 10 itpnl.; and a mran SCOT^^ of 2. ,I ptJrson who n b t a i n d a scar? t r f 2 on cucli nf c ~ ~ ttcms h t l ~ did t not ~inrwer thy rtvnaining two items would ohfain a Scltnl wore of l h but a nlcan of 2 Calculatinr: a scnFc <core in t h i ~ w a y avoids the prot~lern(11 rniqsing d a t ~ whcn ctrn5huctinc .;caleq.

Arjr7r?~,q13 nt qpnrr A relatrd approach rrqi~ires that wc cnlculnte the tnlbnn vr thr r n c d ~ a n(;1.: approprt,lt~~) ot IFI . case< tr-~tliin a gi\i*11<pan t r t flit c;lhc \vltli mlwlnl: 14dtd d n ~ lwhstittltt, this v;llur for thr rn1wnq d ~ t . 3 I valtlt-. Thiq ayprtv;rrh is r t ~ l . ~ t ~ v c slrnplc lv and ,~rclidstlct7,iting varlarlre

I< ; I \\.a\ <,i ,~Rluqtint: a sarnytt- to aHo\ro for yo5sit.lltb Iliac d u c ncv\-ri7';pcln\ta.W c i g h t ~ n ,I ~ +amplt, chcruld m,ihi. I! rnclrta rl-prcs~nl . ~ t i \ eof 11it. populatlc~n it is rlcsignpd to reprt*srtlt st3 that rclialllr vktlrnatt~(,in be m,idt* from l l i ~ a<,~rnp!t' to the pc>yi~laticlnFor I-uamplc, rln,l~inc ; I wftlatliin trl i\*hicll per cell1 or the p o p i ~ l a t ~ o arta n male and 50 per c t ~ i t .Ire icrnalt~.Hnwt*~.t,r, when ,I <amplo ot 1111spoytllallon ~ l r a w ni t 1% tnund tli,if, cithcr I i r ~ c n ~ ~ of s tnun-rcqpimw ~ or bccause of t i r n p u t , (dl yrr cent of camplc ~nt~rnbcrq art, tcmale , ~ n t l li l pcr ct-nt mnlc %nee yt-nt11~ 17itl rr,lnted t c 1 m,lnv otlm.r vnr1ab1t.c fhis b i o ~ ir.111 affrr-t rrthrr varl.~t.rleqtsrm the s a m p l ~ tn thc p c ~ y r i l ~ t i r ~ t ~ r.\tirnrrttbs

\\'eiqhtrnq
t o ttrut

D A T A AKhl,YSIS IN t~CFY(;ITUI7IYAI.DESIGN

To adirt.;! tor tl~r I)i,14 IYC ~x,nllldnrrtl to rvclght each of thc trrnnlcs to count for lpsc 111,111 0 1 1 ~ ~ P T S C I HI % ~ ~ I I C #t ~ ~ m ~c ah l~ would bp ~ ~ ~ ' i g tc) lit~~l count for rnnrl>t h a n oiir person. In thw ~ ~ their a v pryrrrt~on.;in the <ample are adjustc~lto equatc with ihcir proportions in the prjyulation. Tu achievc this rand wc weight each casc hy a spccific weigh^. A wcight is achieved hy dividing the population pcrrcntage for a category by the <amplep~rccnt"~r In this uaqe the formi~la mtirlFor males thrc~ulclbc FiO/ 40 which gi~~r.; .I nrt>ight of 7.25. For ftmalcs the ratio M,OLIIJ br 50/h(l whish gi\.rl;a tr~eichtc ~ 0.53 f (de Vaus, 2olrl). In wave 1 of a pllnel study we ~ ~ ~ on ~ wd l lto d rzpeightthe samplr on khe bnqis of tvhat w t a know about the ~ h a r ~ ~ c t c . r i s t 01~ c the s pnp~~l,liir>n and l l i ~ chilracI~ri<tic~ o f tfinqe \%.hi) rcspnnrf ;II the first yh;l\c (r E. sex, rcgion, ethnlclty). Tliis rcquires that wtt I~ilvrreliable knuwlrdgt. 1)I the populatiim cliaractcristics. In sitbsequent waxFpswe will need to niakr further weighting ad justments to takr ncctwn! of biases that ma), crccp in hecausr of .r ttsiticln. Given tIip amount of' ~nfc~rrnaticln available from the lirqt 1var.c rtrt* \.\,~lj !lave a grc.dt drat of infnrrnation that can tlr lrscd to calclilatr t c c i ~ ! ~ t In s. these cases thr rvciglit IVL>II~LI be a c h i ~ w c dbv dividing thr t\..l\.r 1 ycrccntagr 11!. i l i i~ z , I \ Y 2 (or 7, 4 ) ycrccril.lgt. Thi5 wilI enLlblt,lht, IV.I\ r.; after wavc 1 to rrnl.rin rrpresentativr o l wnvc 1. So Iring an w n v ~I is represel~tativcc l t I hi# populntlon then tl~twtbwrightings cnn hill p rnch wave remain rrprrsc~ntnti\~e of the pryml"ion. Analysis can bfcomc quite complcx w11t.n mtllhple waves arc analyseif a t once ~ i n c reach waiee mav nerd to bc wcigf~ted difft'rcn tlv. Thcsc strategies for wrighting ~ v i t h i n panel <unrrvG are discussed In dttail In Kalton (19S3a: IqSh), 1,~'pkowski (lQS9), I.vlin et a!. (1994) and Ruck et a1. (1 495).

Measuring change

I n studics that txxtcnd over time we can think uf change at two Icvcls: aggregate and individual. Change at the agErrrgate level is al?io scfr+rrcd to as net c h n n ~ cor niacro c l i a n ~ etvh1lr5 , change a t the initividual Ir\,el is also rt.fcrrr\d tc-r a< grrrs~ change o o r rnicrr~ chance. The meaning id the two ways of t l i i n k l n ~ ,3t>outcElange can bc moqt enwly described W Ith a n example. In WJI r 1 of a .;n~dvabout ppoplr in thc paid wutkfirtcc~ \ v t rnlght identih~ thc prrctXntage of peoplc worhlng full time and thc pi>rcmtcigrorki king p.lrt t t n i ~At ~ wave 2 we c o l l ~ r t thr same informarion a n d find that exactly thr w m r pt?rccntages c ~ propl(f work full time and part time as in the F~rstwiivc. While it would b~ correct to conclude t t ~ att the aggregate Ic\lrl lllrm has been no changc hctwccn wave 1 and w n v r 2, it does not fnl low thaf ntr individuals lia\,c changed their level of rvorkfurce parhci~.$tir~n. If 70 pcr cent of full t i m w~nrkrrs ~ had t h a n x ~ d to part time and prr CL-nt ~f part timers h;ld hrct>mc*titlP t~mers wc 1 \ ~ 7 t l l C ( have a large number OF t ~ ~ t l ~ ; ~ ~ rMTJ~CI f r r n ill~ l ~'!tc~ had f cli.~ngecl 111t)irI c ~ c luf workic~rcrp r l i c i p k i o n , but at an ; ~ ~ ~ T c ' R lcvel J ~ c 11 wnuld appear that t h ~ r ~ ~ . ~ 110 1 %
clrangc. At the aggrtal;.alr l ~ v e l changw t y En~li\.ldualsin onc tf~ri.clinn can be cancelled rl~ltby changes in thc opposrtr direction. Distinguishing Lbctw~cn aggregate and Individual change is irnpnrtant both at a theoretical Icvcl ;lnd at the rtwarch dcqign level. it 1% easy to miqinterpret IOI\? Icvrls of aggregate c h a n p , I f tve interpret I Q ~ + ~ regate change as thc ,lb.ience ot individu,?l changc rvc will arriipcat ~ l ~ r l c r c m c l u s i o n ~ahnu t the phenomenon. Fur rtample, the ~ercr.n tagt. of people who rccimivc welfare and the charactcrivtics of such p ~ o p l c arc Fairly stahlc ovcr tiinc.. This has Icd snmc prople to believe that wrl (arc. r~ciplcntsare a stnhlc and unchanging group. In reality, there is a large h~rnover of thosc (In w ~ l f a r e with rela hvcl y Icw continuouslv d c p ~ n d m t on welfare over timr (FIakirn, 1986). Stabilitv at the aggregate lrvcl will indicate snrnethin~: about the nature 0 1 tlir macro system (i.c. thcrr 1s something nbnut tF-tc str~~cturc of the s t ~ r n l and wclnornlc s y t r m that produces a given 1ct.d of pcrrple on ~\.i*li,~rtb) Fln~vp\,pr,it d n ~ not s t r ~l l ur .ihout the indrviduals and the naturc of changes thev experlcncr. The type of rr>t7;lrcll design we adopt urill dcpcnd nn whctlir~rour Sncu~ is on the aggrr*gn!c or the individual It%vrlof changt.. h srrir* nf rcpea ted rros<-sccr rclnal ~ r v c v is s qil ite appropriate for backi tly: ~~~~~cgat' change. Sincv qilch n s u n e y mea\urta.; c . h , ~ n a t~a~grclup Ik=rr,l tvc need ccrrnpa~abl~ groups river time. Hc~wcvcr,11 we are intcrc~trd in change a t the ~ndivrduallevel wc have to u\c n pane! d e s i q bt~;tt~.;c lllf Game rndividual~ arc inrcrlvcd i n cach %\'.~I.L*. D a t a will he analvsrd diffcrentfy dt.pr,nd~ nc on ~Vhrthrr WP a r c t - ~ l m ining aggregate o r individual level c h a n ~ rI4'twn draling with agcrth}:ate

Vot only d n c ~ bias ~ t i our ~ atr~lity ~ t tt, n ~ ~ rcliahlp k c estimatt,.: frrrm a %ampleto the pcrp~~lat~rln but it ran .11+0 d~qttrrtthe patterns of rt*lationships ilnd I-rrtrz~trnvariables ~ 1 1 1 i 1 1 101 sample. Tt 15 not alv,.al;s pm5ibIe to rt-wcil:ht samples reliahlv. tlowl.vrr, we can en5urc th,it the pattern of rr.l;ltion.ihips wtl find betrvcrll varinblc5 is not cllw tn thc rffcct of the variablt* It~rwhtch we have bia<rd information. To do this we cuntrtd qtatisticallv tcsr var1.1bIc4where t\.r rithrr know o r believe thertl m a r be a bias. Bv z t s i n ~ <tnti<tlca! controls [ c k ~ . ~ p l c17) ~r tt-c can Inuk a t r<=latinnships with the r t t t ~ ut' t other rariabfr. rcnm\?rd. Rv r c r n ~ \ ~ i n thc g cficcts ot other .r.ariablt,\ wtac;ln e l ~ m i n q tkc t ~ cfftlct of a ~.ariab!ewJit3r~sarnplt~ hias is evident

I ~ , \ , t h l change M Y ' ;trc reqtricttd t r j nggregatc rntLasurc.;. I-.or e ~ a m p l ttviicn , c ~ , ~ n ~ ~rtqys~jiate ninz rhangt. i v c ~ 7 i l FI-umpnrc yrrltrp nlcans nvcr timc, rhancc.; i n Rrrllry vnrlancc, nvrrall percvnt,rces i n particular cntc:~nrli*ia t rl i l ((*rent timc points ( e . pewcntagc ~ unemployed, ptlrcrntagc on wcl(,in. txtc.). Whcn adoptinn a n aggregat~ change stratcgv we can compare whgrnupr; to ~ c ic f the ertcnt and direct~rmof ag~rcr:atr chan~c differ bchvwn sul,eri~~!ps. For example, we rnifiIit track c h a n c t r , at an agere1 ; ' 1 1 ~ Icvel, in tlw 1ci.cl of rlfrtirnism of b i ~ + i n ~ otvnrxr< ss and empln\.ccr. bcfor~ and aftcs the election n f a labour oriented go\)cmment. Ry cornr,?ring the difftmrences in ~ g ~ r t . g a tch.ingc e for thc rlifftvent group< we can brgin to ~dcntit!. qnmc r j f tlic fiic~orsthat affiact c~ptimtsm 1 1 : Ilrt+

htl\turc

t>f

nominal and

ink-rva! (r#+nhnuotrs .~nt! categtrn .I11

trmr scri#% .inalvs~c;latcnt ~ a r i ~ b4ru1-tural lr rn(dt.1A Z ~ C ~ r* ' Jltlr I AVCI )\ 4, rvpcwrln I\ llh Jummr

Wnrnlnal (L-alegorical)

vnr~ablc+ A N O V A , nun-parantetric AKCIi74,rlrirnrnv varialdt.

Diwriniin~~i~l L1n21y<i5: I ~ ~ ~ t~ ~ r, I ! probit .~n.ilvsis, Eoh~stic
repsci~ui; hnrard/sun~lv:il/

il~vK,'~l?tl'

IL+:~,>~,

f-lowcver, nonca irf thesc htrrltegies ill lows US fn identify the cximt and multidirectional n a h ~ r cof changc n t khe individual Icvcl. Wc cannnt identifv thc d e g c c tn which ind~viduals changc, the numbt*r wIln cl~nncc in nnc rl~rcction or annther. or the mwlt~riirectionaS nature of change (infliws a n d ryutflorvr;). Whcn we can ~cltantih i~idrrvrlrrni< whtl cliange thcn our analyq~r c~ln fw~rs o n identifying t h charactcriqtics ~ 0 1 rlinngerq: t h o ~ c who thaiip' to small and large eutr-ntk, and tlio.;t*who ~-li,irr~c in oppo.;ifr dircctiijil, To uondisvt dnalvqic nt t h i s 1t.t t.! lye rl*qlltrt3pane1 c3altl For Indi\,~iI~rnls.

V i x t ~ r r r -1 1 1 nominal dncl

~ntewsl it-vntini~uus .~nrl

event hictr lry ,~nalysis LLV-lincw .inalv~is;k o ~ i ~ t i t , reFrclw>n. lnzard /.i~~wlr.aI I

rl-en! 111~lorv annlvcis

cateE(.clrlcv I)
Non! tnal (catcmrical)

Log-lint'.~~. analysis; mulli.;laltn~ life t,lhtr n~o~lcls; haznrcl/ sun:ivnl /#>writ histon' an.hv.;ir

Thr. tvpe of analvqis undertaken rvilE depend on the Irtpr* of rlro~i~pr bring considcrcrl Rliq relates to thc icvel of mcasuremcnt of the dcpendcnt
variabl~.

If the depcndcnt variablc is nominal (cn tegorical, ilunlitative) thcn change will simply consist of whether pcople have c h a n ~ e dfrom crne slatr to some otlrcr \tatc (p.c.nlanied to dlvnrced; crnplor-ed tn rlnrrnplovcd). With rlualitatrve ~lrpendcnt \~artahles WT rrill uqe nnalysls m~tliods that can copc with this t y p e of dcpendent vnriahre. 711r particular form of analysis will also dcpend nn thc level of measuremcnt of t h r inclependcnt variables. Mcnard ( l ~ c ~ has l ) summarized the main form.; clf analtp~iq appropriatr tn different ctrcurnstancw and t h r . ; ~ arp proviclcd in Table 4' 1 X dc5cription of cach of thew frrrnrr of analvqrs i s br,vt~ntlthe scope uf this bot)k. Whrw dcpentl~.ntvariablrs are r n e a w r t d at the in t~vvnllevel ivr c.1" rnl>a<tlrcthe alnarr~?t or quantitv of changc rather th.~nstrnpl~trrht.thcr tl1r.v ~ J V P C ~ J R C ~ ~ C or I not. 1Z'r can do thi\ l7uth thr a ~ ~ r c c a I t~ e rt~t~p) and ~nrl~vidual 3 1 ~ ~ ~ For 1 5 . rxample, r t 7 r r n ~ q h trne;l+lirrbthe c f ~ g r r , l a to whlcli cconon~icw ~ l l b e i n ch;anges ~ nrrer time. Assliming that wtmcan accur.~telymeasure cconornjc wellbeing (c.g. income IPvr~l, net wtrrth nf a.;.;tat4 r n dollar.;) !zFe can qirant~fv how rnltcl~tlctter (or wcrrse) off pcnplc artm rrircr timv Approyrratr mcthndr; cbl nnalysln~:chanrc whcn thc chnnjit> \barialslc r s quantitative nrp outlintd in Tatrlc 11.\

This chapter zznill discuss only rneastlring changc in panel dc-ignq. Chapter 12 will ct~nsider wav.; tn which change i s analycd in mpcated cross-sectional dttsips, I n panel dcsi+:ns we havc to work out how change will bc mca<ltred. Thi.; could 1-r~limited to examining agFegate chanec, in which caw a~grrgatemenwres of the dcpcndent variablc would be ccimpar~da t r,qc'Ii wave. For cxnn~ple,wrc rzlould ctbmpare means, v~rianceqor proportions at each wave and srt, haw much these diffcrrd hetwecn wavcs (rablc 9.2). Iiowrver, i f this is thr only way in which panel data were annlv.;c.d tlirrc wn~rldbr little point in tising a panel deign. .A repeatrd c r o w wcticlnal deqijin n4ould do jt15t as rz.ell. WFivn pant-l data are avsilahle thcv will norrnrtlly be analysibd to idtv.rljlv changcb n l the ind~vidunl level a c wtall as t h a ~~ c r e S a t e Irvcl. Thc qi~c*<Iinn thcn I.; holv cl-ran!:c~ . ~ tthe ~ndli-iduallevel i-- lo be mc;rsurt*d.

One ~ i m p l eway nf measuring ch.rnge whcn the I</til' r l ~ o i l ~ sd-ntrb.: t~ dcprndcn! vdr~nblt.is quantitative i s to c.~l{>ulate clttnrv~,~ ~ v r for f c tvch prrsrjn. Riis i.; dcine bv suhtr,>ctingw7a\'c 1 \cores fr11rn tva1.c 2 scnrt-s (nr

LONGTLJDINAL DESIGNS

DATA ANALYSIS T N I,OYGr'TUISINA I . DESIGN

157

l'able 9.2 t%r!/i

17f

e~nnrtrrriigi~gryrrr,qufc~ ~ I I I > )~Il ~ t r~rf~ ~i l ' ~ w ~~ z ~ L ~ ~

Wave t

Wavc 2

Change
+lo

hlean Variancc I'ercentage In catpgory X

25

35
5U

62
45%

-12
+4%

scorc and divide the answer by tRc wavc 1 score. To convert i t to n percentage change we would multiply the answer by 1UU. For example, a perqon's wavc 1 scorc might be 30 and their w a v e 2 score 4U. %n:

49")"

wave 2 score - wavc 1 score 40 - 30 x 100 = wave 1 score 30

=-

whatever waves arc being cumparcd) and treating the difference as reflecting change. However, there are several probIems with this approach. 'The first is [hat any rnpasure will have a degrec of error. By using a measure twice (wave 1 and wave 2) and computing a change score, the unreliability of cach test will be compounded in the change score. Great care s h n ~ ~ be l d taken in using change scorcs unless the measure of the outcome variable is known to hc highly reliable. Another related problem is that extreme scores are more likely to change due to the pheliomena of rcgrcssion to the mean and tnrncation effects (Chapters 5, 6). 11 can tl~erefc>re be difficult tu tell to what extent thc chnngc score reflects reat chaiige and how much it rcflects measurcmfnt errnr. Residual uhi1nge scores A problem with raw change scores is that the amount of change may not be independent of t h e initial scorcs. For example, we might expect more change among thosf who had extreme scores initially or, for some other reason, we might anticipate greater change among people who score m the middle of some scalp. Certainly the dirrction of change is likely to be different depending on where a person ties on some scale. Tl~ose at the ex trcrne ends of a scale can only change in one direction. O n e way to remove the effect nf the initial score is to use residualized or regressed change scores (Kerlingcr, 1973: 337; Menard, 1991: 45). ,Theseidentify cases where a person has changed more than would have been expected on the basis of their initial scorc. Using this procedure we use regression analysis to predict wave 2 scores from wave 1 scores on thc basis of tlic correlation between wave 1 and wavc 2 scores. We then subtract these prrdirted rvave 2 scorcs from the l ~ ~ t l l (wave lr 2 scores. What remains a the residual gain score - the amount of gain that is F F ( ? ~ UP tu the influence of the initial wave 1 score.

10

30

i n o = 33%

The wave 2 score represents a 33 per cent increase over the wave 1 score.

All these ways of L'listing~iishinp between rral c h n n , ~ ond ~ lack of ~ t l i n t ~ i l i f y measuring changc have a common problem: they all rely un possessing seIiable measures of t h e dependent variables. Meeting with this condition can be espec~ally problematic when usmg numeric dependent variables. One approach that can be used to help distinguish between real change and 'change' produced by unreliable instrumenk is to replicate the analysis using different measures. If the same p a t t ~ r n s persist when a number of different measures of the underlying concept are used it is error. less likely that the mcasurcd change is due mainly to n1~asurcmei1t Thc achievement of consistent patterns when different rneasures are employed s~rggeststhat the pattern is not dependent on the particular measures and 1s not simply an artifact of unreliable measures.

Honr much c h a ~ l ~ i sr change? Given that there will always be some degree of measurement error, how is change to be defined? The problem of specifying how much measured change has to occur before we are prepared to h a t it as real change is a difficulty regardless nf whether raw change scores, residualized change scores or percentage change are used. If, for example, an a scale of 0-20 a person changes b y 0.5 of a point, or if a person changes by 1 per cent, do we treat this as change, or is this degree of change likely to be due to measurement error? Should we employ a minimum cutoff point and only define change of more than a certain predefined level as constituting changc? There is no simple or clear-cut answer to these questions. The basic rulc shouId bc that the more reliable the measures the more we can treat small changes as real changes. Sincc many var~ablus in social science are prone to measurement error we should be cautious in what we define a s
change.

P~rrrntngt,clrang~ Another way of measuring change at She individual level is to calculate the percentage change in the initial scnre. This approach is only appropriate for ratio scale variables where there is a non-arbi Wary zpro (this includes most continuous va iables in sncial wlence ~esearch). I-Iere we subtract the wave 1 score r t m the wave 2

Adjlistinp.for 'illflation' Some measures (e.g. income) have to be adjusted ovcr time before we can measure change. In snme cases the actual

LONC17'U13INAI. DESIGNS

txpwss'd In year 71K)O d i ~ l l f i rThus ~ . 51IJOO vi8nr 71Kll) dvrll,~rc rvcwld L-uv 51673 wr~rtlinf goods IF tIr;it 51011f) was syrnt in 190.
/ . T / ~ ~ - L ~ ~i .f G f P lF'Lj+l L~ ~t~i//rf~<

SlW3
5UZII 5MA $796
Y
Y

11"'
x I1

"2

( r e TI)(1-X",.) ( 1 c lW-8" ,)
l [ Y l - ~ ,)

IlCj! ( l f

4)

Q?

$712 $W5
EhhY
5f7Y

Y Y Y

(1 L G

0 Vfl
11 44

0 kt7
I! rr:

<hl)J
SWl

v ll U H

Eupreccrd
\car

rij

?~~fN th~llirrs )
$

Inflation rat+. 1",,l ?OPE

StnlrJni-di:rlhn~t: z-scorcs { ~ n d {vrsmtil~ rnrrk Another form r r t adjusting scores is to cxprrss all scores in terms of whcrp they lie on a di.;tri'hutinn. T h a t is, ~ , r can , sinrrtLrdh9 scores and, rt7thr.r than looking at change in nbsoltitr values, we look at changes in a pcrson's scnre ~ i ~ l n l i vto r clthfr p f o p l ~( w bl,~rsh, ~ 19% for an accessible d i \ c l ~ w i o n OF a ranKim nf method.; of stanclard~/ation). We look a t changcs trr whew a persrln Iics in a distrihuti~)n. Whcn Iuoking a t change, in rrla tivr rather than absolute terms, wc would only count n person at. c h a n ~ i n g if they changed their positinn relative to others, tliat is the!! mnvcd more (or 1~1s.;)than some clthcr penplr. To eyprew c l i a ~ on ~ gan ~ ~nten~al Ic\.el depmdcnt tbariablc in relat~vrtc-rms we can ruprws each pcrson'5 score at r v a v r 1 and wave 2 as a :-<cnrcl. With an ord ~ n a dcpendvnt l variablc we can use a person's perccntilc rank. We W O L I ! thcn ~ measure chanjie by look in^ ; I !changes in z-scorcq or pcrcmtile rank, For e\nrnplc, i f w7e wrrtl lrloking at cha~igcsin incorn? rrtpcrtime wc. could convrrt each person's income into a :-score which ri.i~ulcltell us about tll,lt pcr+rmrsir~cnrnt, rz~l~itive to otl7t.r pcople's incomi, If we had t h e w z-svc-lrcs Cnr khe samr inslrviduats ovrr two waves rvc ctluld identify those whnsr :-score incrcnqed and thosf for whom it dpcri*ased. Wc coulcl also quantify how milch the pcrqon's relativc inuornc position changed (Table 9.4). Altcarila tiuely, we I-(ruld calculate Ihrir percentile rank In tlic lncnme stake.; and see how thcv chang-ed over timc in rank. This mcthod o f measuring change in relative terms is \aluablc when wr need to ~ d l u for ~ t changes in the wav in which the dep~ndcnt variable is measurcd over time. H(>wcvcr, it 15 a mcthod that can br used even when wc d o not necd to adjust for thc way in which thc variable is
measusrd.

inqtn~rnent mny change [e.g. questions may need to bc changed to match tht- age or life 3 t a ~ e of respondcntq) For other variables, such as income, where the rncnning of the categories changes (P.K. income need.; to he adjusted for inflation) the vnr~,lhl.r* itself neccds to be rescalcd to makc mtms>ures cumparable over timr, I'i~r cxamplc, w o may be Innkinq af changes in incnrnc among mrn and tvornen betr\.cm l Y Q 0 and 2i)rl~I. Ilrfore we can rn.~kerneanin~fulcorn~ x w ' i ~ obehzrren ns income levels in EL@9n and ?(1110 wc ntvd to know h t ~ v much the s a m c cjc-hlar amount is tvorth over t ~ m c ,attcr aJju5ting for intlntion. Bccnust- c ~ finflation $1000 in 1990 is not thr cquivalen t of B I O E 3 0 in the year 2000. In Table 9.3 incornc ha5 been adjusted bv the inflation ralli>. Thr incomr in m c h year ha< bren expreswd In 1Y90 dollar<. T l i u ~ , tsr adfustmrnt~ ic>r ~nflation,4F(H)IS in 2Dnn :.i the cqui\,alent ( y f only q.7'8 in lwli dollaw after inflntion. In the second rrample, thy rjpprl.;rtc r.alc~ilation~ hdl\~z. tlren rnadc ic~ththe worth of 510011 in each ycar bring

I
I

I
I

Since thc --woreindicntcs tlic number of qtandard dcviatlonq a prrwn lies abovc o r belor*, the .;ample mean we can usefuliv compare changcs in their paqition over timr. In Table 9.4, c a w 1 remains below the sample mean a t wave 2 but has drawn close. Casc 2 has deterinratcd relative to others by 0.4 of a standard deviation. Casc 7 has improved ller relativc posit ion cijnsiderahl y. Rv using z-score< we autornnticallv adjust for cliffetencck in actual inconlt. due to inflation and the twpral l rncrenw in mean income. This approach enabltr crs to f c u s on chnnccl; in how people ar? doing rcIntivc to others. rlltilrnativcty, we could examine the rrlaticc change of i l t t l ~ v d r r n lbv ~ examining change in r ~ n k ('l'able 9.5).This is comparnblc tn Iclnking at changes In T-scnrts hut dopa not qunntifv the amount of difference behvcen tndividuals. It i~ thrrefore useful wit11 ordinal levcl dependent variahlrs. In T,3ble 9 5, ttlc rrlative po\itinn rrf rase 1 has irnprurl-ed, that of case 7 Ira.; detenuratcrl ~ h a r p l ywhilr that of cnsf has irnproroed markccllv.

160

CONGITUDINAI. DESIGNS

DATA ANALYSF IN LONGlTUDINAL DESIGN

'Table 9.4
Case

Cllarlge.;

irr z-scorcs of rndrcidrtals

Raw score

Tablc '4.6 ~Mrnnzscnrr

7 9?0-2OllU
Raw score 2000 (income)
;-senre ZWO

filr 1 1 1 ~ 1 1~ r t r l fT L W ~ I ~ P I T ,

IVY0 (income)
1 (female) 2 {male)
3 4
(female)

2-wore 1YYU

Change in :-score

1 W

ZOO0

Change

(male) 5 (fernalp)
b
(female)

Table 9 . 7
Year

Fmnle rnrnrnp as a pmprf~iln of male e#r?iings

Colrrinn 1 Averape FT male $ pa
$5.ODO $1 i,OOR S25,OOO
Col~rrnil2

7 (male) 8 (malc) 4 (female) 10 (malc)

Rv~ragcFT female $ pa
S3,400 $10,500 $1R,OUU

CuIumn 3 Female as prnportion r i f mnlc (crdumn Zlcolumn I)

n.68 0.7U 0.72

Cnlumn 4
Abwlutc $ gap

Mean
Standard deviation

1970 1980

-$1.600 -$4,500
-$7.0UO -$%,a05

1990 20011

S?~.OOO $2h,qi11

0.77

Table 9.5 Cf~nvrgcsmrnsur~dhy rhnngrs In rank

Case Raw score 1400 [ incomc)

Rank

Raw scrwc Rank 2000 I ~ n c o m p )

We can also use z-scores (or percentile rank) to see if some groups change more than others. For example, if we were examining whether 'the income gap between men and women had narrowed between 1990 and 2000 we cnuld obtain the z-scores of all men and all women as wave 1 (from a pooled sample of both men and women) and do the samc at wave 2. Using aggregate lcvcl analysis we could then see i f the mean 2scorc of mcn at wave 2 has changed more than the mean z-score of women a t wave 2. i n Table 9.6, women, On avcragc, havc an income of 0.75 (1990) and 0.57 (2000) standard deviations belnu~the mean ~ v h i l e men, on average, have an income of 0.75 (1990) and 0.57 (2000) standard deviations ~ h o r v the mean. Men continue to do better b t ~ t the gap has narrowed slightly between 1990 and 2000 Anothcs approach to adj~rsting for 'inflation' or other change over time is to c h a n g ~ values into proportions (Table 9.7). Absolufe figures might rndicatc a widcning of the income gap between men and women. For

example, the average income gap in w e ~ k l y incomc between men and women might have been $25 in 7970. l n 2000 this might havc increased to $150 per week. However, 1970 dollars are not cornparabie lo 2000 dollars. We could use any of the above approaches to make the necessary adjustmcnk to facilitate comparison. Alternatively, we might express female earnings as n proport~orrofmnlr comings in each year. In this way we automatically remove the effect of inflation. Thus although the dollar gap between the average incomes of men and women is increasing in Table 9.7, the relative pwition of wnmcn is improving. We can also approach the problem of standardization by adapting some other base. For example, we might want to see if the income gap is widening between the lower and higher income earners. Wc could take the bottom 25 per cent of income earners and express their average income as a proportion of the average income of the lop 25 per cent. Tn this example we see in Table 9.8 that in 1970 the top 25 per cent of income earners, on average, earned five times more than the average of the lowest ZFj per cent of income earners. In this hypothetical example the disparity increasts over the years so that by 2000 income earners in the top 25 per cent earned 7.7 times more than the average income earner in the bottom 25 per cent.

EXPLAINING CIIANCE
Assuming that the issues of how to measure change have been dealt with and we are reasonably confident that real change is being measured, wc will probably also want to explain the change. The theories and propositions around which the research is designed will provide guidance as to how to go about testing explanations.

Describing change
C3ncc changp has hrcn conccphlall7c*d a n ~ l mcasuscd, rlr*cisiiln~ nlirct I7e made about hnw to reprt-scnt this changc. 'rlicse a r r \.anuus math1.matical and stnhstical w 7 n y s o f doing thlh ( M ~ n a r d1991: , Wff) bur IIL-re1 will only mention hricfly some ways nf d i s p l n y i n ~ changt* in tabular and ~raphira! form.

I-lowt~vcr,the prcll9lrm I n a n b cuplan~tivnis that w e cannot be 1-rrtain what F,~ctari havr procluced thc ch;lnge. Bccausr w e ldck tht= lrvcl OF control of an cupcrimcntnI d e i g n , any changr could b~ duc to ~ ~ n k n o w n fact orb that m-curred behvrrn the waee.i. Whtn c~plninlngchange tl~ranalysis ivill be bawd on con~pasing different jii-vupc:of proplc. We wiFl we whether the cvtcnt and nature o f ch,i~i~ int one ~ gnrup arc diffvrent from ~ t l i r s grtr-trps Anv diffcreiic~s in ch,lnge yattc.rn< of grcwps i5 ntlrrbutcd to cllli~r ditt~*rcnccs that mav c x ~ s t hctwcen ~ h poup\. r Ihkr., for t.xarnplt, a shidy cxaminlnq thtl impact nf p,lid cmptovment on the sclf-estrcrn clt young people Using n pancl dc>i ~w Hbecnulci rneasurc scllc5tcwn at t ~ v o or mow w.nrcs. 'rc~ tcxsk the rftcd ot crnployrncnt on self-t*sttlem we would identthr thosc young people who gained ernplrrymcnt Getrvccn wavcs, thnse who becamp t~nernploved, those who remained cmploved, a n d thosc wlrn have rcmninctl uncrnplovcd W e might find thnt thosc who eainrd ~rnplovrnent showed a real irnprcl~rrnrnt in sclf-rsfrrrn, and those who hlst thcir jvh shnrv~c1 a fo.;~in self-cstet.m, while thr stabfc emylaycd and ilnemyloyrd showed no sclt-cst~cm c21nngt~. It tvou Id br tempting to ctmclude that it i q tlic diffprcnceq in emplnvnient that account tor the d~fterc~nt ch,~n~e in s self-eqtcvm of the p+c~ups. Thc problem, howevcr, is that whe du not know that ctiangrs in emplrrymennt s t a h ~ s are the only changes t h e people a r c e ~ p c ~ i c n c i n g . Nor d ~ we 1 know that the four p o u p r are comparable E n otlicr wavs ( r . ~ . gpnder, age , ~ n deducation). In experimental deqijins this prclbleln is l~anil lrd 1.y random nllocstion to RrtuFs. Tvplwlly In panel ~1csim;ns this I.; not \x)w~trlc. n l c maln i\rav of r m p n n d i n ~ tn the problem of compamhilih i s to i15c st;ltistic-al controls to cr;~ithli<h cnrnparnbil~t\' among the );snup%. Thic gi?,ll cat1 be achievtbd bv u.;gng one form or anilthcr o f rnr~lt~vnri;ltc amIy'is This s t r a t e p ~~inablcs trs trl rcmclve thc. effwtq u l kntl%zbn d iftertnnct,? h~twt.r.11 the ~ T L ~ I &CP F ~ Sit. i~ estaitrlish~dthat t l i ~ p u p ' ; are qirn11~1r In the> .;pcc~ht-tl rc..;pccts i t is thpn pa.;siblc to cstabllsh hvw much the ~oitpq Cl~itcr tn regard to self-estwm ch.~nges.Thr log^ and tccf~niques of thjs approach will bc dizcii.;<ed more full\. in Chayt~.r 12.

Change r>xrertimr can L l c p r c ~ c ~ n tcficctivcly ~d in tnbu2;lr (ornr. The pnrticul~rf ~ r m ivilZ depend on the n~trnhl*r o f tEnw points oktar ~\.llich ch,lngc is being trackrd and ivhctl~rrchnn~rais L l ~ i r l consider~d ~, ~t t h ~ aggregate or thr ~ndi\,fd u.tl levt~l. T i c c*arlFcr table.: prnvidc illustrsticlns nf w m e of t h c w a y i l l wliicl~ tablrs might displav clinnge. Tlicse tablcs will usu;lllv h a r ~ c !cam c+none axic - r i t l i ~ r across t l i ~ tnp (e.g. t a b l ~ s on cohort annlvsis I n Chnptcr 12) or d ~ w n the .;idt% (e.g. Ta1,fes q.3, ').7 and q-8). The wav in wllirh c h a n ~ e i s indicattrl will vary dc.pendin): m ivhcther agpt-gnte or inclivi~l~~al ch.~n~:t> is being c,on<iderpd 51rntn of thc V J S ~r,anp pi pcic~iblr (rat.5 r,f rt*pnrtiilK change is p r o v l ~ l ~rn d l ~ ~ i b lU r Q. ' 12'11cn dcLllinqwith change ~t the 1nd11~ld11aJ I<,\~rl, .;~ni~l.ir a~~rtrsrlir.; applv. I'mods ovpr w h ~ c h ~ncl~vldual clranqc i:. h t + ~ tr.lc-ked ~ ( c g . vrar) \s 131 bc i n r l i c a t t x i eitV~r>r acmsq thy top i i f thc t,~ble1lr trn thc *id(,. Thc othrr axis will represent tllr particular {ray in which ~nrlividual changt. is being cmccptunli7ed. A ~ a i na , \vide rangy of poq.;il-le ~v;lt.'; of rcprcsentin^ ~iidividualclian~c* i< pc5rsilrlc. %me commtm approachez arc ill~istratrd In Table Cb.10.

Graph5 a r c a g00d way to rtlprcscn t chnngc. The most ccmjmnn h-pe crf p a p h for $ispl;ly~ngchan~e;c iq Ihc trenii graph, This stytion \rill p r o ~ i d t a br~cfintrcldttction tcl trend graphs A filllcr dircusrion of thc.;t* ~raphli and other w ~ y of s dispiavjn~ chnn~tn graplilcal'lv can bc found it1 Hrnrv ( 3 99,s). The sirnplc trend b ~ ~ i p1%-ill h h a \ e mo ayes: tlir X-nxi~ar-ros< thtb Iyclttcln~ant1 the Y-a.;i> on the ~ i d eThv . S-axi:, ~ v i l lrrprcsent time rvliilr thc Y-ari.; will rt.flwt ~iq ~ ~ ~ t i t on itv the vari,~blt that iq b ~ i n ) ex,~rnined : for cl-ranct*.Figure 0 1 ~lluslratesa srmplc twnd gmpli. T~L )'-axil; ' c~3n bc r n ~ a r u r e c u.;ing j a w ~ d ranee e of mraxurcrnenl t l r ~ i h n ! Fig~tri.u.1 it 15 m r a s u r t d as a ratc - t h t ~ nitnlbcr crf r i i v n r c ~ I,clr ~ ~ 111013 pn~~i~Entirm aged 15 or nver. Flnwri.cr, the Y-axi5 cnuld Iiave denoted the* trri~~tllrr of Ji\ nrcc.5, err snmc other rvay t r t lndrcatin~tlir prc\.alenrtb t r f d i vnrce. Figurta 9 1 prllvicics trend data f-r A @j-,31>17~

""-

'

164

LONGITUDINAL DESKNS
r ~ nbou! Table 9.9 Exnrrlyle.: uf tabrtlnr n<n!c;of ~ ? l r w z t i ) dtrta nsg r.rgtitr clrnnp
Year
1998-9

DATA ANALYSIS IN LONGITUDINAL DESIGN

.-

201113-1
10

2002-3

'

Mean
Varianw
9G

5
80
~ mean) n

15
HU

8tl
1on./,

change (

5o0O 13.6 per IORO ?%5,5,0Cf

horn previous per~od

rat^

12 pcr lOOO 510,000

12 8 pcr 1000

Gaps (c.g bctrveen male and ferndie avcrdgc income)

$7,5110

1970

1975

1980

1985

1991)

1995

prnprlrtiun nf male

Year

mtr, Aartra!in, 1970-95 (de Vatrs and Wolcotl, F i p r e 9.1 Crzrdr ~fi~wwrr 19971

% in~pmved[from

+Males

--t Females

All

1
-

prcvious period)
% stable
" 1 0
"dj
"<)

5%
85%

15%
80"/0 5"& 2wyb 5Ys

2%

go

. . -.

- - -

7SV/"
20% 22';'" 7.5%
-u

decl~ned

tW
"'%1 10%~
~5 ppinnh

80

change
cliang~ng by X amrn~nt

.
t

70 -

Vean improvement of improvers Mean dect~ne nf decliners

10 points

35 prr~ntq 10 points

40 p o ~ n b

10 points

K -

+
- -

k 5
z

:so
50-

--

Figure 9.2 s h o ~ r the s different tmnds in employment IeveIs fnr men and women aged 55-59 over the years 1975-2010 and compares these with the overall trmds for the age group. By reporting the trends separately for the different subgroups (men and women) wc can obtain a more finegraintd picture than the trend line for khe age group as a whole provides. The way in which the X-axis and the Y-axis are scaled can distort the trend in the data. Each graph in Figure 9.3 has the same information a s Figure 9.2 but stretching or contracting the scales provides a very different impression of the bend in the employment levels of older men and women. In Figure 9-33 the changes in employment levels appear modest because the X-axis has been stretched and the Y-pxis contractcd. This has the effect of flattening out the trend line, In Fibwre 9.3b the X-

a
/ / -

1 30 I
20 f 975

1980

1985

1990
Year

1995

2000

2005

2010

Figure 9.2 Ernp10,ynvel~t mte uf p e q ~ l en x ~ d 55-53 h!j g m t d ~ r A~rst~n-nl~n, , 19752111 0 (arctrml and predicted) (de Vaus and Wolcntt, 1997)

166

I,ONGITUDINAL DESIGNS

DATA ANALYSIS IN LONG! F'U131NA1, DESIGN

I67

(a)

1
$00 L-

l
-

---.
2 '

s 2 4 year old malo sutrae rate a$ tat1001 male age standnra.rpd so~carle fa!-

-24

B k soh-

war old male u n e r n p l m l ralp ns rutlo 01 uvprall unemploymrrll raw - -

p g

6,

I~
1
;
I

E d 0 1 . 20'
1975

-Males

--

1 7

1980

1985

1990

1995

2000

2005

2010

Year

[ I I

e z

(b)

-Females
-

---

All -

0.5

0
h 9

I-T--l

,
@ ,'
$ 0 '

,-7

. "

'6 2 4

, G O

$ ,$ $ ,'

\d6 4 ,'

8 ,,#
Year

& ,'

@ , '

'u

%
d

z
E
U a)

60-

- -

--

Figure 9.4 Malc s u i r i d ~ duntlrs nrrd n r ~ m p l o ! / m ~ mfr n t ratios, 20-24 yenr d d s , A~tsfralio, 7966-90 (DHSH, 1995)

a"

.. 5 0 ,
I
40
-

axis has been contracted and the Y-axis stretched which has the effect of

20
1975 1980 1985 7990 19952000 2005 2010

Year

Figure Y 3 Etrlplqrlment 2010 trrr'tttal n ~ r d predicted) Wolcntt, 1997)

[i~)

ofpro~lu nged 55-59 lry ,yeyellcIL.r,Aw5trnlia, 1975fIattmed scale, {b) e . ~ r r d d scale (de Vaus and

exaggerating the trend rate of change. Trends for multiple variables can be plotted on h e same graph a s illusrrated in Figure 9.4. The figure plots two variables: suicide rate and unemployment rate of young people. In this figure the trend line for suicide of 20-24 year olds has been plotted alongside the unemployment rates for the same age groups. Plotting trend llnes from different variahles on the same graph can highlight the way in which trends on different variables co-vary. However, one must take care not to assume thrt t this co-variation of trends demonstrates a causal relationship. In F~gure 9.4 each of the variables is measured on the same scale, so each variabIe can be plotted against the same Y-axis. Sometimes wc may wish to comparc trends on two different variables, each of which is measured in different units of measurement. Tn such a case we can use two Y-axes - each with a diffcr~nt scale of measurement. In Figure 9.5 the left hand Y-axis (primary Y-axis) is the seric>us crime rate per 500,000 populahon, while the right hand Y-axis (secondary Y-axis) represents the percentage of ex-nuptial births (birtlls to unmarried mothers). Again,

168

LONGITUDINAL DESIGNS

DATA ANALYSTS I N LONGITUDINAI. DESIGN

165

+ 2

3.5
3
2.5 2

X m
g
9
0

8,000.

m c a .w

m -

51
rn

7,000

2 0
a l
ur

$ g s,ow
j
=
5,000

r
n

X
w

1.5

4,0003,000

0.5
0
1970

.-5I.-

1995

1,000<

Year
-

Flgure 9.1, Crude drirorce rntr, Austndin, 1970 niid 1495 (de Vaus and Wdcott, 1997)
Year

Figure Y 5 Ex-nupfnol brrths nnd serious (Sullivan et al., 1997)

m'mp

rate, Ausfufllia, 19133-93

great care must be taken not to confuse similar trends with causal
connections. Where a longitudinal study provides data for just two time points rather than long term trend data it will normally be more appropriate to use bar graphs to represent the two time points (Figure 9.6). To simply have two time points and join them togcthcr with a line to form a trend graph could be highly misleading since this could hide h e variations in the intervening years (sec Figure 9.1 for these variations).

When measuring change it is important to distinguish between chmgr at the aggregate level and change at the individual level. This distinction has been discussed and its implications for analysis strategies have been considered. Prnblerns of reliably detecting change have been identified and some ways of reducing these problems have been outlined. Finally, the chapter has d~scussedthe need to standardize some measures over time so that real rhangc can be identified. Different standardization strategies were described.

Note
1 It can also occur in expenmental designs but can be avoided by adopting a randr~mized design without a pre-test measure.

Summary

Since a basic aspect of lnngitudinal designs is to describe and explain chang~, the focus of this chapter has been on analysis issues that arise with the measurement of change. One important problem is the problem of missing information which is caused b y incomplatc data collection at some point E n the multiple data colIcction waves of longitudinal studies. The different types of missing data were deqcribed an$ strategies for dealing with missing data during data analysis were evaluated.

PART I V

Groups

T~me 1 (TI)
Initial test

'Intervention'
(XI

Time 2 (T,)

CROSS-SECTIONAL DESIGNS

I
1

F~ve years later

1
1I

Level d marital

No children

LWBI 01 manta!

happiness

1
1

Haw Orst child wrthln last 12 months

1 LWI sl of marital
appiness h,

CROSS-SECTIONAL DESIGN
lave first chrld ween 12 and 60 rnths previously

I
,

Level of marital

('4
Lewl of marrtal happhesq (Y)

Cross-sectirjnal dcsigns have three distinctive feature?: n o time dimension; reliance on existing differences rather than change following intervention; and groups based a n existing diffcre~~ccs ratl~erthan random allocation.

No time dimension
In the standard cross-sectional dcsign data are collected at one point of time. Tn this ruspwt this design differs from standard pancl designs and pre-test and post-test experimental designs where data are collected at a number of different points of time. As such the cross-sectional design can only measure d f i r r n c e s between groups rather than change. , The distinction between measuring difference and measuring change i s important and may be cIarified with an example (Figure 10.1). Suppose we want to Ieam abnut the impact of children on marital happiness. A panel design would track people over time. It would begin with a group of d~ildless married couplcs and measure their marital happiness. Five years after the initta! wave of the study we might revisit the sample and rerneawre their level of marital happiness. Bv this time some couples will have had children and some will h a w remained childless. Snme couples will have had their first child recently (say within the last year) while others will have had their first child earlier (behveen I and 5 years ago) Wc could then think in terms of three groups: the childless, the new parents (first child within last year) and the established parents (first consist of child between one and five years of age}. The analy~is~could looking at the degree o f change irr marifaf l~aypiness0 5 the three groups

between wave 1 and wave 2. Different amounts of change in marital happiness of the three groups cuuld reflect the effect of parenthood. A cross-sectional design for the same research question would involve a one-off study in which we measured the levels of marital happiness of couples, whether they had any children, and, if so, the age of their eldest child. Using this information we could create three groups: those without children, those whose eldest child was less than 12 months old, and those whose eldest child was between one and five years old. We could then compare the level of marital happiness of the three groups to see if there is any difference. If there was any difermce we might be tempted to attribute this te the effect on marital happiness of I~nving

children. Essentially the difference between the two designs is that thc panel design enables us to compare groups in terms of diJferrnf anlnlan ts of chnfzg~ on the dependent variab4e while the cross-sectional dcsign involves comparing the groups simply in terms of diJfcre?lc~ on the dcpendent
variable.

Reliance on existing differences

Because the cross-sectional design entails collecting data at and corlccrnin,y one point of time, all analysis relies on differences in the sample at that point of time. Unlike the panel design the cross-sectional design

d0t.s t ~ o t allow for difft~r~iiccs to emerge over timc. Unlike the ~hxperimpnral d l - s t p where thcsc 1s an active ~ n t e r ~ t n t i o by n the researcht,r t ~ j prodrlcc chnnge or tn crcntp d i ftcr~nces,thc cn>s~-.;cctiona! design rtblicq on elistlny: difirrences rathcr than examining thc Impact of erpcr~mental intemc*ntsclns. Craw-wctinnal dezigni; pmpfny a relati~.cly larwit*e approach to making causal inferenccq (Marsh, 1982: 6) and can be distinguished from cxp~rimcntsin this regard. J4i ke crosr-scctic~naldesibvs, t>xperimcnh ruplain variation in the dependent variable by seeing if tjiis \win tinn is svstcmalicall y linked to \?ariatinn in rile indepenclcn t t~ariahlr 1.. variation in rn;lnt;ll:hapyincsq rnatchtd bv variations In parrbnthnod stntlrs? Howel-er, cxpcrirnents activclv crcnte variation In the indepcnticnt variable tzuhcn*.lscross-sectinn,~l drqipw rely 071 ckisting diffcrcnccs.

to 'groups'

Non-random

None

Measu-e on

Non-random

None

Different existing

cx,,

I
Figure 10.2 Grorr!)~ i t ! cmss-srctit~)ml d,x5isy~i5

The nature of 'groups' in the cross-sectional design

The ~rpcrimenta!desirn rcllkv nn compariiil: groitpc. Tndivirl~ta!~ ~rc' ranrfun~l\,I Elocnted to Rrtwps that prior tc, .my rxyenn~riita 1 Iiitcrvcn tioil f lic L~vn groups 41nuld be identical. Tlie rcsearch~rart i ~ ~ r l ) ~ pmduccs JiiCrrences bctwcr n thrr groups by a dcIi bcrate and con trt9lrd intcrvm tion. This i n t e r v ~ lion n is the indepcndtxn t variable in the cxpvrlmental design. Ideally, this differential treatment is the only difference behvrrln Sroups and therrfnrp any group differences in regard to the uutcorn~\ ar~able should bc attributable to !hi< d i fierential int~nrcntion. In tho cro~q-sectionaldesign the 'groups' art1 cnnstructed on thv ha+ of rxirting diffc.mces in khr* <ample. The sample is divided irp intn groups acctlrding to the catrgnry of the indepr*nilenlvariable to which thcv happt-n to belong. In thc cnrlier exanplv tIic comparison grcnips wroulri br thnsr without anv chi tdren, thosc whnw elc?er;tchild was IPSS than 12 months old, and thnse whnsc eldest child waq between ontaand five ?car< olcl. The critical p i n t , howev~r, i.: that 'because we mtrct rclv on euiqtinq differences there can be no random allocation to groups. The groups nlnv very well hr dlffcrent in other rcspcck apart from t h ~ category of the independcnt variable to which ~ h c y hefong. This, in turn, can m,ikr i t difficult to work olit whether a11v Kroup difference.; on thc outccvtw varic~t-llc are duc t r l tlir independent vnr~nblc or to other rrlated d ~ f f c r ~ n c tbrt\rreen >s thc crzmps. Lqin:: thra framerr-ork crl t ? ~ t lclaqcic exycrimcntal d r s i v ai; ? . ftamc of ref~rt-ncrb t Vie cross-sectional clc5ign can bc rppr<..;cntcd diaerama t icallv as in Figtirc 112.2. In this c a w the 'intcwentinn' 8s simpIy "irelng in a diffcrcn l cntcKnry of thc indr.prwlmt variable. Thc information about thiq and thp outcome mcastrn: iq cnIlected at rhc same point of timp. Cmw-srctional d e s i ~ n s arc not l~mitedto c u m p a s i ~ o y b~twc~ illst n ht70or ~ l ~ r cgroups. t. Thc number of cat~piric.; of the indcprniicnt

variable ~f~trrrnineq tht. nurnbcr elf 'groups'. Oftrn we will Iook at the c~rre2atinn bctuvccn continztocr~r'ariafiles such as, sav, age and Incnmr Icvcl. \\'hen lt~cA.jn): .lt such scrrrcl,~lion, tve r7rv . ~ \ L t i i f i ivhctlicr incrmi, vanes according tn agc. That 15,w r cnmpare all the a c r IrtreIs lor group<) and see whct11t.r income varie~qysttln~a tically across ngc levels. h surnmdry, cross-sectionill d v s i ~ n sdiffcr from t>uperimental and panel desih~s in that they do not have a time dimk1n.;ion. Consequrntl~, they face difficul tie4 in unarnbrg~~c~uslv i d m ti f ~ ng r the timc scqucncc ot events, H c r a u ~ e of this they tncc the prcrtrrern of ~dentifyingcausal direction. Furthemarc, brrcausc criws-sectional dp.;ips re1y im existing differences rathcr than random allocatic>n, mv apparcnt effect of the independcnt variable may in f.tct bc due to other, i~ncontrolleddiffercnccs betwccn the groups. The problrrnq created by this will be discussed more fu fly in thc rncthodolngicnl problcms section o I Chapter 11.

Obtaining a time dimension: repeated cross-sectional studies Cross-section,~l designs are frcqucntl y critici7ed for lacking a timc dimension. Rrcausc. of this shnrtcclming they are lrnnhlc to unarnbijiu~uslv cstabli~hthc time sequcncc rn which vents occur. This is partic~llarlya prnh!rm with one-off crow-sectional stird~ck.H o ~ ~ e \ ~thr er, repeated cmsc-svctionnl deGm ad(! tesses thiq yrrlblcm to a llmi trd er tcnt. This d e s i ~ n invalr,rs collecting ~nfnrrnationat a n~imber o f diffcrcn! time pointh E ~ r r l from difilr7;l ~ n f ~ pRI l r ench Ij~rrr 110111t. Rather t h a n tracking n Ernup of indivtduals cwcr time and sccing how thow individuals c l i n t ~ g the ~ , repeated ern<<-rcctiilnal design nlitnins cornparablr samples at t><~cli timc point. 50 Inng as the sarntx rllrr..;ti~~n.; are aqkeii i l l

~1qt.dfor cc~Vlurk anatvsic. T h ~ allow v tiit, in\ . ~ . s t i ~ a t /tco i r both twt-k change and get sumc Iln<lcr+tandini o f the ~ i ; ~ ! u r c)t t~ clinngr. Cuhurt ~n.al~<15 will be discrw~tadI ~ IChapter 11. Repeated mca5urrs designs can take twc) t 11rms:prospecti\)c n nd rptrr7spcctlve. A pro<ppctivc d c ~ i g n is onr w h t ~ r c wc bcgin now and rcprat the study a t various puints in the futurc. TIic rctrospertive r e p ~ a t r drneilaili-eq design is onc tlwt draws on exiqting data scatsto exarninc pattcrns of change up to tht. prc3rcbnt point. Right m c v wc could use tlic C;cntlral Srxinl Suweys of tliil laqt 25 years tn exarninv patterns of change. Census data cnIlcctcd nvrr thc pact ?car+ can br an,~lvwdin a s ~ r n i h manncr r Zrr rnnn!. other sctc ot data that are ro~ltiiirli' ut>llccttld le.g. annual I ~ I V I F T C ~ ~ \tnhshcs, r n a r r i n ~ r ~ldtil, tcrhl~ly htilti>ti~+E.

each of thew tndtlycndcnt cross-sectional samples rvc can build u p a picture over tirnt* nf whatever v a r i a b l t ~ n7c ~ Irave. an r>b\.iotrs e\;lrnplv o f thic h.pe Thc naticinal c~lisuk ~n racll count? of 'design'. T(1 thc extent that the wrnr q ~ ~ c s t i i marc s <~<ked in cach census, changc*s can be tracked uvcr l o n ~ pertnds. Many cr\un!rirs have regular national hwirll surveys that track a far wider ranjy of matters than any national ccnsus. F n the United Stntcs the General Social Survey has been conducted annually since 1972 and provides an erccllcnt way nf Itwking at changrs in hchaviour, belief<, values and attitudrs over the last quarter cenhlsy. In Britain the Blitish Social Attitudes Sttwrv is 11st.d to achieve a rirntlas md. The General Hntrsehold Sun-ev provld- data for hvn dccadcs and thc Family Erypnrliturc St1rr.e~ vi~'IJ5 timr scries data for over 3 3 ypnrs. The International %rial 5ience Survry lnvolves asking a similar core nf questions each year to new sarnpres in rach of the countries that arc1 members of the Int~rnntional Social Science Programme (35 countrics in 2000). Rcpeatcd croqq-sectionat survcvh provide an rxcclIent snapshot of values at different points i n a pcriml id rapid change. Dia~ramrnaticnllvthe repedtcd cross-scrtional design can be reprew n t d as 3 ?;rricu;cri independent <arnplc.s IFipre 10.7). Each timr point d r a w cm a nC*w samplc. Tvpically, car11 <ample will provide a wide cross-scctinn of rastas so that useful crtlss-wcticonal analvqiq r a n Irc conducted witllin rnch s a n ~ p l eWF . can track changes by compi~ring pattcrt~s in each sarnplc with those in previcn~s or subsequent sarnpTps ( c . ~rather , than just <)TIC aEtl group khev include a widr range of ages 5 0 that at any gilVen point n t ttmp the age differences c a n be examined). M'hile this itt-sikm does not prmit thc trackins ot individual change o%.erhme i t enable-< l o n ~ i t u d i n a lanal!'siq at the aggregate Ic~~ck Ise Chapter 0).Fwr!hrrmorc, these r~pca tcd cn)ss-srctinnal q h r d i ~ s car1 bc

1
I

'

ISSUES IN CROSS-SECTIONAL DESIGN

Cross-scction~ld r 4 ~ n s arc probatrlv thc most widely urrd d r \ i g n r i n wcial researvh. C ~ I I P r ~ a s o n for this prlpularlty is that the!, cnablt- the rc~enrclier ro r\llt,iiii r c ~ z l t s ~ l a t i v c ~quickly. ly 5ince data arc ct~ll~,rtr-J ;~t one point n b titilt1 t h t w is no need In .cv,.,littrlr various tollow-up stngcs or intcrcenti~m+ hr.tt)rr* a n n i v s ~ n q the d.it;t. I t iq aJqci h i e t h ~ t tjt11t.r , t hlngs being r c ~ u a ( l c K. 5,implc s i x , p n y ~ ~ l , ~ t r o sn a,n i ~ l c type), crclss-<r.itronaF (lc-ignq a r ~ . nlrwr- r o \ t rtfcctivr th,ln c t > n ~ y , ~ ~ a tc l l\ty . c r ~ m ~ w t . land l I n n ~ ~ t u d i nra l rll + i ~ n + 'Tl-rls . 1~ becauhcn crti.;s-scctir>n;il des~q~i.; d o not r n t n i l t l i t ~cusi 17i r1~[11~~ift~cl iltlt;l collections, (rar-A~ng rcspnndenb r>r t>f c r p r r i liicntal intervcntio~is. C:ross-scctit~n~ll tlcsign~can bc irlra l for d c d p t i v e ar~alv.;i.: I f we sirnyIy want tcl ~ l t ~ s c r i l lthe t . c h ~ r a c t ~ r i ~i )tt t a c~ p0puldtf~1n. thtbir a t t i h~dcs,t h r i r v c ~ t i n grntcntlon or thtbrr bu! inK Fatterns thtxn Il~rcnlss.;cctional stir\ P I 15 n nlnGt satisfactcln i v ~ o vf obta~nln:: 11734 c f r ~ ~ r ~ p t i v c n ntc~rnlation.nut crt)<~-sechonal design* .Irr not restricted tcr di-.;cr~ptf\~e nnalyqis :I<~ 1 1 t~ 1 ;lrgued below, yrt3pt.r analysis that uses qtatl.;tical controls c n ~ b l c *crtwq-s~cticmaldnl,i 10 p r r r v ~ d cr.aliz;lblc itifc~i+rn;l tion J bout causal prt)crsscs and for testiiig cn t ~ s a l madrls. 'There arc., I i o ~ ' ~ va~number r, oi mrt h n d o l o ~ c a land p r q ~ ical c t iqsues ot w h i c h wp need tri hr aware when ~ i s i n x cmss-secticln~~l designs. An ;EztBarencss of thtvsr* 1s5uc.sshould help m i n i m i ~ e the ~ h o r t c o m i n p of this

Minr, ever, mn*t 01 t1ic.w thscat? Ihictt>r\, m , tcrr;ltron, ~ instrulni3nf d v c n ~ , inort;llilv ant1 tc*tini: ctloct.;) a r r w bi,ca~l<t, of 7 I ~ L >i l \ re not pr(~L~1~-111\ wvtli thc n ~ r r - t ~ n tvtt*rnc*nt ~~b ttt thcsc dt><ig~~+. srtlr5-scctioiial ilc.;i~n.;. 'l'hr main bhrt.,~r\ t i ) ~ h i r ~ t c r n avalidif\ l of cross~ r c t ~ o tdwigns ~al .ilcani frr>it~ hvo ~ourccq: ~ ~ r o b I ~in ~ restabli3liinji n< c.luse w i tlic~ut a tirnc. Jirn~~tision; and problem\ a t thc I c t ~o lf meaning (Mnrqh, 1482). Problen~s.lt thc Ievcf of cal~sc.arca, i l t cclune, an ~ s s ~ t~c\ ~ 1 1 any li ci~-.i~m b u t cnrc+st~c.~lcru;rl d ~ s i v .ire s p r ! i i ~ r l . i r l v prone to problrnis ~t thw level. !'roblrn,s a t tlic Icr.cl of mi,sntnq a r p also common t o ill! dc*i~msand ~ r i c t \ v ~ t fequal ~ furct. for t.\prrlni~-ntat. I o n g ~ t i ~ J ~ , n~ an l tl ~ r ~ ~ r ~ - s c cdt--i):ti\ ~~onnl h.l;lr~h(1 L)F;2) ]\.I\ 1s.l-~tfcn clearl! anrl fort-t*f~rl I\ about thcsc trt.t~13rrlll1i.m.; Shc takcs I.;\ilr> r v i t l ~those who t i in i . ; . ; cross-s~ctit~nal c i r~ vry J ~ r i g n sh e c a ~ l w of thcii. ,~llcgcdprobl(~rns will1 causality and r n t w n i n ~ . Shc argucq that Iliic d t ~ q i j i nis not nt.arly a + flawcd as i t i s fr~~c1~scwtlv purtray~d to bt, Thc f r ~ l l ~ b w i n cliscuwion g Jr,rt%.son Mdrsh'\ ~ r p ~ r n t ~ n t .
+tL~ti<tlc~1 rV<rLv.\rtln,

'fl~crc i s nt?den\ i

r l th,~! ~ irn5s-.;ection.11 51rltlic. I , ~ c c prnbIcm\ ~rl~aritilvin,: cn~~.;nl vari~l,l(-+.F\,t.n I l i o ~ ~ g hl vio t ~ r n t ~ - c ~ l . d t\r,i~~iai)lrs >~ivI m i ~ l i1t ~ crrrrlh. I,itcd cir gruz~ps nligl11~ l i f f t lon r an o u tcnnitb v~ri;lbIcwc cnnnc~t Ijr L ;.I rc that thcsc dil!crencc\ art- r l ~ l c t c ~a mri,si~llilrk bi,lwc.c,n ~ h vari~blt.5 r (RlCilurrk, 19h-l).In CI.r,~ptt~r 14) 1 ~ntrod~~c an c cc l r a r n p l r r ) f the effect af parc~nthr~oit .tnt~is (have nc-, children; plslpst child ~ 1 n t 1trnp t ~ v e x old and eliIcst chiltl bct.c\wn 7 and 7 r.r>r.: tdd) and marital h,>ppint*s<. Fcen i f tht*thrrr*gmup.; 11 i ttered in tcrrnh t ~ nzarltal t happinex\ rvv i ~ o u l d harrc difi~cultv ir~c>rk~ng out whether t h i ~ dil trrtsncc M'RS CIUC' t v t l ~ t i cliift3r~nt r parental +iaf11.;. 'l'hc 1I1i-PP p o u p ~ w ' r i ,likoly tn differ i n w,iy< tvlhcl. 117i117 just thc~r part~nthnod htat~~ -s 1 1 0 ~ I ~ ~ i ti h g y \lave been tc?gcthrlr, their age profllt., Irbvcl uf wtrrkfnrce partrciyntiiin and so forth. 71rf,<t, ~iifferenccs,rathcr than parcr~thood shtli\, c r l i ~ l d account for diffc>rt.na*hin marital htippintlsa.

clr5ien.
In c m ~ . ; - ~ ~ ~ c t ~ r r n n l o~rr~lir.;~.; ~ t i rathrar r ~ ~ tli,~n at tlit. data c v l l r c t ~ o n rt,Ij:tb as it i s in cwpc~rinic~r~l,il r l ~ ~ s i 'Thc ~ ~ s .inlirllt>n . 15 nchieved lv m a h i n ~ t l ~ rgroups a\ \itni!ar 4% pmstble by ~ t , ~ f r ~ t r r - , 1.crno~ing ~iE~/ ~liffrrt.iicr~s I; . hctwccn j i r o ~ ~ p riflr4r s rfirln IIII;'~'bt,rvl cmthl,'r'h'r/W hcn jvc cmnpart- ):rtvu ? MT need t n hnou, Iiirw l t i l l ( . I ~~lifferrncc 41 II~J h r to ~ Y tlfl/r~ I I I I ~ I CI*> i ' ~ / / y n t v d to remnvc n1.Tnv nt' rrr!rurl. 5inct. rrthtsr I h r n ~ ,trial. nut bc cqual tf~r ditfer~nccta \ pr>\-r27ly ~ n cnrnp;rrt7 d !]kt- rv1tF1 like. I n outllnc c ~ fhrviv I.; dnnc \ r - t l l 1 ~ .tli+ru<wd in Chapter 11. I lt~iz,cvcr,tlic h;lkic IticEc can bc 1ntrnJ11~t.tE now For the c l i i l d r t ~ n.~nd r n a r ~ t nli;lppirat.;s l r u a r n l ~ l \uc t~ ~rould cr~n~l,.irr~ tllr marital h.1ppinr.t.; of
r r f crvifr>~~nrlin ~ g. a r i ~ > t lir; l ~ t,lcklcd \
rlrrlir

Thi, prnbtrm dc*kigns a t thy

Wc encnr1ntt.r p r r ~ t ~ l t ~ with ms intcrn,~l \>,ll~dih.rvhcn llit, I O ~ I L - nncl ;friiitlrrt' OF t111. ~irv5ti:n docs n o t enatllv 11s tc? clitjc~sc unarnbigunu4v unp c'rpl.inatlon oi r w r rcsitlts (31 er anctt h r a r t.\planation. C'arnphr~TI and C;t,lnlcr- tlLhr7l) h.n tx ~dcntlfied A n u m b v r c v t t;lctors that threatr-n intcsna! ~ , . ~ l ~ r l dnd ~ t ! - thr<ta Im\.c b w n dt>cusst~l I~Idt%!;liE in Chaytcrs 7 and S in
relntron tcr I o n c r t u l l ~ t ~ and ~ i l experinrcnt~ilrli-.ignq.

tlirtbt> t ; r o u p . 1 1 i c 1 ~ 1\vitliot~t ~ *In{ ~ - t ~ i I d r ~ tli t~ ~c i t, -\Y!I~-vtbldtv\tc11iTd [ \ 11ntlt'r tlnp !par ~ t .lpl t n n ~ Ilio\tb l r\,htr.;v t<lclt.\l c h ~ l d1.. bcluvcrn 1 a n d 5 \.i..trs ( 1 1 1 . 1 . %rrnvl\nc r n r ~ h %a\. t th,\t ,>n\ rliibt.rt~nc~b~ rn ~n;lrlfdlIia~j~jnc+. 3r(' nt\t ~ L I L .to c l i ~ l t l r c Iwt ~ ~ r i t i r to t h c d ~ i f c r ~ ngc n t proiilt..; 43f tlit. thrr-t* gn'~ip\ pm't~ll'.. I h r ~.lrgllr , h ~ i flrt~sc t rt'l~u ha\ r trtal'n 1larcnt.s Iongar art,, r?n arcr;lgc, oldcar than !hr~.;c w ~ t h u u tch11clrt.n Thcby ~ i r q u t h ~, ~ tit I \ tht, aytb ot partners, not thc prcqrnce of c h i l d r ~ ~ n ~,v l i i r ~ infl~rt.nutl< h rn,~ r t t l~happintacs. Il'e idn It,<t thr+ pos.;il~il~l\hv r c m c > v i n ~ the, actv J It trrtncc..: from ottr cornptiri5t>ns. Wc c.ln make a nurnhcr rlt nbsfrictc~lcninparlconc. Firhi wcm 171 1<11tt t v b l . c \ t p(-t>p11+ L ~ ~ l>t>t\vet*n c ~ ~ l 20 *ln{T2 1 , i r i k I \ t y S \ v i ~ ~l1far , t ;(sft!~~)r !hi> p a r l ~ c u l ~ii:~. ~ ~ r grrb11l-rlhtb mnrital h.ipprntn\% ( > + Ihc thrt*t, Errwry, d i f f t . r ~ ~ l l ' h a j t IS, !vt3 rc,slrlct o u r cumparlsclri arruqc the t h r t ~ t ~ - ~ I \ p3 L I to f l ~ t ? \Z ~~ h ,o art, !nrit-11 Ihv 5.1rnc J ~ L ' .1 4 ' ~ 'C(TIII J r r \ p ~ ~ dtliic. t ch3rnparrcon +cparattl\ within caich ag<*h+rcli~p (1, E. 25-2'). 311-11, 74-W). 5tlppc)sc w e l ~ n d tha~, ;rvll!~?r t*.~c.h Srl-rttpinK, thtyrP.lrc strll dilft'rc~ncr\In rnarilal 1 ~ a p p ~ n c r ; s hctkvcrti Ihr- SrtrupG ( I e bt,tzvt~t.ntlic clitldlt-w, I he nt,zt. parl>ntsGindtIic>st, with (llilcl Eigctl 1 10 I; ~cdr.;)dr+pitt, ~lirnrrc o ~ n p a r a l ~agt3 l c ~ prt,irlu.;. M'r (-an thcn *it It.;l~t$ b ~ thn i t thiq rtal;lt~~>nchir Ix~t\vt~t*r~ tnar~t,il h , ~ ~ p t ~;lnCj css r d r ~ ' r ~ t h o t )+!a1115 d i- Itri! tlllr' t v the ~f t l ttbrcnt JCI. prtrt ~ l r + c l t l h t ~thrrt, ~ r ( ~ k 1Tlit, p . IOZI( nt fhiq i \ ~ i n i i l , ~ to r fht> it~i:i~- o t i ? r i ~ t , - ! t ~!:~L>LII?, ~tc tllal I < ~ o n l t s t i ~~l\c'rf vt~~ in c-\p*rrrntan tal d r s j ~ n s . I-hrbnlclrcb~,.rrialllc. Ihat art1 ~.c>t~lrr>llt-~l for <t.~li~tir.all\, lhc rnrlr.taw t bt-an I,e conirelcnf thnlf t h r * fiii,>l rr,lati~>nslirp wr l~r~c I,ctr-,.~.cn l N arid li i h r ~ fwtinq l a t h , 7 I r~*i~jh(m,h 111 b t l t n . ~ Y ~c ~ ~~ n\'. ~ ~l TIM-prohlr~ln will1 till\, htlwtwtnr, ir that wit c.ln rinlr~ ct)~itrcll for rari~blc< !hat 1 ~ 1 I~a\,r. % thottp,lit oi , ~ n d ,~l,o~it 1\,111ihnvt. li,~\.r ~nforrnat~un. Rnntlrrm aFl(>r,7tion in ,In c~\pt~rirnt~rltal de.;tgn r4ft.c titmclv conkrr3lq Ft~rall 1 nr1,117Tc\ - Lnnivn ancl ~ ~ n k n n w\l;ht>n n wt. c t ~ l i t r t ~ fob. ! vari,~L>l<>.; st,it~~ticallr at tht, dst,i an,1!?.<1\ qt.1gp t r y t a n ncvrr br, .;tlr~% t l l j t \tmc11~1\-p c-cmtrtlllvrl f ( v all rclc\.;lnt 1~ari.ll7lcs 'rhcrc nq alwav.; th(* p(>wibilit\r thal ,In> rtb!;ltitlndirp Lrrc h ~ v fnunll r brh\,rt1nS nnll k ' cc>liI~1 be d ~ ~ c L sr7inc I zrncnnkr,! lud t ariablt*. But t l r i ~ is ilot to ctmtlenln crn~s-+-t~ctic~t~aI i i c > ~ c n$1.; s bt~tng of n o u+t. in cb\~lc~rinq c L ~ ~ ~rrala~lt~nd1ipr. \aF In~lc~d .;inc.t* , ira sosi;ll +cier~sc. rrse.in-h t l i ~ c.1pat.1 r ty ttb ~ r ~ l t ~ n .t3iIl1er tvc pr~ci~~~ t>r il t~t111c~ITv lv <i~i % ~v~ i ~ianip~!I~~te wI~ti111on~ ,I< r q ~ i ~ r t ' lby i 1~xpt~t.imtvlS111 t l ~ < ~ g n~cc ..11111ltt.1i Ilit~rei h c~Xlc.n t t - ~ vii , C T T T ~, a l ~ c r ~ ~ , i ttt3 i ~ <.rm--\tb~ t*\ lit>~i,i! r{t>\iqri+ RJ t l i ~ r I han rejty-[in\: crtl--vv~:l lon,i I tltb<iqti+ nc heqn!: t i - t - T c b k + for ~-aut,ll an,iF\.<r+ 1% c n i ~ ~yrvctvd sl -\ qtc.~n,>trc,illt, .~ti~ ~-arcfuII\~ i ! \ 1 1 1 1 tlw , i n . ~<lI ~ \, , ~ n dsoilti-!>! ctati\ticnll~ ttjr t,>~ttirk t?i.it mrchf yl.lu.;~lrl\- t , \ ~ l , i r l l the rtlrrr~T,itic?i~s n r r , ol~srr\.c~. Fir r t l i r ~ r n t r etvc iii~lslc n r l ~ t , ~ n t lrrrnind v ollr~ 4 \ u r<~ !lit, t tmtnti\ c naturt. 01 #in!. rt>nrliih~tw? i t tm dr.11~ In ;Ins s r ~ t ~ n t i t ~ c t~ll~lt~ L)llr .l\ A l t h ~ ~ i ~Findin): gli th,l 1 t ~ v,i w riahlt?q ,I rc ~ r l r r t d ~ ~ tdut>+ r r l ntyt ez!,ihl i\h carlbt, I! t1tw.; nltun that ,i i . l t l s ~ 1 ~ ~ I ~ ~ ~ \ R i J \ ~~I~Ir w > R .~E~ \b l c fTtlrkV~

Model

1 school

Chrlrl's acaOPmlc ach~euem~nt

Parental value placed on ~ducat~on

ach~eveinent

Model 3

Parenlal
rP.sources

Ch~ld's academ~c
achievement

,~r!:t~v<, 'I~nrling,I c c ~ ~ ~ r c l , ~ b ! irot n i v c ~ rkrntlk~nq ~ ,inrE lung c.ancc>r I11 nil w-,ay FrI%vPs that smoking cause3 tung sanccr, 11~1 i f ~ : r l r . ; mran !hat thc l i v ~ n t h t camnot - ~ ~ ~ btl rul1.d n u t ' ( l W 1 721

Crt~v.-<rt-tio~r;II cis til can h ~ l p ~b~f,lbTi<I~ t t t t~C.FLIS~I r e l , t~ ron5hip.: clo v n t cuiqt. L%'hrle i! is truv that "I c ~ ) r r ~ l ~ hctrvci~n ~ t ~ o i tlvn ~ ~~;lriahlc< doe.: not ~~.;tal~lisli caurntion it is a prc-rr>qlrisit<, for r < t a b l i ~ l , ~ ,In cat1\a3 ~ rt*la!tons h i p If thcrv i s nn rrrrrt4,ltitln rva* cart br, cont~dc+nt th~t thcrt. i . : ~ r r r t; I cxrqa1 rcl,lti~>nshi(> I I I r r ~ l r r ~ r i t r r rv.lrinb!c+ y a.; cauqcs can ht,clf jiist ar much +cicnt~f~ importance c as I(~callng C L I U SC ~t ~% ~r.t . ~ ~ nthi* ly trrl~,~cc manuc t,lctrircrs ut+ru~fd conqidcr n fintlinr: of ) r r l ctrrtrl~tit\nbrt\\.tr~n.;lnokin< and lung cdnccr lo Lli>an imporla11 f i ~ r d i n ~
h'f0111 IS:
A PKIOItf R L A s O M l V G A K I ? 4.0 110(' R E A + V V l N G

Thtl moqt ~ ~ s c f tuvl , i ~ti3 $(I al,nrrt tht- cau.;.~l nn,ilv.;i< of crtws-3r,ctirmnl d.1tcl i s tal d w t v ninelt.ls, o r tlo~r-rhart, c>r -path dicigrLin~i. of tlw tvpe elluhtr.~t~,tl in I rqrrta I 1 1 & ~ l r ir%l,ctt l hcrtl In tt71.; Elr~uk 'rhc t a 4 o! !I>cd8?ta .in<ilvct it thrn lo cvaluattb l i n ~ \ n,cll ' t h a w t~iodr~lq Cif tl-11. Jnlti. Tht. Itc.rn1 4 tfl~.clrir+ ,ind nlod~bls2 % a ):uidc. t c ~ ,inalt.~i\i + ,it1 ~ m l ~ r , r t a n t t+>,rl tt>r t 4 i t g Ftar\oii at1.11y5it1t: crt>c+.;e<l~otr,lId a t ~ '.I t ~s tiit, rntwlr.1 !halt ~tnncfs bclwctan t l ~ ri\+r>archer r ,~trili~nhrtdlrtltlnp~r~'ir;inin the n t t c . m ~ ~ t t c dr,!~v ~ c-,?u<.il~ n t t > r e n r t y r\larsli ' 1Q42: 7 2 ) It i s imprlrt.~nIt ~ ~ l ! t ~ ~ . tthe ~lop n1tfdt4 /rr,'orra ~ ~ \ i n crtr\.-crctic>n,~l q 1.1.1td ti, ds,~t.r* C ~ L I L corlclusior?.;. . ~ ~ rht,rc I . ; a dif t ~ r ~ ~be!rvcr.n r ~ c ~ .n J V rorr reahonrilg a n d rrrl Irut r c a + n n i r ~ ~ . ,4 r)rprt rt*n%onrnc in\.nlrcs yroptr.;im:, r ~ n the twsi.: nf thc.c~r~tik.al

ISSUES 1?J CI<CXSSECTIOV AI IIFSI(;N

181

cunsirlcration~ and prcvious rcscarch, that X will cause Y and will thcrcforc be cnrselatccf. This reasoning is ~ndependcnt of the data we may have. Ari hnc reasoning, nn the other hand, is the process whereby, nn the basis of an observed correlahvn in nur data, urc make up a causal story as to why the correlation exists. Thc psoblcm wilh nrl Iloc redsoning is that we can always make LIP a story r e p r d less of the correlation that we find. 1t iq not hard to t h ~ n k up a qtory that is consiste~it wilh a correlr~tioll - it is quit^' a different matter to anticipate, on h e basis of rcasnning and theory, that a correlation should xis st and then find that it does. I n silmmary, cross-sectinnal data can be quite effective in dcmnnstratirlg where a cai~sal relaticlnsliip probably does not exist. F~lrthermosc,it can hclp evaluatc which, of various competing theoretically derived u priuri models fit the data best. Finally, wc can use cross-sedional data to evaluatc and modify a given R prrori model.
Prc)perlv d ~ s i p c d,~nticiyaling , rivdl causal explanations a s far as powiblc and building in wavs to test t h ~ m [crc>rs-sect~onal] , surxbt.ys can provide evidence for arid against different causal ~nodels.The t.\.rdence i s not prouf, huwever; it I S only as goud <I\ L 1 1 c mnclr~lis. ( t I c ~ r s 1982: l ~ , 117)

Evcn in thcrsc c~rcum~trlncrs in wliicl~ we cannot develop a new study to establish causal direction wu.c can shcd stmw I ~ g h lon the rlircctic~~~ by mllcchng rrtrnspectivu infc>rniation and 23v asking people dirrctly about any effect childrcn may have had on their marital l~appiness. This approach does not yield unambiguous data hut, bit by bit, i f builds a picture regarding which causal direction is the more likely. Establishing causal dirrctinrl, however, is not always pmblcmatic. 5)mc independent variablcs arc fixed (r.g. ge~ider, race), others are not r ~ a d i I v subject to our rnan~pulation( e . age, ~ religinus denomination) and for others the time nrdcrings are straightforward (e.g parent's education prrccdcs rcspimdent's educn tion, cduca tinu l r v ~ l ubually prececlrs the tvpe of job we gct).

By establishing causat explanations on thc basis of cilrrelations, crosssectional designs a r c nftcn accused elf treating human action as dcicrmined by cxtcrnal forces and of neglecting thc rule crf huinan agency. , Marsh puts this vicw most cIedrly:
The aim of c x p l n n n t i o ~ is ~not just lo show high cr,rrelatir)ns between v ~ r i a b l ~ s ; sociological cxplnm tirjns IYLIII~L~ i1Imo5t i ~ l t v a y turn ~ (wt to be r~ flop R I I V W 011 ~ ~ thrse grounds. It must also show how the actions of the peuplr involved were the actirms of ctlnsciuus hurndt~ hcings, rcwting Lo a n environment, trying tr, makc sensc c-rf it and pursuing vasiuus goals anrl actions with morc or lcss success. Only cxplanatinns thal take cognizance of the meaningful aspect of social action will satisfy us as human observers. (IqH2: Y 8 j

Another prnblern with cros<-sectional data is that Even i f we estabIish that two variablcs are correlated this does not establish their causal (Iircctio?~. T n order to illustrate this point let us use the parental status and marital happiness example again. Suppose (hat the cross-sectional study showecl thd t the chiIdless couplcs arc less liapyy than those with babies and that corrples with the oldfr cIiildren are t l ~ c happiest. The problem is that evcn i f we assume that this 14eflectsa cai15aI relationship between the two varial-4es we know nothing about the direction of the relationship. What comcs first - childrcn or marital happiness? Do children increase marital happiness nr are happjlv marricd people more likely than the unhappily married to have childrcn? A panel survey could resoIve this questiun. We would know the level of marital happiness before children and again afterwards and scc if thcre was any clerrig~. We could also check whether thc initial level nf happiness was linked wrth the likeliliond of subsequentiy having children. However, t h c r ~ is nu simple way of doing [hi? when all data a i e collected n t and aliorlt the one point (if time Onr way of dppronching this dilemma i s t c3 dcvelnp a priuri rnodcls ta tcst. WhiIc support for a n 17 priuri mociel docs not prove causal direction it a t least prmidcs a kheorcticnl basis for arguing a case and provides empirical data that a t least arc consiqtent with this. Hairing found data consistent with thc thesis o f rr partic~tlar causal directipn the next step would hc to adopt a dcsign that can establrsh for causal direction.

Herbert Blumer (19Fih) c o i n ~ d the term 'variable analysis' to describe the type of analysis and understanding providcd by cross-sectic~nalsurvey designs {hewould also apply ii to longititdinal and experimental designs of the type already c o n s i d r r ~ d )Me . arglles that ai~alysis using discrete, quantifiable variables is inadcquntc for at least thrct rcascms. First,
It l e a w s
oul

internchon In which h u m a n 11fehas

the actual compluxiher of acllvity , ~ n dthe actual prorrsscs ilq bring. (195b: 101)

05

!%cond, i t ignores the actor's ii1tcrprct;ltion of khr r ~ b u tion a and b ~ h n v lour and tlic way in which thcse intcrprctatinns impact on their actions:
The i~-tdepc.ndcnt vnriablr ih put , ~ tt l ~ rhrginni118 of the prucew uf tntcbrprctdhun anti the dependent varlahlc. at the tcwn~lialpart of thr process. Tilt, inten~eningprocess is ignored . . . as sumethii~g that need not b r considcrcd. (lq56. 47)

'

By ignoring the i n t e r \ , m i n g process one cannnt ~ u ~ d c r s t a ntuhy d the independent and dependent variables are or arc not related. Where

I H2

CROSS-SBCTIONAL DESIGNS

ISSUES E N CROSS-SECTIONAL DESIGN

183

ht~rnaii hehaviour is intentional and based on l~ndcrstand~njis of a situation thcn thesc. intentions 2nd uiider~kandingsqt~ouldbe part of the explanation of thc beliaviour, L'lumer argues that variable analysis usually fails to do ihis. Third, Blurner argues that variable analysis w wnches people and thcir b~haviour ouk cjf their contexts. By trying to work with discrete, quantifiable variables onc loscs touch with the 'whole'. By dealing nnlv with artjh'clalr); isolated hits of behaviour the full meaning ot the behaviour will be misscd. By looking a1 particular aspects of proplc's bcliefs and actions without looking a t the context in which thpy take place i t is pmhLiblethat we will niisundcrstnnd lhc meaning of the RcI~aviour. In the cnd the qucslion should b e n o t whether I l ~ c meaningful and intentional dimension of behaviour should constittlte part c ~ fsncial explanations, but whcthcr this aspect of bel~aviour can L 7 c captured using the data coll~ctinn methods usually associa tcd w it11 cross-scctir>nnl designs. Thcrc Ilavc been a variety of ways in which thosc uskng cross-sectional survcys I ~ a v gone e about {'roviding 'lncnnit~gful* rxplnnations. As Marsh observes, thcse
apprn~rllcs differ in whnm t h r y bclirvr thc n>mtjlrig ~ h u u l d be s~rpplredby ant! how rni~ch and what tvpc. of e\,idencc is required before nnr can r~alidly nttribuir mcaning to action (1982: 1111)

She identifies a number of approaches. The first, and least satisfactory, is that which she calls 'facesheet sociology'. Tliis consists of correlating a set of sncio-demographic characteristics (c.g. sex, class, ngc, rcligion and ethnicitv) with the phenorncnnn one wants to explain. Tlie more one can dctect cori.elations the more we have 'explained' variancc in that phenomenon. However-, this approach doe? not provide sociologicalIy meaningful explanations. 'To find that gender is corruIated with rcliginusness is not to u n d e r s t a ~ ~ why d gender is linked to religiousness. We netad to ~znderstalid what it is about gpnder and religion that explains why women are more religious than men. Typically, as Marsh points out, this approach to supplying mean~ng 1s brought from outside the data whcrc, in an nrl lint way, t h ~ rcscnrcher ' si~ppIies the explanation horn thcir stoc-k of plausible sociolngical idpas. While these explanations ma!; be consistent wiih the data they arc rarelv ct~mpelling since many 'cxplanrltiorls' that arc equally consistcnt with the data can bc dc.vt.loped. A second a p p r o a c h i5 to ask indi\,idu;lls tn s u p p l ~the , meaning. We c-a11 ask them w h y they acted a r they did. Tliis approach is criticized h!, thnse who argu13 that uftcn people d o not know why they acted a s they did. Thc response to thiq objection 1s that if people d o not know why thcy acted a s thry did then these 1s not a pmblern sincc wr cdnnot s a v that that bchavtour is in any sense mcaningiul A s e y n d objtction 1.0 asking people tn give their own explanations is that t h ~ y misunderstand

thetr behavio~ir.Experiniental ~ v t d e n c cshows h a t in certain euperimental situatiims we can drmc~nstrate en~piricallywhat causal factor led to given behnvlour but thc indi\liduals themselves are unaware of what this factor is. This objection has same merit for somc behaviour in some situations but it does not mean that individuals, in many situat~ons, cannot supply at lenqt some reasons w h y they have behaved as they did. To say that they do not have access to the full range of factors that affected their actions is not to say that they have 110 understanding rrf their bchaviour. If we were to ask a mother with young children why she wanted to work part time she woulci no doubt be able to identify a number of thc rp;lson3 behind her decision. That she might not be aware of or he n b l ~ tn artrculatc all the intluenccs is not to invalidate the irnpnrtancc of those she has identified. A third objection is that individuals may indeed bc aware of the reasons frjr their acticms but, because the reasons may not he all that nc~hle,social desirability factors wilT prevent them fmm reporting these reasons. Whilc these objection5 havc some merit, it can still be useful to ask people for the masons for their behaviour. Marsh argues that 'actors . . . do have privileged access L o their own experience but not to all the drtcrminanks of their o w n brhavinur' (1982 107). Hcrwever, M a r s h says, w e shnuld nr)t cxagscratc what we will discover and we must b e a t the explanations provided by people thcn-rselves with circumspection. A final approach to includi~lg a meaningful dirnmsiun to explanatinns IS for khr rcsearcher to supply the meanings (1982: 111). Thcrc will be many situations where we simply cannot reliably ask individuals for their reasons. Often we can only identify reasom by Icmkinp, at a number of peopEe in similar stti~ations and reading off commonalties b e t w ~ e n the cases. The indtvidual actor will normally not havc acccss to the additional insights available to thc researcher who can ohserve from a range of cases. Burkheim's (1970) study of suicide provides such an cxamplt.. R v studying thc statistical regularities he identified the social groups that were most 'suigenic' - the groups with the Iiighcst suicide rates. By asking what these suigcnic groups had in common he sought to provide the reasnns for suicide. He did so by proposing that in thew groups people were most likely to experience anomie - the feeling of normlessncss and a sense of not belonging. He argues that thrs subjective feeling is distresqing to the extent that penplc commit suicide. Durkheim suppIics this rncanlng mthcr than i t corning from the data. rt is Durkheim rvho rirylrrs r a t h e r than (~tnylirir.rlll!td ~ m a ~ l s t ~ nthat f ~ , qthc suigenic groups suffer more anomie. However, he ncvcrtheluss attempt5 to lnakc sense of the empirical correlatio~~s b ! supplying a sublectively meaninghi1 a c c n ~ me tf t I ~ c l~nks bthvcen rnernhership nf certain social categories and s~~icide. Marsh (1982) discusses a t lungth a study of the social basis of depression by Bmwn and Harris ( 1 978) where i k was bclicved that those

ISSUES IN C'KCX-SECTltJYrZ1. IlESIGN

165

ctlltcrlng From ~lcyrt~c..iun r \ r l [ not bc full\ ,Ir\,artAo! all tht- tnctors i l > n t r , b u t ! nto ~ lhr-rr rlt:prrsqic>r~. I;rn\tTn a ~ i ~ I4;lrr1% ! adopt an ,ipproa<.h nlrf ~znllkcthat uttl~;.c.d by Uurkhr*ini in his shtdv (Ti suicide - but w ~ t h ant* diffcrcncc. I'licv cuIlwtvci ~nft~rrndhon abo~~ what t tlrr!! thought W O L I I L be ~ ~ h ~ c n c i a lp r e ~ i p i t ~ ~ t ifactors nq behind dt-pression ( l i k tvcnts rvhlch acted 3.; p m ~ o k i n g aplnlc such it< famtlv problems; poor )~t>~tsln ep rn ; p l ~ i ~ mtc strcswrq; n .;cvcrih; of qtrr*ssom; access to quppnrt t ~ ~ t w o r k cThcv ;) tnfrrrrna tion about I he\(. tac tors becarl5r t liry tlio~#ght rt would Fwlp make qcnsr, a t a sublectivc* level, o f why some piwplr were drprcqccd and why othrrs uterc not. I3,v planning ahc.ad and ,~r?ticipating thr pr>s.;ihle rcawnc that might lic brhind depreqsinn thrv trvrc able to dt?\clup a n accn~rnlof drpre3sion t l i ~ tried t SO h u ~ l d in the t,upt>rit.nces uf !ti[, .ictcws. It is not n f~3olproofw a v c ~ f tapping thr .;t~b~rcti\.c.l!+ m c a n i n ~ f ua l y x c t of brhaiuiour but I! dr*n~un~tr;ltes a n attrmpt 111 n cross-s~ctron~il surLbevdcqign tcl Include hi:: form of explanation. I r i summarv, tlliarc i s nnthil~):vnlit-rcnt in cross-st-ctitlnal designs that prc=cludes t r y ~ n gto a c c ~ s s the r n r ~ n ~ i i f idirnenh~or~s ~f~~l of human brliavlour and b u ~ l d ~ n thesr g into t ~ r p l a n a l ~ o nIt s . rnaktx:, qmsc to emplov 3 r,irigc i l f stratqqcv5 .;trice no stnclt. approach i s w ~ ~ h c n~ i tt q sliortcotnrnq~.

d a l th,it ~ .Irka rcflccti\

i3 nf thi. p o p t ~ l a t i ~rm t i t h r rvt3rc. c f c ~ ~ to ~ rcprewnt. n ~ ~ ! For thi\ rcmnn c r c w ~ - ~ c r l ~ \tnrv~ys r ~ ~ l ~ lim'1, ~ l !?C*CC)~C t l i t x d i v p ot ~ ch~~it* For h t l ~ d i ~zvli~re "i acctlr,llcn ricscripticln is r c q ~ ~ i r r(r d K. pol^ tical pnllq, househnlil taxpend~turc . i ~ ~ t v r y xWliilv ). csir\r-srctio~ral rlra51~ms Iiavc potential wt*aknesws in unambigunu4 y irlc.ritiiying cari.;tl+ (a matter of internal \'alirli?) the!. src* qtrilng at descr~ptiun.

Practical issues
Many c ~ thtl f practical i.;\u~-\ Ji.;cusscd in thi.; srctinn a r r no1 prculiar trr cro5.;-sc~-lirjnal dcr;igns I w t a r e impnrt.~nt is5u.c~ for ; 1 2 ! research rcgard lccq of the d e l ~ n .

Rr0presentatir.c 5arnples a r e nccr.;.iarY i f \we are trl ~ c n c r a l i z e from

I+~*'('sLI~~s

oI*tained m a samplc to the \\-tdc*r population that the sarnpEc ik mmnt tu rppresent. Tlir surc.st way o f achithv~ng rcprc.;cnta tivc qarnplt-s 1.1 to

crnplov probabilit!! hampling metl-rods. Thew arc m ~ t h o d s w)icrt~ rach

p1r5on in t I ~ e pclpi~ lntiun to which wc want to ~ ~ n r r a l ihas r e an cquaF o r kt~rlr++n chancrc o f bring selrctcd in thc sample. Cross-sectional r'lrbiygs can liarP more success th.ln okher drrqiqns in act~irr~in tepwwn g ta tlveness. !:xperiments encounter problcrn.; with repr~.;c~nltati\~eness for two main reasons. 'l'hcy often aqk rntrrc of people than do one-off crosssrrtional studicr. 'I hcv also invnlvc. active inkrvcn tions and tlietcfnrc h.lvc trj rely on r,ctlunt~ers m d av;lilnbility samplcq. l'hev conqcqucntly lack rcpresentativtnt=,\ Even ivl~c~rc rcpresentat~vr qnrnples arc rJit.~~ned i n ~ t ~ a l lthis i c;fn hv lost a< F C ' O ~ I P dr~~ cjut p vvtxr t h t cmrrw ot the tn*ytnr~mrnt. . I \ similar yrrlhlrm nf d r n p r ~ t ~ c,in t cornprc~mi*r,panel shldics. T h r morra wavcs a patlt,l s t ~ ~11as ~ l t11.t. y \:t.r,aitcr the c ~ r r n t ~ l n tdropout. i\#~ 'T'rr thc~ t*\lrnt that thrc drop17~1 t lnirnd ilcc~s~tiipnrtan t sarnlllr binses, reprcqrlntativt-nt.ss can br prr~gr~r>tvcIr. ~lndcrrn~nrd. q r t ~ ~ .c t ,r c l q ~ - s w t ~ ~ Jcsign.; ~ r ~ a l t l r l ntlt havc R timty dtrnrmqjon the!' a v c ~ d tlrr, problem nf I,ia.; bt~ung ~ n t r n d t ~ c rbv c l 5arnpltbatlrjtion. So long a4 t h y I in! ll.1 E .;ample is w d l rclectcd thc. cross-sectlr~natsurvcv qhnr~lrl yirFd

Thcrc ic nc~tliing about tlit* It~grc of cm~+srartic~nal desicn.: that r q u i r e s J partir~~F,lr method of coll~.r-!in): the d , ~ t a I'hc c.;scnt~al thin,, in crcvskbcctltrnal <lra,irni, i< t o rvlli,i~n ,I \truc:ur~*<j +(*t 41t ~ l a t , i t I r , ~ l tztiab!e 5v\tematic ct~nlpirison5 bctrz4t-~,n cases, nr )I;rt~iipk ut cnr;c.;. Tlic m~n,~lv.;i+ of cri>s.~-.;t*t?t iorlar data rctltllrra.; data fmm m i~ltiplc cssrs us in^ I lit1 s.lIne variahli..; Anrhvsis rcl1t.s on cornp;lrlrlc c a w s and accoi~nt~~ tor l y :varlntjnli hr.twri~r1cases o n onc variablv in trrrns of variation on anrlthcr variat-It.. For r ~ a m p l t\ri, ~ , t v ~ l loh~ervt,th.~t there is vanation In terms of inct>mc: wrne penplr r a m morc tI~nnothers. Wc wilI tlicn srr i f these variaticriis correspr>nd with \.ariation< i t 1 qornethin~ else: for ru,?inplc,d o prnplv w i t h highrr inr-r>me also tend to h ; l l ~ higher educatrr111 Ihan those wrlh low incnrne? Dtl tljc high incnmt- 1.a rncrs tend tn be malc ~ v h i l c the Ioru incon~taearncr.; tr~nrlto be fVmEllr7 Tl-11.; .;or1 n k cntwlation.il nnnlt~qisspy~tirt- what is wmcffmcs called a ~+ari;lblc tw c.ire rnatrlr - a ~ r l d in ~\.hich wtl have the sanlt, ~nftvnlation tor each caL;t>. In the examplc in Figier~3 1.2, cnse 1 13 a 44 year old, middlc class male with ari annual invomr cri $15,0011. I lc. votcxs Conscrvativc and ~JRrccswith attil.~tdcA but h a s n o opiriion on nttit~rdc B. The d a t a 111 a varial7ltl bv cnse matrix can he collvctcJ i~.;ing anv rnethnd: t~ura+t~onnaircs, f~cc-to-face i n t t ~ r v ~ r w (>175~'r~,lf1o11<, s, L~~c'hiv~l rccnrdc, itblc*rt~onp intr3rt Icrl.; am4 <%ti r h c r c 15 no ri%.1+nn \.r.hv a rniut~zrrrl! ~ n t ~ t l i o d canno( s S r t =uqed tcr cotiipli*tr the m n t r i ~ 11 15, ncvcrthclcs.;, import,~nt thal tht, s,lrncX mix c l C n i t l t h ( ~be ~/~ 11ki.rl lor mcfi c a w . FRTCYJIIIIIIC~ WC' sh~tl!d c ~ v ~ ci>II~rl!ti~ i~l in tclrnlnlion on inc+r>mlrfroln some pt,npl~% sing ~n < ~ t ~ o n v r n osrlt-~=ctmplctinn us qtlr<ticlnn,iiru whilt. rdyrn~ o n ,I f.7ce-tcrfac~rntclrr!crz. for r>thr.r p~*c>ple, and rPlv!nr:on p r sonncl rrrcltd. for nthcr%, nr t,rxation rcturn.: Ftlr a hrthc.r si%t r j t pcoplc Sincc thr rnt-thnd b\* whit-11 data a r c c~dlt~ctt,ci can infl~rt-tlzc the rva\, peoylt, .In<wt3rcluestifrn5 M'tl a! Erast rnwt ran<urccnnqi.;tclnc-v.
'

Case

61I

I I Mate

Mldqle

S4SK

I
4 I

,
I

44

,
I

Agree

No

Consewalrve
I

I openion
56

Female Workrng

1 I

52%

. -

D~saqrne D~sagr~r Uon-

-c
I

A -

1
I

'

Conservnrtve

73ii~ sho~cr L I ~ anv ynrt1cul~1r rl~t,icnllr*ctio~rmtathtld tt.111 ci~,pc.nd on lnnnv f ~ c t r ~ r'1s hcsi, . ~ n c l ~ SIN ~ d c( ~ It hc %amplca,thc cautcnt to whtch they art8 i l u ~ t t ~ r~ t i i tll ~ i lclntn plncv nr , ~ t e widrlv .;c;ltEt~ril, bh(. timc ~ r t d rcs041rcr5 nvailablc, the Jcgrtbr of .;cnsitivitv of tlir rnal(arint lwinx colIestt.ii, ant1 thc c ~ m ~ p l e x 01 ~tv the m,~tcrinl. While ,I varirtv of methods can h.used to ct~llect itlformaticm to form a ~ t n i ~ t z l r cdatn ~ r l wt i t rern,~ins the r.l.;e that s t n l c t u r ~ d q~it~sticrnna~rt~\ art. ihr~ mcist widcllv ustd mt~lhotl.Stru~.t~irrd rlutn.;tionnnirc.; can be a ~ l m i n istcrrd in n number crf dltfercnt rvnvG, t hrm must ccmmon of wl>ith;irc3:(I 1 facr-tn-facc hv kai ncd ~ntcrv lewerr, (2) over the ti*l(&phr>ne with tr,iincd inttbr\.ier\.~~r~, (7) .;tllf-adn1ini5trrcclr by tht. rr<pun~lcmt ivllerta the q u e ti(v111air1,i\ nr~rmallvrt~ci.~\le~I 'qnd returned IIirt~~r):h thr>mail. Fach rncthud 13f ,>drnini5terrng qucqt~r~nnairpz l i , ~ *i t \ a d v w a t ~ * anii < dt'ii*;lct~rsan(+ thi, mt.tliod\ havtb Ihcir <trcncths a n d wcakr~cssc.: (Dillman, 79%; Ro~crs, 107h; Crorta., l't70; C;rovv.; a n ~ fKniin, 7'4:ct). In sclr~.iin a~ p a ~ - t ~ c u mclhod l,~r of nrltrlinistmticw vrltl n c c ~ lto hv aware of th~-+i. rt3lklti~ia ~trcn):thsanrt wt-akntkssrq and huw t h ~ v rntght applv to thc pasticul.il- slir\.r,v in which vrw arc1 t ~ n g + ~ ~'rl~c.;t% c c I . strt-ncthr and weakrtLw-p< c,ln btb p-~>upcrl into f ~ v c bto~d c..~teg~rriec. Tabl~ i I. 1 prtn.irltr5 ,I W I - , I ~ A V \ * tl\ a ~ l ~ ~ a of ! i (I>INV v~ P J C ~ (if flit, tlirc,c*r n p ~ l i t I~I l~ ~ ~d r i ~ t n i k t r ~ ~ L i ( ~ i r;1tta4cm ear11 nt thc critlaria. \ t fttllt>r~frccuwcini s ; l ~ ~ ~ l a b in l t 13illmnn (ItVS) , I T > Liltz ~ V c ~ (~ l~ ~~\~ J 1 1 ) .

Tlitl sample <we dcpenil< 017 fund.;. timi*,acitb.;s ttl pottant ~ a parhclyants, l pl.innt~cl method.; i l t aniilyii\, and Iht- clugrtv' nt p r e c i . ; ~ ~arid ~ n acr.urncy rrquirt*rl. 111 ~ c n p r a l ,rllc Earger thc qamylc the bPttt*r, Iluf br\.rmd 3 ctarttlin point ~ n c r c , ~ s i n thc c s;lrnplr silt, ha.; .;mnllcr and rnrjrc marginal

bt3nrfit~.

The Inr):rr thr, sanlplr v r r thc anore it can I-(. i;uhdEviilcd so thn! mcnnin~ful wbgrnuy cnrnparistir~scan br mad(,. If, 111 a gcner,~lsrirvev OF h r ? ~ ~ w h ( ~ we lds, uPantt>rlto comparta thv wellbeing nf ~hlldscwin couplc h t ~ i ~ s ~ h n~ ld 1s 1th that (1C chilrlrcn in 5olc p-trcnt Iiuuw*hold~, I\.rV wnuld 1lt.t.d t c ~ cniurc Ii1;lt thr urnplr 1% tn~lrll ~ c lai-!:c cntluqh !(t \,~~alrl \ ~ l Cistont f nun~bc>l.+ IT( suit, p n w t hriu.;e't~t)ltls. I f wi, wantrd 1 0 ronlparta Iatht~r ht-,~drd m i l rnut1w.r h ~ . , ~ d c5171r. ~ ! prt'nt ht\r~\rh.rllkl< I\.+: \vtrt~ldt ~ l r d tn <c.crurt a vcrv largt3 tniti,il \amplc. simplv t ~ ttnd t c n t n t ~ hI,~tlirr 1i~a~lrd 5ole parpnf hc*u<chold>. Adopting thy .;tratcgv of ~ncri,,isin):the ~rvernllsample s i x tn ol7tairl .;~rfirc~cnt hard to t ~ n d Krrluyr; c,in b r ineif~cirr~t 1%-c <-an t - i d up \v"h more of thc ' t _ v p ~ ~ rr~spoilrlent* l' t h a ~ iwe a c h t a l ! ~ I I ~ V ~,An . alltarnalivc appronch i~ t(t cwrrsatnpl~.flrc hard t r l h n ~ l poupq I t 1 en<urp \t~tflc~c'r~t

Sat!sl;uc.ton
['or-r
53tu4.l!..hTv
I.rxd

Coat q i l t rrl
C,lr,d
\\'L* ill r v , ~ i i ttr\ Lntw h ~ n v ~ l o ~r tl ~ *e rtw! F ( ~ N I I,itlnn I ~ q u r t1~ 5 I~lichly !cr btb h> f h ~ h s,?n?plt. c*cliin,ltt.. L l \ i n ~ pr02~ahilitv tl~i,orc t' 2~11t3\timntc* 1v1tl1r7 h i ~ h I ~ a v t a l o f collfit4vnc~th,it the* real pup~dlaticmI ~ ~ u~ r t ' lc l lIIC' ~ ~ ~ ~ t$a$ l i i t1.0 n , y1.r c c t ~ nf l l l x n rample e.;tirn,ite (i.e. In this cldrnplc., sr~mtbrr,hcrc \\'!thin tht. rarlgr 01 55 p a r cvnt to h?. pcr

that \rimt-rarr,I .;,irnplv

GIrod
C.1~311

['vnr Giw.>tl
I;{~x!

cr.nt).

Ut~v 0.l If-tv kc\ factors tliat affect< h o ~ v yrrcistal~.\i-ca c a n specify thv pnpulatfrtn t i g ~ ~ r c 1 %<t l ~ c .;ifr nf the <,lrnplc. I'hc 1ar):c.r Ihc .;ample t h ~ a Iiari.ox\rr>r thc 13.ind (callrd thc cunhdrncr inter\ all w ~ t l l ~ ~n { h i c nbc h can t><tilnt~it* tht. y r ~ y ~ r l ~ ~ilgtlr~' t i n n wtlE f~c. In tlic al-rol-t~ i ~ ~ a i n pri-c l i ~rn~ght tiavc- had n qaniple nf a0011t ?4-IC)(J pt.01~11~ Wi tli 13 ~ ~ i mt l p~ l~4 ~ l7 i e w e cnn say tIi,?t thr pnpulntirln tvilF ltc \vi thin n rclativrly n;lrro14, band of +2,0 pcr cent arrwnll thr* sampfe e>tfmato. If wc wanted to narrtrw ~ h band c IVP wo~lldhil\.e to ~ R C T C ~ I S V thv s a m p ! ~ sizt,. 1 ' 1 1 ~~ c n r r a t n~lv i s that t o Ii4tl~~c th~, 1 conf~rlriiceinterval we nctbd to quadruple t l ~ t ,.;arnplc sizc. Tliub to reilrrc~. Ihta sarnplc cmcv froni +2 n ptlr cmt tn ? I 0 ptnrccmt nrr M J O I I T ~ ~ 11~t?d lo irlcrr,tce the s , ~ r n p q I~ zc tr~ "2011 caqrlt. n b ~ ~ l o l ~ s tllcrC l r ' 1.. ,I ptlirit at ~ ' h i ~tlw - h c?tra Ic\cl o f prtt-~\icrni\ tlot \vi)rth thr' tb\trn cnqt. I t htbrt. th.it prrtnt 1s will rl~~pcnd on the h~n~l.: 32'.111;11~1t~ * l n ~tlie i dcxr~,eof yrcii-:inn nbqtrivecl -T,~bltl 1 1.2 pran id<#\ yst~rnatc~s nt' tlir .;,~n,plt. +I/L~\ \.nu t z n ~ ~ f rt~tluii-c d tlt.pc.niJ~ n z {>TI hl lrv m ~ ~ c .snrnpltn~ li tt.rror you 1vottl~4Iw prt)parc.rl to ~ o l c r ~ i7 ~hl. c . tnblc, ~ n d i c ~ ~what t c s thtb conf~dcnrcint~lrvatrr~vrltlht- ,lt hat i s rutrrrccl trr nq tlit. 4$ ptarcrnt ctinf~dtanct I~.\.cl. TI7u.i. tvta can E l r (15 FYI. ctnt conf~dtantthat if I ~ P had ,I r . ~ n d t ~ n\ample i of 100 c n w s the "' p(>pulationy\ljnlatt, W<MI Id I-"~*'""-:*'

nt1mht.r.;. Thiq can a f f ~ c tthe reprc.;entnttvl>ner;s of the qarnple, but this hlr bv r ~ w e i g h t i n g thc sample to take aucounf of un~lerand m crwrnpltnl: (Ch,lptcr '1).
cnn bc adlu.;trlc!

is nit% ,it G 111r. IiLrliIii~~d rjf c a r n p l i n ~ error: thtl hamplc d{*t.\ ntlt L i r ~ + ~ ~rcllt~zt r . ~ t ttlic ~ l pilp~~lah'nrl ~ Irom rvlilch zt i q dra\vn. I1'ht.n r\,tU~ i ~ r<~ndo~n r . s,~niplr,< ~vr ca n c\tirnatc, rvithtn a sptacif~r~d Ic.\,eF (11 prroFlnb~litv, how cTnw t l ~ t * (real but i1nL17nzr.n) fi~;urch in t l i ~ ~rot?lrlnirrvi artm liLt.li. tr, br- to thc fip~rt>q {vt. gfnt f r ~ j r nthr~ ~ a t n p 17 ~h . t ~M.P + ni~):htf ~ n d th;lt hll prr ckcritof n sarnplc <a\- tircv 5i1ppost cnpital pudshrnen( <<itrcn

Il'hvn w.ins c:t,lmylr+thrrta

1YO

CKC7SS-SFCTIONA L DESIGNS

find in the sarnptc. IC 50 per cent of the sample supported a particular polit~cian then wc cur~ld bc a l m u ~certdin t that the real level of ?upport in the population would bc somcwhcrc bctwccn 40 and 60 per cent. If the sarnpIe was increased to 400 then the hand of seasonable certainty would bc 45-55 per cent (50 r 5 per cent).'

Since the analysis of cruss-sectional data requires that we anaEysc variation in one variabEe in terms of variation in annther variable, our measurements must bc sufficiently scnsit~vcto tap the vaiiatioi~that exists. For example, i f we were trying to explain variations in income and sirnpIy collcctcd incomc datn using just two broad categories, SO to $7 n0,UOO and $1 00,000+, wc. would mask most of d-te variation that exists and have litllc variation in 'explain'. By including mast pcople within the one category (under $100,000) we would be treating them a s though they are the same. Conscqucntly, we would bc unable to explain the variation nlifhi~; this category. Similarly, if we warrted to measure marital satisfaction and this was done in such a way that everyone dppeared highly sahsfrcd we w t d d bc ~rnablcla 'explain' variations In marital sa tic;faction. Ef our it~rtrun~ent is too blunt to pick u p variations then we cannot explain variation since we have not measured it i l l the first place. The same applies with the key exp1,anator~ variables: we must ensure variation. For cxarnple, if wc thought that variations in incomc might he due tn gender it is extremely tlnlikely that, in a sample consisting mainly of males, much of the variation in income will be due to gender differences. I f we thought thal incomc diffvrtmces are partly due to age differences but then tested this using a sampIc that was fairlv age homogeneous, wc would find that age was not very important in explaining income variations in thiq samplc.
I~~futurr~aiion for statistical cu~rfmls

forms including iucr~nvc.iiirnce, invasion nf privacy and time. How long should inlcrvicws las! and how long can a qi~estinnnaire be? rt is impossible to he definitive. The optimal lengtli of an interview or qucstionnairc will vary widely among different types o f people. Some people \till be busier and unable to spend time. Others will encounter language difficulties and find the interviewlquestionnairc more demanding. It will depend on how easy the questionnaire is tn follow and how skillcd thc interviewer is. Even more important is how interesting people tind the topic. Undoubtedly, the length of a market rcsearch questionnaire about something of little interest to people will need to be shorter than one dealing with rnattcrs of direct concern and about which they have first-hand experience.

Types of dntn
There is natl~ing about thc logic of cross-sectional designs thal rcquires the use of quantitative or qi~alitativc dnia. Thc kcv thing i s that data are systematic - that we have information on thp same matters for each case. Whether these data are in thc form of quantitative, numerical or coded data, or uhrther they are quote< frnm intcrvicws, cxtlacts froin diaries or c>bbscrvations,is not irnpurtant FLIT the logic of t h e design. Analysis is simpler (but not ncccswrily better) if data are collected using predefined variables and categors~s. This is wliy q ~ e s t i o n n a i rRTC' ~'~ so widely used in cross-sectional designs. But even in structured questionnaires we must make choices between open ended and forced choice questions (see Fodd y , 1993: 126-52 for a ful l ~iiscussion of the purpnrtcd advantages and disadvantages of vpen and closed questions). Structured questionnaires are one of the simplest ways of obtaining a structured data set. Howcver, we might use lcss structured data collection techniques and frnm this construct a structured datn sct. Using indepth interviews, for example, we migl~ t identify from the interviews a set of variables of interest to us. or each of thcsc variables we would then construct a set of categories and then classify each case. For exampfc, I conducted a set of in-depth interviews with adults that examined the nature oi relatianships with their wider family. As part of the interview we talked about relationsliips with parents. After completing the interviews I constructed a set of variables, a number of which were about adult-parent relationships. Prom my interviews I had idmtified what r thought were a number of different type5 of rclationsl7ips. I u s ~ d a range of pit.ct~o f ~nformatiniito identify the type of relationship I thought applied in a particular case. 1 (hen dassificd each case r)n the basis of a whole set of clues scattered thrnugh Lhc intcrvicw. I rcpeated the same process for many other aspects of the interview ( e . ~ relationship with siblings, nature of marriage, degree of involvement in local community, ct.iih;?liiy of work in life). In other words, 1 war: able m rlsc a range of q~~alitativc information t~.?produce variables and to classify individual

Because of the nat~rrcnf crnss-scctio~~al designs we must br ahlt to introduce statistical controls into our analpsis (Chapter 7 2). This is essential if we arc tu get any clarity regarding causal prcresses using cross-scciinnal data. I f we are to intrnducc statistical controls a t the a ~ ~ a l ~stage s i s these must be anticipated brfi~rr collecting d a t a sincr we cat1 nnly conhilf fur factors about wh~cli we liave information. This can only br achieved by first reading the rclcvnnt li tcrnture and i.hinking thsclu~h tthc thcorctical mc>dpls h r are plarnning to evaluate ~ n anticid pa ting what variables will requlrt. coi~.trollingfor statistically.

Whcn data arc collected directly from respondents wedneed to be mindful of the burden placed nn respondents. This hurdcn can take many

I
192
CROSSSEUTIC7NAL DESIGNS

ISSUES IN CI<ClSS-SECTION A I. UFSIGN

193

I
cases.

I was then able to systematically analyse pattcrns in the data (de

Vaus, 1994).

Ethical issues
As with a n y rcscarch design the researcher using a cross-sectional de5ign will need to attend to matters of confidentiality, privacy, avoidance of harm In participants, and infnrmed consent. T n some respects cross-sectional designs can minimize some of the ethical problem5 that can arise with experimcnta1 and longitudinal designs. Since no tracking of participants is required it is much easier to ensure anonymity. Thcrc is no nwd to kccp l~sts r ~ names f or to match names a t all to responses. Furthermore, since the data collection is a 'oneoff', cross-sectional studies ran be less intrusive and, in this respect, can

Bccause cross-sectional designs rely on existing variahons rather than intrr>ducing il~terventions they avoid manv of the potential ethica! concerns about harm to participants occurring as a result of the inten~cntinn. I t is also an advantage that they avoid randcrm allocation tn experimental and control groups. Randurn allocation can give rise to serious ethical problems (see Chaptcr 6).

Summary

be less of an imposition on a person's privacy. In some survfys cenfidcntiality issues can arise even when information is provided anonyrnrrusIy. In a survey of an organization or a specific region or in a national census it is conceivable that individuals could be iden tifird by examining a scl of charactt.ristics. For i n d i l iduals with rcIatively unusual characteristics tt may be possible to idcn tlfv individual people. For example, In a national census data will be collected for each person that lists their geographic locatio~~ (down to relahvel y small geographical arcss called collector's districts), their ngc, gender, occupation, income, education, household characteristics, marital status and so forth. With detailed information of this type it would be possible for someone with access to tht. data to identify some individuals such as the doctor in a particular small town, the school teacher, the clergyman ctc. Sincc some survey data and census data are made publicly available to researchers for Further analysis it is important that no individual is identifiabre by means of cross-classifying detailed information in individual cases. , To guard against this problem, data from census callcctions and surveys wilP be 'anmymized ' before they arc released to other pcc,ple for further analysis. This is achieved in various ways. One way i5 to omit certain key inlormatinn, such as any geographic identifiers, from any public release 06 the data. Another way is to reduce the precisic~n of the data by collapsing categories. For example, instead of providing data u ~ t specific l~ ages o r birth datcs, only age g o u p s (e.g. in five-year age gruups) will bc reyeased. Another stratcgv is to omit any linking informa tion between cases. For example where informatit,n is collected from each member nF a household the publicly released data may makc i t impossible tn identifv which indiviclual cases belong t c ~ the same household. By controlling the deta~l and amount of puhlicly released infomation in these ways it is possible to prevent the iclentificatjon d particular inrlividuats from an anonymnus data set.

Sincc cross-sectional designs lack a time dimension and randomized control groups thev encounter potential shortcomings for causal analysis. 'These shortcomings represent threats to the internal validity of cross-sectional studies. This chapter has outlined thtse shnrtcomings and identified some ways of minimizing their impact. While cross-sectional designs present some peculiar weaknesses in relation to internal validity it has been pointed out that they also have some particular strengths. Because they do not have a time dimension, they do not suffer from many of the other threats to internal validity t h a t we have encountercd with cxpcrimental and Ioiigih~dinal designs. Cross-sectional designs are a simple and cost effective design and consequently are often able to employ exccllcnl s n m p l ~ s and avoid thc problems caused by sample attrition. As such cross-sectionnI designs can have strong external validity. The absence of randomized control groups means that the analysis of data from cross-sectional designs relies on considerable sample variance and on the creation of post hoc comparison groups. This has implications for sample sizc, sample type m ~ d the practical strategies of implementing cross-sectional studies. The chapter has outlined these practical and cthical considerations.

Notes
1 n i s needs to be q~ialificdto the extent that snppmsor variables may mask a causal relationship. Strictlv speaking we would n w d to be able to eliminate the influence nf all suppressor variables before we could bc ccrtaln that the absence of a correlation b ~ l w c c nhvo vnriablt-j really rrflccts the absence RE a C ~ U S ~ I relatiunship. For a fuller cfiscussion of suppressor var~ahles see Rosvnbcrg (1 968: Chaptcr 4). 2 These estimatrs assume thdt we are dealing with variabfes where there 1% a W/30splil. 'Thc cstimatt'd sample sizes will be different In more hnmogcncous pnpula t~ons.

CROSS-SECTIONAL ANALYSIS

Crtr5h-.;tu.trrrnaI Jeqigns arc r ~ d.suited l t r b 14t~~cr1ptive anaivis - at Itust for dcscriptionq uF thing< ; t ~ thev are at n glvcn point of time. Flowrvcr, crii~q-st-ct ~ n n a ld~s~g115 I ~ a v cbrrn cri t i c i l ~ t t tor their weakncss n t the level nf t*rplanatory, causal analysis. Somc rt*scnrc-lier? argue that, for

levels in particular locationr: and , i t a pnrticular \tag{, nl Ihr Fanj~lvlifil cycle. Since thy.;(. sirrvets are nnt primarily designcd ti) rlc\,rlnp cxyl;lnntinns c,f bcha\,loi~r, problcrnc (11 causation, of ~a~1*;12 o r d ~ or r the inflwence of unciintrr>llcd \-ariabl- do not matter. TIlc ki~vthing is to rn.;rlvca that sarnplr%,irr Inrgc enorrgl~5cr that rve can p a t sufficient ntrmbcr.. E n the varicius si~hi:roups to prrrvidtb rt-liable descriptions. The 0tht.r kcv thing is ti) ohtaiii qarnplcs In .;llcli a way that wr3i a n l?;t~ncraI~ze bryclnrl the samplr ttr thc population frtrrn which thc. sa mplr w a s drawn. ?hc an,ilvsis i ~ w then c bvcnrnrs a matter of t171* npprnpriate wnv': tct ~lc.scriPethr data It 1 ;. beyilnJ th~b +ctrpr of this I l i ~ > tt, k go through nll thi* warTsIn tv111t h d ~ i can a be i i r ~ r r i l ~ c r l .

scvcral rr*n.;ons, cross-sectional designs are ~ ~ t ~ r t l i lat c sthe s c.xplnn;rtory q t n of ~ ~ '1 ~ t u d v . To support t h ~ i rl u d ~ e m e n tIhry cite thc diflicirlties c m ~ s - w cI (tma1 data presrv~ilor t ~ t ; t a b l i s l ~ i the* n~ cau~a1 order ot c t . m ts, for cun troll l n for ~ all thta variablrs that miell t 1-c inflz~cncin~ corrcln ticwq, nncl tcrr r%\tnbli<hing,~ z n , i r n % ~ r ~ u ~cntv.,ll i u ~ l y , clrder. .-\-\ltIiou~h t l ~ c j ~ ~ d g ~ mIt~ vst~ eon c l the- I-r~tic~sms is too hnrqli I1it.v arc ~-alnab!t* hc~ca~~~c they C ~ ~ J M ~ l*l c tion n tc1 inl~tlrcn t weakncssrs uf tV~i% cross-sectional dcsign that murt 271' addressccl.
Descriptive analysis

Cnuntin~ i.; 3 Ilnsir aspect of dcsrrjptive analyqi.;. Tlic aim i s to eqtnblisli liow m a n y pcople have pnrticul,lr cliaractrri~ticr, hnve n particular opinion, f r c l ~ s ~ in ca gi~en i \ r a v ;lnJ $0 forth. h,l;rrk(,t x\lrvPyG wilt find out hnn, man!, pcnple like a particrlldr product, a pr)litical poll rvifl c011~lt t ~ o w111.111\. pi,opIr ,"tend to 7 r ) t ~tor n spccifir p;rrtv clr can~ttd;ltca, ,I census w ~ l lctrunt the nt1mhr3r of pcople in tl~cn l,c*ptll;ltiun, tvhilst .I

Cross-sect innal design< arc iJcnll y suited tn ~icbscriptive analysis and arc. widt.1 y ~ i s r d for these ptlryoscq. Thcir wideaprtud use in markct rcqmrcli and pol~ticsl polling b ~ a r this out. 'l'hcsc .irtl contexts in which rt.scnrchrr< want simply to dcscribe such tI11ng5 as: who u s w o r likcs particular products; aspcct'i af consumer brlinvinur; voting intcntinn; level of rtittlrt.>t in politics; trt-lings toward< plrtictrlar politicians; and attih11k.s tcwards a rnngc of iqsues. Typicallv, this FTe elf rnqcarch fc~itsc.; on question? of 'Irr>w m,lny' and 'which tvprs" of pctlpZc do or th~nk rn pirticular ways. 'Fhc ciln.;IiG i h anothcr 1-xnnlplc of a desvriptivtn crclss-sectional tfc~ign. I t s pfinlarv f~mction is it1 prt~vidc <In accur.llc description c,f fhc pnp~l!ntic~n of a cotlntr~f: hurr mJny pcnplc lhttn~arc> in tllr p~rpulatiun, d n ~hmv f man\ people p o c \ c - ~ parhcular chLlmct ~ * r ~ c f i(e.9. t > svnrfrr, arc, e d ~ r c ~ l t ~ rIrl.ing m, in p , l r t ~ ~ = u Itration.; t.~r chtc 1, lloucehold c x p c ~ ~ ~ l i t u r c survcv* provide a filrthrr t*uarnptcof c m r ~ - w c t i c ~ nclata a ! bcinz II~C'LJ tnr dcwript i v r purposes. In thrsr. \11~vcys ~ o ~ ~ ~ r n s nei~m ktot mnp ~ thp cxpcnd iti~rv patterns of indrviduals snit I ~ u u ~ r I ~ r > lSuch d s . .;urvrys ennbli~ .(int>-graineddc.;crip~ir~ns of the expcnd i t ~ i r c patterns of cl i fft-rent tvp~':of Iamilicb and Iwia~c~h~drls and for pivvjlll~1231th different it~comc

hcluse!mld i*xp~.nd i ture SiIrvLay ~ v il l dcscriht. h o ~ vmany pcople 1iai.rparhcular patflmrns of e x p e n d i t ~ ~ r v As a result wc rnunt how many pcoplv gi\re p a r i ~ c ~ ~ answer5 lar brcausc we are intcri>stcd in the t i r . ; l r r l l l r l r r l r i nf cases. Mrhat i s tlic h.pical rt.spon<t. t o the q u ( ~ ~ t ~ { m Hc ' '~ w ppicat i < ~ l r r ttvpical r ~ s p r v ~ ~ Art' people qprt.nd over a Inrgc, number of c;rtccnrirs' Are they ciu+trrc*d in lust a f t w categnrics' I-low ~ i r n i l aor r drssvrni lnr are people in rlw sample? Art. tlwv clustered lowards one end ot n ~ i ~ s t r ~ b u f (o pn . ~ low . inrnnle a d ) . In othcl. wt)rrls in addition tu nimplc counting wtTwill look at thc sirnpr, of any distribution and try to summari7e in rnmningful wavs what tllr distributinn Irx+kslike. Thew ~ n ; l l smay be acliirvrd with varini~s m a y st~tisticq.'To stlmmarire wlir?f i~ ~ I I J J ~ C R In ! a ~iistr!b~tiiun w e liunuld use a n appropriate measurc. of ctmtral tendency (mi*nn,rned~an o r rntrc"l1). 'To sum marirc thc spread and i r n t t ~ ~ l t i l iin l ! ~a d ibtrili~~tion we would I I ~ an npprt~priatcmcasurc of d ~ s p ~ r s i o (ne . ranpa, ~ intcrquartile rangc, variancc and rlt~ndnrd devtntltrn) Other aspect5 of Ihr shape of n clislrlbution call IV .;llinmarizetl cvitli qlatiqtic~ sucli a.: skrwncsr; and kurtrwi.;. mrlrc~ and G r a p h ~ c drrcpr~sentations rli.;tribi~tions can offtlr~c o i i v e ~ be mmo i~nliritl~clt. und~rshn~i,~ than l ~ l cf;iimn?ar\, .tatr.;ttcal mc;lwrL*>. Tic c h ~ r t k , Ivr chart5, 11nc gr;lyIi\, h ~ ~ t n q a r n lieu s , and tvhiqkcr ylllt\, stern and I c ~ f plot.. are ius! somc nf the graphtcal mr;rn<c>f reprecrntinfi distributinnh. Mo.;t cvrnputer b ~ s c d <t*~tisticnl p;ick;ly;.rs and sprcndsht~c~tc producc n rangc3of such praplis. AltcanlativeIy, wv may choose to pn3hrntit the data in tabular form. Frcqiiimcy distribuhon. arta lllc normti! upavof doing this.

CROSS-SECTICINAI, IIESTGNS

CROSSSECTIONAL AN AI.YSIS

i i rm>dincy ~iro<kril,y rr rrlntto~tsl~iy Table 12.1 ,411~ l l r t ~ t r f l t mof Recoded version

M't~endescribing distrtbutions we have to decide how much detail to provide. Continuous variablcs such as age (in years), incnrnc (in dollars) and hours worked each week can be analysed as cnntinuous variables, or collapsed into groups. Altcrnrltivcly, age might bc collapsed into categorie~ S U C as ~ under 20, 20-29, 30-39, 40-49, 50-59, 60-69, 70-79 and go+; income might be collapscd into income groups (Iess than $5000, $50[10-10,000, $10,001 -1 5,000, $15,001-20,000) and analysed in thcst. grouping5. Manv ordinal variables might be collapsed into fewer categ~jries. For example, respondents in a survey are asked liow strongly they agree or disavcc with a particular attitude statement (strnngly agree, agree, can't decide, d~sagreeor strclngly disagree). We would need to make decisions about whether to collapse down to thc cat~gories 'agr~c', 'can't decide' and 'disagree' or, alternatively, if we are only interested in agreement wc could simply use the categories of 'agree' and 'other'. Nominal variables, such as country of birth, may havc a largc number of categ-nriics - many of which can be combined for the purpose of simple and cTear presentation. Wc might group countries into regirms (e.g. Western Furope, Cpntral arid Eastern Eumpc, North Arncrica, Other America, Sor~th Asia, South East Asia, Oceania) or according to levels o f development, or whether or not thcy arc English-spea king (Chapter 6). The d e p e of detail we retain will depend on factors such as sample size, the number of people in particular categories, and the way in which data are to he analysed and presented, and on the main points that need to be highlighted. The way In which variables arc collapsed can have important implications for patterns of results a n d care must be taken to not to simply rerode to cithcr crcatc or mask a pattern. For example, Tahle 12.1 ilIustra tes t h way ~ in which coding can mask a relationship. There am two broad ways in which coliapsing of categories of a variahle will be done.

Male

Fcmnle

Shonglv agree
Agree

5W"
I0

60%
45

M1%
40

Disagree Skrmgly disagrw

30 113

35

' 1-

Dirnpree

40

Using this approach we focus on the distribution of a variable as a basis rather than the substantive meaning for recoding and let the distribut~on of the categories dctcrminc how we combine categories. For example, rather than collapsing income in the way described above we might simply have two categories: people below the median incornc and those above the median it~come. If t h median ~ income was 535,000 then the hvo incoinc categories would be helow $35,0011 and abovc $35,000. Or we might divide thc sample into deciles: the bottom 10 per cent of income earliers, the second bottom 10 per cent ~iincome carriers, through to the top 10 p ~ cent r o f income earners. The specific dividing Iines would be determined by looking at the actual distribution. Using this approach, definitions of low and high income are not imposed arbitrarily by the researcher but are defined mlafiz~e[y on the basis of the distribution. Table 12.2 illustrates liow the same variable might be collapsed differently depending on whether substantive or distributional methods of recoding were used.

Form of datn
As well as deciding on how to present and summarize distributions and how to group values, there may be a need to attend to theform in which the data are analysed. Sometimes it is desirabfe to change the way in which our measurements are expressed. We may need to convert variables so that thcy are a 1 1 measured on a uniform base. This can be necessary when analysing data in repeated crosssectiona1 surveys where the value of, say, lncomc needs to be adjusted fnr inflation. In cross-na tional studies some variables, such a 4 income, may need to be converted into a commun currency (say US dollars). This was Jiscuss~din Chapter 9. Data may also be represented by exprwsing individual scores, not in tern~s of the units of the original variable ( e . ycars ~ for age or dollars for income), but in terms of how the score of an individual (or group) compares with the vpical vaIue for that variable. This might be done by

Using this approach, categories are combined because nf thc similarity of the categories. Thus combining the categories 'agree' and 'strongly agree' bvould be done because thcv botli reflcctcd asreemen t. A1terndtively, depending on the purpose of thr analysis we might corntlint. 'strongly agree' and 'stron~lvdfsagree' as t h y both indicate people who feel strongly about t l ~ c issuc. k i t h a variable such as incorn?, substantive recodlng would combinc categories c>ri the basis of the actual income values. Tl~uqwe rnigI~t collapse income into categories such as below 5113,000, 51 0,000-19,994, S20,000-29,999, $30,0130-39.999, $40,000-49,999, 950,000-59,999, $60,000-69,999. 970,nOO-79,994, $8U,Ol)ll-fi9,9'dY, $90,00099,949, $100,000 and over.

do1F.l r+, Il ~ . i ~ t < l - h r n . ~vtln r k ~ .and fr;inil;. Instca~l rrvt, I\ Iruld Iw tit-al rtiq ~n 1 hv cclmmrm 't+tsrrtrr~c! ' nt dc\.iatinn< Frrlt~l tht, miv,ln A v ~ r i a b l c15 <tL~ndtirdi/l+d \lrnpl.r: 1 v . s i l l ~ t r a i t i n gthc typrc,il \.,iIuc Irnran or mw'rnnl frclnl c;tch pr>rsonrs acrual ccl3rt. on Ihdt x . ~ r ~ ~ b l Tilt' tb, rtb.;~rltinfi vn1ur for cnch ycr+cm I* thr-n d i i . i d t d bv n rntlasurc c ~ \.nrintinii l (5tanriar~1dcvintlrrn 1 1 n?t5;rn rvas itsccl a n ~ lintcrqliartilr, range - nr ~\.linlc*rcr - if [he rncdi;ln trrn< 1 1 4 )

rc~-c.rprc~siiiq 1*,1ch pcrkrrrl's inctrmc If~lr t.\,~~n~~ it1 lt" tr7rtii< l t r l tie\\ m~rcli I o n - r r c>r 111):hc.r i t i c th,In thta avtLrngr \r;llt~c.O n t ~ rz-av I n \\.h~rhthlc is dtlnt. 14 b~ {,\yrr,\+rnq a pcr\on'+ ind lvidunl w r r r Iir tcrn-rs ot t h ~iumtirr ~ (I( <tilli~IilrCI d c \ ~ i i l t i ( ~~t t ~lit'< s ~ ~ O V or P~ P I C I W t h t ~IT~CYII lZrc can go one ~ t r p t i 1 rthr-r anif stnrtrft~rrit:t> ~.ilri,ildp~ Standnr~l . i~atton 1 1 . 1 . ; thv effrct of cxprcssing c~,ich yrrscm's ccnrc rt,lt~liiir to that c3f other.: in the dl.;trihutinn I t has t l i cftrct ~ t ~ cllrninatlng t difltkren~-rs rn thc 11n1t.; of mcas~~rcn t~ acrosk r ~ n cniln hlc*\, nc-ms+trrnt. anti air(?\.; d l ffert-nt iv.~riLillles. I t mah'lcs LIK tu cornpart, ;~pylcsw i t h oranges. U y s t n n t l ~ r d i ~ i n income, g tor curnplr, wt. cc>illd then t>xarnlnihchance+ in !lie incornr* po?\lhon c d + w e c'(lt~ld~ 7 ~ p r t ' s s rncn and wornrn (3vt.r firm- i n diffCrt*nt c t n i ~ l l r i ~If c,~ch man'\ anrl each w c m a t i ' ~ i n stniidar~lizcdtr~rrn, t1ri.n for any ycnr or country rr.I3 ctr~llrfcalsulatc thc a\.tbrage : r t ~ n i l a r J.;crrrc t ~ t mrn arlrl of wonlcn. We c t ~ ~d~ o lthi< ~ l c3vt.r a largca ntirntlcr o f year.; and nnt b r wnrrit~d ahout thc nvcr,lll tnfftuf of inflalron. M'r MFI>LIJII stlt' thlft, 011 nvcrajic, hvrntat? ~ ~ o u l i bc 4 rclci.i% infi incnrnvs t.cblow thc ;IvtaraKtl and m,~le.; nbovc lhc ,jveraKe. I ~ v htnndardiring t t ~ r ~ a l l l 1l.e c s i n n rr.,~dilvstm(, r\.hctht~r thta cap bt"tt\~rcn rncr a ~ i d xvtrrnvn , I n a r r n ~ u i t rerr ~ \i*idrr~i~i) (>v(*r : tin~t.. I n ttrc un+t;liitla rtj i ~ t x i l f~lrtv 11 L\ IW Id be rnclrt- diitlcu It tri r%ht,lblish$In\ Insrtqki<th o r drcllnt, bt-ca~rwb tht, ' \ aIur,' of rncc)mc.: c t l n n ~ c 5'I ht* avtxr,iqc E~CIII~IT ~T~>~I\.CC'I I I I I Cinctlrnlx.: c l b r n r b r l ,rnd wurntan m,iv h ~ v ~ricrt*,i~ecI c I n i1t~c)luIc ~ l o l l tei.ms ~ ~ r ' t l ~ In ~t tcnTatitc *I-rm.: tht. c.1~ nlav ha? t n nnrrorz'td StC~ndarJi7tnl:incomca a l w allor\'+ ztc to clrnlFarP t l w d i f f t r t * t r c ~ ~ ~ Ivtrvrbcn ic>lrntrit*\ Rr. st,1nd,>r~l17117~ lye cciuEd cuniyarL1 t f ~ t rt~l~ti\.~b ~~OSI!IO~ of R nltm ,~ndi\.onlLhn rrr Erlfaln, 3rmt~ica.Gtv-plsrlr., lapan anrl I r a n w withollt t-tainj: t r 0 ~ 1 l d 1 ~ 1~ 1 tlitb d ~t p t r~~ri~ l ~ - ee~~ ~ v p rct l~ ~ in ~lc.,

Tle~(-rihtn c l~ ~+tr~lli~t~ no cir -trw < ; l ~vli.olc~ .;tirrfr; I+ t\~prc,~fl\ t)nlv thra hr,t part I ~ Fa Jrst+ripti\.c. n n a l <i+ ~ florr, clftrn th.111 ii(+l\\.I., \tm.int stat. bur\* r l ~ s ~ r i h t ~ t riit icm frar ~ anlong lit C~lrcnt.;ub.;rls i j t tli<bpctrplc rmnl3rj<ing tht* <t~i<F\.. Fttr r \ < ~ r n p l ccf41 , tllr incurnr pattt~rnk t ~ rnvn f ;~nd rvtmicn ~ l l f t t > r ' By hnw nluch? I n rvhal dircction7 Art- Ihtarc rlifftlrt-nctl.; ,lmclnr: tilt* yclurig, r n ~ r l c l,1 l1: ~ td ~ n rthr. l rldcrl!7 thtav dibtcr asrordrng to rr>gion7 C~vnp,~sini; ~ ~ ~ l ~ on q r a o y t~ m t~ wvar,,rblc p ~ linctrtnc, c t ~ ~ q c r ~ ~ l t i 5 n 3 , rclic~nu\nps\,l i a r ~ p i n r ~ < i\ s f";\r~ntlnll,v l n d~\cripllve cbuerc15ethat rntail.; r<t,~l,li~li~ rl-hirli n ~ tvpcq ptv,pTtl ha! c p,?rtiuul~ir rIi;lrLictcr~~hcs, att i tuilr... cir I)ch.~~~crtir.: It \trrp+ ~ l l r ) ~rrf t ,iftcrtjptin!: t r y c,\~l,rrn r v h ~ snnlt9 >:nrtrp ,ire cliFfr>rcbnt Irvm r~tlirars(,il t l u r ~ ~ g that h IVCIIIIC~ IJC' tht' o l 1 v i ( ~ 1 \ next +try) 12'11arta .;irnlllv ~l~,st-rltlin:: F;Flrrbrn%, \\ Fll~h n~,t\,in tk~rn. ![-ad r ~ b auco~~li (31 t * why p-nul,c ~Clffer.I'Eir, I , Y P T C ~ ~ of drascrilrin~ I+, lur~v.yt-ttv+, tl~c flr.;t rnd ~ ~ ~ t ~ n 5t.l-p c aITrltlr l ~ l h~ 1 t!tb\~clup~ng ~ p ~ l ~ l ~ ~ n a t ?Tor onc ~.x a t n p l v , brfvrc . ; I T ~ I e ~ \y I l~ a ~ n~ ~ n t l rdifft*r(.ii~cc l. i t 1 it~cornr> M ~ L mu<t ~ flrkt ~~t~if~ tliv lis t*xtrllit li ; ~ t l dnaturr 05 tlic LI i l tpc.r<.nct... l ? c ~ r - r i p h t r at-rrrs\ n~ * u l > g r o u p cat^ bi*cwrn l n v ~t~krrig l a ~\.idt* ~ a r i e rrf t~ data analvs~\I c c l i i i i ~ l i ~ bu ~ ts~ l i ( n rtnhcarr-her w i l l a l w a v ~ rmplu!. F-riv~riatc or n ~ ~ l l t ~ \ * a te r !mrthuils ,l o t ontl w r t n r nt-lother. Tlic tt~cliniclirr*s \\rill varv hrlnr siniplc crt~<+=knbulatlrrnq and s w n l ~ a r i ~ o clt nrnc.rnt; to rncjrc. c n r n ~ l t l \ n ~ k itivaria l tr ~ r l i n i q u c s such as niuJt ~ p l c rcgrt~ssici;lnlld d ~<crrrnin;~n t anal\-si<. Thr analr~l.,m i ~ hrtr l v t>n wrnm.lry .;tatisfirs ( r - g. rrrnly;lr1.;rm of rncvtl.;, corn>l,?tinncntbfficirnk, annlv5i.; of \~nriance),uatl ~ l a p h i c a l rrprfit>ntatitlns ( c . ~ wattrr~mrns, . co1nplt.x I ~ n cgwph.;, sirtstcrt*d I w r r l i n rtql nr rrly tin t.~bular prt.w~itaticm ( t ~ t - t ~ ~ vrra<c-tnbtrlatirm ay to m u ltiwf,i\r crc,s~-t.lbulatiotix) T h t . I-htlir-t* l ~ t . t r t th c ~ r V J rtrjkab .~prnml~.I~c\ I\-111 <1ef~-nCI rtn thc c l ~ ~ > ~ ~ n c t r , r i thr~ t ~dnl<i t - ~ It...:. Il~~t.1 mc~a<~irrtnt-nt) ,init 11ie tvpi. of atttl~cnt-c t ~ ir,li1t'F1 ) thl* an,~lysl. 1 s d i r e c t ~ dChapter . rl p r r ~ v i ~ f ian rk o u t l ~ n ~ nl a mric,.p i l f alr,>Tvcrc trnrhnrtluv< ;lntt t l i t , d a t , ~ ~on-.~d~.r,~tfo~i~ Ih;lt tviI1 aill-ct clnc's c h o ~ c v I~t~twt~ t lc i~L~Itl~rn.r ~n t j x (15.
c9t.11

21Hl

CRC7SSSFC I'ION A 1 , DESlCNS

CROSS-SECTION A L ANALYSIS

2I31

penplc resptmd to sets of questions. I t is an approach tr) analysis that is frequently used in s t ~ ~ d i c of s ntiitl~dcsand valtles and personality characteristics. Essentially it involves looking for pattcrns in sets of attitudes people hold. It seeks to drscover whcther there is an underlying structure to tlie pattern of responses to questions. For cxarnple. we might ask people to tell u s what they ronsidcs to be the more important attributes for children to possess (e.g. good manners, obedience, neatness, imagination, independence and self-controI). By observing the pattern of answers, wc might find that stme people emphasize good manners, obedience and self-controf and place littlc emphasis on imagination and independence or vice versa. In other wolds ccrtain variables cluster together. %me people will select charactcristics that reflect the underlying factor of uorifo~mit!/whilc others select charactcristics h a t reflects an ralltunnrny dimension. Similarly, we might hc intercstcd in the types of things pcoplc look fur in a job. We might ask about the importance uf the following things in a job:
1 good pay 2 opportunities to use initiative 3 having rvspc~nsfbility 4 the feering of achieving sornerhing 5 absence of too rnuc1-1pressure 6 gcncrous holidays 7 g o d hours.

of inferential statistics are cc>mrnonlyused in descriptive analysis: inter-

val cstimatcs and tests c ~ significance. f Interval estimates allow ws to estimate within what range of the sample figures the population figures are likely to lie. For example, in a sarnpIe we might find that 60 per cent say they intend to vote for a particular political party. Wc can use interval estimates to estimate within what range of the 60 per cent the population figure is likcly to lie. Using a statistic called the standard crror we would be able to say, f0.r example, that we can be 95 per cent confident that the real population figure will be within a certain rangp of the sampIe cstimate of 150per cent ( e . *2 ~ per cent or within the range of 58 per cent to 62 per cent). Another way of using inferential statistics is to use tests of significance. Tl~cscarc frequently used in conjunction with correlation coefficients. They provide an eshmate of the likelihood that a correlation at least as strong as that i~bscrvcdin the sample will also bc found in the population. For example, a correlation in a sample could reflect a real correlation in the population or it could simply bc thc result of sampling crror. Tests of significance provide an estimate of the likelihood that the sample correlation could be due simply to sampling error. If there were a rcasonalrlt chancc that it could bc due to sampling error W E wnr~ld not feel confident in predicting that the sample correlation would be found in the wider population.

Explanatory analysis
When trying tr, make statements about causal relationships, cmsssectional designs must rely on static comparisons between groups or on correlations hehvcen variables where measurements are made at the same point of time. The problems with this have already been discussed at length (Chapters 3 and 11). The basic strategy by which these problems are handlcd is by introducing statistical controls into the analysis.

We might analysc answers to see if there is a structure or a pattern in the way individuals respond to each of these job characteristics. We might find that there is a structure: sump people tend to list the intrinsic job characteristics (npportuniti~s to use initiative, responsibility and feeIings of achieving something) while others list t t ~ cextrinsic characteristics (pay, holidays, hours and no pressure). By looking for a pattern in the way people respond to sets of questions we are able to develop a more sophisticated understanding of the structure of attitudes and behavioilr.

TIre logic af statistical controls

The logic of statistical controls is best understood within the context of the logic 01 experimmtal dcsigns. The experimental design relies on comparing change over time in two or more groups, which were initially idenhcal in all relevant respects, Different groups are exposed to cxpesimental treatments and a c o n t r ~ Igroup is E X P Q S E ~ to no treatment. Following the treatment, any differences between the groups on the outcome variable is taken to be d t ~ e to the effect of the treatment. However, ii ! h ~ p n u p s were different to begin with we would not know whetf~er any post-treatment difference5 between groups were due tn the treatment or tto the initial differences.

As well as describing patterns in n sample we will frequently want to generalize to the wider population From which the sample is drawn. We 1viI1 want tn know if the pitterns we have described in the sa~nple are likely to hnId in the wider population and, if so, how close the population figures are likely to be to thosp we havc foi~ndin the sample. When we use a probabiIity sample (Chapter 6 ) we cap use inferential stntishcal tcchnjqt~es to answer these sorts of questions. Two main types

I n rror.-w.c.rt i,?tla\ rft.ciqns tht, l o ~ l r 1 , to I-tvnparc crclllps \Y hicli art* ~ ~ ~ (1t1 r b t/,l. r bwis l ul e\~.;tinq ~iiflrrcnces (the indi.p~nrit'nt variatll~:) \t'p t ) ~ ~ Cc>mpJTta r i OLIt<t>m~** ( t lie C I I ' P C ~ L ~ C I I I \)arrt~blc) In t hcsi* grr ii~pq. \ l ~ r .difft,rcncr< rnav be interprettd . + b c ~ n qduv t t l thc influence of dlftcrmt grm111 rnt~rnbt*r3k1p. Thus diff~rrncc.;in lhc incornc (drprnden! \.asinl~le) tlf nralc< and femaler [indc~pcnclcnt i ~ n r ~ ~ brni>:ht l t * ) I3c due t~, the cftect of ~ c n d c r orr incnmp H ~ M ~ ~ the V I male-fi.rnalc .~, difference.; might trc J u t , not to ~ c n d r p7t.r r Gr but ZII oihcr dilfert>nucs bc.lv\rcrn thr g r o u p . T11c nialr p o u p rn~gfithi* (11d1,r. br bcttcr educated, includinlorc t'ul l ttmr* workers, bc rlre1\vn from d iftt-rent c r c ~ ~ p tiom ; l than thr tr-n~ale grrltlF. Tn i\wrk t n ~ t ~.hekht*r m.ilc-fcn~alc rltffYrr.ncr.< In irlcornc ~ l r c di~tatv ~cnt1c.rwe r ~ t w !to ct)mp,ire curnp.~ra~~lr* nl.iIe5 ,111d
~

Thc s t r t m ~ t . an.ilv<i\ ~t rxcusc whrn ~t 1%. f ~ ~ l ' r r L >fo l ~m *. 7 tch on a tvhvlr 5t.t t ~ i

tt~malt~s. ?litb c.xyc.rimt.n tall cit-.to,n norrnallv achievcb.; ctrrnynrat,~ l ~ t yhehzeen CrnuFG by lantium a q s ~ g i ~ m t l r~ ti gro ~t i ~ p s C r o r ~ - ~ c c l i ~rlt~.;ign.: ~ n a l stink to dchi~vc crlrnprlrt7b1l1 h- h v I ~ I ( I ~ C I I F (Chapter I~,~ 3) t w t , slrlctl Ihrt' r t . 1 ~ nn ctiqting diffen-ncr<and all rlatn arc* ctdlccted . ~ a t single time pcr~nl, tliv mntch~ngis dtlne at tht, dala analv\is s~;lgc - ~ l t c r rlata CITC collcctcd s , t~ her than bt-lr r e . Th~k rnatrhing will alzva];? he s I~niittlJ ,~t~ .I d Irs, . i d ( ~ q ~ ~ mt*,ltls a t r ~ Illkin r,indc>inalh~cat~cm o f achic\.jng grtwp corny;lr,~l)il~tv IZtbcnn on11 maitch in r~l;ltionto .iriaI~lcs~ b n i ~\.l,ich ~t ~trtc>rnrntit,nhn-- brtan i-t>flect~d. 7'his nican5 wc n~list anticipate rc,lt.vont r;lriabl~l.;on w'lli~ll+t:rrnips bl10~11C1 ht* n~,~tchcd. Our ,111ticipatirln r \ , l l l aluva~=, be I t ~ s than pcrti3ct. I f M'I* wt1ri3 e~amiiijng !he i r n p ~ c nf t gender nn incorntb wc would ncc.4 to con1p;lri. mcrl and tvonicn ~ ' I 1 ( workcd 1 ,I sirnildr number of h r ~ f per i ~ v r t = kIt wnrilcl be inappropriate tn I-ornpre a grrxlp crf W I ~ C I Iw l i ~ r e m a n y worked part timc ivitll a R ~ I ~ L I F ) rnt-n w h ~ 3 Wcre mainly full time wnskcr<. It maLcs m n ~ wnsc t to comparc. mvii ~ v h o tvnrkrd f u l l titl~e wi tfl u.cirncn who workrcl (~113 timtl and crb~t~n~t, incomc dif lrrcncrs brtwccn !Itrsc ITVCI FroilFs. We ro11IiI thrn c(>mp,lre men and r%,orncsn w h o wnrkcrf say 2q-75 hour< a zveek, and then th(3sc that work 75-21 hours, and so forth. Senqiblt. a n a l y t s rill seek to C ~ ) H I J I R IIikr P z1)!/1t I f income d ifit-rt.nct=spcrsi-t hctwcfn rnvn ant1 warnen ichrn wr match for (control) hot~rq of ~ ' ~ tlit'n r k wc h r i t 7 ~ tllaf ~ flle initial inctlmr, difftbrcnccs an3 1 r r 1 l dllr~ to dkffrwnf t~o~rr.; r>f wvrk trf mptl anti ivlrrncn, Ilthrla~ghthrrt, m<iv Ilr' nthcr %actr,r, ttwt ma\. ,iftrot ink-r>mrrlifttxrent-tl~ l i t ~ t w ~ t 'nlcn n ,ind wotncn, rIur ,1n,rly5i< will a t Ir>,lstt i . l ~ . ( ~ elilninatthrl hours of \x,nrL. a 5 t h r t-\pf,qt~ntit>t~ t<rr the incc-tme d 1fitwnrt3s \Ye micht iv;lnl t t r cl>rnp.lrr tncmnitl drt it.rtwc.rs .Irnlmq rncv and wclrncn ~ T I ~ R I sirn~lar ~rcc~lpatltm.;. Wc r4t)ulcJco~itrollor (rnntc.11) t)c-cupation aritl cnmpart- profe-~c~na E mr1n \v ~ t hprcrtcs~~nnal wclrnen, clerical men with clertcal women, and 50 on. If men nt~tl wiinien in tliu same tvpr t ~ f job ha\ i> di l f ~ r r n rncr)mtas t thcn \vc r%mcrul~l h a v r tcr say that inrclmr J if!t*rcnrr>+ hchs-xn m r n and \rfomrtl art. r r t ~ cluv tn the rliffr~t'nttvpes (if jotw helrl by rncn and wnrnen.

variablrls ~ r ) ? r l r l ! r t r ~ ~ - ( l r r 1 ~ ht* f ~ t . tirnrtnvnri,~blt,.; M.V C C I I troJ ~ a t once, ~ I T c ' more s~miIar the grcrul,q wt5vtrrnparc \+,ill his. I-nr rx,~n~pl(b, wtnniiglit think !hat incomc J ~ l t ~ r c n s klct\\.t.cn cs men .lnd \\,nnj<.n .Ire ttue tn thta curnula t ~ e \ efft3ctq of gi-nclcr drfCprcnncc.; rn hcrurs o f rvt~rk,tvyt*. t > f job<, atnoun1 of wrrrkforct. ~ ~ u ~ r r i c n and c c * 19 li~ca ~ion,ilF,rckgtnund. I\'(, .~rnulrlh ! . to rcmtrvc3all tliehtb fartors frcm uur cornpari.;on.; of rncn and women. We m i ~ l id t ivi~lc o u r qnmplil into m a n y grnupG so ~ h a i n n r Kroup rn1g1-1'be xvorner? ~\.nrkrn); h1)1 tinlrl in yrofr.s51rm~I i c h c \viit> ! l ~ \ ' c trrtiary ~ qualiTh~.;t> rcrulll be fications ant! 11or.c a l Icnst 1-1 irrar.; worktorcc rb\perrclnrt%. compnrcci w i t h ,I c o n i p w h l c group o f men. ttnr~thtl. FaTr I I F KriIup n l l ~ h t bc < i r n ~ f , ~ r thc t o ahrn,c. in 211. r r ~ p e ~ t s . e \ c c p that I tli~ nwn nnd w n m m h,lve trntlc jobs. M . I I ~ r~tticr pair5 01 grtwps c-clulJ hc comparctl, R y ct>ntrcill~n~ for or rc>mt>vinginal~afcrnali. dit1t.i-encus we [.,in ,in.;ivcnr tht* q i ~ c s t l r ~ n . 'M'lrrrl r7llrt.r Ilrlny. nrtut-,~rlt~l, d(x&c gcndrr*mahr*a n v di Fft,rt.rir.t*to i n c o t i ~ rlf, ~' ~%.hi,n a11 irthcr t h i n ~ s ark- ctlir;ll, rncn ant! wonrcn *till havta d ~ t f c r ~ * r i t ~nc~lnic*r, !hr*t~ rvt. h a t t~evt~lenc-i~t>f gcntlt*r~li*crirnin,it~or~ Shr\ nlea,n$ that gt311~Ec.r iirtlf il: rt>spc~t~rlhlta ftrr tllr ~rh.o~iit, 111lfcrr-nccs If, ~ r ~ h ccz-c i r r-tlnlnd !r>rGI sci c)t \,anahlcs, gcntlcr d lfertanctb- 111 rnctrmc 11 ~+alywar, 1k.e \rcwlrl bax t47i1( I h c > i rri 1i,11 i t ~ i . o n ~ ~ i~i,( t ~ > r ~ ,tt*L\rta ~~~t t!l{d4 ,$ (!I(, v'i r~~iL-l~~< \VL> h ~ v t conlrnl lcif. I'lit 11 ~ffcrr-nilv, wtb wc,ulcl s a y tha b t hc. c{lnll.n'flctl iF rrabltbh ; l crpl(ir?~ t h t x irliti a I c 1 I' ttt,ren~-~s. C I I c o i ~ r s rrt , is not pcxqil7le to control for c.1 erv posh~ble v;lriaihlc, so I h i s yrr-ibili~!~ atwal,s rc,main* that any gcandcr r ~ l ~ i t cincc~~nrd di ffrrcnct.~ cor~ldh t n dut. to thc<t* rrncuntrt1lletl vari~bltn+. Ilirria a r r .I vari~t\.of lvily\ in w h ~ c h .;tal~sticalcontrols A r c marJc a n d fhprc. art- a I ~ r i c . l \OF wav\ of interprei-tng tht* ~ . a r i o t ~ pattern\ < of rrsullq il~dt cil~i 1~ t~tltaincd. Ejnv cmmrnon nlctliod o f ct>ri~rrrll in!: for v;lriablr\ ih the ilrabomti~tntcchniqlrc.
The elahoratinn technique

T h ~ r tih , insufficitwtspace licrc to cxplorr all the Jcatall* nf t h i ~ trachniqrlr. An ruccllr~nt Imt~kt h ~ fullv t dt.\cril.rs thv mrthc~tli\ !hat Atr<~~~t~er Us. 1 tl,yrr I$ S I ~ ? ~ A ~J ~I + /TI J! IFL!~~S). II~~

C'rvrilrrt t tl~r' I~rr~rrrrnlt' nijr~l!/.;r~ This will cx;lrnlnin tht. rcla tir>nship bt9trvt.r.n tht. rnrlrptmdtv~tand dt.pcn~lent x.ar~;lfilr.( c . ~ ~ t w d c s rand iilcnrn~,). 'l tiis . ~ n!a I s i s might be i r l t h ~ fnrm , nl corrtalat ~r,n+,t;llll(a.;or

204

CIIOSS-SECTIONAL DESIGNS
Table 12.3 Zero-ordv [?nilcol?rlttronnl t a b l ~ s indlcnliri~ spr,cjfrratiol~
(a) Zcrrj-ordrr tahle

comparison of means (Chapter 6).The rclationship between these two variables is called the zrro-ordt7r relatronship. Mvlfifil r p ! p i U ~ l l t rmltrnl -r~nrinblr, ltltr third iwrinhir~) This is the variable on w h r h you are trying to match thc groups (as defined by the categories of thc indeppcndent variable). It is often callcd the test variable or the Z variable. Your selection of the third variable will be on the basis of theoreticaI modeEs and what you expect could be responsible for the zero-order relationships. This will depend on whether you are proposing that the zero-order relationship is spurious (sec below) or arc propusing an indirect causal relationship (see below). Drdw a r~rodrlof the rclatinnship you are proposing between the three variables (see discussion below). Undeutaki> HIP ~ n a l y s i s This may be in the form of tables, and/or correlations, or comparison of means. Essentially the analysis involves looking a t the rclationship bytween the initial two variable? (X and Y ) separately for each category of the control variabIc (1). These rela honships are called conditic~nalrela tionships. Infcrprrf the rcsrrll< Ways r ) l interpreting differell t results are

zero-order relationship ni gendcr and incnme

I
Low
Inct~rnr

Male

Female

High

fi) Cnnd~tionalhblcs

rt.lat~onr;hir of g ~ n d r r and Income cot~trollingfor occupition

Male

Fernare

Low

discussed below.

There are four main ways of thinking nbout and interpreting the results of elaboration analysis, as follows.

Ln this approach we begin with a zero-order relationship and then specify the particuIar suhgrnups to which it applies and does not apply. For example, there is a general relationship behvren gcnder and income as illustrated in the zero-order cross-tabulation in Table 12.3 there is a greater percentage of malcs than fcmaaIes who are high income earners. Mowcv~r, when we control for occupation (blue colla~ and white collar), and look at both occupational groupings separately, wc find that thc zcrv-order relationship docs not hold consistently within cach occupational group. The separate tables for bIuc coJlar and white collar dre called coniiif imml tnblcs: they show the relatiomhip bctwcen gcndcr and incomc rindcr different conditions (blue collar condition a n d white collar conditinn). The initial pattern of women doing worst pcrsish within the blue collar workers but not ainong thc white collar workers. TIIE zeroorder relationship seems to he specific to particular cwcupational group4 rather than general to all groups. Whert. thr initial relationship holds for sume categories of the contrn! variable (occupatirm in this case) but not others, the independent variable and the control variablc a r e said to i n t ~ r i ~ that l : is, &c rumhinotion of the two variables can creatc a particular effect ( i r.. being female a ~ r d bkue

collar) that neithcr on its own creates. Tl~e concept of interaction has been discussed earlier in this book {Chapter 4).

Replication occurs where the zero-order rclationship persists in ench category of the control variable. Where this occurs then the control variable is not responsibIe for the initial 7ero-ordcr relationship. In thc exampre in TabIc 12.4 location (rural and urban) has been cnntrolled. When comparing males and females in general (the zero-order table) there is n 30 per cent gap betwccn males and females (60 per cent males vcrsus 3 1 3 per cent females have a high income). The same pattern persists among rural men and women and also among urban men and women. In other words the zero-order relationship is replicated within each category of the control variable. When ihis pattern occurs the zero-order reIationship is said not to be due to the control variable.

206

CROSS-SECTIONAL DESIGNS
Controlling for hours
Zero order relationsh~p Hours of work

Gender

Income level

Gender

lnctlmc lcvcl

(2)

Zero-clrdcr tnble: ~ e r n - o r d e r rdat~rtn+hip of gciirlcr and income

RII
Male

Gender
Female

Low

Inconir
Iliglr

(h) Conditional rablcs: rrlatinnship uf ~ n d e snrl r income contmlling fur lt~c~ticm

lncnmc

High

Gender Male
I..ow
Femalc

initial relationship was due to the hour5 of work variable as represented in the figure. This is also illustrated in the cross-tabulations in Table 12.5.In this case tlie zero-order relationship shows that a greater percentage of men than women are high income earners (55 pcr cent versus 40 per cent). The cross-tabulation for part timers shows that virtually the same percentage of men as women are high income earners (31 per cent versus 33 per cent). Similarly, among the frdl timers, the same percentage :ef men and uf women are high income earners (65 per cent versus 65 per cent). In 0t11~r words, in this hypnihctical example, when men and women have comparable hor~rsof work the income differences between men and women disappear. T1-2.czero-order relationship is due to thc relationsh~p betwecn gci-rder and hours of work and t h e relationship behnre~nhours of work and income, The fact that there were far more part time w o m m than part time men in thc sample generated the correlation between gender and income. Had the sample had the same number of past time men and women then the initial relationship between gunder and income would not have appedred.

Income

High

TND~RECT

CAUSAL RELATIONSHIPS

Elaboration analysis can also be used to clarify the nnturc. of the xeroorder relatinnqhip. In this case we might be asserting that gender casually affects income. But we might want to know by what mechanisms it has this effect. Dofs gender affect income because gender affects hours of work, which, in turn, affects income (Figure 12.1)? Herc we are controlling for hours of work in order to see if the relationship between huurs of work and gender is due tu the relationship hrh~rcn gender and hours OF work alld the relationship between liours of work and income. If the relationship between gendcr andfincome disappears anlnng those with cnrnpar~lbl~ hours of work [hen we would say that the

A spurious relationship is one whcrc the correlation b e t w ~ c ntwo variables is not because of any causal relationship between them hut because both the variables are related to, or outcomes of, some third variable (Chapter 3). For example, let us suppose that people who attend religious scl~oolsare more religious than those who attend secular schools. If this reiationship bccause religious schools make people more religious or is it because both factors are produced by some third factor? L n tliis exarnplc the modcl is proposing that the reason for the rclationship between type of school and religiousness is trot because one cailscs thc othvr but becausc religious parents arc more likely than nonreligious parents to send their duldren to reli*ous schools (Figure 12.2). Religious parents are also more likely to raise religious cl~ildren. Thus the reason whv young pecrple in religious schools are more religini~s than those in secular schools is because religious parents sent their children tu such schools nnd raise religious children. Resulk such as those in Table 12.6 wauld be cnnsistcnt with this interpretation. The initia! relationship disappears when .controlling for

zoe

CROSS-SECTIONAL DESIGNS

CROSS-SECTIONAI. ANALYSIS

209

rable 12.6 Zcm-nrdpr

(a)

nlrrt

nlr~drtlunnltnbll,s ~llrrstrrrlti~g a splrrntIrs rrlrztronchtp

(a) Zcrtrt~rdcr tahle zero-ordcr rekitirmsh~p o f schovE t>pea n d rcllpo~rsn-s

Zero-ordrr [able. z~rc-ordcrrchtlonsh~p of gcndcr and inctJrnp

All

Type of s c l ~ w l
lleliginus

Gender
Secular
Male

Female
Child's religionsness

Low

Income
High

High

(b)Condltlonal tables relalionshlp of gcndcr and lncome contrulllnp, for liours of work

(a) cnndltional tdhIcs: reIat~an.;hip of school type and religinulncs.; ctmtrolling for parental
rcliginusnms

I .ow Incotnc
High

Law Cliild's rcligic)usnesc

High

Ftrll i ~ n r r Male

Gender

~ ' O M - W ! I ~ Iprlrcnts OS

lype of school
Religious Sccular

Female

Low
Incr)mc

6 5'%,
Child's
153,

822

I31gh

18"h

N
-

Controlling for parental religiousness

School type (religious or

Parental

parental religiousness. Among those with religious parents we see the qnme level of reli~inusnessamong chi4dren regardless of thc type of school attended: 65 per cent of those children in religious schnols were highIv religious compared to 63 per cent of tllose in secular schools. The same patteyn 1s repeated among those with non-relipjous prlrcnts: 15 prr cent of children attending religious schnols were rcligieus compared to 18 pcr ccnt of thosc in secular schools.
SLIMMARYOF
INTERI'KE'CATION OF DIFFERENT PATTERNS

religiousness

F i p r e 12.2 MorJr,I~irlircnfir(q~l~drrect r r l n l ~ o n ~ l ; ktalren tp sdrod /ypc and stridr~rtrc.lrgror~rnt~~p

The strategv in elabnrdtion analysis is to compare the pattern in the wroorder relationship with that in the conditional rela tionships. 711e

Multivarjahle analysis

r
15

t
not
L ~ U P to t t . 4

i
varr,ll?lr

X-+Y

J2

X-.l'

Inter,irtic ~ r r b!tu-n-n 7 , S-.)' indrfrmrii-nl imari4\blr and 7 Y Y !ch?t u.iri;rblt.; X nnlv . ~ t t r r ' f > I under hptr-iiic t - ~ n % t i lk; w

I-Tabnratinn annly\r< f~ a 5irnplt. form of multivarinte anah <I<. I t ukuallv ft~t~% cjn eq thr. rolr of a third z ~ . ~ r i a lin ~lr ilnder.;tandinq an initial bivatlate rc2ation41ip. Othcr lornls nf rn~rltiv~iri,lkc analvs~ . ;~ LI~~ n.; I partial corrclatiun, rnzaltiple regresrion, dixcr~minant analyqiq, path nnalvsi.: and log-ltnc,is ,lnalys15 allow us ti3 corltrnl ftlr a n r ~ m h e rnf nthcr var~ablt=s slnl~~ltaneo~irl iTariclu.; y. forms elf rnulti\bariaSt! a n ~ l y s i scan tlnal-rlc po~rrerftt!and ~ n f n r m ~ t i v c~ e planatorv analysis of crnss-sectional datn
\%'ccan rcmo1.t the confot~ndin:: cfftact.:of GI $14c r f \~.lriahle.; at k3iire tlitb 'ptrre' et'lec~o f nn indrpcndrnt variablc I ht. mtre variah1t.s we can control i ~ r r a t oncc the more valid our cimprr.;on of gruup.. bt*comrq,and thc* morc- confident rvc can hc that ,In> uhscn.cd t~fft-ct of an lndvpcndent \arrablc is duel to that vnsi~blts and not the ccrnfcrr~ndinq efl~hct.;o f r~tli~ \.ariabl~s. r 7 Thew rn~~ltrvari~itcr analysis tcchniclut~.: can tell u.; abi>ul thi* jrrrfli rfh>ct tlf n wt nf i n ~ l c p c n ~ l m variables. l for ~ ; l set c ~ varirlhlcs f ~ r m u l t n n m ~ u s thew lv rnrrltit a r ~ a t r 3 13y r o n t r r ~ l l i n techniq~rr (wable ~ 11st o ta\antlnr bl~c ~l~sc.rctr> ebtrct ot t * x h .irr.ilili. rvtthin a ret c ~ frndtaprndcnt varrst-rlt~. 12'hilc ~ u n d c r t,d u c a t i o n ancl hi~ui-3n S ~vtrrkma)' each h;l\.c sunlc rlffect 011 ilicnrnc, rnultfvariatr~ nnalvsis isn;lPlc<us to cxplnrr. how mrlch c,lcli rmr ha5 indt-pendt-ntly of tht* others. This str,itt.gy mahles us to ~ d r n t i f y the rrlillrz~ij~rrr/)r>rlntrrr* n!' each o f a st*t o f indcpcndcnt ~,ariahlt.%. 1 h411lti1.ariattb techniqut- t,na'tlle the t~sting of c a 1 1 ~ r?;rrrlel2 as t\.t>ll a.: thr impact o f s!nglt. ~ ~ ~ r i a b R lg ebst t > ~ t i nn q i d e l thtwe ~ t r ~ c h n i q i #can ~% bt* cffcct~vc. in Iiclpfn): idcnlify c r l u ~ ~~~~~~~~r.: ~l or at Ic.a.;l in evaluating how well a m o r l ~ lf ~ t a s siht nf cri~ss-sectinnal rlntn.
1

3 L'

I nnrl t-t t ariahl~ prt,ctutt-c h > t h Y and in !!me

1
t i

Spur~r>r~cncl;+

;~nd focus 1j11

cnmparistsn p r o ~ i d c * the kry tcl interpreting thc wav in which tlic ccmtrol variablr affects the n n ~ i n a irt'Idlion~h~p. Table 12.7 prnirirjes n simple s u m m a n nf M - R ~ Sto interpret the' cornpa~isonbl't~vn~~i ~~TC~-C)T rcla~CT tionships a n d cclnditional relntionships.

Thr ~ l ~ b n r a t i n model n ha%a nurnbrr elf prrs2llcm%that can limit its u w fulness a4 a tz7av of contsnlling ft,r varinbli~q in crosh-wctional analv5iq. h rnajnt prol~lcin is that this sort uf malcliing rcquirrs very Inrgr samplt>s. Bccause it involvt*~subdividing the snrnplc ~ n t osrparntr groups for each catrgnry of tlie test varrabIc rve can quit-kly t n d lrp with quite small ~ ( w for p an.?ly.;is. If thba te%t \*arrat.l~has, <a? fivr caleqoric., caili subcroup c;En t-e ~m~11. I f wc rzant to match fnr h v o \.ariallle\ a t once thi.; probFem is compcwnde~d. 7 he iliificullics crc,ated bv runnrng nut of c a w s ;IrcL so wriot!q thal this form t r f nnalvsrs iq lim~trri Frt~qucntZvto r n a t c h i n ~ cln just onc v a r l ~ h l e nt a time. \ < rvtall .I< rttnnrrlg ntrt oi cnq= thc rr<trlts o b t ~ i n e d from t*labr)r.itir>n ~n;llyst'h can tlrcnme vrrjm complex and cfifficult to ~ntcqlret.\Then 1r.r Iia\-rn man! diflcrrilt cr~nditicma l tatlle.. thc logic invnlvcd in ~nterprt*tinc tliv vt~ryin):pa t t ~ r n s (,in I,ert-lmr \*crycornp2vu. It unn heccsmi. ex treniel v dhfticillt to taupla~n and prtvqtlnt tlie resirltq of anvthing but the quitc I q~rnplc tlYnhrrrat~on analy\e<

Thcrt* is in.;ufEicir*nt qpace hpre to descriht* tht. rnultiv,iriatc fcshniqtrt><.XLin]: simpltl yct sr~phi5ticatcdintroduction< are available ( K e r l i n ~ c and s Pedfta7izr. I(T3; T~lrachnick and Fidell, 1Q89, Grimnl and Yamold, lLW5).

Cohort analysis

Cross-~c.iticlnal rlcsii:n5 art, Ijmitcd in thr extent t 1 5 rvhiih they can pruv~dr~nfnrmation about c h r ~ 1 g trx e r t ~ m t I ~lm\,evcr, . reptattad ~-ro-<cctinnnl < t l ~ d i e scan yrm-idc valuahlt. in<iellt.~ a b i l ~ t tchanqe Cohurt annlvsi'i can I I ~ C J ~ ~ ~ i of i ' rrpeated < cros+\ecticinal .;tutlic.; to drscrrhe o,ygctrp~rfr c h a n ~ colfer trmr and to Fdrntif\p the cxlmt tu which this change is attribtttablc to pt'ri~d cttrcts, agclnj: ,~nd cohort c*ffrcts [C'hnptvr 7).

212

CROSS-SECTIONAL DESIGNS

CROSS-SECTIONAL ANALYSIS

213

For example we might want to study changes in church attendance. what Glenn (1977) calls a sfnndn~ii This can be achieved by constr~icting ccrhurt Inble.
Year

nf data colIection

Construdinx und rendins cohort tables

When a series of cql~ally spaced surveys (e.g.a population census every 10 years) has been conducted, a standard cohort table can be constructed. In order to construct the table wc would establish an age cc/iorf spun that is the same as the interval between each survey. Thus i f data were age cohorts of 20-29, 30-39, collected every 10 years we might cstablisl~ 40-49 etc. The table would consist of age cohorts down the side and data collection years across the tnp. In each cell of the table we would insert the relevant information that applied to the particular age ccnhort in the particular year. If the dependent variable was church attendance, each cell would contain some measure of church attendance (e.g. percentage the average numbcr of times people m attending church a t least montl~ly, that age group attend church each year, a measure of variance regarding church attendance or some other way of summarizing church attendance) for the relevant agc cohort for the particular study year (sec Tables 12.8 to 12.10). We can then read the standard cohort table in three different ways to detect different patterns of change. Clear period effects are evident if, within any particuIar year, there are no differences between the age groups (i.e. within a column) but there are differences between each column. In Table 12.9 we see that in 1Y70 all agc groups had the same proportion of r e ~ J a church r atkenders. Age groups were indistinguishable. This irrelevance of age persists in every decade. Period effeck are evident by looking a t the trend across cnlumns. In cach decade the proportion of regular atlenders changes, and it changes in an identical way for each age group. A n examination of these p e r i d effects maables us to see how people within the same age group differ at different periods. Thus in Table 12.9 we can see that people in their 20s in 1970 are quite different from those in their 20s a t the turn of the century. The age group is constant: the penod in which this age group is living is different.

Ageing effects or developmental effects are thosc where change is attributable to people growing older. These can be detected by reading thr standard cohort table diagonally-from top left to bottom right to identify lntr~coho~t trends. Reading down the diagonals of the table enables US to see how a particular age cohort changes as it ages (Table 2.8). Iiow have people who were in their 20s in 1970 changed as an age group as t h y havc moved into their 30s in 1980, into the 40s by 1390, and into their 505 by the turn of the century? Table 12.8 shows perfect ageing cffccts. Tcn per cent uf the cohort of 20 year olds in 1970 was attending church a t least monthly. Ten years later, when this c o h c ~ would t have been 30 or so years of age, the church attendance of this cohort increased to 15 per cent. Another 10 years later (1990) this cohort - now in i t s 40s - displays a further increase in church attendance. The same ageing pattern can be seen in any left to right dnu~nward diagonal. The concluston one would draw is that as people get older they increase their church attendance. This is consistent regardless of the 1 years one examines, or the age group examined.

Cohort effects arc cvident in Table 12.10. That is, wit hi^ a l l y grzlun vpnr tlie a ~ groups e have different proportions of regular attendcrs: the age group makes a clifftrencc to the Ievcl of frequent attcnders.

214

CKOSS-SECTIONAL DESIGNS

CROSS-SECTIONAL ANALYSIS

215

rl~~rrcl~ nl I ~ m t rwu?i!lrly by ngc Tablc 12.9 Colrcrrl table ofpercrntlrfi, flttc~rdirrx culiort a ~ Ilcnr, ~ d 1 L?711-2110!1: p e r i ~ ~ E ~~JtC Z S

I I

Cohort effects are evident where there is change within a column and within the rows, and the direction of change in CI column (c.g. from low to high per cent) is opposite to the direction OF change w i t h ~ n the rows (e.g. from high to low per cent). M i l e a cnhort table can be very heIpf~tlin identifying ageing, period and cohort effects there is an important difficulty with this approach. It is impossibIc to completely isolate a 'purehaging, period or cohort effect (Glenn, 1977: 130). Any table that demonstrates one kypc of effect invariably display? one other type of effect. For example, Table 12.8, which was designed to iIlustrate ageing effects (change in the diagonal), also di~plays cohort effects (change in cnlumn). TabIe 12.9, designed to display period effects (change across columns), also displays ageing effects. Tablc 12.10 was designed to display cohort effects but aIso displays period effects. While i t is not possible to disaggregate these effccts we can, ncverthelsss, s ~ the c flbs~ncr of particular effects and the ycjssihility of other effects.

TabTe12.10 Cn~iort~nbleofperrmtn~eutt~ndzngchurrhatIm~tmnthl~~h~~~~t. cohort and ypau, 2970-2000. cohort tjftjcls

I
I

Unfortunately, we do not always have tlie luxury of studies being conducted at cvui~Iyspaced or cotlvcnicnt time Lntcrvals. Whcn these conditions do nnt prevail we need to modify the way in which tlie cohort tabEe and the age cohorts are constructed. This is illustrated in Table 12.11. To construct this table we first establish our cohort widths for the first year, 1979 (in this case 10-year age bands). For the ycar of the next data colIection (1984) wc wiIl construct age cohorts of the same age space (i.e. 3 0 years) but the lower and upper limits of the cohort will be adjusted for the interval between the data collcrtions. Thc same procedure would be used to adjust coharts between each data collection phase. For cxampIe, cohort 1 in 1979 comprised the 20-29 age group (10-y~ar span). The next data collection was 1985 (five years later). To track the initial 20-24 year old cohort five years later wc identify Lhc 25-34 age group in 1984. The comparable cohort two years later in 1986 will be 2736. T h w years later, in 1989, the comparable cohort will be 30-39 years. The basic strategy entails establishing a fixcd agc span for thc cnhuri and lower and upper ages for f ~ cohort e equivalent to the then adjusting tl~c gap hc!wcen the data collections. This cnliort table lias to be read a little diff~rently from thr standard cohort table:
1 To identify ageing effects we read orross rows rather than down a cnhort diagonal. T n this case we can see an ageing effect as the percentage of regular attenders increases as w e track the agc cohort

across successive surveys.

Summary

ch,~l.trr ha.; providcd guic~elmras fnr thc dcscripti\'e and prplanatory , ~ n a ~ y ~ i i-f, datGi gcncralrd by croqs-?t>clinnal Arsigns. It has not sctnsirlcrcd the spclclfic tcshniqur;, but manv of thc .;tahstica! $;.l~~dcliiics
l'hlq

PART V

pr~ivirlcdIn Ch,~ptes6 apply to croqc-sectioiial analvs~s. The ctzapter consid~rcd the particular problems of using crom.t.ctiunnl data for causal nn;llysi.; - pmblems- creattd bv the absence nf J tirnc dimension to the d e * j p and the lack of rnndomi7ed control groups The genera! wav nf dealing with thcsc prohlcms is a t the analysis s t a ~ c r a h r ttian at thr r c s x ~ r c l rlrsip i .;tag?. Thc chapter otltlined thc. loqic uf ilsing ~ t ~ l t ~ s t icontrols cal and rnultiv;lr~ate ana1ysis r;o that cm.;~-wct~clnnl data c.in bc uwCul for ci-tuqal analysis. tt tllrn described on(-particulnr form rrF rnultit~arintean,llvsis - elahoratinn analyqis - to prilvidc an itnderct,lndin~ ctt the \ r a w in which the pmblrms of cauqal annlvqi~ cnn 1~ tacklta~l.Finally, the chapter d ~wussedn rnetlioil oi a n a l y i n g d a t a f rmn rrgptba tcd crow-section~ls t l ~ ips. d In p.irticular r t fc~uscilon cohrwt nnalvsr> - how to conqkr~ict and intcrprct ct~hclrttahl~.< and tn be awnre c ~ Ihc f 111nihctl t h i ~ typc of analys~z.

CASE, STUDY DESIGNS

CASE STUDY DESIGN

For rn~inv~ ~ ~ 1 1t . h 5 c ~a w *iudy h a s been Ihc rlglv ducklling of rt.sr>arrh d e s i ~ nMust rtwurch rncthudu t r r t 5 eithtbr ignore c6i*csturli~~+ or cotifuse [hen1 \ \ , Ith ntlit-r tvpcs o t 5 ~ ~ ; i rt-*tvrcI~. al \I Iwn cawa *tud! JL,+~LW~ 11~1t.e been disrussed thry havr generally been scrn - trom a r n r t h u d o l o ~ ~ c a l p i n t nl view - as 'soft'options %me curnmrntatca< believe* cnse 5 t u ~icq l +auld bc w e d rmly for cxplora tcsv research: k e genera tc Iir.pothesrs tnr futurc tcqting ~ 'th i rnLJrv r1gor011rrcsearch designs. The rnrthodoloby of c a w study design has not bccn neglcctrd because c a w stud ics h,>\rt, bccn ~ ~ n i m p o r t nin r ~~ t n c i a scicnccq. l Indcrd they have been f~uldarnt*nt,llto thy <tlbstantirfc 2nd mcthodolo~ical dct'clnprnent o f the st~cia l scicr~ct~s. In hcrial anlhropology studies of trihrs linve bfcn c a w studies prrr t,.rrr/lrrlcin.Community .itudies sirch as the Lvnds' h4iddlr,to:l*~1 /lu?ql and small GrOUF s t ~ ~ d i ~ like . ; M'hyte'r; 5trr7t.t Corrrrr S o o r f v (1943) have rnacir major crmtributions to lllc development of ii(7ci~~Iogy as R d iscipIiiir. P~yc'l~onnrlTysi~: ha4 thrivcd on a case shtdy approach (f r e d , l Q 5 ) Educativnal r e c a r r h , cvalnahon rcwarch and organi~ntionalrtksearch haire all made ~ r t m s i v e 11qr of caw studim to instcr f h ~ i de~cli>prnen r t. The I ~ c k i3f a - \ sbernatic- rll~cu.;.;ifm of w.t. studr. d r > i v q 11.1q hequn to be a d ~ l r t w e d In rtacent trrnr>\. I'in ( 1 'J89; 7Qq3)h a < provided ,I pnrticuIarl\useful tl-catrn~nt of casc \ ~ L I C I V Jrasigns ;lnd .I porvi,rful dcfrwce of their
value.

This rhaptcr rramincc n range of w ~ v s of dcvel~ryinc:case ~ ~ I I J V d e s i ~ q Thi5 . i< a difftnrrmt task to that ~xndcrtakcrl for cuprrimental, loneihlclinal nncl crc>ss-\r~ction~l ~lrqignsI~iyn~r.;e, ns Yin points nut, 'unirkc 4,tht.r sr.;parch strr7tept,s, the prjtcntial "c,rtalog" r r f rewnrch desi~n.;has n d !,t*t been dtnc.elript>d'(1Q.q'): 27).

CASE '4 UDY 171.51GNS

CASE q1'UUY DIISIGN

A caqc I K tht 'i~blect't ~ f study. I t is thc unit ~ n f analysis (we Chapter 2 ) about tvhich tvc collect inforrn~tion.In c~lse stlady desiqnc it is the unit that r \ r seek trr ~nderstanda s a w h i d ~The . unlt of analvsiq mav he a person about whom we trv to build up an understanding that is ~nfnrmra~i by thc ~.rrntcr!in which the wholc caxr ruistq. Rut c a w studicr arc. not rcstrictd to 111Ai15irlrrnl~. \ V r cam scltct man!, other Ivpes nf c~lscs- units cbf analysis. A marriagp, a famiFv or R I1oust4iofd m.qv serve a 5 a caw. Plnrvs si~cha.; a black of linl~ses,n resldcntlal cornmunrh.. ; l region o r a cotlrltn- can all scnrc.I.; caw>,a< can argnn~:i~tiaprs SLECII a s a busincs~, R school, government dppartmrnt or a union A case rntght bc an r ~ ~ ~ quch i r fas a rl~vorcr rather thnn t l pcrsnn ~ ~ I\-ho divnrce5. 12ltcrnntivrly, a rl~~c.rsior~ ( c . dcci~ictns ~ a b m t dcrrunqi7ing a n organization) mzght bc the r m 1~nf an,llysis for the casc xtudv. Tlir caw ~tudv ccruld lnvol\,e understanding thc clccision a3 a wholc, eiarnining thc prnccss b\. i \ . h ~ t h i t \\.as rnadc, the particlyanfq. the conqequcncw etc 'T~nlr ~ x ~ ~ might m i q b r thc unlt 4)F analyis. Wc might s t ~ ~ d tlw y lYfiO~. o r thv !()YO.; t1ar.l1 n i t h t w c ~ l n bc tlrr '!!lint;' .~llouttvh~chwc c,-r>llect infom,~tinn.

A wt*II-desigiird caw s h d y will av(11tI examining lust somc nt thc constit~~c,nt t.lcm~*nss. It will build up a picturc rd the c;l<ixbv taking lntc, account ~nformnt~im p i n t 4 horn many Icvels. The final CAW sZ~~il\. will tcll us nlclre than, and snrncthlng qualit,~tive\y different froin, that which a n y ct>nslituentelement of the c,~.;c could tell 11s. 111 the c a w of a school, the i n 4 ; ~ h t w s i n d from shrdcnts, teachcrs. parcnts, a d n l ~ n i s t r ~ t rand w communi& mrmtrers ~ t ' i l l probahl!r differ and, when taken trrgcther, proviclr a much fuller, rntjrc complex understantling of tlir who]!>than would tlic pempcctlvc prnvided bv any particular element of thc c a w . Ihc ~ ~ I i ( > IS le grixater than the <urn ot its parts. Sinct- many c,i\cs will consist of diffrrc~nt elcmtank, diffrrent n~t-thot.ls u i d a t a collectitm mav br requircd for thc t-! ifierer~telement.;. A siir\.cv of ktudentq might be apprtlpriatr; c>bst,rv;ltion of classror~msand qtaff nicetin$'; might nlso bp wc~rthwhi!e;wliilc interviews rntght. bp n ~ o o d way nt KninIne ~ n f n m n t i o n fmm t~achcrq.An a n ~ l v sof ~~ &ool ; nrnrds and archives ctn~ldprnvide utcful infomatrt>rl about thc Iiiktorical cnntext within whrch thc schotd operatrbs.

Case studies and theory

,\ l thnugli somv case study rescarcIicr3 ccmduct c-asc sturl i r s a.: 111uugh ihey o ~ i l v have tn colleri tlie facts about the caw and writ^. abnut thcse in an c n y n l n g wav, the task of t h r case studv reqenrchcr 1.; fundamcntallv theor~tiinl. Coll~=cting and anal !sing informa tion from case stud i r q must bc g ~ ~ i d rby r l thrrwy.

It is htblpful to disting~~ish betwcen ca*eq as ;Y whole a n d rasps h a t cnnsist nf varinus levcls or components. Yin (1'189) ~rscs thc terms 'l~olistic' and 'embedded' d e s i m s to refer to this Jiqtinctinn. Some cases cnnsist ni multiple lcvels or components. For example, a school as a case includps teacliing staff, administrative staff, staff at different level5 of seniori? and experience, students, students differcnt year levels, parents, go\.ernrnent and community members and so forth. A school can be coiiceived of at thc 'holistic' level where wtLfor115 on charncterir;tic~ of the school that apply to that l e ~ c lThr . school exists as an c n t ~ t !and ~ has schrml le\.ct characteristics ( t . . ~ size, . type of school, locahcln, culturc nf school, a s ~ of t rules, a struchlrc, a svstrm of rnana~crncnt,a qchool phi losophv, qtrmgt lis and weaknc~wsi. RLIt thew ;Ire a l v ) man\, .;tlbler-ch or elempnks ttl, ihc- sclrool. A full p i c h ~ r r irt t h r cchool in all ~ t \cornplcxitv wclt~ld snly be uhi.~ined uf we collcctcd ~t~forrn titm a from a w ~ d ranee r of the constiturnt elemrnts (ernbcd~led ttnib) nf the largrr untt. % : rnasri;lRe proc idcs ,inother rxamplr. A t thp linlistic Ii~vcIMT would treat thc m ~ r s i a i :a~ s a whole in trrms of things \llch as i t + length, stage. type, Ir)\.clof cnnil~ct and intimar\', equality. rncdt-s o i int~ractinn rtc. \Ye could a l w treat Oir huqhand and the w~tt* as rrnbt>tlded u n ~ t s and hutld tip a ~ L I C I Ifulltnr and diftcrent plrturc nf the m a r r i , ~ ~ bv e incorporating the c-xplarrcncecand p ~ r ~ p c c tof i \the ~ ~11~1.bnntl ~ and ;he t ' v t f r (szrt.~tn~ts).

Case studv c l r s i ~ n s diffcr from tlic desihws discuwed earlit>rin this book In that thy\ seck to achic*v~ botll more cumplex a n d fuller t~uplanazion~ of phenomena. Thry scck to achicvc idiogmphic 3 s well nq nnmt,tllptic rvplanations (SPP Chapttbr 14). 'Tl~cv also differ in the wav in which they ~0 abn~lt t y i n g to achicvc c a u ~ a cbxylanaticms l (5cr Chapters 14 and 15). Caze \ t l ~ d yrcwarch in socia! ~ i e n c e s must have a thcorctical d imensmn. W ~ t h n ~ ai t theor~ticnldimcnqion a casc s t u d y will br of Iittlr, value tor ~ v ~ d c~enrral17a ,r ticm - one c-nt tliv goals of social ccienitl rcwarch. T h i ~ w c h o n vr)nsidrr%Ihscr I , nys in tlehich tlrrilry i s used in ~lt..;igrl~n~ cast.

5ludies.

This approach, which ic spcn by Yln 11'18q1 a s bring at Ihv hcast r r f cnsc ~tudirb ~c , ~ i n sivith a thecrrt,, clr a set ot rival ~hcorri>\, rcgardlng a particul,lr phennrncncm. On biw basis ot a theor!# w r p r ~ , J ~ that c t n case with a ~ a r t i c u l ~ set ~ r of rharactcr~stics ill h a w .I particular ntltc'ome. T!lr t l i t ~ r 3 r vma\ br sirnylt, or cnrnl.leu.

111
L--

CASE 57 U13t DESIGN5

C'ASF STUDY I1I:SIC;N

27.3

f'or ruarnplc~, tht- c a w ?hzcly mrglit begin ueith tht. talloxc~ngqutb>t~on: is the cjfr3it ~ l de\-crlvcrl, i c i h r ~ base2 ~l icmtrnl of staffing on tlic qt~alityof e d u c ; ~ t ~ nin n a scIio~17'Dt'\~oll,rd or ~cI11>id hasrd staffing
'\\'?l;lt

cc~ntsulinvolvch 13~ich <chon1 appili nting, dism k i n g ,tnd promoling stalf a s well as scttlng pay levels ant1 Icnchtng conditiitns. Devolvtd qtaffing svstcms can br contrasted wit11 cmtraFi7ed systcrnq where public service burt=aucrats managc these staff in^ rnJttcrs at head ofiice l e v ~ l Wc . rn~ght I7ccln rvith thc prrrpr~Gtiont h t ~ Jr\'nl~.edcontrol will yir.Id much Iwttpr r d ilea tional nutcumcs than c e n t r a l v d contm!. T l ~ c rcnqoning \vnitld be i lint Iclcal conlrol makes peoplc h r more accorrntablc, enables a schnnl to build a staff prr)Filc appropriate to its necds, and makes it much ~ n s i c r to rrward qtaff wht, nrc a c h i e v i n ~ the- oiitcornes dcsirrd bv the schord. The ~ n ~ p l i cthconp it hcrc is that edizcatinnal clualih iq fundn&entallr. a rcsult of t ~ n c h e quality r Ins opposed to thc nature of studcn ts, school rP\cllr re[.<, nr~,lnitationals t n ~ c t u r eparcntal , ini.olvemmt) ,-inti !hat teachcr q u a l i b iz a function of tmchcr accoz~ntnhility and teacher 'fit' with t h schtjol. ~ The selectinn o C a case to test this theory would 11. i111 u s kr, find a sclioul tlrnt has intrtld~icccl'I cievolvcd staffing system. 1x1, thnroughlv invt~qtig d ~ n f the i school (including thc 'cmhrdded elcrnrlit?' such a s tpnc'hcrq, man~tycmcnt, partlnt.;, past and prthwntstlrdrnf5) \rfc 1vnu2d buqF(1 u p ? , pictlir~ of t11~ rlu;lltty irf edur.-ltioii In the schcrol (houueverdcfintrll ; ~ n d n i ~ out l ~ ~ any l111L.s [hat exist bc*twt-ci~ the cducatinnal q u a l ~ l y, ~ n %hc ~l staffing systpm 11 pi~qsibIe w~ wtn~lrl collect histnric,ll data re la tin^ to the timtl before tlic dt*vnlved s y ~ t c n l wa5 introductld lo see if tlicrc w , ~ s c\r~rlcncc of improvement after it\ introductic~n Wtn ivcluld also qcck to ~dcntifyw h a t r.l.;r% happen in^ In the schnol that might hay[* Icd to irnprt~r.erncn!s In thr yualihf of education err factom that r n i ~ l i thave prc\,cntcd imprnvements dcqpitr the introduction of the rlevolibcd
systpm.

RegLtr~ITt-+of the c r , r i ~ l i l t v * of~ ~ tlw tl~ct~rv this , ; ~ ~ ~ r oto ~i case cl~ studit- bt,crri.; with i lC.C~ 1+k ~ - \ ~ ) ~ r t a t r il ~r~ . ri~ i\ *, ~ Fr~lrn d prr.\ trru5 rwcarch and /nr thc~rricq. lye approach t h e c a w y t t ~ d vwith thc purptr.;c nf test in^ OUT tl~txor! in a reaj l i l t > <ituation. Clc,?rlv the selection t ~ fthr real life situation (tlw cast) wnuld nr.cd to match thc c a n d i t ~ o n s undtxr which the

theory priipcnes particular r.rutcomus (Chapter 74).

THEDRY Iil'll

n f Y G CASE V r t r n I F 5

us in^ ; I thc=tjrvbuild in^ appr<x~ch tn c a w rturiics we seltxct cnscs to help

Jevclop and refine tllc prr~p.,m~tir>ns and d ~ v c l o p a tllcor!. that fits the case5 wiL\ttrtly. For exaniy-rl~., cvc migh l .;larl with thc silnplt. pi.oposition that this intmductirln drvnlved svstc\nlr wi I t irnprrwc. cd iicationnl nutcom~.;. LVc might .;c*Ft~-t a schmd rvlwrc 5uch a .;vstcm has Lwcn intrcducrJ and find that t*rl~~caticmal outcrvncq did, in f;lcf, improve. W e rn~glitthm find a scl~rx)lrn which [fir inttndrrctivn (]I fht. de\~ol\'ed staffing \vstcm did t ~ o f~>rt>duce better educat ic~naln ~ ~ l c n m rIl'hy r . did i t producc grind otltcomr7.i i l l wmt. cascs and not others? Our an,~lvsi.; of cnch c,l\r would aim tt, highli~lit clitfrrt~n~+t~s bt*hvt.cn cases 5%-hcrc i t did and Jiii nnt work. I h r analvqiq wntild , ~ l < o id~ntifv cornrnt~nal~ti~ls among r,twh+ \\ here rlcz.r>l\.t-d qt,+ttingna* ~ u c z c w f u Tand commn~r.~l itit,.; arnnng c,l.tn\ whcrc ~t 1x.n.; u n ~ ~ r c c c ~ s f ( tt, t .l ~ . t hc wav in which tlic -y.;tem was irnplrmtmt~d). Tlirsca case shidivs coi~ldthen bc used tcl dcvttlop a sct o f 1>ro1~o~i tion5 r ~ l > t ol thy ~ conditions and con tcxt rmder wliicl~dcvolvt.rl .rtakfing system.;, Ir,i~l t o impmvrmr>n t~ (Chapter 15). Rie Jiffcrcnce b~t\vrtln thc t h ~ o r y trsting and thc~rrv building a p p r c ~ a c h cis ~ that in thr fr~rrnerw e Irr4~r)l with a set c)f qt~itpspecif~c propc~qitic>n.; and thm qrc if thcsc wnrk in rrlal world .;it~tntions.Ln the theory building model wr begin with t ~ n l vn question and perhaps a basic propor;itinn, lnnk a t rt=al cases and rwri trp with '1 m ~ r w specific theon rrr s r t of propnsition.; it< n rcsrrlt r)! rtarnining ; r c t ~ a cascq. l

Tlie point nf tlic care s h ~ d v r n ) i ~ l dbe to qrc i f thc thenry actti;lllv worked in a r ~ a li l fc. situation. I f i t did work then thc theory i s supprwtcd (nvt proven) 1 f i t did not work thrn we wouId <ct,k tn understand, f r t m il c < ~ r c f anafvv.; ~tl (if the case, w ! ~ y thc predicted outcome f i e t t ~ r cifrtcatic~n) did not evcntuatc. Is it trccalise the theory 1s completely wrnng? IJclrs the thct,ry r r q t ~ i r e some rc?finernt.nt? Is lhc I hmry applicable only under specific c~rc~tmstsnces? In.;tead of thi+ wrnptc prnpnqftinn ive mie;ht dcvclclp a rnnre complcu modrl that spco flrlt what wc+ultl h ~ p p c nrr-ill1 t h t . intrcldrlctmn of n dt~\ol\-cd stakting stern. The rtrnd4.I rnlght a n t i c ~ p n t po5itil.e ~ effect< in p,irticnlar ar;pcch of educatior~al nutcomes and ncg.11ir.c e f f e cin ~ r,tht=r% It inight specify pnritive effvcts c~nlyafter a givrn ptriod of lirnt*, and only when thc d c v u l v t d systcm ;1. i r n p l ~ r n e n t ~in d n particular wnv (r..g, ! r ~ t hbu!l teachrr and cnrnrnunitv cor~sultahon,wrth appropriate. cliccks anrj balance?). In t)tller weord<,thz= thtw? xve spvcitv might nnttciy.~tc I h n ~when cond~tinn. ; i + H + C + D are met thvn thc devolved ?v.;tt=rn 1 ~ 1 1 1~ r h i c v c I n ~ p r r l r ~ r n e nin t s aspcctq f and l ' aftpr a p'crincf ot V ZivrArq.

Whilc tlic t w o above c a w stud y approachrs ,Ire rlrrory ~*c,ffirr*ri in that thc goal is to i ~ s c the case to trst, rchnc and ~+cvclop thenr~ti'al gcn~rali7ations, fh~l clrnical ca5r >tudv use:: theoricc: vrrv differentlt.. Clinical c a s t s t u d i ~ ;lrcx s ri~~i te . ~~tr~ Ttl !~ t USP ' ~ ~ther~r~tah to understand a c.;lst. Thc w.lrr in xvhich a r l ~ n i c ~ n deals n wlt'fi .I iiicnt illu5h.1ti.c t l i i q s ~ l of e cast. sttidy. 4 child mar V I ~ I a I p ~ v c h o l o ~ ~because .;t I r c is prrf(mning poorly at srlinol and h,i+ ht,~.nmed ~ s r u p l ~ The v ~ ~ tnqk . nf thc cl~nical psycl1c71uci<t is to work olrt w h a t is going c~nin this casc ,~ndw h y it is happ~nin<. t Z goor3 clit~ici~ln ~ 1 1 sfart 1 ~ ' t t 1 1~vmptams. wlrat is happening a t <chr?()!' In {\.hat nril,i.; of schm~l iq tht~clitld perfr3rniin.: ptrv-lyTThc. clinician rr.il! undt~trbttdlv ti.~vt.a rancp ot pi)<~ib!c e ~ ~ I i l n a t l t ,tor i i ~Forv schcml pt%rfrrrmance anrl d ~ ~ r u p t i v ~e EII;FI.IO n !I I schncll. ~ 5hr1 \\,ill gather

CASE 5-I'Ci131'DF.SlC;U

~nfc\rmatitbnt t j l ~ u l l ~ lip l ti p~clure d what I \ going tm.%hi,will J ~ * t . c l ~ ~ p liunchc\ and collect fiirth(~r information t o tcst thrsc hit~icli~?s. Thla p s t rlholoeir-t m.lv give tftv child variou.: tt9.;t< to establi.:h hi< intelliqc~ncc q~~otlcn!. a n d .>ny G P P C I ~ ~ I Ct ~ m i n Ji<abilitis. ~ Thr ps~*clioEo~ist d(w4 to scc ii a cngnitivc ta\plnnatinn crt thc learning dtll~cuftiesh t ~ Shc . may have thc r h ~ l d ' s eyesight or hearinq trsted to src if sight or nuditorv problems arc. contributing ta his learn~ny: disabilities. Thr psychologist rnavcvaluatcl h e child's rclatinnship sk~llh to establid~ if pmblems in this qPhere c n n t r i h t c to thr Ernrnlng difilc~rltic3.A I t ~ r n a t ~ v c lshc y rni~ht probe into thc child's familv rrlafir~n.;hipi, to sec i f st~mctfiingthat i < p i n g on ,it home IS creating probIerns at .;chooI. P e r h a p the child ~ n t l ll~c t e a c h ~do r nnt get otz , i n ~the l prol3lcn1s 5pring from ilicir rclatinn.;liip ~litficultit*~. It is also pcwihle that thc child is sufkring from anuirty, d~presGionor some other clinical svndrtlmr. The pvint I \ that the clrn~cian will havr ; l battcy id pt)tcntial e\planntlons for thca child\ prclhll*mq.Her task iq to build up a full picturc nl fhr~ rase so that she can c ~ a l u a t r which explanation best fits the Carts of tlrr case. Having correctly dtngnrrsed thta n n ~ u r eo r cnli\cl n f the cliilrl'r problem4 tht* clinician can hrgin trcatnwnt. ln th15ta\.lrnplc the ~i>a'i i< try undcr~tanrlthe caw ;lnd 41dvca prtilyl~*m for this ca\c. I he puq7nGr ls not to tt*<! or d t ~ v - l o jt~ l i t w r ~ ebut ~ to lt+t2 rbuisting t111~)rics. Thc clinician works m,ith p l a ~ ~ s i b rilL1l l ~ hypr>thcsi,s a n d prngrc.;aivcly collects information ti3 hclp sort crut which fits k t . The samr logic can ayplv In consultancv case shidic*. An org-ani~~tirm r n i ~ h be t sufferins a g t a t drat ofconflicl and FO@rrnnralr. A c c ~ n ~ u l t a n t may have ~1r.lnyc? of pnwihle cxplanation~for h i s and should g o a l w l ~ t the task of ci~llectinginformation about the organizaiiirn that can hrlp establish wllich of the rxplanahons fits t l i ~ particular ~ rirpnizatinn. 'Thc goal it: not tri dcvelup a thcnry of organiratir~nalconflict and moralr It~rq fyut 10 idcntih. tvhich o f .;c~vcral aIternnti\.ctheories rnakr- most 5enw in this pasticul;lr case.

Acceptance of cultural goals

Accepts means of ach~evrng the goals

Accepts

Rejects

but thi.4 cannot pc)ssitrly mean that the case study ccmsists of r.rlr,twf/~i~~cp abo~ttthc case. T o dcscribe e ~ e r v t h i nis ~ irnpossibtc: there must t xa
~CKU 5.

Urscription must nrjt, indwd cannot, be athmrchcal. ll!~ always selccl 2nd organizc t h ~ which t wc dtlscrib. Dcwriptions will h i ~ h l i g h t aspcctq of the c a w . I t will be more likc a paint in^ of A landscape than a pt~utc~qr~~ itpwill h , btl a n interpretation rathtnr than a mirr~,r image. lYrn r n ~ ~ hdo Z [hi< ~r+inyPYFITCFI tl~ct>ricsvr p r t ~ - t \ i s t i n crmc~ptual ~ catcp~r~(:\,or u<lnji implicit thrllrr~.<nf what i.; rclcvant and what ~ilt~pvic are l ~ jrnpwtant. For cx,iml~le, whcn drqcribing n ptar.;nn 1 might, a s sociologist, highlight thri r personality, rthnicitv, agc, ~ e n d e r and the like becauw. 1 str these as thc critical clcmcnts that cc~nsrltute a social pcrsnn. Anothrr person, a binlogist, m i ~ h t Focus much more on physical, ge-enehc dttributes. Each of the descr~ptions rcBects a dfffcrcnt focus of ~ n t c r e ~ and t a differt9nt nssewment rrf what i s r c l r v ~ n tin describing n person.

Although t h ~ q chapter and the fc~llowing twn chapters foctrs on explnnatory case ~ t t ~ d i ei t ;is important to di~ctt-;s illt~scriptivc \ b d v dc<~rn<. I t is i r n y n r t ~ n to t cmrsirlt-r descriptive in\(. qtudles for two rm-ons T'hr t~rst is t h ~ unlt's~ t we can )=and dc<cr~ptions of cn<cn5 wt. will he In nil pusition tci 'lchieve good p ~ p l a n a t i i mtising ~ those cast%%. The sccnlid rcarcm is thal .I disccrssic~n t ~ dcscriptirr t casr r;h~ditl.:further highliclith Il~e irnporta:irt~uf tfienr\. In c ~ . ; c q.t u ~ l vrtb<tanrch.

Thc p r o b l m ~ in any dercription is w h c s ~ to~begin and whcrc tn end. This i~ r.\pccia[ lv kc. ~ . i t h cact1 ~ ~ U L ~ I A P S c i l< ~ t u d deal< ~ i.r,1t14 the il'l!o!t, c,~st.

Dcscripti1.e case atudic.; m a y consiqt of qingle or mill tiple cases. One way of reporting multiplr case studics is to use typolngics and idral typcs. Typologies may bc thcnretically derived or cn~piricall y dcrivcd . A thc~rcticallvderiv~rf t \ . p o l o g ~i4 nne that is lo~icallvor thenr~ticallv pnwthlc*. Fnr ex;irnplt=. CIcrton 11Qh31 d c v e ! n ~ d YIIW p o l o ~ of tvpcs of de~,ianl.;bawd on thc notions ~ r fcultural and institl~!ionalized tJrr4ofrGid a c h i e u i n ~tl~rr.;t. goals (Fijir~rr13.2). 'Tlitls the ct~~~fi)rtr~r.;l iq a theorr.trca1 constri~ct dcfined as somt~onc who acctlpts the g(lnS::of culturc a n d thr tt:r7trpr_i by whicli the cultrtrt*pr~scribes t h t k nchievement o l thcsc ~ndF<. rhc fllTro:lrlfc,t a~.c.t*pts the c r i a l ~ but not thr. prescriber1 rnc6ans for aclrirvjng them. T l ~ c rrIrr(ll15t reject< L-u ttural ~rralqhut neverthvfcz.; goes along w ~ t hthe prcwr~trcdcultural brhaz.~our. Thc rintrrutisf rtaitbctq both thc gnals and thc mcans of n c h ~ e v i n those ~ goal<. Typologies construdcd in this tlcductive wav rt*yl.c~rnt a sct nl ideal epc. (Cowr, 1977: 117-4). Ideal tvptnsarc a thenrt-hc~l descripticrn c + f srts
cqtjd~E;

CASE STU I I Y DESIGN

227

ut chararlteri.stics we e ~ p e c !trill t grr together tn a c a w . Thcv represent a p i r r or theoretical tvpp t h ~ i~ t ilcot necessarily found in that purr fc~rrnin empirical reality. For example, while Wcbcr identified the charactenqtics of a ' p w ' form oi bureaucracy ( G ~ r t h and Mill$, 1946: 1Yh-204) we may find no cxan~plcs of aciuaI bureaucracies that function in exactly the way Wehcr postulates. Deductive typologies and ideal types can providc a useful way of analysing casc studies. The ideal tvpe can serve as a yardstick a~ainst. which tn compare actual cascs (Chapter 15). The template pmvided by the ideal type can guide the way in which we investigate the casc; i t can guide what we Pook for.

An inductivelv derived typologv is one in which we start with a cluesticln and then examine cases in the light uf thc question. A comparison of cases can then highlight rlusfrrs nf similar cascs. For example, 1 commenced a set of casc sttldies with thy question, 'How d o aduIts pet on with their cldcrly parents?' I s h ~ d i c d a mnge of cases and found that cascs diffcred along a rangc of dlme~isions (u.g.lcvcl of conflict, tension, drpendencc, balance, r n j o y m c i ~ t, inh~nacyand intrusit cness). I then noticed that cases tended to cuhibi t diqtinct cl t~sters of ~Iiaracteristics. On thc basis of thesc conimnn elements I d-l~rlract~rized particlllar cascs as heing of a particular type. Mrurking from a question and actual cases 1 then deveioped a fourfold case based typology of adul t-parent relahcmships that T called parent centred, child c e n t r ~ d , remote, ~nterdcpendent (de Vaus, 1494). The particular characteristics of each type nted not concern us here. The point is that typologies can bc developed in different ways - tither deductivtly nr inductivclv. R~girdlessof the way in which they are developed they can provide a helpful template for conducting, arralysing and reporting (Chapters 14 and 15) descriptive case studics.
Other elements o f case study designs
WIlcn designing case studies tlierc, are n numhes of elements, apart from !how discusscd abnve, that can he built intn thr design.

example. In other sltuatron? we mnv rely on a single cr-rtrcnl case. This can be appropriate when rve have a clear tlieorv with well-Filrmulated propositions and wc haire a single cdse that rnects all the requirements of thc theory. Such a case can provide a rnodcrately convincing test elf a complex theory (Yin, 1989). Multiple cases, strategically sel~ctcd, c;ln providc a rnrrch tougher tcst of a theory and can help specify the different conditions under which a thcorv may nr may not hold. Fusthcrmorc, lnultiplc cascs a r c essential if the case studies are heing r~scd for inductive FuToses (see Chapter 15 O n andlytir induction). Given sufficient resot~rces and access to cases, multiple casc designs will norrnnllv bc more powerful and cnn~incing and providc rnorc i n s i ~ h t s than single case rlesi~ws. I Iowever, when using a multiple case dcsikqi tvc should endeavnur to treat each case as a single cnsc so that we are able to establish a full account of that case before engaging in cross-case ctmparisons. The unity of thc singIc casc shauld bc respected (Stake, 1994; Yin, 1989: 56-77,

h paral1c.l dcsign IS one where all the case st~rd ies arr clone at once (e.g. ddtcrent in17rstigators eC3chdoing a cast.). On1y ( ~ thc t ct)nrplct~r>n uf thc investigation of tach r a w s h ~ d varc cornparisnnq madc bctwren the cases. Thls strategy can be appropriate when the project adopts a simple theory testing apprc~ach. A sequential d e ~ i ~ i.; nonc where case studies follow one another. Using this approach one investigator could, in princ~ple, conduct cach case study. An advantage of the sc.qucntial approach is that the selectinn of each casc and some of the issues examined can be infc~rrncdby putzlcs identified in earlier cases. One case can ihrow up idcaq t h a t can influence the selection of subscqucnt casts and that can be followed up in these later cases (Chapters 14 and 15). When s d n p t i n ~ a more inductivc, theory building approach a scquen tfal d e s i p is mnre appropriate than a parallcl approach.

A case study desigr ran bc based on slnglc or multiple cascs. A single casc design will norrn~lly bc less cornpcll~ng than muEhple casc d e s i p s . Using the logic of rep!ication (Chapke~ 14) a single caw represents only one rcplicatfnn and does not necessarily provide a tough test of a thcorv. However, W E mav have little choice. Limited access to cases or the extreme nature of the c a w may mean that w e ran nnlf study a single

Most case shrdics, a n d all explanatory caw studies, will incorporate a time dimension. Without tliis dimcnqion anv adcquatc causal cxplanatinn is not possible. Casc studics can provide a p w d wav of carrfulfy mapping t11c scqlrcnce of events, which is the basis of c n u s d explanations. It has to be decided whether this t i m ~ dimension is obtained rctrospectivel y nr pro5pectively. A rctraqprctirlc design involves collcciing, on thc one occasion, information relating to an extended period. This r ~ q u i r e s the reconstruction of the history of the case This rnlght be done ttirough the use nf archival records and drvcurnents, or interv~ews with peuplc whn participated in or chserved past events.

T'lriq J t , \ i ~ n )la\ lht*clbvit~us prut7lcnls as\r~crali,d with 11)~s 01 c ~ ~ r l c t l c ~ , rec~rnstruchr,nr l t thc pact rn t l i ~ Ihgllt of t h v prra?en!, a n d m i + t a k l n ~he %pqupnnn in rr,hnt=h eiprnt+rvc~~rreii. l-lt)~vc\~cr, in manv qitualions tf~rrc, is l~ttlc cl~oicc ll~rt t o dr,lw on pcopltb'.; ahiliiv to rccall thr p + t With <.ilse +tuifrc<tlrc ilsc ut rnliltiplc sourccc of rvicfmcc can t-educe t h t ~ r n b l c r n s that qcl 1vit11 t h i ~ approach. A ~ ~ r n ~ ~d u c~ yn s ~ involvcs ~ t ~ r t s ~ c k i n gc h a n ~ e s forward over t i r n ~ .t t ha.; thc obx tolls atl\'nntayr of enatlFin~, thc inur.;h-q,ltcw to I(wk at e\.rnts thev tnccur rather lhan relying rm partial and rcron.;tructcd nccoui~ts. I3t*pc*ndlngon the i<srrc.; uncfcr consirleratic~nand thtl t y p s 17f c,rses, pri)<pcrtt\.rcast* \tud1e~ might la+t for l;r.lrs. 'The u b ~ i t , u s~Fisailvan~agc r a n bc I l ~ c t i lnc and rc<ourct's rryuirrd. 'I hew c u n ~ t m i n t s can <cvt>relv l i n ~ the ~ t nrtmber . ~ n d rangeof ca.;~.q that can bv5h1~lied. Thi\, in turn, can rcqult In \'rl.v I1rnrtc.d teql. ol a tlicor\e. lipgarrl lcrs nf w l i ~ t h ~ ar r e t ~ t ~ s p v c t ior v ~ pr(xpt=ct ivc approash i~ ~ ~ d o p t rthr. d . gi>:clis trl build 1 1 p J d e a r ant1 reaqnnnbly ~letai lcd picbrirc, r > f thc qetlilcncr In whfcll cvr-nt.; took place a n d nl thr conlcxt In which thcv i ~ r ~ t ~ r rTu ~ i ?b.( ~ i l dmeaningful c a ~ ~ + t1xplanationq aI we must grt tlic serlucncc and thc ct~ntextr ~ ~ h/I t .con\-incing c a u ~ a r\planatrt~n l ivjll bt, ~ b l t .to frack t l ~ i b 'ql(vy' bv which Ijnc (or a sct t 1 0 c'vcn t(3) rb17tl1nrtlip producing .I particular rjutcomr.

- ---

--

-------_I
-- .-h s m w e

----- -

E rplannlny" - - --

.-1-

Ll~rrnct~vn - _-

--

c
1-

T~mp

Ravosp~fw~

- - --

--

J
I

Relm4~~-1w.-

--- -1 rlp'Annlnry

I ,

P~FIIPE~IV~

-I
I

C a r crde,

) ; ; I ?

I--,-

1 - 7 : 1 3 1
I-

I
-

I
*-

I
phml,om , s w , w n a

1
I
-

I
parfilb

pwatta

t --i

I wJwb8ql .-

-1-

I w l m a 8I 1 1- I

Type5 of case study d ~ c i g n s

UnliCt- 0tht.r types OF r c ~ n r c h design there is, as ycat,n u invcntrvy of 5pc.s 13f ca-e studv d r ; ~ g n (Yln, IcES<)). l Ir~wrl-er, t h r ilisci~ssrtmr ~ clcf rnvnt.; r l l caqe .;ludy d ~ s i ~ n discus'irdl .; in thiq chapter prn\ridrl%a wnv i l f t h l n k ~ n g.~houtthe difftrent ways in which casv s t ~ t d i t \can be strut'hrcd. ' I ht*w cltlrncnts .,arc: Jr5criptilfe o r rxpfanatnr). tlirory testing or thi.nnl building r singlc case or rnultipie casr = hrd~sticor cmhcddt-d units of analysis pr~11'1 or ~ e q u e n t i ~case il <t~~dicl< rtltrthycct~ve o r prorprctivr.
a

A cm~~-clci.;sif~ratrtln ot thew cltlmc.nts rr.;r~lts in thc Ir~yic~rl ~o+sibilitv of dlttcrt3nt ~ a r i a t ~ n n of < c a w .;tud\ clr.;igns ( f ~ ~ u r 1 . 7p 1). T~II.; iiqsrc i . ; ~r~ccntt.d nut tc, rnerruhrltn \t.lth the ranee of pr)~qilrilit~r>q, nrtr t t > dt~vrlop h4 diffr*rent names for ditfcrent ca*ta s t t ~ d y desi~ns, but to yrovirfe a rvav of t h t n k ~ n g about different altrrn~tir.rnk for <haping a c a w +i~tdv dcricn. The d ~ i s i o n s madc In rillatinn to thc+y six rlrmrnls \\,ill rta~ultin d~ffvrcnt was.; of conducting kt~crase +tt~dv, nnd r j t analyqing I the d a t a crrlltrtcd in the stitd~
h . F

What a case study is not

Method of

Tlme 1 ( T,)

Intewent~on

Time 2 (T,)
Post-test

Bt.fnr~ cnding this chapter i t is nwrth saying what case study designs are
1111k.

groups
'Experemental'
Group rs untfom
In terms of the proposed causal

>re on

Not thr onr-shut casc 5trd!1

The taxonomy of r e s ~ n r c hd c s i p s outlined by Campbell and Stanley (lqh3) and hv Cook and Campbell (1979) has led to misunderstandings of what a case study design can be. Using the template of the classic experimental design they identify a design which they refer to as the onrslrtrt msi3sf~r~-2r!. This is rcprespntrd cliagramat~cally in F~gurc 13.3. As a simple example of this type of d e s i k ~ we could consider a group of children whnsc parents divorced w~thinLhc last year and for whom wc obtained a measure nf emotional adjurtment. In effect this design consists nf members o f an 'experimental' group only and relies only nn

variable
/)

vanable

'post-in tervention' rnfnrmation. The earlier discussions of the logic of txperlmcntal, longih~dinal and cross-sectional designs indicate how this deqign tells us nothing of value ahout thc impact i ~ fparental divt~rccon the eniotiot~~l adjustmrnt of children. We cannot tcll from this design whcthcr children with divorced partanthare in any way ~ l ~ s t ~ n r t frum i u p othrr ct~ildrcn, and whct1lt.r hle1.e is any tl~njtgr! in emotional adjustment follt~wing divorce. Representing this type of design as an ekamyle of case study design is n of a wellunfortunate and provides a rnislcading picture of thc ~ o t e tial cot~ceivedcase study design. lndccd Cook and Campbcll (1979) haxfe acknoruledged this and have renamed thc design as a L I I I E - ~ ~ U I Ip~1st-Ir5t P only design, and now stress that it should not bc confused wit11 true case study design.

thc log~crvI7~reby ~ n i t i allypotlles~s l clr research qur<t~nn\ can be s~rbju~tcd L o empiric~ltcsting. Dccidin~betwccrl single and multiplc casr sturlit.s [st,? latrrl, srlecting s ~ r c r f cnscq ~ c 10 he .;tu~lied,dcvclopin~ a case study prutcjml and dehl-ring r~alcvant data collection stratrgres 1e.g. the period to b~ covered rrtrospech've, prospectirc] are all part o f case study d e ~ i v (1993 . 33)

Any method of data collection can be used wikhin a case study design so long as ~t is practical and cthical. Tndeed, nnp of the distinguishing features of thc case study method is that multiple methuds of data colltrtion will often be employed (see Chapter 14). Case studies seek to build up a ELITI picture of a casc, its subunits (see earlicr discussion on units of analysis and embedded designs in this chapter) and i b contcxt.

Casc studies have frequently been cqua ted with qualita t ~ v e methods, e<peclally participant nbscrva tion and unstructured, i n - d ~ th p intcrviews. This identification is no doubt partly because some of the early, influential case stt~dies used participant obseriratinn rncthcjds (e.g. Sfnxct G ~ r ) l rSorir.t!l r and Middlefuua),Sirnilarly, most ethnograpllic casc studies used in sncial ,~nlt~rupologp employ participant obser~.atior~, and ethnographieq and case stiidies have sornctimrs been lakcn to be the same tliing.' I n other contexts case studies are cqrlated wlth unstructured indppth intervlrws that enable the rcscarcher tn bii~ldup a much fuller picture of a rase than is possible with more quantitdtive based methods of data collection. In general there k a tendency to equate 'thick dcscripticln' (Ccertz, 1973) with case studies. I f WL' equate case s t u d ~ e with s a particular data cnllcction method we misunderstand case study d e s i p . Yin argtles that casr s'tudy dp5z4yn iq

Summary
Case study deqigns constitute a major deqign for social r w ~ a r c h Thcy . offer a flrxiblr approach, which can result in an extensive variety nf particular designs. They make usc of a wide variety of data collccticm

methods and they are particularlv suited to using a di\*erw range of units of analysis. C a w studies In social science rcscarch should bc fundernen tally theoretical. Case study designs are particularly suited to situations involving n small number of cases with a large number of variables. Thc apprc>ach is appropriate for the invcshgatlon of case5 when i t is necessary to understand parts of FI case withln the conkpxt of thr whole. Casc studies are dc.~ip,wcRudto qtudy vvholcs rather than parts. They are ;lFst~partic~r larly

232

CASE STUDY DESIGNS

appropriate when we need tn inlFestigatr phenomena wherc it is not possible to introduce interventions. While experimental1 y based designs allow us to exclude the inflr~cnccof many variables, case study designs are particularly u s c h i when we do not wish or are unable to screen out the influence of 'external' variabres but when we wish to examine their effect an Hie phenomenon we arc investigating.

14
ISSUES TN CASE STUDY DESIGN

Note
1 Yin (1993:h0ff) careh~IIv distinguishes ethnographies from case studies. One reason Yin provides for this distinction is that cthnoPaphies are not suited to theory testing wllrreas case studies arc. A further basis lor the distmction is that cthnopaphies do not assume a s~ngle objective reality that can be invcsttgated following Ihe traditional mles of scientific inquiry.

A11 research d c s i p s shnuld be internally valid and cxtcrnatly valid, should produce reliable results and should hc amenable to replica tion. Case study designs are often secn to be deficient in all these areas. However, careful attention to theqe matters at the design stagc can deal with many of these criticisms.

Methodological issues

Thr intcrnal validity of many of the research designs examined in Parts 1r to IY of this book depends on having groups that arc comparable e x c ~ p t in relation to specific key variables. Internal validity in these earlier designs relies on screening out the influence of variables other than the key causal variables. They involve focusing on a small number nf variablm and removing the influence of other variables by cnutrols of one sort or another. Threats to internal validity stem from the danger that factors other than our key variable am producing any changes we observe
IDIOGRAPHIC AND NOMOTHETIC EXPLANATIONS

By focusing on a very restricted range of variables the earlier designs achieve a narrow or nomofhetic explanation. That is, h e y achieve partial explanations of a class of cases rather than a 'full' explanation of a particular case. They involve an examination of fewer causal [actors and a larger number of cases. For example, a nornothetic explanation might cxarninc. divorces as a class of cases and identify key factors that contribute to divorce overall ( c . ~ age of marriage, external family stressnrq etc.). The nomnthetic approach contrasts with iditynrphir expldnation. Idiographic explanation focuses on particular events, or cases, and seeks to deveIop a compIcte explanation of each case. For cxampIe, an idiugraphic cxplanntinn of a divorce would examine a particular divorce and develop a frill picture of why flmt divorce

occurred.

231

CASE STUDY DESIGKS

ISSUES 1N CASE STUDY IDF.SIGN

Case sti~dics can be utllizcd for both types of explanation. A case qtudv adopts an idiographic approach wIicn a full and contextualized understanding of a case is arhicvecl. But case study designs can adopt R nomothetic approach. Thiq nccurs whcn particular cases are used to achicve a more generalized understanding of broader thcwretical propositions (see drscussion of case studies and theory in Chapter 13). By dcvcloping a full, well-rounded causal account, case studies can achieve high internal validity. By seeing particular causal factors in combination with other causal factors we call assess bath the relative importance of particular causes and the way in which various cazlseq interrelate. As such, case studies can achievc a sophisticated and b a l a n c ~ daccount of causal processes and, in so doing, avoid mistaking cause with cvrrcIation.

They stress thc importance of looking at parts w i t h ~ nthe contcxt of the whole.' Thp casc also must bc seen within the contcxt in which it exists. By examining this context fully the rcscarcher can gain a fuller and more rounded picture of the causal processes surrounding a particular phenomenon. Yln argues that
a major rational? (or using [case studies] is whcn your investigation must cover both a particular phmontpnoi; and the roiltcxf within which the phenomenon is occurring eithcr bccausc (a) the context is hypothesized to cmtajn important rxplanatory ~nformatiun about the phenomenon or (b) khe boru~~daries hehveen phtnomennn nnd contcxt are n n t cle~rly evident. (1993: 31 )

WIIOLES, NUT IUST

PARTS

The designs discussed in earlier parts of this book Fwus on 7torinblw rather than cusps. They examine how different haits nf cases arc associated with one another (Mitchc.11, 1983: 192). Tvpically, howcver, thesc trails are not considcrcd within the contexl of the case c d rvhicll thcy are part. By wrenching traits out of the context in which they occur we strip them of much of their meaning and ccrnsequcntly risk misreading therr meaning and sipificance and thus misunderstanding their causes. Dlumer (1956)uses the term 'variabIe analysis' to describe analysis that f0cusc.s on variables rather than cases. HE argues that although such analysis can establish statistical relationships between variabIes they do not providc much insight into causal processes. Hc argues that: put a t the beginn~ng part of the proccqs of interpretation and the dependent variable IS put at the terminal part of the prccss. The intervening prncess i s ignored . . . as sometliing that need not be considered. (IY56: 97)
Thc i~ldrpendentvariablr i s

Although this is a somewhat simplistic version of the statistical analysis uf relationships between variables, it contains some truth. Whilc many of the deslg~is covered earlier in this bonk can isolate variables that produce part~cular outco~nes, they are not so gi>i>d a t telling LIS 71jh!/ they produce these outcomes. Case sti1ciie3,on the crther hand, en~phasiye an understanding nf the whole cnse and swing t h r case within its widcr context. Goudc and Hatt (1 952) describe cnse studies as
a w ~ uf v organinng soc~al rlatn - . to preserve the rrnrtnry cknmctrr n f t h r socml olyn-t IY'IJTX S ~ I E ~ I R. !. il is a11 approach whrch vlews a n y socyl unit as n whole (1952. 331, their rtalics)

The study of context is important because behaviour takes place within a context and its meaning stems largely from that context. The same behavrour can mean very different things depending on its coi~text. Furthermore, actions have meanings to pcnple performing those actions and this must form part of our understanding of the causcs and meaning of any behaviour. To simply look a1 bckaviour and give it a meaning rather than take the meaning of the actors is to miss o u l on an important source of ~tnderstanding of human bcllaviour. An example can illsistrate this point. 'rl~lc research li tera turc on extended families assumes [hat cxchange of help and carc between the ge~cncrations(e.g, adults and older parcnts) reflects the quality and strength nf intcrgcncrationrtl ties. However, if we examine the meaning of intergenera tional help within its context we might interpret its meaning differently. Within a particular Family thcrc may be a great deal of intergenerational help: t h adults ~ in the family are attentive to the elder1y parents, visiting often and helping nut where they can. Rut is this a sign of family solidarity, closeness and caring, or is it a sign o f control, dependence, manipulation and blackmail (adults help out of guiIt, conccrn about hhcritaiice, worry that siblings will be favoured unless they help)? b the help reciprocal? What wcrc the patterns of help in the past? What is the history of parent-child relatinnships in this family? What particular forms of help arc given? What type of help is withheld? Why do people give the help? How happiIy is it accepted? What hnppcns if help is not offered? What are the norms regarding family loyalty and help within thc community, class and ethnic gmup in which this family 1s situated? What are the rrrles nf inheritance in this society? What are the legal obligations, if any, of children tn carc for elderly parentq? W e would wc could build u p a need to address thcse and other questinns b e f ~ r c picture of the rneantng uf intergenerational help and the yossihle causcs of help in a particular casc. To isolate the behaviour from this broader context and to strip it of the meaning p v e n to it by actiws is to invitc misunderstanding, and thus threaten the internal v a l i d ~ q OF the study. To take a further example: our research goal may bc to gain an undcrstanding of drug addiction. To make sense of adclickion in a particular

236

CASE STUDY DESIGNS

ISSUES IN CASE STUDY DESIGN

237

case we would need to understand its social and institutional context

Was addiction the outcome of medical treatment to contrul pain (addiction from trcatrnent)? Did it originate wrth emotional distress following relationship breakdown (addiction as escape)? Was drug use a taken for granted part of the immediate social context in which the person lived {addiction as conformity)? Did the person comc from a very anti-drug background (addiction as rebellion)? The context is all-important for understanding the phenomenon (addiction) and presumably is crucial in shaping appropriate ways of managing or treating the addiction, In summary, explanations bastd on case studies involve much more than explaining variation in one variable in terms of variation in another variabfe. Case sh~dydesigns are devised ti> yield a sensible, plausible account of events and in this way achieve internal validity. They achieve explanations by building a full picture of the sequence of events, the context in which they occur, and the meaning of actions and events a s interpreted by participants and their meaning as given by a context. In the end an adequate causal explanation is onc that makes sense. It invnlv~s telling a plausible, convincing, and logically acceptable story of how events unfold and how thcy are linked to onc another.

drpend, in part, on the type of case, the data collection methods, and the nature of the outcome variables. A life history of a dead person, which relies on documentary evidence, is unlikeIy to suffer From reactive effects. Retrospective designs will be less prone to reactive effects than prospective studies. But a study of a tightly knit group in which the researcher uses participant observation and interrrifws will inevitably affect the way in which the group operates. We can try to reduce the threats of reactivity by using unobtrusive data collection methods (Kellehear, 1993). For some types of cases this can work well, bl others it may not be possible to use unobtrusive methods for c i t h ~ rpractical or ethical rcasons. If data can only be ohtained by interviews or 0bsEr~ationit is very difficult to avoid the reactivity of being a 'foreign object' (Denzin, 1978: 200).

HISTORY AND

MATURATrON

The way in which 'l~istory'and maturation can threaten the validity of causal explanations has been discussed in previous chapters. Exptrimental designs seek to deal with the problems of hiqtory and maturation by using randomized control p u p s and focusing on d$ferenc~s in change between these groups rather than on absolute chang~ (Chapters 4 and 5). Case study designs try to deal with the 'problem of history' and mattrration by looking duscly at the wider context and exploring the extent to which these sorts of concurrent cvcnts contribute to observed outcomes. That is, rather than diminatinx historical /cnnt~xtual and maturational factors from the analysis (by contrnlling them out) the case study designs incllide them in order to enhance our understanding. In this way we can arrive at a fuller and richer understanding while, at the same time, avniding the threats t h a t history and maturation present to the internal validity of our case study conclusions. The quality of the case stucly will largely rest on how well i t identifies these liisto~ical/ contextual and maturational factors and includes them in any explanation.

While caw studies may achieve excellent internal validity by providing a profound understanding of a casc, they havc been widely criticized as lacking external validity. A profound understanding of a casc, it is nrgucd, provides no basis For generalizing in a wider population beyond that case. A case is just that - a case - and cannot be representative of a larger univers~ of cases.

Simply doing a case qtudy can produce changes in the case and we can confuse the effects of doing the study with the effccts of pther variables. Whether or not this will be a problem in a particular case study will

If is correct to say that case study designs cannot provide a basis for making statistically valid gmcralizatinns beyond that particular case (see Chapter 15 on statistical analysis). A case study may consist of just one case (one community, one organization, one pcrsnn) and we cannot be confident, in any statistical sense, that the case represents a wider class of cases. Even if we conduct multiple case studies, say 10, our n remains too small for credible statistical gencralizatinn. Ijowever, case studies do not strive for this type of external vaIidity. There are two types of generalization: statistical and theoretical. Sfufisfic111 genmnliznti~~z is achieved by itsing represents tive random sampleq. Oh the basis of statistical probability we generaIizc our findings to a wider population that our sample i s designed to represent. Theoretical generalization involves generalizing from a study to a t h ~ ~Rather y. than asking what a shtdy tells u s about the wider pupulation (statistical gcneralizntion) we ask, 'What does this case tell us about a specific theory (or theoretical prc>position)?' In Chapter 13 1 argued that casc study d e s i p s are fundamentally theoretical. They are des~gned to heip develop, refine and test theories. They d o this rising the logic of repfication.

'
238 CASF STUDY DESIGNS

ISSUES IN CASE STUDY IIFSIGh

239

'

To understand the ccrnccpt of theoretical generalization it is necessary to tlnderstand the logic of rqdicnfinn in research design (Yin,1989). Th~s is the logic that lies a t the heart of generalizing from experiments. Since experiments typically do not use representative probability samples they provide nn basis for statistical generalization to a wider population. An experiment might employ an excellent design that ensures a high degree of internal validity but unless it is based on a probability sample wc do not know if those findings will occur in the wider population. There may he something about the particular sample that means that the findings wilI appIy only to the people who participated in the experimcnt. Exprrimo~ts using non-probability samples argue for external validity on thc basis of replicntion !ugic. That is, if the experiment can bc repeated again and again under thc same conditions and produce the s i l m ~ results we can be confident that the experimental results will hold up more generally. If the experirncntaI results hold up under different conditions and with different types of experimental participants our confidence in the generalizability of the r e d ts gou-s. If the experimental results cannnt be rcplirated under particular conditions then we will need to specify conditions to which thc cxperiment cannnt be generalized. We gain cnnfidencc in the value o f and robustness of experimental findings by repeating the experiment. Mrhcre the experiment is repeated with different samples and under different condi tinns we get morc of a sense of the limits of generalizability. Repeated experiments provide some idea of the range of generalization. If an experiment only works once with a particular group of people and the results cannot be reproduced we would have little confidence in its findings nr its applicability more generally. Experimental science is based on the Togic of replication. No finding is accepted on the basis of a single experiment. The same logic applies to case studies. Indeed a single case stcrdy can be thought of in much the same way as a single experiment. In Chapter 13 1 argued that the findings of a case study are tested by repeating case studies (more of this later). We can examine different types of cases under different conditions and maybe even using different methods. 'She more the cases behave in a way that is consistent with thc way we would expcct on the basis o f our theory, the more confident we are about our theory. 1f we find that some rases do nut behave in the way WE would have predicted we need to modify our theory to acconlmodate that case in lhc same u7av that thcories wilI be modified to account for the experiment that docs not give thc expectcd results.

selection of cases contributes to Iit~nlI and tlrrorctict~lreplica tion (Yin, 1989). Case study d ~ s i g n involve s sciccting cases for thenrctical and targeted purposes. We select a case because it tests whether a theory works in particular, real wnrld situaticms. Alternatively, we select a case because we tliink it might disprove a proposition, or because we want to scc if the theoretical proposition works under particular conditions (like repeating an experiment under different conditions). The strategic selection of cascs means that we know something of the chnracteristics of a case before the case study proper bcgins. Since cases aw selected because they meet particular requirements (as opposed to bting randomly selected) we muqt first know sumcthing about each case. This may rcquire extensive case screening before actual cases are finally selected (see later in this chapter). How cascs are selected strategically can be illustrated by using the earlier example nf devolved s~qffir~g systems. If we begin with the propnsihon []-tat 'the introduction of devnlved staffing systems in schools inlproves thc qualit-?;of education' our first case study will target a school in which wrclr a system had bccn intrt~duccd. We w,ould predict that in this school there would be evidence of e d ~ r c a tional impro\~ement. Further~ncre,we would predict that an intensive s h d y of the school wcmuld show the following:

1 The improvement in education could be traced back to the ncw

systcn1.

2 The reasons the new system was effective wcre thosc that were anticipated in the theory (better fit of teachers with school, greater fcrlings of accountability and greater belief that effort was rewarded).

STRATEGIC SELECTION

OF CASES

The external validity of cdqe sh~dies is enhanced bv h e strnfe'yic selection of cascs rather than by the statistical selection of cases. The strategic

If we found that thc case matched what the theory predicted, a titern1 replication of the theory would have been achieved (Yin, 1989: 54). Since this outcome may br due to chance we could seek out some other schools in which the devolved system was introduced and cxarninc its impact on the quality of education in those schools. If the same patterns occur we have further literal replications. We could then attempt to tesl the theory in a difkrcnt way. Sincc we had concluded that the imprnvement in education was due to the introduction of the devolved system we would expect, on the basis nf cHlr theory, that where a centralized system was rctaincd there would be no improvement in education. W e w n ~ ~ deliberately ld seek out instances of schools still functioning with a centralized system to see whethrr our theoretical predictions were supported. A s a tougher test wc might look for cases that mn counter to the hasic proposition: that is, cases of a failed dcvulvcd system. Alternatively, we might Eook for cascs of centralized systems that, over the samc period, also delivered irnprovcmcnts in educational qualiv. However, our

thcor] nlaj Ilr ~ z ~ thb7r c h ~t sLlnarcou~rt ftw tlic5t' ' ~ l r v ~ n r caw< ~t' It m i ~ h t kp~riF\ z l i ~ t tht basic rclatinnship will tjnly \uor'Cc i1ndt.r pnrtiw lar ccviditiorir. I f we find t h ~ tht* t hawc propclritirln.; do not hold up and thlq i. yredjctablt- lrn tlie bawr of thr theon' M'C hrlve acfdcd furthcr support to our thevry. .Ibis i q ~ ~ ~ l!{1trt?rt+f lt=~ r ti - d rt$it-o?io~~ (Yln 1USq, 54).

Practical issues

5incc c a w 5 arL1 u w d for thrortttical rather than s t a t i s h d ~cneralwatinn tlir*rihiq littit> p i n t in srlccting caws b c ~ a u . ; they ~ arc ul stTrnc scnsp reprt*~r,n ta t ivc of w m c ~virlerpnylifatitm. Nor d n lvtUnerd h , IlIr>k For ' t ~ ~ p ~ c cnsr>s2 nl' W c h a w no sttrr. wav nf k n t w i n q i\*llt~thcr '1 c a w i.; truly typical a n d no n r a v OF rqtirnatin~is5 h ~ ~ c ~ l 7'11~ ~ i jt ~~\ t. r t I\\ ~ +irrnt5 pr3cy'1kh to find t \ ~ i c a l c.isi~s rcflcrtq contllsirrn abrlut w h ~ t hlitthcll / I W ? ) isill< ianuntar;l hve nntl nnnlvtlc matlcr (T! ~ n ~ i ~ ~ c ot r iwh'it c ~ n Ytn ( 1 q H Q 1 c,llT.; .;tati.;tical ~ r ~ ~ e r a l i ; l a.111d l~on thccrrrtical ~enc.r;lli/aticm. 5incr t hc. purpwe crf rase 5tutf it? I \ ~ r t o t gcncwl17r t c ~a widrr sample of m s r s (r~nunicrativc ~ndllclirrnor rtahstical gencral~znt~nn) t h t w IS little point in t y i n g to find a typical c a w for a cnqt- shldy. T1ir nwrl i.: to fin~icaws thal will prnvide valid and challenq~ng tc.;t< of . itlicorv. Cllnwr and Strauss (l'jh71 hn\,c uqed thi. term lilrarrfii.nl w t ~ i p l i r i ytci de%cribe tlie normal methud o f selezt~ngc a w s in case sttrdv dtbs~gns. Flakim 11 9x6) uses thr tcwn frrrlrsrd .ca~rrydili,pto dcscribc thc snme prooeG3. Thc~c terms r ~ f c rtc) tlic strattyv uf scltacting caw? that wi31 prciviilc illuminating examples of a tr-pe of caw (x In de~rriphvrcasc qtllciy d r s r ~ n sn ) r that will provide nppsnpriate t c ~ t s nf ,t t h r o y .

T h e i > no ctrrrcct rrl~rnbcrnf caw< to i n i l ~ r d ein a c.l\cl .;t~td\-di*'.iqi. C,IGL~ ktudt rlt..;isn< can conqi~l of a sinelc E ~ P v . ~ r mt11t1yIr~ ~ c w t - (Uh;lytcr ~ 121. I4'ith r n ~ t l h y lcaw ~ ~ *tlldic< tl~cnlimbcr nf is a nlntt1.r r ~ f judgcm~nt()'in, I t r R q ~ .X). IZqisnlfrcant factor in d r t c r m ~ n ~ nthc* ) : number r p t co'itn.;~ ~ hc l h-le l ri~orlr with whrch t h e prtlpositions a s r t r l 1-c Icqttld. Ll<inr: the Ior;ic o f repllcation n singlc rcp2icatrun tctl5 r l s ~c-rrnttIi!t~g but n ~ p m t t dr c p 1 i ~ d t ~ ogi\ n ~c' marc corltidrlncc* in trnd ingq l i w f r seck tlict~rctic~il replication5 wrbr v i l l npetl to cnndl~ctaddf tinnrtl caqe s t u r l i r ~ Whcrtl lvr fii~dcase* tIi:lt not fit with nur tbxprct.ltions thi\ rn,I!. p~crri\~cr thra r~rri! to rr~nductaddit~cln~ll c;lsta sltt~riib'; to enable c r q trl r,,tl..... 1..... .,L-- . . I . .. -. .
.I
1
S T c

c,wc, ~ j w i ~ n !L> e~ cf;<[?7'tjipr' l mlr p r o p t ~ ~ i t i o n lvil! , 4~!w i ~ b thv n ~ ~ r n l v r o f cnsv ~ h l d t e s rcr can crrnduct. Thc cnmple\itv ol o u r prt*dictinn< m,iv jliflurnce tht. c t ~ n f i d c n cwc ~~ havc in utrr rrplic.~t~nns Thc rntlrc ccwnplcx our prediction< the rntrrc cmfidi*nt n7c c,ln bc in the 51nglc c a v where nur prcd~ctions arc replicated. Wherc* rlur prcrliction.; .lrt3 qirnple ( e . ~wlicrl . A taxistq then 7 ruill tucur) n . ; i n ~ l e case study ma\. lint reprissrnt a t o u ~ h tmt and rve nm!. want multiplc rekll1cahinn5befnre wc fpcl confident In the rcqlilt5, Illhcrca our p w d ~ c t i o n s arp more cornprrx and demanding ( c . ~ wlrrn . A+l;+C+D exist\ thm \I'J,Y+\'+Z t\'ilZ orcur) i t is wlakivc~ly tsnlikvl). that t l ~ i ~ c o n , p l ~ rp t t r r r t ivil l crccur ~ l ~lo lc ~F~nr~ct.. 14cci~rd in):ty w c rnav itwl rllr~tc ct3nhrlri~t in tlic tt~iyhnlws of thta tcst that Ihl, ~ i 1 1 ~ cast> l e f t t ~ r l vprnuidtxs. How mjny castn~ t ~ t d i c rul.cnnduct s rvrll nl.;od~pc.nd un o u r knrrwlcilge nf cutc.rnal far~ors that rni5:ht .ittcrl results. That 15, i! t4.e expect that our rcbsuli.; will hrdd u p i ~ n d c ln wide ranqc of c.utcrnnI cnntliticin< wc nlav ntTd letvckr casc \hldie<. \Y licrt- w=,1rtx rtnctxrta~nabnut the ~xtt.rn,ll cnndit~c>li+ tnirler which our prtdict~int~.i will hold wtamsv ncr~l nlclrt3c;lsr ~ h l ctc, l ~~nrlc tl>c<r r rl iff~rerltct $nrlll~on.; !o see how c~ln.;i.;tt~nllyocrr findinj:s hllltl IYin, 1"s" tTc>r i%\,~n~p r\.c l e ,may be un<itrtsrtmIri-th~r thl' Fink I l t m k txrrr.nJcvol\.tvl ht,i t l In:: Lv5tl.m~~ n c cd l itcatit~nal pc.rtorn~,-lnct~ wil! linld rtyartlltnk.; of rckfii(~n Ic.g ii,nl:r c ~ t v~ , t ~ k i ~ ~ rur*al r h . ~anti n , r t - n ~ o t r Or, ) a p m , IYL' mav not knari, it thrpo\rtivt& impact wiII hnlil ,rt ~litftar~nt Ic,\.rls (pnmilr!, <el-ondarv or t r r t i ~ w l ,vr In the d ifferpnt t\.pcas n i .;chnot sv~terns fpr~vatc, rPli~iol~ (1r $ sinti>).In thr tacc of unccrta~ntita.: w r m,ly necd to inrPiitl~ c.lrt, studit- that accommrdatr* scrrnc of thrw=cJ ifferencc~ ti3 s ~ ic t tlic prediction5 can bc rPpllcatcd in thcsc rlitfrrrnf cr~nditinns. A furthr%r tartclr that may affcc-t the n u r n h ~ r of cnqe.; Ihr ct~ncr~ptual knn~ctvnrk with n~hicl.r w r Art, ~ v n r k i n For ~ . example, i f qtwial cia+\ $\.a< ;Irr in>pr,rtnnt concept in tllr shrrlv ~t r v w l d hc neccqsarv to ~ n c l t ~ ct~sta.; de frnm vach cyf the drffrwnt s o c ~ a lclasses. Tlie number of cnscXs would dfpi-nd tvi hntv .;wial clnsc wras concephiali~ecfand (HI ]low many cln\< cat~goricsthi:, concrp~tralizntion included ( c . ~ .jlrst rniddlc~L - I J . ; ~ and working clasc, or a more r ~ f i n o d claw c l , ~ s ~ i f ~ c n tthat i ( ~ ntnclude< upper c l a w and \.arton.; cla.;~iSicntinnq rvitlirn thc middlc ant1 working cla<ws). Wr co~ililgo on fortwt-r and chrck t h ~ n g s mat under anrr nlrmbcr ot rlifi~rent contliticln* in the end we mu<! m a k e ~ l ~ d ~ c r nat)ouf t ~ n tI~ ~krlv \ a r i a t i c ~ n;rnd ~ hallc good rt.a<rm\ for t.atx,cttnq [lint thrsca difft.rt~n~ ci~ndrhc~ns r n ~ ~ *3ffect h t the p ~ t t c r n s W . r nlstl nimt I7e gi~rdedb) n halt ih prnctlcnl w ~ ~ t h t~ hnc-rinrtraintq ~ of time, mnncv and .rccr.;s tin relevant C J ~ L ~ ' ; . l'hc critic-al thin: i c t t ) 5trlrct the mo<t ~ t r a t e ~ c-;lscs ic trr twt 4 ~ 1 r proprrhrhon\ rathcs than aiming ftir a l a r ~ c numbtar of r a w < .
i t .

govt.min1: sd~rrcrT\I l . 1 ~ bc'ttcr ~ t.ci~icn tlnrlal ou tcnm~.;w e nerd to knnw <chlcl-rschur>l<h a v e ~\*hicll sytcrn. IYtb r n i ~ h t 2150 need to kn(~r%d, for .;rlerliun pr~rl-rosr.;, how l o i ~ g a ? C ~ O O I 1135 113d ; I particuhr system. Wc nilght w a n t 10 srlect schnol5 that had good and poor n l t ~ c ~ t i c ~ n n l rrutc*crmP.;.Tn do this WC' I \ ~ ( I u ! ~ to havc this informatron I-efc>nx the c a s t ~tll~cficln prortacq t r d placr*. S t r , ~ t c c ~wh;e c .;~lcction can r ~ q ~ l i rcr~nsiclcrnl.rl~* r gmttnJ\rclrk t r i itfent~fyilrc. L-liaracteri~lics of a largr n u m b r r rbt casrs 111 ordtnr to scrB which unrs thc critrria for wlrrtlcm. Thy.; process can br timt. con<rtrni~i~: but is critical ancl must bt. built intn t ~ n l c line< anti b~,dp=ts C~sc scrcvnin!: m a v br don~l in n u m h r r of rv;lvs, Jepl-nd~ngon t1lr t1.p~ case. sun cy ~ l u ~ % t i o n n a iannirah n>, tcprrrb clt a n rrrgsniration, ilrclliv~l rccords, d,~tab,iaeso r a wide range n F other sourct7s nray yield 111~ rrfwant screcnlng ir~tormattnn. Evcn when .;crtvninq i> intrt>ducvd we mav still cnd up .;clectiny: inapprrrpria tt. cases, Ni~vesthelr';s, thesr cases mav bc u s r h l l for other yrrrpvsc\. /\lt~mat~veSy, rvv mav d ~ c i d c tn disccrntin~lcthnt parh-cular c;tse rlildv .;inrr i t docs not 11c.l~ answer the rwiginnl r1.searl.h questions

ntlrnl7t.r of cast.'; sclc.ctt-d, and the partic~trarcases scli-ctetl, will bc afkctrd 1.j. the , ~ r n n ~ tOF n t tinlr and mt,nc.v ailabltx. IYtlll-t.uectrtcd t-ase sl~~ iw r l can I?(- verv time con<i~minh+. Cnnrt~clucntly, only a limiteil numbi*r can brt cnnductcd. Thy ~ ~ 1 s factor t l l ~ g h l ~ g h evtm ts mure the iniportnncc of the qh-atry$c selcctim~of caw.;: since ~ r n t va fcw case studit..: can be cnmplctcd, each {me hsr to 'earn it.: kccp'. .hccc.;s to caw.; iq the ofhcr critical factor that affect! both t h number ~ of casr.; shldied m d thr par!icuj,~l-c a w 5 .;i%lertcd.Certain cases nlav be idt*ntif~cd as bcing stratqica'l lv irical frlr tlic dcqim but pmdicai rnat1r.r~ mav mcan that w c h c,ises arc. not ; ~ v a i l a b lfor ~ rcsr-arch plirpows. G,~lniny: acccss tu cases that h n v ~ been idcnkihcd a< strdregicn I1 y approto bc approached 1 ~ 1 t cirnsidtv-ablr h care and rffort. priate n w d ~
'niib

~nfc~rrnnticm. G a i n i n ~rclia1,lc informat~rm a p r r r c q ~ ~ i + i tfnr e ~ood rrsc;lrch. Thi* mc;ln5 that tlic infurn~ntlon njuct nvt s i r n ~ l v be .In artifact r ~ thtb t p n r l i c ~ ~ l c1rcumst;lnces ar of itq c~llt'ct~on. I4.c n i ~ ~ 1 111 1 bt, c o n l ~ d ~ n t that the s~irnt*~nfr~rmation rvauld cnTerKc i f ~t were s o u ~ h again t in compar'jble circumstances. Wc certainlv d o not wan1 cnrnpdrison< bctwccn castk< tc~ 't7c in\,;lliclnred bv sloppv collrction o r dl ffrrent approache< twin): applied in each caqe ~ t u d v bec.1~1 that ~ rnakcs cases ~ncnmparablc*For r*xarnpl~, it wnulri be cornplctrly i n c ~ p p r o p r ~ a to t p Cump,ire n1nrrrngc.s wherc in onr cdse marital sati1;f,3ctmn was ascertained b\. using a rtnlcturcd hatterv nf r~t~e<tionnairr kern4 while in anothcar satisfaoticln was n i c a s ~ t r ~by ~ d ~n\cstic.Ttor chscarvatlon, rlr in ; I thirrl ~~~tiqf'lrtion itpa!: detcrminrd t,n tlic b;i.~s r,f rrvpclrtt ol othr.r pc~>plc* (e.~ kiibnd.;, ch~ld rcn, parmts). A1 thrn~gliearl1 c a w s i i ~ d y would provirlr in lr~rrna tion ilhc>~it 'm;lrit,>l sati~factinn, thp w a y this conccrt i s mrasured us in^ thrb differcnt tcd~niqrrt..: mivansthat, in rFfpct, it i* betng cctncep!uali;.erl very d iffcrtxntlv in vnch casc3. Swond, it is imperative that a n v >ttld\ 1% capal-rlc of rcplic.~ticm. h l e e t i n ~[his c~,nditic>n rtq~~irv thai a if crihcr invr';sil;att~>cont1ucti.d a s~niilar 3ttidy ~l~e!' ~ t , o l ~ find l d thr same psttprns. Succ{-.\fr~! repjic-atiil~~ i . ; .I +altyudrcl a ~ ~ ~ ifrazld. n t t I t JISO cnahlcs r>thtnr- t 1 5 ~ c 1 1r tl1tb pattcm r>n!lm orcur5 wI1r.n a yarl icul,ir person cnn~llictk thc ol-~;rrvatlon+ or intcrvtcws. I 1 finrfinp arcx onlt. sap,~blc of b v i n ~ f c m r d El\. ontLpcrwn thcv prt)~idtn little I ~ a s ~ for c ~ent>rali/ing trl thcarctical proyositinns.

I.

Thc trmptation is t r ~ 'grt your hands dirty' carlv on - to ~ c otu t thert* ~ n d It-,~rnFrclni thr caci.4 in full confrdcnce that the truth of thc c + ~ s c rviFl ~clmcl~orv cnrrrge. Tc~c*d t r n pi*r>pYt* cornmpnce cast. stud ie3 rvithou t knowing what their re.;c;lrr+li q~lc.sbrlni s o r what p r o p o s i t ~ i ~ n thev s art.
el alunting Thiq is recipe for disaster. lt i q wrcntinl that wc h a w a clrar r r v n r c h question brforr heginning a casc stutlv.: l n d ~ c t wr i cannot i v c n hegin to sclrct case\ until r v r ha1.r a clear ~ t a t r r n e n of t the r e a r c h q u ~ q t i o nWe . ntvd to go bevi~nrl having a clucrtinn. Wc slio~tldbe ablc to fr,rmulat~ sonle ~ n i h a l propositinn+ - some Initial an.;rtmcrst r ~ the question that the

c.1.;~~ttlcliv4 m.Ty

i n v o l ~J~ nl~rnbr-rot ditfcrt-nt irn*e.;ti~att)rs. It1 'p,lmlTc!' de.~gns rt is alnrnqt inevrtablt- that a numhcr of i n t e s t i ~ a t o r s 75,111 ~ n v r l l ~ ~e rprrliap.; l 1 ~ 1 t3 h ifi~tt'rent i n l - r . t i ~ ~ t r condrrctinq v ench studv. In rntlltiplt. c a w d~m.;ignse ~ c l cast i sttldv .;hot~ldhi, condtrctt~di r a ~ cnnqiGtrnt iva!' slncc rnconsist~ncybctwern cnqeq can i n v ~ l i d a t rcnsc camparlwn.; Yin (lcjqu) tlrg,,, t h , ~ 1 o f 5 wsc ~ C . ~ C J V protoc~1T~ hrlp acliir,vc this son51stcncy. 1'1,-1nntnq t o encurc conq~stcncv ,icrtn< c a s ~ ~ttsrjics am1 ncrtKs inve5tl~ a t n r isr ~ n ~ y r ) r t nFclr n l Iwcl rea.;cln.;. hrst, 11 i*~rlinncc~ tlrc rt~liabilitv (>four

cast.

~t~rclic ~< '111 ht.1~ L I C ~test. Shere will br thnsv who urKr that fEcSdlvnrh 41n11ld b r a frr=(,o f ~lienric~ since thr1r;c irnpo.;~ nkrr p r r c n r ~ c c ~ t i o n ak n ~ yrt.-t.x~\tinq l t-atccorics on t47e clala. %mc will urgtm that WP.L;IICIU I L ~g o intu t h ~i~elcl ' trct, f r ~ ~ itlrrt,sctt~.~l! ii cncurnbranccr a.: tliwe sirnrlv bltnrl u.; to ii-hat r\.C r n i ~ h r~therrvlsc t we and make 115 dtanf to wliat prkr\pli& arr I r v i n ~ to tcZl rl<. CJrlr jr~b,the): will say, i < to ,lllnw clthcrs t o tell u< their story rather than u> irnprlsin); a thrclr~tical~ n t p r p r ~ t a t i oon n their <tory, or uqtnq their \tory ttjr thtbclreticnl pilrycrres.

244

CASE STUDY DESIGNS

ISSUES IN CASE, STUDY DES1C;S

245

In mv view this is not thc task of thc social scientist. rt might be thc role of a biographer, a novclist or an activist. The role of a social scientist is tc> develc~p and evaluate theoretical generalizations that enable us to understand whole dasses of cases - nut, in the final analysis, individual cases. We do not have to have a welL-fo~mulatcd theory to tcst. Our questions may be such that there a r e no obvious theories to test. l3ut this dws not release u s horn the nrcd to be well read and well prepxed theoretically beforc going into the field. Gasteur observed that, 'Where observation is concerned chance only favours the preparcd mind' (quoted in Mitchell, 1983: 203) If wc are unaware of rclcvani theories, concepts, dcbates ancl thc Iikc, we will probably miss the significance of much of what we might come across. Without having some idea of what we are looking for we will not know what we have found.

the face o f cnnsidcrable social incyuality, does the communily maintain a strong sense of cohesion?' These questions guided the study a n d provided the framework around which the study was reported. Regardless of which approach is adopted it is highly desirable that the logic of the research design remains clear. The reader shcnlld bc clear about the prnposition(s), and why a particular case was selected to trst the proposition. Tht reader should be able to see that there is a structure and a logic to the selection of cases - that they have bcen sdected for a purpose rather than simply because they happen to be

available. Any report that merely provides a number r)f stories is incomplete. In reporting ctlse studies it is critical [hat wr relate tl~e cases to propositii>ns and that we seek to draw comparisons between cascs and arrive at generalizations or more refincd yropoqitions as a requit of the caw studies. W c need to tell the story of the set of cases - the generalizations, thc propositions and the questions they answer and the propositit>ns we end up supporting.
Ethical issues 'Thr s a m e ethical principles apply Lo case study designs as to the designs discusscd in previous chapters. The particular ethical iqsues confronted will vary, depending 011 the typc of case study design adopted and t l ~ r particular form of data collection used for that design. For cxarnple, the ethical issues w ~ l lbc cliffereit if the casc study involves an active intervention or a passive intervention, or requires that W E obtain a retrospective account of an event, intervention ur change. The ethical issues will also differ according ta whether data are cnllccted by interviewing individuals, using informants, analysing official records, passive nb3cnration, conducting a survey of casc elements, or using participant observation. Since case studies frequently employ a range of diffcrcnt data collection techniqucs fur the one study it is likely that a greater range of ethical issues will arise when using a case study design than with other designs TIicre is no point In repeating the ~tlncal matters identified earlier in this book. You sliould read thesc earlier sections carefully and you should see how they apply to the specifics of the part~crrlar case study Jesim you adopt. H o w e v ~ rsince , case study designs frequent1y involve ~k--trlining more in-dcpth information and Crcqllcntly employ participant nbscrvation techniques, some ethical issues arise that are less of an ~ s s u e with other designs. I argued earIier that reactivity can be a significant thrcat to the intcmal validity nf casc s t ~ l d y desip;ns: thc pwsence uf a r~searchcr can alter thc dynamics of the cascs being nbserved. This can be partlcuIarl y so where participant observation techniques arc used. One way in which some

Case s t ~ l d i e can ~ involve the collection of a vast amount of information. This ~nfrsrmatic~n must be carefully prcrccs5ed and distilled bcfnre it can he yrescntcd. W e cannot simply describe the casc and somehow let the facts speak for d~ernsclvcs. Any attempt ti) prcsent all the facts will result in an indigestible mess that is unreadable and unread. The facts do not speak for themselves and we should not pretend othenvise. Describing a case always involves selection and ordering and at least implied construction of causal sequences and interpretations. Therc are many different ways in which case studies might be presented. We might describe each case as a whole s n that the reader builds up a clear picture nf the casc. Alternatively, we might describe good examplcs of particular types of cases and then cornpatc thcsc different types of cdses and draw general conclusions regarding our theories. Hochschild's The Second Shiff (1989) prnvides a good example of this type of reporting. She describes a series of marriages and from each of her cases she extracts ~ C S S O ~ about S the domestic division of labour and the procrsses by wluch marital inequaIity in this sphere is maintained. Alternatively, we might repnrt the results of a series of case studies by extracting ilrern~ffrom the case studies and reporting the findings of the case studies thema ticall y. For example, Vnughn (IY84) cnnd~~cted rrtruspectivc case studies of coupIes w hoqe Intimate relat~onships had cnded. From thcse casc studies she Identified a set of underlyng patterns and stages beneath cach disinfegrahng relationship. Another approach is to focus on a series of qlrfstionl; and rcport the cases in terms of how they shed light nn thesc questions. For example, Dempsey (1990) conducted a single case shldy of a small country town The guiding questions in the case study were questions such as, 'I-low unequal arq people In this comrn~tnity?','What are the key bases of social clifferentiation?', 'How, in

I
246
CASF STU13Y DFSlGNS
I

ISSUES IU CASF STUUY I3ES1GN

247

mlgl~t h y to reduce the effects nf rractivitv is to disguise their ~dent~t ay s rcscnrchers to avoid their r r s ~ a r c l ~ cidentity r intruding. For example, an rnvestigator may want to examine sexual harassment in the workplace. If he went into a companv revealing his idcntity and purpose it is likely that people in the nrganization would behave differently while the investigator is present and thus undermine the study. As an alternative the investigator might undertake the research by obtaining a job with the company and covertly observing what goes on. The ethical issues of deception, and failure to obtain informed conscnt, obviously a r i ~ in e this situation. The dilemma it higl~lights is the tradeoff that occurs between ethics and internal validity: keeping to cthical guidelines can threaten the internaI valldity of the study. There can be little doubt that revealing one's identity and gaining informed consent can undcrmine a great deal of what Punch (1994) calls 'strcct style' ethnography. rn some cases revealing one's identity can effectively kill the research prnjcct. In these situations careful consideration 1111ist be given to the merits of some form of dcccption in the light of the likely benefits of tlie research and thc potential for harm to the participants. Whilc it is easy for a researcher to rationalize that the bencfits will oi~tweighany potential fur harm, thcrc 1s a Iot of sense in obtaining advice or even in having dccisions made by third parties such as ~nstitu tionaI ethics committees. lnvestiga tors will need to resolve cornpcting ethical cr~nsideratinns. To what extent should the ethics of informed consent and voluntary participation take priority over tlie value of research exposing I~ighly unethical behaviour? Even if a researcher revealed his/her identity and gained permission to undertake the study, the question arises of how to get informed consent, and From whom to obtain that consent. If we were conducting a study of sexual harassment in a workplace, would we obtain pcmisslc~n from the gatekeepers in the organization, or from all people whom we might observe in the coursc n i thc study? Do people need to he informed that their public behaviour is being observed for the purpose of research? Clearly m many cases this is simply impractical (u.8. observing the behaviour of people at a party, in a dernonstratron, shopping at sales). In my view abservtng public behaviour raises few cthical problems. So long a s individuals arc not identifiable there should be n o danger of harm to participants. After all, if the bchaviour is public there is l-rardlv anv invasion o f privacy. Anothcr ethical problcm that is more likelv to arise whcn using nbservat~clnin a case sh~rlyis the problem of what to d o when we observe illegal or i2arrnful bchavirrur. Should the cthical isstres of informed consent and voluntary participation take priority over exposing highly unethical h ~ h a v i n u r ? For cxample, what wuuld be your ethical responsibility if your case studies of families revealed cascs of dpmestic v~olence or c h ~ l d abuse? What should y c d~ o when conducting a case study of a

you learn of the identity of a person sell~ng dangerous drugs to students? What do you d o if you discover, during your rcscarch, that a group participant is about to commit a csirnc? Is tlre betrayal nf trust of thnse around you more unethical than one's responsibility to those in danger? What is the responsibility of the researcher to othcr rcscnrchcrs? If one rcsearchcr betrays the trust of a community, even if for ethical reasons, does this thm undermine the capacity of other researchers to study in that community? Once a group has learned tcr distrust social researchers it can be very difficult to conrfuct any further research with that group, and this may havc further ethical consequences. The behaviour of some researchers w ~ t hindigenous groups has led to profound mistrust by some indigcnaus co&unitic.s of outside researchers. How do you resolve the nbjcctivt. of feminist research to promote identification, trust, empathy and non-exploitative rcla tionships between the investigator and thnse participating in the research on the one hand, and tu expose unethical and harmful belraviour on thc othcr? We lack clear-cut answers tu these dilemmas. They highlight the fact that ethical principles can often conflict with une ai~olher,and that ethical issues can compete w i t h rnrthndological and practical principles.
schnnl and

Summary

This chapter has emphdsized that thc intcrnaI validity of case studies relies on a Ingic that sets this design apart from the other designs discussed in this book. Case studies focus on a large number of variables and thc way in which they intcrrclate. In contrast, the other designs rely on cnntroIling out the influence of other variables and on focusing on the influence uf a very limited number of variables. One of the strcngkhs of case studics is that they atternplt to understand the significance of particular factors within the context of the whole case rather than by screening out this context. As such case studies havc the pc~tcntialfor good internal validity based on a more thorough understanding of the meaning of partic~~lar behaviour and events than the other designs prclvidc. The external validity of case studirs is based on the logic of replication rather than on sampling logic. Case qtudics are used to generalize to a theory rather than to a populahon. Cases in casc study designs are selected strategically rather than statisticaIly and a r c selected as critical tests of theoretical propositions. Iswes in selecting cases, the seTectinn of research methods to collect information about the case, and condtlcting case studies were discussed. Finally, particrilar ethical questions that can arise in casc studies, particubsly Lhosr employing participant obscsvation and ethnogapliic methods, wcrc raised.

--

t Thr. crlriccpt nf n wholt- IS, .is C;c?c>rlr dlld tlntt (1Y51) point oul, n ctmqtrflct 1tqt.lt It 1.; r l q I! Iin <It-hntbq i \ har the '~r,lu>le I.' 7'11eill-lincat~rm I,~*twt>rn t h ia5t1 ~ It h ~wlx7Te) . and I!< context 1%. intlwti arl~~tr,ir>. 2 'Tvyical' casr.: qhr~illdnot bt, ct~nlu.;rclwith selt*cting '~rlcalhpical' c a w < C.-X(-: th.it mn! n,prr<ent ail idral tvpe (Chapter 13) 7 It ma\ br. arlrrtyrlatt. to cnndut t c . 1 ~ ~tl~di w.ithnt~t e claur rr1~c;rrch qut~clion< tlr prrlpr><rtions w h m w r art. ~iolnr: erplntatorv case fttrd,t,\. I1 wrL .lrr iiclng fht\ c a w 5ti1d\ I n clarilv cnrrcpfts, Ihtn lo l ~ ~~t d pe n l t ~ l~ u~ ~ ~ \ ~ !l.lml-m~cht (HIs t-nttv thr held b~.Cort= a n,~t.;lrrh tluccllon l i n t berw iitiall/t~l R u t fht.; t y p ~ L ~~ C .lst> 41 L I V, ~ dt**~clleJ i n 1 1 1 . 1 ~pr~x1 I I CJ ~ ~ r ( w v r ~ t qttrxstitln, l I + I t.rv ttrfft,re~~t trom a chirp
stk~J\. d

CASE STUDY ANALYSTS

r w ~ n4 .

1-GIW

chlifv dr.vyrr

IS Ihc

. ~ n ~ . ~ril 't a~ r crh

qur*\lltm.: It reftarnt o tht-

\lrurtllnxr~f a <tudv lhat M 111 cbnat,le 11, loclc and qmrcturtnof thc ktul!\.

Ir,

dr\.cIopccl! a~lIv XlrbthuLl.s I nr nnalvsing r.a.;t. stt~ilit+ rc lc<.; ~ v ~ t ~ ~ m n t i c than ,Ire thc t ~ . c h ~ i i q ~ tljr ~ canalv51ng s data ct~llccttd with orher tvpta.;of rewL~rch dtlqiFns Analvstq wiih thtw othcr. dc.~gns typicnIlv use.; slatisiical tt1chn1q~rcb.;that rclv ciri I ~ ~ i v i rmanv ~ g r,ises ;lr~drrvolve ,irclund

cc>rnp,lrlng grutrp* nt cases. Furilirrrnort., ,ittc.~npt.; ti) ~ r n c r a l i r rfrom ihesc Jc.;~i:ns ~lrcb a s ~ d on prn!lL?tl~!lh tht.~lr\. rvhich. in turn, rt-li~qnn havinq ~ u t l i c i r ~ [Ci.tI \ I . ~ tram nblrrc'lltn gentsrali7e irritli a y v c n Ievrl of

prcclslnn , ~ n d ccrlnnitv. Tiit= r ~ ~ rrli~ ~dt,,~gn\ ~ , i (lt,~tt \\,ITII v , ~ r I ~ 3 v 1 1 r rclv 0 7 1 c t ~ ~ n p ~ ~ r !l~-ta>t,f?i iv+~is ;:rtillp< u<tng a I~m~ttarl n i ~ ~ n l ot ~rr I ariallles. Caw \ t u ~ l t e s , h a w c ~ c r , ,~ch~cx\t. euplanattnr~sb! intt-n.itvc1 .inalvqi< of m a n y \'arl,lblr.; i ~ ~ i t l r t r.I r casts ~ ~ P I I ~ c I a~wI sc ~ i ' tI' I S ~ Cto ~ replicate rrsults - z ~ c l t a 5 n mcans of locat my: cn11w1c \ y l d ~ ~ a t i c ~ n In s .acld~ tlnn rn~tltiplc c a w s h d i e s are usrf111 tc~r mnLt nq cnmparxwn< h r h v t ~ i nn w c and t hu.; t.\trapula t ~ n to g bro~cler tlicoric.;

Statistical analvsis

AltIinugIi the logic nf thth e,~rlicrd c s i ~ n sd m . ; nijt rtlcluirc statistical analysis, s~rt-hanalysi.: is I$*IIIPIV ~ i w d M . ~ t h o d sw c h a s cnrrclahl-rn~, c r i ) w - t d b ~t ~ im, l ~ sr.grrssit,n and conlparlsvn of mean< rtvly on splc of r;l.;e< and o n Zlllildmq a e ~ i c a lp ~ t t u t . rc>r dc=ccribin~ trenik acrnw ~ l i r r f t ! p f rcast+^ ~ r,~ihcr kh.ln rrflcrting tlie charactcrist~cso i any particular
CrlhC.

Be)(auct* CAW qtudv ~ l e c ~ q n fix-~sr: < un inrl i v i ~ua l l c a w q t hct. ernflov dikfcrt*nl mcthcldz 11t analrsi+ A numhcr ( d s ~ ~ c methodc h h w e I7crn dcvelopcti cjr a r r In thc prtlcc.;% uf I w i n ~ rlcr~clopc,d. U;l?c study nnnlyq~s rr\~olvc ,~rountl ~ a s w s z l n q tlir ' f i t b t ~ t ~ v r ~ ac nn ~ n !\.id d ~ r a icasta and tlrc tht,or\ or theories b r i n ~ rr.;tr.il. Tht. glral nf c,ast. ct~tdvanal!*.;iq I < bh~*nrctical ~ r ~ n c r a l i 7 a t i nrcjthvr n than ~tnti.;tical~ p n t ~ a l i 7 a t i c(Cliaptrr ~n 14). Tlt~s goal ,~itrlctsthl. wnv In M~IIICII W ~ g I r~ aL~t>u! our analysiq. Rcc;luw thk. ftwus i< on ~ndiric!uaI c n ~ 5 I ! I + innppr~7yrlafi., I\-hcn a n , ~ l ~ c r nIracc t : rlata, to t o l ~ t r t the tli~rnbcrof cases zhar hari particular cli.rrac~cr~~tic-< crr beIi,lvt*d in J parlic~il.irwav Tht*<mall number of cnsc.;

and till> a t v p ~ c dn a t l ~ r cof Zht* c&isc.ssrlt.rtorl rn,~kt> it rncnnin~lcs<trr ,in;llv.t* wbc studies b\. c n u n t ~ l l ~ . Ti115i < nclt ttt sav t h ~ casi. t \tudv analwis cannot hnve a guantitati\.v or statiqtlcnl clrrnmt. "4 ca.srb can be dtv5urr/rt-if qtati5hcally. .A p r t t c u l a r rt'ginn ~ n i ~ be h ta case In a r r w ~ r c h p m i ~ tEt . can bc dcscribc~llrn tcrrns of ~ t s .;i/c, wvalth, avtrJgL3 ,Ige, nncmploymcnt ratc, crime rate ,>nd t h ~ ' like. Howcvcr, the f o c u ~ nf the analysis is t o d ~ s c r i b cthe rcgicin - to b~i~l ud p .T pictitre of tll~ltCSW. F~lrthrrmrirc,where cn.;t7.; include emtreddrd units of nnalv<is ( s r r Chaph.r 11) thc analysiq of qcrrne levels may wcll include the sort rrl analv+~< ~vp~caTIy a~srrci,~tod tvith other rltvsns. For crmmplc, n ewe shidv ot ; ! rrglon rniyht includt. a sumev ot rt,.;iJt>ntsin the rcgiull (thi, reqlclcntq t-tjnfi an crnbcdcl~d unit thin thp case). The surl.ty of resident.; wnuld include thc tvpr of analysts tvpically used in ,I crtrsqsectional c-lcqrgn. Howcvcr, tht. ftwus of the ovt-rall analyws w n u l ~ l hi- nn the cnsc ns a whole - the region. The survey r c ~ i ~ l twould .; simply hr rmr piccc 1r-t t11c j~gsawto builrl i ~ p a pcture of tlic hole case. In Chaptlar 13 1 empha.;i~i*d that case srudicc arc h~nrlamclit.>lly thcurvtic,~]. Tlicir analyk~s is a l w Fnndament.~tlytl~i>orrtical. The annlrf<~s nwd+ t o 1 . c built aruuntl thr' theoretical ~ r t ~ p a c l t i u n bctng s tt.ctt.rl nr de\~eItrp~~l.

impcrssiblc. Any dcscsiption nl J I I V cilw ~ I W H V inv~rlvcs S a ~rlrcfrl~rs (IF facts. Ihih .;clract~ou will bc bnscrl 011 what wr ~;vL- a s relevant and important T h i ~ 7 tery act of .;~It~rtinq nlcans that tvil art1 niaking decisions about wl~iltI $ rtllcr-ant. This s~*lrctic~n process tvill lllr I ~ c a v ~ I intlt~cnctbd y by o u r ~ r n p l i c ~theories. t Fi~rthrrmore,anr- mporlinq of a cast. W I ~ I involve onitbrrll,' tlic selected fact+.'T'hc ~ n e \ . i ~ a b.;c*lrct l e ~ v i hand . ordering will mcan tlrat all descriptions arc. our .rl~scripticm.-.,r~lther t h ~ ntlrt, description OC tlir case. The incvi tahlr strbjective elcmcnt o f case s hrdv nnalvsis does not mcan that such an<~lv<is I < thercforc nut rr.c>rth~vhil~. Wl~rlti t ~413~s mean is that \r7e.iho~ilcE bl* r*\plicit abntat thr hn\it nn ~ c h i c h ~ v c .a r c c o n s t n z c h n ~ lh~=
description.

Meaning and context

In Ctiaptvr 13 I she~sec!!h.it caw studies arc particularlv suitcd to rescnrch problem5 rvhvn the phenomenon In which we are in tcrvstrd either be distingui~lit~il from its contcvt irr mu5t bc seen wirhin its contrut Many social scientists believe that s d ~ q i l a t r causal cxpl;lnntic>ns neccl lakt1 account nf the meaning that partrcip~nts in a social sitijr7ti0n attributr to t h d r b ~ h a \ ' r m ~Thev r. n r p e that tlir meaning of br.lia~.iour helps 11.; 'makc sense' of n.hy cTnp event pmducir particular outcomes. Thcrcforcb when conductirl~caw studips it i q useful tr) ~ o l l ~ t informaticrn about the <utljccti\-e meaning rrf brh;lvinur for pnrticiyantr; and build this subjechve datil into our analvsiq of the case.

Analysis in descriptive case studies

C a w dta.;cr~pttonss h n ~ ~ lbv t l structured around theoretical idt*as. 'T'hcre are <rhohrs, however, w h o ,irgue that castx dcwriptinns shoi~ld~ i n i p l v dc~csiht> thr case and not \trek tn interpret it. Thrs view argllcs thnt we should lct t t ~ e 'facts' 5pc;lk f t ~ r th~rnwlvcs a n d a\.ntd lqpoqing our own ~ntcrpr~,t.llicin nn 'the t.ict\'. %tch an approach i.; both u n d r + ~ r ~ b and l~-

One \vav conktructing ric..;criptiun r* tn zl5c rtlc,il t \ yt.5 ((see Chnptrr 13).This i n v o l r t ~ conqtrucfing ~ tlicc>rcticallyJrrivrd Jcscriphcms o f h ' p ~ s of cases. On thv b ~ > irlf s thct)n,trca~l rtvr<idcrahnn* , ~ n o~t l ~ Ln~lorvlcJ~c r nf tl?e exiqting rtlwarcli I~ternt~rrc M'C m n l i ~ the t clcnli*nh OF a type ( p . ~an , authorilnrinn pcrson, a perfectly l~urcaucratic org~nixa lion, n democratic ~Eectclralqvqtrm, an cga1itari;ln m a r r i a g ~ ) . The anal~rtic str~tegy is tn u s r this icjcla1 type as a t~v-nplate ti1 5uitJt' tlir analyqi~ n l at1 actual case. Wc can list the templntc. tn srp how cln<cly nur actual case fit.; the template. M'c. might have good reasons for eupecting that p a r t i c ~ l ~cia r w s will reyrescmt the characteristic.: of a particular Vpv. For exarnplr, wc might cxprct, for various rea<ons, that a government departrncnl in ,I Weqtrninst~rform uf government might bc close to a11 ideal type of n bt~reaucracy a'i delincd b y Wcbcr {Ccrth and Mills, 1Y4h). Using o u r trrnplnte rrf an deal type bureaucracv wu can examine n selected d r p ~ r t m m (case) t to WT 1 1 1 1 ~CIOSPI~ ' i t appr(~xirnnte< the ideal Fpe. AS ~vt)ll ; l i ; ~dcntifiinf: rvhrrc the case !its t ! I ~ C , I ! rve would ~ l w use the ternplatr to hclp h i ~ h l l ~ where ht the c a w dtvcsgi- from the ideal. Rcpeated ~ ~ 1 qtudieq s t ~ { e . n~r ~ l n g c of government dl-pastrnents) might .;how uq thnt empirical ekamplc.; ~+cinsi<tently divcrgi>from the i d r a l in st) far as mini-lcrial intrrirtanlion.: might nvtxrridr, tlic ideal rrf rulra~ o v e r n e dcfcci<inn r n , ~ k i nwmliich ~ i : . a charactpri.;tic of thc idcal tvyr L~ureaucr;lc7*. I t qhotrld be r ~ m r m b r r r d Rorrci.cr, , Ihat the ideal type .tnTI qtands a < an ii1tnaT t?pe r e ~ a r i i l t , - of t h ernpiric~l ~ tiiirlinp from J %rtnf cases. Thr ~ d ~ + ta l ;l p rernd~n..;r 'purr' ideall7ed , ~ h \ t r , ~ c t ~ c ~ n . The uw n T tht* Idea1 type prtrv~cfic< a rvay nf Ir>tiL.ing a t and o r p n i n n ~ : the annlysis Ft~rthe cfescript~ons o f actual cast>$. U.;lng this appmacli wnx avoid dc'icriptinn that simply dcrcribcq w h a t ~ v wtb ~ r l~appen to tind irl~t about thc c ~ s c o r sirnplv reports the features t h , ~ t catrlz our attention. Instead, the dt..;cripticw i c < t t ~ t r t i ~ r c d tanned , nnd p l ~ r p o s t h ~ l .

252

CASE STUDY DFSIGNS

CASE STUDY ANALYSIS

A typologv consists of a set of types. Thus we might develop typologics of ditferent sorb of personality, forms of government, tvpcs of organizational structure or types of marriage. This set of types may he based on a set of ideal types or he derived empirically (see Chaptcr 13).Typologies and ideal types provide cxccllent methods of analysis in which the 'wholeness' uf n case is preserved. Rather than just concentrating on traits, rt typology attempts to build up an overall picture of a case taking a wide range of characteristics into accnunt. Where we ernplc~y a theoretically derived typology the task of analysis is simply an extension of the ideal type analysis described abovc. Where we use our case studies to derive empirically grounded typologies ( i t . those based on our actual cascs) we nccd to adopt a different appronch to analysis. These approaches are only possible if we have multiple cases.

particular explanation. Altcrnativcfy, it may simply bc designed to provide a record of events without an cxpl~cit attempt to explain or inter-rct those events.

Explanatory case studies

As in descriptive analysis, in explanatory analysis theory plays a key role. Thcorctical propositions will either direct the analysis or bc the goal
of

~nalysis.

Clrrstrr n l ~ n l ~ t t ~pprcmches lc Tt~cse approaches involve identifying a sct of variables we want to use as the basis for our typology. POT example, in a study of an organization we might have collected from i w r case studies information abntrt the way deciqions arc made, the way in which rules a n d rcgulahons are used, the degree of h~erarchy, dcgrec of rationality rtc. rn a set nf case studies wc could then group case5 that had similar constellations of chardcteristics. I f we have a small number of cases that we know well we can do [his manually. I f we have a larger number of cases we can use a statistical technique called cluster analysis to a c h i ~ v c Llic same end. Eqsentially, this technique involves identifying the characteristics on which we want the typology to be based, coding cach case in terms nf cach of thnse characteristics, and then feeding that information into a clustcr analysis computer program. The program will then identify which cases are mnst ,similar to one another. It will cluster cases into gruugs that have 'relati~bely similar sets o f characteristics.

When multiple case studies are being conductecl the aim of the first stcp in the anaIyqis is to understand each cast. 11s 11 alllulr. Only after a picturc of each case has bccn constructed is i t appropriate to ennipare the cases (Yin, 1989; Stake, 1994). The analysis of each case must be organized around questions and theoretical coi~cepts.Each case should be used either to test or to build th~oretical propositions. When each case is ana tysed aro~~ncl a co~nrnon framework it is thcn possible to comparc the cascs to arrive a t higher levd generalizations, or to provide tougher tests nt thpnries. If cach caw is evclrnir~edin an idiusyncratic way then i t will be very difficult to compare cascs meaningfully. Thr way in which we approach the analysis of rxplanrrtury case studies will vary somewhat depending on whcther we are using them for theory testing or theory building.

Thcrc am a number of ways of using case studies when testing theoretical propositions. Kin (1989) outlines two approachcs: pattern matching and time series analysis. In my view time series analysis is simply a y p e ol pattern matching but in the discussion below r wit1 treat them separately.

Ratlicr than presenting cases in terms of types we can represent a case as a history of some sort or another. Tf we are dcaling with individuals the case description might be prcscnltcd as a biography - an account of a life. The biogmphy will necessarily be selective and will represent thc persun in a particular light. It i5 likely that the biography will go beyond sin~plc description and will include either implicit or expliclt thcorirs of why the perscm's life went in particular directions. Histories of events, nrganizat~ons,policies, or whatever thc unit 0 6 analysis might be, reprcqent a way of describing a case whcrc the emphasis is on the sequencr of events. The sequential a'ccount may seek L n cx\dnin the course nf events and may bc d e s i ~ n e dto support a

Pattern matching is a form of theory testing analysis that esbblishes a dctailcd set of predictions before the cnsc study is conducted. These prediction5 stem From a theoretical model and thercfnrc represent a clear theory besting approach. Thc analysis could proceed by e5tabIishing a .Get of altprnative pattern4 we wnuld predict on the basls of c o m p ~ t i n g tIleorics. Thr form and cornplcxiiy of pattern matching can vary considerably. The basic principle is that t l ~ e more elaborate the predicted pattern (so long as it still follows logically from the theory) the tnughcr the test of a theory. Having predicted a particular pattern we would {hen conduct the case study to see if the case does, in fact, match the predicted pattern. If the case matches the preclicted pattcm then the casc supports the thcorl~

CASE STUDY ANALYSIS

Independent variable

I
Pattern 1

Independent variable

Pattern 2
Pattern 5
Pattern 8

Pattern 3

Dependent variable
Yh

Dependent vanable

Y,

Pattem 4
Pattern 7

Pattern 6
Pattern 9

in thc same way that a successfuI experiment supports a thenry. If,

however, the caw docs not match the predicted pattern the theory requires modification. Yin (1989) describes pattern matching that varies in complexity according to the number of independent and dependent variables included in the predicted patterns.
SinipEi, pntfer~ls At its simplest level pattern matching involves one independent variable with huo values (e.g. male and female) and one dependent variable with two possible values (bchnvus in one of two ways in a particular situation). In this case there are four possible difiercnt patterns (Figure 15.1). For a givcn case with a given d~aracteristic we could predict nnt of hvo patterns. Which pattern we predicted would depend on theoretical considcrations. The pattern wc might predict for a particular case that we know has value X, on the indepcnd~nt variable is that the case will have value Yb on [he dependent (outcome) variable. The prediction takes the form: if X , then Yb. For example, our theory of work perfomancc in schools might lead us to expect that when staffing appointments, dismissals and promotions * are managed at the local school level rather than by a highly bureaucratic, centraliled and remotr system (X variable), staff teaching commitment ( Y ) will be high. If devolved staffing systcrns are symbolized as X, and high level of teacher commitment is symbolized as Yb, nur prediction is:
When X , (local based staffing system) exi4ts then Yh [high levels of tenchcr cornm~tmcnt) w1l1 finllllow.

pattern I while a competing theory might predict pattern 3. The case study would enable us to see which pattern emerged and therefore which of the rival theories was supported in this case. For examplc wc might propose the following two rival theories: Thtvry A !,oral control will lead to higher commitment to work bccausc effort and 'fit' is swn and rewarded (and lack of effort and not fitting in with school needs is punished). Theory R 1,ocal co~itrolmakes people feel more demoralized, and vr~lncrahlcto local politics and prejudices, and dues not recognize wider professional development etc. This leads to a lack of commitment and a lack of professinnalism and to plaving politics to win Favour rather than fostering performance.

We would also cxpcct that when Xb (centralized system) exists then Y , (low levels o f teacher commitment) will follow. Our test, even in this very simple examply of pattern matching. would be more interesting and more powerful if we were tTting alternative theories that predicted different patterns. One theory might predict

A slightly more complex version of ~ssentially the same situation would arise if either of the variables had more categories fe.g, medium leveIs of performance and a hybrid of local and centralized staffing modeIs). The more values either variable or construct possessed, the greater the number of possible patterns we could predict. For example, if both variabIcs had three possible values there are nine possible patterns we could find in any particular case (Figure 15.2). Tn this situation we would still predict a particular pattern for a givcn cast.: if they were X ,we might predict Y . , Fnr example, we might have a theory about the impact of parenting styles on child behaviour. Each of these two factc~rsmight havc thrcc c a t e p i c s a s illushatcd in Figure 15.3. Using this example we might predict that authoritarian families will produce children who are rebellious (pattern 1) A case study of an authoritarian family will help address whether this proposition receives support. Repeated case studies of autl~oritarianfamihes wilI provide a firmer test. A further proposition map be that permissive families will produce children who constantly engage in limit testing brhaviour (pattern 9).

Parenfal style of aulhoritv

ae&rl~ous -1 Pattern I

Panern 2

Panem 3
Low

1
I

Hfqh

,
I

101

Hlqh

The bcqt test of the theory would hc a writ-\ o f c a w stud~cs yredictin~ particular ypes OF child hhaviour for each parenting style nntl for r~rt.ll of the rnlztuaIlv cscIusivc prcrpnqit~nnsto h c confirnlcd by thc rdevant c~we sltld ius.
Mclrc c t > v r / l l :pnl ~ /rUvr;s: ~ ~ r r r l t ~ /irilir,pr,~~rlrn! ~lr' rrrrri,rl~lr~s rt more ~.[,mplea srl ~ r fpattrrn.. can hc prcrlictc~c!if h.rmo indcpcndent rari~blcqa r p i~scd. a diftcren t outcom t1 is prcd~cted for each a ~ ~ ~ ~ b r ~ r of ~r~ indgendcnt rirn

1, 2 I Ctr~ldsets I pattern
0

, pallem

Pattern

2
7 :

own t~rnfts

Tests

'

Pattern
19

Patfern

I Pnltnrn I P
7 ,

vrrr*

Pmnm

Palloin 24

70

v,~riablrs (Fiprc 15.4). Here wc might make different prediction<, depending on the cornbination of parcntat authc~rityspir combined with the child's leve! (if anxiety. We might predict a diffr~rentoutcomc pattcrn of pcrmisqivc parenting d c p c n d i n ~ on whether we a r e looking at n farnil11 where thr child IS ,inxinu< rrr s~?ct~rr/cunfident. We rniglit predict that t h anxiouq ~ child w ~ t hpermissive parents will exhibit the ' t ~ s t sparental l i r n ~ t r ' responw {patt.cn 24) while the confident /r;ccure chiFil wi tll pcrmissivc parents may set her own limits (pattern 17). We cnutd extpnd this example tu include more indryendent variabEcs and thus anticipate even more pl~srible pnttcrns. Figurr 15.4 ill Listrates ihtt logit-nl possibi litie- - thr different patterns one cclttld conce~vabIy find. O f course, our thmretkat rcasnniny: will I ~ v d u s to prtdict pnrtrcrrlar pattwns i n particul,ar circurn~tnnces,!hat is, for particul.7r h-pcs of casw ;I< illustralrrl in tl~c shad~d cclls in Figi~rc15.4. Harjnt: cstabli<hcd tht* pmsitrlt*pattern* ,lnJ the. prediitl-t! pattern+ nTcurould then wck 011 t C ~ S that ~ S rnahl~ u s to test the pmprw~tions. For taliarnplt., we wrluld conrtz~cta c.,isc stud\, c m a familv in wliiclt authcwitrrrian parentint: and child anuiec occur. O u r p r e d ~ ~ + t ~ I\-ot~ld r , n be that the ch~li4~roulcl hc v c y ctmlpl~ant. if khis proved t o be h ~ c w . e would hnve achieved a Iltrrnl rr,j~l~cntinrr. We m i ~ l i tthen Iwk for a casr of authnritasian parrmntln:: ~ ' l t h a cliilil I \ not an~iou5. Our predrcfitm here mij:lit be that the chitd twirl not bc c n m p l i a n t ~ This would hc a tlrc*orr'irti~lrrgplirr~lrnrr. TIl.lt is, whcrv tl-re prticular combinatinn of factnss

is not present we will nnt get n spcciflr outcome (see I.itt*r for ft~Ilcr discuwion). Another approach to cnllecting and anaIvsing case studv data i i s to lonk for cases with particular outcomes (c.g. Itiglllv corny l lank children) and predict !hat thesp children wilI be anxinus and hnvc authoriiarian parent<. In term.; of the prediction5 in F i ~ l ~ 1 r5.4 c ruc have predictctl this, but t f we find case<of f i ~ ~ hcompriant ly childr~n who are n r ~ t h e anxious r nor h ~ v au e thnri tarian parents thcn o ~rt t htwry 11rt.d~ refining011~. way of tc?tin~, a theoretical propns~tion is. t i i demnn\tmte. t h r o u ~ h c a w %tudic<,t f r d t ~vhcncertain crmditlclns apply (e.g. at~tl~ori!;lr~an pmrcnting and child anxiety) tllvrc will J I bc ~ ; l sprcific c~~rtcr~rnr (h~t;hly ctlmplinnt child). The prohleni with this w r t of r n , ~ t c h i n iq ~that i l only sptu-ifies sttfhc lr'tlf c o r l ~ f r l t o for ~ 1111 cwtcorne. It dtw\ not It-I! U G ~ v l l t ~ f h ~ ~ the<c ;ire the ~)!tl~/conditinns under which the outcome occllr5. As hitch i t does ilot rcprta.;cnta rlrmandln); t ~ q ol t n propositfnn. A nmorc. dpnl<~t1rling tc+t tvnulrl involve Innkinq Inr caws !rehere tlii* particular outcrvnt3 is; prrwnt and tlien a5king whrthrr thiq cltitcomc rlirl!, mcilrs unrft*r the khcorc*ticall y prcdictcd ci~ndi tions.

IndependPnT vnnables

Independen! vanahles

-Condlt~on 1

Condition 2

---r

'

Values of dspendenl vanables

1 1
H~gh level of worker
auTonomy

Hkqh degree d dlrect control over worker

l qual~ty
Staff

Work

A further test nf n propositinn is whcthrr nPc. find cases whcrc all the condition.; are met hut tht. outcome does rrnf occur. For cxnrnplt., d r ~ we find c a w s of authoritarian parenting and anxious childrrn where childrcn are not compliant? IF we do ihcn clitr proposition is not. supported and our theory will n w d to be revised to accommodatt this tvpe

follow in^ characteristics on the fivc outcome variables (A, B, C, D, E): A I,

H7, C , , LI7, El. For cases that m r t t condition 2 wr predict a diffc*rc*nt pattern c ~ f c,utcornes: A t , R1,Cq, L& and E l . Thi% cnn I 7 r illushated rvi th an cxarnple ( F i ~ u r c 1 5.h). Ttie theory h i n g used hew is one that prcd~ctsthat autonomy lends to better outcomrss than h ~ g h Icvrls of supen~isiun. While there may bc nn payoff in terms of cfficien~y,worker autonomy hns pusitive hcn~fi t\ in terms of rnoralc, work ct~ntnitment, ~nitiativrand improved work quality. I n (II~PI' words wc predict that c~rganiza tions ( c n w s ) that permit high Ic~relsrrf nit tc~nomyarncln): mcrnbcrs of tlicir wc*rkfrrrce will d isplnv a parbcular 1rr7ifr~rrt of outconlc.; hie11 marale + 111~17qlinllh' of worI\ + IIIW ~ t a fkirnnxbcr i + modetalc Ir\.cl<(jt efhaenu?~ A h!gh le\-cis of innov;~!~nn. 0rg~ni;tatirlnz u-ith high dc~rr*cs of supewl+lr,n and direction arc p ~ e ICICCJ d to bchave differrnttv. fhcy are prcdic?r=dto have a drffcrrnt < t a t of outcolnr chnrrlct~ristics: Pow morale + low qualrty of work +high staft turnover + moderate Ievelc nC cfficitncy + low Fcvcnls of ~nnovatiun.

of cart*.

Alone ca~~rldrr p t t e r ~ t sr r r ~ r l f i y !d~~ y ~ n c i P t rnrmblmrf The prcvintas section focu.;t%don situations in which we prcdict a ~ i n g l eoutcomc variabIe undrr a pcrticular srt of ctlnditions (rnultip lc independent variables). We can tanvisage situations wlirrc the oppt>s~tc applics - wherc wc have a qlx\cif i c cunclition (onc indcpcndent vnrrnblc) and predict that Iliis cundititrn wiF1 producc a particular set. of nutcnmcs. T h a t is, we rvill yrcd ict a p f l l l r . ~ r fo f or~tcomcs l~nclrr a ),ileen cnnditinn (Fiqurr I5+.5), 12'c ivrll then match that prcd~ctc~l pattern ~ v i t h case5 that mect the condition. firrr EVP prcd~cta pattrrn of outcomtXsin c a w < that meet condition 1 ancl n cliffwent pnttcrn tnr cases that mcrt condition 2. Thc pnttcm is a p~rticitlar rornI~i?rnfifl)l r b l ~>l~l.comes. In t l i i q c a w wc prcdict t h ~ t for cases rnr.rat~tig condition 1 (on the independrnt vrrriabld) we will f ~ n d the

1561)

CASE STUl)\r 1 )I!SIC;N';

H I7 1 t I ! Tlic mo<t rtlmpfcr x.cn~cjnt i t p t t t 8 m match~n r\~ ill ( x - t l ~n.hc.n r tvc 11ai.v a i t ' ! of ~r~dtxpcndent 1.arinblt.5 and a i r ~ tof dcpcndrnt ~.ariableq. In this <rtllnrlon rtotQ rndy bc a b h tcl. prc~lrut that ~i.llcn c a w has a yartic11l;lr GI,! o f c ' . J ~ Iattr~butes s~~ tht'v tvill iie matched k o a pnrticlilar srr of outcornr.;. For cxampIe, wc might predict that whcn an organization is fairly <mall, and pc~<t\t..;~c.s ,I ' f l a t ' org;lni;.nt l(rnt1l s l r u r t u r ~ , r r s p ~ c l c drons~lltJtixre mechnni.;m.; and workcr rcwnrd pliln<, then there will hc a vn1ur.d set nf outcornt~~. Thc.;c might be high r\,ork~rcnmmitmmt, tx\iclIent qiialit!' o u p u t , +lrt*riqtic.\ betryeen ~\,orkrv-\ t~ut<irlc the rcrrrLplnct~, rc.,~iFv acucptance vt innrrr.mitic-m, low level.: 0 1 a!~v~ntceisrn and n ~ ) lntlu\tr~;tl ii~<pi~taticln.
I t i St r1ciIy speaking tht.ory t c ~ t i n ginvc)lvcs specify 1iig a lthctjretical prc>pn<ition, t r s t i n ~ i t and i+itlicr ncccpting or r t > ~ r c t i n the ~, p r ~ p o ~ i t ~In on rt~~ilih+, hlnvevcr, i f a propt,\itii~n I < not sl~pyt,rtt=il hv ; l (-n.;t+ stud! tFirn tht. 11Fht qtep is to rlliint, f l ~ c tlicon. so tlia t t t can ~ a k p aiiount of the, t~kcrption yro\.id~ij tr, fhr c,I*e. In this iw\. tht. Iv+r>pt>kltlcw cover< a tv~dt,r ,ind ~ v l d t 5ct r c r f c,v.c. .intf I~ect~meq rnllna~wnvt*r!til This psocrsc is callt~frrrlndtfrc i ~ r t l ~ ~ r i and rr)~ will ~ be outl1nc.d mnrt. tttl.lr. Iatcr in the s c ~ t i n n 111at ~Ical::with tlicnory tl~tildrng.

TIME SERTES

ANAl.I'SI5

Trend nnalv..!c ~ t ~ be l d far n l ~ r tctirnplr\ ~ R a t h e r thein . ~ n i ~ ~ I v yrtd~rtin a ~rliftt.ri-nt t r t d tin the wrnc vnrvalylc tor rllftPrpnt i . ~ > t - 5 .,I rcm~plexvrr5inn r l t ~rtur~d analys~swoul~lprta~ltct trend< tr>r a ~t ( i t t.ciriablt>:: in any g i v m r.15e. For exarnplc, w r might prrcltct t'ti,it In a gixfcn cmrrse tllc cnrnlmvnts will incr~nsr, tht* ratio of females to ninli,.; will improve with thr* increased cnrnlrnrnt Icvcls, there rvill t-c an inrrtwsc in rnattlrc aEp sturlrnts, and t l i p r r wilt t ~ c nn rcdiictinn in thc q~~~~ ofl studcilts. itv In azintlier cnursc wc might p r c d ~ c n t dlftcrcnt corntvnatinn o f trend<. ,I <tc,ldv enrolment, a rlrcljnc in qtandardz, n ~lvclint? in mature agc ctudrntc, ,mcI fewer fcrnalc stl~llcnts. Trend analyst-\ can b t cithcr ~ ~ l t c r r l r l ~ lrir r-~ rrrtrr~~~*rr~r~~fr.i1. t Intrrr~~lltt.~i time series ;~nalvsis rtafpr.;to a situation rvficrr ,I \pcrific event tL1h.c pl,~cr' .;onrcwher~ within the srqrlcnce of evrnts. This rnahles u s tv cxnrni nc. t hr p a t i e r n of cvcnts hratrlrr>and after the intrrri~ptinil(or intcrvcntron). It1 thiq respect an interruptrd sime serics analysis I < ~ i m i l a r to n bt*lr>rt* and ,iftcr design. Althi1~1~;11 i f dt,cs not invthvtb a control group, tvv wtluFcI cclnduct stuclics ot c a w \ tt~herethc intPrrlipticirl did , ~ n ddid riot t,rkr% I~l,icc \I'hilr tvta c.>~rnrit cnntrol for r r t l ~ c rJr il~arrncc~ bctrrccn c.l.tn. collect dcta~lracl~tllorrnattnnabout all cn\c%\ to mnble uq to c o i i ~ ~ < l t ~ r M t ~ c t h r rthesc c7lllt.r r i l l tr,rcwces Irathcr tki,~n Iht. ~nterruytit%n) rnl<lll rt.spcm.;ilde for [Ilk* rrllp,ircnt rffect nf thc ~ntc~rrupt~r~n. Ititerruptcd tsrrlci .~n,lly.;iq u,ould invnlvc l ~ r t d i c t ~ n a gprr.2 al~cl1 ~ + 1 inlrrrupkioil pL?ltCrn(r.g. dccline prc and lncrcahe post, fcrlloivctl I>\' longer term s t ~ h i l i t v and ) matching t h i ~ prcdictrd pattern with th,lt fnr
actual cascq. Not ajl trend qtudic< w ~ l lhave n n intr*rrliption. %me will s i n l p l ~ . prpdict a trend t h a t $\+ill be anticipated in .I yartlcular ctlnh3ut l or cwrnple, we r n i ~ h tanttcipste from what IVP knaw about lht. rit.rnt3~ r a p h r cand ernnnmic prtifile of a r ~ g i o nIthc* case) that it 1.; FiLc*l). to

:llthough Yin (1WQ)treats this methikt c>I analj.qi.i as qcyarato Iriini thy pattern matching dl-ccribcd dl-ovc, thc POSIC iz Identical and i t % h ( t ~ ~ l d tl~ereforer;implv hc treated a s a fomi ni p t t r m matching. Inqkrad nf predicting a particirl~rp;lttt1m for a ~ r otf v.~riabIesit prctlist.; n p.irt1cular trend or ~ q u r n c t , of event5 The* nn.ilytic strategy ~ n i , o l v rprc~ cl~ctfnga p a r t i ~ u l ~ pattern ~r of c h a n ~ cnvcmrtime. This type OF p a t l c r n annlysiis can takt\ rrnr of two forms: trrjfrl an,llvsis and cfrrorrt)lt!~~inrl (cl\,t>nt ~t7rluencc) aiinlv.;~~,
7rt.rtrl nrmlrr~rs 7'rrnR analysis i< a n cuaniin~tinn of thc directinn nf c!i,lnpc in partirufar 1-ariable or w t nf \.,lrlablc.;. Mre addrr- I ~ I rllic-q* trrvn ol rvhctlwr tlic Irrsnrl i< i~pr\,ard (htt,cp tIr gr,-1ilu.71), shot\'* r l r ) ch,tncc. variable ( u p ant? clo\+,n) or drrwnwar~l(htc~bp trr graduall. Prcd icted trc*nrl.;c;lii rangy f r u n ~ tlic srrrrplt, t c ~the higlily t - o t ! r / l f r . ~ .TFir simplest fnrnlr p n d i c l a trend in tlnc ili*pc~nJrrit vari;lblt~. Wta ~ n ~ i : thrn l~l prcd ict a d it'ferviit par lcrn of change t ) i i t311s\,a la tllc in dificrcnl crlnttlrtq. Ior e ~ a m p l c , w r nrkli-anticipate that cnrt~lmrr)t< in a partluttlar t v p , c j f tli111-erqih.cotlr+LaIthta c a w ) ~ v i l l irwrcn.;c . In ~lir chnrF tern hut d+-c-l~rlc in f h c Imger t t = m ~ rlnc,tht*r course m ~ g h t ~ r ~ l d ~ cto t umarntaln d ~tabtc en rdrnent !cvt4c ~ r ~ l ~ ,~tir~thcr ilc tteprc ) C r turr+r i c prcdictkd t o txtyrar!tlrlcc 'J-l.irp Incrcaqeq in [.ti rt~/nient Ic\.cls.
? L

t.xperience l u n ~ term pc~pulakinn dccl~ne. Ano1llc.r region with n rl I ft~>n>nf inix of denrogmyhic ,inti ectmomic chamctcri.;tic~ mrght be prrtlictcti to c x y ~ r i c n c cgrowth w h ~ yet l ~ another rvgion with a different prtjfilr might be pred ictcd to stasnate.
Cl;rtvmlii~yicnl nrtnli/rl< Chrnnolo~ical ,analv\i< involves prt=d~t-tinl: n .;r7r!~trvi~t-~7 of e ~ ~ c n tI q or rl-en a sequencc nt trt'ndq) in\.ol\ins a n?irn1.rzr (if dlitcrcnt evtant.; or i,,iriL~blcs That is, \\T w c ~ l r lprcdict ~ r . h , ~ t \ tr~i!lrf clinngc, nt what t-vr-litc \teould take plactl, in w h ~ turclttr, 111~1 .;txqllcance nf evcwt* might bc a corrst7nrrrl rbfl'fTc.r srqucnct. nr it rnrght Ilc ,I Jcscriptir~ S~ P L ~ ~ I I V ~ C -tl1,lt L' proposes partic~rlnv<tr7"yrS~ in a prorr%+ l'xnrnplcs of n s t . ~ ~ vcvioii r~l of chrr)nt~loj:lrill;111,11~si< might 111, rm~t1~,lc llrat prcrpmc predirtnhlc Gtazeq in b t ~ o n i ~ r l a p , marijuana w r r t l < t - ~ - L c ~ r , ICkhhk. ~ t d q e =~n t l l t a rlrvntpgsatinn vf int !mato rc~latiniwhiy.: (\'auqhn. I W h ) , stages in thv procrw ot adiustmcnt to rt-tlrcrn~nt (Xtchltav. TLr7rlh,cbr chnngt+ in the rcl=ltton.;hip~ beh\.een ,~rlults avJ thcir pnrcntk .I\ pEirtant+ ilKe l Massdcn ant1 1 2 t ~ a m < 1947). ,

2hZ

CAST: STUDY IIPSIGIGS

Rrg~rd lcb~s ot whether thC analvsi.; involvec a causc and effcct aeqi~wicvof evenis, or a set c l t predictcd stages, the analytic q u ~ s t i o n i . ; wr!l~fh~.r tl~c predictcd p n l l ~ ~ l 'rlfclra71,iy h occurs in the casts w h ~ r e theory rvt3uld rxpcct it to occur. Or does t h c pattern nccur rvht~nw e would predict it .;hould ttrrr c ~ m r ? T l i e Corms of the* predicted patterns can vary. Yin (1989) indicates four tvpt's of w;iy5 in wli~c-lr event< might br predicted ttl change in r ~ l a t i o n in ~ach ntl~rr.We can predict chmnolo~ie.; in which:
must always occur ht-fore othrr cv~nts,with tlic ntvrr<cb +,rf~/r,r~~t' bt.~nc impmsibl~.. wmtx event? nitlst ahvnvx h t followrd bv other r\ ents, on n m n t i r r ~ r . ~ ~ ~ t /
.ir>mc
t,n51.,

nnt find

cause."If

case% ~vht'teMT 1?,1\.~' thc c f t t ~ t swifhclt~t our prt>\t~rned we ,foil tn find any huch caws we havc achieved lurther thcnret~cslrepl~cntic~n.

Analysis for theory

building: analytic induction

At ib heart, analytic induction ic 'a s t r a k c ~ vof analvsis that dircrts the

somc events can only follow other evpntp after n spccifled pt.;qn,~t+ nf timt*;nr rrrtain f i n i l * / r 7 r l m lI n a cnsP study mav be marked I-y clawe.; t > f events that diftrr ~ u b = , t a n t ~ a l from lr t h w e of u t h r r time prnani.. [stages]. ( 1 999: 17'3)

I n Chaptt*r 13 1 n r g u ~ d thar gi>neraIizin): from caw ~tudie*; rrdit.s on t h r repiication logic 01 cxperimrnts rather than the .;tatistical 317gic o f surveys. M'r gain mntidcncr in e ~ p ~ r l r n v n t a rcsu l It? not ji~st from thc elegai~cil of the cxpcriment but from our capacity to predictably replica tc r~sulb and to pwtlictably f ~ i to ! replicate results (i.c. mr anticipate that the intcmcnhon wll ha1.e its effect ttnder spcci fic conditirms hut not under other conditions). Similarly we gain confidpncc in case study findings wlicn we can accuratclv predict which k p c s of caw.; rvill d i ~ p l a v particular patterns and ~vhictrcasee will ~ i o fd~\plays ~ ~ f fpatternq. ic Where. we have a predfcterl .wt of outctxnes and a singIe causal factor a similar logic holds. We would ask the following <or% of qu~stions:
1

inrtrs tigator to iorrnulatc g e n ~ r a l i x a t i o nthat ~ apply to all instances ot prckblc~n' (Denzin, 1978: 191). It is a rncthod that can bc 11st.d to rirliicve d c ~ c r i p h v r gmcrali7ation.; o r tn arrl1.e at ca~rca l explanatinn5. I t ;.E n q t r a t c p that move< from i n d ~ v ~ d uc aa l w s and ~ c h to s identifv what tlic cases havc in common. Thc common talerncnt prclvidcs tht- basis ot tli~oretica2~eneraliz~ition (see Iyiguru 1.2). Analytic inductinn Y G a stTafPg~ that s r ~ k . :In arrive a t genrral~zationq that appZv tn d l ca<i3s,b n thi~ rr.+pect it tI4ffers from the s t r a t r g i e esf nri.~lysisdisc~~ssed In previous wctions of this hook. In Chapter 3 1 dislinfiii<htd hetwecn deterministic and prnbahilistic notion5 of causnt~rm.rJw I T I I > ~ V Gof ilrldl\-if: ~ I ~ S C L in I Cprcviouq S ~ ' ~ wctions a w bawd on pn~babil~~ causat~c,n: tii r4.e cst~rnatc1~1irtht.rone group is mclrc Zikely tll.in 0 t h ~ ~T ~ O Y P S to I ~ t h d v c in a part~culnrway. Analytic ind~~ction, hnwcuer, st.ckq to achirvc univprsal gencr;lli;lations. Denzin ( l Q i R : I i ) 3 sunimnri7cs c i kc! ~ ~tcpq in thr process of analytic indirction:
tl1c1

DOCS the full set of outcomc charactrristics occur when thr prcsurneJ c a u w 1 factor is present? I T so we have confirmation of our theory.

7 Mlc wc~uldthen find another case where the prr.;urned causal factor i~ PPPSIVI~ and stc whether the full wt of outcomtSs is also prt-sent in that cart1. Z i so rvc h a v e a litcral replic~tionof thc previntrs case a n d furtht-r confirma tion uf o u r theory. 3 D{JwrJ get caw< whew the presumcrf causal factor is prcscnt but nnlv w r r r . of the pr~bdictedcrutcime characteristic+ are preq~nt7 t f we F~nd surli I-ases t h r n rc-e hn1.c Ialled to rraplicate thr throry and we would c11hcr rqect or modify tlic theory. I F wr could find no c ~ s c s where thc c a u $*.a< ~ prcwnt and the full qet of nutcornrs was rrrrt present tlicn I\.(, hazre a t hccrrchcal rt>rlication. 4 1Z'r rvould thvn seek to dtnd a caw in which the presumed cai~\al factor ES !lo! prtxscnt. Wi* would rxpcct that thc fullI set of outcc~mrs ~iluld not 01-stir whet1 the, catlsu M3,r.. not prc-rcnt T h a t i.;, we shn~rld

to explain (the d ~ p e n d c n variable). t 1 Specify what it is you are s c r k i n ~ 2 Formulatr an initial and prc~\,isionalpc>r;sible c\planatinn of thc phentln~rnnn vou are seekin< to explain (your thcory). 3 Conduct a shtdy of a caw scnl~lctedto tcst your thi~ory. 4 Review (and revisv i f necessary) your pro~isionill theory in the Iight of the caw o r exclude thc casc as tnap~mpriatr. 5 Conduct further case studiw 10 teqt the (seviwd) propositinn and reformulate the proposition as requirrcl, h Continuc with caw studies ( i n c l u d i n ~ lmkin~ for cases thal might disprnvc the proposition) nnd revl\t. the prnposition until you ~chievcaa causal proposition that accounts for all the casrs.

bc illuatr,ltt.*d uqinc the hvp.~"th~tical eiampIe in Cftr7ptt'r 13 about tlita impact (rn the qtralin' of c d ~ ~ c a t i n r)! n d~\~c>lvcd \.rr.;ur; ccntralizcd sclionl st,^ tfin): sy5tcrn5. In this c*xample tlic quality ot ~ d u c n t i o nis the pl~mnrncnonwe arc wrking tcl csplai~i (.;try 1). O~rr prnvisinnal (partial) cxpfanatinn iq t h a t ~Fr~,nI\-cd .;tnffing s\..;tems 1 ~ 1 1 1 ~7rnducvgreater i r n p r o w ~ ~ inn thr ~~ ~ U J I iry o f t.rIlrcaZ-ton in ,I schoo[ Ihan will ~~t~r-rt~~lizcrl .;v~tern$(%trap 2 ) . 1%'~ would thprcforc anticipate that cases [st-hnols) w i tli devolvtd +ystcms ~vouFclhn1.1. bcttrr ~ ~ u ; l educaticm li~ tllnn thore W Ith ccnSrali7rif svstcm.:
'nie prncr,w might

( ( ~ t h c rt h ~ n ~ ks i n g e q ~ l a l ) ./\ltcrn,~hvt~Fv, we might anticipntc that schools th;lt intrcduced a dc\,ol\,cd st,~ffrngsystr-rn wcluld exhibit an lmprcwemcnt In thr qlralrty c ~ feducatitm and thnt this improvcmcnr M*LIUICI b~ greater than in schools that did not intmducr such a ~ s t e m . WP w o i ~J l thtln sclcct CI caw to tcst our prol.713~1tinn (~tt7p 3), W r might find a schcml that ha< rect~ntl!. ~ntrodlsced a Im-alfy hilsed staffing .;vstenl. Having d c v c l o p ~ da detinition of what c o n s t i t u t ~ s educational quality and worked ntEt horv tn measure thiq, we could then conduct a caqr rtud v to src if the introd i~ction of the ncw staffing systrm had led to tl~r prcdtcted Irnprtm.crnentk. Let tts auppi).~ that thc prrndictclri improvcnients had rn fact tnkcn place. This lends support to our pwpositicm hut the support is t~ardlv ovc*nuhclming and probably wonld no1 convince a .;ceptic (stcp 4) Thc sceptic ct~uldsav that one case hardlv proIVrsthe a gencra! Irnprtlvclm~.ntin c*ducationrrl point, that there rndy have h r ~ n qualitv in all --chonl?: over thc sarnt. prriod, and that qchonls that had kept the old system m i ~ h thave also exhitlitcd an improvement in c d u c a t i ~ n qzlalih*. ~~l At tl~c* vcrv Pen\t thcy m r ~ h .;,iv t that thrv c~>uld hnd a schnol with a ct*ntr,ll svrtcm Illat had t m p n ~ v c d thcir qu,lIity of cld~iiahon OVPT the <amc perltld. They might a k t , itg gut^ th;lt t h c ~ c't7~tltl ~ find cxan~plcsof schnnl+ whcre the lot-al .;v<tem ruaq introducrd w ~ t h d isnqtrnus conqequcnces for cducarional qua lie. Wc wnuld ntrc=d to d n more work - further casc st~ldtes(step 5 ) . Initially we rn~ghtlook for furthcr casw o f n r w t y ~ntrt>duccdTordl staffing svstems to check that the initial casc studv was not ju4t good luck. Let us imagine that we then ccrme acmss a qchool thnt o u r critics a l l i t d d tci - one wherr the ~ n t r o d i ~ c t i o of n the ncw .;=tern had heen a disaster. I t had led to tcachcr, sturient , ~ n d parent disratisfaction, I o s ~ of morale atrd a ctear decline in many aspccts of educational qt~altty. How do we make scnsr of this caw? What is dilfert-nt about this case that shouId makc it behave d i f f ~ r m t l yfrom that which w r predicted pravisionnlly, and so diffcrentlv from t h e other caw< wc havr alrcndv 5tudicd7 Since our study should protluce a compl~tr picture OF each casc and its f o c i ~ and l rultrrml: context we rvill prubahlv pick up some initial clues. How is the 'drviant' sd~ool different from the others? W c might notice ~ m fvaturt.q c ahnut thc pnlcrs.; by \vhlch the ~ l e v n l \ . ~scheme d ~vas ~ntroducc*rl in the school in which it ~ 7 ~ a 1s Inilurc. Fur cxampIe, tcachrrs report t h ~ rt t svas impn.;tld nn thcm Tlruy wPrr not c(>nsultcdand thrre arc nrr p r o c r s r s 01 appcal a~ain.;! thc principal'.; decrsions. O u r linrnch thcn 15 t31;11 t h r success of thrs type of ~chcrncdepends cln thr waV in tvhich it i q irnplemrqtp~-l M'e tllcn Inok back s t our cnrlier caws to qcr i f this c ~ ~ l a i n t ls~ c diftcrent outcnrnc.; Dn the s~~ccc..;.;ful cnseq ,tan4 11uta.: hal47ngdiffcr~nt tmplc.mt3ntationpsnccsw.; tn the clisa<trnu<ctlse? I-et 11.; s l p p o s e that the s u r c e s ~ f u lc~weq irnplr*m~nt~,d t h r scheme nnlv affcr 3 great deal ot fl'tltlc~n ,lnlj qlrr~rt! \ . i s w~drl+~,rcn~l cnn<lbn.;llrnl,nut tht3 c l c ~ ~ r ~ b l l v h
.-?"-'

of thc ~ i l i c m e thc process bv which i t \\lac tn c > p r r a t ~ M'c ~ wo~tlri thcn Ilr in a position (step 6 ) tcr mollify o u r initinl, pro\,~sional prt>pow tion ti) read ~ o m c t h i n xI ~ k r :'Whtn ~mplcrnenfcdin a clirnatil of con.;ul tatit~n

and ctmsrnsus, de\.nlr,ed staffing svqtcmc wrll prtxlirce imprcrr.ernr*ntqin cducntion;ll quality.' Mnr\+evr,r, our ncrt caw CII~~SCS problems. Ills. have Iwated ~ n o t h c r casc ~ ~ h c tht. r c lnc~lly baked staffing s y t e r n also h a s brcn a d~s;~.;tcr. In thi5 case, t-etlwcver, thr 5yskern w a s irnplrmentcd only after cascf 111 const~ltaiiunnrld w ~ t h thr full agrrt8rncnt of ~ t a f f,lnd ollicr.;. mi.; is not \chat ~ v expcitcd. c ll'hy tines !I115 caw not fit' rtlrther r u m Ination ot the case rt.r.c*,ilsthat t l ~ t~chr,nl * hw had a nrbwprincipal appointed sincc l lie .;ystrarn IVJS ~ntrmiuced.This principal i< not t n ~ s t e da n ~ il< bcl~cvetito plav faunuritr.; H r is a l w bel~errcdto victimize particular popular staff. Mavbe thcse aw the rcnsons whv the systrrn d t ~ lnot work. Chrcking back nn t*arlirr cascs, the succew stories all had princip.lt.; rvhn wrbrc widcly liked and trnstc~i. arc n t r thp two failurcs h a v e anvthing in c n r n r n ~ n ? Althr~rlghthe dilfrrent in ccrtain ways, what they h ~ v in e common i s th,ll teacl~er\1117 not IIJVC L ~ ntv . ( i t i t j r f ~ ~f~rc t ~ fit,+ + s!t,td7i~;. 111 oiit, C J ~ I *~t t v ~ q f ~ m - ~ t>n c l tlitvn withoul ~.onsuttatiim.In thc uthcr, t h ~ .prrncipal has trndi~rmini>d tl11.1r ccmi~denccin the t'airntls o f t h system. ~ IVe might tE7c.n marl I @ rwr revised prnprlsition to srlrndhing like: ' Whrrc tt*ilciwrshavt. confidence in thr falrnw- of a devolved ~ t a f t i n g sytern ~t will Imd to in1pr4n*cmrnts in the qualitv of rducation,' We could gn nn with additional casc st~rdics to c h t ~ ortt k how well this yrtrposition Ilnlds. E n additicm lo looking at caw5 whcre n local q t a f f ~ n g yyqiprn was inttoduced we should look at some cases whcrp the centrali7rd syqtcrn was rctainrd. We would ~ v p e c tthat r 4 would ~ not sec the s a m e Icvcl of improvement in edt~cation in thew s c h o n l ~ I . f this prr~vcd tn bc sn it wi7t1ld strengthen our ctlnfidrnce in our initinl throry about thc inipact of local st,iffin~ systcms. We r n i ~ h look t for cases that could di;pro~vo u r thenry. For e u ~ m y l r~f , wi> coulcl find a school that had kept thc centraliwd syqtcrn and had shnwn an irnprcrvrmmt in educatinnal qualitv wc rn~ght2 x 1 a b l ~ ,to f~~rtlle improve r our theory. WP might shrdy a schnol w h ~ r t we * hnvc idt~ntificdthat there Il,i< h e n nn Irnpravrmtnt I n cducatitln btst thc ctntltrali;rtd ~ t a f f i n g s)rstpm had b c w rt\taincii. Thiq wt>~lld tvll a ~ a i nour ~l r c v i ~ c dprcrp{~rh~rn. Hr~rvevcr,further in\ c..;tijiation of thc <cI?tlol rn rsllt ~Iior* that ~ wlien w e Irwk a t thv qchtml in i t s widt\r contcvt ~ h e r c15 J l~o<tite rclaticmship bctwecn thc school ~ n Ithc d Ivs,il cc,nrmr~n~tv. 1Z'c I~am that thrre had b c ~ n n previous attempt b7. t h e schnnl r{n~ncil (on which thrrc. werv manv local cornmun~tl, rcprescntattves) tc, ~ r l t r o d ~ ta re Iucai stnffinq cchrmc. TcacFlcrs had sl~cce+<fullv rc>+i\trdt h i q ant1 Fc~llthat tl~tl ccntrali/~dst0iihngsyqtrm was protest~ngthem. Mfhtli M.P It.ai-n t h a w thing.; a b n t ~ trr-hat nt firkt 5 i ~ l l t1 ( ~ k Itkc s n 'tltw~arilcaqr*' th,il dt+prn\rc*c:o u r propositirm, in facl it t 11rthr.r ctinl~rni.;

266

CASE STUDY DESIGNS

o u r propositinn that strfsses the notion that teachers must have confidence in the system. Perhaps this case would cause us to further broaden our proposihnn to emphasize the importance of teachers having confidence in the system and to identify t h e factors that produce that cnnfidcnce rather than strcss the importance of a local or centralized systenr in itself.

REFERENCES

Summary
This chapter has stressed the pi)iiint that case study analysis should he fundamcntally a theoretically informed undertaking. This applies equally to descriptive and explanatory caw studies. The use of ideal types as a method of analysing descriptive case studies was di.sfrrssed. Two broad methods of analysing casc studies for explanatory purposes were outlincd. Where case studies are used to test a theory, pattern matching provides an appropriate way of analysing and comparing cases Pattern matching can range from matching very simple predicted patterns to highly complex patterns involving mrlltiple independent and dcpendent variables. Where case xtudics are used to build a theory, analytic induction provides a uscful wav of analysing and using each
case study.

Note
1 Of course this Iogic dssumes that our causal variable is the only factor that can produce a given imtcnme. If we find caws where the effects occur but our favoured cause is absent then we have alternative explanations of thc phenomenon. O u r research design should he de5igned to elim~natesuch alternative

explanations

Aronson, t a n d Mills, j. (1954) 'me effcct of severity of initiahon on liking for a group', Joirrrlrll r l f Ablrr>r8trrllni7d Sacrol Ps!/c!t~/~gyr 5% 177-81. Alchley, R.C. (1976) Tlrr S o c r ~ / r ) nf ~ qXpt~r~tnenl. Cambridge, MA: Schcnkman. Baumrind, U. (1964) 'Srlme thoughts on ethics of research: after reading Milgram's "Behavioral study of ubedicnce"', A~ntlrfrfln P~.vc!zo~o&'~s~, I$. 421-3. Becker, H.S. (1966) Orlfsi'ler~Stlrrit~,~ Irl tile Socii~lo~ey 01Dmiance. New York: Free Press. Blalock, I I , (1964) Corrsal I n f e r c ~ ~ c c in s Non Expi~uii~rrntal Re.~wrcli. Chapel Mill, NC: Universitv of North Carulind Press. Iqlnu, P. ( I Q h 4 ) Sxcllnrfgr nnd Pr~tlli,rit1 Sorrn! L ~ f cNew York Wlley Rlurner, H 11956) ' S o c ~ n l o g ~ c analysis ~~l. and thp "vnriable"', A m r r ~ c n ~ Sorir~logicfll i Kr'url.rtt, 2 1 683-40. Rrutvn, C;.W and Harris, 'r. (1978) TIie Socml C7rr~if1s r ~ fDt-prcssinr>:a Sfrrrly of Fs!jrhintrrc Dl~ardrr111 M7onre1r.I.ondom: Tavistock. Ruck, N., Ermisch, J a n d j~nkrns, S. (1995) Choosing n Lvngitudinal S u m q Design: Tlic lssrt~'s, ESRC Rosearch Centre on MicrwSoclal Change, Univers~ty of Essex. Campb~11,D.T. (1989) 'Foreword', in R.K. Yin (rd.), Carse 5tud.y R~rsenrch:D~sign and MriholJs. Rcvcsly Hills and Imndon: Sage. Cdmpb~ll,D.T. and Stanley, J.C. (196.7) E x p ~ r r t ~ ~ r t ~ nnd t f l l Q~rasi-Experimentfll Ursr~~trs,!i~r R ~ w r r c h .Bor;tc>n'Houghton Mifflin. Cohcn, J. (1987) Statistjcnl Poxcr Annlysiclfor the Brlra7)iourrll Scrc!~c~s. Ncw York: Academic Press. Cook, T.D. and Campbell, D.T. (1479) Quasi-Exp~ri~tzmtat~on Dbipu und Annlysis Issurs for Fidd Srttinp. Boston: Huughton MiMin. Cnser, L.A. (1977) Mnslrrs of So~-inlng~~-i~l Thought: Idens Sn Historical and Socrnl Glnl~xt New York: Flarcourt, Brace, Jo~,anovich. Davrs. 1.A (1985) Tlre Loxic of Cnirsnl Order. Beverly I-Iills, CR: Sagc. Dempsry, K. (1990) Smnllfuictt: n Stlrrdrt o f Snrinl I ~ ~ r q l m l r t Cohrsinri y, nnil 6clorrapii~,q Melbournr. Oxford Unikcrsity Press. L1~n7rn,N . (197s) Tliv Rr..sp~truhAct- a I'l~rvr+,ticnl rtrfrr~iilrctrnnlo Rrseflrcl~Mcthods. U e w York McGra~v-I-lill. Il~,partmr~ of ~t f l u m a n %;ertmicesnnd I3ralth (1995) Ynl/th S z ~ i ~ i d1t1 t' Ar~~trnlin 0: Br~cl;qm~/i~d Mrr~~ryrnl?h. Canberra: Australian Government Publishing Servirc. dc V a t ~ s ,D.A, (lq9.l) Lurtlng Go: Rrl~rt~nrrsf~r~~:: b~fior'rlrAifrrlts und thcrv Pnrewls. Melbourne: Oxfnrd Universib Press de Vaus. [).A. (2001 ) S A L Y V C rI iIl~ Social Rt,srurcllr l..ondnn: Routlrdge. d e Vaus, U.A. and LVolcott, r. ( e d s ) (1997) Arrsfr~lrlrrrF ~ n ~ i I'roftlr~: ly Sncrnl nnd Drrrlnyrrtyl~rr Pnlterns- Melbnurnr: Australian Instih~teof Family Stndics.

2hH

REFERENCES

Uillman, D (1978).2117il r~nd TC~PIIII(IIII' ~ F I P ? T I / S flrr . T~trd S ~ P S I , ~ I I Mtatlrod. New York: Wiley. Durkheim, F. (1970) S~liclrif.London: 12outlcdgc and Kegan Paul. Foddv, W. (19931 Cor~sfr~ictrn,y Questinn<fi?r Tr~tt,mic-;cls f l r d Qurstiorrnn~res Thmq 17nd Flucllcr it! Soclnl Res~~orcl;. Canibrtdge: C a m b r ~ d g c Unrversity Press. Freeman, D. (1984) Mnr~rrrrtMmd find Sntlion: flrr Mnkiug niid Un~nokinxof (in A~rt!~rr~pulo,qit-ol Myth. l+armnndsworth: Penguin. Freud, 5. (1955) T ~ VCasp O H~st[?rrt~ (Lritlr Hnns and Rot M111iJ. Londun: Hugarth Press and the Instih~te of Psycho-Analysis. Geertz, C. (2973) 'Thick descriphon: towards an interpretive theory of culhre', in C. Geert;. (ed.), Tlru I~tt.r;~r~totion of Cult1rn:c: Sl,lt>ctrdFsqalrs. New York: Basic. pp. 3-32. Gerth, M.H. and Wright Mills, C. (1946) Fm~nMnx W r h ~ r :Essatl:: it1 Sociolo,y!/. Oxford Oxford Universitv Press. Glaser, U.G and Strauss, A.L. (1967) Tht. Disr-oorrrn/ o f Crour~dedTl~cluoy:Stratcpes for Qlmlitutrrw Research. Chicago: Aldine. Glenn, N.V. (1977) Coljorf Atrrrlvsrs. Uebe~l y H~lls, CA: Sage. Goldenberg, S. (1992) U~rnkirrgMet/~odolo,yrcally. New York: HarperCullins. Goode, W.J. and Hatt, 1' K. (1952) Mr,tlrorls rtr Socirr! Rt-srori-11. 1,ondnil. McCrawHill Graetz, 13. (1993) 'Health consequences of ~ r n p l o v r n t ~ n at n d unrmploymenl: longitudindl cvidcnct. for voung mcn and women', Sarml Scrcilce and Medlcirre, 36. 715-24. C r ~ m m ,I G, and Yarnold, R.R. 11995) Rrnding nrld LFnderstn~~rlrng Xru!t~rlar!ah, Slntrctir,q. Wadlington: American Psychological As<nriatinn. Grnvcs, R.M. (1979) 'Actors and questions in telephone and personal interview SU~I'P~S Pnblrc ', Opri~ionQnnrtrrly, 43, 190-20.5. Groves, R.M. and Kahn, R.L. (1979) Sur~reysI?y Trlrphonr. u N ~ ~ t ~ oConipnrisorl ~ial u ~ r H rPerso~~al Intrruiml~.New York: Academic Press. Hage, 1. and Meeker, B.F. (1988) Sorinl Crru~nl~ty. London- George Allen and Unwil~. Hakim, C. (1986) Rusedrrh Des~gn:Stnttugrus and Clroiut.~In the Design rf Social 12rswrrJt. London: Allen and Unw~n. I-lmry, G.T. (1995) Grirphrng Data: Trchrrr~ytr~s of Disflh!/ and At~alysis.Thousand Odks, CA: Sage. Hkrtel, B. (1976) 'Minimizing error ~arinnce introduced by missing data analysis in survcy analysis', Soc~nlo,~ica! M~thods nrld ResmrcIt, 4: 459-74 Hochschild, A 11989) TIw S~,co?ldShift New York: Avon. Holnnn, R. (19Y1) T l t ~ Ftllrr~ nf Sllcrnl Rrsenrch. London I.ongrnan. t!clnlans, C;. (1961) Sncrnl Rrltavici~rr:[IFElrmtbnhory l ortrr?. Yew York Harcorirt, Brdce, luvanovich. Tnhnsnn, h, (1977) Sac:,ll Sfnt~strc5 rt<rf/~c~ f r Tt3~?rs. r N e w Y n r k McGraw-Hill. Kalton, G. (19833) 'Compensating for miss~nfis u n e y data', Research Report Scr~cs, lnstih~te of Soc1a1 Rcscarch, University o l kllchzqan Kalton, C; (1983b) ltrtrod~tcfiurtto SItn~e!/So)npli?r,g. Beverl' HIITS,CA: Sage. Kalton, G. (1086) 'Ciandling wave notl-response in pancl surveys', Jorrrrrfll ~ r f C?fir>nlStof7sf1rs, 2: 303-14 Kt'llehear, A. (1947) 7 . l ~ililclbtr~r.iuc Rmei~rt-lwr: Grr~dcto ~ u f h d d hSt . Leonnrds: Allrn nrtd Unwin

(if Br~ltlr~lrc~rir,~l Krsrarrlr New York Hott, Krrl~ngrr. 1.R (1973) Frlrrndnt~n~~s R~nehart and Winston orrr /r N c r ~ York Kerliilpr, t- h (1'179) Br'lrnrrror~unlRcssn~rrlzr a CunctVptunl A l r p ~ Holt, Rtnrhart and Winston. Kerllngt=r, F.N and Pedhazur, E.J (1973') Mrrlhylr Rrgrt,r~m)r $17 Lir'l~arj~tlrrrn! Rr,~cnrcl~. New York: Holt, Rinehart and W ~ n s t o n Kimmel, A.J. (1988) Eth~rs nud Vulurs rn Applied Snctnl Jirst,arc!r Reverlv Hill< and London: Sage. I rpkowsk~, J1 4 .(1989) 'Treatment of wave non-response In pancl surveys', in 13 Kasprzyk, G. Duncan, G Kalton and M P S ~ n g h(cd?), PrfnfulSIITTII,~F New Ynrk iq~ley Lewrq, O (1951) L ~ f il r r n Mrxrcnrt Vllln~r. Trprrztlnn Rrsttrrl~~d. Urhana, IL: Universlty of Fllinois Prcss. Loether, H 1. and McTavish, U G (1974a) Itlfcrrnl~nlStattstrcs for Soric~lug!~ts. nn Inlrod~rctiot~. Boston. AElyn and Bacon F .nether, F1 J and ,McTavirh, D G (1974b) Ursrripltur~Statistrrq Jor Socfo!o~ylst~ nit Iiitrodrtctron I3ostnn: Allvn and Bacon. Meriell-Lynd, H (1g29) M~~Jdlufoilln: n Strirly 1r1Modcr~rAntrrrcnn I ynd, R.S. Uzr!tlric. New Ynrk: Harc~rurt, Bractl and World I.vnn, P., Purdon, S., tlcdges, B dnd Mcaleese, 1 (199C) 'An asws3mcnl of nlterna!rve ~ ' c i g h t l n g+trdttbg~rs',Yolith Colinrt Surr cy Report, Fcf iicd h o r ~ Dt.partmrnt, Sl-reff~ld Mardm, D. and Abrarns, 5 . (1E)87) ' L ~ b c m t n ~ compnntolls, , t n t r u d ~ r sand cuckoos in t h r nest: a soc~ology of caring relatlr>nshlp$over the I ~ f cycle', c in P. Allatt, T Ked, A Rr\)nlan and R Byihewaj ( c d ~ )Wolnrr~ , orld t/tr L l f i Tyi!~ Tmnstfit>n::ni~d Turnrng-Pt>117tsLondon Macrn~lldn Vvlarsh, C (19B2) T~IC Satnjcy M ~ t h o d the Co?;tnhntrotr o f S u n ~ r ~ , fu c Si~rnlo~rcfll Eup1n)ratEun. London: Georgc Allen and U n w u ~ . Marsh, C. (1988) txplorrng Uatfl. nn Introdurhan fo Uotn Rrlalqstc for Socrul Screntr~tsCambrrdge. IJolzty Mmd, M. (1947) Covtlil,qof Agu rn Sanron o S t ~ r i i of j A d u l ~ ~ ~fwd i~n Scu c~ 111 Prltrrrtirlp Surrttrt? Fhrrnundswurth' P c t i p ~ i n . Menard, S (1991) LOngitz~CIr~lol Re~tvrch. hrerubury Park, CA. Sage. Merton, R.K (1968) Socral Throry and Sarol Slrrrttr~vr Ncw Ynrk F r w Press M ~ l g r a n ,S. (19M) 'Issues in the studv of obedience a reply to Rsurnr~nd', Arnrurc(ulrll Psyd~ol~)grst, 14 848-52. Mjlls, C W. (1959) TIlr Socrologri-01 Imnglrraflon New York- Oxford University Press Mi tchrll, J C. (1983) ' C a ~ e and situation analks~s',Socrt~/ryrrol Rnllnrl, 31 : 1x7-211. Moser, C A and Kaltun, C (1 971) S I I M~Hrodlr ~ ~ 111 SOCIIII I~f~~strxnt1011 London Hrlnemann Mlreller, I H., Schuessler K r and Cosmcr, H.L. (1977') Strrhsf>cn/ K I ' I ~ S O I I I I!II ~ Srrc~olo~~r Ibston Houghtrjn h'liifl~n I'unch, M (1W4) 'l%I~tlc+ and etll~cs In q ~ ~ a ~ ~ t ~ t rr<enrch', n t i v c In N Denzin ntld Y. L~ncoIn(cds), Hiri~iflmoF. ot Qtralifoti~r Reqarch Thou5and Oaks, CA S d g ~ pp 83-97 Rrdftcld, R. (1'130) Tl~puztlnn,o M1,rrcolr Yrllyiy 17 5tlrti1, of Frdk l @ Chicago Unrvi.r<~ty rrf Ch~cago Pres rurd ~l~i Iht' i t Wnrhrr. Riwthlrsbcrgrr F.l and Dicksnn, W T (1919) M ~ r r f a ~ ~ ~ r Carnbridgp, MA: Harvard Universlh, Prrs5.

IZEFERENCES

Rogers, T.F. (1976) 'Interviews hy telephone and i n personm quality ot responses atid firld prrfnrrnancc', Ptrhlir Opitrton Q~tnrtesll/,4 Q 51-65, liose, O., Duck, N.A. and Corti, L. (1941) 'Design !<sues in the British Household Pallel Study', Working Papers of thc FSRC Research Centre nn Micro-%cia1 Changc, Papcr 2, Universitv OF Essex, Colchcst~r. Rosenberg, M (1968) Tlre L n ~ r cnJ S u n ~ t y AAn,y~ts. New Yurk: Basic. Sprctor, P I . (1981) Rrsrnrcf~Dr51~~1s. Beverly Hills, CAm Sage. Stake, R.E. (lqY4)'Case studies', in N.K. b n 7 i n and Y. Lincoln (eds), Harrdhnok of Qtinlitnfr-tw Rcsenrch. Thmsand Oaks, CA: Sage. pp. 236-47. Sullivan, L., Maley, B. and Warbp, M (19Y7) Sfafe o f tl~rNat~nn: Stotiqficnl Irrditsrtrbrs rifAu.;tnrlia'c We11 Rr2ing.Melhorlrne:Crntrc for hdepcndcnt Stucl~es. Supprs, P. (1470) ,4 I'roballrEistrc TI~rnr!, of Catisnlilu. Amsterdam: Nnrth Holland. Tilbachn~ck,B.G. and Pidell, L.S. (1983) U s i q M~ultivnriateStntislics. New Yo&:

NDEX

and Row. 1a jfeI, 1-1. (1984) Tt~c Sncinl Ditnrnsiorr ' European Wev~loplnerlts i~ Socral PsyrJro10~~. Cambridge: Cambridge University Press Vaughn, D. (1986) U)7corrplmniy: l'rrr~zrtlgPoirrts in Il~tinmtrR~iatrotrsh~ps. New Ynrk: Oxford Un iversitv Press. h'hytt*, W.F. (lq43) S t r ~ e lComtxr Snrirty. tltt, Soclol S ~ ~ I I C ~ ofI on I M Itnlimi S l ~ t r n . Chicago: University of Chicago Press Yin, R , K . (198q) Ct75r. Strid!/ Rr.~cnrrl~: DPS!,~ rind F Mrfhods. Rcverly Hllls and
Harprr

A prlnrl reawning crrjsc wct~onal LIL*~ 178-9 I~S, Abstrncted ernplrlcrcm, 2

Ad hnc rPd5Clnln):
cro.;s seft~onal dcs~gns, 174-80, 182 Agc ct,lir~rt \pan, 71 2, Ageing efiectc, 115, 124, 211,212-17,

215 Inn#iludmal designs, 172

hclltmr~n4 FrOUF 11t.slgn.173 Ag~regaie cllanpe Ag~rcjiatc chdnge, 122, 128-9. 131, 135, 137,
1h7-54, 211

Bias, TI attrltlnn, 75-h c f k ~ on t descriptic~n,136 effm-t on exylanatic~n, I?h ~dcntifving missing data bins. 149 r~mol~irrg stnt?shcnHy, 1.75, 136, 152 tcstinx for effects, 77 r w i~ l t I ng, i 15 1 -2
CAI'I, 1.11-2 CASC bSL1dlCh
cl~nlc~d 2 ,2 3 4

London: Sage Yin, R.K. (1493) Applirnhvr~~ o f Casr S t ~ t d yRrsmrcJz. Beverly HiIIs and London: s<1ge

dc~cnptn~ ac n ~ l y s i s25n-3 ,

~rdplis,163 methods of analy<lq, 151-4 repeated crosssrrtlorl dw~gns, 153 h h l ~ 167 ~ , 16-4 Alltxation to grotips, 44-h,48 hnalvhc mdurhnn, 263-6 six steps. 263 Anr>nyrn~v

rxplana torv nnalvc~s, 253-6A prcwntlng, 214-5 report!ng, 244-5

thcorv building, 267-6 I henry tesllnR. 25-67 Casc study d c ~ i .50-1,21$-% ~ , fypPS, 728-') a n , 141-2 Ceuyal direct~nn cross scctiunal dcsikms, 180-1

experiments. 87-8 cross s ~ t i n n ad l e s i ~ ~192 s,

cw also ronfida~tinl~tv nnonvmizing data crt~s sectional designs, 192 Arhficiality, T',
hssoc~ation,measures of

C~usal effec! dirt~t, 65, 71

indirect, 71

cr~' crvrdatinn, ctr-variation Attrilrnn asscssin~ cffects, 123

bias, 75-h
i n ~ t i nsample l s17e.14.1

total, 71 Caufal explanation, 2-4 diffcrcnt tvpcs, 37-Y dirrct effects, 2-4, 36-4 ~ndirect. 3-9 ~nft~rnn c;l~~sc, g 4,344
mle of cornparisnn groups. 79-47 Cau-al rids t i n n s h ~ p ~ crltrrla fnr l ~ f t ~ r c n c9 c, . ( > direct and ~ n d ~ r ~76-9 rl,
Causat~on, 4-5, 3 - 5 1 a prlrti-I reasoning. 178-9

niinimizing c i f ~ ~135-6 h, rnlntm171ng,I T - q nori-r,jtidr\m, 135 panr.1, 138-4

rrplarem~nt~ 136 ,

rrspondrnt hurdrti, 139-411 tmckirl~ panel m ~ m b e r s 138-9 . n n ~ non t response, 147 Avcragc pnrl~alt,iblc, 104

;rd hex rensonlng, 17Y-HI). Tll2 correlation and cause, 178 dctcrminirtlc, 3-5
~ < t n b l i < h ~ causc, n~, 7 4 4 7 , 178-81

INDEX

INDEX
Defmi tions norn~nal, 24-7 upfrahonal, 24,27 Demand characteriqtics, 24 Dependent variable, 1936 Description, 1-2, 114-5 cross-sectional, 194-2MF deductive typologies, 225-6 descriptive. research questions, 17-19 empirical typologes, 225 idea1 type$, 225-6 inducttve typologiec, 226 stanstics, 89-90 theoretica1 typologies, 225 theory, 224-5 lypologi~s, 2246 w~th case studies, 224-5 hwriphrre analysis cross sectional data, 194-218 Distr~butions, 195 ccntral tendpncy. 195 frequency distributinn~,195 graphs, 145 [deal typ! analysis, 251 irnplicrl theories, 251 kurtosis, 195 shape of distrlbuhons, 195 skeu-nesq, 195 spread, 195 suhjectiv~ ty. 250-1 typologim, 252 Developmental effects
see ageing e f k k Dirtct relaltinnships, 36-9, 65,71 SPP nlso causal cffect D~stributions, descrhing centra1 tendency, 92-3 proportions, 95 shape, 94-5 var~abllity, 934 Double-blind, 83 Dropout w e atli-liton nynam~c samples, 137, aggregate change, 137
Effects

Uausatlon (cottt

cmss sectional d~stgns, 177-81 matching. 178 model f ~ t t ~ n17Y-8n g, probahllrstic, 4-5 rcc~procal, 36 statt~tical controls, 177-9 type? of causal patterns, 36-9 S Pa ~l~n saucal dirirhon Ctnhal tendency, measures of, 92-3 Change graph? SL,C graph" Change measurement, 153-63 calcuIating raw change scr>rrs,155-6 cross-sect~o~%al, 171-2, 17?-5 experiments, 53-6 measurement error, 156 multrd~rect~onal, 154 perccntagc change, 156-7 problems of raw change scores, 1% qualitative change. 154 quantitative changc, 154 raw change scores, 155-6 reliability and raw chanxe, 156 residual~~e change d scores. 156 ' t s nlsu ag-grepitcl rhange, indtvrdual change Change scorec, 1 10 Chanh~, descrthmg, 163-8 graphs, 1h3 tables, 163 Chronotog~cal anafys~s, 261-2 Cohort analvsis, 115-18, 21 1-17 age cohort span, 212 ageing cffectc, 11 5, 21 1 aggregate change, 211 cohort effects, 21 1,217-15,216-17 constructing tables, 2112 perir~d efi~cts, 115-lh,Z11 problemq, 217 reading cohort tables, 212-17 repeated cross-sectional dmign, 21 1 standard cohort table, 212 unevenly spaced surveys, 215-17 Cohort scquent~al des~gn, 124-6, 133 panel agetnE, 124-5 Cohort tahlrs st*^ cohort anaivs~s C~7llapsingvar~ableh ser recod ing Cornpanson" dtffercnt methods, 40-7 group?, 09-47 meaningful, k7-B multiple, 41 -2
8

rnultipt~ c a w studitr, 22h means, 1YQ on? shot rase slurly designs, 230 time points, 42-3 Computer assistcd dota crdlcction, 142-2 Conccpb clnrifyirig, 24-7 dimensions, 25,Zh nature of, 24 CATI, la 1-2 UAPI, 141 -2 advanta~es in I o n ~ i t u d i n ~ studics l Cnnditinnal tahl~s, 204-5 Confidence interval, scc sampling Confidence Iwel, srr sampling Cunfidcntialitv data storage, 87 expcrim~nts, 87-8 cmss sectiorral designs, 192 Inn~ihtdinaldesigns, 146 1cg.tl contcxt, 87 limits, 87 s~r, lrlsll anonymity Context c a w study nnaly\is, 234-6, 250

seplrcatron, 205-h q p c ~ f ~ c a t i o204-5 n, spunous rclahonehiy, 204. 207 steps, 203-4 types of causal pattern, 36-9 zcro-order r~lationqhips, 2114-10 set also d~rect relationship, causal effect,
causation

Control groups
strengthening rntcrnal vaIidity, 73-5 Correliition, 95,

Cost
studies, 24% declustering. 141 1ongihidinA designs, 140-1
casc

Co-va ria lion, 34-li

Covert t>hsc.rvntion,246 case studies. 246 Critical cases, 227 Cross sccfior~al designs, 50. 170-218 cf lr)n$udinal, 1711-1 group differences, 170-1 intcrvcntinns, 172 nature of 'groups', 172-3 rcpeatcul stud it!^, 173-5 tlmr rlimension, 173
Data serurily

[ongitudinal d c s i k ~ s146 ,
.;PP

c~l>iidp~itialt!v

Dcucptiun, 246
cast= R ~ I ~ ~ 246 P s , disguis~ng identity, 236

Srt, c'nl.~o irnfrvmcd ctjnscnt Declusteri~rg, 141 DPductive rpasoning, 6-#, 253-62
typnlosirs, 226

short and long term, 79 Elaboration analysis, 1IM.203-10 mndlbona I tables, ZD4-5 control variables, 2M indirect causal relationships 204,
206-7

Ernbddcd units uf analysis, 220-1 Equallty of vanance, 103, Error SPP measurement error; sampling ermr Ethics experiments, 83-8 anonym~ty, 192 anonymr71ng data, 192 ct~nfidcntialityand anmymlty, 146, 192 covcrt observa tinn, 246 data security, 146 d~ceptlon, 246 dtsp~ismgidentitv, 216 harm lo participank, 145, 193 illegal/harmful behav!our,246 cave studies, 246-7 crmss sechnnal designs, 192-3 informcul conwnt, 145,246, 85-6 longitudinal designs, 144-6 partrcipant observation, 246 respons~bility to other researchers, 247 voIuntary participabon. 79.145 see informed consent, voluntary participation, anonymity, harm to participants Experiments, 53-1 12 analysis, 89-111 and explanation, 70 artificiality, 77 clas~ic design, 48-9, 53-9 ethics. 83-8 field, 57-8 Iaboratorv, 56-9 natural, 58-q 5eattEc/Dcnver lncome Maintenance E x p r ~ r n e n t57 , types of dc-ign, 53-69 Explana han, 1, 2-5 cf prediction, 3-4 cxp1anatc)ry narrowness o f cupcrimcnts,
711-2

interaction effects, 204-5 problrms, 210

~diographrc rxplanation. 221, 233-4 nomuti~chc cxplanahon. 221, 273-4 roie o f expcrimmts, 70 w~th case studies, 221-4.253-63

274

INDEX

c r r w wrtionaE ct t.xp~.rimcnts,IK4 rrt~)rs sccti(tnaE c f pallel dudips, 184 casc studl: clesip, 237-411 cross scctlcmnl design.;, IH4-5 cxperimmt5,7h-8 longit~rdin~rl rl~~sigps, 135-K inln~gratlcmand outrnigratron, 137 panvl attrition, 135-h renctiviiy, 7h repwx.nt;ltivcnL%s, 1 P4-5 5t;l trstrcal cpnrralixalirm, 237, 2111, 249 iesting for bins cffcctl;, 77 thcorr~ticnl~ m r m l i f a l i n n227, , 240,
Z4Y

tl.1wthon1c ctfcct. 77 History, 132 ~mxpvrimcnts, 73 CJ\C s t u ~ l ~ t236 s. c r w s scctir~na I d~sikm5, 171 [ongittdinal designs, 132 Mnli?Iir units o l annl\rsis, 22U-1

tlirents tu. 76-7 unrr-pr~wntativfsanlplecl, 76-7 Ertrancoiir variahlcs, 19-,20, 37-4 Factor ,~nalysis,lL)B 1:;lclnriat a n ~ l y s i ui i vnrianr~ ~ L > L -I r w - \ v a ~ an;llyc~k of vari~ncc r:;lctori,>[designs, fji-H, 103 I',~llacy o f aiiii-minfi tlw ron.;rqutqt, 13-14 F;rl.;iiira lion, 14- 15, 267, 2h.5 closed thcr~rieu,14- IF; Fcminlst rprrarch ethics, 217 ]:loor and cciljnfi cffccts .,itrtruni-alim ~ i l m - l s I:OCUSW~ s,?mpling, 240 Framcb nf refcrenw, 3 9 4 7 5m lrlst~cornpiris~~ns Frcqu~ncy iiistril~\itiont;, 145

arv nlslo 0 p t - rtirlnali~atinn ~ lndlrect causal relationships. 36-9, 71 t.laboration ~nalyi.;,204, 2Dh-7 lndlvidual level chc~nge. 122, 129-9.134. 175,
157-54

grapt15, Ib3 1npthr3d.; o f nnalvsic, 1 5 1 4 ~nulttdircctional ct~~angi-, 151 panel d r s ~ g n s 157 , t<lt)lt.h,163, 1M Induction ,inalytic, 2411,263 -6 mumeratlvc, 240 lndtlcti vc typologies, 225-6 Inductive reafonlng, 5-h st? nlso nnalvhc ~nducticln, mdwctlo~i Inference srr causal rrlatii)nsh~ps, thcnry htt~td~n~ Inferential stati~tich, 89-90, 99 Informed consent, 7H,P7, Zlh rasp studleq, 24h d e c ~ p t i o n7 ,R how much infin-n~ntiun,R.4 erperunents, 64-6 l(m3rttldinal dc~ikms, 145 IirnlnK, H? who frrlnl, 8.5-6 lnm~~rntiom, 120, 1 : 7 It~~trunirn c tl ~ r a v ckpvrrmcntal, 71-5 SFPSS srrtronal d ~ s ~ p nI77 s, I a ~ n p t t t d ~d nc ~slr p \ , 1W,131-4

Intcr.~ct~rln cfftrt.;, 42, 55. 65-6. 73, 74. 103, 1114 clabr~ration ~ n ~ l y s i20.1-5 s, Internal vnltdity, 27-8 ad hr>c t*xplanaho~rf, 182 cause and correlation, 234 uar~able analvsis, 3 - 6 crlntext in cace studies, 234-h cross sectiorral dcsibm, 177 pffectof no randoln~7eJ control group, 131 cqtahlishtng rauw, 177-81 pft.lbl~shlngmvaning, 18 1-4 !11510l y, 77-4, 132, 177, 276 ideo~aphic explanations, 233-4 race s t u d ~ e s233-7 , cxpcnmcnk, 72-6 long~tudlnal deslgna. 131-5 Instrument d ~ c a y , 7-1-5,1334, 177' rnnttrrat~on, 73-4, 132 n?r*asurt,ment frror, 134 mnrtnlitv, 75-6, 1.14-5, 177 nnmntlietic explanat~nns, 273-4 sclc~hr>n, 73 stalislical regressron, 75, 134, 177 tcstlng/pand cnnditionlng. 71, 17.7. 177 unlly of casw, 234-5 Interval estlmntrs, 2tH Itjtewals attrition, 79 hetwwn t.xpcrimt.nta1 s f a ~ e s79 ,
memory rflert~, 80

Inter~~en vnriablf, ~ n ~ 19,3h I n t ~ n ~ e n t r o n47. s , 172 nctivc, natural, 47 types ol, 48 uneven, 82-3 Ilem non-response, 147 I-adder of abstractton, 24 Level of rnpacuremrnl, 40-2 interv,~l,91 nnminnf, 91-2 ordin.11, 91 Lifc ct,ur\r analvsi.;, I lt; I .~teraEr~pTlcatior1,262-3 Lon~itudlnal dc\irm\, 4g-50, 1 1 3-hq c f cvpcrlrnrnral d e s i ~ n 113 , dlnieiisions rrf, I l ? l p n f l cf trend, 113 prospectwe c f rctrr~~pcc~ 113-14 rc, purpr~o rf , 11;-lH Macro changr srr aggreaatt? change Mnun effcvtb, h5

M,i>kin~ relationfhips. I<>(>-7 Ma(rlicd fluup<, 44-5, 75 r x p m t fdctn, 44-5 cross sectional d ~ r ~ g n171; s, ~labnratlnnanalysis, 203-1 R matchcd block d e s l g c , 47-6 post test, 33, 4H.hl p r p - k t , 14 sandomisa tion, 45 statislical controls, 202-3 Matur.it~nn wsc sfudics, 23h cross seciiunal des~jins, 177 cxpcrirnents, 73-4 Iurigrtud~naldesigns, 132, 133 Meaning, 5, 11 establish~ng meanins, 181- 4 Facesl~wtscxrolt>gy, 182 case snldv annlyhis, 2 3 4 4 , 250 cross swllnnal designs, 1814 pro\~dinfimcaning, 1H2 variable analvs~s.181 Mmsr~rt.menlprnlr, 24-71, 84, I DJ, I Sb, 157 conqtant error. 31 corrt4a led PrIvr 72 term* of, 31 -2 Inn~~tudunnl drsipis, I34 random crrrlr, 71 Measuring changt, bcr rhange mmPa5uwrnent Micro c l ~ a r ~ g e qt-r indlv~dual cliar~~r Missing data, 117-53 alprage o l span rmprltatlon, 150 bias, 148 dealing with, 14s group means ~mputatinn,150 ignoring missrng data, 151 imputatirm Fmm earllr=rwaves, 151 ltem non-response, 147 prrwise, 151 randnm wstgnmcnt within ~ O I I P F irnpiitat~o~ 150 ~, regrrssion haqecl i r n p ~ t ~ l b o151 n, wmplp mean< Irnpul.ltion, 1.19 sample s v e , 148 scalc ct?n.;tructinn, 14X Gnurrpg, 14; unit IIU~ rt3sponsc,I47 w a v r nnn rpsponse. 137 CIY R / W nnn resptmfe Mortdlih, 134-i cross x ~ t i ~ n idl ~ l s i ~ m177 5, e k p t i m c n t s . 75-6 lo~lgit~~d d+l+it;ns, i ~ ~ . ~ t134-5

INDEX

277

Multiple jiroups multiple c a w study design, 22h multtple coI>orl dcstgn. 124 multtplc comparison test?, 101 multtple group cnpcr~rnental dwp, 62-3 Multiple time points multiplc potnt panel d e s g n , 119-20 multipkr yre test experimental dfijgn, 62 Multivar~nte anaTvsi5, 211 mu1 tiplc. regrcsslon. 104 Net change
sw

aggregate change

Nomlnal d ~ f ~ n i t i n24-7 n, Nomothctic explanation, 22,22 1, 233-4 Yon-pararnetr~cstatist~cs. 1U2 Yon-probabilily snmplmg, 90 Yorrnal distrrbutiun, 1UZ-3 assuniption of. l(12 crttical ratio, 1W t-tPStS. 1110 Yurmall~ed distributions. 102 One grrl~tp cornparirons with known value, 97.99 One shot case s k d y design, 230-1 Operational definlt~on, 24, 27 Operationalization, 24-27 Clutmigration, 120,137 Oversampt~ng, 143, 187-8
P a i d sample design, 102 Panel agcing, 124-5 Panel condit~oning, 143 assessing cffi~k, 123 Panel data, analysing, 155 Parallel casc study d c s i p s , 227,242-3 investigators. 242 research protwds, 242 Parametr~c stat~stics, 102 Part~clpant obccrvat~tm, 246 case s h l d i ~246 ~, Pattern matching, 2553-60 P?r~Wit;lg~ change, l i h - 7 calculal~ng,156-7 rnPnsuremcnt crioc, I57 probpems, 157 reliabil~v, 157 Pcnod efferts. 121, 115-6, 221, 21 1-14,216, Placebo, 81,83, l'!ous~blc rival hypotheses, 9, 1 1-14. case shldies, 224 sourcc nf* 21,22-4

s~thst<>nkial arrJ theoretical types, 22-4 tedinical and rnctlio~icllu~,ical, 21 Post test only wirh conlrul group ex~~rintn~ MI ts, Prcdictton, 3-4 I'r~vacy 5r.r' confidcntiality Probability sampl~ng, 79, 90 I'roof, 15 Provisronill nature of suppnrt, 15 Proportions, 95 Propositions. b H Pruspcrti\ c d c r i ~ m cnw studv dec~gnq, 227-8 disndvantagcs, 127 rnult~plr poitll panel, 119-20 simple pinrl, 118-19 QuaI!latlrre, c f quanhtativc. 10 change, 154 data, 191 casc stud ips, 230- 1 ~~~erimm 83 ts, Quantitative ct qual!lativp, 111 change, I54 data. 191 rnsc stltdtcs, 249-50 methods, 1U Quahr longitudinal designs, 12H-4 aggregate change, 12R-9 indivtdual change, 128-9 Qwstronnaire length, 190-1

Random error, 31 I<andrjm samplingscc prob,ttrll~ty sampling Iiandomizcul assignment, 45 attrition, 8l,1(2 control v o u p resentment, 81 gatekeeperp, HI group S ~ J P ,81 I ~ m j t of s random alltwation, 81 problems, 71 -2, 80-2 r c f t r ~ l s 81-2 , tinderestiinatin~ ~ , i u s , cffprt, ~l 71 -2 R a w diange Ccrjrc5, 155-6 ~alcul~itin)?;. 155-h cxtwrne scorph, 156 mpasurpment frrnr, 1 % pmblems. 156

RPactivity case studies, 236 Recall anchor events, 128 memosahle events, 128 problems with, I40 recall problems, 127 rel~ospectiw desips, 140 r c v e s c telescoping, 140 telescoping, 140 Rpcoding, 196 dishlbutionaI approach, 197-8 masktng relabonships, 196-7 substantive approaches, 196-7, 198 Rword I i n k a ~ dcsips, 128 confidet~tialitv, 128 Jteltabllity, 29, 30-1, fK changc scares, 156. 157 IZepeated cmsy-sectional d~siw, 129, 137, 21 1 Replacement for attrillon, 137 analysis irnplicatirlns, 122 strategies, 121 hplicahtnl, 77, 78, 226 case studies, 226, 237-8 elabora t ~ r m analpcis, 203-6 exper~menh, 90 literal, 279, 262-3 theoretical, 239,262-3 Rppnrting case stud~es, 24-5 Research design cf method, 9-10 cf work plan, 9 definition, 8-9 dimension5 of, 47-8 Research question, descriphve, 17-19 explanatory, 19-21 focuss~ng, clarifying, 17-21 Rcsidualizcd change scorns, 156 Respondent burden, 13940,148, 140-1 lictrmpectivc design4 anchor ~ v e n h 128 , casc studics, 227-8 experiments, h0-1 memorable cvenh, 128 panelf, 126-H recall problems, 127 re% erse telpscnp~ng, 140 telescoping, I40 Rotating panel design, 122-5, 133 attntlun, 122, 12.3 avording burnout, 122 condihoning effects, 123 external vnltdity. 137

hard

k n frnd groups,

123

pooling panels, 127

purpose, 122 %mplc size attrition, 143 crms h~chonal deslgns, 187-90 dctcrrninants of, 187-188 elaboration analysis. 210 experiments, 78-9 missing data, I 4 8 oversampling, 143, 187-8 random, 79 single and multlple case stttdy dcsignq, 22h-7 statistical power, 78 subgroups, 187-8 Sampling, 14 access to cases, 242 caw s c w n i n g , 24 1 case qtudy designs, 240 confidence tnlerval, 189 cnnfidenrp Ievc.1, 189 external valrditv, 29 focuswd, 2-11 number o f cases, 2411-1 oversampltng, 143, 187-8 prvcislon of estimates, 188 sampl~ng Prror, I # selection in case studies, US-40
theoretical, Z40

typical caws, 240 unrepresentative, 29 variance, 190 weighting, 188 Sampl lng error, H9,92,99, 188, longh~cllnal deslgm, 143 Scale constr~rction,148. 151 Stepticism, 11-15 Scheffe test. 101 %,lection o f cases, 2313-41 access to cases, 242 case screening, 239, 241 -2 case studies, 238-40 number of cases, 240-1 sdrnpling, 240 I;electmn, 75 %;eft-fulhll~ng propl~ecy. 83 '+qucntial case study dcs~gns. 227 Shape nf distribution$, 94-5 S~mple panel design attrlhon, 120.121
h a s , 120

inmigmhun and outrnigratiun. 120 samplc size, 120

S i n ~ p t tp~iriel * tlcc1i:n ( t - m ~ l , ) wit h ~ C ~ I J C P T I I N ~ I , 1 3 - 2 $viIlinul rrplacrmm~t. 120 S~rn~rlatcd bcforr-.liter d r s i ~ n , l?Q i i n ~ l ca5p r shldy dcsi~m. 2 2 ~ Sin~lc culinrl cle~ign, 177-4 a ~ t - i ncif~~crt.;, fi 124 pcr~ud ~jil'cl~ 124 .

llir.or!., 5 cu po.;L fnrlo, h

Validllv (cr~rt ) crit'ric~n groups, 30

rrit~rinnv a l i d i l ~2Y , rxterndl, 2 8 ' 1
internal, 27-H natui-c of, 28 thwats, 24 sirb nlslr internal validity, external validity Variability, mcasurcs r)i, 43-4

ektcrn.~lv a l ~ ~ i i t34 v, intcmal i7alidit\.%4

lon~iludinal dcsijinc, 145 probleml: frnm, %.I

post tnclun1, h
c~se study .~nalvsis,240-2, 753-hl) case study drsihm, 121 -h ~ t . 1 . nl.40 tl1ecv-y h~tildrng, thmry testing

W a w non rc\pon\r, 147

U'3vr'l inh?nf;rl\ htween, 142-3 nilmber In long~tudrnnl studzes, 1 4 2 pinpl r o n d ~ h o n l n143 ~

Thcorv building, 5-6

annlylic induction, ?6D, 263-6 crlht' ~iudivs, 227,227, 2h3-3 Theory testing, 6-3, 11, 253 rase sludy tlesisms, 221 -3, 227,

pr~ihlcn~>, 124
Solomon .I-grtwp desijin, h3-5,

HI).

103
lestin~ ell<brls, 71, fh Sprri ficatitw, 2114-5, Spur~oul;rcl,ilinnshipx. 33, 37-'1. 44 vlahnwt~nn nnatysis, 2LI-I. ZU7

Variable analys~s,1H1,234-6 Variahlc. bv casc matrix, 185-6

Variance, samplc, 100 iQoluntay p>rticipilivn

253-hll Tl~ick ~lcscripti~~n, 2.711

'I'irnr orderetl an,~lysis, 35, 1 1 :

rcspclndent hurdcn, 143 clletghts, 7h.3 7h. kHH mi>sing ci,?t,? bid., 151-2

bins, 84
experiments, K - 4

Sti~ffi~ig
lnn~itudlnal designs, 144 Standariiiratltin, 157-hl inllalinr~ adl~~slmcnt. 157-0 inhtruincnth, 17s ~ * ~ c ~ mnk, ~ l t I~ V l- h c1 prtiynrlion.;, Ih(V-1 SC.I I II S, 157 -h 1 vnriablt,~. ILK-s ~-5cnrcs, 159 - h l Statistical cnntrol5. 162, IW atlju5tinp ftlr mi.~hing data bias, 152 cC ~xpeririierital controls, 201 -2 1~1,lhr)ralion analysis, 313-1 [I crms scctiunal cicsihms. 1 7 7 4 trlgic of, 201 mi> tct~ing,202 nu~TIiylr, 200 Stnt~stiwl gcncr.~lizatir>n, 237, 210, 24Y St,~tisficnlinfcrencc, 2110-1, i n t ~ r v aestirnnles, l 201 trsb of significance, 2 1 1 1 : Shhstical pwrer, 78 Stntistic,ll mgrc~sim, 75, 134, cro'rs wctifmal designs, 177 Irrngitudinal dr.;ips, 134 St<~ti.sticaE curnmnrics, 97, ' 1 9 Srlhgrrrup an,~ly.;ih, 1.V-H

biogmyhy, 251 cnsrnl;t~rd ivs, 752-.3

chmnnln~ical ,~nnl ysis, 261-2 ct~ronoIr)girs, 2.52 hislories. 252 i ~ i t r - r n i ~ t tinw c d s~ries, Zhl linir ~ t ~ r i annl\sis. ~'. 2M1-2 twnd onalysi\, 2MI Tmc klng paricl rncrnbl,crs. 13S-? I - l t ~ cvht, 1111 dccl~r%tcrrng, 1 4t Tr.insk~rntt~d distrjhut~on';, 102 nor~nnlizcrl cli<hibutiom, 102 Translorrnrrl variables, 1'17-8 Trvilhn~nb s r t 7 intervenkions Tr'nd andysis, 111, 2hn T'ruiication etlects, I 10- I Tu.rrway an.iIysis r,f vasi.inw, 104,

110
Typcho~ies,225-6, 252 cluskr analysis, 252 deductive, 225-h rmpirical. 225, 252 ind~ictivc, 226 fhenr&ic,~l, 225 Unit non-rrcpon.;~, 137 Units r 3 f ani~lvsix,18 rrnbrdd~clunit.;, 22i1-1 holi5tic ~ l r ~ iZZnk. I c a w sludirs. 2 0 Unobtr~rsir,~ tnctlr~xlb. 237 L!jirt~llal~~litv