DEPARTMENTOFSOCIOLOGY

RESEARCHDESIGN
ChristinaHughes
C.L.Hughes@warwick.ac.uk
ResearchDesign:Overview
Researchdesignisthebasicplanforapieceofresearch.This
planincludesfourmainideas(Punch,1998).Theseare:

ThestrategyhowwillIproceedfrominitialidentificationof
researchquestionsthroughtocollectingdataandevidenceand
writingup?
Conceptualframeworkwhatkindsoftheoriesorassumptions
amIbringingtomyanalysis?
Whoorwhatwillbestudied?
WhichtoolsandprocedureswillIusetoundertakemy
research?
Whilstresearchdesignrepresentsthefirststageofaprojectit
involvesthinkingthroughthewholeprocessofresearchfrom
beginningtoend.Forthisreason,akeyaspectofresearch
designisthatofevaluationasyouwillwanttoknowhow
adequatetheresearchyouintendtoundertakeis.Mosttextbooks
onissuesofevaluationanddesignfocusprimarilyonthemore
positivistapproachesintheseareas.Thisissummarisedwellby
Punch(1998)whichisakeysourceIshallusehere.Youshould
note,nonetheless,thatacriticalunderstandingofevaluationand
designcannotbedevelopedwithoutrecognisingthe
assumptionsthatarebroughttobearaboutthenatureofsocial

realityandthepurposesofresearch.Greene(1994)indicates
howthekeyvaluesassociatedwithdifferentresearchparadigms
impactonthetypesofevaluativequestionsasked.Forexample,
typicalevaluativequestionswithinpostpositivismare:Are
desiredoutcomesattainedandattributabletotheprogramme?
Isthisprogrammethemostefficientalternative?Withina
criticalframeworkakeyevaluativequestionwouldbeInwhat
waysarethepremises,goalsoractivitiesoftheprogramme
servingtomaintainpowerandresourceinequitiesinsociety?
Forinterpretivists,acentralquestionwouldbeHowisthe
programexperiencedbyvariousstakeholders?Asyouwillsee
thesequestionsdonotreplacemoregeneralquestionsofvalidity
andreliabilitythatareposedinrelationtotheresearchfindings
andtheassociatedprocessesofknowledgegeneration.Both
thesequestionsandspecificparadigmrelatedquestionsare
asked.
Thesematerialscontributetothefollowinglearningoutcomes:
Tohaveanunderstandingofthekeyprinciplesofresearch
design;
Tobeabletocriticallyevaluateyourownresearch.
HowdoIdesignmyResearchProject?

Trytothinkofyourendpointsatthebeginning.Whilstresearch
isoftenviewedasalinearseriesofseparatestepsasBlaxter,
HughesandTight(1996:10)commentitshouldbeviewedasa
spiral.Viewedfromthisperspective,researchiscyclical,canbe
enteredatalmostanypoint;isaneverendingprocess;will
causeyoutoreconsideryourpractice;andwillreturnyoutoa
differentstartingplace.
Punch(1998:255)suggeststhattheconcernsofresearchdesign
aretoanswerthefollowingquestions:

Whoorwhatwillbestudied?
Whatstrategiesofinquirywillbeused?
Whatmethodswillbeusedforcollectingandanalysing
empiricalmaterials?
Thedesignmustfitwiththeresearchquestionsposed.
Thus,willtheresearchusequantitative,qualitativeortextual
approaches?Willtheresearchcombinemethods?Ifso,how?Is
thedesignabsolutelyprespecifiedorwillitbeemergent?Isita
mixtureofboth?Whatevertheresponsestheymustfitwiththe
researchquestionsposed.
ClarkandCauser(1991)suggestthefollowingneedtobe
consideredwhendesigning,planningandconductingresearch:
Whataretheobjectivesoftheresearch?Thisquestionneedsto
beconsideredintermsofidentifyingatopicarea,definingthe
researchquestionsandrationaleandidentifyingkeyconcepts.
Whichmethods,ormixofmethodsareappropriate?For
example,areyougoingtoconductaoneoffsurveyoraseries?
Doyoufocusonasinglecasestudyorasetofcases?
Howareyourconceptsgoingtobeoperationalized?Itis
alwaysnecessarytospecifyhowthecentralconceptsarebeing
appliedtoconcretesocialphenomena.
Whoorwhatwillbestudied?Indocumentaryresearch,for
example,theresearcherneedstochoosethemostappropriate
documents.
Canyouaccessthenecessarypeople,sitesandmaterials?
Doyouneedtopilotfirst?Inlargescalesurveyresearchitis
usuallythoughttobeanabsolutenecessitytopilotresearch

instrumentssuchasquestionnaires.Iftheresearchisbasedon
informalobservation,thepossibilitiesforpilotingareless
certain.
Istheresearchfeasiblegiventimeandotherresources
available?
Doinitialobjectivesneedtobemodifiedinthelightofthe
practicalitiesoftheresearch?
Whataretheappropriatemethods?
Doyouneedtobuildincrosschecks(askingarangeof
questionsonthesameissue,orusingdifferentmethodssuchas
documentarysupportofbiographicalinformation)toenhance
validity?
Doyouhaveanadequatetimetable?
Weshouldaddthefollowing:
Analysiswhatmethodswillbeused?
Ethicswhataretheissuesthatneedtobeconsidered?
HowGoodisthisResearch?CriteriaforEvaluation

Anindicationofthecomplexityofthequestionofevaluative
criteriaincontemporaryqualitativeresearchisgiveninPartV
oftheDenzinandLincolnHandbook(1994:479557).
ModifyingHammersley(1992),theydescribefourbasic
positionsonthequestionofevaluativecriteria:positivism,
postpositivism,postmodernismandpoststructuralism.(They
alsolistthecriteriaimportanttoconstructivists,critical
theoristsandfeminists.)Inthisbook,wherequantitativeand
qualitativeresearcharebroughttogetherwithinthepragmatic

questionmethodframeworksuggested,thecriteriapresentedin
thischapterfitbestwiththepositivistandpostpositivist
positionstheydescribe.(Punch,1998:262,n7)
Punch(opcit)raisesaveryimportantaspectofevaluation
criteria.Thisisthatthecriteriaconsideredrelevanttodeciding
whetherornotwecanbeconfidentaboutthefindingsof
researcharelinkedtotheepistemologicalandpolitical
frameworksoftheresearchdesign.Punchreproducesthemore
dominantcriteriaassociatedwithpositivismandpostpositivism
thatareusedwhenansweringthequestion`Howgoodisthis
research?'

Beforeprovidingthecriteria,Punchoutlinestwoconceptsthat
establishthebackgroundtoevaluativecriteria.Theseare:
disciplinedenquiryandthefitbetweenthecomponentpartsof
aresearchproject.
Disciplinedenquiry

Disciplinedinquiryhasaqualitythatdistinguishesitfromother
sourcesofopinionandbelief.Thedisciplinedinquiryis
conductedandreportedinsuchawaythattheargumentcanbe
painstakinglyexamined.Thereportdoesnotdependforits
appealontheeloquenceofthewriteroronanysurface
plausibility....Whateverthecharacterofastudy,ifitis
disciplinedtheinvestigatorhasanticipatedthetraditional
questionsthatarepertinent.He[sic]institutescontrolateach
stepofinformationcollectionandreasoningtoavoidthe
sourcesoferrortowhichthesequestionsrefer.Iftheerrors
cannotbeeliminatedhetakesthemintoaccountbydiscussing
themarginforerrorinhis[sic]conclusions.Thus,thereportof
adisciplinedinquiryhasatexturethatdisplaystheraw
materialsenteringtheargumentandthelogicalprocessesby

whichtheywerecompressedandrearrangedtomakethe
conclusioncredible....Disciplinedinquirydoesnotnecessarily
followwellestablished,formalprocedures.Someofthemost
excellentinquiryisfreerangingandspeculativeintheinitial
stages,tryingwhatmightseemtobebizarrecombinationsof
ideasandprocedures,orrestlesslycastingaboutforideas.
(CronbachandSuppes,1969inPunch,1998:2512).
Whatisimportantaboutdisciplinedinquiryisthatitsdata,
arguments,andreasoningbecapableofwithstandingcareful
scrutinybyanothermemberofthescientificcommunity.
(Shulman,1988inPunch,1998:252)
Thefitbetweenthecomponentpartsofaresearchproject

...inmostcasesthebestwaytodo[this]istogettheresearch
questionsclear,andthenalignthedesignandmethodswith
those.Wehavenotedthereciprocalinfluencebetweenquestions
andmethods,andlimitationsastowhatcanbedoneintermsof
designandmethodimposelimitationsonquestionsthatcanbe
asked,buttheargumentisthatthisdirectionofinfluenceshould
beminimized.Themaininfluenceshouldbefromquestionsto
methods.(Punch,1998:252).
Puncharguesthattheissueof`fit'isamajorconcernin
assessingthevalidityoftheresearch.Hearguesthatinprojects
wherethecomponentparts(iedesign,methodsandquestions)
donotfittogether.
SpecificCriteriaforEvaluation
Punch(p253)offersthefollowingspecificcriteria:

1.Thesetupoftheresearch

2.Theempiricalproceduresusedintheresearch(design,data
collection,sampleanddataanalysis)
3.Thequalityofthedata
4.Thefindingsandconclusionsreachedintheresearch
5.Thepresentationoftheresearch
1.Thesetupoftheresearch
Isitclearwheretheresearchiscomingfromintermsof
approaches,methodsandparadigms?
Isthetopicareaclearlyidentified,iewhatisthisresearch
about?
Aretheresearchquestionsappropriate(ietheyareclear,easily
understoodandunambiguous,theconceptsarespecificenough
toconnectthedatatoindicators,thedatawillanswerthe
questions,thequestionsarerelatedtoeachother,thequestions
areworthwhileandinteresting).
Istheresearchsetincontext?Contexthastwosenseshere.The
firstrelatestoatheoretical,practicalorprofessionalconcern.
Thesecondmeansthattheresearchconnectstoarelevant
literature.

2.EmpiricalProcedures
Arethedesign,datacollectionanddataanalysisprocedures
reportedinsufficientdetailtoenabletheresearchtobe
scrutinizedandreconstructed?
Isthedesignofthestudyappropriatetotheresearchquestions?

Arethedatacollectioninstrumentsandproceduresadequate
andappropriatefortheresearchquestions?
Isthesampleappropriate?
Arethedataanalysisproceduresadequateandappropriatefor
theresearchquestions?

3.QualityoftheData
Isthereevidenceofcare,controlandthoroughnessinthe
collectionofdata?
Isthedatareliable?Isthere,forexample,internalconsistency
andstabilityovertime?
Isthedatavalid?Thus,howdoweknowthisinstrument
measureswhatwethinkitmeasures?Howwelldothedata
representthephenomenaforwhichtheystand?
Hasthepossibilityofreactivityinthedatabeenconsidered,
reportedandtakenintoaccountandwhatstepshavebeentaken
tominimizeitseffects.Forexample,inquantitativeresearch
reactivitymeanstheextenttowhichmeasuringsomething
changesit(attitudescalesmayprovokeorinfluencetheattitude
inquestion).Inqualitativeresearchaparticipantobserver's
presencemaychangethebehaviourofthoseobserved.

4.FindingsandConclusionsreachedintheresearch
Havetheresearchquestionsbeenanswered?
Howmuchconfidencecanwehaveintheanswersput
forward?

Whatcanbeconcludedfromtheresearchonthebasisofwhat
wasfound?
Thesecondtwoquestionsraiseissuesofinternalandexternal
validity.Internalvalidityreferstotheinternallogicand
consistencyoftheresearch.Forexample,inquantitative
researchthisrelatestowhethertherelationshipsbetweenthe
variableshavebeencorrectlyinterpreted.Inqualitativeresearch,
theinternalvalidityreferstotheextenttowhichthefindings
faithfullyrepresentandreflecttherealitythathasbeenstudied.
Thismeansthatallthepartsfittogetherandwhetherrival
hypotheseshavebeeneliminated,negativeevidencehasbeen
consideredandfindingshavebeencrossvalidatedwithother
partsofthedata.
Externalvalidityreferstoquestionsofgeneralizability.In
quantitativeresearchthisgenerallyreferstoissuesarisingfrom
samplegeneralizability.Inqualititativeresearchitbroadlyrefers
tothetransferabilityofthefindingstoothersettingsand
contexts.

5.PresentationoftheResearch
Isthereawrittenreport?
Whatotherwaysmighttheresearchbepresented?
References
Blaxter,L,Hughes,CandTight,M(1996)HowtoResearch,
Buckingham,OpenUniversityPress
Clark,JandCauser,G(1991)Introduction:ResearchStrategies
andDecisionsinGAllanandCSkinner(Eds)Handbookfor
ResearchStudentsintheSocialSciences,London,Falmer,pp
163176

Greene,J(1994)QualitativeProgramEvaluation:Practiceand
Promise,inNDenzinandYLincoln(Eds)Handbookof
QualitativeResearch,ThousandOaks(Calif),Sage,pp530
544
Punch,K(1998)IntroductiontoSocialResearch:
QuantitativeandQualitativeApproaches,London,Sage