SECTION 2: SAMPLING DESIGN IN INTEGRATED IMPACT ASSESSMENT

For those accustomed to conventional statistical sampling methods it may at first sight
appear that the above discussion merely serves to yet further complicate what is
already a very complex process:
the range of potentially relevant variation between individuals within any stakeholder
categorisation is almost as great as the number of respondents
interviewing all individuals within units like households or enterprises and including a
broader range of stakeholders as well as control samples increases the numbers of
people to be included
taking into account temporal variation increases the number of interviews

However incorporating the above concerns does not necessarily increase the length of
time or resources needed. It does though require effective integration of different types
of methodology and different types of sampling at different stages of the assessment
process.
There are many different types of sampling methodology even within the broad
categories of random and non!random sampling methods. There are also a number of
different possibilities for sampling in longitudinal studies. The different types of
sampling technique are summarised in "ox #. $ore details on different techniques
their specific uses and potential pitfalls are given in the %ppendix.
BOX 2: TYPES OF SAMPLING
Random sampling
Simple random sampling: a group of people are selected at random from a complete
list or map of a given population.
Systematic random sampling: a group of people are selected in a systematically
random manner &eg every tenth name' from a complete list of a given population.
Stratified random sampling: when populations are divided into subgroups depending
on particular characteristics.
Cluster sampling: where clusters are randomly selected and all individuals or
households in particular clusters are interviewed.
Random walk: when the interviewer follows a random route.
Staged sampling: where samples are selected within samples e.g. random sampling or
walk within a cluster.
!" #$!s%ions
(hat might have occurred to make the sample atypical of the wider group)
*ould certain types of participant be less likely to be selected than others)
*ould pragmatic criteria such as cost of time constraints introduce bias into the
sample selection)
Non&'andom sampling
Quota sampling: quotas for certain types of people or organisations are selected for
interview.
Purposive sampling: similar to quota samples but where respondents within each
quota are selected to represent diversity.
Chain sampling or snowballing: % first contact is selected and interviewed and then
asked to suggest other interviewees and so on.

Genealogy-based sample: entire families and their relatives may be selected.
atched samples: similar pairs of villages pro+ects or types of respondents are
selected in order to compare them.
!" #$!s%ions
,oes the sample cover those whose views and opinions are particularly important or
normally overlooked in particular women and the poorest groups)
(hose views and opinions will not be covered by a given sample and does their
exclusion matter)
,oes the sample cover all groups likely to have differing opinions will views)
,oes the sample help us understand the linkages between different units of analysis
&such as individuals and organisations')
R!p!a% sampling m!%(ods
Repeat survey: where the entire survey processes repeated including the sampling.
Panel or cohort surveys: the same sample of people or organisations is contacted
several times over a relatively long period.
Rotating survey: a combination of panel and repeat survey methods where one
fraction of the sample is changed each time the survey is repeated.
!" #$!s%ions
what is the likely effect of changes in the sample on the results)
what about systematic changes for particular groups e.g. out migration on marriage
and-or in search of work)
2)* Sampling iss$!s in $sing s%a%is%i+al m!%(ods
.tatistical sampling is the selection of:
/ a portion... from a population the study of which is intended to provide statistical
estimates relating to the whole / &0xford 1nglish ,ictionary'.
It is used when it is important to assess and quantify the generalisability of particular
impacts and use statistical methods to establish relationships between different
variables for example access to credit and increased income. The aim is to obtain as
representative a picture as possible of the extent to which particular impacts occur and
for which stakeholders. .tatistical sampling is a specialist topic for which a statistical
expert will be required. % full discussion of the issues involved is outside the scope of
this paper what follows focuses on general points which it is useful for the general
reader to bear in mind
2
.
.tatistical random sampling can be used either to produce a broad profile of a
population or to provide data for statistical tests. In either case inferences are often
made &either explicitly or implicitly assumed' from the sample to the wider population
rather than simply reporting on the particular sample. (here inferences are to be made
the requirements of statistical analysis mean that:
sampling needs to be done in as near an approximation to random sampling as is
feasible within cost and time constraints. Ideally all units in the target population should
have an equal known chance of being selected. This requires both a list or mapping of
the complete target population from which random selection can be done and
minimising of refusals or non!contacts.
sample si!e is critical and needs to be large enough to establish plausible
associations and minimising the effects of chance errors. The broader the range of
issues covered the larger the sample required to establish plausible associations. 1ach
control variable used increases the minimum sample si3e that is required. This means
that issues and questions need to be kept to a minimum for survey of the total sample.
However beyond a certain sample si3e further increases in numbers of respondents
have proportionately less contribution to decreasing errors. .o it is not necessarily the
case that the larger the sample the more reliable the results
1
For detailed discussion of these issues see a standard textbook on statistical methods for social
scientists such as that by "lalock 2452 listed in the resources.
issues covered must be very focused because large samples required in
complex investigations also mean large costs
Ideally also there should be:
control groups for comparison
baseline samples which can then be followed up in subsequent assessments. This
should be large enough to allow for dropouts.
There is no "magic ideal number# which gives sample si3e for all assessments.
6enerally $%% to &%% respondents are required including controls is considered
sufficient. %s indicated in "ox # there are a number of different random sampling
techniques which can be used. %s indicated in the %ppendix each has advantages and
disadvantages depending on the particular circumstances under investigation.
In practice a number of elements may serve to introduce biases in the sample no
matter how carefully planned and whatever techniques are used:
lists are rarely neutral 7 as discussed above even telephone directories generally
only list heads of household voters lists exclude the homeless or migrant populations
lists of registered entrepreneurs may not cover informal sector enterprises or give
names of business partners etc.
non-responses are often not random' 1ven if lists are comprehensive resource
and time constraints frequently lead to exclusion of those whom it is too expensive or
inconvenient to pursue ie precisely those groups identified by *hambers above.
decisions may need to be made about outliers &exceptions or untypical cases'
whether they can be explained and included in the analysis or ignored and excluded.
(hether they are to be excluded from statistical analysis but followed up by other
methods.
%s noted above it may not be possible to fully address these problems within budgetary
constraints. It is however crucial to consider the degree to which any biases might affect
the subsequent analysis.
2)2 Sampling iss$!s in $sing ,$ali%a%i-! m!%(ods
In using (ualitative methods sampling is done in a non!random purposive manner in
order to probe particular issues. These may provide essential background information
for design of statistical surveys or they may follow up on investigation of statistical
correlations unexpected findings and so on. Investigation may use a combination of
different types of interview on specific topics direct observation and-or compiling of
details case studies. It may include repeat interviews with a small sample in different
seasons and-or in!depth interviews in different contexts.
It is critical that sampling is done with careful selection of key informants and cases.
This may be done in a number of different ways depending on the purpose of the
investigation. In some assessments the types of cases which are likely to be of
particular interest may be known in advance and the task is therefore to find examples
through non!random sampling techniques like those indicated in "ox #. .amples are
likely to include a range of different types of respondent for example:
(uota or purposive sample of primary stakeholders to illustrate different types of
impacts in detail or to increase understanding of the situation of stakeholder categories
who may be underrepresented in the statistical sample
respondents who are not direct pro)ect beneficiaries selected either by quota or
purposive sampling or snowballing or genealogy!based sampling
case studies of unusual extreme or deviant cases &programme dropouts failures
or successes'
8ualitative methods can also be used:
to follow up or pilot surveys for a random selection of respondents in a statistical
survey or a purposive selection of respondents who seem particularly interesting.

for cluster samples e.g. in!depth interviews of all members of a women/s group.
%gain however as the interviews are more in!depth and detailed the number and si3e
of such clusters will be much smaller than statistical survey.
%t the same time qualitative investigation also uses opportunities for random
investigation thrown up by ad hoc chance encounters. These chance encounters may
include the above. "ut they also provide a very useful way of crosschecking information
where the reliability of information gained in pre!planned interviews may be somewhat
in doubt.
In sampling for using qualitative methods more detailed focus is given to a smaller
number of respondents. It is therefore important to be aware of the degree to which
particular respondents and participants can or cannot be taken to be representative of
particular views and interests or particular stakeholder categories. If they are not typical
and have been selected to illustrate particular problems this must be borne in mind and
made explicit in the subsequent analysis.
2). Sampling iss$!s in $sing pa'%i+ipa%o'" m!%(ods
.ampling in participatory methods is generally less predictable than in individual
interview situations. 9articipatory methods can be used to rapidly explore particular
issues through bringing together different groups of stakeholders. They are particularly
useful in rapidly examining contexts in order to focus use of statistical or qualitative
methods. It can also be used as a means of disseminating information as part of
programme policy planning. % key aim is generally also to ensure the representation of
disadvantaged people and to link impact assessment to policy formation and practice. %
prime focus is therefore to bring together different stakeholders.
There are a range of ways in which participatory methods can be used to complement
other sampling methods for example:
9articipatory diagramming methods can be used with very small groups of people
within households or enterprises and-or as part of a survey. In this case sampling can
be done either randomly or non!randomly as with qualitative methods to follow up or
pilot as a complement to statistical methods.
:oting techniques can be used for specific questions in a way similar to cluster
sampling in surveys.
9articipatory methods like mapping can be used to generate lists from which random
samples can be selected. Transect walks can form a basis for random walk sampling.
However participatory methods are also frequently used to bring together different
stakeholders in a specified location rather than as part of interviews in their own homes
or workplaces. In this case those conducting the assessment have much less control
over who comes and who does not come and there is always likely to be an element of
self!selection. ;evertheless it is possible to think through much more carefully the ways
in which participatory exercises are promoted located and timed to make them more
accessible to particular stakeholder groups &and also less accessible to others if
necessary'. It is also possible to invite only specified representatives of particular
stakeholder groups and make the meeting as attractive to them as possible through
particular content linking with decision!making inviting outside speakers or +ust offering
a good meal.
It is easier to have more systematic sampling for participatory methods where there are
existing participatory structures which can be used to disseminate invitations and-or
identify the best people to attend. It is particularly crucial with participatory methods to
be aware of and record who is actually present who participates in discussions and
how this influences the ways in which information is obtained and-or decisions reached.
It is also important to build up a cumulative understanding of context and also contacts
in order to make sampling progressively more precise and effective.