1. What is maximum variation sampling?

Instead of seeking representativeness through equal probabilities, maximum variation sampling seeks it by including a wide range of extremes. The principle is that if you deliberately try to interview a very different selection of people, their aggregate answers can be close to the whole population's. The method sounds odd, but works well in places where a random sample cannot be drawn. This is an extension of the statistical principle of regression towards the mean - in other words, if a group of people is extreme in several different ways, it will contain people who are average in other ways. So if you sought a "minimum variation" sample by only trying to cover the types of people who you thought were average, you'd be likely to miss out on a number of different groups that make up quite a high proportion of the population. But by seeking maximum variation, average people are automatically included. A maximum variation sample (sometimes called a maximum diversity sample or a maximum heterogeneity sample) is a special kind of purposive sample. Normally, a purposive sample is not representative, and does not claim to be. However, a maximum variation sample, if carefully drawn, can be as representative as a random sample. Despite what many people (with a little knowledge of statistics) believe, a random sample is not necessarily the most representative, specially when the sample size is small.

2. When to use maximum variation sampling

There are two main occasions for using maximum variation sampling:

When the sample size is very small, or When no population information is available (and it is not difficult to find population members with the selected characteristics)

2.1. Maximum variation sampling for small sample size

By "small" here, I mean less than about 30. ("About 30" means anything from about 20 to about 50 - there is no sudden change as the sample size increases.) Regardless of the actual number, random sampling doesn't work well for these small samples: there's a high chance of getting a sample that's not representative, even though it was chosen at random. When the sample is as small as 3 (for a set of consensus groups) random sampling is far too dangerous. Instead, you could use quota sampling or maximum variation sampling. If you have enough data on the population, quota sampling is fine. For example, if you are sampling 20 people from the population of a town, a simple form of quota sampling is to choose 10 men and 10 women. But quota sampling - from published or guessed population data - isn't always relevant. That's when maximum variation sampling is most useful. For example, when you're choosing a sample for a set of consensus groups, you normally take three types of people that will be as different as possible on the issue being researched.

2.2. Maximum variation sampling in the absence of population data

Though random sampling is considered the ideal sampling method, sometimes it's not possible to take a random sample. In some countries, census information is either not available, or so many years out of date that it's useless. Even when recent and detailed census data exists, there may be no maps showing the boundaries of the areas to which the data applies. And even when there exist both good census data and related maps, there may be no sampling frames. The good news (from a sampling point of view) is that these conditions usually apply in very poor and undeveloped countries with large rural populations. In my experience, there's not a wide range of variation in these populations. The more developed a country, it seems, the more differences there are between its citizens. Therefore, where random sampling is not possible, perhaps it's not so necessary. But in poor countries where sample frames are nonexistent, maximum variation sampling can be very effective, using the multistage method explained below.

3. What is the best sample size for maximum variation sample?

For a single-stage sample, or at the grassroots level of sampling, it's best to limit a maximum variation sample to no more than about 50 units. Above that number, interviewers get confused, and other methods, such as quota sampling and radial sampling, are simpler, and often more comprehensive. By combining those sub-samples of 50 or less in a multistage sample, the total sample can be thousands of people - but because of the additional effort involved, you wouldn't do that unless there was no alternative. The largest I've tried was about 200, in clusters of 12 but a quota sample (e.g. age group by sex by occupation type) might have been as representative, and would have needed a lot less supervision of interviewers.

4. How to select a maximum variation sample

With maximum variation sampling, you try to include all the extremes in the population. For example, in a small village, for a radio audience survey, you could ask to interview...

the oldest person in the village who listens to radio the oldest who does not listen to radio the youngest who listens to radio a person who listens to radio all day a person who often talks about radio programs he or she has heard a person who listens to radio in the middle of the night a person who has never listened to radio in his or her life the person with the most radios (a repairman, perhaps) the person with the biggest aerial a person who is thought to be completely average in all ways a person who spends a lot of time in the street and in public places a person who works nearly all the time

...and so on - changing "person" to "man" or "woman" alternately, to ensure equal representation of both sexes. Of course this only works when such information about other people is widely known. The above list of people could be produced in a village, where many people know many others, but would be much more difficult in a large city. Often it's useful to have a preliminary brainstorming session with an initial group of local informants (who should not be eventual respondents). Present an initial list of personal types to them, similar to the above, but suitably modified for the purpose of your study. Ask them to come up with some more types of person, and to tell you if some of the types you invented make no sense in that area. But unless you begin with an example, I've found that people find it difficult to understand what you're asking. One problem with drawing a sample as above is the informants you use to identify the people with those characteristics. It's tempting - because it's easy - to go to the local government office and ask the officials to name people of those types. You may get a list of them quickly, but in one important way there will not be maximum variation: suggested respondents will all be known to the local government officials. Your net can be cast more widely by sequential sampling (snowball sampling), getting only a few suggested respondents from each source. In other words, informant A suggests respondents B and C from your list of characteristics, B suggests D and E, C suggests F and G - and so on. Given the principle of "six degrees of separation," and the fact that respondents are not being asked to suggest their friends, but people with specified characteristics, the maximum variation method should give most people in the survey area a chance of being included in the sample. Did you notice the flaw in that argument? The problem is that the more people a potential respondent is known to, the more likely that person is to be selected for the survey. Therefore, the list of personal types needs to explicitly include socially isolated people, by adding criteria such as...

an older man who has very few visitors a younger man who doesn't talk much an older woman who lives by herself and has no family members living nearby a young woman who is unmarried and hardly ever leaves home

...and so on - varying the above descriptions to suit the culture. (In some parts of the world you will never find a woman living by herself.) Where there is segregation along religious, linguistic, or tribal lines, you will need to start separate threads of inquiry in each of those cultural groups. If you ask for a particular type of person, and the informant can't name somebody exactly like that, it's fine to accept an approximation, based on some other criterion that seems relevant. This can introduce other dimensions of diversity that you didn't initially think of.

5. Selecting the dimensions of variation

In the above example, the 12 different kinds of radio listener (plus another 4 kinds of social isolates) were found through imagining the social circumstances that might affect radio listening. The list wasn't exhaustive or systematic, but if you want to be sure that no group of people has been omitted, you can use dimensional analysis to create a more comprehensive list. It's done like this... Step 1 is decide what sample size you want. For example, let's say it's 20. This determines the number of dimensions: 20 is 2 to the power of what? The closest answer is 4, because 2 x 2 x 2 x 2 = 16. So you can use 4 dimensions to get 16 cases, then add a few more factors, such as socially isolated people. (For a sample of 32, use 5 dimensions, and for 64 use 6. Above 100 or so, quota sampling usually works better.) Step 2 is to decide on those dimensions. Think of some characteristics of people that (a) differ widely between people in relation to the subject you're researching, and (b) are known to a wide range of other people. For example, if the subject is how much time people spend listening to radio, it may not be useful to choose gender as a dimension, because in most countries men and women spend about equal time listening to radio. However, whether or not people have a radio at home makes a big difference to their listening time. Other visible factors that affect radio listening are whether people have TV at home, and how much time people spend away from home, in places without a radio. Another factor is how much they like listening to the local programs, but that's not easily observable, so you might need to use a proxy variable, such as how often they say they talk about radio programs. Now we have the 4 variables, each with two extreme answers. Give each possible answer a letter code, starting from A, like this...

Have a radio at home: yes (A) or no (B) Have TV at home: yes (C) or no (D) Stay at home most of the time (E) or away from home most of the time (F) How often they talk to others about radio programs: "most days" (G) or "hardly ever" (H)

Taking each of the 4 variables in turn, there are 16 possible categories (2 x 2 x 2 x 2). These are ACEG, ACEH, ACFG, ACFH BCEG, BCEH, BCFG, BCFH ADEG, ADEH, ADFG, ADFH BDEG, BDEH, BDFG, BDFH For example, BDGH = somebody who has no radio at home, no TV at home, is away from home most of the time, and hardly ever talks about radio. Step 3. All you have to do now is find somebody matching that description - and repeat that task for the 15 other types of people. What if you can't find people who meet some of those descriptions? This can happen - for example, it might be hard to find somebody who stays home most of the time, and doesn't have radio at home, but talks about it a lot. In this case, you'll end up with more than one person in some of the 16 categories. No great problem: just make sure that people in the same category are very different in some other way that seems relevant to your study.

Step 4. Finally, don't forget to add the 4 people who seldom communicate with others. That brings your sample up to 20. You want more than 20? Just add some more people, as long as they are as different as possible from each other in some relevant way. Though this systematic method of selecting respondents is easier when rostering interviewers, I haven't found that it produces a more diverse sample than the more random method described in section 4 above.

6. Multi-stage maximum variation sampling

When you are selecting a multistage sample, the first stage might be to draw a sample of districts in the whole country. If this number is less than about 30, it's likely that the sample will be seriously unrepresentative in some ways. Two solutions to this are stratification and maximum variation sampling. For both of these, some local knowledge is needed. When you are surveying a large geographical area, a maximum-variation sample can be drawn in several stages. The first stage is to decide which parts of the population area will be surveyed. For example, if a survey is to represent a whole province, and it's not feasible to survey every part of the province, you must decide which parts of the province (let's call them counties) will be included. Selecting them is done like this...

6.1. Stage 1
1. Think of all the ways in which the counties can differ from the province as a whole - specially ways that are related to the subject of the survey. If a survey is about FM radio, and some areas are hilly, reception may be poorer there. If the survey is about malaria, and some counties have large swamps with a lot of mosquitoes, include one such county and one that's the opposite. If the topic is related to wealth or education levels (as many research topics are), find out which counties have the richest and best-educated people, and which have the poorest and leasteducated. Try to think of 5 to 10 factors that are relevant to the study. 2. Then try to gather objective data about these factors. Failing that, try to find experts on the topics, or people who have travelled around the whole province. Using this information, for each factor make a list of the counties which have a high level of the factor (e.g. lots of mountains, lots of swamps, or wealthy) and counties which have a low level of the factor (e.g. all flat, no swamps, or poor). 3. The counties mentioned most often in these lists of extremes should be included in the survey. Mark these counties on a map of the province. Has any large and well-populated area been omitted? If so, add another county, which is as far as possible from all the others mentioned.

6.2. Stage 2
When the counties (or whatever the areas are called) have been chosen, the next stage is to work out where in each county the cluster should be chosen. Continue the maximum-variation principle by using the same principle inside each selected county. If a county was chosen for its

swampiness and flatness, choose the flattest and swampiest area in the country. If it was chosen for its mountains and wealth, choose a wealthy mountainous area. To find out where these areas are, you may need to travel to each county and speak to local experts.

6.3. Stage 3
When you have chosen the towns and rural localities, you can either continue using maximum variation sampling, or you can choose another method, such as quota sampling, block listing from aerial photographs, or radial sampling. If you use maximum variation sampling for the final stage, you'll normally choose a number of clusters (streets or neighbourhoods), then choose respondents in each cluster using the principles explained in section 4 or 5 above.

Would you like to read more about maximum variation sampling? Sorry, but you can't! This page is not very detailed, but even so, it seems to be the most detailed explanation of maximum variation sampling ever written. The next most detailed (and most widely cited) seems to be in Michael Quinn Patton's book Qualitative Research and Evaluation Methods, on pages 234-235 in the 2001 edition (less than one page in total). Also, this page focuses on maximum variation sampling for surveys. Using it for qualitative research, such as in-depth interviews and case studies, would require slight variations. I'll write a separate page on that, when the need arises. Suggested citation for this page: List, Dennis (2004). Maximum variation sampling for surveys and consensus groups. Adelaide: Audience Dialogue. Available at www.audiencedialogue.org/maxvar.html, 12 September 2004. Other principles of sampling mentioned above (random sampling, quota sampling, stratified sampling, and snowball sampling) are described in Chapter 2 of Know Your Audience.