Beruflich Dokumente
Kultur Dokumente
-o-
3. experimental research?
Science revolves around experiments, and learning the best way of conducting an experiment is
crucial to obtaining useful and valid results. When scientists speak of experiments, in the strictest
sense of the word, they mean a true experiment, where the scientist controls all of the factors and
conditions. Real world observations, and case studies, should be referred to as observational
research, rather than experiments. For example, observing animals in the wild is not a true
experiment, because it does not isolate and manipulate an independent variable.
THE BASIS OF CONDUCTING AN EXPERIMENT
With an experiment, the researcher is trying to learn something new about the world, an
explanation of ‘why’ something happens. The experiment must maintain internal and external
validity, or the results will be useless. When designing an experiment, a researcher must follow all
of the steps of the scientific method, from making sure that the hypothesis is valid and testable, to
using controls and statistical tests.
Whilst all scientists use reasoning, operationalization and the steps of the scientific process, it is
not always a conscious process. Experience and practice mean that many scientists follow an
instinctive process of conducting an experiment, the ‘streamlined’ scientific process. Following the
basic steps will usually generate valid results, but where experiments are complex and
expensive, it is always advisable to follow the rigorous scientific protocols.
Conducting a experiment has a number of stages, where the parameters and structure of the
experiment are made clear. Whilst it is rarely practical to follow each step strictly, any aberrations
must be justified, whether they arise because of budget, impracticality or ethics.
STAGE ONE
After deciding upon a hypothesis, and making predictions, the first stage of conducting an
experiment is to specify the sample groups. These should be large enough to give a statistically
viable study, but small enough to be practical. Ideally, groups should be selected at random, from
a wide selection of the sample population. This allows results to be generalized to the population
as a whole. In the physical sciences, this is fairly easy, but the biological and behavioral sciences
are often limited by other factors.
For example, medical trials often cannot find random groups. Such research often relies upon
volunteers, so it is difficult to apply any realistic randomization. This is not a problem, as long as
the process is justified, and the results are not applied to the population as a whole. If a
psychological researcher used volunteers who were male students, aged between 18 and 24, the
findings can only be generalized to that specific demographic group within society.
STAGE TWO
The sample groups should be divided, into a control group and a test group, to reduce the
possibility of confounding variables. This, again, should be random, and the assigning of subjects
to groups should be blind or double blind. This will reduce the chances of experimental error, or
bias, when conducting an experiment.
Ethics are often a barrier to this process, because deliberately withholding treatment is not
permitted. Again, any deviations from this process must be explained in the conclusion. There is
nothing wrong with compromising upon randomness, where necessary, as long as other
scientists are aware of how, and why, the researcher selected groups on that basis.
STAGE THREE
This stage of conducting an experiment involves determining the time scale and frequency of
sampling, to fit the type of experiment. For example, researchers studying the effectiveness of a
cure for colds would take frequent samples, over a period of days. Researchers testing a cure for
Parkinson’s disease would use less frequent tests, over a period of months or years.
STAGE FOUR
The penultimate stage of the experiment involves performing the experiment according to the
methods stipulated during the design phase. The independent variable is manipulated, generating
a usable data set for the dependent variable.
STAGE FIVE
The raw data from the results should be gathered, and analyzed, by statistical means. This allows
the researcher to establish if there is any relationship between the variables and accept, or reject,
the null hypothesis.
These steps are essential to providing excellent results. Whilst many researchers do not want to
become involved in the exact processes of inductive reasoning, deductive reasoning and
operationalization, they all follow the basic steps of conducting an experiment. This ensures that
their results are valid.
4. Exploratory research?
Exploratory research is conducted to clarify ambiguous problems. Management may have
discovered general problems, but research is needed to gain better understanding of the
dimensions of the problems. Exploratory studies provide information to use in analyzing a
situation, but uncovering conclusive evidence to determine a particular course of action is not the
purpose of exploratory research. Usually, exploratory research is conducted with the expectation
that subsequent research will be required to provide conclusive evidence, It is a serious mistake
to rush into detailed surveys before less expensive and more readily available sources of
information have been exhausted.
In an organisation considering a program to help employees with childcare needs, for example,
exploratory research with a small number of employees who have children might determine that
many of them have spouses who also work and that these employees have positive reactions to
the possibility of an on-site child-care program. In such a case exploratory research helps to
crystallize a problem and identify information needs for future research.
Exploratory research methods:
The quickest and the cheapest way to formulate a hypothesis in exploratory research is by using
any of the four methods:
• Literature search
• Experience survey
• Focus group
• Analysis of selected cases
Literature Search
This refers to “referring to a literature to develop a new hypothesis”. The literature referred are –
trade journals, professional journals, market research finding publications, statistical publications
etc Example: Suppose a problem is “Why are sales down?” This can quickly be analyzed with the
help of published data which should indicate “whether the problem is an “industry problem” or a
“firm problem”. Three possibilities exist to formulate the hypothesis.
• The company’s market share has declined but industry’s figures are normal.
• The industry is declining and hence the company’s market share is also declining.
• The industry’s share is going up but the company’s share is declining.
If we accept the situation that our company’s sales are down despite the market showing an
upward trend, then we need to analyse the marketing mix variables.
Example 1: A TV manufacturing company feels that its market share is declining whereas the
overall television industry is doing very well.
Example 2: Due to a trade embargo imposed by a country, textiles exports are down and hence
sales of a company making garment for exports is on the decline.
The above information may be used to pinpoint the reason for declining sales.
Experience Survey
In experience surveys, it is desirable to talk to persons who are well informed in the area being
investigated. These people may be company executives or persons outside the organisation.
Here, no questionnaire is required. The approach adopted in an experience survey should be
highly unstructured, so that the respondent can give divergent views. Since the idea of using
experience survey is to undertake problem formulation, and not conclusion, probability sample
need not be used. Those who cannot speak freely should be excluded from the sample.
Examples :
1) A group of housewives may be approached for their choice for a “ready to cook product”.
2) A publisher might want to find out the reason for poor circulation of newspaper introduced
recently. He might meet (a) Newspaper sellers (b) Public reading room (c) General public (d)
Business community; etc.
These are experienced persons whose knowledge researcher can use.
Focus Group
Another widely used technique in exploratory research is the focus group. In a focus group, a
small number of individuals are brought together to study and talk about some topic of interest.
The discussion is co-ordinated by a moderator. The group usually is of 8-12 persons. While
selecting these persons, care has to be taken to see that they should have a common
background and have similar experiences in buying. This is required because there should not be
a conflict among the group members on the common issues that are being discussed. During the
discussion, future buying attitudes, present buying opinion etc., are gathered.
Most of the companies conducting the focus groups, first screen the candidates to determine who
will compose the particular group. Firms also take care to avoid groups, in which some of the
participants have their friends and relatives, because this leads to a biased discussion. Normally,
a number of such groups are constituted and the final conclusion of various groups are taken for
formulating the hypothesis. Therefore, a key factor in focus group is to have similar groups.
Normally there are 4-5 groups. Some of them may even have 6-8 groups. The guiding criteria is
to see whether the latter groups are generating additional ideas or repeating the same with
respect to the subject under study. When this shows a diminishing return from the group, the
discussions stopped. The typical focus group lasts for 1-30 hours to 2 hours. The moderator
under the focus group has a key role. His job is to guide the group to proceed in the right
direction.
Analysis of selected cases
Analysing a selected case sometimes gives an insight into the problem which is being
researched. Case histories of companies which have undergone a similar situation may be
available. These case studies are well suited to carry out exploratory research. However, the
result of investigation of case histories arc always considered suggestive, rather than conclusive.
In case of preference to “ready to eat food”, many case histories may be available in the form of
previous studies made by competitors. We must carefully examine the already published case
studies with regard to other variables such as price, advertisement, changes in the taste, etc.
3. MEASUREMENT CONCEPTS
Measurement
Measurement is the process observing and recording the observations that are collected as part of a
research effort. There are two major issues that will be considered here.
First, you have to understand the fundamental ideas involved in measuring. Here we consider two of major
measurement concepts. In Levels of Measurement, I explain the meaning of the four major levels of
measurement: nominal, ordinal, interval and ratio. Then we move on to the reliability of measurement,
Second, you have to understand the different types of measures that you might use in social research. We
consider four broad categories of measurements. Survey research includes the design and implementation
of interviews and questionnaires. Scaling involves consideration of the major methods of developing and
implementing a scale. Qualitative research provides an overview of the broad range of non-numerical
measurement approaches. And unobtrusive measures presents a variety of measurement methods that
affiliation." That variable has a number of attributes. Let's assume that in this particular election context the
only relevant attributes are "republican", "democrat", and "independent". For purposes of analyzing the
results of this variable, we arbitrarily assign the values 1, 2 and 3 to the three attributes. The level of
measurement describes the relationship among these three values. In this case, we simply are using the
numbers as shorter placeholders for the lengthier text terms. We don't assume that higher values mean
"more" of something and lower numbers signify "less". We don't assume the the value of 2 means that
democrats are twice something that republicans are. We don't assume that republicans are in first place or
have the highest priority just because they have the value of 1. In this case, we only use the values as a
shorter name for the attribute. Here, we would describe the level of measurement as "nominal".
First, knowing the level of measurement helps you decide how to interpret the data from that variable. When
you know that a measure is nominal (like the one just described), then you know that the numerical values
are just short codes for the longer names. Second, knowing the level of measurement helps you decide
what statistical analysis is appropriate on the values that were assigned. If a measure is nominal, then you
know that you would never average the data values or do a t-test on the data.
There are typically four levels of measurement that are defined:
• Nominal
• Ordinal
• Interval
• Ratio
H.S.; 1=some H.S.; 2=H.S. degree; 3=some college; 4=college degree; 5=post college. In this measure,
higher numbers mean more education. But is distance from 0 to 1 same as 3 to 4? Of course not. The
In interval measurement the distance between attributesdoes have meaning. For example, when we
measure temperature (in Fahrenheit), the distance from 30-40 is same as distance from 70-80. The interval
between values is interpretable. Because of this, it makes sense to compute an average of an interval
variable, where it doesn't make sense to do so for ordinal scales. But note that in interval measurement
ratios don't make any sense - 80 degrees is not twice as hot as 40 degrees (although the attribute value is
twice as large).
Finally, in ratio measurement there is always an absolute zero that is meaningful. This means that you can
construct a meaningful fraction (or ratio) with a ratio variable. Weight is a ratio variable. In applied social
research most "count" variables are ratio, for example, the number of clients in past six months. Why?
Because you can have zero clients and because it is meaningful to say that "...we had twice as many clients
of measurement, assumptions tend to be less restrictive and data analyses tend to be less sensitive. At
each level up the hierarchy, the current level includes all of the qualities of the one below it and adds
something new. In general, it is desirable to have a higher level of measurement (e.g., interval or ratio)
Scaling
Scaling is the branch of measurement that involves the construction of an instrument that associates
qualitative constructs with quantitative metric units. Scaling evolved out of efforts in psychology and
education to measure "unmeasurable" constructs like authoritarianism and self esteem. In many ways,
scaling remains one of the most arcane and misunderstood aspects of social research measurement. And, it
attempts to do one of the most difficult of research tasks -- measure abstract concepts.
Most people don't even understand what scaling is. The basic idea of scaling is described in General Issues
in Scaling, including the important distinction between a scale and a response format. Scales are generally
divided into two broad categories: unidimensional and multidimensional. The unidimensional scaling
methods were developed in the first half of the twentieth century and are generally named after their
In the late 1950s and early 1960s, measurement theorists developed more advanced techniques for creating
multidimensional scales. Although these techniques are not considered here, you may want to look at the
method of concept mapping that relies on that approach to see the power of these multivariate methods.
General Issues in
Scaling
S.S. Stevens came up with what I think is the simplest
But what does that mean? In most scaling, the objects are text statements, usually statements of attitude or
belief. The figure shows an example.There are three statements describing attitudes towards immigration.
To scale these statements, we have to assign numbers to them. Usually, we would like the result to be on at
least an interval scale (see Levels of Measurement) as indicated by the ruler in the figure. And what does
"according to a rule" mean? If you look at the statements, you can see that as you read down, the attitude
towards immigration becomes more restrictive -- if a person agrees with a statement on the list, it's likely that
they will also agree with all of the statements higher on the list. In this case, the "rule" is a cumulative one.
So what is scaling? It's how we get numbers that can be meaningfully assigned to objects -- it's a set of
But first, I have to clear up one of my pet peeves. People often confuse the idea of a scale and a response
scale. A response scale is the way you collect responses from people on an instrument. You might use a
dichotomous response scale like Agree/Disagree, True/False, or Yes/No. Or, you might use an interval
response scale like a 1-to-5 or 1-to-7 rating. But, if all you are doing is attaching a response scale to an
object or statement, you can't call that scaling. As you will see, scaling involves procedures that you do
independent of the respondent so that you can come up with a numerical value for the object. In true scaling
research, you use a scaling procedure to develop your instrument (scale) and you also use a response scale
to collect the responses from participants. But just assigning a 1-to-5 response scale for an item
refers to a set of
used for a single item
items
Purposes of Scaling
sometimes we do scaling to test a hypothesis. We might want to know whether the construct or concept is a
single dimensional or multidimensional one (more about dimensionality later). Sometimes, we do scaling as
part of exploratory research. We want to know what dimensions underlie a set of ratings. For instance, if you
create a set of questions, you can use scaling to determine how well they "hang together" and whether they
measure one concept or multiple concepts. But probably the most common reason for doing scaling is for
scoring purposes. When a participant gives their responses to a set of items, we often would like to assign a
single number that represents that's person's overall attitude or belief. For the figure above, we would like to
be able to give a single number that describes a person's attitudes towards immigration, for example.
Dimensionality
A scale can have any number of dimensions in it. Most scales that we develop
the construct can be measured well with one number line or whether it may
dimensional. We can measure the concept of height very well with only a
concept -- you are either more or less thirsty at any given time. It's easy to see
that height and weight are unidimensional. But what about a concept like self esteem? If you think you can
measure a person's self esteem well with a single ruler that goes from low to high, then you probably have a
unidimensional construct.
What would a two-dimensional concept be? Many models of intelligence or achievement postulate two major
dimensions -- mathematical and verbal ability. In this type of two-dimensional model, a person can be said to
possess two types of achievement. Some people will be high in verbal skills and lower in math. For others, it
will be the reverse. But, if a concept is truly two-dimensional, it is not possible to depict a person's level on it
using only a single number line. In other words, in order to describe achievement you would need to locate a
They called this general theory of meaning the semantic differential. Their theory essentially states that
you can rate any object along those three dimensions. For instance, think of the idea of "ballet." If you like
the ballet, you would probably rate it high on activity, favorable on evaluation, and powerful on potency. On
the other hand, think about the concept of a "book" like a novel. You might rate it low on activity (it's
passive), favorable on evaluation (assuming you like it), and about average on potency. Now, think of the
idea of "going to the dentist." Most people would rate it low on activity (it's a passive activity), unfavorable on
evaluation, and powerless on potency (there are few routine activities that make you feel as powerless!).
The theorists who came up with the idea of the semantic differential thought that the meaning of any
concepts could be described well by rating the concept on these three dimensions. In other words, in order
to describe the meaning of an object you have to locate it as a dot somewhere within the cube (three-
dimensional space).
Unidimensional or Multidimensional?
What are the advantages of using a unidimensional model? Unidimensional concepts are generally easier to
understand. You have either more or less of it, and that's all. You're either taller or shorter, heavier or lighter.
It's also important to understand what a unidimensional scale is as a foundation for comprehending the more
complex multidimensional concepts. But the best reason to use unidimensional scaling is because you
believe the concept you are measuring really is unidimensional in reality. As you've seen, many familiar
concepts (height, weight, temperature) are actually unidimensional. But, if the concept you are studying is in
fact multidimensional in nature, a unidimensional scale or number line won't describe it well. If you try to
measure academic achievement on a single dimension, you would place every person on a single line
ranging from low to high achievers. But how do you score someone who is a high math achiever and terrible
verbally, or vice versa? A unidimensional scale can't capture that type of achievement.
There are three major types of unidimensional scaling methods. They are similar in that they each measure
the concept of interest on a number line. But they differ considerably in how they arrive at scale values for
different items. The three methods are Thurstone or Equal-Appearing Interval Scaling, Likert or "Summative"
Types of Surveys
Surveys can be divided into two broad categories: the questionnaire and the interview. Questionnaires are
usually paper-and-pencil instruments that the respondent completes. Interviews are completed by the
interviewer based on the respondent says. Sometimes, it's hard to tell the difference between a
questionnaire and an interview. For instance, some people think that questionnaires always ask short
closed-ended questions while interviews always ask broad open-ended ones. But you will see
questionnaires with open-ended questions (although they do tend to be shorter than in interviews) and there
Survey research has changed dramatically in the last ten years. We have automated telephone surveys that
use random dialing methods. There are computerized kiosks in public places that allows people to ask for
input. A whole new variation of group interview has evolved as focus group methodology. Increasingly,
survey research is tightly integrated with the delivery of service. Your hotel room has a survey on the desk.
Your waiter presents a short customer satisfaction survey with your check. You get a call for an interview
several days after your last call to a computer company for technical assistance. You're asked to complete a
short survey when you visit a web site. Here, I'll describe the major types of questionnaires and interviews,
keeping in mind that technology is leading to rapid evolution of methods. We'll discuss the relative
advantages and disadvantages of these different survey types in Advantages and Disadvantages of Survey
Methods.
Questionnaires
When most people think of questionnaires, they think of the mail survey. All
administer. You can send the exact same instrument to a wide number of people. They allow the respondent
to fill it out at their own convenience. But there are some disadvantages as well. Response rates from mail
surveys are often very low. And, mail questionnaires are not the best vehicles for asking for detailed written
responses.
A second type is the group administered questionnaire. A sample of respondents is brought
together and asked to respond to a structured sequence of questions. Traditionally, questionnaires were
administered in group settings for convenience. The researcher could give the questionnaire to
those who were present and be fairly sure that there would be a high response rate. If the
respondents were unclear about the meaning of a question they could ask for clarification. And,
there were often organizational settings where it was relatively easy to assemble the group (in a
What's the difference between a group administered questionnaire and a group interview or focus group?
In the group administered questionnaire, each respondent is handed an instrument and asked to
complete it while in the room. Each respondent completes an instrument. In the group interview or focus
group, the interviewer facilitates the session. People work as a group, listening to each other's comments
and answering the questions. Someone takes notes for the entire group -- people don't complete an
interview individually.
A less familiar type of questionnaire is the household drop-off survey. In this approach, a researcher goes
to the respondent's home or business and hands the respondent the instrument. In some cases, the
respondent is asked to mail it back or the interview returns to pick it up. This approach attempts to blend the
advantages of the mail survey and the group administered questionnaire. Like the mail survey, the
respondent can work on the instrument in private, when it's convenient. Like the group administered
questionnaire, the interviewer makes personal contact with the respondent -- they don't just send an
impersonal survey instrument. And, the respondent can ask questions about the study and get clarification
on what is to be done. Generally, this would be expected to increase the percent of people who are willing to
respond.
Interviews
Interviews are a far more personal form of research than questionnaires. In thepersonal interview, the
interviewer works directly with the respondent. Unlike with mail surveys, the interviewer has the opportunity
to probe or ask follow-up questions. And, interviews are generally easier for the respondent, especially if
what is sought is opinions or impressions. Interviews can be very time consuming and they are resource
intensive. The interviewer is considered a part of the measurement instrument and interviewers have to be
researcher to gather information rapidly. Most of the major public opinion polls that are reported were based
on telephone interviews. Like personal interviews, they allow for some personal contact between the
interviewer and the respondent. And, they allow the interviewer to ask follow-up questions. But they also
have some major disadvantages. Many people don't have publicly-listed telephone numbers. Some don't
have telephones. People often don't like the intrusion of a call to their homes. And, telephone interviews
research contexts. You'll see that there are very few simple rules that will make the decision for you -- you
have to use your judgment to balance the advantages and disadvantages of different survey types. Here, all
I want to do is give you a number of questions you might ask that can help guide your decision.
Population Issues
The first set of considerations have to do with the population and its accessibility.
For some populations, you have a complete listing of the units that will be sampled. For others, such a list is
difficult or impossible to compile. For instance, there are complete listings of registered voters or person with
active drivers licenses. But no one keeps a complete list of homeless people. If you are doing a study that
requires input from homeless persons, you are very likely going to need to go and find the respondents
personally. In such contexts, you can pretty much rule out the idea of mail surveys or telephone interviews.
• Is the population literate?
Questionnaires require that your respondents can read. While this might seem initially like a reasonable
assumption for many adult populations, we know from recent research that the instance of adult illiteracy is
alarmingly high. And, even if your respondents can read to some degree, your questionnaire may contain
difficult or technical vocabulary. Clearly, there are some populations that you would expect to be illiterate.
We live in a multilingual world. Virtually every society has members who speak other than the predominant
language. Some countries (like Canada) are officially multilingual. And, our increasingly global economy
requires us to do research that spans countries and language groups. Can you produce multiple versions of
your questionnaire? For mail instruments, can you know in advance the language your respondent speaks,
or do you send multiple translations of your instrument? Can you be confident that important connotations in
your instrument are not culturally specific? Could some of the important nuances get lost in the process of
People who do research on immigration issues have a difficult methodological problem. They often need to
speak with undocumented immigrants or people who may be able to identify others who are. Why would we
expect those respondents to cooperate? Although the researcher may mean no harm, the respondents are
at considerable risk legally if information they divulge should get into the hand of the authorities. The same
can be said for any target group that is engaging in illegal or unpopular activities.
Is your population of interest dispersed over too broad a geographic range for you to study feasibly with a
personal interview? It may be possible for you to send a mail instrument to a nationwide sample. You may
be able to conduct phone interviews with them. But it will almost certainly be less feasible to do research
that requires interviewers to visit directly with respondents if they are widely dispersed.
Sampling Issues
The sample is the actual group you will have to contact in some way. There are several important sampling
What information do you have about your sample? Do you know their current addresses? Their current
Can your respondents be located? Some people are very busy. Some travel a lot. Some work the night shift.
Even if you have an accurate phone or address, you may not be able to locate or make contact with your
sample.
Who is the respondent in your study? Let's say you draw a sample of households in a small city. A
household is not a respondent. Do you want to interview a specific individual? Do you want to talk only to the
"head of household" (and how is that person defined)? Are you willing to talk to any member of the
household? Do you state that you will speak to the first adult member of the household who opens the door?
What if that person is unwilling to be interviewed but someone else in the house is willing? How do you deal
with multi-family households? Similar problems arise when you sample groups, agencies, or companies.
Can you survey any member of the organization? Or, do you only want to speak to the Director of Human
Resources? What if the person you would like to interview is unwilling or unable to participate? Do you use
If you have an incomplete list of the population (i.e., sampling frame) you may not be able to sample every
member of the population. Lists of various groups are extremely hard to keep up to date. People move or
change their names. Even though they are on your sampling frame listing, you may not be able to get to
them. And, it's possible they are not even on the list.
Even if you are able to solve all of the other population and sampling problems, you still have to deal with
the issue of response rates. Some members of your sample will simply refuse to respond. Others have the
best of intentions, but can't seem to find the time to send in your questionnaire by the due date. Still others
misplace the instrument or forget about the appointment for an interview. Low response rates are among the
most difficult of problems in survey research. They can ruin an otherwise well-designed survey effort.
Question Issues
Sometimes the nature of what you want to ask respondents will determine the type of survey you select.
Can you anticipate the most frequent or important types of responses and develop reasonable closed-ended
questions?
Sometimes you are dealing with a complex subject or topic. The questions you want to ask are going to
A screening question may be needed to determine whether the respondent is qualified to answer your
question of interest. For instance, you wouldn't want to ask someone their opinions about a specific
computer program without first "screening" them to find out whether they have any experience using the
program. Sometimes you have to screen on several variables (e.g., age, gender, experience). The more
complicated the screening, the less likely it is that you can rely on paper-and-pencil instruments without
Is your survey one where you can construct in advance a reasonable sequence of questions? Or, are you
doing an initial exploratory study where you may need to ask lots of follow-up questions that you can't easily
anticipate?
If your subject matter is complicated, you may need to give the respondent some detailed background for a
question. Can you reasonably expect your respondent to sit still long enough in a phone interview to ask
your question?
If you are asking people about the different computer equipment they use, you may have to have a lengthy
response list (CD-ROM drive, floppy drive, mouse, touch pad, modem, network connection, external
speakers, etc.). Clearly, it may be difficult to ask about each of these in a short phone interview.
Content Issues
The content of your study can also pose challenges for the different survey types you might utilize.
If the respondent does not keep up with the news (e.g., by reading the newspaper, watching television news,
or talking with others), they may not even know about the news issue you want to ask them about. Or, if you
want to do a study of family finances and you are talking to the spouse who doesn't pay the bills on a regular
basis, they may not have the information to answer your questions.
Even if the respondent understands what you're asking about, you may need to allow them to consult their
records in order to get an accurate answer. For instance, if you ask them how much money they spent on
food in the past month, they may need to look up their personal check and credit card records. In this case,
you don't want to be involved in an interview where they would have to go look things up while they keep
Bias Issues
People come to the research endeavor with their own sets of biases and prejudices. Sometimes, these
Respondents generally want to "look good" in the eyes of others. None of us likes to look like we don't know
an answer. We don't want to say anything that would be embarrassing. If you ask people about information
that may put them in this kind of position, they may not tell you the truth, or they may "spin" the response so
that it makes them look better. This may be more of a problem in an interview situation where they are face-
Interviewers may distort an interview as well. They may not ask questions that make them uncomfortable.
They may not listen carefully to respondents on topics for which they have strong opinions. They may make
the judgment that they already know what the respondent would say to a question based on their prior
With mail surveys it may be difficult to know who actually responded. Did the head of household complete
the survey or someone else? Did the CEO actually give the responses or instead pass the task off to a
subordinate? Is the person you're speaking with on the phone actually who they say they are? At least with
personal interviews, you have a reasonable chance of knowing who you are speaking with. In mail surveys
Administrative Issues
Last, but certainly not least, you have to consider the feasibility of the survey method for your study.
• costs
Cost is often the major determining factor in selecting survey type. You might prefer to do personal
interviews, but can't justify the high cost of training and paying for the interviewers. You may prefer to send
• facilities
Do you have the facilities (or access to them) to process and manage your study? In phone interviews, do
you have well-equipped phone surveying facilities? For focus groups, do you have a comfortable and
accessible room to host the group? Do you have the equipment needed to record and transcribe responses?
• time
Some types of surveys take longer than others. Do you need responses immediately (as in an overnight
public opinion poll)? Have you budgeted enough time for your study to send out mail surveys and follow-up
reminders, and to get the responses back by mail? Have you allowed for enough time to get enough
• personnel
Different types of surveys make different demands of personnel. Interviews require interviewers who are
motivated and well-trained. Group administered surveys require people who are trained in group facilitation.
Some studies may be in a technical area that requires some degree of expertise in the interviewer.
Clearly, there are lots of issues to consider when you are selecting which type of survey you wish to use in
your study. And there is no clear and easy way to make this decision in many contexts. There may not be
one approach which is clearly the best. You may have to make tradeoffs of advantages and disadvantages.
There is judgment involved. Two expert researchers may, for the very same problem or issue, select entirely
different survey methods. But, if you select a method that isn't appropriate or doesn't fit the context, you can
doom a study before you even begin designing the instruments or questions themselves.
Types Of Questions
Survey questions can be divided into two broad types: structured and unstructured. From an instrument
design point of view, the structured questions pose the greater difficulties (see Decisions About the
Response Format). From a content perspective, it may actually be more difficult to write good unstructured
questions. Here, I'll discuss the variety of structured questions you can consider for your survey (we'll
Dichotomous Questions
When a question has two possible responses, we consider it dichotomous. Surveys often use dichotomous
questions that ask for a Yes/No, True/False or Agree/Disagree response. There are a variety of ways to lay
We can also classify questions in terms of their level of measurement. For instance, we might measure
occupation using a nominal question. Here, the number next to each response has no meaning except as a
placeholder for that response. The choice of a "2" for a lawyer and a "1" for a truck driver is arbitrary -- from
the numbering system used we can't infer that a lawyer is "twice" something that a truck driver is.
We might ask respondents to rank order their preferences for presidential candidates using
an ordinalquestion:
We want the respondent to put a 1, 2, 3 or 4 next to the candidate, where 1 is the respondent's first choice.
Note that this could get confusing. We might want to state the prompt more explicitly so the respondent
knows we want a number from one to 4 (the respondent might check their favorite candidate, or assign
higher numbers to candidates they prefer more instead of understanding that we want rank ordering).
We can also construct survey questions that attempt to measure on an interval level. One of the most
common of these types is the traditional 1-to-5 rating (or 1-to-7, or 1-to-9, etc.). This is sometimes referred to
as a Likert response scale (see Likert Scaling). Here, we see how we might ask an opinion question on a
1-to-5 bipolar scale (it's called bipolar because there is a neutral point and the two ends of the scale are at
by the respondent on a set of bipolar adjective pairs (using 5-point rating scale):
Finally, we can also get at interval measures by using what is called a cumulative or Guttman
scale (seeGuttman Scaling). Here, the respondent checks each item with which they agree. The items
themselves are constructed so that they are cumulative -- if you agree to one, you probably agree to all of
Sometimes you have to ask the respondent one question in order to determine if they are qualified or
experienced enough to answer a subsequent one. This requires using a filter or contingency question.
For instance, you may want to ask one question if the respondent has ever smoked marijuana and a
different question if they have not. in this case, you would have to construct a filter question to determine
Filter questions can get very complex. Sometimes, you have to have multiple filter questions in order to
direct your respondents to the correct subsequent questions. There are a few conventions you should keep
• try to avoid having more than three levels (two jumps) for any question
Too many jumps will confuse the respondent and may discourage them from continuing with the survey.
• if only two levels, use graphic to jump (e.g., arrow and box)
The example above shows how you can make effective use of an arrow and box to help direct the
If you can't fit the response to a filter on a single page, it's probably best to be able to say something like "If
YES, please turn to page 4" rather that "If YES, please go to Question 38" because the respondent will