SURVEY RESEARCH DESIGN IN LANGUAGE LEARNING

A. Survey Research
Survey research is a commonly used method of collecting information
about a population of interest. The major purpose of surveys is to describe the
characteristics of population.
The three major characteristics that most surveys possess:
1. Information is collected from a group of people in order to describe some
aspects or characteristics (such as abilities, opinions, attitudes, beliefs, and/or
knowledge) of the population of which that group is a part.
2. The main way in which the information is collected is through asking
questions; the answers to these questions by the members of the group
constitute the data of the study.
3. Information is collected from a sample rather than from every member of the
population.
There are many different types of surveys, several ways to administer
them, and many methods of sampling. There are three key features of survey
research:

Questionnaires -- a predefined series of questions used to collect information

from individuals. This method gathers responses to questions that are essay or
agree/neutral/disagree style.

Interviews - questions posed to an individual to obtain information about him

or her. This type of survey is like a job interview, with one person asking
another a load of questions.

Sampling -- a technique in which a subgroup of the population is selected to

answer the survey questions; the information collected can be generalized to
the entire population of interest.

B. Steps in Surveys Research

1. Defining the Problem
The problem to be investigated by means of a survey should be sufficiently
interesting and important to motivate individuals to respond.
2. Identifying the Target Population
Almost anything can be described by means of a survey. That which is studied
in a survey is called the unit of analysis. Although typically people, units of
analysis can also be objects, clubs, companies, classrooms, schools,
government agencies, and others.
3. Choosing the Mode of Data Collection
There are four basic ways to collect data in a survey: by administering the
survey instrument live to a group; by mail; by telephone; or through face-toface interviews.
4. Selecting The Sample
The subjects to be surveyed should be selected (randomly, if possible) from
the population of interest. Researchers must ensure, however, that the subjects
they intend to question possess the desired information and that they will be
willing to answer these questions. Individuals who possess the necessary
information but who are uninterested in the topic of the survey (or who do not
see it as important) are unlikely to respond.
5. Preparing the Instrument
The most common types of instruments used in survey research are the
questionnaire and the interview schedule
C. Questionnaire Design
The two most common types of survey questions are closed-ended
questions and open-ended questions.
a. Closed-Ended Questions

The respondents are given a list of predetermined responses from which to

choose their answer

The list of responses should include every possible response and the
meaning of the responses should not overlap

An example of a close-ended survey question would be, "Please rate how

strongly you agree or disagree with the following statement: 'I feel good
about my work on the job.' Do you strongly agree, somewhat agree,
neither agree nor disagree, somewhat disagree, or strongly disagree?"

A Likert scale, which is used in the example above, is a commonly used

set of responses for closed-ended questions

Closed-ended questions are usually preferred in survey research because of

the ease of counting the frequency of each response

b. Open-Ended Questions

Survey respondents are asked to answer each question in their own words

Responses are usually categorized into a smaller list of responses that can
be counted by the study team for statistical analysis
If you use questionnaires, you will sit down and write up some questions

that you need answers to. This can go in several ways:

Open ended questions where the participant fills in the answer with their
thoughts. For example, 'What do you think of pet ownership?' This is
useful for a descriptive study, but there is very little here that you can
analyze statistically.

Multiple choice questions allow for statistical analysis, such as 'Do you
think pet ownership is a good thing for people - agree, neutral, or disagree.'
However, you may miss some personal feelings or thoughts on the
situation.

Using questionnaires allows a researcher to utilize several strengths. For example:

It allows for minimal contact between researcher and participant.

Multiple avenues, such as handing them out in person, using snail mail,
email, and online survey engines, can be used.

Participant's answers are readily recorded on the forms.

Questionnaires aren't all sunshine and happy times though. There are some weak
points that need to be addressed. For instance:

The questions and instructions must be written extremely clearly or

participants will answer in incorrect ways.

There is little ensuring participants finish a survey, meaning they may

return it half finished and therefore useless.

The cost of printing out hundreds of questionnaires can quickly become

very expensive.

D. Considerations for Designing a Questionnaire

It is important to consider the order in which questions are presented.
Sensitive questions, such as questions about income, drug use, or sexual
activity, should be put at the end of the survey. This allows the researcher to
establish trust before asking questions that might embarrass respondents.
Researchers also recommend putting routine questions, such as age, gender,

and marital status, at the end of the questionnaire.

Double-barreled questions, which ask two questions in one, should never be
used in a survey. An example of a double barreled question is, "Please rate
how strongly you agree or disagree with the following statement: 'I feel good
about my work on the job, and I get along well with others at work.'" This
question is problematic because survey respondents are asked to give one

response for two questions.

Researchers should avoid using emotionally loaded or biased words and
phrases

E. Survey Administration
Surveys can be admininistered in three ways:

Through the mail

o Advantage: Low cost
o Disadvantage: Low response rate

By telephone
o Advantages: Higher response rates; responses can be gathered
more quickly
o Disadvantage: More expensive than mail surveys

Face-to-face
o Advantages: Highest response rates; better suited to collecting
complex information
o Disadvantage: Very expensive

F. Sampling Procedures
One of the primary strengths of sampling is that accurate estimates of a
population's characteristics can be obtained by surveying a small proportion of the
population. Four sampling techniques are described here:
a. Simple Random Sampling

Simple random sampling is the most basic form of sampling

Every member of the population has an equal chance of being selected

This sampling process is similar to a lottery: the entire population of

interest could be selected for the survey, but only a few are chosen at
random

Researchers often use random-digit dialing to perform simple random

sampling. In this procedure, telephone numbers are generated by a
computer at random and called to identify individuals to participate in the
survey

b. Cluster Sampling

Cluster sampling is generally used when it is geographically impossible to

undertake a simple random sample

Cluster sampling requires that adjustments be made in statistical analyses

For example, in a face-to-face interview, it is difficult and expensive to

survey households across the nation. Instead, researchers will randomly select
geographic areas (for example, counties), then randomly select households within
these areas. This creates a cluster sample, in which respondents are clustered
together geographically.
c. Stratified Sampling

Stratified samples are used when a researcher wants to ensure that there
are enough respondents with certain characteristics in the sample

The researcher first identifies the people in the population who have the
desired characteristics, then randomly selects a sample of them

Stratified sampling requires that adjustments be made in statistical

analyses
For example, a researcher may want to compare survey responses of

African-Americans and Caucasians. To ensure that there are enough Afrian-

Americans in the survey, the researcher will first identify the African-Americans
in the population and then randomly select a sample of African-Americans.
d. Nonrandom Sampling

Common nonrandom sampling techniques include convenience sampling

and snowball sampling

Nonrandom samples cannot be generalized to the population of interest.

Consequently, it is problematic to make inferences about the population

In survey research, random, cluster, or stratified samples are preferable

G. Measurement Error
Measurement error is the difference between the target population's
characteristics and the measurement of these characteristics in a survey. There are
two types of measurement error: systematic error and random error.
a. Systematic Error

Systematic error is more serious than random error

Occurs when the survey responses are systematically different from the
target population responses

For example, if a researcher only surveyed individuals who answered their

phone between 9 and 5, Monday through Friday, the survey results would
be biased toward individuals who are unemployed

Sources of bias include

o Nonobservational error -- Individuals in the target population are
systematically excluded from the sample, such as in the example
above

o Observational error -- When respondents systematically answer

surveys question incorrectly. For example, surveys that ask
respondents how much they weigh will probably underestimate the
population's weight because respondents are likely to underreport
their weight
b. Random Error

Random error is an expected part of survey research, and statistical

techniques are designed to account for this sort of measurement error

Occurs because of natural and uncontrollable variations in the survey

process, i.e., the mood of the respondent

For example, a researcher may administer a survey about marital happiness.

However, some respondents may have had a fight with their spouse the evening
prior to the survey, while other respondents' spouses may have cooked the
respondent's favorite meal. The survey responses will be affected by the random
day on which the respondents were chosen to participate in the study. With
random error, the positive and negative influences on the survey measure balance
out.
H. Ethics of Survey Research
Informed Consent
Respondents should give informed consent before participating in a survey. In
order for respondents to give informed consent,

The researcher must inform the respondents of the study's purpose,

content, duration, and potential risks and benefits

The researcher must inform the respondents that they do not have to
answer all the survey questions

The researcher must inform the resondents that they can stop participating
in the study at any point

Confidentiality and Anonymity

It is absolutely imperative that researchers keep respondents' identities
confidential. To ensure confidentiality, researchers should not link respondents'
identifiers to their survey responses when using data. Common identifiers include
names, social security numbers, addresses, and telephone numbers.
Anonymity
Anonymity is an even stronger safeguard of respondent privacy. If a researcher
assures anonymity, it means that the researcher is unable to link respondents'
names to their survey
I.

Advantages and Disadvantages of Survey Research

Advantages

Sample surveys are a cost-effective and efficient means of gathering

information about a population

Survey sampling makes it possible to accurately estimate the

characteristics of a target population without interviewing all members of
the population

Survey sampling is particularly useful when the population of interest is very

large or dispersed across a large geographic area.
Disadvantages

Surveys do not allow researchers to develop an intimate understanding of

individual circumstances or the local culture that may be the root cause of
respondent behavior

Respondents often will not share sensitive information in the survey

format

A growing problem in survey research is the widespread decline in

response rates

REFERENCES
Suryabrata, S. 2005.Metodologi educational research (Competence and Practice).
Earth Literacy .. Jakarta: King Grafindo Persada.

Latief, Mohamad Adnan, 2012. Research Methods on Language Learning an

Introduction. Malang: UM Press. 2016.

SURVEY RESEARCH
DESIGNS IN LANGUAGE LEARNIING
Lectured by Mrs. Epiasiasi, M. Pd.

Compiled by:
1.
2.
3.
4.

Bahya Alfitri
Panca Indah Wulan S
Saris Frima
Yesa Listika

2113061
2113059
2112083
2113047

ENGLISH EDUCATION STUDY PROGRAM

LANGUAGE AND ARTS EDUCATION DEPARTMENT
INSTITUTE OF TEACHER TRAINING AND EDUCATION
TEACHER ASSOCIATION OF REPUBLIC OF INDONESIA
(STKIP-PGRI) LUBUKLINGGAU
2016