RESEARCH DESIGN

Meaning, Purpose, and Principles .

Research design is the plan, structure, and strategy of investigation conceived so as to (a) obtain answers to research investigations and (2) control variance. The plan is the overall scheme or program of the research. It includes an outline of what the investigator will do from writing the hypothesis and their operational implications to the final analysis of data. The structure of the research is more specific. It is the outline, the scheme, the paradigm of the operation of the variables.

A research design has two (2) important purposes: a) To provide answers to research questions. The research design tells us what observations to make, how to make them, and how to analyze the quantitative representations of the observations. An adequate design also outlines possible conclusions to be drawn from the statistical analysis. b) To control variance
The main technical function of research design is to control

variance. A research design is a set of instructions to the investigator to gather and analyze his data in certain ways. It is, therefore, a control mechanism. The statistical principle behind this mechanism is: Maximize systematic variance, control systematic extraneous variance, and minimize error variance. Systematic variance refers to the variance of the dependent variable influenced by the independent variable. Control of extraneous variables means that the influences of independent variables extraneous to the purposes of the study are minimized, nullified, or isolated.

 Error variance is the variability of measures due to random

fluctuations whose basic characteristic is that they are unpredictable. Two important determinants of error variance are: (1) individual differences among subjects, and (2) errors of measurement. are several ways to control extraneous variance, namely:

There a)

b)

Eliminate the variable as a variable by choosing subjects so that they are homogenous as possible on that independent variable. The problem with this approach is that we lose the power of generalization. Through randomization. If randomization was thoroughly accomplished, then the groups can be considered statistically equal in all possible ways. This does not mean, however, that the groups are equal in all the possible variables because by chance the groups can be unequal but the probability of their being equal is greater than the probability of their not being equal. Hence, randomization is the best way of controlling all possible extraneous variables.

c.)By building the extraneous variables right into the design of the study as independent variables thereby making it possible to extract from the total variance of the dependent variable, the variance due to extraneous variables. d.)By matching subjects. The variable on which the subjects are matched must be substantially related to the dependent variable or the matching is a waste of time and can be misleading. The problem with matching is that it is difficult to find matched subjects on more than two variables. e.)By statistical control, e.g., analysis of covariance.

Internal and External Validity

The

research design must be constructed so as to maximize internal and external validity.

Internal

validity asks the question: Did, in fact, the independent variables make a difference in the dependent variable in this specific instance?
External

validity asks the question: To what populations, settings, treatment variables, and measurement variables can this effect be generalized?

There are eight (8) classes of extraneous variables which, if not controlled in the research design, may produce effects becoming confounded with the effect of the independent variables:
History,

refers to the specific events occurring between the first and the second measurements in addition to the experimental variables.
Maturation,

refers to processes within the respondents operating as a function of the passage of time per se (not specific to the particular events) including growing older, growing hungrier, growing more tired, and the like.
Testing,

refers to the effects of taking a test upon the scores of a second test.

Instrumentation, refers to changes in obtained measurement due to changes in instrument calibration or changes in the observers or judges. Statistical regression, refers to the phenomenon operating where group have been selected on the basis of their extreme scores. Selection, refers to biases resulting from differential selection of respondents for the comparison groups. Experimental mortality, refers to the differential loss of respondents from the comparison groups. Selection-maturation interaction, refers to the interaction effects between the aforementioned variables which can be mistaken for the effects of the independent variables.

There are four (4) jeopardizing factors to external validity (or representativeness), namely: 1. The reactive or interactive effect of pretesting, in which a pretest might increase or decrease the respondents sensitivity or responsiveness to the experimental variable and thus make the results obtained for a pre-tested population unrepresentative of the effects of the experimental variable for the un-pretested universe from which the experimental respondents were selected.

Pretesting modifies the subject in such a way that he responds to the experimental treatment differently than will unpretested persons in the same population. 2. The interaction effects of selection biases and the experimental variable.

3. Reactive effects of experimental arrangements/procedures, which would preclude generalization about the effect of the experimental variable upon persons being exposed to it in non-experimental settings. It means effects arising from the experimental setting which will not occur in non-experimental setting. 4. Multiple-treatment interference refers to effects due to multiple treatments applied to the same subjects where prior treatments influence subsequent treatments in the series because their effects are not erasable.

Research Designs
1.According to Kidder, the design of the study maybe either any of the following: a. Pre-experimental One shot case study

 One group pretest/posttest Static group comparison b. Quasi-experimental Interrupted time series Regression discontinuity Pretest-posttest non-equivalent control group c. Experimental design Randomized two group Before-after two group Solomon four group Factorial design (e.g. split plot) Repeated measure (One unit of analysis and several treatments)

d. Survey research Static group comparison Panel design Cross-sectional pseudo-panel e. Participant Observation Complete participation Observer as participant Participant and observer Complete observer
2.There are over 20 possible designs based on the foregoing presentation. But the most commonly used especially in evaluation researches are presented below. The following notations are used to diagram each design:

R
_ _ _ _

-means random assignment

-drawn horizontally, separating the two groups, indicates that the groups not randomly assigned, that is, the groups are non-equivalent -indicates measurement of some kind, an observation -indicates the program to be evaluated

O X

a. Classical experimental research design The classical experimental research design is also called the true control group, pretest-posttest design. It is an ideal design for assessing the impact of projects because it effectively controls plausible alternative causal factors needed to make valid causal inferences of project impact. It can be shown using the diagram below:

EG: R O1 CG: R O1

O2 O2

b. Pretest-posttest, non-equivalent groups design This design differs from the foregoing in that experimental and control groups are not randomly assigned. Because of this, the control group is a not a true control group, that is, it is not statistically equivalent to the experimental group. The following diagram illustrates this design: EG: O1 X O2 CG: O1 O2 c. Posttest only, non-equivalent groups design This design is resorted to when it is not possible to collect information on the experimental and control groups prior to the introduction of the project. In most cases, evaluation is not a built-in component in the project cycle and baseline data were not collected at the inception of the project.

EG: CG:

O1 O2

d. The true control group, posttest only design The true control group posttest only design is the same as posttest only, non-equivalent groups design only that it is randomized. Using this design in evaluating development projects requires an identical entity or unit as control group. This can be illustrated through the following: EG: R X O1 CG: R O2 e. Before-and-after design This is one of the simplest designs for establishing causal attribution between two or more variables. Only the experimental group is observed before and after the intervention is introduced. Hence, establishing causal attribution faces some difficulty.

The before-and-after design can be shown in a diagram through the following: EG: O1 X O2 f. Posttest only, one group design In this design, only the experimental group is observed but only after exposure to the project. To be able to make inferences regarding project impact, measured outcome indicators are compared with some comparable areas or groups using secondary data. The following diagram will illustrate this: CG: X O2

g. The single group time series design A number of observations or measurements are done prior to and after exposure to the treatment/intervention. This enables the evaluators to validate their findings. The following diagram shows the longitudinal character of this design.
EG: O1 O2 O3 X O4 O5 O6

Basic Research Methods

1. True Experimental Research a. The purpose of true experimental research is to investigate possible cause-and-effect relationships by exposing one or more experimental groups to one or more treatment conditions and comparing the results to one or more control groups not receiving the treatment. b. Important characteristics include:
Rigorous management/control of the independent and dependent variables. Use of a control group as a baseline against which to compare the groups receiving the experimental treatment. Random selection of subjects, random assignment of subjects to groups, and random assignment of experimental treatments to groups.

c. All variables of concern are held constant except the treatment variables are deliberately manipulated or allowed to vary. Thus, the internal validity of experimental design is high. d. Its major disadvantages relate to external validity, namely, artificial nature of the experimental setting, demand characteristics, and evaluation apprehension. Demand characteristics- refer to the fact that the experimental setting may evoke certain demands--- that is, expectations on the part of the subjects to act in the way they think the experimenter would wish. Evaluation apprehension- refers to the concern that a subject has about being observed and judged while in the laboratory setting. That is, subjects may try to present themselves in a favorable light

2. Field Experiment n contrast to the laboratory experiment, the setting of the field experiment is a natural one and the subjects are not generally aware that they are subjects in an experiment. The researcher who uses a field experiment to test a hypothesis is looking at behavior in its natural setting. The advantages of using a field experiment include the following: By focusing on behavior in a natural setting, the experimenter can be much more certain of the external validity of his findings. Problems of the artificiality of the experimental setting, demand characteristics, and evaluation apprehension are eliminated. The problems associated with this research method are as follow:

Because

the researcher is working in a complex natural setting where many events may occur simultaneously, he must be sure that the independent variable is sufficiently visible for potential subjects to have the opportunity to react. Otherwise, a failure to respond becomes difficult to interpret. The dependent variable must be selected carefully. The researcher must be able to readily observe and reliably judge the dependent variable behavior. Ethical concern. Is it reasonable for the investigator to involve individuals in an experiment without their knowledge or permission? Researcher has no control over majority of events in the environment. Unexpected events may reduce or destroy the effectiveness of the manipulation.
Example:

In 1968, more than 4,000 passengers on the IND subway in New York City saw another passenger fall on the floor of the train. The passengers did not know it that they were subjects in a field experiment designed to assess helping behavior (Piliavin et al, 1969). The experimenters arranged the situation so that the characteristics of the person needing help would vary: he was either black or white, and was either apparently drunk or slightly disabled. After the person fell, the experimenters, who were also present in the subway car, noted the number of people who offered to help, their race, and the speed of the helping response.

3. Quasi-Experimental Research The purpose of this method is to approximate the conditions of the true experiment in a setting which does not allow the control and/or manipulation of all relevant variables.
This

research is characterized by methods of partial control based on a careful identification of factors influencing both internal and external validity.
The

strengths of quasi-experimental research is that it allows the study of very strong variables that cannot be manipulated or controlled by the experimenter. They also have high external validity.
The

weaknesses of the design have to do with the lack of control over the independent variables and the absence of random assignment of subjects.

4. Ex-Post Facto Research It is a systematic empirical inquiry in which the scientist does not have direct control of independent variables because their manifestations have already occurred or because they are inherently not manipulable.
Inferences

about relations among variables are made without direct intervention from concomitant variation of independent and dependent variables.
It

involves the investigation of possible cause-and-effect relationships by observing some existing consequences and searching back through the data for plausible causal factors. This is in contrast to experimental design which collect its data under controlled conditions in the present. The weaknesses of ex-post facto research include the following:

Lack

of control over independent variables. Within the limits of selection, the investigator must take the facts as he finds them with no opportunity to arrange the conditions or manipulate the variables that influenced the facts in the first place. The difficulty in being certain that the relevant causative factor is actually included among the many factors under study. The complication that no single factor is the cause of an outcome but some combination and interaction of factors go together under certain conditions to yield a given outcome. A phenomenon may result not only from multiple causes but also from one cause in one instance and from another cause in another instance. When a relationship between two variables is discovered, determining which is the cause and which is the effect maybe difficult. The fact that two or more factors are related does not necessarily imply a cause-and-effect relationship. They all simply maybe related to an additional factor not recognized or observed.

5, Survey Research
It

is a research method which studies large and small populations by selecting and studying samples chosen from the populations to discover the relative incidence, distribution, and interrelations of sociological and psychological variables.
Oftentimes

called sample survey. There are five basic types of surveys based on the method of obtaining information, namely:
Personal interview is the most powerful of the five types.

Interview schedules are difficult to construct, they are time consuming, relatively costly, but there is no other method that yield the information the way they do. Mail questionnaire has serious drawbacks, namely, the possible lack of response, the inability to check the responses given, and generally poor returns, which is why it should not be used if a better method can possibly be used and if it is to be used, it should be used in conjunctions with other techniques.

Panel survey interviews a sample of respondents and then re-interviewed and studied at later times, thereby, enabling the researcher to study changes in behavior and attitudes. Telephone surveys have little to recommend them beyond speed and low cost. Controlled observation The advantages of survey research are as follow:
It

has the advantage of wide scope: a great deal of information can be obtained from a large population. It allows for very specific formulation of issues of concern. Survey research information is accurate--within sampling error, of course.

The

disadvantages of survey research are as follow: Survey information ordinarily does not penetrate very deeply below the surface. It is demanding of time and money. Interviews are susceptible to interviewer bias. The survey interview can temporarily lift the respondent out of his own social context, which may make the results of the survey invalid. It requires a good deal of research expertise. The researcher must know sampling, question and schedule construction, interviewing, data analysis, and other technical aspects of the survey.
Case

and Field Study The purpose of case and field study research is to study intensively the background, current status, and environmental interactions of a given social unit---an individual, a group, an institution, or a community.

Case

studies are in-depth investigations of a given social unit resulting in a complete, well-organized picture of that unit. Depending upon the purpose, the scope of the study may concentrate on specific factors or take in the totality of elements and events.
Compared

to a survey study which tends to examine a small number of variables across a large sample of units, the case study tends to examine a small number of units across a large number of variables and conditions.
Case

studies are particularly useful as background information for planning major investigations in the social sciences. Because they are intensive, they bring to light the important variables, processes, and interactions that deserve more intensive attention. They pioneer new ground and are often the source of fruitful hypotheses for further study.
Case

study data provide useful anecdotes or examples to illustrate more generalized statistical findings.
Because

of their narrow focus on a few units, case studies are limited in their representativeness and external validity.

Case studies are particularly vulnerable to subjective biases. The case itself maybe selected because of its dramatic, rather than typical attributes, or because it neatly fits the researchers preconceptions. Subjective interpretation tends to influence also the results of the study.
Archival Research a. It refers to the analysis of any existing records that have been produced or maintained by persons or organizations other than the researcher. In other words, the original reason for collecting the records was not that of the researchers. Archival research has two important advantages:
b.It allows the researcher to test hypothesis over a wider range of time and societies than would otherwise be possible. Demonstrating the validity of a hypothesis in a number of different cultures and historical periods gives us considerable confidence in the validity of the hypothesis.

It

uses unobtrusive measures. Because the information used in archival research was originally collected for some other purposes, there is little or no chance that demand characteristics or evaluation apprehension will be problems for the present researcher.

c. The major disadvantage of this method has to with difficulties associated with data availability. The researcher is often to the mercy of those who collected the data. Sometimes, of course, creativity and ingenuity will help the researcher to locate the kinds of data needed. In other cases, missing or inaccurate records will prevent an adequate test of the researchers hypothesis. Even if the material is available, it is sometimes difficult to categorize it in the way necessary to answer the research question. Careful methods of coding, content analysis, and the use of complicated statistical analysis are often needed.