RESEARCH DESIGNS AND METHODS

Regina F. Valencia Ph.d Ed.Dev. Ed.Dev.

OUTLINE OF TOPICS


Topic 1: Quantitative Design vs. Qualitative Design in Research Endeavors Topics 2 & 3: Quantitative and Qualitative Methods: Nature, Applications and Promises

QUANTITATIVE vs. QUALITATIVE



Research design = the investigator s overall strategy to answer the research question - determining the appropriate research design for research problems leads to effectiveness and efficiency

QUANTITATIVE vs. QUALITATIVE



SIMILARITIES - Work with empirical data - Involve careful & systematic study - Attempt to describe social world - Attempt to add to human knowledge - Have a community of scholars - Must be valid and reliable

QUANTITATIVE vs. QUALITATIVE

Quantitative
Objective Hard science Literature review must be done earlier in the study Tests theory

Qualitative
Subjective Soft science Literature review may be done as study progresses or even afterward Develops theory

QUANTITATIVE vs. QUALITATIVE

Quantitative
One reality: focus is concise and narrow Reduction, control, precision Measurable Mechanistic: parts equal the whole

Qualitative
Multiple realities: focus is complex and broad Discovery, description, understanding, shared interpretation Interpretative Organismic: whole is greater than the parts

QUANTITATIVE vs. QUALITATIVE

Quantitative Qualitative

Report statistical analysis; Report rich, narrative Basic element of analysis is individual interpretation; numbers Basic element of analysis is words/ideas Researcher is separate Uses subjects ContextContext-free Researcher is part of the process Uses participants Context-dependent Context-

QUANTITATIVE vs. QUALITATIVE

Quantitative
Create hypotheses Reasoning is logistic and deductive Establishes relationships, causation Uses instruments

Qualitative
Create research questions Reasoning is dialectic and inductive Describes meaning, discovery Uses communication and observation

QUANTITATIVE vs. QUALITATIVE

Quantitative Strives for generalization Methods: descriptive, correlational, mathematical (statistical), quasiquasi-experimental and experimental Sample size: 30-500 30(distinct) Qualitative Strives for uniqueness Methods: phenomenological, grounded theory, ethnographic, historical, philosophical, case study Sample size: not a concern (seeks information-rich sample) information-

QUANTITATIVE vs. QUALITATIVE

Quantitative
Types of questions: non-probing non-

Qualitative
Types of questions: probing

Types of analysis: Types of analysis: statistical summarization subjective, interpretive Easy to replicate Counts the beans Tough to replicate Provides information as to which beans are worth counting

QUANTITATIVE vs. QUALITATIVE

..In selecting research designs, consider ASKING yourself.. 1. What information is needed to make current decisions about a product or program? 2. Of this information, how much can be collected and analyzed in a low-cost and practical manner (e.g., using questionnaires, surveys, etc.)? 3. Which methods will get most, if not all, of the needed information?

TOPIC 2:

QUANTITATIVE METHODS
NATURE, APPLICATIONS AND PROMISES

QUANTITATIVE METHODS
SURVEY RESEARCH
- examines samples for the purpose of inferring conclusions about the populations from which samples were drawn -serves two principal functions: description (produces a profile of pertinent population characteristics) and explanation (explores the reasons for the descriptive data)

QUANTITATIVE METHODS
SURVEY


Advantages: - data collected in realistic setting - relatively inexpensive - yields large amount of data - particularly useful in describing the characteristics of a large population

QUANTITATIVE METHODS
SURVEY


Disadvantages: - hard to establish causality - potential biases exist - non-response threatens validity non- subjects often unable to recall information about themselves & their past activities - deception - minimal room for feedback and probing

QUANTITATIVE METHODS
SURVEY


What can surveys measure? - Knowledge - Belief - Attitudes - Orientations - Behavior

QUANTITATIVE METHODS
SURVEY


Types, according to time element: - Cross-sectional surveys - Longitudinal surveys Types, according to design: - Self-administered surveys - Interviewer-conducted surveys

QUANTITATIVE METHODS
SURVEY


Self-administered surveys -Mail surveys -Online surveys (e-mail and Internet) -Group administered surveys Interviewer-conducted surveys - Telephone interviews and CATI - Personal survey interviews

QUANTITATIVE METHODS
SURVEY


Survey Questionnaire - this highly determines the success or failure of the survey research, thus calls for researcher to ask and frame the right questions, to identify the best questionnaire structure and the proper means of distribution

QUANTITATIVE METHODS
SURVEY


Types/Formats of Survey Questions Multiple-choice Checklist Fill-in-the-blank Contingency or filter questions Matrix questions Double-barreled questions Leading questions Open-ended questions Close-ended questions

QUANTITATIVE METHODS
SURVEY


Guidelines in structuring the questions: 1. Simple is best: short and clear. 2. Avoid: Negatives DoubleDouble-barreled terms Biased or loaded terms Leading questions 3. All questions should be relevant. 4. Respondents must be competent to answer. 5. Close-ended questions are generally preferred. Close-

QUANTITATIVE METHODS
SURVEY
 1)

2)

How do you organize the questions? Adopt a general organizational pattern that complements your research objectives (i.e. funnel and inverted funnel order). Topically related questions should be grouped together.

QUANTITATIVE METHODS
SURVEY
3) Easy-to-answer questions should be placed Easy-tofirst. 4) Questions should be ordered in a manner that avoids establishing a response bias .

QUANTITATIVE METHODS
SURVEY


Questionnaire Design Design depends on survey type Introduction Instructions Question order Layout Length PrePre-testing

QUANTITATIVE METHODS
SURVEY


Sample study: The Magic of Lumen s Wais Tips: The effectiveness of the SURF Wais Tips Ni Lumen Serial Advertisements in influencing the buying public A Survey (Boncodin, 2002)

QUANTITATIVE METHODS
CONTENT ANALYSIS
-Technique for making inferences by objectively and systematically identifying specified characteristics of messages

- To describe the characteristics of communication, asking what, how, to whom, why is said and with what effect the phenomenon results to

QUANTITATIVE METHODS
CONTENT ANALYSIS


Content Analysis Research Model I. Conceptualization stage Problem identification Literature review Hypotheses or RQ specification II. Design stage III. Analysis stage

QUANTITATIVE METHODS
CONTENT ANALYSIS


Content Analysis Research Model II. Design stage Define message population Define and select a sample Sources and dates Define unit of analysis Create content categories Mutually exclusive Exhaustive Reliable Develop quantification scheme Train coders and run pilot study (especially in large-scale content largeanalyses; inter-coder reliability) inter-

QUANTITATIVE METHODS
CONTENT ANALYSIS


Content Analysis Research Model III. Analysis Stage Code content Assess reliability Analyze data (statistical tests) Reporting

QUANTITATIVE METHODS
CONTENT ANALYSIS


Advantages: Non-reactive measurement NonTime series data available Data reduction Widely applicable Disadvantages: Focus on quantification, only frequency Latent or real meaning of data content is not addressed

QUANTITATIVE METHODS
CONTENT ANALYSIS


Sample studies: - A Content Analysis of the Roles and Images of the Women of Sex and the City (Bartolome, 2002) - Green TV: A Content Analysis of Environmental Issues Presented on News and Current Affairs Programs of ABS-CBN, ABSGMA & NBN (Lavares, 2002)

QUANTITATIVE METHODS
EXPERIMENTAL RESEARCH -Functional research design whose purpose is to explain the impact that certain input variables (independent) exert on outcome variables (dependent) - Since it focuses on determining causation, better suited for explanatory purposes

QUANTITATIVE METHODS
EXPERIMENTAL RESEARCH


Advantages: Establishes causality (IR & DV) Control Replication Disadvantages: Artificiality Experimenter bias Limited scope Potential ethical concerns on subjects

QUANTITATIVE METHODS
EXPERIMENTAL RESEARCH


Basic Designs: PretestPretest-posttest control group PosttestPosttest-only control group Solomon four-group design four-

QUANTITATIVE METHODS
EXPERIMENTAL RESEARCH


Conducting experiments: 1) Select a setting 2) Select experimental design 3) Identify the independent and dependent variables 4) Determine IV manipulation Straightforward Staged (Deception) 5) Select and assign subjects to conditions 6) Conduct a pilot study 7) Run the study

QUANTITATIVE METHODS
EXPERIMENTAL RESEARCH


Sample study: Naivete: An Experimental Study on the SelfSelf-Concept of Children Based on Their Socio-Economic Status (Cruz, Socio2003)

OPEN FORUM QUESTIONS PLEASE..

Data Collection Methods

NATURE, APPLICATIONS AND PROMISES

QUALITATIVE METHODS
- Create events in which FOCUS INTERVIEWS one person (interviewer)

encourages another person (interviewee) to articulate interests or experiences freely

- conversation with a purpose

QUALITATIVE METHODS
FOCUS INTERVIEWS


Basic objectives:

- learning about things that cannot be observed directly by other means - understanding a social actor s perspective - verifying, validating or commenting on data obtained from other sources

QUALITATIVE METHODS
FOCUS INTERVIEWS


Basic objectives: - testing the assumptions that the researcher has made prior to the interview - eliciting the distinctive language (i.e. jargon, vocabulary, idioms and forms of speech) used by interviewees in their own settings

QUALITATIVE METHODS
FOCUS INTERVIEWS


7 Stages of an Interview Investigation

Thematizing Designing Interviewing Transcribing Analyzing Verifying Reporting

QUALITATIVE METHODS
FOCUS INTERVIEWS


Types of interview questions: Introducing questions Follow-up questions FollowProbing questions Specifying questions Direct questions Indirect questions Structuring questions Interpreting questions

QUALITATIVE METHODS
FOCUS INTERVIEWS
 1)

Quality Criteria for an Interview The extent of spontaneous, rich, specific and relevant answers from the interviewee. The shorter the interviewer s questions and the longer the subjects answers, the better. The degree to which the interviewer follows up and clarifies the meanings of the relevant aspects of the answers.

2)

3)

QUALITATIVE METHODS
FOCUS INTERVIEWS


Quality Criteria for an Interview

4) The interviewer attempts to verify his or her interpretations of the subject s answers in the course of the interview. 5) The interview is self-communicating it selfis a story contained in itself that hardly requires much extra descriptions and explanations.

QUALITATIVE METHODS
FOCUS INTERVIEWS


Qualification Criteria for the Interviewer

Knowledgeable Structuring Clear probing Gentle Sensitive Open Steering Critical Remembering Interpreting

QUALITATIVE METHODS
FOCUS INTERVIEWS
 -

Sample studies: ReRe-constructing the World Trade Center: A Study of Social Construction in the News: The Case of Sept. 11th Attacks in the US (Macalincag, 2002) Click & Shop `Till You Drop: A Study on the emergence of E-Commerce as a Ecommunication and marketing strategy (Malana, 2002)

QUALITATIVE METHODS
FOCUS GROUP DISCUSSION / - manner of obtaining data FGD on a given topic through
discussion among members of a group that have been chosen based on specific criteria

QUALITATIVE METHODS
FGD


Characteristics of FGD Involves small groups of people (6-10) (6Homogeneous, but unfamiliar Inductive, interactive Comparative (if involves multiple groups) - Has a focus and direction - In-depth but limited generalizability In- Can be either stand alone method or in conjunction with others -

QUALITATIVE METHODS
FGD


Planning FGD - Determine the purpose - When and when not to use FGD - Participants: quality and quantity - Design options SingleSingle-category design Multiple-category design Multiple- Get advice from target audience - Write up a plan

QUALITATIVE METHODS
FGD


Advantages: 1) Social interaction among participants make the latter feel that they can disclose more. 2) The researcher who most often is the moderator is allowed to probe deeper into the discussion. 3) High face validity, thus more feedback. 4) Relatively low cost. 5) Speedy results.

QUALITATIVE METHODS
FGD


Disadvantages: 1) Researcher has less control in the group interview as compared to the individual interview. 2) Data are more difficult to analyze. 3) Technique requires carefully trained interviewers/moderators. 4) Groups are difficult to assemble. 5) Logistical constraints.

QUALITATIVE METHODS
FGD


Categories of FGD Questions: Opening questions Introductory questions Transition questions Key questions Ending questions Summary question Final question

QUALITATIVE METHODS
FGD


Qualities of good FGD questions: Conversational Commonly used language Easy to say Clear Short OpenOpen-ended Include clear directions

QUALITATIVE METHODS
FGD


A good questioning route: Has an easy beginning Is sequenced Moves from general to specific Uses time wisely

QUALITATIVE METHODS
FGD


FGD Participants

Group composition Size of a focus group Finding participants Selecting participants

QUALITATIVE METHODS
FGD


FGD Participants Improving attendance - Select appropriate times and location - Make personal contact - Send formal invitation - Reminder phone call Incentives

QUALITATIVE METHODS
FGD


Moderating FGDs A pivotal role: the moderator Moderating team Stages of moderating: PrePre-session Small talk Snacks

QUALITATIVE METHODS
FGD


Moderating FGD Session Recording discussion Getting started Anticipating the flow of discussion Pause-andPause-and-probe technique Respond appropriately, consistently Dealing with problem personalities Concluding the focus group

QUALITATIVE METHODS
FGD
 -

Sample studies: Sure Ka Ba Is That Your Final Answer Goodbye!: A Qualitative Study of Primetime TV Game Shows (Bringas, 2002) The Man of the House: An Exploratory Study on the Lives of Househusbands (Lantin, 2002)

QUALITATIVE METHODS
CASE STUDY

- preferred strategy when:

1) how or why questions are being posed 2) when the investigator has little control over events 3) when the focus is on a contemporary phenomenon within some real-life context

QUALITATIVE METHODS
CASE STUDY


Traditional complaints against this method: 1) Lack of rigor as a method may lead to researcher bias 2) Provides little basis for scientific generalization 3) Takes too long and results in massive, unreadable documents later on 4) High degree of difficulty in implementation

QUALITATIVE METHODS
CASE STUDY


Desired skills of a case study investigator: - Knows how to ask questions properly - Good listener, too - Adaptive and flexible - Has a clear grasp of the issue being studied - Lack of bias

QUALITATIVE METHODS
CASE STUDY


Sources of evidence in case studies: Documentation Archival records Interviews Direct observations Participant-observation notes ParticipantPhysical artifacts

QUALITATIVE METHODS
CASE STUDY

What makes an exemplary case study? must be significant must be complete must consider alternative perspectives must display sufficient evidence must be composed and written in an engaging manner

QUALITATIVE METHODS
CASE STUDY
 

Sample study: Icon of the Masses: The Judy Ann Santos phenomenon: An analysis of Judy Ann Santos wide mass appeal on the Filipino audience (Cruz, 2002) Shooting the President: A Case Study on the Photographs of Pres. Arroyo Published on the Philippine Daily Inquirer (Luciano, 2003)

QUALITATIVE METHODS
FEASIBILITY STUDY
-Thorough and systematic analysis of all factors that affect the possibility of success of a proposed undertaking. -Data, facts and other findings presented in the study then become the basis for deciding whether the project is to be pursued, abandoned, or revised - Is a synthesis of separate studies usually dealing with the market, technical, financial, socio-economic, and management aspects of the project

QUESTIONS, AGAIN PLS?