CMN 102 Manual Fall 2010

CMN 102: Empirical Methods in Communication
Fall 2010 Dr. Robert A. Bell Department of Communication UC Davis
Table of Contents
Nature of Inquiry .............................................................................................................. 3 Causation ...................................................................................................................... 18 Ethics and Politics In Social Research .......................................................................... 27 Research Design ........................................................................................................... 28 Conceptualization, Operationalization, and Measurement ............................................ 40 Questionnaire Construction ........................................................................................... 91 Sampling ..................................................................................................................... 110 Experimental Methods................................................................................................. 126 Survey Research ......................................................................................................... 151 Ethnography of Communication .................................................................................. 176 Content Analysis ......................................................................................................... 191 Evaluation Research/Quasi-Experimental Design ....................................................... 203
NATURE OF INQUIRY
I. Why Do Research?
A. Limitations of Common Sense.
1.
Much of What We Know To Be True Isnt.
2.
Individualistic Nature of Common Sense.
B.
How Real is Reality?
1.
Experiential Reality: Knowledge Based on Direct Experience.
2.
Agreement Reality: Knowledge Based On Social Consensus.
a.
Tradition.
b.
Authority.
C.
Errors and Biases In Casual Human Inquiry.
1.
Inaccurate Observations.
2.
Overgeneralization. (See handout on replication as a guard against overgeneralization, page 8).
3.
Selective Observation: Ill See It When I Believe It.
4.
Ex Post Facto Hypothesizing (Made-Up Information).
5.
Illogical Reasoning. (See handout on page 9.)
6.
Ego-Involvement in Understanding.
7.
Premature Closure of Inquiry.
8.
Mystification.
II.
Theory: The Foundation of Social Science.

A. Definition: A theory is a conceptual representation or explanation of a phenomenon.
1.
All Theories Are Abstractions.
2.
Theories Are Inherently Partial.
3.
Theories Are Constructions.
B.
Theory as Explanation of Regularities in Human Symbolic Behavior.
1.
The Charge of Triviality.
2.
Exceptions Dont Deny The Rule.
C.
A Variable Language For Theory Construction.
1.
Variable: A Feature of a Class Of Objects Which Can Vary (i.e., Which Is Variable).
2.
Attributes: The Qualities Which Compose a Variable. (Alternatively, the Values That a Variable Can Assume.) See exercise on page 12.
D.
The Traditional (Deductive) Model of Scientific Method.
1.
Starting Point: Phenomenon of Interest.
a.
Casual Observation.
b.
Theory.
c.
Competing Theories.
d.
Problem In Need Of An Answer.
2.
Step 2: Theory Construction.
a.
Concepts.
b.
Theoretical Statements.
i.
Axioms.
ii.
Propositions.
iii.
Hypotheses. (See Example On Page 13).
3.
Step 3: Operationalization of a Theorys Constructs To Create Variables.
4.
Step 4: Observation (Data Collection and Hypothesis Testing).
5.
Summary: Wallaces Model of Science.
I N D U C T I O N
Theory
Empirical Generalization
Hypotheses
D E D U C T I O N
Observations
III.
Inductive Theory Construction.

A. Grounded Theory: Inducing Theory From Data.
B.
Inductive Theory and Field Research.
IV.
Some Characteristics of Good Theory.

A. Three Primary Characteristics of Good Theory.
1.
Generality.
2.
Accuracy.
3.
Parsimony (Simplicity).
B.
The Rub: Tradeoffs Among Generality, Accuracy, and Parsimony. (See Handout On Page 16.)
One Guard Against Overgeneralization is Replication

Below is a typology of replication based on four kinds of similarity between the baseline study and the replicative study.* Replications differ along four dimensions, producing 16 types of replication, each with its own advantages. The four replication dimensions are as follows: TIME: Same Versus Different Time Period. PLACE: Same Versus Different Location. SUBJECTS: Same Versus Different People. METHOD: Same Versus Different Research Procedures.
Time Same (Concurrent) Place: Methods Same Subjects Same People Different People Different Same People Different People (a) (e) (I) (m) (b) (f) (j) (n) (c) (g) (k) (o) (d) (h) (l) (p) Same Place Different Place Different (Iterative) Same Place Different Place
Adapted From: Howard M. Bahr, Theodore Caplow, and Bruce A. Chadwick. (1983). Middletown III: Problems of replication, longitudinal measurement, and triangulation. Annual Review of Sociology, 9, 243-264.
Some Common Reasoning Fallacies*
Begging the Question This fallacy is committed when we make a claim and then attempt to support it by advancing evidence which is equivalent to the original claim. Example: A: B: A. The President is telling the truth when she says she wants to help people like me. Why do you say that? Because she would not lie.
Appeals To Authority The use of authoritative testimony in reasoning is legitimate. But when authority appeals are used to close off further discussion, reasoning becomes fallacious. Example: A: B: A: The Surgeon General of the United States tells us that socialized medicine would be a disaster. I believe that there is much merit to this approach. So you think that you know more about this matter than the Surgeon General?
Arguing Against The Person This fallacy occurs when a person rejects evidence solely because of its source. Example: A: B: As President Jones once observed, tax increases actually decrease government revenues. If Jones said it, it must be wrong.
Argument From Ignorance This fallacy has been committed when one accepts a claim because its opposite cannot be proved. Example: Because you have seen no evidence that tax increases raise government revenues, you conclude that tax increases will decrease government revenues.
Adapted From: S. Toulmin, R. Rieke, & A. Janik. (1984). An introduction to reasoning (second edition). New York: Macmillan.
Consensus Bias This instance of illogical reasoning occurs when we accept something as true (or false) because most people think it is true (or false). Example: A: B: A: A person has a constitutional right to own a gun. Why do you think so. A poll I just read about says that most people believe this is the case.
Hasty Generalization This fallacy, sometimes referred to as jumping to conclusions, involves making a generalization on the basis of too few instances. Example: She is rich and she is an atheist. I guess rich people tend to reject God.
Alternatively, an individual may draw a general conclusion from atypical examples. Example: I did not like that sushi plate. I dont like Japanese food.
Confused Temporal Succession When a person assumes that event A has caused event B simply because A preceded B, this fallacy has been committed. Example: President Obama gives a marvelous speech in Egypt. The next week, his approval ratings climb. One concludes that the speech brought about the rise in ratings, ignoring other events which occurred prior to the poll.
False Analogy When one argues from analogy, one is focusing on the attributes that two entities share, ignoring those attributes not in common. This may lead to a faulty conclusion. Example: Example: You are good at ping-pong. I bet youd make a great tennis player. Any family that spends more than it brings in will eventually be ruined financially. If the government continues to deficit spend it, too, will run out of money. [Families cannot print money; governments can.]
10
Composition When one assumes that what is true about the parts of a whole is true for the whole, one may not be correct. Example: Since the individual cells of the human body are not visible to the naked eye, people are invisible. Because sodium and chlorine are deadly poisons, sodium chloride (table salt) must be poisonous.
Example:
Division When one asserts that what is true of the whole must be true of the parts, the division fallacy may be committed. Example: Example: Because the pie is round, its pieces must be round. Because sodium chloride is edible, sodium and chloride must be edible.
Gamblers Fallacy This fallacy has occurred when a person assumes that the probability of an event is affected by another event which is actually independent. Example: You have just flipped three heads in a row. You assume that the chance of flipping tails on the next toss is extremely high. In reality, the odds have not changed; they are still 50-50. The last three people you have asked out on a date have said yes. You assume that the next person will turn you down. After all, your good luck cannot go on forever.
Example:
11
Variables and Attributes

In the table below, jot down the name of the variable for the attributes given. Variable Attributes Asian, Black, Caucasian, Hispanic, Native American, Other Freshman, Sophomore, Junior, Senior, Graduate Student NBC, ABC, CBS, FOX, CW AM, FM Small (under 10 employees) Medium (11-100 employees) Large (101-1000 employees) Very Large (over 1000 employees) Army, Navy, Air Force, Marines Assistant Professor, Associate Professor, Full Professor Liberal, Moderate, Conservative Black, Blonde, Brown, Red, Grey, White
In the table below, write down a working set of attributes for the variable given. Variable Marital Status College Major Political Party Affiliation Loneliness Nationality State of Residence Birth Year Music Style TV Program Type Attribute single, married, separated, divorced, widow/widower
12
Uncertainty Reduction Theory*

Axioms 1. Given the high level of uncertainty present at the onset of the entry phase [of a relationship], as the amount of verbal communication between strangers increases, the level of uncertainty for each interactant in the relationship will decrease. As uncertainty is further reduced, the amount of verbal communication will increase. As nonverbal affiliative expressiveness increases, uncertainty levels will decrease in an initial interaction situation. In addition, decreases in uncertainty level will cause increases in nonverbal affiliative expressiveness. High levels of uncertainty cause increases in information seeking behavior. As uncertainty levels decline, information seeking behavior decreases. High levels of uncertainty in a relationship cause decreases in the intimacy level of communication content. Low levels of uncertainty produce high levels of intimacy. High levels of uncertainty produce high rates of reciprocity. Low levels of uncertainty produce low reciprocity rates. Similarities between persons reduce uncertainty, while dissimilarities produce increases in uncertainty. Increases in uncertainty level produce decreases in liking. Decreases in uncertainty level produce increases in liking.
2.
3.
4.
5.
6.
7.
Propositions 1. Amount of verbal communication and nonverbal affiliative expressiveness are positively related. [Note how this proposition has been logically deduced from Axioms 1 and 2. If Axioms 1 and 2 are correct, the proposition must also be true. Attempt to identify the axioms from which the remaining propositions have been derived.] Amount of communication and information seeking behavior are inversely related. [Deduced from Axioms 1 and 3.] Amount of communication and intimacy level of communication are positively related. [Deduced from Axioms 1 and 4.] Amount of communication and reciprocity rate are inversely related. Amount of communication and liking are positively related.
2.
3.
4. 5.
Source: C.R. Berger & R.J. Calabrese. (1975). Some explorations in initial interaction and beyond: Toward a developmental theory of interpersonal communication. Human Communication Research, 1, 99-112.
13
6. 7.
Amount of communication and similarity are positively related. Nonverbal affiliative expressiveness and intimacy level of communication content are positively related. Nonverbal affiliative expressiveness and information seeking are inversely related. Nonverbal affiliative expressiveness and reciprocity rate are inversely related. Nonverbal affiliative expressiveness and liking are positively related. Nonverbal affiliative expressiveness and similarity are positively related. Intimacy level of communication content and information seeking are inversely related. Intimacy level of communication content and reciprocity rate are inversely related. Intimacy level of communication content and liking are positively related. Intimacy level of communication content and similarity are positively related. Information seeking and reciprocity rate are positively related. Information seeking and liking are negatively related. Information seeking and similarity are negative related. Reciprocity rate and liking are negatively related. Reciprocity rate and similarity are negatively related. Similarity and liking are positively related.
8. 9. 10. 11. 12. 13. 14. 15. 16. 17. 18. 19. 20. 21.
14
Theoretical Concepts are Operationalized To Create Variables

Directions. Describe a procedure for operationalizing each of the following constructs. Concept Intelligence Respondent Sex Attentiveness Satisfaction with Ones Physicians Communication Behavior Attitude Change Persuasive Message Comprehension Network Density Fear Appeal Strength Amount of Television Viewing Popularity Family Income Response Latency Physical Attractiveness Operationalization Subjects score on a specified IQ test, administered in a standardized way. Respondents self-report of being either a male or a female, as indicated with a checkmark in one of two boxes.
15
Tradeoffs In Theorizing
Two Oclock Theory Is General and Accurate, But Not Simple. (Many Concepts Are Needed To Insure Accuracy Across a Variety of Contexts. Generalized Accuracy is Obtained By Sacrificing Parsimony.)
Six Oclock Theory is Simple and Accurate, But Not General. (A Theory Can Be Both Simple and Accurate Only if it Applies to a Narrow Range of Situations. Thus, Generalizeability Must Be Sacrificed.)
16
Ten Oclock Theory is General and Simple, But Not Accurate. (A Theory Can Be General and Simple Only By Excluding The Many Variables Which Allow For Accuracy Across a Variety of Settings.)
17
CAUSATION
I. Causality and Explanation.
II.
Determinism.
A. Deterministic Explanation.
B.
Objection: Causality Versus Free Will.
C.
Response: Reasons Have Reasons (Causes).
D.
Weak Determinism: Probabilistic Causal Model.
III.
Nomothetic and Idiographic Explanation. (See handout on page 20.)

A. Nomothetic Explanation.
B.
Idiographic Explanation.
IV.
Criteria For Causality.

A. Temporal Order Rule: Cause Must Precede Effect.
B.
Correlation: The Two Variables Must Be Empirically Related.
1.
What Different Correlations Look Like (See handout on page 22.)
18
2.
A Word on Interchangeable Indexes.
C.
Absence of Third Variable. (See exercise on page 25.)
19
Two Approaches To Communication Research*

Social Definition Approach (Idiographic) Related Labels Ontology (Study Of The Nature Of Existence)
Interpretivism, Interactionism, Ethnography, Cultural Studies Social Reality is constructed through interaction. Social actors negotiate the meanings of actions and situations. Social reality is not a thing to be interpreted; it is those interpretations. The basis of behavior is the social actors interpretations of the conditions in which they find themselves. Experience is a process of interpretation rather than direct perception of the world.
Social Facts Approach (Nomothetic)

Realism, Variable-Analytic, Positivism, Positivism Neo-
The things researchers study exist and act independently of them and their activities. The social world is an objective (real) structure of relations which is not accessible to direct observation. These very real processes generate events, only some of which are observable. Empirical world: that part of reality that can be observed. Actual world: the empirical world plus real but unobservable social processes. Knowledge is generated by building models to explain observable phenomena in terms of the underlying structures and mechanisms which constitute reality. These models consist of law-like statements about the tendencies of things. The assumption is made that if these models can explain regularities in the empirical world, they may be useful models of the real (i.e., of reality). There is no way to verify whether or not a model of reality is true.
Epistemology (Study of the Basis of Human Knowledge)
Knowledge is derived from (a) everyday meanings and interpretations through the description of these meanings and interpretations (from natives points of view) and from (b) translating these meanings into theoretical constructs. Such constructs are generated from everyday social constructs (i.e., everyday language) so that the integrity of the phenomenon can be retained.
Adapted From Norman W.H. Blaikie (1991). A critique of the use of triangulation in social research. Quality and Quantity, 25, 115-136.
20
Social Definition Approach Data

Rule-guided behavior and the intentions and motives which shape these behaviors; peoples own interpretations of their actions, the actions of others, and social situations. Descriptions cast in culturally appropriate terms. Description of cultural rules and norms which constitute the meaningfulness of interaction.
Social Facts Approach

Surface features (observable aspects) of the social world, as represented by our variables.
Explanations/ Theory
Theories are descriptions of the social structural relations that underlie reality.
Validity
Agreement among participants in a social system (including the investigator and the natives) that an account of their world is acceptable.
If the models constructed to explain observable features of the social world work, they are considered to be potentially valid accounts of reality.
21
Visualizing Different Correlation Values

Range
The correlation coefficient ranges from +1.0 to -1.0. A correlation with the absolute value of 1.0 is called a perfect correlation. When two variables have a correlation of 1.0 or -1.0, this means that a subjects value on one of the variables can be perfectly predicted (i.e., predicted without error) based on knowledge of his/her score on the other variable. The sign of r tells us if high scores on one variable are associated with high scores (positive sign) or low scores (negative sign) on the second variable. A correlation of 1.0 is just as large and informative as a correlation of +1.0. We never get perfect correlations in the social sciences. If we did obtain such a correlation, a scattergram of the relationship might look something like this:
Zero Correlation
When two variables have absolutely no relationship to each other, the correlation is r = .00. In this instance, knowledge of a subjects score on one variable does not improve in any way an investigators ability to predict that persons score on the second variable. Below is a scattergram of a bivariate distribution for which r = .00.
22
Intermediate Correlational Values

In most instances, we will obtain correlations that have an absolute value somewhere between 0 and 1.0. For example, this is what a correlation with an absolute value of .60 might look like:
23
Statistical Significance
You might have heard news anchors report the findings of a scientific study, and then describe its results as statistically significant or not statistically significant. This notion of significance has relevance to the interpretation of correlation coefficients. Imagine that a sample of subjects has been drawn from a population in which the correlation between two variables is r = .00. What is the chance of obtaining a correlation of exactly .00 in the sample? Not very high. After all, there will always be some sampling error in any study. When we compute a correlation coefficient, we will typically test it for statistical significance. When we do so, we are testing the null hypothesis that the correlation in the population is r = .00. If our result is significant, this means that the non-zero correlation we have obtained is probably not due to chance. Statistical significance does not mean that the correlation is really important or even large. When our sample size is large, even small correlations will usually be statistically significant. This is as it should be. After all, few bivariate relationships are exactly .00 in populations of interest. Unfortunately, news folks often equate statistical significance with scientific importance. They dont know better, but now you do!
24
Correlation Interpretation
Case 1
An investigator has administered a questionnaire to 1000 males which included measures of (1) frequency of pornographic viewership and (2) sexual objectification of females. He finds a correlation of r = .78 between exposure to pornography and the sexual objectification measure. Our investigator concludes that pornographic viewing leads men to think of women as sexual objects. Offer two other possible interpretations of this relationship.
Case 2
An investigator has found in a study of 250 children that those kids who watch Sesame Street on a regular basis have a better command of written English than do children who do not watch Sesame Street. She concludes that Sesame Street effectively teaches reading skills to children. Offer two other possible interpretations of this relationship.
Case 3
An investigator finds through her analysis of crime records that violent nighttime crimes are less likely to occur when there is a full moon than when the moon is not bright or when the night is cloudy. Our investigator concludes that a full moon has a sedating effect on potential criminals. Offer two other possible interpretations of this relationship.
Case 4
An investigator has measured the marijuana use of 1000 UCD students. He finds that the more marijuana students smoke, the poorer they perform in school. He concludes that marijuana smoking lowers student performance. Offer two other possible interpretations of this relationship.
25
Case 5
An investigator has found that people who watch a lot of television tend to think of the world as a scary and dangerous place. She concludes that television viewing leads people to exaggerate the seriousness of crime in our society. Offer two other possible interpretations of this relationship.
Case 6 An investigator observes that there is a strong positive correlation between ownership of a cellular phone and ownership of an expensive performance car; stated differently, people tend to own both or neither. He concludes that ownership of an expensive performance car leads people to purchase a cellular phone. Offer another interpretation of this relationship.
Case 7
A finance professor finds that individuals who watch the TV show Wall Street Review are much wealthier than individuals who do not watch this show. He concludes that watching Wall Street Review makes people wealthy. Offer two other interpretations of this relationship.
Case 8
A researcher discovers that patients who have uncommunicated desires for physician action (e.g., desires for tests, new medications, and the like) report less trust in the quality of the care they received from their physicians after their medical office visits. He concludes that the unexpressed desires erodes patients trust. Offer two other interpretations of this relationship.
26
ETHICS AND POLITICS IN SOCIAL RESEARCH
I.
Ethical Issues.
A. Voluntary Participation.
B.
No Harm to Participants.
C.
Anonymity and Confidentiality.
D.
Deception.
E.
Analysis and Reporting of Results.
F.
Role of the Institutional Review Board.
G.
Professional Codes of Ethics.
II.
Politics of Social Research.
27
RESEARCH DESIGN
I. Research Design.
II.
First Things First: Identify the Purpose of Your Investigation.

A. Exploratory: Whats Going On Here? (Problem Definition)
B.
Descriptive Research: What Is...
C.
Explanatory Research: Why...
III.
Specify the Unit of Analysis and the Unit of Observation.

A. Definitions.
1.
Unit of Analysis: the what or who that is being studied.
2.
Unit of Observation: the unit for which data are collected. (See Handout On Page 33.)
B.
Common Types of Units of Analysis Found in Communication Research.
1.
The individual.
2.
Dyads.
3.
Social Collectives: Groups, Organizations, Communities, etc.
28
4.
Artifacts.
a.
Physical artifacts.
b.
Social episodes and interactions.
C.
Units of Analysis and the Ecological Fallacy.
D.
Units of Analysis and Reductionism.
IV.
The Time Dimension.

A. Why Time Considerations Are Important.
1.
Time and Causality Inferences.
2.
Developmental Questions.
3.
Generalizeability of Findings over Time.
B.
Management of Time in Research Design.
1.
Cross-Sectional Research.
2.
Longitudinal Research.
a.
Trend Studies: Investigations of changes in a population over time.
29
b.
Cohort Studies: Investigations of changes in specific subpopulations (cohorts) over time.
c.
Panel Studies: Investigations of changes in a specific sample of individuals over time. (Same individuals serve as subjects on all occasions.)
3.
Approximating Longitudinal Studies With Cross-Sectional Data.
a.
Simple Logic (Assumed Order of Events).
b.
Age Comparisons.
c.
Recollections.
4.
Exercises. (See Handout On Page 35.)
V.
The Case For Designing Multi-Method Research.

A. Multi-Method Research Defined.
B.
Logic of Multi-Method Research.
1.
Limitations are Inherent in Any Method.
2.
Confidence in Findings.
Webb (1966): When a hypothesis can survive the confrontation of a series of complementary methods of testing, it contains a degree of validity unattainable by one tested within the more constricted framework of a single method.
C.
Neutralization of Weaknesses.
30
Denzin (1970): The flaws of one method are often the strengths of another, and by combining methods, observers can achieve the best of each while overcoming their unique deficiencies.
D.
Expanded Scope.
E.
Four Styles of Research. (See Handout On Page 37.)
F.
Four Potential Goals of Inquiry. (See Handout On Page 38.)
1.
Naturalism: Studying communication in a way which preserves its natural integrity.
2.
Nonreactivity: Studying communication in a way which does not affect the behavior of the people studied.
3.
Generalizeability: Studying communication in a way which allows for the application of findings beyond the individuals investigated.
4.
Causal Explication: Studying communication in a way which allows for the explication of cause and effect.
G.
Tradeoffs in Research.
H.
Combining Methods.
1.
Concurrent Multi-Method Research.
2.
Iterative Multi-Method Research.
31
3.
Multi-Method Replication of Others Work.
I.
Multi-Method Research as One Aspect of Replication (revisit handout on page 8).
32
Units of Analysis and Units of Observation

Unit of Analysis? Unit of Observation?
Study Description
An investigator wishes to test the hypothesis that there is a negative correlation between marriage length and frequency of argument; he expects to find fewer arguments in older marriages. He collects estimates of argument frequency from each partner in 200 marriages and averages these estimates to create an argument frequency score, which is then correlated with marriage length. An investigator wishes to test the hypothesis that patients in HMO health plans make fewer requests of their physicians for treatments and procedures than do patients in traditional fee-for-service health plans. She tape records the conversations of consenting patients at a large clinic which sees both kinds of patients. She then compares the number of requests made by HMO patients with the number of requests made by fee-for-service patients, controlling for other factors (e.g., gender, health status, age). An investigator is interested in testing the hypothesis that the Wall Street Journal is a more conservative paper than the New York Times. One hundred editorials have been randomly selected from each paper from the population of all editorials published in the last calendar year. Each editorial is then rated on a liberal-conservative scale which has been well validated. The ratings for each paper are averaged to derive an overall measure of political ideology for each paper. A conservative television watchdog group is outraged by the language used during prime time on the Fox and CW television networks. They decide to do a content analysis to reveal just how naughty the language is on these two networks, relative to the other broadcast networks (ABC, NBC, CBS). These citizens count the number of swear words used on every prime time program broadcast by the six networks over a period of two weeks. A bad words per minute of programming score is computed for each program. An overall network naughtiness score is then computed for ABC, NBC, CBS, CW, and FOX by averaging across all of the programs aired by each network (after weighing for program length). The group then compares the network naughtiness scores of the four nicer networks with the scores of the two evil networks to test its perceptions.
33
Study Description
A music critic is angry because his favorite rock group, the Rolling Stones, is always used as the point of comparison for a new band, the Stationary Pebbles. Our peeved critic intends to show that the imagery used by the Rolling Stones is completely different than the imagery used by the Stationary Pebbles. He does so by using computer-assisted content analysis. Specifically, every song ever recorded by both groups is run through the computer designed to analyze its lyrics on a number of dimensions. The values obtained are then averaged for each band to make a comparison possible. An interpersonal communication researcher hypothesizes that nonverbally sensitive individuals make better police officers. She obtains the performance ratings for 150 cops and also has these individuals complete the Profile of Nonverbal Sensitivity (PONS) Test. She then classifies the officers into two groups: the nonverbally sensitive and the nonverbally insensitive. These two groups are then compared on the performance ratings to determine if the more sensitive officers are better at their jobs than the less sensitive officers.
Unit of Analysis?
Unit of Observation?
34
The Time Dimension in Research: An Exercise

1. An investigator knows from past research that adolescent drug use is inversely related to school performance. The direction of causality is not clear, however. Do children perform poorly in school as a result of drug use, or do they turn to the drug scene in an effort to cope with difficulties at school? The investigator decides to answer this question by following a group of students from grade five through grade twelve. These students will be interviewed about their use of controlled substances twice a year for the eight-year duration of the investigation. Data obtained from these interviews will be used in conjunction with school records on performance. What kind of research design will the investigator be using? a. b. c. d. e. 2. Cross-Sectional Research Longitudinal: Trend Study Longitudinal: Cohort Study Longitudinal: Panel Study Attempt to Approximate Longitudinal Research With Cross-Sectional Data
An interpersonal communication researcher believes that dysfunctional family communication patterns can be passed from generation to generation. To test this possibility, he intends to measure the degree to which a sample of 1000 junior high school students are satisfied with the communication they have with their parents. He will also ask the parents of these students to report the extent to which they were satisfied as adolescents with the communication they had with their parents (the students grandparents). He will thus be able to correlate family communication satisfaction across two generations. What kind of research design will the investigator be using? a. b. c. d. e. Cross-Sectional Research Longitudinal: Trend Study Longitudinal: Cohort Study Longitudinal: Panel Study Attempt to Approximate Longitudinal Research With Cross-Sectional Data
3.
A mass communication researcher believes that exposure to soap operas leads people to develop distorted views of reality. For example, he believes that people who watch a lot of soap operas, in comparison to people who do not, will overestimate the proportion of married couples who engage in extra marital affairs and the proportion of individuals who are doctors and lawyers. This researcher will attempt to examine his hypotheses by conducting a nationwide survey in which respondents complete a measure of soap opera viewing and make estimates such as the ones described above. Soap opera viewing will then be correlated with these estimates. What kind of research design will the investigator be using? a. b. c. d. e. Cross-Sectional Research Longitudinal: Trend Study Longitudinal: Cohort Study Longitudinal: Panel Study Attempt to Approximate Longitudinal Research With Cross-Sectional Data
35
4.
A political communication scholar is interested in the long-term effects of coming of age during the Watergate era on cynicism toward government. In 1974, he identified through a random sampling procedure a group of high school students (aged 15-18) for study and administered a questionnaire to them. Using the same sampling procedure, he generated a second sample of respondents in 1984 from this age group (now aged 25-28). These respondents completed the same questionnaire administered ten years earlier to the first sample. In 1994, he generated a third sample of individuals aged 35-38 and administered his questionnaire to these folks. In the year 2004, he came up with a fourth sample of people aged 45-48 for study. In the year 2014, he will come up with a fifth sample of people aged 55-58. What kind of research design is this investigator using? a. b. c. d. e. Cross-Sectional Research Longitudinal: Trend Study Longitudinal: Cohort Study Longitudinal: Panel Study Attempt to Approximate Longitudinal Research With Cross-Sectional Data
5.
An interpersonal communication researcher believes that our society has become more and more impersonal over the past two decades, which should reflect itself in increasing levels of loneliness. She learns that a major polling firm has made a practice of asking nationwide samples of Americans to report their levels of loneliness in its annual Americas Quality of Life Survey. The investigator obtains the mean loneliness score for each survey conducted over the past two decades to test her perceptions. What kind of design is she using? a. b. c. d. e. Cross-Sectional Research Longitudinal: Trend Study Longitudinal: Cohort Study Longitudinal: Panel Study Attempt to Approximate Longitudinal Research With Cross-Sectional Data
6.
A marital communication researcher is interested in the phases of marriage. He asks a sample of married individuals to write a history of their relationships. Each is instructed to provide a chronological account of the entire relationship, from first meeting to the present. What kind of design is this investigator using? a. b. c. d. e. Cross-Sectional Research Longitudinal: Trend Study Longitudinal: Cohort Study Longitudinal: Panel Study Attempt to Approximate Longitudinal Research With Cross-Sectional Data
36
Four Styles of Research

Research Style
Fieldwork What They Do
Observe people and events in their natural settings.
Unobtrusive Measurement
Make observations in a manner which does not affect the behavior of the people being observed.
Survey Research
Employ sophisticated sampling procedures to generate representative samples of the populations of interest. Study sample to understand population. Sampling techniques, standardized interviewing (face-toface and by phone), self-administered questionnaires. Analyses are typically quantitative.
Experimental Research
Manipulate (independent) variables to assess their causal effects on other (dependent) variables.
Primary Tools
Observation, participant observation, ethnographic interviewing. Focus groups may also be used. Analyses are typically qualitative.
Content analyses of archives, covert observation and coding of behavior, examination of physical traces (accretions and erosions). Analyses are typically quantitative.
Experimental manipulations within carefully constructed situations. Careful control of extraneous factors. Analyses are quantitative.
Priority
Naturalism Study people in a way which respects the situated nature of interaction.
Nonreactivity Study people without affecting their behavior.
Generalizeability Make inferences about many people on the basis of a substantially smaller sample.
Causal Explication Draw conclusions about cause-andeffect relationships.
37
Four Styles of Research and Four Investigative Values
Research Style
Fieldwork Naturalism
This is the strength of ethnographic fieldwork.
Unobtrusive observation can be very naturalistic, provided that contextual information is respected by the coding procedure. The meaning of naturalism is unclear when applied to content analyses of texts.
Survey Research
Not at all naturalistic. Survey researchers typically employ standardized procedures for all respondents. Information obtained is often extracted from the larger social context.
Laboratory experimental research is not at all naturalistic. Field experiments can be very naturalistic. Indeed, subjects in field experiments may not even know that they are being studied.
Non-reactivity
Reactivity can be very high. Informants are usually aware of the investigators presence and purposes. Investigators behavior may also affect the informants behaviors. The fieldworker attempts to develop a relationship with the informants which encourages honest, accurate reporting. Covert observations may be made to assess behavior when reactivity concerns are considerable.
By definition, unobtrusive measurement research is nonreactive.
The interrogative nature of survey research is unlike everyday interactions. Although open-ended questions can be asked, most typically, people are asked to report their thoughts and feelings using investigator-defined categories.
Reactivity potential is high. People who know they are in an experiment will develop hypotheses about the purpose of the study. These guesses may affect their behavior. Deception may be used to thwart subjects efforts to understand the purposes of the experiment, but the deception itself may lead to suspicions which affect the subjects behaviors. Reactivity is less of a concern in the naturalistic field experiment.
38
Research Style
Fieldwork Generalizeability
Fieldwork is typically limited to the understanding of a particular cultural group. The goal of generalizeability violates the theoretical assumptions of ethnographic fieldworkers.
In theory, there is no reason why unobtrusive measurement cannot be applied to artifacts which represent a large and diverse population. In practice, this objective may be exceedingly difficult to achieve.
Survey Research
This is a primary goal and the strength of survey research. The overriding objective is to obtain a sample which is representative of the entire population to which one wishes to generalize.
In theory, generalizeability is possible. Once could, for instance, conduct an experiment using subjects drawn from a nationwide probability sample. In practice, the cost of doing so is prohibitive. Experimental researchers are often accused of relying too heavily upon convenience samples (such as college undergraduates), which restricts generalizeability.
Causal Explication
Ethnographic fieldworkers do not typically view the world in terms of cause and effect. Causal explication is neither a goal nor a value of ethnographers.
One could manipulate a variable of interest and examine its effects using unobtrusive dependent measures. Overall, unobtrusive measurement does not lend itself well to causal explication. The need for control will often clash with the goal of nonreactivity.
Survey researchers are usually more interested in descriptive research questions than explanatory ones. As such, causal explication is often not a goal. Longitudinal survey research could, however, be employed to provide a basis for inferences about cause-andeffect.
This is the strength of the experiment. By manipulating one or more variables in a carefully controlled setting, the investigator can examine how the manipulation affects other variables of interest. In essence, every experiment is a mini longitudinal study.
39
CONCEPTUALIZATION, OPERATIONALIZATION, AND MEASUREMENT

I. Conceptualization.
A. Concepts.
1.
Conception: Those images we have in our head.
2.
Concept: The term we use to represent the image.
3.
Three Classes of Things.
a.
Directly Observables.
b.
Indirect Observables.
c.
Constructs (Theoretical Creations).
B.
Conceptualization: Construct Definition.
1.
Definitions, Reality, and Reification.
2.
Kinds of Definitions.
a.
Real Definitions.
b.
Nominal Definitions.
c.
Operational Definitions.
3.
Dimensions (see handout on page 44).
40
4.
Indicators.
C.
Summary.
Conceptualization
II.
Fundamental Issues in Operationalization.

A. Levels of Measurement. (See handout on page 48.)
Nominal Definition
Operational Definition
Measurement
B.
Structuring of Responses.
1.
Response Choices.
a.
Agreement.
b.
Evaluation (Good-Bad).
c.
Frequency (Never-Often).
d.
Proximity (Identical-Different).
2.
Representing the Extremes.
3.
Variations Between the Extremes.
C.
Single or Multiple Indicators.
41
III.
Scale Construction.
A. Some Common Scaling Procedures.
1.
Thurstone Scaling (see handout on page 51).
2.
Likert Scaling (see handout on page 55).
3.
Semantic Differentials (see handout on page 77).
4.
Bipolar Scales (see handout on page 80).
5.
Guttman Scaling (see handout on page 84).
6.
Direct Magnitude Estimation (see handout on page 86).
B.
Indexes Versus Scales.
IV.
Measurement Quality.
A. Reliability of Measurement.
1.
The Meaning of Reliability.
2.
Assessment Procedures.
a.
Test-Retest Method.
b.
Parallel (Alternative) Forms Reliability.
42
c.
Internal Consistency.
B.
Validity of Measurement.
1.
The Meaning of Validity.
2.
Types of Validity.
a.
Face Validity: Is Our Measure Consistent With Common Agreements?
b.
(Criterion-Related) Predictive Validity: Does Our Measure Predict Some Future Event That Is logically Related to the Construct the Measure Is Intended to Tap Into?
c.
Construct Validity: Is Our Measure Associated with Other Constructs to Which it Should Be Logically Related?
d.
Content Validity: Does Our Measures Capture the Full Range of Meaning of the Concept Is Purports to Assess?
V.
Typologies (see handout on page 90).
43
Dimensions of Concepts: Some Examples

Love Hatfields Types*
Within Hatfields model, there are two types of love: passionate and companionate. Passionate Love. Passionate love is a state of intense longing for union with another. Reciprocated love (union with the other) is associated with fulfillment and ecstasy. Unrequited love (separation) with emptiness, anxiety, or despair. A state of profound physiological arousal. The components of passionate love can be classified into three groups: cognitive, emotional, and behavioral. Cognitive components of passionate love include intrusive thinking or preoccupation with the partner, idealization of the other or of the relationship, and desire to know the other and be known. Emotional components include attraction to other, especially sexual attraction; positive feelings when things go well; negative feelings when things go awry; longing for reciprocity; desire for complete and permanent union, and physiological arousal. Behavioral components include actions toward determining the others feelings, studying the other person, service to the other, and maintaining physical closeness. Passionate love is measured with the Passionate Love Scale. (This instrument will be made available to you later in the quarter.) Companionate Love. This kind of love is defined as the affection we feel for those with whom our lives are deeply entwined. This kind of love involves friendship, understanding, and concern for the welfare of the other. The components of companionate love are classified as cognitive, emotional and behavioral. Cognitive components include great knowledge of one another, due to the willingness to selfdisclosure intimately and honestly. The primary emotional component is deep caring. Because companionate love involves so much caring, partners have the ability to inflict great hurt upon each other. At the behavioral level, intimate feel comfortable when in close proximity, and perhaps touch. Hatfield believes that the first love scale developed, Rubins Love Scale, measures companionate love.
Elaine Hatfield. (1988). Passionate and companionate love. In R.J. Sternberg & M.L. Barnes (Eds.), The psychology of love (pp. 191-217). New Haven: Yale University Press.
44
Sternbergs Triangular Theory of Love*

Sternberg believes that love can be understood in terms of three components, which together can be viewed as forming the vertices of a triangle, as depicted in the figure below. Intimacy refers to close, connected, and bonded feelings in loving relationships. The signs of intimacy include desiring to promote the welfare of the loved one; experiencing happiness with the loved one; having high regard for the loved one; being able to count on the loved one in times of need; mutual understanding with the loved one; sharing ones self and ones possessions with the loved one; receiving emotional support from the loved one; giving emotional support to the loved one; having intimate communication with the loved one; and valuing the loved one in ones life. The passion component refers to the drives that lead to romance, physical attraction, sexual consummation, and the like in a loving relationship. The decision/commitment component consists of two aspects, one short term and one long term. The short-term one is the decision that one loves someone. The long-term aspect is the commitment to maintain that love. Sternberg and his colleagues have developed scales to measure each of the three love components explicated in his model. Derived Typology. Sternberg has advanced a typology of love based upon his triangular model: Single Component Types of Love
Intimacy (Intimacy Alone) Infatuation (Passion Alone) Empty Love (Commitment Alone)
Intimacy
Passion
Decision/ Commitment
Dual Component Types of Love

Romantic Love: Intimacy + Passion Companionate Love: Intimacy + Commitment Fatuous Love: Passion + Commitment
Consummate Love
Passion + Intimacy + Commitment
Robert J. Sternberg. (1986). A triangular theory of love. Psychological Bulletin, 93, 119-135.
45
Self-Disclosure Wheeless* proposes that self-disclosure is composed of five dimensions: Honesty: Amount: Positivity: The extent to which ones disclosures are truthful (versus dishonest). The amount/frequency of ones disclosures. The extent to which one s disclosures are positive (versus negative) about oneself. The degree to which ones self-disclosures are intentional revelations (versus unintentional accidents). The intimacy level of ones disclosures.
Intent:
Depth:
Scales have been developed to measure each component. Assertiveness Assertiveness is the ability to express ones feelings or wants, even when such expressions may lead to loss of reinforcement or punishment. Lorr and More believe that there are at least four dimensions of assertiveness: directiveness, independence, social assertiveness, and defense of rights and interests. Each class of assertiveness is conceptualized both as a skill and as a disposition to behave in specified ways to a specific class of stimuli. The basic framework is outlined in the table below:
Lawrence R. Wheeless. (1976). Self-disclosure and interpersonal solidarity: Measurement, validation, and relationships. Human Communication Research, 3, 47-61. Maurice Lorr and William W. More. (1980). Four dimensions of assertiveness. Multivariate Behavioral Research, 15, 127-138.
46
Assertiveness Dimension Directiveness
Stimulus Situation Problematic Interpersonal Situations Situations involving others that require direction, initiative, control and assumption of responsibility.
Response Pattern Taking leadership, direction, control and responsibility for action (versus refusing or evading action involving direction, control, or decision making). Expressing or defending opinions, resisting peer pressure or authority (versus yielding or conforming to pressure or authority). Introducing self; starting, stopping or continuing conversations with strangers or acquaintances; making friends (versus remaining in the background; maintaining silence; doing nothing). Defending rights, rejecting unjust demands (versus accepting violations of rights).
Independence
Conformity Pressure or Influence Group or individual pressure to conform or to comply.
Social Assertiveness
Social Situations Situations involving two or more people that call for initiation and conduct of social interactions.
Defense of Rights and Interests
Violation of Rights Situations involving violation of rights or unjust demands.
A well-validated measure designed to assess each dimension has been reported by Lorr and More.
47
Levels of Measurement
Nominal
Numbers are assigned to attributes of variables only as labels or names (nominal means in name only). The assignment of numbers is arbitrary. Example: The numbers on the uniforms of baseball players. Assignment is arbitrary and does not reflect amounts of some underlying property (e.g., a player does not get assigned number 10 because she has more of something than the player who is given number 8).
Ordinal
Assignment of numbers to attributes on the basis of a rank order of the amount of some property possessed by each attribute. Example: Your preference list of candidates for the Presidential election. Suppose you listed McCain first, Barr second, Obama third and Nader fourth. We know that you like McCain more than Barr, Barr more than Obama, and Obama more than Nader but we have no idea how much distance separates these candidates.
Interval
Assignment of numbers to variable attributes on the basis of the amount of some property possessed by each attribute AND in a manner which preserves information about differences/distances among those attributes. Example: The Fahrenheit temperature scale. The difference between 20 degrees and 40 degrees is identical to the distance between 80 and 100 degrees.
Ratio
Assignment of numbers to the attributes of a variable with a true zero point. Numbers are assigned on the basis of the amount of some property possessed by each attribute AND in a manner which preserves information about differences/distances among those attributes. Example: Kelvin temperature scale, for which the coldest possible temperature is given the value of 0. The difference between twenty and forty degrees is identical to the distance between 80 and 100 degrees. Furthermore, 20 degrees is one-half as hot as 40 degrees, and 80 degrees is four-fifths as hot as 100 degrees.
48
Characteristics of Nominal, Ordinal, Interval, and Ratio Scales
Characteristic
Mutually Exclusive and Exhaustive Attributes Attributes May Be RankOrdered Equality of Intervals True Zero Point
Nominal
Yes No No No
Ordinal
Yes Yes No No
Interval
Yes Yes Yes No
Ratio
Yes Yes Yes Yes
49
Levels of Measurement Exercise

Measurement Example
A researcher measures each subjects height in centimeters. A researcher assigns the number 1 to female subjects and the number 2 to male subjects. Army recruits are given an intelligence test which ranges from 50 to 150. Scores reflect the number of correct answers given on the test. A researcher computes the number of words spoken per minute by subjects in a study on regional dialects. Students at UCD are assigned a number to reflect their class standing as follows: 1=freshperson, 2=sophomore, 3=junior, 4=senior, 5=graduate student. A friend asks you to rate the physical attractiveness of several celebrities, using a 1-10 point scale. Elementary school students are given a 20-problem cognitive development test. Because parents dont like to be told that their child had a score of zero, the scores were transformed by subtracting the number of incorrect answers from 50. Thus, the range of test scores was 30-50. An investment newsletter ranks the top one hundred mutual funds on the basis of 5-year total return. A mass communication researcher has coders assign the number 1 to programs judged to be sitcoms, 2 to news shows, 3 to dramas, 4 to documentaries, 5 to soap operas, 6 to variety/talk shows, and 7 to any program not fitting categories 1 through 6. A marketing service measures the market share of the top ten soft drink beverages. A national testing agency assigns SAT scores to students on the basis of their performance on the SAT examination. A U.S. agency classifies people on the basis of their state of residence as follows: 1=Alabama, 2=Alaska, ..., 50=Wyoming. A researcher counts the number of different brands of prescription drugs advertised in leading magazines in each of the past ten years. A relationship researcher has classified romantic relationships as being in the (1) dating stage; (2) serious courtship stage; or (3) long-term commitment stage.
Nominal Nominal
Circle One...
Ordinal Interval Ratio Ordinal Interval Ratio
Nominal
Ordinal Interval Ratio
Nominal
Nominal
Nominal
Nominal
Nominal
Nominal
Nominal
Nominal
Nominal
Nominal
Nominal
50
How To Construct A Thurstone Scale

Step 1: Item Generation
Generate a list of statements relevant to the attitude object. For this example, we will construct statements pertaining to attitudes toward nuclear power. 1. 2. 3. 4. 5. 6. 7. 8. 9. 10. 11. 12. 13. 14. 15. I could live happily next to a nuclear power plant. Nuclear power will ultimately lead to the extinction of man. I would be delighted if my child were to become a nuclear engineer. Only nuclear power can solve our nations energy problems. Nuclear power plants cause cancer and other dreaded illnesses. Nuclear energy is clean and efficient. Nuclear power is safer than the burning of fossil fuels. Nuclear power is too costly to be of much value. Nuclear power is a great threat to our environment. People who oppose nuclear power dont know what they are talking about. Nuclear power is a more viable energy alternative than solar energy. The U.S. Government should do everything it can to encourage the expansion of our nuclear power capabilities. Nuclear energy is not cost-effective, relative to other energy sources. The Federal Government should prohibit the construction of additional nuclear power plants. The Federal Government should order the dismantling of all nuclear power reactors.
Note: In practice, investigators are encouraged to generate an initial item set of 100+.
Step 2: Item Sorting, Based on the Equal-Appearing Intervals Criterion

Have a large group of people (Thurstone suggested 100) place each item into one of the following categories, using the equal interval criteria: Most Negative: ___ 1 ___ 2 ___ 3 ___ 4 ___ 5 ___ 6 ___ 7 ___ 8 ___ 9 Most :Positive
Step 3: Compute Each Items Scale Value

Typically, the median is used; the mean may also be used.
51
Step 4: Compute a Measure of Agreement For Each Item

The standard deviation is usually computed for each item to assess the extent to which judges agree in their categorization decisions.
Items with large standard deviations should be discarded; their placement on the continuum of favorability cannot be reliably determined.
Step 5: Final Item Selection

Select a subset of about twenty items that represent the full negative-to-positive continuum. Only those items for which judges demonstrated high agreement should be selected.
Step 6: Scale Administration

Now you are ready to administer the final scale to a second group of people. Have each person check off those statements with which s/he agrees. His/her attitude score is simply the median scale value of the items s/he checks off.
52
Thurstone Scaling: An Example: Results of a Thurstone Scaling of Evaluation Adjectives*
Term/ Phrase Superior Fantastic Tremendous Superb Excellent Terrific Outstanding Exceptionally Good Extremely Good Wonderful Unusually Good Remarkably Good Delightful Very Good Fine Quite Good Good Moderately Good Pleasant Reasonably Good Nice Fairly Good Slightly Good Acceptable Average All Right
Housewives 20.12 (1.17) 20.12 (0.83) 19.84 (1.31) 19.80 (1.19) 19.40 (1.73) 19.00 (2.45) 18.96 (1.99) 18.56 (2.36) 18.44 (1.61) 17.32 (2.30) 17.08 (2.43) 16.68 (2.19) 16.92 (1.85) 15.44 (2.77) 14.80 (2.12) 14.44 (2.76) 14.32 (2.08) 13.44 (2.23) 13.44 (2.06) 12.92 (2.93) 12.56 (2.14) 11.96 (2.42) 11.84 (2.19) 11.12 (2.59) 10.84 (1.55) 10.76 (1.42)
Executives 18.22 (2.82) 18.69 (3.68) 18.67 (2.01) 19.00 (2.10) 18.72 (2.25) 18.81 (2.19) 19.31 (2.01) 17.03 (4.12) 17.33 (3.09) 17.97 (2.35) 16.47 (2.99) 17.44 (2.63) 16.61 (2.45) 16.83 (2.52) 15.61 (2.72) 13.69 (2.90) 13.81 (3.25) 11.42 (2.99) 13.61 (2.43) 11.89 (3.37) 11.44 (2.79) 11.94 (3.84) 10.25 (3.14) 10.67 (3.34) 9.97 (2.34) 10.17 (3.28)
Graduate Business Students 19.45 (1.78) 20.15 (1.37) 19.70 (1.18) 19.40 (1.95) 19.58 (1.97) 19.08 (1.61) 19.58 (1.26) 17.68 (2.26) 17.45 (2.26) 18.45 (1.99) 16.78 (2.12) 17.20 (2.32) 16.60 (2.24) 17.00 (2.18) 14.60 (3.00) 15.70 (2.08) 14.78 (2.27) 12.60 (2.55) 13.48 (2.33) 13.85 (2.19) 12.70 (2.65) 12.40 (2.24) 11.88 (2.62) 10.72 (1.96) 10.82 (1.43) 10.95 (2.15)
Undergraduate Business Students 18.96 (1.67) 19.20 (1.87) 18.92 (1.75) 19.60 (2.42) 19.44 (1.42) 18.60 (1.63) 19.40 (1.35) 17.88 (1.72) 18.00 (1.50) 17.52 (2.10) 16.20 (1.80) 17.08 (1.89) 16.76 (1.51) 16.80 (1.44) 15.32 (2.21) 15.60 (1.94) 14.56 (1.96) 13.04 (1.43) 14.48 (2.14) 14.20 (1.71) 13.72 (1.77) 13.12 (2.11) 12.32 (1.52) 11.40 (2.02) 10.76 (1.05) 11.40 (1.26)
Taken From: J.H. Myers & W.G. Warner. (1968). Semantic properties of selected evaluation adjectives. Journal of Marketing Research, 5, 409-418.
53
O.K. So-so Neutral Fair Mediocre Not Very Good Moderately Poor Reasonably Poor Slightly Poor Poor Fairly Poor Unpleasant Quite Poor Bad Very Bad Unusually Poor Very Poor Remarkably Poor Unacceptable Exceptionally Poor Extremely Poor Awful Terrible Horrible
10.28 (1.67) 10.08 (1.87) 9.80 (1.50) 9.52 (2.06) 9.44 (1.80) 6.72 (2.82) 6.44 (1.64) 6.32 (2.46) 5.92 (1.96) 5.76 (2.09) 5.64 (1.68) 5.04 (2.82) 4.80 (1.44) 3.88 (2.19) 3.20 (2.10) 3.20 (1.44) 3.12 (1.17) 2.88 (1.74) 2.64 (2.04) 2.52 (1.19) 2.08 (1.19) 1.92 (1.50) 1.76 (0.77) 1.48 (0.87)
10.11 (2.48) 8.81 (2.75) 9.56 (1.90) 9.56 (3.67) 8.11 (2.74) 6.47 (2.41) 6.83 (3.50) 6.31 (2.19) 7.19 (2.36) 5.19 (2.86) 6.67 (2.81) 4.36 (3.02) 4.56 (2.58) 3.67 (2.54) 2.22 (2.34) 3.08 (1.79) 3.14 (2.39) 2.75 (1.70) 3.53 (3.42) 3.19 (2.23) 2.83 (2.14) 2.25 (1.46) 2.22 (2.63) 2.22 (2.51)
10.58 (2.12) 9.52 (1.47) 10.18 (2.01) 9.20 (2.05) 8.90 (2.36) 6.40 (2.05) 6.28 (1.87) 5.82 (1.74) 7.25 (2.00) 4.72 (2.51) 6.25 (1.63) 4.68 (2.63) 3.62 (1.67) 3.85 (1.81) 2.70 (2.16) 3.48 (1.68) 3.35 (1.99) 3.12 (1.70) 3.98 (2.79) 3.22 (1.82) 3.10 (1.72) 2.48 (1.72) 2.05 (1.43) 1.62 (1.15)
11.28 (1.21) 10.36 (1.15) 10.52 (1.16) 10.24 (2.20) 9.36 (2.20) 7.92 (2.02) 7.24 (1.59) 6.16 (1.57) 8.48 (1.83) 5.24 (1.51) 6.72 (1.74) 5.52 (2.06) 4.56 (1.78) 4.24 (1.88) 3.08 (1.50) 4.16 (1.57) 3.68 (1.52) 3.92 (1.68) 5.56 (3.06) 3.52 (1.96) 3.24 (1.76) 2.68 (1.86) 1.88 (1.24) 2.00 (1.35)
Note: Numbers in parentheses are standard deviations.
54
How To Construct A Likert (Summated) Rating Scaling

Step 1: Conceptualization
Identify the content domain of the construct you wish to measure, including its dimensions and indicators.
Step 2: Item Generation

Write items which reflect the full domain of the construct to be assessed. It is useful to think of these items as being a sample of a universe of items which could conceivably be used to assess the construct. It is important to include both positively- and negatively-worded statements to prevent a response set that is, a tendency for the subject to check off the same response category without much thought or reflection. For example, if we wanted to construct a measure of patients satisfaction with their physician, we would include positive items such as: I am satisfied with the care my doctor provides. My doctor is a very skilled professional. Negative items would also have to be included: I have considered changing doctors. My doctor is uncaring.
Step 3: Administration of Questionnaire

The items are presented, typically in questionnaire form, to the group whose attitudes you wish to assess. How large must this group be? There are many rules of thumb; one suggests that you should use at least 15 subjects per item, but in no case have fewer than 100 individuals. The relationship between number of items and necessary sample size is not linear. As the number of items increases, the number of individuals needed per item decreases. Typically, a five-category or seven-category response format is used:
Strongly Agree Agree Unsure Disagree Strongly Disagree
Very Strongly Agree
Strongly Agree
Agree
Unsure
Disagree
Strongly Disagree
Very Strongly Disagree
55
Step 4: Reverse Scoring of Items

The meaning of an endorsement of a particular category for example, strongly agree, depends on whether or not the statement is favorable or unfavorable toward the attitude object. As such, the value assigned to a particular response depends on the direction of the item:
Strongly Agree Positive Statement (e.g., The President is managing the economy effectively.) Negative Statement (e.g., The President is managing the economy ineffectively.) 5 Agree 4 Unsure 3 Disagree 2 Strongly Disagree 1
Step 5: Item Selection Analyses

Items are selected for the final scale after careful statistical analyses of respondents patterns of results. Initially, the matrix of correlations among all items will be calculated, and this matrix will be subjected to an item analysis to determine the reliability of the scale. Standard computer packages will provide a wealth of information about each item, including its correlation with the other items (referred to as the corrected item-total correlation) and the Cronbach alpha reliability of the scale if the item were not included. A good item is one which is strongly correlated with the other items (after all, each item is assumed to be assessing a slightly different aspect of the same construct); its deletion from the scale reduces the overall alpha value. It is often common to analyze ones items through a statistical procedure called factor analysis to determine if they in fact assess a single component (factor). If a subset of your items are more strongly correlated with each other than with the remaining items, you may have a twofactor scale. That is, you may actually have two scales, not one. (For example, the original version of the self-monitoring scale has been shown to be composed of perhaps three relatively independent subscales, which have been labeled acting, other-directedness, and extraversion.)
Step 6: Final Scale Development

Once you have analyzed the reliability and factor structure of your items, you will hopefully be able to select a subset of items which reliably tape your unidimensional construct. The fact that scales in the social and behavioral sciences are always in a state of constant refinement does not detract from the value of the present version of a particular instrument.
56
Summated Rating Scales: Some Examples

The Affinity-Seeking Instrument (ASI)*
Directions. Below you will find several statements that concern how you interact with others. For each statement, please tell us how accurately the behavior or feeling describes you by indicating whether you agree or disagree with it. More specifically, indicate (by circling) whether you strong agree with the statement, agree, are unsure, disagree, or strongly disagree.
1. I seldom know what to say or do to get others to like me. 2. If I put my mind to it, I could get anyone to like me. 3. I have trouble building rapport with others. 4. I have difficulty getting others to want to spend time with me. 5. If I want someone to like me, I can usually create positive feelings between us. 6. I feel tense when I am with people I do not know well. 7. I am good at getting others to want to hang around with me. 8. I do not seem to know what to say and do to make myself popular with others. 9. When necessary, I can put on an act to get important people to approve of me.
Strongly Agree
Agree
Unsure
Disagree
Strongly Disagree
Strongly Agree
Agree
Unsure
Disagree
Strongly Disagree
Strongly Agree
Agree
Unsure
Disagree
Strongly Disagree
Strongly Agree
Agree
Unsure
Disagree
Strongly Disagree
Strongly Agree
Agree
Unsure
Disagree
Strongly Disagree
Strongly Agree
Agree
Unsure
Disagree
Strongly Disagree
Strongly Agree
Agree
Unsure
Disagree
Strongly Disagree
Strongly Agree
Agree
Unsure
Disagree
Strongly Disagree
Strongly Agree
Agree
Unsure
Disagree
Strongly Disagree
Source: R.A. Bell, S.W. Tremblay, & N.L. Buerkel-Rothfuss. (1987). Interpersonal attraction as a communication accomplishment: Development of a measure of affinity-seeking competence. WJSC, 51, 1-18.
57
10. I am not very good at putting on a show to impress others. 11. I am very good at playing roles to draw people to me. 12. I can present myself as more likeable than I really am. 13. I can put on excellent social performances to get others to approve of me.
Strongly Agree
Agree
Unsure
Disagree
Strongly Disagree
Strongly Agree
Agree
Unsure
Disagree
Strongly Disagree
Strongly Agree
Agree
Unsure
Disagree
Strongly Disagree
Strongly Agree
Agree
Unsure
Disagree
Strongly Disagree
Note: Items 1 through 8 compose the Affinity-Seeking Competence scale. The Strategic Performance scale is composed of items 9 through 13 Reverse scoring must be applied to items 2, 5, 7, 9, 11, 12, and 13.
58
Assessing Alternative Responses To Conflicts in Friendships*

Direction: Please think of a situation that took place sometime during the past five weeks in which a friend said or did something that was upsetting to you. By friend we mean a person to whom you feel close who is not a relative or romantic partner. Below you will find descriptions of actions that people sometimes take when they have conflicts with a friend. For each statement, please indicate whether you strong agree, agree, disagree, or strongly disagree that it describes how you responded to this particular conflict. (Exit Items)
Please Circle One Response For Each Statement
1. 2. 3. 4. 5. 6.
I told my friend that I no longer wanted to be friends with him/her. I told my friend that I was thinking seriously about ending the relationship. I permanently cut off all contact with my friend. I told other people that my friend and I had gone our separate ways. I walked away from the friendship. I told my friend that we were not suited to each other and should give up trying to be friends. I let my friend know that I did not want him/her to ever talk to me again. I threatened to end our relationship if my friend did not change his/her ways.
Strongly Agree Strongly Agree Strongly Agree Strongly Agree Strongly Agree Strongly Agree
Agree
Disagree
Strongly Disagree Strongly Disagree Strongly Disagree Strongly Disagree Strongly Disagree Strongly Disagree
Agree
Disagree
Agree
Disagree
Agree
Disagree
Agree Agree
Disagree Disagree
7. 8.
Strongly Agree Strongly Agree
Agree
Disagree
Strongly Disagree Strongly Disagree
Agree
Disagree
Adapted From: J.G. Healey & R.A. Bell. (1990). Assessing alternative responses to conflicts in friendship. In D.D. Cahn (Ed.), Intimates in conflict: A communication perspective (pp. 25-48). Hillsdale, NJ: Lawrence Erlbaum.
59
(Voice Items)
1. 2. 3. 4.
We solved our problems by talking things over in a calm and polite manner. We were each open to the others suggestions for resolving problems. We negotiated a solution to our conflict. I told my friend that there were problems in our friendship and suggested that we work them out. We talked openly and honestly about our differences in hopes of restoring the friendship. I made sure my friend realized that resolving our differences was important because I valued the friendship. We worked together to settle our differences. I told my friend what was bothering me and asked for his/her opinions on the matter.
Strongly Agree Strongly Agree Strongly Agree Strongly Agree
Agree
Disagree
Strongly Disagree Strongly Disagree Strongly Disagree Strongly Disagree
Agree
Disagree
Agree Agree
Disagree Disagree
5. 6.
Agree
Disagree
Agree
Disagree
7. 8.
Agree Agree
Disagree Disagree
(Loyalty Items) 1. 2. 3. 4. 5. 6. 7. 8. I said nothing and waited for things to get better. I did not take things personally and just kept quiet so that things might improve. I hoped that the situation would resolve itself. I tried to pretend that the incident never occurred out of concern for the friendship. I allowed things to cool off rather than taking any action. I did nothing and remained loyal to the friendship. I said nothing and dealt with the situation by adopting a strategy of forgive and forget. I kept my concerns to myself to give my friend a chance to reconsider his/her actions.
Strongly Agree Strongly Agree Strongly Agree Strongly Agree Strongly Agree Strongly Agree Strongly Agree Strongly Agree Agree Disagree Strongly Disagree Strongly Disagree Strongly Disagree Strongly Disagree Strongly Disagree Strongly Disagree Strongly Disagree Strongly Disagree
Agree
Disagree
Agree Agree
Disagree Disagree
Agree
Disagree
Agree
Disagree
Agree
Disagree
Agree
Disagree
60
(Neglect Items)
1. 2. 3. 4. 5. 6. 7. 8.
I allowed my friendship to deteriorate. When my friend tried to discuss our problems, I refused to cooperate. I said nasty things about my friend to other people. I ignored my friend for awhile. I said and did things out of anger to make my friend feel bad. While in the presence of my friend, I acted as though he/she did not even exist. I mentioned bad things that my friend did in the past to hurt and embarrass him/her. Out of anger, I said things to damage the reputation of my friend.
Strongly Agree Strongly Agree Strongly Agree Strongly Agree Strongly Agree Strongly Agree Strongly Agree Strongly Agree
Agree Agree
Disagree Disagree
Strongly Disagree Strongly Disagree Strongly Disagree Strongly Disagree Strongly Disagree Strongly Disagree Strongly Disagree Strongly Disagree
Agree
Disagree
Agree Agree
Disagree Disagree
Agree
Disagree
Agree
Disagree
Agree
Disagree
Scoring Procedure: strongly agree = 4; agree = 3; disagree = 2; strongly disagree = 1.
61
Public and Private Self-Consciousness Scales*

Directions. Below you will find several statements that concern thoughts and feelings you may experience. For each statement, please indicate how accurately the behavior or feeling describes you by indicating whether you agree or disagree with it. More specifically, indicate (by circling) whether you strong agree with the statement, agree somewhat, are unsure, disagree somewhat, or strongly disagree. Private Self-Consciousness Items 1. I reflect about myself a lot. 2. I am generally attentive to my inner feelings. 3. I am always trying to figure myself out. 4. I am constantly examining my motives. 5. I am alert to changes in my mood. 6. I tend to scrutinize myself. 7. Generally, I am aware of myself. 8. I am aware of the way my mind works when I work through a problem. 9. I am often the subject of my own fantasies. 10. I sometimes have the feeling that I am off somewhere watching myself.
Strongly Agree Strongly Agree Agree Somewhat Agree Somewhat Unsure Disagree Somewhat Disagree Somewhat Strongly Disagree Strongly Disagree
Unsure
Strongly Agree
Agree Somewhat
Unsure
Disagree Somewhat
Strongly Disagree
Strongly Agree
Agree Somewhat
Unsure
Disagree Somewhat
Strongly Disagree
Strongly Agree
Agree Somewhat
Unsure
Disagree Somewhat
Strongly Disagree
Agree Somewhat Agree Somewhat
Unsure
Disagree Somewhat Disagree Somewhat
Unsure
Strongly Agree
Agree Somewhat
Unsure
Disagree Somewhat
Strongly Disagree
Strongly Agree
Agree Somewhat
Unsure
Disagree Somewhat
Strongly Disagree
Strongly Agree
Agree Somewhat
Unsure
Disagree Somewhat
Strongly Disagree
Source: A. Fenigstein, M.F. Scheier, & A. H. Buss. (1975). Public and private self-consciousness: Assessment and theory. Journal of Consulting and Clinical Psychology, 43, 522-527
62
Public Self-Consciousness Items

11. I am concerned about what other people think of me. 12. I usually worry about making a good impression. 13. I am concerned about the way I present myself. 14. I am self-conscious about the way I look. 15. I am usually aware of my appearance. 16. One of the last things I do before leaving my house is look in the mirror. 17. I am concerned about my style of doing things.
Strongly Agree Agree Somewhat Unsure Disagree Somewhat Strongly Disagree
Strongly Agree
Agree Somewhat
Unsure
Disagree Somewhat
Strongly Disagree
Strongly Agree
Agree Somewhat
Unsure
Disagree Somewhat
Strongly Disagree
Strongly Agree
Agree Somewhat
Unsure
Disagree Somewhat
Strongly Disagree
Strongly Agree
Agree Somewhat
Unsure
Disagree Somewhat
Strongly Disagree
Strongly Agree
Agree Somewhat
Unsure
Disagree Somewhat
Strongly Disagree
Strongly Agree
Agree Somewhat
Unsure
Disagree Somewhat
Strongly Disagree
63
Rosenbergs Self-Esteem Scale*

Directions. Below you will find several statements that concern feelings you may have about your own sense of worth. For each statement, please indicate how accurately the statement reflects your true feelings by indicating how strongly you agree or disagree with it. 1. I have a number of good qualities.
Strongly Agree Tend To Agree Tend To Disagree Strongly Disagree
2.
I wish I could have more respect for myself. I feel I am a person of worth, at least on an equal plane with others. I feel I do not have much to be proud of.
Strongly Agree
Tend To Agree
Tend To Disagree
Strongly Disagree
3.
Strongly Agree
Tend To Agree
Tend To Disagree
Strongly Disagree
4.
Strongly Agree
Tend To Agree
Tend To Disagree
Strongly Disagree
5.
I take a positive attitude toward myself.
Strongly Agree
Tend To Agree
Tend To Disagree
Strongly Disagree
6.
I certainly feel useless at times.
Strongly Agree
Tend To Agree
Tend To Disagree
Strongly Disagree
7.
All in all, I am inclined to feel that I am a failure. I am able to do things as well as most other people. At times I think I am no good at all.
Strongly Agree
Tend To Agree
Tend To Disagree
Strongly Disagree
8.
Strongly Agree
Tend To Agree
Tend To Disagree
Strongly Disagree
9.
Strongly Agree
Tend To Agree
Tend To Disagree
Strongly Disagree
10.
On the whole, I am satisfied with myself.
Strongly Agree
Tend To Agree
Tend To Disagree
Strongly Disagree
Note: Reverse scoring should be applied to items 1, 3, 5, 8, and 10.
Source: M. Rosenberg. (1965). Society and the adolescent self-image. Princeton: Princeton University.
64
Self-Disclosure Scales*
Intended Disclosure Scale

1. When I wish, my self-disclosures are always accurate reflections of who I really am. 2. When I express my personal feelings, I am always aware of what I am doing and saying. 3. When I reveal my feelings about myself, I consciously intend to do so. 4. When I am self-disclosing, I am consciously aware of what I am revealing.
Strongly Agree Strongly Agree Strongly Agree Strongly Agree Agree Agree Agree Agree Disagree Disagree Disagree Disagree Strongly Disagree Strongly Disagree Strongly Disagree Strongly Disagree
Amount
5. I do not often talk about myself. 6. My statements of my feelings are usually brief. 7. My conversation lasts the least time when I am discussing myself. 8. I often talk about myself. 9. I often discuss my feelings about myself. 10. Only infrequently do I express my personal beliefs and opinions. 11. I usually talk about myself for fairly long periods at a time.
Strongly Agree Strongly Agree Strongly Agree Strongly Agree Strongly Agree Strongly Agree Strongly Agree Agree Agree Agree Agree Agree Agree Agree Disagree Disagree Disagree Disagree Disagree Disagree Disagree Strongly Disagree Strongly Disagree Strongly Disagree Strongly Disagree Strongly Disagree Strongly Disagree Strongly Disagree
Positiveness-Negativeness
12. I usually disclose positive things about myself. 13. On the whole, my disclosures about myself are more negative than positive. 14. I normally reveal bad feelings about myself.
Strongly Agree Strongly Agree Strongly Agree Agree Agree Agree Disagree Disagree Disagree Strongly Disagree Strongly Disagree Strongly Disagree
Lawrence R. Wheeless (1976). Self-disclosure and interpersonal solidarity: Measurement, validation, and relationships. Human Communication Research, 3, 45-61.
65
15. I normally express my good feelings about myself. 16. I often reveal more undesirable things about myself than desirable things. 17. I usually disclose negative things about myself. 18. On the whole, my disclosures about myself are more positive than negative.
Agree Agree Agree Agree
Disagree Disagree Disagree Disagree
Control of Depth
19. I intimately disclose who I really am, openly and fully in my conversation. 20. Once I get started, my self-disclosures last a long time. 21. I typically reveal information about myself without intending to. 22. I often disclose intimate, personal things about myself without hesitation. 23. I feel that I sometimes do not control my selfdisclosure of personal or intimate things I tell about myself. 24. Once I get started, I intimately and fully reveal myself in my self-disclosures.
Strongly Agree Strongly Agree Strongly Agree Strongly Agree Strongly Agree Agree Agree Agree Agree Agree Disagree Disagree Disagree Disagree Disagree Strongly Disagree Strongly Disagree Strongly Disagree Strongly Disagree Strongly Disagree
Strongly Agree
Agree
Disagree
Strongly Disagree
Honesty-Accuracy
25. I cannot reveal myself when I want to because I do not know myself thoroughly enough. 26. I am often not confident that my expressions of my own feelings, emotions, and experiences are true reflections of myself. 27. I always feel completely sincere when I reveal my own feelings and experiences. 28. My self-disclosures are completely accurate reflections of who I really am. 29. I am not always honest in my self-disclosure.
Strongly Agree Strongly Agree Agree Agree Disagree Disagree Strongly Disagree Strongly Disagree
Strongly Agree Strongly Agree Strongly Agree
Agree Agree Agree
Disagree Disagree Disagree
Strongly Disagree Strongly Disagree Strongly Disagree
66
30. My statements about my own feelings, emotions, and experiences are always accurate selfperceptions. 31. I am always honest in my self-disclosures. 32. I do not always feel completely sincere when I reveal my own feelings, emotions, behaviors, or experiences.
Strongly Agree
Agree
Disagree
Strongly Disagree
Agree Agree
Disagree Disagree
67
Snyders Self-Monitoring Scale*

Directions. The statements on the following pages concern your personal reactions to a number of different situations. No two statements are exactly alike, so consider each statement carefully before answering. Please indicate the extent to which you agree or disagree with each statement. More specifically, for each one please note if you... VSA SA MA U MD SD VSD Very Strongly Agree Strongly Agree Mildly Agree are Unsure Mildly Disagree Strongly Disagree Very Strongly Disagree
Circle the response that best describes how much you agree or disagree with each statement, as it relates to you. 1. I find it hard to imitate the behavior of other people. 2. My behavior is usually an expression of my true inner feelings, attitudes, and beliefs. 3. At parties and social gatherings, I do not attempt to do or say things that others will like. 4. I can only argue for ideas which I already believe. 5. I can make impromptu speeches even on topics about which I have almost no information. 6. I guess I put on a show to impress or
VSA SA MA U MD SD VSD
VSA
SA
MA
MD
SD
VSD
VSA
SA
MA
MD
SD
VSD
VSA
SA
MA
MD
SD
VSD
VSA
SA
MA
MD
SD
VSD
VSA
SA
MA
MD
SD
VSD
Source: M. Snyder. (1974). Self-monitoring of expressive behavior. Journal of Personality and Social Psychology, 10, 526-537.
68
entertain people. 7. When I am uncertain how to act in a social situation, I look to the behavior of others for cues. 8. I would probably make a good actor. 9. I rarely need the advice of my friends to choose movies, books, or music. 10. I sometimes appear to others to be experiencing deeper emotions than I actually am. 11. I laugh more when I watch a comedy with others than when alone. 12. In a group of people I am rarely the center of attention. 13. In different situations and with different people, I often act like very different persons. 14. I am not particularly good at making other people like me.
VSA SA MA U MD SD VSD
VSA
SA
MA
MD
SD
VSD
VSA
SA
MA
MD
SD
VSD
VSA
SA
MA
MD
SD
VSD
VSA
SA
MA
MD
SD
VSD
VSA
SA
MA
MD
SD
VSD
VSA
SA
MA
MD
SD
VSD
VSA
SA
MA
MD
SD
VSD
15. Even if I am not enjoying myself, I often pretend to be having a good time. 16. I am not always the person I appear to be. 17. I would not change my opinions (or the way I do things) in order to please someone else or win their favor. 18. I have considered being an entertainer. 19. In order to get along and be liked, I tend to be what people expect me to be rather than anything else.
VSA
SA
MA
MD
SD
VSD
VSA
SA
MA
MD
SD
VSD
VSA
SA
MA
MD
SD
VSD
VSA
SA
MA
MD
SD
VSD
VSA
SA
MA
MD
SD
VSD
69
20. I have never been good at games like charades or improvisational acting. 21. I have trouble changing my behavior to suit different people and different situations. 22. At a party I let others keep the jokes and stories going. 23. I feel a bit awkward in company and do not show up quite so well as I should. 24. I can look anyone in the eye and tell a lie with a straight face (if for a right end). 25. I may deceive people by being friendly when I really dislike them.
VSA
SA
MA
MD
SD
VSD
VSA
SA
MA
MD
SD
VSD
VSA
SA
MA
MD
SD
VSD
VSA
SA
MA
MD
SD
VSD
VSA
SA
MA
MD
SD
VSD
VSA
SA
MA
MD
SD
VSD
Note: Reverse scoring must be applied to items 5-8, 10, 11, 13, 15, 16, 18, 19, 24, and 25.
70
Busss Shyness and Sociability Scales*

Directions. Below you will find several statements that concern how you interact with others. For each statement, please tell us how accurately the behavior or feeling describes you by indicating whether you agree or disagree with it. More specifically, indicate (by circling) whether you strongly agree with the statement, agree somewhat, are unsure, disagree somewhat, or strongly disagree.
Sociability Scale
1. I like to be with people. 2. I welcome the opportunity to mix socially with people. 3. I prefer working alone rather than with others. 4. I find people more stimulating than anything else. 5. I would be unhappy if I were prevented from making many social contacts.
Strongly Agree Strongly Agree Agree Somewhat Agree Somewhat Unsure Disagree Somewhat Disagree Somewhat Strongly Disagree Strongly Disagree
Unsure
Strongly Agree
Agree Somewhat
Unsure
Disagree Somewhat
Strongly Disagree
Strongly Agree
Agree Somewhat
Unsure
Disagree Somewhat
Strongly Disagree
Strongly Agree
Agree Somewhat
Unsure
Disagree Somewhat
Strongly Disagree
Shyness Scale
6. I feel tense when I am with people I do not know well. 7. I feel inhibited in social situations.
Strongly Agree Agree Somewhat Unsure Disagree Somewhat Strongly Disagree
Strongly Agree
Agree Somewhat
Unsure
Disagree Somewhat
Strongly Disagree
8. I am socially somewhat awkward.
Strongly Agree
Agree Somewhat
Unsure
Disagree Somewhat
Strongly Disagree
Source: A.H. Buss. (1980). Self-consciousness and social anxiety. San Francisco: W.H. Freeman & Company.
71
9. I am often uncomfortable at parties and other social functions. 10. When conversing, I worry about saying something dumb. 11. I feel nervous when speaking to someone in authority. 12. I am more shy with members of the opposite sex. 13. I have trouble looking someone right in the eye. 14. I do not find it hard to talk to strangers.
Strongly Agree
Agree Somewhat
Unsure
Disagree Somewhat
Strongly Disagree
Strongly Agree
Agree Somewhat
Unsure
Disagree Somewhat
Strongly Disagree
Strongly Agree
Agree Somewhat
Unsure
Disagree Somewhat
Strongly Disagree
Strongly Agree
Agree Somewhat
Unsure
Disagree Somewhat
Strongly Disagree
Strongly Agree
Agree Somewhat
Unsure
Disagree Somewhat
Strongly Disagree
Strongly Agree
Agree Somewhat
Unsure
Disagree Somewhat
Strongly Disagree
Note: Items 1 through 5 compose the sociability scale. Items 6 through 14 make up the shyness measure. Reverse scoring should be applied to all items except 3 and 14.
72
Sexual Communication Satisfaction Scale*

Directions. Please indicate the extent to which you agree with each of the statements that follow, as it relates to your current romantic relationship. Circle the response to the right that best reflects your level of agreement or disagreement.
1. 2. I tell my partner when I am especially sexually satisfied. I am satisfied with my partners ability to communicate her/his sexual desires to me. I do not let my partner know things that I find pleasing during sex. I am very satisfied with the quality of our sexual interactions. I do not hesitate to let my partner know when I want to have sex with him/her. I do not tell my partner whether or not I am sexually satisfied. I am dissatisfied over the degree to which my partner and I discuss our sexual relationship. I am not afraid to show my partner what kind of sexual behavior I find satisfying. I would not hesitate to show my partner what is a sexual turn-on for me.
Strongly Disagree Strongly Disagree Disagree Disagree Unsure Unsure Agree Agree Strongly Agree Strongly Agree
3. 4. 5. 6. 7.
Strongly Disagree Strongly Disagree Strongly Disagree Strongly Disagree Strongly Disagree
Disagree Disagree Disagree Disagree Disagree
Unsure Unsure Unsure Unsure Unsure
Agree Agree Agree Agree Agree
Strongly Agree Strongly Agree Strongly Agree Strongly Agree Strongly Agree
8. 9.
Strongly Disagree Strongly Disagree Strongly Disagree Strongly Disagree Strongly Disagree
Disagree Disagree Disagree Disagree Disagree
Unsure Unsure Unsure Unsure Unsure
Agree Agree Agree Agree Agree
Strongly Agree Strongly Agree Strongly Agree Strongly Agree Strongly Agree
10. My partner does not show me when she/he is sexually satisfied. 11. I show my partner what pleases me during sex. 12. I am displeased with the manner in which my partner and I communicate with each other during sex. 13. My partner does not show me things she/he finds pleasing during sex.
Strongly Disagree
Disagree
Unsure
Agree
Strongly Agree
Source: L.R. Wheeless, V.E. Wheeless, & R. Baus. (1984). Sexual communication, communication satisfaction, and solidarity in the developmental stages of intimate relationships. Western Journal of Communication, 48, 217-230.
73
14. I show my partner when I am sexually satisfied. 15. My partner does not let me know whether sex has been satisfying or not. 16. I do not show my partner when I am sexually satisfied. 17. I am satisfied concerning my ability to communicate about sexual matters with my partner. 18. My partner shows me by the way she/he touches me if she/he is satisfied. 19. I am dissatisfied with my partners ability to communicate her/his sexual desires to me. 20. I have no way of knowing when my partner is sexually satisfied. 21. I am not satisfied in the majority of our sexual interactions. 22. I am pleased with the manner in which my partner and I communicate with each other after sex.
Disagree Disagree Disagree Disagree
Unsure Unsure Unsure Unsure
Agree Agree Agree Agree
Disagree Disagree
Unsure Unsure
Agree Agree
Strongly Disagree Strongly Disagree Strongly Disagree
Disagree Disagree Disagree
Unsure Unsure Unsure
Agree Agree Agree
Strongly Agree Strongly Agree Strongly Agree
Reverse scoring should be applied to items 3, 6, 7, 10, 12, 13, 15, 16, 19, 20, and 21.
74
UCLA Loneliness Scale*

Directions: Please indicate how often you feel the way described in each of the following statements. Indicate by circling whether you never, rarely, sometimes, or often feel the way described.
1. 2. 3. 4. 5. 6.
I feel in tune with the people around me. I lack companionship. There is no one I can turn to. I do not feel alone. I feel part of a group of friends. I have a lot in common with the people around me.
never never never never never never
rarely rarely rarely rarely rarely rarely
sometimes sometimes sometimes sometimes sometimes sometimes
often often often often often often
7. 8.
I am no longer close to anyone. My interests and ideas are not shared by those around me.
never never
rarely rarely
sometimes sometimes
often often
9. 10. 11. 12. 13. 14. 15. 16. 17. 18.
I am an outgoing person. There are people I feel close to. I feel left out. My social relationships are superficial. No one really knows me well. I feel isolated from others. I can find companionship when I want it. There are people who really understand me. I am unhappy being so withdrawn. People are around me but not with me.
never never never never never never never never never never
rarely rarely rarely rarely rarely rarely rarely rarely rarely rarely
sometimes sometimes sometimes sometimes sometimes sometimes sometimes sometimes sometimes sometimes
often often often often often often often often often often
Source: D. Russell, L.A. Peplau, & C.E. Cutrona. (1980). The revised UCLA Loneliness Scale: Concurrent and discriminant validity evidence. Journal of Personality and Social Psychology, 39, 472480.
75
19. 20.
There are people I can talk to. There are people I can turn to.
never never
rarely rarely
sometimes sometimes
often often
Reverse scoring must be applied to positively worded items: 1, 4, 5, 6, 9, 10, 15, 16, 19, 20.
76
How To Construct Semantic Differential Scales

The semantic differential is purported to be a method for assessing the connotative meaning of concepts. The concept appears at the top of a page and is followed by a set of bipolar adjectival scales, usually with seven or nine points. The respondent is asked to rate the concept based on what it means to him/her. For example, the concept Barack Obama might be measured as follows: Barack Obama Bad: Wise: Valuable: Trustworthy: Unpleasant: Active: 1 1 1 1 1 1 2 2 2 2 2 2 3 3 3 3 3 3 4 4 4 4 4 4 5 5 5 5 5 5 6 6 6 6 6 6 7 7 7 7 7 7 8 8 8 8 8 8 9 9 9 9 9 9 :Good :Unwise :Worthless :Untrustworthy :Pleasant :Passive
It has been argued that most of the variance in meaning can be accounted for by three underlying dimensions: evaluation, potency, and activity. Items reflecting the evaluation dimension include good/bad, valuable/worthless, clean/dirty, etc.; items tapping the activity dimension include active/passive, fast/slow, etc.; potency items include strong/weak, potent/impotent, and powerful/powerless, etc. The steps in preparing a semantic differential are essentially the same as those required for constructing a Likert scale; indeed, most scholars consider the semantic differential to be a type of summated scale. Reverse Scoring Items with the negative adjective pole on the right hand side (such as unwise and worthless) would be subjected to reverse scoring, as follows: 1=9, 2=8, 3=7, etc.
77
Semantic Differential Scales: An Example
Common Scales for the Assessment Of Source Credibility Trustworthiness

Untrustworthy: Dishonest: Disreputable: Undependable: Irresponsible: Unsafe: Unjust: Unfair: Deceptive: Insincere: Dishonorable: -5 -5 -5 -5 -5 -5 -5 -5 -5 -5 -5 -4 -4 -4 -4 -4 -4 -4 -4 -4 -4 -4 -3 -3 -3 -3 -3 -3 -3 -3 -3 -3 -3 -2 -2 -2 -2 -2 -2 -2 -2 -2 -2 -2 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 0 0 0 0 0 0 0 0 0 0 0 1 1 1 1 1 1 1 1 1 1 1 2 2 2 2 2 2 2 2 2 2 2 3 3 3 3 3 3 3 3 3 3 3 4 4 4 4 4 4 4 4 4 4 4 5 5 5 5 5 5 5 5 5 5 5 :Trustworthy :Honest Reputable :Dependable :Responsible :Safe :Just :Fair :Truthful :Sincere :Honorable
Expertise
Inexpert: Uninformed: Unqualified: Unintelligent: Unauthoritative: Inexperienced: Unskilled: Untrained: Stupid: Inept: Novice: Amateurish: Incompetent: -5 -5 -5 -5 -5 -5 -5 -5 -5 -5 -5 -5 -5 -4 -4 -4 -4 -4 -4 -4 -4 -4 -4 -4 -4 -4 -3 -3 -3 -3 -3 -3 -3 -3 -3 -3 -3 -3 -3 -2 -2 -2 -2 -2 -2 -2 -2 -2 -2 -2 -2 -2 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 1 1 1 1 1 1 1 1 1 1 1 2 2 2 2 2 2 2 2 2 2 2 2 2 3 3 3 3 3 3 3 3 3 3 3 3 3 4 4 4 4 4 4 4 4 4 4 4 4 4 5 5 5 5 5 5 5 5 5 5 5 5 5 :Expert :Informed :Qualified :Intelligent :Authoritative :Experienced :Skilled :Trained :Smart :Proficient :Veteran :Professional :Competent
78
Bipolar Scales
Bipolar scales are very similar in format to semantic differentials. With a bipolar scale, respondents are asked to indicate where they place themselves on a numerical scale which is anchored at each end by a word or phrase. For example, a person might ask to indicate the extent to which a particular position reflects her own opinion: BARACK OBAMA W OULD MAKE AN EXCELLENT U.S. PRESIDENT. I tend to agree with this statement : I tend to disagree : with this statement
There is nothing sacred about a nine-point scale. Subjects may be asked to indicate their position on a continuum defined by any number of intervals. Nor is there any reason why only positive numbers must be used in numerical bi-polar scales. Indeed, there are times when negative values may make more sense to subjects: It has been suggested that Barack Obama would make an excellent President of the United States. How do you feel about the idea of a President Barack Obama? Very Negative : -7 -6 -5 -4 -3 -2 -1 0 Neutral 1 2 3 4 5 6 7 : Very Positive
Recently, some investigators have abandoned numerical categories in lieu of more precise continuous measures. For instance, a respondent could be asked to indicate the point on a continuum represented by a line which reflects his or her viewpoint. The distance between the starting point on the line and the point of intersection is measured in mm. to derive a value for the item:
It has been argued that an expanded system of mass transit which linked Sacramento to the San Francisco Bay Area would lead to many people to move from the Bay Area to the Greater Sacramento Area. As a resident of Sacramento, how would you feel about such an outcome? Please draw a slash across the point on the line which best reflects your opinion. Very Negative Very Positive
|-------------------------------------------------------------------------------------------| Neutral
79
Bipolar Scales: Examples

Perceived Homophily [Similarity] Scale*
Attitude Similarity
Doesnt think like me: Behaves like me: Similar to me: Unlike me: Perceives things like me: Personality different from mine: Does things like I do: Shares my beliefs: Shares my attitudes: Likes things I dislike: 1 2 3 4 5 6 7 8 9 1 2 3 4 5 6 7 8 9 1 2 3 4 5 6 7 8 9 1 2 3 4 5 6 7 8 9 1 2 3 4 5 6 7 8 9 1 2 3 4 5 6 7 8 9 1 2 3 4 5 6 7 8 9 1 2 3 4 5 6 7 8 9 1 2 3 4 5 6 7 8 9 1 2 3 4 5 6 7 8 9 :Thinks like me :Doesnt behave like me :Different from me :Like me :Doesnt perceive things like me :Personality similar to mine :Does things unlike I do :Doesnt share my beliefs :Doesnt share my attitudes :Dislikes things I like
Value Similarity
Morals unlike mine: Sexual attitudes different from mine: Doesnt share my values: Treats people like I do: Doesnt share my emotions: Politics different from mine: 1 2 3 4 5 6 7 8 9 1 2 3 4 5 6 7 8 9 1 2 3 4 5 6 7 8 9 1 2 3 4 5 6 7 8 9 1 2 3 4 5 6 7 8 9 1 2 3 4 5 6 7 8 9 :Morals like mine :Sexual attitudes like mine :Shares my values :Doesnt treat people like I do :Shares my emotions :Politics like mine
Appearance Similarity
Looks different than me: Different size than I am: Same weight I am: Wears hair like I do: 1 2 3 4 5 6 7 8 9 1 2 3 4 5 6 7 8 9 1 2 3 4 5 6 7 8 9 1 2 3 4 5 6 7 8 9 :Looks similar to me :Same size as I :Different weight than I am :Wears hair different than I do
James C. McCroskey, V.P. Richmond, & J.A. Daly. (1975). The development of a measure of perceived homophily in interpersonal communication. Human Communication Research, 1, 323-332.
80
Background Similarity
From social class similar to mine: Culturally different than me: Economic situation different from mine: Status different from mine: Family like mine: Background different from mine: 1 2 3 4 5 6 7 8 9 1 2 3 4 5 6 7 8 9 1 2 3 4 5 6 7 8 9 1 2 3 4 5 6 7 8 9 1 2 3 4 5 6 7 8 9 1 2 3 4 5 6 7 8 9 :From different social class than mine :Culturally similar to me :Economic situation similar to mine :Status like mine :Family different from mine :Background similar to mine
81
Passionate Love Scale (Short Version)*
Directions: We would like you to describe how you feel when you are passionately in love. Some common terms for this feeling are passionate love, infatuation, love sickness, or obsessive love. Please think of the person whom you love most passionately right now. If you are not in love right now, please think of the last person you loved passionately. If you have never been in love, think of the person whom you came closest to caring for in that way. Keep this person in mind as you respond to the thirty statements that follows. Please be assured that there is no correct way to respond. To indicate the extent to which each statement describes how you feel when passionately in love, please use the following scale:
Not At All True Of Me 0 1 2 3 4 5 6 7 8 9 10 Definitely True Of Me
Write the number which best describes how true the statement is of you in the box to the left of the statement.
1. 2. 3. 4. 5. 6. 7. 8. 9. 10. 11. 12. I would feel deep despair if this person left me. Sometimes I feel I cant control my thoughts; they are obsessively on this person. I feel happy when I am doing something to make this person happy. I would rather be with this person than anyone else. Id get jealous if I thought this person were falling in love with someone else. I yearn to know all about this person. I want this person physically, emotionally, mentally. I have an endless appetite for affection from this person. For me, this person is the perfect romantic partner. I sense my body responding when this person touches me. This person always seems to be on my mind. I want this person to know me my thoughts, my fears, and my hopes.
Source: E. Hatfield & S. Sprecher. (1986). Measuring passionate love in intimate relations. Journal of Adolescence, 9, 383-410.
82
13. 14. 15.
I eagerly look for signs indicating this persons desire for me. I possess a powerful attraction for this person. I get extremely depressed when things dont go right in my relationship with this person.
83
An Overview Of Guttman Scaling

The goal of attitude assessment via Guttman Scaling is to measure an attitude by identifying a set of statements that can be ordered along a dimension of difficulty of acceptance. For instance, the statements below would constitute a Guttman scale if endorsement of a particular item was associated with endorsing all previous items. Thus, items A-E collectively constitute a Guttman scale if most individuals responses resemble one of the following six patterns:
A no yes yes yes yes yes
B no no yes yes yes yes
C no no no yes yes yes
D no no no no yes yes
E no no no no no yes
Score 0 1 2 3 4 5
___ A. ___ B. ___ C. ___ D. ___ E.
I would not object to sitting next to a __________ on a bus. I would not object to having a __________ live next door. I would have no objections to inviting a __________ to dine in my house. At a party with my friends, I would not hesitate to ask a __________ to dance if I were attracted to him or her as a person. I would have no objections to my son or daughter dating a __________ as long as s/he were a good person.
84
Guttman Scaling Example: Assessing Physicians Attitudes Toward Abortion*
Directions. Below you will find descriptions of conditions which might lead a pregnant woman to ask you to perform an abortion. Please indicate (by circling) whether you would approve or disapprove of an abortion under each condition.
Most Difficult to Accept

approve approve approve approve approve approve approve approve approve approve approve disapprove disapprove disapprove disapprove disapprove disapprove disapprove disapprove disapprove disapprove disapprove
1. 2. 3. 4. 5. 6. 7. 8. 9.
The womans career or education would be disrupted. The woman is too young to have the child. The woman is financially unable to support the child. The woman is too old to have the child. The woman does not want the child. The womans unmarried status would be a problem. The pregnancy or childbirth would threaten the mental health of the woman. The pregnancy or childbirth would threaten the physical health of the woman. The woman is pregnant as a result of rape or incest.
___ | | | | | | | | | | | | | | | ___ Easiest to Accept
10. There is a risk of congenital abnormality. 11. The pregnancy or childbirth is a threat to the womans life.
Source: Koslowski, M., Pratt, G.l., & Wintrob, R.N. (1976). The application of Guttman scale analysis to physicians attitudes regarding abortion. Journal of Applied Psychology, 61, 301-304.
85
An Overview of Direct Magnitude Estimation

History
Magnitude estimation is a technique developed in psychophysics to measure perceptions of the magnitudes of such variables as loudness, brightness, warmth, weight and other physical sensations. In this use, the subject is given a reference stimulus, such as a light of a certain intensity, and told to consider its intensity to be reflected by a particular number e.g., 100. The subject is then told to use this reference when judging the intensity of other lights. For example, a light thought to be twice as intense as the reference light would be given a value of 200; a light half as intense should be assigned a value of 50; a light one-tenth as intense would be given the value of 10.
Application to Social Stimuli

Recently, efforts have been made to determine if subjective social stimuli, for which there is no objective measurement available, can be assessed with magnitude estimation. The answer appears to be yes. The technique has been used to assess aesthetic value of art pieces, preferences, the value of money, the seriousness of crime, the social status of occupations, and social attitudes. The argument advanced in favor of this procedure is that DME is more precise than the categorical scaling methods described above. For instance, in the typical Likert scale, a person must place her/his attitude in one of five (or seven) categories. With DME, the person can use as precise a yardstick as s/he desires.
Example
I would like to ask your opinion about how serious you think certain crimes are. The first situation is this: A PERSON STEALS A BICYCLE PARKED ON THE STREET. This crime has been given a seriousness score of 10. Use this first situation to judge all others. For example, if you think a particular crime is 20 times more serious than the bicycle theft, the number you tell me should be around 200, or if you think it is half as serious, the number you tell me should be around 5, and so on. There is no upper limit. Use ANY number so long as it shows how serious you think the situation is. If you think something is not a crime, give it a zero rating. We are no ready to begin.
86
COMPARED TO THE BICYCLE THEFT SCORE OF 10, HOW SERIOUS IS . . .
___ 1. A person using force robs a victim of $25. The victim struggles and is shot to death. ___ 2. A person disturbs the neighborhood with loud, noisy behavior. ___ 3. A person steals property worth $1000 from outside a building. ___ 4. A man forcibly rapes a woman. Her physical injuries require medical treatment. ___ 5. A five-year old child steals a candy bar in a drugstore. ___ 6. A student breaks into her professors office to steal a test. ___ 7. A terrorist randomly shoots people at an airport check-in point. Twelve people are killed and five others are critically wounded. ___ 8. A poor man steals money from a convenience store to buy food for his hungry children. The man pretends he has a gun. No one is injured. ___ 9. A student rips an article out of a book at a public library.
87
Reliability and Validity Assessment: An Example

Situation: An investigator has developed an approach to measuring Fear of Communication. Is her new approach reliable and valid? Reliability: Repeatability of Results Test-Retest Method An investigator has created a new, 40-item measure of Fear of Communication. A group of subjects completes the 40-item scale. A few weeks later, the same group completes the same scale. Their scores at time 1 are correlated with their scores at time 2. If the measure is reliable there should be a strong, positive correlation between the two scores. Parallel (Alternative) Forms Method Two 40-item scales are developed to measure Fear of Communication: Form A and Form B. A group of subjects completes both forms. Their scores on A and B are correlated. If the measures are reliable, there should be a strong, positive correlation between the two scores. Internal Consistency Method (Most Common Approach) An Investigator Has Created A New, 40-Item Measure Of Fear Of Communication. Since Each Of These Items Is Assumed To Be An Indicator Of The Same Construct Fear Of Communication, They Should Be Highly Correlated with each other (i.e., internally consistent). Our investigator computes the alpha reliability coefficient for the 40 items, which is based on the matrix of correlations among the 40 items. (High intercorrelations, high reliability; low intercorrelations, low reliability.) Validity: Does the Fear of Communication Scale Measure What it is Intended to Measure? Face Validity Are the 40 indicators on the scale consistent with common agreements about what fear of communication means? This is a guess about validity based on ones common sense. It is the weakest form of validity. Criterion-Related (Predictive Validity) Our investigator determines if subjects scores on the new Fear of Communication measure predicts their willingness to give a talk before a large group for extra credit. If the scale measures fear of communication, individuals with high scores should be less likely to volunteer for the experience. Construct Validity Our investigator administers the new scale along with measures of related constructs, such as Shyness, Self-Perceived Social Skills, and Social Anxiety. If the scale is truly valid, scores on it should be strongly and positively correlated with shyness and social anxiety; a strong, negative correlation between the fear measure and self-perceived social skills should be obtained. Content Validity Our investigator makes a judgment of the extent to which the full meaning of the construct fear of
88
communication is represented by the 40 indicators. Are there aspects of FOC that are not reflected in the indicators? Are there dimensions of FOC not assessed?
89
Typologies
The purpose of a typology is to categorize observations into a system of types. Although there are several ways to form a typology, a common approach is to create types by examining the intersection between two (or intersections among more than two) variables. In most cases, the results of this strategy is a nominal-level category system. These types can themselves be thought of as the attributes of a variable. An Example One example of a typology formed through the intersection of two variables is Rusbults Exit-VoiceLoyalty-Neglect Model of responses to dissatisfaction in relationships. Rusbult argues that two dimensions (variables) distinguish the various responses one might make when dissatisfied with a relationship. Active-Passive: One either does or does not take action which addresses the problem at hand. Constructive-Destructive: Ones response is intended to either preserve or destroy the relationship. When these two variables are dichotomized and crossed, the result is a typology of four kinds of responses to dissatisfaction:
Active
Exit
Voice
Destructive
Neglect Loyalty
Constructive
Passive
90
QUESTIONNAIRE CONSTRUCTION
I. Introductory Observations.
II.
Guidelines For Asking Questions About Behavior.

A. Use Memory Cues (Aided Recall).
B.
Make Each Question as Specific as Possible.
A question should be specific with regard to who, what, where, and when.
1.
Who: To Whom Does the Question Apply?
2.
What: Is the Object of the Question Clear?
3.
Where: In What Location Must the Behavior be Performed to be Relevant?
4.
When: What is the Time Frame for Behavior Performance?
a.
Sources of Confusion.
b.
Appropriateness of Time Frame as a Function of Salience.
c.
Prevent Telescoping.
Telescoping: the tendency to place an event in a time period when it actually occurred prior to that event.
91
i.
Multiple Interviews.
Interview the respondent about the behavior in question. The data for the first interview are not used, but serve to bound subsequent reports. (e.g., The last time we talked, you told me that you had done x...Since our last interview, have you...[questions about behavior follow].
ii.
Bounded Recall In One Interview.
Bound recall with an initial question. (For instance, in an interview which took place in May, the respondent could be asked to report about the occurrence of behavior X in the month of April. The subsequent question then asks about performance of the same behavior in May.)
iii.
Using Records.
d.
Minimize Omissions.
Omission: A failure to report a behavior which occurred during the specified time frame.
i.
Use Their Personal Records.
ii.
The Personal Diary Approach.
e.
Focus on the Past or Present.
C.
Some Strategies For Asking Threatening Questions About Behavior.
1.
Use Open Questions
2.
Frame Issue as No Longer Controversial.
92
3.
Use Respondents Language
4.
Consider Using Respondents As Informants.
5.
Deliberately Loading the Question is Sometimes Okay!
a.
Everybody does it.
b.
Assume the Behavior.
c.
Use of Authority to Legitimize Threatening Behavior.
d.
Reasons Why Not.
6.
Did you ever, even once...
7.
Casual Approach.
8.
Precede Question With A Contrast Question.
9.
Anonymity Through Self-Administered Questionnaires.
10.
Using the Random Response Technique.
III.
Guidelines For Measuring Knowledge

A. Discourage Guessing.
93
B.
Consider the Merits of All Question Types.
1.
Open Questions.
2.
Dichotomous Questions.
3.
Multiple Choice Questions.
C.
Reduce Threat
1.
Casual Approach.
Do you happen to know ... As far as you know, is ... From what you have heard or read...
2.
Legitimize Ignorance.
D.
Check For Lying
E.
Consider Using Audio/Visual Aids.
IV.
Guidelines For Asking Questions About Attitudes.

A. Clearly Specify the Attitude Object
1.
Define Terms, When Necessary.
94
2.
Assess the Specific Parts of the Attitude Object.
B.
Assess All Relevant Components of Attitudes: Cognitive, Affective, and Behavioral.
C.
Assess Attitude Strength.
1.
The Strength Dimension Can Be Built Into The Question.
2.
A Separate Question Can Be Used To Assess Strength.
D.
Avoid Double-Barreled Questions
1.
Only One Attitude Object For Each Question, Please.
2.
Consider Separating Source From Opinion.
E.
Offer Two Views, Not Just One.
95
Unwise
Should the Federal Government allow...? Should the Federal Government forbid...? Do you favor...? Do you oppose...? Are you satisfied with...? Are you dissatisfied with...? Do you support...? Do you oppose...?
Better
Should the Federal Government allow or forbid... Do you favor or oppose...?
Are you satisfied or dissatisfied with...?
Do you support or oppose...?
F.
Include The Middle Point in Bipolar Questions.
G.
Be Sensitive to Word Effects.
1.
Loaded Language.
2.
Effects of Enhanced Outcomes Versus Simple Labels (see handout on page 102).
H.
Avoid the Error of Incomplete Alternatives.
I.
Avoid Double Negatives.
J.
Dont Dismiss the Value of Open-Answer Questions.
1.
Advantages of Open Questions.
96
2.
Disadvantages of Open Questions.
K.
When Using Closed Question Formats, Be Open To Creative Alternative to the Traditional Approaches..
1.
Reiteration of the Traditional Approaches (Likert, Semantic Differential Scaling, etc.)
2.
Some Creative Alternatives.
a.
Use Metaphors
i.
The Rating Thermometer.
ii.
The Ladder Scale.
b.
Rankings.
97
c.
Paired Comparisons.
V.
Guidelines For Organizing the Questionnaire.

A. The Introduction.
1.
Legitimize the Sponsor.
2.
Motivate Participation.
B.
Provide Clear Instructions.
1.
How responses should be recorded.
2.
Legitimize all responses.
3.
Overview questionnaire: Lay of the land.
C.
Be Sensitive To Question Order Effects (see example on page 106).
1.
The Strain Toward Consistency.
2.
General Questions Before Specific Questions.
D.
Organization.
1.
Organize Questions into Content Subsections.
98
2.
Start Off With a Particularly Interesting Question/Issue.
3.
Save Threatening Issues For the Latter Part of the Questionnaire.
4.
Provide Transitions.
5.
Clearly Communicate Skip Patterns.
6.
Save Demographic Questions For Last.
E.
Length.
F.
Variety.
1.
Use a Variety of Question Formats.
2.
Use a Variety of Response Formats.
V.
Pretesting. (See handout on page 107.)
99
Whats Wrong With These Questions About Behavior?

On the basis of class lecture, please identify the flaws in the following questions. Each question is deficient in several ways.
1.
Over the past three months, how many movies did you view?
2.
How often do you recycle?
3.
Over the past five years, how many appliances did you purchase?
4.
How much money have you spent on clothing during the past year?
5.
How many long distance phone calls do you make in the typical week?
6.
How often do you attend movies and theatrical performances never, rarely, sometimes, or often?
100
How Can These Questions About Threatening Behaviors Be Improved?

On the basis of the guidelines discussed in class, please consider how the following questions about potentially threatening behavior could be improved.
1.
How many packs of cigarettes do you smoke in the typical day? None, 1 pack, or 2 or more packs?
2.
Now we would like to ask you a couple of questions about your attitudes toward fidelity in marriage.
A. Do you approve or disapprove of extramarital affairs? ___Approve ___Disapprove
B.
Have you ever been unfaithful in your marriage? ___No ___Yes
3.
Do you regularly purchase pornographic materials for your personal use?
4.
Have you ever beaten your child so hard that he or she became physically injured by your abuse?
101
Question Enhancement Effects*

When seeking public opinion on social issues, survey researchers must use labels to represent the issue. Such labels can be simple or enhanced. Examples of simple labels include welfare, space exploration, and aid to the cities. Conversely, public opinion researchers may use an enhanced outcome label to describe the desired effect of a course of action. Thus, welfare might be labeled as providing assistance to the needy. Rasinski conducted split-ballot experiments in 1984, 1985, and 1986 to assess the effects of simple versus enhanced outcome wording on the publics support for government spending. This is what he found:
Percent Saying Too Little Is Being Spent Crime Year 1984 1985 1986 Enhanced: Halting Rising Crime Rates 69.3% 67.3% 66.8% Simple: Law Enforcement 56.5% 57.8% 53.9%
Drug Addiction Enhanced: Dealing with Drug Addition 1984 1985 1986 64.4% 66.6% 60.7% Simple: Drug Rehabilitation 49.4% 58.0% 56.3%
Adapted from: K.A. Rasinski. (1989). The effect of question wording on public support for government spending. Public Opinion Quarterly, 53, 388-394.
102
Percent Saying Too Little Is Being Spent Welfare Year 1984 1985 1986 Enhanced: Assistance To The Poor 64.1% 65.2% 62.8% Simple: Welfare 25.2% 19.8% 23.1%
Cities Enhanced: Solving Problems of Big Cities 1984 1985 1986 52.0% 45.7% 48.2% Simple: Assistance To Big Cities 21.3% 20.8% 17.7%
Social Security Enhanced: Protecting Social Security 1984 68.2% Simple: Social Security 53.2%
103
Whats Wrong With These Questions About Attitudes?

1. 2. 3. 4. 5. Do you tend to consider yourself to be a Republican, Independent, or Liberal? Are you a Republican or a Democrat? Do you approve of the Presidents plan to reduce the pay of federal workers and to cut the federal deficit? How much do you enjoy attending theatrical performances and movies? Not at all, somewhat, or yes. Do you approve or disapprove of President Smiths plan to give every young American a chance to go to college through a program of national service in which students would receive financial assistance for battling crime, serving as teaching aides, becoming one of his interns at the White House, or cleaning up the environment? Are you satisfied with the quality of the instruction your children receive in the areas of math, reading, and writing? Do you agree with the argument made by some communist feminists that the U.S. Government should guarantee equal pay for equal work, even though such a policy might lead to massive layoffs? Do you favor or oppose a law which would not allow people to defer paying taxes on IRA capital gains until retirement? All in all, do you approve or disapprove of the job that President Smith and Vice President Jones have done since taking office? As you may have read, many people have claimed to have been abducted by space aliens during the past few years. Do you believe that these people are telling the truth, or are they insane? All things considered, do you agree or disagree with the following statement: The federal government should not be allowed to forbid companies to lay off workers who wish to take time off to care for a sick family member.
6. 7.
8. 9. 10.
11.
104
Some Common Response Categories*

Choice Type Agreement Response Categories Agree-disagree Approve-disapprove Favor-oppose Are you for or against? Evaluation Excellent-good-fair-poor Good idea-bad idea Too many-not enough-about right Too much-to little-about right amount Better-worse-about the same Useful for comparisons with past or expectations for future. Can be expanded to five points by asking very much better-slightly better-etc. Useful in ratings of importance and satisfaction. Good for ratings of activity frequencies. Useful in evaluating peoples satisfactions with amounts. Good, all-purpose evaluation rating scales. Comments Useful follow-ups include: How Strongly? and How Much?
Very-fairly-not at all Frequency Regularly-sometimes-seldom-never Often-most of the time-some of the time-rarely-never More likely-less likely-no difference
Good for measuring probability of the respondents action, given some hypothetical circumstance.
Adapted from: P.B. Sheatsley (1983). Questionnaire construction and item writing. In P.H. Rossi, J.D. Wright, & A.B. Anderson (Eds.), Handbook of survey research (pp. 195-230). Orlando: Academic Press.
105
Question Order Effects: The Classic Example*

Year: 1948 Order: Communist Reporter/American Reporter 1. Do you think the United States should let Communist newspaper reporters from other countries come in here and send back to their papers the news as they see it? 2. Do you think a Communist country like Russia should let American newspaper reporters come in and send back to America the news as they see it? 36.5% % Saying Yes
65.6%
Order: American Reporter/Communist Reporter 1. Do you think a Communist country like Russia should let American newspaper reporters come in and send back to America the news as they see it? 2. Do you think the United States should let Communist newspaper reporters from other countries come in here and send back to their papers the news as they see it? 89.8%
73.1%
Source: H.H. Hyman & P.B. Sheatsley. (1950). The current status of American public opinion. In J.C. Payne (Ed.), The teaching of contemporary affairs. Twenty-first Yearbook of the National Council of Social Studies, 11-34.
106
Guidelines For Pretesting Your Questionnaire

Importance of Pretesting
Think of questionnaire pretesting as nothing more than taking a preliminary draft of the questionnaire out for a trial drive to diagnose potential problems with it. Every questionnaire must be pretested, even when written by a very experienced survey research professional. No exceptions!
Pretest For What? Purposes of Questionnaire Pretesting Assessment of the Quality of Specific Questions
Variation. If almost everyone responds to a question in the same way (i.e., there is little variation in their responses), this is a good sign that something might be wrong with the question. Perhaps the question is worded in such a way that people are not being truthful out of concerns over how they will be judged. Of course, a lack of variation is not always a problem. If 95% of the population reports being able to read, we wont be troubled by this lack of variation. Meaning. We need to pretest on meaning to determine if each question is being understood as intended. What is clear and obvious to the question-writer may be interpreted differently by respondents. Pretesting on meaning is needed to identify such confusions and correct them! Pretesting on meaning is also important because many respondents will answer a question even if they dont understand it. In the survey research literature, many comical examples have been reported of questions being interpreted in other-than-intended ways. For instance, the question Should corporations be driven by profits? has been interpreted by some subjects as Should corporations be driven (controlled) by prophets? Questions about heavy traffic may be assumed by some subjects to refer to cars and trucks; other respondents especially those living in bad neighborhoods may interpret the question as being about drug trafficking. Family planning may be interpreted as birth control by some individuals, but as family decision-making by others. Difficulty. A good question asks for information that a respondent can provide. A question such as How many loafs of bread do you and your family consume each year? is probably a bad one; even the most cooperative respondent might have trouble coming up with a valid answer. Respondent Interest. Finally, every question needs to be pretested to determine its effect on respondents levels of interest in the survey. Efforts should be made to enliven dull questions. This is why varying question formats is especially useful!
Assessment of the Questionnaire Format

It is not enough to pretest each question. questionnaire must also be pretested. The structure and organization of the overall
Flow and Naturalness of the Sections. One must assess the extent to which each section flows well. Does it cohere? One must also determine if there is a natural flow across sections. This is
107
where overviews and transitions can be important. Like a good term paper, a questionnaire must overview its content and provide transitions that link its distinct parts. Question Order. If one question seems to shape how a completely different question is interpreted, the two questions may need to be reordered or separated. For instance, suppose you ask respondents how satisfied they are with what they have to pay to attend UCD, and then ask them to evaluate UCD in general. If they tend to answer the second question in terms of the high cost of tuition, a red flag should go up. This problem could be solved simply by asking the more general question first. Clarity of Skip Patterns (Branching Questions). Very often, we find it necessary to ask different questions to different people, based on their answers to an earlier question. For instance, if I ask you if you have purchased a car during the past 12 months, I will probably want to follow up by asking you questions about that purchase. These questions are irrelevant if you did not purchase a car during the specified time period. In this case, you will need to be given clear instructions to skip to the next relevant question or even to the next section. A questionnaire must thus be pretested to determine if subjects understand your instructions concerning such branching questions and follow them correctly. Length. Respondents responses to a questionnaire have more to do with how interesting it is than with how long it is. However, as a general rule, a questionnaire administered in a face-to-face interview should probably be no more than 50 minutes long; longer questionnaires may need to be administered in a couple of sessions. For phone surveys, the general norm is about 10 minutes. Respondent Interest. Just as you need to pretest each question to determine its impact on respondent interest, so too must you assess the changing nature of respondents interests over the duration of the questionnaires administration. If their interest begins to fall at some point, that part of the questionnaire may need to be given more spice. Respondent Well-Being. Does the questionnaire leave respondents bummed out? If so, do something! Insert a few positive, upbeat questions.
Pretesting Strategies Input from Professionals

It is useful to begin pretesting by asking one or more professional survey researchers with experience writing questions to look over your survey and make suggestions. Experienced interviewers can also provide useful feedback.
Trial Runs with Respondents from Population

Input from professionals is important, but cannot substitute for pretesting with respondents drawn from the same population from which the real sample will be derived. The participating pretest is one in which the respondent knows that his/her purpose is to help evaluate the questionnaire. After each question, the interviewer can ask the respondent questions such as:
108
What did the whole question mean to you? How would you say it? What did ______ make you think of? What was it you had in your mind when you said __________? Would your answer have been different if the question had been worded this way: ______________________? A second approach, often used later in the pretesting processes, is the undeclared pretest. With this tack, the interviewer pays careful attention for points of confusing, respondent boredom, the effects of the questionnaire on respondents mood, etc. However, the interviewee is never told that s/he is merely a pretest respondent.
109
SAMPLING
I. Introductory Comments.
A. Who Samples?
B.
Why Sample? Costs, Time, and Necessities.
II.
A Working Vocabulary.
A. Population: The entities about which one wishes to draw conclusions.
B.
Sample: The subset of the population which is actually studied, and from which results are generalized to the population.
C.
Element (Unit of Observation): The unit about which information is collected and that provides the basis for our analysis.
D.
Sampling Unit: The elements considered for selection at some stage of the sampling procedure.
1.
The Sampling Unit in Single-Stage Sampling.
2.
The Sampling Unit in Multiple-Stage Sampling.
E.
Statistic: The summary description of a given variable in a SAMPLE.
F.
Parameter: An estimate of a variable in the POPULATION.
110
G.
Sampling Error: An estimate of the degree of error to be expected for a given sample.
H.
Representativeness and External Validity.
III.
Nonprobability Sampling. A nonprobability sample is one which has been selected not on the basis of a randomized mechanism, but on the basis of convenience or systematically employed nonrandom criteria.
A. Nonprobability Sampling Strategies.
1.
Convenience Samples: Selection based on availability for the study.
2.
Quota Sampling: Interviewers select individuals for the sample in the same proportions as the population proportions on easily identified variables (e.g., age, gender, race).
3.
Disparate Purposive Sampling: Examine the Most Extreme Cases.
a.
Best Case/Worst Case.
b.
Most Dissimilar.
4.
Typical Case(s) Sampling: Cases are selected for study on the basis of the investigators judgment of representativeness.
5.
Critical Case Sampling: Cases are selected because they have had predictive power in the past not because they are seen as being representative of the population.
6.
Snowball Sampling.
111
B.
Nonprobability Sampling Techniques: An Exercise (page 117).
IV.
Probability Sampling. In probability sampling, each unit in the population has a known, nonzero probability of being included in the sample.
A. Equal Versus Unequal Probability Sampling.
1.
Equal Probability Sampling: Sampling designs which give each member of the population the same probability of being selected for the sample.
2.
Unequal Probability Sampling: Sampling designs which give each member of the population an unequal probability of being selected for the sample.
B.
Probability Sampling Techniques.
1.
Simple Random Sampling: This is an equal probability sampling procedure. All members of the population are listed, or at least physically present. That is, you could reach any and all of them if you so desired.
a.
Sampling fraction: The probability of selection, determined by dividing the desired sample size by the number of units in the population.
b.
Some Techniques for Random Selection From Sampling Frame.
i.
Let the computer do it.
ii.
Creative Use of Random Numbers Table.
iii.
Manual procedures.
c.
Advantages and Disadvantages.
112
2.
Systematic Sampling.
a.
Procedure.
i.
Assemble a list (the sampling frame) of all of the units of the population.
ii.
Randomly select a starting point.
iii.
Determine your sampling fraction (1/n).
iv.
Select every nth case.
b.
Potential Problems.
i.
Is your sampling frame inclusive of every unit in the population?
ii.
Is the sampling frame (list) well mixed?
periodicity concerns
3.
Stratified Sampling: Random Sampling of Strata (Subpopulations): This sampling approach involves the simple random sampling of the subpopulations of the population of interest.
a.
Equal Versus Unequal Probability Stratified Sampling.
b.
4.
Cluster Sampling: Simple or Stratified Random Sampling of Entire Groups.
a.
Procedure.
113
i.
Divide survey population into unique, nonoverlapping groups prior to sampling. (Groups are typically preexistent, naturally occurring groupings.) These groups are the primary sampling unit.
ii.
Randomly sample a subset of the clusters.
iii.
Survey every member of the selected clusters.
b.
Examples.
c.
Distinction Between Stratified Sampling and Cluster Sampling.
i.
Similarities.
ii.
Dissimilarities.
d.
Stratified Cluster Sampling.
5.
Multi-Stage Sampling.
a.
Procedure For Two-Stage Sampling.
i.
Survey Population is Organized into Groups (Clusters), which are sampled to obtain Primary Sampling Units (PSUs).
ii.
The Members of these PSUs are Sampled (Simple or Stratified Sampling) To Obtain the Final Sample.
b.
A Three-Stage Sampling Example.
c.
Sampling Interviewees In the Household PSU (see handouts beginning on page 119).
114
i.
Kish Method of Respondent Selection.
ii.
Troldahl-Carter Interviewee Selection Method.
iii.
Bryant Method of Respondent Selection.
iv.
Some Less Obtrusive Interviewee Selection.
C.
Probability Sampling Techniques: An Exercise (page 124).
V.
Sources of Error in Sampling Designs.

A. Nonsampling Bias.
Nonsampling bias is the difference between the true target population value (e.g., means, percent agreement/disagreement, etc.) and the population value we would obtained if the data collection operations were carried out on the entire population.
Nonsampling bias results from explicit decisions and implementation of decisions during data collection efforts that are not directly related to the selection of the sample.
1.
Nonsampling Bias Due To Poor Definition of the Study Population.
a.
Target Population: To Whom We Want To Generalize Findings.
b.
Study Population: The Population Which is Actually Represented, Given the Researchers Procedures.
2.
Nonsampling Bias Due To Nonresponse Bias.
a.
The Unreachables.
115
b.
The Uncooperatives.
3.
Nonsampling Bias Due to Measurement Error.
B.
Sampling Bias: Sampling bias is the systematic error that results from a sampling approach that over-represents or under-represents a portion of the study population.
C.
Sampling Variability.
Sampling variability is the fluctuation of sample estimates that results from the random selection process.
Sampling variability is inevitable because statistics from randomly selected samples will vary from one sample to the next due to chance. (Remember: You have only one of a very large number of samples which could have been drawn.)
1.
Factors Which Increase Sampling Variability.
a.
Small Sample Size (N) (as sample size increases, sampling variability decreases).
b.
Large Variance in the Variable (sampling variability decreases as sample homogeneity decreases.)
2.
Sampling Variability and Confidence Intervals.
VI.
Sample Size: How Many Respondents?
116
Nonprobability Sampling: An Exercise

Directions: Identify the nonprobability sampling strategy used in each of the scenarios described below. Procedure 1. Dr. Oliver Outcast has decided to launch a program of research into intergalactic space abductions. He decides that he should begin by interviewing individuals who believe that they have been abducted. He first interviews a local woman whose kidnapping was featured in a recent article in the town newspaper. He then asks this person for the names of other individuals who have contacted her with reports of similar experiences. These individuals are then interviewed and asked to provide the names of other abduction victims they know of. What kind of sampling procedure has Oliver employed? 2. An investigator is interested in the relationship between shyness and career choice. She hypothesizes that careers which involve much social contact (e.g., sales) will be judged as less attractive by people who are shy than by people who are not shy. Conversely, careers which involve less social contact (e.g., laboratory technician) should be rated as more attractive by individuals who are shy than by individuals who are outgoing. Our underfunded investigator selects a sample of college students enrolled in her departments classes to test her hypotheses because they are available and can be induced to participate with an offer of extra credit. What type of nonprobability sampling design did our investigator use? 3. Due to the recent tuition increases, you decide to get a second job to finance your UCD education. Before you know it, youve landed a job with Jones Marketing, which has an office at the Woodland Fair Mall. The manager puts you on a project intended to assess peoples interests in a new service called Mechanics On Call. With this service, a certified mechanic will come to your house and change the oil and other fluids in your car for only $49.95. What a deal. Your task on your first day of work is to interview 50 men under the age of 35, 50 men over the age of 35, 50 women under the age of 35, and 50 women over the age of 35. What kind of design is Jones Marketing apparently making use of? Sampling Design (Circle Answer)
CONVENIENCE MOST/LEAST SIMILAR TYPICAL CASES CRITICAL CASES SNOWBALL QUOTA
117
Procedure 4. A news organization covering a state election for Governor attempts to forecast the results of the election by interviewing an assortment of citizens from two counties which appear to be demographically similar to the state as a whole. What sampling strategy is this organization using?
Sampling Design (Circle Answer)

CONVENIENCE MOST/LEAST SIMILAR TYPICAL CASES CRITICAL CASES SNOWBALL QUOTA CONVENIENCE MOST/LEAST SIMILAR TYPICAL CASES CRITICAL CASES SNOWBALL QUOTA CONVENIENCE MOST/LEAST SIMILAR TYPICAL CASES CRITICAL CASES SNOWBALL QUOTA
5. A news organization covering a state election for Governor attempts to forecast the results of the election by interviewing an assortment of citizens from one county which has always voted for the winner in past elections. What sampling strategy is this organization using?
6. An organizational behavior scholar wishes to identify those factors which distinguish well managed organizations from poorly managed organizations. She selects the 30 organizations from the S&P 500 with the greatest percentage increase in five-year earnings and the 30 S&P 500 organizations with the largest average percentage decline in five-year earnings. She then interviews a broad spectrum of individuals from within each organization. Our investigator has made use of what kind of sampling procedure?
118
Interviewee Selection Procedures

Kish Method of Respondent Selection*
The Kish method is the most rigorous interviewee selection procedure ever developed. It is also the most intrusive, and is thus not used very much anymore. In particular, the technique requires that personal information be collected about the members of the household before rapport has developed. Step 1: After the introductory comments, obtain a list of all of the individuals in the household who meet the age requirement for the survey, and then ask for the sex and age of each member. Step 2: Delete from the list those individuals who are ineligible to participate in the study because of other criteria relevant to the survey (e.g., age, voter registration, or whatever). Step 3: Rank order the members of the household on the basis of their age and gender, beginning with the oldest man and ending with the youngest woman. For instance: List All Persons Age 18 and Over in this Dwelling Unit Person Husband Wife Oldest Son Youngest Son Husbands Dad Daughters Husband Daughter Sex M F M M M M F Age 55 55 24 22 80 30 25 Rank 2 6 4 5 1 3 7 X Interviewee
Step 4: Select person to interview on the basis of one of eight versions of a selection table that is printed on the questionnaire. (Each selection table is used either 1/6th or 1/12th of the time [see table below]. The selection table which is used for a particular household is based on random assignment of a table to the household.)
Source: L. Kish. (1965). Survey sampling. New York: John Wiley.
119
If the Number of Adults in Household Is: Proportion of Assigned Tables 1/6 1/12 1/12 1/6 1/6 1/12 1/12 1/6 Table No. A B1 B2 C D E1 E2 F 1 2 3 4 5 >5
Select Adult Numbered: 1 1 1 1 1 1 1 1 1 1 1 1 2 2 2 2 1 1 1 2 2 3 3 3 1 1 2 2 3 3 4 4 1 2 2 3 4 3 5 5 1 2 2 3 4 5 5 6
Troldahl-Carter Interviewee Selection Procedure*
Step 1: Ask how many persons live in the household age ___ or older. For example, How many persons 18 years of age or older live in your household, including yourself? Step 2: Ask how many of these adults are men. Step 3: Determine the respondent on the basis of one of four matrices of possible respondents (a matrix should be randomly assigned to a household). One of the four matrices developed by Troldahl and Carter is reproduced below. See the source above for the other three matrices.
Source: V.C. Troldahl & R.E. Carter (1964). Random selection of respondents within households in phone surveys. Journal of Marketing Research, 1, 71-76.
120
Question 1 Question 2 How many of these adults are men? 0 How many persons 18 years of age or older live in your household, including yourself? 1 Adult Interview the woman Interview the man 2 Adults Interview the oldest woman Interview the man Interview the oldest man 3 Adults Interview the youngest woman Interview the man Interview the youngest man Interview the youngest man >3 Adults Interview the youngest woman Interview the oldest woman Interview the youngest man Interview the oldest man Interview the oldest man
1 2 3 >3
Bryant Method of Respondent Selection*
This procedure represents a modification of the Troldahl-Carter method. It is sometimes referred to as the Troldahl-Carter-Bryant Procedure. Its purpose is to correct for the undersampling of males which appears to occur with the Troldahl-Carter approach. Step 1: Ask how many individuals aged ___ or older live in the household. Step 2: Ask how many of these individuals are women. Step 3: Select a respondent on the basis of one of the four matrices. (Matrix use alternates.)
Source: B.E. Bryant. (1975). Respondent selection in a time of changing household composition. Journal of Marketing Research, 12, 129-135.
121
No. of Women Aged ___ + In Household 1 Matrix A 0 1 2 3 >3 Matrix B 0 1 2 3 >3 Matrix C 0 1 2 3 >3 Matrix D 0 1 2 3 >3 man woman man woman man woman man woman
Number of Persons Aged ___ + In Household 2 oldest man man youngest woman 3 oldest man youngest man man oldest woman >3 youngest man woman youngest man man or youngest man youngest woman youngest man woman oldest woman youngest man oldest man man youngest woman oldest man woman oldest man man or oldest man oldest woman youngest man man youngest woman oldest man woman oldest woman oldest woman oldest man youngest man oldest woman youngest woman youngest woman oldest man woman oldest woman youngest man woman youngest woman youngest woman youngest man oldest man youngest woman oldest woman oldest woman
122
Less Obtrusive Interviewee Selection
Birthday Methods
One recently developed approach is to ask to talk to the person within the sampling unit who most recently had a birthday. A variant of this approach is to ask for the individual who will have the next birthday. Preliminary research suggests that this approach leads to rather unbiased (i.e., representative) samples, but much more research is needed before it can be used with great confidence. Theoretically, the method should work. The only potential bias is that people born in certain months may be over represented, but since astrological sign does not correlate with much of anything, the consequences of such a bias should be minimal.
Hagen/Collier Method
With this approach, the interviewer directly requests to talk with one of four individuals: (1) The oldest male over age ___; (2) the oldest female over age _+__; (3) the youngest male over age ___; or (4) the youngest female over age ___. In households with only one male or one female, that individual is both the oldest and youngest representative of his/her gender. Of course, the request is made to talk to the oldest/youngest male/female who live in the household not the oldest/youngest male/female who happens to be home at the time of the contact. The four different requests must be used in proportion to the known distribution of males and females in the sampling area. The method will probably lead to the under representation of middle-aged individuals of both genders.
123
Probability Sampling: An Exercise

Directions: Identify the probability sampling strategy used in each of the scenarios described below.
Procedure 1. A U.C. professor is interested in assessing students attitudes toward an increase in tuition fees. He calculates that he will need to question one out of every 25 students currently enrolled in the nine-campus system. The professor has his computer randomly generate a number between 1 and 25 for every student in the system. He then writes down each number within this range on a slip of paper, places these numbers in his baseball cap, and asks a colleague to pull one number out of the hat. The number 17 is selected. All students who were assigned the number 17 by the computer were then mailed a cover letter, questionnaire, and stamped return envelope. What sampling design did he use? Suppose that the good professor had instead taken a randomized and enumerated list of all students in the U.C. system and randomly selected a starting point on that list (say, student number 21,058). He then selects that student for inclusion in the sample, and every 25th student thereafter (cycling back through students 1-21,057). What sampling design did he use? Now suppose that our professor had first assigned each student to a group based on major. Thus, he had a group of physics majors, biology majors, geography majors, etc. He then selected four percent (1/25) of the students in each major, using simple random sampling. What design did he use? An educational researcher is interested in teachers attitudes toward competency requirements for K-12 educators employed in the State of California. He obtains a list of all public schools in the state and then randomly selects 25% of these for inclusion into his sample. He then obtains a list of all of the teachers in each school selected and randomly draws 10% of the schools educators for inclusion in his final sample. What design did he use? An educational researcher is interested in teachers attitudes toward competency requirements for K-12 educators employed in the State of California. He obtains a list of all public schools in the state and then randomly selects 10% of these for inclusion into his sample. He then mails questionnaires to every teacher in the schools selected. What sampling design did he use? Sampling Design (Circle Answer)
SIMPLE RANDOM SYSTEMATIC STRATIFIED CLUSTER MULTISTAGE
2.
3.
SIMPLE RANDOM SYSTEMATIC STRATIFIED CLUSTER MULTISTAGE SIMPLE RANDOM SYSTEMATIC STRATIFIED CLUSTER MULTISTAGE
4.
5.
124
Procedure 6. A media researcher wishes to compare the current occupational portrayal of blacks and whites in U.S. print ads. She identifies all magazines with a circulation greater than one million in the last calendar year, and then randomly selects 20% of these magazines for analysis. Every ad which appeared in the issues of these magazines during the year is content analyzed. What design did she use? A sociologist is interested in doing a survey of his fellow sociologists beliefs about the important issues in the field. He defines his population as all people who are members of the American Sociological Association (ASA). A list of the current members is obtained from ASA headquarters. The investigator samples 25% of the membership by flipping two coins for each name listed. If heads came up on both coins, the member was included in the final sample. Suppose our investigator had broken down his lists into smaller lists, based on each members primary interest (e.g., race relations, organizational sociology, health sociology, etc.), and then applied simple random sampling to each list. What kind of sampling design would this approach represent? Lets assume that our investigator took an alphabetized list of members and enumerated each name. He then selected a random starting point through the use of a random numbers table; lets assume that number 329 came up. He then selected member 329, and every fourth name thereafter (333, 337, 341, etc.), cycling back through the top of the list until he reached member 328. What sampling design did our investigator use?
Sampling Design (Circle Answer)

7.
8.
SIMPLE RANDOM SYSTEMATIC STRATIFIED CLUSTER MULTISTAGE SIMPLE RANDOM SYSTEMATIC STRATIFIED CLUSTER MULTISTAGE
9.
10. A major survey research firm is interested in adult (i.e., aged 18 or older) Americans attitudes on a variety of law and order issues. She begins by obtaining a list of zip codes in use throughout the nation, and then randomly selects 25% of these codes. She then obtains a list of households within each of the selected zip codes and randomly selects 5% of the households from each zip code area, weighting each area for the number of households in it. Finally, she uses the last birthday technique to identify one person within the household for inclusion in the final sample. This interviewee is also weighted to take into account the number of eligible individuals within the household. What sampling design did our investigator use?
125
EXPERIMENTAL METHODS
I. An Overview of Experimentation.
A. Select a Question for Study.
B.
Formulate a Theory-Based Hypothesis as a Proposed Solution to the Problem.
C.
Design and Execute An Experiment Designed to Test the Hypothesis.
D.
Test the Hypothesis.
E.
Examine the Generality of the Tentatively-Supported Hypothesis in New Situations Through Replication.
F.
Examine the Theoretical Implications of the Findings.
II.
The Research Question.

A. Inquiry Starts with Ignorance.
B.
Manifestations of Problems.
1.
A Gap in Knowledge.
2.
Contradictory Results In Previous Research.
3.
Inability to Explain an Observation.
126
4.
Theories.
a.
Competing Predictions or Explanations.
b.
Derivations/Tests of Theories.
C.
Unsolvable Problems and the Limits of Experimentation.
III.
The Hypothesis.
A. Definition: An hypothesis is a testable statement of a potential relationship between two (or more) variables.
B.
Examples of Hypotheses.
1.
Covariation: An hypothesis that states that two (or more) variables will be statistically correlated with each other.
Example: High self-monitors are less likely to be detected when given the task of deceiving others than will low self-monitors.
2.
Causal Hypotheses.
Example: High self-monitors are more skilled at suppressing behaviors indicative of stress than low self-monitors, thereby enhancing their abilities to lie without detection.
C.
Types of Variables.
1.
Independent Variable: An aspect of the environment that is experimentally manipulated to study its effects on other variables.
127
2.
Dependent Variable: The variable that we propose will be affected by our manipulations of the independent variable.
3.
Mediator Variable: A variable may be said to function as a mediator to the extent that it accounts for the relationship between the independent variable and the dependent variable.
Independent Variable(s)
Mediating Variable(s)
Dependent Variable(s)
See exercise on page 136.
4.
Manipulation Checks: Measurements intended to determine if the experimenters manipulation (operationalization) of an independent variable worked as intended.
D.
Operationalization.
IV.
Some Experimental Designs.

A. Two-Group Designs.
1.
Two-Randomized Group Design (Posttest Only).
a.
Design.
128
R M O R O
Experimental Group Control Group
R = Randomization M = Manipulation of Independent Variable O = Observation of Dependent Variable (Posttest)

b. Example: Training and Detection Deception.
i.
Hypothesis.
Individuals who are instructed in the behavioral cues of deception will be more likely to detect deception than will individuals who have not received such training.
ii.
Procedure.
iii.
Internal Validity.
2.
Two Randomized Groups Design (Pretest/Posttest).
a.
Design.
129
R O1 M O2 R O1 O2
R = Randomization O1 = Observation of Dependent Variable (Pretest) M =Manipulation of Independent Variable O2 = Observation of Dependent Variable (Posttest)
b. Example.
c.
Advantages and Disadvantages, Relative To Posttest-Only Designs.
3.
Two-Matched-Group Design.
a.
The Basic Design.
Matching O (Pretest) Matching
O O
[NOTE: MATCHING SUBSTITUTES FOR RANDOMIZATION.]
b.
Example.
B.
N - Group Designs: The above designs can be generalized to more than two groups.
1.
Example #1: Three-Group Posttest Only Design.
130
R M O R M O R O
Experimental Group I Experimental Group II Control Group
R = Randomization M = Manipulation of Independent Variable O = Observation of Dependent Variable (Posttest)
2.
Example #2: Four-Randomized Group Pretest/Posttest Design.
R O1 M O2 R O1 M O2 R O1 M O2 R O1 O2
Experimental Group I Experimental Group II Experimental Group III Control Group
R = Randomization O1 = Observation of Dependent Variable (Pretest) M = Manipulation of Independent Variable O2 = Observation of Dependent Variable (Posttest)
131
3.
Example #3: Three Matched-Groups Design.
Matching O (Pretest) Matching Matching
M M
O O O
Experimental Group I Experimental Group II Control Group
C.
Factorial Designs: For >1 Independent Variable
1.
Example: 2 factor design.
Hypotheses: Training enhances deception detection. Deceiver preparation enhances detection.
Training Condition Preparation Condition Preparation No Preparation No Training n = 50 n = 50 Training n = 50 n = 50
2.
Example: Three Independent Variables (2x2x5)
Hypothesis 1:
Metaphor facilitates comprehension through increased attention to the message. (e.g., The U.S. is a beacon for democracy.) Source status facilitates comprehension through increased attention to the message. Source physical attractiveness decreases comprehension through decreased attention to the message.
Hypothesis 2:
Hypothesis 3:
132
Source Status: Low Metaphors None Two Four Six Eight Source Status: High Metaphors None Two Four Six Eight
3.
Source Physical Attractiveness Low n = 15 n = 15 n = 15 n = 15 n = 15 High n = 15 n = 15 n = 15 n = 15 n = 15
Source Physical Attractiveness Low n = 15 n = 15 n = 15 n = 15 n = 15 High n = 15 n = 15 n = 15 n = 15 n = 15
Interactions (assumes at least 2 IVS).
a.
Two-Way Interaction Example.
Hypothesis:
Training leads to greater detection of deception, but only when the deceiver is led to believe that severe consequences will occur if the deception is detected.
133
Design and Expected Results:
Training Condition Consequences None No Training I Same as in Cells II and III III Same as in Cells I and II Training II Same as in Cells I and III IV Highest Detection Accuracy Here
Severe
b. Three-way Interaction Example.
Hypothesis:
Training leads to greater detection of deception, but only when the deceiver is led to believe that negative consequences will occur if the deception is detected and if the deception is planned in advance.
Deception: Spontaneous Consequences None Severe
Training Condition No Training Training
Deception: Planned Consequences None Severe
Training Condition No Training Training
Training Helps Here
134
V.
Hypothesis Testing and Threats To Validity. (Please go to handout on page 137.)
VI.
Replication: The only way to be confident.
Theory
VII. Theory and Research.
Research
135
Identify the Independent, Dependent and Mediator Variable In Each Hypothesis
Hypothesis 1. When a message is counterattitudinal, a humorous style will lead to more attitude change than will a serious style because humor will inhibit counter-arguing. An interviewer who initially expresses skepticism about a job candidate, but then endorses that candidate will be more influential in her recommendation than will an interviewer who is positive from the outset. This greater effectiveness is due to the perception that the skeptical interviewer employs more rigorous evaluation standards than the nonskeptical interviewer. Fast talkers will be more persuasive than slow talkers, due to the attribution of intelligence to fast speaking communicators. The door-in-the-face request tactic will lead to greater compliance than a simple request tactic by elevating message recipients selfpresentation concerns. Strong fear appeals are more effective than weak fear appeals because of the effects of fear on subjects assessments of personal vulnerability.
Independent Variable?
Dependent Variable?
Mediator Variable?
2.
3.
4.
5.
136
Some Threats to Validity in Experimental Research*

Internal Validity Definition
Internal validity refers to the ability of an investigator to rule out alternative interpretations of the presumed causal relationship between our manipulated variable(s) (i.e., the independent variable(s)) and the dependent measure. Essentially, questions about internal validity refer to the issue of whether our manipulations were, in fact, responsible for differences we observed in the dependent measure.
Common Threats to Internal Validity

1. History (Uncontrolled Event).
If an event occurs between our pretest and posttest (or between our manipulation of the independent variable(s) and our posttest) which is not part of our experimental treatment, then we have a problem of drawing the conclusion that the difference observed on our dependent measure is due to our manipulation of the independent variable(s). Example: You are interested in the effects of a university-based health promotion campaign on students smoking behavior. During your promotion activities, the U.S. Government begins its own anti-smoking campaign. Randomization can often deal with this threat to internal validity. If all of our subjects have been exposed to the extraneous event, then its effect should be expressed equally across all treatment groups. To use the above example, the effects of the governments efforts should be felt equally by all subjects, including those in the control group. 2. Maturation.
Maturation becomes a threat to internal validity when an observed effect might be due to the respondents growing older, wiser, stronger, more independent, etc. Example: As adolescents grow older, they become more independent and may be more inclined to say no to cigarettes rather than to peer pressure. Randomization will usually solve this problem: maturation effects should be just as strong in ones control group. 3. Testing.
This information has been abstracted from: T.D. Cook & D.T. Campbell (1976). The design and conduct of quasi-experiments and true experiments in field settings. In M.D. Dunnette (Ed.), Handbook of industrial and organizational psychology (pp. 224-246). New York: Rand McNally.
137
Testing may be a threat to internal validity when the very act of taking a test affects the variable being assessed. Example: Our students who take a pretest about their smoking attitudes may come to reconsider their views and habits as a result of the questionnaire experience itself. There are experimental designs which systematically examine the possibility of testing effects. A simple two-group design, for instance, could be substituted for a four-group design involving two experimental groups, only one of which is pretested, and two control groups, only one of which is pretested. 4. Instrumentation.
This threat occurs when our measuring instrument changes between pretest and posttest. We thus get differences on the posttest measure which might be erroneously attributed to the manipulation of the independent variable(s). If measurement is by mechanical means, calibration of the equipment is critical. Even paper and pencil tests might take on new meanings as time passes. Example: In a longitudinal study of developmental aspects of sex roles, an investigator finds that her questionnaire uses terms which have taken on new connotations. 5. Selection.
This threat occurs when the people in the various treatment groups are not identical. Of course, the purpose of randomization of subjects to treatment groups is to prevent this problem. Randomization is normally quite effective, especially when our sample size is large. One procedure which can be used to minimize the possibility of this threat to internal validity is the matched group design. An investigator could also collect demographic and other kinds of information to ensure that randomization is successful. The possibility of a selection bias is particularly great in field research, in which preexisting groups are often assigned to particular treatment conditions. Example: An investigator wishes to examine the effectiveness of a smoking prevention education program. She could not randomly assign subjects to groups because of the diffusion of treatment problem described below. Rather, she will have to assign several schools to the treatment group and several schools to the control group. Despite great efforts to construct equivalent groups, can she really be sure that an experimental group composed of Woodland High School and Dixon High School is equivalent to a control group composed of the students at Davis High School and Winters High School? Probably not. There may be differences in socio-economic status, race and ethnicity, parental educational levels, etc.
138
6.
Attrition.
If particular kinds of individuals drop out of the different treatment groups, then you end up with treatment groups that are not identical, even if they initially were equivalent. Attrition analyses are critical; the investigator must examine the possibility that a different kind of person was more likely to leave the investigation by comparing drop outs to those who remain in the study. Example: Heavy smokers might drop out of the smoking prevention treatment group because they find the educational materials to be threatening, but heavy smokers in the control group will probably remain in the study. Their habit, after all, is not under attack. 7. Diffusion of Treatment.
This threat to validity occurs when the individuals in one treatment become inadvertently exposed to the treatment from another experimental condition. Randomization will not save the experimenter on this one! Example: Your experimental subjects share all of the information you have given them about smoking effects with the control subjects, who happen to be their friends. This kind of problem can be precluded by separating treatment groups for example, by having experimental subjects in Sacramento and control subjects in Culver City. This creates the potential for a selection bias, however. 8. Compensatory Equalization of Treatment.
When an experimental treatment appears to be doing good, administrators may become upset that the control group isnt getting equivalent treatment. They may then do things which serve to equalize the experimental and control groups. In doing so, they may make it impossible to determine if the treatment did in fact produce benefits. Example: The experimental subjects in a study of a new pharmacological agent appear to be made healthier. Pressures mount to administer the drug to control subjects. 9. Compensatory Rivalry.
If the assignment of people to treatment versus control groups is made public, the conditions for social competition are generated. Example: Students in one class room are given a special reading program. The students in the control classroom decide that they will work very hard to show that they are no dummies. The effect is that both groups show performance gains. It thus becomes impossible to have confidence that the treatment did in fact benefit students. 10. Resentful Demoralization of Respondents Receiving Less Desirable Treatments. Instead of adopting a we will show them! attitude (compensatory rivalry), subjects in the control condition (or in other less desirable treatment conditions) may feel defeated or resentful. One effect of resentful demoralization is a change in the behavior of the control subjects, which seriously undermines their usefulness as a comparison group.
139
Example: If Woodland High School learned that they had been denied a neat anti-smoking program, they might become demoralized and conclude that if no one else cares about their health, then why should they care. 11. Inadvertent Differential Treatment of Subjects in Ways Unrelated to Experimental Manipulations. In the process of administering different treatments, it is possible that biases unrelated to the experimental manipulations can be introduced. Example: The health education coordinators who administer one type of anti-smoking education program are more experienced than are the coordinators who administer the other kind of education program. Are observed effects due to the program or to the experience levels of the coordinators? We will never know. Example: Something significant happens during one of the treatment sessions, but not during the other treatment sessions. For instance, one treatment condition is administered at 7:00 a.m.; all others take place at a humane time of the day. Example: One treatment condition includes an obnoxious subject who makes the experience a negative one for everybody in that experimental group.
Construct Validity Definition

Construct validity pertains to the quality with which one has operationalized his or her construct. When a particular operational definition can be construed in terms of more than one construct, we say that the construct is confounded. This is a different issue than internal validity. Internal validity asks: Did manipulated variable X really bring about the observed changes in dependent variable Y? Construct validity asks: Did manipulated variable X really assess construct X, and did dependent measure Y really assess construct Y? Example: In a health communication campaign, was the information responsible for the effectiveness of the treatment, relative to the control group? Or is the result due to the helpful and caring attitude of the health educator, whose approval the students desired? Multiple Operationalism: Since no operational definition provides a perfect fit with the theoretical construct, what are researchers to do? They should use multiple measures of each construct whenever possible. Multiple operational definitions (e.g., a behavioral measure of shyness, a peer measure of shyness, and a self-report of shyness) should share common variance attributed to the construct shyness, and should also be expected differ from each other in unique ways that are irrelevant to shyness.
140
Threats
1. The independent variable did not vary what it was meant to vary.
Example: A high fear appeal message proves not to elicit higher levels of fear in audience members than the low fear appeal. Manipulation checks should be routinely taken to assess the effects of ones manipulations. For instance, when one manipulates the strength of fear in a persuasive message, then one should attempt to obtain measures of experienced fear, perceived vulnerability to the stated threat, etc. Likewise, when an investigator alters descriptions of a sources background to affect perceptions of credibility, then subjects should be asked to make ratings of credibility. Manipulation checks are a necessary feature of any carefully planned experimental design. 2. When manipulating the independent variable, other constructs were also inadvertently manipulated.
Example: A manipulation of source trustworthiness should not involve the alteration of information related to source expertise and power. Example: When manipulating argument quality, you should do so in a way which does not affect argument length (strong and weak arguments should be of equal length). 3. Your dependent measure does not adequately assess the dependent construct.
Example: Your multiple choice test for message comprehension does not really reflect differences among subjects in comprehension. 4. Your dependent measure assesses factors not relevant to the construct it is suppose to measure.
Example: Subjects scores on your multiple choice test of message comprehension reflects, in addition to message comprehension, prior knowledge on the topic, intelligence, test taking abilities, and reading skills.
Statistical Conclusion Validity Definition

Experiments are conducted to make decisions for example, to draw a conclusion about the validity of the hypotheses. We are typically concerned with the mere existence of some phenomenon; an inference about its existence is usually based on inferential statistics and tests of statistical significance. We can go wrong in two ways: Type I Error: We conclude that the phenomenon does exist when it really does not. Type II Error: We conclude that the phenomenon does not exist when it really does.
141
The probability of making these two errors in decision-making can be minimized, but never completely eliminated. These errors can be diagramed as shown in the box to the right.
Reality Phenomenon Exists Phenomenon Exists Phenomenon Does Not Exist Correct Inference Made Phenomenon Does Not Exist Type I Error
Researchers Statistical Inference
Type II Error
Correct Inference Made
Threats: Type I Error

1. Fishing [Data Snooping] and the Error Rate Problem.
When we go fishing for a statistically significant effect (for example, by comparing all possible pairs of means or by examining every correlation among our variables), we increase our chances of making a Type I error unless we make adjustments to our decision-making criteria. In most cases, an investigator will accept as true a phenomenon if there is less than a five percent chance of obtaining a particular effect (such as a correlation of a particular size or a difference among means of a particular magnitude) given an assumption that the sample on which the test was based was taken from a population in which the phenomenon does not exist. Thus, the probability of a Type I error for that particular test is .05; we would say that our effect is significant at the .05 level of statistical significance. The problem comes when we start to make a lot of tests, each with a .05 chance of being a Type I error. One way to deal with this problem is to adopt a more rigorous significance level for each test so that our overall (experiment-wise) probability of a Type I error remains at p < .05.
Threats: Type II Error

1. Limited Statistical Power.
The probability of making a Type II Error increases when we have limited statistical power, due to a small sample size. By analogy, trying to find a real effect with a small sample size is like trying to see something through weak binoculars. 2. Low Reliability of Dependent Measures.
If our dependent measures are not very reliable, they may not be powerful enough to detect real differences in the sample. To continue our analogy, trying to find a real effect with unreliable 142
measures is like trying to see something through dirty binoculars. 3. Low Reliability of Treatment Implementation.
Independent variables can also be unreliable. The experimental treatment is unreliable if it is implemented in a nonstandard way. For example, if one experimenter implements the independent variable differently than other experimenters, unreliability has been introduced. Likewise, if any one experimenter manipulates the treatment in different ways for different people we also have nonstandard implementation of our independent variables. This lack of standardization introduces noise into the experiment, and makes it difficult to detect real treatment effects.
External Validity Definition

External validity refers to the extent to which we can generalize the results we obtained from a particular sample to (1) the population from which the sample was drawn and (2) other populations.
Threats
1. Interaction of Treatments and Treatments.
When two treatments have been introduced in a study (as is done in factorial designs), we may not be able to generalize the effects of one of the treatments to situations in which the other treatment is not present. Example: You find that strong fear appeals work in a study in which both fear appeal strength (strong versus weak) and evidence type (personal testimony versus expert testimony) have been manipulated. Can you generalize to other situations involving no use of evidence? 2. Pretest Effects on Responses to the Treatment.
If we have used a pretest, is it possible that this testing affected how people responded to the experimental treatment? Example: In a study on peoples responses to different kinds of requests for help, subjects are asked to complete an Altruistic Tendencies Scale to make sure that the experimental and control groups do not differ on general helpfulness. If I observe differences among my control and experimental groups, there is the possibility that the pretest may have sensitized experimental subjects to the requests and altered their behavior. We are thus left with the question of whether the results could be generalized to situations in people have not been led to examine their altruistic tendencies prior to receiving a request for assistance. 3. Generalization To Other Kinds of People.
Can the studys results be generalized to groups not included in the design: e.g., to people of a different sex, age, race, social group, culture, educational background, socio-economic class, etc.?
143
Given resource limitations, experimental researchers must often make use of convenience samples, which are not typical of the population at large. For instance, much research has been conducted on college undergraduates, who are on average younger, brighter, whiter, better educated, and more affluent than mainstream America. 4. Generalization to Other Kinds of Settings.
Do the effects observed generalize to settings not examined in the experiment? Example: If participatory decision-making was found to increase employee satisfaction in a study involving small organizations, would the same effect be found in large organizations? Military organizations? Not-for-profit organizations? Example: If I find that funny television ads gain more attention from consumers than serious ads, can I generalize this finding to print and radio? 5. Generalization to Other Times.
Does the effect of a particular treatment generalize to other times? Example: Can we generalize the results of a study conducted 15 years ago on the causes of divorce to the present time? Example: Can we generalize the results of a study on gender differences in compliance within work groups that was conducted 30 years ago to the present time?
144
Experiments in Communication: Some Examples

Earl Examines The Effects of Being a Potty-Mouth on Persuasion
Earl Experimentalist is curious about the effects of profanity on persuasion. Based on research on the double standard of swearing, he hypothesizes that: Profanity has no effect on the persuasiveness of a male source, but has a negative effect on the persuasiveness of a female source. This effect is due to the debilitating impact of female cursing on attention to the message. Earl decides to conduct an experiment to test his hypothesis. He identifies 200 people and randomly assigns half of them to his experimental group and the other half to his control group. Subjects in the control group are asked to read an editorial purportedly published in Mother Jones. The editorial is highly critical of the U.S. Governments failure to deal with toxic waste sites. Subjects in the experimental group read the same editorial, except that five naughty words were incorporated into it. One-half of the subjects in the experimental and control groups were led to believe that the editorial was written by Sally Smith, an environmental scientist. The other subjects in the experimental and control groups were told that the editorial was written by Sam Smith, an environmental scientist. Thus, 50 subjects heard a profane message from a female source, 50 heard a profane message from a male source, 50 heard a clean message from a female source, and 50 heard a clean message from a male source. The design looked like this: Sex of Source Present Female Male n=50 n=50 Profanity
Absent n=50 n=50
After reading the editorial, each subject reported his or her attitude toward the Governments cleanup efforts on a series of Likert scales. Earl summed across these responses to create a measure of subjects attitudes toward the Governments cleanup policy. In addition, each subject was asked to list all of the arguments he or she could think of that were presented in the editorial. Finally, each subject made an estimate of the number of dirty words in the editorial and indicated if the source of the message was male or female.
145
1.
Identify the Independent Variable(s) examined by Earl:
2.
Identify the Dependent Variable(s):
3.
Was a mediating variable assessed in this study? If so, what was it?
4.
Were manipulation checks taken?
5.
What type of experimental design did Earl use?
6.
How did Earl attempt to hold constant potential factors that could make interpretation of the results difficult?
7.
Suppose Earl found that (a) subjects who read the editorial from the female source were more persuaded when she did not swear; (b) subjects who read the editorial from the male source showed the same amounts of persuasion in the profane condition as in the control condition; and (c) subjects who heard a profane message recalled the same number of arguments as subjects who heard the nonprofane editorial. Does this pattern of results support the hypothesis?
8.
Has Earl taken into account the possibility of interactions?
9.
In what ways is the generalizeability of this study limited?
146
Dr. Bell Examines Fund-Raising Strategies

Dr. Bell is interested in the individual and joint effects of two strategies for getting people to donate money to a local AIDS charity: foot-in-the-door and pregiving. With the foot-in-the-door strategy, the solicitor asks the target person to perform a small task which should lead him or her to conclude that the cause is valuable. This person is then asked to make a monetary donation. Dr. Bell operationalized foot-in-the-door (FITD) by asking respondents to sign a petition encouraging the governor to continue his support of local AIDS prevention programs, and then requesting for a donation of money. Self-perception theory predicts that the person who has just signed the petition should come to see himself/herself as having a positive attitude toward such programs (I signed the petition, so I must be for this cause), and should thus be more willing to make a donation. With pregiving, the target person is given something prior to being asked to perform the target request. Dr. Bell operationalized this strategy by giving some subjects a brochure on AIDS which was described as containing valuable information about this terrible disease. Pregiving is assumed to work by making people feel obligated to give; after all, the norm of reciprocity tells us to give to those who have been kind to us. Dr. Bell was also interested in the effects of combining these two strategies, both of which have been effective. He believed that there would be an effective and an ineffective way to combine these tactics. This order effect can be described as follows: (1) People who initially signed a petition, then given a brochure, and finally asked for money would donate the most. These people should not only come to see themselves as supporting this cause (the self-perception effect), they should also see themselves as owing something to the solicitor because of the gift of the brochure (the norm of reciprocity effect). (2) People who were given a brochure prior to signing a petition, and then asked for money were not expected to be charitable. This is because the two strategies should cancel each other out when presented in this order. The person may feel an obligation to the solicitor because of the gift of a brochure, but this obligation should be satisfied by the signing of the petition. Furthermore, the person would come to see his/her signature as a payback for the brochure, not as an indication that s/he really supports this cause. A total of five conditions (four experimental groups and one control group) were included in the research design: Control: FITD: These people received only the target request i.e., the request for money. These people were asked to sign a petition, and then received the target request for money.
Pregiving: These people were given the brochure and then asked for money. Order I: These people were asked to sign a petition, were then given the brochure, and then received the request for money. These people were given the brochure, then asked to sign the petition, and then heard the request for money.
Order II:
147
A script was carefully devised for each condition to ensure that extraneous factors were not introduced into the study. The research teams practiced extensively until they were able to administer the memorized scripts flawlessly and naturally. The investigators hypotheses can be summarized as follows: (1) The FITD strategy was expected to be more effective than the control (target request only) strategy and the Order II strategy. (2) The Pregiving strategy was expected to be more effective than the control strategy and the Order II strategy. (3) The Order I strategy was expected to be more effective than all of the other message strategies. The investigator had three teams of solicitors, each composed of two adults, go door-to-door to administer the five message strategies outlined above. The experiment was conducted in two suburban Northern California communities. Each team used a random procedure to assign each household to one of the five conditions. The strategy for that condition was then implemented. Thus, all research teams administered each of the five conditions on an equal number of occasions. One of the members of each team served as requester, while the other member performed the role of record-keeper. The record-keeper was responsible for the random assignment of a household to one of the conditions, and also recorded the amount of money donated (the range proved to be $0 to $26).
1.
Identify the Independent Variable in this study:
2.
What is the Dependent Variable?
3.
Were any Mediating Variables assessed?
4.
What type of design did Dr. Bell use?
5.
What did Dr. Bell do to hold constant potential factors that could make interpretation of the results difficult?
6.
What could he have done to increase control?
7.
Suppose Dr. Bell had used five research teams, and had assigned one condition to each team. Thus, team one could have administered all of the control condition requests, team two could have been responsible for the FITD requests, etc. Would this have been a good or bad idea?
8.
148
Suzy Looks at Liking

Suzy Scholar is interested in the effects of Source Likeability on persuasiveness. She knows that the research generally suggests that likeable sources are more persuasive, but she wants to know why the effect occurs. Specifically, she believes that a likeable source is generally more persuasive because audiences give more attention to his or her message. However, unlike other researchers, Suzy does not assume that increased attention necessarily increases ones persuasiveness. Greater attention should increase persuasiveness only if ones arguments are strong. In the case where ones arguments are weak, the increased attention given to likeable sources may actually decrease their persuasiveness. Suzy hypothesizes: Likeable sources will be more persuasive than unlikable sources when their arguments are strong, but less persuasive when their arguments are weak. This effect is due to the greater attention given to likeable sources. Suzy constructs two written messages on the need for federal support of child care for working mothers. The message is purported to have come from a speech given by a noted federal official, James Johnson. One of the messages is filled with arguments that have been determined by pretesting to be perceived as very strong. The other message is based on arguments determined in advance to be very weak. Suzy selects 100 subjects from a larger pool of subjects who had previously indicated neutral attitudes on this topic. She then randomly assigned 50 subjects to the strong arguments condition and 50 subjects to the weak arguments condition. Half of the subjects in the strong arguments condition and half of the subjects in the weak arguments condition are given information about the source designed to lead them to believe that he is a despicable person. The biography describes Johnson as someone who is disliked by his subordinates, has been divorced four times, and is known around the capital city as a man who isnt afraid to step on people to get things accomplished. The remaining subjects were led to believe that Johnson is considered a saint by his workers, has been married to the same person for 35 years, and is known around Washington D.C. a man who cooperates effectively with people to get things accomplished. Source Likeability Argument Strength
Strong Low High n=25 n=25
Weak n=25 n=25
After reading the speech, subjects were asked to report on a series of semantic differentials their evaluation of the need for a federal program to support child care for working mothers. They also rated how strong they thought Johnsons arguments were in support of his positions. In addition, they were asked to write down all the arguments they could remember that were used by Johnson. Finally, they made a rating of how much they liked Johnson on a single 10point scale.
1.
Identify the Independent Variable(s) in this study:
149
2.
What is the Dependent Variable?
3.
Is there a Mediating Variable?
4.
Were any manipulation checks made? If so, what were they? Does reliance upon pre-testing in any way mitigate the need for manipulation checks?
5.
What type of design did Suzy use?
6.
What did Suzy do to hold constant potential factors that could make interpretation of the results difficult?
7.
What could she have done to increase control?
8.
150
SURVEY RESEARCH
I. Introduction.
A. What is a Survey?
A survey involves the use of a self-administered questionnaire and/or personal interview methods to collect information from a sample of respondents for the purpose of describing the larger population from which these people were selected.
B.
Reasons for Conducting Surveys.
1.
Demographic Analysis.
2.
Measurement of Social Attitudes and Beliefs.
3.
Consumer Research.
4.
Policy Formation.
5.
Academic Research.
C.
The Components of Surveys.
1.
Research Objective.
2.
Sampling Design.
151
3.
Questionnaire Design.
4.
Question Administration
a.
Self-Administered.
b.
Personal, Face-to-Face Interview.
c.
Telephone.
5.
Data Analysis.
II.
Self-Administered Mail Surveys.

A. Some Advantages of Mail Surveys.
1.
Low Cost.
a.
Reduced Personnel Needs.
b.
No Travel Costs.
2.
Nationwide Mail Surveys are No More Costly than Local Surveys.
3.
Effective for Embarrassing and/or Self-Incriminating Topics.
B.
Some Disadvantages.
1.
Sampling Frames are Often Incomplete.
152
2.
Not Appropriate for Populations with Low Literacy Rates.
3.
Not Particularly Effective When Open-Ended Questions Need To Be Asked. (Inability to Probe.)
4.
Not Appropriate For Complex Topics.
5.
Not Appropriate For Complex Survey Formats Involving Many Contingency Questions (Skip Patterns).
6.
Not Appropriate When Survey Must Be Completed Quickly.
7.
Greater Chance of Low Response Rates.
C.
A Solution to the Response Rate Problem: Dillmans Total Design Method (TDM).
1.
Why People Respond: Three Factors.
a.
Attention to Request.
b.
Positive Reward/Cost Ratio.
i.
Minimizing Costs.
ii.
Maximizing Rewards.
153
c.
Obligation.
i.
Norm of Reciprocity.
ii.
Worthy Sponsor.
2.
The TDM Procedure. (See handout on page 163.)
a.
Questionnaire Construction Guidelines.
b.
Implementation Guidelines.
c.
Cost Estimates. (See guide on page 167.)
3.
Evidence of TDM Effectiveness.
D.
Effectiveness of Other Strategies for Increasing Response Rates.
1.
Deadlines.
2.
Incentives.
a.
Monetary Incentives.
b.
Nonmonetary Incentives.
E.
Other Uses For Self-Administered Questionnaire Surveys.
1.
Simultaneous Assessment of Multiple Members of a Household.
2.
Supplements to Face-to-Face Interviews.
154
3.
Drop-Off and Pick-Up Procedures.
4.
Simultaneous Administration to Large Groups.
III.
Standardized Interview Surveys.

A. Standardization.
1.
Reliability Requires A Standard Procedure of Interviewing.
A standardized questionnaire results in a standardized study only if the questionnaire is administered in a stable fashion.
a.
Standardization Across Interviewers.
b.
Within-Interviewer Standardization.
B.
Uses of Standardized Interviewing.
1.
When
2.
When Not.
a.
Exploratory (Problem-Definition) Research.
b.
Case Studies.
C.
Pros and Cons of the Personal, Face-to-Face Interview.
1.
Some Advantages.
155
2.
Some Disadvantages.
D.
Guidelines For Interviewer Behavior.
1.
Read Questions as Worded.
2.
Standardize the Probing of Inadequate Answers.
a.
Clarification of Meaning.
b.
Vocabulary Problems.
c.
Unacceptable Response Category Invoked.
d.
Probing Open-Ended Questions.
e.
Probing Dont Know Responses.
3.
Standardized Recording of Answers. (See handout on page 168.)
4.
Interpersonal Neutrality Must Be Maintained.
a.
Professional Distance.
b.
Disapproving Responses Must Be Avoided.
c.
Approving Responses Must Be Avoided.
E.
Training the Respondent
156
1.
Initial Briefing. Interviewers should begin the interview with a statement that informs the respondent how the interview will be conducted and what is expected of the respondent.
SAMPLE INITIAL BRIEFING: Since many people have not been in an interview exactly like this, let me read you a paragraph that tells a little bit about how it works. I am going to read you a set of questions exactly as they are worded so that every respondent in the survey is answering the same questions. Youll be asked to answer two kinds of questions. In some cases, youll be asked to answer in your own words. For those questions, I will have to write down your answers word for word. In other cases you will be given a list of answers and asked to choose the one that fits best. If at any time during the interview you are not clear about what is wanted, be sure to ask me. Please be assured that there are no right or wrong answers to any of the questions I will be asking you.
2.
When the Respondent Does Not Fulfill His/Her Role: Common Problems and Standard Training Responses. (See handout on page 169.)
F.
Training and Supervision of Interviewers: Some Basic Principles.
1.
Training Must Involve Active Participation on the Part of the Interviewers.
2.
Training Takes Time.
3.
Training Must Focus on Both the Hows and the Whys of Standardization.
4.
Ongoing Supervision of Interviewers is a Must.
IV.
Phone Surveys.
A. Telephone Trivia. (See handout on page 171.)
B.
Some Advantages of Phone Surveys.
157
1.
Lower Costs, Relative to Personal Interviews.
2.
Explicit Sampling Frame Not Necessary (Random Digit Dialing).
3.
Access to Most of General Population.
4.
Short Data Collection Period.
5.
Interviewer Available for Clarifying, Motivating Participation, Etc.
6.
Quality Control Through Supervision Is Easily Accomplished.
7.
Higher Response Rates than Usually Obtained with Mail Surveys.
C.
Some Disadvantages of Phone Surveys.
1.
Members of the Target Population Who Have No Phones Will Not Be a Part of the Study Population.
2.
Response Rates Will Usually Be Lower than Those Obtained Through Personal Interview Surveys.
3.
Questionnaire Length Constraints.
4.
Measurement Procedures Cannot Take Advantage of Visual Aids.
5.
Interviewer Observations Are Constrained.
158
D.
Generating Sampling Pools Through Random Digit Dialing.*
1.
Define the Geographical Boundaries of the Target Population.
2.
Gather the Area Codes and Prefixes Used Within the Geographical Region.
3.
Examine Need for Screening (Eligibility) Questions.
4.
Assign Random Suffixes for Each Prefix to Create Seed Numbers.
a.
Common Techniques For Suffix Generation.
i.
Let the Computer Do it.
ii.
The Add-A-Digit Procedure.
b.
Why Telephone Directory Sampling is a No-No.
What follows is just one of several approaches to RDD.
For a detailed discussion of this issue, see R.A. Bell. (1989). A comment on Krokoffs Recruiting Representative Samples for Marital Interaction Research. Journal of Social and Personal Relationships, 6, 231-234.
159
5.
Randomly Select an Equal Number of Seed Numbers from Each Prefix.)
RDD Is Self-Weighting Exchange % Working Numbers 50% 45% 20% 12% 38% 41% 10% No. of Calls Made 200 200 200 200 200 200 200 No. of Interviews Likely To Be * Completed 100 90 40 24 76 82 20
444445447449555556 558*
Expected value of the mean number of interviews completed over a series of independent samples.
6.
Using Working Seed Numbers to Generate a Block of New Numbers.
D.
Generating Samples When the Sample Frame is Explicit.
1.
Simple Random Sampling.
2.
3.
Stratification By Prefix.
160
E.
Processing Sampling Pools: Call Sheets and Disposition Records. (See handout on page 173.)
F.
Contact.
1.
Introducing the Survey.
a.
Mandatory Information.
Verification of phone number Who is calling? (Interviewers full name.) Identify the research organization or sponsor of the research. Study topic. Selection procedures. Protection of respondent confidentiality procedure. Approximate length of interview. Opportunity to refuse. Opportunity to ask questions. b. Optional Information.
Offer of incentive. How results will be used. Format of interview (open or closed items). Any unusual expectations of the respondent. Detailed discussion of research purpose.
c.
Sample Introductory Message. (See example on page 175.)
2.
Selecting Respondents: Household Sampling Revisited.
161
3.
Fallback Statements: Standardized Answers to Common Questions.
Explanation of Survey Use of Findings Random Digit-Dialing Respondent Selection Procedure Contact Person: Name and Phone Number of Project Director.
4.
Geographical Screening Question.
G.
Special Considerations for Phone-Administered Questionnaires.
162
Dillmans Total Design Method For Mail Surveys*
Obligation
Attention To Request Favorable R/C Ratio Norm of Reciprocity Worthy Sponsor
Questionnaire Guidelines
1. Designed as small booklet (6 X 8 inches). Does not seem imposing. Marketing materials wont look legitimate. Get people to want to look over the questionnaire.
2.
No slick reproduction; avoid resemblance to an advertising brochure.
3.
Cover (first) page should have a catchy title and an attention-grabbing illustration. No questions, please. Instructions go here. No questions on the last page (back cover). Use this page to invite additional comments and express appreciation for the respondents participation. The first few questions should be the most interesting, topic-relevant, and respondent-relevant. Give special attention to the first question; it should apply to everyone.
4.
Less imposing. (Most people will initially look at the first and last page.)
Shows you really care about their opinion and are nice.
5.
This might be fun (rewarding)! This study also appears to be potentially important (selfinterest)!
These people are asking about an issue which affects me personally!
Don A. Dillman. (1983). Mail and other self-administered questionnaires. In P.H. Rossi, J.D. Wright, & A.B. Anderson (Eds.), Handbook of Survey Research (pp. 359-377). Orlando: Academic Press.
163
Obligation
Attention To Request 6. Use lowercase letters for questions and uppercase letters for answers. Write questions so that responses can be recorded in a straight vertical line (prevents missed questions). Dont split a question between pages. Use transitions to signal change of focus, justify the importance of a new set of questions, etc. Use visual cues (arrows, indentations, spacing) as necessary to reinforce directions. Favorable R/C Ratio Ease of use. The wellwritten and visuallyinviting questionnaire will not seem like a very difficult task to complete. Norm of Reciprocity Worthy Sponsor
TDM Implementation Procedures

1. A one-page cover letter is included to describe why the study is socially useful, why each respondent is important, and who in the household should respond. Cover letter should be printed on the sponsoring organizations official stationery. The exact mailing date should be printed on the cover letter. Respondents name and address are typed on the letter; generic salutations such as Dear Friend are avoided. The sender (project director) should sign the letter personally with blue ink; press hard enough to leave an indentation. Avoid appearance of an impersonal mass mailing. If they took the time to direct this letter specifically to me, I should help them out. Socially important work. I might benefit. I should help them out because they are doing work which will help us all.
2.
These are the kind of people who deserve support.
3.
164
Obligation
Attention To Request 4. The purpose of the identification number printed on the questionnaire should be explained. Favorable R/C Ratio Respondents might assume the worse when they see the ID number and conclude that participation is too risky. Seems less imposing. No postage must be paid by respondent. Norm of Reciprocity Worthy Sponsor
5.
The mailout packet, which includes the cover letter, questionnaire, and postage-paid business reply envelope (6 X 3 inches) is mailed in a monarch-size envelope (7 X 3 inches). The respondents name and address should be individually typed on the mailout envelope. Never use address labels. This does not appear to be a mass mailing. Maybe I should open the letter instead of trashing it.
6.
7.
Affix first-class postage to the mailout letter.
It cost you something to get this to me. I should help * you out.
Research has shown that a postage stamp is no more effective than the U.S. Post Offices nonprofit permit stamp.
165
Obligation
Attention To Request 8. After one week, send reminder postcard to all recipients of the questionnaire. Three weeks after the first mailout, send a second cover letter and questionnaire to everyone who has not responded. Seven weeks after the first mailout, a second cover letter complete with another questionnaire is sent by certified mail. They might have forgotten about the study or lost the first questionnaire. Favorable R/C Ratio Norm of Reciprocity Theyre really putting a lot of energy into getting my views. The least I can do is help them out. Worthy Sponsor
9.
Certified mail thats costly. The least I can do is respond.
166
Estimating Costs in Mail Surveys

What follows is a list of the various costs associated with mail surveys. Some of these items may not be necessary for your particular survey. Although the following list is extensive, you may have costs not included below.
General Costs
Draw systematic sample or purchase a sample listing from a consulting firm. Purchase mailout envelopes. Purchase business reply envelopes. Print Questionnaire. Graphics design for cover. Telephone (toll charges).
Supplies (staples, printer supplies, software, computer supplies, etc.). Type, proof, and store names in computer. Professional supervision of clerical staff. Professional consultation fees (data analysis, survey design, etc.)
First Mailout
Print cover letter. Address letters and envelopes. Postage for mailout.
Prepare mailout packets. Postage for returned questionnaires (business reply envelopes). Process, precode returns.
Postcard Follow Up
Postcards (Purchase or Print) Address postcards Prepare mailout
Process, precode returns Postage for returned questionnaires (business reply envelopes). Postage for returned questionnaires (business reply envelopes). Process, precode returns.
Second Mailout
Print cover letter. Address letters and envelopes. Postage for mailout. Prepare mailout packets.
Third Mailout
Print cover letter. Address letters and envelopes. Postage for mailout.
Prepare mailout packets. Postage for returned questionnaires (business reply envelopes). Process, precode returns.
167
Standardized Interviewing
Recording Answers During the Standardized Interview In the standardized interview, the goal is to obtain records of the interviewees responses in a way which does not involve interviewer judgments. The following guidelines should be observed at all times:
Question Form Open Factual Write down all information relevant to the questions objectives. Closed Check off the answer chosen by the respondent. If the respondent is not certain which category fits, the question should be treated as an open-ended question and the interviewees opinion should be written down verbatim. The final decision about how to treat the answer should be made back at the office during the coding operation. Opinion Record answer verbatim. Paraphrasing and summaries are to be avoided. Check off the answer chosen by the respondent. The respondent should be probed until an answer is given. No category should be checked off unless an answer is given.
Information Requested
168
Training the Respondent: Standard Responses to Common Problems
Problem QUESTION MAY HAVE ALREADY BEEN ANSWERED BY RESPONDENT
Interviewer Response The next question is one you have already dealt with to some extent. However, the way the interview works is that I need to have you answer each question specifically, so that we can compare the answers you give with the answers everyone else gives. Also, sometimes we find the answer is different to a specific question, even though it seems that the question has been answered before. So, let me read the question as it is worded here, and I would like you to give me the answer to make sure we have it right. I see what your problem is with the question. Even though these questions are carefully tested, sometimes we have one that is not quite clear to some people, or which does not quite fit everybodys situation. Again, though, the way a survey works, we need peoples best answers to the questions as they are written. That way we can compare your answers with other peoples. If we change the question for each respondent, we wouldnt be able to analyze the answers. Let me read the question again, and you give me the best, most accurate answer you can, given the way it is written. With this kind of question, answers are analyzed according to which of these alternatives people choose. I need to have you choose one of these specific answers so that we can compare your response with those that others give. We know that in some cases none of the answers will fit the way you feel exactly; but other people will have that problem, too. The important thing is that we keep the question-and-answer process consistent across everybody, so we can see similarities and differences in the answers people give.
LACK OF QUESTION CLARITY
DISSATISFACTION W ITH ALL RESPONSE CATEGORIES
169
RESPONDENT FEARS RESPONDING INCORRECTLY
Well, we would like it if you would make your very best estimate. Even though it may not be exactly right, no one is in a better position than you are to make this estimate. Just do the best you can. On factual questions, things like how many times you have seen a doctor or been in the hospital, it is fine for you to get help from anyone who can be helpful, because we want the most accurate information we can get. However, when we ask for somebodys feelings or opinions, there really is no one except you who can give us that answer. Again, it is a matter of being consistent in how we treat everybody. I will talk about anything you want after the interview, but not before it is over. The reason is that we have found that in some cases when interviewers give their opinions and ideas during an interview, we influence the answers we get. This whole interview process is set up so that the only thing that influences the answers if your situation and what you have to say.
A NONRESPONDENT W ISHES TO HELP THE RESPONDENT ANSWER AN OPINION QUESTION
RESPONDENT W ANTS TO KNOW

HOW THE INTERVIEWER FEELS ON AN ISSUE
170
Telephone Trivia
Telephone surveys became justifiable in the United States in the 1970s, when the proportion of U.S. households with telephones finally exceeded 90%. America is connected. Approximately 98% of all households now have connected phone lines. Approximately one-third of all U.S. phones are for nonresidential uses (business and government). Nearly one-half of households will soon have two or more telephone lines. Those people without phone lines are disproportionately. . . single adults minorities undereducated low income employed in nonmanagerial/nonprofessional positions When a telephone is answered, the median number of rings it will take is about 3-4; more than 90% of the time, an answered telephone will be picked up within seven rings. The proportion of households with unlisted numbers has been estimated to be between 40-50%; this proportion is increasing. Who has unlisted numbers? The proportion of people with unlisted numbers is much higher in Urban areas; unlisted numbers in rural areas are very rare. Believe it or not, there is a greater tendency for lower-income people to have unlisted numbers than higher-income individuals. In particular, people holding professional or managerial positions are four times more likely to be listed in the directory than individuals without such positions. College graduates are four times more likely to be listed than nongraduates. The proportion of white households with unlisted numbers is lower than the proportion of minority households. For example, blacks are 250% more likely to have unlisted numbers than whites.
171
Women are more likely to have unlisted numbers than men. Single adults are more likely to have unlisted numbers than married adults. Divorced and separated individuals are four times more likely to have unlisted numbers than married individuals. Single-parent families are also more likely to have unlisted numbers than two-parent families. Young people are more likely to have unlisted numbers than senior citizens. Fear of crime is strongly and positively correlated with having an unlisted number.
172
Phone Survey Record-Keeping: Sample Call Sheet
Project Title: ____________________________________________ Telephone Number (__ __ __) __ __ __ __ __ __ __ Contact Attempt Questionnaire Number: ___ ___ ___ ___ Disposition Code Interviewer I.D. #
Date
Time
1 2 3 4 5 6
___ / ___ ___ / ___ ___ / ___ ___ / ___ ___ / ___ ___ / ___
___:___ ___:___ ___:___ ___:___ ___:___ ___:___
a.m. p.m. a.m. p.m. a.m. p.m. a.m. p.m. a.m. p.m. a.m. p.m. COMMENTS
___ ___ ___ ___ ___ ___ ___ ___ ___ ___ ___ ___
___ ___ ___ ___ ___ ___ ___ ___ ___ ___ ___ ___
1 2 3 4 5 6
_________________________________________________________ _________________________________________________________ _________________________________________________________ _________________________________________________________ _________________________________________________________ _________________________________________________________
173
Disposition Codes*
No Contact 10 11 No answer after 7 rings Busy, after one immediate redial Answering machine (residence) 23 13 Household language barrier Describe language needs, if possible, on Comment Line of Call Sheet 14 15 15H Nonresident answered Refusal Refusal: hung up without comment 20 21 22 12
Not Part of Target Population Disconnected or Not Working 30 Temporarily Disconnected Nonresidential Number (Business, Govt, etc.) No one meets eligibility criteria 33 31
Contact
Contact only Selected Respondent Not Currently Available Selected Respondent Not Available During Survey Period Selected Respondent Not Available Due to Health Language Barrier with Selected Respondent Describe language needs, if possible, on Comment Line of Call Sheet
32
34
35
Refusal by Selected Respondent Partial Interview
36
36R Partial Interview; Refusal to Finish 37 Completed Interview
The disposition codes described here are based on the system reported by P.J. Lavrakas. (1987). Telephone survey methods: Sampling, selection, and supervision. Newbury Park, CA: Sage. For a less detailed disposition coding system see J.H. Frey. (1983). Survey research by telephone (p. 162). Newbury Park, CA: Sage.
174
Sample Introductory Message*

To Person Who Answers:
Hello. Is this __ __ __ - __ __ __ __ ? [phone number] This is __________ calling from the Telephone Survey Center of the University of the Redwoods. We are conducting a survey of Northern California residents on their opinions and perceptions of the quality of life in the state. Your number was selected at random by our computer. The way this survey works, the person I need to talk with is not necessarily the person who answers the phone. For this interview, the I need to talk with the oldest/youngest male/female aged 18 or older in your household. Would this person be available? IF PERSONS WHO ANSWERS IS RESPONDENT: We have just a few questions we would like to ask of you. Your answers will be confidential and the interview will take only about 5 minutes. Can you help us out? Thank you. Please feel free to ask questions at any time.
TO RESPONDENT (ONCE S/HE HAS BEEN BROUGHT TO THE PHONE): Hello. This is __________ calling from the Telephone Survey Center of the University of the Redwoods. We are conducting a survey of Northern California residents on their opinions and perceptions of the quality of life in the state. Your number was selected at random by our computer. We have just a few questions we would like to ask of you. Your answers will be confidential and the interview will take only about 5 minutes. Can you help us out? Thank you. Please feel free to ask questions at any time.
IF RESPONDENT IS NOT AVAILABLE: Could you please give me the name of the person I have asked to speak with? __________ When would be a good time for me to call __________. [Enter suggestions on Comment line of Call Sheet.]
This example relies upon the Hagen-Collier Respondent Selection Procedure
175
ETHNOGRAPHY OF COMMUNICATION
I. Overview.
A. The Ethnographic Approach.
1.
Ethnography: The analysis of a culture in an effort to understand the world view of the members of that culture.
2.
Culture: The accumulated knowledge that natives use to interpret their experiences and generate behavior.
3.
Ethnography as Method.
a.
Participant Observation.
b.
Ethnographic Interviews: Open-ended and emergent logic.
c.
Focus Groups.
d.
Analysis of Documents and Other Texts.
4.
Types of Ethnographies. (See handout on page 184.)
B.
Recent Examples. (See handout on page 186.)
C.
Assumptions.
1.
Theoretical Foundations.
2.
Propositions.
176
a.
People act toward things on the basis of the meaning that those things have for them.
b.
Meanings are cultivated through social interaction with others.
c.
Meanings are generated and modified through an interpretive process used by the person dealing with the things s/he encounters.
3.
Some other implications.
a.
Shared identities are created and maintained through communication.
b.
When interpretations differ, individuals must negotiate the meanings of things.
c.
Behavior is freely chosen.
d.
We dont reflect much upon our meanings unless they are in some way questioned.
D.
The Stance of the Ethnographer.
1.
Enter into the culture with an attitude of complete ignorance.
2.
We learn from the members of that culture; they are the teachers and the ethnographer is the student.
3.
Since a culture is a system of meanings, we must study the cultures symbols.
4.
The ethnographer can learn another culture.
5.
The ethnographer must make inferences, and must subject these inferences to the judgments of native informants.
177
6.
Because physical actions have no inherent meanings, the ethnographers observations must be treated as pre-data until confirmed by native informants.
E.
Justifying Ethnographic Research.
1.
Understanding humans for its own sake.
a.
Accounting for regularities which exist across cultures.
b.
Explaining variations (cultural differences).
2.
Identifying the cultural boundaries of social and psychological theories of human behavior is an important goal.
3.
Understanding problems in intercultural communication.
4.
Solving social problems.
II.
Guidelines In the Planning Fieldwork.

A. Be Sensitivity to Ethnocentric Tendencies.
B.
Be Aware of the Assumption of Cultural Uniformity.
C.
Ethnographer Fit (Compatibility) With the Culture.
D.
Anticipate Culture Shock.
E.
Preparation For Record Keeping. (See figure on page 188).
178
1.
The Observation Journal.
2.
Personal Journal.
3.
Interview Journal.
F.
Research Plan.
1.
Ethnographic Sampling.
2.
Field Behavior: How Will You Be Interacting with the Natives.
3.
Location of Activities: The Communication Matrix (page 190).
4.
Limiting the Scope of Research.
III.
Selecting Informants.
A. Some Sampling Procedures.
1.
Network Sampling.
2.
Network Analysis
3.
Area Sampling.
4.
Recognize the Value of Talking with the Very Successful and the Very Unsuccessful.
179
B.
Recognize that Not all Informants will Prove to be Useful.
IV.
Observation.
A. Some Barriers to Effective Participant Observation.
1.
Language Difficulties.
2.
Ethnographer Ethnocentrism.
3.
Ethnographer Physical Appearance.
4.
Customs.
B.
Types of Observation.
1.
Overt Versus Covert.
a.
Overt.
b.
Covert.
2.
Level of Systematization.
a.
Descriptive: Understanding the big picture.
b.
Focused: In search of answers to specific questions.
c.
Selective. In search of specific attributes of a behavior, activity, routine, practice, and the like.
180
C.
Role of Observation in Ethnography: Observation as Predata.
D.
Reasons for Observation.
1.
No Adequate Native Language Exists To Describe An Aspect Of The Culture.
2.
Much Knowledge Is Tacit.
3.
Unwillingness Of Informants To Speak About A Certain Matter.
4.
Cultural Traditions Which Make Getting Informal Disconfirmation Difficult.
5.
To Understand Better The Use Of Symbols.
E.
Recording Observations.
1.
The Observation Journal.
2.
Film and Photography.
V.
The Ethnographic Interview.

A. Preparing the Interview Guide.
B.
Phases of the Interview. (See handout on page 191.)
1.
Contact.
181
2.
Pre-Interview.
a.
Project Explanation.
b.
Recording Explanations.
c.
Native Language Explanations.
d.
Interview Explanations.
e.
Question Explanation.
3.
The Interview.
a.
Question Types.
i.
Descriptive Questions: Requests for Descriptive Information.
ii.
Structural Questions: Requests for Information Components of the Cultures Symbol System.
About
the
iii.
Contrast Questions: Request for Information About How Cultural Symbols Differ.
b.
Progressions.
C.
Question Strategies.
1.
What to Ask?
a.
Informants Expertise.
b.
Flexibility.
182
c.
Focus.
2.
Order.
3.
Question Checklist.
183
Typology of Ethnographies*
The categories that follow are not mutually exclusive.
Processual Ethnography Holistic Ethnography

An extensive, thorough description of an entire cultural group. These ethnographies can be extensive or sketchy (e.g., book-length or article/chapter length).
Holistic Ethno-historical Ethnography

The cultural reality of the ethnographic present is described as the historical result of events in the past.
Holistic Ethnography as a System of Rules

This perspective is influenced by structural linguistic theory and aims to describe a culture by stating the rules one must follow to behave like a native. Examples include ethnographies of kinship terminologies and ethnographies of the decision-making process of a culture.
Ethnography of Daily Activities and Cycles

Although daily activities are discussed by most ethnographers, this approach treats the day as the basic unit of observation and description. This approach would also include descriptions of the yearly cycle of activities.
Biographical Ethnography
These life histories, which include biographies and autobiographies, show the influence of psychological anthropology. An example would be a biography of an Indian chief.
Adapted From O.Werner & G.M. Schoepfle. (1987). Systematic fieldwork (Vol. 1). Newbury Park: Sage.
184
Cross Sectional Ethnography

An ethnography in which a cross section of the members are studied, rather than a group of interacting members for instance, a study of prostitutes in which each informant is interviewed and no observations are made of prostitutes interacting with colleagues or clients. Another version of this approach would be the study and comparison of several different groups (e.g., prisons, schools, and street gangs).
Film or Photographic Ethnography

Photoethnographies take the form of books filled with photographs of natives living everyday life. Ethnographic films usually complement the written ethnography.
Encyclopedic Ethnography
Thesauri or dictionary of any language community. An example is Werners The Anatomical Atlas of the Navajo. This approach is static; it does not show document language in use.
185
What Ethnographers Study: Topics From The Journal of Contemporary Ethnography

What follows is a list of the titles of articles appearing in The Journal of Contemporary Ethnography over the past few years. This list excludes the titles of theoretical essays and book reviews.
Anger Management Among Abortion Escorts Prison Adaptation Strategies of 1st-Time, Short-term Inmates Somewhere Out There: Parental Claiming in the Preadoption Waiting Period Nonmainstream Body Modification: Genital Piercing, Branding, Burning, and Cutting Youth Theater and the Articulation of Cultural Capital: Refocusing Bourdieu Through Ethnography Metaphors of Stigma: How Deinstitutionalized Mentally Retarded Adults See Themselves Undercover Deception: Reconsidering Presentations of Self Interpretive Restraint and Ritual Tradition: Marysvilles Festival of Bok-Kai Backboards and Blackboards: College Athletes and Role Engulfment Appalachian Women: Violence and Social Control Bad Blood: The Moral Stigmatization of Paid Plasma Donors Organizational Politics and Technological Change Women at the Wall: A Study of Prisoners Wives Doing Time on the Outside Stigma and the Public Self: Notes on Communication, Self, and Others Stigma and the Moral Careers of Ex-Mental Patients Living in Board and Care Going Into the Closet With Science: Information Control Among Animal Experimenters Making It by Faking it: Working Class Students in an Elite Academic Environment Chinas Lost Generation: The Status Degradation of an Educational Cohort Danger at Sea: Social Hierarchy and Social Solidarity Preprofessional Socialization and Identity Transformation: The Case of the Professional ExParents Rationales for Operating Home Schools Dogs and Their People: Pet Facilitated Interaction in a Public Setting The Active Worker: Compliance and Autonomy at the Workplace
186
Why Cant We be Friends: Expectations Versus Experiences in the Volunteer Role Ladies, Flirts, and Tomboys: Strategies for Managing Sexual Harassment in an Underground Coal Mine Cooling Out Men in Singles Bars and Nightclubs: Observations on the Interpersonal Survival Strategies of Women in Public Places Developing Trust: Patient-Practitioner Encounters in Natural Health Care The Story of Edward: The Everyday Geography of Elderly Single Room Occupancy (SRO) Hotel Tenants Working At and Working: Computer Fritters Ethnography and AIDS: Returning to the Streets Confronting Deadly Disease: The Drama of Identity Construction Among Gay Men with AIDS Constructing AIDS Policy in the Public School: A Multimethod Case Study Takin It to the Streets: AIDS Outreach as Ethnography Ethnopharmacology: An Interdisciplinary Approach to the Study of Intravenous Drug Use and HIV Visual Ethnography: Tools for Mapping the AIDS Epidemic Deskilling, Decollectivization, and Diesels: Toward a New Focus in the Study of Changing Skills Women in Outlaw Motorcycle Gangs Narcotics Addicts Hustling Strategies: Creation and Manipulation of Ambiguity Boyhood, Organized Sports, and the Construction of Masculinities American States of Mind: Political Beliefs and Behavior Among Private and Public Workers Duty Bound: Elder Abuse and Family Care The Charaka Complex: Maturation Out of Delinquency in an Israeli Slum Turn-ons for Money: Interactional Strategies of the Table Dancer Guarding Against Boredom: Security Specialists in the United States Air Force Jocularity, Sarcasm, and Relationships: An Empirical Study Psychiatric Dirty Work Revisited: Conflicts in Servicing Nonpsychiatric Agencies The World of the Yeshiva: An Intimate Portrait of Orthodox Jewry
187
Preparation For Record Keeping: Three Journals
188
Residence Style and Ethnographic Style*
The Ethnographer Lives:

In Grass Hut Close Proximity to The Natives (GRASS HUT) Live In Ethnography Example: Full immersion in a South Pacific Cultural Group. At A Distance Commuter Ethnography Example: School ethnography; present when school is in session.
The Ethnographer Works In:
White Room (NEUTRAL PLACE AWAY FROM NATIVES)
Ethnographic Experiments Example: Psychological anthropologist who lives with the natives, but does testing away from the natives group.
Secret Ethnography Example: A clientsponsored study of an organization, in which members are interviewed in private places
Adapted From: O. Werner & G.M. Schoepfle. (1987). Systematic Fieldwork (Vol. 1). Newbury Park: Sage.
189
The Communication Context Matrix: An Example*
The Possibilities: Initial Communication Context Matrix for Six Navajo School Ethnographies
Students Students School Community In Class and school In class, halls, and yard In homes
School
Community
Ethnographers
In meetings school board meetings interview Chapter meetings and school board interview research staff meetings
Ethnographers
Interview and classroom
Final Communication Context Matrix After Restricting Ethnographies to the View of Students
Students Students School Community Ethnographers In class, hall, and yard In class, halls, and yard not covered Interview and classroom
School
Community
Ethnographers
not covered not covered contact and permission contact and permission permission research staff meetings
Adapted From O. Werner & G.M. Schoepfle. (1987). Systematic fieldwork (Vol. 1). Newbury Park: Sage.
190
CONTENT ANALYSIS
I. Historical Sketch of Content Analysis.
A. Quantitative Newspaper Analysis.
B.
Early Content Analysis.
1.
How Early Content Analysis Differed From Quantitative Newspaper Analysis.
a.
Academic Interest.
b.
Move To Latent, Concept-Based Analysis.
c.
Statistical Sophistication.
2.
Factors Accounting for the Emergence of Content Analysis.
a.
Development of Electronic Media.
b.
Emergence of Social and Political Problems for Which the Mass Media Received Blame.
c.
Development of More Sophisticated Research Procedures.
C.
Propaganda Analysis.
1.
World War II Influences.
a.
Harold Lasswell and Associates Refine the Tools of Content Analysis (Experimental Division for the Study of Wartime Communications, Library of Congress).
191
b.
Hans Speiers Investigative Team (Federal Communications Commission): Predicting Nazi Movements on the Basis of the Enemys Broadcasts.
2.
Some Lessons from World War II Studies.
D.
Content Analysis Goes General.
E.
Computer-Assisted Content Analysis.
II.
Some Uses of Content Analysis (see handout on page 197).
III.
Definitional Considerations.
A. Content Analysis Defined: Content Analysis is a research technique for making replicable (reliable) and valid inferences from data to the context from which those data were taken.
B.
Comparison With Other Research Approaches.
1.
Content Analysis is Unobtrusive.
2.
Content Analysis Accepts Unstructured Materials.
3.
Content Analysis is Context Sensitive.
192
IV.
Unitizing.
A. Sampling Units: The specific entities which are sampled; the chunks in which content comes.
B.
Recording Units: The specific segment of the sampling unit about which information is recorded (unit of observation).
C.
Context Units: The contextual limits which may be considered when categorizing (coding) the recording unit.
See handout on page 198.
V.
Sampling.
A. Rationale For Sampling.
B.
Most Common Sampling Schemes Used in Content Analysis.
1.
Simple Random Sampling.
2.
3.
Stratified Sampling.
4.
Proportional Sampling.
5.
Multistage Sampling.
193
C.
Sample Size.
VI.
Recording
A. Recording as Categorizing.
B.
Category Formation Criteria.
1.
Mutual Exclusivity.
2.
Exhaustiveness.
C.
Coding.
1.
Observers (Judges).
2.
Training.
3.
Some Ways of Communicating The Meaning of Categories.
a.
Definitions.
b.
Extensional Lists.
c.
Decision schemes. (See page 199 for an example.)
D.
Data Sheets. (See example on page 200.)
194
1.
Administrative Information.
2.
Data Organization Information.
3.
Data.
VII.
Types of Questions.
A. Questions About Frequencies.
1.
Questions About Absolute and Relative Frequency.
2.
Questions About Changes in Frequency Over Time.
B.
Associations/Correlations Among Categories.
C.
Covariation Between a Category in the Content Analysis and Noncontent Variables.
D.
Image/Portrayals Via Discriminant Analyses.
E.
Clustering.
VIII.
Reliability Assessment: An attempt to determine the extent to which our categorizing reflects real variations in real phenomena.
A. Ingredients of Data-Making.
1.
Real Variations in Real Phenomena.
195
2.
Flaws in Our Measurement System.
3.
Idiosyncrasies of the Coders.
4.
Recording Errors.
B.
Duplication of Effort.
C.
Reliability As A Necessary But Not Sufficient Condition for Validity.
D.
Types of Reliability. (See handout on page 202.)
1.
Stability (Intra-Coder Reliability).
2.
Reproducibility (Inter-Coder Reliability).
3.
Accuracy (Conformity to a Standard).
IX.
Validity.
A. Definition.
B.
Some Issues in Validity, as it Relates to Content Analysis.
196
Some Uses of Content Analysis

Purpose
Identification of Trends in Content Trace Development of Scholarship
Example
Are females being portrayed in more professional roles now than ten years ago? What theories are being researched in the field of __________, and how does this attention compare to the past? Are there invisible colleges in the field of __________? [bibliographic citation analysis] Are advertisers in Britain more or less likely to use humor than advertisers in the USA? Are there any differences in humor style usage? How do editorials on television compare with newspaper editorials? Have network executives kept their promise to decrease the level of violence on television programming? How often do Saturday morning commercials make use of the same characters that are found in the shows that provide the context for the commercials? You have peoples open-ended descriptions of the problems they feel most challenge this nation. These must be placed into standardized categories so that tallies can be computed across respondents. How do the pro-life and pro-choice movements portray each other? What level of education must one have achieved to be able to comprehend the New York Times? How are visual elements used to augment the verbal message of network news? Does the enemy give away its upcoming attacks? What can be learned about the attitudes of these clinical patients from a content analysis of the images they see in the Jones Protective Inkblot Test? What values are expressed on television, and are these values portrayed positively or negatively? What issues are receiving the most attention from network news departments this month? How does this coverage compare to the past?
Examination of International Differences in Content Media Comparisons Audit Communication Content Against Objectives Communication Policy Formation Code Responses to OpenEnded Questions Expose Persuasion Techniques Readability Assessment Examine Stylistic Features Identify the Intentions of Communicators Assessment of Psychological States of Persons and Groups Assessment of Cultural Patterns of Population Groups Reveal the Focus of Attention
197
Unitizing in Content Analysis: Some Examples
Sampling Unit
Recording Unit
Context Unit
The Television Program
Character
Scene
The Political Speech
Word
Sentence
The TV Ad
Utterance
Previous Utterance
The Romantic Novel
Statement of a Relational Nature Between Paramours
Scene in Which Exchange Takes Place.
The Newspapers
Political symbol
Editorial In Which Symbol is Used
The Presidential News Conference
Verbs
Sentence
The Magazine
Headlines
Story
198
Recording Via Decision Schemes

Example: Places of Action in Comic Strips
Code Planet Earth United States Historical Urban Rural Contemporary Future Foreign Nation Historical Urban Rural Urban Rural Urban Rural Contemporary Future InterStellar Planet Deep Space Urban Rural Urban Rural A B C D E F G H I J K L M N
199
Sample Data Coding Sheet
Project Name: Demographic Analysis of TV Characters Variable Program ID # State of Coding Categories/Codes (Circle Code or Enter Appropriate No.) Assigned Number: ___ ___ ___ 1 Coding Completed 2 Coding Verified 3 Data Entry Complete 1 Bill 2 Cathy 3 Janie 4 Paul Assigned Number: ___ ___ ___ ___ ___________________________________ ___________________________________ ___________________________________ ___________________________________ 1 Female 2 Male 3 Unsure 1 2 3 4 5 6 7 8 Black White Asian Hispanic Native American Other: ____________________ Not Human: Animal Not Human: Alien, Monster, etc. Column No. 1-3 4
Coder Character ID # Character Name/ Description
5 6-9 No Data Entry For This Variable
Character Sex
10
Character Race
11
Lawbreaker?
1 No 2 Yes Describe Infraction: ________________
12
200
Project Name: Demographic Analysis of TV Characters Variable Occupation Categories/Codes (Circle Code or Enter Appropriate No.) 1 Professional 2 Managerial 3 Clerical 4 Sales 5 Craftsman 6 Service 7 Private 8 Operative 9 Laborer 10 Household 11 Unemployed 12 Child 13 Not Determinable Describe Work Activities: 1 Regular Character 2 Guest Appearance 1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 Situation Comedy Family Drama Crime Animated Adventure Documentary News Sports Programming Early Morning (6 - 8:59 a.m.) Morning (9 a.m. - 11:59 a.m.) Afternoon (Noon - 4:59 p.m.) Early Evening (5 p.m. - 6:59 p.m.) Evening (7 - 10:59 p.m.) Late Evening (11:00 p.m. - 1:59 a.m.) Graveyard (2:00 - 5:59 a.m.) Column No. 13-14
Role Program Type
15 16
Program Time (Beginning)
17
Station
Enter Local Channel Number (02 - 99)
18-19
201
Types of Reliability*
Errors Assessed Types of Reliability Stability Reproducibility Accuracy Reliability Designs test-retest test-test teststandard Intra-Judge Inconsistency yes yes yes Inter-Judge Disagreements no yes yes Deviations From a Standard no no yes Relative Strengths Weakest Intermediate Quality Strongest
Adapted From K.K. Krippendorff (1980). Content analysis: An introduction to its methodology. Beverly Hills: Sage.
202
EVALUATION RESEARCH/QUASI-EXPERIMENTAL DESIGN

I. What is Evaluation Research?
A. Definition: Evaluation research is a kind of applied research which has as its purpose an examination of the outcomes of a social intervention.
1.
Applied Research
2.
Purpose
3.
Outcomes
4.
Social Intervention
B.
Examples of Evaluation Research.
II.
What is Quasi-Experimental Design?

There are many natural social settings in which the research person can introduce something like experimental design into his [her] scheduling of data collection procedures (e.g., the when and to whom of measurement), even though he [she] lacks the full control over the scheduling of experimental stimuli (the when and to whom of exposure and the ability to randomize exposures) which makes a true experiment possible. Collectively, such situations can be regarded as quasi-experimental designs. Donald T. Campbell & Julian C. Stanley* A. Some Factors Which May Make Randomization Impossible.
1.
Experimental and Control (or Comparison) Groups Occur Naturally and Cannot Be Reconstituted.
D.T. Campbell & J.C. Stanley. (1963). Experimental and quasi-experimental designs for research. Boston: Houghton Mifflin Company.
203
2.
Intervention Began Prior To Investigation.
3.
Ethics Require That Subjects Have Right To Select Themselves Into or Out of a Particular Treatment Group.
4.
All Subjects Have Been Exposed to the Social Intervention.
B.
Quasi-Experimental Designs and Potential Sources of Invalidity.
Three factors make quasi-experimental designs potentially more subject to sources of invalidity.
1.
Absence of a comparison/control group.
2.
Necessary Reliance Upon a nonequivalent comparison/control group.
3.
Lack of control over the timing of the experimental stimulus (the intervention).
III.
Connections Between Quasi-Experimental Design and Evaluation Research.

A. Why are Quasi-Experimental Designs Used So Often By Evaluation Researchers?
B.
Do Evaluation Researchers Rely Upon Other Research Techniques?
IV.
Stages of Experimental- and Quasi-Experimental Evaluation Research.

A. Problem Formation.
B.
Measurement. 204
1.
Specify the Nature of the Intervention.
a.
Identification of Groups.
b.
Extent of Exposure to Intervention.
2.
Operationalize the Outcome Variable(s).
3.
Develop Measures For Relevant Experimental Context Variables.
4.
Develop Measures For Relevant Subject Characteristics.
C.
A Measurement Example.
D.
Develop an Experimental or Quasi-Experimental Design.
V.
Experimental Design Revisited: Uses of Experimental Designs In Evaluation Research.
VI.
Three Common Quasi-Experimental Designs.

A. Nonequivalent Control Group Design.
1.
Basic Design.
205
Groups Typically Constitute Naturally Assembled Collectives; Randomization Not Possible
Experimental Group Comparison Group
O O
O O
2.
B.
Time-Series Designs (Single Group).
1.
Basic Design.
O1 O2 O3 O4 X O5 O6 O7 O8 Time >
2. Point of Comparison.
3.
Some Strengths.
a.
Maturation Effects are Controlled.
b.
Testing Effects Can Be Examined.
c.
Selection Biases and Mortality Can Be Dealt With.
4.
Most Likely Source of Internal Invalidity: History (Uncontrolled Event).
5.
Some Possible Findings (see handout on page 208).
206
C.
Multiple Time-Series Designs (Multiple Groups).
1.
Basic Design.
Experimental Group: Comparison Group:
O1 O2 O3 O4 X O5 O6 O7 O8 O1 O2 O3 O4 O5 O6 O7 O8
Time: >
2.
Advantages.
3.
Disadvantages.
207
Time-Series Quasi-Experimental Design: Some Possible Outcome Patterns*
Based on Figure 3 in D.T. Campbell & J.C. Stanley. (1963). Experimental and quasi-experimental designs for research. Boston: Houghton Mifflin.
208

CMN 102 Manual Fall 2010

Hochgeladen von

Dokumentinformationen

Copyright

Verfügbare Formate

Dieses Dokument teilen

Dokument teilen oder einbetten

Freigabeoptionen

Stufen Sie dieses Dokument als nützlich ein?

Sind diese Inhalte unangemessen?

Copyright:

Verfügbare Formate

CMN 102 Manual Fall 2010

Hochgeladen von

Copyright:

Verfügbare Formate

CMN 102: Empirical Methods in Communication

Fall 2010 Dr. Robert A. Bell Department of Communication UC Davis

Much of What We Know To Be True Isnt.

Individualistic Nature of Common Sense.

How Real is Reality?

Experiential Reality: Knowledge Based on Direct Experience.

Agreement Reality: Knowledge Based On Social Consensus.

Errors and Biases In Casual Human Inquiry.

Overgeneralization. (See handout on replication as a guard against overgeneralization, page 8).

Selective Observation: Ill See It When I Believe It.

Ex Post Facto Hypothesizing (Made-Up Information).

Illogical Reasoning. (See handout on page 9.)

Premature Closure of Inquiry.

Theory: The Foundation of Social Science.

All Theories Are Abstractions.

Theories Are Inherently Partial.

Theories Are Constructions.

Theory as Explanation of Regularities in Human Symbolic Behavior.

The Charge of Triviality.

Exceptions Dont Deny The Rule.

A Variable Language For Theory Construction.

The Traditional (Deductive) Model of Scientific Method.

Starting Point: Phenomenon of Interest.

Problem In Need Of An Answer.

Step 2: Theory Construction.

Hypotheses. (See Example On Page 13).

Step 3: Operationalization of a Theorys Constructs To Create Variables.

Step 4: Observation (Data Collection and Hypothesis Testing).

Summary: Wallaces Model of Science.

Inductive Theory Construction.

Inductive Theory and Field Research.

Some Characteristics of Good Theory.

One Guard Against Overgeneralization is Replication

Some Common Reasoning Fallacies*

Variables and Attributes

Uncertainty Reduction Theory*

Theoretical Concepts are Operationalized To Create Variables

Objection: Causality Versus Free Will.

Response: Reasons Have Reasons (Causes).

Weak Determinism: Probabilistic Causal Model.

Nomothetic and Idiographic Explanation. (See handout on page 20.)

Criteria For Causality.

Correlation: The Two Variables Must Be Empirically Related.

What Different Correlations Look Like (See handout on page 22.)

A Word on Interchangeable Indexes.

Absence of Third Variable. (See exercise on page 25.)

Two Approaches To Communication Research*

Social Facts Approach (Nomothetic)

Epistemology (Study of the Basis of Human Knowledge)

Social Definition Approach Data

Social Facts Approach

Visualizing Different Correlation Values

Intermediate Correlational Values

ETHICS AND POLITICS IN SOCIAL RESEARCH

Anonymity and Confidentiality.

Analysis and Reporting of Results.

Role of the Institutional Review Board.

Professional Codes of Ethics.

Politics of Social Research.

First Things First: Identify the Purpose of Your Investigation.

Descriptive Research: What Is...

Explanatory Research: Why...

Specify the Unit of Analysis and the Unit of Observation.

Unit of Analysis: the what or who that is being studied.