POEC 5310: Research Design I

Fall 2005

Tuesdays 7-9:45 Prof. Kimberly Kempf-Leonard

CBW 1.104 972-883-4969; kleonard@utdallas.edu; 2.120 Green Hall

Course Description
This course on research design is the first in a series of graduate courses offered in the School of
Social Sciences to teach graduate students about methodology used in the social sciences. The
task of the methodologist is to explain what the investigator is doing and to determine whether
the procedures will lead the investigator to the kinds of statements he or she wants to make. This
course explains strategies for devising social science studies, compares the relative benefits of
various designs, and identifies the tools necessary actually to conduct studies that will yield data
worthy of analysis, publication, tenure, and promotion! This material will be valuable not only
for students who will conduct research and administrators who must evaluate the research of
others, but also for those who, in the course of daily lives, wonder how and why people behave
and organizations develop as they do.

NO PREREQUISITE: Any graduate student in the School of Social Sciences is welcome to

enroll and encouraged to take the class early in their graduate studies.

The word methodology is used in various ways within the social sciences. As such, and
especially in the orientation of this course, it is worth considering that among the many design
strategies, methods of data collection, and analytical techniques, there are standards and “rules of
thumb,” but also exceptions to these “rules” and lack of consensus about what is ideal. The more
issues you understand and appreciate, the better methodologist you will become.

Too often students jump into analysis of data without understanding the pitfalls and
idiosyncracies of how the data were collected and what the variables really mean. This course
should help you to avoid such mistakes. Take to heart the following messages from prominent
social science methodologists:

Methodology … [has] developed as a bent of mind rather than as a system of

organized principles and procedures. The methodologist is a scholar who is
above all analytical in his approach to his subject matter. He tells other scholars
what they have done, or might do, rather than what they should do. He tells them
what order of finding has emerged from their research, not what kind of result is
or is not preferable. This kind of analytical approach requires self-awareness on
the one hand, and tolerance, on the other. The methodologist knows that the same
goal can be reached by alternative roads.
P. Lazarsfeld & M. Rosenberg (1955) The Language of Social Research,p 4.
 Riddle: When it comes to reading the report of an empirical study, what is the
difference between a layman, a researcher, and a methodologist?
Answer: The layman reads the text and skips the tables; the researcher reads the
tables and skips the text; and the methodologist does not care very much about
either the text or the tables, as long as they both agree with each other.
T. Hirschi & H. Selvin (1973) Principles of Survey Analysis, p. 8

‘abstracted empiricism’ helps insure ‘that we do not learn too much about man
and society’
C.W. Mills (1959) The Sociological Imagination, p. 75

[In] the interplay between design and statistics, design rules!

W. Shadish, T. D. Cook, & D.T. Campbell (2002). Experimental &
Quasi-experimental Designs for Generalized Causal Inference, p. xvi

Avoid the fallacy fallacy. When a theorist or methodologist tells you that you
cannot do something, do it anyway. Breaking rules can be fun!
T. Hirschi (1973). “Procedural Rules and the Study of Deviant Behavior.”
21 (2) Social Problems, pp.171-2.

Required Readings
Please purchase and bring to class the two books listed below. Other assigned readings will be
distributed in class or made available on library reserve.
De Vaus, David A. 2001. Research Design in Social Research. Thousand Oaks, CA: Sage.
(ISBN: 0-7619-5347-7; www.oup.com)

Bryman, Allan. 2004 Social Research Methods, 2nd edition. Oxford: Oxford University
Press. (ISBN 0-19-926446-5; www.sagepub.co.uk)
[first edition is fine too: 2001. (ISBN: 0-19-874204-5)]

Recommended Resource: The Encyclopedia of Social Measurement (2005)

available in the reference section of the library

Expectations of the students

Active attendance. Students should come to class meetings ready to discuss the topics and
assigned materials for the week. Research methods are best learned by doing, so we will have
several opportunities to practice. Assignments culminate in a final research design proposal in
which students convey their skills in research design and readiness to proceed to other
methodology courses.

Progress and achievement. The course activities and assignments are designed to provide a
foundation in research design and multidisciplinary methods for graduate students in the social
sciences. Activities and assignments will require different levels of attention, time, and skill; in
part, because students enroll in the course with various backgrounds, knowledge, and biases
about social science methodology. I hope every student meets with me to discuss his or her
interests in methodology, progress in the course, and most certainly if problems arise.

2
Colleagial interaction. Especially given our different substantive and methodological interests, it
is important that class participants be open-minded and respectful of one another. In addition to
being courteous and responsible, it is important for learning that students attend class, so
repeated absence or tardiness, regardless of the reason, will not be considered favorably.
Moreover, except in unusual circumstances, late submissions and make up assignments will not
be allowed.

Grades. Half of the course grade will be based on the final research proposal, due Nov. 29,
which should also become part of your portfolio. The other half of your course grade will be
based on assignments that lead up to the research proposal, including the following assignments
and exercises:
1. 5-8 journal articles on topic of your choosing (due Sept. 13)
2. hypothesis for research proposal (due Sept. 27)
3. Summary table of subjects and variables (due Oct. 4)
4. triangulation of methods (due Nov. 8)

Course Structure
The outline of course topics and reading assignments are listed below. The course will be a combination
of lecture, discussion, and activities. During most weeks, I will begin the session by presenting
techniques, issues, and concerns that I consider most important on the methodological topics for that
week. We will then have an open discussion and relate the issues to substantive examples within the
social sciences. Students are especially encouraged to share examples in class; the more practice we get at
evaluating research design, the better methodologists we will become!

1: Aug. 23 Purposes & Paradigms of Social Science Research

Exploration, description, comparison, explanation, prediction, & evaluation
Read: de Vaus, Ch. 1: The Context of Design

Stages of Social Science Research

1. Define a research problem
2. formulate a meaningful hypothesis
3. Conduct a literature review to determine what is known about the research
problem.
4. Identify dependent, independent, and intervening variables.
5. Formulate a research design.
- - - - - - - -- - - - - - - - - - - - -- -
6. Conduct the study.
7. Analyze and interpret the results.
8. Implications for future research and/or policy

2: Aug. 30 Epistemology & the Role of Theory in Social Science Research

Induction, deduction, hypotheses & Propositions, and the scientific method
Read: Bryman, Ch. 1: Social Research Strategies
G. Taubes (2002) “What if It’s All Been a Big Fat Lie?” New York Times
Magazine, 7/7

Come prepared to discuss concerns for the social scientist about epistemology
and researcher objectivity. What lessons might be learned from the article on
dieting research?

3
3: Sept. 6 Standards for Assessing Social Science
Reliability, validity, generalizability, objectivity, ethics, & methods
Read: de Vaus, Ch. 2: Tools for Research Design
Bryman Ch. 3: Nature of Quantitative Research
Bryman Ch. 13: Nature of Qualitative Research

4: Sept. 13 Features & Types of Designs

Causation, triangulation of methods, and feasibility
Read: de Vaus, Ch. 3: Causation and the Logic of Research Design
de Vaus, pp. 203 -211, section on elaboration
Bryman Ch. 21: Breaking down the quantitative/qualitative divide
Bryman Ch. 22: Combining quantitative and qualitative research

Assignment 1 due today: Decide on a social science topic that you intend to investigate
this semester. Choose a topic of interest to you and about which you will be able
to locate related refereed journal articles because this topic will become the focus
of many assignments and lead to the research proposal that contributes heavily to
your course grade. Identify five to eight articles from refereed journals that
describe some sort of research on this topic, provide me with a bibliography and
abstracts of each. It will be important that you have access to the complete
article.

5: Sept. 20 Concepts and Variables

Hypothesis construction, operationalization, levels of measurement, scales, indices &
typologies
Read: Bryman (pp. 65-70) concepts & measurement

Come prepared to share the research problem you will propose to study. You
should propose to study a relationship that is explanatory, predictive, or
evaluative. No exploratory, descriptive, or comparison proposals will be allowed.

7: Sept. 27 Research Subjects

Populations, sampling strategies, units of analysis, attrition, weighting, Human subjects
Read: Bryman Ch. 4: “Sampling”
Bryman Ch. 25 “Ethics in social research”

Assignment 2 due today: Turn in draft of the hypothesis you anticipate to propose. Write
a paragraph describing why this is a good example of a hypothesis. Be certain to
identify important concepts and the causal relationship specified in the
hypothesis.

8: Oct. 4 Experimental designs

Types, elements for construction, strengths and weaknesses, & implementation issues
Read: de Vaus, Ch. 4: Types of Experimental Design
de Vaus, Ch. 5: Issues in Experimental Design

Come prepared to talk about how your topic might be investigated through an
experimental design and what difficulties would need to be overcome.

Assignment 3 due today: Construct a table that provides an overview of the articles you
are reviewing. This table should identify the research subjects and how they were
identified. You will also need to distinguish the independent and dependent

4
 variables, and how they were operationalized. I will provide you with a template
for this assignment, or you may devise your own.

9: Oct. 11 Cross-sectional Designs & Case Study Designs

Reliance on existing differences, data collection methods, and practical issues
Read: de Vaus, Ch. 10: Cross-Sectional Design
de Vaus, Ch. 11: Issues in Cross-Sectional Design
de Vaus, Ch. 13: Case Study Design
de Vaus, Ch. 14: Issues in Case Study Design

Come prepared to discuss various units of analysis you could propose to examine
and what a potential case study for your topic look like.

10: Oct. 18 Longitudinal designs

Types, purposes, strengths and weaknesses, implementation issues, and examples
Read: de Vaus, Ch. 7: Types of Longitudinal Design
de Vaus, Ch. 8: Issues in Longitudinal Design

11: Oct. 25 Methods of Quantitative & Qualitative Data Collection I:

Unobtrusive observation, experience, & physical traces.
Read: Bryman Ch. 8: Structured Observation
Bryman Ch. 9: Content Analysis
Bryman Ch. 10: Secondary analysis & Official Statistics

12: Nov. 1 Methods of Quantitative & Qualitative Data Collection II: Obtrusive Asking.
Read: Bryman, Ch. 5: Structured Interviewing
Bryman Ch. 6: Self-Completion Questionnaires
Bryman Ch. 7: Asking Questions

13: Nov. 8 Methods of Quantitative & Qualitative Data Collection III: Field research
Read: Bryman Ch. 14: Ethnography & Participant Observation
Bryman Ch. 15: Interviewing in qualitative research
Bryman Ch. 16: Focus Groups

Assignment 4 due today: Write a short paper in which you describe two different data
collection methods you could use to measure critical concepts and examine the
relationship of interest about your topic. Describe the data collection instrument
you would use to measure key variables, and justify your decisions.

14: Nov. 15 No Class Meeting (annual meetings of American Society of Criminology in Toronto)

15: Nov. 22 Finish up; discuss research proposal projects…

Evaluation & Applied Research (if time permits)

Read: P. Rossi, H. Freeman, & M. Lipsey (1999). Evaluation (ch.1, 2 & 12)

Nov. 29: Proposals are due by 7pm. (Paper copies only)