Chapters 4, 5, and 13: Nonexperimental Methods I & II: Descriptive Methods, Qualitative Research, Correlational Studies, Ex Post Facto

Studies, Surveys and Questionnaires, Sampling and Basic Research Strategies

Whats so Bad about Lab Experiments? Laboratory setting is artificial Participants know they are being studied Demand characteristics Experimenter Expectancies Smart vs. Dull rats Reactivity Socially desirable responses!

Whats so Bad about Lab Experiments?

Low ecological (external) validity Reduced experimental realism. May be measuring what a participant can do, not what they really do do. Participants often unrepresentative Volunteer Bias Mortality effects Limited population-College Students perhaps?

Field Experiments
Natural settings leads to high external validity Higher ecological validity Higher temporal validity Better generalizability Can use samples which can not come to lab Greater experimental realism Useful to ensure generality of lab studies

Field Experiments Why are they Bad? Hawthorne effect Ethical issues Invasion of privacy Manipulating variables might be dangerous Failure to obtain informed consent Less internal validity Loss of control MANY extraneous variables

Descriptive Methods

Descriptive methods do not involve manipulation of an independent variable. When we use descriptive methods, we can only speculate about causation that may be involved. Comparisons with the Experimental Method We are not able to exercise any control with regard to gathering these data and cannot make any statements regarding cause-andeffect.

Descriptive Methods

What

are the advantages 1. Descriptions are informative 2. Descriptions are the starting point for building operational definitions 3. Description is sometimes the only ways to study behavior

Descriptive Methods Presentation of Results

Deciding how to present the results of your research project. Qualitative presentation of results: Report consists of a description of the behavior in question (a narrative record) and the conclusions prompted by this description. Quantitative or numerical presentation of results: Need to know how behavior under investigation is going to be measured and how these measurements will be analyzed.

Descriptive Methods Archival Research

Archival and Previously Recorded Sources of Data Refers to use of data recorded by other individuals for other purposes public health and census data The General Social Survey is an archival source that can be accessed online http://www.icpsr.umich.edu/gss/home.htm

Descriptive Methods Archival Research

Potential Problems With Archival and Previously Recorded Sources of Data You will not know exactly who left the data you are investigating. The participants may have been selective in what they chose to write. This problem is also known as selective deposit. Archival and previously recorded sources of data may not survive long enough for you to make use of them.

Descriptive Methods Observational Techniques

Good We see behavior which is real. Bad No informed consent Susceptible to experimenter expectancies We usually cannot randomly sample Usually we cannot quantify responses Low internal validity

Descriptive Methods Observational Techniques

Case Studies: Involves intense observation and recording of behavior of a single (perhaps two) participant(s) over an extended period of time. There are no guidelines for conducting a case study and the procedures employed, behaviors observed, and reports produced may vary substantially. Frequently used in clinical settings to help formulate ideas and hypotheses for further research.

Descriptive Methods Observational Techniques Naturalistic Observation: Involves seeking answers to research questions by observing behavior in the real world.
The

first goal of naturalistic observation is to describe behavior as it occurs in the natural setting without the artificiality of the laboratory. The second goal of naturalistic observation is to describe the variables that are present and the relations among them.

Descriptive Methods Observational Techniques Psychological Detective

Why should the researcher be concealed or unobtrusive in a study using naturalistic observation? The reactance or reactivity effect Refers to biasing of the participants responses because they know they are being observed. The reactivity effect is also known as the Hawthorne effect because of the location of the original study.

Descriptive Methods Observational Techniques

Participant Observation: The researcher becomes part of

the group being studied.

Often used when the goal of the research project is to learn something about a specific culture or socioeconomic group. Ethnography is a form of participant observation based on the anthropological tradition of research. Observer as participant refers to a researcher who primarily observes a situation but who interacts with the others (Glesne, 1999). Participant as observer refers to the researcher who becomes a part of the culture by working and interacting extensively with the others (Glesne, 1999).

Descriptive Methods Observational Techniques Psychological Detective

What are the drawbacks and weaknesses of the participant observer technique? An extended period of time may be necessary before the participant observer is accepted as a member of the group that is under study. Cannot make cause-and-effect statements.

Descriptive Methods Observational Techniques

Using More than One Observer There are two main reasons for using more than one observer: One observer may miss or overlook a bit of behavior. There may be some disagreement concerning exactly what was seen and how it should be rated or categorized.

Descriptive Methods Observational Techniques

Using More than One Observer When two individuals observe the same behavior, it is possible to see how well their observations agree. The extent to which the observers agree is called interobserver reliability. Low interobserver reliability indicates that the observers disagree about the behavior(s) they observed. High interobserver reliability indicates agreement.

Descriptive Methods Observational Techniques

Here is the formula for calculating interobserver reliability:

# of times observers agree ______________________ X 100 = percent of agreement # of opportunities to agree

85% agreement is generally considered to be an acceptable minimum level for interobserver reliability.

Descriptive Methods Clinical Perspective

Clinical perspective Schein (1987) argued convincingly that the clinical perspective or model is not a subcategory of participant observation because:

Descriptive Methods Clinical Perspective

A client typically chooses the clinician, whereas the participant observer chooses the others to be studied. Unlike participant observers, clinicians cannot be unobtrusive because they have been asked to participate in the situation. Although the participant observer can remain passive, clinicians must intervene in the situation. The participant observers goal is understanding, whereas the clinicians goal is helping. Participant observers validate their findings by replication while clinicians validate their findings by being able to predict the results of a given intervention.

Descriptive Methods Choosing Behaviors and Recording Techniques

Time Sampling: Involves making observations at

different time periods in order to obtain a more representative sampling of the behavior of interest.
of time periods may be determined randomly or in a more systematic manner. The use of time sampling may apply to the same or different participants.
Selection

Descriptive Methods Choosing Behaviors and Recording Techniques

Situation Sampling: Involves observing the same behavior in several different situations. This techniques offers the researcher two advantages:
By sampling behavior in several different situations, you are able to determine whether the behavior in question changes as a function of the context in which you observed it. You are likely to observe different participants in the different situations and because different individuals are observed, your ability to generalize any behavioral consistencies across the various situations is increased.

Descriptive Methods Surveys, Questionnaires, Tests, and Inventories

A test should measure the nature and extent of individual differences A good test differentiates (discriminates between groups) people based on their true scores Why Use Tests? To help determine outcomes of experiments To diagnose strengths and weaknesses To assist in placing individuals in appropriate programs To assist in selecting applicants To evaluate a programs effectiveness

Descriptive Methods Surveys, Questionnaires, Tests, and Inventories Surveys typically request our opinion on some topic or issue that is of interest to the researcher. Types of Surveys Descriptive Analytic

Descriptive Methods Surveys, Questionnaires, Tests, and Inventories

Descriptive seeks to determine what percentage of the population has a certain characteristic, holds a certain opinion, or engages in a particular behavior. Examples: The Gallup Poll Nielsen television ratings

Descriptive Methods Surveys, Questionnaires, Tests, and Inventories

Analytic seeks to determine what the relevant variables are and how they might be related. Questions need to be chosen carefully. Pilot testing is generally necessary to determine the type of questions that will be used in the final survey instrument.

Descriptive Methods Surveys, Questionnaires, Tests, and Inventories

Developing a Good Survey or Questionnaire (see table 5-1, p. 100). Step 1 decide what type of instrument to use. How will the information be gathered? Will you use a mail survey, conduct a phone interview, or conduct the survey in person? Will you use trained interviewers?

Descriptive Methods Surveys, Questionnaires, Tests, and Inventories

Yes-No Questions The respondent answers yes or no to the items. Forced Alternative Questions The respondent must select between two alternative responses. Multiple-Choice Questions The respondent must select the most suitable response from among several alternatives. Likert-Type Scales The individual answers a question by selecting a response alternative from a designated scale. A typical scale might be the following: (5) strongly agree, (4) agree, (3) undecided, (2) disagree, or (1) strongly disagree. Open-Ended Questions A question is asked to which the respondent must construct his or her own answer. Step 2 identify the types of questions to use.

Descriptive Methods Surveys, Questionnaires, Tests, and Inventories

Step 1 decide what type of instrument to use. How will the information be gathered? Step 2 identify the types of questions to use. Step 3 write the items. They should be clear, short, and specific.

Use familiar vocabulary and be at the reading level of the individuals you intend to test.

Step 4 Pilot-test and seek opinions from knowledgeable others.

It is important to ask others, especially professionals who have expertise in your area of research interest, to review your items. They may be able to detect biases and unintended wordings that you had not considered.

Descriptive Methods Surveys, Questionnaires, Tests, and Inventories

Step 5 determine the relevant demographic data to be collected. Includes items such as age, sex, annual income, size of community, academic major, and academic classification. Step 6 determine administration procedures and develop instructions. Instructions must be clear, concise, easy to follow, and thoroughly rehearsed (for in-person or phone interviews). How will informed consent be obtained?

Descriptive Methods Surveys, Questionnaires, Tests, and Inventories

Survey Administration Procedures Mail Surveys Personal Interviews Telephone Interviews

Descriptive Methods Surveys, Questionnaires, Tests, and Inventories

Mail Surveys Advantages

The

researcher does not have to be present while the survey is being completed. Surveys can be sent to a much larger number of participants than a single researcher could ever hope to contact in person.

Disadvantages
The

researcher cannot be sure who actually completes the survey. There is no guarantee the respondent answered the questions in the same order in which they appeared in the survey. Low return rate.

Descriptive Methods Surveys, Questionnaires, Tests, and Inventories

Personal Interviews Advantages

When

a trained interviewer administers a survey in a respondents home, the response rate climbs dramatically. The trained interviewer is able to clarify ambiguous questions, make sure that all questions are answered in the proper sequence, and generally assist with any problems that the respondents may experience.

Disadvantages
The

potential for considerable expenditure of time and money exists. People not always at home.

Descriptive Methods Surveys, Questionnaires, Tests, and Inventories

Telephone Interviews Advantages It is easy to create random samples with random-digit dialing. 95% of households in the U.S. have telephones. Computer technology allows responses to be entered directly as they are made.

Descriptive Methods Surveys, Questionnaires, Tests, and Inventories

Telephone Interviews - Disadvantages Incoming calls may be screened and potential respondents lost. It is easy to say no to a phone interviewer. Cannot use visual aids to clarify questions. Cannot evaluate nonverbal cues such as facial expressions, gestures, and posture. It is more difficult to establish rapport.

Descriptive Methods Surveys, Questionnaires, Tests, and Inventories Tests and Inventories Tests and inventories are designed to assess a specific attribute, ability, or characteristic possessed by the individual being tested. Characteristics of Good Tests and Inventories Validity a test or inventory has validity when it actually measures what it is supposed to measure.

Descriptive Methods Surveys, Questionnaires, Tests, and Inventories

Types of Tests and Inventories Achievement Tests

Are given when an evaluation of an individuals level of mastery or competence is desired. The score that distinguishes passing from failing determines the minimum level of achievement that must be attained.

Aptitude

Tests Tests or inventories

Are used to assess an individuals ability or skill in a particular situation or job. Measure specific aspects of an individuals motivational state, interpersonal capability, or personality.

Personality

Descriptive Methods Surveys, Questionnaires, Tests, and Inventories To Use Multiple Choice Items or Not?

Disadvantages Advantages Limit students options Versatile No opportunity to Easy to score practice writing Simple to take Some people dont do Poor writers not penalized well on them Good items used again Limits content to be Good distracters are assessed diagnostic Items must be well Hard to fake correct answer written

Descriptive Methods Surveys, Questionnaires, Tests, and Inventories Attitude Tests

Item
The day before Thanksgiving should be a holiday. Final exams should be elective. The dining room should serve gourmet food. My parents dont appreciate how smart I am. My professors dont appreciate how smart I am, either.

Agree

No Strong Feeling
____ ____

Disagree

____ ____

____ ____

____ ____

____ ____

____ ____

____

____

____

Descriptive Methods Surveys, Questionnaires, Tests, and Inventories Likert Scales Statements indicating an attitude toward a topic are written Items with clearly positive or negative attitudes are selected Statements are listed with a space for respondent to indicate degree of agreement

Descriptive Methods Surveys, Questionnaires, Tests, and Inventories A Likert Scale

Directions: Indicate to what extent you agree or disagree with the statements listed below by circling one of the following: SA means that you strongly agree with the statement (value = 5) A means that you agree with the statement (value = 4) U means that you are undecided about the statement (value = 3) D means that you disagree with the statement (value = 2) SD means that you strongly disagree with the statement (value = 1)

Government has no business funding child care programs. Child care should be supported by federal, state, and local tax dollars.

SD SD

D D

U U

A A

SA SA

Items can be weighted Weights of unfavorable items are reversed Average score is computed

Descriptive Methods Surveys, Questionnaires, Tests, and Inventories Personality Tests

Projective tests Present respondent with ambiguous stimulus Rorschach Structured tests Questions are objective MMPI

Descriptive Methods Surveys, Questionnaires, Tests, and Inventories Questionnaires What they are
and pencil tests with structured questions Self-administered Advantages Can be mailed out Survey broad geographic area Cheaper than one-on-one interview Respondents may be more honest Data easy to share with other researchers Disadvantage Low return rate (sample unrepresentative)
Paper

Descriptive Methods Surveys, Questionnaires, Tests, and Inventories Basic Assumptions of a Questionnaire Does not make unreasonable demands upon the respondent Does not have a hidden purpose Requests information that respondents presumably have Contains interesting questions Does not request information that could be obtained by other means

Descriptive Methods Surveys, Questionnaires, Tests, and Inventories Creating Questionnaires

Should closed or open-ended questions be used? Is reliability or breadth of information most needed? Close ended questions are objective

Are

more reliable

Open-ended questions are subjective

Can

get a greater breadth of information Yield statistically unmanageable data

Descriptive Methods Surveys, Questionnaires, Tests, and Inventories Creating Questionnaires

Should an interview or a questionnaire format be used?

How

strong are demand characteristics? Must responding be structured for the participants? What breadth of information is needed?

Interviews are preferred when the researcher must react to a participants responses. But questionnaires are more reliable, especially if demand characteristics are an issue.

Descriptive Methods Surveys, Questionnaires, Tests, and Inventories Creating Questionnaires

Are questions worded correctly? Are double-barreled questions avoided? "Sales reps are polite and responsive." Are leading questions avoided? "How satisfied are you with the compensation policies of our company? Implies satisfaction even before answering Are Barnum statements avoided? Do you sometimes worry? Are undefined terms avoided?

Descriptive Methods Surveys, Questionnaires, Tests, and Inventories Creating Questionnaires

What is the response scale? Are measurements sensitive to subtle differences? Are multiple-choice or Likert-type questions appropriate? What description should anchor Likert-type questions? How many choices should each scale contain? Is an odd or even number of choices needed?

Descriptive Methods Surveys, Questionnaires, Tests, and Inventories Creating Questionnaires

How will questions be administered?

Are

practice questions needed? Are funnel or filter questions needed? Are demand characteristics a problem? Are ethical obligations being met? Have order effects been avoided? Are alternate forms needed? Are catch trials needed? Are clear instructions provided? Is a pilot study needed?

Qualitative Research

is defined as an inquiry process of understanding a social or human problem, based on building a complex, holistic picture, formed with words, reporting detailed views of informants, and conducted in a natural setting (Creswell, 1994, p. 2). The qualitative researcher believes that a full description of human behavior includes peoples feelings in addition to what they are doing and how they are doing it. The qualitative research style is much less formal and impersonal, and the reader of a qualitative research report can expect to find such additions as definitions that evolved during a study (Creswell, 1994, p. 7).

Qualitative Research

The qualitative researcher is committed to studying particular people in specific settings. Qualitative researchers prefer to use inductive logic. Qualitative research begins with guiding hypotheses reflecting a global issue of interest (Marshall & Rossman, 1989). Qualitative researchers typically analyze their data simultaneously with data collection, data interpretation, and narrative reporting writing (Creswell, 1994, p. 153). Grounded theory is one of the most popular forms of contemporary qualitative research.

Qualitative Research
Grounded

Theory (Strauss & Corbin,

1990)
The

ultimate goal of this approach is to derive theories that are grounded in (based on) reality. Grounded theory is not advocated for all types of research questions. Knowing the literature too well can hamper the creativity necessary to doing grounded theory research. The heart of the grounded theory approach occurs in its use of coding, which is analogous to data analysis in quantitative approaches.

Qualitative Research

Types of coding in grounded theory research (Strauss & Corbin, 1990): Open Coding: The researcher labels and categorizes the phenomena being studied. Axial Coding: Involves finding links between categories and subcategories from open coding. Selective Coding: Entails identifying a core category and relating the subsidiary categories to this core.

Qualitative Research

Grounded Theory (Strauss & Corbin, 1990) From the process of selective coding, the grounded theory researcher moves toward developing a model of process and a transactional system, which essentially tells the story of the outcome of the research.
Process refers to a linking of actions and interactions that result in some outcome . A transactional system is grounded theorys analytical method that allows an examination of the interactions of different events. The transactional system is depicted in a conditional matrix such as that shown in figure 4.2

Qualitative Research

Participatory Action Research (Fine et al, 2003) Goal of a PAR project is usually to evaluate and understand the impact of some social program on the community. Typically involves the participants as equal researchers in the study. PAR approach can use one or more specific methods to gather information. These may be both quantitative and qualitative.

Correlational Research
Looks to the degree of relationship between two variables Relationships are only discovered ex post facto (after the fact, as opposed to a priori), observations are collected, then the correlation is computed. The absolute magnitude of a correlation indicates the degree of relationship.

Correlational Research Weaknesses

CORRELATION DOES NOT IMPLY CAUSATION!!!

The

absence of random assignment allows for potential confoundings The order of occurrence of the variable is unknown

Possible Causal Chains

Correlational Research Why do them?

Ethics Inexpensive Lab experiments are sterile Allows us to summarize an entire relationship with one number. Lead to better prediction than experiments
# hours spent volunteering

# hours spent volunteering

Low Medium Family income High

30 25 20 15 10 5 0

40 35 30 25 20 15 10 5 0 0 50000 100000 150000 Family income

Correlational Research

The sign of the correlation indicates the direction of the relationship. With positive correlations, as one of the variables increases, so does the other, conversely, as one of the variables decreases, so does the other. With negative correlations, as one of the variables increases, the other decreases.

Correlational Research

The Pearson r is by far the most commonly used correlation Can only be used if both variables are measured with interval or ratio scales of measurement and a linear relationship is assumed.
25 20 15 10 5 0 0 50000 Income 100000 150000

Oh no! What about this one!

Happiness

Correlational Research Beware of the Truncated Range!

25 20

Happiness

15 10 5 0 0 10000 20000 30000 40000 50000 Income

Correlational Research Beware of the Truncated Range!

25 20

Happiness

15 10 5 0 40000

50000 Income

60000

70000

Correlational Research Beware of the Truncated Range!

16 14 12

Happiness

10 8 6 4 2 0 70000 90000 110000 Income 130000 150000

Correlational Research Linear Regression Attempts to predict the value of one variable based on the value of another by creating a line of best fit. Y = bX + a Standard error of the estimate Coefficient of determination (r2)
16 14 12 10

Happiness

8 6 4 2

0 70000

90000

110000 Income

130000

150000

Correlational Research Reliability and Validity

Reliability:

The ability of a test to measure the same thing over and over. Validity: The ability of a test to measure what it is intended to measure. All tests have SOME reliability and validity, some are better than others though.

Correlational Research A picture is worth a thousand words

A reliable measure may or may not be valid but a valid measure must be reliable.

Correlational Research Reliability

Conventional Views on Reliability A score consists of two things, true score + error If error is high, reliability is low. Temporal Stability-The same form given on separate occasions should yield similar scores. Form Equivalence-If given alternate forms of the same test scores should be similar. Internal Consistency-Within the test itself, scores on individual questions should be similar to each other.

Correlational Research Reliability

Reliability is a necessary but not sufficient condition for validity. For instance, if the needle of the scale is five pounds away from zero, I always over-report my weight by five pounds. Is the measurement consistent? Yes, but it is consistently wrong! Is the measurement valid? No!

Correlational Research Types of Reliability

Inter-rater Reliability: The extent to which two raters are positively correlated. Test-retest Reliability: The consistency with which participants obtain the same overall score when tested at different times. Split-half Reliability: The consistency with which participants scores on some trials match their scores on other trials.

Correlational Research Increasing Reliability Increase sample size Eliminate unclear questions Standardize testing conditions Use both easy and difficult questions Minimize the effects of external events Standardize instructions Maintain consistent scoring procedures

Correlational Research Types of Validity

Internal Validity: The extent to which a good (clean) measure (or experiment) has been used, a clean measure is far more likely to reflect the concept intended.
External Validity: The extent to which a measure truly measures what is intended.

Correlational Research Types of Validity

Face Validity

The extent to which a measurement procedure appears to measure what it is intended to measure. Not always useful because things that look right may not be and things that dont look right might be.

Correlational Research Types of Validity

Construct Validity

Construct validity refers to the degree to which inferences can legitimately be made from the operationalizations in your study to the theoretical constructs on which those operationalizations were based.

In order to demonstrate construct validity, a measure must establish both convergent and discriminant validity.

Correlational Research Types of Validity

Convergent Validity
The extent to which scores from one procedure are positively correlated with scores obtained from another procedure that is already accepted as valid.

Correlational Research Types of Validity Discriminant Validity

The extent to which the scores obtained from one procedure are not correlated with the scores obtained from another procedure that measures other variables or constructs.

My new test which is intended to measure selfesteem is discriminantly valid if the scores do NOT correlate with scores on Rotters Locus of Control test.

Correlational Research Types of Validity Criterion Validity

The extent to which the scores from a procedure correlate with an observable behavior, such that the procedure is capable of distinguishing between subjects on the basis of that behavior. Two different types, predictive and concurrent validity.

Correlational Research Types of Validity

Predictive Validity

The extent to which a procedure allows for accurate predictions about a subjects future behavior.

Correlational Research Types of Validity

Concurrent Validity

The extent to which a procedure correlates with the present behavior of participants.

Correlational Research Where do you get data? Often, data can be gathered from other places. If studying relationship between ACT score and GPA, all I have to do is look it up! Other cases you may need to measure! If possible, use a published measuring device such as an IQ test. If no measure exists, make one!

Sampling Techniques
Probability Sampling Every potential participant in the population has an equal likelihood of being selected for participation in the study. Nonprobability Sampling Every potential participant does not have an equal likelihood of participating.

Sampling Techniques Probability Sampling

Simple

Random Sampling ALL members of population have equal chance, this is the ideal.

Sampling Techniques Probability Sampling

Systematic Random Sampling o Every nth member selected.

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50

Sampling Techniques Probability Sampling

Stratified Random Sampling Identify important subgroups, then proportionate random selection from them. If TWU has 60% female and 40% male, then, if you need N = 100, randomly sample 60 females and 40 males.

Sampling Techniques Probability Sampling

Cluster Sampling Certain groups are randomly selected, then all members of the group participate (i.e. a state might go to a few randomly selected schools to see if a program works).

Sampling Techniques Nonprobability Sampling

Convenience sampling Using those participants which are readily available.

Sampling Techniques Nonprobability Sampling Quota Sampling We ensure that the sample has the same percentage of each subgroup as the population. If the left/right handed ratio is 10/90, then we would ensure that we choose 1 lefty for every 9 right handed people. The people are chosen by convenience.

Sampling Techniques Nonprobability Sampling

Snowball Sampling Researcher identifies participants based on identification by previous participants. Useful if your population is hard to find. Members of rare populations tend to know each other and know how to find others like themselves. If you are studying people with Hodgkins Disease, most participants will know others from support groups or other places.

Small N Designs Why are they Good?

Sometimes that is the best we can do! Limited populations Control participant variables Reduces error variance No tricky statistics Reliability of effects are automatically checked

Small N Designs
Baseline

designs designs designs

A-B
Reversal

A-B-A
Multiple-baseline

A-B-A-B

Variations A-B-A-C-A-B-A

Quasi Experiments
Participants

assigned to conditions based on preexisting traits Quasi-experiments are ultimately correlational

9 8 7 6 5 4 3 2 1 0 0 20 40 Age 60 80

8 7

Driving Abililty

Driving Ability

6 5 4 3 2 1 0 Young Adult Middle Adult Age Old Adult

Quasi-Experiments Sources of Quasi-Independent Variables

Quasi-independent variables come from three sources Participant variables Environmental events Passage of time

Quasi-Experiments Participant Variables

Age

Gender
Height

Often

we want to create groups based on a pretest score perhaps top and bottom 25% on a selfesteem test

Quasi-Experiments Environmental Variables

Might

come from common experiences People that live in LA vs. NY People that experienced an earthquake Time-series designs

Quasi-Experiments Environmental Variables-Time Series Designs

Quasi-Experiments Environmental Variables-Time Series Designs

Nonequivalent

Control Group Design

SF
Seattle Before After

Quasi-Experiments Environmental Variables-Time Series Designs

Interrupted

time-series design

Quasi-Experiments Environmental Variables-Time Series Designs

Multiple

time-series design

Presidents

Others

Quasi-Experiments Time Variables

Maturation

studies Changes with Experience Changes with time

Quasi-Experiments Time Variables

Longitudinal

designs
O O O O O O

Quasi-Experiments Time Variables

Quasi-Experiments Time Variables