Sie sind auf Seite 1von 14

Dr.

William Kritsonis
Professor
McNeese State University
Procedures in Educational Research

Study Guide 1 – Introduction to Educational Research


1. Why should we study educational research?
2. What are the definitions of the five general kinds of research?
3. Why is it important that both basic and applied research be done?
4. What is the difference between formative and summative evaluation?
5. What is the key question associated with each of the following forms of
evaluation: needs assessment, theory assessment, implementation assessment,
impact assessment, and efficiency assessment?
6. What are the different sources of knowledge? Which ones are especially
important for educational researchers?
7. What is the key difference between inductive reasoning and deductive reasoning?
8. Describe the two forms of the scientific method, and explain why both are
important.
9. Explain why researchers do not use the word “proof” when they write up the
results of their research in journal articles.
10. What criteria can you use to determine the quality of a theory or an explanation?
11. What does the principle of evidence state?
12. What are the five main objectives of educational research? (Hint: the first letters
form this acronym, EDEPI.)
13. Why is each of the five main objectives of research/science important?

Study Guide 2 – Quantitative, Qualitative, and Mixed Research


1. Describe the key features of quantitative and qualitative research.
2. Describe the key features of mixed research.
3. What is the difference between a categorical variable and a quantitative variable?
Think of an example of each.
4. Why is experimental research more effective than non-experimental research
when a researcher is interested in studying cause and effect?
5. What are the main problems with the simple cases of causal-comparative and
correlational research?
6. What are two variables that you believe are positively correlated?
7. What are two variables that you believe are negatively correlated?
8. What are the different types of qualitative research, and what is the defining
feature of each of these?
9. What are the different types of mixed research, and what is the defining feature of
each of these types of research?
10. What are the three research paradigms in education and what are the major types
of research in each of these paradigms?
Study Guide 3 – Developing Research Questions and Proposal Preparation
1. What sources of research ideas have been identified by investigators?
2. How would you get a research idea from each of these sources?
3. How do research ideas and questions differ from ideas and questions that cannot
be empirically researched?
4. What is the purpose of conducting a review of the literature in a quantitative
study?
5. What is the purpose of conducting a review of the literature in a qualitative study?
6. What are the information sources you would use in conducting a literature review
and what is the advantage of each?
7. Why is ERIC such a valuable resource for educational researchers?
8. What is the difference between directors, search engines, and meta-search
engines?
9. What are the advantages and disadvantages of using the public internet in
conducting a literature search?
10. How would you evaluate the validity of information obtained over the Internet?
11. What factors should you consider in determining whether it is possible for you to
conduct a study?
12. How do research problems in qualitative and quantitative research differ?
13. How does the statement of the purpose of a study differ in qualitative and
quantitative research?
14. How do research questions differ in quantitative and qualitative research, and
what is their purpose?
15. Why should research questions in quantitative research be very specific?
16. What is a hypothesis, and what is one criterion that it must meet?
17. Why hypotheses are typically not formulated in qualitative research, and what is
typically used instead?
18. What are the components of a research plan, and what is contained in each
component?
19. What is a meta-analysis, and why is the conclusion reached in a meta-analysis
study more valid than the conclusion reached in a single study?
Study Guide 4 – Research Ethics
1. What is the definition of ethics, and how does this definition relate to research?
2. How do the three approaches that are used in considering ethical issues in
research differ?
3. How do societal concerns relate to research ethics?
4. What are the professional issues involved in research ethics, and what is the
appropriate ethical behavior related to each of these issues?
5. Why is treatment of the research participant an ethical issue to be considered in
educational research when the potential for physical and psychological harm is
minimal?
6. What must a researcher do to ensure that his or her study is ethical?
7. What kinds of information does a consent form have to include?
8. Under what conditions can an investigator get a waiver of the requirement of
informed consent?
9. What is the difference between consent from a minor’s legal guardian and assent
from the minor, and why are both important?
10. What is the difference between active and passive consent, and what are the
advantages and disadvantages of each?
11. What is deception, and when is it used in a research study?
12. What are the ethical obligations of a research that makes use of deception?
13. Why can participants still feel pressured to participate in a study even after the
researcher has state that they can withdraw or decline to participate?
14. What are the issues relating to freedom to withdraw with respect to minors?
15. Why do educational researchers have to be concerned with protecting participants
from mental and physical harm in their studies?
16. What is the difference between confidentiality and anonymity?
17. What is the purpose of the IRB?
18. What kinds of information should be contained in a research protocol submitted to
the IRB?
19. What are exempt studies, and what type of studies meet the exempt criterion?
20. What is expedited review, and what type of studies would receive expedited
review?
Study Guide 5 – Standardized Measurement and Assessment
1. What is measurement?
2. What are the four different levels or scales of measurement and what are the
essential characteristics of each one?
3. What are the twelve assumptions underlying testing and measurement?
4. What is the difference between reliability and validity? Which is more important?
5. What are the definitions of reliability and reliability coefficient?
6. What are the different ways of assessing reliability?
7. Under what conditions should each of the different ways of assessing reliability
be used?
8. What are the definitions of validity and validation?
9. What is meant by unified view of validity?
10. What are the characteristics of the different ways ob obtaining validity evidence?
11. What are the purposes and key characteristics of the major types of tests discussed
by researchers?
12. What is a good example of each of the major types of tests that are discussed by
researchers?

Study Guide 6 –Methods of Data Collection


1. What is a method of data collection?
2. What are the six main methods of data collection? (Hint: the first letters make the
rather awkward acronym, SQIFOS)
3. What principles should you follow when constructing a questionnaire?
4. What is an example of a leading or loaded question?
5. What is an item stem?
6. If you are conducting an exploratory research study, are you more likely to use
closed-ended questions or open-ended questions?
7. How many points should a rating scale have?
8. When should you use a contingency question?
9. Explain how to pilot test a questionnaire or an interview protocol.
10. What is the difference between a quantitative and a qualitative interview?
11. Why would a researcher want to conduct a focus group?
12. What are the main differences between quantitative and qualitative observations?
13. What are the four main roles that a researcher can take during qualitative
observation?
14. What is the difference between frontstage and backstage behavior?
15. What are some examples of secondary or existing data?
Study Guide 7 - Sampling
1. What type of sampling produces representative samples?
2. What is a representative sample, and when is it important to obtain a
representative sample?
3. What is the difference between a statistic and a parameter?
4. What is a sampling frame?
5. How doe you select a simple random sample?
6. What do all of the “equal probability selection methods” (i.e., EPSEM) have in
common?
7. What are the three steps for selecting a systematic sample?
8. How do you select a stratified sample?
9. What is the difference between “proportional” and “disproportional” stratified
sample?
10. When might a researcher want to use cluster sampling?
11. Are convenience samples used very often by experimental researchers?
12. If your goal is to generalize from a sample to a population, then which is more
important: random selection or random assignment?
13. If your population size is 250,000, then how many participants will you need, at a
minimum for your research study?
14. Sampling in qualitative research is similar to which type of sampling in
quantitative research?
Study Guide 8 – Validity of Research Results
1. What is a confounding variable, and why do confounding variables create
problems in research studies?
2. Identify and define the four different types of validity that are used to evaluate the
inferences made from the results of quantitative studies.
3. What is statistical conclusion validity, and what is the difference between null
hypothesis significance testing and effect size estimation?
4. What is internal validity, and why is it so important in being able to make causal
inferences?
5. What are the two types of causal relationships, and how do these two types of
causal relationships differ?
6. What type of evidence is needed to infer causality, and how does each type of
evidence contribute to making a causal inference?
7. What is an ambiguous temporal precedence threat, and why does it threaten
internal validity?
8. What is a history threat, and how does it operate?
9. What is a maturation threat, and how does it operate?
10. What is a testing threat, and why does it exist?
11. What is an instrumentation threat, and when would this threat exist?
12. What is a regression artifact threat, and why does this threat exist?
13. What is a differential selection threat, and when would this threat exist?
14. What is meant by an additive and interactive effect as a threat to internal validity?
15. How does differential attrition threaten internal validity?
16. What is external validity, and why is it important?
17. What is population validity, and why is it difficult to achieve?
18. What is ecological validity?
19. What is temporal validity?
20. What is treatment variation validity, and why can this be a threat to external
validity?
21. What is outcome validity?
22. What is construct validity, and how is it achieved?
23. What is operationalism, and what is its purpose?
24. What is multiple operationalism, and why is it used?
25. What is meant by research validity in qualitative research?
26. Why is researcher bias a threat to validity, and what strategies are used to reduce
this effect?
27. What is the difference between descriptive validity, interpretive validity, and
theoretical validity?
28. How is external validity assessed in qualitative research, and why is qualitative
research typically weak on this type of validity?
Study Guide 9 – Experimental Research
1. What is an experiment, and what are the significant components of this
definition?
2. What are the different ways a researcher can use to manipulate an independent
variable?
3. What is meant by the term experimental control, and how is experimental control
related to differential influence within the experiment?
4. What is random assignment, and what is the difference between random
assignment and random selection?
5. How does random assignment accomplish the goal of controlling for the influence
of extraneous or confounding variables?
6. How would you implement the control technique of matching, and how does this
technique control for the influence of confounding variables?
7. How would you use the control technique of holding the extraneous variable
constant?
8. When would you want to build the extraneous variable into the research design?
9. What is counterbalancing, and when would you use it?
10. What is the difference between a carryover effect and an order effect?
11. What is analysis of covariance, and when would you use it?
12. What is a research design, and what are the elements that go into developing a
research design?
13. When would the one-group posttest-only design be used, and what are the
problems encountered in using this design?
14. When would you use the one-group posttest-only design, and what are the
potential rival hypotheses that can operate in this design?
15. When would you use the posttest-only design with nonequivalent groups, and
what are the potential rival hypotheses that can operate in this design?
16. What makes a design a strong experimental design?
17. What is the difference between an experimental and a control group?
18. What functions are served by including a control group into a research design?
19. What potentially confounding extraneous variables are controlled in the pretest-
posttest control group design?
20. What potentially confounding extraneous variables are controlled in the posttest-
only control-group design, and how does the design control them:
21. What is a factorial design, and what is the advantage of this design over the two-
group posttest-only design (e.g., the posttest-only control group design with two
groups)?
22. What is a main effect?
23. What is an interaction effect, and what is the difference between an ordinal and a
disordinal interaction?
24. What is the difference between a factorial and a repeated measures-design?
25. What are the advantages and disadvantages of factorial and repeated-measures
designs?
26. What is a factorial designed based on a mixed model, and when would it be used?
Study Guide 10 – Quasi-Experimental and Single-Case Designs
1. What is a quasi-experimental design, and when do you use such a design?
2. What requirements must be met to reach a valid causal inference when using a
quasi-experimental design?
3. What is a nonequivalent comparison-group design and what are the essential
features of this design?
4. How are rival explanations addressed when using the nonequivalent comparison-
group design?
5. What types of biases can exist when using the nonequivalent comparison-group
design?
6. What is the best way of determining whether a threat is plausible when using the
nonequivalent comparison-group design?
7. What are the essential design characteristics of an interrupted time-series design?
8. How is a treatment effect demonstrated when using an interrupted time-series
design?
9. How are potential confounding variables ruled out when using the interrupted
time-series design?
10. What are the essential characteristics of the A-B-A design?
11. How does the A-B-A design rule out rival hypotheses and demonstrate the effect
of an experimental treatment condition?
12. What are the primary problems that can exist when using the A-B-A design, and
how can they be solved?
13. How does the multiple-baseline design demonstrate a treatment effect?
14. What is the primary problem that can be encountered in using the multiple-
baseline design?
15. When would you use the changing-criterion design?
16. What are the essential characteristics of the changing-criterion design?
17. Identify and discuss the four methodological issues that must be considered when
using a single-case research design.
Study Guide 11 – Nonexperimental Quantitative Research

1. The three necessary conditions for establishing cause and effect:


Condition 1: Variable A and variable B must be related (the
relationship conditions).
Condition 2: Proper time order must be established (the temporal
antecedence condition).
Condition 3: The relationship between variable A and variable B
must not be due to some confounding extraneous or “third” variable (the
lack of alternative explanation condition).

2. Why must a researcher sometimes conduct non-experimental research rather than


experimental research?
3. Why must researchers watch out for the “post hoc fallacy”?
4. Name a potential independent variable that cannot be manipulated.
5. Explain the problems with the simple cases of causal-comparative and correlation
research. Why is a researcher not justified in making a cause and effect claim
from these two cases?
6. Explain exactly how strong experimental research fulfills each of the three
necessary conditions of cause and effect.
7. On which of the three conditions for cause and effect is non-experimental
research especially weak? On which one of the three necessary conditions is non-
experimental research strong?
8. Explain why you cannot make a defensible “causal claim” based on an observed
relationship between two variables (e.g., gender and achievement) in non-
experimental research.
9. What is the purpose of the techniques of control in non-experimental research?
10. Which form of non-experimental research tends to be the best for inferring cause
and effect: cross-sectional research, trend studies, panel studies (i.e., prospective
studies), or retrospective research studies? Why?
11. Explain the difference between a direct effect and an indirect effect.
12. List an advantage and a disadvantage of causal modeling.
13. What are the two dimensions used to classify non-experimental research into nine
designs?
Study Guide 12 – Qualitative Research
1. What are the key characteristics of qualitative research?
2. Explain the role of induction in qualitative research?
3. Why is it said that qualitative research does not follow a series of steps in a
“linear fashion”?
4. Why is qualitative research important for educational research?
5. What are the key characteristics of phenomenology?
6. How does the researcher analyze the data collected in a phenomenology?
7. What are the key characteristics of ethnography?
8. What is the difference between a “macro” and a “micro” culture?
9. How do people become members of cultures?
10. What is the difference between the emic and the etic perspective?
11. What are they key characteristics of case study research?
12. What is a case?
13. Define intrinsic case study, instrumental case study, and collective case study.
14. What are the key characteristics of grounded theory?
15. What are the four important characteristics of grounded theory?
16. When does the researcher stop collecting data in grounded theory research?

Study Guide 13 – Historical Research


1. Define historical research.
2. Why would a person want to conduct historical research?
3. How can historical research tell us anything about the present?
4. What are the steps involved in the conduct of historical research?
5. Identify the sources of historical research topics?
6. What type of information is used when conducting a historical research study?
7. Where would you find the historical information needed for a historical study?
8. What is the difference between a primary and secondary source?
9. What is the difference between external criticism and internal criticism?
10. What is meant by positive criticism?
11. What is meant by the terms “vagueness” and “presentism” and how do they relate
to positive criticism?
12. What is negative criticism, and how does a person conducting a historical study
achieve negative criticism?
13. What methodological problems might a person encounter when synthesizing
material and preparing the narrative report?
Study Guide 14 – Mixed Method and Mixed Model Research
1. What position does the mixed researcher take on the compatibility thesis and
pragmatist philosophy?
2. Why is the fundamental principle of mixed research important?
3. Give an example of a within-stage mixed model research study.
4. Give an example of an across-stage mixed model research study.
5. What is the difference between mixed model research and mixed method
research?
6. What kind of study does this notation imply: qualQUANqual? Can you
think of why a researcher might use such a design?
7. What is the difference between a sequential and a concurrent design feature?
8. What are the eight stages of the mixed research process?
9. Explain each of the five rationales for conducting a mixed research study.
10. What is the difference between quantizing and qualitizing, and are these used in
mixed method or mixed model designs?
11. What kinds of validity might be relevant to a mixed design?
12. What are the four potential problems involved in writing and attempting to
publish a mixed research report?

Study Guide 15 – Descriptive Statistics


1. What is the difference between descriptive statistics and inferential statistics?
2. List the three steps in constructing a frequency distribution.
3. What types of graphical representations of data are typically used in conducting research?
4. Which graphical representation is used to examine the correlation between two
quantitative variables?
5. What is a measure of central tendency, and what are the common measures of central
tendency?
6. When is the median preferred over the mean?
7. If the mean is much greater than the median, are the data skewed to the right or skewed to
the left?
8. What is a measure of variability, and what are the common measures of variability?
9. How are the variance and standard deviation mathematically related?
10. If a set of data is normally distributed, how many of the cases fall within one standard
deviation? How many fall within two standard deviations? How many fall within three
standard deviations?
11. What is a measure of relative standing, and what are the common measures of relative
standing?
12. How do you calculate a z-score?
13. What are some of the different ways to examine the relationships among variables?
14. If you calculate the percentages in a contingency table down, then should you make your
comparisons down the columns or across the rows?
15. What is the difference between simple regression and multiple regressions?
16. How is the regression coefficient interpreted in simple regression?
17. How is the regression coefficient interpreted in multiple regression?
Study Guide 16 – Inferential Statistics
1. What is the difference between a statistic and a parameter?
2. What is the symbol for the population mean?
3. What is the symbol for the population correlation coefficient?
4. What is the definition of a sampling distribution?
5. How does the idea of repeated sampling relate to the concept of a sampling
distribution?
6. Which of the two types of estimation do you like the most, and why?
7. What are the advantages of using interval estimation rather than point estimation?
8. What is a null hypothesis?
9. To whom is the researcher similar to in hypothesis testing: the defense attorney or
the prosecuting attorney? Why?
10. What is the difference between a probability value and the significance level?
11. Why do educational researchers usually use .05 as their significance level?
12. State the two decision making rules of hypothesis testing.
13. Do the following statements sound like typical null or alternative hypotheses?
(A) The coin is fair. (B) There is no difference between male and
female incomes in the population. (C) There is no correlation in
the population. (D) The patient is not sick (i.e.., is well). (E)
The defendant is innocent.
14. What is a Type I error? What is a Type II error? How can you minimize the risk
of both of these types of errors?
15. If a finding is statistically significant, why is it also important to consider practical
significance?
16. How do you write a null and alternative hypotheses for each of the following:
(A) The t-test for independent samples. (B) One-way analysis of
variance. (C) The t-test for correlation coefficients?
(D) The t-test for a regression coefficient.
Study Guide 17 – Data Analysis in Qualitative Research
1. What is interim analysis?
2. What is memoing?
3. Why is it important to transcribe qualitative data when possible?
4. What is the difference between segmenting and coding?
5. What is the difference between inductive and a priori codes?
6. What is the difference between co-occurring and facesheet codes?
7. Explain the process of enumeration.
8. What is a hierarchical category system, and why can it be useful to construct
hierarchical systems?
9. How do qualitative researchers show relationships among categories?
10. How are network diagrams used in qualitative research?
11. What are the five types of validity that are of potential importance in qualitative
research, and what are their definitions?
12. What are the thirteen strategies that are used to promote validity in qualitative
research, and what are their definitions?
13. What are some of the capabilities of computer programs for data analysis?
14. What are some of the leading qualitative data analysis computer programs?

Study Guide18 – Preparation of the Research Report


None

Copyright © 2004 William Kritsonis, PhD


ALL RIGHTS RESERVED/FOREVER