One

Statement

Problem

Analyzing and critiquing Study 1

1. Cite problem statement as stated in study.

Does appear on the first page?

No (1)

Yes (5)

2. Rewrite problem statement in your own word

Was the author version clear enough to do this easily?

5 4 3
2 1 NA
3. Evaluate problem in term of each of criteria below. Give both a verbal
description, particularly of shortcoming, and a numerical rating
a. Workability
5 4 3 2 1 NA
b. Critical mass
NA

4 3

2 1

c. Interest

4 3

2 1 NA

d. Theoritical value

4 3

2 1 NA

e. Pratical value
NA

4 3

2 1

Two
Variables
Analyzing and critiquing Study 1
1. What is (are) the independent variable(s)?

2.

What is (are) the moderator variable(s), if any?

3. What is (are) the dependent variable(s)?

4. What are the important control variables?

5. What are two possible intervening variables?

6. Are there any important variables that might have been included but
were not? If so, list them and label them as to type

Three
Hypotheses
Analyzing and critiquing Study 1
1. Were hypotheses cited in the study?
Yes (5) No (1)
If yes, cite them.

2. Rewrite hypotheses in your own words.

f. Was the author version clear enough to do this easily?

2 1 NA

4 3

3. For each hypotheses answer the following:

a. Is it specific directional (5), General directional (4), or null (3)?*
5

4 3

2 1 NA

b. Is it supported by sufficient literature or argument? If 3 or lower

offer specific critism)
5
3 2 1 NA

c. Is it clear and understandable? (If you rated 3 or lower, indicate how

you think it might have been improved)
5 4
3 2 1 NA
d. It is consistent with the methodology and design? (If you rated 3 or
lower, indicated in what ways they are inconsistent.)
5 4 3 2 1 NA
*)A specific directional hypothesis call for differences and specific their
direction; a general directional hypothesis call for differences but does not
specify their direction; a null hypothesis call for no differences

Four
Supportive
materials
Analyzing and critiquing Study 1
1. What, in a sentence, is the context of the problem?

Is the context clear and sufficiently establish?

3 2 1 NA

2. a.

What is the magnitude of the literature review?

1 NA

4 3 2

b. How empirical and up to date is it?

1 NA

4 3 2

c. Does it bear on the variables and the hypotheses?

3 2 1 NA

d. Is it well organized (introduction, subheadings, summary)?

3 2 1 NA

e. Try to summarize it in a sentence or two.

1 NA

4 3 2

3. a.

Do the researchers attempt to establish the significance of the study

using argument
and/or supportive literature?
b. Is the attempt convincing?
1 NA

4 3 2

c. Summarize it in a sentence or two

d. Suggest a point of significance that might be added

Five
Operational
definition
Analyzing and critiquing Study 1
1. How did the researchers operationalize each of the independent
variables?

Does each OD seem sufficiently operational and unique?5

NA
(If 3 or lower, give specific criticism)

4 3 2

2. How did the researchers operationalize each of the moderator variable, if

any?

Does each OD seem sufficiently operational and unique?5

NA
(If 3 or lower, give specific criticism)

4 3 2

Does each OD seem sufficiently operational and unique?5

NA
(If 3 or lower, give specific criticism)

4 3 2

3. How did the researchers operationalize each of the dependent variables?

4. How did the researchers operationalize each of the important controle

variables?

Does each OD seem sufficiently operational and unique?5

NA
(If 3 or lower, give specific criticism)

4 3 2

Note: When DO are lacking, try to write them in yourself based on the
methods used.

Six Measurement and manipulation of

independent & dependent variables
dependent

Analyzing and critiquing Study 1

1. For each independent variable answer each of the following:

a. Was it measured or manipulated?

b. Describe briefly how this was done

c. How suitable (or valid and reliable) can you tell the manipulation or
measurement was on the basis of both face validity and
documentation?
5 4 3 2 1 NA

2. For each dependent variables answer each the following:

a. Describe briefly how it was done?

b. How valid and reliable was its measurement as documented?

5 4
3 2 1 NA
(If 3 or lower, give a specific criticism or suggestion for improvement)

Seven Measurement and manipulation of

moderator & control variables

Analyzing and critiquing Study 1

1. For each moderator variable answer each of the following:
a. Was it measured or manipulated? (If no moderator variable was used,
write none)

b. Describe briefly how this was done

c. How suitable (or valid and reliable) can you tell the manipulation or
measurement was on the basis of both face validity and
documentation?
5 4 3 2 1 NA
(If you rated 3 or lower, make a suggestion for improvement)

2. For each major control variables (up to three) answer each the following:
a. Was it measured or manipulated?
b. Describe briefly how it was measured.

c. How suitable (or valid and reliable) can you tell the manipulation or
measurement was on the basis of both face validity and documented?
5 4 3 2 1 NA
(If 3 or lower, give a specific criticism or suggestion for improvement)

Eight

Research
Design

Analyzing and critiquing Study 1

1. Was the design a true experimental one, quasi experiment, ex post facto,
or a non design?
Diagram the design.

2. How did the researchers control for selection bias? Be specific.

3. How did the researchers control for history bias? Be specific.

4. Were there any other sources of bias particularly controlled for or

particularly uncontrolled for?

If yes, describe the procedure(s) or lack of any

5. How internally valid was the design?
1 NA
(If 3 or lower, describe how it might have been improved)

6. Describe the population to which the results can be generalized.

7. How externally valid was the design?
5
1 NA
(If 3 or lower, suggest an improvement to increase it)

Nine

4 3 2

4 3 2

Statistical
Analysis

Analyzing and critiquing Study 1

1. Were statistical test used?
If yes, name them

Yes

No

2. Were the statistical test used proper ones or the most suitable ones to the
design?
5 4 3 2 1 NA
(If 3 or lower, suggest another)

3. Was their use clearly described?

NA
(If 3 or lower, describe the source of confusion)

4 3 2

4. Should the researchers have done additional statistical tests or presented

additional statistical results?
Yes No
If yes, name them

5. Was there a presentation of statistical results in graphic or tabular form?

If yes, could it (they) be understood and interpreted?
5 4 3 2
1 NA
(If 3 or lower, describe the source of confusion and suggest how it might
have been overcome).

Ten

Presentation and
discussion results

Analyzing and critiquing Study 1

1

a. Briefly describe the findings of the study

c. Does the article include an adequate and clear summary of the authors
findings?
5 4 3 2 1
NA
(if 3 or lower, offer a specific criticism)

d. Were the findings clearly related to the studys problem and

hypotheses?
5 4 3 2
NA
(If 3 or lower, offer a criticism)

a. Give a brief interpretation of the findings to indicate why they

came out as they did or what they mean.
b. Did the authors offer and support meaningful and convincing
interpretation of the findings that helped you understand the results?
5 4 3 2 1 NA
(If 3 or lower, offer a specific criticism or a useful interpretation)

3. a. Briefly state a conclusion that could be drawn from the study and a
recommendation based on that conclusion.

b. Were meaningful conclusion and recommendations offered?

3 2 1 NA
(If 3 or lower, offer a suggestion).