Beruflich Dokumente
Kultur Dokumente
Validity Outline
1. Definition 2. Validity: Two Different Views 3. Types of Validity
A. Face B. Content C. Criterion
i. Predictive vs. Concurrent ii. Validity Coefficients
D. Construct
i. Convergent ii. Discriminant
Validity Definition
Validity measures agreement between a test score and the characteristic it is believed to measure The basic question is: are you measuring what you think youre measuring?
10
11
Cronbach (1980): Construct is basic, while Content & Criterion are subtypes.
12
13
14
Is each item relevant to domain? Is domain adequately covered or are parts of it left out? But if you are going to ask these questions, why not do it when creating the test?
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
CSEPT: We obtain evidence of validity by simultaneously defining the construct and developing instruments to measure it. This is bootstrapping.
30
31
32
33
34
35
Convergent evidence
36
37
Convergent validity
Example Health Index Scores correlated with age, number of symptoms, chronic medical conditions, physiological measures Treatments designed to improve health should increase Health Index scores. They do.
38
Discriminant validity
low correlations between new test and tests believed to tap unrelated constructs. evidence that the new test measures something unique
39
40
41
42
Person B
Neutral
Test 1 0 1 2 3 4 5 6
Test 2 0 1 2 3 4 5 6
44
45
46
47
48
49
50
Review
CSEPT: 1. Validity is a characteristic of evidence, not of tests. 2. Valid evidence supports conclusions drawn using test results 3. Validity is determined by social consequences of test
Borsboom et al. 1. Validity is not a methodological issue, but a substantive (theoretical) issue 2. A test of an attribute is valid if (a) the attribute exists, and (b) variation in the attribute causes variation in test scores
51
Review
CSEPT: 4. Validity can be established in three ways, though boundaries between them are fuzzy:
A. Content-related evidence B. Criterion-related evidence C. Construct-related evidence
Borsboom et al: 3. Its all the same validity: a test is valid if it measures what you think it measures 4. Validity is not mysterious
52
Review
CSEPT 5. Content-related evidence: do test items represent whole domain of interest? 6. Criterion-related evidence: do test scores relate to a criterion either now (concurrent) or in the future (predictive)? Borsboom et al. 5. These questions are properly part of the process of creating a test
53
Review
CSEPT 6. Construct-related evidence is obtained when we develop a psychological construct and the way to measure it at the same time. 7. A test can be reliable but not valid. A test cannot be valid if not reliable. Borsboom et al. 6. A test must be valid for a reliability estimate to have any meaning
54
Review
Blanton & Jaccard (2006) warn against over-interpretation of scores which are based on an arbitrary metric For an arbitrary metric, we have no idea how the test scores are actually related to the underlying dimension