A Review of Assessment Learning1

Item Analysis - refers to the process of examining the students response to each item in the test. Two Characteristics of an Item:
Desirable --------> RETAINED Undesirable -----> REVISED or REJECTED

Three criteria in determining the desirability and undesirability of an item:

a. Difficulty of an item. b. Discriminating power of an item. c. measures attractiveness

Difficulty Index refers to the proportion of the number of the students in the upper and lower groups who answered an item correctly.

Index of Discrimination
Discrimination Index is the difference between the proportion of high performing students who got the item right and the proportion of low performing students who got an item right.

Positive Discrimination if the proportion of students who got an item right in the upper performing group is greater than the proportion of the low performing group.

Negative Discrimination- if the proportion of the students who go an item right in the low performing group is greater than the students in the upper performing group. Zero Discrimination- if the proportion of the students who got an item right in the upper performing group and low performing group are equal. Maximum Discrimination- is the sum of the proportion of the upper and lower groups who answered the item correctly.
Discriminating discrimination discrimination. Efficiency divided by is the index of the maximum

Measures of Attractiveness - count the number of students who answered the incorrect option in both the upper and the lower group.
Steps of Item Analysis 1. Rank the scores of the students from highest score to lowest score. 2. Select 27% of the papers within the upper performing group and 27% with the lower performing group. 3. Set aside the 46% of papers because they will not be used for item analysis. 4. Tabulate the number of students in the upper group and lower group who selected each alternative. 5. Compute the difficulty of each tem. 6. Compute the discriminating powers of each item. 7. Evaluate the effectiveness f the distracters.

Validity of a Test
Validity - refers to the appropriateness of scorebased inferences; or decision made based on the students test result.
Important Things to Remember About Validity 1. Validity refers to the decisions we make and not to the test itself or to the measurement. 2. Like reliability, validity is not all or nothing concept; it is never totally absent or absolutely perfect. 3. A validity estimate, called validity coefficient, refers to specific type of validity. It ranges between 0 to 1. 4. Validity can never be finally determined; it is specific to each administration of the test.

Types of Validity
1. Content Validity - a type of validation that refers to the relationship between a test and the instructional objectives, establishes content so that the test measures what is supposed to measure. related Validity- a type of validation that refers 2. Criterionto the extent to which scores from a test relate to theoretically similar measures. 3. Construct Validity- a type of validation that refers to the extent to which a test measures a hypothetical and unobservable variable or quality. 4. Predictive Validity- a type of validation that refers to the extent to which a persons current test results can be used to estimate accurately what that persons performance or other criterion, such as test scores will be at the later time.

Factors that Affect the Validity of a Test

1. Poorly constructed test items 2. Unclear directions 3. Ambiguous items 4. Reading vocabulary too difficult 5. Complicated syntax 6. Inadequate time limit 7. Inappropriate level of difficulty 8. Unintended clues 9. Improper arrangement of items

Test Design to Improve Validity

1. What is the purpose of the test?
2. How well do the instructional objectives selected for the test represent the instructional goals? 3. Which item format will best measure achievement of each objective?

4. How many test items will be required to measure the performance adequately on each objective?
5. When and how will the test be determined?