MEASURING

COMPLEX ACHIEVEMENT with ESSAY QUESTIONS

Essay
a short literary composition on a particular theme or subject, usually in prose, and generally analytic, speculative, or interpretative.

Whats the diff?

Objective Test items measure a variety of simple learning outcomes ex. recall of memorized information Interpretive exercises measures complex achievement.
Higher mental processes Understanding Reasoning Critical thinking Scientific thinking Creative thinking Problem solving

Essay Question:
Especially useful for measuring those aspects of complex achievement that cannot be measured well by more objective means:

the ability to supply rather than identify

interpretations and applications of data the ability to organize, integrate, and express ideas in a general attack on a problem.

INTEGRATE BIG IDEAS.

The BIG IDEA OF ESSAY IS...

Forms of Essay
Restricted Response Essay Question
Usually limits both the content and response. The content is usually restricted by the scope and the topic to be discussed. It is most useful for measuring learning outcomes requiring the interpretation and application of data in a specic area. Example: ON MEASURING THE ABILITY TO INTERPRET AND APPLY DATA 1) Describe TWO situations that demonstrate the law of Supply and Demand. 2) Explain how the RH bill is both an bane and a boon in the country.

Extended Response Essay

Allows students to select any factual information that

they think is pertinent, to organize the answer in accordance with their best judgment, and to integrate and evaluate ideas they deem appropriate.

However, too creative expression and other higher-order

skills makes this type inecient for measuring more specic learning outcomes.

Example: Compare medias freedom of speech in the 70s to the present day. Cite examples.

Restricted-Response Essay Questions

The restricted-response question usually limits both the content and the response. The content is usually restricted by the scope of the topic to be discussed. Limitations on the form of response are generally indicated in the question.

Restricted-Response Essay Questions

Examples 1. Describe two situations that demostrate the application of the law of supply and demand. Do not use those examples discussed in class. 2. Why is the barometer one of the most useful instreuments for forecasting weather? Answer in a brief paragraph.

Extended-Response Essays

The extended-response question or assignment allows students to select any factual information that they think is pertinent, to organize the answer in accordance with their best judgment, and to integrate and evaluate ideas as they deem appropriate.

Extended-Response Essays

Advantage: - ability to analyze problems, organize their ideas, describe in own words, and/or develop coherent argument. Disadvantage: - inefficient for measuring more specific learning outcomes and introduces scoring difficulties.

Extended-Response Essays

Example 1. Compare developments in international relations in the administration of President Gloria Macapagal-Arroyo and President Ejercito Estrada. Cite examples when possible.

Extended-Response Essays

Example 2. Evaluate the significance of the sea captains pursuit of the white whale in Moby Dick. 3. Describe the influence of Mendels law of heredity on the development of biology as a science.

Learning Outcomes Measured

Restricted-Response VS Extended-Response

 Restricted-response is similar to

objective interpretive exercise in measuring complex learning outcome.

Difference: Objective Interpretive select answer Restricted supply answer

Restricted-response essay measures ability to:

Explain cause-and-effect relationships Describe applications of principles Present relevant arguments Formulate tenable hypotheses Formulate valid conclusions State necessary assumptions Describe the limitations of data Explain methods and procedure

 Extended-response essay

assessments measure more general learning outcome.

Extended-response essay measures ability to:

Produce, organize, and express ideas Integrate learning in different areas Create original forms (e.g., designing an

experiment) Summarize (e.g., writing a summary of a story) Construct creative stories (e.g., narrative essays) Explain concepts or principles (e.g., expository essay) Persuade a reader (e.g., persuasive essay)

Restricted- Response Essay

Asks the learner to demonstrate an

understanding of the given material Measures comprehension More structured, more specific Useful for measuring learning outcomes requiring the interpretation and application of data in a specific area

Extended-Response Essay
Students use judgment; integrates and

evaluates ideas they deem appropriate Emphasizes integration, organization, and originality Creative process, places more value on higher-order skills

Advantages of Essay Questions

Measures complex learning outcomes that

cannot be measured by other means Emphasizes the integration and application of thinking and problem-solving skills Enables the direct evaluation of writing skills Easy Construction Direct contribution to student learning

Limitations of Essay Questions

Unreliability of scoring X clear identification of learning

outcomes being measured X well-defined scoring rubrics Amount of time required for scoring the responses Limited sampling of content that essay questions provide

Suggestions for constructing essay questions

1.

Restrict the use of essay questions to those learning outcomes that cannot be measured satisfactory by objective items.

2. Construct questions that will call forth the

skills specified in the learning standards.

3.

Phrase the question so that the students task is clearly indicated.

 4. Indicate an appropriate time limit for

each question.
5. Avoid the use of optional questions.

Scoring Criteria
Clear specification of criteria leads to an

improved Reliability and Validity of Assessment

Informing the students of these criteria

before administering the exam will give them an idea what to expect and what to do.

Scoring Rubrics (grading tools)

Rubrics is a way of describing evaluation

criteria based on the expected outcomes and performances of students. (Ebert-May, 1997)
Anchor Responses -- most clearly

correspond to the levels of scoring rubric.

Scoring Rubrics (grading tools)

Must be descriptive (clear) rather than

judgmental in nature
Reliability, Comparability and Fairness

of Scores

 Poor:
Excellent

Better:
Writing is clear and thoughts are

complete.

Restricted-response essay questions

1. Why is the barometer one of the most

useful instruments for forecasting weather? Answer in a brief paragraph.

2. Describe two situations that demonstrate

the Application of the law of supply and demand. Do not use those examples discussed in class.

Restricted-response essay questions

Make a list of possible answers

Full credit Partial credit

 1. What is geography? Dont use the definition

discussed in class.
Study of Earth as the home of man. study of the surface of the earth and its relation to

the activities of man.

study of man and earth study dealing with man and his relationship with

Extended-response essay questions

1. Evaluate the significance of the sea

captains pursuit of the white whale in the Philippine deep.

2. Describe the influence of Mendels laws

of heredity on the development of biology as a science.

Analytic Scoring Rubrics

Separate scores for certain characteristics

gives students clearer feedback on their strengths and weaknesses.

Finer distinctions or specifications are

useful especially in scoring expository essays and descriptive summaries.

Analytic Scoring Rubrics

State of Oregon statewide writing

assessment rubrics:
1. Ideas 2. Organization 3. Voice 4. Word Choice 5. Sentence Fluency 6. Conventions

Analytic Scoring Rubrics

Narrative Essay rubrics developed by Wilf,

Gearhart, Quellmalz and Whittaker (1992) 1. Theme 2. Character 3. Setting 4. Plot 5. Communication

Holistic Scoring Rubrics

Can be constructed more rapidly Can be used to score a set of essay

responses more rapidly than analytic scoring rubrics.

Doesnt provide accurate feedback, but

teachers could write comments.

http://www.azed.gov/wpcontent/uploads/PDF/OfficialScoringGuide6ptHolistic_6_10.pdf

Suggestions for scoring essay questions

answer in advance.

1. Prepare an outline of the expected

2. Use a scoring rubric appropriate to the type of question used.

Example: Rubric for oral presentation (http://

www.cbd.int/ibd/2008/resources/teachers/ appendix3.shtml)

3. Control the influence of irrelevant factors such as: a. Legibility of handwriting b. Spelling c. Sentence structure d. Punctuation e. Neatness

4. Evaluate answers without looking at the students name

5. Obtain two or more independent ratings when important decisions are to be made.

Suggestions for scoring essay questions

1. Prepare an outline of the expected answer in advance.

2. Use a scoring rubric appropriate to the type of question used.

3. Control the influence of irrelevant factors such as: a. Legibility of handwriting b. Spelling c. Sentence structure d. Punctuation e. Neatness

4. Evaluate all responses to one question before going on to the next one.

5. Evaluate answers without looking at the students name.

6. Obtain two or more independent ratings when important decisions are to be made.