1

Planning the Test

One essential step in planning a test is to decide why you are giving the test. (The word "test" is used although
we are using it in a broad sense that includes performance assessments as well as traditional paper and pencil
tests.)

Are you trying to sort the students (so you can compare them, giving higher scores to better students and
lower scores to poor students)? If so you will want to include some difficult questions that you expect only a
few of the better students will be able to answer correctly. Or do you want to know how many of the students
have mastered the content? If your purpose is the latter, you have no need to distribute the scores, so very
difficult questions are unnecessary. You will, however, have to decide how many correct answers are needed to
demonstrate mastery. Another way to address the "why" question is to identify if this is to be a formative
assessment to help you diagnose students' problems and guide future instruction, or a summative measure to
determine grades that will be reported to parents.

Airasian (1994) lists six decisions usually made by the classroom teacher in the test development process:
1. what to test, 2. how much emphasis to give to various objectives, 3. what type of assessment (or type of
questions) to use, 4. how much time to allocate for the assessment, 5. how to prepare the students, and 6.
whether to use the test from the textbook publisher or to create your own. Other decisions, such as whether to
use a separate answer sheet, arise later.

You, as the teacher, decide what to assess. The term "assess" is used
here because the term "assess" is frequently associated only with traditional
paper and pencil assessments, to the exclusion of alternative assessments
such as performance tasks and portfolios. Classroom assessments are
generally focused on content that has been covered in the class, either in the
immediate past or (as is the case with unit, semester, and end-of-course tests)
over a longer period of time. For example, if we were constructing a test for
preservice teachers on writing test questions, we might have the following objectives:

The student will:

1. Know the advantages and disadvantages of the major selection-types of questions.

2. Be able to differentiate between well and poorly written selection-type questions.
3. Be able to construct appropriate selection-type questions using the guidelines and rules that
were presented in class.

We could have listed only the topics we have covered (e.g., true-false questions, short-answer questions,
multiple-choice questions, and test format) instead of the objectives.

Now that we have made the what decision, we can move to the next step: deciding how much emphasis
to place on each objective. We can look at the amount of time in class we have devoted to each objective. We
can also review the number and types of assignments the students have been given. For this example, let's
assume that 20% of the assessment will be based on knowing the advantages and disadvantages, 40% will be
on differentiating between well written and poorly written questions, and the other 40% will be on writing
good questions. Now our planning can be illustrated with the use of a table of specifications (also called a test
plan or a test blueprint) as shown in example 1a below.

1a
Table of Specifications
# items/
Objectives/Content area/Topics Knowledge Comprehension Application
% of test

1. Know the advantages &

disadvantages of the major selection- 20%
types of questions.

2. Be able to differentiate between

well and poorly written selection-type 40%
questions

3. Be able to construct appropriate 40%

selection-type questions using the
 2

guidelines and rules that were

presented in class.

Back to Top of Page

A table of specifications is a two-way table that matches the objectives or content you have taught with
the level at which you expect students to perform. It contains an estimate of the percentage of the test to be
allocated to each topic at each level at which it is to be measured. In effect we have established how much
emphasis to give to each objective or topic.

We also have to take into account the type of thinking skills we wish to assess. Whether you use Bloom's
taxonomy or another structure, the levels of learning can help you identify
the types of questions (or other type of assessment) that are appropriate. For
ease of use we have used only three levels: knowledge (recall or
recognition), comprehension (or understanding) and application (or skill),
and labeled the columns accordingly. The important thing is to use levels of
thinking that are relevant for your students and have been incorporated in
your instruction. At this stage it can be helpful to mark an "x" or make a
check mark in the cells to show the levels at which each objective will be measured, as shown in example 1b.

1b
Table of Specifications
# items/
Objectives/Content area/Topics Knowledge Comprehension Application
% of test

1. Know the advantages & x

disadvantages of the major selection- 20% 20%
types of questions.

2. Be able to differentiate between 40%

well and poorly written selection-type x x
questions

3. Be able to construct appropriate 40%

selection-type questions using the x
guidelines and rules that were 40%
presented in class.

We can see that objective 1 (20% of the test) is to be measured totally at the Knowledge level, and
objective 3 at the Application level, allowing us to go ahead and enter that information in the Knowledge
column for objective 1 and the Application column for objective three in example 1b. Objective 2 is to be
measured at two different levels. So we must decide how to divide the 40% of the test for objective two
between Knowledge and Comprehension.

We may feel a need to include some questions in which students have to recognize guidelines for writing
questions (knowledge) along with examples of good and poor questions from which the students are to select
the better constructed questions (comprehension). If we have spent similar amounts of time and emphasis on
the two, we may allocate 20% to the Knowledge component and 20% to Comprehension for objective 2
(example 1c).

1c
Table of Specifications
# items/
Objectives/Content area/Topics Knowledge Comprehension Application
% of test

1. Know the advantages & x

disadvantages of the major selection- 20% 20%
types of questions.

2. Be able to differentiate between

x x
well and poorly written selection-type 40%
20% 20%
questions

3. Be able to construct appropriate 40%

selection-type questions using the x
guidelines and rules that were 40%
 3

presented in class.

At this point we recognize that 40% of our test is to be on knowledge, 20% on comprehension, and 40%
on application. This does not mean that we must have 40 knowledge questions; it does mean that the score on
the test will reflect knowledge and application in equal amounts, and comprehension to a lesser degree than
knowledge or application.

It may be that at this point you want to compare the test(s) provided by the textbook publisher with your
completed table of specifications. If they match and you think the questions are well written, you may decide
to use the test (or parts of the test) provided with the text. On the other hand, you may find that it will be
necessary for you to create a test to provide an accurate assessment of what the students in your class have
learned.

One question frequently asked is how many questions are needed to adequately sample the content
representing an objective or topic. Increasing the number of questions increases the probability that we will
have a good estimate of what the child knows and can do.

The number of questions and the type(s) of questions used both affect
the amount of time needed for completion of the test. Nitko (2001, p. 117),
provides some estimates of time to complete various types of questions for
junior and senior high school students. Oosterhof (2001, p. 161), gives
similar estimates but indicates that elementary students and poor readers
might need more time.

True-False questions 15-30 seconds per question

Multiple choice (recall questions that are brief) 30-60 seconds
More complex multiple choice questions 60-90 seconds
Multiple choice problems with calculations 2-5 minutes
Short answer (one word) 30-60 seconds
Short answer (longer than one word) 1-4 minutes
Matching (5 premises, 6 responses) 2-4 minutes
Short essays 15-20 minutes
Data analyses/graphing 15-25 minutes
Drawing models/labeling 20-30 minutes
Extended essays 35-50 minutes

These estimates provide information needed to decide what type(s) of questions and how many of them
to use. More true-false questions can be answered during a given period of time than multiple choice or short
answer questions. However, our choice of question types must be based on the level of learning at which we
are assessing our students. We can decide to use true-false and short-answer questions for the knowledge
component, and multiple choice for the comprehension. We will award the lowest number of points per
question (1 per question) for the easiest questions (in this case the true-false and short answer). This helps us
determine the number of true-false or short answer questions to include, as shown in example 1d.

Measuring objective 3, in our example, may be fairly time-consuming if we provide students with a
written selection and ask them to create a number of questions based on that selection. While this is a paper-
and-pencil task, it is basically a performance task. The table of specifications can include a mixture of
assessments so long as they are appropriate to your objectives and content. We are asking students to
demonstrate their ability to do something that our instruction has, hopefully, prepared them to do. Because this
is going to take more time than answering questions by selecting answers, we may decide to weight each
question constructed by the student at 5 times the number of points given for other questions.

1d
Table of Specifications
# items/
Objectives/Content area/Topics Knowledge Comprehension Application
% of test

1. Know the advantages & x 20%

disadvantages of the major selection- 20% 10 Q 10 pts
 4

types of questions. 10 Q x 1

2. Be able to differentiate between x x

40%
well and poorly written selection- 20% 20%
15 Q 20 pts
type questions 10 Q x 1 5Qx2

3. Be able to construct appropriate x

selection-type questions using the 40% 40%
guidelines and rules that were 4Qx5 4 Q 20 pts
presented in class.

TOTAL 20 Q 5Q 4Q 29 Q 50 pts

Some authors recommend putting the number of items in each cell, either along with or in place of the
percentage. If all questions are weighted equally in the scoring, this is appropriate. If, however, some more
difficult item types are given more points than other questions that are more susceptible to guessing (such as
true-false), the number of questions can be misleading and should be replaced by the number of points that can
be earned. A true-false question might be worth one point while a multiple-choice question or a short-answer
question might be worth two points.

In looking at the total number of questions at the bottom of each column, we see that our test needs 20
knowledge questions (that we have decided will be true-false) worth 1 point each, five comprehension
questions (multiple choice, 2 points each), and four application tasks (developing questions task, 5 points
each). In looking at the numbers of points in the right-hand column, we have a total of 50 points, with 20% for
objective 1 and 40% each for objectives 2 and 3. If we have carefully considered what we have been trying to
achieve with our students and that is accurately reflected in this table, our test will be a valid measure of
student learning and the effectiveness of our instruction.

In estimating the time needed for this test, students would probably need from 5 to 10 minutes for the 20
True-False questions (15-30 seconds each), 5-7 1/2 minutes for the five comprehension questions (60-90
seconds each), and 20-30 minutes (rough estimate) to read the material and write the four questions measuring
application. The total time needed would be from 30 to 48 minutes. If you are a middle or high school teacher,
estimated response time is an important consideration. You will need to allow enough time for the slowest
students to complete your test, and it will need to fit within a single class period.

Another consideration in planning a classroom test may be alignment with standardized tests used in your
state to measure similar areas of student learning. How are those tests constructed? What objectives are
measured on those tests? How are they measured; i.e., what kinds of items are used and what levels of learning
(knowledge, comprehension, application, etc.) are emphasized? On your classroom test you need to measure
what you have taught in the ways you have taught it, but in both the teaching and the testing, consider that
your work is part of a broader educational system.

The final step in planning the test will be to write the test questions. If more information is needed on
item writing, please consult the other modules that correspond to the types of questions of interest to you.
Accommodations
Accommodations may be needed for some of your students. It is helpful to keep those students in mind as
you plan your assessments. Some examples of accommodations include:
Providing written instructions for students with hearing problems

Using large print, reading or recording the questions on audiotape (The student could record the
answers on tape.)
Having an aide or assistant write/mark the answers for the student who has coordination problems, or
having the student record the answers on audiotape or type the answers
Using written assessments for students with speech problems
Administering the test in sections if the entire test is too long for the attention of a student
Asking the students to repeat the directions to make sure they understand what they are to do
Starting each sentence on a new line helps students identify it as a new sentence
Including an example with each type of question, showing how to mark answers
B2. Constructing and Assembling The Test
Before beginning to construct your own test, you may want to compare your table of specifications with
 5

test items provided by the publisher or other sources to see what, if anything, from those sources can be
incorporated into your assessment.
* Begin with simpler item types, then proceed to more complex, from easy to difficult, from concrete to
abstract. Usually this means going from selection to supply-type items. Selection-type items would
usually begin with the most limited selection type (true-false) and progress to multiple choice or
matching in which options can be used more than once. The objective is to determine what the student
knows. If more difficult items appear early in the test, the student may spend too much time on them and
not get to the simpler ones that s/he can answer. For the test we were planning in example 1d of this
module, we would begin with true-false, followed in order by short answer, multiple choice, and the
performance tasks.
* Group items of the same type (true-false, multiple choice, etc.) together so that you only write directions
for that item type once. Once you have a good set of directions for a particular type of item, save them so
you can use them again the next time you use that same type of item.
* Check to see that directions for marking/scoring (point values, etc.) are included with each type of item.
* Provide directions for recording responses, and have students circle or underline correct responses when
possible rather than writing them to avoid problems arising from poor handwriting.
* If a group of items of the same type (multiple choice, etc.) carry over from one page to another, repeat the
directions at the top of the second page.
* All parts of an item should be on the same page.
* If graphs, tables, charts, or illustrations are used, put them near the questions based on them (on the same
page, if at all possible).
* Check to see that items are independent (one item does not supply the answer or a clue to the answer of
another question).
* Make sure the reading level is appropriate for your students. (This may be a problem with tests supplied
by textbook publishers).
* Space the items for easy reading.
* Leave appropriate space for writing answers if completion/short answer, listing, or essay questions are
used. (Younger children need larger spaces than older students because their print/handwriting is larger.)
* When possible, have answers recorded in a column down either the left or right side of the paper to
facilitate scoring.
* Decide if students are to mark answers on the test, use a separate answer sheet, or use a blank sheet of
paper. Usually separate answer sheets are not recommended for students in primary or early elementary
grades.
* Include on the answer sheet (or on the test if students put answers on the test itself) a place for the
student's name and the date.
* Make an answer key. (This is easy to do as you write the questions.)
* Check the answer key for a response pattern. If necessary, rearrange the order of questions within a
question type so the correct answers appear to be in a random order.
* Set the test aside for awhile.
* Re-read the questions; proofread the test one last time before duplication. If possible, have someone else
read the test as well.
* Prepare a copy of the test for each student (plus 2 or 3 extra copies). Questions written on the board may
cause difficulties for students with visual problems. Reading the test questions to the students (except in
the case of spelling tests) can be problematic for students with deficiencies in attention, hearing,
comprehension, or short-term memory.
* Plan accommodations for individual students when appropriate.

B3. Test Administration

A teacher's test administration procedures can have great impact on student test performance. As you will
see in the guidelines below, test administration involves more than simply handling out and collecting the test.

Before the test:

* Avoid instilling anxiety

* Give as many of the necessary oral directions as possible before distributing the tests, but keep them to a
minimum.
 6

* Tell students purpose of the test.

* Give test-taking hints about guessing, skipping and coming back, etc.

* Tell students the amount of time allowed for the test. You may want to put the length of time remaining
for the test on the board. This can be changed periodically to help students monitor their progress. If a
clock is prominently available, an alternative would be to write the time at which they must be finished.

* Tell the students how to signal you if they have a question.

* Tell the students what to do with their papers when they are finished (how papers are to be collected).

* Tell the students what they are to do when they are finished, particularly if they are to go on to another
activity (also write these directions on the chalkboard so they can refer back to them).

* Rotate the method of distributing papers so you don't always start from the left or the front row.

* Make sure the room is well lighted and has a comfortable temperature.

* If a student is absent, write his/her name on a blank copy of the test as a reminder that it needs to be made
up.

After Distributing Test Papers

* Remind students to put their names on their papers (and where to do so).

* If the test has more than one page, have each student check to see that all pages are there.

During the Test

* Minimize interruptions and distractions.

* Avoid giving hints.

* Monitor to check student progress and discourage cheating.

* Give time warnings if students are not pacing their work appropriately.

* Make a note of any questions students ask during the test so that items can be revised for future use.

After the Test

* Grade the papers (and add comments if you can); do test analysis (see the module on test analysis) after
scoring and before returning papers to students if at all possible. If it is impossible to do your test analysis
before returning the papers, be sure to do it at another time. It is important to both evaluation of your
students and improvement of your tests.

* If you are recording grades, record them in pencil in your gradebook before returning papers. If there are
errors/adjustments in grading, they (grades) are easier to change when recorded in pencil.

* Return papers in a timely manner.

* Discuss test items with the students. If students have questions, agree to look over their papers again, as
well as the papers of others who have the same question. It is usually better not to agree to make changes
in grades on the spur of the moment while discussing the tests with the students but to give yourself time
 7

to consider what action you want to take. The test analysis may have already alerted you to a problem
with a particular question that is common to several students, and you may already have made a decision
regarding that question (to disregard the question and reduce the highest possible score accordingly, to
give all students credit for that question, etc.).

Check Your Knowledge

DIRECTIONS: FOR EACH QUESTION BELOW, WRITE THE LETTER CORRESPONDING TO
YOUR ANSWER ON A SHEET OF PAPER. CLICK ON THE CORRESPONDING BUTTON FOR
THE ANSWERS.

1. When is it appropriate to construct a table of specifications?

A Before you write the test questions
B After you write the test questions
C After the students have taken the test
D ONLY when you have to create the test

2. Are tests from book publishers better than those you develop?
Top of Form
Yes, because they wrote the textbook
A
Yes, because they have professionals who write the questions
B
No, because they don't know what you have taught and what has been emphasized
C

3. When should tests from the publisher be used?

Top of Form
When you don't have time to construct one
A
Any time
B
None of the time
C
When they correspond to your table of specifications
D

4. What is the first information entered into the table of specifications?

Top of Form
Number of questions per cell
A
Objectives or topics
B
Percentage of test per topic
C
Total number of questions on the test
D

5. What is the primary consideration when selecting types of questions to use?

Top of Form
How many questions can the students complete in a given amount of time?
A
Are the students average, above or below average?
B
How can the accomplishment of objectives best be measured?
C
How old are the students?
D
 8

6. What test information should be supplied to students when they take a test?

Top of Form
How much time they have for the test
A
How much each question is worth (points)
B
What happens to students who fail the test
C
Both A and B above
D

Which of the following is/are recommended as they are stated when constructing a test? Record an answer for
each item as either Recommended (R) or Not Recommended (NR).
Top of Form
7 Put all true-false questions together
R NR

Top of Form
8 Use the test from last year
R NR

Top of Form
9 Make an answer key before the test is administered
R NR

Top of Form
10 Estimate how long it will take students to complete the test
R NR
Practice Activity
Print this page out from your browser to work with.
1. Outline the steps you follow when you plan tests for your students, and compare them to the steps
presented in this module.

2. Select a test that is provided by the publisher of your textbook in the Teacher's Edition of the
textbook or in supplementary materials. On a sheet of paper list the number of each question on a
separate line down the left side of the paper. Then determine which topic or objective each question
addresses and the level of thinking (knowledge, comprehension, application).

Question # Topic or Objective # Level of Thinking

Analyze the test in terms of the number of questions addressing each topic or objective, and the
level(s) of thinking. To what extent does the coverage of the test (in terms of emphasis on the
various topics and the levels of thinking) correspond to your instruction?
Practice Activity – Feedback
1: Compare your planning process with that involved in developing the table of specifications. Which of the
following does your process include: (Check all that apply.)
 Determining test content by looking at objectives, topics, and class activities
 9

 Determining the amount of emphasis to place on each topic based on the time and emphasis placed on
each topic during instruction
 Deciding what levels of thinking should be addressed in the assessment for each topic based on class
activities and objectives
 Deciding what types of questions best measure the levels of thinking you want to measure
 Determining alignment with standardized tests (objectives, item types, emphases)
 Estimating the amount of time needed for the test

If you have included most or all of these steps, your test planning process is well thought out. The results
from the students should provide a valid measure of how and what you are teaching and how much the
students are learning.

2. After analyzing the publisher's test you may decide that it is a good match with your teaching, that the
test (or part of it) would be useful if you supplemented it, or that you really need to create your own
assessment. Any of these three options is acceptable so long as the test is an accurate measure of what
has been taught.

Classroom Application
1. Self-Assessment/Inventory: When you plan, prepare, and administer a classroom test, which of the
following do you routinely do? (On a separate sheet of paper, write the letters of all procedures you use.)
a) Estimate the amount of time students will need to complete the test, based on its composition and
make sure sufficient time is available

b) Provide a copy of the test for each student

c) Make sure copies of the test are clear and easy to read

d) Prepare an answer key before the test is administered

e) Check for a discernible pattern to the answers

f) Check for overlap of question content within item types or across the whole test

g) Vary the students to whom you start distributing tests

h) Try to avoid or minimize distractions and interruptions during the test

i) If a pattern of INCORRECT answers to one or more questions develops as you grade a test,
check your answer key to make sure the answer key is correct

j) Analyze the tests and items after the tests have been scored

k) Return the graded tests in a timely manner

l) Discuss the tests with your students after they have been graded

2. Critique a test from the publisher of one of the textbooks you use or a test you have created.
As you critique the test, answer the following questions.

Does the test provide a reasonably acceptable representation of the level(s) of thinking and learning you
used in instruction and learning activities with your students?

Is the test a reasonably accurate representation of the content you have taught and the varying degrees of
emphasis on the respective topics?

Is the assessment limited to important learning targets?

Is the assessment format familiar to the students?

 10

Is the vocabulary appropriate for your students?

Are directions included and do they contain the information students should have?

Is information on scoring (points for scoring, etc.) included?

Are questions grouped by type?

Do questions progress from easiest to most difficult and from those requiring the least time to those
requiring more time to answer?

Is each question and all of its responses presented on the same page (not split onto two pages)?

Is sufficient space provided for short answer, completion, or essay responses?

Is there a proper place for the student's name and the date?

Have appropriate accommodations been made (or suggested) for students when necessary?

Is the test, or some items/parts of the test, aligned with what students will be asked to know and do on
standardized tests?

Share the results with your mentor teacher or a colleague. Ask your mentor teacher or a colleague what they
do that differs from the procedures you use. Also, they probably can also share with you methods they have
tried that did NOT work before settling on those they currently use.