HMEF5053 Measurement and Evaluation in Education

Ong Yee Sing, August 2018
HMEF5053 Measurement and Evaluation in Education

Sample questions and answers
Tidied and answered by Ong Yee Sing, August 2018. Correctness is not
guarantee.
Topic 1
1. What do you understand by the following terms?
a. Measurement (Pengukuran)
b. Evaluation/Assessment (Penilaian/Pentaksiran)
c. Test (Ujian
Measurement refers to the process by which the attributes or dimensions of

something are determined (assigned numbers) such as size, and quantity using
standard instruments such as rulers, scales, thermometers, examination questions
etc. are used to obtain information about what we are measuring. The process
of measuring must be valid and accountable, using suitable instruments and
skills of using these instruments. We are not assessing. E.g. we measure the
temperature of the room.
Assessment is a process by which information is obtained relative to some

known objective or goal as the information are interpreted to make decisions.
E.g. we assess whether my student required special attention based on his result
on dyslexic test.
While evaluation the information obtained from measurement are put against
certain criteria to determine whether the criteria have been achieved, so we
can assign grade to the thing that we are interested. E.g. I evaluate the
successfulness of my teaching based on the students ability to solve a problem
on the Boyle’s Law.
A test is a set of tasks or questions intended to elicit particular types of behavior

when presented under standardized conditions to yield a score that has
desirable psychometric properties. E.g. a class test is held to examine students’
knowledge on meiosis based on their ability to answer a set of 20 questions.
1
2. What are the four principles of assessment? [Aug 2018]
(i) What is to be assessed has to be clearly specified.

The specification of the characteristics to be measured should
precede the selection or development of assessment procedures. In
other words, in assessing student learning, the intended learning or
goals or outcomes should be clearly specified
(ii) An assessment procedure should be selected based on its relevance
to the characteristics or performance to be measured.
There must be a close match between the intended learning
outcomes and the types of assessment tasks to be used.
(iii) Different assessment procedures are required to provide a complete
picture of student achievement and development.
This is because no single assessment procedure can assess all the
different learning outcomes in a school curriculum.
(iv) The assessment must be aligned to instruction. What to be assessed in
the classroom must be consistent with what has been taught and vice
versa
3.
Educators and schools assess students to help learning, and to help teaching.
With regards to learning, assessment provide information to help making

decision in the progress of learning.
 Diagnostic assessment can be performed at the beginning of a lesson to
diagnose weak students for remedial action and strong students for
enrichment programme.
 These assessment can also discover exceptional children, who are
physically, mentally, emotionally or behaviourally different from the
normal population, such as those who are dyslexic, or with other learning
disability, so that they can be assigned to a special treatment or
education program.
 Assessment can also be used to providing students with certificates by
comparison of performance among students such as Malaysian
2
Certificate of Education (SPM), so that the students can be selected for

further studies, scholarships or jobs.
 Assessment can be used to determine the placement of students so that
students can be streamed according to their abilities and performance,
e.g. for class placement or arts or science stream.
 Assessment enable parents to communicate and understand the progress
of their children’s learning as a method to show the growth of their
children.
 Assessment also allows the school administration and counselling to
determine how the school is performing in relation to other school to
prepare intervention strategies if the performance of the school has
dropped.
With regard to teaching, assessment provides information regarding the class

preparation.
 Assessment determine the achievement of objectives to see if the desired

learning outcomes are reached.
 Assessment determine whether the current teaching methods is effective
against this batch of students.
 Assessment determine whether the students possess relevant prior
knowledge.
 Assessment determine whether the current teaching materials used are
effective.
 Assessment determine the ability of teacher to teach by comparing the
performance of their students.
4.
Criteria Formative assessment Summative assessment

Timing Beginning of school year Conducted at the end of a
and conducted throughout teaching-learning phrase (e.g. end
the teaching-learning of semester or year)
process
Aim To improve the quality of Grading to determine whether the
student learning programmes was successful; to
certify students who mastered
3
specific competencies with letter

grade, and improve the curriculum
Method Paper and pencil tests, Paper and pencil tests, oral test
observations, quizzes, administered to the group
exercises, practical sessions
administered to the group
and individually
Example Quizzes, essays, diagnostic Final exams, national
tests, lab reports, anecdotal examinations, qualifying tests
records
5.
[Aug 2018]
Criteria Norm-referenced test Criterion-referenced test

Aim  Compare a student’s  Compare a student’s
performance with that of performance against
other students some criteria
 Select students for  Determine the extent to
certification which a student has
acquired the knowledge
or skill e.g. arithmetic
operations, reading
difficulty
 Improve teaching and
learning
Types of Questions from simple to Questions of nearly similar
questions difficult difficulty relating to the
criteria
Reporting of Grades are assigned No grades are assigned
results
Content Wide content coverage Specific aspects of the
coverage content
Examples UPSR and other national Class tests, driver’s license test
examinations, final
examinations
Topic 2
4
1. What are the three learning outcomes?
The three learning outcomes are the cognitive, affective and psychomotor
learning outcomes, that can be describe as KSA (knowledge, skills and attitude)
domains.
The cognitive learning outcomes – knowledge or mental skills
Affective learning outcome – growth in feelings or emotional areas
Psychomotor learning outcome – manual or physical skills
2. List the taxonomy of all the 3 learning outcomes.
Each domain consists of subdivisions, starting from the simplest behaviours to the
most complex, thus forming a taxonomy of learning outcomes. Each taxonomy
of learning behaviour can be thought as “the goals of the schooling process”.
Cognitive learning outcomes
When assessing cognitive outcomes, we assess the learners’ ability in using the
facts, concepts or principles of the subject area. Bloom’s Taxonomy is a
classification of the different objectives and skills that educators set for their
students (learning objectives). It is developed by a group of psychologists
leaded by Benjamin Bloom in 1956. The six levels in Bloom’s classification are
ranked from the lowest level to the highest level with increasing difficulties in
mental abilities: knowledge, comprehension, application, analysis, synthesis and
evaluation.
Like other taxonomies, Bloom’s is hierarchical, meaning that learning at the

higher levels is dependent on having attained prerequisite knowledge and skills
at lower levels. Hence, depending on the student’s ability (primary school or
5
university students), some levels of Bloom’s taxonomy can be skipped when

preparing the lesson objectives. These lesson objectives, written using this
taxonomy, can be used at the end of the lessons to assess students’
achievement in the learning outcomes.
Level Description Example

Knowledge Recall specific Quote formula for
knowledge density
Comprehension Show understanding of Explain in one’s on words
the meaning of for the steps in making a
concepts and principles solution
by translation of
concepts into one’s own
worlds, interpolation of
missing information, and
interpretation of given
information
Application Apply a rule or principle Use the formula for
into novel or new projectile motion to
situation or unprompted calculate the maximum
use of an abstraction distance a long jumper
can jump
Analysis Identify component Recognise logical
parts and describe their fallacies in reasoning,
relationship, and determining needs for
separating material or training based in
concepts into information gathered
component parts
Synthesis Put parts together or Wrote a creative short
creating new meaning stories
or structures
Evaluation Make judgement about Evaluate and decide on
materials, methods, the most effective
values of ideas etc. solution to a problem
and justify the choice
The differences between original Bloom’s and revised taxonomy includes:
 The revised taxonomy used verbs instead of nouns to indicate that

learning an active process.
 The last two categorises are rearranged in the revised taxonomy.
Original Knowledge Comprehension Application Analysis Synthesis Evaluation
6
Revised Remember Understand Apply Analyse Evaluate Create
 A cognitive and knowledge matrix is added in the revised taxonomy.

Factual, conceptual and procedural knowledge was mentioned in
Bloom’s taxonomy but was not discussed in detailed. The revised
taxonomy added metacognitive as well.
o Factual - The basic elements students must know to be acquainted
with a discipline or solve problems.
o Conceptual – The interrelationships among the basic elements
within a larger structure that enable them to function together.
o Procedural - How to do something, methods of inquiry, and criteria
for using skills, algorithms, techniques, and methods.
o Metacognitive – Knowledge of cognition in general, as well as
awareness and knowledge of one’s own cognition.
The Cognitive Dimension
The Knowledge
Remember Understand Apply Analyze Evaluate Create
Dimension
Factual
Conceptual
Procedural
Metacognitive
Affective learning outcome [Aug 2018]
Like cognitive objectives, affective objectives can also be divided into a

hierarchy (according to Krathwohl 1964). This area is concerned with feelings or
emotions. Again, the taxonomy is arranged from simpler feelings to those that
are more complex.

Receiving This refers to the Listen to others Aware that
learner’s sensitivity to with respect certain things
the existence of stimuli are
– awareness, willingness honest/dishonest
7
to receive, or selected
attention.
Responding This refers to the Gives a Saying honesty is
learners’ active presentation better and
attention to stimuli and behaving
his/her motivation to accordingly
learn – acquiescence,
willing responses, or
feelings of satisfaction.
Valuing This refers to the worth Propose a Consistently (but
or value a person plan to social not always)
attaches to a particular improvement telling the truth
object, phenomenon or
behaviour.
Organisation This refers to the Accepts Being honest in
learner’s internalization responsibility a variety of
of values and beliefs for his or her situation
involving (1) the behaviour
conceptualization of
values; and (2) the
organization of a value
system according to
priority.
Characterisation This refers to the Show self- Honest in most
learner’s highest of reliance when situation,
internalization and working expects others
relates to behavior that independently to be honest
reflects (1) a and interacts
generalized set of with others fully
values; and (2) a honesty
characterization or a
philosophy about life.
At this level the learner
is capable of practicing
and acting on their
values or beliefs.
Psychomotor learning outcome
The psychomotor domain includes physical movement, coordination and use of

motor-skill areas.
8
Perception Use sensory cues to guide Adjusts the height of the ladder
motor activity in relation to the point on the
wall
Set The mindsets (mental, Recognises their abilities and
physical and emotional limitation
sets) that predetermine a
person’s response to
different situation
Guided Practising (imitation, trail Performs a mathematical
response and error) when equation as demonstrated
beginning to learn a
complex skill
Mechanism Intermediate stage in Rides a motorcycle
learning a complex skill
when the learned
response have become
habitual and can be
performed with some
confidence
Complex overt Motors act that involves Displays competence while
response complex movement that playing the piano
are automatic (quicker
and better than
mechanical)
Adaptation Modify movement Respond effectively to
patterns to fit special unexpected experience
requirements
Origination Creating new movement Creates a new dance routine
or pattern to fit a
particular situation or
specific pattern
3. Of the three types of learning outcomes, the assessment of affective LOs is

the least emphasised in school. Why? [Aug 2018]
 The belief that the development of appropriate feelings is the task of

family and religion
 The belief that appropriate feelings develop automatically from
knowledge and experience
 Difficulties in develop attitudinal and value-oriented instructions as
affective goals are intangible and cannot be attained in the typical
periods of instruction offered in schools
9
 Affective characteristics are considered to be private matters

 No sound methods to gather information about affective characteristics
4.
Categories Explanation Example

Audience The group of learners for whom the Form two students
objective is written for
Behaviour Describe what the learner will be able Identify
to do after the instruction – must be
measurable and observable
Condition Defines the circumstances under which With the use of a
the objective must be completed, i.e. map
the available of materials
Degree The standards (time limit, percentage Within 5 minutes
etc.) that the learner must meet to
reach an acceptable level of
performance
Given a list of 20 Asian countries (condition), the students (audience) will identify
(behaviour) at least 15 of the corresponding capital cities (degree).
Topic 3
1. What are the advantages and disadvantages of matching questions?
Advantages Disadvantages
Understanding of relationships Limited to materials that can be listed
in two columns
Ability to apply knowledge Giveaway in the last item through
elimination
10
Grouping MCQ questions Difficult to differentiate between

effective and ineffective items
Easy to write and score Testing of trivial facts
Highly efficient to sample a large Encouraging rote memorisation
amount of knowledge in a short time
2. List important considerations when constructing matching questions.
 Clear instruction
 Homogenous information in a column
 Include more responses than premises or allow the response to be used
more than once
 Put the items with more words in column A
 Correct answers should not be obvious
 No clues or keywords to the correct answer
 All responses and premises should appear on the same page
3. What is True/False questions? Describe and provide examples.
11
A statement is presented and students must indicate whether the statement is

true or false. There is only two possible alternatives. It can be used to test
knowledge and judgement in a short time.
Example: a whale is a mammal because it gives birth to its young.
4. Limitations of Ture/False questions.
 Guessing
 Tendency to take quotations from text – encourage tote memorising and
include trivial material
 Difficult to write statement for complex material
 Does not discriminate among students of different abilities as well as other
question types
5. What is objective question?
An objective test is a written test consisting of items or questions which require

the respondent to select from a list of possible answers. Hence, an objective
item or question provides the respondent with a list of possible answers. It cannot
be influenced by the personal preferences and prejudices of the marker and
not open to varying interpretations.
Types of objective questions includes multiple choice questions, matching

questions and true or false questions. All these questions provide items for
students choose the correct answer from. For exaxmple, in true or false question
such as “Putrajaya is the capital city of Malaysia.” has two possible solution: true
or false. The students cannot provide other answers. The answer is “objective”
and not open to interpretation, unless essay questions or other “subjective” tests,
where students are free in the response and the markers have to interpret the
response against a selected marking scheme.
6. What are the strengths and advantages of MCQ? [Aug 2018]
12
(a) Learning outcomes from simple to complex can be measured;

(b) Highly structured and clear tasks are provided;
(c) A broad sample of achievement can be measured;
(d) Incorrect alternatives or options provide diagnostic information;
(e) Scores are less influenced by guessing than true-false items;
(f) Scores are more reliable than subjectively scored items (e.g. essays);
(g) Scoring is easy, objective and reliable;
(h) Item analysis can reveal how difficult each item was and how well it
discriminated between the strong and weaker students in the class;
(i) Performance can be compared from class to class and year to year;
(j) Can cover a lot of material very efficiently (about one item per minute of
testing time); and
(k) Items can be written so that students must discriminate among options
that vary in degree of correctness.
7. What are the weaknesses or disadvantages of MCQ? [Aug 2018]
(a) Constructing good items is time-consuming;

(b) It is frequently difficult to find plausible distractors;
(c) MCQs are not as effective for measuring some types of problem-solving
ability and ability to organise and express ideas;
(d) Scores can be influenced by reading ability;
(e) There is a lack of feedback on individual thought processes as it is difficult
to determine why individual students selected incorrect responses;
(f) Students can sometimes read more into the question than was intended;
(g) It often focuses on testing factual information and fails to test higher levels
of cognitive thinking;
(h) Sometimes, there is more than one defensible “correct” answer;
(i) They place a high degree of independence on the student’s reading
ability and the constructor’s writing ability;
(j) Does not provide a measure of writing ability; and
(k) May encourage guessing.
8. What are the principles of writing MCQs? Give examples. [Aug 2018]
Textbook p59
See also:
https://www.researchgate.net/profile/Sitanshu_Kar/publication/304349657_Basic
13
_Principles_of_constructing_multiple_choice_questions/links/576c985b08ae193ef
3a9a683/Basic-Principles-of-constructing-multiple-choice-questions.pdf
9.
A Table of Specifications is a two-way chart which describes the topics to be

covered by a test and the number of items or points which will be associated
with each topic. It is a matrix with listing of the content areas or topics as rows,
and levels of thinking or process skills (such as recall, comprehend, apply,
analyse, synthesise or evaluate) as columns.
Tables of Specification typically are designed based on the list of course

objectives, the topics covered in class, the amount of time spent on those
topics, textbook chapter topics, and the level of understanding you want the
students to have about each topic.
The purpose of a Table of Specifications is to identify the learning objectives

being measured and to ensure that a fair and representative sample of
questions appear on the test. A Table of Specifications helps to ensure that
there is a match between what is taught and what is tested.
Topic 4
1. Discuss 2 methods of scoring essay test with appropriate examples.
14
Analytical marking Holistic marking

most frequently used in large-scale public
examinations (such as the SPM and STPM)
and also by teachers in the classroom
marking scheme with proper mark allocations reading an entire response and assigning it to
for elements in the answer a category identified by a score or grade by
judging the total quality of the answer relative
to other student responses or the total quality
of the answer based on certain criteria that
you develop
Points are given to items corresponded to the Points are given based on an overall
marking scheme impression
Easier to use as it divides the task into smaller Hard to sustain the level of concentration
components simplifies the task. required to assess such work holistically
Students may dump irrelevant information Looking for a logical reasoning within the
into the answers together with the correct answer
answer
15
16
2. What is essay questions.
17
Based on Stalnaker's definition, an essay question should meet the following

criteria:
 Requires examinees to compose rather than select their response.

o Objective test questions (MCQ, true-false, matching), on the other
hand, require students to select the answer from a list of possibilities.
 Elicits student responses that must consist of more than one sentence.
o In theory, there is no limit to the length of the answer. However, in
most cases, its length is predetermined by the demand of the
question and the time limit allotted for the test question.
 Allows different or original responses or pattern of responses.
o Not a factual questions, but to give reasoning etc.
 Requires subjective judgment by a competent specialist to judge the
accuracy and quality of responses.
o The nature of essay questions is such that only specialists in the
subject can judge to what degree responses (or answers) to an
essay question are complete, accurate and relevant.
3. List strengths and limitations of essay question.
Advantages
1. Assess higher-order or critical thinking skills and complex learning

outcomes. They allow one to assess students’ ability to synthesise,
organise, and express ideas, and evaluate the worth of ideas.
2. Evaluate student thinking and reasoning by demonstrating their reasoning
and thinking skills in their answers.
3. Provide authentic experience as constructed responses are closer to real
life than selected responses. Problem solving and decision-making are
vital life competencies. In most cases these skills require the ability to
construct a solution or decision rather than select a solution or decision
from a limited set of possibilities.
18
Disadvantages
1. Assess a limited sample of the range of content due to the time it takes for
students to respond to essay questions and for graders to score responses.
2. Are difficult and time consuming to grade. Different readers may vary in
their grading of the same or similar responses (interscorer reliability) and
one reader can vary significantly in his grading consistency depending on
many factors (intrascorer reliability). Gender and ethnic bias, the length
and legibility of the response, and the personal preferences of the grader
with regards to the content and structure of the response are some of the
factors that can lead to unreliable grading.
3. Provide practice in poor or unpolished writing. The way in which students
construct their responses to essay questions differs in several ways from
real-world writing tasks.
Misconceptions
(a) By Their Very Nature, Essay Questions Assess Higher-order Thinking
Whether or not an essay item assesses higher-order thinking depends on the

design of the question and how students’ responses are scored.
(b) Essay Questions are Easy to Construct
Essay questions are easier to construct than multiple-choice items because

teachers do not have to create effective distractors, but they still require a lot of
effort and time.
(c) The Use of Essay Questions Eliminates the Problem of Guessing
The use of essay questions introduces bluffing (vague generalities, padding,

name-dropping, etc.), another form of guessing.
(d) Essay Questions Benefit All Students by Placing Emphasis on the Importance
of Written Communication Skills
The focus on written communication skills is also a serious disadvantage for

students who have marginal writing skills but know the subject matter being
assessed.
(e) Essay Questions Encourage Students to Prepare More Thoroughly
19
Students’ extent of preparation is based more on the expectations teachers set

upon them (higher-order thinking and breadth and depth of content) than the
type of test questions they expect to be given in examinations.
Topic 5
1. Differentiate between traditional assessment (TA) and authentic assessment
(AA). [Aug 2018]
Traditional Assessment Authentic Assessment
 Purpose: to evaluate if the students  Purpose: to measure students’

have learned the content; to proficiency by asking them to
determine whether or not the students perform real life-tasks; to provide
are successful in acquiring knowledge; students many avenues to learn and
to ascribe a grade for them; to rank demonstrate best what they have
and compare them against standards learned; to guide instruction; to
or other learners provide feedback and help students
manage their own learning; to also
evaluate students’ competency
 Provides teachers a snapshot of what  Provides teachers a more complete

the students know picture of what the students know
and what they can do with what
they know
 Measures students’ knowledge of the  Measures students’ ability to apply

content knowledge of the content in real life
situations; ability to use/apply what
they have learned in meaningful
ways
 Requires students to demonstrate  Requires students to demonstrate

knowledge by selecting a proficiency by performing relevant
response/giving correct answers; tasks showing application of what
usually tests students’ proficiency has been learned
through paper and pencil tests
 Students are asked to choose an
answer from a set of questions (True or
False; multiple choice) to test
knowledge of what has been taught.
20
 Provides indirect evidence of learning  Provides direct evidence of

learning/competency; direct
demonstration of knowledge and
skills by performing relevant tasks
 Requires students to practice  Provides opportunities for students to

cognitive ability to construct meaning/new knowledge
recall/recognize/reconstruct body of out of what has been taught
knowledge that has been taught
 Tests and strengthens the students’  Tests and strengthens the students’
ability to recall/recognize and ability to reason and analyze,
comprehend content, but does not synthesize, and apply knowledge
reveal the students’ true progress of acquired; Students’ higher level of
what they can do with the knowledge cognitive skills (from knowledge and
they acquired. Only the students’ comprehension to analysis, synthesis,
lower level of thinking skills, application, and evaluation) are
(knowledge and comprehension), are tapped in multiple ways.
tapped.
 Hides the test  Teaches the test
 Teachers serve as evaluators and  Involves and engages the students

students as the evaluatees: teacher- in the teaching, learning and
structured assessment process: student
structured
 Assessment is separated from  Assessment is integrated with

teaching and learning. Test usually instruction. Assessment activities
comes after instruction to evaluate if happen all throughout instruction to
the students have successfully learned help students improve their learning
the content. and help teachers improve their
teaching.
 Provides limited ways for students to  Provides multiple avenues for

demonstrate what they have learned students to demonstrate best what
they have learned
 Rigid and fixed  Flexible and provides multiple

acceptable ways of constructing
21
products or performance as
evidence of learning
 Standardized; valid and reliable  Needs well defined criteria/rubrics

and standards to achieve reliability
and validity
 Curriculum drives assessment.  Assessment drives curriculum and

instruction.
Examples: Examples:
 True or False; multiple choice tests  demonstrations
 standardized tests  hands-on experiments
 achievement tests  computer simulations
 intelligence tests  portfolios
 aptitude tests  projects
 multi-media presentations
 role plays
 recitals
 stage plays
 exhibits
2. What is authentic assessment? Discuss and support your answer with relevant
examples.
Authentic assessment is a form of assessment in which students are asked to

perform real-world tasks that demonstrate meaningful application of essential
knowledge and skills.
To create an authentic assessment:
(a) Identify which standards you want your students to meet through this
assessment;
(b) Choose a relevant task for this standard, or set of standards, so that students
can demonstrate how they have or have not met the standards;
(c) Define the characteristics of good performance on this task. This will provide
useful information regarding how well students have met the standards; and
(d) Create a rubric, or set of guidelines for students to follow so that they are
able to assess their work as they perform the assigned task.
22
Example of authentic assessment strategies includes exhibition of an athletic skill;

and production of a short musical, dance or drama.
Topic 6
1. Discuss methods for assessing group project work.
Students should be made aware of assessment before starting the project
 assessment method
 criteria (product and/or process)
Product versus process
 assessing the product - measuring the quantity and quality of individual

work in a group project
 assessing the process - evaluating individual teamwork skills and
interaction
Assessment by instructor versus by group members
 by instructor - instructor assigns all marks

 by group members - group members evaluate their contributions to the
group and assign marks
Product assessment by instructor

 Equally shared mark: All group members receive same grade
 Easiest to implement - does  Individual contributions are

not require any additional not reflected in the distribution
work aside from marking the of marks
projects  Poor students may benefit
 Appropriate if group work from the work of hard-working
mark is a minor part of total students
mark for the course  Good students may be
 Group responsibilities are dragged down by poor
enforced - group succeeds or students
fails together  Does not motivate students
23
 Exam questions: Questions should be specifically about the project, and

are answerable only by students who have been thoroughly involved in
the project
 May increase interest in the  Students may ignore group in

project - students may be order to study for the exam on
more motivated to learn their own
about the work of their fellow  May mean additional work for
group members instructor when preparing the
exam questions
 May not be effective -
students may be able to
answer the questions by simply
proofreading
the project
 Splitting tasks: Project must be divisible into multiple tasks of the same
complexity and each student is responsible for one task. Final mark is part
group mark (e.g., 50%) and part individual task mark (e.g., 50%)
 objective way of  May increase interest in the project -

determining individual students may be more motivated to
participation learn about the work of their fellow
 individual component group members
grade may provide  Students may ignore group in order
additional motivation to study for the exam on their own
 May mean additional work for
instructor when preparing the exam
questions
 May not be effective - students may
be able to answer the questions by
simply proofreading the project
24
 Direct evaluation: Instructor judges individual merits by oral interviews,

periodic reports, meeting minutes and observation.
 oral interviews are a good way  very time consuming

of getting information on  information obtained is
individual participation often subjective and/or
 enables instructor to give each may be inaccurate
student more specific feedback  class size may make it
infeasible
Product assessment by peer evaluation

Issues with peer evaluation:
 Should we use self-assessment?

 Should instructor adjust marks?
 Should it be done individually or collectively by consensus?
 Distribution of a pool of marks: Award the group a mark equal to (group

mark) X (no. of group members) and let group divide marks among
themselves.
 easy to implement  open to subjective evaluations by

 peer assessment students (e.g., giving friends high marks as
may motivate opposed to those who contributed the
students to most)
contribute more to  opens the doors to personal conflicts
the group between group members
 may foster competition
 may be difficult for students to evaluate
each other without objective criteria
25
 Individual weighting factor as points awarded for a list of tasks and

individual mark = (group mark) X (peer assessment factor)
 provides students with objective criteria  time consuming for

by which to judge individual instructor
contributions  rating scale may be
misinterpreted
 tasks all have the
same weight
Process assessment
List of skills to assess, such as:
 adoption of complementary team roles

 cooperative behaviour
 time and task management
 creative problem solving
 use of a range of working methods
 negotiation
Process assessment by instructor
Direct evaluation of team behaviour using teamwork logs - sample questions:
 what steps have you taken to organize your teamwork?

 what steps have you taken to monitor the effectiveness of your team?
 what steps have you taken to improve the effectiveness of your team?
 what problems have you encountered in working as a team and how did
you tackle them?
 if you were to embark on a second, similar task as a team, what would be
different about the way you go about working, and why?
 makes students reflect on their  reviewing logs can be very

operation as a team time-consuming
 logs provide plenty of  students may need training in
information that can be used order to be aware of what
as a basis for assessment goes on in the teams
26
Process assessment by peer evaluation
 Individual assessment (see Appendix for example)

 how members view each member of the team
 use lists of key group work traits
 average of individual marks must be the same as the group mark
 gives a personalized view of  time consuming and complex;

each member's contributions instructor must check results
 list of traits provides students  list of traits may not give a true
with objective criteria measure of the group work
process
 students may misinterpret traits
2. Discuss steps for designing project work. [Aug 2018]
The teacher should develop an outline that explains the project’s essential
elements and his or her expectations for each project.
1. Identify Learning Goals and Objectives.

 What specific skills or concepts will students learn? [cognitive skills,
social and affective skills, metacognitive skills, types of problems,
concepts and principles to be applied]
 Steinberg (1998) the Six A’s Project Checklist: authenticity,
academic rigour, applied learning, active exploration, adult
relationships, assessment practices
2. Situation or Problem
 A sentence or two describing the issue or problem that the project is
trying to address.
 For example, the pollution levels in rivers
3. Project Description and Purpose
 A concise explanation of the projec’s ultimate purpose and how it
addresses the situation or problem.
 For example, students will research, conduct surveys and make
recommendations on how students can help reduce pollution of
rivers. Results will be presented in a newsletter, information
brochure, exhibition or website.
4. Performance Specifications
 A list of criteria or quality standards the project must meet.
5. Rules
27
 Guidelines for carrying out the project include timeline and short-
term goals, such as to have interviews and research completed by
a certain date.
6. List of Project Participants with Roles Assigned
 Roles of team members and if members of the community are
involved, identify their roles.
7. Assessment How the student’s performance will be evaluated.
 In project work, the learning process is being evaluated as well as
the final product.
3. Can you give an example of project work commonly carried out by your
pupils in school ?
Most projects have certain common defining features (Katz & Chard, 1989):
 Student centred;
 A definite beginning, middle and end;
 Content is meaningful to students; directly observable in their environment;
 Real-world problems;
 First-hand investigation;
 Sensitivity to local culture and culturally appropriate;
 Specific goals related to curriculum;
 A tangible product that can be shared with the intended audience;
 Connections among school, life, and work skills;
 Opportunity for reflective thinking and student self-assessment; and
 Multiple types and authentic assessments (portfolios, journals, rubrics, etc.).
Examples includes:
 Develop a newsletter or website on a specific issue relevant to the school

or community (school safety, recycling, how businesses can save energy
and reduce waste, etc.);
 Compile oral histories of the local area by interviewing community elders;
 Produce a website as a “virtual tour” of the history of the community
 Create a CD of students graduating from primary or secondary school;
 Create a wildlife or botanical guide for a local wildlife area;
 Create an exhibition on local products, local history and local
personalities using audiotapes, videotapes and photographs; and
 Investigate pollution of local rivers, lakes and ponds.
28
[Aug 2018]
Project work is a learning experience which enables the development of certain

knowledge, skills and attitudes which prepares students for lifelong learning and
the challenges ahead. These objectives may not be achieved by other
instructional strategies. Thus, projects are aimed at:
(a) Developing the skill of planning, structuring and taking responsibility for a
larger piece of work and providing experience of working in project form;
(b) Deepening knowledge within a subject or between subjects;
(c) Providing students with opportunities to explore the inter-relationships and

inter-connectedness of topics within a subject and between subjects;
(d) Encouraging students to synthesise knowledge from various areas of

learning, and critically and creatively applying it to real-life situations; and
(e) Encourage collaboration skills and communication skills when working in a

team.
29
Does the project emanate from a problem that has

meaning to the student?
Authenticity
Is it a problem or question that might actually be tackled
by an adult at work or in the community?
30
Do students create or produce something that has

personal and/or social value, beyond the school setting?
Does the project lead students to acquire and apply
knowledge central to one or more discipline or content
areas?
Does it challenge students to use methods of inquiry
Academic Rigor central to one or more disciplines? (e.g., to think like a
scientist)
Do students develop higher order thinking skills and habits
of mind? (e.g., searching for evidence, taking different
perspectives)?
Does the learning take place in the context of a semi-
structured problem, grounded in life and work in the world
beyond school?
Does the project lead students to acquire and use
Applied
competencies expected in high performance work
Learning
organizations (e.g., teamwork, appropriate use of
technology, problem solving and communication)?
Does the work require students to develop organizational
and self-management skills?
Do students spend significant amounts of time doing field-
based work?
Does the project require students to engage in real
Active
investigations, using a variety of methods, media, and
Exploration
sources?
Are students expected to communicate what they are
learning through presentation and/or performance?
Do students meet and observe adults with relevant
expertise and experience?
Adult Do students have an opportunity to work closely with at
Relationships least one adult?
Do adults collaborate on the design and assessment of
student work?
Do students reflect regularly on their learning using clear
project criteria that they have helped to set?
Do adults from outside the classroom help students
develop a sense of real world standards for this type of
Assessment
work?
Will there be opportunities for regular assessment of
student work through a range of methods, including
exhibitions and portfolios.
31
Topic 7
1. Which of the portfolio development process is the most important? [Aug 2018]
A portfolio can be defined as a container that holds evidence of an individual’s

skills, ideas, interests and accomplishments. The organised collection of contents
such as text, files, photos, videos and more to tell that story, are generically
referred to as artefacts, and are the evidence of what students have learned.
These artefacts are usually accompanied by students’ reflection. Portfolios
typically are created for one of three purposes; to show growth, to showcase
current abilities, and to evaluate cumulative achievement.
The portfolio development process consists of four basic steps: collection,

selection, reflection, and connection.
1. Collection: This step simply requires students to collect and store all of their
work.
2. Selection: This will depend on whether it is a process or product portfolio
and the criteria set by the teacher. Students will go through the works
collected and select certain works for their portfolio.
3. Reflection: It is reflection that differentiates the portfolio from a mere
collection of student work. Reflection is often done in writing but it can
also be done orally. Students are asked why they have chosen a
particular product or work (e.g. essay); and how it compares with other
works, what particular skills and knowledge were used to produce it (e.g.
the essay) and how it can be further improved.
4. Connection: Students are encouraged to make connections between
their schoolwork and the value of what they are learning and the real
world.
I think the selection process is the most important, because the students
determine set the foundation on how the portfolio will looked in this process
when they decide on the basis of selection.
2.
32
(i) Promoting student self-evaluation, reflection and critical thinking;

(ii) Holding students accountable for mastering content standards in a subject
area;
(iii) Measuring performance based on genuine samples of student work;
(iv) Providing flexibility in measuring how students accomplish their learning
goals;
(v) Promoting communication between teachers and students, enabling the
sharing of the responsibility for setting learning goals and for evaluating
progress towards meeting those goals;
(vi) Giving students the opportunity to have extensive imput into the learning
process; and
(vii) Facilitating cooperative learning activities, including peer evaluation and
tutoring, cooperative learning groups and peer conferencing.
3. What are the critics regarding using portfolios as an assessment tool? [Aug
2018]
(i) Extra time needed to plan an assessment system and conduct the assessment
especially for large assessment;
(ii) The gathering of all the necessary data and work samples can make portfolios bulky
and difficult to manage;
(iii) Scoring portfolios involves extensive use of subjective evaluation procedures such as
rating scales and professional judgement, and thus open up the assessment to the
question of reliability; and
(iv) Scheduling individual portfolio conferences is difficult and the length of each
conference may interfere with other instructional activities.
Chapter 8
[Aug 2018]
Reliability is the degree to which an assessment tool produces stable and

consistent results.
 Types of Reliability: inter-rater reliability and intra-rater reliability.

 Methods of testing reliability: Test-Retest, Parallel or Equivalent Forms and
Internal Consistency (Split-Half, Cronbach’s Alpha)
33
Validity refers to how well a test measures what it is purported to measure.
 Three types of validity have been identified: construct validity, content

validity and criterion-related validity (Predictive Validity and Concurrent
Validity).
“Reliable but invalid”
 the test is consistently and systematically measuring the wrong construct

 If the scale is reliable it tells you the same weight every time you step on it
as long as your weight has not actually changed. However, if the scale is
not working properly, this number may not be your actual weight.
“Invalid but reliable”
??
Chapter 9
34
[Aug 2018] [no calculation in Aug 2018]
Item analysis is a process which examines the responses to individual test items
or questions in order to assess the quality of those items and the test as a whole.
In Classical Test Theory (CTT) the statistics produced from analysing the test
results based on test scores include measures of difficulty index and
discrimination index. Analysing the effectiveness of distractors also becomes
part of the process.
 Item analysis is especially valuable in improving items or questions that will

be used again in later tests
 It can also be used to eliminate ambiguous or misleading items in a single
test administration.
 Item analysis data can provide a basis for efficient class discussion of the
test results. For example, teachers can highlight the misinformation or
misunderstanding reflected in the choice of particular distracters on
multiple-choice questions or frequently repeated errors on essay-type
questions, thereby enhancing the instructional value of assessment.
 Item analysis reveals technical defects in the items or the marking
scheme, so students’ marks can also be rectified to ensure a fairer test.
 Item analysis data can be used for remedial work. The analysis will reveal
the specific areas that the students are weak in.
 Item analysis data can reveal weaknesses in teaching and provide useful
information to improve teaching. For example, despite the fact that an
35
item is properly constructed, it has a low difficulty index, suggesting that

most students fail to answer the item satisfactorily.
 Item analysis procedures provide a basis for teachers to improve their skills
in test construction. When revising the items, they gain experience in
rewording the statements so that they are clear, rewriting the distracters
so that they are more plausible and modifying the items so that they are
at a more appropriate level of difficulty.
Chapter 10
a. 𝑥̅ = 67.687
∑(𝑥−𝑥̅ )2
b. 𝑆𝐷 = √ = 15.333
𝑁−1
𝑥−𝑥̅
c. 𝑧 = = 1.95
𝑆𝐷
d. 𝑡 = 10(𝑧) + 50 = 68.52
36
Mean, median and mode measured the central tendency of the data.
 Mean
 The mean is the average, i.e. the sum of all the values (marks) divided by
the total number of items (students) in the set.
 Median
 The median is determined by sorting the score obtained from lowest to
highest values and taking the score that is in the middle of the sequence.
 Mode
 The mode is the most frequently occurring score in the data set. If you
have two modes, it is bimodal.
 Standard deviation
 Standard deviation refers to how much the scores (obtained by students)
deviate or differ from the mean.
Descriptive Statistics
You use these statistical techniques to describe how your students performed. For
example, you use descriptive statistics techniques to summarise data in a useful way
either numerically or graphically. The common descriptive techniques used are the
mean or average and standard deviation. Data may also be presented graphically
using various kinds of charts and graphs.
Inferential Statistics
You use inferential statistical techniques when you want to infer about the population
based on your sample. You use inferential statistics when you want to find out the
differences between groups of students, the relationship between variables or when
you want to make predictions about student performance. For example, you want to
find out whether the boys did better than the girls or whether there is a relationship
between performance in coursework and the final examination. The inferential statistics
often used are the t-test, ANOVA and linear regression.
37

HMEF5053 Measurement and Evaluation in Education

Hochgeladen von

Dokumentinformationen

Originaltitel

Copyright

Verfügbare Formate

Dieses Dokument teilen

Dokument teilen oder einbetten

Freigabeoptionen

Stufen Sie dieses Dokument als nützlich ein?

Sind diese Inhalte unangemessen?

Copyright:

Verfügbare Formate

HMEF5053 Measurement and Evaluation in Education

Hochgeladen von

Copyright:

Verfügbare Formate

Ong Yee Sing, August 2018

HMEF5053 Measurement and Evaluation in Education

Measurement refers to the process by which the attributes or dimensions of

Assessment is a process by which information is obtained relative to some

A test is a set of tasks or questions intended to elicit particular types of behavior

2. What are the four principles of assessment? [Aug 2018]

(i) What is to be assessed has to be clearly specified.

With regards to learning, assessment provide information to help making

Certificate of Education (SPM), so that the students can be selected for

With regard to teaching, assessment provides information regarding the class

 Assessment determine the achievement of objectives to see if the desired

Criteria Formative assessment Summative assessment

specific competencies with letter

Criteria Norm-referenced test Criterion-referenced test

1. What are the three learning outcomes?

Psychomotor learning outcome – manual or physical skills

2. List the taxonomy of all the 3 learning outcomes.

Like other taxonomies, Bloom’s is hierarchical, meaning that learning at the

university students), some levels of Bloom’s taxonomy can be skipped when

Level Description Example

The differences between original Bloom’s and revised taxonomy includes:

 The revised taxonomy used verbs instead of nouns to indicate that

Original Knowledge Comprehension Application Analysis Synthesis Evaluation

Revised Remember Understand Apply Analyse Evaluate Create

 A cognitive and knowledge matrix is added in the revised taxonomy.

The Cognitive Dimension

Affective learning outcome [Aug 2018]

Like cognitive objectives, affective objectives can also be divided into a

Level Description Example

Psychomotor learning outcome

The psychomotor domain includes physical movement, coordination and use of

Level Description Example

3. Of the three types of learning outcomes, the assessment of affective LOs is

 The belief that the development of appropriate feelings is the task of

 Affective characteristics are considered to be private matters

Categories Explanation Example

Grouping MCQ questions Difficult to differentiate between

2. List important considerations when constructing matching questions.

3. What is True/False questions? Describe and provide examples.

A statement is presented and students must indicate whether the statement is

Example: a whale is a mammal because it gives birth to its young.

4. Limitations of Ture/False questions.

5. What is objective question?

An objective test is a written test consisting of items or questions which require

Types of objective questions includes multiple choice questions, matching

6. What are the strengths and advantages of MCQ? [Aug 2018]

(a) Learning outcomes from simple to complex can be measured;

7. What are the weaknesses or disadvantages of MCQ? [Aug 2018]

(a) Constructing good items is time-consuming;

A Table of Specifications is a two-way chart which describes the topics to be

Tables of Specification typically are designed based on the list of course

The purpose of a Table of Specifications is to identify the learning objectives

Analytical marking Holistic marking

2. What is essay questions.

Based on Stalnaker's definition, an essay question should meet the following

 Requires examinees to compose rather than select their response.

3. List strengths and limitations of essay question.

1. Assess higher-order or critical thinking skills and complex learning

(a) By Their Very Nature, Essay Questions Assess Higher-order Thinking

Whether or not an essay item assesses higher-order thinking depends on the

(b) Essay Questions are Easy to Construct

Essay questions are easier to construct than multiple-choice items because

(c) The Use of Essay Questions Eliminates the Problem of Guessing