Moderation of HSC Assessment Marks

BOSLO Videoconference 27-29 July 2010

Any Questions?
To ask a question please send an SMS to:

0417 244 139

These questions will be answered at the end

OR Email: kim.owens@bos.nsw.edu.au
Make sure your microphone is OFF during the video conference

Standards-referenced approach to assessment

Assessment is the process of gathering information about student achievement HSC student achievement is described in the Performance Bands, based on:
knowledge, skills and understanding

Student achievement in Design and Technology

Band Design and technological development (dot point)

6 5 4 3 2

critically analyses the factors which influence analyses the factors which influence explains the factors which influence describes the factors which influence identifies some factors which influence

External examinations
Examination committees work closely with performance bands; they need to:
Set items that allow students to show their level of knowledge, skills and understanding Set marking guidelines that distinguish between different levels of achievement

School-based assessment
Teachers must also work closely with performance bands; you need to:
Set tasks that allow students to show their level of knowledge, skills and understanding Set marking guidelines that distinguish between different levels of achievement

For more details

HSC Assessment in a Standards-Referenced Framework A Guide to Best Practice

School Assessment marks: Internal meaning

From standards-based assessment marks the Board of Studies can accept/preserve:
The rank order of students The relative gaps between students

Internal meaning: A School and B School

A School Greg Hugh Irene Jane Kylie Leon Mark 96 84 80 79 66 51 Rank 1 2 3 4 5 6 B School Mark Rank Ulrika Vivienne Walter Xavier Yvonne Zadie 85 84 83 62 52 35 1 2 3 4 5 6

School Assessment marks: External meaning?

The Board of Studies cannot assume that the submitted marks are aligned to the performance bands exactly
Eg. We dont know that all AND only students with Band 6 achievement have marks 90-100

External meaning: A School and B School

A School Mark Band Greg Hugh Irene Jane Kylie Leon 96 84 80 79 66 51 6? 5? 5? 4? 5? 3? 2? B School Mark Band Ulrika Vivien Walter Xavier Yvonne Zadie 90 89 88 62 52 35 6? 5? 5? 6? 5? 6? 3? 2? 1?

Role of Assessment Moderation

School assessment marks are moderated using external examination marks The role of moderation is to:
Connect assessment marks with the standards Retain schools rank orders and relative gaps

Moderation at the Board of Studies

Assessment marks are moderated using a process designed by Dr Robert MacCann, the former Head of Measurement and Research Services Branch The process uses a quadratic polynomial
a curved adjustment

The Quadratic Equation

For each school group, three values need to be calculated:

a, b

and

y = ax 2 + bx + c
Once calculated, any school assessment: x can be moderated to y

Quadratic Equation Conditions

We apply three conditions to each school group:
1. Highest assessment mark to highest exam mark 2. Assessment mark mean to exam mark mean 3. Lowest assessment mark to lowest exam mark

Solving the Quadratic Equation

Once the conditions have been applied, we use formulae to determine

a, b and c

See Dr MacCanns paper for these formulae:

The moderation of Higher School Certificate assessments using a quadratic polynomial transformation: A technical paper

Moderation of A School
Assessment Examination Student Greg Hugh Irene Jane Kylie Leon Mark 96 93 84 83 80 80 79 79 66 71 51 64 Rank 1 2 3 4 5 6 Mark 93 84 76 82 64 71 78 Rank 1 2 4 3 6 5

Mean 76 78

Moderation of A School
100

Moderated Assessment

93
90 80 70 60 50 40 40

78 64

Note: most programs have fewer decimal places than the BOS system; this can result in imprecise estimates for a, b, c

51
50 60 70

76
80 90

96
100

School Assessment

Special Moderation Cases

Sometimes a change in process is required We will work through the most common special cases here Further details can be found in Dr MacCanns paper

One student
100

Name John

Assess Exam 63 60 60 60 60

Moderated Assessment

90 80 70 60 50 40 40 50 60

Mean 63

60

63
70 80 90 100 School Assessment

Two students: low exam OK

100

Name 82 Maud

Assess Exam 86 82 82 79 79 79

Moderated Assessment

90 80

Dorothy 90 LowEst 78

79
70 60 50 40 40 50 60 70

LowEst =

min SA max Exam max SA 86 78 = 82 90

86
80

90
90 100

School Assessment

Two students: low exam not OK

100

Name 82 Maud

Assess Exam 86 82 82 78 72 72

Moderated Assessment

90 80 70 60 50 40 40 50 60 70

Dorothy 90 LowEst 78

78

LowEst =

min SA max Exam max SA 86 78 = 82 90

86
80

90
90 100

School Assessment

Assess Exam Assess Exam

Ties on first place

100

89 89

85.5

80 87 77 78 79 62 77 84

76 74 73 69 68 66 61 55

75 53 57 58 69 57 41 43

Moderated Assessment

90 80 70 60 50 40 30

85.5

88 82

67

79 79 78

41 55
30 40 50 60 70

78

75
80

89
90 100

School Assessment

Illness/Misadventure: low exam

100

IM? N 80 Y N N N 47 35 52 61
30 40 50 60 70 80 School Assessment

Assess Exam 86 80 81 73 59 44 58 42 52 35 52 80 46 73 38 45 41 32 47

Moderated Assessment

90 80 70 60 50 40 30

N 86
90 100

Mean 61 47

Illness/Misadventure: high exam

100

IM? N 80 Y N N 52 N N 35 52
30 40 50 60

Assess Exam 86 80 81 74 59 45 58 43 52 52 80 76 73 38 45

Moderated Assessment

90 80 70 60 50 40 30

41 35 32 65 Mean 61 47 52 47 52

65
70 80

86
90 100

School Assessment

IM on first place: low exam

50

46 42 37

IM? Y N N N

Assess Exam 48 43 42 39 49 29 49 46 46 43 43 37 37 42 42

Moderated Assessment

40 30 20 10 0 0

Mean 41

39
10 20 30 40

43 41
50

School Assessment

Depressed exam performance

Every year, a few students have an atypically low exam mark: depressed We compare all students moderated assessment and exam marks Depressed cases are moderated like IMs, but they retain their own low exam marks

Adjustments to bottom mark

A moderation curve that is decreasing or flat at any section is not permitted To fix this, we adjust the lowest moderated assessment mark (not the top or the mean) Dr MacCanns paper provides further details

Contact details:
Metropolitan Liaison Officers Margot Braithwaite Phone: (02) 9367 8387 Mobile: (0408) 021 193 Jennifer Michalski Phone: (02) 9367 8445 Mobile : (0417) 270 680 Kim Owens Phone: (02) 9367 8311 Mobile: (0417) 244 139
Regional Liaison Officers Meredith How - North West TAMWORTH Phone: (02) 6755 5043 Mobile: (0428) 667 668 Lindy Billing Riverina WAGGA WAGGA Phone: (02) 6937 3889 Mobile: (0408) 693 063 Edwina McCoy North Coast COFFS HARBOUR Phone: (02) 6659 3274 Mobile: (0428) 660 126 Ian West Hunter NEWCASTLE Phone: (02) 4924 9976 Mobile: (0419) 499 176 Alanah Miszuk Western BATHURST Phone: (02) 6334 8048 Mobile: (0418) 635 920 Debra Watson - South Coast WOLLONGONG Phone: (02) 4224 9940 Mobile: (0428) 424 354

10