Standardisation

Information and Guidance for OCN North East Region Centres

Version 1 June 2010

CONTENTS
OVERVIEW AND BACKGROUND............................................................................ 2
Levels of Standardisation ...................................................................................................2

AIMS AND PURPOSE OF STANDARDISATION .....................................................3

Purpose.................................................................................................................................3 Aims......................................................................................................................................3 Outcomes..............................................................................................................................4

NOCN REQUIREMENTS FOR RECOGNISED CENTRES AT THE DIFFERENT LEVELS OF STANDARDISATION............................................................................ 5

Centre (internal) Standardisation........................................................................................5 Regional Standardisation....................................................................................................5 National Standardisation.....................................................................................................6

CENTRE PRACTITIONER FORUMS........................................................................7 RETENTION AND COLLECTION OF EVIDENCE FOR STANDARDISATION.........8

Collection and storage of samples for standardisation ...................................................8 What the sample should include.........................................................................................8

PLANNING AND CONDUCTING A CENTRE STANDARDISATION EVENT .........10

Who should attend a standardisation event?..................................................................10 Documents required for the meeting................................................................................11

APPENDIX 1.......................................................................................................... 14 RETENTION OF LEARNER EVIDENCE AND NOCN STANDARDISATION REQUIREMENTS.....................................................14 APPENDIX 2.......................................................................................................... 17

PARTICIPANT FEEDBACK FORM......................................................................... 15 APPENDIX 3.......................................................................................................... 20 FACILITATOR FEEDBACK FORM......................................................................... 18

................................................................................................................................................. ................................................................................................................................................. ................................................................................................................................................. ................................................................................................................................................. ................................................................................................................................................. ................................................................................................................................................. ................................................................................................................................................. ................................................................................................................................................. ................................................................................................................................................. ................................................................................................................................................. .................................................................................................................................................

Overview and Background

The process of Standardisation, in this context, is to ensure that any and all of the judgments about assessed work which are made by Tutors, Internal Verifiers and Quality Reviewers or External Verifiers are both consistent and are evidenced. The aim of this process is therefore to improve overall consistency by agreeing the specific standards which are to be met and identifying benchmark (see below) tasks and evidence. This process must occur internally within individual centres own processes and will also be provided for centres by NOCN and OCN North East Region. It is important to note that: 1. Within this context the term benchmark means a point of reference by which all standards can be referenced and 2. Standardisation is not about re-assessing the samples that are provided for discussion but about measuring them against the agreed examples (benchmarks). Therefore, whilst the format and focus of standardisation events may vary, the purpose is to draw out and agree the standards that individuals are applying when assessing and monitoring any assessed work. In other words the process should clearly explain: what is expected to be seen in particular pieces of assessed work = the standards when this work is considered to be good enough and when it is not = that is when it does or does not meet standards (the benchmarks) This overarching question is therefore equally applicable to all units, regardless of their level, credit value or curriculum area. The samples which are provided at standardisation events are simply a means of providing examples for discussion of those standards.

Levels of Standardisation

There are 3 levels or tiers of standardisation: Centre (sometimes called internal) Regional (OCN North East Region) National (NOCN)

The standardisation activities for each of the 3 levels will be related and should both influence and be influenced by, the outcomes of the others. All N/OCN recognised centres are required to carry out appropriate centre standardisation and also to contribute to regional and national standardisation, as requested, as part of their Centre Recognition compliance. This document outlines what is required by OCN North East Region centres for all levels of standardisation and offers specific guidance on the implementation of centre standardisation for individual centres.

Aims and Purpose of Standardisation

Purpose
The overall purpose of standardisation at all levels is: To ensure consistency in the application of level and credit across N/OCN provision To promote and achieve consistency of practice in assessment and verification To inform the quality improvement cycle which will include training and development issues To ensure that a full range of assessment activities allow for the generation of appropriate assessment evidence To share good practice To identify, develop and use national benchmarks and to achieve processes for those to be applied and reviewed

Aims
The aims of any standardisation event at any level will include some or all of the following: To provide an opportunity to ensure the consistency of the award of credit to learners across different assessors/verifiers/centres and to agree the standards to be achieved To compare the different assessment strategies used across a unit/s and to ensure that: o The assessment strategy, including any benchmarked tasks, allow for the generation of sufficient/appropriate/authentic evidence to be produced o The assessment strategy and the assignment tasks which are used prevent any unintended barriers to achievement for a particular learner and pay due respect to equality and diversity issues To ensure that the assessment tasks are written in language that is appropriate to the level and is free from bias

To ensure consistency in the assessment judgements that are being made, based on the available evidence To ensure the adequacy of the feedback provided to learners To ensure the adequacy of the feedback provided to assessors and the robustness of the internal verification of the assessment judgements To review the appropriateness and currency of the units and their fitness for purpose in the current context and make any necessary recommendations To identify and share best practice

Outcomes
The outcomes of the standardisation process should be used to: Improve the assessment practice of tutor/assessors within centres and bring about consistency of assessment decisions to safeguard the integrity of the award of credit Improve consistency in the judgements of verifiers Influence and improve the qualification or course design and any prescribed assessment tasks within the qualification. This would normally be done at national level but will be influenced by feedback from the standardisation activities at other points. At centre level this process can be used to consider the appropriateness of the current courses being delivered and therefore make any changes.

NOCN Requirements for recognised centres at the different levels of standardisation

Centre (internal) Standardisation
In any centre which is recognised to deliver N/OCN courses, it is essential that the internal verification process includes an opportunity for the standardisation of their practice in relation to assessment judgements. This should happen in all cases where assessment decisions and recommendations for the award of credit to learners are made. This will be determined by the number of tutors/assessors and the courses which are being delivered in the centre. It is usual that the Internal Verifier for the course/s has the responsibility for arranging and facilitating these standardisation events. The guidelines contained within this booklet have been designed for use when organising these internal standardisation events to meet the specifics of individual centres. These activities and events will be monitored by the Quality Reviewer or any person with Authorised Internal Verifier Status (AIVS) where that is appropriate and will be included in Quality Reviewer reports, including any quality improvement actions required to develop the processes.

Regional Standardisation
OCN North East Region organises regional standardisation events, which cover qualifications and qualification units within specific sectors, within an annual cycle. Individual centres will be informed of the dates of these events at the beginning of September in each year and informed when/if they are required to supply samples of work for particular events. This will be done in sufficient time for this to be included in the planning processes. Further information about requirements can be gained from the Quality Reviewer. Centres will be asked to participate in these events by contributing samples of evidence (where appropriate), and through the attendance of individuals who have Approved Internal Verifier status, where this is appropriate, especially if they are approved for a particular sector area involved. How will this be done? When samples are required from a centre the Quality Reviewer is responsible for agreeing what samples will be required and when they will be collected. The samples should of course have been through the centres own internal standardisation processes so these events should be planned accordingly. This process of identifying and collecting samples will therefore normally be included within the Quality Assurance planning and agreed visits. Samples may be requested at other times but Quality Reviewers will arrange how these will be collected (see section on sample selection). Please note therefore that centres are not required to attend the OCNs regional standardisation events except where there is an appropriate AIVS individual. However it is particularly important that those people with Approved Internal Verifiers status participate in these regional standardisation events. (We will provide certificates of attendance for CPD

evidence if required.) Invitations will be made, when appropriate, but please contact us for further information. However in addition centre practitioner forums for identified sectors or qualifications will be held, where appropriate (see below). Feedback from both National and Regional standardisation events will be available for all centres following each event.

National Standardisation
The National Open College Network (NOCN) organises standardisation events for all its qualifications by sector on a three year cycle. This programme will therefore inform OCN North East Regions standardisation programme including the regional events, as above, and then any subsequent information that is provided for centres. Samples from those collected for, and discussed within, the regional events will be selected to send to the National Standardisation events as OCN North East Region examples. OCN North East Region Quality Staff, including Quality Reviewers with a specialism in the sector area, attend these national events. Feedback from these events will be distributed to OCN North East Region centres following all national events and will often determine the need for any centre practitioner forums.

Centre Practitioner Forums

Centre Practitioner forums, such as the one currently held for the NOCN Languages qualification, form part of the overall OCN North East Region standardisation strategy. The purpose of these events is to offer the opportunity for centres to get together to share good practice, to ensure any specific requirements are met and to consider possible development work in relation to the qualification or curriculum or sector area. These events also serve an as opportunity to distribute and discuss any outcomes from the other levels of standardisation. Centres will be informed of any planned practitioner forums along with the regional standardisation schedule. Centres can make suggestions for a topic or curriculum area which could be covered within these events by discussing this with the Quality Reviewer.

Retention and collection of evidence for standardisation

The NOCN retention policy is attached as Appendix 1. This policy sets out the requirements for retaining learner details along with the samples of evidence which are discussed within this guidance and should be read in conjunction with this document. This will have been explained within the centre recognition process and will be discussed and monitored as part of the Quality Reviewers visits.

Collection and storage of samples for standardisation

All samples should be taken from the centres current learner evidence and the Quality Reviewer will explain what format the samples should take. These can be supplied as photocopies and all original copies will be returned. All samples are stored electronically at OCN North East Region. As well as providing samples for national standardisation events, this will also allow us to build up a range of samples which can be used to compare standards over time and to create benchmark samples which can then be used as guidance for centres. These samples will be anonymised. For further information or guidance about retention of learners work and details please ask the Quality Reviewer or Quality Accreditation Manager.

What the sample should include

At all levels of standardisation it is important that in order to be effective, the sample provided should be as complete as possible. This is to ensure that informed judgements can be made about all of the evidence to support the purpose of the standardisation process which is explained in the overview section. It is good practice within all standardisation activities but it is a requirement for those supplied for OCN North East Region that the sample should support the evidence of achievement for a whole unit of assessment and should include: A Sample Cover Sheet on each unit indicating the rationale for selecting the sample A copy of the unit descriptor including the unit code

The whole assessment strategy for the unit being standardised including the tasks/ assignment briefs /benchmarks All of the evidence generated by a learner/s against all of the assessment criteria for the unit/s Clear record of the assessment decision including feedback to the learner in relation to the achievement /non achievement of the assessment criteria Internal verification records including feedback to the assessor and the internal verifiers judgements on the security of the award of credit

 Evidence where possible of subsequent monitoring of any actions identified by the internal verification process A centre can develop their own paperwork for internal events but the Quality Reviewer will confirm that any paperwork is appropriate and explain what is required when asked to supply samples for OCN North East Region.

Planning and conducting a centre standardisation event

The following guidance aims to support centres in implementing standardisation activities within their own programmes. In order to facilitate an effective standardisation meeting it is important to undertake some preparatory work prior to the meeting.

Who should attend a standardisation event?

This will depend on the size of your centre. You should agree who will facilitate the event and invite tutors and internal verifiers who are currently or may be involved, in the planning and delivery of your courses. This might be specifically useful if you offer separate courses within different sector areas at the same level. Feedback to tutors who cannot attend will form part of the actions following standardisation. The facilitator should be prepared to compile a report of the event with discussions, agreements and actions. Sample selection You should give clear instructions about what the sample selection should include (as above). Unit selection You should select the units which are to be standardised and ensure that over time all units, courses and levels being delivered are examined through the standardisation process. The aim is to ensure that all units are reviewed at some time. This will be individual to you as a centre and will of course depend on the length of your courses, the different courses including the different levels and the overall number of units which you deliver. This may also depend on what has been requested from you by OCN North East Region. At regional and national level, standardisation events are normally organised by sector and therefore specific units of qualification which you may be approved to deliver will be requested. Sample size The number of samples to be scrutinised should be small and manageable. It is usually more productive to have a smaller rather than larger number of units for examination but it is important that in all cases you should make sure that a cover sheet for the sample is complete. The purpose of this cover sheet is to indicate a rationale for why the sample has been chosen. So for example, does the sample represent one or more of the following factors: Strong evidence of achievement Weak evidence of achievement or a possible borderline case Evidence which is sufficient to ensure achievement of all of the assessment criteria at the specified level Evidence which is not entirely appropriate for the level Evidence which is not sufficient to demonstrate achievement of all the assessment criteria The use of an innovative assessment strategy An example of good and constructive feedback to a learner An example of good and constructive feedback from the verifier to the assessor 10

Example of Non achievement Any other valid reasons for the selection

Within a centre the sample should include evidence assessed by each of the assessors involved. Within a centre the sample should include evidence verified by each of the internal verifiers.

Documents required for the meeting

Sample of work selected You should provide copies of the samples so that all participants in the standardisation event are able to scrutinise each one. You may wish to make these anonymous. However if you are required to provide samples for regional standardisation then names of the learners will be required. Participant Feedback Form Appendix 2 Facilitator Feedback Form Appendix 3

The following documents will also be a useful source of reference for participants and are available for download from OCN North East Region website: NOCN Assessment Definitions Assessment Guidelines Qualification Guide For standardisation of units of NOCN qualifications, you will need the appropriate qualification guide Standard agenda The following is an example of a possible suitable agenda for a standardisation meeting: Standardisation Meeting Curriculum area/s: Date and Venue: Agenda: 1. Overview and expected outcomes 2. Standardisation consideration of the samples provided and completion of participant form. 3. Verbal feedback agreeing the standard/adequacy of evidence and supporting documentation/best practice this information may be used to complete an overall report.

11

4. Selection of sample - to be used for regional standardisation if this is required. Completion of sample cover sheet for retained samples if required. This can be done in conjunction with your Quality Reviewer. Conducting the event 1. Overview and expected outcomes At the beginning of the meeting the facilitator should ensure that participants are clear about the purpose of the standardisation event and what their role is. This should also include how the participant feedback form should be completed and explain how that information will be used in producing the facilitators report. Reference should be made to the guidance documents above and how they can be used by the participants during their consideration of the samples. 2. Consideration of samples Each participant should examine each sample and record their responses to the questions on the participant feedback sheet i.e. all participants should consider all and the same samples. This is likely to take some time which should be set aside for the activity. 3. Verbal feedback: agreeing standards and the adequacy of supporting documentation Once all the samples have been scrutinised and a participant report has been completed, the facilitator should lead a discussion of the findings. This will probably best be achieved by considering each sample and sharing the participants views. It is important to remember that the main outcome of this discussion should focus on agreeing and defining the acceptable standards including the rationale for those decisions. The facilitator should record the outcomes of this discussion and produce a summary report of the findings. 4. Selection of sample to be used for the next level of standardisation (if required) The outcomes of the discussion should enable the group to agree on an appropriate sample to be retained and used at the next level of standardisation if this is required. In this case the facilitator should then complete a sample cover sheet for the chosen samples that clearly identifies the rationale for the choice and how it helps to contribute to a consistency in agreeing standard statements/benchmarked tasks/or dissemination of best practice. If these arent required for regional events then the centre may retain them for their own purposes. Reporting the Outcomes Participant Feedback Form (Appendix 2): this can be done in groups All participants should complete a Participant Feedback Form to record their findings for each unit scrutinised Facilitator Feedback Form (Appendix 3)

12

The facilitator should take feedback against the participants findings unit by unit. The Participant Feedback Forms can be retained by the facilitator. The facilitator should then complete the Facilitator Feedback Form to summarise the findings of the meeting and record recommendations/good practice identified to go forward to the next level of standardisation if required. The sample to go forward to the next level of standardisation should be selected to reflect the issues identified in this report Sample cover sheet The appropriate sample cover sheet should be attached to the sample selected to go forward to the next level of standardisation. This should clearly record the rationale for selection. Training on conducting a standardisation event is one session on OCN North East Regions provider training programme. For further details please see our website

13

Appendix 1

Retention of Learner Evidence and NOCN Standardisation Requirements

This information supplements the requirements set out in the NOCN Centre Handbook and the OCN North East Region Standardisation Guidance document. Centre Evidence Centres must keep complete and accurate records, for at least three years from the end of year to which they relate, for all qualifications and make these available to NOCN or OCN North East Region on request as part of their centre recognition agreement. Centres are required to retain the following records: Name of learner, date of birth and contact address Title and accreditation number of each qualification and unit studied Name(s) of assessor(s) Assessment records, including assessment decision and reason for decision Name(s) of internal verifier(s) Internal Verification reports

Evidence for Standardisation All Centres should hold internal standardisation events as part of their quality assurance practice and covering their full range of provision (for example a minimum of three pieces of evidence from every tutor/ assessor). These should include: The Unit Learners Work Assessment Task Tutor Feedback

Centres offering specific qualifications may be requested subsequently to provide evidence for regional standardisation events. Identified centres will be supplied with full information on the type and amount of evidence and the date when they will need to submit this, directly from OCN North East Region. Quality Reviewers will be responsible for identifying and collecting this specific evidence from Centres. OCN North East Region will forward a selection of these samples to NOCN for national standardisation events and retain these and other samples for its own use. Feedback from these events will be available. Summary When the requirements of regional standardisation have been met, centres are no longer required to retain samples of learner work. It is recommended however that centres do retain some samples for their own quality assurance purposes as part of centre standardisation. For further advice and support please contact your Quality Accreditation Manager, Quality Reviewer or OCN NORTH EAST REGION Quality Development Manager. Provider training sessions covering standardisation are available ~ for further details please contact helen.smith@OCN North East Region.org.uk 14

Appendix 2

Participant Feedback Form

Qualification/Course title Unit Title Level Credit

1.

Which learning outcomes do you think are being addressed by the evidence presented?

2.

Is the assignment/task for the unit set at the correct level?

Yes or No

3.

Do the tasks give the learner the opportunity to fully meet the assessment criteria?

4.

Is the assignment/task set in an appropriate format for the learners? 15

Appendix 2
Yes or No

5.

Is the language set in a way that prevents any unintentional barriers?

6.

Has the learner achieved the appropriate standard with acceptable evidence for the learning outcomes?

Yes or No

7.

Is there sufficient evidence to cover the learning outcomes?

8.

How is this related to the different levels?

9.

Is the feedback to the learner adequate and constructive?

16

Appendix 2

10.

Any other issues?

Signed:

Print Name:

Date:

17

Appendix 3

Facilitator Feedback Form

Event Title Date Facilitator Title of the units being standardised Unit Title Unit code Credit Value Credit Level

Summary of findings 1. Standards Did the scrutiny of the samples highlight any areas of concern about the consistency of standards being demonstrated?

Was there a clear distinction between the evidence of achievement demonstrated at different levels?

Are the expected requirements at each level identifiable?

2.

Appropriateness of assessment tasks 18

Appendix 3
Do the tasks provide learners with an opportunity to meet all of the assessment criteria? Consider reasonable adjustments.

3.

Evidence of Achievement

Issues identified relating to the: Validity?

Sufficiency?

Reliability?

Authenticity?

Other?

4.

Tutor Records and Feedback

5.

Internal Verification Records 19

Appendix 3

6.

Curriculum plan - areas for development

7.

Good Practice

Please give clear details of any good practice identified in any of the samples:

Signed:

Print Name:

Date:

20