part C The process of materials evaluation

7 Piloting - a publisher’s view

Peter Donovan

Introduction
There is a range of situations in which teaching materials are trialled in
the course of their development. Individual teachers often prepare
exercises or sets of material for their own classes, which can be refined I
over a period of time on the basis of ‘hpw it went’ with a number of |
classes. Rather than trying to arrive at a final, definitive state for the ;
materials, such development is often a continuing open-ended process of
refinement and adaptation to different groups of learners. In some
situations, a group of teachers may prepare materials for use by
themselves and by other teachers in their own institution, which may
then be finalised in a more permanent state for ongoing use after a
trialling period. In larger scale materials development projects, a pilot or
trialling phase may be built In before the materials are revised and
disseminated more widely and formally.
This chapter attempts ro describe the issues involved in piloting on
a wider and more formal scale, during the course of development of
materials leading up to their commercial international publication.
Several characteristics of this process distinguish it from the situations
described above. Firstly, the original authors of the material are
frequently physically distanced from the piloting event. While they
might be able to teach some or all of the materials themselves (and it is
undoubtedly desirable that they do so), the pilot material will be
channelled through the publisher’s editors and marketing staff to
teachers who will not know the authors, nor their intentions except as
they are expressed and realised in the materials themselves. The authors
will therefore typically have no direct influence over most of the
piloting situations or regular contact with them. (Although large-scale
piloting projects where authors move between teaching and materials
writing do exist, thest* are less common and tend to be focused on
requirements for a particular educational sector, often sponsored by a
ministry, a publisher, an aid agency, or a combination of partners.)
 Secondly, the piloters themselves will usually have a free choice as
to ^ whether to use the materials or not (unlike teachers who may be 1
required by their institution or their peers to use material being specially
J produced for their benefit). Thirdly, and related to the latter point, the -
1 footers may not directly need the pilot materials as part of their %
teaching programme. Institutions typically have their own requirements
; satisfied through their choices of existing published material. Pilot |
materials are offered as an optional extra, which teachers may decide to |
incorporate in their programmes for n given period out of interest, for 1
variety, because of a relationship with a publisher, or because they 1
might fill a need which is not currently catered for. It would be difficult
1 for a publisher to coerce individual teachers into participating in a \
piloting project, although individual teachers might not be able to resist
J the decision of a head of department or institution to take part.
Pre-publicnrion piloting is usually for a specific, limited period,
which j has to reconcile the requirements of the publisher’s development
1 schedule with the constraints of the teaching situation — the period in
i which the materials can be used during the teaching year, term or \
semester, and the number of teaching hours which can be devoted to it.
1 1c will involve the teacher providing feedback in the form of;answers
to | a questionnaire, and sometimes in addition a written report ,of the
pilot I experience. This may, optimally, be supplemented by informal
inter- \ views with the authors, editors, or marketing staff, for more
detailed and anecdotal comment. A final distinction is that in somj
situations i payment or some other benefit such as books or materials
may be J offered to the teacher or institution involved in the piloting,
usually in | return for particularly detailed reporting on the material and
the j experience over and above the completion of questionnaires.*
What therefore distinguishes the process of piloting materials in j
preparation tor their formal publication, as opposed to other modes of |
piloting or trialling, can be summarised in terms of factors of proximity,
. channels of communication, volition, time and benefit.

What is piloted?
Given that the piloter is usually at a distance from the author and
publisher, it follows that what is supplied for piloting should be a
to be used from the pilot package itself, without the intervention of the author
or editor, making the teacher’s buok an essential part of the package. There will
need to be clear indications of the intended audience, ages, level, teaching rime,
needs and expected outcomes, and so on. Thu learners must have clear printed
prges and audio material to work from, and so the work must have goner
through at least a preliminary editorial stage before release. IdcaT^ as much of
this material as possible should be in the teacher’s hands when making a
decision on whether to pilot. The teacher should also have the questionnaire for
the pilot from an early stage - either when making a decision, or from the start
of the pilot programme - in order to be able to see what the aims and scope of
the pilot are, and to be alert to the issues on which feedback is required.
Whilst pilot materials around ten years ago might have been made up of a
4
cu.t and paste* version of the author’s typescript, rising expectations from
users and the widespread use of ‘desk-top publishing’ packages on authors’
personal computers have led to pilot materials appearing in more professional
form, sometimes laid out in ways reflecting the design of the final materials,
and incorporating representative artwork scanned into the pages. Pilot materials
are, however, likely to be provided in black and white only, for reasons of cost.
Exceptions might be some types of material for younger learners, or specific
pages on which colour is essential for teaching purposes. (While the reasons for
a basic level of presentation are accepted by the majority of piloters, feedback
often contains references to the standard of design, and hopes that the final
production will be of higher quality, in colour, with improved layout, etc.)

Examples of piloted materials

Below are examples of pages from two different types of pilot materials, and an
example of how one of them has appeared in final published form. Figure i is an
example of a current supplementary (grammar) pilot, for learners aged around ti.
Figure i consists of some pages from the pilot of Cambridge English for Schools,
again for learners aged around n, which was produced in the early 1990s. Figure 3
shows the corresponding material in its final published form, in 1996. As you look
at the materials notice the differences between the pilot and the published versions.
 The process of materials evaluation

5B What's in your fridge?

Kim’s mum is angry. There is orange juico ami water on the
kitchen Coor. The dirty glasses are on the table. The dirty
jug is in the sink. Tho bowl is under the table. There’s an
orange behind the door, the empty sugar bag is next to the
bin fcnd there's an orange between the oven and the
(ridge...
1. Read the description and answer the
questions.
1 Where’s the knife?

2 Where's the jug?

I;‘s ...................... .. the sink.
3 Where are the dirty spoons?
Theyre ........ the fridge.
j§ Write the correct words for the

........... by beside in front of above

Check your answers on page 60.
 3. Write the questions.

It's under the TV.

It's in the bin.

They’re by the telephone.

4
jod c aj
It's above the ove

They're between the crisps and the salad.

4. Look at the picture and finish dta sentences*

«
1 The post office is . the river.

2 The seats are ....... .... the bridge.

3 .............................................. The tennis courtis the

park.
The book shop is ........ the bank
and the police station.
5 Tt e bus stop is ... ........... the bank.