MASARYK UNIVERSITY FACULTY OF EDUCATION

DEPARTMENT OF ENGLISH LANGUAGE AND LITERATURE

Some aspects of assessing pronunciation in EFL classes

Diploma thesis

Brno 2007

Supervisor: PhDr. Jaroslav Ondrek

Written by: Simona ebestov

Declaration I hereby declare that this thesis is my own and that I used only the resources stated.

Simona ebestov .......................................

Acknowledgements I wish to thank to PhDr. Jaroslav Ondrek, the supervisor of my thesis, for his valuable advice and guidance on the whole work. Thanks also belong to Petra Hoej and tpnka Zemanov for their help with the assessment and my special thank goes to all my students who willingly cooperated.

Contents 1 Teaching and learning English pronunciation..............................................................6 3 Practical part.................................................................................................................29 4 Research part..............................................................................................................48 Conclusion.......................................................................................................................73 Appendix 2......................................................................................................................78

Introduction English language plays a dominant role in modern

world. Each of us has ever been encountered with the necessity to learn the language for many reasons. For many people is the most important with function other of any when is language communication the trend of people English

travelling, studying or working in a foreign country. Therefore, learning increasing and the opportunities for the actual use of the language are becoming more frequent. The topic of this work is based largely on the communicative function of language speaking and listening. We intend to deal with an aspect of English which is, in our opinion, often neglected in English lessons pronunciation testing and its assessment. The course, idea arose from our be belief that from teaching the very

pronunciation should be a part of a syllabus of any English moreover, should included beginning. One reason for this statement is that it is generally believed that the age factor is influencial and small children can learn pronunciation more easily than the older or adults. The next benefit of pronunciation learning is its possible effect on listening skills. In order that our communication is comfortable for both sides, we need to understand as well as our speaking should be understandable. So the practice of our pronunciation helps other people to understand us better. Nowadays, almost every coursebook offers pronunciation practice but any suggestions on giving such a structured feedback like those on other aspects of language are rare. We will try to present ways of testing pronounciation and try to deal with the possible ways of the consequent assessment. 5

Apart

from

the

objective

methods

of

testing

pronunciation which are not usually available in schools, there are other more often used subjective methods. These methods are based on the assessors impression, so certain criteria should be followed to reach as reliable results as possible. The first part of this work will present the tasks needed to be considered for efficient learning and teaching the role of the teacher and the learner and the factors which have an effect on the whole pronunciation teaching and learning process. Next, we will give an overview of the testing methods in connection with the two approaches to impression-based assessment atomistic and holistic. The last research part will implement the theory into practice and find out about what results can be actually brought by the pronunciation testing and what problems in assessment may be encountered.

1 in

Teaching and learning English pronunciation Pronunciation seems to be sometimes a neglected part our English it for lessons. Many many reasons. teachers are As most not used to our teachers in

teaching

schools are not native speakers, there is no need to set native-like pronunciation as one of our goals, moreover, most teachers themselves do not feel perfect in this language component and thus feel reluctant to show it. But we do not need to be perfect to enable our pupils to achieve their best. The next obstruction for teachers is the lack of pronunciation tests and general unfamiliarity with their assessment. speaking to Nevertheless, giving students should feedback on be involved. correct our

Learning pronunciation will elevate their level of speaking and undoubtedly will improve their listening skills. 6

Before teaching pronunciation, many aspects should be taken into account. Among the most important ones are the roles of the teacher and the learner. On the one hand, what they aim to achieve and, on the other hand, what they are willing to give to succeed in achieving it. Before setting up goals and working out a plan, the teacher should know about their learners skills and limitations. Needless to say, the overview of the various aspects of English pronunciation sounds, stress, rhythm and intonation is essential. 1. 1 The role of the teacher As pronunciation is a very complex language component, there are many tasks for the teacher. Firstly, helping learners hear and produce sounds from their native language point of view. on. Secondly, It is it is establishing to set out what to concentrate advisable priorities

according to the acceptability to the English speaker as well as to what is beyond good intelligibility and is not necessary to take ones stand on. Thirdly, devising activities, adjusting them for different learning styles, and according to the aspects which influence pronunciation learning as it is going to be described in the following chapters. The last but not least important task is providing feedback and assessing learners performance and progress. Assessing ones own production of speech is very difficult as we tend to hear ourselves in a distorted way and thus it is complicated to compare with the correct production. Moreover, the feedback on how the learner is doing and progressing is an essential motivation factor in further pronunciation learning.

1. 2 which

The role of the learner The learners role is the same as in any other subject means taking ones own responsibility and being

willing to learn. Here, the teachers possibilities are limited, but still the choice of appropriate activities, motivating learners and building the general awareness of usefulness may be supportive. For example, one of the methods for increasing motivation can be a class-discussion based on ones own experience with foreigners and their pronunciation, what is acceptable and what is disruptive while talking to a foreigner in any language. The issue of motivation within all subjects of learning is definitely very complex, depending on many inner and outer factors and should not be neglected. 1. 3 Setting goals in English pronunciation People learn languages for many different purposes. And therefore, the goals for individual learners may vary. From the teachers point of view, the following aspects should be taken into account: the age, natural ability and motivation of the learners which is to be the base for answering the questions about how much time we will devote to teaching pronunciation and what level is needed for obtaining efficient communication. This is difficult, since, in contrast with e. g. grammar or vocabulary plans, pronunciation does not enable this particular progressive pace as all phonetic and phonological features occur from the very beginning. Nevertheless, we can count on the subconscious acquisition of the sound of English which will be beneficial for both, teachers and learners, later on. We can delimit two extreme on the targets other in learning many pronunciation. On the one hand, some learners aim to obtain native-like pronunciation, hand, learners purposes are more practical in the way that as 8

long as their speaking is comprehensible, they do not have the need to improve. Both these opinions have their advantages and disadvantages. In practice, many learners do not achieve native-like pronunciation and the question is, if its obtaining is necessary. At the same time, learning pronunciation does not only improve speaking, but has a great influence on our listening skills, so its practice is useful. According to Gimson (1994, p. 273), the first extreme target is achieving just such a level of pronunciation which enables understanding. Gimson mentions so called Minimal General Intelligibility as the lowest requirement. This Minimal General Intelligibility: possesses a set of distinctive elements which

correspond in some measure to the inventory of the RP phonemic system and which is capable of conveying English listener a message efficiently standpoint, time to from given in a native the the listeners has had that to

context of the message is known and that the tune speakers pronunciation. (p. 273) In its opposition, Gimson describes High Acceptability as: a form of speech which the native listener may not identify as non-native, which conveys information as readily as would a natives and which arrives at this result through precision in the phonetic realization of phonemes and by confident handling of accentual and intonational patterns. (p. 273)

As

the

previously

mentioned

statements

were

the

extremes in attitude to pronunciation learning, our aim should be somewhere in between, which means that we should aim to reduce the time to tune in for the listener as well as to put down the strain for the speaker in order to make the conversation comfortable for both sides. 1. 4 affect What affects pronunciation learning Like in other subjects, there are many factors which pronunciation learning, but some of them are specific to this language section as well as to learning foreign language as a whole. It involves both learning as an organized and intentional process, and the unintentional language acquisition, however, which is proceeds not less subconsciously beneficial. and intuitively, Both,

learning and the language acquisition, are dependable on the inner and outer factors, which the teachers should be aware of when setting out goals and preparing methods and materials. The factors which cannot be influenced neither by the learner, nor his or her surroundings are the native language, the age and to a certain extent the phonetic ability of the learner. In contrast, the amount of exposure to English, the attitude and motivation may be supported and changed a lot either by the teacher, or the learner, but usually in cooperation and support from both sides. 1. 4. 1 The native language

The influence of the native language is unquestioned. It involves the individual sounds as well as combinations of sounds and features such as rhythm and intonation. The more differences there are between the native and the target language, the more difficulties will be encountered by the learner. The learners errors derive from various 10

sources. When a particular sound does not exist in the mother tongue, the learners tend to substitute it by the nearest equivalent from their native language. For example, the sound [] will often be substituted by [d] or [z] as these are the nearest. and This the first language too. application Still, the affects the rhythm intonation

influence of the native language is only one of the factors and does not need to be crucial. 1. 4. 2 The age

It is commonly assumed that the age factor has a great influence on pronunciation. If someone has a native-like pronunciation in a second language, they are very likely to have started learning it as children. And conversely, people who started to learn a second language in adulthood, will scarcely achieve a native-like accent, although their grammar and vocabulary may be perfect. However, the results of the studies on the age-relation issue which have been carried out are conflicting. One should take into account that it is very difficult to control other factors like motivation, ability, opportunity to hear and use the language and the learners attitude, therefore the results are bound to be distorted to a certain extent. Don Snow (1992) comments on this issue in a contradictory way: Research comparing children that to adults has and

consistently

demonstrated

adolescents

adults perform better than young children under controlled conditions (e.g., Snow & HoefnagelHoehle, 1978). One exception is pronunciation, although even here some studies show better results for older learners. (p. 129)

11

The

idea

of

pronunciation

learning

being

somehow

dependent on the age has been supported by some researchers who claim that language learning has a sensitive period and that after a certain age people lose some abilities. This certain age is said to be between ten and thirteen years. (Kenworthy, 1987, p. 6)

The

age-relation

is

supported also by others:

Pronunciation is one area where the younger-isbetter learner assumption begins a may have validity. language, Research the more (e.g., Oyama, 1976) has found that the earlier a second native-like the accent he or she develops. (Snow, 1992, p. 129) From the teachers point of view,

we assume, as the or the and older

that for they adult silly.

learning pronunciation the age of primary and level pupils is more convenient,

lower-secondary

are to In

less do

shy may their

and find we

often which strange take

more

willing

activities can

learners of

children,

advantage often not

spontaneity in

willingness to sing and chant, which is appropriate learners. Research

suggests that older students will

show quicker gains, though younger children may have an

12

advantage in pronunciation. (Snow, 1992, p. 129) However,

when the adult learners are motivated and want to approach native-like pronunciation, they are able to focus better on their problematic areas and this fixed-target acquisition may be also very efficient.
1. 4. 3 Phonetic ability

It is generally assumed that some people have a better ability to hear foreign coding languages ability who than or others. This aptitude for oral mimicry, phonetic auditory proved tests Although people by which the can discriminating researchers measure results this proved ability designed that some ability.

better discriminate between two sounds, still it is not a crucial condition of obtaining training good itself pronunciation has a as significant

effect. (Kenworthy, 1987, p. 6)

13

The

point

is,

whether

different

types

of

learners

should be taught with different methods. Kenworthy (1987) divides these learners into poor discriminators and good discriminators and explains: One study has indicated in which that those with sounds them good are to

phonetic drills, again. exploit

abilities tasks

benefit

from

pronunciation

particular

heard and the learner has to imitate again and Their all innate the abilities enable to opportunities compare what

they are doing with the model presented. Poor discriminators

do drills

not seem to an

seem very to

to

benefit In their before of this the

from

drills

much. cause

fact,

attempts they of a a

stabilize accurate Because is involved, which

reach

production seems

sound.

complexities factor

very much out of the control of the teacher. We can only operate on the assumption have the that our learners basic

equipment and provide a variety of tasks so that something will

14

suit the needs and abilities of each learner. (p. 7)

1.

4. 4
meanings

Exposure to the language

and intensity. The meanings may vary from to

The exposure to the target language has a wide range of living in the country where the language is spoken

talking to native-speakers or using authentic materials, e. g. films, literature, music, television, so the term is quite wide in its meaning. Living in the country of the target language does not always mean the actual use of the language. Many people may live in a non-English-speaking environment, or use their mother tongue with their families. Conversely, many learners who live in a nonEnglish speaking country may use English in many daily situations like school or work, so the amount of exposure basically depends on its quality and quantity. Logically, being exposed to English in the English environment should have a positive effect on

the

learners skills.

pronunciation Research,
however,

and

listening
that this

indicates

increased exposure to English does not necessarily speed the acquisition of English. (Snow, 1992, p. 129) It is considered to be a contributory factor, but is not the most important. 1.

4. 5
In the

Attitude
acquisition of

and

sense
of a

of
foreign

identity
pronunciation language the attitude and sense of identity of the learner 15

plays a major role. It is a personality-determined factor and we can hardly predict whether people having come to the environment with different accent will modify their own.

Some seem to be impervious and even after a long time will absorb only some turns of phrase and the pronunciation of a few individual words; others seem very their
Using

receptive accent
or not

and

begin as
accent

to

change as they

almost
using the

soon
may

step off the plane.

(Kenworthy, 1987, p. 7) show different

attitudes to people. When imitating or adopting the accent of the host country, we show

our

positive

relation to their language and culture as well as respect for them as people. It is sign of how much we would like to integrate ourselves in the new place.
As far as teaching pronunciation is concerned, the attitude and sense of identity factor is

closely

connected next.
1.

with

the

inner

motivation

factor which is going to be explained

4. 6

Motivation
16

For

some

learners

pronunciation

is to

more be an be

important than for others. The learners who consider are pronunciation of their eager usually important learning part English to

corrected and concerned about how good or bad their pronunciation is. This can be also expressed by unwillingness to talk when being uncertain about ones pronunciation: If we cannot say it perfectly, we wont say it at all. In these learners the teacher can perform magic, because the learner the wants to cooperate and improve. To maintain this motivation from learners perspective, the teacher should set out goals and give the learner feedback, so that both are aware of some kind of progress. Conversely, the other extreme is, when the learner is not aware that their way of speaking irritation causes or difficulty,

misunderstanding for the listener. The

17

teachers factor of

possibilities motivation

in

motivating not be

learners are limited, nevertheless, the should neglected as it is considered to be one of the most important factors affecting pronunciation learning.
2

Giving feedback on pronunciation Teaching a language or is inevitably involves testing, offering

feedback they on are

tied either

with

feedback

which and
giving on

formal

informal

assessing
As we have

achievements,
already with mentioned, information

suggestions for making improvements.

pronunciation is essential for maintaining motivation and providing students how

progressing and what they should focus on. Compared

to

learning grammar or vocabulary, where students are able to assess themselves objectively having the correct answers at their disposal, self-evaluation in pronunciation is complicated since it is very often distorted by the students own ear and phonetic ability.
18

The

importance

of

teachers on
point task

role

in

giving
also seems from to be

feedback
the a teachers difficult

this
of at

language
Yet, assessing in the much

component is thus

unassailable and crucial. view, least,

pronunciation, in comparison with other language components as, literature negligence which we is the is a have had fact skill at our disposal not

attention is paid to this issue. The main reason for this that which speaking, is too comprising complex

pronunciation,

to

enable

any

realiable

analysis

which

could be considered to be objective.

19

2. 1

Assessment methods Learning pronunciation is based on both production and reception/ identification, therefore, it is closely connected with oral exams and listening tests. Both speaking and listening activities may be used for testing and assessing our learners progress, although they do not have to provide us with comparable and agreeable results. Contrary to production, reception can be tested and quantified by counting the correct answers in a written test, whereas production is more difficult to assess as it involves testing speaking or reading which, from the listeners point of view, include many other aspects of communication, not pure sounds. It is always subjective to a large extent and so the problematic matter of assessing pronunciation production is its reliability. We are bound to rely on the impression of the assessor. Therefore, both the procedure and the assessment should be defined as concretely as possible. Depending on the aim of our assessment, two approaches of testing pronunciation are going to be described atomistic and holistic. Before dealing with these, we look at the most common concept of pronunciation testing.

2. 2. 1

Impression-based pronunciation testing

Taking into account our school environment, teaching and technical conditions in our classes, testing based on the teachers impression rather than using any objective scientific methods is inevitable. Nevertheless, we should be aware of the possible difficulties. According to SzpyraKozlowska et al. (2005), the impression-based pronunciation testing has many drawbacks. She mentions that different assessors may have different criteria of evaluation and it is also more general and therefore can be imprecise. In 20

opposition to such an opinion, Kenworthy (1987, p. 20) believes that also assessments which are impressionistic and therefore subjective can be both dependable and accurate. What She claims speaks in that the favour impression-based rankings of the impression-based

tend to agree with other objective ranking techniques. pronunciation testing, is the fact that it is typical of many international examinations in English, like Cambridge English Examinations, where the assessors evaluate "intelligibility and the amount of strain a candidate's pronunciation puts on the listener. (Szpyra-Kozlowska et al., 2005) For First example, is the in easily requirements English are understood, on L1 pronunciation features may in be Certificate defined: "although

pronunciation

intrusive"1, in Certificate in Advanced English, the same is defined as follows: "L1 accent may be evident but does not affect the clarity of the message"2 and in Certificate of Proficiency in English: "pronunciation is easily understood and prosodic features are used effectively; many features, including pausing and hesitation, are "nativelike"."3 Obviously, the definitions of the requirements in Cambridge English Examinations are very general and impressionistic in nature. The examiners who are very often non-native speakers of English are instructed as follows: "when assessing pronunciation, examiners should try to put themselves in the position of a non-EFL specialist, native speaker of English and assess the amount of strain on the listener and the degree of patience an effort required to understand the candidate. (as cited in Szpyra-Kozlowska et al., 2005)
1

Paper 3: Speaking. Cambridge Common Scale for Speaking [online; quoted 25. 1. 2007] http://www.cambridgeesol.org/support/dloads/ket/KET_HB_sampleS.pdf 2 ibid 3 ibid

21

On the basis of this instruction, the assessment may have more interperetations depending on the assessor and is bound to be always subjective to some extent, but we can conclude that it is a method which can provide us with information about the progress and achievement in quite a plausible way and it enables us to compare the learners performances within time or class. 2.

2. 2

Atomistic testing

Atomistic approach requires a detailed marking scheme in which

specific aspects of pronunciation separately. word lists short or pairs It means on also and aloud based it

are

evaluated

reading phonemic enables

oppositions, minimal or testing

sentences sentence

containing

appropriate sentence

accentuation

stress

intonation. This approach is claimed to be more objective than the holistic one as it judges only segments of speech particular vowels, consonants, stress, rhythm, intonation, etc. Nevertheless, the drawbacks of this approach are the demands on the assessor in our school environment the teacher. It requires recording the learners speech samples
22

and repeated listening to them, so it is extremely time-consuming and thus unsuitable for large classes.
3. 2. 2. 1 Methods of testing

Atomistic testing may apply to both production and identification of sounds, stress patterns and intonation. It is advisable for the teacher to choose a good material and set out clear criteria beforehand. Atomistic testing focuses always on a segment of speech, therefore we often have to omit those wrongly pronounced segments of language, which have not been aimed to assess. This is the only way both to compare our learners performances, and to avoid any confusion from the overload of information. Repetition On the beginner level, the easiest test to prepare is repetition exercise. It is useful for learners who cannot read or who are beginning with English. It is based on hearing sounds, stress and intonation, and imitation which gives the teacher checking the gist of learners items potential than and all phonetic ability. The test may consist of single words or sentences particular rather pronunciation aspects at the same time in order to be as much objective as possible. Hearing identification Another way of testing beginners as well as more advanced learners is hearing identification (Madsen, 1983, p. 61). Good pronunciation is dependent on our ability to hear the language. It can consist of recognizing sounds in minimal pairs, the fall or rise in intonation or indentifying stress in words or sentences. 23

Reading aloud Commonly reading aloud. used way of to pronunciation Madsen (1983) assessment three is According points

should be kept in mind: (1) When using lists of sentences, evaluate only one or two points per sentence; (2) use natural language; (3) avoid signalling to the student which pronunciation point you are testing. (p. 66) Since reading tends to be longer and involve many points to assess at the same time, it is advisable to record the learners performances in order to listen to them repeatedly and have the possibility to compare. The material to read should enable natural sound, e. g. a letter, instructions etc., and students should have time to read the text silently before reading for assessment to get the context. and The reading to aloud test testing almost provides aspects good of control enables all

pronunciation including stress and intonation as well as vowels and consonants. Nevertheless, we have to count on the fact that reading and speaking skills are not the same and, inevitably, is the intonation not as and sentence as rhythm in in reading usually natural normal

conversation.

2. 2. 3 A

Holistic testing higher level and of achievement of is the testing learners acceptability

intelligibility performance.

In this

holistic approach

to pronunciation 24

testing examiners are asked not to pay too much attention to any one aspect of a candidates performance, but rather to judge its overall effectiveness. (Alderson, 1996, p. 289) The advantage of this procedure is that it can be administered to large groups and is not as time-demanding as the atomistic approach. As it has been already mentioned this approach is used many international exams in English, where the pronunciation and is involved of in the so called intelligibility acceptability candidates

speaking performance. The definition of intelligibility is very general: Intelligibility is being understood by a listener at a given time in a given situation. So its the same as understandability. (Kenworthy, 1987, p. 13) To put intelligibility more concretely, we can say that: The more words a listener is able to identify accurately when said by a particular speaker, the more intelligible that speaker is. (Kenworthy, 1987, p. 13) In practice, words are made up of sounds and if the speaker substitutes one sound for another and the result is that the listener hears a different word, it means that the speaker is unintelligible or not intelligible comfortably. However, when the wrong substitution occurs in a speech, but the word is anyhow understood, then we can say that the speech is intelligible. Considering this should be the base for setting out the aims of our pronunciation lessons, which does not necessarily need to be native-like speaking performance. According to Kenworthy (1987; p. 14), as intelligibility is very complex, there are many factors influencing it. One of them is the counts of sameness. This applies to the Czech practice for example in the sound [] which is often substituted for [d] as it is the nearest equivalent in the Czech language. But, when the listener is 25

aware

of

this

and The

the

incorrect factors

substitution affecting

does

not

produce another word, he or she will not have difficulties understanding. other learners intelligibility are, for example, the over-use of selfcorrections, hesitations or grammatical restructurings. All previously mentioned factors may be annoying or disruptive for understanding as well as for the whole communication. Another speaker factor which can cause problems is that the person speaks too fast. According to Kenworthy (1987, p. 14) it is usually like not the speed itself which causes difficulties, speaking, but incorrectly sentence applied other rhythm, and features of intonation

stress,

which makes it difficult for the listener to pick out the most important bits of the message. So far, all the above mentioned factors were speakerfocused. But we have to be aware of the fact that there are always mentions the at least two two participants on the listener to use in a communication. first, clues the when Intelligibility also depends important ability listener. Kenworthy

factors: contextual

listeners familiarity with the foreign accent and, second, listeners listening. (Kenworthy, 1987, p. 14) To conclude, understanding is dependent on the amount of exposure and being used to the foreign accent. A teacher should be aware of this fact. Being exposed to the learners pronunciation, the teachers assessment is very likely to be distorted. The issue of intelligibility is very complex and is a major part of communication. Therefore, the goal is not only the correct production of sounds, stress patterns and intonation, pronunciation intelligibility. 26 but efficiency can be of communication as without irritation and difficulties understanding. So the goal of defined comfortable

2. 2. 3. 1

Methods of testing

As we have already mentioned, the main criterion for holistic testing is the efficiency of communication between two people. Therefore, should be the used best in method is interactive of the testing including more than only one participant. All the activities the interaction assessor or another student to involve both sides of the communication the speaker and the listener to function as an oral interview including natural situations and asking questions. Re-telling stories This kind of test involves first reading a story silently and then telling the story using ones own words and sentence structures. The assessor may interefere giving further questions. Description of pictures Pictures people or may be or used for for description of of two objects, similar scenes, comparison

pictures, in which the learner looks for similarities and differences. Sequence of pictures This test is based on telling a story involving linking words expressing the cause and the result. It can be applied to only one student or a pair where each of them is given one half of the pictures and they should decide on the correct sequence of the story. Pictures with speech bubbles

27

In this test students are required to guess what the people in the pictures are saying. It may be used individually or in pairs. Using maps Many students books involve a unit dealing with giving directions. This activity is to be done in pairs, where one gives the directions and the other one follows them. 2. 2. 4 Atomistic versus holistic approach

When deciding between atomistic and holistic testing, the purpose of testing should be considered. As far as reliability is concerned, in Szpyra-Kozlowska (2005) Hughes (1991) says that atomistic tests are more reliable for diagnostic purposes in the language classroom and in cases in which scoring is carried out by different assessors, whereas holistic approach is faster and more appropriate for experienced assessors.

28

Practical part Introduction In this practical part we are going to deal with the

3. 1

practical aspects of pronunciation testing and assessment in class. On the basis of the theoretical part, a qualitative research has been carried out to test students pronunciation and their progress within a period of time. The idea of assessing students pronunciation arose out of its teaching. In my English lessons we did pronunciation exercises but the process was somehow not complete. The feedback for the learners as well as for me as a teacher was missing apart from me informally responding to the individual performances. The conception of pronunciation in our coursebooks did not enable me more than assessing the learners immediate performance. Before starting our research, the students had been taught pronunciation but had not been provided with any statement on if they were progressing or not, or just a list of points what they led should us to focus try on. to So the need a of a structured feedback and the lack of instructions in our coursebooks design complete pronunciation course involving all the necessary steps. Having studied literature dealing with the issue of pronunciation tests, I found the instructions scattered in many books, always being presented just in bits and pieces. Therefore, I tried to put them together to be used in my lessons in order to see to what extent they are beneficial for the learners time and and achievable demands on as the far as their are reliability, concerned. The research itself and its results will be presented as the last item of this part and will be preceded by three assessors

29

parts generally relating to the three stages used in the research. The first part is the evocation stage. It deals with the factors affecting been described pronunciation learning in the theoretical which have to be already part,

precise, it deals only with such factors which the teacher or the learner have some influence on. Further, we present activities and recommendations which may be used in the classroom. The second part is called the realisation stage. This stage involves and phase the its and the focus of realisation We on omit the of the of teaching teaching testing towards pronunciation pronunciation proceeding testing.

ways

from

level

sounds

production

testing intelligibility and considering their weaknesses and suitability. The third part is the reflexion stage. It presents the description of criterial levels for atomistic and holistic assessing and the requirements on the choice of the assessors. 3. 2 Evocation Setting out goals and criteria of assessment as well as teaching pronunciation itself requires considering all aspects affecting pronunciation and its learning. Having been generally explained in the teoretical part, some of these aspects of are the given and cannot be changed Among by any the participant learning process. these

following are listed: the age, the native language and the phonetic ability of the learner. Since we cannot influence them, it is essential that we take them into account when planning a course. The factors which the teachers or the learners have some, however, limited control of, are the exposure to English, the attitude and the motivation. 30

Nevertheless, the exposure to English during the course of learning it, is somehow given depending on the English or non-English environment, but we can still advise our students on other possibilities which can improve their English outside the class, e. g. help them use authentic materials like film, newspaper or find a native speaker to communicate with. The teachers control is quite limited, because we cannot influence whether our learners really do so. But we can contribute by using the right motivation methods, either self-made, or inspired by those presented in literature. In the theoretical part an opinion was expressed that motivation higher belongs of to the crucial p. and factors 57) claims usually affecting that has "a a pronunciation. level Kenworthy (1987,

awareness

concern

positive effect on motivation and therefore achievement." For this reason the stage of building awareness should not be neglected. What These may be be particularly used in such helpful are the in General the awareness-building activities (Kenworthy, 1987, p. 54). should situations which teacher encounters any negative attitute from the learners side towards the pronunciation learning. Kenworthy mentions two of such activities: Questionnaire based discussion and A tape-based of the activity. The learners spoken first is with a based on a questionnaire, the second centres on samples of the native languages heavy foreign accent. Both aim to help learners develop a concern for the pronunciation of English through their personal experiences of language in use. (ibid) The descriptions of the activities have been inspired by various resources and the instructions are only suggested and open to any adjustments.

31

3. 2. 1 Activity 1 Aim: Increasing interest pronunciation Resource: Questionnaire,

and

motivation, and

discussion

on in

questions

instructions

Kenworthys Teaching English Pronunciation, Dalton, Ch., Seidlhofer, B. Pronunciation (1994, p. 155) and Ur, P. A Course in Language Teaching (1997, p. 51) Time: at least 30 minutes Language used: mother tongue is acceptable for meaningful discussion Procedure: 1 Filling in the questionnaires individually; 2 Comparing the answers in pairs or groups; 3 Creating statements and questions based on the pair/group discussion; 4 Whole-class discussion of the prepared statements and questions. Output: Teacher's evaluation: - How involved were the students in the discussion? - Did agreement or disagreement predominate, and what do you think are the reasons? - Were there questions which did not lead to an interesting discussion? If so, why? - Are there questions which you would like to add?

32

Activity 1 - Questionnaire4
1 Imagine you are talking in your own language with a foreigner. The person doesnt speak your language very well and is very difficult to understand. What do you do? Do you: (a) pretend you understand even when you dont? (b) ask him or her to repeat everything slowly and carefully? (c) try to get away? 2 What do you say when the foreign speaker apologizes for his poor accent? Do you: (a) tell him his accent is very good even when it isnt? (b) tell him that his poor accent doesnt matter? (c) tell him that his accent is very bad and that he must work hard to improve it? 3 How do you feel when a foreigner pronounces your name wrong? (a) very angry (b) it bothers me a little (c) it bothers me a lot (d) it doesnt bother me at all 4 How do you feel when you meet a foreigner who speaks your language with a very good accent? (a) surprised (b) pleased (c) not surprised (d) full of admiration (e) dont care or think about it 5 In the future, who will you speak English to? (a) mostly English5 people visiting my country who dont know my language (b) mostly English-speaking people in this country (Britain, USA, etc.) (c) mostly non-English6 people who dont know my language and whose language I dont know, so that we speak English together (d) dont know 6 Do you think it is more important to have good pronunciation when: (a) you are speaking English to English people? (b) you are speaking English to non-English people? 4 presented in Kenworthy (1987, p. 54) 5 We would suggest using English-speaking people for all English native-speakers rather than English people which we consider to stand only for the people from England. 6 Similarly, we would suggest using non-English speaking people for the same reason as in the previous footnote..

33

To sum up, on the basis of this activity, the teacher should receive information on the students attitude towards learning pronunciation which is helpful to design the course according their own needs and goals. 3. 2. 2 activity". It Activity 2 consists of a recording of a strongly

Another awareness building activity is "a tape-based foreign-accented speech and following activities based on either discussion of given questions, or also a table to be filled in preceding further discussion. As an example, Penny Ur (1991, p. 51) presents a box for the students to work with:

WORKSHEET: RECORDINGS OF FOREIGN PRONUNCIATION Speakers mother tongue: Words/phrases mispronounced Define or describe the mistake

Aim:

Increasing

interest

and

motivation,

stimulate

discussion of the implications for communication and social interaction of heavily foreign-accented speech. 34

Resource: Kenworthy (1987, p. 55), Ur, P. (1991, p. 51) Time: at least 45 minutes Language used: Mother tongue is acceptable for meaningful discussion Procedure: 1 Introduce the task asking: "I'd like to know what you think about this speaker?" 2 Play the tape (2 - 3 minutes) of foreign accents (people who are not very proficient in the target language) and let students fill in the box or write down notes concerning the aspects which make the people's performances sound foreign; 3 Play the tape as many times as needed; 4 Let the students compare their notes in pairs or groups; 5 Let them discuss the answers to the following questions: 1. What seem to be the most common errors? 2. How would you describe this person's accent? 3. Would you like to have a conversation with this speaker? 4. (If we have a tape of a foreigner speaking Czech) Do you dislike hearing your language spoken in this way? 5. Why do you think the person pronounces so badly/in this way? (a) The person doesn't really care about pronunciation. (b) The person hasn't been told how bad their accent is. (c) It's very difficult to pronounce this language well. (d) The person hasn't been told about pronunciation. 6 Whole-class discussion

35

Output: Identifying words which are very important and need to be pronounced well. The teacher gets a general idea of how responsive the learners are likely to be to pronunciation work, both as a group and as individuals. An equally useful component of the "evocation stage" is eliciting information from the learners learning about their previous pronunciation and English experience.

Which questions we prepare depends on our needs, but we should know if they want to improve their pronunciation, what they think their weak and strong points are and how much they want to improve. Getting included mentioned in this one kind of or the done of information in may the be form either of a "building-awareness separately techniques"

above,

questionnaire or an informal discussion. It is beneficial for the learners themselves as well as for the teachers. The task makes the learners consider and define their individual goals which itself has a motivating effect. The evocation stage should provide us with the input information for the realization stage including setting out goals and criteria on the of further of testing to and be assessing, devoted to deciding amount time

pronunciation and the form of a test. 3. 3 Realisation The main part of the realization stage may involve teaching pronunciation and carrying out a particular test. As this work we does not focus to on teaching various pronunciation techniques of itself, are going present

testing. All we can find in literature are just recommended activities which may be used for testing rather than any strict rules and exact instructions, so the teacher is free in their choice and may adjust any material from literature 36

according to their own individual needs or create some themselves. Before designing a test it is important to consider the aim. Like in any other language component, there are two kinds of tests. The first one is a progress test, which monitors the students progress. The second one a proficiency test - deals with the learners general level of a language skill without reference to any course, which is presented in the international exams. In the practice of pronunciation testing this division between the two kinds of tests infers the division between atomistic and holistic testing although it does not have to obtain in all cases. But the progress test is usually based on what has been taught during the course and in most coursebooks the pronunciation teaching course design has some structure and deals with the aspects of pronunciation in successive steps. So it is obvious that the test will only include the points which have been covered. The proficiency test does not take into account what has been taught and is designed only according to the general level of the learners all English. of It examines intelligibility where the main including aspects pronunciation

criterion is comfortable communication between, the speaker and the listener. Another way of dividing tests is whether they test reception or production. Reception deals with hearing identification atomistic of sounds, and stress and intonation in in the the testing listening comprehension

holistic one. The advantage of the reception testing is that the result can be quantified simply by counting the correct examining quantified answers. amount but is That in is also it the cannot reason More be on for their is and sufficient course since books. difficult measured the

production

entirely

dependent

assessors 37

impression. We will deal with the tests on production and point out the rules which should be followed in order to receive as reliable results as possible. We will now look at the activities which may be used for testing. They have already been briefly presented in the theoretical part from the point of view of the holistic and atomistic way of assessing. In this part we are going to introduce them more thoroughly and focus more on their advantages and disadvantages in connection with their particular purposes. 3. 3. 1 Limited-response activities

In limited-response activities the teacher has total control of the learners utterance. These activities are usually easier to prepare as well as assess and their results are comparable with other learners. Test 1 - Words in isolation This test is aimed at examine the learners ability to distinguish between sounds. It is advisable to start pronunciation learning based on minimal pairs and correct production of sounds. This is just an example which may focus on a limited number of sounds. Students read a list of words containing minimal pairs. The assessment form shows which sound is to be assessed and the assessor puts a tick or a cross next to each word. The test can be done in one lesson in class, or, what is more reliable, we can record the learners performances and let them be assessed by more people or by someone else, not the teacher of the class who has become familiar with the students accents. Assessment form7 thing

think
7

Inspired by URs publication A Language Teaching (1997).

38

pen sick ran vest Test 2 - Pictures

pan thick rang west

For pupils who cannot read, we can use pictures for identifying objects, where there in each picture there is a possible source of confusion. For example, a picture of a pen can cause a confusion with the word pan. Which pictures we choose is dependent on the words which our learners know. Since of reading the is not involved, is the advantageous spelling feature students. Test 3 - Words in sentences8 Reading sounds is aloud sentences because containing they the problematic context and preferable provide activity avoiding any

pronunciations and thus its suitability for also dyslexic

reading will be more natural. There were several people standing in the

hole/hall. Are you going to sail/sell your boat today? Do you like this sport/spot? Test 4 - Reading Aloud Reading examining aloud is one On of the most common it ways of pronunciation. the one hand, provides

control, on the other hand there is still a big difference between reading and normal conversation, so we have to count on the fact that it is not spontaneous. Learners are often confused by the spelling and as they see pauses between words they find it difficult to link, so we should
8

Inspired by HEATONs Classroom Testing (1990).

39

be aware that in real conversation the performance would be better. Nevertheless this kind of test may give us an approximate level of the learners pronunciation. It is important to choose the right material to be read. The best is to prepare something that may actually be read aloud. It can be a letter, instructions or a dialogue. The following example is a short letter or a message. The student will read only the left side, but the assessor may get also the points which should be assessed. It is better to divide the text into more lines and assess only one or two points per line, because it is impossible to notice everything and give such a thorough feedback when there are many students in the class. The result of such a kind of assessment would be too chaotic, because in case of more assessors, do each not of them will us to point out and omit something else and the results received by this kind of assessment enable compare students performances. Text Its my girlfriends birthday next week and I want to give her a surprise with a special present or a special evening somewhere. What should I do? Examiners notes g:l,frend; b:de wi:k; wi ; or ; i:vn; falling intonation

The above mentioned activities were based on a limited response. Considering their advantages, they are easy to prepare and their we assessment enables are going to comparison between free-response students, as the text is the same for everyone. Further, present activities in which there is not one correct answer but the student is free in the choice of his or her answer.

40

3. 3. 2 Free-response activities Free-response activities do not test

pronunciation

during reading, but the students own utterance. Student is free to choose a formulation for his or her answer. On the one hand, the advantage of this method compared to limited response students other and is avoiding spelling pronunciations more natural. and On the the that speech generally should take reason a sounds

hand, we for this

into account of

the fact is

nervousness of a learner may affect his or her performance lot practice required beforehand. Test 5 - Re-telling stories As the title tells us, this test involves re-telling a story or an incident. A student reads the text silently and then is supposed to re-tell it in their own words. If the text is chosen appropriately, the student should use such elements of pronunciation which we want to examine. The reading comprehension should not affect the result in case of inaccuracy on the summary. Test 6 - Using pictures Pictures are useful for testing speaking skills because their description does not involve any reading, which could affect the performance. There are various ways how we can use them. For a description students should have time to see them beforehand for a few minutes to prepare and order their thoughts. The international Cambridge examinations include pictures for comparison in their speaking activity. The purpose of this activity is to find out the differences or similarities and their common topic. This activity can be done comfortably as a part of a normal lesson better than a 41

part

of

test

and

it

also

enables

more

than

one

participants to work together. Another variation of using pictures is the sequences of pictures which is similar to telling a story. Students are to look for the links between the pictures rather than describe pictures in isolation. For either individual or pair work there are pictures with speech bubbles. In this activity students are supposed to guess what the people are saying. Almost every coursebook includes a giving directions task and maps can be used for this purpose as well. 3. 3. 2. 3 Testing intelligibility

So far, we have mentioned two levels of pronunciation testing. The first dealt with limited-response activities which as are suitable as in for testing all The type aspects second for is of type pronunciation, but the use of stress and intonation is not natural us and as normal conversation. more The the third main provided stress well with activities suitable testing testing is not the so

intonation.

pronunciation in interaction. This involves production as comprehension of the and criterion It is intelligibility learners speech.

detail-focused as the previous two but involves all aspects of pronunciation segmental and suprasegmental. The activities are mostly interview-like, either it is the interview with the examiner, or with the peers. For this technique, pictures, role-plays or discussion may be used, but so that the speakers performance is as spontaneous as possible, it is importnat to choose a topic which the learner is familiar with. In testing speaking, pronunciation is just one of several aspects which are evaluated and which make the speech comprehensible. So, apart from accent, there is 42

grammar, vocabulary, fluency and comprehension which make the picture of the students performance complete. This assessment involved, kind is of testing out is As demanding there defined are as far many as is the very concerned. aspects

setting

clearly

criteria

important, moreover, one assessor only is not enough for achieving relatively reliable results. But this technique enables the learner to show their best in pronunciation and so the results could be considered to be the most valid and reliable. 3. 4 Reflexion

Having presented variations of pronunciation tests, we are coming to their final part which is the feedback. This part will deal is with a assessment and the task because of it the is assessor. As has already been mentioned, giving feedback on pronunciation very difficult task impression-based. Another stumbling block is the complexity of this language component, so what should be taken in mind first is the aim of a particular test and on its basis setting out concrete criteria, otherwise the assessor will get overloaded with much information at the same time. Assessing the limited-response tests is probably the easiest. As it focuses on a limited number of items, we may just tick or cross the correct or wrong production of sounds. Free-response tests are more difficult to assess and so it is useful to record the learners performances and listen to them more times. In the atomistic assessment there will be only a few criteria, because it is very timeconsuming to cover everything. In the holistic assessment the criteria are differently defined as they include both speaking and listening. Description of criterial levels 43

Setting out criteria of assessment should refer to the design of the test. errors and Learners when to are likely than a to when make more pronunciation self-conscious Hughes reading speak the on speaking so it is of

spontaneously, but there are many students who would become shy topic, recommended to use both methods. (1989) describes criterial levels obtaining valid and reliable scoring as follows: Scoring will be valid and reliable only if: Clearly recognisable and appropriate descriptions of criterial levels are written and scorers are trained to use them. Irrelevant features of performance are ignored. There is more than one scorer for each performance. (p. 110) The first point to consider before setting out other criteria is the standard according to which we are going to judge the learners performances. Is it the native-speaker standard? It could goal be, but we consider as it to is be a discouraging for our learners this almost

impossible for them to achieve. Moreover, what we intend to suggest, are the possible ways of testing and assessing pronunciation prevalence speakers, teachers criteria comparison of to of so in it give with our might our environment teachers seem learners should be of quite which English daring in so entails over of the that the that it the Czech way of the is Czech native-

feedback This set out

native-speaker.

implies

marking

possible for the learners to obtain the highest score which can correspond with comfortable intelligibility.

44

From the didactic point of view, the progress should be appreciated more than the achievement itself, as hard work and willingness to improve is more important in life than having just a good starting level i. e. good phonetic ability. As far as the approach is concerned, the descriptions may be atomistic or holistic. In order to reach reliable scoring we may use both methods as a check. The atomistic approach usually checks only a limited number of items, e.g. production of particular sounds, stress, linking and the result is either correct, or wrong, whereas the holistic focuses on the quality rather than the total of correct answers. In the theoretical part, we have already mentioned the scale in Cambridge examinations and now we will look at another example the American Foreign Service Institute interview procedure. Two testers are required to assess the candidates holistically and rate them on a sixpoint scale for each of the following: accent, grammar, vocabulary, fluency and comprehension. Both results then should agree and as Hughes (1989) mentions there is a high level of agreement between holistic and atomistic scoring. As an example, we present the six-point scale for accent: 1. Pronunciation frequently unintelligible. 2. Frequent gross errors and a very heavy accent make understanding difficult, require frequent repetition. 3. Foreign accent requires concentrated listening, and mispronunciations lead to occasional misunderstanding and apparent errors in grammar and vocabulary. 4. Marked foreign accent and occasional mispronunciations which do not interfere with understanding. 45

5. No conspicuous mispronunciations, but would not be taken for a native speaker. 6. Native pronunciation, with no trace of foreign accent. (Hughes, 1989, p. 111) We may notice that the criteria above refer mainly to how difficult the speaker is not only to be understood, but also to listen to. It is not an easy task to classify a performance into one of the groups and what makes this kind of assessment even more demanding is the number of assessors needed. But the issue of the scorers is going to be tackled later on. Irrelevant features of performance In assessing pronunciation it is important to focus only on such aspects or items of pronunciation which have been decided to be tested, otherwise the results will not be reliable, but chaotic. This is important to be stressed particularly in pronunciation testing, as it is difficult to ignore other errors in a performance. An example has already been mentioned in Test 5 Re-telling stories where the attention should be paid to pronunciation, not to wrong comprehension. And the same applies to the test of distinction between the sounds: if [] and [] is tested, the result should not be affected by wrong stress, for example. The scorer Unlike assessing other language components, assessing pronunciation has its unique requirements and demands on the scorers. The first problem is, whether the class teacher is the best assessor. No publication or article being referred to in this work took the view that the class teacher is 46

suitable for assessing their learners. It is difficult for the teacher to fulfil this requirement, but still, there are some possibilities. We believe that, if necessary, the class teacher is able to assess according to the atomistic criteria, meaning that they are able to distinguish betweent the correct and wrong production of sounds, stress and intonation. For that reason, if necessary, the class teacher may be helpful. In order to get more reliable results it is better to ask a colleague from the school to assess for checking. The holistic assessment is more complicated. Here, one teacher is never reliable, if two have different opinions, the result is also pointless, because we can count only the points of agreement between both assessors and the rest does not give us any information which implies that three assessors are needed to provide some level of reliability. To make the method even more difficult, Kenworthy (1987, p. 20) claims that all English teachers themselves are not suitable judges, as they have a great exposure to non-native accents and so they have developed special skills as listeners which make them atypical listeners and so unsuitable. The ideal judges are listeners who have not had an abnormal amount of exposure to non-native speech nor any previous contact with the speakers being assessed. [] Judgements by teachers of English are of limited value. (Kenworthy, 1987, p. 20). At least the good news is according to Kenworthy that non-native listeners can be used as judges, e. g. other learners of English in the class. We think that this requirement is very difficult to fulfil in our school environment, so we should at least stick to the rule of three scorers.

47

Research part On the basis of the theoretical part a qualitative

research has been carried out in order to consider the importance of pronunciation testing in EFL classes. Our research tried to implement the three previously mentioned stages: evocation, realisation and reflexion into practice. As theory often differs from practice, we were not able to fulfil of the all the requirements, part that and any are rules methods open to and and any recommendations, but followed the opinion expressed at the beginning techniques practical are just recommended

necessary adjustments in order to be administered according to our individual needs and conditions which were limited. Our limitations lay in the time possibilities and the number of students at our disposal. The whole procedure was carried out over a span of four months including all the mentioned stages. The whole procedure consisted of several parts. The first part dealt with receiving and analysing the input information using the building-awareness activity, questionnaires and pre-research. The second part involved the research itself including the first and the second recording, pronunciation practice, and subsequent analysis of the results. The third part dealt with assessment and analysing the results of the three assessors. There were sixteen students involved in the research so we are aware of the fact that such a limited number does not allow us to fully categorize our findings as far as the progress is concerned. For the assessment there were three assessors nonnative teachers of English. One can argue that the choice of non-native English teachers, who may be familiar with 48

foreign accent as they have a large exposure to it in their own classes, will lead to unreliable results. There is a point in this so argument, our way our however, of testing for will assessing hardly in be our much should English classes conditions

different,

pronunciation

correspond to the possibilities in our classes. Our research will analyse two aspects of our testing. The first is the feedback on the two performances of each student including the results of the atomistic and holistic testing. The second aspect to be analysed are the results of the three assessors who were involved. On the basis of our results we will infer the rate of agreement separately in atomistic and holistic part.

4. 1

Plan and goals Firstly, we will deal with the methods and results of

our research. In my classes, having introduced the new item in our lessons it as focused I pronunciation out that their it challenging learning most and only and few consequent welcomed remained recording indifferent. found students

they found

Consequently,

performances

obviously reflected their attitude and motivation. The second point to be discussed in this work deals with the issue of the assessor. We will try to point out what the concrete problems in assessing are, based on our experience, and how much our assessors results differ from one another. 4. 2 Methods

49

At the beginning, it is important to mention that we have which the been restricted limited equipment us and as only to far as We what we both did had at our and any disposal. The first item to be mentioned is the time teaching, not was have not recording are concerned. The next important item is space. so the recording studio, sound always

perfect. Nevertheless, we dare say that we did not have any better or worse conditions than in most schools, so in practice we can hardly count on having much different conditions. 4. 3 Evocation and pre-research

50

The evocation stage consisted of several steps. Most of the students had not had any experience with being taught pronunciation and they seemed quite unfamiliar with this language component. So the first task was to make an attempt to motivate them to learn. In my classes we used the building awareness activity questionnaire - presented in one of the previous chapters. It can be discussed in class informally, learners can fill it in and compare in pairs or groups before the whole-class discussion. It is not necessary to carry out the discussion in English, since learners should not be restricted only to their own level of English, but the aim for the learners is to become aware of the meaningfulness of pronunciation learning. The questions in the activity fall into two groups: the first part is aimed to find out about the learners own experience and the following set of questions is aimed to stress the importance or need of pronunciation. It is difficult to make a concrete conclusion out of an informal discussion but I found it contributing to show the pronunciation issue also from the point of view of the listener. When imagining or evoking a situation with a foreigner speaking Czech with a very strong foreign accent, students agreed on the fact that it may make the conversation difficult and very demanding. To conclude our discussion, we reached the view that learning pronunciation is as important for speaking as for listening and that at least, basic knowledge of the English pronunciation principles may improve also our listening skills. Having introduced the learners to our general goals and made them think of for the possible we benefits moved on of to 51 pronunciation learning themselves,

gather

information

about

our

students

concerning

the

factors affecting pronunciation. We have decided again on the questionnaire form eliciting information about the age and the length of learning English, the current or any previous exposure to English, the actual experience with learning pronunciation and their personal goals concerning personal opinions of their strong points and weak points which they would like to practise. We are going to describe the input information which we consider to be relevant to get the overview of the age and the possible influence of other factors. Level and age The students differed in the level of English as well as in the age at which they began to learn English. As far as the level of English is concerned, students belonged to three groups: A2, B1 and B2. The age at which they began learning English ranged from 6 to 50, so the whole group was very heterogeneous. Seven out of sixteen students began learning English in adulthood and the rest nine students before. Exposure to English None of the students reported using English out of the classroom or having been to an English-speaking country for more than a few weeks, so this factor can be considered as insignificant.

Native language The native-language factor will be mentioned later on in connection with the problematic sounds. All students were Czech and therefore in most cases the problems with

52

production of sounds, stress and intonation were similar within the whole tested group. Information are aware of on the the fact learners that these phonetic factors ability, are very

attitude and motivation is difficult to describe. But we imprortant and that is also why we used the evocation phase before the research itself. Pre-research Prior research information to had of starting been each the main out research in part, to and a on prethe carried learners order gather

pronunciation

basis of this a decision was made on setting out the goal and the criteria for our further steps. The pre-research part was a reading-aloud task. The learners were given a text to read. This was a short dialogue. They were given time to read the text silently before the recording.
A Are you working at the moment? B Yes, Ive got a job with a record company called the Mad Cat. A Really? Is that normal working hours? B Yeah, nine to five. A Do you share an office with other people? B Uhuh, there are three of us. A And do you get on well? B Yes, we have a good working relationship. (Natural English. Pre-intermediate. OUP9)

http://fds.oup.com/www.oup.com/word/elt/products/nepre_ts11.doc?cc=cz [online; quoted 16. 2. 2007]

53

The recordings were given to three assessors, who were asked to write down their first impression on the performances in an unstructured way. In the preresearch no strict criteria were given as their setting out was supposed to be the result of the procedure. Here is the example of one assessment given by one assessor:
Student 14 - links quite well (apart from three of us) - no weak forms - recognizable w, velar n; recognizable difference in there vs three - schwa short, long not clear - mad is not open enough - no aspiration

To analyse the data received by the three assessors, I used a table in which all the mentioned strong and weak points were written down. The pluses stand for the aspects which were considered to be pronounced correctly and the minuses for those considered wrong. Further on, those points on which at least two assessors agreed were counted and the results were analysed. Pre-research: results
S 1 Linking [w], [v] [] [] [] [] []
+++ +++ ++ +

S 2
------

S 3
++ + ++ + + + + -

S 4
++ ++ ++ + ++ +

S 5
++ + ++ +++ ++ +

S 6
++ + -++ + + +

S 7
-+ ++ ++ --

S 8
+++ -+ -

S 9
-++ + + ---

S 10
-+ + -+ + -+ + +++

S 11
------

S 12
------

S 13
+-+ ++ -

S 14
--++ --

S1 5
-----

54

[:] [] Aspiratio n

+
---

---

++ --

+ --

+ -+ +

--

+ + +

---

---

Aspect of Number of pronunciation students who pronounce d wrongly Linking - 10 [w], [v] -7 [] -0 [] -5 [] -6 [] -4 [] -1 [:] -2 [] -9 Aspiration -0

Number of students who pronounce d correctly +5 +8 +0 +6 +2 +4 +0 +1 +0 +0

The order of the most problematic areas 1 3 9 5 4 6 8 7 2 9

According to the table the most problematic area for the students was considered to be linking, production of [] and [w], [v]. But the number of other wrongly pronounced sounds was also not insignificant. Therefore, the conclusion and the goal of our further steps was decided to be linking on the first place and practising some of the problematic sounds using exercises on minimal pairs. 4. 4 Realisation According to the results of the pre-research, we have designed a test consisting of several parts. The first two parts applied the atomistic approach, the second three parts holistic. The atomistic method of testing examined the production of problematic sounds in minimal pairs and

55

linking in short phrases. All the students performances were recorded and given to three assessors to be evaluated. In the holistic testing given we used to a reading-aloud read the test activity. Students were time

silently before the recording.

Test on Pronunciation 1 sick free den ran sink vest vet pen men thick three then rang sing west wet pan man

drink a cup of tea put it on Are you in the same place? Dont crash into anything! read a book a big umbrella going out There isnt any doctor. I couldnt do anything about that. The story is very interesting. I am writing to ask for information about your language courses. I am especially interested in an intensive course of two or three weeks. I am thirty-one and I work in the library at Milan University. I can read English quite well but I need to improve my listening and speaking. I have looked at your website, but there is no information about intensive courses next summer. Could you please send me information about dates and prices? I would also like some information about accommodation. If possible, I would like to stay with a family. My wife is going to visit me for a weekend when I am at the school. Could she stay with me in the same family?

56

Consequently, each of them was recorded reading aloud all the three parts. The recordings were given to three assessors to be evaluated and the results from the first recording were given to students with comments on their strong and weak points and suggestions for their further pronunciation practice. After examined the three same months of practice, points, students just were

recorded again reading the same format of the test which pronunciation different words for minimal pairs, phrases for linking and text were chosen. The whole procedure was aimed to be the same and so were the criteria of assessing. The test was supposed to find out about the students progress.

Test on pronunciation 2 veil vent think day beg send sun sin closing whale went sink they bag sand sung sing clothing

never again I saw it. Why am I leaving? may ask Hes quite old. An American car. Good evening! for ever Are you enjoying it here? The shop isnt open yet. too expensive Mind your own business. Thursday evening For me the first good thing about the weekend is that I dont have to go to work. I like my job, but I have to spend all day inside, in an office, and Im a person who loves being outside. Another good thing is that I dont have to get up early. During the week I have to get up at half past six every day. Its not too bad in the summer but I hate it in the winter when its dark

57

in the morning. But above all, I like the weekend because I have time to do all the things I really enjoy doing, like listening to music, reading, or going out with friends.

4. 4. 1 The

Atomistic testing results atomistic testing results consist of the

minimal-pair part and the part of linking. Each examined point was evaluated by the three assessors who either put a tick or a cross next to each point, according to whether they considered the points to be rather correct or wrong. We use the word rather because in many cases it was difficult the to decide on the correctness as the not all of the the sounds were produced absolutely right or wrong. Therefore, assessors evaluation reflected quality examined point which in many cases was not perfectly clear. Minimal pairs In eighteen testing words minimal nine pairs, minimal students pairs, were given the examining

students problematic sounds. Such sounds were those, which had been most often mispronounced in the pre-research: [v], [w]; [s], []; [d], []; [e], []; [n], []. The table below presents the results of the two recordings. In the second and the fourth column, there is the total of correct answers, i. e. answers which were ticked (considered correct) by at least two assessors. The next columns show each students percentage obtained in the test and the last two columns refer to the differences between the first and the second recording.

Student

1st recording Total: 18 17 12

2nd recording Total: 18 18 13

Difference Difference in correct (%) answers +1 +1 +5,6 +5,5

A2/1 A2/2

94.4 66.7

100 72,2

58

A2/3 A2/16 A2/9 B1/10 B1/11 B1/13 B1/14 B1/15 B1/18 B2/6 B2/8 B2/17

14 11 17 10 16 13 12 14 8 17 12 13

77.8 61.1 94.4 55.6 88.9 72.2 66.7 77.8 44.4 94.4 66.7 72.2

15 15 15 15 16 18 14 12 12 18 16 17

83,3 83,3 83,3 83,3 88,9 100 77,8 66,7 66,7 100 88,9 94,4

+1 +4 -2 +5 0 +5 +2 -2 +4 +1 +4 +4

+5,5 +22,2 -11,1 +27,7 0 +27,7 +11,1 -11,1 +22,3 +5,6 +22,2 +22,2

Over

the

three

months

of

practice,

all

students

apart from three improved by 16% on average. Nevertheless, the assessors reported that in spite of such improvement in this part, they had not noticed such significant progress in the production of sounds in holistic part. Linking The test on linking was evaluated in the same way as the test on minimal pairs. Students were given about thirteen phrases in which fifteen points on linking were tested. The assessors ticked or crossed all the points according to their impression.

Student

1st recording Total: 15 5 2 10 4 0 9 7 7 3 3 2 15 4

2nd recording Total: 15 12 10 13 6 6 14 10 13 10 10 5 14 7

Difference Difference in correct (%) answers +7 +8 +3 +2 +6 +5 +3 +6 +7 +7 +3 -1 +3 +46.7 +53.4 +20 +13.3 +40 +33.3 +20 +40 +46.7 +46.7 +20 -6.7 +20

A2/1 A2/2 A2/3 A2/16 A2/9 B1/10 B1/11 B1/13 B1/14 B1/15 B1/18 B2/6 B2/8

33.3 13.3 66.7 26.7 0 60 46.7 46.7 20 20 13.3 100 26.7

80 66.7 86.7 40 40 93.3 66.7 86.7 66.7 66.7 33.3 93.3 46.7

59

B2/17

26.7

12

80

+8

+53.3

As the result chart shows us, all students apart from one improved significantly. The progress was by about 35% on average. Conclusion on atomistic testing The results obtained show improvement in both parts of atomistic testing in most students. The improvement in linking was more significant than in minimal pairs. This may be caused by the fact that minimal pairs containing problematic sounds are more difficult for students to learn as they are often completely new to students, whereas in linking students are supposed to use what they already can just in a different way. The rule for linking is quite simple for them and all they need is just concentrate on its application. 4. 4. 2 The Holistic testing results task for students in the holistic test was

reading aloud a short text. The holistic assessment was divided into five parts: overall impression, comprehensibility, production of sounds, stress and intonation. All of these five parts were evaluated according to a five-point scale in which each point was defined in the assessor sheet. The assessors were supposed to use the five-point scale for each part separately in accordance with their impression of the whole performance. Overall impression The overall impression examined how natural or difficult to understand the students performance was. The

60

scale for the assessors was the same as in the oral part 10 of the Cambridge examination: How does the performance sound to you? 1) pronunciation is heavily influenced by L1 features and at times be difficult to understand 2) pronunciation is generally intelligible, but L1 features may put a strain on the listener 3) although pronunciation is easily understood, L1 features may be intrusive 4) L1 accent may be evident but does not affect the clarity of the message 5) pronunciation is easily understood and prosodic features are used effectively; many features, including pausing and hesitation, are native-like The without reading. assessors at In this task the was to which evaluate the this part were were

looking

text

students

part the

marks of

all assessors

included in the results regardless of whether they differed or not. The average mark was counted out of the three. The table below shows the average mark of each performance from the first and the second recording as well as the difference between them. Plus stands for improvement,

minus for worse performance.

Overall Student impressio n1 A2/1 2.7 A2/2 2 A2/3 4 A2/4 3.3 A2/16 2.3 A2/9 2.3 B1/10 2.7
10

Overall impressio n2 3.3 3 3.7 2.7 2 2.7

Difference +0.6 +1 -0.3 +0.4 -0.3 0

Paper 3: Speaking. Cambridge Common Scale for Speaking [online; quoted 25. 1. 2007] http://www.cambridgeesol.org/support/dloads/ket/KET_HB_sampleS.pdf

61

B1/11 B1/13 B1/14 B1/15 B1/18 B2/6 B2/7 B2/8 B2/17

2.3 2 2 2.3 1.7 4.7 3.3 3.3 3.3

2.3 2.7 2.7 2.7 2.3 5 4 3.7

0 +0.7 +0.7 +0.4 +0.6 +0.3 +0.7 +0.4

The

results

showed

slight

improvement

within

one

level in most students and one student obtained the highest mark native-like performance. Comprehensibility The comprehensibility part tested how much of the performance was understandable for the assessors. In this part the assessment was based again only on hearing without reading the text. The scale for assessment was given as follows: How much do you understand? 1) not most of the performance 2) not quite many words/phrases 3) not a few words/phrases 4) everything with difficulties 5) everything

Student A2/1 A2/2 A2/3 A2/4 A2/16 A2/9 B1/10 B1/11 B1/13 B1/14 B1/15 B1/18 B2/6

Comprehensibility Comprehensibility Difference 1 2 4.3 3.7 5 3.7 4.3 3.3 3.7 3.7 4.3 3.7 4 2.7 5 4.7 4.3 5 4.7 4 4.3 4 4 3.7 4.3 4.3 5 +0.4 +0.6 0 +0.4 +0.7 +0.6 +0.3 -0.3 0 +0.3 +1.6 0

62

B2/7 B2/8 B2/17

5 4.7 5

5 5

+0.3 0

Also in this part there was a slight improvement in most students and the comprehensibility compared to the overall impression has brought much higher scores for more students which infers that despite a lower score in the overall impression, the comprehensibility can still be considered perfect. Production of sounds, stress and intonation The last three parts were assessed at the same time and the assessors of the were asked to of mark their overall and impression production sounds, stress

intonation according to the following scale: 1) Bad 2) Quite good 3) Good 4) Very good 5) Excellent

Production Production Student of sounds of sounds Difference 1 2 A2/1 2.7 3.7 +1 A2/2 2.7 3 +0.3 A2/3 4.3 4 -0.3 A2/4 3 A2/16 2 3 +1 A2/9 2.7 2.7 0 B1/10 2.7 3 +0.3 B1/11 2.7 2.3 -0.4 B1/13 3.3 3 -0.3 B1/14 2 2.7 +0.7 B1/15 2.3 3 +0.7 B1/18 2 3 1 B2/6 5 4.7 -0.3 A2/7 2.3 B2/8 3.3 3.7 +0.4 B2/17 3.7 4.3 +0.6

63

As the production of sounds was more focused than the overall impression and the comprehensibility, the marks for this aspect were usually lower. But again, the results showed slight improvement in most cases.

Student A2/1 A2/2 A2/3 A2/4 A2/16 A2/9 B1/10 B1/11 B1/13 B1/14 B1/15 B1/18 B2/6 A2/7 B2/8 B2/17

Stress 1 2.7 2.3 3.3 3 1.6 2 2 1.7 2.3 1.7 1.7 1.3 4 3 2.7 2.3

Stress 2 3 3 3 2.3 1.7 2.3 2.3 2.7 2.3 2.7 2.3 4.7 3 3.7

Difference +0.3 +0.7 -0.3 +0.7 -0.3 +0.3 +0.6 +0.4 +0.5 +1 +1 +0.7 +0.3 +1.4

The assessment of stress showed improvement in all students apart from one. Stress compare to the production of sounds showed worse results in most cases than in the test on production of sounds. We suppose that the reason may be too much concentrating on fluent reading and correct production of sounds and linking, as these aspects had been involved predominantly in the previous practice.

Student A2/1 A2/2 A2/3 A2/4 A2/16

Intonation 1 2 2 2.7 3.7 1.6

Intonation 2 2.3 2 3.3 2

Difference +0.3 0 +0.6 +0.4

64

A2/9 B1/10 B1/11 B1/13 B1/14 B1/15 B1/18 B2/6 A2/7 B2/8 B2/17

2 2 1.7 1.7 1.7 1.7 1.3 3.3 3.3 2.7 3

1.7 2 2 2 2 2.7 1.7 4 2.7 4

+0.3 0 +0.3 +0.3 +0.3 +1 +0.4 +0.7 0 +1

Out

of

all

the

previous

parts,

the

test

on

intonation proved the lowest scores. The reason may be the same as in the test on stress. Moreover, the assessors reported that they had found this part most difficult to assess, as it is almost impossible to concentrate only on intonation and they usually noted down their first impression although there may have been other factors which have distorted the evaluation.

Overall Student impressio n A2/1 +0.6 A2/2 +1 A2/3 -0.3 A2/16 +0.4 A2/9 -0.3 B1/10 0 B1/11 0 B1/13 +0.7 B1/14 +0.7 B1/15 +0.4 B1/18 +0.6 B2/6 +0.3 B2/8 +0.7 B2/17 +0.4

Comprehensibility Sounds Stress Intonation +0.4 +0.6 0 +0.4 +0.7 +0.6 +0.3 -0.3 0 +0.3 +1.6 0 +0.3 0 +1 +0.3 -0.3 +1 0 +0.3 -0.4 -0.3 +0.7 +0.7 +1 -0.3 +0.4 +0.6 +0.3 +0.7 -0.3 +0.7 -0.3 +0.3 +0.6 +0.4 +0.5 +1 +1 +0.7 +0.3 +1.4 +0.3 0 +0.6 +0.4 +0.3 0 +0.3 +0.3 +0.3 +1 +0.4 +0.7 0 +1

Total +2.6 +2.6 -0.3 +2.9 +0.4 +1.2 +0.8 +0.8 +2.2 +3.4 +4.6 +1.8 +1.7 +3.4

Conclusion on holistic testing According to the results in this part, most students progressed in all five parts. The average highest score was reached in the comprehensibility part which can be considered good news as, for communication, it is more important than the way someones speech sounds. 65

Conclusion on both atomistic and holistic - parts The results of both parts did not prove any significant link between the atomistic and the holistic part. This may be caused by a small number of students tested or the fact that many students were able to learn to produce the problematic sounds and linking in words or short phrases. Accordingly, the progress in the atomistic test was significant. But the holistic test revealed that many students were not able to apply the items from the atomistic test. Some may have achieved a very high score in the minimal pairs and linking, but still the holistic test showed wrong pronunciation of these items. To sum up, according to our results, the improvements in atomistic test may be a question of shortterm practice, whereas we consider any progress in holistic testing a long-term matter as more factors are involved like subconscious acquisition within some time and the attitude and motivation of the learner. 4. 5 Reflexion The last stage of our research dealt with the issue of assessing. The feedback reflecting the students results has already been mentioned in the previous chapter. In this chapter we will explain the reasons for our choice of the assessors in reference to the criterial suggestions presented formerly and analyse the results of the assessors in the atomistic and holistic parts of the test. The question that we would like to find the answer to is whether according to our results our choice of assessing methods can provide us with somehow reliable, nevertheless, impression-based results which may help us in further teaching English pronunciation.

66

Firstly, we will deal with the choice of assessors. Having followed the rule of more than one assessor for each performance we have opted for the number of three assessors in case of different results of the two assessors. For practical reasons three non-native teachers of English were chosen. According to some authors this decision may not be the ideal one, but not everybody opposes it. We think that it was the best option which we had at our disposal and also the easiest option when testing pronunciation in school. Hardly ever can we count on a sufficient number of native speakers in our schools, therefore, in our opinion, we have administered an assessing method which we consider to be the most likely applied in school practice in testing pronunciation. The most difficult task for us to tackle was setting out concrete We will criteria see for the how assessment difficult it and was following for the them. later

assessors to decide whether, e. g. the particular sound was produced correctly or wrongly and decide on the level in holistic assessment. The last point supporting reliability is closely connected to the previous one and means that any irrelevant features of a learners performance which have not been set as criteria should be ignored. The assessors have been instructed according to the suggestions mentioned above and in the next chapter we will look at their agreeable or different results of assessment. 4. 5. 1 The Assessors agreement in atomistic testing test intended for the atomistic assessment

consisted of two parts. The first was aimed to examine the students recognition between problematic sounds and their native-language nearest equivalents and the second part focused on linking. The assessors were instructed to put a 67

tick or a cross next to each examined word or phrase and to evaluate only the relevant features. This at first obviously concrete criterion turned out to be difficult to fulfil. The table below presents the results of one hundred percent agreement of the three scorers.

Atomistic Testing Part 1 - Minimal pairs

Assessors 100% agreement (number of the same answers out of 18) Assessors 100% agreement (number of the same answers out of 18)

Studen t

Percentage of assessor s 100% agreeme nt (%)

Percentage of assessors 100% agreement (%)

1st recording 1 2 3 4 16 9 6 7 8 17 10 11 13 14 15 18 15 12 15 14 16 13 18 14 10 11 9 14 11 12 16 12 83,3 66,7 83,3 77,7 88,9 72,2 100 77,7 55,6 61,1 50 77,7 61,1 66,7 88,9 66,7 15 15 14 13 16 18 17 15 14 14 14 12 15 14

2nd recording 83.3 83.3 77.8 72.2 88.9 100 94.4 83.3 77.8 77.8 77.8 66.7 83.3 77.8

Assessors agreement in total 74%

Assessors agreement in total 82%

As we can see the total of assessors 100% agreement is 74 % in the first recording and 82% in the second one. Their agreement varied on the scale between 50% and 100% which is quite a significant difference. Even though the criteria for assessment had been explained and all the assessors had discussed them beforehand, they reported that 68

what

they

had

found

the

most

difficult

was

such

a and

production of a sound which they felt was not completely correct but at the same time not completely wrong therefore some inclined to consider it correct and some wrong. The next problem was the correct production of the particular sound on the one hand, but mispronunciation of a different part of the word or phrase which sometimes led to the impression that the word had been pronounced wrongly. The second recording compared to the first one showed better results in the assessors agreement. According to the assessors, this is because as most students pronunciation got better, more sounds were clearer and thus easier to assess. Atomistic Testing Part 2 - Phrases on Linking
Assessors 100% agreement Student (number of answer s out of 15 ) 8 13 7 8 14 15 11 10 9 7 4 10 8 13 10 15 Assessors 100% agreement (number of answer s out of 15) 12 10 9 5 10 14 7 12 11 14 12 10 10 9

Percentage of assessors 100% agreement (%)

Percentage of assessors 100% agreement (%)

1st recording 1 2 3 4 16 9 6 7 8 17 10 11 13 14 15 18 53,3 86,7 46,7 53,3 93,3 100 73,3 66,7 60 46,7 27 66,7 53,3 86,7 66,7 100

2nd recording 80 66.7 60 33.3 66.7 93.3 46.7 80 73.3 93.3 80 66.7 66.7 60

Assessors agreement in total 67.5%

Assessors agreement in total 69%

69

The assessors reported the same reasons for different evaluation in this test as in the first one. The results differed mainly because of omitted and changed words in the phrases so that the phrase lost its sense, however, the student used linking correctly. When comparing the agreement in assessment between the first and the second recording, we come to very similar numbers, The impression-based assessment is highly subjective and the either-or The evaluation or can never be completely is too reliable. tick cross assessment

restricted to such a subjective method of assessment. In our opinion, the impression-based assessment should not be restricted only to tick and cross answers as it has much wider scale. Nevertheless, according to the results in the table, we are convinced that at least three assessors are necessary to balance the impression of only one.

70

4. 5. 2 In

Assessors agreement in holistic testing the holistic assessment the three scorers were

supposed to evaluate the reading-aloud task according to the five-point scale in which the mark 5 represented the highest score. As we have already stated that the standard for the holistic assessment should not be native-like performance, but comfortable intelligibility, the results varied on the scale from the lowest to the highest scores. The assessors task was to choose such a definition of the recorded performance which represented their impression best. The table below shows the rate of agreement of the three assessors in the five parts of holistic testing. Although the categories Production of sounds, Stress and Intonation tend to belong to the atomistic part, we decided to put them at the end of the holistic test as the task for the assessors was to give a mark for the overall impression of these three categories.

Holistic Testing (the figures stand for the number of students out of 16) Overall impression 100% agreement 2/3 agreement Different results Understandability Production of sounds Stress Intonation

4 12 0

4 10 2

4 11 1

1 10 5

0 10 6

Apart from the reasons for different assessments of the scorers, we have come to another factor which influenced the holistic assessment. The assessors reported that in spite of good production of sounds, stress and 71

intonation it was often the fluency and the speed of ones reading which had a disruptive effect. A lot of pausing and hesitating in the performance often made the impression and thus the final mark worse. Conclusion on the agreement in assessment As we have seen, the agreement of the assessors was not absolute in any of the parts. This comes out of the impression-based assessment. Except for the criteria given to the assessors, we have to count on each assessor having his or her subjective criteria which are included in their assessing. We have mentioned that in the atomistic test the choice between tick or cross is too limited as we found out that it was the quality of the tested item which was considered produced rather rather correct for or at wrong. least If two the of item the was three well

assessors it was considered correct. Due to the differences in the three scorers assessments, we assume that three scorers are more reliable than only one as more scorers offer more opinions and thus balance the subjective view of only one impression. This conclusion has its relevance to the holistic assessment as well.

72

Conclusion The aim of this work was to present various testing activities and ways of their assessing. We believe that be pronunciation included in as any one of language design. components should course

Moreover, structured pronunciation teaching should involve giving structured feedback to learners as it simply makes the whole process complete for both sides. Pronunciation testing as one of the ways of feedback presupposes setting out goals which help the learners be aware of what itself is aimed becomes to be more achieved, therefore, meaningful for them. the In practice

addition, structured pronunciation teaching is beneficial in the way that it helps the learners build the general awareness of the system of pronunciation as an important language and communication component. In our opinion, setting out goals and subsequent working on their achieving, comparing results within time and class is a significant motivation factor itself which may enhance the whole process. Arranging the right conditions will enable our students to achieve their best. We admit that many of the testing methods which have been presented are too time demanding to be administered in large classes, however, mostly not more than, for example, a correction of a written assignment. Another problem could arise with the number of assessors. On the one hand, it is assumed that the number of three is supporting the reliability of the assessment, on the other hand, this is just a recommended procedure. The only recommendation is to try to follow the advice given and adjust the activities to the possibilities we have at our disposal. Any feedback which is aimed to help our learners and which is accomplished thoughtfully 73

and responsibly is better than none. To compare, we often assess spoken or written work of our learners, which is also In subjective my own and impression-based, I have found therefore, pronunciation pronunciation is not as exceptional as it might seem. experience, activities and the whole research procedure particularly contributing as far as not only speaking and listening skills attitude process, with their of my to the students many other are concerned, but also their activities did weak not involving seem reading,

speaking and listening. At the beginning of the testing students and and the particularly on the enthusiastic, but having received their first assessment strong points, information their This overall higher impression scores in comprehensibility, following further was a

practice began to be more focused and motivated to achieve recording. turning point in our pronunciation practice, because it is natural that people are eager to know about how they are proceeding and consequently, willing to improve. I dare say that they have become slightly less scared of the listening tasks as they are aware of the existence of weak forms, linking and other aspects of pronunciation which used to In my make them confused good and discouraged about great from concentrating before the actual listening. opinion, enjoyable leisure the point pronunciation grammar and teaching in general is that it may be a great time-filler providing meaningful activities, after any vocabulary drills and on the top of it, it can be used as a concentration-demanding activity since it does not necessarily require too much strain and attention. The suggestion which should be followed in all circumstances so as to motivate the learners and progress, is maintaining nice atmosphere, encouragment and giving a 74

chance to everyone to experience their own success and achieve their best.

75

Appendix 1

Assessor: Instruction on assessment Atomistic Testing Part 1 Minimal Pairs Put a tick or a cross according to the correctness in pronunciation. sick free den ran sink vest vet pen men thick three then rang sing west wet pan man Student: ______________

Part 2 Phrases on Linking Put a tick or a cross according to good or bad linking. drink a cup of tea put it on Are you in the same place? Dont crash into anything! read a book a big umbrella going out There isnt any doctor. I couldnt do anything about that. The story is very interesting. Holistic Testing 1st listening: Do not read the text. How does the performance sound to you? Scale: natural -------------------------------------------------- very difficult to understand 5 4 3 2 1 1) pronunciation is heavily influenced by L1 features and at times be difficult to understand 2) pronunciation is generally intelligible, but L1 features may put a strain on the listener 3) although pronunciation is easily understood, L1 features may be intrusive 4) L1 accent may be evident but does not affect the clarity of the message 5) pronunciation is easily understood and prosodic features are used effectively; many features, including pausing and hesitation, are native-like 2nd listening: Do not read the text. How much do you understand? 1) not most of the performance 2) not quite many words/phrases 3) not a few words/phrases 4) everything with difficulties 5) everything k t w d g t w j p t w

76

Part 3 Reading Aloud I am writing to ask for information about your language courses. I am especially interested in an intensive course of two or three weeks. I am thirty-one and I work in the library at Milan University. I can read English quite well but I need to improve my listening and speaking. I have looked at your website, but there is no information about intensive courses next summer. Could you please send me information about dates and prices? I would also like some information about accommodation. If possible, I would like to stay with a family. My wife is going to visit me for a weekend when I am at the school. Could she stay with me in the same family? 3rd listening: Write down words/phrases you found difficult to understand. 4th listening: Scale: (1) Bad (2) Quite Good (3) Good (4) Very Good (5) Excellent Evaluate the students: production of sounds stress intonation 1 1 1 2 2 2 3 3 3 4 4 4 5 5 5

Further Notes:

77

Appendix 2
Assessor: Instruction on assessment Atomistic Testing Part 1 Minimal Pairs Put a tick or a cross according to the correctness in pronunciation. veil vent think day beg send sun sin closing whale went sink they bag sand sung sing clothing Student: __________

Part 2 Phrases on Linking Put a tick or a cross according to good or bad linking. never again I saw it. Why am I leaving? may ask Hes quite old. An American car. Good evening! for ever Are you enjoying it here? The shop isnt open yet. too expensive Mind your own business. Thursday evening Holistic Testing 1st listening: Do not read the text. How does the performance sound to you? Scale: natural -------------------------------------------------- very difficult to understand 5 4 3 2 1 1) pronunciation is heavily influenced by L1 features and at times be difficult to understand 2) pronunciation is generally intelligible, but L1 features may put a strain on the listener 3) although pronunciation is easily understood, L1 features may be intrusive 4) L1 accent may be evident but does not affect the clarity of the message 5) pronunciation is easily understood and prosodic features are used effectively; many features, including pausing and hesitation, are native-like 2nd listening: Do not read the text. How much do you understand? a) everything b) everything with difficulties c) not a few words/phrases d) not quite many words/phrases e) not most of the performance r w j j t n d r w p w r j

78

Part 3 Reading Aloud For me the first good thing about the weekend is that I dont have to go to work. I like my job, but I have to spend all day inside, in an office, and Im a person who loves being outside. Another good thing is that I dont have to get up early. During the week I have to get up at half past six every day. Its not too bad in the summer but I hate it in the winter when its dark in the morning. But above all, I like the weekend because I have time to do all the things I really enjoy doing, like listening to music, reading, or going out with friends. 3rd listening: Write down words/phrases you found difficult to understand. 4th listening: Scale: (1) Excellent (2) Very Good (3) Good (4) Quite Good (5) Bad Evaluate the students: production of sounds stress intonation Further Notes: 1 2 1 1 3 2 2 4 3 3 5 4 4 5 5

79

Bibliography ALDERSON, C. J., WALL, D. & CLAPHAIM, C. Language Test Construction and Evaluation. Cambridge: Cabridge University Press, 1996. 310 p. BACHMAN, L. F. Fundamental considerations in language testing. Oxford: Oxford University Press, 1990. 408 p. BACHMAN, L. F., PALMER, A. S. Language testing in practice. Oxford: Oxford University Press, 1996. 377 p. BAKER, A. Ship or sheep. 2nd edition. Cambridge: Cambridge University Press, 1981. 168 p. BAKER, D. Language testing. London: Edward Arnold, 1989. 110 p. DALTON, C., SEIDLHOFER, B. Pronunciation. Oxford: Oxford University Press, 1994. 191 p. GAIRNS, R., REDMAN, S. Natural English. Pre-intermediate. Students book. 1st edition. Oxford: Oxford University Press, 2005. 176 p. GILBERT, JUDY B. Speaking Clearly. Students Book. 1st edition. Cambridge: Cambridge University Press, 1990. 136 p. GIMSON, A. C., CRUTTENDEN, A. Gimsons Pronunciation of English. Sevenoaks: Arnold, 1996. 304 p. HANCOCK, M. English Pronunciation in Use. 1st edition. Cambridge: Cambridge University Press, 2003. 200 p. HEATON, J. B. Classroom Testing. London: Longman, 1990. 127 p. HEATON, J. B. Writing English Language Tests. 1st edition. London: Longman, 1988. 160 p. HUGHES, Arthur. Testing for Language Teachers. 1st edition. Cambridge: Cambridge University Press, 1989. 172 p. JENKINS, J. The Phonology of English as an International Language: new models, new norms, new goals. 1st publ. Oxford: Oxford University Press, 2000. 258 p. KELLY, G. How to teach pronunciation. 1st edition. Harlow: Longman, 2000. 154 p. KENWORTHY, J. Teaching English Pronunciation. Harlow: Longman, 1987. 164 p. McNAMARA, T. Language testing. 1st edition. Oxford: Oxford University Press, 2000. 140. p.

80

MADSEN, S. H. Techniques in Testing. New York: Oxford University Press, 1983. 212 p. OXENDEN, C., LATHAM-KOENIG, C., SELIGSON, P. New English File. Preintermediate. Students book. 1st edition. Oxford: Oxford University Press, 2005. 159 p. ROACH, P. English Phonetics and Phonology: a practical course. 3rd edition. Cambridge: Cambridge University Press, 2000. 283 p. SCRIVENER, J. Learning Teaching. 2nd edition. Oxford: Macmillan, 2005. 431 p. SNOW, Don. Myths and Misconceptions About Second Language Learning [online]. Washington: Center for Applied Linguistics, 1992 [quoted 5 February 2007]. Available from: <http://www.cal.org/recources/digest/myths.html>. SOARS, J. et al. Headway Intermediate Pronunciation. Oxford: Oxford University Press, 1990. 112 p. SZPYRA-KOZLOVSKA, J., FRANKIEWICZ, J., NOWACKA, M., STADNICKA, L. Assessing assessment methods on the reliability of pronunciation tests in EFL [online] . Lublin: Maria Curie-Sklodowska University, Lublin, Poland: 2005. [quoted 14th February 2007]. Available from: <http://www.phon.ucl.ac.uk/home/johnm/ptlc2005/pdf/ptlcp37.pdf> UNDERHILL, A. Sound Foundations. Oxford: Macmillan, 2005. 210 p. UNDERHILL, N. Testing spoken language. Cambridge: Cambridge University Press, 1987. 117 p. University of Cambridge. ESOL Examinations. Paper 3: Speaking. Cambridge Common Scale for Speaking [online, quoted 14th February 2007] Available from: http://www.cambridgeesol.org/support/dloads/ket/KET_HB_sampleS.pdf UR, Penny. A Course in Language Teaching. Cambridge: Cambridge University Press, 1997. 375 p.

81