North South University Department of English ENG 573: Research Methods in TESOL SUMMER 2012

Research Paper

Conducting an error gravity study for the Secondary School Teachers in evaluating a writing task

Submitted to: MAHJABEEN HUSSAIN (MJn) Lecturer Department of English North South University

Submitted by: SK. SHAFIQUR RAHMAN ID: 121 1218 055

Date of Submission: 16th August 2012

A. Abstract: This study addresses the response of ESL (English as Second Learners) students error gravity in both grammatical and writing forms to a naturally occurring sample of second language writing in a secondary school teaching environment. Using a guided grammatical scale protocol, the secondary school participants are led to participate a one-page essay that is produced by the students whose first language is Bengali. The faculty evaluates the writing holistically (on a scale from one to ten) and is invited to identify the most troublesome errors. Results indicate that the scoring does not affect holistic scores; however, the English faculties of secondary school do rate the composition differently. While qualitative analysis demonstrates that, not surprisingly, individual editing styles among faculty are quite variable, there is a tendency across faculty to edit semantic gaps as opposed to grammatical items. This indicates a preference by the faculty to clarify content, a finding that supports prior research. This study also includes the most frequent grammatical errors done by the secondary school students, which help the English faculties to bear more concentration and the students selfcorrection in a better writing skill achievement. The opinion through a bounded system case study, a term originally coined by Louis Smith, results into a systematic way of giving importance on particular grammatical parts and structures of meaningful writing skills. The participants of this study are 5 secondary level students and 3 secondary level teachers. The materials used in this study are- a case study followed by the researcher while evaluating the students writing task; a class observation form filled by the researcher that gives a description of a grammar classroom, teaching skill in learning grammatical items, materials used by the teacher in a grammar teaching class room, different types of learners and teachers cooperation with them; 3 questionnaires filled by the secondary level teachers that give their opinion in importance of grammar in a writing task. Through this procedure, we hope to reach the result of the study that gives us an identification of teaching grammar according to its seriousness that improves the secondary school students writing task.This
Sk. Shafiqur Rahman | 2

study of error gravity for the secondary school teachers will be beneficial for TESOL while teaching and evaluating writing skill.

B. Motivation for the research: When I was at class eight, in our English language class, our English teacher tasked us to write a 250-word composition on Climate Change as a home work. I wrote the composition and gave it to my home tutor for a previous evaluation, and he scored my composition six out of ten, and showed me some correction of my writing and advised me some grammatical and textual units. The next day, I submitted my original uncorrected script of the composition to my course teacher and he scored my paper eight out of ten. Then I surprised and a question turning around my brain that why the same script getting different scores. By then, I planed upon an idea, and I again submitted the same uncorrected paper to the same course teacher of my school after four months, and that time he scored my paper six out of ten. Then I understood there are no rules of scoring writing. Scoring writing goes through a holistic approach. As it is called impressionistic scoring, writing creates different impressions to different people at different time. For the tradition of evaluating writing ability in second language learning, I plan an action research to develop an error gravity frequency table of evaluating a writing task for the secondary school teachers so that the scoring in a writing task can be done analytically in a context of different test givers, period and kind.

Sk. Shafiqur Rahman | 3

C. Purpose of the research: To make an error gravity frequency measurement table through a case study by doing an action research To get an idea of the writing ability of the secondary school student To attain a comparison between what is being learnt to the students and to what extent they are competent in learning grammar through live observation as a part of class room observation To what extent CLT is helping secondary school students learning of particular grammatical items

D. Research questions: 1. What is the difference between mistake and error? 2. What types of errors are considered most serious by the Secondary School English Teachers? 3. What is the degree of seriousness in error making among the high school students in their writing tasks? 4. What are the levels of giving marks of the high school English teachers to their students considering the errors in writing skill? 5. What are the respective expectations of the high school English teachers in making correct and meaningful sentences in a Second Language Learning environment? 6. What is the level of learning grammar among the high school students in a Second Language Learning environment?

Sk. Shafiqur Rahman | 4

E. Literature Review: 'Error' is made unconsciously and corrected by the same person who has made the 'error'. Error is in the level of performance; however, 'mistake' is because of the lack of knowledge. Therefore, mistake is in the level of competence. None of the above is intentional. 'Fault' is completely different from these two. It is because of being careless or doing something intentionally, in this case the person who had done the fault has to be blamed.

In a well-known experiment, mistakes made by Greek secondary-school children were shown to Greek teachers of English, British teachers of English and British non-teachers. Members of each group graded the mistakes on a scale from 1 (least serious) to 5. Interestingly, the mistakes which the Greek teachers regarded as most serious were often those that troubled the native speakers least, and vice-versa. Some examples, with the average grading given by the Greek teachers (GT) and the British non-teachers (BN): *We agreed to went to the cinema by car. (GT4.6; BN2.2) *We didnt knew what had happened. (GT4.4; BN1.8) *Dizzys from the wine we decided to go home. (GT4.2; BN2.1) *The people are too many so and the cars are too many. (GT3.0; BN4.3) *The bus was hit in front of. (GT2.6; BN4.3) *There are many accidents because we have not brought (meaning broad) roads. (GT2.4; BN4.1)

The native speakers generally gave higher marks to mistakes, which impeded their understanding: discussing the reasons for their assessments, many mentioned intelligibility. The non-native teachers seemed more disturbed by infringements of common grammar rules; in discussion, they referred frequently to basic mistakes. They seemed most upset by the fact that learners continued to break rules which had been taught at an earlier stage of the

Sk. Shafiqur Rahman | 5

course, and which they should therefore have mastered. They were effectively teaching grammar because it was there. (Hughes and Lascaratou 1982)

Grammar can be reassuring and comforting. In the convoluted landscape of a foreign language, grammar rules shine out like beacons, giving students the feeling that they can understand and control what is going on. Although this feeling is partly illusory (structural competence only accounts for a proportion of what is involved in mastery of a language), anything that adds to learners confidence is valuable. However, the security-blanket aspect can lead students and their teachers to concentrate on grammar to the detriment of other less modifiable but equally important aspects of the language. People often regard grammar as a single interconnected system, all of which has to be learnt if it is to work properly. This is an illusion. Grammar is not something like a car engine, where a fault in one component such as the ignition or fuel supply can cause a complete breakdown. It is more realistic to regard grammar as an accumulation of different elements, some more systematic than others, some linked together tightly or loosely, some completely independent and detachable. We teach or should teach selected subsystems, asking for each: 1) How much of this do the students know already from their mother tongue? (A German speaker, unlike a Japanese learner, knows the main facts about English article use before his/her first lesson.) 2) How much of the rest is important? 3) How much of that have we got time for? To try to teach the whole system is to ignore all three of these questions. Where grammar is given too much priority, the result is predictable and well known. Course books become little more than grammar courses. Students do not learn English: they learn grammar, at the expense of other things that matter as much or more. They know the main rules, can pass tests, and may have the illusion that they know the language well. However, when it comes to using the language in practice they discover that they lack vital elements, typically vocabulary and fluency: they can recite irregular verbs but cannot sustain a conversation. (As J K Jerome put it a century
Sk. Shafiqur Rahman | 6

ago, few people care to listen to their own irregular verbs recited by young foreigners.) Such an approach is also psychologically counterproductive, in that it tends to make students nervous of making mistakes, undermining their confidence and destroying their motivation. (Michael Swan, 2002) Many approaches within second language research have presented insights into the intricacies of native/non-native interactions; one of those has been broadly termed the study of error gravity. This type of research has aimed to establish hierarchies of error types so that language teachers might focus on areas of grammar and pronunciation judged by native speakers to be most disruptive to communication. Measures of comprehensibility, acceptability, and naturalness are among the constructs used to assess sensitivity to particular L2 errors. (Rifkin & Roberts, 1995) Using an error evaluation approach, error gravity hierarchies have been investigated for second and foreign language classes in English (Johansson, 1978; Khalil, 1985; Sheorey, 1986; Santos, 1987;Tomiyana, 1980);. In most of this research, isolated spoken or written sentences or contrived prose passages have been used as stimuli. Such studies have provided useful insights into native speaker perception of errors in L2 writing, but the approach, which often presents only one error per sentence carrier, may only partially capture a realistic evaluation of L2 performance. In order to help composition instructors had better prepare their learners, we argue for the importance of assessing response to L2 error in a more naturalistic manner. Rather than asking native writers to evaluate L2 errors in terms of abstract concepts such as acceptability or comprehensibility, a naturalistic performance by the evaluator (i.e., marking and editing any error they perceive) holds the potential to produce a more realistic accounting of response to L2 writing. In the current study, we aim to more approximate actual evaluation of and response to L2 writing by asking university faculty to

Sk. Shafiqur Rahman | 7

holistically grade and then edit an actual composition written by a student for whom English is not a native language. F. Method and Materials: 1. Classroom observation: (Appendix C) Target institution: Rajuk Uttora Model College Target population: Students of class nine Target subjects: students of class nine and their English Language faculty Observer: The Researcher Goals of observation: number of students in an English language classroom; Evaluation of efficacy of grammar and semantic materials provided by the course teacher; Practice session that helps the practical use of taught grammar materials and its effectiveness in the researchers case study materials Procedure: live observation, notes and logs of the researchers, the researchers participation with the student activity, feedback

2. Questionnaire for the English language faculty (Appendix B) Likert scales, Guttman scaling and scaled questions is followed to achieve leading questions; and highbrow, complex , irritating and negative questions are kept out of the questionnaire Questionnaires are associated with class room observation, case study materials and interviews with individual respondents; Validity, reliability, lack of redundancy, repetition, independence of items, irrelevant information are maintained; Little explanation, giving free choice, few verbal links and independent opinions are given importance;

Sk. Shafiqur Rahman | 8

As a secondary school English language faculty, the teachers academic, professional and non-academic achievements will be considered as ones level of proficiency

Sex and age are identified in a separate option in the questionnaire

3. Case study: Step 1: 5 students of class nine are given a blank A4 paper; they write an essay on the give topic mentioned on the top of the paper; the course teacher evaluates them and identifies the errors with circles using red ball-point pen; and on the pointed box of the paper the course teacher gives his/ her comments; As students are reluctant to give their name on the paper, they are identified as participantsP1, P2, P3, P4 and P5; among them P1 and P2 are female and the rest P3, P4 and P5 are male Step 2: The researcher counts the particular errors for each paper with countable frequency and give his opinion on the comment box. Study) G. Procedure: Classroom observation 1. Notes and logs of the researchers: Number of the students in the classroom: 42 Materials used by the teacher: A grammar book- A Practical English Grammar by (Appendix A: Case

Thomson and Martin; Several work shits for pair work in classroom participation Topics discussed by the teacher: Complex sentence structures; Linkers and the use of

linking words and Use of punctuation Tasks for the students: Teacher gives several work shits to pin point the errors in a

complex sentence, Fill in the blanks with linking words and editing punctuation marks
Sk. Shafiqur Rahman | 9

Evaluation of the task: Teacher observes the pair work situation and calls for the

correct answers 2. live observation: Classroom environment: Well- lighted large classroom Response of the students: Some hyperactive students ask questions and teacher helps

them Cooperation of the teacher with the students: Teacher cooperates with the learners Teacher-student mutual understanding: well Handling of weaker students: Teacher knows his weaker students and takes care in

classroom participation willingly 3. Follow-up and paper selection process: Teacher asks students to write an essay on Traffic Congestion and its Possible Solution and tells me to collect five papers among them, I selected 2 hyperactive learners, 2 shy learners and one weak learners paper by the suggestion of the teacher for my Case Study procedure. Among them, two are female and three are male. Case study: Teacher collects the five papers, identifies the errors with red pen, and scores them. The following table shows some details of the five participants. Table- G1 Participants P1 P2 P3 P4 P5 Category Hyperactive Shy weak Shy Hyperactive Scores (out of ten) 08 07 05 08 07 Sex Female Female Male Male Male

Sk. Shafiqur Rahman | 10

Identifying errors: the researchers activity: Each 0 presents for each error in the frequency column. Based on the total number of frequency, degree of seriousness is identified. Table-G2 Grammatical Errors Subject verb agreement Error Frequency Total number of frequency Degree of seriousness o o o o o o o o o o o o o o o o o o o o o Not serious Alright Serious Very serious

P1- 00 P2- 000 P3- 0000 P4- 0 P5- 000 P1- 0 P2- 0 P3- 000 P4- 0 P5- 00 P1P2- 0 P3- 0 P4P5- 0 P1-0 P2-00 P3-00000 P4-00 P5-0 P1-0 P2-0 P3-00 P4-0 P5-0 P1P2P3-0 P4P5P1P2-0 P3-0 P4-00 P5-00

13

Use of tense

Not serious Alright Serious Very serious Not serious Alright Serious Very serious Not serious Alright Serious Very serious Not serious Alright Serious Very serious Not serious Alright Serious Very serious Not serious Alright Serious Very serious

Word order

Spelling mistake

11

Use of determiner

Use of capitalization

Correct position of adverb

Sk. Shafiqur Rahman | 11

Punctuation

Period (.)

Comma (,)

Hyphen (-)

Quotation mark ( )

Direct and indirect sentences

Passive sentences

Incorrect idiomatic expressions

Antecedents of pronouns

Use of appropriate linkers in making complex sentences

P1-0 P2-0 P3-00 P4-00 P5-00 P1P2P3-0 P4P5P1-0 P2-0 P3-0 P4-00 P5-00 P1P2P3P4P5P1P2P3P4P5P1P2P3P4P5P1P2-0 P3-0 P4P5P1P2P3P4-0 P5P1P2-0 P3-0 P4P5-0 P1-0 P2-0 P3-00 P4-0 P5-0

o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o

Not serious Alright Serious Very serious Not serious Alright Serious Very serious Not serious Alright Serious Very serious Not serious Alright Serious Very serious Not serious Alright Serious Very serious Not serious Alright Serious Very serious Not serious Alright Serious Very serious Not serious Alright Serious Very serious Not serious Alright Serious Very serious Not serious Alright Serious Very serious

Sk. Shafiqur Rahman | 12

Redundancy

Parallel structure

P1P2P3P4-0 P5-0 P1P2-0 P3-0 P4-0 P5P1-0 P2-00 P3-000 P4-0 P5-00 P1P2P3-0 P4P5-

o o o o o o o o o o o o

Not serious Alright Serious Very serious Not serious Alright Serious Very serious

Appropriate prepositions

Not serious Alright Serious Very serious Not serious Alright Serious Very serious

Semantics

Questionnaire for the English language faculty: In L2 learning situation writing skill is mostly dependent on grammatical accuracy. Three secondary school teachers give their opinion on the importance of grammar learning in building up successful writing skills answering different questions of the questionnaire. This particular procedure relates researchers case study results with the opinion of the related faculty members. Questionnaire query results1. Do you think learning grammar according to its importance in a writing task is necessary in a secondary school L2 learning environment? o 3 teachers say- yes 2. Which type of scoring do you prefer in a writing task evaluation? o 2 prefer holistic and 1 prefers analytical scoring 3. Do you think errors in semantics are more important than grammatical mistakes? o 2 say they are equally important, one says no

Sk. Shafiqur Rahman | 13

4. What kind of classroom environment do you prefer to teach grammar in a teacherstudent cooperation group task? a) Large (50 students) b) small (10 students) c) medium (25 students) o Two prefers large classroom and one, medium 5. Do you think scores should be given on the correct use of grammar in a writing task? a) Yes b) No c) there are other facts like:.. o Two say yes and one says there are other facts like: total meaning Opinion of the teachers: 1. Only grammatical accuracy comes first in scoring a writing task o Two say agree and one, disagrees 2. The writing should only convey meaning / follow semantics where grammatical errors are not given importance o Two say disagree and one, strongly disagrees 3. Grammar and semantics go side by side o Two say agree and one, strongly agrees 4. Subject verb agreement is the prime rule of grammar o Three say strongly agree 5. Punctuation problem can be considered at the secondary school level o Two say neutral and one, agrees 6. Must have the knowledge of tense at the secondary school level o Two say strongly agree and one, agrees 7. Must have the knowledge of capitalization at the secondary school level o Two say agree and one stays neutral 8. Spelling mistakes at the secondary school level can be considered o Two say agree and one, strongly agrees 9. Must have the knowledge of word order at the secondary school level o Two say agree and one stays neutral 10. Writing skill reflects the total skill of grammar and meaning o Three say strongly agree

Sk. Shafiqur Rahman | 14

H. Analysis: Through Classroom observation procedure, we see grammatical error gets importance in writing task. In the provided work shit for classroom task, students practice pin pointing error so that they can find grammatical error in their writing. Among the selected five participants of the writing test, there is a mixture of different categories of learners- hyperactive, shy and weak so that I can get different types of error in their writing. According to the degree of seriousness in Table-G2, we find that most frequent grammatical errors are- subject-verb agreement, use of tense, spelling mistake, Use of determiner, correct position of adverb, punctuation (comma), appropriate prepositions. P1 and P4 score highest (8 out of 10) and their frequency of error is quite high in these very serious area than other areas.

Most frequent errors

14 12 10 8 6 4 2 0 very serious

Sk. Shafiqur Rahman | 15

Scores of the participants

9 8 7 6 5 4 3 2 1 0 P1 P2 P3 P4 P5

Arithmetic mean of the scores is (8+7+5+8+7)/5= 7 Median of the scores is (5,7,7,8,8)=7 Mode of the scores is 7 and 8 Therefore, in this test, 7 is an ideal score. As, P2 and P5 both score 7. According to Table-G2, Most frequent error of P2 and P5 is in Subject- verb agreement. That is under very serious in the degree of seriousness. In the questionnaire, the degree of satisfaction table shows that under opinion no 4. ,Subjectverb agreement is the prime rule of grammar , 3 teachers of secondary school strongly agree with it. Therefore, the case study data that we get from the students and the opinion # 4 of the questionnaire that we get from the teachers meet at a point that subject- verb agreement is the most frequent error in a writing task. From the teachers expectation- What are the qualities of a good writing?1. Proper form; less grammatical error; meaning of a sentence should be clear 2. Less grammatical error 3. Less grammatical error; proper form and meaning From the questionnaire, teachers common expectation is less grammatical meaning.

Sk. Shafiqur Rahman | 16

I. Results of the research: 1. What types of errors are considered most serious by the Secondary School English Teachers? subject-verb agreement, use of tense, spelling mistake, Use of determiner, correct position of adverb, punctuation (comma), appropriate prepositions 2. What is the degree of seriousness in error making among the high school students in their writing tasks? subject-verb agreement

3. What are the levels of giving marks of the high school English teachers to their students considering the errors in writing skill? From the questionnaire, Q#2, two teachers prefer with holistic and one analytical marking system. 4. What are the respective expectations of the high school English teachers in making correct and meaningful sentences in a Second Language Learning environment? Less grammatical error 5. What is the level of learning grammar among the high school students in a Second Language Learning environment? As the most frequent grammatical error is Subject-verb agreement and the teachers expectation Subject-verb agreement is the prime rule, the level of learning grammar among the high school students is at the beginner level.

Sk. Shafiqur Rahman | 17

References: Hughes, A. and C. Lascaratou. (1982). Competing Criteria for Error Gravity English Language Teaching Journal, 36/3, 175182. Johansson, F. (1978).Studies of error gravity: Native reactions to errors produced by Swedish learners of English.Gothenburg Studies in English 44. Gothenburg, Sweden: Acta Universitatis Gothoburgensis. Khalil, A. (1985). Communicative error evaluation: Native speakers evaluation and interpretation of written errors of Arab EFL learners. TESOL Quarterly, 19(2), 335 351. Rifkin, B., & Roberts, F. D. (1995). Error gravity: A critical review of research design, Language Learning, 45(3), 511537. Swan, Michael. In Methodology in Language Teaching, ed. Richards and Renandya, CUP 2002, pp.148152) Sheorey, R. (1986). Error perceptions of native-speaking and non-native-speaking teachers of ESL.ELT Journal, 40(4),306312. Santos, T. (1988). Professors reactions to the academic writing of nonnativespeaking students. TESOL Quarterly, 22(1),6990. Tomiyana, M. (1980). Grammatical errors communication breakdown. TESOL Quarterly, 14(1), 7179.

Sk. Shafiqur Rahman | 18

Appendix A: Case Study This study is on purpose of a research project named conducting an error gravity study for the Secondary School Teachers in evaluating a writing task. The researcher is an MA in TESOL student of North South University and this study is a part of his research work. The researcher is asking your kind cooperation and giving his words that all the personal information will be kept secret. Traffic Congestion and its Possible Solution Sex: male / female

Name:

Mark obtained: Teachers Comment: Thank you for your nice cooperation
Sk. Shafiqur Rahman | 19

Identifying errors: the researchers activity Grammatical Error Frequency Errors Subject verb agreement

Total number of frequency

Degree of seriousness o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o Not serious Alright Serious Very serious Not serious Alright Serious Very serious Not serious Alright Serious Very serious Not serious Alright Serious Very serious Not serious Alright Serious Very serious Not serious Alright Serious Very serious Not serious Alright Serious Very serious Not serious Alright Serious Very serious Not serious Alright Serious Very serious Not serious Alright Serious Very serious

Use of tense

Word order

Spelling mistake

Use of determiner

Use of capitalization

Correct position of adverb

Punctuation

Period (.)

Comma (,)

Sk. Shafiqur Rahman | 20

Hyphen (-)

Quotation mark ( )

Direct and indirect sentences

Passive sentences

Incorrect idiomatic expressions

Antecedents of pronouns

Use of appropriate linkers in making complex sentences Redundancy

Parallel structure

Appropriate prepositions

Semantics

o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o

Not serious Alright Serious Very serious Not serious Alright Serious Very serious Not serious Alright Serious Very serious Not serious Alright Serious Very serious Not serious Alright Serious Very serious Not serious Alright Serious Very serious Not serious Alright Serious Very serious Not serious Alright Serious Very serious Not serious Alright Serious Very serious Not serious Alright Serious Very serious Not serious Alright Serious Very serious

Sk. Shafiqur Rahman | 21

Appendix B: Questionnaire for the English language faculty Questionnaire On Conducting an error gravity study for the Secondary School Teachers in evaluating a writing task Dear Respondent, I am Sk. Shafiqur Rahman, student of MA in TESOL, North South University Bangladesh. I would be grateful if you could spare a few minutes to complete this questionnaire. The researcher is asking your kind cooperation and giving his words that all the personal information will be kept secret. (Please answer all the questions correctly and for any query ask to surveyor) Name: Gender: Male/Female Faculty position: .. Academic, professional and non-academic achievements

Please tick the appropriate answer: 6. Do you think learning grammar according to its importance in a writing task is necessary in a secondary school L2 learning environment? a) Yes b) No (If Yes then answer the questions) 7. Which type of scoring do you prefer in a writing task evaluation? a) Analytical b) Holistic c) Other 8. Do you think errors in semantics are important than grammatical mistakes? a) Yes b) No c) Equal 9. What kind of classroom environment do you prefer to teach grammar in a teacherstudent cooperation group task? b) Large (50 students) b) small (10 students) c) medium (25 students) 10. Do you think scores should be given on the correct use of grammar in a writing task? b) Yes b) No c)there are other facts like:..

Sk. Shafiqur Rahman | 22

Please mark () the appropriate box to indicate your degree of satisfaction. Opinions 1. Only grammatical accuracy comes first in scoring a writing task 2. The writing should only convey meaning / follow semantics where grammatical errors are not given importance 3. Grammar and semantics go side by side 4. Subject verb agreement is the prime rule of grammar 5. Punctuation problem can be considered at the secondary school level 6. Must have the knowledge of tense at the secondary school level 7. Must have the knowledge of capitalization at the secondary school level 8. Spelling mistakes at the secondary school level can be considered 9. Must have the knowledge of word order at the secondary school level 10. Writing skill reflects the total skill of grammar and meaning Strongly agree Agree Neutral Disagree Strongly disagree

What are the qualities of a good writing? :

Sk. Shafiqur Rahman | 23

Signature: ........................................ Appendix C: Classroom observation 1. Notes and logs of the researchers: Name of the faculty: Number of the students in the classroom: Materials used by the teacher: Topics discussed by the teacher: Tasks for the students: Evaluation of the task:

Date: .............

2. live observation: Classroom environment: Response of the students: Cooperation of the teacher with the students: Teacher-student mutual understanding: Handling of weaker students:

3. The researchers participation with the student activity: 4. Feedback:

Sk. Shafiqur Rahman | 24