FIELD STUDY REPORT

In this modern era English is believed to be the backbone of countries’ economies,

English has become the language of technology and technology is considered to be a power;
importance of English has significantly risen to a high level. Especially in third world countries
due to the poor quality of education in government sector English proficiency levels of students
are not high enough to meet the demands of 21st century. In case of Pakistan Bashiruddin (2003),
pointed out that “Pakistan has complex and interesting linguistic contexts. Its national language
is Urdu and its official language is English. In the four provinces there are four main native
languages: in Sindh, Sindhi, in Punjab, Punjabi: in the NWFP (KPK), Pashto: and in Baluchistan,
Balochi. Each of these native languages has many dialects. In this context English on the one
hand conflicts with a sense of national identity and on the other hand, is strongly associated with
social advancement (p.05)”.
Most Pakistanis; mostly from the lower socio-economic strata attend Government Urdu-
medium schools. The students have no exposure to English in their everyday communication
(Bashiruddin, 2003 p.07).
In current curriculum English has been given priority as a subject. It has been mentioned
that it should reach the masses, so that there is no discrimination amongst the rich and poor in
Pakistan in terms of opportunities for personal, professional and economic development. Mostly
in Pakistani Government schools students are tri-lingual they speak their regional language, then
Urdu is the national language and English.
So, in order to see the proficiency of Pakistani students from different regions a cross-
sectional survey has been conducted in Lahore and Karachi.
This field report paper discusses purpose of the study, highlights main research question
and hypothesis and the methodology, it will highlight the ethical issues and limitations of the
study. Furthermore, it also describes the data analysis method and procedure, and conclusion.
Main research question
What is the difference in overall English proficiency test scores of class 8th students between
Lahore and Karachi TCF government schools?

Subsidiary Questions

1
(I)What is the difference between Karachi and Lahore English reading proficiency test scores of
class 8th students in TCF government schools?

(II)What is the difference between Karachi and Lahore English writing proficiency test scores of
class 8th students in TCF government schools?

Hypothesis

1. There is no difference in overall English proficiency test scores of class 8th students
between Lahore and Karachi TCF government schools.
2. There is no difference between Karachi and Lahore English reading proficiency test
scores of class 8th students in TCF government schools
3. There is no difference between Karachi and Lahore English writing proficiency test
scores of class 8th students in TCF government schools?
In this study students’ test score is dependent variable and regions are independent variable.
While teaching method and teachers professional qualification is extraneous variable.

Purpose of the study: purpose of this cross-sectional survey is to find out the differences
between Karachi and Lahore TCF Government schools’ students of class VIII in performance in
English test.

Research design
Survey study method better fitted the requirements of this study. According to Fink and
Koseoff, (1998), survey is a way of gathering information from people about their ideas,
feelings, social and educational background, and beliefs. As Mujis (2004) narrated that “survey
research is well suited to descriptive studies, or where researchers want to look at relationship or
differences occurring in particular real life context” (p. 36). The main aim of this study was to
find the differences of two regions in English test scores. Since, this research required collecting
data at a single point in time therefore; cross-sectional survey design was chosen, because cross
sectional survey is one that produces a ‘snapshot’ of a population at a particular point in time
(Cohen, Monison, & Marrison, 2000). Furthermore, the cross -sectional survey method also
made the job lighter while collecting data as it is the quicker to conduct and cheaper to
administer (Fink and Koseoff, 1998).

2
Sampling:
Our sampling was convenient for this study which was made ready for us by our
facilitators to study the test scores of students in English proficiency test. As Brayman (2001),
stated that “A convenience sample is one that is simply available to the researcher by virtue of its
accessibility (p. 100)”. 62 class 8th students selected from Karachi and Lahore out of which 33
students selected from Lahore region and 29 from Karachi region. Out of 62, 49 were male
students and 13 were female. All students were tri-lingual. English was their third or fourth
language. All students were from the same socio-economic background. Most of the students in
Lahore speak three languages which are consisted of Punjabi, Urdu and English. However, in
Karachi students speak Urdu, Sindhi, Pashto and English.
Demographic detail of sample
Gender Karachi Lahore Grade
Male 16 33 Class 8th
Female 13 0 Class 8th
Total 29 33 ……………..

Measurement tool (English proficiency test)

Measurement tool (test) consisted of five parts with 17 test items with equal marks on
every part. Tool was designed to see the difference of performance in the reading proficiency and
writing proficiency levels of the students. There were twenty marks for the test. Each part has
four marks. So marks were equally distributed. First four parts were dealing the reading skills of
students and the last part was dealing the writing skill. There was no negative marking.

Reliability test
In order to check the reliability of the tool, through SPSS I run reliability test. Which
shows that reliability of the tool is (alpha=0.386) which is less than 0.7; this shows a weak
reliability of the tool used for testing the English proficiency of the students. Kline (1999) notes
that although the generally accepted value of 0.8 is appropriate for cognitive tests such as
intelligence tests, for ability tests a cut-off point 0.7 is more suitable.
Table 1.1

Reliability Statistics

3
 Cronbach's
Alpha N of Items

.386 17

Data collection process

It was face-to-face administrated test; we went into groups to respective schools. As
Gorard (2001) stated that “Face-to-face researcher can read the questions aloud, explain any
difficulty points if necessary, and record the responses in as much detail as desired (p. 82)”. The
children filled the test under examination set-up. Before distributing the test, it was introduced to
the children. Purpose of the test was explained and importance of their participation was also
highlighted. The same process of administration was used to avoid the unnecessary research’s
bias. The instructions were kept same for all the children to avoid possible influence on children
response. The instructions for the test were read to children and written examples were also
shared to guide the children how to answer the questions. All students were seated properly. As
we were in group of four administrators so we divided rows of students. There were 29 students
in our class.

Ethical consideration
Research ethics is the making and implementing of moral judgments about the aims and
methods of a study (Aubrey et al; 2000). Before conducting the survey all participants were told
that their data will be kept confidential. Participants also were informed about the test that they
will not be penalized on their mistakes in test by their teachers. Before sending us to the schools
our facilitators had already taken consent from the school administration.

Limitation:
As the sampling of this cross-sectional survey was convenient so the results of this survey
cannot be generalized because we do not know of what population this sample is representative.
And also sample was very small (62). As this analysis has been done by a novice quantitative
researcher there is a possibility that analysis may not reflect depth.
Data entry

4
 After data collection process, a chart for data analysis was developed on Statistical
Package for Social Scientists version 16.0 (SPSS 16.0). According to Gaur & Gaur (2006) SPSS
is the most reliable software for analyzing quantitative data. Responses were coded according to
the scoring scheme (correct=1, incorrect=0). Dummy variables were coded for categorical
indication; for example Lahore 1 and Karachi 0, we started entering the data as soon as the
papers were marked. After entering the data, it was checked for errors by running the
frequencies. We checked each variable to look for the value that fall out-of range of possible
values for a variable. Through frequency distribution I detected incorrect entries and rectified
them accordingly before carrying out further analysis.

Descriptive analyses of overall test scores of EPT1

After entering the data in SPSS I run descriptive statistics in order to find the range, mean and
dispersion of overall test scores.
Table: 1.2

Table 1.2 clearly shows that Karachi students have scored better (M=9.0) than Lahore
students (M=7.5) in terms of minimum marks both regions are at same level, but in maximum
marks Karachi has done better than Lahore. However, standard deviation shows that scores of
Karachi students are more dispersed (SD=2.9) than Lahore students (SD=2.37). In other words
there are more within group differences in Karachi as compared to Lahore students. It shows
interesting result that though in Karachi students have done better, their marks dispersion is also
more. It means that there are some students who are far better in English language than their
classmates.

Exploratory analyses for normality

1
English proficiency test

5
 In order to see whether data are normal or not I checked the skewness and standard error
of skewness through exploratory analyses of overall scores (see table 1.4). I also checked the
normality and skewness in Lahore (see table 1.5) and Karachi (see table 1.6).

Table 1.3
As histogram (fig: 1.3) is showing that data are normal, and bell curve also suggests that
data are normal. In order to check the skewness of data I also see the standard error of skewness
and value of skewnees. As the value of skewness is (0.076) which is less than the double of
standard error of skewness (0.304), it confirms that data are normal. (See Table 1.4)
Table 1.4
Variable Skewness Std: error of skewness Comment
Overall test scores (0.076) (0.304) Normal

Checking the normality of overall EPT scores in Lahore and Karachi

In order to see whether data are normal or skewed in both regions or not, I checked
normality through histogram (SEE 1.5 & 1.6). Histograms are showing that data in both regions
are normal. I also checked the skewness and standard error of skewness of both regions (see
table 1.7).

6
 Figure 1.5 Figure 1.6

REGIONS SKEWNESS STANDARD ERROR OF SKEWNESS REMARK

LAHORE -0.147 0.434 NORAMAL
KARACHI -0.41 0.409 NORMAL

Table 1.7

Table 1.7 confirms that data are normal in both regions, as values of skewness are less
than the double of standard error of skewness.

Inferential analyses of EPT overall scores

According to Kerr, Hall and Kazub(2002)When employing an independent t-test there are
two sets of variances and the assumption is that these are not significantly different(p.71). And
Levens test shows that variances in two groups are not significantly different (p>0.05) (Table
1.8). Therefore, independent t-test can be used. Independent T-test shows that the overall
difference in test scores between the TCF government schools of Karachi and Lahore in English
proficiency test is significant [t (60) = -2.09, p=0.041] (see table 1.8).
The overall test shows that students in Karachi TCF School have done better in their
English proficiency test; therefore, we would reject the first hypothesis which says that there is
no difference between Karachi and Lahore students’ English proficiency. We found that there is

7
significant difference in overall EPT test scores between Lahore and Karachi TCF Government
Schools’ students of class VIII.

Levens test
df
t Sig (two-tailed)
f sig

Total score
Equal variance assumed -2.09 60 0.041
3.54 0.065
Equal variance not -2.59 53.2 0.044
assumed

Table 1.8

Descriptive analyses of reading scores in EPT between Lahore and Karachi

In order to see the difference between Lahore and Karachi in reading proficiency, I found
the mean of the questions related to reading (Part one to Part 4).

Table 1.9

Table 1.9 shows that Karachi students have done better (M=8.206) comparing to Lahore
students (M=7.2121) in their reading proficiency test scores. However, standard deviation shows
that Karachi students’ scores are slightly dispersed (SD=2.366) as compared to Lahore
(SD=2.161). It reveals that some students in Karachi have scored a bit higher as compared to
their classmates. While there is sameness in Lahore reading proficiency test scores.

Exploratory analyses for normality of reading scores in EPT

8
 In order to see whether distribution data are normal or skewed, I checked the normality of
both regions through exploratory analyses. (See Table 2 for Lahore: table 2.1 for Karachi)

Table 2 Table 2.1

variable regions skewness Standard error of skewness comment

Reading scores Lahore 0.84 0.409 normal
Karachi -0.161 0.434 normal

Table 2.2

Table 2.2 shows that in both regions data are normal. As values of skewness are less than
the double of standard error of skewness.
In order to see the overall normality of scores in reading test scores between Lahore and
Karachi I also checked overall normality of scores (see Table 2.3).

9
 variable skewness Standard error of skewness remark
Reading score 0.016 0.304 normal
Table 2.3

As (table 2.3) shows that as values of skewness is less than the double of standard error
of skewness it confirms that overall data are normal.

Inferential analyses of reading scores in EPT

In order to see whether this difference in reading is significant or not I run independent t-test.

Table 2.4

Levens test for

equality of variances df
t Sig (two-tailed)
f sig

Mean score
Equal variance assumed -1.73 60 0.09
0.222 0.639
Equal variance not -1.72 57.207 0.09
assumed

10
 Table 2.4 shows that this difference is not significant [t (60) =-1.73, p>0.05]. Therefore,
we fail to reject the second hypothesis which says that there is difference between Karachi and
Lahore reading proficiency test scores.

Descriptive analyses of writing test scores of Karachi and Lahore in EPT

In order to see the difference between writing proficiency test scores, I found the mean
and standard deviation of test scores between Karachi and Lahore.

Table: 2.1

Table 2.1 shows that Karachi students have done better (M=0.8) than Lahore students
(M=0.36) in their writing proficiency test scores. However, standard deviations shows that
Karachi writing test scores are more dispersed (SD=1.54) as compared to Lahore (SD=1.16). It
means that some students have scored higher marks in writing proficiency and some have scored
lower marks.

Exploratory analyses of writing scores in EPT

Through exploratory analyses I checked the normality and skewnes of the data in both
regions. Both in Lahore and Karachi data are positively skewed (See Tables 2.2 and Table 2.3).
According to Antonius (2003), “Normal distributions are a very important and common pattern
that occurs in a wide range of distributions of quantitative variables (p. 105).”

11
 Table 2.2 Table 2.3

variable regions skewness Std:of skewness Comment

Writing scores Lahore 2.983 0.409 Skewed/not normal
Karachi 1.622 0.434 Skewed/not normal
Table 2.4

As table 2.4 shows that value of skewness is more than the double of standard error of
skewnes in both cases, so we can conclude by saying that in writing test scores data are not
normal. I also checked the overall normality of writing scores, which also confirms that data are
not normal. (See table 2.5)

12
 variable skewness Std:error of skewness Comment
Writing score 2.130 0.304 Not normal/skewed
Table 2.5

Inferential analyses of writing test scores in EPT

In order to check the homogeneity of variances, I run independent t-test, as Field (2005)
stated that “Independent t-test is used to test different groups of peoples, so it also assumes that
variances in these populations are roughly equal (p. 287)”.

Levens for variance equality t-test for equality for means

F Sig t df Sig(two tailed)
Mean score equal variance assumed 5.043 0.028 -1.24 60 0.218
Equal variance not assumed
-1.222 51.775 0.227
Table 2.6

Levens test clearly shows that the variances distributions is significant (p=0.028) which
is less than (p=0.05). This shows that we are not supposed to use independent t-test. In order to
see whether the differences of the means in writing proficiency is significantly different or not
between the two regions we would use the Mann-Whitney test.

13
 Figure 2.7

Mann-Whitney test shows that the difference in writing proficiency test scores between
Karachi and Lahore TCF schools is not significant (Mann-Whitney U=409.500, p=0.127).

Therefore, we failed to reject the third hypothesis which says that there is no difference between
Karachi and Lahore English writing proficiency test scores of class 8th students in TCF
government schools. We found that the difference in writing proficiency test scores is not
significant.

Conclusion and Discussion

There are few test items (see table 2.8) in which Lahore has done well as compare to
Karachi, and the difference was found to be significant as well, while in one test item (part 2
item 3) Lahore did not score(See appendix A).

Test items Lahore Karachi t-test

mean SD Mean SD
Part 2 question 1 0.93 0.24 0.65 0.48 0.004
part 2 question 3 0.00 0.00 0.55 0.506 0.001

part 3 question 2 0.789 0.51 0.517 0.508 0.025

Table: 2.8

Table 2.8 shows that in part 2 question 1 which is reading comprehension Lahore has
scored (M=0.93) better than Karachi (M=0.65). Moreover standard deviation also shows scores
of Karachi students’ scores are more dispersed than Lahore (Lahore SD=0.24, Karachi=0.48).
While t-test also shows that this difference is significant (p=0.004). Surprisingly in part 2

14
question 3 Lahore students did not score, so there is significant difference in scores (p=0.001). In
part three question 2 which is a reading comprehension item Lahore students scored more than
Karachi students (Lahore M=0.789, Karachi M=0.517). It is also clear from the table that
Karachi students’ scores are more dispersed as compare to Lahore students (Lahore SD=0.415,
Karachi SD=0.508). It shows that few students in Karachi scored higher marks as compare to
Lahore students and difference also was found to be significant (p=0.025).
Overall results show that Karachi students overall have done better than Lahore students
(Karachi, M=9.00, Lahore, M=7.57), while Karachi students’ scores are more dispersed as
compare to Lahore (Karachi SD=2.97, Lahore SD=2.3).This difference was found to be
significant [t (60) = -2.09, p=0.041]. In reading and writing Karachi also has done better than
Lahore. (Karachi (reading) M=9.00, (writing M=0.793), Lahore (reading) M= 7.5 (writing) M=
0.3), however this difference was not found to be significant (reading t (60) =-1.73, p>0.05) for
writing (U=409.500, p=0.127). It means in Karachi there is a hierarchy of proficiency in English
in which some students are at lower level and some are at higher level. While in case of Lahore
students are more or less similar in terms of proficiency.
As I mentioned earlier that in Lahore students were tri-lingual as their mother language
was Punjabi their national language was Urdu and English was their third language. If we
compare Lahore students with Karachi most of the Karachi students were Urdu speaking so
English was their second language. And continuous dispersion in Karachi scores is evidence of
this fact that Urdu speaking students had been doing better as compare to their colleagues,
because they have only two languages to deal with English and Urdu. As in Karachi there are
few students whose native language was Pushto, Urdu and Sindhi.
Other factors also can influenced the test, according to Bachman(2004), “ A variety of
factors , such as topical knowledge, affect, cognitive style and anxiety, in addition to the
language ability we want to measure, may affect language test performance (p. 28).
Interesting thing is that though overall there is significant difference between Karachi and
Lahore scores but there is no significant difference in reading and writing scores. Table 2.8
shows that in part 2 and 3 Lahore has done better than Karachi and difference is also significant,
so there is a possibility that these findings might have neutralized the overall reading and writing
scores between Lahore and Karachi. There is a possibility that Karachi teachers are more
professionally trained than Lahore teachers.

15
RECOMMENDATION FOR THE TOOL
Most problematic part for the students was part two. In part two there was no key
mentioned to interpret the information given in the table (see appendix A), and most of the
students could not understand the table. In future it can be very good for students to mention key
before giving them table to interpret.

16
REFRENCES
Antonius, R. (2003). Interpreting quantitative data with SPSS. Sage publications: London

Aubery, C., David, T., Godforey, R. &Thompson, L. (2000).Early childhood educational

research: Issues in methodology and ethics.London: Routledge/Falmer press

Bachman, L.F. (2004). Statistical analyses for language assessment. Cambridge University
press.

Bashiruddin, A. (2003). Learning English and learning to teach English; the case of two
teachers of English in Pakistan. Unpublished doctoral thesis, Ontario institute for studies
in education of the University of Toronto.

Brayman, A. (2001). Social research methods (2nd Ed.) Oxford University press

Cohen, L., Manison, L., & Marrison, K. (2000). Research method in education: (5th Ed.) New
York Rutledge Falmer.

Field, A. (2005). Discovering statistics using SPSS. (2nd Ed) sage publication

Fink, A., & Koseoff, (1998). How to conduct survey: A step by step guide: United Kingdome:
Sage publication.

Gorard, S. (2001). Quantitative methods in educational research: the role of numbers made easy
Continuum London

Green, S. B., & Salkind, N. J. (2008). Using SPSS for Window and Macintosh: Analyzing and
Understanding data (5th Ed.). Upper Saddle River, NJ: Pearson Prentice Hall

Gaur, A. S., & Gaur, S. S. (2006). Statistical methods for practice and research: A guide to data
analysis using SPSS. New Dehli: Sage publications.

17
Kerr, W, A. Hall, K, H. & Kozub, S, A. (2002).doing statistics with SPSS. Sage publications
London

Kline, P. (1999). The handbook of psychological testing (2nd Ed). London: Routledge.

Muijs, D. (2004). Quantitative research in education: Sage Publication: London.

Siegel, S., & Castellan, N.J. (1988). Nonparametric statistics for the behavioral sciences
(2ndEd.). New York, NY: McGraw-Hill.

18
APPENDIX A

19