Feras Fanari

Action Research Report

Aug 2015
Department of Levantine (B4) - UMA

ICPT Effectiveness

Student survey was conducted online (SurveyMonkey).

Gathered information from four classes:

Class AP00614 (End of semester three).
Class AP55314 (End of semester three).
Class AP00613 (MSA Post DLPT).

Individual respondents are not identified in reporting survey findings. All of

the survey's results are presented in anonymous summary, such as
statistical tables, comments and charts.

Identify the problem:

Some Levantine students failed the 301 ICPT test. Therefore, I am trying to
identify the problems that the students are encountering while taking this
test. This will help me look for better methods and resources to better
prepare them for this type of language proficiency exam in the future so that
they all achieve great results. Additionally, it will shed more light on some
issues with the format of the ICPT test itself that might be troubling to the
students.

Feras Fanari
Action Research Report

Aug 2015
Department of Levantine (B4) - UMA

Q1- How well were you prepared for the ICPT examination?

Q2- How much authentic listening and reading material was given to you in
the classroom to prepare for ICPT?

Q3- I was introduced to GLOSS and / or JLU language training web

portals:

Feras Fanari
Action Research Report

Aug 2015
Department of Levantine (B4) - UMA

Q4 - I utilized GLOSS / JLU materials to prepare for ICPT tests.

Q5- What authentic resource was presented or utilized the most?

Q6- Of all of the above mentioned authentic resources, which one benefited
you the most? Explain your answer.
3

Feras Fanari
Action Research Report

Aug 2015
Department of Levantine (B4) - UMA

In hindsight gloss and JLU

The passages followed by multiple choice questions. The passages on the ICPT were not difficult
to understand, the difficulty primarily comes from the questions, which would be tricky to
answer even if the passage was in English.
For reading - Self study of packets, the book, and news Listening - Review of old material and
documentaries and interviews. I listen to a lot of MSA/Dialect mixed videos and it helps a lot,
because higher level test material uses a lot of more official MSA words.
News Articles
I dont know
A combination of the two was beneficial. JLU was probably more beneficial than GLOSS in my
opinion. Reading and listening to authentic material was helpful as well. Be sure to retain
vocabulary you don't know when covering this material, otherwise it will be lost.
JLU benefited me the most, especially for the later ICPTs, and especially the online assessments.
Gloss
JLU/Gloss These resource were very helpful, but they were given to us in semester 3
JLU
Ajyal, because its cultural and they speak fast enough to challenge a person without wanting to
tear their hair out.
I like the short clips of Roya and other series due to authentic usage vocab.
JLU
I really enjoy watching drama series on mtv lebanon and it helps with authentic listening and
also encourages me to look up more words, and the same goes for reading BBC Arabic and using
Quizlet vocabulary online. Also I found that going through and reviewing the listening passages
from old units was very helpful.
MTV Lebanon. There's a show on there that I like and can understand for the most part.
I think they all benefitted me, however Ruya TV and some of the Maraya series where the most
interesting. JLU was good material but a little boring.
News articles

Feras Fanari
Action Research Report

Aug 2015
Department of Levantine (B4) - UMA

Q7- Of all of the above mentioned authentic resources, which one benefited
you the least? Explain your answer.
Ru'ya was not used
Arabic TV series, because the vocabulary is very cultural and different from what is encountered
on the tests.
The listening from the curriculum that is too unclear to understand the letters. I can't hear the
difference between a and and don't feel some of them are helpful at all because even the
teachers have trouble understanding without consulting the scripts. VERY few ICPT passages
had poor audio quality, maybe 3-5/50
MTV Lebanon
I dont know
GLOSS was not as helpful for me. I liked how JLU simulated the ICPT with multiple choice
questions. However, the timer can be good or frustrating depending on what you are trying to
accomplish.
GLOSS was not beneficial in the beginning of the course.
Levantine movies. I'm an MSA student.
Levantine drama movies.
Gloss
Gloss and JLU. There are way too many glitches and mistakes in the system for it to be an
effective source for studying.
No comment.
Ajyal... I don't think we used this. If we did I was unaware of its name.
I stopped using the things that benefited me the least, like JLU and Gloss. But now we use JLU
in class.
The videos with poor quality that I couldn't understand at all.
JLU b/c I can do that on my own without a teacher
The ones not provided or mentions

Feras Fanari
Action Research Report

Aug 2015
Department of Levantine (B4) - UMA

Q8- Did you have a problem or difficulty in understanding the Arabic or the
English format in ICPT tests?

Comments (total of 9):

1- When I had difficulty understanding the Arabic, it was most likely due to insufficient
vocabulary or audio passages were too quick and I couldn't remember all the information
I heard. In English, sometimes it's hard to know what the question is asking... you may
hear something in a passage, but you're unsure if that properly correlates to what the
English is trying to say.
2- It depended on the test, for some of them the English questions were more
straightforward, but for a lot of them, especially the later ones, even if I could completely
understand the Arabic, the English questions were very confusing, and did not always
seem to be the best way to test the Arabic comprehension.
3- The English questions were designed to trick SM's, including using language that is not
commonly used even at the college level.
4- The English is designed to confuse the student instead of test them on their understanding
of the target language. There have been instances where the English vocabulary is even
too advanced for me, so I don't know what the question is asking.
6

Feras Fanari
Action Research Report

Aug 2015
Department of Levantine (B4) - UMA

5- English answers don't always seem to match up with what I hear, which makes choosing
an answer difficult. This could be a problem with my listening.
6- It wasn't so much the format as it was the content in some cases.

7- Sometimes the English questions don't make much sense in connection to the Arabic
passage.
8- I don't think it is fair when the student has to choose between the "more right" and the
"less right" answer. It should either be right or be wrong. I found myself wasting time
trying to decide between 2 answers on several questions in which both of the answers
seemed could be correct.
9- The wording of the questions in English is not natural.
Q9- What was the most challenging aspect of the ICPT test? Explain.
13 Responses:
1- Tricky questions, in which all answers are present in the passage, and choosing the
correct answer depends on interpreting the subtleties in the same way as the question
writer. This can present additional difficulty if the author was not a native English
speaker who may not be aware of the connotations of some words, which may introduce
confusion.
2- I dont know
3- The most challenging aspect of the ICPT (in the later levels 301/302) is time
management, particularly for the reading. It's also difficult to prepare for these tests
because it is based on your cumulative knowledge. You can't just study one particular
subject.
4- For reading, finishing in the time allotted for the test. For listening, only listening once
for the lower level passages, and only having a very short break in between for the higher
level passages.
5- Having the ability to only listen twice.
6- The fact that it sometimes seems like there is more than one right answer.
7- Similarity of some of the answer choices
8- The way the questions are worded. Sometimes, they are indecipherable.
9- Listening. I struggle with listening. I have to focus intensely to grasp the understanding of
whole sentences, even if I know many words individually. The level 2/2+ area was the
most difficult for me.
10- When there is more than one correct answer, or one that is so close to being correct but it
is less correct than the other. That is the frustrating part.
7

Feras Fanari
Action Research Report

Aug 2015
Department of Levantine (B4) - UMA

11- The time. It was sometimes hard to stay focused throughout the whole test.
12- Its just a tough test, the listening is difficult because you can only listen twice, and the
reading because there isn't enough time.
13- The English questions because of the way the answers are written, since it's still not a
natural way.

Q10- Which type of test (unit vs. ICPT) helped you better understand your
language learning strengths and weaknesses?

Comments (total of 9):

1- Going over questions you got wrong is the best way to learn.
2- They both helped, I suppose. The unit tests require many different types of skills
including translation, transcription, comprehension questions, etc..) this is good for
seeing where your weaknesses are. The ICPT is a better indicator of how you will
perform on the DLPT however. The unit tests are better for testing your ability in a
particular subject matter. However, the ICPT will test your ability to have a conversation
during the OPI.
3- The unit tests are more straightforward tests of the language comprehension, whereas the
questions on the ICPT can be confusing, or they can hint toward an answer that I would
not have otherwise been able to understand.
4- The unit tests are geared towards comprehension as opposed to being able to distinguish
one specific word out a 2/2.5 minute passage
5- I feel that the unit tests better assessed my abilities however the ICPT helped prepare for
the DLPT
6- We actually get feedback on the Unit Tests. We are allowed to review the answers and
understand why we got them correct or incorrect.
8

Feras Fanari
Action Research Report

Aug 2015
Department of Levantine (B4) - UMA

7- The unit tests were weighted differently for each unit. Often times, listening would be
easy for a unit and reading would be more difficult. The next unit test would be opposite.
I would find reading to be easier and listening to be more difficult. TTT was also a factor
that was hard to predict success for struggle.
8- I believe that the unit test is more demanding; it wants you to answer in short-answer
form rather than multiple choice. There is no guessing with a 25% chance of getting the
answer correct in the Unit Tests, either you understood it or you didn't. That showed me
exactly what I needed to look more into. However, the ICPT breakdown sheets of our
grades and the total missed or total correct in each Level is very helpful to me, so now I
know which levels of content I need to focus and work on.
9- I didn't like the Unit test because it asks for very specific things sometimes and not
necessarily a general idea or meaning. I also don't like having to handwrite answers while
I'm listening, its hard to write in English and listen to Arabic at the same time. Overall I
enjoyed taking the ICPTs more.
Q11- What do you suggest to be done differently to better prepare you for
the ICPT?
13 responses:
1- Use past ICPT questions as practice. Going over the ICPTs so we can learn why our
answers were wrong is the best way to learn.
2- Nothing
3- The best way to study for the ICPT is to immerse yourself in news and various multiple
choice exercises.
4- Multiple choice passages, especially with questions similar to the style used on the ICPT,
would be very helpful, to get used to the format of the tests, and so that it is more about
the language comprehension than about if the questions are confusing or not.
5- In addition to comprehension materials, having multiple choice materials will better
prepare SM's for the style of the ICPT and DLPT
6- More authentic materials earlier in the course.
7- Mock DLPT materials.
8- There is no preparing.
9- Reviewing past vocabulary, focusing heavily on the level 2 area. Since this is the level
that is most important in achieving a desirable grade in both the ICPT's in semester 3 and
on the DLPT. Most people seem to struggle with that level but do just fine in lower level
9

Feras Fanari
Action Research Report

Aug 2015
Department of Levantine (B4) - UMA

material and most of the time better in the 2+/3 area (since that's the materials we are
covering in class. But the middle gets lost and I think that reviewing it would really help.
10- Give us snacks like goldfish and fruit snacks. Also nap time would be nice. But really, I
think what the teachers do is sufficient, and all of the authentic material is really helpful
considering a good chunk of the test is authentic and in some cases taken right out of an
old school tv show.
11- More time to prepare and more authentic material.
12- I also think some of the materials presented in class could be reviewed for accuracy.
Definitely we should only listen to passages 2 times just like on the ICPT. Any more than
that and the teacher should listen to it with the class and explain the passage. Another
suggestion is that we try to focus on certain topics on certain days. We are exposed to so
many different materials and vocabulary but it may help me if it was categorized
/organized into subjects/ topics. Its hard for me to remember all higher level vocabs
when its not organized into topics.
13- More review on basic vocabulary from semester 1 and early semester 2. I would've done
more level 2 stuff.

Plan of Action Summary:

After reviewing the student input from this survey, I have the
following suggestions:
1- Some students expressed that getting feedback on the unit tests was very helpful to their
language learning experience, since they were allowed to review the answers and learn from
their mistakes. Therefore, I recommend using past ICPT questions as practice and review.
Many students expressed that going over previous ICPTs can help them learn why their
answers were wrong. This would be a good method to learn.
2- Exposing the students to Mock DLPT style authentic materials with multiple choice
exercises towards the beginning of semester II. Having multiple choice materials will better
prepare the students for ICPT / DLPT style of questions.
3- Based on student input regarding the English question format of the advanced ICPT tests, I
recommend a re-evaluation of the ICPT questions. A majority of students complained that the
English is not natural to them, confusing and seems to be written by non-Standard American
10

Feras Fanari
Action Research Report

Aug 2015
Department of Levantine (B4) - UMA

College educated test writers. Some other students expressed that there is a loss of relevance
between the Arabic text and the English questions. I polled the opinion of some faculty
members regarding this issue and they believe that this discrepancy might be due to
translation flaws or a misinterpretation which might have led to an inaccurate formulation of
questions in relation to the Arabic text. In such case, it would be a possible option to task an
education linguist specialist who is well versed in both, American English and Arabic to
review the ICPT questions. Several faculty members at UMA were educated in American
universities, lived in the United States and were immersed in the American culture long
enough and therefore, are qualified to do such task.
4- Consider a reduction in the total number of unit tests to balance the number of ICPT tests,
especially in the 3rd semester. Several students indicated that the ICPT tests were a better
preparation tool for taking the final DLPT. On the other hand, some other students preferred
unit tests as an indication of their language learning capabilities.

11