NATALIA GONZLEZ ABURTO

THEORETICAL
FRAMEWORK

ASSESSMENT QUALITIES ON SLA

Theoretical framework
When talking about assessment terms like evaluation or testing come to mind and that
is because they are all terms involved in the process of learning, which is why it is
important to tell the difference between each one in order to understand how it affects
the learner.
First, evaluation is more focused on making a judgment or determination concerning
the quality of a performance, work product or use of skills against a set of standards.
Evaluations are designed for and intended to document the level of achievement that
has been attained. (Center for School Success, 2011)
Now, according to Brown (2004), tests differ from assessment as tests are a subset of
assessment, which means that they are just another way to assess. They are one of
the many existing assessment tools.
Thus, Brown (2004: 4), states that tests are prepared administrative procedures that
occur at identifiable times in a curriculum when learners muster all their faculties to
offer peak performance, knowing that their responses are being measured and
evaluated. And assessment, on the other hand, is an ongoing process that
encompasses a much wider domain. Whenever a student responds to a question,
offers a comment, or tries out a new word or structure, the teacher subconsciously
makes an assessment of the students performance.
Since the definition of assessment has been understood, the next step for teachers is
to comprehend the principles of language assessment. To design good assessment
tools, teachers should pay attention to some test qualities that will be further explained
below.
Brown (1994: 3) refers to a test as a method of measuring a persons ability or
knowledge in a given domain.

As it is a method it is a set of techniques and procedures that measures a persons

abilities, knowledge, competence and a given domain for which it is necessary to have
a set of criteria that make that measurement to be good, which are the following:
Practicality: Brown (2004), Bachman (1996), and Coombe et al. (2007) agree on
practicality being the ease and convenience of administration of a test, in terms of
material resources and time (for the testing situation and also for checking). For
example, a test may take 10 minutes to be completed, but 1 hour to be checked; in this
case, this test is impractical. Bachman (1996), particularly, makes a differentiation
between practicality and the rest of the test qualities. For this author, this quality is the
only one that is not related to the use of the test scores, but has only to do with the
conditions under which the test will be implemented, and the relationship between the
resources used and those that are available at the testing situation. Coombe et al.
(2007) point out that some of the aspects of practicality depend on the educational
institution, others on the teachers, and others on external factors.
Brown (2004) defines a reliable test as a test that will basically have the same results if
it is given to two similar sets of students or the same set on different occasions,
Bachman (1996), quite similarly, defines reliability as the consistency in results when a
test is applied under different testing and scoring conditions. We can differentiate two
types of reliability: Test reliability, which was already mentioned above; and also scorer
reliability, which refers to the consistency in results when a test is checked by more
than one person, and how subjective (or objective) the marking criteria are (Brown,
2004). Coombe et al. (2007) describe three types of fluctuations in the results of a test
due to high or low reliability: Fluctuations in the learner (how differently a learner will
perform at different test-taking situations), fluctuations in scoring (how differently two or
more scorers will grade a students performance based on the same test), and
fluctuations in test administration (how some aspects of test administration affect the
scores).

Validity: Brown (2004: 22) defines validity as the extent to which inferences made from
assessment results are appropriate, meaningful, and useful in terms of the purpose of
the assessment. A test is valid when it measures what is intended to measure. For
example, a literature essay is more intended to measure comprehension and analysis
of a literary piece, so it would be wrong to focus more on grammar than the actual
interpretation and ideas about the text (Brown, 2004). Some of the features that make
a test highly valid may be: a clear, and familiar format and test tasks, a reasonable
amount of time to complete the test, clear instructions, a reasonable challenge
presented by the difficulty of the test (Brown, 2004: 27). Coombe et al. (2007) adds that
the approach of teaching that is being used will also be a factor in validity. Bachman
(1996), on the other hand, refers to validity as the degree to which the interpretations
made about a test scores correspond to the aims of the test.
According to Bachman (1996: 19), simply using a test does not make it useful.
Bachman argues that the primary purpose of tests is to measure; and that two of the
qualities (reliability and validity) that we will see next, are critical for tests as they are
seen as essential measurement qualities.
Bachman (1996), states that test usefulness provides a metric by which all aspects of
test development and use can be evaluated.
Whereas Bachman (1996), defines usefulness as the main concept defined by all the
other test qualities, Coombe et al. (2007), see it as the first, and most important test
quality. A specific purpose, particular group of test-takers and a specific language use
need to be kept in mind when designing a test with high usefulness.
Bachman (1996) proposes that a model of test usefulness should include, apart from
the ones already mentioned from this authors work, the following qualities:
Authenticity: Bachman (1996), introduces the quality of authenticity and describes it as
the degree to which the test tasks reflect real life target language use (TLU) tasks. In

other words, an authentic test is a test that has tasks that represent situations of
language use in real life. For example, a multiple choice language test is not very
authentic if we are assessing communication skills. Coombe et al. (2007), point out that
this is important in order to expose students to the language in a more meaningful way,
with activities that mirror real life situations.
Interactiveness: Another quality introduced by Bachman (1996), is interactiveness,
which refers to the way in which the test engages the test-takers and requires them to
use and connect their different areas of language knowledge: language ability, topical
knowledge and affective schemata. No explicit references to interactiveness were
found in Brown or Coombe et als papers; however, this quality might be considered as
part of one of the qualities that these authors propose.
Impact: According to Bachman (1996) tests have impact at two levels: a micro level,
which refers to those individuals who are affected by the results of the test (for
example, test-takers of a standardized or placement test to enter a university course);
and a macro level, which refers to the impact that tests have on society and
educational systems (for example, if the long-term consequences of a university entry
test determine that the people who enter a certain university course are not the most
suitable for that course.) None of the other authors mention this test quality, however, if
we consider that impact refers to the information that assessment gives to those who
take part in it in order to make future decisions on education and further assessment,
Coombe et al. (2007) introduce a concept that could be an equivalent: Washback,
which is defined as the effect that testing has on teaching and learning. This affects the
students and the teacher, it gives information on what is being done well and what
needs improvement from both sides. Finally, it is important to highlight not only what
needs improvement, but also what objectives have been met, so that the students have
a sense of accomplishment.

The last two test qualities to be mentioned are introduced by Coombe et al. (2007) and
are not really mentioned by the other authors.
Transparency: It refers to the availability for the students of information about
assessment: expected outcomes, assessment criteria, time given to complete the test,
how much each test item weights, among others. If a test is transparent, it will involve
the students more into the assessment process (Coombe et al., 2007).
Security: The last test quality of this model is security, and it is defined as part of
reliability and validity. This test feature has to do with the need for recycling tests and
not letting them be a part of the public domain in order to avoid the passing of
information from one year to the next (for example, if a student keeps a test from a
previous year and gives it to another student in the future, and they find out the exact
same test is applied each after year with no modifications). (Coombe et al. 2007)
As a conclusion, it can be said that there are many factors to consider when designing
any assessment tool. Some of the test qualities mentioned above are related to the
interaction between the test and the student (interactiveness), or between the test and
real life language use (authenticity), and some others consider the resources and the
place and time of the test-taking situation (practicality); however, all of them are equally
important depending on what we require from the test, and what we aim to accomplish
with our assessment.

References
Bachman, L. F., & Palmer, A. S. (1996). Language testing in practice: Designing and
developing useful language tests. Oxford University Press, USA.
Brown, H. D., & Abeywickrama, P. (2004). Language assessment: Principles and
classroom practices. New York: Longman.
Brown, H. D. (1994). Teaching by principles (pp. 152-157). Englewood Cliffs, NJ:
Prentice Hall.
Center

for

School

Success.

(2011).

Retrieved

from

http://www.centerforschoolsuccess.org/index.php?
option=com_content&view=article&id=86&Itemid=150
Coombe, Ch., et al. (2007). A Practical Guide to Assessing English Language Learners.
Michigan: University of Michigan Press.