TFR Guide Assessment Glossary 2009-08-01 TVT

Assessment Glossary
The Follow-Up
Report
Assessment,
Research, and
Planning Glossary
(A Guides and Handbooks Report)
Compiled by Todd V. Titterud

Revised 08/01/2009
The Follow-Up Report: Guides and Handbooks 1

Assessment Glossary
Assessment, Research, and

Planning Glossary (Compiled by Todd V. Titterud, 05/21/2007, Revised: 08/01/2009)
There are numerous glossaries available online covering technical and common terms prevalent in the fields of
assessment, evaluation, planning, administration, research design, statistical analysis, and testing. This
glossary was compiled from materials originally downloaded for personal use from many different sources over
several years and citations for the original sources are not available. While the glossary is neither definitive nor
all inclusive, it does combine terms and their usage from all of these interrelated fields into a single reference.
Much of the terminology in these fields is fluid and evolving rather than standardized. Many terms have
alternate definitions and there are multiple terms with similar or equivalent meanings and usage. It is often
recommended that institutions establishing their own cyclical regime of institutional research, assessment, and
planning develop their own glossary of adopted definitions for key terms in the process.
|A|B|C|D|E|F|G|H|I |J |K|L |M|N|O|P|Q|R

|S|T|U|V|W|X|Y|Z|
A
A-B-A design
A single-case experimental design in which the response to the experimental treatment condition is compared to
baseline responses taken before and after administering the treatment condition
A-B-A-B design
An A-B-A design that is extended to include the reintroduction of the treatment condition
A priori codes
Codes that were developed before examining the current data
AA or AS degree requirements
The areas in which students must demonstrate achievement in learning to earn an AA or AS Degree; generally
demonstrated through successful completion of required and elective courses
Ability
Present or potential capacity of an individual to perform a task or to use skills, including ones that are intellectual
and physical
Ability or trait parameter

In item response theory, a theoretical value indicating the level of a participant on the ability or trait measured by
the test; Analogous to the concept of true score in classical test theory

Assessment Glossary
Ability testing
The use of standardized tests to evaluate the current performance of a person in some defined domain of
cognitive, psychomotor, or physical functioning
Absolute decision
Decision based on the absolute level of performance in reference to a criterion; A performance decision which is
not norm referenced
Abstract
A brief summary of what is in an article
Abuse
The use of an existing authority for purposes that extend beyond or contradict the intentions of the grantors of
that authority; The furnishing of excessive services to beneficiaries of government programs, violating program
regulations, or performing improper practices which do not involve prosecutable fraud
Academic aptitude test

Test which predicts achievement in academic pursuits; Design needs to minimize the effect of exposure to
specific materials or courses of study on the examinee's score
Academic program review

The periodic peer evaluation of the effectiveness of an educational degree program; usually encompassing
student learning, faculty research, scholarship, and service, and assessment resources
Access
The ability to gain the attention and to influence the actions of key decision makers
Accessible population
The research participants who are available for participation in the research
ACCJC
Accrediting Commission for Community and Junior Colleges
Accommodations or adaptations
Modifications in the way assessments are designed or administered so that students with disabilities or limited
English proficiencies can be included in the assessment; A reasonable modification in an assessment
instrument or its administration made to compensate for the effects of a qualified disability without altering the
purpose of the assessment instrument; For example: Braille forms or tests in native languages
Accountability
Responsibility for implementing a process or procedure, for justifying decisions made, and for results or
outcomes produced; Use of results for program continuance or discontinuance; Public reporting of student,
program, or institutional data to justify decisions or policies; Using results for determining funding; Demand by
community stakeholders (public officials, employers, taxpayers, donors, investors, etc.) for educational
institutions to prove that the money invested in education has led to measurable learning; The responsibility
placed on an individual or group for their own or others actions, conduct, performance, projects, etc.; The
political principle that agencies or organizations are subject to some form of external control, causing them to
give a general accounting of and for their actions; The demonstration or proof to external constituencies that the
programs, services, and management of an institution are responsible and effective
Accounting
The process of classifying, measuring, and interpreting financial transactions

Assessment Glossary
Accreditation
A voluntary process involving an association of schools and/or colleges to encourage high standards of
education; Accreditation indicates that the Commission judges that the institution, in a manner consistent with
Commission standards, offers its students on a satisfactory level the educational opportunities implied in its
objectives and is likely to continue to do so; A status awarded by a certification agency to a candidate that has
demonstrated compliance with the standards set forth in the certification program; Certification awarded by an
external, recognized organization, that the institution or program meets certain requirements overall, or in a
particular discipline
Accrediting body
A voluntary, non-governmental association established to administer accrediting procedures; A listed accrediting
body is one that is officially listed by the Secretary of Education because it is used as part of the Department of
Education’s processes for determining institutional eligibility for certain federal funds
Accrediting or Accreditation Liaison Officer (ALO)

The individual in an institution assigned to carry on continuing relations with an accrediting body
Acculturation
The process of adapting to a new culture
Accuracy
Used in survey research to refer to the match between the target population and the sample; Degree to which
the data and information collected about the performance of a student are correct measures of performance and
are free from error; Extent to which an evaluation conveys technically adequate information about the
performance and qualifications of a student; Extent to which an evaluation is truthful or valid in what it says
about a program, project, or material
Achievement
Ability to demonstrate accomplishment of some outcome for which learning experiences were designed; What a
student has learned as a result of formal instruction
Achievement or proficiency levels

Descriptions of student or adult competency in a particular subject area, usually defined as ordered categories
on a continuum, often labeled from "basic" to "advanced," that constitute broad ranges for classifying
performance
Achievement test or testing

Assessment method designed to measure student competency or acquired knowledge, skills, attitude, or
behavior in relation to specified learner expectations; An assessment that measures a student's acquired
knowledge and skills in one or more common content areas (for example, reading, mathematics, or language);
A standardized test designed to efficiently measure the amount of knowledge and/or skill a person has acquired,
usually as a result of classroom instruction; May be used to produce a statistical profile used as a measurement
to evaluate student learning in comparison with a standard or norm; A test to evaluate the extent of knowledge
or skill attained by a test taker in a content domain in which the test taker had received instruction; An objective
examination that measures educationally relevant skills or knowledge about such subjects as reading, spelling,
or mathematics
Acquiescence bias
Bias created when respondents in a survey answer positively to two equally conflicting questions
Acquiescence response set

The tendency to either agree or to disagree
Action learning

Assessment Glossary
Involves a group of people coming together regularly to work collectively on problems in their community,
workplace or project in order to achieve improvement by helping each other learn from their experiences;
People involved in action learning increase their skills and knowledge by learning with and from each other,
working on real issues or problems, reflecting on their experiences, and taking action based on group decisions;
Values associated with action learning include respect for others, honesty and integrity, collaboration, and
developing trusting relationships with others
Action plan
A detailed description of the strategies and steps used to implement a strategic plan; Tactics describing who,
what, when, where and how activities will take place to implement a strategy
Action research
A multi-stage type of research in which a problem is researched, changes are made, the problem is researched
again, more changes are made, and so on through a number of cycles, until the problem is solved; Applied
research focused on solving practitioner’s local problems; Collecting data in the field to allow for continuous
adjustments and improvements; Targeted research done to facilitate immediate decision-making; Goal oriented
research done to support effective change; Research that allows the researcher to take an active role as an
involved participant in the process of planning and implementing changes; School and classroom-based studies
initiated and conducted by teachers and other school staff in an effort to improve classroom instruction and
educational effectiveness; Research addressing both the practical concerns of people (in a community,
organization, etc.) and the goals of research through people working together on projects; A flexible method of
integrating research into projects, involving community participants and generating action which is essential for
good planning and development; Effective action research depends on the agreement and commitment of
participants; Useful for working on complex social problems or issues that need systematic planning and
analysis; Involves an ongoing cycle of planning, acting, observing, reflecting (and back to planning) involving
critical reflection as an important step in each cycle; Tends to be cyclic (similar steps tend to recur in a similar
sequence so that actions are responsive to emergent learning), participative (clients and informants are involved
as partners or active participants in the research process), qualitative (deals more often with language than with
numbers), and reflective (critical reflection on the process and outcomes are important parts of each cycle);
Inquiry-based research conducted by practitioners for the purpose of making change; Practitioners examine
their existing practices, form a question about their practice, collect and analyze or interpret data, and use the
research results to enhance and improve their professional practice
Active consent
A process whereby consent is provided by signing a consent form
Active inquiry
Students actively participate in learning through exploration and research; Teachers using active inquiry in their
classrooms guide and support students to ask and answer their own questions related to what is being studied
Activity-based costing
Processes or activities which use resource inputs to produce outputs; One of the operating expenses of a
university which must be considered in managerial/funding decisions
Ad hoc research
Research conducted in response to a specific, one-time-only need
ADA
Americans with Disabilities Act
Adaptive testing
A sequential form of individual testing in which successive items in the test are chosen based primarily on the
psychometric properties and content of the items and the participant's response to previous items

Assessment Glossary
Additive and interactive effects

Refers to the fact that the threats to internal validity can combine to produce an additive or multiplicative bias
Adjusted validity or reliability coefficient

A validity or reliability coefficient, usually a product-moment correlation, which has been adjusted to offset the
effects of differences in score variability resulting from different populations
Administration
The management and direction of the affairs of an organization; A collective term for all policy making officials or
executive management of an organization
Administrative agency
A government organization set up to implement a law; Any civilian government body (board, bureau,
department, or individual), other than a court or legislature, that deals with the rights of private parties by
adjudication, rule making, investigation, prosecuting, etc.
Administrative feasibility
Extent to which appropriate data are readily available or can be obtained, produced, or interpreted with available
resources such as staff expertise, time, and equipment
Administrative order
A directive carrying the force of law issued by an administrative agency after adjudication
Administrative outcomes
Operational and specific statements derived from a unit’s core functions that describe the desired quality of key
services within an administrative unit and define exactly what the services should promote
Administrative responsibility
Accountability for the conduct of the evaluation process and for the decisions made based on the evaluation
results
Administrator
Anyone with fiduciary responsibility; A manager; The head of a government agency
Advanced placement exams

Test and assessments given at the beginning of an instructional segment to determine to what extent students
have already achieved the objectives of the planned instruction, so that they may be place in more advanced
course work
Adversarial groups or teams

Group of people who enter into cross-examination of counter plans, strategies, or outcomes
Advisory committee
A group of individuals appointed or elected to recommend and implement policy related to program operation; A
group or panel of internal or external members with no decision making authority, assembled to identify and
discuss specific issues and make recommendations
Advocacy groups or teams

Groups of people who are brought together to develop competing strategies for achieving a given set of
objectives
Affective

Assessment Glossary
Any category of feeling, as distinct from cognition or behavior; The display of affect is a set of physical changes
which indicates an emotional state; Outcomes of education involving feelings more than understanding (e.g.
likes, pleasures ideals, dislikes, annoyances, and values)
Affective dimension
The psychological concept that refers to a person's feelings, emotions, or degree of acceptance or rejection of
some object
Affective domain
The range of feelings and emotions: Includes interests, attitudes, motivations, values, and appreciations
Affective outcomes
Outcomes of education involving feelings more than understanding; For example: Likes, pleasures, ideals,
dislikes, annoyances, and values
Affinity group
A type of focus group in which respondents already know one another
Affirmative action
Any program, policy or procedure implemented to correct past discrimination and prevent current and future
discrimination within an organization or system
Affirmative action plan

An organization's written plan to remedy past discrimination against, or underutilization of, women and
minorities; Usually consists of a statement of goals, timetables for achieving them, and specific program efforts
Age equivalent
The chronological age in a defined population for which a given score is the median (middle) score
Age norms
Values representing typical or average performance of members of age groups
Agency
Any department, office, commission, authority, administration, board, government owned corporation, or other
independent establishment of any branch of government in the United States; A formal relation whereby one
person is authorized to act for another
Agenda setting
The process of deciding what issues will be considered at a formal meeting; The process by which ideas or
issues come up through the various political or administrative processes to wind up on the agenda of a political
institution or policy making body
Aggregate
A group of persons having certain traits or characteristics in common without necessarily having direct social
connection with one another (e.g. all female physicians or all cities with a population over 20,000)
Aggregated scores
The combined scores for a population of students; Often expressed as an average; Requires that all the scores
be based on the same or equivalent assessments administered in uniform ways
Aggregating
Combining two or more related scores into one total score

Assessment Glossary
Aims
The broad, long-term goals of a project
Alignment
The process of assuring that learning outcomes, local curriculum and instruction and the system of assessment
all support and match each other; How ideas, goals, outcomes, etc. compare or relate across different areas,
levels, courses, etc.; Process of linking content and performance standards to assessment, instruction, and
learning in classrooms; Process of strengthening the linkage between job responsibilities and teacher evaluation
systems, or between curriculum/instruction and assessment; The process of bringing various elements together
to move in the same direction
Allocational effects
The ways in which policies influence of the use of resources
Alternate forms
Two forms of the same test, both of which measure the same construct; Two or more versions of a test that are
considered interchangeable, in that they measure the same constructs in the same ways, are intended for the
same purposes, and are administered using the same directions; A generic term used to refer to any of three
categories: Parallel forms: Tests with equal raw score means, equal standard deviations, equal error
structures, and equal correlations with other measures for any given population, Equivalent forms: Tests
without the statistical similarity of parallel forms, but the dissimilarities in raw score statistics are compensated
for in the conversions to derived scores or in form-specific norm tables, or Comparable forms: Tests that are
highly similar in content, but the degree of statistical similarity has not been demonstrated
Alternative assessment
Assessment measures other than traditional, standardized, norm- or criterion-referenced paper and pencil
testing; Nontraditional means of recording evidence of learning; Requires students to actively accomplish
complex and significant tasks, while bringing to bear prior knowledge, recent learning, and relevant skills to
solve realistic or authentic problems; Entails direct observation of student performance; For example: Coding
live art criticism discussions, portfolio reviews, rating performances or art products on criteria established by
teachers and students, journals, and authentic task assessment; See also authentic and performance
assessment
Alternative hypothesis
Statement that the population parameter is some value other than the value stated by the null hypothesis
Ambiguous temporal precedence

The inability to specify which variable is the cause and which is the effect
Amount technique
Manipulating the independent variable by giving the various comparison groups different amounts of the
independent variable
Analysis
Assessments performed by an organization to provide a basis for decision making; Organizing and reviewing
information so it has meaning; Conclusions and hypothesis derived from the review of collected and organized
data or information; The interpretation of data; The treatment of data and information in order to elicit certain
statistical data, assessment results, and evaluative conclusions
Analysis of covariance
A statistical method that can be used to statistically equate groups that differ on a pretest or some other
variable; Also used to examine the relationship between one categorical independent variable and one
quantitative dependent variable controlling for one or more extraneous variables

Assessment Glossary
Analysis of variance
A statistical method used to determine whether a relationship exists among two or more variables by formulating
concurrent comparisons of the variables
Analytic rubric
An analytic rubric articulates levels of performance for each criterion so the teacher can assess student
performance on each criterion
Analytic scoring
A method of scoring in which each critical dimension of performance is judged and scored separately, and the
resultant values are combined for an overall score; A type of rubric scoring that separates the whole into
categories of criteria that are examined one at a time; Evaluating student work across multiple dimensions of
performance rather than from an overall impression; A type of rubric scoring that separates the whole into
categories of criteria that are examined one at a time; Products or performances are evaluated for selected
dimensions, with each dimension receiving a separate score; For example: A piece of writing may be evaluated
on several categories, such as organization, use of details, attention to audience, and language
usage/mechanics; Analytic scores may be weighed and totaled; Approach to scoring or rating that considers
various parts or aspects of the attribute or performance being assessed, for use in profiling strengths and
weaknesses or in obtaining an overall summary; Scores may be recorded as a check mark for presence or
absence of an attribute, marked on a numerical or descriptive rating scale, or put in the form of a brief comment;
The use of a scoring key (e.g. rubric) containing an ideal response to judge the competence or proficiency of
student responses on an assessment; The use of specific criteria or features to evaluate and assign points to
each essential part of a product or performance
Analytical skills
The ability to discover the underlying structure of an argument, a communication, a problem, or a solution
Analytical thinking
The ability to analyze facts, generate a comparison, and draw correct inferences or conclusions from available
information
Anchors
A written descriptor for a point on a rating scale; Descriptive point on a scale or continuum; Representative
products or performances used to illustrate each point on a scoring scale; Actual samples of student work which
illustrate the essential characteristics of work typical for each scoring level on a scoring rubric; Raters use
anchors to score student work, usually comparing the student performance to the anchor; For example: If
student work was being scored on a scale of 1-5, there would typically be anchors (previously scored student
work) exemplifying each point on the scale; Anchors for the highest score point are sometimes referred to as
exemplars
Anchor test
A common set of items administered with each of two or more different forms of a test for the purpose of
equating the scores of these forms
ANCOVA (Analysis of Co-Variance)

Same method as ANOVA, but analyzes differences between dependent variables
Androgogy
Adult instructional methods; methods designed to have more appeal or effectiveness with adult learners; The art
and science of teaching adults; The art and science of helping adults learn
Anecdotal record
A short, written report of an individual's behavior in a specific situation or circumstance; a short narrative of an
event or activity that may be used to support generalizations about the performance of a teacher

Assessment Glossary
Angoff
A standard-setting procedure, which requires participants to review test items and to provide estimation of the
proportion of a subpopulation of examinees who would answer the item correctly
Annotated rubric
Notes from an assessment development group, often after a field test and initial scoring, which explain the
meaning of criteria or distinctions between the criteria on a rubric; Used to increase scoring reliability and to train
others to score consistently
Announced visit or observation

Student observation or visit that is prearranged with the student to be evaluated and for which the faculty can
prepare; Class observation or visit that is prearranged with the instructor to be evaluated and for which the
instructor can prepare
Annual update or report

A brief report, submitted annually, from each academic program based on its assessment plan and which
outlines how evidence was used to improve student learning outcomes through curricular and/or other changes
or to document that no changes were needed
Anonymity
A situation in which it is not possible to identity an individual; Evaluator action to ensure that the identity of
subjects cannot be ascertained during the course of a study, in study reports, or in any other way; Keeping the
identity of the participant from everyone, including the researcher
ANOVA (Analysis of Variance)

A method of statistical analysis broadly applicable to a number of research designs and used to determine
differences among the means of two or more groups on a variable; A simple summary of the variation in the
experimental data; One of the most widely used statistical methods; The independent variables in an ANOVA
are usually nominal, and the dependent variable is usual an interval; A class of parametric statistical procedures
capable of processing more than two columns of group-difference data in one row (one way ANOVA) or two or
more columns of such data in two or more rows (two way or factorial ANOVA)
Apparency
A clear, understandable representation of the data
Apparent authority
The appearance that an individual has the authority or power to act as an organization’s agent even though the
organization has bestowed no such authority or power to that individual
Applicability
The extent to which information on a particular outcome measure meets the needs of multiple stakeholder
groups; For example: The extent to which data generated from a critical thinking, problem solving, or writing
assessment yield information that can be used by multiple groups, such as faculty and administrators who wish
to improve programs, or government officials and prospective employers who desire documentation of skill level
achievement or attainment
Application service provider (ASP)

Third-party organizations that deliver software applications and related services over the Internet, allowing an
organization to outsource some or all of its information technology needs
Applied research
Research focused on answering practical questions to provide relatively immediate solutions; Research
conducted with the intention of applying the results to some specific problem; In applied research the research

Assessment Glossary
questions are generally determined by policy makers, decision makers, or other stakeholders rather than by
researchers; Systematic study to gain knowledge or understanding necessary to determine how a recognized
and specific need may be met
Appreciative Inquiry
A large-group method, often used at times of organizational change, to discover the most valuable aspects of
the organization's past that should be carried forward into its future
Appropriation
Money that has been set aside and allocated for a specific function
Aptitude
The potential for acquiring abilities or developing competencies; A student's capability or potential for performing
a particular task or skill
Aptitude test
A test consisting of items selected and standardized so that the test predicts a person's future performance on
tasks not obviously similar to those in the test; A test intended to measure the test-taker's innate ability to learn,
given before receiving instruction; A test that estimates future performance on other tasks not necessarily
having evident similarity to the test tasks; Tests that focus on information acquired through the informal learning
that goes on in life; Often aimed at indicating an individual's readiness to learn or to develop proficiency in some
particular area if education or training is provided; May not differ in form or substance from achievement tests
while differing in use and interpretation
Arbitrary
A characteristic of a decision or action that is uninformed or is based upon one person's judgment or discretion
Archival records
Existing biographical, academic, or other file data available from the college or other agencies and institutions
Archival search
An examination of existing records, reports, and documents pertaining to the object of the evaluation
Archived research data

Data originally used for research purposes and then stored
Arithmetic mean
A kind of average obtained by dividing the sum of a set of scores by the total number of scores; Usually referred
to as the mean
Articulation of quality
An expanded articulation of an ideal future condition of quality, arising from the words and phrases of a strategic
plan
Artifact
A product developed by the student or another individual; An artificial statistical phenomenon or result;
Assessment or statistical findings which result from errors or weaknesses in the research design or testing
instrument; Something that is artificial, contrived, or fictitious; A research conclusion which arose due to a bias
or confound of some sort, rather than because of the causal relationship being tested
Assessment
The collection and analysis of evidence of effectiveness for all parts of the institution, including but not limited to
evidence of student learning and achievement; Process of gathering, describing, or quantifying information
about performance; The process of observing learning by describing, collecting, recording, scoring, and

Assessment Glossary
interpreting information about a student's or one's own learning; Process of collecting information about a
student to aid in decision making about the progress and development of the student; The process of or
instrument for measuring, quantifying, and/or describing those aspects of teaching related to the attributes
covered by the evaluation; Systematic collection, review, and use of information about educational programs
undertaken for the purpose of improving student learning and development; An effort to gather, analyze and
interpret evidence that describes institutional, departmental, divisional, or program effectiveness; Ongoing
process to assure and document learning; Ongoing process aimed at understanding and improving student
learning; To thoroughly and methodically analyze student accomplishment against specific goals and criteria;
The systematic collection and analysis of information to improve student learning; Assessment is not an end in
itself nor is it merely to gather information, rather the purpose of assessment is educational or learning
improvement; Involves making expectations explicit and public, setting appropriate criteria and high standards
for learning quality, systematically gathering, analyzing, and interpreting evidence to determine how well
performance matches those expectations and standards, and using the resulting information to document,
explain, and improve performance; Individual student accomplishments are assessed and summarized to
ascertain the achievement of groups or subgroups of students
Assessment or evaluation audit

An independent examination and verification of the quality of an evaluation plan, the adequacy of its
implemented, the accuracy of results, and the validity of conclusions; An independent quality check and
verification of the assessment and evaluation of a teacher
Assessment for accountability

Assessment of some unit (e.g. program, department, college, or entire institution) to satisfy stakeholders
external to the unit itself; Results are summative and are often compared across units
Assessment for improvement

Assessment that feeds directly, and often immediately, back into revising the course, program or institution to
improve student learning results; May be formative or summative
Assessment instrument
The evidence of learning that the student will produce (e.g. test, essay, portfolio, demonstration) which will be
evaluated by faculty with respect to a rubric
Assessment literacy
The possession of knowledge about the basic principles of sound assessment practice; Includes knowledge
about terminology, the development and use of assessment methodologies and techniques, familiarity with
standards of quality in assessment, and familiarity with alternatives to traditional measurements of learning
Assessment method
A strategy or technique evaluators may use to acquire evaluation information; For example: Observations, text-
and curriculum-embedded questions and tests, paper-and-pencil tests, oral questioning, benchmarks or
reference sets, interviews, peer- and self-assessments, standardized criterion-referenced and norm-referenced
tests, performance assessments, writing samples, exhibits, portfolio assessment, and project or product
assessments
Assessment of individuals
Uses the individual student, and his/her learning, as the level of analysis; May be quantitative or qualitative,
formative or summative, standards-based or value added, and used for improvement; Needs to be aggregated if
used for accountability purposes; For example: Improvement in student knowledge of a subject during a single
course, or improved ability of a student to build cogent arguments over the course of an undergraduate career
Assessment of institutions
Uses the institution as the level of analysis; May be quantitative or qualitative, formative or summative,
standards-based or value added, and used for improvement or for accountability; Ideally institution-wide goals

Assessment Glossary
and objectives serve as a basis for the assessment; For example: How well students across the institution can
work in multi-cultural teams as sophomores and seniors
Assessment of programs
Uses the department or program as the level of analysis; May be quantitative or qualitative, formative or
summative, standards-based or value added, and used for improvement or for accountability; Ideally program
goals and objectives serve as a basis for the assessment; For example: How sophisticated a close reading of
texts senior English majors can accomplish (if used to determine value added, results would be compared to the
ability of newly declared majors)
Assessment plan
A plan written in the first step of the assessment cycle in which faculty draft an SLO for a course or program,
outline how it will be assessed, and state how the resulting data will be used to improve student learning: A
document that outlines the student learning outcomes (for academic programs) or unit outcomes (for support
units), the direct and indirect assessment methods used to demonstrate the attainment of each outcome, a brief
explanation of the assessment methods, an indication of which outcome(s) is/are addressed by each method,
the intervals at which evidence is collected and reviewed, and the individual(s) responsible for the
collection/review of evidence; The written assessment document with the mission, goals, and objectives of the
institution, the outcomes to be assessed, and the assessment strategies; A document which identifies expected
outcomes for an educational program and outlines how and when the identified outcomes will be assessed
Assessment report or record

Annual report of the previous year’s assessment activities; The summary of the results and uses of results from
the prior year’s assessments; An annual document, based on the assessment plan, that presents and explains
assessment results and shows how assessment results are being used to improve the educational program
Assessment results
The differences or similarities between intended and actual outcomes of the assessment
Assessment system
Combination of multiple assessments into a comprehensive reporting format that produces comprehensive,
credible, dependable information upon which important decisions can be made about students, schools,
districts, or states; May include norm-referenced or criterion-referenced assessment, alternative assessment,
and classroom assessments
Assessment task
An illustrative task or performance opportunity that closely targets defined instructional aims, allowing students
to demonstrate their progress and capabilities
Assessment validity
The degree to which an assessment measures what is intended, as opposed to what is not intended, or what is
unsystematic or unstable
Assimilation
The process of being fully absorbed and incorporated into a culture
Assumptions
Conclusions drawn without data to support them; Something consciously or unconsciously believed to be true
without reference to supporting evidence; Recognized or unrecognized beliefs which influence cognitive,
affective, and observable behavior; Explicit assumptions are unproven or unprovable beliefs or conditions stated
by a researcher to have influenced an analysis or research design; Implicit assumptions are beliefs or conditions
inherent or evident in an analysis or research design which are not explicitly stated by the researcher; Implicit
assumptions not recognized by the researcher may result in researcher bias; Implicit assumptions not
recognized by the research audience or consumer, or the unconscious assumptions of the research audience,

Assessment Glossary
may result in the misinterpretation of research results and actions or decisions not supported by the research
findings
Attainment
To gain or accomplish credit or a credential, such as a certificate or degree or educational level
Attention assessment
The process of collecting data and making an appraisal of a person's ability to focus on the relevant stimuli in a
situation; May be directed at mechanisms involved in arousal, sustained attention, selective attention and
vigilance, or limitation in the capacity to attend to incoming information
Attention bias
A type of bias where the act of observation itself becomes an important independent variable or treatment (e.g.
the Hawthorne effect)
Attribute
A characteristic, capacity, or perceived quality of an individual, thing, or place; Individual attributes include, but
are not limited to, attitude, ability, behavior, skill, knowledge, or interest
Attrition
Loss of students through means other than graduation; Loss of subjects from the defined sample during the
course of a research study
Audience
Those persons to be guided by the results of student evaluations in making decisions about the development
and progress of students and all others with an interest in the evaluation results and findings; Those persons
who will be guided by the evaluation in making decisions and all others who have a stake in the evaluation;
Those individuals who have a potential interest in the results of teacher performance assessment and
evaluation and in the quality of teaching
Audit
A systematic assessment of a organization’s management, finances, operations, controls and scope in which
policies and procedures are carried out; Inspection of the accounting records and procedures of a business,
government unit, or other reporting entity by a trained accountant for the purpose of verifying the accuracy and
completeness of the records
Audit review approach

Form of evaluation which involves measuring how well project objectives were met and the impacts of projects,
focusing on questions such as “what did we set out to achieve?” and “what are the signs that we have done
this?”, and using methods such as structured surveys
Audit trail
A record depicting all the chronological stages of a particular transaction
Authentic
A characteristic of assessments that have a high degree of similarity to tasks performed in the real world; The
more authentic the assessment, the less inference required to predict student success after graduation
Authentic assessment
A form of assessment in which students are asked to perform real-world tasks that demonstrate meaningful
application of essential knowledge and skills with performance typically scored on a rubric to determine how
successfully the student has met specific standards; A form of performance assessment in which the activity
simulates the contexts, tasks and ways people in “real-life” situations use knowledge and skills; An authentic
task is one in which the student addresses a complex problem situation under real-life conditions; Assessment

Assessment Glossary
tasks that evoke demonstrations of knowledge and skills in ways that they are applied in the “real world”;
Assessment that fits meaningful, real-life learning experiences; May include recording evidence of the learning
process, applications in products and performances, perception of visual and audio relationships, integrations of
new knowledge, reflecting profitably on one's own progress, and interpreting meaning in consideration of
contextual facts; Method of assessment in which the student is expected to demonstrate his or her competence
and proficiency through the completion of a task that mimics a job, higher educational, or life skill
Authentic performance assessment

An evaluation of a student's ability to perform in the classroom a complex task that is common in the field or
“real world”; Substitute for truly authentic real world tasks which occur infrequently in most classrooms
Authentic task
An assignment given to students designed to assess their ability to apply standards-driven knowledge and skills
to real-world challenges; A task is considered authentic when students are asked to construct their own
responses rather than to select from ones presented and the task replicates challenges faced in the real world
Authority
Power defined according to a legal and institutional framework, and vested in a formal structure (a nation,
organization, profession); Power exercised through recognized, legitimate channels
Average
A general term applied to the various measures of central tendency; The three most widely used averages are
the mean (or arithmetic mean), the median, and the mode; When the term "average" is used without designation
as to type, the general assumption is that it refers to the arithmetic mean
Average deviation
A statistical term referring to the average of the deviation, taken from some fixed value, usually the
mathematical mean of the data
Award units
The generic term which describes various measures of the amount of value (or credit) received by a student for
the successful completion of a course; A measure of the amount of value (or credit) received by a student for
the successful completion of one or more educational experiences, such as a course or other type of learning
experience; The number of units to be awarded for the successful completion of a course is usually determined
by a faculty committee (or equivalent body) upon consideration of factors such as difficulty of the course, the
time expected of a student in contact with the faculty and otherwise (i.e. Contact Hours/Clock Hours and Total
Hours), and the type of educational experience (e.g. lecture, laboratory, independent study); When data
pertaining to Award Units is transmitted outside the institution, the transmitting institution should specify the
Award Unit used (e.g. Semester Credit Hour, CEU), and define the unit, where applicable, in terms of the total
number of Award Units required for program completion and, if appropriate, the real-time factors of the Award
Unit (i.e. Contact Hours/Clock Hours and Total Hours)
Awareness of the hypothesis

Confounding due to a research participant's understanding of the research purpose, consciously or
unconsciously, influencing the behavior being measured
Axial coding
The second stage in grounded theory data analysis

Assessment Glossary
B
Backload
Amount of effort required after the data collection
Backstage behavior
What people say and do only with their closest friends
Balanced scorecard
A popular strategic management concept, the balanced scorecard is a management and measurement system
that enables organizations to clarify their vision and strategy and translate them into action; The goal of the
balanced scorecard is to tie business performance to organizational strategy by measuring results in four areas:
financial performance, customer knowledge, internal business processes, and learning and growth
Bar graph
A graph that uses vertical bars to represent the data
Barriers to entry
Factors that reduce entry into an industry or market
BARS
See behaviorally-anchored rating scale
Baseline
The behavior of the participant prior to the administration of a treatment condition; Base level of previous or
current performance that can be used to set improvement goals and provide a basis for assessing future
progress
Basic research
Research aimed at generating fundamental knowledge and theoretical understanding about basic human and
other natural processes; Systematic study and investigation undertaken to discover new knowledge, facts or
principles (the pursuit of knowledge for the sake of knowledge)
Battery
A set of several tests designed to be administered as a unit; A group of several tests standardized on the same
sample population so that results on the several tests are comparable; A group of carefully selected tests that
are administered to a given population, the results of which are of value individually, in combination, and totally;
Sometimes loosely applied to any group of tests administered together, even though not standardized on the
same subjects; Most common test batteries are those used to measure school achievement, which include
subtests in the separate learning areas
Behavior
The combined observable responses of a person to internal and external stimuli
Behavior management
Structure and support for students to develop appropriate behavior by building on student strengths and
developing their confidence in their own abilities
Behavior modification
A conscious attempt to change or eliminate an individual’s undesirable behavior by specifying expected
behavior and reinforcing and rewarding desired behavior

Assessment Glossary
Behavior summary scale (BSS)

Type of rating assessment in which judgments about teacher performance are related to general or abstract
benchmarks representing various levels of performance
Behavioral observation scale (BOS)

Type of rating assessment in which judgments about teacher performance are related to a series of statements
describing specific examples of observable teacher behaviors
Behavioral observations
Measuring the frequency, duration, typology, etc. of student actions, usually in a natural setting with non-
interactive methods (e.g. formal or informal observations of a classroom); Observations are most often made by
an individual and can be augmented by audio or videotape
Behavioral risk management

The process of analyzing and identifying workplace behavioral issues and implementing programs, policies or
services most suitable for correcting or eliminating various employee behavioral problems
Behaviorally-anchored rating scale (BARS)

Type of rating assessment in which judgments about teacher performance are empirically linked to specific
examples of incumbent performance at each level of effectiveness on the rating scale
Behaviors
Actions which are specific and observable; Specific, observable actions of a student in response to internal and
external stimuli
Bell curve or bell-shaped curve

A frequency distribution graph of statistics; A normal distribution shaped like a bell; The curve representing the
normal distribution of a rating or test score
Benchmark
A referenced behavior for comparing observed performance at a given level; The standards used as a basis for
comparison or measurement; A detailed description of a specific level of student performance expected of
students at particular ages, grades, or development levels; A description or example of student, candidate, or
institutional performance that serves as a standard of comparison for evaluation and judging quality; A
mathematical average of past performance levels; The first or last set of data used for comparisons; A criterion-
referenced objective; Performance data used for comparative purposes; Past performance used as a baseline
against which to compare future performance; An exemplary or best practice performance level used as a target
to strive for; Exemplary samples of performance or production; An average or standard used for comparisons; A
set of benchmarks can be used as "checkpoints" to monitor progress toward meeting performance goals within
and across grade levels; An actual measurement of group performance against an established standard at
defined points along the path toward the standard with subsequent measurements of performance using the
benchmarks to measure progress toward achievement; A criterion-referenced objective; Performance data that
are used for comparative purposes
Benchmark tasks
Pieces of student work selected by a group of lead teachers as exemplifying a certain score level; See also
Anchor test
Benchmarking
An actual measurement of group performance against an established standard at defined points along the path
toward the standard; Subsequent measurements of group performance use the benchmarks to measure
progress toward achievement; The systematic process of comparing an organization’s products, services and
practices against those of a competitor organization or other industry leader to determine what allows it to

Assessment Glossary
achieve high levels of performance; Gathering information about model efforts or best practices by other
organizations engaged in similar endeavors to help establish project targets and goals; The process of
continuously comparing and measuring an organization against recognized leaders and similar organizations to
gain information that will help the organization take action to improve its performance; A process of comparing
organizational performance against other, similar operations or jurisdictions, a standard, or a past time-frame,
while maintaining direct comparability of performance measures
Benefit
An advantageous consequence of a program or action
Benefit-cost ratio
The ratio of the present value of benefits over the present value of costs
Best practices
Practices that enable an organization or individual to achieve superior performance results
Bias
Systemic variance that skews the accurate reporting of data in favor of a particular group; A constant error; Any
systematic influence-on measures or on statistical results-irrelevant to the purpose of the evaluation; A
systematic tendency toward a lack of objectivity, fairness, or impartiality on the part of the assessor or evaluator,
often based on personal preferences and inclinations; Systematic error in the assessment instrument and
procedures, or in the interpretation and evaluation process; Occurs in testing when items systematically
measure differently for different ethnic, gender, or age groups; Types of potential bias in research include:
attention bias, centripetal bias, confounding, cultural bias, demand characteristics, expectancy bias,
measurement bias, recall bias, sampling bias, volunteer bias, and withdrawal bias
Biased sample
A sample that is systematically different from the population
Binomial sign test

A single sample inferential statistic for non-parametric data at the nominal or ordinal level; Can also be used as
a two-sample inferential statistic for non-parametric data in a related design
Bivariate analysis
Pertaining to two variables only
Blind
The giving or taking of measurements without knowledge of the true purpose of the research; See double blind
and single blind
Bloom's taxonomy
Six levels of cognitive ability arranged in order of increasing complexity (from 1=low to 6=high): 1. Knowledge:
Recalling or remembering information without necessarily understanding it (includes behaviors such as
describing, listing, identifying, and labeling); 2. Comprehension: Understanding learned material (includes
behaviors such as explaining, discussing, and interpreting); 3. Application: The ability to put ideas and
concepts to work in solving problems (includes behaviors such as demonstrating, showing, and making use of
information); 4. Analysis: Breaking down information into its component parts to see interrelationships and
ideas (related behaviors include differentiating, comparing, and categorizing); 5. Synthesis: The ability to put
parts together to form something original (involves using creativity to compose or design something new); 6.
Evaluation: Judging the value of evidence based on definite criteria (behaviors related to evaluation include
concluding, criticizing, prioritizing, and recommending)
Body of evidence
All factual information which bears on the determination reached in a study

Assessment Glossary
Body of work (BoW)

A standard-setting procedure in which panelists examine complete student response sets and match each
student response set to a performance level category based upon previously agreed upon descriptions of what
students at the different levels should know and be able to do
Bond
Promise to repay a certain amount (principal) at a certain time (maturity date) at a particular rate of interest
Bonferroni's correction for multiple comparisons

An adjustment to the confidence level required when a single scientific hypothesis is being investigated using
multiple inferential statistics where the risk is the fact that the probability of a Type 1 error (or p-value) increases
with every new statistical procedure
Bookmark
A procedure used to establish a criterion-referenced cutoff of a score or rating determined by content area
experts
Boolean operators or connectors

Words used to create logical combinations (e.g. “and”, “or”, or “not”)
Bootstrapping
A non-parametric approach for making statistical inferences employing large numbers of repetitive computations
to estimate the shape of a statistic's sampling distribution
BOS
Behavioral observation scale
Box plot or box-and-whisker plot

Graphical aid (drawn either horizontally or vertically) designed to display both the central tendency and the
dispersal of a given distribution by displaying five values: the lower and upper extremes (the smallest and
largest observations, which define the range), the median (the midpoint of the distribution), and the lower and
upper quartiles (defined by the 25th and 75th percentiles)
Bounded rationality
Seeking the best possible solution, but not necessarily the most rational from a purely economic standpoint
Bracket
To suspend your preconceptions or learned feelings about a phenomenon
Branding
The process of identifying and differentiating an organization’s products, processes or services from other
competing organizations by associating it with a name, phrase or other mark; Process of creating a unique
name and image for a product, organization, or individual that attracts and retains loyal customers through
advertising or marketing campaigns having a consistent theme
Brainstorming
Decision-making technique where an individual or group generates multiple ideas and suggestions about a
particular topic or theme within a very short time frame; A not very successful method for generating ideas, in
which a small group of people come up with ideas as fast as they can and participants may build on each others'
ideas, but not criticize them
Break-even analysis

Assessment Glossary
A measure used to determine the approximate sales volume or income required to cover the costs associated
with producing a particular product or service
BSS
Behavior summary scale
Budget
A numerical summary of an organization’s available resources and how those resources are to be allocated
based on anticipated future expenditures for various items; A plan in numbers
Budget padding
Proposing a higher budget than is actually needed; Frequently justified by the assumption that all proposed
budgets will be reduced prior to authorization regardless of actual needs
Bureaucracy
A formal organizational arrangement characterized by division of labor, job specialization with no functional
overlap, exercise of authority through a vertical hierarchy (chain of command) and a system of internal rules,
regulations, and record keeping
Business literacy
The knowledge and understanding of the financial, accounting, marketing and operational functions of an
organization
Business planning
A plan that determines how a strategic plan will be implemented; Specifies how, when, and where a strategic
plan will be put into action; Also known as tactical planning
Buy-in
Obtaining agreement from key stakeholders that the proposed plan is acceptable

Assessment Glossary
C
CAAP (College Assessment of Academic Proficiency)
A national standardized test by ACT (Academic College Testing) that measures general education proficiencies
in Writing, Reading, Math, Critical Thinking and Science
Calibration
The process of setting the test score scale, including mean, standard deviation, and possibly shape of score
distribution, so that scores on a scale have the same relative meaning as scores on a related scale
Canned, packaged, or off-the-shelf instruments or software

Preexisting tests or software that is designed to be applicable to a variety of situations or applications
Capacity
Instructor’s potential for acquiring skills and competencies through such means as self-study, on-the-job
training, mentoring, coaching, and professional development activities
Capacity building
The development of an organization’s core skills and capabilities (e.g. building leadership, management, finance
and fund-raising, programs, and evaluation) in order to build the organization’s effectiveness and sustainability;
The process of assisting an individual or group to identify and address issues and gain the insights, knowledge,
and experience needed to solve problems and implement change
Capital
Assets that are available for on-going business needs to produce income; The fixed non labor factors used in
the production of goods and services, and the value of such factors
Capital budgeting
The separation of expenditures that produce long-term benefits, especially those involving the construction of
facilities, from the annual operating costs of an organization; The process for reviewing expenditure decisions
for capital projects and deciding on the methods for financing them
Capital expenditure
A dispersement for the purchase, replacement, renovation or expansion of facilities that cannot be charged as
an expense under generally accepted accounting standards
Capital expense budget

Budgets used to determine capital expenditures for constructing or upgrading facilities and equipment for which
a predetermined dollar value has been assigned based on its life expectancy
Capital grants
Grants for use in construction or renovation
Capital improvement
Addition to an organization’s fixed assets; A permanent addition to an organization’s fixed assets resulting from
an outlay of funds over a specific and finite period of time
Capital plan
A plan for maintaining assets to continue programs
Capricious

Assessment Glossary
That which is unpredictable, unaccountable, impulsive, or without a supporting rationale, or which is

characterized by implied unruly or whimsical behavior
Capstone assessment
Assessment of outcomes structured into learning experiences occurring at the end of a program; Capstone
experiences involve demonstration of a comprehensive range of program outcomes through some type of
product or performance; Capstone outcomes may be those of the degree program only or a combination of
those from the program, general education, and the institution as a whole
Capstones
Capstone experiences for graduating seniors are designed to demonstrate comprehensive learning in the major
through some type of product or experience; In addition to emphasizing work related to the major, capstone
experiences can require students to demonstrate how well they have mastered important learning objectives
from the institution’s general studies program
Carryover effect
A sequencing effect that occurs when performance in one treatment condition is influenced by participation in a
prior treatment condition(s)
Case
A bounded system
Case study
A form of qualitative research that is focused on providing a detailed account and analysis of one or more cases;
An intensive, detailed description and analysis of a single project, program, or instructional material in the
context of its environment; The collection and presentation of detailed information about a particular participant
or small group, and frequently includes the accounts of the subjects themselves; A study containing qualitative
data (e.g. observations and information drawn from interviews) about one subject; A research method which
focuses on the characteristics, circumstances, and complexity of a single case, or a small number of cases,
often using multiple methods; Case study findings may raise awareness of general issues, but the aim is not to
generalize the findings to other cases; Case studies are conducted primarily using qualitative research
techniques, but do not exclude quantitative research methods
CAT
Classroom Assessment Technique; Also Computer Administered Test
Categorical or project grant

Grants requiring that the money may be spent for only a limited purpose
Categorical variable
A variable that varies in type or kind
Category midpoint
The midpoint between the upper and lower measures in a category range; The midpoint of categories is used to
calculate means and standard deviations in cases where the data categories were grouped or collapsed; The
arithmetic average of the upper and lower boundaries in a category; For example: The midpoint in the pre
grouped category "9 to 11 years of school" is (8.5+11.5)/2 = 10, and for a single score, such as "3," the
boundaries would range from 2.5 to 3.5 with an average midpoint of (2.5 + 3.5)/2 = 3
Causal-comparative research
A form of non experimental research in which the primary independent variable of interest is a categorical
variable
Causal description

Assessment Glossary
Describing the consequences of manipulating an independent variable
Causal explanation
Explaining the mechanisms through which and the conditions under which a causal relationship holds
Causal line
A temporal series of events so related that, given some of them, something can be inferred about the others
whatever may be happening elsewhere; A succession of lesser cause-effect events: Event A causes State B,
which triggers Event C, etc.
Causal model
A model which represents a causal relationship between two variables
Causal modeling
A form of explanatory research where the researcher hypothesizes a causal model and then empirically tests
the model; Also called structural equation modeling or theoretical modeling
Causal relationship
The relationship established showing that an independent variable, and nothing else, causes a change in a
dependent variable; Also establishes how much of a change is shown in the dependent variable
Causation
A situation in which one factor determines the outcome of another factor or result; Fundamental notion of
science that some things just happen randomly while others are contingent upon prior events taking place or
specified antecedent conditions being reached
Causality
The relation between cause and effect
Cause and effect relationship

Relationship in which one variable affects another variable
Ceiling
Highest limit of performance that can be assessed or measured by an instrument or process; The upper limit of
performance or ability that can be measured effectively by a particular test; Individuals have reached the ceiling
of a test when they perform at the top of the range in which the test can make reliable discriminations;
Individuals who perform near to or above this upper limit are said to have reached the ceiling, and the
assessment may not be providing a valid estimate of their performance levels; Such individuals should be given
a more difficult assessment of the same attribute if it is necessary to differentiate between higher levels of
performance
Cell
A combination of two or more independent variables in a factorial design
Census
Survey of a whole population; The entire population of the group in question is included in the study; A study of
the whole population rather than a sample
Central tendency
Measures that indicate the middle or center of a distribution; A measure of central tendency provides a single
most typical score as representative of a group of scores; The "trend" of a group of measures as indicated by
some type of average, usually the mean or the median
Central tendency effect

Assessment Glossary
A type of rater effect or artifact in which an assessor or an evaluator tends to rate instructors or students toward
the mid-point of a scale or to judge a performance as average or neutral when it is actually well above or well
below the middle level of the scale; Consequently assessment results will lack validity because observers lack
inter-rater reliability
Centrality
Core purposes
Centralization
The process of consolidating all decision-making authority under one central group or location
Certification
Official recognition of advanced status, outstanding performance or a high level of expertise in the profession;
Sometimes used as a synonym for credential or license; A document issued to a person completing a course of
study not leading to a degree; Verification that a level of achievement has met designated standards
Centrifugal and centripetal bias

Types of bias where the research site itself results in samples which are unrepresentative of the population at
large; For example: Respondents at low or high prestige institutions may differ from the population as a whole
because low prestige sites may risk a centrifugal bias (due to avoidance by respondents of certain
characteristics) and high prestige sites may risk a centripetal bias (due to attracting respondents of certain
characteristics); Both of these biases illustrate how a random sample can create sampling errors when the
experimental sample is drawn from a subset of the parent population created through a self-selection processes
CIRP (Cooperative Institutional Research and Effectiveness Program)

A national annual survey conducted by the Higher Education Research Institute (HERI) designed specifically for
first-time college attendees (includes new transfers into a college); The survey is administered within the first
two weeks of school and measures demographic, attitudinal, and educational self-assessment characteristics of
an entering cohort
Chain of command
Structure of decision-making responsibilities from the highest levels of authority to the lowest levels
Change verbs
Action words that tend to describe how things become different; Useful in writing mission statements, goals, and
strategic objectives
Changing-criterion design
A single-case experimental design in which a participant’s behavior is gradually altered by changing the criterion
for success during successive treatment periods
Checklist
An assessment that is based on the examiner observing an individual or group and indicating whether or not the
assessed behavior is demonstrated; List of performance criteria for a particular activity or product on which an
observer marks the student's performance on each criterion using a scale that has only two points; For example:
Present or absent, and adequate or inadequate; A list of items to be completed for use in the evaluation of
constructed response format items; Characteristics or behaviors marked as either present or absent
Chi-square or chi-squared test

Statistical test used to determine whether a relationship observed in a contingency table is statistically
significant; Most common method of statistical analysis for frequency data; Given an array of actual cell
frequencies, a null hypothesis expected distribution is computed, and the difference between the two is tested to
see if it could have occurred by chance

Assessment Glossary
Churn rate
The number of participants who discontinue their use of a service over a given period of time divided by the
average number of total participants
Classical test theory or classical true score theory

The view that an individual's observed score on a test is the sum of a true score component for the participant,
plus an independent measurement error component; A few simple premises about these components lead to
important relationships among validity, reliability, and other test score statistics; A testing theory which
postulates error scores are random with a mean of zero, and uncorrelated with error scores on a parallel test
and with true scores; Postulates that true scores, observed scores, and error scores are linearly correlated and
are represented by the equation x = t + e, where x is the observed score, t is the true score, and e is the error
score
Classification accuracy
The degree to which neither false positive nor false negative categorizations and/or diagnoses, occur when a
test is used to classify an individual or event
Classroom assessment
An assessment developed, administered, and scored by a teacher or set of teachers with the purpose of
evaluating individual or classroom student performance on a topic; The systematic and on-going study of what
and how students are learning in a particular classroom; Often designed for individual faculty who wish to
improve their teaching of a specific course; Differs from tests and other forms of student assessment in that it is
aimed at course improvement, rather than at assigning grades; Simple, non-graded, anonymous, in-class
activities that help instructors gather feedback from students on the teaching-learning process and determine
how well their students are learning what they are being taught; The purpose of classroom assessment is to
provide faculty and students with information and insights needed to improve teaching effectiveness and
learning quality; College instructors use feedback gleaned through Classroom Assessment to inform
adjustments in their teaching; Faculty also share feedback with students, using it to help them improve their
learning strategies and study habits in order to become more independent, successful learners; Classroom
assessment is a type of formative evaluation
Classroom assessment technique (CAT)

Assessment techniques embedded into the classroom-based, or other, learning process; Assessment
conducted in the classroom because students benefit from opportunities to give and receive feedback on their
learning before being evaluated or graded and they also need to learn self-assessment; Faculty benefit from
gathering data which address questions about issues or problems in their teaching
Classroom management
The management of student behavior and learning activities by teachers by, for example, designing learning
activities in ways that structure relationships to support learning, the use of rewards and sanctions, and
negotiating classroom codes of conduct
Classroom research
A collaborative process for investigating teaching and learning issues
Client
Anyone whose interests are served by an organization, or who receives or uses an organization’s resources or
services; Clients can be internal to an organization (e.g. one department may be the client of another
department) or external to the organization
Closed-ended question
A question that forces participants to choose a response
Closing the loop

Assessment Glossary
Using assessment results for program change and improvement; Demonstrating the linkage between
assessment and change; Communicating to constituents the changes made due to an assessment; Using
assessment results for program change and improvement; The results of one year’s work in strategic planning
indicate what the next year’s plan should focus on
Cluster
A collective type of unit that includes multiple elements
Cluster analysis
This is one of the four recognized types of multivariate analysis (the others being principal components analysis,
factor analysis, and discriminate analysis)
Cluster sampling
Type of sampling in which clusters are randomly selected; Sampling from selected clusters of potential subjects
within the target population, rather than the population as a whole (thus saving time and expense, but at the risk
of introducing a sampling bias)
Co-curricular programs
Out-of-class activities (e.g. student affairs programs and activities)
Co-occurring codes
Sets of codes that partially or completely overlap
Coaching
Classroom observation and feedback on specific aspects of practice by colleagues
Code
A numeric or alphabetic symbol used to represent attributes or words; To translate a given set of data or items
into a set of quantitative or qualitative symbols
Coding
The process of transforming data, evidence, information, judgments, notes, and responses to numeric and/or
alphabetic codes; Marking segments of data with symbols, descriptive words, or category names; Making sense
of a text by identifying, summarizing, and labeling its components relative to a research question
Coefficient
A value expressing the degree to which some characteristic or relation is to be found in specified instances; For
example: The coefficient of correlation is a value expressing the degree to which two variables vary
concomitantly
Coefficient alpha
A formula that provides an estimate of the reliability of a homogeneous test or an estimate of the reliability of
each dimension in a multidimensional test; A class of procedures used to estimate reliability from the internal
consistency of a measure; An internal consistency reliability coefficient based on the number of parts into which
the test is partitioned (e.g. items, subtests, or raters), the interrelationships of the parts, and the total test score
variance; Also called Cronbach's alpha, and, for dichotomous items, KR 20
Cognitive
Issues that relate to a person's ability to solve a problem using rational thought
Cognitive ability
The psychological concept that refers to such processes as perceiving, knowing, recognizing, conceptualizing,
judging, and reasoning

Assessment Glossary
Cognitive ability testing

Using a test instrument during the personnel selection process to measure a candidate’s learning and reasoning
abilities
Cognitive assessment
The process of systematically gathering test scores and related data in order to make judgments about an
individual's ability to perform various mental activities involved in the processing, acquisition, retention,
conceptualization, and organization of sensory, perceptual, verbal, spatial, and psychomotor information
Cognitive complexity
The ability to differentiate, abstract, and conceptualize
Cognitive development
Development explained through sequential stages in which individuals encounter problems or ideas which
cause cognitive conflicts that demand the individual to accommodate or change their way of thinking to become
more complex
Cognitive domain
The range of knowledge and knowledge-related skills needed for learners to achieve different types of
instructional objectives; Knowledge skills include perception, knowledge of facts, acquisition of skills, and
higher-order inference
Cohort
A group whose progress is followed by means of measurements at different points in time; A group of study
subjects, who are selected based on predetermined criteria, and who are followed over a period of time
Collaboration
To work together sharing ideas and resources, especially in a joint intellectual effort
Collective bargaining
The process by which an agent chosen by employees negotiates a formal labor agreement or settles day-to-day
labor disputes on behalf of the employees in the areas of wages, benefits, working conditions, and
administrative policy with parties representing the top elected or appointed executives of an organization
Collective case study

Studying multiple cases in one research study
Collegial
Participative and mutually respectful
Combined model
A scoring or evaluation procedure that uses features of both compensatory and conjunctive models
Commercial, norm-referenced, standardized exams

Exams with scores based on a comparison to a reference or normative group: Usually group administered
multiple-choice "objective" tests in one or more curricular areas purchased from a private vendor
Communication
Human communication is a process during which source individuals initiate messages using conventionalized
symbols, nonverbal signs, and contextual cues to express meanings by transmitting information in such a way
that the receiving party constructs similar or parallel understandings
Comparability

Assessment Glossary
The similarity of the phenomena (e.g. attributes, performance, assessments, data sources) being examined;
The amount or degree of comparability is often used to determine the appropriateness of using one
phenomenon in lieu of another and to help ensure fairness
Comparative data
Data from two or more similar groups which have been exposed to different conditions of the independent or
intervention variable
Comparative experimental model

Studies that assign a program, project, or instructional material to one group of persons and compare their
subsequent performance on some structured task to that of another group that was not exposed to the program,
project, or instructional material
Comparison group
A group of subjects who are similar to the group being studied, with the exception of exposure to the
independent variable; A group that provides a basis for contrast with an experimental group (the group of people
participating in the program or project being evaluated); The comparison group is not subjected to the treatment
or independent variable, thus creating a means for comparison with the experimental group that does receive
the treatment
Compatibility thesis
The idea that quantitative and qualitative methods are compatible
Compensation
Money or another item of value given or received as payment or reparation for a service or loss
Compensatory model
An evaluation or scoring procedure that permits trade-offs of one attribute against another; A model in which low
performance on one attribute can be offset by high performance on another; Most compensatory models have
an absolute minimal level of performance for each attribute, below which trade-offs are not permitted
Competence based
A set of knowledge, skills, or abilities usually demonstrated on a measure
Competencies
What students should know, be able to do, and value; The knowledge, skills and abilities required to perform a
specific task or function; See also Core competency
Competency
The demonstration of the ability to perform a specific task or achieve a specified criteria; A discrete unit of
learning resulting from the influence of educational experiences and mastered by an individual student; A skill,
knowledge, or experience that is suitable or sufficient for a specified purpose; Level at which performance is
acceptable; A group of characteristics, native or acquired, which indicate an individual's ability to acquire skills in
a given area; A knowledge, skill, ability, personal quality, experience, or other characteristic that is applicable to
the profession of teaching
Competency-based assessment
Measures an individual's performance against a predetermined standard of acceptable performance; Progress
is based on actual performance rather than on how well learners perform in comparison to others; See Criterion
referenced assessment
Competency test
A test intended to establish that a student has met established minimum standards of skills and knowledge and
is thus eligible for promotion, graduation, certification, or other official acknowledgment of achievement

Assessment Glossary
Competitive advantage
Characteristics that allow an organization to outperform its rivals; What an organization does better than its
competitors
Competitive strategy
How an enterprise competes within a specific industry or market; Also known as business strategy or enterprise
strategy
Complete participant
Case study method where the researcher becomes a member of the group being studied without telling the
members they are being studied
Complete-observer
Case study method where the researcher makes observations as an outsider without telling people they are
being observed
Completion
The conclusion of an educational program and acquisition of a degree; Receipt of credential (e.g. certificate,
degree)
Complex-generated response
An assessment that asks a student to perform or produce in order to demonstrate knowledge and skills; Such
assessments will not have one right answer, but instead will result in student work across a range of quality; The
assessment requires that the student engage in a task of multiple parts or steps; Scoring of the assessment
involves teacher judgment based on stated criteria for performance.
Component
One of the parts or processes in an evaluation plan, effort, or system
Composite score
A single score used to express the combination, by averaging or summation, of the scores on several different
tests; A score that combines two or more scores or results for the same or related attributes; A score that
combines several scores according to a specified formula
Comprehensive
All dimensions of a learning goal with regard to scope, content, specificity, skills, and types of thinking required
are addressed
Comprehensive sampling
Including all cases in the research study
Computational skills
The ability to identify problems in data, to reason numerically, and to apply and use data to solve problems
Computerized adaptive test

A method by which a computer selects the range of questions to be asked based on the performance of the
participant on previous questions
Computerized or computer assisted assessment

The use of computers to measure performance on some attribute, not necessarily an attribute related to
computers and technology; A common term used to describe the use of computers to support assessments

Assessment Glossary
Concept
An abstract, general notion; A heading that characterizes a set of behaviors and beliefs
Concurrent validity
The relationship of one measure to another simultaneous measure or variable assessing the same or a related
attribute; The means of determining a test or other assessment tool’s validity by comparing test scores against
actual job performance
Concurrent validity evidence

Validity evidence based on the relationship between test scores and criterion scores obtained at the same time
Conditioning
Associating a response with a previously unrelated stimulus
Confirmability
The findings of the study can be confirmed by another person conducting the same study; A
characteristic of objective research studies
Confidence interval or limits

The range around a numeric statistical value obtained from a sample within which the actual corresponding
value for the population is likely to fall, at a given level of probability; The range of scores or percentages within
which a population percentage is likely to be found; Usually expressed in a "plus or minus" fashion; A sample-
based estimate as an interval or range of values within which the true or target population value is expected to
be located (with a specified level of confidence given as a percentage); Confidence limits are the end points of a
confidence interval
Confidence level
The specific probability of obtaining some result from a sample if it did not exist in the population as a whole;
The level at or below which the relationship will be regarded as statistically significant
Confidentiality
Not revealing the identity of the participant to anyone other than the researcher and his or her staff; The
protection of data and information from persons other than those authorized to have access; Situation in which
the identity of students will not be released to other individuals or institutions beyond the teacher or others who
evaluate students; Maintaining the privacy of specific data
Configural scoring rule

A rule for interpreting a pattern of scores on two or more assessments or parts of one assessment for the same
teacher
Confirmatory factor analysis

Factor analysis working with a priori expectations in which specific expectations concerning the number of
factors and their loadings are tested on sample data
Conflict management
Conflict is viewed as the interaction of interdependent people who perceive opposition of goals, aims and
values, and who see the other party as potentially interfering with the realizations of these goals; Conflict
resolution styles include: Competition: A win-lose strategy dealing with conflict from a dominance mind set;
Accommodation: A win-lose style approaching conflict management with a more yielding mindset; Avoidance:
Involves not talking about the issue in question or avoiding the other party altogether; Compromise: When both
parties are willing to give and take in order to manage the conflict effectively; Collaboration: A win-win conflict
management style whereby both parties work toward understanding the position of the other, in order to reach
agreement

Assessment Glossary
Conflict of interest
A situation in which an evaluator's private interests affect her or his evaluative actions, or in which the evaluative
actions might affect private interests; A situation in which the private interests of someone involved in the
assessment or evaluation process (e.g. interviewer, rater, scorer, evaluator) have an impact (either positive or
negative) on the quality of the evaluation activities, the accuracy of the data, or the results of the evaluation
Confound
To confound is to fail to detect a confounding bias prior to conducting research, resulting in unreliable cause-
and-effect interpretations; A confound is the confounding variable which created the bias
Confounded
The situation in which the effect of a controlled variable is inextricably mixed with that of another, uncontrolled
variable
Confounding bias
A type of bias in which one or more initially unrecognized confounding variables turn out to have affected the
obtained results, thus rendering cause-and-effect interpretation unreliable.
Confounding variable
An unforeseen and unaccounted-for variable that jeopardizes reliability and validity of an experiment's outcome;
An uncontrolled variable that systematically varies with the independent variable; A type of extraneous variable
that was not controlled for and is the reason a particular “confounded” result is observed; A variable which
influences the dependent variable while systematically varying with the independent variable
Confucian dynamism
A cultural dimension that values a long-term orientation, persistence and thrift
Congruence analysis
The verification of data by using more than one instrument or source of data for assessing performance on the
same criterion
Conjunctive model
An evaluation or scoring procedure that requires the student or teacher to attain a minimal level of performance
on all attributes assessed.
Consensus
Agreement
Consensus group
A type of group discussion in which participants try to form a consensus on an issue
Consent
The granting of permission by an individual concerning the collection, use, retention, or access to assessment
data and information
Consequential basis of validity

The assemblage of information on the theoretical and value implications of the way that the results of testing are
used, and the appraisal of both the potential and actual social consequences of the testing, including side
effects
Consequential validity
The aspect of validity dealing with positive and negative intended and unintended issues associated with the
use and interpretation of tests

Assessment Glossary
Consistency
Obtaining the same or similar results across multiple administrations or scoring of an assessment;
Implementation of procedures in an identical or near identical manner across individuals or over time; A type of
rater effect in which an assessor or an evaluator tends to rate or to interpret different data and information in a
similar way; A consistent rater tends to assign the same grade or rating to all assessment results and products
without regard to their quality or to the scoring rubric
Constant
A single value or category of a variable; A variable which only has a single value; A certain characteristic (like
grade level) can be a variable in one study and a constant in another study
Constant comparative method

Data analysis in grounded theory research
Constituency
A group served by an organization or institution
Construct
Specific concept or variable of interest in a given research effort; An abstract or general idea inferred or derived
from specific instances; An attribute of an individual or a phenomenon that is not directly observable, but which
is theoretically based or inferred from empirical evidence; A characteristic or trait of individuals inferred from
empirical evidence; The concept or characteristic that a test is designed to measure; Something that exists
theoretically but is not directly observable; A concept developed for describing relations among phenomena or
for other research purposes; A theoretical definition in which concepts are defined in terms of other concepts;
For example: Intelligence is a construct because it is widely assumed to exist but cannot be directly observed or
measured; See Hypothetical construct
Construct domain
The set of interrelated attributes (e.g. behaviors, attitudes, values) that are included under a construct's label; A
test typically samples from a construct domain
Construct equivalent
The extent to which the construct measured by one test is essentially the same as the construct measured by
another test; Also, the degree to which a construct measured by a test in one cultural or linguistic group is
comparable to the construct measured by the same test in a different cultural or linguistic group
Construct irrelevance
Occurs when the assessment tool used to measure an educational or psychological construct includes items or
measures extraneous, or not relevant, to the construct and that cause scores to be different from what they
should be
Construct response item

An exercise for which examinees must create their own responses or products rather than choose a response
from an enumerated set
Construct under representation

Occurs when some of the aspects that represent the construct to be addressed are not included in the
assessment used to measure it
Construct validity
The degree to which a test measures what it claims or purports to be measuring; The extent to which a test or
other assessment instrument measures a particular trait; The degree to which inferences can legitimately be
made from the operationalizations in your study to the theoretical constructs on which those operationalizations
were based; Demonstrated through patterns of inter correlations among measures with high correlations

Assessment Glossary
between theoretically similar measures and low correlations between theoretically dissimilar measures; Seeks
an agreement between a theoretical concept and a specific measuring device, such as an observation; Used to
indicate that the test scores are to be interpreted as indicating the test taker's standing on the psychological
construct measured by the test; A theoretical variable inferred from multiple types of evidence, which might
include the interrelations of the test scores with other variables, internal test structure, observations of response
processes, as well as the content of the test; All test scores are viewed as measures of some construct, thus the
phrase is redundant with validity, and the validity argument establishes the construct validity of a test; The
degree to which one can infer certain constructs in a psychological theory from the test scores; The degree of fit
of a measure and its interpretation with its underlying explanatory concepts, theoretical rationales, or
foundations
Constructavism or constructavist teaching/learning

Constructavism is a learning theory that asserts that students learn best when they construct their own
understanding of the world around them; Constructavist teaching is student-centered and attempts to create
contexts in which students actively grapple with big issues and questions instead of being passive recipients of
teacher knowledge
Constructed response item or format

An assessment unit with directions, a question, or a problem that elicits a written, pictorial, or graphic response
from a student; An exercise for which examinees must create their own responses or products rather than
choose a response from an enumerated set; Short-answer items require a few words or a number as an
answer, whereas extended-response items require at least a few sentences; The examinee produces an answer
or response to a given stimulus or test question; Sometimes called "open-ended"; Also known as production
format
Constructivist theory
Posits that people build new information onto pre-existing notions and modify their understanding in light of new
data, and in the process their ideas gain in complexity and power; Constructivist theorists dismiss the idea that
students learn by absorbing information through lectures or repeated rote practice
Consultant
An individual who works independently to assist and advise client organizations with various organizational
functions and responsibilities on a fee-for-service basis
Consultative
Conducted in a manner that solicits input from various groups, but does not require actual participation in
decision-making
Contamination
A tendency for the assessor’s data, the scorer’s rating and judgments, or the evaluator’s conclusions to be
influenced or confounded by irrelevant knowledge about the student, other personnel, or other factors that have
no bearing on the student’s level of performance
Content analysis
Qualitative research method for analyzing written or verbal material to generate reliable evidence about a large
sample; Analyzing the content of media (e.g. publications and recordings) to determine the main themes being
represented, usually by counting the number of times a word or theme appears; Analysis of text documents or
verbal recordings; The process of identifying and listing, in accordance with a parsimonious classification
system, categories of expression contained in a variety of information sources; Usually researchers establish a
set of categories and then count the number of instances that fall into each category; The crucial requirement is
that the categories are sufficiently precise to enable different coders to arrive at the same results when the same
body of material is examined; Requires high inter-rater reliability to be valid; May be quantitative, qualitative, or
both; Typically, the major purpose of content analysis is to identify patterns
Content domain
Assessment Glossary
The set of organized categories characterizing subject matter under which behaviors, knowledge, skills, abilities,
attitudes, and other characteristics may be represented in specifications for assessment instruments by which
items are classified
Content learning
The acquisition of knowledge from an experience, as distinct from the process of an experience
Content-related evidence
Validity evidence based on a judgment of the degree to which the items, tasks, or questions on a test
adequately represent the construct domain of interest
Content representativeness
The extent to which the knowledge or skills being assessed represent an adequate sample from the domain of
interest
Content standards
A defined domain of educational objectives; Statements that describe what students should know or be able to
do within the content of a specific discipline or at the intersection of two or more disciplines; Broadly stated
expectations of what students should know and be able to do in particular subjects and grade levels; Define for
teachers, schools, students, and the community not only the expected student skills and knowledge, but what
schools should teach; Often describe the information and skills essential to the practice or application of a
particular discipline or content domain
Content validity
The appropriateness of the domain definition and the sampling of content, and/or the extent of congruence
between the scope of a content area that an instrument or process claims to cover and what it actually does
cover; Extent to which a measurement reflects the specific intended domain of content; The degree to which a
test or other assessment instrument used during the selection process measures the skills, knowledge and
abilities or other related job qualifications; A test has content validity if the item measures what it is intended to
measure; A term used to refer to a kind or aspect of validity required when the test user wishes to estimate how
an individual performs in the universe of situations the test is intended to represent
Contestable markets
Markets where profits are held to a competitive level due to the ease of entry into the market
Context
The set of circumstances or acts that surround and may affect a particular student, learning situation,
classroom, or school; The environment within which the teacher works; The combination of the factors
accompanying the study that may have influenced its results; For example: The geographic location of the
study, its timing, the political and social climate at that time, other relevant professional activities that were in
progress, and any existing pertinent economic conditions
Context sensitivity
Awareness by a qualitative researcher of factors such as values and beliefs that influence cultural behaviors
Contextual variables
Indicators or dimensions that are useful in describing the facts or circumstances that surround a particular
learning situation and influence a student's performance in that situation
Contextualization
The identification of when and where an event took place
Contingency planning

Assessment Glossary
The process of identifying an organization’s critical information systems and business operations in order to
develop and implement plans enabling those systems and operations to resume following a disaster or other
emergency situation
Contingency approach
The use of different administrative strategies under different conditions; The study of the relationship between
factors, such as the task an agency performs or the technology it uses, and the style of supervision, type of
organizational design, and other administrative strategies that will work best given those factors
Contingency question
An item that directs participants to different follow-up questions depending on their response
Contingency table
A table displaying information in cells formed by the intersection of two or more categorical variables
Continuous improvement
Using evaluation over time to show change for the better; cyclical evaluation of outcomes in order to repeatedly
increase achievements over time
Continuous quality improvement (CQI)

Management technique focused on meeting or exceeding customer requirements by continuous improvement
and innovation in products, processes, and services; Includes a sense of collective responsibility for learning,
mutual accountability for results, habitual listening to stakeholders, data based decision making, and an ethos
of continuous personal and institutional improvement
Continuous variable
A variable that may have fractional values (e.g. height, weight and time); Variables which can take any value
and have accuracy of measurement
Contrast effect
A type of rater affect or artifact in which an assessor or an evaluator tends to compare one student member to
another student rather than comparing that student’s level of performance to the standards
Control group
A group in an experiment that does not receives a treatment in order to compare the treated group against a
norm; A group as closely as possible equivalent to an experimental group which is exposed to all the conditions
of the investigation except for the treatment, program, project, or instructional material being studied; A group of
subjects, matched to the experimental group, which does not receive the treatment of interest
Convenience sampling
People who are available, volunteer, or can be easily recruited are included in the sample
Convergence group
A group that is responsible for incorporating the important features of alternative strategies proposed by
advocacy teams into a compromise strategy
Convergent evidence
Evidence based on the relationship between test scores and other measures of the same construct
Convergent validity
General agreement among ratings, gathered independently of one another, where measures should be
theoretically related

Assessment Glossary
Conversion tables
Tables used to convert a student's test scores from scale score units to grade equivalents, percentile ranks, and
stanines
Core competency
A broad, institutional-level outcome statement describing what students are able to do at the end of their
experience at the college; Core competencies are categories within which course and program level learning
outcomes are classified and may not be assessed directly
Core functions
The major responsibilities of the unit stated in a few succinct statements; Also know as Primary Functions
Core values
Enduring beliefs or principles that the institution’s members hold in common and endeavor to put into action;
Core values guide faculty, staff, administrators, and sometimes students in performing their work
Corporate citizenship
The contribution a company or organization makes to society through its core business activities, social
investment and philanthropy programs, and its engagement in public policy; The manner in which a company
manages its economic, social and environmental relationships and the way it engages with its stakeholders (e.g.
shareholders, employees, customers, business partners, governments, and communities) and has an impact on
the company’s long-term success
Corporate or organizational culture

The beliefs, values and practices adopted or exhibited by an organization that directly influence employee
conduct and behavior
Corporate governance
A set of processes, customs, policies, laws, and institutions affecting the way a corporation or organization is
directed, administered, or controlled
Corporate image
The way in which an organization is viewed by clients, employees, vendors or the general public
Corporate social responsibility

Concept that organizations have an obligation to consider the interests of customers, employees, shareholders,
communities, and the environment in all aspects of their operations
Corporate values
The prescribed standards, behaviors, principles, or concepts that an organization regards as highly important
Corporation
A group of individuals legally empowered to transact business as one body; Legal entities created by states
through the approval of the charters submitted by their founders
Corrective scoring
A calculation used to offset the effects of guessing in objective tests
Correlation
The extent or degree to which two variables are related; The degree to which two or more sets of
measurements vary together; The tendency for certain values or levels of one variable to occur with particular
values or levels of another variable; The degree of relationship (linear or curvilinear) between two variable,
scores, or assessments; Statistical measurement of the relationship between the changes in two or more
measured variables; A common statistical analysis (usually abbreviated as r) that measures the degree of

Assessment Glossary
relationship between pairs of interval variables in a sample and ranging from -1.00 to zero to +1.00; A non-
cause and effect relationship between two variables; A positive correlation exists when high values on one
scale are associated with high values on another; A negative correlation exists when high values on one scale
are associated with low values on another
Correlation coefficient
An index that indicates the strength and direction of the relationship between two variables; A measure of
association between two variables that can range from -1.00 (perfect negative relationship) to 0 (no relationship)
to +1.00 (perfect positive relationship)
Correlational research
A form of non experimental research in which the primary independent variable of interest is a quantitative
variable
Corroborating evidence
Comparing documents to each other to determine whether they provide the same information or reach the same
conclusion; Documentation that confirms, strengthens, and/or provides support for other documentation on the
same attribute, competency, or situation
Cost
The quantity of resources required in order to achieve a desired end
Cost-benefit analysis
A means of measuring the costs associated with a specific program, project, activity or benefit compared to the
total benefit or value derived; The analysis of well-defined alternatives by comparing their costs and benefits
when both costs and benefits are expressed in monetary terms; Each alternative is examined to see whether
benefits exceed costs, and the ratios of the alternatives are compared; The alternative with the highest benefit-
to-cost ratio is then selected; A management tool that involves calculating or estimating the monetary costs and
potential benefits of a proposed course of action; Technique designed to measure relative gains and losses
resulting from alternative policy or program options
Cost-benefit ratio
The proportional relationship between the expenditure of a given quantity of resources and the benefits derived
their expenditure; A guideline for choosing among alternatives within the rational model of economic decision
making
Cost center
An enterprise that has a manager who is responsible for cost performance and controls most of the factors
affecting cost; A separate cost accounting unit charged with providing a method of isolating and tracking costs
as a means to enhance management controls by identifying responsibility for various expenses
Cost effectiveness
The extent to which one program, project, or instructional material produces equal or better results than another
that costs about the same amount of time, effort, and resources; The extent to which an object produces the
same results as another but is less costly; Cost effective does not mean affordable because highly effective
interventions may be so expensive that without additional funding they are still cost prohibitive to an institution
even when they are the most cost effective
Cost-per-hire
Calculated measurement of the direct and indirect costs associated with filling a vacancy
Counseling
Intervention involving the provision of advice or support on a personal basis, by someone who has been trained
to provide that support

Assessment Glossary
Counterbalancing
Administering the experimental treatment conditions to all comparison groups, but in a different order
Course
An organization of subject matter and related learning experiences provided for the instruction of students on a
regular or systematic basis, usually for a predetermined period of time
Course or course-level assessment

Applying assessment techniques to the stated learning objectives of a given course; Assessment to determine
the extent to which a specific course is achieving its learning goals
Course embedded assessment

Collecting assessment data information within the classroom because of the opportunity it provides to use
already in-place assignments and coursework for assessment purposes; Involves taking a second look at
materials generated in the classroom so that, in addition to providing a basis for grading students, these
materials allow faculty to evaluate their approaches to instruction and course design; Reviewing materials
generated in the classroom to provide a basis for grading students and using them to evaluate approaches to
instruction and course design
Course objectives
Statements of what the students are expected to know or learn by the end of a course; Often focus on what the
instructor does rather than what the student will be able to do (i.e. focus on input rather than output); Often
content-based and not necessarily competency-based; Often not measurable or assessable
Course portfolio
May be developed by students or faculty; Students create portfolios by gathering a body of evidence of their
own learning and competences in a course; Used to help students become reflective thinkers; Faculty develop
course portfolios to document their own leaning in a course and then share it with students; Course portfolios
might include information illustrating how the course fits within the institution or program, as well as evidence of
student learning, the instructor’s reflections and self-assessments, and perhaps the reactions of peers in the
discipline
Course SLOs or student learning outcomes

What the student will be able to produce at the end of a course; General statement of the skills and abilities a
student will possess upon successfully finishing a course; Lowest level at which SLOs are usually assessed;
You do not “teach to the SLOs” but rather write the SLO to reflect what you think is most important for the
student to learn or be able to accomplish; Writing course-level SLOs involves considering the overarching goals
of the course, matching these goals with a particular assessment method, and articulating these overarching
goals in an SLO statement; Courses may have multiple SLOs.
Coursework cluster analysis

Looking at the patterns of coursework that students have taken and, based on what is known about the learning
outcomes associated with each of the courses a student has passed, determining whether a student is
considered as having mastered the skills and knowledge necessary to graduate
Covariate
How one variable varies with another; Usually used in removing associated effects of an external variable such
as age or family income from other variables in a study; A product of the correlation of two related variables
times their standard deviations; Used in true experiments to measure the difference of treatment between the
variables
Covariation

Assessment Glossary
A state that exists when two things vary together; The degree of covariance is expressed through measures of
association
Creaming
The process of focusing on participants who are easy to serve, with the possible consequence of neglecting
those participants who are most in need of services
Creativity
In an educational setting, creativity pertains to the development of flexible thinking patterns; Entails aspects of
problem solving and motivation to find solutions as well as the generation of multiple hypotheses
Credibility
A researcher's ability to demonstrate that the object of a study is accurately identified and described, based on
the way in which the study was conducted; Believability or confidence by virtue of being trustworthy and
possessing pertinent knowledge, skills, and experience
Credible
Worthy of confidence and acceptance by others; Usually based on the expertise, trustworthiness, and/or
reliability of the source of the evidence or judgment; Credible does not necessarily mean accurate or valid
Crises management
The practice of implementing organization-wide policies and procedures containing pre-established guidelines
for responding to catastrophic events or tragedies (e.g. fires, earthquakes, severe storms, workplace violence,
kidnappings, bomb threats, and acts of terrorism) in a safe and effective manner
Crises planning
Creating formal written plans that establish specific measures or actions to be taken when responding to
catastrophic events or tragedies in the workplace
Crises prevention
The process of an organization implementing specific plans and procedures designed to circumvent certain
disasters or emergencies
Criteria or standards
Characteristics of good performance on a particular task; Guidelines, rules, characteristics, dimensions, or
principles by which student responses, products, or performances are evaluated; Description of performance or
other indicators that demonstrate how students or service units will meet the expectations stated in the
outcomes; Descriptive benchmarks against which performance is judged; A dimension along which performance
(e.g. effective teaching) is rated or judged as successful or meritorious; Qualitative or quantitative statements
used to measure whether the program standard of competency achievement has been met; Criteria indicate
what we value in student responses, products or performances; May be holistic, analytic, general, or specific;
May be described in terms of measures such as rubrics or grades; Often stated in terms of percentages,
percentiles, or other quantitative measures; Sometimes called performance standards
Criterion
The singular of criteria; A behavior, characteristic, or quality of a product or performance about which some
judgment is made; What a teacher intended to teach and what is checked to see if students did indeed learn
what the teacher thought was being taught; A dimension along which performance is rated or judged as
successful or meritorious; A standard by which something can be judged; The standard or benchmark that you
want to predict accurately on the basis of the test scores
Criterion domain
The construct domain of a variable used as a criterion

Assessment Glossary
Criterion of falsifiability
The property that statements and theories should be refutable
Criterion referenced assessment

Performance interpreted in relation to pre specified standards; Assessment where an individual's performance is
compared to a specific learning objective or performance standard and not to the performance of other students;
Assessments which determine what the assessed can do and what they know, not how they compare to others;
Compares a student's performance to specific standards; Reports on how well students are doing relative to a
predetermined performance level on a specified set of educational goals or outcomes included in the curriculum;
Tells us how well students are performing on specific goals or standards rather than just telling how their
performance compares to a norm group of students nationally or locally
Criterion-referenced measurement
Measurement that provides data on the extent to which particular objectives or criteria are achieved
Criterion referenced score interpretation

A score interpretation that does not depend upon the score's rank within, or relationship to, the distribution of
scores for other examinees
Criterion referenced tests

Tests whose scores are interpreted by referral to well defined domains of content or behaviors, rather than by
referral to the performance of some comparable group of people; A test in which the results can be used to
determine a student's progress toward mastery of a content area; Performance is compared to an expected
level of mastery in a content area rather than to other students' scores; A measurement of achievement of
specific criteria stated as levels of mastery; Test that allows its users to make score interpretations in relation to
a functional performance level, as distinguished from those interpretations that are made in relation to the
performance of others; Examples include comparisons to cut scores, interpretations based on expectancy
tables, and domain-referenced score interpretations; A measurement of achievement of specific criteria or skills
in terms of absolute levels of mastery; The focus is on performance of an individual as measured against a
standard or criteria rather than against performance of others who take the same test, as with norm-referenced
tests
Criterion related validity

The correlation or extent of agreement of the test score from an assessment with one or more external variables
that measure the attribute being assessed; Used to demonstrate the accuracy of a measuring procedure by
comparing it with another procedure which has been demonstrated to be valid; Also referred to as instrumental
validity
Criterion-related validity evidence

Validity evidence based on the extent to which scores from a test can be used to predict or infer performance on
some criterion such as a test or future performance
Critical case sampling

Selecting what are believed to be particularly important cases
Critical ethnography
A style of discourse and analysis embedded within conventional ethnography in which the researcher chooses
between conceptual alternatives and value-laden judgments to challenge research, policy, and other forms of
human activity; Critical ethnographers attempt to aid emancipation goals, negate repressive influences, raise
consciousness, and invoke a call to action that potentially will lead to social change
Critical incident
A significant and observable episode or performance (effective or ineffective) in a student’s career that alters the
direction of subsequent behaviors, activities, or events

Assessment Glossary
Critical incident appraisal

The use of documentation concerning critical incidents when evaluating and making decisions about a student’s
current and potential performance
Critical information
Knowledge about the subject being evaluated or the assessment instruments, evaluation process, and/or
working conditions and learning/teaching context that must be known to avoid incorrect interpretations of results
Critical reflection
Action research process where participants and researchers in a project think back over a project, critically
comment on what has happened in the project, and then use the new learning to improve future activities
Critical research paradigm

Focuses on a critical understanding of the situation or practice being researched in order to plan for
transformative action; The critical research approach is situated in the context of actual people and practices
and emphasizes social change
Critical success factors

The key items that must be met in order to successfully achieve a specific objective; Those few things that must
go well if a project or organization is to succeed; Also called key success factors
Critical thinking
The ability to integrate previous knowledge to identify and weigh possible responses to a novel situation; Ability
defined by the seven major categories of interpretation, analysis, evaluation, inference, presenting arguments,
reflection, and dispositions; Within each of these categories are sets of skills and sub skills
Cronbach's alpha
One the measures of internal consistency that falls under the broad heading of coefficient alpha
Cross-case analysis
Searching for similarities and differences across multiple cases
Cross-sectional research
A study, which measures a population at a specific point in time or over a short period of time; An alternative to
longitudinal research
Cross-sectional survey
Involves observations of a sample, or cross-section, of a population or phenomenon that are made at one point
in time
Cross-tabulation, crosstabs, cross partitions, or crossbreaks

A way of arranging data about categorical variables in a matrix so that relations can be more clearly seen
Cross validation
The application of a scoring system or set of weights empirically derived in one sample to a different sample
drawn from the same population to investigate the stability of relationships based on the original weights
Crosswalk
A research tool used to guide analysis and reporting, particularly when there are multiple data sources; A
crosswalk does not present any of the findings or results, just the types of information that has been gathered
from the different data sources

Assessment Glossary
Cultural competence
A set of values, behaviors, attitudes, and practices which enable people to work effectively across racial/ethnic
and cultural lines
Cultural differences
The diverse set of behaviors, beliefs, customs, traditions, language, and expressions characteristic of people
who are members of the same group; Often associated with groups of people defined by a particular race,
ethnicity or national origin
Cultural relativity or relativism

The perspective that the appropriateness of any behavior must be evaluated from the cultural context of its
occurrence; Opposite of ethnocentrism, which interprets and judges the behavior of others from the perspective
of the observer’s culture; Technique used by anthropologists to control for their own ethnocentrism (or observer
bias) when studying other cultures and interpreting the behavior of individuals from other cultures; Principle that
an individual human's beliefs and activities should be understood in terms of his or her own culture; Studying
another culture from its point of view without imposing our own cultural views; Most commonly misinterpreted
and falsely defined as the belief that there are no universal principles of right or wrong because the meanings of
‘right’ and ‘wrong’ are solely determined by one’s culture; Often misused to dismiss without valid justification
those values which are inconvenient to achieving the goals of an individual or group
Cultural sensitivity
An acceptance and appreciation for the customary beliefs, social forms, and material traits of different racial,
religious, or social groups as well as of one’s own cultural group
Culture
A system of shared beliefs, values, practices, perspectives, folk knowledge, language, norms, rituals, and
material objects and artifacts that the members of a group use in understanding their world and in relating to
others; A group’s shared assumptive reality; The shared values, understandings, assumptions and beliefs held
by members of a certain group or society that produce the norms that shape the behavior of people in that
group; The collection of beliefs, expectations, and values learned and shared by group members and passed on
from one generation to another
Culture-fair test
A test devised to exclude specific cultural stimuli so that persons from a particular culture will not be penalized
or rewarded on the basis of differential familiarity with the stimuli
Culture of assessment
An environment in which continuous improvement through assessment is expected and valued
Culture of evidence
An environment in which using research and/or assessment results to guide policy and management decisions
is expected and valued
Curricular validity
The extent to which the items on the assessment or test measure the content of a local curriculum, or the extent
of agreement between the test coverage (topics, breadth and depth, skills, cognitive complexity) and the goals
and objectives of the curriculum
Curriculum
The knowledge, skills, attitudes, behaviors, and values students are expected to learn from schooling; Includes
statement of expected student outcomes, descriptions of material and activities, and the planned sequence that
will be used to help students acquire the expected outcomes; A comprehensive overview, including activities
planned for delivery to the students, the scope of content, the sequence of materials, interpretation and balance
of subject matter, and motivational, instructional, and assessment techniques to be used; A set of ordered

Assessment Glossary
intended learning outcomes; All of the instruction, services, and activities provided for students through formal
schooling including but not limited to: content, teaching methods and practices, instructional materials and
guides, the physical learning environment, assessment and evaluation, time organization, leadership, and
controls; Curriculum includes planned, overt topics of instruction as well as unseen elements such as norms and
values taught by the school and through classroom interactions between the teacher and learner, hidden social
messages imbedded in the curriculum materials themselves, and the material that is not included in the overt or
planned curriculum
Curriculum alignment
The degree to which a curriculum's scope and sequence matches a target outcome or assessment measure
Curriculum embedded or learning embedded assessment

Assessment that occurs simultaneously with learning, such as by completing projects, portfolios and exhibitions;
Occurs in the classroom setting with students unable to tell whether they are being taught or assessed;
Assessment tasks or tests developed from or inherent within the curriculum or instructional materials
Curriculum framework
A document outlining content strands and learning standards for a given subject area which provides a structure
from which lessons and curricula can be organized and presented to the student
Curriculum map or mapping

A matrix relating program-level student learning objectives or outcomes (usually enumerated in individual rows)
to the courses and/or experiences that students take in progress to graduation (usually captured in columns); A
matrix showing the coverage of each program learning outcome in each course; May also indicate the level of
emphasis of each outcome in each course; A matrix representation of a program's learning outcomes showing
where the outcomes are taught within the program
Cut score or critical score

A score which a student needs to achieve to demonstrate minimal competency; A specified point in a predictor
distribution of scores below which candidates are rejected or considered not to have reached a minimum
standard of performance; Score used to determine the minimum performance level needed to pass a
competency test; A specified point on a score scale, such that scores at or above that point are interpreted or
acted upon differently from scores below that point
Cutting score
A score that marks the difference between two levels of teaching performance (e.g. good and excellent); When
the difference is between minimally acceptable and not acceptable, or pass and not pass, it is referred to as a
cut score, critical score, or passing score
Cycle
Collecting data and learning from it over and over; A round of assessment, evaluation/analysis,
intervention/planning and communication which is repeated

Assessment Glossary
D
Data
Discrete units of information, usually numbers; Recorded observations, usually in numeric or textual form;
Information collected by a researcher; The information and evidence gathered during the assessment process
for use in determining the level of learning outcomes
Data access
Conditions under which access to information is provided, including who has the access; Extent to which the
evaluator will be permitted to obtain data during the course of an evaluation
Data analysis
Process of organizing, summarizing, and interpreting numerical, narrative, or artifact data, so that the results
can be validly interpreted and used to guide future development of students
Data acquisition skills

The ability to use references (including the Internet) to find information and to judiciously select valid data
sources
Data collection procedures

Set of steps used to obtain quantitative or qualitative information about the knowledge, skills, attitudes, or
behaviors possessed by a student
Data integration
The merging of related data for use in scoring, judging, and evaluating
Data mining
The use of technological applications to aid and support decision making by locating, manipulating and
synchronizing information buried deep within corporate databases
Data set
A set of data; A collection of related data items
Data sources
Persons, documents, products, activities, events, and records from which the data are obtained
Data triangulation
The use of multiple data sources
Data warehouse
A system used to collect operational data from different functions, areas or business units into a central
database
Database
A collection of information/data, often organized through tables, within a computer's mass storage system;
Databases are structured in a way to provide for rapid search and retrieval by computer software
De-layering
An organizational restructuring strategy meant to reduce the organization’s existing levels of managers or
supervisors

Assessment Glossary
Debriefing
A post study interview in which all aspects of the study are revealed, any reasons for deception are explained,
and any questions the participant has about the study are answered; The simplest form is a short recapitulation
of what subjects have done and why they have done it; Part of the research procedure which is an important
aspect of ethicality in research; See also dehoaxing
Deception
Misleading or withholding information from the research participant; The deliberate concealment of the true
purpose of a piece of research, often aided by a cover story delivered at the briefing; Generally only considered
ethical if the deception is necessary to avoid demand characteristics, reduce experimenter effects, or otherwise
control confounding
Decentralization
The process of assigning decision-making authority to lower levels within the organizational hierarchy
Decile
Any one of the nine points (scores) that divide a distribution into ten parts, each containing one-tenth of all the
scores of cases; Every tenth percentile (the first decile is the 10th percentile, the second decile is the 20th
percentile, the third decile the 30th percentile, etc.)
Decision consistency coefficient

Similar to the reliability coefficient; A calculated value that tells the extent to which the decision results (e.g.
classifications) would be the same if the process were repeated; Often used in either-or decision situations (e.g.
mastery-non mastery); A coefficient of zero (0) means no consistency and a coefficient of one (1.0) means fully
consistent
Decision tree
Technique that identifies various possible outcomes given the risks associated with each decision option
Decision rules
Rules for choosing between optional interpretations or courses of action given certain evidence; For example: A
rule by which teachers pass or fail students in a course based on their test scores and other performances in the
course, or a rule by which an evaluator decides that the difference between the test scores of students exposed
to different programs is statistically significant
Deductive reasoning
The process of drawing a specific conclusion from a set of premises; A form of reasoning in which conclusions
are formulated about particulars from general or universal premises; The ability to extract certain rules based on
a sequence of experiences or observations and apply those rules to other similar situations; Deriving
conclusions and hypotheses from the data; Data is collected and analyzed before assuming a hypothesis
Deductive method
A top-down or confirmatory approach to research
Defensible
An action, conclusion, or statement that is explainable or justifiable, based on a solid foundation and policy,
explicit evaluation procedures, valid assessments, and sound evaluation and decision-making practices
Degree
A title conferred on students by a college, university, or professional school on completion of a program of study
Degrees of freedom (df)

Assessment Glossary
The number of components of a statistic which are free to vary; A mathematically complex concept which is
usually simple to determine and use (e.g. the degrees of freedom for a variable sampled at n different intensities
is simply n-1)
Dehoaxing
Informing study participants about any deception used and the reasons for its use; Important component of
debriefing when the research involved any intentional deception
Delegation
The process of assigning tasks or projects to subordinates and clearly dictating expected outcomes and time
frame for completion
Deliberative poll
A method of obtaining informed polling data where a group of randomly selected citizens meet, discuss an issue
for some time, and then vote on the issue
Demand
The number of units of a particular product or service that can be sold at the price the company is planning to
charge
Demand characteristics
Confounding variable resulting from the experimental set-up itself, including the behavior or appearance of the
researcher(s); Subtle environmental factors which interact with the motivational state of subjects during the
research experience to make the observed behavior non-natural in some important respect; The demonstrable
fact that the research setting may evoke behavior you did not intend to evoke and which is not a result of the
independent variable
Demand forecasting
A strategy intended to project an organization’s anticipated workloads over a specific period of time in order to
determine future personnel needs
Demographics
The characteristics of human populations and population segments, especially when used to identify consumer
markets
Demonstrated competence
A student who has shown the ability to perform a specific task or skill at a specified level using some
predetermined measure
Deontological approach
An ethical approach that says ethical issues must be judged on the basis of some universal code
Departmentalization
The process of dividing an organization’s labor, functions, processes or units into separate groups
Dependability
Being able to account for changes in the design of the study and the changing conditions surrounding what was
studied; A measure of how consistent the results obtained in an assessment are in a criterion-referenced
evaluation; Consistency of decisions in relation to pre specified standards
Dependent variable (DV)

A variable that is presumed to be influenced by one or more independent variables; A variable that is considered
to be an effect or a variable that is predicted; A variable that receives a stimulus or treatment and is measured
for the effect the treatment has had on it; A measure that is assumed to vary as a result of some influence (often

Assessment Glossary
referred to as the independent variable); In experimental research, the researcher assumes the dependent
variable or factor is caused by (or depends upon) another variable (the independent variable); Sometimes called
the outcome variable
Delphi technique
A method for obtaining group consensus involving the use of a series of mailed questionnaires and controlled
feedback to respondents which continues until consensus is reached
Deployment
How a system utilizes tools throughout the operation
Depth or in-depth interview

Qualitative research technique which uses long, probing interviews without a formal questionnaire; Also known
as a depth
Depth of knowledge
The extensiveness of knowledge and understanding within a certain area of content
Derived score
A score to which raw scores are converted by numerical transformation (e.g. conversion of raw scores to
percentile ranks or standard scores); A test score pertaining to a norm group (such as a percentile, stanine, or
grade equivalent) that is an outgrowth of the scale scores; May be useful as descriptors, but because they are
not calibrated on an equal-interval scale they cannot be added, subtracted, or averaged
Description
Attempting to describe the characteristics of a phenomenon
Descriptive research
Research focused on providing an accurate description or picture of the status or characteristics of a situation or
phenomenon
Descriptive statistics
Statistics that focus on describing, summarizing, or explaining data; A way of summarizing data by letting one
number stand for a group of numbers; Set of mathematical procedures designed to present research data in
summary form which is not part of hypothesis testing (e.g. mean, median, mode, range, and standard deviation);
Commonly presented in tables and graphs to summarize data; Most common graphical displays are bar charts,
box-and-whisker charts, histograms, and pie charts
Descriptive validity
The factual accuracy of an account as reported by the researcher
Descriptors
Statements of expected performance at each level of performance for a particular criterion in a rubric; A set of
signs used as a scale against which a performance or product is placed in an evaluation; Allows assessments to
include clear guidelines for what is and is not valued in student work; Typically found in analytic rubrics
Desensitizing
Helping study participants deal with and eliminate any stress or other undesirable feelings that the study might
have created
Design
A plan related to operations; Organization and structuring of resources and actions to achieve a desired end
outcome; A representation of the set of decisions that determine how a student evaluation is to be conducted
(identifying purposes and use of the information, developing or selecting of assessment methods, collecting

Assessment Glossary
assessment information, judging and scoring student performance, summarizing and interpreting results,
reporting evaluation findings, and following up on evaluation results); A plan for conducting an evaluation (data
collection schedule, report schedules, questions to be addressed, analysis plan, management plan, etc.);
Designs may be either preordinate or emergent
Design flexibility
A quality of an observational study that allows researchers to pursue inquiries on new topics or questions which
emerge from the initial research
Desired outcomes
The results or products that a training program, process, instructional unit, or learning activity strives to achieve,
as defined in measurable terms; See also Goals and Objectives
Determinism
The proposition that all events have causes
Development
The progression of an individual through various phases of life
Developmental assessment
Evaluates students throughout their undergraduate experience rather than only at the end of their program; A
true longitudinal approach would evaluate students at several points in the program, providing baseline
information about students’ knowledge and skills, and identifying changes in students’ growth and development
Developmental delay
Failure of a developing organism to reach, achieve, or display some physical, cognitive, or behavioral
developmental norm at the expected chronological age
Developmental norm
Age-related expectation of mental or physical ability derived from prior experience or research with the parent
population; Vitally important in detecting developmental delay and part of the standardization process required
for major psychometric tests; See Norm
Developmentally appropriate
Characteristic of an assessment task that reflects the skills and knowledge which teachers and students, with a
given level of training and experience, have had a reasonable chance of acquiring or learning
Deviation
Short for deviation from the mean; The distance between the mean and a particular data point in a given
distribution
Diagnostic test
Tests designed to identify where a student is having difficulty with an academic skill; A test used to analyze or
locate an individual's specific areas of weakness or strength, to determine the nature of his weaknesses or
deficiencies, and, wherever possible, to suggest their cause; A test intended to locate learning difficulties or
patterns of error; May be used to identify a need for remedial instruction; Primarily used to identify needs and to
determine prior knowledge of participants; Usually occur prior to a learning experience; An intensive, in-depth
evaluation process with a relatively detailed and narrow coverage of a specific area; Usually the purpose of this
test is to determine the specific learning needs of individual students and to be able to meet those needs
through regular or remedial classroom instruction; Measurements and measurement packages designed to
assist the assessment phase of patient or student management; Required to have demonstrated or known
sensitivity (ability to detect someone who needs to be detected), specificity (ability to exclude people who need
to be excluded), positive predictive value (a measure of how many of those detected as positive actually are

Assessment Glossary
positive), and negative predictive value (a measure of how many of those detected as negative actually are
negative); Examples include intake assessments, placement tests, and entrance exams
Diagramming
Making a sketch, drawing, or outline to show how something works or to clarify the relationship between the
parts of a whole
Dialogue
Self-reflective exchanges engaged in by the college community and characterized by a free exchange of ideas
without the purpose of defending or deciding on a course of action; Group discussions, often with facilitation,
among colleagues designed to explore complex issues, increase group intelligence, and facilitate group
learning; An essential process in SLO development and assessment; ACCJC accreditation theme
Differential attrition
A differential loss of participants from the various comparison groups; Participants who drop out are different
from those who stay in the study
Differential functioning
Characteristic of an assessment approach, task, instrument, or evaluation system that yields higher results for
one group than another group, even though both groups have the same level of ability or competence on that
aspect of teaching
Differential impact
The varying impact of policy or research on different groups; Impact which is not systematic across groups
Differential influence
When the influence of an extraneous variable is different for the various comparison groups
Differential item functioning (DIF)

A statistical procedure used to identify items that function differently for different groups; Not just item bias
Differential prediction
Extent to which a measure estimates future performance on the same attribute differently for two or more
groups of teachers who vary on relevant characteristics, such as years of teaching experience or special training
completed
Differential selection
Selecting participants for the various treatment groups that have different characteristics
Differentiated instruction
A model of instruction that responds to the different readiness levels of students in the classroom; It supports a
more individualized education for students by creating a variety of activities and assignments that meet
students’ diverse needs
Difficulty or facility
A statistical property indicating the level of a question from 0.0 to 1.0; Calculated as the average score for the
question divided by the maximum achievable score; A facility of 0.0 means that the question is very hard (no-
one got it right), 1.0 means that it is very easy (no-one got it wrong), and 0.5 is considered ideal
Diffusion curve
The rate over time at which innovations are copied by competitors
Digital divide

Assessment Glossary
The increasing differentiation between groups or categories of people resulting from the differing ability of their
members to utilize recent technological advances to their own benefit
Dimensions
Aspects or categories in which performance in a domain or subject area will be judged; Separate descriptors or
scoring methods may apply to each dimension of the student's performance assessment; Desired knowledge or
skills measured in an assessment which are usually represented in a scoring rubric; For example, a
measurement of student teamwork skills on a performance assessment might include 6 dimensions: adaptability
(recognizing problems and responding appropriately), coordination (organizing team activities to complete a task
on time), decision making (using available information to make decisions), interpersonal (interacting
cooperatively with other team members), leadership (providing direction for the team), and communication
(clearly and accurately exchanging information between team members)
Direct assessment
Method of gauging student achievement of learning outcomes through evaluation of student work products;
Gathers evidence about student learning based on student performance that demonstrates the learning itself;
Can be value added, related to standards, qualitative or quantitative, embedded or not, and/or using local or
external criteria; Examples include written assignments, classroom assignments, presentations, test results,
projects, logs, portfolios, and direct observations
Direct costs
The costs directly attributed to a particular product, program, service, or activity
Direct evidence, measures, or methods

Evidence that shows directly that a student has learned; Evidence produced directly by students; Measures of
student leaning requiring students to display or demonstrate their knowledge and skills as they respond to the
instrument; Processes employed to assess student learning directly by requiring students to demonstrate
knowledge and skills; Students or learners display knowledge and skills as they respond directly to the
instrument itself; Examples include essays, tests, portfolios, and demonstrations
Direct measures of learning

See direct evidence
Directional alternative hypothesis

An alternative hypothesis that contains either a greater than sign (>), or a less than sign (<)
Disaggregation
The division of a collective body into its component parts
Disaster recovery plan

A set of guidelines and procedures to be used by an organization for the recovery of data lost due to severe
forces of nature such as earthquakes, fires, tornadoes, floods or hurricanes
Discourse community
A community of scholars and researchers in a given field who respond to and communicate to each other
through published articles in the community's journals and presentations at conventions; All members of the
discourse community adhere to certain conventions for the presentation of their theories and research
Discrepancy
Difference in results between two or more raters or scorers on the same assessment, or between two or more
evaluators concerning the same teacher
Discrete variable

Assessment Glossary
A variable that is measured solely in whole units (e.g. gender and siblings); Variables which can have only
certain values (e.g. whole numbers); One of the two sub-classes of interval/ratio data; See also Continuous
variable
Discriminant analysis
A statistical technique which allows the researcher to categorize differences between two or more groups with
respect to several variables simultaneously; One of the four types of multivariate methods
Discriminant evidence
Evidence that the scores on your focal test are not highly related to the scores from other tests that are
designed to measure theoretically different constructs
Discriminate validity
The lack of a relationship among measures which theoretically should not be related
Discrimination index
An index that indicates how well an item distinguishes between the students who understand the content being
assessed and those who do not; Positive discrimination indicates that the item or task is discriminating in the
same way as the assessment method of which it is a part
Discrimination parameter
The property that indicates how accurately an item distinguishes between examinees of high ability and those of
low ability on the trait being measured; An item that can be answered equally well by examinees of low and high
ability does not discriminate well and does not give any information about relative levels of performance
Disjunctive model
An evaluation or scoring procedure that requires the student or teacher to achieve a minimal level of
performance on only one of the attributes assessed
Disordinal interaction effect

An interaction effect that occurs when the lines on a graph plotting the effect cross
Dispersion
A measure of how tightly a distribution is clustered around its mean
Disproportional stratified sampling

Type of stratified sampling in which the sample proportions are made to be different from the population
proportions on the stratification variable
Dissemination
The communication (written, oral, and/or other) of the actions of evaluators, assessment outcomes, or audit
results to all right-to-know audiences in order to foster knowledge of the findings
Distance or distributed learning

Education which occurs outside of the traditional classroom setting
Distinctive competence
Special skills and resources that generate strengths that competitors cannot easily match or imitate
Distractors
The incorrect alternatives or choices in a selected response item
Distribution or frequency distribution

Assessment Glossary
The range of values of a particular variable; A tabulation of the scores (or other attributes) of a group of
individuals to show the number (frequency) of each score, or of those within the range of each interval
Diversification
A strategy to increase the types of business, services or products a company is providing
Documentation
Proof of an event; Evidence that something happened; Collection or compilation of all tangible materials,
records, and forms used in an assessment or evaluation; Naturalistic assessment process which involves
recording classroom observations over time, across learning modalities, and in coordination with colleagues;
The body of literature (e.g. tests manuals, manual supplements, research reports, publications, user's guides,
etc.) made available by publishers and test authors to support test use
Documenting
The process of recording and providing tangible evidence and information about the performance of an
individual, course, program, department, treatment, institution, process, or other entity
Domain-referenced test
A test that allows users to estimate the amount of a specified content domain that an individual has learned;
May be based on sets of instructional objectives; A test in which performance is measured against a well-
defined set of tasks or body of knowledge (domain); Domain-referenced tests are a specific set of criterion-
referenced tests and have a similar purpose
Dotted line relationship

An organizational structure whereby a person reports to one manager but also has responsibilities to another
Double blind
A research design in which both the experimenters and participants are ignorant of the true purpose of the
research; Used when experimenter effects or other factors might bias the results a study
Downsizing
The process of reducing the employer’s workforce through the elimination of positions, management layers,
processes, functions, etc.
Dropout
A person who does not complete a study or program
Dynamic information
Information that is characterized by continuous change, information activity, or progress
Dynamic systems
Systems in motion; Most dynamic systems, like all living systems, are open systems; A complex interactive
system evolving over time through multiple modes of behavior and following certain rules (e.g. the cardio
vascular system); Qualitative observational research of dynamic systems is not concerned with having straight-
forward, right or wrong answers, thus change in a study is common because the researcher is not concerned
with finding only one answer
Dynamic or dynamical systems theory

An area of mathematics used to describe the behavior of complex systems by employing differential and
difference equations

Assessment Glossary
E
Ecological validity
The ability to generalize the study results across settings
Economic benefit
The amount of money an individual receives from working; The gains in income or profits over time due to an
action or decision
Economic indicator
Statistical data representing a broad range of economic trends
Economic return
Financial benefit of higher paying employment as a result of educational attainment; Financial benefit of an
investment of resources (e.g. time, effort, or money)
Economies of integration
Cost savings generated from joint production, purchasing, marketing, or control
Economies of scale or size

A theory that as a company gets larger, its operations and associated unit costs decrease; Production costs per
unit decreases as the number of units produced increases; When increased levels of production result in
decreased average costs of production
Economies of scope
Cost reductions allowed by the products of two or more enterprises being produced from shared resources
Education-to-education
Programs and services provided by a learning-dedicated environment to facilitate learning and development
through a successful transition of students from school to further education
Education-to-work
Through contextual, applied, and focused educational experiences students are provided with the knowledge
and skills for employment
Educational objective
A statement describing the knowledge, skill, attitude, or behavior a student is expected to learn or perform and
the content on which it will be performed as a result of instruction
Educational providers
Institution, whether university, corporate, or virtual, whose programs or services produce student learning or
development
Educational success
Increasing student learning and/or development; Meeting competency; Obtaining credit, credential, or degree
Effect size
The difference between the mean of the experimental groups and the mean of the control groups in standard
deviation units
Effect size indicator

Assessment Glossary
A measure of the strength of a relationship
Effective
An attribute of institutions, teachers, programs, and approaches that meet their stakeholders' needs; The mark
of effectiveness in higher education is desirable change in students
Effectiveness
Extent to which an individual, program, activity, or organization is achieving or failing to achieve its stated
objectives; A measure of the likely actual benefit arising from a given remediation program or intervention
Effects
Measures of the change in response produced by a change in the level of a factor
Efficacy
A measure of the theoretically maximum benefit arising from a given remediation program or intervention under
ideal conditions
Efficiency
Relationship between inputs and outputs; Resource management indices
Efficient
Producing more educational output per dollar; An attribute of institutions, teachers, programs, and approaches
that balance effectiveness against considerations of costs (i.e., are effective with a minimum use of resources)
Elasticity of demand
The change, expressed as a percentage, in the quantity of demand for a product or service divided by the
change in the price being charged
Electronic portfolio or e-portfolio

A collection of work developed across varied contexts over time saved in an electronic format; May advance
learning by providing students and/or faculty with an easier way to organize, archive and display pieces of work;
Increasingly popular because they offer practitioners and peers the opportunity to review, communicate and
assess portfolios in an asynchronous manner
Element
The basic unit that is selected from the population
Email or e-mail discussion group

A discussion group whose messages are distributed by automated, mass-distributed email programs
Embedded assessment
Means of gathering information about student learning that is built into and a natural part of the teaching-
learning process; Often uses for assessment purposes classroom assignments that are otherwise evaluated to
assign students a grade; Can assess individual student performance or aggregate the information to provide
information about the course or program; Can be formative or summative, quantitative or qualitative (e.g. an
existing course research paper is graded for content and style as part of the course grade, but it is also
assessed for ability to locate and evaluate Web-based information as part of a college-wide information literacy
SLO assessment); Collecting assessment data information within the classroom using in-place assignments and
coursework; Using materials generated in the classroom to allow faculty to evaluate instruction techniques and
course design in addition to providing the basis for grading students; Embedded assessments need not be a
part of the student’s grade; Collecting assessment data information within the classroom because of the
opportunity it provides to use already in-place assignments and coursework for assessment purposes; For
example: Including questions for assessment purposes within existing course exams; Involves taking a second

Assessment Glossary
look at materials generated in the classroom so that, in addition to providing a basis for grading students, these
materials also allow faculty to evaluate their approaches to instruction and course design
Emergency planning
The process of establishing specific measures or actions to be taken when responding to catastrophic events or
tragedies (i.e. fire, earthquake, severe storms, workplace violence, kidnapping, bomb threats, acts of terrorism,
or other emergency situations) in the workplace
Emergent design
An implementation plan in which the specification of every step depends upon the results of previous steps;
Sometimes also known as a cascading or rolling design
Emic
The intrinsic differences within a culture which are meaningful to group members
Emic perspective
The insider’s perspective
Emic term
A special word or term used by the people in a group
Empathic neutrality
A quality of qualitative researchers who strive to be non-judgmental when compiling findings
Empirical
Based on observation, experiment, or experience
Empirical data
Data obtained by actual observation rather than by conjecture; Data from the evidence of the senses
Empirical research
The process of developing systematized knowledge gained from observations that are formulated to support
insights and generalizations about the phenomena under study
Empirical statement
A statement based on observation, experiment, or experience
Empiricism
The idea that knowledge comes from experience
Employer of choice
A term used to describe a public or private employer whose practices, policies, benefits and overall work
conditions have enabled it to successfully attract and retain talent because employees choose to work there
Employer satisfaction
Employer approval of the worksite performance of educational/technical program completers in their employ
Empowerment evaluation
An evaluation approach that includes collaborative and training functions within a goal of the empowerment of
management and program staff to continuously assure quality of services
Enabling techniques
A set of methods usually used in focus groups to help people produce ideas and give their opinions indirectly

Assessment Glossary
Enlightenment effects
Type of confounding where the possibility that prior exposure to the study area might influence the behavior
being tested
Enterprise resource planning (ERP)

A series or set of computer applications used to help monitor and manage the operations of an enterprise
Entitlement grants
Grants that provide assistance to persons who meet certain criteria
Entitlement programs
Programs that provide a specified set of benefits to those who meet certain eligibility requirements
Entry level
Information collected by programs about potential students to determine their qualifications for entering a
program; In some cases, information is also used to select the appropriate level for students to begin their
studies; Information collected for entry-level decisions about individual students can provide a good starting
point for assessing academic programs; Lowest level of employment in an industry or career path
Enumeration
The process of quantifying data
Environmental analysis
An analysis of the environmental factors that influence a firm’s operations
Environmental opportunity
An attractive area for a firm to participate in where the firm would enjoy a competitive advantage
Environmental scanning
A review process of examining environmental factors at a given point in time and in a specific setting; A process
that systematically surveys and interprets relevant data to identify external opportunities and threats; To monitor,
evaluate, and disseminate information from the external environment to key people within an organization
Environmental threat
An unfavorable trend or development in the firm’s environment that may lead to an erosion of the firm’s
competitive position
Envisioning or visioning
A set of qualitative techniques used for helping organizations or communities clarify their visions of desirable
futures
Epistemology
The study of how knowledge is generated and justified; How something is learned and believed as true
Equal employment opportunity

Refers to efforts to eliminate employment discrimination on the basis of race, ethnic background, sex, age, or
physical handicap; Ensures that all persons have an equal chance to compete for employment and promotions
based on job qualifications
Equal-interval scale
A scale marked off in units of equal size that is applied to all groups taking a given test, regardless of group
characteristics or time of year

Assessment Glossary
Equal probability selection method

Any sampling method where each member of the population has an equal chance of being selected
Equating
A statistical process used to convert scores on two or more alternate forms of an assessment instrument to a
common score for purposes of comparability and equivalence; Statistically adjusting the scores of a test to make
scores from two tests comparable
Equitable
That which is fair, impartial, and just, and which provides equal opportunity for all
Equity
A quality or state that is fair, impartial, and just; A criterion for allocating resources on the basis of fairness;
Concern for fairness; Assessments are free from bias or favoritism; Assessments are reviewed for stereotypes,
situations that may favor one culture over another, excessive language demands that prevent some students
from showing their knowledge, and the potential to include students with disabilities or limited English
proficiency
Equity of opportunity
Providing all students with the same opportunity to succeed
Equity of outcomes
Ensuring that all students meet or exceed the same high standard
Equivalence
The comparability of two or more parallel measures that have been designed to assess the same aspect of
learning/teaching and to yield similar evaluation results regardless of the measure used or the scoring/rating
procedure applied; Two non-identical items which have the same meaning
Equivalency reliability
The extent to which two items measure identical concepts at an identical level of difficulty
Equivalent-forms reliability
The consistency of a group of individuals’ scores on two equivalent forms of a test measuring the same thing
Error
The difference between a person’s true score and his or her observed score; The extent to which a score,
assessment, or calculation is incorrect or inaccurate; Difference between an observed score and a predicted or
estimated score
Error of measurement
The difference between an observed score and the corresponding true score or proficiency; The difference
between a learner/teacher's obtained score and his/her true score on an assessment that is due to factors
beyond the control of that teacher; Such factors include lack of reliability in the assessment instrument or
process, variability of settings of the assessment, limited sampling of teacher performance, bias of the assessor,
rater effects, and interactions among such factors
Error variance
The estimate of variance of the distribution of the population of individual cases based on the variation among
the scores within a group sample; Same as within group variance
Essay test

Assessment Glossary
A test that requires students to answer questions in writing; Responses can be brief or extensive; Tests for
recall and ability to apply knowledge of a subject to questions about the subject, rather than ability to choose the
least incorrect answer from a menu of options
Estimate
An indication of the value of an unknown quantity based on observed data; An approximation of a true score,
parameter, or value; The particular value of an estimator that is obtained from a particular sample of data and
used to indicate the value of a parameter; Because no instrument or statistical procedure can provide an exact
(or true) score or value, essentially all data are estimates; The smaller the error of measurement, the more
precise the estimate of the true score or value
Estimation
Estimation is the process by which sample data are used to indicate the value of an unknown quantity in a
population; Results of estimation can be expressed as a single value (point estimate) or a range of values
(known as a confidence interval)
Estimator
An estimator is any quantity calculated from the sample data which is used to give information about an
unknown quantity in the population (e.g. the sample mean is an estimator of the population mean)
Ethical
Performing the evaluation or behaving in accordance with a moral code of conduct that addresses such issues
as the well-being of the evaluated, the good of the institution and its community, and the innate rights of
individuals
Ethical skepticism
An ethical approach that says concrete and inviolate moral codes cannot be formulated
Ethics
The principles and guidelines that help us to uphold the things we value; A philosophic principle concerned with
opinions about appropriate and inappropriate moral conduct or behavior by an individual or social group;
Process by which we clarify right and wrong and act on what we take to be right; Code of practice imposed upon
practitioners by their professional body and/or employer
Ethics committee
A formally constituted panel which conducts required reviews of the ethical grounds of research proposals prior
to their approval
Ethnocentrism
Judging people from a different culture according to the standards and values of your own culture; The belief of
the members of a group that their culture has the best beliefs, values and norms
Ethnography
Research design in which the researcher studies an intact cultural group in a natural setting during a prolonged
period of time by collecting, primarily, observational data; The research process is flexible and typically evolves
contextually in response to the lived realities encountered in the field setting; A form of qualitative research
focused on describing the culture of a group of people; A type of qualitative research which treats a group of
people as an anthropologist would an unknown tribe using detailed descriptions of how they live and act
Ethnology
The comparative study of cultural groups
Ethogenic study

Assessment Glossary
An ongoing observational approach that focuses on processes rather than products
Etic perspective
An external, social scientific view of reality
Etic term
Outsider’s words or special words that are used by social scientists
Evaluation
A study to determine the extent to which a program or project reached its goals; Judgment regarding the quality,
value, or worth of a response, product, or performance based on established criteria; An assessment at a point
in time of the value, worth or impact of a project or program; Qualitative and quantitative descriptions of pupil
behavior plus value judgments concerning the desirability of that behavior; Using collected information to make
informed decisions about continued instruction, programs, activities; To measure, compare, and judge the
quality of student work, schools, or a specific educational program; Using assessment findings (evidence or
data) to judge program effectiveness; Using assessment findings as a basis for making decisions about program
changes or improvement; Analyzing results and drawing conclusions; Decisions made about assessment
findings; Deciding about the value of programs/program outcomes; Systematic investigation of the worth or
merit of a student's performance in relation to a set of learner expectations or standards of performance;
Systematic process of determining the merit, value, and worth of someone or something; Any effort to use
evidence to improve effectiveness; May involve recommendations for changes; May utilize notable events and
subjective impressions in addition to various measurements; The purposes of evaluation include finding out if
client or community needs were met, improving a project or program, assessing the outcomes or impacts of a
program, finding out how a program is operating, assessing the efficiency or cost-effectiveness of a program,
and/or understanding why a program does or does not work
Evaluation apprehension
Type of confounding where naturally apprehensive or secretive personalities will not behave normally during a
test; Potential bias indicated by comments such as "I better watch what I say in front of you”; Potential bias
created when instructors do not trust administrators or the assessment process and believe or fear that the
results of student learning assessments will be used to evaluate their own performance; May result in the
confounding of standardized tests when instructors “teach to the test”
Evaluator
Anyone who accepts and executes responsibility for planning, conducting, and reporting evaluations; A person
who assembles data and information collected about a subject, analyzes them, makes judgments as to whether
that subject's performance level meets the pre-specified standards, prepares a summary report, writes
recommendations, and who may provide feedback to the subject, directly or through another person
Event sampling
Observing only after specific events have occurred
Evidence
Proof, data, or documentation that fulfills a purpose, supports a conclusion or tests a hypothesis; Quantitative
and qualitative data which an institution as a whole uses to determine the extent to which it attains the
performance goals it establishes for itself
Evidence-based practice (EBP)

Properly informed professional decision making; The conscientious, explicit, and judicious use of current best
evidence in making decisions; A systematic approach to integrating current scientific evidence; Only as good as
the evidence available at the time and the process used to identify and select appropriate evidence
Evidential basis of validity

Assessment Glossary
The assemblage of information about the construct validity of the test scores and measurements, as well as the
relevance of the measurement to its applied purpose and its utility in an applied setting; Construct validity, in this
case, includes information collected from content and criterion-related validation procedures;
Ex post facto research

After the fact or retrospective research; Studies investigating possible cause-and-effect relationships by
observing an existing condition or state of affairs and searching back in time for plausible causal factors; A
method of identifying the possible antecedents of an event that has already happened and which, thus, cannot
be engineered or manipulated by the researcher; Research where the independent variable or variables have
already occurred and in which the researcher starts with the observation of a dependent variable or variables;
One of the recognized subtypes of the experimental method
Excellence
A quality or state of high or superior performance, or of having virtues and values surpassing most others
Excess capacity
The ability to produce additional units of output without increasing fixed capacity
Executive report
An abbreviated report that has been tailored specifically to address the concerns and questions of a person
whose function is to administer a program, department, project, or institution
Executive summary
A brief, non technical summary statement designed to provide a quick overview of the full-length report on which
it is based
Exhaustive
A set of categories that classify all of the relevant cases in the data; Property that the response categories
include all possible responses; Sets covering the complete range of data values
Exhibition
An inclusive demonstration of skills or competencies; May be live performances, interdisciplinary, and require
individual creativity as well as a display of developed skills; May include individual or group/collaborative
projects produced over an extended period of time
Existence versus frequency

A key question in the coding process of content analysis; The researcher must decide if he/she is going to count
a concept only once (for existence) no matter how many times it appears, or if he/she will count it each time it
occurs (for frequency)
Exit interviews or surveys

Information obtained from students on completion of their study or leaving the institution; Includes information
about student growth and change, satisfaction with academic programs, their experiences in their majors,
reasons for leaving (if non completer), and their immediate and future plans
Exit level
Information obtained about students on completion of their study to assess the changes in learning as a result of
their program of study
Exit outcomes or proficiencies

What all students should know and be able to do when they graduate from school
Expectancy data
Data illustrating the number or percentage of people that fall into various categories on a criterion measure

Assessment Glossary
Expectations
Realistic goals or outcomes
Expected growth
The average amount of change in test scores that occurs over a specified time interval for individuals with
certain individual characteristics such as age or grade level
Expected outcome
A specific and desired benefit that occurs to participants of a program following application of an intentional
action; Generally phrased in terms of the changes in knowledge, skills, attitudes, behavior, condition, or status
that are expected to occur in the participants as a result of implementing the action
Expensing
The depreciation of the entire cost of an asset during the first year of purchase
Experience
That which is acquired through exposure to, or participation in, an activity or process
Experience curve
Systematic cost reductions that occur over the life of a product; Product costs typically decline by a specific
amount each time accumulated output is doubled
Experiment
An environment in which the researcher objectively observes phenomena that are made to occur in a strictly
controlled situation in which one or more variables are varied and the others are kept constant; A researcher
creates an environment in which to observe and interpret the results of a research question, participants in a
study are randomly assigned to groups, and in an attempt to create a causal model groups are treated
differently and measurements are conducted to determine if different treatments appear to lead to different
effects; Performance of a planned set of trials; Any process or study which results in the collection of data, the
outcome of which is unknown; In statistics, the term is usually restricted to situations in which the researcher
has control over some of the conditions under which the experiment takes place
Experimental control
Eliminating any differential influence of extraneous variables
Experimental design
The art of planning and executing experiments; The plan of an experiment, including selection of subjects, order
of administration of the experimental treatment, the kind of treatment, the procedures by which it is
administered, and the recording of the data (with special reference to the particular statistical analyses to be
performed); Refers to the way that experiments or studies are structured to provide meaningful data relevant to
the research questions being investigated
Experimental group or sampling unit

The group that receives the experimental treatment condition; A unit is a person, animal, plant or thing which is
actually studied by a researcher; The basic objects upon which the study or experiment is carried out; The
entities to which the treatments are applied; A group of subjects assigned to receive a treatment (the
independent variable), the effects of which (the dependent variable) are measured and compared to those
observed for a control (comparison or non treatment) group; The group of subjects who receive the
experimental treatment or process which is under investigation
Experimental methods

Assessment Glossary
Class of research design intended to approximate to the ideal of the true experiment and characterized by
structured observation of the effects of one or more deliberately manipulated independent variables on a single
dependent variable, while the effects of other (ideally all) possible causation is tightly controlled
Experimental research
Research in which the researcher manipulates the independent variable; Scientific investigation in which an
investigator manipulates and controls one or more independent variables to determine their effects on the
outcome or dependent variable; Research which seeks explanations for cause-effect relationships by
manipulating one or more factors (independent variables) while holding other factors constant
Experimenter or experimenter bias effect

The unintentional effect that the researcher can have on the outcome of a study; Consequence of carelessness
and lack of attention to detail by experimenters which bias the results of their research (e.g. by failing to prevent
confounding or not enforcing experimental controls)
Experimenter expectancy
Type of confounding where the expectations of experimenters subtly influence the behaviors being measured
Expert
One who has demonstrated a high level of proficiency in a knowledge area or skill set; An individual who has
demonstrated expert level performance on an assessment; Also used to describe an individual with sufficient
knowledge in an area to evaluate the content relevance and content representativeness of a measure and/or to
help set standards in an area
Explanation
Attempting to show how and why a phenomenon operates as it does
Explanatory research
Testing hypotheses and theories that explain how and why a phenomenon operates as it does
Exploratory factor analysis

An analysis of latent factor structures conducted without a priori constraints or hypotheses
Exploration
Attempting to generate ideas about phenomena
Extant data
Existing data, information, and observations that have been collected or that are available for use in the
assessment and evaluation processes
External assessment or evaluation

Evaluation conducted by an evaluator from outside the organization within which the object of the study is
housed; Use of criteria or instruments developed by individuals or organizations external to the one being
assessed; Use of evaluators not employed by the assessed institution; Usually summative, quantitative, and
often high stakes; Sometimes used to refer to an assessment of external factors (i.e. an environmental scan)
External audit
An assessment conducted by an independent external auditor as a means of providing a professional opinion
on the accuracy and fairness of a company’s financial statements based on generally accepted auditing
procedures; A systematic assessment of a organization’s management, finances, operations, controls, and
scope in which policies and procedures are carried out, performed by professionals who are independent of the
entity being audited; The periodic external scrutiny of corporate governance and management systems, financial
statements, and underlying financial systems, and the performance, performance management, and reporting of

Assessment Glossary
public bodies; Periodic or specific purpose (ad hoc) audit conducted by external (independent) qualified
professionals
External auditor
An audit professional who performs an audit on the financial statements of a company, government, individual,
or any other legal entity or organization, and who is independent of the entity being audited
External benchmarking
The process of comparing an organization’s current policies and practices to those of a competitor
organization(s) to determine current and future trends in areas of employment and business practice
External criticism
Determining the validity, trustworthiness, or authenticity of the source
External examiner
Using an expert in the field from outside your program, usually from a similar program at another institution to
conduct, evaluate, or supplement assessment of your students
External reliability
The degree of consistency of a measure over time; See also Internal reliability
External validity
The extent to which the study results can be generalized to and across populations of persons, settings, times,
outcomes, and treatment variations; The extent to which the results of a study can be generalized or transferred
to another setting; The extent to which assessment findings and conclusions from a study conducted on a
sample population can be applied to the population at large; The ability to apply the findings in one context to
another similar context
Externalities
A cost or benefit imposed on one party by the actions of another party; Costs are negative externalities and
benefits are positive externalities
Extraneous variable
A variable that may compete with the independent variable in explaining the outcome; any variable other than
the independent variable that might influence the dependent variable
Extrapolate or extrapolation
To infer an unknown from something that is known; To estimate the value of a variable outside its observed
range; To predict future results from prior observed results
Extreme case sampling

Identifying the “extremes” or poles of some characteristic and then selecting cases representing these extremes
for examination

Assessment Glossary
F
Face validity
The perceived extent of acceptability or legitimacy of an instrument or process to teachers, administrators,
policymakers, students, parents, the general public, and other stakeholders concerned with educational
evaluation and the quality of learning/teaching; Extent to which an instrument superficially appears to an
examinee as measuring the construct being investigated; The degree to which a measure appears to the
subject being measured as measuring what it claims to measure; Face validity may motivate subjects to perform
well or it may promote deliberate misrepresentation by a respondent; Apparent or presumed validity which has
not been tested or validated; An evaluation of a test based on inspection only; Making a decision regarding the
appropriateness of a test or other assessment instrument based on appearance rather than objective criteria
Facesheet codes
Codes that apply to a complete document or case
Facilitation
Coordinating rather than leading an exercise so that all group members are encouraged to participate in the
discussion or activity; Helping a group of people come to conclusions
Facilitator
A person who makes it easier for other people to accomplish objectives by offering advice and assistance in
solving problems, either with technical issues or with other people
Factor
Input variable; Any variable, real or hypothetical, that is an aspect of a concept or construct; In measurement
theory, a statistical dimension defined by factor analysis; In mental measurement, a hypothetical trait, ability, or
component of ability that underlies and influences performance on two or more tests and hence causes scores
on tests to be correlated; In analysis of variance, an independent variable; In factor analysis, a cluster of related
variables that are distinguishable components of a larger set of variables; A number by which another number is
multiplied, as in “gas prices tripled or increased by a factor or 3 within five years”
Factor analysis
A statistical procedure that analyzes correlations among test items and tells you the number of factors present,
and whether the test is unidimensional or multidimensional; A statistical test that explores which variables in a
data set are most related to each other; A survey factor analysis can yield information on patterns of responses,
not simply data on a single response; Allows larger tendencies to be interpreted, indicating behavior trends
rather than simply responses to specific questions; Any of several statistical methods of describing the
interrelationships of a set of variables by statistically deriving new variables, called factors, that are fewer in
number than the original set of variables; Factor analysis reveals how much of the variation in each of the
original measures arises from, or is associated with, each of the hypothetical factors; Any of several methods
used for reducing correlated data to a smaller number of factors where a small number of factors considered to
be the basic variables accounting for interrelations in the data are extracted from a correlation matrix; Method
requiring the accumulation of scores on a number of simultaneous variables for each subject and then
performing multiple correlations; See also Principal components analysis
Factorial design
A design in which two or more independent variables, at least one of which is manipulated, are simultaneously
studied to determine their independent and interactive effects on the dependent variable; Participants must be
randomly assigned to the levels of at least one of the independent variables
Factorial design based on a mixed model

Assessment Glossary
A factorial design in which different participants are randomly assigned to the different levels of one independent
variable but all participants take all levels of another independent variable
Fair or fairness
Impartiality; Such aspects of the assessment program and evaluation system as equal opportunity to acquire the
knowledge and skills to be assessed, use of developmentally appropriate assessments, sound procedures,
appropriate use of evaluation results, and reasonable demands on the subjects being evaluated in terms of such
factors as time, costs, and personal resources required; Assessment or test that provides an even playing field
for all students; Teachers, students, parents and administrators agree that the instrument has validity, reliability,
and authenticity, and they therefore have confidence in the instrument and its results
False negative
When assessment results inaccurately indicate a person, program, or organization should not be placed into a
category; An incorrect diagnostic judgment that an entity does not fall within a target category
False positive
When assessment results inaccurately indicate a person, program, or organization should be placed into a
category; An incorrect diagnostic judgment that an entity does fall within a target category
Fat organization
An organization with a structure consisting of several layers of management
Fatigue effect
A type of confounding possible with prolonged or physically demanding research procedures resulting in
reduced or poorer performance on later items in a procedure or instrument
FERPA
Family Educational Rights and Privacy Act of 1974
Feasibility
The extent to which an evaluation is appropriate and practical for implementation; Capability of being
accomplished or brought about
Feasibility study
A study designed to discover if a business, product, project, or process justify the investment of time, money,
and other resources
Federal mandates
Any provision in a bill or joint resolution before Congress or in a proposed or final Federal regulation that would
impose a duty that is enforceable by administrative, civil, or criminal penalty or by injunction (other than a
condition of the bill or joint resolution or implementing regulation)
Feedback
Information that provides the performer with direct, usable insights into current performance, based on tangible
differences between current performance and hoped-for performance; The information and recommendations
provided to a learner/teacher about his/her performance based on the results of that learner/teacher's evaluation
and designed to help the learner/teacher improve his/her performance and make decisions concerning
professional development and improvement
Feedback forms or feedback questionnaires

Forms that gather information about participants and their assessments of an activity, course or program
Field notes

Assessment Glossary
Notes taken by an observer; Notes taken by researchers to record unstructured observations they make in the
field and their interpretation of those observations
Field test
Studies of a program, project, or instructional material in settings like those where it is to be used
Financial aid
Funds awarded to a student on the basis of his/her demonstrated financial need for the purpose of meeting
postsecondary education expenses
Financial assistance
Any support received by a student from a source other than parents, spouse, or his/her own resources, to help
meet the student's postsecondary education expenses
Financial audit
A review of a company’s financial position serving as confirmation that a company’s financial statements are
accurate
Financial budget
A budget that contains a company’s balance sheet detailing how a particular operations’ plan will affect
company resources and liabilities
Financial need
The difference between the cost of a student's postsecondary education and the expected family contribution
from the student and his/her family to be applied toward the student's postsecondary education expenses
Financial statement
A report containing financial information derived from an organizational accounting record
First impression effect

A rater effect in which an assessor or an evaluator tends to base judgments and evaluations on early opinions
rather than on a complete picture and tends to distort subsequent information to fit the initial opinion
First-in, first-out (FIFO)

An inventory valuation technique that presumes inventory purchased first is used or sold first
First mover advantage

The competitive advantage held by a firm from being first in a market or first to use a particular strategy
Fiscal
Related to finance or finances
Fixed assets
An accounting term used to describe tangible property used in the day-to-day operation of a business; Typically
includes items such as real estate, equipment, machinery, fixtures, and furnishings
Fixed budget
A type of budget that is developed based solely on fixed costs and does not alter with fluctuations in activity
Fixed costs
The day-to-day costs of doing business that don’t fluctuate based on the number of goods produced or services
performed

Assessment Glossary
Flexible cognition
A learning framework that encourages higher-order mental skills; The ability to spontaneously restructure one’s
knowledge, in many ways, by adapting to the demands of radically changing situations
Flat organization
An organization characterized by having only a few layers of management from top to bottom
Floor
The lowest limit of performance that can be assessed or measured by an instrument or process; The lowest limit
of performance that can be measured effectively by a test; Individuals have reached the floor of a test when they
perform at the bottom of the range in which the test can make reliable discriminations; Individuals who perform
near to or below this lower limit are said to have reached the floor, and the assessment may not be providing a
valid estimate of their performance levels; Such individuals should be given a less difficult assessment of the
same attribute if it is necessary to differentiate between lower levels of performance
Focus group analysis or discussion

A qualitative research process designed to elicit opinions, attitudes, beliefs, and perceptions from individuals to
gain insights and information about a specific topic; A form of structured group discussion involving people with
knowledge and interest in a particular topic and a facilitator; A carefully planned discussion to obtain perceptions
on a defined area of interest in a permissive, non threatening environment; A group selected for its relevance to
an evaluation that is engaged by a trained facilitator in a series of discussions designed for sharing insights,
ideas, and observations on a topic of concern; A common type of group discussion, in which a moderator
encourages a small group of people (usually 8 to 10) to gradually focus on a topic; Typically a group of 7-12
individuals who share certain characteristics relative to a particular topic related to a research or evaluation
question in which discussions are conducted by a trained moderator to identify trends/patterns in perceptions;
Moderators provide direction and set the tone for the group discussion, encourage active participation from all
group members, and manage time; Moderators must not allow their own biases to be expressed verbally or
nonverbally; Careful and systematic analysis of the discussions provides information that can be used to
evaluate and/or improve the desired outcome; A small group of people, led by a facilitator, discuss their ideas
about a particular issue
Follow up
Actions taken to maintain the strengths and address the weaknesses that were identified in the evaluation
Forced choice response or assessment

A format for a test, assessment, rating, or survey item where the respondent is given a limited number of options
from which to select an answer; In cases where there is a correct or best answer, the other options are referred
to as distractors; Testing where responses to an item, questions or prompts are placed against a set answer
key; Scoring does not require judgment on the part of the scorer because there is one right answer to the item;
In test construction, forced choice or multiple-choice items require an examinee to choose an answer from a
small set of response options; Examples of forced-choice items are multiple-choice questions, true/false items,
checklists, and a five-point rating scale
Forecasting
A business analysis conducted in order to assess what future trends are likely to happen, especially in
connection with a particular situation, function, practice or process that is likely to affect the organization’s
business operations
Formal
The conducting of an assessment or evaluation activity in accordance with a prescribed plan, structure, or
advance notice
Formal award

Assessment Glossary
A diploma, certificate, or degree awarded in recognition of the successful completion of a program of studies;
Also included under this heading are certificates awarded for attendance or for the completion of a course
Format
The structure of assessment instruments, evaluation forms, and materials; Includes shape, size, mode of
delivery (e.g. paper versus computer, in-person versus telephone, audio tape versus video tape), and general
design or layout of the materials and equipment
Formative assessment or evaluation

Improvement-oriented assessment; Evaluation conducted while a creative process is under way, designed and
used to promote growth and improvement in a student's performance or in a program's development; Evaluation
designed and used to improve an object, especially when it is still being developed; Process of judging an
ongoing, changing process or product for diagnosis, revision, description, information, or comparison;
Observations which allow one to determine the degree to which students know or are able to do a given learning
task, which identifies the part of the task that the student does not know or is unable to do, and which suggests
future steps for teaching and learning; Ongoing, diagnostic assessment providing information, or feedback, to
guide instruction and improve student performance; The gathering of information about student learning during
the progression of a course or program; Improvement-oriented assessment that usually takes place continually
throughout a lesson module, course, or program; The use of a broad range of instruments and procedures
during a course of instruction or during a period of organizational operations in order to facilitate mid-course
adjustments; Usually conducted repeatedly to tract improvement or change in the learning of students over
time; May be short term focused to assess whether some or all students in a group need additional assistance
to achieve a course outcome; May be long term focused to assure students progress through their courses
toward achieving program outcomes; Formative research (often called formative evaluation) is conducted during
the planning and delivery of a program and it produces information that is used to improve a program while it is
in progress; Formative research can be contrasted to summative research, which assesses a program upon its
completion
Fraudulent activity
Fabrication or alteration of results
Frequency analysis
Frequency analysis measures the number of times a particular distracter, or combination of distracters, was
selected by a group of participants
Frequency distribution
Arrangement in which the frequencies of each unique data value are shown; A statistical description of raw data
in terms of the number of items characterized by each of a series of values of a continuous variable
Frontload
Amount of effort required in the early stage of assessment method development or data collection
Frontstage behavior
What people want or allow us to see
Full costs
A cost management and cost-benefit analysis of the sum of direct and indirect costs; Costs and advantages
may be considered in terms of environmental, economical, and social impacts
Fully anchored rating scale

All points are anchored on the rating scale
Functional-level testing

Assessment Glossary
The administration of an instrument or assessment process whose difficulty level is appropriate for the
individuals being tested, but not necessarily for the age or grade level or group membership of that individual;
Functional-level testing refers to the individual being tested, not to the group for whom the assessment was
designed nor the group to which the individual belongs
Fundamental principle of mixed research

Advises researchers to mix research methods or procedures in a way that the resulting mixture or combination
has complementary strengths and non overlapping weaknesses; The researcher should use a mixture or
combination of methods that has complementary strengths and non overlapping weaknesses

Assessment Glossary
G
Gain scores
The difference between a student's performance on a test and his or her performance on a previous
administration of the same or parallel test
Gantt chart
A common way of showing tasks over time used for project planning and keeping track of the status of individual
tasks within a project; A horizontal bar chart developed by Henry L. Gantt as a production control tool; Used in
project management to provide a graphical illustration of a schedule that helps to plan, coordinate and track
specific tasks in a project; A graphical time-line which represents the durations of multiple tasks or activities
used in planning and coordination; Gantt charts are constructed with a horizontal axis representing the total time
span of the project broken down into increments (days, weeks, or months), a vertical axis representing the tasks
that make up the project, and horizontal bars for each task connecting the starting and ending times; Gantt
charts often include milestones (important checkpoints or interim goals), delegated responsibilities (who does
what when), current completion status (with charts updated by filling in task bars to a length proportional to the
amount of work that has been finished), and dependencies (activities which are dependent on the prior
completion of other activities) or other relationships between tasks
Gap analysis
A means of measuring and evaluating differences between an organization’s current position and its desired
future position; The identification of the difference between the desired and current state
Garbage can theory of organizational choice

A theory of organizational decision making applicable to organizations where goals are unclear, technologies
are imperfectly understood, histories are difficult to interpret, and participants wander in and out; Such
"organized anarchies" operate under conditions of pervasive ambiguity, with so much uncertainty in the decision
making process that traditional theories about coping with uncertainty do not apply
General education
The common knowledge, skills, and developmental characteristics of the college educated person
General linear model (GLM)

The mathematical model underlying most comparative statistics, involving proportional relationships among
variables of interest; A mathematical procedure that is the “parent” of many statistical techniques
General obligation bond

A bond that guarantees that all taxpayers will be responsible for the bond’s principal and interest payments
Generalizable or generalizability
Results of an assessment are generalizable when the score on one assessment can accurately predict a
student score on a different assessment covering the same knowledge or skill; Generalizability across time is
promoted by ensuring that assessments focus on general level concepts or strategies, not on facts, topics, or
skills, which are found only at one level or in one class; The extent to which research findings and conclusions
from a study conducted on a sample population can be applied to the population at large; The extent to which
the performances sampled by a set of assessment activities are representative of the broader domain being
assessed; The extent to which information about a program, project, or instructional material collected in one
setting can be used to reach a valid judgment about how it will perform in other settings; The appropriateness of
using results from one context or purpose in another context or for another purpose
Generalizability coefficient

Assessment Glossary
A reliability index encompassing one or more independent sources of error; Formed as the ratio of the sum of
variances that are considered components of test score variance in the setting under study to the foregoing
sum, plus the weighted sum of variances attributable to various error sources in this setting; Indices which arise
from the application of generalizability theory are typically interpreted in the same manner as reliability
coefficients
Generalizability study (G-Study)

A type of investigation that applies generalizability measurement theory; Provides researchers with an indication
of the extent to which a sample of measurements generalizes to a universe of measurements
Generalizability theory (G-Theory)

An expansion of classical test theory (CTT) that uses variance components to create coefficients, parallel to
reliability coefficients in CTT that informs researchers about the generalizability of measures
Generalize
Making statements about a population based on sample data
Generative skills
Skills or knowledge that are durable over time in face of changes in technology, work process, and societal
demands: They cross functions and serve as the foundation for effective adaptation to changes in role
requirements
GIGO
Short for "garbage in, garbage out", indicating that your results are only as good as your input
Goal articulation
A process of defining and clearly expressing goals generally held by those in an organization or group; Usually
regarded as a function of organization or group leaders and a key step in developing support for official goals
Goal congruence
Agreement on fundamental goals in the context of an organization; Refers to agreement among leaders and
followers in the organization on central objectives; In practice, its absence in many instances creates internal
tension and difficulties in goal definition
Goal free evaluation

Evaluation of outcomes in which the evaluator functions without knowledge of the purposes or goals; Focuses
on actual outcomes rather than intended program outcomes
Goals
Desired end results; A statement of intent or an end that a person or a group strives to attain; Statements of
expectations of general capabilities or student outcomes resulting from planned educational experiences;
Intended results stated in general terms; A written statement describing the direction that indicates success or
improvement in organizational performance; Broader and less specific statements than objectives or outcomes;
Derived from or aligned with the mission statement of the institution and department or unit; In terms of student
behavior, delineates a less focused outcome than an objective; Instructional goals: General or global
statements about knowledge, skills, attitudes, and values expected of students completing a course, discipline,
or program; Non-instructional goals: General or global statements about level of productivity, quality or
effectiveness of services provided by non-instructional units; Goals for Learning: Expressions of intended
results in general terms used to describe broad learning concepts or competencies (such as clear
communication, problem solving, information literacy, and ethical awareness)
Going native
Identifying so completely with the group being studied that you can no longer remain objective

Assessment Glossary
Governance
A separate management system or leadership process used to lead, direct, command, and control an
organization
Grade or mark
A rating or evaluation of a student's achievement in a course
Grade or grade level equivalent

A score that describes student performance in terms of the statistical performance of an average student at a
given grade level; The estimated grade level that corresponds to a given score; School grade level for a given
population for which a given score is the median score in that population; A grade equivalent score does not
equate to performance in the classroom
Grade or grade level norms

The average test score obtained by students classified at a given grade placement; Interpreting scores on a test
in reference to the average performance of students at each grade level
Grade-point average (GPA)

A measure of a student's average performance in all courses taken during a reporting period, term, or academic
year—or accumulated for several terms or years—except for those courses explicitly excluded by the institution
from grade-point average calculations; This numerical average is found by (1) multiplying the numerical grade or
a number assigned to a non numerical grade (for each course to be included in average), by the number of
award units (for each course in the average), (2) summing these results, and (3) dividing this sum by the total
award units for all courses included; Grade points for a course are obtained by multiplying the number of award
units given for successful course completion by the numerical grade (or equivalents for non numerical grades)
awarded for student performance (e.g. A =4, B = 3, C = 2, D = 1, and F = 0)
Grading
Values placed on varying levels of achievement in course assignments and/or examinations; Graded course
assignments and tests can be used for assessment if they are clearly linked to course or program goals and if
the assessment is based on established criteria and standards
Grading system
The process by which a teacher arrives at the symbol, number, or narrative presentation that is used to
represent a student's achievement in a content or learning area
Graduation rate
Extent to which students at an institution of postsecondary education complete an academic program, or enroll
in a subsequent educational program for which the prior program provided substantial preparation, within 150
percent of the normal time for completion of the program
Grant
A giving of funds for a specific purpose
Graphic organizers
Mental maps that help students make their thinking visible; Representations of the process skills of sequencing,
comparing, contrasting, classifying, inferring, drawing conclusions, problem solving, and thinking critically
Grounded theory
A qualitative approach to generating and developing a theory from the data that the researcher collects; A
general methodology for developing theory that is grounded in data systematically gathered and analyzed;
Theory derived from continuously testing an assumption so that it evolves over time in response to collected
information until it is no longer an assumption; Practice of developing other theories that emerge from observing
a group in which the theories are grounded in the group's observable experiences, but researchers add their

Assessment Glossary
own insight into why those experiences exist; Researcher attempts to derive a theory by using multiple stages of
data collection and the refinement and interrelationship of categories of information; Two primary characteristics
of this design are the constant comparison data with emerging categories, and theoretical sampling of different
groups to maximize the similarities and the differences of information; An approach to qualitative research
where the researchers try to approach a problem with no preconceptions and to build their theories solely from
the data gathered
Group affiliation
Membership association; Civic development or citizenship
Group discussion
A generic type of qualitative research in which a small group of people provide information by discussing a topic
Group moderator
The person leading the focus group discussion
Group think
A tendency of individuals to adopt the perspective of the group as a whole which tend to occurs when decision
makers don’t question underlying assumptions
Grouped frequency distribution

Distribution where the data values are clustered or grouped into separate intervals and the frequencies of each
interval are given
Growth over time

Increases in knowledge or development that occur over time, typically measured through pretest/posttest
experimental designs
Guessing parameter
The probability that a student with very low ability on the trait being measured will answer the item correctly
Guideline
A procedural suggestion intended to help evaluators and their audiences to meet the requirements of the
evaluation standards; Strategy to avoid mistakes in applying the standards
Guiding principles
Underlying tenets or assumptions that describe effective learning, teaching and assessment within each subject
area
Guttman scale
A type of response scale where items assessing a unidimensional construct are ordered by difficulty (using
classical test theory); When an examinee fails to endorse an item it is expected that he/she will not endorse any
harder item, and when an examinee endorses an item it is expected that he/she will endorse all easier items;
This type of scale was a predecessor to item response theory

Assessment Glossary
H
Habits of mind
A fluid and life-long approach to learning that involves reflection, inquiry, and action; An approach that favors
uncovering concepts rather than covering content and encourages the learner to think about how they acquire
knowledge
Halo effect
A type of rater effect in which an assessor or an evaluator tends to base overall judgments or evaluations on
selected pieces of information rather than on all available relevant information
Hawthorne effect
Phenomenon whereby behavior can be changed by the mere act of observing it; Changes in outcomes are the
result of the experimental group receiving a treatment but are not a result of the specific treatment received;
Similar to the placebo effect in medical research; A form of reactivity whereby subjects improve an aspect of
their behavior being experimentally measured simply in response to the fact that they are being studied rather
than in response to any particular experimental manipulation; See Attention bias
Hidden curriculum
Beliefs, values and understandings that are passed on to the pupil or student, not explicitly, but unconsciously
and implicitly through the organization of the institution
Hierarchy
A characteristic of formal bureaucratic organizations; A clear vertical "chain of command" in which each unit is
subordinate to the one above it and superior to the one below it; One of the most common features of
governmental and other bureaucratic organizations
High inference
The types of judgments, decisions, and conclusions that are based on complex inductive reasoning and that
require a high degree of subjectivity on the part of the individual
High performance organization

Systems promoting a culture of learning and improvement
High stakes assessment or evaluation

Evaluations that lead to decisions that, if incorrect, harm students and are detrimental to their future progress
and development; Any testing program whose results have important consequences for students, teachers,
schools, and/or districts, such as promotion, certification, graduation, or denial/approval of services and
opportunity; When assessment results are tied to student graduation or to funding for schools from states or
other agencies; Using the results of an assessment to set a hurdle that must to be cleared for completing a
program of study, receiving certification, or moving to the next level; Usually such assessments are externally
developed, based on set standards, carried out in a secure testing situation, and administered at a single point
in time; Examples include mandated exit exams required for graduation and bar exams required for practicing
law in a specific jurisdiction; Can corrupt the evaluation process when pressure to produce rising test scores
results in "teaching to the test" or making tests less complex
High stakes test

A test whose results have important, direct consequences for the examinees, programs, or institutions tested
Higher order cognitive development

Assessment Glossary
Defined as the development of understanding one's own and others' knowing and serves as the supporting
structure for sustained intellectual inquiry and analysis
Higher order thinking

A complex level of thinking that entails analyzing and classifying or organizing perceived qualities or
relationships, meaningfully combining concepts and principles verbally or in the production of art works or
performances, and then synthesizing ideas into supportable, encompassing thoughts or generalizations that
hold true for many situations
Holistic rubric
A holistic rubric assigns a level of performance by assessing performance across multiple criteria as a whole
instead of listing separate levels of performance for each criterion
Holistic perspective
Taking almost every action or communication of the whole phenomenon of a certain community or culture into
account in research
Holistic scoring or holistic method

Method of scoring essays, products, and performances in which a single score is given to represent the overall
quality of the essay, product, or performance without reference to particular dimensions; Assigning a single
score based on an overall assessment of performance rather than by scoring or analyzing dimensions
individually; Used when the product is considered to be more than the sum of its parts and so the quality of a
final product or performance is evaluated rather than the process or dimension of performance; Scoring
procedure yielding a single score based upon an overall impression of a product or performance; Evaluation
score is based on an overall impression of student performance rather than multiple dimensions of performance;
Type of grading in which an assignment is given an overall score with possible scores described in a rating
scale; In a holistic rating scale a high score indicates achievement of all aspects of the assignment, while a low
score means few if any of the desired outcomes have been achieved; Holistic score levels need to be specific
enough to reveal meaningful, diagnostic information when the scores are aggregated
Horizontal organization
A flat organizational structure that consists of fewer hierarchal levels and which often relies on the use of cross-
functional teams
Human capital
The collective knowledge, skills, and abilities of an organization’s employees
Human relations theories of organization

Theories stressing workers' noneconomic needs and motivations on the job which seek to identify those needs
and how to satisfy them; Organizational theories which focus on working conditions and social interactions
among workers
Human resources planning

The process of anticipating future staffing needs and ensuring that a sufficient pool of talent possessing the
skills and experience needed will be available to meet those needs
Hybrid organization
An organization whose structure is comprised of both vertical and horizontal models
Hypothesis
A tentative explanation based on theory to predict a causal relationship between variables; Statement of the
casual relationship between two (or more) variables (e.g. “if X, then Y” or “if variable A is X, then variable B will
be Y”)

Assessment Glossary
Hypothesis testing
The act of applying objective and peer-replicable testing to theoretical beliefs; Normally supported by using
inferential statistics in attempts to disprove the null hypothesis and provide an estimate of the confidence level in
the form of a p-value
Hypothetical construct
A presumed internal quality of a system, beyond direct observation, whose presumed operation accords with
available empirical data; A construct is some postulated attribute of people, assumed to be reflected in test
performance; Constructs are part of a theory and may, in turn, map onto one or more variables, each of which
may be operationalized as observable measures in a number of different ways; See Construct

Assessment Glossary
I
I-E-O model
Astin's Input-Environment-Outcome model of student development; This model describes the development of
students as being an interaction of the college environment and the characteristics students bring with them to
the environment
Identity statement
A concise statement of salient and distinguishing characteristics of an institution (who we are and how we are
different)
Impact data
Summary data that describe the likely results of a decision (e.g. number of students likely to fail to meet a given
cut score)
Impact evaluation
Evaluations that look specifically at whether or not the program achieved its goals and had its intended effect;
An evaluation measures the final results of a program or initiative; See also Outcome evaluation
Impacts
Changes that occur in the family, community, and larger society as a consequence of participation in adult
literacy education
Implementation
Putting something into place by doing what was planned
Improvement
Making something better by moving closer to a goal
In-house instruments or software

Non-proprietary instruments/software which are tools developed by institutions for internal use, not researched,
or purchased from an outside source; In-house assessment tools are sometimes preferred because they are
designed to exactly match an institutional purpose
In-person interview
An interview conducted face-to-face
Inclusive approach
Planning or decision-making which involves people and groups from diverse backgrounds or with differing
interests that enables and encourages all participants in a project, process, or organization to be listened to and
heard; Inclusive methods respect and recognize the differences between people, taking different views into
account; Involves creating a relaxed and enjoyable atmosphere that allows ample time for participation, avoids
the use of jargon or discriminatory language, and enables the development of trust and understanding between
people or groups
Income statement
A financial document showing a company’s income and expenses over a given period of time
Incompatibility thesis
The proposition that one cannot mix quantitative and qualitative research

Assessment Glossary
Inconsistency
A type of rater effect in which an assessor or an evaluator tends to rate or to interpret similar data and
information in different ways
Incrementalism
A model of decision making that stresses making decisions through limited successive comparisons, in contrast
to the rational model; Focuses on simplifying choices rather than aspiring to complete problem analyses, on
"satisfying" rather than "maximizing"
Incubator culture
A corporate culture high on personal and egalitarian dimensions
Independent variable
A variable that is presumed to cause a change in another variable; A variable that is part of the situation that
exist and from which originates the stimulus given to a dependent variable; A variable that is considered to be a
cause or a variable that is used for prediction; The variable or factor that is assumed to cause or influence the
dependent variable(s) or outcome; The independent variable is manipulated in experimental research to
observe its effect on the dependent variable(s)
Indicator or indicators
Statistics that reflect something about the performance or status of some aspects of an educational program; A
specific description of an outcome in terms of observable and assessable behaviors; Specifies what a person
who possesses the qualities articulated in an outcome knows or can do; Measures for individuals or
organizations that provide information about measurable traits, situations, knowledge, skills, performances,
resources, inputs, or outputs; Specific examples and explicit definitions that can be used in rating students' level
of achievement relative to specified skills, strategies, and knowledge; Several indicators are generally needed to
adequately describe each outcome
Indirect assessment
Assessments that deduce student achievement of learning outcomes through the students’ reported perception
of their own learning; May also be the opinions or thoughts of others about student knowledge, skills, attitudes,
learning experiences, and perceptions
Indirect costs
Expenses incurred for the purpose of common general activities and cannot be identified or charged directly to a
specific project, product, or service
Indirect effect
An effect occurring through an intervening variable
Indirect labor
Labor that is necessary to support product production or service delivery but is not directly involved with the
actual production or delivery process
Indirect evidence, methods, or measures

Unobtrusive measures that occur naturally in a research context; Data collected by the researcher without
introducing any formal measurement procedure; Evidence that shows student learning indirectly; Measures from
which student learning can be inferred rather than directly demonstrated; Acquiring evidence about how
students feel about learning and their learning environment rather than actual demonstrations of outcome
achievement; Methods that involve perceptions of learning rather than actual demonstrations of outcome
achievement; Students or others report their perception of how well a given learning outcome has been
achieved; Quantitative examples include course grades, transfer rates and retention data; Qualitative examples
include surveys of students, alumni, or employers, exit interviews, focus groups, and reflective essays;

Assessment Glossary
Processes employed to assess a program and/or student learning indirectly by gathering indirect information
related to student learning or program success
Indirect measures of learning

See indirect evidence
Individualism
The cultural value dimension that emphasizes self-interest, independence, or self-reliance over group, social, or
collective values, processes, or goals
Inductive analysis
A form of analysis based on inductive reasoning; A researcher using inductive analysis starts with answers, but
forms questions throughout the research process
Inductive codes
Codes that are generated by a researcher by directly examining the data
Inductive method
A bottom-up or generative approach to research
Inductive reasoning
Reasoning from the particular to the general; A form of reasoning in which a generalized conclusion is
formulated from particular instances; Drawing a hypothesis/assumption and collecting data to see if it is correct
or not; Reasoning from the specific to the general
Inclusivity
Broadness in orientation or scope; Frequently used in terms of broadness of culture and ethnicity
Industrial psychology
Applied psychology concerned with the study of human behavior in the workplace and how to efficiently manage
an industrial labor force and problems encountered by employees
Inference
A logical conclusion or judgment that is explicitly supported by data, evidence, and information gathered as part
of the evaluation, assessment, or research process
Inferential statistics
The division of statistics focused on going beyond the immediate data and inferring the characteristics of a
population based on samples; Research statistics; A measure of the confidence we can have in our descriptive
statistics; The statistics we use to test hypotheses about a population; Inferential statistics is always guessing
from incomplete information and the trick is being able to make definite statements regarding the accuracy of
the guess; Class of mathematical procedures designed to establish the likelihood of a causal relationship
existing between blocks of empirical observations
Influence
Attempting to apply research to make certain outcomes occur
Informal
The conducting of an assessment or evaluation activity without a prescribed plan or structure, or with little or no
advance notice
Informal test

Assessment Glossary
A non standardized test that is designed to give an approximate index of an individual's level of ability or
learning style; often teacher-constructed
Informal conversational interview

Spontaneous, loosely structured interview
Information
Knowledge about the attributes and performance of a learner/teacher based on assessments, documentation,
and data sources used in the assessment and evaluation processes
Information competency, literacy or skills

Combined capability to access, evaluate, and use information in fulfillment of coursework and independent
study; The ability to acquire, evaluate, organize, maintain, interpret, and communicate knowledge
Information needs
Information requirements of the evaluator, clients, and other pertinent audiences to be met by the evaluation
Information sources
The persons, groups, instruments, recordings, and documents from which data are obtained
Informed consent
Agreeing to participate in a study after being informed of its purpose, procedures, risks, benefits, alternative
procedures, and limits of confidentiality; Agreement by a subject, prior to the collection of information and/or its
release in evaluation reports, that their names and/or other confidential information supplied by them may be
used in specified ways, for stated purposes, and in light of possible consequences; The agreement between
concerned parties about the data-gathering process and/or the disclosure, reporting, and/or use of data,
information, and/or results from an assessment or evaluation; Agreement of a subject to participate in an
experiment after having been informed of all the potential negative consequences of the experimental treatment;
An individual’s agreement to allow something to transpire subsequent to the individual having been informed of
associated risks and alternatives; The process of obtaining voluntary participation of individuals in research
based on a full understanding of the possible benefits and risks of the research; Informed consent is usually
demonstrated by signing a consent form
Informed decision making

Using data and information, not assumptions, to make decisions
Infrastructure
The fundamental physical structures of a city, county, state, country, or other geopolitical entity (includes roads, utilities,
water, sewage, etc.)
Input variables
The activities, materials, and teacher behaviors designed to improve student learning and behavior
Inputs
The “nouns” of projects; The resources that are used to make the project happen (e.g. people and equipment);
The personal, background, and educational characteristics that students bring with them to postsecondary
education that can influence educational outcomes
Institutional assessment
Assessment to determine the extent to which a college or university is achieving its mission
Institutional effectiveness (IE)

The measure of what an institution actually achieves; How well the institution meets its goals and mission; How
well an institution succeeds in accomplishing its mission as determined by the relative accomplishment of

Assessment Glossary
explicit objectives within a given time frame: An analysis of institutional effectiveness data measures (i.e.,
effectiveness, efficiency, and excellence) via outcomes assessment identifies opportunities for improvement;
Extent to which an institution meets its unique mission; Usually includes the functions associated with
assessment and institutional research; A measurement of institutional achievement done for the purpose of
continual improvement; A continuous set of the processes which include planning, assessment, and review
aimed at ongoing improvement; The ongoing quest for quality and the demonstration of how well the institution
is fulfilling its mission and realizing its vision
Institutional effectiveness plan

The institution’s documented assessment plan that includes reporting plans of annual goals and objectives and
follow-up assessment reports of designated departments; The written plan encompassing the college’s policies
and procedures for assessment, and the reporting of assessment results
Institutional outcome
Benefits or changes in a population, organization, or community due to program activities and typically
expressed in aggregate form; These benefits may include research, economic, civic, service, and community
outcomes, as well as student learning outcomes
Institutional performance
The degree to which institutions and organizations of higher education have achieved their goals and objectives
as evidenced by student outcomes and efficiency indicators
Institutional portfolios
A means of assessing the impact of the entire educational experience on student learning; May be used to drive
internal improvement and external accountability
Institutional Priorities Survey (I.P.)

A proprietary survey by Noel-Levitz for Faculty and staff that measures the satisfaction and opinions of a wide
range of support services at a college; Frequently used as a companion survey to the Student Satisfaction
Inventory (SSI) to compare staff and student perceptions about an institution’s culture and environment
Institutional research
Applied study of a college's functioning in terms including measures of resources, inputs, and outputs and
occasionally outcome assessment
Instruction
The methods and processes used by teachers to change what students know, what they can do, the attitudes
they hold, or their behavior; Systematic provision of information, opportunities, and resources to promote the
development of a repertoire of knowledge and skills
Instructional assessment
Determining the extent to which students have met learning outcomes (at the course, department, or program
level)
Instructional goal
A statement of what students are expected to learn in a given lesson, unit, course, program, or across
educational and training programs
Instructional objectives
More detailed expressions of educational objectives
Instructional unit
Any department directly related to students learning in the classroom

Assessment Glossary
Instructional validity
The degree to which the items on a test measure what is actually being taught, and/or what the individuals being
assessed have had an opportunity to learn
Instrument
An assessment device adopted, adapted, or constructed for the purposes of the evaluation; Research tool used
to assess variables during an evaluation; Device used to collect data, information, and evidence; For example:
Tests, questionnaires, application forms, interview schedules and protocals, checklists, rating scales, focus
group protocols, and observation records
Instrument or assessment tool

A method of gathering data about student performance; For example: A questionnaire, a test, a checklist of
stages in solving a problem, or a criterion-referenced rating scale for an art product or performance
Instrumental case study

Case study interested in understanding something more general than the particular case
Instrumentation
Any change that occurs in the way the dependent variable is measured
Intangible assets or capital

Assets or items of value which are not physical in nature (e.g. patents, trademarks, and brand recognition)
Intellectual capital
Broad term combining the idea of the intellect with the economic concept of capital referring to the collective
knowledge of an organization applied to some money-making or other useful purpose
Intellectual property
Property protected under federal law which includes trade secrets, confidential or proprietary information,
copyrightable or creative works, ideas, patents, or inventions
Intelligence
The ability to think abstractly and to learn readily from experience
Intelligence test
A psychological or educational test designed to measure intellectual processes in accord with some evidence-
based theory of intelligence; A test that measures the higher intellectual capacities of a person, such as the
ability to perceive and understand relationships and the ability to recall associated meaning; May measure the
ability to learn
Inter-coder reliability
Consistency among different coders
Inter-rater coefficients
Special type of reliability coefficient used to determine the extent to which two or more raters are consistent in their scoring of
students; Often used to determine whether two judges grade in the same way (e.g. would students receive the same grade if
their responses were graded by two different teachers).
Inter-rater or inter-observer reliability or agreement

The degree to which different raters or observers give consistent estimates of the same phenomenon; Achieved
when different coders arrive at the same results when the same body of material is examined; The extent to
which two or more individuals agree which addresses the consistency of the implementation of a rating system
Inter-scorer reliability

Assessment Glossary
The degree of agreement or consistency between two or more scorers, judges, or raters
Interaction
A measure of the extent to which the effect of one factor is different for different levels of another factor
Interaction effect
When the effect of one independent variable depends on the level of another independent variable
Interdisciplinary
The covering of knowledge and skills from several academic subject areas and/or domains
Interdisciplinary or integrated assessment

Assessing students’ abilities to apply concepts, principles, skills, and processes from two or more subject
disciplines to a central question, theme, issue, or problem
Intermediary
A third party who serves as a facilitator for negotiating a deal or other transaction between parties
Interim statement
A report containing financial information covering a period of less than one year (i.e. quarterly reports)
Internal audit
The process of conducting an in-house examination of one or more of an organization’s processes, functions,
programs, etc.
Internal assessment
Analysis of an organization’s position, performance, problems, and potential
Internal consistency or reliability

The consistency with which a test measures a single construct; The extent to which all questions or items
assess the same characteristic, skill, or quality; A form of validation of a multiple item test where the criterion is
the total score on the test itself; The extent to which a scale or instrument is measuring a single idea
Internal consistency coefficient

An index of the reliability of test scores derived from the statistical interrelationships of responses among item
responses or scores on separate parts of a test; One of many types of reliability coefficients in classical test
theory; A measure of the degree to which items on an instrument are inter-correlated
Internal evaluation or audit

An evaluation, assessment, or audit conducted by a staff member or unit from within the organization being
studied
Internal reliability
See Internal consistency
Internal reliability coefficient

See Internal consistency coefficient
Internal scanning
Looking inside the business and identifying strengths and weaknesses of the firm
Internal validity

Assessment Glossary
The ability to infer that a causal relationship exists between two variables; The rigor with which the study was
conducted (e.g. the study's design, the care taken to conduct measurements, and decisions concerning what
was and wasn't measured); Extent to which the designers of a study have taken into account alternative
explanations for any causal relationships they explore
Interpersonal communication
Interpersonal communication occurs when two people engage in voluntary, ongoing, interdependent
interactions, which involve meaningful interpretation of their verbal and nonverbal behaviors
Interpersonal skills
The personal ability to create unique shared meaning with others.
Interpretation
The act of explaining test scores to students so they understand exactly what each type of score means; For
example: Explaining that a percentile rank refers to the percentage of students in the norm group who fall below
a particular point, not the percentage of items answered correctly
Interpretive validity
Accurately portraying the meaning given by the participants to what is being studied
Interrupted time-series design

A design in which a treatment condition is assessed by comparing the pattern of pretest responses with the
pattern of posttest responses obtained from a single group of participants
Interval data
A collection of observations of an interval variable
Interval scale
A scale of measurement that has equal intervals of distances between adjacent numbers
Interval variable
A variable in which both the order of data points and the distance between data points can be determined (e.g.
percentage scores and distances)
Intervening or mediating variable

A variable that occurs between two other variables in a causal chain
Intervention
Doing something to change the possibility of the result; Actions taken to improve goal achievement; Changes
made to improve outcomes or performance; The act of remediation itself, or the action taken to remediate a
condition; The treatment which is actually delivered to the person; The alternative to doing nothing and letting
nature take its course
Intervention study
Research designed to establish the efficacy and/or effectiveness of a remediation, treatment, program, etc
Interview or interviewing
Asking questions in a systematic way to determine another’s perspective or experiences; Asking individuals to
share their perceptions of their own attitudes and/or behaviors or those of others; A face-to-face or long-
distance (e.g. by telephone) discussion between two or more people to collect information and the opinions of
the people being interviewed about certain topics or events; Evaluating student reports of their attitudes and/or
behaviors in a face-to-face-dialogue; A survey research technique; Interviews are often audio- or video-taped for
later transcription and analysis; A series of orally-delivered questions designed to elicit responses concerning
attitudes, information, interests, knowledge, and opinions; May be conducted in person or by telephone, and

Assessment Glossary
with one subject or a group of subjects; The three major types of interviews are as follows: Structured: All
questions to be asked by the interviewer are specified in advance; Semi-structured: The interviewer can ask
other questions and prompts in addition to the specified questions; Unstructured: The interviewer has a list of
topics, but no or few specified questions
Interview guide
A list of topics to be covered in an interview which is similar but less structured than a questionnaire and without
multiple-response questions.
Interview schedule
A formal instrument that specifies the precise wording and ordering of all the questions to be asked of each
respondent during a structured interview
Interview protocol
Data-collection instrument used in an interview
Interviewee
The person being asked questions
Interviewer
The person asking the questions; The assessor who conducts the interview, either in a ace-to-face setting or by
telephone, and makes a record of the responses
Intra-coder reliability
Consistency within a single individual in coding qualitative data
Intra-rater or Intra-observer reliability or agreement

The degree to which the same rater or observer gives consistent estimates of the same phenomena; Achieved
when the same coder arrives at the same results when the same body of material or level of phenomena is
examined at different times or under different conditions
Inventory
A questionnaire or checklist, usually in the form of a self-report, that elicits information about an individual's
personal opinions, interests, attitudes, preferences, personality characteristics, motivations, and typical
reactions to situations and problems; A catalog or list for assessing the absence or presence of certain attitudes,
interests, behaviors, or other items regarded as relevant to a given purpose
Investigation
A systematic examination, observation, or inquiry; May be a type of assessment task or activity, or a process
conducted during or after the administration of an assessment as part of a quality check
Investigator triangulation
The use of multiple investigators in collecting and interpreting the data
Investment center
An enterprise that has a manager who is responsible for profit and investment performance and who controls
most of the factors affecting revenues, costs, and investments
Involvement
Participating in or being engaged in an activity or relationship; In higher education: Student, faculty,
administrator, alumni, or employer participation in the teaching and learning enterprise
Ipsative scale

Assessment Glossary
Assessment or measurement by comparison with one’s self; Involves the individual using his/her values or
behaviors as the yardstick by which the comparisons are made
IQ or I. Q. tests
The first of the standardized norm-referenced tests, developed during the nineteenth century; Traditional
psychologists believe that neurological and genetic factors underlie "intelligence" and that scoring the
performance of certain intellectual tasks can provide assessors with a measurement of general intelligence;
There is a substantial body of research that suggests that IQ tests measure only certain analytical skills, missing
many areas of human endeavor considered to be intelligent behavior; An IQ is considered by some to be fixed
or static, but an increasing number of researchers are finding that intelligence is an ongoing process that
continues to change throughout life
Irregularity
A variation form established rules, standards, principles, or procedures in a manner that can lead to unjustifiable
actions or indefensible decisions
Irrelevant information
Information which is not coded or recorded because it is viewed as not relevant to the research question; In
content analysis one must decide what to do with the information in the text that is not coded: One's options
include either deleting or skipping over unwanted material, or viewing all information as relevant and important
and using it to reexamine, reassess and perhaps even alter the one's coding scheme
Item
An individual question or exercise in an assessment or evaluative instrument; A single question, problem, or
task used to assess a student; A question or problem on a test
Item analysis
A technique employed to analyze student responses to objective test items; Analyzing each item on a test to
determine the proportions of students selecting each answer; Can be used to evaluate student strengths and
weaknesses; May point to problems with the test's validity and to possible bias; Used both to improve the quality
of items and to enhance the interpretation of results; May show the difficulty of the items and the extent to which
each item properly discriminates between high achieving and low achieving students
Item banks or pools

A collection of test questions with known statistical properties from which a sample of items can be drawn to
create a measure
Item bias
An item is biased when it systematically measures differently for different ethnic, cultural, regional, or gender
groups
Item characteristic curve (ICC)

The graphical representation of item characteristics determined through the application of item response theory
Item response theory (IRT)

Test theory approach based on the observation that the probability of an examinee correctly responding to an
item is determined by an interaction of examinee ability and item characteristics; A method for scaling individual
items for difficulty in such a way that an item has a known probability of being correctly completed by an adult of
a given ability level
Item stem
The set of words forming a question or statement
Iterative

Assessment Glossary
Term used in research to refer to the repetition of a cycle of processes with an eye toward moving ever more
closely toward desired results

Assessment Glossary
J
Jackknife
A resampling technique in which researchers rerun analyses several times dropping one participant from the
analysis each time
Job analysis
A technique for studying a teaching job in terms of the knowledge, skills, and abilities (KSAs) needed to perform
the job in a minimally acceptable manner as well as the functions and tasks that are performed as part of the
job; May involve observation of teachers while they are doing their jobs as well as interviews with these
teachers, and possibly their supervisors, and review of such documents as job descriptions, school procedures,
and teacher products
Job description
A summary of the qualifications, duties, responsibilities, physical and mental demands, and working conditions
associated with a specific job
Johari window
A leadership disclosure and feedback model used in performance measurement featuring the four quadrants or
windows of knowing (Quadrant I represents the area of free activity or public area and refers to behavior and
motivation known to self and known to others; Quadrant II represents the blind area where others can see things
in ourselves of which we are unaware; Quadrant III represents the avoided or hidden areas, things we
know but do not reveal to others; Quadrant IV represents the areas of unknown activity, in which neither the
individual nor others are aware of certain behaviors or motives)
Joint Committee on Standards for Educational Evaluation

Coalition of professional associations founded in 1975 and accredited by the American National Standards
Institute (ANSI) in 1989 that have published evaluation standards in the areas of accuracy, feasibility, propriety,
and utility for use in student, program, and personnel evaluations (see http://www.wmich.edu/evalctr/jc/)
Joint employment
The relationship between a professional employer organization or employee leasing firm and an employer,
based on a contractual sharing of liability and responsibility for employees
Journals or journaling
Students' personal records and reactions to various aspects of learning and developing ideas; A reflective
process often found to consolidate and enhance learning; A way to assess student learning by having students
record their new learning, questions to pursue, and observations about themselves as learners.
Judgment
An appraisal, decision, or opinion about the performance level of a learner/teacher with respect to the
knowledge, skill, ability, behavior, or attribute being assessed
Jurisdiction
The limits or territory within which authority may be exercised; In bureaucratic politics, the area of programmatic
responsibility assigned to an agency by the legislature or chief executive; A term used to describe the territory
within the boundaries of a government entity (e.g. a local jurisdiction)
Jury trial for projects

Project evaluation process patterned after jury trial procedures for clarifying issues, introducing and assessing
evidence, and reaching conclusions; Sometimes known as the Adversary Model of Evaluation

Assessment Glossary
Just in time
An inventory control and management strategy in which raw materials are delivered from the supplier just prior
to their use in the manufacturing process

Assessment Glossary
K
k
The size of the sampling interval used in systematic sampling
Kendall coefficient of agreement (u)

A test to detect an underlying logical pattern in a series of repeated paired comparisons
Kendall coefficient of concordance (W)

A test for the correlation of more than two variables used for ordinal data
Key informant interviews

A method of collecting information about a community or group of people by finding and interviewing a limited
number of individuals determined or expected to be well informed and unbiased
Key performance indicators (KPI)

Quantifiable, specific measures of an organization’s performance in certain areas of its business; The purpose
of KPIs is to provide the company with quantifiable measurements of items it has determined to be important to
the organizational or business long-term goals and critical success factors; Used to understand and improve
organizational performance and overall success; Also referred to as key success indicators
Knowledge
The sum of the information and experience the learner/teacher has acquired or learned and is able to recall or
use; Cognitive objective that involves recalling or remembering information without necessarily understanding it
and includes behaviors such as describing, listing, identifying, and labeling
Knowledge assets
The parts of an organization’s intangible assets relating specifically to knowledge, expertise, information, ideas,
best practices, intellectual property, and other capabilities
Knowledge integration
Broadly defined as the assimilation, extraction, transformation, and loading of information from disparate
systems into a single, more unified, consistent, and accurate data store used for evaluating, manipulating, and
reporting information
Knowledge management (KM)

The process of creating, acquiring, sharing and managing knowledge to augment individual and organizational
performance
Knowledge mapping
A process used to create a summation of the knowledge an organization will need in order to support its overall
goals, objectives, strategies, and missions
Known groups evidence

Evidence that groups that are known to differ on the construct do differ on the test in the hypothesized direction
KSAs
Acronym for knowledge, skills, and abilities
Kuder-Richardson formula or reliability coefficient (KR20, KR21)

Assessment Glossary
A statistical formula used to compute an estimate of the reliability of a homogeneous test; One of many formulas
for determining the internal consistency of a dichotomously scored measure; A measure of inter-item
consistency

Assessment Glossary
L
Labor productivity
The correlation between a given output and the percentage of labor time used to produce the output
Last-in, first-out (LIFO)

An asset-management and valuation system that assumes the assets acquired last will be used, sold or
disposed of first; See also first-in, first out (FIFO)
Late mover advantage

The competitive advantage held by firms that are late in entering a market; Late movers are often able to imitate
the technological advances of other firms or reduce risks by waiting until a new market is established
Latent semantic analysis (LSA)

A kind of machine learning that simulates the human brain's understanding of the meaning of words and
paragraphs; Currently employed in the rating of short writing samples; Used in computerized grading of student
writing; Proposed basis for a new generation of smart search engines
Latent variable
An indirectly observable variable or factor
Lead time
The total time required to produce a product or service
Leadership
The process by which an individual influences a group and directs it toward a specific goal or organizational
mission
Leading question
A question that suggests a certain answer
Lean manufacturing
A management principle that focuses on the reduction of waste to improve overall customer value
Learning
The process by which a relatively lasting change in potential behavior occurs as a result of practice or
experience
Learning communities
Concurrent student enrollment in two or more disciplines connected by a common theme; Sometimes targeted
to a particular student group such as entering freshmen
Learning curve
A graph that depicts the relationship between the rate at which knowledge or a skill is learned and the time
spent acquiring it
Learning gain
A positive change in learning outcomes measured following instruction or educational experiences; Difference
between pretest and posttest; Longitudinal change in learning outcome measures
Learning organization or institution

Assessment Glossary
An organization where people continually expand their capacity to create the results they truly desire, where
new and expansive patterns of thinking are nurtured, where collective aspiration is set free, and where people
are continually learning to see the whole together; An organization that has developed a continuous ability to
learn, adapt and change
Learning outcomes
Specific knowledge or skills that students actually develop though their college experience; The products of
instruction or exposure to new knowledge or skills; Outcome behaviors: Observable behaviors or actions on the
part of students that demonstrate that the intended learning objective has occurred; Operational statements
describing specific student behaviors that evidence the acquisition of desired knowledge, skills, abilities,
capacities, attitudes, or dispositions; Behavioral criteria for determining whether students are achieving the
educational objectives of a program, and, ultimately, whether overall program goals are being successfully met;
Often treated as synonymous with objectives, although objectives are usually more general statements of what
students are expected to achieve in an academic program which are less measurable than outcomes
Learning standards
Learning standards define in a general sense the skills and abilities to be mastered by students in each strand
at clearly articulated levels of proficiency
Learning styles
Different preferred approaches to learning (e.g. visual, auditory, kinesthetic, etc.)
Legislation
A proposed or enacted law or group of laws
Leniency
A type of rater effect in which an assessor or an evaluator tends to rate a learner/teacher too high or to judge
the performance level as better than it actually is
Lesson
The content that is to be taught or the activity that is to be done during a specific period of instructional time
Letter grade
A summary evaluation of a student's proficiency or competency expressed on an alphanumeric or numeric scale
(e.g. the common grading scale of A = 4, B = 3, C = 2, D = 1, and F = 0)
Level
Category into which a factor has been divided
Level of analysis
Chosen by determining which word, set of words, or phrases will constitute a concept in a content analysis;
Generally 100-500 concepts is considered sufficient when coding for a specific topic
Level of confidence
The probability that a confidence interval to be constructed from a random sample will include the population
parameter
Level of generalization
In content analysis a researcher must decide whether concepts are to be coded exactly as they appear, or if
they can be recorded in some altered or collapsed form
Level of significance
The probability that observed or greater differences occurred by chance

Assessment Glossary
Level of implication
In content analysis one must determine whether to code simply for explicit appearances of concepts, or for
implied concepts, as well
Level of measurement
The relationship among the values that are assigned to the attributes for a variable; Knowing the level of
measurement helps you decide how to interpret the data (e.g. when a measure is nominal then you know that
the numerical values are just short codes for the longer names); Knowing the level of measurement helps you
decide what statistical analysis is appropriate (e.g. if a measure is nominal, then you know that you would never
average the data values or do a t-test on the data); A hierarchy is implied in the levels of measurement: At the
lower levels assumptions tend to be less restrictive and data analyses tend to be less sensitive, at each level up
the hierarchy, the current level includes all of the qualities of the one below it plus something new; There are
typically four levels of measurement: Nominal: Numerical values just "name" the attribute uniquely with no
implied ordering of the cases; Ordinal: Attributes can be rank-ordered but distances between attributes do not
have any meaning nor do averages; Interval: The distance between attributes has meaning, the interval
between values is interpretable, and averages have meaning but measurement ratios do not have meaning (e.g.
80 degrees is not twice as hot as 40 degrees); Ratio: Has an absolute zero that is meaningful and averages
and fractions (or ratios) are meaningful; Generally it is more desirable to have a higher level of measurement
(e.g. interval or ratio) rather than a lower one (nominal or ordinal)
Licensing agreement
An agreement between two enterprises or individuals granting the legal right to use a patent or trademark
Lifelong education
Students committed to lifelong learning as a result of their college experience, will express an openness and
desire for intellectual experiences, express confidence in their ability to independently acquire new knowledge,
and project a life of learning activities; Intellectual curiosity or intrinsic motivation to learn
Lifelong learning
A broad concept where education that is flexible, diverse, and available at different times and places is pursued
throughout life; Concept associated with the four pillars of education for the future: learning to know, learning to
do, learning to live together (and with others), and learning to be; The need for incumbent workers to
continuously develop new technical skills and competencies as industry and technology changes and/or as
workers change jobs
Lifelong learning skills

A skill that can be used in virtually all aspects of life or can be used in many situations throughout a person's
lifetime (e.g. the ability to make and carry out effective plans)
Likert scale
A method to prompt a respondent to express their opinion on a statement being presented; Likert scales are
often 4 point scales (strongly agree, agree, disagree, strongly disagree) or 5 point scales (strongly agree,
agree, neutral, disagree, strongly disagree), but sometimes can have as many as 10 potential choices; An item
type used on objective measures allowing respondents to indicate their level of agreement with a statement by
marking their response on a five point scale, usually ranging from strongly agree to strongly disagree; A
summated rating scale
Line-item budget
Budget format for listing categories of expenditures along with amounts allocated to each
Linear regression
A linear equation for forecasting criterion scores from scores on one or more predictor variables; A procedure
often used in selection programs or actuarial prediction and diagnosis

Assessment Glossary
Listening skills
Attending to and interpreting verbal messages from other people
Literature review
An in-depth review of key works and information available on a topic; A researcher typically conducts a literature
search and uses the resulting information to write a literature review; The section of a research report that
provides the framework of the research investigation and summarizes the literature the researcher studied in
order to develop the research study
Literature search
A comprehensive survey of publications on a specific topic that usually results in a list of references; The
process of conducting a literature search familiarizes a researcher with the body of work related to the research
interest; A researcher may use the list of references from a literature search as source material to write a
literature review
Litmus test
A test that uses a single key indicator to prompt a decision
Loaded question
A question containing emotionally charged words
Local assessment
Means and methods that are developed by an institution's faculty based on their teaching approaches, students,
and learning goals
Local norms
Norms that have been obtained from data collected in a limited locale, such as a school system, county, or
state; May be used instead of, or along with, national norms to evaluate student performance
Local responsiveness
The global strategy that emphasizes responsiveness to local markets and needs
Localization
The strategy of applying locale-specific terminology and data to a specific product or application in order to meet
the language, cultural, and other requirements of a specific market
Locally developed exams

Objective and/or subjective tests designed by the faculty of the program or course sequence being evaluated
Location parameter
A statistic from item response theory that pinpoints the ability level at which an item discriminates, or measures,
best
Long-term assets
The value of a company’s non-liquid assets such as property, equipment, and other capital minus depreciation
Long-term debt
Loans and other financial repayment obligations with a maturity date in excess of one year
Long-term liabilities
Debt and other repayments recorded on the balance sheets which are due in more than one year
Long-term orientation

Assessment Glossary
The cultural dimension that values building for the future and making investments that will be successful over
time
Longitudinal
Takes place over time
Longitudinal data
Data collected on the same individuals over time for use in a longitudinal study; Data collected from the same
population at different points in time
Longitudinal research
Data are collected at multiple time points and comparisons are made across time
Longitudinal study
A study that investigates development, learning, or other types of change in individuals or other variables over
time; An intervention study where treatment effects are reassessed at intervals over many years, not just at the
end of the initial intervention period; Considered the preferred method of evaluating educational initiatives where
the final results need to work their way in real time through the normal developmental life-cycle of the subjects;
Examples include cohort, cross-sectional, ethogenic, and trend studies
Low context cultures

Cultures that primarily value clear and unambiguous messages and emphasize time management, deadlines
and punctuality
Low inference
The types of assessment tasks, judgments, decisions, and conclusions that require a low degree of subjectivity
on the part of the judge or evaluator
Low inference descriptors

Description that is phrased very similarly to the participants’ accounts and the researchers’ field notes
Lower limit
The smallest number on a confidence interval

Assessment Glossary
M
Main effect
The effect of one independent variable
Major assessment
Degree of student learning due to academic baccalaureate programs; See also Program assessment
Management by objectives (MBO)

Management technique, outlined by Peter Drucker in the 1950s, to increase organizational performance by
aligning goals and subordinate objectives throughout the organization through employee participation in setting
performance goals, courses of action, and time lines followed by performance measurement and evaluations;
Requires SMART goals, management buy-in, a systems perspective, alignment with the corporate or
organizational culture, and ethical safeguards and oversight; A management technique designed to facilitate
goal and priority-setting, development of plans, resource allocation, monitoring progress towards goals,
evaluating results, and generating and implementing improvements in performance
Management consultant
An individual who works independently to assist and advise clients with managerial responsibilities regarding
various organizational issues
Manipulation
An intervention studied by an experimenter
MANOVA (Multivariate Analysis of Variance)

An extension of ANOVA methods to cover cases where there is more than one dependent variable and where
the dependent variables cannot simply be combined; Technique used for assessing group differences across
multiple metric dependent variables simultaneously, based on a set of categorical (non-metric) variables acting
as independent variables; Multivariate significance test analogues of univariate ANOVA experimental designs; A
test of the significance of group differences in some m-dimensional space where each dimension is defined by
linear combinations of the original set of dependent variables; Analysis of variance where multiple dependent
variables are bundled together into a weighted linear combination or composite variable (e.g. canonical variates,
roots, Eigen values, vectors, or discriminant functions)
Map
A chart that summarizes the major elements of a system and shows the relationships between the parts of a
system
Margin of error
One half of the width of a confidence interval
Market life cycle

The period of time during which a considerable portion of the buying public remains interested in purchasing a
given product or service
Market penetration
The degree to which a product is recognized and purchased by customers in a specific market
Market segmentation
The process of dividing a market with a purpose of gaining a major portion of sales in a subgroup within a
category instead of a more limited share of purchases by all category users

Assessment Glossary
Market share
The percentage of the total sales (from all sources) of a given type of product or service attributable to a specific
company
Market value
The most probable price (in terms of money) that buyers are willing to pay for particular goods or services
Marginal mean
The mean of scores in the cells of a column or a row
Master list
A list of all the codes used in a research study
Mastery testing
An assessment process designed to determine whether a teacher or student has acquired a predetermined
level of content knowledge or skills; A test designed to indicate that the participant has or has not mastered
some domain or knowledge or skill, generally indicated by a passing score or cut score
Matched t-test
A statistical test used to compare two sets of scores for the same subject; A matched pairs t-test can be used to
determine if the scores of the same participants in a study differ under different conditions; For example: A
matched t-test could be used to determine if people write better essays after taking a writing class than they did
before taking the writing class
Matching
Equating the comparison groups on one or more variables that are correlated with the dependent variable;
Process of corresponding variables in experimental groups equally feature for feature; An experimental
procedure in which the subjects are so divided, by means other than lottery, that the groups are regarded for the
purposes at hand to be of equal merit or ability
Matching variable
The variable the researcher uses in the control technique called matching in order to eliminate it as an
alternative explanation
Materials-based assessment
Evaluation of learners on the basis of tests following the completion of a particular set of curriculum materials; A
commercial text and its accompanying workbook is an example of this type of assessment
Materials evaluation
Evaluations that assess the merit or worth of content-related physical items, including books, curricular guides,
films, tapes, and other tangible instructional products
Matrix organization
An organizational structure where employees report to more than one manager or supervisor
Maturation
Any physical or mental change that occurs over time in a participant and affects the participant’s performance
on the dependent variable
Maturity
To evolve to full growth and development, stability, integration, allocentrism, autonomy, and symbolization

Assessment Glossary
Maximum likelihood estimation (MLE)

A method of estimating population characteristics from a sample by choosing the values of the parameters that
will maximize the probability of getting the particular sample actually obtained from the population
Maximum variation sampling

Purposively selecting a wide range of cases
MBO
See Management by objectives
MBO-based evaluation
A teacher evaluation approach based on a set of pre-specified objectives prepared for or in collaboration with
the teacher
Mean
Arithmetic mean; The arithmetic average of a set of numbers; Standard measure of central tendency used for
interval or ratio data; Also called "the first moment" or "center" of a distribution of numeric scores; For a chosen
variable, sum the scores for each case and divide the sum by the number of cases and the result is the
arithmetic mean; A measure of central tendency calculated by dividing the sum of all the values by the number
of the values; May be overly affected by extremely low or high scores
Mean deviation
A measure of variation that indicates the average deviation of scores in a distribution from the mean;
Determined by averaging the absolute values of the deviations.
Measurability
Refers to how the outcome is operationally defined and measured, including the methodological soundness of
the chosen measures
Measure
A standard procedure for quantifying a sample of behavior from a larger domain; Often used interchangeably
with test and instrument; An instrument or device that provides data on the quantity or quality of that aspect of
teaching performance being evaluated; To classify or estimate, in relation to a scale, rubric, or standard, the
degree of quality or quantity of that aspect of learning/teaching being evaluated; The process of collecting data
using appropriate techniques
Measure of variability
A numerical index that provides information about how spread out or how much variation is present
Measurement
Assigning numbers to observations according to rules; The process of assigning numbers or categories to
performance according to specified rules; The numerical estimation of the ratio of a magnitude of an attribute to
a unit of the same attribute; May be quantitative or qualitative; Honest collection, storing, retrieving and
interpreting of information to help an institution meets its mission; May be a frequency count of kinds of
responses, a rating indicating the level of attainment of a desired quality, or the percentage of correct answers
on a multiple choice test; In criterion-referenced assessment, may be specified as degrees of attainment based
on what is possible and stated in terms of percentages of attainment along the continuum or section of a
continuum (such as what can be expected for a particular grade level), where the continuum represents ultimate
mastery of the criterion; Quantitative description of student learning and qualitative description of student
attitude; Design of strategies, techniques and instruments for collecting feedback data that evidence the extent
to which students demonstrate the desired behaviors; A quality or quantity arrived at by observation of an
operationalized measure of a variable in known circumstances
Measurement-based evaluation

Assessment Glossary
A teacher evaluation approach that is based on the use of methods which are not dependent on the expertise of
the evaluator
Measurement error
Difference between the observed value of a measurement and the “true” value
Measurement error variance

That portion of the observed score variance attributable to one or more sources of measurement error; The
square of the standard error of measurement
Measures of central tendency

The single numerical value considered most typical of the values of a quantitative variable (e.g. mode, median,
and mean); Sometimes called a measure of "central location"; A single category or "most typical" score from the
distribution of your variable
Measures of relative standing

Numerical values which provide information about where a score falls in relation to the other scores in the
distribution of data
Median
The middle score in a distribution or set of ranked scores; The middle value in a series of values arranged in
rank order; The point on a scale that divides a group into two equal subgroups; Not affected by extremely low or
high scores; Measure of central tendency used with ordinal, interval, or ratio data; When a set of cases have
been rank-ordered (from highest to lowest or lowest to highest) the median is the middle score; The median is
also the 50th percentile; Cases or categories MUST be rank ordered or the median is meaningless; With true
ordinal data the median category should be a verbal category; Do NOT use a numerical category from your
output if the median category is a word or words, rather than a true interval-level number; It is a very common
mistake to assign a number for the median to ordinal data; Use the verbal label for the median category in
ordinal data; For example: "Agree" is a word, not a number
Mental models
A group or network of interrelated concepts that reflect conscious or subconscious perceptions of reality;
Internal mental networks of meaning constructed as people draw inferences and gather information about the
world
Mentor teacher
An experienced, often specially trained, teacher who works with new teachers, interns, or regular teachers in a
professional improvement program; Mentors serve as resources, coaches, advisors, and confidants to other
teachers and may be involved in formative evaluation activities as well as in the development and
implementation of assistance plans
Mentoring
The provision of support by experienced teachers to promote the development of new or less experienced
teachers
Merit pay
Salary increments allocated to a teacher based on an evaluation that demonstrates the teacher's superior level
of performance; Increases in salary and wages that are tied to actual quality of work performed
Meritorious performance
Performance level significantly exceeding the minimally acceptable standard which may be worthy of
professional recognition, career advancement, reward, or merit pay
Meta analysis

Assessment Glossary
A systematic way of compiling results across studies in order to clarify the findings in an area of research; A
quantitative technique used to integrate and describe the results of a large number of studies
Meta cognition or metacognition

The knowledge of one's own thinking processes and strategies, and the ability to consciously reflect and act on
the knowledge of cognition to modify those processes and strategies; Refers to an individual's ability to think
about his/her own thinking and to monitor his/her own learning; Metacognition is integral to a learner's ability to
actively partner in his or her own learning and facilitates transfer of learning to other contexts
Meta evaluation
An evaluation of an evaluation
Method of assessment
Tests and procedures used to measure student performance in meeting the standards for a learning outcome;
The techniques or instruments used to measure learner/teacher attributes and behaviors; For example: Rating
scales, observation checklists, structured interviews, and portfolios
Method of data collection

The specific means used to document learner/teacher performance; Includes the data forms and procedures
necessary to define the specifics of the valuation model or system; Five common methods are test scores,
observation, reflection, ratings, and portfolios; Technique for physically obtaining data to be analyzed in a
research study
Method of evaluation or assessment

The approach used to conduct the evaluation (e.g. the use of formal classroom observations followed by an
interview with the supervisor and an oral examination by a team of peers); Techniques or instruments used in an
assessment or evaluation
Method of working hypotheses

Attempting to identify all rival explanations
Methods triangulation
The use of multiple research methods
Methodology
How you collected the data; A set of analytical methods, procedures and techniques used to collect and analyze
information appropriate for evaluation of the particular program, project or activity; Methodology is also a section
of a research report in which the researcher outlines the approach used in the research, including the method of
recruiting participants, the types of questions used, etc.
Metrics
The written, active, and operational components of an objective which are well defined and measurable in their
accomplishment
Mini-max budgeting
A budgeting tactic in which two separate budgets are developed based on maximum assumed outputs and
minimum assumed outputs
Minimally acceptable
A performance level that meets the minimum standards, as defined by its criteria; Any lower level of
performance is not acceptable in terms of the purpose of an evaluation
Minimum competency

Assessment Glossary
A level of knowledge, skill, or ability (usually demonstrated on a measure) that has been determined to be the
minimum required for successful use of that knowledge, skill, ability, or personal trait
Minimum efficient scale

The smallest output for which unit costs are minimized
Mission or mission statement

A general statement of purpose guiding the practices of an institution or program; A program mission statement
should provide an overview of the department/program’s philosophy, goals, and objectives, and embody the
program’s purpose and the faculty’s priorities for the program; An institutional mission statement is the internally
defined statement that documents the vision of the institution and answers the question: What is the
organization attempting to accomplish?; Institutional mission statement objectives are broadly defined
statements of intention defined by the strategic planning process which are not limited to student learning
outcomes; A statement illustrating what the company is, what the company does and where the company is
headed; A brief, comprehensive statement of purpose of an agency or program; Declaration of an organization’s
purpose or reason for being; The broadest possible statement of the organization's purposes, goals, values and
functions; Describes the underlying design, aim, or thrust of an organization and contains the basic purpose or
philosophy of the organization
Mixed or mixed method research

Use of both quantitative and qualitative data collection and data analysis techniques; Research in which the
researcher uses the qualitative research paradigm for one phase of a research study and the quantitative
research paradigm for a different phase of the study; A quantitative phase and a qualitative phase are included
in the overall research study; Research in which quantitative and qualitative techniques are mixed in a single
study; Research that involves the mixing of quantitative and qualitative methods or paradigm characteristics
Mixed model research

Research in which the researcher uses both qualitative and quantitative research within a stage or across two of
the stages in the research process; Quantitative and qualitative approaches are mixed within or across the
stages of the research process
Mixed purposeful sampling

The mixing of more than one sampling strategy
Mixed scanning
A model of decision making that combines the rational-comprehensive model's emphasis on fundamental
choices and long-term consequences with the incrementalists' emphasis on changing only what needs to be
changed in the immediate situation
Mode
The score or value that occurs most frequently in a distribution; The most frequently occurring value in a set of
ranked data observations; A measure of central tendency usable for all types of data; The category that
contains the largest frequency or the greatest number of scores; The mode is the score that occurs the most
often in your variable of interest; If your variable is a nominal measure, the mode is the only measure of central
tendency that you can use (but you still can do percents, rates, ratios and compare groups on a nominal
variable)
Model
An example of a coherent method, approach, procedure, or strategy of learning/teaching or of
assessment/evaluation, as defined by its key or unique assumptions, propositions, attributes, supportive theory,
research, practical precedent, or foundation, and which implicitly defines accomplished or good
learning/teaching
Modeling

Assessment Glossary
The use exemplary teachers and mentors to demonstrate practices of good teaching to other teachers for the
purpose of improvement or of repertoire expansion
Moderator
The researcher who leads a focus group and facilitates discussion
Moderator variable
In regression analysis, a variable that serves to explain, at least in part, the correlation of two other variables; A
variable that changes the relationship between other variables; A variable involved in an interaction effect
Modern organization theory

A body of theory emphasizing empirical examination of organizational behavior, interdisciplinary research
employing varied approaches, and attempts to arrive at generalizations applicable to many different kinds of
organizations
Modifications
Recommended actions or changes for improving student learning, service delivery, etc. that respond to the
respective measurement evaluation
Modus operandi analysis

Deducing the cause of effects based upon analysis of events, process, or properties associated with the effects;
Analogous to procedures used in detective work
Monitoring
The checking on a process or a person to verify that progress is being made, required activities are occurring,
assessment and evaluation procedures are being implemented, suggested teaching practices are being tried,
prior information is still applicable, earlier decisions can still be justified, and/or standards are being met;
Continuous research which regularly measures the change in one or more indicators; Assessing the inputs and
activities of a project
Moral development
The transformation that occurs in a person's form or structure of thought with regard to what is viewed as right or
necessary
Mortality
Type of bias resulting from subjects not completing the study period; May be literal mortality (as in patient
deaths in medical research) or figurative (as in student attrition in educational research); See Attrition
Motivation
The entire constellation of factors, both intrinsic and extrinsic, that causes a student to behave in a particular
way at a particular time; With regard to student outcomes the focus is upon testing behaviors and the
development of life-long learning
Multi-factor test
An instrument that measures two or more constructs which are less than perfectly correlated
Multi-group research design

A research design that includes more than one group of participants
Multi-modal methods
A research approach that employs a variety of methods
Multi-trait multi-method

Assessment Glossary
An approach to construct validation where the construct of interest is measured using more than one
experimental method and other related constructs are measured using the same methods so that a priori
assumptions about the relationships between the constructs can be investigated
Multidimensional assessment
Assessment that gathers information about a broad spectrum of abilities and skills
Multiple-baseline design
A single-case experimental design in which the treatment condition is successively administered to different
participants, or to the same participant in several settings, after baseline behaviors have been recorded for
different periods of time
Multiple choice or multiple response tests

A test in which students are presented with a question or an incomplete sentence or idea, and are expected to
choose the correct or best answer/completion from a menu of alternatives; A question, problem, or statement
(called a stem) appears on a test followed by two or more answer choices, where the incorrect choices (called
distractors) usually reflect common errors, and the examinee's task is to choose from the alternatives provided
the best answer to the question posed in the stem
Multiple measures
The array of different types of evidence that are collected or assessment instruments that are used to better
assess a learner/teacher's knowledge, skills, and performance; Multiple measures of the same attribute provide
a more comprehensive, reliable, and valid measure of that attribute than any one measure alone; Multiple
sources of assessment data gathered from a variety of sources
Multiple operationalism
The use of several measures of a construct
Multiple regression
Regression based on one dependent variable and two or more independent variables
Multiple time-series design

An interrupted time-series design that includes a control group to rule out a history effect
Multiple-treatment interference
Occurs when participation in one treatment condition influences a person’s performance in another treatment
condition
Multivariate analysis
May be one of several methods for examining multiple variables; Strict usage: Research designs with two or
more independent variables and two or more dependent variable; Loose usage: Research designs with two or
more independent and/or dependent variables; Examples include path analysis, factor analysis, multiple
regression analysis, MANOVA, LISREL, canonical correlations, and discriminant analysis
Multivariate methods
A collection of techniques appropriate for the situation in which the random variation in several variables has to
be studied simultaneously; The main multivariate methods are principal components analysis, factor analysis,
discriminant analysis, and cluster analysis
Municipal bond
An often tax-exempt bond issued by federal, city, county, state or local government agencies to finance capital
expenditures for public projects
Mutually exclusive

Assessment Glossary
Property that sets, categories, or intervals do not overlap; Response categories that are separate and distinct;
Two or more events, conditions, or variables which cannot occur at the same time

Assessment Glossary
N
n
The sample size
N
The population size
Narrative inquiry
A qualitative research approach based on a researcher's narrative account of the investigation; Not to be
confused with a narrative examined by the researcher as data
Naturalistic assessment
Assessment evaluation rooted in the natural setting of the classroom: Involves observation of student
performances and behavior in an informal context done as students go about their daily work
Naturalistic generalization
Generalizing on the basis of similarity
Naturalistic inquiry
Observational research of a group in its natural setting; Qualitative researchers studying things in their natural
settings, attempting to make sense of, or interpret, phenomena in terms of the meanings people bring to them
Naturalistic observation
Observation done in “real world” settings; One of the three basic approaches to scientific research which is
characterized by recording of behavior as it occurs in a more or less naturalistic setting where there are no
attempts at intervention
National Institute for Literacy (NIFL)

An independent federal organization created by the National Literacy Act of 1991 to serve as a focal point for
public and private activities that support the development of high-quality regional, state, and national literacy
services
National Reporting System (NRS)

An outcome-based reporting system for the state-administered, federally funded adult education program
required by Title II of the Workforce Investment Act; The goals of the NRS were to establish a national
accountability system for education programs by identifying measures for national reporting and their definitions,
establishing methods for data collection, developing software standards for reporting to the U.S. Department of
Education, and developing training materials and activities on NRS requirements and procedures
Need
Something that is wanted or required to increase a person's quality of life, general wellbeing or happiness; The
difference between a desirable and an actual state of affairs; Needs may be classified as felt, expressed,
normative, comparative, individual, group, organizational, clinical, administrative, subjective, and/or objective
Needs analysis
A method of analyzing how an employee skill deficit can be addressed through current or future training and
professional development programs, how to determine the types of training/development programs required,
and how to prioritize training/development
Needs assessment

Assessment Glossary
A systematic exploration of the way things are compared to the way they should be; Evaluation of the service
delivery capacity of a community or organization in relation to the current or future demand or need for those
services; A process for identifying and prioritizing gaps in results based on the cost to meet the need versus the
cost to ignore the need; A systematic study of a problem or innovation, incorporating data and opinions from
varied sources, in order to make effective decisions or recommendations about what should happen next; A
structured process to determine the needs of a designated survey field (e.g. individuals, an agency, a system,
etc.)
Negative case sampling

Selecting cases that disconfirm the researcher’s expectations and generalizations
Negative correlation
The situation when scores on two variables move in opposite directions
Negative predictive value (NPV)

Diagnostic test that measures how well an instrument detects true negatives when all its decision negatives are
considered; A high NPV indicates that the false negative problem is under control
Negatively skewed
Skewed to the left
Negotiation
The process of bargaining among parties to reach a mutually agreeable solution
Network diagram
A diagram showing the direct links between variables or events over time
Neural network
A computer driven statistical model that can learn to recognize patterns in data
Next-in, first-out (NIFO)

An inventory valuation method that values inventory at projected levels of cost that reflect replacement values
over the near term
No significant difference
A decision that an observed difference between two statistics occurred by chance
Noise
Random error; Random outcomes or measures
Nominal data
A form of categorical data that classifies items into categories or groups, placing no real significance on the
order of the groups
Nominal group technique (NGT)

Form of group discussion or decision-making where participants first work independently, then present their
ideas one at a time to the group which then consolidates and ranks the ideas or proposed actions; A consensus
planning tool used to identify the strengths of an organization, department, or division, whereby participants are
brought together to discuss important issues, problems, and solutions
Nominal scale
A scale of measurement that uses symbols (such as words or numbers) to label, classify, or identify people or
objects

Assessment Glossary
Nominal variable
A variable determined by categories which cannot be ordered (e.g. gender and color)
Non-directional alternative hypothesis

An alternative hypothesis that includes the “not equal” sign
Non-discrimination
Evidence that differences of race or ethnicity, gender, or disability do not bias results of assessment instruments
or procedures
Non experimental research

Research in which the independent variable is not manipulated and there is no random assignment to groups
Non-instructional assessment
Determining the extent to which service outcomes in support of student learning are achieved
Non-instructional Unit
Business units of a school that impact students in ways not directly in the classroom
Non-linear regression
A non-linear equation for forecasting criterion scores from scores on one or more predictor variables; A
procedure often used in selection programs or actuarial prediction and diagnosis
Nonequivalent comparison-group design

A design consisting of a treatment group and a nonequivalent untreated comparison group both of which are
administered pretest and posttest measures
Nonparametric statistics
A family of statistics that are based on the nominal categorization rank ordering of data points; These statistics
do not require the sample to meet any distributional assumptions; Statistics which can be used with nominal and
ordinal data; Also sometimes usable with interval and ratio data when other assumptions cannot be met;
Examples include the chi-square test and the Spearman rank difference correlation coefficient; Mathematically
"less powerful" than their parametric equivalents (less likely to lead to significant results than parametric tests)
Nonreactive measures
Assessments done without the awareness of those being assessed
Nonverbal communication
The non-spoken aspects of communication that include such culturally sensitive factors as kinesics (body
language, gestures, eye contact, facial expressions, and other bodily movements), proxemics (conversational
distance and other uses of space), paralanguage (tone, pitch, accent, and other nonverbal aspects of speech),
chronemics (timing, frequency, tempo, rhythm, and other uses of time), haptics (touching), design factors
(physical signals such as clothing, accessories, and interior design), etc.
Norm
A single value or a distribution of values constituting the typical performance of a given group; A distribution of
scores obtained from a norm group; The midpoint or median of scores or student performance in that group;
Performance standards that are established by a reference group and that describe average or typical
performance; Usually norms are determined by testing a representative group and then calculating the group's
test performance
Norm group

Assessment Glossary
The group used as a point of comparison for a test; A random group of students selected by a test developer to
take a test to provide a range of scores and establish the percentiles of performance for use in establishing
scoring standards
Norm referenced
Describing a students’ performance on an assessment by comparison to a normed group; An interpretation of
scores on a measure that focuses on the rank ordering of students not their performance in relation to criteria
Norm referenced assessment

An assessment where student performance or performances are compared to a larger group; Usually the larger
group or "norm group" is a national sample representing a wide and diverse cross-section of students; Students,
schools, districts, and even states are compared or rank-ordered in relation to the norm group; The purpose of a
norm-referenced assessment is usually to sort students and not to measure achievement towards some
criterion of performance
Norm-referenced measurement
Measurement that provides data on how well individuals do in relation to others in the group
Norm referenced test interpretation

A score interpretation based on a comparison of a test taker's performance to the performance of other people
in a specified reference population
Norm referenced test or testing

A score that compares a student's performance to that of people in a norm group; The student or group scores
will not fall evenly on either side of the median established by the original test takers which serve as a
performance standard; Norm-referenced tests highlight achievement differences between and among studies to
produce a dependable rank order of students across a continuum of achievement from high achievers to low
achievers; An objective test that is standardized on a group of individuals whose performance is evaluated in
relation to the performance of others
Normal curve equivalent

Arithmetically manipulated assessment data so that scores range from 1-99; Normalized standard scores with a
mean of 50 and a standard deviation of approximately 21; Used to compare different tests for the same student
or group of students and between different students on the same test
Normal distribution
The curve representing the normal distribution of a rating or test score; A unimodal, symmetric, bell-shaped
distribution that is the theoretical model of many variables; A normal frequency distribution representing the
probability that a majority of randomly selected members of a population will fall within the middle of the
distribution; Represented by the bell curve; Also called a Gaussian distribution
Normal distribution curve

A bell-shaped curve representing a theoretical distribution of measurements that is often approximated by a
wide variety of actual data; Often used as a basis for scaling and statistical hypothesis testing and estimation in
psychology and education because it approximates the frequency distributions of sets of measurements of
human characteristics
Normal form
The criteria applied to relational databases to insure good design and efficient management of the database;
Following normal form avoids problems such as having the same data stored in multiple locations
Normalized standard score

A derived test score in which a numerical transformation has been chosen so that the score distribution closely
approximates a normal distribution, for some specific population

Assessment Glossary
Normative forecasting
A method of projecting future needs in order to determine what developments will be required to meet those
needs
Norming
A process by which instructors come to an agreement about how the rubric will be used and standards are
applied to evaluate assessments; Norming is done to avoid inter-rater error (i.e. large differences from evaluator
to evaluator in how assessments are scored) and help achieve inter-rater reliability
Norming group
The specific group for which the test publisher or researcher provides evidence for test validity and reliability
Norms
Statistics or tabular data that summarize the distribution of test performance for one or more specified groups;
Statistics that describe the test performance of specified groups, such as pupils of various ages or grades in a
standardization group for a test; Norms may be designed or assumed to be representative of some larger
population, such as pupils in the country as a whole; A performance standard that is established by a reference
group and that describes average or typical performance; Usually norms are determined by testing a
representative group and then calculating the group's test performance; A set of scores that describes the
performance of a specific population of students at a particular grade level on a selection or constructed
response set of tasks; The written and unwritten rules that specify appropriate group behavior
Novice
A person who has not had the education or training necessary to be considered an expert
Null hypothesis
The hypothesis of no difference or no differential effects; A statement about a population parameter which is a
deliberate statement of the contrary of what you really suspect to be the case; Useful when applying inferential
testing because group difference statistics, by their mathematical nature, make the initial presumption that two
samples are from the same population until proven otherwise, and report when the means of the two samples
move sufficiently apart for that presumption to be dismissed
Numeric response
A response style where the participant enters a number to indicate their choice
Numerical rating scale

A rating scale with anchored endpoints

Assessment Glossary
O
Object of the evaluation
What one is evaluating; For example: A program, project, or instructional material
Objective
A characteristic of an assessment, observation, or conclusion that minimizes the impact of bias and subjectivity,
and that yields results which can be empirically verified
Objective mastery
Test items are written measuring particular objectives and if enough items measuring the same objective are
answered correctly, then objective mastery is concluded
Objective measures
Tests or measures that can be scored without the need for subjective judgments (e.g. multiple-choice format)
Objective percent correct

The percent of the items measuring a single objective that a student answers correctly
Objective test
A test for which the scoring procedure is completely specified enabling agreement among different scorers; A
correct-answer test; Style of testing that measures the participants knowledge of objective facts, the correct
answers, to which, are known in advance
Objective scoring
Different scorers or raters will independently arrive at the same score or ratings for a student's performance;
Most often associated with assessment methods comprised of selection items
Objectives
Objectives describe the skills, tools and/or content that students master which enable them to fulfill, perform,
and/or accomplish the expected learning outcomes of a course or program of study; A specification of what is to
be accomplished, the time frame in which it is to be accomplished and by whom; The specific knowledge, skills,
or attitudes that students are expected to achieve through their college experience; Sometimes used to mean
the same as expected or intended student outcomes: Intended results in precise terms which specify what
needs to be assessed and serve as a more accurate guide in selecting appropriate assessment tools; The pre-
specified intended outcomes of a program, process, or policy; Specific and measurable targets for
accomplishing goals
Objectives for learning

See objectives
Objectives-referenced test
A test whose scores are referenced to the attainment of the objectives the test was designed to measure, rather
than to the performance on the test by some comparison group of people
Observable
That which can be seen and documented by another person (e.g. the tone of the teacher's voice can be
observed and recorded, but the thinking of the teacher that determined the tone of voice cannot be observed)
Observation

Assessment Glossary
One of several methods used to collect data about a learner/teacher's performance; Includes the recording of
evidence and notes while watching the behavior; May utilize audio or video recordings; A data collection
procedure or tool in qualitative research; An everyday skill which is methodologically systemized and applied in
qualitative research; Unobtrusive watching of the behavioral patterns of people; A research technique in which
no direct questions are asked, but people are watched and their behavior recorded; A method of data collection
in which data are gathered through visual or auditory observations; Observations may be structured or
unstructured
Observer
The person who collects evidence and notes about what he/she is observing, either in a classroom or another
setting; The observer is an assessor, but may or may not be an evaluator
Observer effect
The degree to which the assessment results are affected by the presence of an observer; The degree to which
the presence of an observer influences the outcome
Obtained score
The actual results for a learner/teacher on an assessment; Due to measurement error and other factors, the
obtained score may not be the same as the learner/teacher's true score
Occupational exploration
An awareness of a wide variety of occupations and the skills, knowledge, and attitudes needed to obtain them
Occupational skills attainment

Proficiency to perform actual tasks and technical functions required by certain occupational fields at entry,
intermediate, or advanced levels
OCR
Optical Character Recognition; A method whereby a computer can recognize text and other marks that have
been scanned
Ombudsman
Permanent office that receives complaints and acts on behalf of stakeholders to secure information, request
services, or pursue grievances
OMR
Optical Mark Reader; A device that scans paper forms (normally bubble sheets) and recognizes the marks
made on the form
On-demand assessment
An assessment that takes place at a predetermined time and place, usually under uniform conditions for all
students being assessed; An assessment process that takes place as a scheduled event outside the normal
routine; An attempt to summarize what students have learned that is not embedded in classroom activity; For
example: The SAT, district and state tests, intake assessments, exit exams, and most in-class unit tests and
final exams
One-group pretest-posttest design

A research design in which a treatment condition is administered to one group of participants after pretesting,
but before posttesting on the dependent variable
One-group posttest-only design

Administering a posttest to a single group of participants after they have been given an experimental treatment
condition

Assessment Glossary
One-stage cluster sampling

A set of clusters is randomly selected and all of the elements in the selected clusters are included in the sample
One-tailed test
A directional application of one of the two-group inferential statistics (i.e. the t-test, the Wilcoxon, or the Mann-
Whitney); Statistical procedure that only has to deal with the separation of two distributions down one or other of
the asymptotes (or "tails" of a bell curve), but not both
One-way analysis of variance

Statistical test used to compare two or more group means
Ongoing
Addressed regularly as part of the business of the college rather than in response to periodic external
requirements
Ongoing assessment
Tracks student learning by performance on tasks that are part of the natural lesson plan process; Ongoing
assessment provides accumulated data for summative analysis by producing samples of work clustered by the
outcome demonstrated
Open coding
The first stage in grounded theory data analysis
Open-ended
Assessment questions that are designed to permit spontaneous and unguided responses; A question that
allows participants to respond in their own words; A format of a test, assessment, or survey item that calls for
the answer to be supplied by the respondent rather than selecting from a list of options; For example: Essay
questions, short-answer questions, drawings, and fill-in-the-blank items
Open inquiry approach

Form of evaluation which aims to improve or change projects by asking open-ended questions (e.g. “what's
working?”, “what's not working?”, “how could we improve things?”, and “what are the community's needs?”) and
using methods such as focus groups and interviews
Open question
See open-ended
Open systems theory

A theory of organization that views organizations not as simple, "closed" bureaucratic structures separate from
their surroundings, but as highly complex, facing considerable uncertainty in their operations, and constantly
interacting with their environment; Assumes that organizational components will seek an "equilibrium" among
the forces pressing on them and their own responses to those forces
Operating budget
A detailed projection of all projected income and expenses during a specified future period
Operating cash flow

The total of company net profit, depreciation, accounts payable, and accrual changes, less inventory and
accounts receivable changes
Operating costs or expenses

The day-to-day expenses incurred in running a business, organization, or project; Typically includes the costs of
such items as personnel, materials, overhead, depreciation, and interest

Assessment Glossary
Operating grants
Grants for use in the development and operation of specific programs
Operational definition
A definition of a term or object achieved by stating the operations or procedures employed to distinguish it from
others; A very precise statement about how observed behaviors or events will be interpreted as representing a
designated construct; Definitions for terms and research variables specific to one program or project; A
definition used within a program or project
Operationalism
Representing constructs by a specific set of steps or operations
Operationalize
Defining a term or object so that it can be measured; Generally states the operations or procedures used that
distinguish it from others; The defining of a psychological or physical attribute by the way it is measured; The act
of assigning a particular physical dimension as a measure of a particular research variable
Operating budget
Tactical resource requirements, plus tactical priorities, equals the operating budget
Operations management
Decision-making that focuses on the performance and efficiency of the production process
Opinion poll
A type of survey in which people's opinions are asked
Opportunity costs
The amount that a particular product or resource could have earned in its next best use
Opportunity or opportunistic sampling

The act of selecting a research sample according to who is available to take part in it, rather than according to
more precisely derived criteria; Selecting cases when the opportunity occurs
Opportunity to learn (OTL)

To provide students with the teachers, materials, facilities, and instructional experiences that will enable them to
achieve high standards; What takes place in classrooms that enables students to acquire the knowledge and
skills that are expected; May include what is taught, how it is taught, by whom, and with what resources
Options
Alternatives available to students to select from in multiple-choice items
Oral examination
An assessment of student knowledge levels through a face-to-face dialogue between the student and examiner-
usually faculty
Oral histories
Interviews with a person who has had direct or indirect experience with or knowledge of the chosen topic
Order effect
A sequencing effect that occurs from the order in which the treatment conditions are administered
Ordinal data

Assessment Glossary
Data or numbers that can be placed in rank order; The main difference between ordinal data and nominal data
is that ordinal data contain both an equal-to and a greater-than relationship, whereas nominal data contain only
an equal-to relationship
Ordinal interaction effect

An interaction effect that occurs when the lines on a graph plotting the effect do not cross
Ordinal scale
A rank-order scale of measurement
Ordinal variable
A variable in which the order of data points can be determined but not the distance between data points (e.g.
letter grades)
Organization chart
A graphic representation outlining how authority and responsibility are distributed within an organization
Organizational change
A theory of organization that focuses on those characteristics of an organization that promote or hinder change;
Assumes that demands for change originate in the external environment, and that the organization should be in
the best position to respond to them
Organizational culture
An organization’s attitude and values regarding itself, employees, customers, and the general public;
Encompasses the manner in which things are done within the organization based on defined policies and
practices; Basic patterns of attitudes, beliefs, and values that underlie an organization's operation
Organizational design
The process of establishing and arranging the elements of an organization’s structure
Organizational humanism
A set of organization theories stressing that work holds intrinsic interest for the worker, that workers seek
satisfaction in their work, that they want to work rather than avoid it, and that they can be motivated through
systems of positive incentives (e.g. participation in decision making)
Organizational or organization development (OD)

Organization improvement through action research; A system-wide application of behavioral science knowledge
to the planned development and reinforcement of organizational strategies, structures, and processes for
improving an organization's effectiveness; Body of knowledge and practice that enhances organizational
performance and individual development which views the organization as a complex system of systems that
exist within a larger system, each of which has its own attributes and degrees of alignment; Collaborating with
organizational leaders and their groups to create systemic change and root-cause problem-solving on behalf of
improving productivity and employee satisfaction through improving the human processes through which they
get their work done; A long range effort to improve an organization's problem solving and renewal processes,
particularly through more effective and collaborative management of the organizational culture, often with the
assistance of a change agent or catalyst and the use of the theory and technology of the applied behavioral
sciences (primarily industrial/organizational psychology, industrial sociology, communication, cultural
anthropology, administrative theory, organizational behavior, economics, and political science); A planned
approach to far-reaching organizational change designed to enable an organization to respond and adapt to
changing market conditions and to set a new agenda; A long-term, systematic, and prescriptive approach to
planned organizational change which applies the concepts of the social and behavioral sciences to the problem
of changing an organization and treats the organization as a complex social and technical system that should
have enough flexibility to change its design according to the nature of its tasks and external environment; A
theory of organization that concentrates on increasing the ability of an organization to solve internal problems of

Assessment Glossary
organizational behavior as one of its routine functions and concerned primarily with the identification and
analysis of such problems
Organizational planning
The process of transforming an organization’s goals, objectives, philosophy and mission into practices and
policies
Organizational structure
The intentional arrangement of positions in an organization to accomplish the goals of the company; Types of
structures include functional, geographic, product, hybrid, and matrix
Organizational transformation
Organization-wide changes (such as restructuring operations, introducing new technologies, processes,
services or products, implementing new programs, re-engineering, etc.)
Orientational research
Research done for the purpose of advancing an ideological position; Sometimes called critical theory research
Out-of-level testing
Administering a test that is designed primarily for people of an age or grade level above or below that of the test
taker
Outcome evaluation
Evaluations that look specifically at whether or not the program achieved its goals and had its intended effect;
See also Impact evaluation
Outcome measure
Instruments used for gathering information on student learning and development; A performance measure that
quantifies the extent to which results are achieved, and that measures effectiveness
Outcome validity
The ability to generalize across different but related dependent variables
Outcome variables
The results or products of teaching; For example: Student mastery of skills, completion of courses, teacher-
developed instructional materials, student projects, and student performance on standardized tests
Outcomes
An operationally defined educational goal, usually a culminating activity, product, or performance that can be
measured; The specific knowledge, skills, or developmental attributes that students actually develop through
their college experience; The "end-products" of the entire instructional process; Specific, measurable behaviors
reflecting desired student knowledge, skills, values, or levels of service provided by a course, discipline,
program, or institution; The long-term end goals of an institution, program, course, project, or department;
Outcomes reflect the actual results achieved, as well as the impact or benefit, of a program; Assessment results
Outcomes assessment
A strategy used to evaluate and measure the results of an instructional method or program
Outlier
A number that is very atypical of the other numbers in a distribution
Output or outputs
Anything an institution or system produces; A type of performance measure that focuses on the level of

Assessment Glossary
activity in a particular program; Direct products of a program's activities which are usually measured in terms of
the volume of work accomplished; For example: The number of classes taught, counseling sessions conducted,
educational materials distributed, and participants served; Outputs have little inherent value in themselves but
may lead to outcomes or benefits
Output measure
A performance measure that measures efficiency by quantifying how many products are produced or services
performed
Overall performance level

A combination of the cut-scores or proficiency levels of the various assessments used to determine whether
students do not meet, meet, or exceed the standard set for a whole learning outcome; Different assessments
may be given greater weight when determining an overall performance level.
Overview
Conceptual/introductory statement that gives essential definitions, provides a general rationale, and presents
summarized procedures, common problems, and special difficulties that are applicable

Assessment Glossary
P
p-value
Standard mathematical shorthand for reporting a probability; A number between zero and one tells us that the
event may occur, and tells us if the event is more likely (a probability nearer to one, with 1 being certainty) or
less likely (a probability nearer to zero, with 0 being totally improbable) to occur; If we are tossing a fair coin the
probability of tossing a head is 0.5, and the probability of tossing a tail is also 0.5; The probability of an event
ranges between zero and one, where zero probability means that the event will never occur, and a probability of
one means that the event is certain to occur; Sometimes probability is described as a percent, ranging from 0%
to 100%, instead of a p-value; May also be a test item statistic that represents the percentage of students who
answered that item correctly out of a particular population group: May be calculated for a national
standardization sample or for a class or school level population where a test has been administered; Calculated
by dividing the number of correct responses on an item by the total number of students tested; May be
expressed as a decimal value or as a percentage (by multiplying the decimal value by 100);
Panel data
Refers to data that is collected on the same variables across different cohorts (e.g. collecting incoming freshman
SAT scores each year)
Panel study
The same individuals or group of respondents are repeatedly studied or surveyed at successive points in time to
detect changes in their behavior or opinions
Paradigm
See Research paradigm
Paradigm shift
A term used to define the process of change in patterns of thought or behavior; A change in how we think about
something
Parallel forms
Two or more forms of a test constructed to be as comparable and interchangeable as possible in their content,
difficulty, length, and administration procedures and in the scores and test properties (e.g. means, variance, and
standard error of measurement); Two (or more) item sets, matched for difficulty, in a psychometric package
used to avoid practice effects which might prevent using the same assessment twice on the same subjects
Parameter
A numerical characteristic of a population; A coefficient or value for the population that corresponds to a
particular statistic from a sample and is often inferred from the sample; A value used to represent a certain
population characteristic; For example: the population mean is a parameter that is often used to indicate the
average value of a quantity; A quantity, property or fact that gives information about the whole population; A
population parameter is usually unknown and has to be estimated
Parametric statistics
Used with interval and ratio data and usually with data that were obtained from groups randomly assigned,
normally distributed, and with equal variability between groups; Preferred statistics to use because they are
more "powerful" than nonparametric statistics; Examples include: t-tests, analysis of variance, and Pearson
correlation coefficient
Pareto chart
A bar graph used to rank in order of importance information such as causes or reasons for specific problems so
that measures for process improvement can be established

Assessment Glossary
Parochialism
A personal view of the world based only on one’s own perspectives and values; This view does not recognize or
respect other perspectives or ways of living and working
Partial correlation
Used to examine the relationship between two quantitative variables controlling for one or more quantitative
extraneous variables
Partial regression coefficient

The regression coefficient obtained in multiple regression
Participant
A person that participates in a testing, assessment, or survey process by answering questions
Participant-as-observer
Researcher spends extended time with the group as an insider and tells members they are being studied
Participant feedback
Discussion of the researcher’s conclusions with the actual participants
Participant observation
Observation of a group in which the observer is immersed in the day-to-day lives of the people or through one-
on-one interviews with members of the group
Participatory learning and action (PLA)

An umbrella term for a wide range of similar approaches and methodologies involving the full participation of
people in the processes of learning about their needs and opportunities, and in the action required to address
them
Participatory research
A method in which practitioners and learners consciously participate together as researchers in the research
process; Participatory research is often action research with a goal of achieving change
Particularism
A cultural perspective that is contingency-oriented and uses different evaluative standards based on
relationships and situations
Passing score
A score or level of performance that represents the difference between those learner/teachers whose level of
performance is minimally acceptable and those whose performance is not acceptable; Also called Cut or
critical score
Passive consent
A process whereby consent is given by not returning the consent form
Path analysis
An application of multiple regression analysis used to develop and test causal models using correlational data
Path coefficient
A quantitative index providing information about a direct effect
Pattern

Assessment Glossary
A series of similar behaviors or common attributes over a period of time or across different settings or contexts
Pattern matching
Predicting a pattern of results and determining whether the actual results fit the predicted pattern
Pedagogy
Instructional methods, usually not related to or specific to adult learning; The art and science of teaching; Some
pedagogical skills apply across teaching situations whereas others apply only to specific subject areas
Peer assessment
An assessment method in which students within a similar educational setting make and report judgments about
other students' performances; Evaluation of learning by one’s peers
Peer institution
An institution with similar or aspirational characteristics whose comparative quantitative data is used to establish
goals, to set priorities, to measure progress, and to influence institutional improvement
Peer review
Used as part of an accreditation process, part of an academic or student affairs program development, or for an
internal review process; The evaluation of an educational institution by other educators; Performance
evaluations done by co-workers; The evaluation of a teacher by other teachers; Usually done to provide
feedback to the evaluee for purposes of professional development and improvement or to provide subject-
matter and context-related expertise; Discussing one’s interpretations and conclusions with one’s peers or
colleagues; An evaluation of a study by recognized authorities in a subject area prior to publication
Percent Score
The percent of items that are answered correctly
Percentile
The percent of people in the norming sample whose scores were below a given score; A ranking scale ranging
from a low of 1 to a high of 99 with 50 as the median score; A percentile rank indicates the percentage of a
reference or norm group obtaining scores equal to or less than the test-taker's score; A percentile score does
not refer to the percentage of questions answered correctly, it indicates the test-taker's standing relative to the
norm group standard
Percentile rank
The percentage of examinees in the norm group who scored at or below the raw score for which the percentile
rank was calculated; A number indicating an individual's performance level or score in relation to its standing in
the distribution of scores of a representative group of individuals; A percentile rank of 95 means that the
individual did as well as or better than 95% of the group upon whom the percentile ranks are based; Percentile
ranks divide a distribution into 100 equal parts; Percentile ranks cannot be arithmetically manipulated due to
their varying interval nature
Perception
Use of previous knowledge to gather and interpret the stimuli registered by the senses; Combines aspects of
both the outside world (the visual stimuli) and one’s inner world (prior knowledge)
Performance accountability
A means of judging policies and programs by measuring their outcomes or results against agreed upon
standards; A performance accountability system provides the framework for measuring outcomes - not merely
processes or workloads
Performance appraisal or review

Assessment Glossary
The systematic process of determining the merit, value, and worth of a teacher's current performance and
estimating his/her potential level of performance with further development; Specific evaluation with respect to an
individual's progress in completing specified tasks
Performance auditing
Analysis and evaluation of the effective performance of agencies in carrying out their objectives
Performance based assessment

Direct, systematic observation and rating of student performance of an educational objective, often an ongoing
observation over a period of time, and typically involving the creation of products; May be a continuing
interaction between teacher and student, and should ideally be part of the learning process and a real-world
performance with relevance to the student and learning community; Broad term encompassing many of the
characteristics of both authentic assessment and alternative assessment; Assessment technique involving the
gathering of data though systematic observations of a behavior or process and evaluating that data based on a
clearly articulated set of performance criteria to serve as the basis for evaluative judgments; An observation of
the process of creating an answer or product that demonstrates a student's knowledge and/or skills; A formal
assessment method in which a student's skill in carrying out an activity and producing a product is observed and
judged; The process of using student activities or products, as opposed to tests or surveys, to evaluate students’
knowledge, skills, and development; Directly observable, student-generated evidence of learning (e.g. creating a
object or product, performing a task or behavior, and interpreting or critiquing the products or performances of
others); An assessment activity that requires students to construct a response, create a product, or perform a
demonstration; Since performance-based assessments generally do not yield a single correct answer or require
a particular solution method, evaluations of student performances are based on judgments guided by criteria
using a rubric, or analytic scoring guide to aid in objectivity; Examples include: Reflective journals (daily/weekly);
capstone experiences; demonstrations of student work (e.g. acting in a theatrical production, playing an
instrument, observing a student teaching a lesson); products of student work (e.g. Art students produce
paintings/drawings, Journalism students write newspaper articles, Geography students create maps, Computer
Science students generate computer programs, etc.); Important elements include clear goals or performance
criteria clearly articulated and communicated to the learner, the establishment of a sound sampling that clearly
envisions the scope of an achievement target and the type of learning that is involved, attention to extraneous
interference, and establishment of a clear purpose for the data collected during the assessment before the
assessment is undertaken, while keeping in mind the needs of the groups involved
Performance budget or budgeting

Budget format organized around programs or activities (rather than the objects it purchases), including various
performance measurements that indicate the relationship between work actually done and its cost; A budgeting
system predicated on the relationship between inputs and outputs and aligning goals with key management
activities
Performance criteria or standard

Set standards by which student performance is evaluated which help assessors maintain objectivity and provide
students with a target or goal; Established levels of achievement, quality, or proficiency; Explicit definitions of
what students must do to demonstrate proficiency at a specific level on the content standards which set
expectations about how well students should perform; The observable aspects of a performance or product that
are observed and judged in a performance assessment; Standards which help assessors maintain objectivity
and provide students with important information about expectations by giving them a target or goal to strive for
Performance evaluation
The process of determining the merit, value, and worth, based on assessment results, of some performance
attribute(s) of the teacher being evaluated
Performance goal
Specific statement of what is to be accomplished by an evaluee (e.g. growth in knowledge, development of a
skill, changes in practice), how the goal will be met (e.g. activities, resources), when the goal will be met, and
how achievement of the goal can be assessed or determined

Assessment Glossary
Performance indicators
A set of measures with which postsecondary institutions, systems, states, or sectors evaluate and report their
performance and often used to measure efficiency or effectiveness; Quantitative measures of success used to
determine progress toward achievement of strategic goals
Performance measures
Tools or information used to measure results and ensure accountability; A yardstick or standard used to
measure progress toward achieving a strategic or tactical objective; A test-taking method in which the
participants perform some real-life behavior that is observed by the researcher
Performance standards
The levels of achievement students must reach to receive particular grades or to be allowed to move to the next
unit in a criterion-referenced assessment system; Explicit definitions of what students must do to demonstrate
proficiency at a specific level on the content standards; A statement or description of a set of operational tasks
exemplifying a level of performance associated with a more general content standard; The statement may be
used to guide judgements about the location of a cut score on a score scale; The term often implies a desired
level of performance
Performance task
Assessment activity, or set of activities, related to one or more learning outcomes, that elicits one or more
response to a question or problem; A carefully planned activity that requires learners to address all the
components of performance of a standard in a way that is meaningful and authentic; Performance tasks can be
used for both instructional and assessment purposes
Performance test
Designed to evaluate general intelligence or aptitudes; Consists primarily of motor items or perceptual items
because verbal abilities play a minimal role
Periodic budgeting
A budget developed for the purpose of covering a specific period of time (e.g. annual budget)
Periodicity
The presence of a cyclical pattern in the sampling frame
Persistence
Continual enrollment in pursuit of a credential or credit
Personal growth and change

Aspects of students’ relationship systems, self-systems, cultural, intellectual education, occupational, political,
social, religious values and activities and gender role orientations that are influenced by college attendance and
completion
Personal documents
Anything written, photographed, or recorded for private purposes
Personality
The relatively permanent patterns that characterize and can be used to classify individuals
Personnel evaluation
The systematic determination of the merit, value, and worth of the job-related performance of an employee
Petty cash
Refers to the small quantity of cash typically kept on hand by a company to cover incidental expenses

Assessment Glossary
Phenomenological studies
The examination of human experiences through detailed descriptions of the people being studied; Procedure
involves studying a small number of subjects through extensive and prolonged engagement to develop
patterns/relationships of meaning
Phenomenology
Study of objects and events as they appear to the experiencing observer; The description of one or more
individuals’ consciousness and experience of a phenomenon; A qualitative research approach concerned with
understanding certain group behaviors from that group's point of view; A form of qualitative research in which
the researcher attempts to understand how one or more individuals experience a phenomenon
Physical assets
A term used to describe a company’s or individual’s tangible assets such as property, equipment, furnishings,
inventory and supplies
Physical data
Any material thing created, altered, or left behind by people which might provide information about a
phenomenon of interest to a researcher
Pilot or pilot test

Small scale trial of an assessment instrument to test its validity and usability before the full scale assessment is
run; Small scale trial of a treatment on a sample population to test its effectiveness prior to a full scale
implementation on the parent population; Small scale trial of a plan or program to test the design and
procedures prior to a full scale implementation; A brief, simplified preliminary trial study designed to learn
whether a proposed project, program, or evaluation seems likely to yield valuable results; A large-scale
administration of an assessment, usually with several classes of students if not all students in a program, to
detect any flaws in the assessment before the assessment is considered "done" and is fully implemented
Pilot study
A “scaled down” version of a major effort conducted before a larger study to test feasibility
Pilot testing
A brief, simplified preliminary trial study designed to learn whether a proposed evaluation seems likely to yield
valuable results; A test administered to a representative sample of participants solely for the purpose of
determining the properties of the test; A preliminary try-out of a new or revised assessment or process;
Considers such areas as comprehensiveness and clarity of directions, format of assessment materials,
adequacy of resources or equipment to be used for the assessment, quality of assessor/evaluator training
programs, and timing of assessment tasks
Pipeline analysis
The analysis of student flows into, through, and out of an academic program; Includes analysis of retention
rates, course-taking patterns, and graduation rates; May highlight: sources of student matriculations, time
durations, and unit achievements while students are enrolled, typical paths of student course enrollments,
illustrative trends in student flows over time, and potential targets for improved student flows
Pitfall
A not easily recognized difficulty believed to be associated with a particular standard; Mistakes that would be
made out of ignorance of the import and intent of a standard
Placebo
A dummy treatment; A treatment known by the practitioner to have no effect on the condition or variable being
measured but which the subject is told or believes to be effective

Assessment Glossary
Placebo group
Group in an intervention study which is given a dummy treatment, and (usually) kept unaware of that fact; See
also Control group
Placebo effect
The measurable, observable, or felt improvement in health or behavior not attributable to an administered
medication or invasive treatment; Improvement in a measure recorded for the control group which was given a
treatment known to have no effect on the measure
Placement exams
Instrument to determine if students’ qualifications for entering a program or course are at an appropriate level to
begin their studies
Plan of assistance or assistance plan

A strategy for professional development and growth designed to address a teacher's deficiencies in meeting
designated performance standards, based on the results of an evaluation; The plan of assistance should
indicate goals and objectives for improvement, an action plan for improvement, what staff and resources are
available, the timeline for development activities, benchmarks for ensuring that professional growth is occurring,
and measures for verifying achievement of the goals and objectives
Planning
The development of a design or scheme of arrangement with a definite purpose; Institutional planning includes
the educational programs, the physical plant, budgets and finances, and is intended to accomplish the purposes
of the institution.
Planning horizon
The farthest date that the strategic plan addresses; The target date by which the organization expects to
accomplish its intended outcomes
Planning-programming-budgeting system (PPBS)

Effort to connect planning, systems analysis, and budgeting in a single exercise
Pluralism
A social and political concept stressing the appropriateness of group organization and diversity of groups and
their activities as a means of protecting broad group interests in society; Assumes that groups are good and that
bargaining and competition among them will benefit the public interest
Point estimate
The estimated value of a population parameter
Point estimation
The use of the value of a sample statistic as the estimate of the value of a population parameter
Policy
The principles, plan, or procedures established by an agency, institution, or government, generally with the
intent of reaching a long-term goal; Statement of goals and intentions with respect to a particular problem or set
of problems; A written statement that reflects the employer’s standards and objectives relating to various
employee activities and employment related matters
Policy analysis
Process of researching or analyzing public problems to provide policy makers with specific information about the
range of available policy options, and the advantages and disadvantages of different approaches

Assessment Glossary
Political economy approach

Focusing on politics and economies as categories for analyzing organizational behavior
Politics of the budgetary process

The requirement that administrators act as advocates for their own programs during the appropriation process
by soliciting outside support, protecting their budgetary base, and inching ahead with new programs; A
budgetary system that deals with complex problems by relying upon incremental methods of decision making,
information drawn from past experience rather than analysis, and satisfactory rather than optimal standards of
quality
Poll
Usually the same as opinion poll, but sometimes loosely used to mean any type of informal survey
Population
All the members of a uniquely definable group of people or things; The target group under investigation; A
population is any entire collection of people, animals, plants or things from which we may collect data; The
entire group we are interested in, which we wish to describe or draw conclusions about; Populations are often
too large or expensive to measure directly and “sample” populations must be used to infer the characteristics of
the “parent” population; The entire set of objects you are studying; Each member of a population is called an
individual but need not be a person
Population validity
The ability to generalize the study results to individuals who were not included in the study
Portal or community portal

A Web-based application that provides personalization, single sign-on, and content aggregation from different
sources and hosts the presentation layer of information systems; A group or community access gateway to the
World Wide Web, usually requiring a secure login, which brings together local information and offers services
like email and discussion forums to extend community activity and interaction; Portals integrate various
information sources and services, such as search engines and directories using customized content, layout, and
navigation to suit group or organizational needs; Local needs and interests are central to community portals
Portfolio
A collection of a student's work specifically selected to tell a particular story about the student; An accumulation
of evidence about individual proficiencies, especially in relation to learning standards; A purposeful, integrated
collection of student work showing effort, progress, or achievement in one or more areas; A systematic and
organized collection of a student's work that exhibits to others the direct evidence of a student's efforts,
achievements, and progress over a period of time; A collection of student work to show not only learning
outcomes but also the progress or process of learning; The collection should involve the student in the selection
of its contents; May include information about the performance criteria, the rubric, or criteria for judging merit;
May include evidence of student self-reflection or evaluation; May include representative work, providing a
documentation of the learner's performance and a basis for evaluation of the student's progress: May include a
variety of demonstrations of learning gathered in the form of a physical collection of materials (e.g. videos, CD-
ROMs, reflective journals, etc.); Portfolios may be used not only as a course-level assessment but also at the
program-level to show learning progress throughout a whole program; May be a collection of student work over
time showing change or a collection of student work showing breadth of learning outcomes; Program portfolios
include collections of multiple student work samples usually compiled over time and rated using rubrics; The
design of a portfolio is dependent upon how the scoring results are going to be used; Portfolios can be designed
to assess student progress, effort, and/or achievement, demonstrate the student’s learning process and
conscious involvement in the process, and encourage students to reflect on their learning
Portfolio assessment
Method of assessment that relies on a collection of student- and/or teacher-selected samples of student work or
performance in order to evaluate individual student achievement; A portfolio becomes a portfolio assessment
when the assessment purpose is defined, criteria are made clear for determining what is contained in the

Assessment Glossary
portfolio, and criteria for assessing either the collection or individual pieces of work are identified and used to
make judgments about performance; Commonly assessors work together to establish a consensus of
standards or to ensure greater reliability in evaluation of portfolios
Position classification
Analyzing and organizing jobs on the basis of duties, responsibilities, and the knowledge and skills required to
perform them
Positioning statement
A statement defining a company’s target market, brand reference, need fulfillment, and needed support to
communicate what the company stands for
Post-test or posttest
A test to determine performance after the administration of a program, project, or instructional material; The
measurement of a dependent variable, which occurs after an intervention, usually for the purpose of comparing
to a pretest measure on the same dependent variable
Postsecondary education
Program at an accredited degree-granting institution that leads to improved skills, a certificate of competency, or
an academic degree; College or university education
Positive correlation
The situation when scores on two variables move in the same direction
Positive predictive value (PPV)

A measure of how good a test is at detecting true positives when all its decision positives are considered; A high
PPV indicates that the false positive problem is under control
Positively skewed
Skewed to the right
Positivist research paradigm

Approach which states that only observable and measurable data should be taken into account in research;
Positivism is an approach originally developed in the natural sciences and transferred to social studies; Positivist
social science research uses random samples, operationally defined variables, and statistical analysis;
Positivists attempt to use only data provided by direct observation to reach “positive” facts
Post hoc fallacy

Making the argument that because A preceded B, A must have caused B
Post hoc test

A follow-up test to the analysis of variance
Posttest-only control-group design

Administering a posttest to two randomly assigned groups of participants after one group has been administered
the experimental treatment condition
Posttest-only with nonequivalent groups design

Comparing posttest performance of a group of participants who have been given an experimental treatment
condition with a group that has not been given the experimental treatment condition
Power
The ability of an experimental design or inferential statistic to detect an effect of a variable when one is present;
The likelihood of rejecting the null hypothesis when it is false

Assessment Glossary
Power distance
A cultural value dimension that focuses on the degree to which power differences are accepted in a culture
Power test
A test that samples the range of an examinee's capacity in particular skills or abilities and places minimal
emphasis on time limits; A test intended to measure level of performance without regarding speed of response;
Test in which there is either no time limit or a very generous one; Items are usually arranged in order of
increasing difficulty; A pure power test is sometimes defined as one in which every examinee has sufficient time
to complete the test
Practice
The manner in which evaluations or assessments are actually conducted, whether or not the practice is in
accordance with the policy and/or follows the procedures; What practitioners (e.g. researchers, teachers, office
holders, etc.) actually do
Practice effect
Type of confounding possible when the measure in question is itself a learnable mental or physical skill
Practical significance
A conclusion made when a relationship is strong enough to be of practical importance
Practitioner
An individual who practices a learned profession
Pragmatism
The philosophical position that what works is what is important
Pragmatist philosophy
A philosophy that says to use or do what works
Pre-test or Pretest
A test to determine performance prior to the administration of a program, project, or instructional material; The
measurement of a dependent variable prior to an intervention, usually for the purpose of comparing to a posttest
measurement of the same dependent variable
Pre-test/post-test or pretest-posttest
An assessment technique in which students are given an assessment prior to a treatment or course (the pre-
test) to provide a baseline measurement and are given the same or similar assessment at the end of the
treatment or course (the post-test) to provide an impact measurement; Usually a valid means to show learning
in a course because with prior knowledge established through the pre-test it is possible to show learning took
place in the course itself by comparing the pre- and post-test results
Precision
In survey research, the tightness of the confidence limits
Precision of measurement
A general term that refers to the reliability of a measure, or its sensitivity to measurement error
Predefined versus interactive concept

In content analysis one must determine whether to code only from a pre-defined set of concepts and categories,
or if one will develop some or all of these during the coding process

Assessment Glossary
Prediction
Attempting to predict or forecast a phenomenon
Predictive research
Research focused on predicting the future status of one or more dependent variables based on one or more
independent variables
Predictive validity
Ability of a measure to predict some future behavior; The ability of a score on one test to forecast a student's
probable performance on another test of similar skills; The relationship of a measure to performance in a future
context such as a new work setting or to the results obtained on a future measure assessing a similar or a
different (but presumably related) attribute; A term used to refer to a type of criterion-related validity that applies
when one wishes to infer from a test score an individual's most probable standing on some other variable called
a criterion; Called criterion-related evidence to emphasize that it refers to one type of evidence within a unitary
conception of validity; Refers to evidence based on relations to other variables that include test-criterion
relationships; Predictive evidence indicating how accurately test data can predict criterion scores that are
obtained at a later time; Used in the test validation process to measure the relationship between test scores and
actual job performance
Predictive validity evidence

Validity evidence based on the relationship between test scores collected at one point in time and criterion
scores occurring at a later time
Predictor domain
The construct domain of a construct used as a predictor
Predictor variable
Variable believed to predict the magnitude of a second variable; See Independent variable
Predictors
A set of variables believed to predict the magnitude of a second variable, using a criterion
Premium pricing
High pricing by a company because of the perceived superiority of a product or service or the wish to intimate
superior quality
Preponderance
Emphasis or weight given to data and information on a teacher attribute in an evaluation, after considering the
quantity and frequency of the behavior, the importance of the attribute to the job of teaching, and the potential
impact of the behavior or characteristic on students
Prerequisite knowledge
The prior knowledge that is necessary in order to learn how to solve problems or to acquire new knowledge and
skills
Presence or absence technique

Manipulating the independent variable by presenting one group with the treatment condition and withholding it
from the other group
Presentism
The assumption that the present-day connotations of terms also existed in the past
Pretest

Assessment Glossary
An administration of test items to a representative sample of participants solely for the purpose of determining
the characteristics of the item
Pretest-posttest control group design

A research design that administers a posttest to two randomly assigned groups of participants after both have
been pretested and one of the groups has been administered the experimental treatment condition
Price escalation clause

A contract provision calling for an increase in price should an increase in certain types of costs occur
Primary or direct data

Information related to a single student’s learning; Original data collected as a part of a research study
Primary source
A source in which the creator was a direct witness to, or in some other way directly involved in or related to, the
event being studied
Primary standards
Those standards that apply to the evaluation process and assessment methods rather than to the
learner/teachers being evaluated or to their performance levels, and which should be met or addressed before
the assessments are administered and the evaluation process is implemented
Primary trait analysis

Assignment-specific factors or traits considered in scoring an assignment are stated in a hierarchical scale of
three to five incremental levels of achievement quality, and for each level on the scale there is a specific
statement that describes expected behavior (criterion) at that level; Used to help teachers develop their criteria
for grading
Primary trait method

A type of rubric scoring constructed to assess a specific trait, skill, behavior, or format, or the evaluation of the
primary impact of a learning process on a designated audience
Primary trait scoring

The assignment of scores to one or more designated attributes of each task or performance measure; Scoring
procedure in which products or performances are evaluated by limiting attention to a single criterion; These
criteria are based upon the trait determined to be essential for a successful performance on a given task and
scorers measure performance only on that trait
Principal components analysis (PCA)

A factor analytical method of screening a large number of simultaneous (i.e. multivariate) measures for those
which are best regarded as outcomes of a broader underlying factor because they vary together all or most of
the time; Method to find new variables which have the highest possible variance and better represents the
general differences between individuals than any other linear combination of the original variables
Principle of evidence
The philosophical idea that research provides evidence, not proof
Principle of falsification
Assertion that the scientific method is ultimately based on our ability to prove an assertion is false by finding a
counterexample to it, not to prove one is true
Prior learning assessment

Assessment Glossary
Techniques to assess student understanding and recall of material learned in previous, related courses, so that
faculty can teach accordingly; Helps faculty determine the most effective starting point for a given lesson and
the most appropriate level at which to begin instruction
Priority goals
Most important aims that a teacher or evaluator is trying to accomplish; An important aim or purpose of
instruction
Privacy rights
A learner/teacher's privilege to have his/her performance on an assessment or evaluation results to be
confidential and not disclosed to unauthorized parties without prior permission or consent
Pro forma
A term used to describe the presentation of data, usually financial statements, where the data reflect information
as if the state of the world were different from that which is, in fact, the case (e.g. projected profit and loss
statements given one or more projected or alternative levels of revenue)
Proactive
Acting in advance to deal with an expected difficulty; Behavior which seeks to find opportunities in order to take
advantage of them and to anticipate threats in order to neutralize them
Probabilistic
Stating what is likely to occur, not necessarily what will occur
Probabilistic cause
A cause that usually produces an outcome such that changes in variable A tend to produce changes in variable
B
Probability
The chance that a phenomenon will occur randomly; A term used in statistics that depicts the likelihood of some
event occurring; See p-value
Probability proportional to size

A type of two-stage cluster sampling where each cluster’s chance of being selected in stage one depends on its
population size
Probability sampling
Every individual in a specified population has a known probability of selection
Probability value
The probability of the result of your research study, or an even more extreme result, assuming that the null
hypothesis is true
Probes
Prompts to obtain response clarity or additional information
Problem-based learning
The uses of simulated anchors, scaffolding, and working on related projects, along with opportunities to engage
in tasks, as a way to facilitate student learning
Problem of induction
Things that happened in the past might not happen in the future

Assessment Glossary
Problem solving skills

Defining the problem, being able to obtain background knowledge, generating possible solutions, identifying and
evaluating constraints, choosing a solution, functioning within a problem solving group, evaluating the process,
and exhibiting problem solving dispositions; Problem solving and Critical thinking skills overlap
Procedures
The directions for implementing all aspects of an assessment or evaluation process in accordance with the rules
and guidelines given in an institution's policy; Specifications of how the assessment or evaluation is to be
conducted, designated timelines, persons responsible, forms to be used, documentation to be provided, the
analysis plan, and the steps to be followed
Process
How inputs are converted to outputs within an educational or other production system through linked activities; A
generalizable method of doing something, generally involving steps or operations which are usually ordered
and/or interdependent; How you decide to accomplish some task; How something is done; A process can be
evaluated as part of an assessment, but the assessment process primarily focuses on products instead of
processes
Process evaluation
Investigates issues regarding the program’s current operations or the implementation of new initiatives;
Usually focuses on what the program does, who does it, and how it is done
Process reengineering
The process of improving business practices or methods by creating and implementing new processes or
making changes to existing processes
Process skills or process assessment

Assessing a student's skills in progressing through a series of actions or operations; Process skills that teachers
seek to assess relate to thinking abilities, applications of procedural knowledge, and interactions with others; For
example: Critical thinking, creative thinking, problem solving, decision making, goal setting, cooperation, relating
to others, leadership, and management
Process standards
Statements that describe skills students should develop to enhance the process of learning; Process standards
are not specific to a particular discipline, but are generic skills that are applicable to any discipline
Process variables, measures or indicators

The manner in which teaching is conducted; Includes instructional strategies, sequencing of curricular content
and skills, behavior management techniques, assessment and monitoring practices, and the use of materials
and equipment; Indicators of instructional good practice that have been shown in research to have a reliable
impact on educational outcomes but often neglected in outcomes assessment; Many faculty have difficulty
relating to outcomes measures but are intuitively intrigued by process measures
Processes
A series of actions or operations leading toward a particular result; In education, the processes of learning,
seasoning of character, creativity at the edges of knowledge and imagination, and responsible advocacy of
values are considered critically important
Proctor
An individual who supervises a written examination or test to maintain a fair and consistent testing environment,
but takes no part in the examination process
Procurement budget

Assessment Glossary
A budget that displays the timing, quantities, and projected costs for each type of raw material required to meet
inventory and production needs
Producer price index (PPI)

An inflation indicator issued by the U.S. Bureau of Labor Statistics used to assess the economy’s wholesale
price levels
Product
The tangible and stable result of a performance or task
Product or project assessment

Assessment of the product which results from a project assigned to an individual or group based on a topic
related to the curriculum
Product differentiation
A marketing strategy that attempts to make clear distinctions between comparable products serving the same
market segment through the use of innovative design, packaging, branding, positioning, etc.
Product life cycle

A marketing concept in which company products or brands (including educational institutions and programs)
follow a series of phases: introduction, growth, maturity and decline
Productivity
The amount of education, research, and training within a university, as a function of resources and the needs of
students, parents, and taxpayers; How much is accomplished given the cost of accomplishing it
Productivity measure
A performance measure that quantifies how much product is produced at what cost, and that measures
economy; A measure of need fulfillment or goal achievement relative to its cost
Professional development
A process designed to improve specific professional competencies or the overall competence of a teacher or
other professional
Proficiency
State of having or demonstrating a high degree of knowledge or skill in a particular area; Sufficient expertise in a
knowledge area or adequate mastery of a skill with regard to a standard; Performing in a given art, skill, or
branch of learning with correctness and facility; Achieving competency on predetermined standard
Proficiency level
The equivalent of a cut score (on a forced-choice assessment) but for a performance/complex assessment; Set
by determining the required performance criteria (such as the required level on a rubric) for a specific grade
level; For example: A proficiency level could be the achievement of all the criteria required for a scoring level, or
it could be a set number of points achieved by combining scores for each feature on a rubric
Profile
A representation of a learner/teacher's performance on a number of attributes, measures, or dimensions that
use the same scale; A graphic compilation of the performance of an individual on a series of assessments; A
graphic representation of an individual's scores (or their relative magnitudes) on several tests (or subtests) that
employ a single standard scale
Profit
The return received on a business enterprise after all of its operating expenses and financial obligations have
been met

Assessment Glossary
Profit and loss statement (P&L)

An official financial statement prepared on a quarterly or annual basis showing earnings, expenses and net
profit; A financial document summarizing company revenue and expenses during a specified period of time; A
company’s net profit is determined from this financial report by subtracting total expenses from total revenue
Profit budget
A set of estimated financial statements and schedules for the following year
Profit center
An enterprise that has a manager who is responsible for profit performance and who controls most of the
factors affecting revenues and costs
Profit margin or margin

Company earnings expressed as a percentage of revenue
Profitability
A common corporate objective that focuses on a company’s or industry’s efficiency at generating income
Progeny quality of life

The tendency for children of college-educated parents to graduate from high school, attend college, and have
higher cognitive development; Also the tendency for daughters of college-educated mothers to avoid unmarried
teen pregnancy
Program
A set of courses which culminates in a degree, certificate, or preparation for degree or certificate; A plan or
system under which action may be taken towards a goal; A set of courses constituting an area of specialization;
A coherent set of learning and developmental objectives
Program accreditation or specialized program accreditation

A process of program, unit, or discipline review where the examination of effectiveness is conducted within the
context, requirements, and standards of a discipline-based accrediting body
Program assessment
Assessment to determine the extent to which students in a departmental program can demonstrate the learning
outcomes for the program
Program budgeting
A budgeting method that deals mainly with broad planning and the associated costs of functions, activities,
programs and services
Program completion level

A measure of progress, expressed as a percentage, toward completion of a program of studies; Derived by
dividing a student's total number of formal award units previously earned and allowed by the institution, by the
total number of units needed for completion of his/her program
Program evaluations
Evaluations that assess ongoing activities which provide services
Program improvement
Increasing the likelihood of achieving program objectives by using information acquired through assessment
Program review or academic program review

Assessment Glossary
The process of evaluating the quality and effectiveness of a program; A process of self-study, evaluation, and
planning completed by each academic and service area program on a regular cyclical basis; A systematic
process of data collection, analysis, and interpretation for effective planning and accreditation review; An
analysis of a program’s performance with respect to particular indicators, including student learning outcomes;
In higher education, a program review could consist of anything from a judgment by experts of the worth of the
curriculum based primarily on observations to a comparison between student performance indicators and
objectives and actual student performance on either standardized or evaluator-constructed measures
Program SLO or student learning outcome

What the student will be able to produce at the end of a program; This is the middle level at which SLOs are
usually assessed; Writing program-level SLOs involves considering the overarching missions of the respective
program, matching these missions to courses in the program, and coming up with a cumulative assessment
which may or may not be the same as a course-level assessment; Program-level SLOs, like core competencies,
may also serve as “folders” or “categories” under which course-level SLOs are assessed; Program-level SLOs
may or may not be directly assessed
Progress measures
Specific performance targets that impact the entire organization, address its strategic goals, are stated in terms
of measurable and verifiable outcomes, and challenge the organization to be more responsive to the
environment to achieve its desired future
Project
A form of complex performance assessment involving several types of activities and products for completion; A
complex assignment involving more than one type of activity and production; Most projects involve planning and
usually end with a report (oral or written) or product
Project budgeting
A budget that is developed around a specific type of project rather than time period
Project evaluation
Evaluations that assess activities that are funded for a defined period of time to perform a specified task;
Examples include a three-day workshop on behavioral objectives, a two-year development effort, or a three-year
career education demonstration
Project logic
A model that arrays the resources, activities and goals of a project to allow the relationships to be clearly viewed
and understood
Projection
A prediction or an estimate of a future occurrence, based on current data or trends
Projective measures
A test-taking method in which the participants provide responses to ambiguous stimuli
Projective techniques
Psychological techniques often used in focus groups where participants are asked to make imaginative
comparisons
Prompt
A verbal statement or question that provides a cue, reminder, or inspiration, or that motivates the
learner/teacher being assessed into action; A statement to which a student responds in an assessment; The
directions that ask the student to undertake a task
Proportional stratified sampling

Assessment Glossary
Type of stratified sampling in which the sample proportions are made to be the same as the population
proportions on the stratification variable
Proprietary information
Information associated with company products, business, or activities, including such items as financial data,
trade secrets, product research and development, product designs, marketing plans or techniques, computer
programs, processes, and know-how that has been clearly identified and communicated by the company as
proprietary, a trade secret, or confidential
Proprietary technology
Technology such as software applications that is unique and legally owned by a single enterprise or individual
Proprietary test or survey

Measurement instrument owned by a company or individual
Propriety
The extent to which an evaluation will be conducted legally, ethically, and with due regard for the welfare of
those involved in the evaluation as well as those affected by its results; The extent to which the evaluation has
been conducted in a manner that evidences uncompromising adherence to the highest principles and ideals
(including professional ethics, civil law, moral code, and contractual agreements)
Prospective study
Another term applied to a panel study
Protocol
The rules and formalities that guide the administration and scoring of an assessment and the implementation of
an evaluation; A record or document of evidence and information relating to an assessment or evaluation
Psychological attributes
Traits that characterize an individual's behavior
Psychometric
Properties of the items and test such as the distribution of item difficulty and discrimination indices
Psychometric analysis
The analysis of the items and test such as the distribution of item, difficulty, and discrimination indices
Psychometrician
A qualified person who analyses the psychometrics of a test or item
Psychomotor domain
The range of locomotor behaviors needed to explore the environment and perform tasks as well as the sensory-
motor activities that are essential to learning and communication
Psychosocial
Of or pertaining to the interaction between social and psychological factors
Public domain
Materials such as literary works, music, films, photographs, art, etc. that are no longer covered under copyright
law and can be published without obtaining permission or paying a fee
Public social benefits

Assessment Glossary
The advantages of postsecondary education that accrue to society in general that are not related to economic,
fiscal, or labor market effects
Public speaking
One-to-many communication in a face-to-face situation
Publish
To release and make public, in hardcopy, electronic, or web-based formats, an assessment to make it widely
available
Published test
A test that is publicly available because it has been copyrighted and published commercially
Purchasing power
The measure of the value of a particular currency expressed in terms of the amount of goods or services that
one unit of currency can purchase
Purpose
The primary reason or intended use that provides direction for the design, interpretation, and use of an
assessment and evaluation system; The objectives of an evaluation and the intended use of its reports
Purpose of a research study

A statement of the researcher’s intent or objective of the study
Push polling
A form of pseudo-research intended to change opinions rather than measure them, often by asking leading
questions
Purposive sampling
The researcher specifies the characteristics of the population of interest and locates individuals with those
characteristics

Assessment Glossary
Q
Qualitative assessment
Collection of data that does not lend itself to quantitative methods but rather to interpretive criteria; Methods that
rely on descriptions rather than numbers: For example: Ethnographic field studies, logs, journals, participant
observation, and open-ended questions on interviews and surveys
Qualitative evidence, data, or information

Data relating to, or involving quality or kind, which cannot be represented numerically; Data or evidence that is
narrative or descriptive in form; Information presented and/or summarized in narrative form such as written
expressions descriptive of a behavior or product; Facts and claims presented in narrative, not numerical, form;
The facts and evidence that describe a learner/teacher's performance and that typically are recorded in written,
audio, or visual form; Qualitative evidence usually involves fewer cases than quantitative data, but shows much
more specific information and may be very subjective; For example: Portfolios of work, narrative description and
evaluation of a performance, learner description and analysis of a learning experience
Qualitative methods
Ways to dig deeply into a question, usually related to words rather than numbers, and with evolving theories
using small sample sizes; Methods that rely on descriptions rather than numbers (e.g. ethnographic field
studies, logs, journals, participant observation, and open-ended questions on interviews and surveys); The
metrics used to measure value through the use of means other than numerical factors to quantify
accomplishments, progress, shortfalls, or specific results
Qualitative observation
Observing all potentially relevant phenomena
Qualitative research
Empirical research in which the researcher explores relationships using textual, rather than quantitative data;
Case studies, observation, and ethnography are considered forms of qualitative research; Results of qualitative
research are usually not considered generalizable, but are often transferable; Data typically in narrative form;
Data gathered by use of observations and interviews; Results are usually contextual or unique to an individual
and setting; Research in which questions are open-ended and results are expressed in non-numerical terms;
Research that produces findings not arrived at by means of statistical procedures or other means of
quantification; Qualitative data can be collected through interviews, focus groups, observation, or review of
documents
Qualitative research question

An interrogative sentence that asks a question about some process, issue, or phenomenon to be explored
Qualitative or nominal variable

A variable whose values can be from some finite set of different possibilities
Qualitizing
Converting quantitative data into qualitative data
Quality
The degree to which a service meets the needs of the constituent; The degree to which an institution of higher
education is fitted to its purpose, satisfactory to the clients, and of a quality grade equivalent to other institutions
of higher education
Quality assurance

Assessment Glossary
Activities or programs the purpose of which is to demonstrate and ensure that products and services meet
specifications and are consistently of high quality
Quality audit
The process of examining the elements of a quality management system in order to evaluate how well they
comply with quality system specifications
Quality check
The process of verifying both the accuracy of specific data and information, and the appropriateness of the
techniques used to collect, score or rate, and to analyze the data and information
Quality control
Activities or programs whose purpose is to ensure that all quality specifications for products or services are
being met and are of consistently high quality
Quality improvement
Any system or process designed to enhance an organization’s ability to meet quality requirements
Quality program indicator

A variable reflecting effective and efficient program performance
Quantitative assessment
Collection of data that can be analyzed using quantitative methods (methods that rely on numerical scores or
ratings)
Quantitative evidence, data, or information

Data or evidence that is numerical in form; Data which can be represented numerically; Information presented
and/or summarized in numerical form; Facts and claims that are represented by numbers; The facts and
evidence that describe a teacher's performance and that typically are recorded in numeric, statistical, or graphic
form, or can be meaningfully represented by numbers; Quantitative evidence usually involves a great number of
cases and is used to show general patterns and trends rather than specifics and tends to be much more
objective than qualitative evidence
Quantitative methods
Ways to collection great amounts of information at a more shallow level, usually related to numbers over words,
testing of assumptions, and aggregate information with large sample sizes; Methods that rely on numerical
scores or ratings
Quantitative observation
Standardized observation
Quantitative research
Empirical research in which the researcher explores relationships using numeric data; Results can often be
generalized, though this is not always the case; Data is usually numerical and gathered by quantifying
observations through administering tests or other instruments; Results are usually generalizable or an attempt to
find laws and generalizations; Research that produces data that can be statistically analyzed and whose results
can be expressed numerically
Quantitative research question

An interrogative sentence that asks a question about the relationship that exists between two or more variables
Quantitative skills

Assessment Glossary
The ability to apply arithmetic operations, either alone or sequentially, using numbers embedded in printed
materials to solve a problem
Quantitative theory
A theory in which terms are expressed numerically rather than verbally; Quantitative theory states that reality is
objective, singular, and apart from the observer and that research is a deductive process with context-free
cause and effect relationships; Generalizations from these relationships lead to prediction, explanation, and
understanding
Quantitative or interval variable

A variable whose values are numbers; A variable that varies in degree or amount
Quantitizing
Converting qualitative data into quantitative data
Quartile
The 3 points on a percentile scale which divide the scale into equal quarters; One of three percentiles in a
frequency distribution used to define one of four equivalent components; The breakdown of an aggregate of
percentile rankings into four categories: The 1st quartile is the 25th percentile, the 2nd quartile is the 50th
percentile or median, and the 3rd quartile is the 75th percentile
Quasi-experiment or quasi-experimental research design

An experimental research design that does not provide for full control of potential confounding variables
primarily because it does not randomly assign participants to comparison groups; Similar to true experiments
but uses nonrandomized groups; Incorporates interpretation and transferability in order to compensate for lack
of control of variables; An investigation that has all the elements of an experiment, except that subjects are not
randomly assigned to groups; Common in field research and program evaluations where, unlike true
experimental designs, research subjects cannot be randomly assigned to their respective grouping in the study
(for actual or ethical reasons)
Questionnaire
Written documents that contain a series of questions that are answered by respondents; One or more questions
presented and answered together; A list of questions concerning a particular topic administered to a group of
individuals to obtain information concerning their preferences, beliefs, interests, and behavior; An instrument
consisting of a series of queries and statements that is used to collect data and information from a
learner/teacher; A self-report data collection instrument filled out by research participants
Quick ratio
The measure of a company’s liquid assets to current debts used to determine the company’s ability to meet its
financial obligations; Calculated by subtracting inventories from current assets and then dividing the results by
current liabilities
Quintile
The breakdown of an aggregate of percentile rankings into five categories: the 0-20th percentile, 21-40th
percentile, 41-60th percentile, 61-80th percentile, and 81-100th percentile
Quixotic reliability
Situation where a single manner of observation consistently, yet erroneously, yields the same result
Quota sampling
The researcher determines the appropriate sample sizes or a quota for the groups identified as important and
takes convenience samples from those groups

Assessment Glossary
R
Random
Affected by chance
Random assignment
A procedure that makes assignments to conditions on the basis of chance and in this way maximizes the
probability that the comparison groups will be equated on all extraneous variables
Random error
Any unsystematic error; A quantity (often observed indirectly) that appears to have no relationship to any other
variable
Random sampling or sample

In research and statistical studies, a random sample is one where the researcher ensures that each member of
that population has an equal chance of being selected; Process used in research to draw a sample of a
population strictly by chance, yielding no discernible pattern beyond chance; Drawing a number of individuals
from a larger group or population, so that all individuals in the population have the same chance of being
selected; Drawing a number of items of any sort from a larger group or population so that every individual item
has a specified probability to be chosen; Random sampling can be accomplished by first numbering the
population, then selecting the sample according to a table of random numbers or using a random-number
computer generator; The sample is said to be random because there is no regular or discernible pattern or
order; Random sample selection is used under the assumption that sufficiently large samples assigned
randomly will exhibit a distribution comparable to that of the population from which the sample is drawn; Sample
selection according to a random process, with the selection of each entity in no way dependent on the selection
of other entities
Random selection
Randomly selecting a group of people from a population
Randomization
Used to allocate subjects to experimental and control groups; The subjects are initially considered not unequal
because they were randomly selected
Randomized controlled trial (RCT)

Robust and well-tried research design for evidence-based practice where participants are drawn at random from
the largest practicable pool (i.e. they are not pre-selected subjects who are in some way likely to fit the
treatment being evaluated), and includes control groups who do not receive the treatment in question but who in
every other respect are treated identically to the treatment group (to insure against making Type I errors should
a variable other than the treatment be responsible for changes in the treatment group); RCTs are also expected
to be blind or double-blind where necessary, to generally avoid bias and confounding, to do their best to design
out practice effects, order effects, fatigue effects, and ceiling effects, and to maximize (or at least quantify) the
many subtypes of research validity and reliability
Randomizing
A method of picking assessment items and presenting them in no particular order to reduce the likelihood of
cheating
Range
The extent of variation in the measurement of a variable; A statistical variation measure based on the distance
between the highest and lowest score in a series of scores; Difference between the smallest and the largest

Assessment Glossary
value of a measure; Difference between the highest and lowest scores in a distribution (e.g. if the highest score
on a test is 95 and the lowest score is 45, the range is 50); Sometimes expressed as a formula (e.g. 95-45) or
phrase (e.g., 95 to 45)
Rank order
A rating method where the performance of a group, process or product is arranged in a particular order, such as
highest to lowest
Ranking
The process of ordering or arraying from highest to the lowest; The ordering of responses into ascending or
descending order; A learner/teacher's level of performance is compared to others rather than being judged
independently of how others perform
Ranked or ordinal variable

Variables that come in a definite order, so that you might make them into numbers (e.g. Excellent = 4, Good = 3,
Fair = 2, Poor = 1), but it would not make sense to do arithmetic with them (e.g. Fair + Poor = Good)
Rate
To assign judgments or estimations to the magnitude of degree of some aspect of learning/teaching behavior or
performance; The percentage of people in a group who have a specific characteristic
Rate of return
The return rate earned on an investment on an annual basis, expressed as a percentage
Rater
A person who evaluates or judges student performance on an assessment against specific criteria
Rater drift
The tendency for assessors and evaluators to unintentionally redefine criteria and standards over time or across
a series of ratings
Rater effect
The tendency of an assessor or an evaluator to rate a performance at a level that does not accurately or
consistently reflect the performance level of that learner/teacher; There are several types of rater effect, all of
which are possible sources of systematic measurement error
Rater training
Process of educating raters to evaluate student work and produce dependable or reliable scores; Typically uses
anchors to acquaint raters with criteria and scoring rubrics, open discussions between raters and the trainer help
to clarify scoring criteria and performance standards, and opportunities for raters to practice applying the rubric
to student work; Often includes an assessment of rater reliability that raters must pass in order to score actual
student work
Rating
A systematic estimation of the magnitude or degree of some attribute of learning/teaching, using a numerical or
descriptive continuum
Rating scale
A scale based on descriptive words or phrases that indicate performance levels; Scale in which qualities of a
performance are described (e.g. advanced, intermediate, novice) in order to designate a level of achievement;
May be used with rubrics or descriptions of each level of performance; Subjective assessments are made on
predetermined criteria for documenting where learners fall on a continuum of proficiency; A continuum of
response choices; Subjective assessments made on predetermined criteria in the form of a scale; Rating scales

Assessment Glossary
include numerical scales or descriptive scales; Forced choice rating scales require that the rater determine
whether an individual demonstrates more of one trait than another
Ratio analysis
Use of ratios to measure a company’s financial performance by conveying the relationships between an
organization’s accounting numbers and their trends over time; Used to establish values and evaluate risks
Ratio scale
A scale of measurement that has a true zero point as well as the characteristics of the nominal (labeling), ordinal
(rank ordering), and interval (equal distance) scales
Rational model of decision making

Derived from economic theories of how to make the "best" decisions; Involves efforts to move toward
consciously-held goals in a way that requires the smallest input of scarce resources; Assumes the ability to
separate ends from means, rank all alternatives, gather all possible data, and objectively weigh alternatives;
Stresses economic rationality in the process of reaching decisions
Rationale
Written statements providing the reasons for steps taken and choices made
Rationalism
The philosophical idea that reason is the primary source of knowledge
Raw score
The measure prior to scaling; A score obtained from a test, assessment, observation, or survey that has not
been converted to another type of score such as a standard score, percentile rate, ranking, or grade; The
number of items that are answered correctly out of the total possible; By itself, a raw score provides little useful
information about an individuals' performance; Examples of raw scores include a count of the number of correct
answers on a vocabulary text, a tabulation of the occurrence of a certain type of event during an observation, or
an initial rating on a portfolio document
Reactence
Type of possible confounding where stubbornness on the part of subjects will result in responses to the
research instrument that are deliberately opposite to those which might otherwise have been made
Reactivity
An alteration in performance that occurs in people as a result of being aware of participating in a study
Readiness test
A test of ability to engage in a new type of specific learning; Level of maturity, previous experience, and
emotional and mental set are important determinants of readiness
Reading skills
Locating, understanding, and interpreting written information in documents such as manuals, graphs, and
schedules
Repeated measures design

Type of experimental design where the scores from two or more different groups of subjects are measured on
two or more occasions and compared using a test for the difference of means (e.g. in testing the hypothesis that
sober subjects are better at mathematics than drunk ones, a repeated measures design would test the same
subjects on different occasions when they are drunk and when they are sober)
Reasonableness

Assessment Glossary
A characteristic of scoring criteria ensuring that the judging of student work is appropriate for the conditions
within which the task was completed
Reasoning
The process by which one is motivated to, and looks for, evidence to support and refute a statement or
proposition
Reception analysis
A type of audience research that focuses on what audiences perceive in the media - as opposed to what
broadcasters think they produce
Recommendations
A set of suggestions derived from the learner/teacher evaluation results; For formative teacher evaluation, they
may include a list of professional development activities and a plan of assistance; For summative teacher
evaluation, they may consist of personnel actions such as tenure, dismissal/termination, reassignment/transfer,
contract renewal, or promotion
Reconciliation
The process of confirming that “checkbook” balances correspond to bank statements
Record
The written or taped data, evidence, judgments, notes, recommendations, and other statements for use in the
teacher evaluation process; To register and store data and other information
Reduction in force (RIF)

Layoffs of employees necessitated by reductions in budgets or decreases in student enrollment; RIF decisions
are frequently based on seniority rather than level of performance and staffing needs; An involuntary separation
of an employee or groups of employees due to economic pressures, lack of work, organizational changes, or
other reasons of business necessity that require a reduction in staff
Redundancy payment
Refers to an amount paid to an employee who has been laid off, typically calculated in relation to length of
employment
Reengineering
The redesigning of business and work processes, policies, or organizational structure
Reference group
The norm group that is used to determine the percentile ranks
Reference population
The population of test takers represented by test norms; The sample on which test norms are based must
permit accurate estimation of the test score distribution for the reference population; Reference populations may
be defined in terms of examinee age, grade, or clinical status at the time of testing, or other characteristics
Reflection
The process by which a learner/teacher reviews his/her past performance as a means of improving future
performance
Reflective essays
Generally brief (five to ten minute) essays on topics related to identified student learning objectives, although
they may be longer when assigned as homework; Students are asked to reflect in writing on a selected issue,
and content analysis is used to analyze the results

Assessment Glossary
Reflective practice
A mode that integrates or links thought and action with reflection; Involves thinking about and critically analyzing
one's actions with the goal of improving one's professional practice; Requires individuals to assume the
perspective of an external observer in order to identify the assumptions and feelings underlying their practice
and then to speculate about how these assumptions and feelings affect practice
Reflexivity
Self-reflection by the researcher on his or her biases and predispositions
Regression analysis
A set of statistical procedures used to explain or predict the values of a dependent variable on the basis of the
values of one or more independent variables
Regression artifacts, effect, or regression to the mean

The tendency of very high scores to become lower and very low scores to become higher on posttesting of other
or the original measure; The tendency of examinees scoring above or below the mean of a distribution on a
pretest to score closer to the mean on the posttest; A statistical phenomenon that occurs whenever you have a
nonrandom sample from a population and two measures that are imperfectly correlated, resulting in the
sample's posttest mean being closer to the posttest population mean than their pretest mean was to the pretest
population mean; A statistical phenomenon that can make natural variation in repeated data look like real
change
Regression coefficient
A multiplier of an independent variable in a linear equation relating a dependent variable to a set of independent
variables; The coefficient is said to be standardized or under-standardized accordingly as the variable it
multiples has been scaled to a standard deviation of 1.0 or has some other standard deviation; The predicted
change in Y given a one unit change in X; When the regression line is linear (y = ax + b) the regression
coefficient is the constant (a) that represents the rate of change of one variable (y) as a function of changes in
the other (x); The slope of the regression line
Regression-discontinuity design
A design that assesses the effect of a treatment condition by looking for a discontinuity in regression lines
between individuals who score lower and higher than some predetermined cutoff score
Regression effect
See Regression artifacts
Regression equation
The equation that defines the regression line
Regulation
A detailed set of orders that clarify and implement a particular section of law; Government activity designed to
monitor and guide private economic competition; Specific actions (characterized as economic regulation) have
included placing limits on producers' prices and practices, and promoting commerce through grants or
subsidies; Other actions (termed social regulation) have included regulating conditions under which goods and
services are produced and attempting to minimize product hazards and risks to consumers
Regulatory policy
Policy designed to limit the actions of persons or groups in order to protect all or parts of the general public
Relative decision
Decisions based on the use of the standing of individuals relative to one another
Reinforce

Assessment Glossary
To strengthen a learned way of behaving by some response-contingent external or internal influence
Relative score interpretations

The meaning of the score for an individual, or the average score for a definable group, derived from the rank of
the score or average within one or more reference distributions of scores
Relevance
The extent to which the domains and indicators covered by a teacher evaluation system apply to a teacher's
professional functions in terms of both importance and frequency
Reliability
The consistency or stability of test scores; Degree to which an assessment yields dependable and consistent
results; The measure of consistency for an assessment instrument; Instruments should yield similar results over
time with similar populations in similar circumstances; The degree to which test scores for a group of test takers
are consistent over repeated applications of a measurement procedure and hence are inferred to be
dependable, and repeatable for an individual test taker; The degree to which scores are free of errors of
measurement for a given group; A measure of how consistent the results obtained in an assessment are in a
norm-referenced evaluation situation; Consistency of a student's ranking within the group of students against
which the student is being compared; The degree to which an assessment or instrument consistently measures
an attribute; Extent to which an experiment, test, or any measuring procedure yields the same result on
repeated trials; Degree to which the results of an assessment are dependable and consistently measure
particular student knowledge and/or skills; When the same results are obtained over time, the way of collecting
the results are considered reliable; For example: Repeated measurements show the same achievement or
several observers of a classroom situation closely agree with ratings recorded for individuals on the same
criterion; There are several types of reliabilities, including Intra-Rater (the degree to which the measure yields
consistent results over different administrations with the same teacher performing at the same level by the same
assessor), Inter-Rater (the degree to which the measure yields similar results for the same teacher at the same
time with more than one assessor), Internal Consistency (the degree to which individual observations or items
consistently measure the same attribute), and Test-Retest (the degree to which the measure produces
consistent results over several administrations assessing the same attribute of a teacher)
Reliability coefficient
A unit-free index that reflects the degree to which scores are free of measurement error; Sometimes a product-
moment correlation; In classical test theory, the term represents the ratio of true score variance to observed
score variance for a particular examinee population; A calculated number whose value must be between 0 and
1 which describes the consistency of the assessment results such that the larger the number's magnitude, the
more consistent the assessment (e.g. if the coefficient value were 1, all students' scores would be expected to
rank exactly the same way on retesting); A coefficient of correlation between two administrations of a test; The
conditions of testing administration may involve variations in test forms, raters or scorers, or the passage of time
that give rise to qualifying adjectives being used to describe the particular coefficient (e.g. parallel form
reliability, rater reliability, and test-retest reliability); See Reliability estimates
Reliability estimates
There are four general classes of reliability estimates, each of which estimates reliability in a different way:
Inter-rater or Inter-observer reliability (used to assess the degree to which different raters or observers give
consistent estimates of the same phenomenon), Test-Retest reliability (used to assess the consistency of a
measure from one time to another), Parallel forms reliability (used to assess the consistency of the results of
two tests constructed in the same way from the same content domain), and Internal consistency or Item
reliability (used to assess the consistency of results across items within a test); see also Reliability coefficient
Reliable measures
A measure that provides consistent indications of the characteristics being measured; Measures that produce
consistent responses over time
Remediation

Assessment Glossary
Those techniques or strategies designed to improve a learner/teacher's performance in general deficiencies or

specific areas of weakness
Repeated measures
A single sample of individuals is tested more than once on the dependent (outcome) variable; The same
subjects are used for every treatment
Repeated-measures design
A design in which all participants participate in all experimental treatment conditions
Repeated sampling
Drawing many or all possible samples from a population
Replicable
An attribute of an assessment, observation system, or evaluation indicating that the process used to obtain the
data and evidence is explicit and can be repeated
Replicate
To repeat an evaluation with all essentials unchanged
Replication
Research examining the same variables with different people and, often, in slightly different ways
Replication logic
The idea that the more times a research finding is shown to be true with different sets of people, the more
confidence we can place in the finding and in generalizing beyond the original participants
Report card
Summary of student achievement, either formative or summative, describing student progress and development
with respect to learner expectations, cognitive ability, and expected behavior
Reporting
The process of communicating results and recommendations to the designated individuals or groups
Reporting scheme
A record of evidence describing a learner/teacher's performance; There are two major types of reporting
schemes: Complex (a record of evidence describing performance that provides detailed information about the
performance and the context) and Reductive (a record of evidence describing performance that simplifies the
data collection through classifying, coding, or analyzing them)
Representative sample
A sample that resembles the population; A method of dividing a population into strata and then taking a random
sampling from each subpopulation
Representativeness
A factor of performance tasks and of scoring criteria ensuring that the task and criteria focus on the significant
elements, concepts, and strategies in the outcome(s) assessed
Request for price (RFP)

A request made by a private company or government agency to potential vendors or contractors regarding a
price quote for specified products or services
Request for proposal (RFP)

Assessment Glossary
A formal invitation containing a scope of work which seeks a formal response (proposal) describing both
methodology and compensation to form the basis of a contract; A document an organization sends to a vendor
inviting the vendor to submit a bid for a product or service
Requisition
A written request to purchase a good or service
Research
Studying, developing, or testing a theory or phenomena by gathering data in a systematic way; The principle
method for acquiring knowledge and uncovering the causes for behavior
Research based teacher evaluation (RBTE)

Teacher evaluation approach based on "empirically-validated" criteria or indicators of competence derived from
research studies of effective teaching practices
Research design
The outline, plan, or strategy used to answer a research question; Science and art of planning procedures for
conducting studies so as to get the most valid findings; Set of instructions for gathering and interpreting
evidence
Research ethics
A set of principles to guide and assist researchers in deciding which goals are most important and in reconciling
conflicting values
Research hypothesis
The hypothesis of interest to the researcher and the one he or she would like to see supported by the study
results
Research in practice
Research in practice includes reading and responding to research, reflecting on practice in light of research,
applying research findings to practice, and doing research about practice; It includes a range of ways that
practitioners might engage in research, but also explicitly identifies reading and reflection as important ways for
practitioners to engage with research
Research literature
Set of published research studies on a particular topic
Research method
Overall research design and strategy
Research paradigm or approach

A perspective based on a set of assumptions, concepts, and values that are held by a community of
researchers; The collective set of attitudes, values, beliefs, procedures, and techniques that create a framework
of understanding through which theoretical explanations are formed
Research plan
The outline or plan that will be used in conducting the research study
Research problem
An education issue or problem within a broad topic area; An issue that lends itself to systematic investigation
through research
Research proposal

Assessment Glossary
A written document that summarizes the prior literature, identifies the research topic area and the research
questions to be answered, and specifies the procedure that will be followed in obtaining an answer to these
research questions
Research purpose
See Purpose of a research study
Research question
A clear statement in the form of a question about the specific issue that a researcher wishes to answer in order
to address a research problem; Questions that ask what variables can and will be manipulated and studied; See
also Quantitative research question and Qualitative research question
Research topic
The broad subject matter area to be investigated
Researcher-as-detective
Metaphor applied to the researcher when searching for cause and effect
Researcher bias
Tendency for a researcher to obtain results consistent with what he/she wants to find because the beliefs,
values, or desires of the researcher are reflected in their choice of methodology or interpretation of findings
Resource requirements
A quantitative description (usually in money) of the capital, equipment, information, labor, and land that
achievement of a tactical objective will require
Respondent
A person that participates in a survey process by answering questions
Response
Result of a trial with a particular treatment
Response bias
The tendency subjects have to produce experimental responses, which are socially desirable, or that they think
the experimenter expects; A participant's tendency to respond in a particular way or style to items on a test that
yields systematic, construct-irrelevant error in test scores
Response rate
The percentage of people in a sample that participate in a research study; In survey research, the actual
percentage of questionnaires completed and returned
Response set
The tendency to respond in a specific direction regardless of content
Responses
The answers to test, interview, or questionnaire items
Responsibility
That which a person is expected and obligated to do and for which he/she is accountable
Responsibility center
An enterprise whose performance is evaluated separately and is held responsible for its contribution to the firm’s
mission and goals; See cost and profit center

Assessment Glossary
Resource allocation
The determination and allotment of resources or assets necessary to carry out strategies and achieve
objectives, within a priority framework
Resource plan
A component of the program definition statement stating how the program will be resourced and specifying what
supporting services, infrastructure, and third party services are required
Resources
A source of supply or support; In a university setting, resources include personnel, raw materials and supplies,
and the services of capital goods
Restriction in range or variability

Reduction in the observed score variance of an examinee sample, compared to the variance of the entire
examinee population, as a consequence of constraints on the process of sampling examinees
Restructuring
Changing an organizational structure in order to make it more efficient and cost effective
Results
Findings from a study; What has been learned by students, departments, or institutions; The consequences and
outcomes of a process or an assessment; May be tangible such as products or scores, or intangible such as
new understandings or changes in behavior; Outcomes or assessment data obtained about student learning or
development; Frequencies obtained from performance indicators
Retention
The continuation of a student's involvement in an academic endeavor until graduation
Retention rate
Percent of students remaining within an institution once matriculating at that institution
Retrenchment
The act of reducing expenditures in order to attain financial stability
Retrospective questions
Questions asking people to recall something from an earlier time
Retrospective research
The researcher starts with the dependent variable and moves backward in time
Return on assets (RA)

An internal measure of a company’s profitability, equal to a fiscal year’s profits divided by its total assets,
expressed as a percentage (net profit divided by total assets)
Return on investment (ROI)

A ratio of the benefit or profit derived from a specific investment compared to the cost of the investment itself
Return on net assets

The ratio, expressed as a percentage, of profit gains or asset increases realized by a company during a financial
year
Revenue

Assessment Glossary
The amount of income generated by a product or service over a period of time
Revenue budget
The estimated dollar value based on a projection of future sales of products or services
Review
To examine again or to look at thoroughly in order to assign a grade, make a judgment, come to a conclusion, or
evaluate
RIF
See Reduction in force
Right-to-know audience
A group of people entitled to be informed about the results of an evaluation or assessment; See Stakeholders
Rightsizing
An approach to reducing staff whereby jobs are prioritized in order to identify and eliminate unnecessary work;
This method uses a selection criteria based on individual jobs rather than people in order to avoid possibly
laying off the wrong employees; Frequently misused by organizations when layouts are due to a simple
reduction in force (RIF) without any job analysis prioritization
Rigor
Degree to which research methods are scrupulously and meticulously carried out in order to recognize important
influences occurring in an experiment
Risk analysis
The process of trying to determine in advance any potential negative occurrences and resulting impact on
business operations that such occurrences would constitute
Risk avoidance
Strategies and tactics designed and implemented to circumvent or eliminate potential financial and other risks
Risk management
The use of insurance and other strategies in an effort to minimize an organization’s exposure to liability in the
event a loss or injury occurs
Role expectations
Source of two types of possible confounding, where respondents in the role of a "good subject" may try to
respond in the way they think they ought to respond rather than how they would normally respond, and those in
the "bad subject" role may try to respond in any way which is not how they would normally respond
Root cause analysis

A procedure for ascertaining and assessing the causes of operational problems to assist in determining causes
and implementing prevention strategies
Rubric or scoring guide

A scoring scale used to evaluate student work composed of at least two criteria by which student work is to be
judged on a particular task and at least two levels of performance for each criterion; Some of the definitions of
rubric are contradictory; Generally, a rubric is a scoring guide used in subjective assessments; Implies that a
rule defining the criteria of an assessment system is followed in evaluation; May be an explicit description of
performance characteristics corresponding to a point on a rating scale; A scoring rubric makes explicit expected
qualities of performance on a rating scale, or the definition of a single scoring point on a scale; A description of a
specific level of performance within a performance scale; Scoring guidelines written and shared for judging
performance that indicate the qualities by which levels of performance can be differentiated, and that anchor

Assessment Glossary
judgments about the degree of achievement; A kind of holistic or primary trait scoring in which detailed criteria
are delineated and used to discriminate among levels of achievement in assignments, performances, or
products; A generic scoring tool used to evaluate a student’s performance in a given outcome area, consisting
of a fixed measurement scale and a list of criteria that describe the characteristics of products or performances
for each score point on that scale; A brief statement organized in descending order of accomplishment
describing a certain quantity or quality of work, learning, or behavior; Specific sets of criteria that clearly define
for both student and teacher what a range of acceptable and unacceptable performance looks like; Criteria
define descriptors of ability at each level of performance and assign values to each level; Levels referred to are
proficiency levels which describe a continuum from excellent to unacceptable product; A set of primary traits and
guidelines for scoring and evaluating each assessment as agreed upon by a particular faculty group; A rubric
makes explicit and specific statements about the expected qualities of performance at each point in the scale
and for each primary trait or standard being evaluated; Rubrics can be used to help ensure consistency among
raters and increase inter-rater reliability; An objective evaluation tool using specific criteria; A set of categories
that define and describe the important components of the work being completed, critiqued, or assessed; Each
category within a rubric contains a graduation of levels of completion or competence with a score assigned to
each level and a clear description of what criteria need to be met to attain the score at each level; A description
of the criteria which will be used to grade or assess student work or activities and the grading scheme based on
those criteria; Rubrics may be analytic (specifying many traits which will be evaluated, and with the score on
each trait added together to produce the overall grade for the work or activity) or holistic (providing holistic
descriptions of the type of work produced at each level
Rule making
Administrative establishment of general guidelines for application to a class of people or a class of actions at
some future time
Rule of parsimony
Preferring the most simple theory that works

Assessment Glossary
S
Sales budget
A budget that displays a company’s projected sales by month, product type, and geographical area
Salient or salience
A striking point or feature; Something distinctively different or important
Sample
The set of elements taken from a larger population; A subset of cases drawn from a population; A part of a
population; A subset of the entire, total, or parent population of scores; A sample is a group of units selected
from a larger group (the population); A collection of individuals, drawn from the population of interest, generally
to try to understand the population as a whole without having to examine every individual; Usually research is
done with a sample, rather than the entire population, in the hope of drawing valid conclusions about the larger
group or parent population; A selection of a specified number of entries called the sampling unit (participants,
items, etc.) from a larger specified set of possible entities, called the population
Sampling
The process of drawing a sample from a population; A way to obtain information about a large group by
examining a smaller, randomly chosen selection (the sample) of group members; When done correctly, the
results will be representative of the group as a whole; May also refer to the choice of smaller tasks or processes
that will be valid for making inferences about the student's performance in a larger domain; Matrix sampling asks
different groups to take small segments of a test so that the results will reflect the ability of the larger group on a
complete range of tasks
Sampling bias
A bias arising from logically flawed or carelessly executed sampling, resulting in a sample which does not fairly
represent the population in question
Sampling distribution
The theoretical probability distribution of the values of a statistic that results when all possible random samples
of a particular size are drawn from a population; The sampling distribution describes probabilities associated
with a statistic when a random sample is drawn from a population; The sampling distribution is the probability
distribution or probability density function of the statistic; Derivation of the sampling distribution is the first step in
calculating a confidence interval or carrying out a hypothesis test for a parameter
Sampling distribution of the mean

The theoretical probability distribution of the means of all possible random samples of a particular size drawn
from a population
Sampling error
The difference between a sample statistic and a population parameter; The degree to which the results from the
sample deviate from those that would be obtained from the entire population because of random errors in the
selection of respondents and the corresponding reduction in reliability; The deviation between the characteristics
of a sample and a population
Sampling frame
A listing that should include all those in the population to be sampled and exclude all those who are not in the
population; A list of all the elements in a population
Sampling interval

Assessment Glossary
The population size divided by the desired sample size; Symbolized by “k”
Sampling of performance
The selection of an array of learning/teaching performances and settings to be evaluated so that they are wide
enough in range and large enough in numbers to cover the scope of the performances addressed by the
evaluation system in terms of representativeness and comprehensiveness, and to permit valid inferences about
performances to be made
Sampling with replacement

Sampling technique where it is possible for elements to be selected more than once
Sampling without replacement

Sampling technique where it is not possible for elements to be selected more than once
Sanction
A penalty for poor performance; Something given to encourage improvement when performance or service
levels are not satisfactory
Sarbanes-Oxley Act
The Sarbanes-Oxley Act of 2002 was enacted to increase the accountability of organizations to their
stakeholders; Two provisions are of special interest to HR professionals are the whistleblower protection
provision and the 401(k) blackout notice provision
Satisficing
The tendency to select the first option that meets a given need or to select the option that seems to address
most needs rather than the “optimal” solution; Critique of the economics model of profit maximization used in
rational decision making theory which proposes that people in real world decision making situations use a form
of bounded rationality and actually seek solutions which are only sufficiently satisfying (i.e. satisficing); A limited
search through familiar patterns of behavior for a course of action that meets pre-established, minimum
standards of performance, rather than an exhaustive review of all alternatives in search of the optimal solution to
a particular problem; A decision making strategy that attempts to meet criteria for adequacy, rather than to
identify an optimal or maximizing solution; The process of decision making that characterizes most
governmental action
Satisfaction
A student's attitude towards their educational environment and its components
Scalability
The degree to which a computer application or component can be expanded in size, volume, or number of users
served and continue to function properly
Scale
A classification tool or counting system designed to indicate and measure the degree to which an event or
behavior has occurred; The system of numbers, and their units, by which a value is reported on some dimension
of measurement; In testing, scale sometimes refers to the set of items or subtests used in the measurement and
is distinguished from a test in the type of characteristic being measured; A standard of measurement using
equal intervals and allowing progressive classification (e.g. a thermometer or a Likert Scale); A series of
numerical or descriptive ratings on a continuum used to assess or judge specific levels of performance; The
following are the types of scales of measurement: Nominal scale: Naming used to label, classify, or categorize
data (e.g. gender, Social Security Number, number on an athletic jersey, locker number, and an address);
Ordinal scale: Classification function plus observations are ordered but the distance between adjacent values
not necessarily the same (e.g. Olympic medals, finishing place in a race, and a class ranking); Interval scale:
Classification, plus ordered, plus equal intervals between adjacent units (e.g. all test scores are assumed to be
at an interval scale, temperature Fahrenheit, and temperature Centigrade); Ratio scale: All of the above plus

Assessment Glossary
the scale has an absolute and meaningful zero; Most physical measures are at the ratio level of measurement
(e.g. height, weight, distance, time, pressure, and temperature on the Kelvin scale)
Scale scores
Scores on a single scale with intervals of equal size; A score to which raw scores are converted by numerical
transformation (e.g. conversion of raw scores to percentile ranks or standard scores); Units of a single, equal-
interval scale that are applied across levels of a test; The scale can be applied to all groups taking a given test,
regardless of group characteristics or time of year, making it possible to compare scores from different groups of
examinees; Scores appropriate for various statistical purposes since they can be added, subtracted, and
averaged across test levels; Scores based on a scale ranging from 001 to 999; Useful in comparing
performance in one subject area across classes, schools, districts, and other large populations, especially in
monitoring change over time
Scaling
The process of creating a scale or a scaled score; Scaling may enhance test score interpretation by placing
scores from different tests or test forms onto a common scale or by producing scale scores designed to support
criterion-referenced or norm-referenced score interpretations
SCANS (Secretary's Commission on Necessary Skills)

An initiative of the United States Department of Labor that identified the skills workers need to perform in the
world of work and which made recommendations for changes in secondary education to facilitate the
development of these skills; The SCANS report was published in 1991 and identified five competencies
(allocating resources, working with others, using information, understanding systems, and using technology) and
three foundational skill sets (basic skills, thinking skills, and personal qualities)
SCANS competencies
Five essential requirements for students going directly to work and those planning further education: resources,
interpersonal, information, systems, technology
SCANS skills
Three foundational skills sets identified by the United States Department of Labor as needed for effective job
performance today: basic skills, thinking skills, personal qualities
Scatter plot
A graph used to depict the relationship between two quantitative variables
Scenario planning
The process of identifying economic, social, political and technological trends and exploring the implications of
projecting them forward
Schedule
Part of the procedures of an evaluation/assessment; The designated dates and times for various activities
related to an evaluation/assessment, the location of such activities, and the people who will be involved
School profile
A description (graphic, numeric, or variable) of the status of a school with respect to a set of concepts or
variables
Scientific management
A formal theory of organization developed by Frederick Taylor in the early 1900s; Concerned with achieving
efficiency in production, rational work procedures, maximum productivity, and profit; Focuses on management's
responsibilities and on "scientifically" developed work procedures, based on "time and motion" studies
Scope of content

Assessment Glossary
The extent of coverage, by an instrument or process, of all knowledge, skills, abilities, and behaviors to be
measured, in terms of both breadth and depth of coverage
Score
A rating of performance based on a scale or classification; A specific value in a range of possible values
describing the performance of a student; The number of points earned on a measure or the degree of success
on an assessment of teacher attributes or performance; The result obtained by a student on an assessment,
expressed as a number; Scores are usually expressed in numerical terms, but sometimes in descriptive terms
or graphically
Scorer
An assessor who summarizes the results of an assessment for use by an evaluator or decision maker
Scoring
The process of determining the value of a performance on an indicator or criterion; A package of guidelines
intended for people scoring performance assessments; May include instructions for raters, notes on training
raters, rating scales, samples of student work exemplifying various levels of performance
Scoring criteria
Rules for assigning a score or the dimensions of proficiency in performance used to describe a student's
response to a task; May include rating scales, checklists, answer keys, and other scoring tools; In a subjective
assessment situation, a rubric
Scoring dimension
An attribute or facet of behavior or performance in a domain; Usually determined through logical or statistical
analysis, and sometimes reported as part scores
Scoring key
A list of correct answers for selection items or the scoring guide to be followed in scoring or judging responses
to constructed response items
Scoring protocol
The established criteria, including rules, principles, and illustrations, used in scoring responses to individual
items and clusters of items; Usually refers to the scoring procedures for assessment tasks that do not provide
enumerated responses from which test-takers make a choice
Scoring rubric
A set of rules, guidelines, or benchmarks at different levels of performance, or prescribed descriptors for use in
quantifying measures of learner/teacher attributes and performance; A set of related scoring scales used for
judging student work and awarding points to reflect the evaluation of the work; The principles, rules, and
standards used in scoring an examinee performance, product, or constructed response to a test item; May vary
in the degree of judgment entailed, in the number of distinct score levels defined, in the latitude given scorers for
assigning intermediate or fractional score values, and in other ways
Scoring scale
Assessment criteria formatted as levels of quality ranging from poorest to best, used to judge student work on a
single feature, and may combine several traits within a feature; Scoring levels on the scale are assigned points,
each level specifying the characteristics of the quality of content or skills needed to attain the points
Screening or screening test

A fast, efficient measurement for a large population to identify individuals who may deviate in a specified area,
such as the incidence of maladjustment or readiness for academic work; A test that is used to make broad
categorizations of examinees as a first step in selection decisions or diagnostic processes

Assessment Glossary
Scripting
The writing of evidence and notes throughout an assessment activity (e.g. interview, classroom observation)
about what is happening; Does not include making judgments or interpreting the evidence and notes; May be
continuous throughout the activity or intermittent at pre-specified time intervals (e.g. 3 minutes on, 2 minutes off)
Search conference
A qualitative research technique where a large group of people meets to thrash out an issue which is often used
in community planning
Secondary analysis
The re-analysis of already existing quantitative data rather than text or verbal data; Often involves combining
information from multiple databases to examine research questions
Secondary data
Aggregate information related to students, departments, or other groups; Also known as indirect data; Existing
data originally collected or left behind at an earlier time by a different person for a different purpose
Secondary data analysis

A reanalysis of data using the same or other appropriate procedures to verify the accuracy of the results of the
initial analysis or for answering different questions
Secondary source
A source that was created from primary sources, secondary sources, or some combination of the two
Secondary standards
Those standards that apply to the evaluation process and assessment methods rather than to the
learner/teachers being evaluated or to their performance levels; Standards for which it is desirable, but not
crucial or important, that they be met or addressed before the assessments are administered and the evaluation
process is implemented
Segmenting
Dividing data into meaningful analytical units
Selected response item or format

A question or incomplete statement that is followed by answer choices, one of which is the correct or best
answer; Also referred to as multiple-choice; Format which presents alternative responses, from which the
student chooses the correct or preferred answer; An exercise for which examinees must choose a response
from an enumerated set (multiple choice) rather than create their own responses or products (performance
assessment)
Selection bias
Bias that occurs when the method of selecting study participants is skewed, resulting in participants coming
from a particular social or economic group, and not from others
Selection-history effect
Occurs when an event occurring between the pretest and posttest differentially affects the different comparison
groups
Selection-instrumentation effect
May exist if the nature of the dependent variable or the way it is measured varies across the nonequivalent
groups
Selection item
Test item or task to which the students respond by selecting their answers from choices given

Assessment Glossary
Selection-maturation effect
Occurs when different comparison groups experience a different rate of change on a maturation variable
Selection-regression effect
May exist if the two groups are from different populations (e.g. as when the experimental treatment group is
from a population of individuals with low reading scores and the comparison group from a population of
individuals with high reading scores)
Selection response
A response style where the participant selects from a pull-down list
Selective coding
The final stage in grounded theory data analysis
Selective reduction
Central idea of content analysis; Text is reduced to categories consisting of a word, set of words or phrases, on
which the researcher can focus; Specific words or patterns are indicative of the research question and
determine levels of analysis and generalization
Self adaptive
Category of computer-adaptive tests in which the test taker selects to receive a more difficult or easier item
based on feedback regarding their performance on the previously administered item
Self-assessment
An assessment method in which students make and report judgments about their own performance; The
process of evaluating one's own learning; Often includes the ability to judge one's own achievements and
performances, understanding how the product or performance was achieved, understanding why one followed
the process one did, and understanding what might be done to improve the process, product, or performance;
The process of judging one's own teaching performance for the purpose of self-improvement; Instructional self-
assessment techniques include: self-viewing on a videotape, observing and modeling exemplary teachers, filling
out self-rating forms, completing open-ended self-reports, keeping a log, compiling a portfolio, or using self-
study handbooks and materials
Self-concept
A person's judgment of his or her competence or skills in comparison to those of others
Self-completion or self-administered
A questionnaire designed to be filled in by respondents; A survey using a questionnaire designed to be filled in
by respondents
Self-efficacy
Students’ judgment of their own capabilities for a specific learning outcome
Self-esteem
Overall assessment of personal adequacy or worth
Self-evaluation
The process of reviewing one's own behavior and student learning outcomes for the purpose of monitoring and
changing one's own teaching performance
Self-report
Student evaluations of their own learning or abilities; A test-taking method in which the participants check or rate
the degree to which various characteristics are descriptive of themselves

Assessment Glossary
Self-report instrument or measures

A device in which persons make and report judgments about the functioning of their project, program, or
instructional material; Those instruments in which teachers record their own recollections of events, feelings,
judgments, and attitudes
Semi-structured interview
An interview that uses a list of topics to be covered instead of a fixed questionnaire
Semantic differential
Rating scale which contains a series of items with each concept bounded by bipolar adjectives representing
contrasting views of the performance or product; A scaling technique in which participants rate a series of
objects or concepts
Sensitivity
A mathematically derived index of how good a test is at detecting true positives; Instruments with high sensitivity
are called for when false negatives are either expensive or dangerous because they result in missed or delayed
opportunities for improvement, but they may result in a high number of false positives; The awareness and
understanding of other people's feelings, attitudes, social and cultural backgrounds, ethnic traditions and
customs, languages, interests, rights, and needs
Sequence response
A response style where the participant orders a list of objects or text to formulate their response
Sequential time
People with this view regard time as incremental, do one thing at a time, make appointments and generally stick
to schedules
Sequencing effects
Biasing effects that can occur when each participant must participate in each experimental treatment condition
Serial effect
In survey research, a situation where questions may "lead" participant responses through establishing a certain
tone early in the questionnaire; The serial effect may accrue as several questions establish a pattern of
response in the participant, biasing results
Service delivery
The method used to provide services to a client
Service learning
Typically student affairs program related to enhanced social responsibility through community based
environments
Setting
The temporal and physical environment of an event or activity
Shared beliefs
The specific cultural conventions or statements that people who share a culture hold to be true or false
Shared values
The culturally defined standards about what is good or bad and desirable or undesirable
Short-term debt

Assessment Glossary
Debts payable in full within a period of one year or less
Short-term orientation
The cultural perspective that values quick and immediate results, not long-term benefits
Significance
Refers to the likelihood that relationships observed in a sample could be attributed to sampling error alone
Significance level
The cutoff the researcher uses to decide when to reject the null hypothesis; Usually expressed as a p-value;
Also called the alpha level
Significance testing
A commonly used synonym for hypothesis testing; In significance or hypothesis testing, the researcher sets the
significance (alpha) level, analyzes the data to obtain the probability value, and then the researcher compares
the empirical probability value with the preset significance level to determine whether the finding is statistically
significant
Significant difference
A decision that an observed difference between two statistics probably did not occur by chance
Similar-to-me effect
A type of rater effect in which an assessor or an evaluator judges more favorably those people seen as similar
to himself/herself
Simple case
When there is only one independent variable and one dependent variable
Simple case of causal-comparative research

When there is one categorical independent variable and one quantitative dependent variable
Simple case of correlational research

When there is one quantitative independent variable and one quantitative dependent variable
Simple random sample

A sample drawn by a procedure where every member of the population has an equal chance of being selected
Simple random sampling

The term usually used for random sampling without replacement
Simple regression
Regression based on one dependent variable and one independent variable
Simulation
An imitation of a typical task or situation to assess how well a learner/teacher might implement such a task or
perform in an actual situation; Competency-based measure where a person's abilities are measured in a
situation that approximates a "real world" setting; Primarily used when it is impractical to observe a person
performing a task in a real world situation (e.g. on the job)
Simulation study
A study in which the probable effects of alternative solutions to a problem are identified by using symbolic
representations of real activities, situations, or environments

Assessment Glossary
Single-case experimental designs

Designs that use a single participant to investigate the effect of an experimental treatment condition
Single Blind
A study in which either the experimenter(s) or the participants are ignorant of the true purpose of the research;
See also Double blind
Single case study

The researcher explores a single entity or phenomenon ("the case") bounded by time and activity (a program,
event, process, institution, or social group) and collects detailed information by using a variety of data collection
procedures during a sustained period of time
Situational specificity
The extent to which it is appropriate to use an assessment conducted in one setting or context for other settings
or contexts
Skewed
Not symmetrical
Skewed distribution
Any distribution which is not a normal distribution; A distribution that is not symmetrical along the x-axis
Skill or skills
The ability to use knowledge in a practical manner; Observable behaviors that demonstrate levels of
competence (i.e., knowledge, comprehension, application, analysis synthesis, and evaluation)
SLO
See Student learning outcome
Small group
A collection of three or more individuals who interact with one another to achieve a common task or goal
SMART goals, objectives, or outcomes

Specific, Measurable, Achievable, Relevant, and Time bound; Specific, Measurable, Attainable, Realistic, and
Timely; Specific, Measurable, Agreed upon, Realistic, and Time based; Specific, Measurable, Attainable,
Relevant, and Time sensitive; Specific, Measurable, Acceptable, Realistic, and Targeted; Specific, Measurable,
Attainable, Results oriented, and Targeted; Specific, Measurable, Action oriented, Reasonable, and Time
specific; Specific, Measurable, Actionable, Relevant, and Timed; Specific, Measurable, Aggressive but
Attainable, Results oriented, and Time bound; Specific, Measurable, Attainable, Rewarding, and Timed; etc.
Snowball sampling
Each research participant is asked to identify other potential research participants
Social awareness
Students' sense of how they are related to human society as a whole; Implies sensitivity to the welfare of other
members of our society; Students might demonstrate social awareness by, for example, joining a group to clean
up the environment
Social capital
The part of the intangible capital of an organization that relates to personal networks, mentoring relationships,
know-who and corporate culture
Social desirability response set

Assessment Glossary
The tendency to provide answers that are socially desirable
Sociodrama
Dramatization and role playing used to teach an audience about the findings of an evaluation and to illustrate
their potential applications
Solomon four group design

An experimental design using three control groups to avoid the possibility that the pre-test might itself affect the
dependent variable (The traditional experimental group is given the pre-test, the main treatment, and the post-
test in the normal way; the traditional control group is given the pre-test, a control treatment, and the post-test; a
second control group is not pre-tested, but does get the main treatment and the post-test; and a third control
group only gets the post-test); In this design, the differences between the experimental group and the second
control group or those between the first and third control groups are assumed to be caused by the pretesting
Solvency ratio
One of the many ratios used to gauge a company’s ability to meet its long-term financial obligations based on
current assets and liabilities
Sourcing
Information that identifies the source or attribution of the document
Span of control
One of the early principles of administration which states that there is an upper limit to the number of
subordinates any administrator can directly supervise (generally set at twelve|) and advises administrators to
eliminate any violations of this principle by reducing the number of officials reporting to them by either merging
certain offices or stretching out the hierarchy
Spearman-Brown formula
A formula derived within classical test theory that projects the reliability of a shortened or lengthened test from
the reliability of a test of specified length; A statistical formula used for correcting the split-half reliability
coefficient (because of the shortened test length created by splitting the full length test into two equivalent
halves)
Specialist
Defines an individual who has expertise and responsibility within a specific occupation, job function, field of
study, or research
Specific culture
A culture that focuses on interactions for specific purposes and separates work, family, and social roles
Specifications
A delineation of the major attributes of an assessment to be developed, including breadth and depth of content
to be covered, level of difficulty, format of the assessment materials, supplies and equipment needed, level of
complexity, administrative process (e.g. individual or group, location, timing), scoring procedures, and numbers
and types of items and tasks
Specificity
A measure of how good a test is at detecting negatives in a population of condition negatives; High specificity is
called for in tests where false positives are either expensive or dangerous because false positives result in
inappropriate treatments, unnecessary referrals, harder experiences, or failure
Specimen set
A sample set of testing materials that are available from a commercial test publisher; May include a complete
individual test without multiple copies or a copy of the basic test and administration procedures

Assessment Glossary
Speed or speeded test

A test in which performance is measured by the number of tasks performed in a given time; A test in which
performance is measured primarily or exclusively by the time to perform a specified task; A test scored for
accuracy where the test taker works under time pressure; A test is considered a pure speed test if everyone
who reaches an item gets it right but no one has time to finish all items; Thus, score differences depend upon
the number of items attempted
Speededness
A test characteristic, dictated by the test's time limits, which results in a test taker's score being dependent on
the rate at which work is performed as well as the correctness of the responses; The term is not used to
describe tests of speed; Speededness is often an undesirable characteristic
Split-half reliability
A measure of the consistency of the scores obtained from two equivalent halves of the same test; A statistical
procedure by which a random half of the test items is correlated with the other half: Often a more useful
measure than test-retest reliability because the data comes from a single test session and there will be no
practice, illness progression, recovery, mood, or similar effects
Split-halves reliability coefficient

An internal consistency coefficient obtained by using half the items on the test to yield one score and the other
half of the items to yield a second, independent score.; The correlation between the scores on these two half-
tests, adjusted via the Spearman-Brown formula, provides an estimate of the alternate-form reliability of the total
test (The Spearman-Brown formula is a formula derived within classical test theory that projects the reliability of
a shortened or lengthened test from the reliability of a test of specified length); An estimate of reliability
determined by applying the Spearman-Brown formula for m=2 to the correlation between two halves of the same
test (e.g. the odd-numbered items and the even numbered items)
Spontaneous performance assessment

A measure based on observing, without prompting or pre-announcing, what a subject does during non-
assessment activities; Measure based on accidental or unplanned observations which may not be
representative or generalizable
Spurious relationship
When the relationship between two variables is due to one or more third variables
Stability reliability
The agreement of measuring instruments over time; The stability of an instrument is often expressed by the
coefficient of stability which is used to express the degree of the relationship that exists between scores on the
same test (or parallel forms of the same test) observed at two points in time
Staffing metrics
Measures used to determine costs associated with recruitment and hiring, time-to-fill and time-to-start for open
positions, and recruiter workload/activity
Stakeholders
Any person legitimately involved in or affected by the evaluation; Individuals or groups who may affect or be
affected by program evaluation; Those individuals who have a substantial interest in an assessment/evaluation
and in the quality of learning/teaching; Anyone who has a vested interest in the outcome of a program, project,
or plan; The various groups with an interest in the quality, governance, and operation of a program (e.g. the
public, employers, customers, clients, and third party payers); Individuals and groups inside and outside the firm
who have an interest in the actions and decisions of the firm; The many different persons or interest groups that
are involved in a policy decision and are affected by the results; Those people, institutions and interests which

Assessment Glossary
have an interest in, or contribute to, the organization's success or failure in achieving its mission; See also
Right-to-know audience
Standard or standards
A description of the expected level of performance that describes minimum competence in relation to a critical
score or other measure of student performance; A principle commonly agreed to by experts in the conduct and
use of evaluation for the measure of the value or quality of an evaluation; A pre-determined criterion or
expectation of a level of student learning; A level of accomplishment all students are expected to meet or
exceed (e.g. a passing score); Standards do not necessarily imply high quality learning, sometimes the level is a
lowest common denominator; Standards do not imply complete standardization in a program since a common
minimum level could be achieved by multiple pathways and demonstrated in various ways; There are several
categories of standards related to different purposes: Desired performance standards (Specified levels of
performance regarded as accomplished or effective and typically are used for such purposes as promotions,
awards, and certification), Developmental standards (Specified improvement levels to be attained which may
be used for professional development and self-assessment), Legal standards (Specified guidelines and
requirements related to an assessment and evaluation that are specified in the law, governmental policies and
regulations, school district policies, and court decisions), Minimum standards (Specified levels below which
performance is not acceptable and are used for such purposes as licensure and job assignments),
Professional standards (Specified guidelines related to an assessment and evaluation that are specified by
the individuals and associations in the career area affected, directly or indirectly, by the assessment), and
Technical standards (Specified guidelines related to an assessment and evaluation that are specified by
psychometricians, statisticians, test publishers, and specialists in the domain covered by the assessment); See
also Primary and Secondary standards
Standard deviation (SD)

A measure of the variability or dispersion of a distribution of scores; Most widely used measure of dispersion of
a frequency distribution; The square root of the variance; The standard deviation is a calculated number that
describes the extent to which scores are dispersed (spread out) from the mean; Nearly all scores are typically
within 3 standard deviations of the mean; A measure of variation that indicates the typical distance between the
scores of a distribution and the mean; Determined by taking the square root of the average of the squared
deviations in a given distribution; Can be used to indicate the proportion of data within certain ranges of scale
values when the distribution conforms closely to the normal curve; Equal to the positive square root of the
population variance; The more the scores cluster around the mean, the smaller the standard deviation; For a
normal distribution, approximately two thirds (68.3 percent) of the scores are within the range from one standard
deviation below the mean to one standard deviation above the mean (often stated as plus or minus one
standard deviation); Computation is based upon the square of the deviation of each score from the mean;
Sometimes called sigma
Standard error
The standard deviation of a sampling distribution; Statistical estimate of possible size error present in a test
score or other group measure
Standard error of measurement

The standard deviation of an individual's observed scores from repeated administrations of a test (or parallel
forms of a test) under identical conditions; The standard deviation of the distribution of errors of measurement
that is associated with the test scores for a specified group of participants; The difference between an observed
score and the corresponding true score or proficiency; The standard deviation of an individual's observed scores
from repeated administrations of a test or parallel forms of a test, under identical conditions; Because such data
cannot generally be collected, the standard error of measurement is usually estimated from group data
Standard error of the mean

A computed value based on the size of the sample and the standard deviation of the distribution; Indicates the
range within which the mean of the population is likely to be found from the mean of the sample at a given level
of probability

Assessment Glossary
Standard operating procedures (SOP)

A prescribed written procedure outlining how recurring tasks, duties, and functions are to be performed
organization-wide
Standard or standardized score

A score that is expressed as a deviation from a population mean; A score expressing an individual's distance
from the mean in terms of the standard deviation of the distribution; Generally requires the mathematic
conversion of raw scores and a distribution that approximates a normal curve; Scores that have been converted
from one scale to another to have a particular mean and standard deviation; An indicator of the relative standing
of a score within a normal distribution of scores, defined by its mean and standard deviation; By transforming
raw scores to standard scores, the user can interpret intervals between any two score points in relation to a
reference population; Z scores are a commonly used standard score transformation, providing a normal
distribution with a mean of zero and a standard deviation of one; If it is important to compare performance on
different assessments, then the scores from each test or assessment need to be standardized since each
assessment may have a different number of items or rating scale points and each may have been administered
to different groups; Standardizing the scores within each distribution becomes necessary for purposes of
comparing performance on different assessments; By transforming raw scores for each assessment into
standard scores, it becomes possible to determine a learner/teacher's relative strengths and weaknesses within
the groups upon which the scores were standardized
Standard setting
Process by which an expectation of performance level (standard) is determined
Standardization sample
That part of the population that is used in the norming of a test (i.e. the reference population); The sample
should represent the population in essential characteristics, some of which may be geographical location, age,
graduation, or participation in a specific type of program
Standardized or standardization
The use of consistent procedures for administering, scoring, reviewing, interpreting, and reporting the results of
an evaluation; Presenting the same stimulus to all participants; In test administration, maintaining a constant
testing environment, and conducting the test according to detailed rules and specifications, so that testing
conditions are the same for all participants; In statistical analysis, transforming a variable so that its standard
deviation is 1.0 for some specified population or sample; In scoring, ensuring that candidate responses are
judged using predefined criteria in order to provide a consistent basis for evaluating all candidates; A consistent
set of procedures for designing, administering, and scoring a measure; The purpose of standardization is to
assure that all students are evaluated under the same conditions so that their scores or ratings have the same
meaning and are not influenced by differing conditions; Standardized procedures are very important when
scores and ratings will be used to compare individuals or groups; Design and implementation of consistent
specifications for procedures, practices, materials, machinery or other equipment, or other types of products and
services; The process of calibrating a measure to a given population so that the future performance of other
samples can be norm-referenced
Standardized conditions
The administration of an assessment process or instrument to all learner/teachers in the identical manner (e.g.
same instructions and timing, comparable settings, and use of trained assessors and evaluators)
Standardized open-ended interview

A set of open-ended questions are asked in a specific order and exactly as worded
Standardized tests or testing

A test composed of a systematic sampling of behavior, having data on reliability and validity, administered and
scored according to specific instructions, and capable of being interpreted in terms of adequate norms; A test
that is administered and scored under the same conditions for all students; A sample of items or situations with
definite directions for administration and scoring most often accompanied by data on reliability and validity and

Assessment Glossary
sometimes by normative information; Assessment methods, either criterion- or norm-referenced, designed to be

administered, scored, and interpreted in the same way regardless of when and where it is administered; An
objective test that is given and scored in a uniform manner; Tests which are carefully constructed with items
selected after trials for appropriateness and difficulty, issued with a manual giving complete guidelines for
administration and scoring in an attempt to eliminate extraneous interference that might influence test results,
and often with scores that are norm-referenced; A form of measurement that has been normed against a
specific population; Standardization is obtained by administering the test to a given population and then
calculating means, standard deviations, standardized scores, and percentiles, with equivalent scores produced
for comparisons of an individual score to the norm group's performance
Standards
The broadest of a family of terms referring to statements of expectations for student learning, including content
standards, performance standards, Opportunity to Learn (OTL), and benchmarks
Standards-based reform
Program of school improvement involving setting high standards for all students and a process for adapting
instruction and assessment to make sure all students can achieve the standards
Stanine or stanine score

Abbreviation for “standard nine point scale”; A score from a nine-point scale used in standardized tests;
Normalized standard scores with a range of 1 to 9, a mean of five, and a standard deviation of two; One of the
steps in a nine-point scale of standard scores; They are status scores within a particular norm group where the
first stanine is the lowest scoring group and the 9th stanine is the highest scoring group (generally the first
stanine is classified as "poor", the second and third are grouped together as "below average", the fourth through
sixth are classified as "average", the seventh and eighth as "above average", and the ninth as "superior”); The
word stanine draws its name from the fact that it is a STAndard score on a scale of NINE units; In a normal
distribution the nine stanines take up 4%, 7%, 12%, 17%, 20%,17%, 12%, 7%, and 4% respectively of the
distribution
Starting point
A randomly selected number between one and k
State
An unstable cognitive, affective, or psychomotor characteristic possessed in different amounts by the same
person at different times and situations; Distinguishable, but less enduring ways in which individuals vary
Statement of financial position

A financial statement of a company’s incomes and outflows during a certain period of time; A term generally
used by nonprofit organizations as a reference to a balance sheet
Statement of work (SOW)

A detailed pragmatic statement of a company’s needs and requirements on which prospective suppliers base
their bids or proposals to provide products or services
Statistic
A numerical characteristic of a sample; A quantity, property or fact that gives information about a sample; A
summary number typically used to describe a characteristic of a sample and from which inferences about the
population represented by the sample are made; A quantity calculated from a sample of data and used to give
information about unknown values in the corresponding population; For example: the average of the data in a
sample is used to infer information about the overall average in the population from which that sample was
drawn
Statistical analysis

Assessment Glossary
A mathematical process using statistical techniques to tabulate, depict and describe a collection of numeric (or
quantitative) data
Statistical conclusion validity

The ability to infer that the independent and dependent variables are related and the strength of that relationship
Statistical inference
Using information from a sample to draw conclusions or inferences about the population from which the sample
was taken
Statistical significance
A situation or occurrence where a researcher can show through specific tests for significance that it is unlikely
that the obtained results occurred by chance
Statistically significant
The probability of a finding being wrong or random is nil (p-value of .05 or less); A research finding is probably
not attributable to chance; The probability that the association between the factor and the outcome is due to
chance is less than a specified level (usually p < 0.05, 0.01 or 0.001); The claim made when the evidence
suggests an observed result was probably not just due to chance (i.e., there is a real relationship present)
Status report
A description of the implementation of the plan's assessment methods, the findings (evidence) from assessment
methods, how the findings were used in decisions to maintain or improve student learning (academic programs)
or unit outcomes (support units), the results of previous changes to improve outcomes, and the need for
additional information and/or resources to implement an approved assessment plan or gather additional
evidence
Stem
A question or statement followed by a number of choices or alternatives that answer or complete the question or
statement; Most commonly found in multiple-choice questions
Sticky dots
A method of voting used in consensus groups and co-discovery conferences where each participant is given a
fixed number of small round colored self-adhesive labels used to vote for statements on posters by putting a dot
next to the statements they most agree with; Also called dotmocracy
Stop out
When a student leaves school for a term and then returns to school
Straight line depreciation

A depreciation method that produces a regular charge that is proportional to the life expectancy of a fixed asset
Strands
A way of organizing what adult learners need to know and be able to do within core curriculum; Strands need
not be taught sequentially or separately; They identify processes, themes, content, and skills
Strategic
Maneuvering yourself into a favorable position to use your strengths to take advantage of opportunities
Strategic alliances
Two or more companies coming together to enhance business operations
Strategic alternatives

Assessment Glossary
Alternative courses of action that achieve business goals and objectives by using organizational strengths to
take advantage of environmental opportunities
Strategic audit
A checklist of questions that provide an assessment of a firm’s strategic position and performance
Strategic decisions
A series of decisions used to implement a strategy
Strategic factors
Strategic issues expected to have a high probability of occurrence and impact on the firm
Strategic fit
Fit between what the environment wants and what the firm has to offer
Strategic goals
Fundamental issues the university must address; They give direction for accomplishing the mission, contain
specific and meaningful planning challenges, and result from previous analysis; Strategic goals may be SMAR T
goals or desired ends which are not necessarily attainable or quantifiable
Strategic issues
Trends and forces which occur within the firm or the environment surrounding the firm
Strategic management
The act of identifying markets and assembling the resources needed to compete in these markets; The set of
managerial decisions and actions that determine the long-run performance of the firm
Strategic myopia
Management’s failure to recognize the importance of changing external conditions because they are blinded by
their shared, strongly held beliefs
Strategic objectives
Means of achieving, or moving toward, a goal that are measurable and quantifiable; They focus efforts on
demonstrable results and provide broad categories for planning resource allocation; A results-oriented objective
Strategic performance measure

A yardstick or standard used to measure progress toward achieving a strategic objective; A measure of how well
we are doing
Strategic plan
A practical, action-oriented guide based on an examination of internal and external factors that directs goal-
setting and resource allocation to achieve meaningful results over time; A document used to organize the
present on the basis of projections of the desired future
Strategic planning
The process of developing strategies to reach a defined objective; A comprehensive planning process designed
to determine how the firm will achieve its mission, goals, and objectives over the next five, ten, or more years;
Matching organizational objectives and capabilities to the anticipated demands of the environment to produce a
plan of action that will ensure achievement of objectives; A continuous process where people make decisions
about intended future outcomes, how outcomes are to be accomplished, and how success is measured and
evaluated; The process of identifying an organization’s long-term goals and objectives and then determining the
best approach for achieving those goals and objectives
Strategic predisposition
Assessment Glossary
A tendency of a firm by virtue of its history, assets, or culture to favor one strategy over competitive possibilities
Strategic priorities
A ranking of an organization's strategic objectives by importance; A description of what an organization's
decision-makers believe is more important or less important; A basis (criterion) for evaluating the tactical
planning and budget-setting process
Strategic staffing
The practice of hiring smaller core numbers of permanent employees and using temporary employees to fill
more highly specialized positions within the organization
Strategic thinking
Making decisions today which will affect organizational performance years in the future
Strategy or strategies
Broadly stated means of deploying resources to achieve strategic goals; A pattern in a stream of decisions and
actions; Strategies may be dominant (optimal regardless of the action taken by one’s rival), emergent (an
unplanned strategy that emerges from within the organization), intended (a planned strategy developed through
the strategic planning process), and realized (the real strategy of a firm, whether planned or emergent)
Stratification variable
The variable on which the population is divided
Stratified coefficient alpha

A modification of coefficient alpha that renders it appropriate for a multi-factor test by defining the total score as
the composite of scores on single-factor part-tests
Stratified random sample

A grouping achieved by dividing the population to be surveyed into a number of non overlapping classes or
categories which together include all cases, followed by taking cases at random from within the categories; A
set of random samples, each of a specified size, from several different sets, which are viewed as strata of the
population
Stratified sampling
Dividing the population into mutually exclusive groups and then selecting a random sample from each group
Stringency
A type of rater effect in which an assessor or an evaluator tends to rate a learner/teacher too low or to judge the
performance level as poorer than it actually is; Sometimes referred to as "negative leniency”
Structural equation modeling (SEM)

Like path analysis, a method for studying the direct and indirect effects of variables hypothesized as causes of
variables treated as effects; Unlike path analysis, this method allows for causal pathways to be specified
between factors in addition to manifest variables and it also allows for the estimation of measurement and latent
errors; Structural equation modeling is thought to subsume path analysis as an analytic approach; See Causal
modeling
Structured interview
The interviewer asks all respondents the same questions using an interview schedule
Structured observation
The researcher determines at the outset precisely what behaviors are to be observed and typically uses a
standardized checklist to record the frequency with which those behaviors are observed over a specified time
period

Assessment Glossary
Structured performance assessment

A measure based on the administration of an assessment instrument or task at a specified time and in a
designated manner
Student affairs
Division of a higher education institute charged with providing programs and services that assist students in
growing, progressing, or increasing his or her developmental capabilities as a result of enrollment, usually in
out-of-class activities
Student competency
Statement of broad knowledge, skills, attitudes, or behaviors that program majors should be able to demonstrate
upon completion of the degree program
Student development
The ways that a student grows, progresses, or increases his or her developmental capabilities as a result of
enrollment in an institution of higher education
Student evaluation
The process of systematically collecting and interpreting information that can be used to inform students about
the progress they are making toward attaining the knowledge, skills, attitudes, and behaviors to be learned or
acquired; and to inform the various personnel who make educational decisions (instructional, diagnostic,
placement, promotion, graduation) about students
Student evaluation system

All the procedures (including developing and choosing methods for assessment, collecting assessment
information, judging and scoring student performance, summarizing and interpreting results, and reporting
evaluation findings) and the policies that evaluators use to evaluate their students
Student involvement
Student engagement in project, activities, or their own learning
Student learning
The acquisition of knowledge or behavior as a result of participation in programs and services
Student learning outcome (SLO)

Knowledge, skills, abilities, and attitudes that a student has attained at the end (or as a result) of his or her
engagement in a particular set of collegiate experiences; What students are expected to produce by the end of a
course, program, college experience, degree, or certificate program, or set of interactions with student services;
Covers acquisition, retention, application, transfer, and adaptability of knowledge and skills; A general statement
of the measurable skills and abilities a student will possess upon successfully finishing a course of study which
is broader in scope than objectives and focuses on the important general abilities a successful student will take
away from the class, program, or college for use in his/her future life; The measures of student achievement of
knowledge, skills, and other educational outcomes such as improved student attitudes and behaviors that
should have been taught to them by the teacher being evaluated
Student load
The sum of the award units of all courses in which a student is enrolled during a specified time period, as
defined by the institution
Student outcome
The achieved results or the actual consequences of what a student has demonstrated or accomplished; May be
academic or occupational accomplishments, as well as the intellectual, personal, civic development, attitudes,
values, and beliefs that students attain as a result of postsecondary education

Assessment Glossary
Student outcomes assessment

The act of assembling, analyzing, and using both quantitative and qualitative evidence of teaching and learning
outcomes, in order to examine their congruence with stated purposes and educational objectives and to provide
meaningful feedback that will stimulate self-renewal
Student outcomes assessment cycle

The process by which SLOs are identified, measured, and analyzed, and the results used to improve student
learning is a three-step cycle: Identify: Instructors meet and work together in identifying what the students
should produce at the end of a set of learning experiences (be it at the course or program level), SLOs are
drafted, and an assessment plan written which states what the assessment will be (i.e. how the SLOs will be
measured) and what the rubric will be (i.e. which standards will be used to determine attainment of the
outcome); Assess: The semester following the identification of the SLOs and drafting the assessment plan, the
proposed assessment is conducted and the data collected; and Reflect: In the third semester of the cycle, the
data is compiled and the instructors come together again to discuss the results through dialogues which should
include discussions about the meaning of the results and how they can be used to improve student learning.
Student performance level (SPL)

A standard description of an ESOL (English for Speakers of Other Languages) student's language ability at a
given level in terms of speaking, listening, reading, writing, and the ability to communicate with a native speaker;
A profile of skill levels for a student can thus be assigned and used for placement, instructional, or reporting
purposes
Student Satisfaction Inventory (SSI)

A proprietary survey by Noel-Levitz that measures student satisfaction with a wide range of college services
including academics, educational support, and administration; Frequently used as a companion to the
Institutional Priorities Survey that measures satisfaction of faculty and staff in many of the same areas
Students with disabilities (SWD)

A broadly defined group of students with physical and/or mental impairments, such as blindness or learning
disabilities, that might make it more difficult for them to do well on assessments without accommodations or
adaptations
Study
Research study; The principle method for investigating the relationship among variables
Style-based evaluation
A teacher evaluation approach that uses criteria which specify in what way or how teachers are to perform their
job (e.g. announced objectives at the start of each lesson, use of cooperative learning techniques) rather than
what teachers are responsible for doing
Subculture
A culture embedded within a larger culture; A smaller grouping within a culture that has a set of similar
characteristics which distinguishes it from the larger culture
Subject
An individual who is studied
Subjective
A characteristic of an assessment, observation, or conclusion that yields results which cannot be empirically
verified by another person
Subject matter expert (SME)

An individual who has expertise in a business process or specific area

Assessment Glossary
Subjective measures
Tests or measures that are scored or rated with some degree of subjective judgment (e.g. a performance rubric)
Subjective scoring
Different scorers and raters may differ on a student's score or rating; Most often associated with constructed
response assessments
Subjective test
A test in which the impression or opinion of the assessor determines the score or evaluation of learner
performance or achievement; A test in which the answers cannot be known or prescribed in advance
Subsidiary
A company that has more than half of its stock owned by another company or is completely owned by another
company
Succession planning
The process of identifying long-range needs and cultivating a supply of internal talent to meet those future
needs; Used to anticipate the future needs of the organization and assist in finding, assessing, and developing
the human capital necessary to the strategy of the organization
Sufficiency
A judgment on whether an assessment task is comprehensive enough to produce a sample of student work
broad enough in depth, relative to a body of knowledge or skill, to be considered an adequate measure of
whether the student has attained the knowledge or achieved the skill; For forced choice assessments, the
number of items used to decide this is the crucial issue for sufficiency
Summary
A concise report encompassing the major results of an assessment or evaluation
Summary annual report

A summarized report containing information on the financial status of an employee benefits plan
Summative assessment, evaluation, or research

Evaluation focused on determining the overall effectiveness and usefulness of the evaluation object; Evaluation
at the conclusion of a unit or units of instruction or an activity or plan to determine or judge student skills and
knowledge or effectiveness of a plan or activity; Assessment of outcomes which are the culmination of a
teaching/learning process for a unit, subject, or program; An evaluation designed to present conclusions about
the merit or worth of a student's performance; Accountability-oriented assessment; Evaluation designed to
present conclusions about the merit or worth of an object and recommendations about whether it should be
retained, altered, or eliminated; Culminating assessment for a unit, grade level, or course of study providing a
status report on mastery or degree of proficiency according to identified learning outcomes; The use of data
assembled at the end of a particular sequence of activities, to provide a macro view of teaching, learning, and
institutional effectiveness; The gathering of information at the conclusion of a course, program, or
undergraduate career to improve learning for the next cohort of students taking the course or program;
Assessment or evaluation that is used to show learner achievement and is done at the conclusion of a course or
some larger instructional period (e.g. at the end of the program); The purpose is to determine success or to what
extent the program, project, and/or course met its goals; Formative and summative evaluations should be used
to complement each other.
Sunshine law
Statutes granting the public access to government agency meetings and records
Supply and demand

Assessment Glossary
The amount of goods or services available to consumers at a specific price and the level of consumer need or
demand for those goods or services at the specified price
Support unit review

A process employed to examine the operational effectiveness (strengths and weaknesses) and student learning
support of non-instructional units or multi-unit functions; Non-instructional equivalent of program review.
Surrogate task
A substitute or replacement for an actual activity or task, but one that is very similar to the actual task and that
uses the same types of skills, behaviors, materials, and equipment
Survey
Research tool that includes at least one question which is either open-ended or close-ended and employs an
oral or written method for asking these questions; The goal of a survey is to gain specific information about
either a specific group or a representative sample of a particular group; Results are typically used to understand
the attitudes, beliefs, or knowledge of a particular group
Survey instrument
The research tool used to conduct a survey (e.g. a mailed questionnaire, a telephone script, etc.)
Survey research
A term applied to non-experimental research based on questionnaires or interviews
SWOT analysis
An abbreviation used to denote analysis of an organization’s internal Strengths and Weaknesses and external
Opportunities and Threats; Analysis of the strategic factors within an organization (Strengths and Weaknesses)
and within the organization’s environment (Opportunities and Threats) usually summarized in a 2x2 matrix or
table; Also called an Internal/external assessment
Synchronic reliability
The similarity of observations within the same time frame; NOT the similarity of things observed
Synchronous time
People with this view often see time as flexible and intangible with commitments desirable but not absolute
Synthesis
The selection, organization, and analysis of the materials collected; The process of combining data and
information from multiple sources, or of ratings and judgments on separate scoring dimensions in order to arrive
at a conclusion or result; Combining elements into a whole by using an original structure or solving a problem
that requires a combination of several principles sequentially into a novel situation
System
Set of regularized interactions configured or "bounded" in a way that differentiates and separates them from
other actions that constitute the system's environment
Systematic
Any methodical procedure or plan; In higher education, a systematic approach to management implies a careful
analysis of choices about organization, effectiveness, priority, and decision-making in order to allocate scarce
resources
Systematic error
A score component (often observed indirectly), not related to the test performance, that appears to be related to
some salient variable or sub grouping of cases in an analysis; An error that is present every time an instrument
is used

Assessment Glossary
Systematic innovation
The purposeful and organized search for changes and the systematic analysis of the opportunities these
changes might offer for economics and social innovation
Systematic sample
A sample obtained by determining the sampling interval, selecting a random starting point between 1 and k, and
then selecting every kth element
Systemic validity
The negative and positive consequences of testing that need to be monitored in order to evaluate the long-
range value of the test
Systems appraisal
Review of institutional learning over time (generally 3 years)
Systems perspective
Recognition that organizations are complex social systems with dynamically interrelated and interdependent
groups of individuals, structuresm and processes that interact with each other and their environment; Holistic
consideration of the contextual functioning of interacting elements within a system and the relations of the
system to other systems in its environment
Systems theory
A theory of social organization, holding that organizations (like other organisms) behave according to inputs
from their environment, outputs resulting from organizational activity, and feedback leading to further inputs;
Also holds that changes in any one part of a group or organizational system affects all other parts; A framework
by which one can analyze and/or describe any group of objects that work in concert to produce some result;
Interdisciplinary study of the nature of complex systems in nature, society, and science

Assessment Glossary
T
t-score
A derived score on a scale having a mean score of 50 units and a standard deviation of 10 units
t-test
Statistical test used to determine if the scores of two groups differ on a single variable (e.g. to determine
whether writing ability differs among students in two classrooms)
t-test for correlation coefficients

Statistical test used to determine whether a correlation coefficient is statistically significant
t-test for independent samples

Statistical test used to determine whether the difference between the means of two groups is statistically
significant
t-test for regression coefficients

Statistical test used to determine whether a regression coefficient is statistically significant
Table of random numbers

A list of numbers that fall in a random order
Table of specifications
A two-way grid, sometimes called a test blueprint, that lists major areas of content to be covered by the test as
row headings and major kinds of abilities to be developed and tested as column headings
Tactical objective
A written statement describing an intended output; A product-oriented or productivity-oriented objective;
Describes how a strategic objective will be accomplished.
Tactical performance measure

A yardstick or standard used to measure progress toward achieving a tactical objective
Tactical planning
A continuous process where people make decisions about how outcomes are to be accomplished, what
products will be produced, how success is measured and evaluated, and how budgetary resources are allocated
Tactical priorities
A ranking of an organization's tactical objectives by importance; Descriptions of which organizational outcomes
its decision-makers believe are more or less important in achieving the organization's mission; A basis for
determining the organization's operating budget
Talent
An unusually high level of proficiency in performing a task or using a skill; May be in the affective, cognitive,
and/or psychomotor domains
Talent development
A genuine intent to foster the long-term learning or development of others; Includes the ability to judge talent
(using a perspective that is detached from one's personal opinions) and capitalize on its potential, with the focus
on the developmental intent, not one's ability or willingness to train

Assessment Glossary
Tangible net worth

The sum total of a company’s assets less intangible assets and financial liabilities
Target cash balance

The amount of cash a company would like to have readily available
Target population
The larger population to whom the study results are to be generalized
Task
An activity, exercise, or question requiring students to solve a specific problem or demonstrate knowledge of
specific topics or processes; A goal-directed assessment activity or project, which prescribes that the student
use their background knowledge and skill in a somewhat long-term process to solve complex problems or
answer a multi-faceted question; A complex activity requiring multiple responses to a challenging question or
problem; A single question, problem, or activity used to assess a student; An assessment activity or assignment
to be completed by the learner/teacher being evaluated
Task analysis
Defining standards and conditions of a particular task and identifying the distinguishing factors between tasks
Task force
Group brought together to work on specific organizational problems
Task list
The list of steps in a project
Taxonomies
In the context of SLOs, a taxonomy is a classification of skills and abilities based on level of achievement, where
the levels move from very simple processes to very complex ones; Levels range from relatively simple
recognition of value to complex processes such as critical evaluation of ethical situations; Taxonomies may be
cognitive (classifications of cognitive skills and abilities ranging from relatively simple tasks such as recognition
to very complex processes such as critical analysis), psychomotor (classifications of physical skills and abilities
ranging from relatively simple imitation to creative mastery), or affective (classifications of skills and abilities
that relate to values and emotions)
Teachable
The practicality or feasibility of providing instruction on a topic, knowledge area, or skill in terms of the readiness
of the learners, the expertise of the teachers, and the availability of resources (e.g. equipment, time, funding)
Teacher effectiveness
Attribute of a teacher who has the capability or potential of having a positive impact on student learning,
behavior, and attitudes
Teacher evaluation
Systematic appraisal of a teacher's performance and/or qualifications in relation to the teacher's defined
professional role, job responsibilities, and school or district mission
Teacher evaluation system

A complete approach to the evaluation of teachers including its purpose, the rules and regulations that apply,
the target group to be evaluated, the domains to be covered, the procedures and methods to be employed, the
instruments to be used, the persons to be involved, and the types of reports and feedback to be provided
Teacher improvement

Assessment Glossary
The accomplishment of goals and objectives for professional development, growth in knowledge, acquisition of
skills, and changes in practices
Teacher norms
The expectations of teacher behavior in a given context or setting that are usually learned during student
teaching or in the first year of teaching in a new school
Teacher outcomes
The impact of intervention or experience on teachers' knowledge, attitudes, behaviors, or skills
Teacher-outcomes evaluation
Approach to evaluating teachers based on student performance, usually with standardized achievement test
results
Teaching-improvement loop
Teaching, learning, outcomes assessment, and improvement defined as elements of a feedback loop in which
teaching influences learning, and the assessment of learning outcomes is used to improve teaching and
learning
Team teaching
A system whereby two or more teachers pool their skills, knowledge, etc. to jointly develop, plan, and teach
combined classes
Telephone interview
An interview conducted over the phone
Temporal validity
The extent to which the study results can be generalized across time
Tenure
Employment status conferred upon a teacher by state law or institutional regulation after successful completion
of a probationary period which provides substantial, but not complete, protection against arbitrary or capricious
dismissal and termination, and entitles the teacher to due process procedures and other protections that may
not be available to the non-tenured teacher
Tenured faculty
Faculty who have completed a probationary period (usually two to five years) and are now considered
permanent employees of an institution or district with all applicable benefits and rights as specified in state law,
district policy, and collective bargaining agreements
Tertiary sector
The administrative and service sector of the economy; The part of the economy made up of nonprofit
organizations
Test
An assessment instrument consisting of a sample of items or tasks from a particular domain and that can
provide an estimate of performance in that domain; To administer an instrument or implement an assessment
process; Set of questions or situations designed to elicit responses that permit an inference about what a
student knows or can do; Tests generally utilize a paper and pencil format, occur within established time limits,
restrict access to resources or reference materials, and yield a limited range of acceptable responses
Test bias

Assessment Glossary
Used to refer to an instrument, which does not represent the ability or achievement of different groups equally;
Biased instruments favor one group or groups over another group or groups even when controlled for ability
level
Test center
A facility that provides computers and proctoring services in which to conduct tests
Test documents
Publications such as test manuals, technical manuals, user's guides, specimen sets, directions for test
administrators and scorers, and previews for participants that provide the information necessary to evaluate the
appropriateness and technical adequacy of a test for its intended purpose
Test equivalence
Assurance that examinees taking one version of a test do not have a relative advantage over those taking
another version
Test modification
Changes made in the content, format, and/or administration procedure of a test in order to accommodate test
takers who are unable to take the original test under standard test conditions
Test-retest reliability
A measure of the consistency of scores over time; The extent to which two administrations of the same test to
the same group of subjects yield consistent results; Used to assess the consistency of a measure from one time
to another; A reliability coefficient obtained by administering the same test a second time to the same group
after a time interval and correlating the two sets of scores; The degree of agreement is the square of the
correlation coefficient derived from correlating results from the same test administered twice to the same
subjects, generally at least three months apart;
Test-retest reliability coefficient

A type of reliability coefficient obtained by administering the same test a second time, after a short interval, and
correlating the two sets of scores; When the time interval between the two testings is considerable, such as
several months, a test-retest reliability coefficient also reflects the stability of the examinee trait being measured
in addition to the consistency of measurement provided by the test; Sometimes the term test-retest is also used
to describe the administration of different forms of the same test, in which case this reliability coefficient
becomes the same as the alternate form coefficient (see Parallel forms)
Test score pollution

An inflation of test scores that is the result of practices, usually associated with high-stakes testing, which are
designed to increase test scores or performance ratings without improvement of actual performance on the
attributes being assessed
Test theory
Includes the logic and statistical models that underlie standard practices in test use and design including
methods devised to estimate, overcome, or minimize the extent to which extraneous problems or errors
influence measurements taken in a given situation
Testing
Measurement of educational variables
Testlet
A group of items within a test that measure the same factor or which relate to the same stimulus; Items that are
part of a testlet are not statistically independent
Think tank

Assessment Glossary
A group organized for the purpose of intensive research and problem solving, especially in the areas of
technology, social or political strategy, or demographics
Theoretical sensitivity
When a researcher is effective at thinking about what kinds of data need to be collected and what aspects of
already collected data are the most important for the grounded theory
Theoretical saturation
Occurs when no new information or concepts are emerging from the data and the grounded theory has been
validated
Theoretical validity
The degree to which a theoretical explanation fits the data
Theory
An explanation or an explanatory system that discusses how a phenomenon operates and why it operates as it
does; A generalization or set of generalizations used systematically to explain some phenomenon; A particular
interpretation of a body of empirically verified observations; An attempt to make sense of a number of confirmed
hypotheses by drawing them together into a more meaningful whole
Theory-based evaluation
Faculty evaluation approach based on certain theories of teaching or of learning
Theory triangulation
The use of multiple theories and perspectives to help interpret and explain the data
Theory x and theory y

Two opposing assumptions about people at work that lead to opposing styles of management; Theory X
assumes that most people hate work, avoid responsibility, prefer to be directed, and have to be controlled and
coerced to put out a fair day's work; Theory Y assumes that people will seek responsibility, demonstrate a high
degree of imagination, and exercise self-direction if they have a creative, challenging job to which they can
become committed
Thick description
A rich and extensive set of details concerning methodology and context provided in a research report
Think-aloud technique
Participants verbalize their thoughts and perceptions while engaged in an activity
Thinking skills
Higher-order cognitive skills that enable human beings to comprehend experiences and information, apply
knowledge, express complex concepts, make decisions, criticize and revise unsuitable constructs, and solve
problems
Third party
Person(s) other than those directly involved in the educational process (e.g. employers, parents, consultants)
Third variable
A confounding extraneous variable
Third variable problem

An observed relationship between two variables that may be due to an extraneous variable

Assessment Glossary
Three necessary conditions for causation

In the transference theory of causation, the three things that must be present to contend that causation has
occurred are the relationship condition (variable A and variable B must be related), the temporal antecedence
condition (proper time order must be established), and the lack of alternative explanation condition (the
relationship between variables A and B must not be due to some third variable)
Thrift institution
The general term used to describe banks, lending institutions, and credit unions
Time-interval sampling
Checking for events during specific time intervals
Time management
The discipline of using time efficiently and well in order to achieve professional, personal, or organizational
objectives
Time series study or analysis

A study in which periodic measurements are obtained prior to, during, and following the introduction of an
intervention or treatment in order to reach conclusions about the effect of the intervention; A research design in
which behavior of subjects in naturally occurring groups is measured periodically both before and after
introduction of a treatment; Multiple repeated measures
Time-to-degree
The average number of semesters enrolled between student's first term and the term they graduate (elapsed
semesters)
Timed Tests
A test administered to a test taker who is allotted a strictly prescribed amount of time to respond to the test
Timeline
A calendar or list of dates showing the evaluation stages and activities, and indicating the dates by which they
should be implemented and be completed
Timeliness
Coming at an opportune time, or providing information at a point when it can readily inform the
assessment/evaluation or decision-making process
Tolerance
Recognition of and respect for the opinions, practices, or behaviors of others
Topology
Mapping of the relationships among subjects
Total debt
A metric used to establish a company’s financial risk by determining the total amount of assets being financed
by debt; Calculated by adding short-term and long-term debt and then dividing by total assets
Total quality management (TQM)

TQM is a management system taken from business and now being applied to higher education; TQM applies
concepts of control, quality, process, and customer service to management; The quality movement brings with it
a sense of collective responsibility for learning, a habit of listening to the people we serve, a preference for data,
an ethic of continuous improvement, a determination to develop fully the talent of every learner, and an
acknowledgement that we are professionally accountable to one another and to those we serve for results; A
structured system that satisfies internal and external customers and suppliers by integrating the business

Assessment Glossary
environment, continuous improvement and breakthroughs with development, improvement, and maintenance
cycles
Track record
A summary of past events and accomplishments related to an individual's performance
Tracking
A form of monitoring where a series of surveys which repeat a set of questions are administered over time to
measure the change in responses
Tracking systems
Systems set up to monitor progress, compile management information, and keep goals on track
Trade secret
Any formula, pattern, device, or compilation of information used in one’s business that gives the individual or
entity an opportunity to obtain an advantage over competitors that do not know or use this formula, pattern,
device, or compilation of information
Trademark
The name of a particular product or service legally registered as the exclusive property of a specific business
enterprise
Traditional or conventional assessment

Assessments utilizing traditional testing instruments; Traditional testing instruments include: Forced-choice,
machine-scorable, pencil-and-paper tests (e.g. matching, true-false, multiple choice) and restricted-completion,
short-answer, or essay questions
Training
The provision of instruction and planned activities to facilitate the learning of specific knowledge, skills, abilities,
attitudes, and behaviors; Intervention involving the provision of information, materials, or opportunities for
practice and experiment on specific aspects of teaching or learning
Traits
Distinguishable, relatively enduring ways in which one individual differs from another
Transcript analysis
Reviewing a student’s transcript file to answer questions about a student’s educational experiences and to help
faculty examine and understand the patterns of courses that best enhance the learning and development of a
student
Transcription
Transforming qualitative data into typed text
Transferability
Ability to apply the results of research in one context to another similar context; The extent to which a study
invites readers to make connections between elements of the study and their own experiences; The degree to
which the knowledge and skills demonstrated in solving an assessment task can be used in solving other work-
related tasks and real-world activities; One of several characteristics used to evaluate assessments
Transportability
The appropriateness of extending the use of a policy, instrument, assessment procedure, or evaluation system
across different teachers, student groups, subject areas, instructional approaches, learning activities, school
settings, states, etc.

Assessment Glossary
Translation rules
If one decides to generalize concepts during coding for a content analysis, then one must develop a set of rules
by which less general concepts will be translated into more general ones
Treatment
The stimulus given to a dependent variable; Set of repeatable operations under the control of the experimenter
that may be applied to experimental units or treatment groups; What researchers do to subjects in the
experimental group, but not to those in the control group
Treatment group
In a study, the group of subjects who receive the intervention of interest
Treatment variation validity

The ability to generalize across variations of the treatment
Trend
A general direction or movement; In statistics, a trend is a statistically detectable change over time
Trend analysis
The process of forecasting an organization’s staffing needs by analyzing past employment patterns in order to
identify trends that may be expected to continue; Forecasting technique that relies primarily on historical time
series data to predict the future; Detailed examination of a company's financial ratios and cash flow for several
accounting periods to determine changes in a borrower's financial position; Scientific study of changes in social
patterns (e.g. changes in fashion, technology, and consumer behavior)
Trend study
Independent samples are taken from a population over time and the same questions are asked
Trial
Application of one factor combination on one experimental unit
Triangulation or triangulate
The use of a combination of assessment methods in a study; A process of combining methodologies to
strengthen the reliability of a design approach; The use of multiple sources and methods to gather similar
information (e.g. an assessment that incorporated surveys, interviews, and observations); The attempt to obtain
more valid results by using multiple sources of data about one aspect of performance, multiple methods of
collecting data, and/or multiple interpretations of the same data; The collection of data via multiple methods in
order to determine if the results show a consistent outcome; The building of multiple sources of information or
ideas to support a central finding or theme; Using a variety of different research approaches or methods with
similar but different results to plot, define, or approximate the parameters of the area, field, or realm wherein lies
the actual or true results; Multiple forms of evidence that point to the same conclusion
True negative
A correct diagnostic judgment that an entity does not fall within a target category
True positive
A correct diagnostic judgment that an entity does fall within a target category
True score or value

The unattainable ideal of a score which has no measurement error; A hypothetical score that represents an
assessment result which is entirely free of error; Sometimes thought of as the average score of an infinite series
of assessments with the same or exactly equivalent instruments, but with no practice effect or change in the
person being assessed across the series of assessments

Assessment Glossary
Turnkey program
A tailor-made prepackaged program, developed by an external contractor for an organization that is ready for
immediate operation without a need for modifications
Two-stage cluster sampling

A set of clusters is randomly selected and then a random sample of elements is drawn from each of the clusters
selected in stage one
Two-tailed test
A non-directional application of one of the two-group inferential statistics (i.e. the t-test, the Wilcoxon, or the
Mann-Whitney) where the statistical procedure has to deal with separation of the two distributions down both of
the asymptotes (or "tails" of the bell curve)
Type I error
Rejecting a true null hypothesis; Error where the null hypothesis is rejected when it is true, thus causing the
hypothesis to be accepted when it should be rejected
Type II error
Failing to reject a false null hypothesis; Error where the null hypothesis is accepted when it is false, thus causing
the hypothesis to be rejected when it should be accepted
Type technique
Manipulating the independent variable by varying the type of variable presented to the different comparison
groups
Typical case sampling

Selecting what are believed to be average cases
Typology
A classification system that breaks something down into different types or kinds
Typology theory
The examination of individual differences in how people view and relate to the world

Assessment Glossary
U
Underserved populations
Groups identified as potential clients who are not receiving the full extent of services an organization provides,
often because of a history of exclusion due to race or ethnicity
Undervalued
An asset that is available for purchase at a cost that is lower than the product’s actual value
Unemployment insurance (UI)

A statutory benefit designed to provide workers who have been laid off a weekly income during short periods of
unemployment funded by state and federal taxes paid by employers
Unique case orientation

Perspective adopted by many researchers conducting qualitative observational studies; Researchers adopting
this orientation remember every study is special and deserves in-depth attention; Especially necessary for doing
cultural comparisons
Unobtrusive measures
Measures that do not require the researcher to intrude into the research context; Unobtrusive measurement can
reduce the biases that result from the presence of the researcher or measurement instrument; Unobtrusive
measures also reduce the degree of control the researcher has over the type of data collected; Three types of
unobtrusive measurement are Indirect measures, Content analysis, and Secondary analysis
Unintended consequences
Any unplanned or unanticipated outcomes that occur as a result of implementing an assessment or evaluation;
For example: The use of student test scores as part of teacher evaluation results in test score pollution practices
and increased teaching-to-the-test activities and less reliance on the curriculum guide for planning instructional
activities and lessons
Unit cost
The per-item cost
Unit of analysis
The least divisible element on which measures are taken and analyzed
Univariate analysis
Studying the distribution of cases of one variable only
Universalism
The belief that common evaluative standards, rules, and practices should apply to everyone regardless of
circumstance
Universe
The domain of all possible items for a given construct; The domain of all possible test items for a given construct
Upper limit
The largest number on a confidence interval
Unrealized capital gain

Assessment Glossary
An increase or decrease in the value of a security that is not genuine because the security has not actually been
sold
Unrestricted sampling
The technical term used for sampling with replacement
Unsecured debt or loan

A debt that is not secured by any property (collateral)
Unserved populations
Groups identified as potential clients who are not receiving any of the services an organization provides, often
because of a history of exclusion due to race or ethnicity
Unstructured interview
An interview done without using a questionnaire or list of topics; Normally used when respondents are asked to
describe an important or recent event in their life; The researcher asks open-ended questions which allows
respondents to talk freely and to influence the direction of the interview since there is no predetermined plan
about the specific information to be gathered from those being interviewed
Unstructured observation
The researcher uses direct observation to record behaviors as they occur, with no preconceived ideas of what
will be seen; There is no predetermined plan about what will be observed
Usability testing
Methods that evaluate products by observing and measuring the ease and ability of the intended users of a
product to achieve or accomplish the intended purpose of the product; Similar to the machine-user interface or
man-machine research of ergonomics and human factors engineering; Currently most associated with the
evaluation of websites, web applications, and computer interfaces or programs
Uses and gratifications

Form of audience research which studies how people use the experience or consumption of media programs
(for information, aesthetic feelings, personal needs, social needs, and/or escapism); See also Reception
analysis
Utilitarianism
An ethical approach that says judgments of the ethics of a study depend on the consequences the study has for
the research participants and the benefits that may arise from the study
Utility
The extent to which an evaluation will serve the relevant information needs of students, educators, and other
appropriate users; The extent to which an evaluation produces and disseminates reports that inform relevant
audiences and have beneficial impact on outcomes; The practical value of an assessment/evaluation system
with respect to such factors as time requirements, logistics, resources needed, costs and benefits, and
applicable regulatory and institutional policies, as well as the technical concerns of validity and reliability; A
characteristic of scoring criteria that ensures the criteria are diagnostic and can communicate information about
performance quality with clear implications for improvement; An evaluation, often in cost-benefit form, of the
relative value of using a test vs. not using it, of using a test in one manner vs. another, or of using one test vs.
another test

Assessment Glossary
V
Vagueness
Uncertainty in the meaning of words or phrases
Validation
The process of gathering evidence that supports inferences made on the basis of test scores; The process of
determining the appropriateness, meaningfulness, and usefulness of a measure, an instrument, or an
assessment process, and of the inferences resulting from it; The process of investigation by which the validity of
the proposed interpretation of test scores is evaluated
Validity
The accuracy of the inferences, interpretations, or actions made on the basis of test scores; Refers to whether
or not an assessment measures what it is intended to measure; Extent to which test scores actually reflect what
they were meant to measure; The capacity of a measuring instrument to predict what it was designed to predict;
Stated most often in terms of the correlation between values on the instrument and measures of performance on
some criterion; The degree to which inferences drawn about a student's knowledge, skills, attitudes, and
behaviors resulting from the assessment methods used are correct, trustworthy, and appropriate for making
decisions about students; The extent to which the test scores or responses measure the attribute(s) that they
were designed to measure The soundness of the use and interpretation of a measure; Degree to which a study
accurately reflects or assesses the specific concept that the researcher is attempting to measure; A method can
be reliable, consistently measuring the same thing, but not valid; Validity has three components: Relevance (the
option measures your educational objective as directly as possible), Accuracy (the option measures your
educational objective as precisely as possible), and Utility (the option provides formative and summative results
with clear implications for educational program evaluation and improvement): There are also several different
types of validity: See also Concurrent validity, Consequential validity, Construct validity, Content validity,
Criterion-related validity, Curricular validity, Evidential basis of validity, Face validity, Instructional
validity, Predictive validity, Systemic validity
Validity coefficient
A measure of the degree of validity, usually expressed as the correlation between the measure in question and
another measure or a variable; A correlation coefficient computed to provide validity evidence, such as the
correlation between test scores and criterion scores
Validity evidence
Empirical evidence and theoretical rationales that support the inferences or interpretations made from test
scores
Validity generalization study

An investigation of the degree of generalizability or of transportability of a policy, instrument, process,
procedure, or evaluation system
Value added
The effects educational providers have had on students during their programs of study; The impact of
participating in higher education on student learning and development above that which would have occurred
through natural maturation, usually measured as longitudinal change or difference between pretest and posttest;
A comparison of the knowledge, skills, and developmental traits that students bring to the educational process
with the knowledge, skills and developmental traits they demonstrate upon completion of the educational
process; The change in an attribute or product that can be linked to an intervention (e.g. the change in student
test scores before and after completion of a course or the change in the effectiveness of a teacher's classroom
management skills as a result of participation in a workshop series); Increase in learning that occurs during a
course, program, or undergraduate education; May focus on the individual student (e.g. how much better a

Assessment Glossary
student can write at the end than at the beginning) or a cohort of students (e.g. whether senior papers
demonstrate more sophisticated writing skills, in the aggregate, than freshmen papers); Requires a baseline
measurement for comparison
Value chain
The process of manufacturing a product in its entirety, from the procurement of raw materials to the time of
utilization
Value dimensions
Meaningful differences in key values and beliefs among cultures identified by empirical research
Value proposition
The core benefit of a product or service
Value standards
Statements that describe attitudes teachers would like students to develop towards learning
Values clarification techniques

Procedures used to help groups recognize the different values held in the groups, to discern conflicts among
these values, and to consider how these conflicts might be resolved
Values statement
A narrative description that outlines an organization’s beliefs and guiding principles; A written statement
describing the direction that indicates success or improvement in internal organizational characteristics, or in the
expression of beliefs the organization holds; A document outlining and representing the core priorities in the
organization’s culture
Variable
A condition or characteristic that can take on different values or categories; Any quantity that can assume more
than one state or numerical value; Observable characteristics that vary among individuals; A quantity you can
measure or question you can ask about each member of a population (The answer to the question for a
particular individual is called the value of the variable for the individual, or simply an observation); A measured
characteristic that can assume various values or levels; See also Discrete variable and Continuous variable
Variable costs
Total costs minus any fixed costs
Variability
Degree to which characteristics are observed to vary
Variance
A measure of variation within a distribution determined by averaging the squared deviations from the mean of a
distribution; A measure of variability; The average squared deviation from the mean; A measure of the average
deviation from the mean in squared units; The square of the standard deviation; A measure of variability of test
scores, computed as the sum of the squares of the deviations of raw scores from the arithmetic mean, divided
by one less than the number of scores; The smaller the variance, the closer the individual scores are to the
mean
Variance forecast
A measure that utilizes a demand and availability forecast to determine whether an organization is able to meet
future manpower needs
Variate
The quantitative measure of a variable

Assessment Glossary
Variation
The dispersion of data points around the mean of a distribution
Vertical management
A traditional organizational structure consisting of primary functions (e.g. engineering, manufacturing, finance,
etc.) with each function having its own manager
Vertical organization
An organizational structure consisting of many layers of management or other positions of authority
Verification
The process of checking the accuracy of data and information about someone's experience, training,
performance, and other attributes
Verisimilitude
Having the semblance of truth; Probability that the research findings are consistent with occurrences in the "real
world"
Vested rights
An individual’s right to pension benefits that are paid up and therefore are not contingent upon the employee’s
continued service to the employer
Vision statement
A narrative description of a future state to help keep project teams focused; A concise statement of what needs
to change, typically the promoter, sponsor, or leader’s agenda; A compelling, conceptual image of the desired
future; A realistic, credible, attractive future for the institution that sets forth the ideal state the institution aims to
achieve and provides long-term direction, guidance, and inspiration for the organization; A vision statement is a
description of what an organization wants to become or hopes to accomplish in the future (typically in the next
10 years)
Visioning
See envisioning
Visitor or event survey

A survey of visitors at a venue or event
Vocational and technical education

Organized educational activities that offer a sequence of courses that provides individuals with the academic
and technical knowledge and skills the individuals need to prepare for further education and for careers in
current or emerging employment sectors; Includes a competency-based applied learning that contributes to the
academic knowledge, higher-order reasoning and problem-solving skills, work attitudes, general employability
skills, technical skills, and occupation-specific skills
Volunteerism
Organizational support, often in the form of paid leave or sponsorship, for employees pursuing volunteer
opportunities or performing community services
Voting participation
A behavioral outcome associated with citizenship skills

Assessment Glossary
W
WASC
Western Association of Schools and Colleges
Web based assessments or tests

Assessments delivered via the Internet, or an Intranet, in which the items reside on a server and are packaged
with HTML to allow a participant to respond using a browser; Used to refer to instruments (test, surveys, and
questionnaires) which are deliverable over the World Wide Web and viewable through web browsers
Web surveys
Participants read and complete a survey form that is developed for and located on the web
Weighted average
An average of quantities that takes into account their relative importance in a data set instead of treating each
quantity equivalently
Weighted score or scoring

A score adjusted by such factors as the importance of the attribute assessed to a performance, the reliability
and validity of the assessment from which the score was derived, or a combination of such factors; A method of
scoring a test in which the number of points awarded for a correct (or diagnostically relevant) response is not the
same for all items in the test; The scoring formula may award more points for one response to an item than for
another
Weighting
The amount of emphasis given to a particular set of information; For grading purposes weighting usually entails
multiplying all scores for one component by a numerical value to increase the emphasis it receives over other
data; A method to combine the results of two or more assessments used in calculating the percent who meet
the standard for a learning outcome; If some assessments are deemed more important than the cut-scores on
those assessments may be given greater consideration or weight in determining the overall performance level
Wellness program
Programs such as on-site or subsidized fitness centers, health screenings, smoking cessation, weight
reduction/management, health awareness, and education that target keeping employees healthy, thereby
lowering costs associated with absenteeism, lost productivity, and increased health insurance claims
Wholly-owned subsidiary
A company that is entirely owned by another company
Windfall profit
An unforeseen profit that occurs as a consequence of events beyond the recipient’s control
Withholding tax
Income tax withheld from an employee’s wages and paid directly to federal, state and local government
agencies by the employer
Within-stage mixed model research

Quantitative and qualitative approaches are mixed within one or more of the stages of research
Word attack skills

Means by which a person recognizes and perceives the meaning of words

Assessment Glossary
Word response
A response style where the participant enters a word to indicate their choice
Work-based learning
Supervised student learning that takes place at a work site through unpaid or paid activities
Work ethic or behavior

A set of personal attributes required for success at work; Includes the ability to identify established rules,
regulations, and policies, to practice cost effectiveness and time management, to assume responsibility for
decisions and actions, to display initiative and assertiveness, and to demonstrate a willingness to learn
Work-readiness
Individual’s preparation and awareness of the world of work, labor market, occupational information, values
clarification and personal understanding, career planning and decision making, and job search techniques,
including daily living skills, personal skills, work ethic and initiative, reliability, and accepting responsibilities
involved in maintaining a job
Work sample
An assessment method that uses actual and typical on-the-job activities or tasks
Work simulation
A surrogate or imitation of a work sample task
Work-to-education
The process of acquiring greater learning and development to meet increased workplace need and demands
Workforce flexibility
The contribution of postsecondary education to the adaptability of the workforce to keep pace with change
Workforce planning
The assessment of current workforce content and composition issues used to determine what actions must be
taken to respond to future needs
Working capital
The amount of cash available to a business for routine operations
Written surveys or questionnaires

Documents which ask individuals to share their perceptions about the study target, such as their own or others
skills/attitudes/behavior, or program/course qualities and attributes.
Writing skills
Communicating ideas and information through documents such as letters, manuals, reports, and graphs

Assessment Glossary
X
x
Unknown quantity to be determined by completion of a mathematical formula
X-axis
Width; The horizontal axis of a two-dimensional Cartesian coordinate system; The horizontal axis on a graph,
chart, or plot

Assessment Glossary
Y
Y-axis
Height; The vertical axis of a two-dimensional Cartesian coordinate system; The vertical axis on a graph, chart,
or plot
Y-intercept
The point where the regression line crosses the Y-axis
Year-end closing
Financial reports prepared and released at the end of a company’s fiscal year

Assessment Glossary
Z
Z-score
A type of standard score scale in which the mean equals zero and the standard deviation equals one unit for the
group used in defining the scale
Z-axis
Depth; The third dimension in a three dimensional plane or coordinate system
Zero-balance account
A type of bank account whereby funds are automatically transferred into it from another account when claims
against it arise, versus maintaining funds in the account on a regular basis
Zero-base budgeting
Budget format that presents information about the efficiency and effectiveness of existing programs and
highlights possibilities for eliminating or reducing programs by assuming that the minimum funding level for the
agency is zero, thereby requiring agency administrators to justify all expenditures by the same standard of
review that normally are applied only to new programs or increments above the base; A budgeting system that
starts with no authorized funds as a starting point; In a zero-based budget, each activity or program to be funded
must be justified every time a new budget is prepared and resources are allocated accordingly
Zero growth
An economic term indicating a decline in output for two or more consecutive quarters
ZMET
Zaltman Metaphor Elicitation Technique; An elaborate way of discovering people's feelings about a concept or
issue by producing a picture of their mental associations

TFR Guide Assessment Glossary 2009-08-01 TVT

Hochgeladen von

Dokumentinformationen

Copyright

Verfügbare Formate

Dieses Dokument teilen

Dokument teilen oder einbetten

Freigabeoptionen

Stufen Sie dieses Dokument als nützlich ein?

Sind diese Inhalte unangemessen?

Copyright:

Verfügbare Formate

TFR Guide Assessment Glossary 2009-08-01 TVT

Hochgeladen von

Copyright:

Verfügbare Formate

Assessment Glossary

Compiled by Todd V. Titterud

The Follow-Up Report: Guides and Handbooks 1

Assessment, Research, and

|A|B|C|D|E|F|G|H|I |J |K|L |M|N|O|P|Q|R

Ability or trait parameter

The Follow-Up Report: Guides and Handbooks 2

Academic aptitude test

Academic program review

The Follow-Up Report: Guides and Handbooks 3

Accrediting or Accreditation Liaison Officer (ALO)

Achievement or proficiency levels

Achievement test or testing

Acquiescence response set

The Follow-Up Report: Guides and Handbooks 4

The Follow-Up Report: Guides and Handbooks 5

Additive and interactive effects

Adjusted validity or reliability coefficient

Advanced placement exams

Adversarial groups or teams

Advocacy groups or teams

The Follow-Up Report: Guides and Handbooks 6

Affirmative action plan

The Follow-Up Report: Guides and Handbooks 7

Ambiguous temporal precedence

The Follow-Up Report: Guides and Handbooks 8

ANCOVA (Analysis of Co-Variance)

The Follow-Up Report: Guides and Handbooks 9

Announced visit or observation

Annual update or report

ANOVA (Analysis of Variance)

Application service provider (ASP)

The Follow-Up Report: Guides and Handbooks 10

Archived research data

The Follow-Up Report: Guides and Handbooks 11

Assessment or evaluation audit

Assessment for accountability

Assessment for improvement

The Follow-Up Report: Guides and Handbooks 12

Assessment report or record

The Follow-Up Report: Guides and Handbooks 13

Audit review approach

The Follow-Up Report: Guides and Handbooks 14

Authentic performance assessment

Awareness of the hypothesis

The Follow-Up Report: Guides and Handbooks 15

The Follow-Up Report: Guides and Handbooks 16

Behavior summary scale (BSS)

Behavioral observation scale (BOS)

Behavioral risk management

Behaviorally-anchored rating scale (BARS)

Bell curve or bell-shaped curve

The Follow-Up Report: Guides and Handbooks 17

Binomial sign test

The Follow-Up Report: Guides and Handbooks 18

Body of work (BoW)

Bonferroni's correction for multiple comparisons

Boolean operators or connectors

Box plot or box-and-whisker plot

The Follow-Up Report: Guides and Handbooks 19

The Follow-Up Report: Guides and Handbooks 20

Canned, packaged, or off-the-shelf instruments or software

Capital expense budget

The Follow-Up Report: Guides and Handbooks 21