Trends in Food Science & Technology 20 (2009) 300e312
Viewpoint
Systematic
assessment of core
assurance activities
in a company specific
food safety
management system
P.A. Luninga,*, W.J. Marcelisb,
J. Rovirac, M. Van der Spiegela,
M. Uyttendaeled and L. Jacxsensd
a
Product Design and Quality Management Group,
Department of Agrotechnology and Food Sciences,
Wageningen University, P.O. Box 8129, NL-6700 EV
Wageningen, The Netherlands (Tel.: D31 317 482087;
fax: D31 317 483669; e-mail: pieternel.luning@wur.nl)
b
Management Studies Group, Department of Social
Sciences, Wageningen University, P.O. Box 8130,
NL-6700 EW Wageningen, The Netherlands
c
Department of Biotechnology and Food Science,
University of Burgos, Pza. Misael Bañuelos s/n, 09001
Burgos, Spain
d verify that such safety and legal quality requirements are
Department of Food Safety and Food Quality,
Laboratory of Food Preservation and Food
Microbiology, Faculty of Bioscience Engineering, Ghent
University, Coupure Links, 653, 9000 Ghent, Belgium
The dynamic environment wherein agri-food companies oper-
ate and the high requirements on food safety force companies
to critically judge and improve their food safety management
system (FSMS) and its performance. The objective of this study
was to develop a diagnostic instrument enabling a systematic
assessment of a company’s food safety management system.
This paper discusses core assurance activities, their contribution
* Corresponding author.
0924-2244/$ - see front matter Ó 2009 Published by Elsevier Ltd.
doi:10.1016/j.tifs.2009.03.003
to assurance, and how to judge these activities in a company’s
FSMS. The diagnosis shows which core assurance activities
a company executes and at what level, therewith creating a ba-
sis for discussion about systematic improvements.
Introduction
Food quality and safety are important drivers for the
organisation and management of food production systems
in agribusiness and food industry. The typical characteris-
tics of food production, such as microbiological and
chemical food safety risks, restricted shelf life, seasonal
harvesting, heterogeneity of raw materials, and complexity
of supply chains, put high demands on control and assur-
ance of food quality and safety (Ababouch, 2006; Da
Cruz, Cenci, & Maia, 2006; Luning & Marcelis, 2009;
Van der Spiegel, Luning, Ziggers, & Jongen, 2003). More-
over, governments have been increasingly concerned about
the fact that existing safety requirements have been ineffec-
tive in reducing the growing burden of foodborne illnesses.
This, in combination with the international developments
linking food safety with trade, resulted in new food legisla-
tion focused on assuring high levels of food safety (Kven-
berg, Stolfa, Stringfellow, & Garrett, 2000; Martin et al.,
2003; Van der Meulen & Van der Velde, 2004). The current
General Food Law in Europe now prescribes that food and
feed business operators at all stages of production, process-
ing and distribution, within the business under their control,
shall ensure that foods or feeds satisfy the requirements of
food law, which are relevant to their activities, and shall
met (General food Law regulation No 178/2002 article 17).
This imposes a general responsibility to ensure safety of
any food brought to the market. Any adverse effects of
food may constitute therefore an offence committed by
the business operator (Van der Meulen & Van der Velde,
2006).
Severe food legislation requirements in combination
with changes in food supply chains, health and demo-
graphic situations, lifestyle and social situations, and
environmental conditions have led to significant efforts in
development of quality management systems in agribusi-
ness and food industry worldwide (Efstratiadis, Karirti, &
Arvanitoyannis, 2000; Lee & Hathaway, 2000; Luning,
Devlieghere, & Verhé, 2006; Ropkins & Beck, 2000). Now-
adays, food industries have implemented various GMP and
 P.A. Luning et al. / Trends in Food Science & Technology 20 (2009) 300e312
HACCP guidelines (like General Principles of food hygiene
(CAC, 2003)), GFSI guidance document (GFSI, 2007), and
quality assurance standards (like BRC, 2008; IFS, 2007;
ISO 9001:2008, 2008; ISO 22000:2005, 2005) into their
quality management system. The performance of such
systems in practice is however still variable. Several studies
reported positive effects of implemented HACCP principles
according to Codex Alimentarius (e.g. Cenci-Goga et al.,
2005; Khatry & Collins, 2007; Mantovanelli, Marino,
Comi, Vallavanti, & Dolzani, 2001; Naugle, Barlow, Eblen,
Teter, & Umholtz, 2006; Violaris, Bridges, & Bridges,
2008). Others discussed that poor implementation and exe-
cution of such systems are still reasons for rework, too high
contamination levels, customer complaints, recalls, and
even foodborne outbreaks (e.g. Holt & Henson, 2000;
Luning & Marcelis, 2006; Motarjemi & Käferstein, 1999;
Naugle et al., 2006; Sumner, Raven, & Givney, 2004;
Sun & Ockerman, 2005). A wide range of studies have
identified several factors contributing to unsatisfactory per-
formance of quality management systems. Typical factors
include, inappropriate assessment of critical control points
and their monitoring systems, inadequate cooling capacity,
poor cleaning programs, ineffective heat treatments (Holt &
Henson, 2000; Mitchell, 1998; Panisello & Quantick, 2001;
Todd, 2003; Walker, Pritchard, & Forsythe, 2003), insuffi-
cient record keeping and documentation (Panisello &
Quantick, 2001; Vela & Fernández, 2003; Walker et al.,
2003; Worsfold, 2006), lack of validation and verification
(Taylor & Kane, 2005), and poor food handler behaviour
(Basx, Ersun, & Kıvanç, 2006; Legnani, Leoni, Berveglieri,
Mirolo, & Alvaro, 2004; Michaels et al., 2004; Vela &
Fernández, 2003; Walker et al., 2003). These studies have
addressed as well typical problems related to control activ-
ities (e.g. monitoring systems, preventive measures, etc.) as
problems related to assurance activities (e.g. verification
and validation).
The continuous pressure on quality management sys-
tems and the dynamic environment wherein systems oper-
ate, such as emerging pathogens, changing consumer
demands, developments in preservation techniques require
that they can be systematically diagnosed to determine op-
portunities for improvement (Manning, Baines, & Chadd,
2006; Mortimore, 2000; Van der Spiegel, Luning, de
Boer, Ziggers, & Jongen, 2006; Wallace, Powell, & Holy-
oak, 2005). In a previous study, we developed a diagnostic
instrument focused on assessment of microbiological safety
control activities in a company’s food safety management
system (FSMS) (Luning, Bango, Kussaga, Rovira, & Mar-
celis, 2008). However, each FSMS covers not only control
activities but also typical assurance activities (CAC, 2003;
Luning & Marcelis, 2007, 2009).
The objective of this study is to extend the food safety
control diagnostic instrument to enable also the assessment
of those activities in a company’s FSMS aimed at assuring
that food safety requirements are met. Firstly, we define the
term FSMS and subsequently describe the principle of the
301
diagnostic instrument. Then we discuss which activities
are crucial for assurance and develop grids to assess core
assurance activities at different levels. Finally, we evaluate
the usefulness of the diagnostic instrument for assessment
of FSMS, and briefly describe future research activities.
Food safety management system
Before starting with the diagnostic instrument, we clar-
ify the term ‘food safety management system’, consisting
of the terms food safety and safety management system.
Food safety can be considered as a specific aspect of food
quality (Luning & Marcelis, 2009), and is defined accord-
ing to Codex Alimentarius (CAC, 2003) and adopted in
ISO 22000:2005 (2005) as the assurance that food will
not cause harm to the consumer when it is prepared and/
or eaten according to its intended use. Quality management
refers to all activities that organisations use to direct, con-
trol, and co-ordinate quality, including formulating a quality
policy, setting quality objectives, quality planning, control,
assurance, and improvement (ISO 9000, 2005; Luning &
Marcelis, 2007). Organisations use a quality management
system (QMS) to direct and control implementation of
quality policies and achievement of quality objectives
(ISO 9000, 2005). A QMS includes the organisational
structure, responsibilities, processes, procedures, and
resources that facilitate the achievement of quality manage-
ment (Luning & Marcelis, 2009). This definition refers to
the quality management system of the company. Stake-
holders (e.g. government, retailers) commonly require
a specific set-up of the company’s quality management sys-
tem. These requirements are described in legislation (e.g.
EU Regulation 852/2004) and the well-established QA
guidelines (e.g. GMP and HACCP principles according to
Codex Alimentarius (CAC, 2003)) and QA standards (e.g.
ISO 9001:2008, ISO 22000:2005, BRC, etc.) (Escriche,
Doménech, & Baert, 2006; Jacxsens, Devlieghere, & Uyt-
tendaele, 2009; Luning, Van der Spiegel, & Marcelis,
2006). They all impose demands on the organisation of
the company’s system. A food safety management system
(FSMS) involves that part of the QMS that is specifically
focused on food safety. In this study, we focus on those
activities that specifically aim at controlling and assuring
microbiological food safety, leaving (yet) chemical and
physical hazards out of the scope.
Structure diagnostic instrument
Based on a comprehensive literature search in the fields
of food quality management, quality management systems,
and microbiological food safety, we developed a diagnostic
instrument to assess FSMS. Fig. 1 presents the core activi-
ties in an FSMS contributing to food safety control (FSC)
and food safety assurance (FSA). The diagnostic instrument
addresses explicitly control and assurance, because these
are two distinct quality management functions, each with
another objective, different activities, and decisions (Lun-
ing & Marcelis, 2007). Control aims at keeping product
302 P.A. Luning et al. / Trends in Food Science & Technology 20 (2009) 300e312
Fig. 1. Core control and assurance activities as addressed by the FSMS diagnostic instrument.
properties, production processes, and human processes be-
tween certain acceptable tolerances, whereas the objective
of assurance is to control the system and to provide
evidence and confidence to stakeholders about meeting
the set requirements. Control activities concern the ongoing
process of evaluating performance of both technological
and human processes and taking corrective actions when
necessary, whereas assurance activities deal with setting re-
quirements on the system, evaluating its performance, and
organising necessary changes (Evans & Lindsay, 2005;
Luning & Marcelis, 2007, 2009).
The principle of the diagnostic instrument encompasses
a systematic analysis of which activities are addressed in
the company’s FSMS, and assessment of the level at which
these are executed. The diagnostic instrument involves
a comprehensive checklist of core activities of an FSMS,
and grids with for each activity descriptions of three differ-
ent levels. The diagnostic instrument distinguishes three
different microbiological control strategies, i.e. preventive
measures, intervention processes, and monitoring systems,
which contribute in different ways to microbiological
food safety (lower part of Fig. 1). Fig. 1 shows, for each
strategy, the core technology dependent activities and man-
agerial activities that affect microbiological performance of
the control system, whereby an explicit distinction in
design and operation of core control activities is made
because actual system performance depends on both (Lun-
ing & Marcelis, 2006; Mortimore, 2001). Details about the
control activities are in our previous paper discussed (Lun-
ing et al., 2008). Which core assurance activities should be
addressed to analyse an FSMS will be discussed in more
detail in the next section.
Core assurance activities
The diagnostic instrument addresses the following core
assurance activities, i.e. defining system set-up, validation,
verification, and documentation and record keeping
(Fig. 1). The basic assumption is that assurance, as de-
manded by stakeholders, actually is achieved when the
FSMS has been implemented according to their require-
ments and when it is ‘in control’.
Defining the system set-up
Companies nowadays operate in a dynamic and interna-
tional environment wherein high demands are put on assur-
ance of safety of food products (Beulens, Broens, Folstar, &
Hofstede, 2005; Trienekens & Zuurbier, 2008). Each com-
pany needs to translate external assurance requirements
into requirements on the company’s FSMS. These require-
ments come from stakeholders, such as government obli-
gating implementation of GMP/HACCP principles,
retailers requiring certification against BRC or IFS stan-
dards, and or headquarters of multinationals demanding
a uniform company assurance system based on ISO
22000:2005 (Cairns, Harris, Hutchison, & Tricker, 2005;
Kvenberg et al., 2000; Lee & Hathaway, 2000; Martin
 P.A. Luning et al. / Trends in Food Science & Technology 20 (2009) 300e312
et al., 2003). In this translation process, current control and
assurance activities are compared to the (new) stakeholder
requirements, and new specifications on the FSMS activi-
ties are then set, and subsequently translated into new
concrete system requirements (like new, adapted hygiene
procedures, new sampling plans, more accurate equipment,
etc.). This translation process is an important assurance
activity, because it determines the specific set-up of the
company’s FSMS (Fig. 1). Subsequently, when the system
is functioning in practice, additional requirements derive
from feedback information and data from the operational
food safety control system (e.g. data about systematic fail-
ures, changes in recipes, production process modifications,
etc.). This translation process is defined as core assurance
activity because it determines if the system is modified
(e.g. to overcome undesirable situations) and or adapted
(e.g. to new production circumstances) in such a way that
it complies with the assurance demands as set by stake-
holders (Fig. 1).
Validation and verification
To provide evidence and confidence to stakeholders that
safety requirements will be met, the performance of the
control system must be evaluated on its principal effective-
ness and proper execution (CAC, 2008; Cormier, Mallet,
Chiasson, Magnusson, & Valdimarsson, 2007; Doménech,
Escriche, & Martorell, 2008; ILSI, 1999, 2004; Stecchini
& Del Torre, 2005). Two crucial activities to support this
assurance objective are validation and verification (CAC,
2008; Jacxsens et al., 2009; Kvenberg & Schwalm, 2000;
Wallace et al., 2005). Both lead to critical judgment of
the FSMS and supply essential information and data for
necessary system changes (Fig. 1).
The terms verification and validation create sometimes
confusion in practice (ILSI, 1999; Jacxsens et al., 2009;
Mayes, 1999; Panisello & Quantick, 2001; Scott, 2005;
Sperber, 1998). The Codex Alimentarius (CAC, 2003)
defined, within the scope of HACCP, validation as obtain-
ing evidence that the elements of the HACCP plan are
effective, but it is considered as part of the 6th principle
‘establishment of verification procedures’. The new Codex
document on validation (CAC, 2008) defines validation as
obtaining evidence that a control measure or combination
of control measures, if properly implemented, is capable
of controlling the hazard to a specific outcome. Verification
involves the application of methods, procedures, tests and
other evaluations, in addition to monitoring to determine
compliance with the HACCP plan. Verification confirms
the effectiveness of the established system and ensures af-
terwards, with appropriate frequency that the provisions
laid-down are still being conformed to CAC (2003) and
ILSI (2004). In ISO 22000:2005 (2005) validation is as
a process defined that is used to ensure that food safety con-
trol measures are capable of being effective. The validation
process uses evidence to determine whether control mea-
sures are capable of controlling food safety hazards and
303
ensuring that products are safe. Verification is as a process
defined that uses objective evidence to confirm that speci-
fied requirements have been met. In other words, validation
activities involve checking in advance the effectiveness of
designed control measures, whereas verification activities
concern checking afterwards if control activities are operat-
ing in practice as designed. Since both have a distinct func-
tion, we have appointed them explicitly in the diagnostic
instrument. The established QA standards (e.g. BRC, ISO
22000:2005) do not provide detailed requirements on vali-
dation. Some authors and guidance documents indicated
that hazard analysis, determination of CCPs, specification
of critical limits, and establishment of monitoring systems
are important control activities that should be validated
(CAC, 2008; ILSI, 2004; Mayes, 1999; NFPA, 2002; Scott,
2005; Sperber, 1998). In the diagnostic instrument, we spe-
cifically address the validation activities that are aimed at
assessing the effectiveness of core control measures, as
identified in our previous study (Luning et al., 2008)
(Fig. 1).
For verification, several studies, documents, and some
QA standards and guidelines provided suggestions and or
checklists about which elements should be taken into
account. Mayes (1999) described typical examples of veri-
fication activities, such as review of HACCP system and its
records, review of deviations and product dispositions, and
confirmation that CCPs are under control. ILSI (2004) pre-
scribed verification procedures to establish conformity with
the system and its record, involving operations (including
calibration), deviations, corrective actions, and other mea-
sures, internal audits, supplier audits, and changes to the
system. Others emphasised the importance of verifying
prerequisite programs (like, cleaning and disinfection, per-
sonal hygiene requirements, etc.) (Escriche et al., 2006;
FDA, 2006; Sperber, 1998). The BRC Global Standard
for Food Safety (BRC, 2008) defined a detailed checklist
of elements to verify.
In our diagnostic instrument, we specifically address the
verification activities of core control measures, whereby we
made a distinction between verification of equipment and
people related performance, as shown in Fig. 1.
Documentation and record keeping
Documentation and record keeping support the basic as-
surance activities defining control system set-up, validation,
and verification. Therefore, it is as two other crucial activi-
ties assigned in our diagnostic instrument. We distinguished
documentation and record keeping because they have two
different objectives. Documentation aims at keeping knowl-
edge and information, whereas record keeping aims at
collecting data. Procedures, manuals, (work) instructions,
flow diagrams, research reports, complaints, statistical anal-
yses, etc. are typical sources of information and knowledge
for documentation whereas process and product data, spec-
ifications of packaging materials, records of distribution and
storage are typical sources for the record-keeping system
304 P.A. Luning et al. / Trends in Food Science & Technology 20 (2009) 300e312
(Jacxsens et al., 2009; Luning & Marcelis, 2009). Documen-
tation and record keeping are essential for assurance be-
cause it supports a transparent system and enables
stakeholders (government, auditors, certification authorities,
etc.) to evaluate and or certify the system, which contributes
to providing confidence (Duxbury, 2005; Jacxsens et al.,
2009; Walker & Jones, 2002).
Summarising, the assurance function concerns the ongo-
ing process of monitoring (new) assurance demands by
stakeholders, collecting data and information from the in-
ternal control system, evaluating its principle and actual
performance (by validation and verification activities),
applying corrective measures (i.e. setting new or modified
system requirements) and documenting essential informa-
tion and data to provide evidence and confidence.
Assessment of assurance activities
Now we have addressed the core assurance activities, the
next question is how to assess them. Similar to the assess-
ment of core control activities (Luning et al., 2008), we
have developed grids with descriptions of different levels
of assurance activities (low, medium, and high). Before ex-
plaining the levels of activities, we firstly discuss the crite-
ria we used to judge them.
Assurance activities can be as research activities consid-
ered that should result in practical decisions. To differenti-
ate levels in assurance activities, we start from two general
criteria used in this type of research, i.e. validity and reli-
ability. Validity concerns rightness and precision, and reli-
ability includes consistency and ability of verification
(Luning & Marcelis, 2009). We used these general criteria
to differentiate ‘what’ (content) and ‘how’ (structure) of as-
surance activities in more detail (Table 1). With content of
assurance activities, we refer to the information discussed
and analysed. Using these general criteria, we formulated
the criteria specificity of the information (validity) and
knowledge on which the information is based (reliability).
With structure of the assurance activities, we refer to the
way they deal with the information. Using the general cri-
teria, we formulated the method of executing the activities
(validity), and the controllability of the activities (reliabil-
ity). The underlying criteria in Table 1 are used to describe
Table 1. Underlying criteria to differentiate assurance activities in the assessment grid.
Criteria for judging assurance activities
Levels
Content of activities
High
Medium
Low
Structure of activities
High
Medium
Low
the different assurance activity levels in the assessment grid
(Table 2), and will be described in more detail below.
The principal assumption underlying the diagnostic in-
strument is that assurance activities performed on a higher
level will provide better guarantees to meet food safety re-
quirements due to a more effective and reliable FSMS.
Assessment of ‘defining system set-up’
Table 2 shows the overall grid to assess the core assur-
ance control activities in a company’s FSMS at different
levels. To assess ‘defining system set-up’ the two activities
‘translating external requirements’ and ‘using feedback in-
formation and data’ are distinguished.
Typical content elements, representative for the high
level, are specific information and scientific knowledge
(Table 1). In the grid (Table 2), for the core assurance activity
‘defining system set-up’ we specified these representative
elements to aspects like ‘evaluated on own specific food
production characteristics’, ‘analysis of changes in stake-
holder requirements’, and ‘information from validation
and verification reports’. Typical structure elements, repre-
sentative for the high level, are criticising, procedure driven
activities and systematic, fully independent. This is specified
in the grid for ‘defining system set-up’ in aspects like ‘eval-
uated on critical aspects’, ‘presence of procedures with as-
signed responsibilities’, ‘systematic analysis’, and ‘well
documented’. Medium levels are associated with the con-
tent elements standard information and expert knowledge,
which are indicated in the grid with aspects like ‘standard
data from food production system’, ‘changes when new
standards are officially required’, ‘acting on changes’, and
‘using external expert support’. Typical structure elements,
representative for the medium level are analysing, feedback
driven activities, and regular, partly independent, which is
denoted in Table 2 with aspects like ‘regular analysis’, ‘pro-
cess and product data’, ‘not systematically documented’,
and ‘external expert support’.
Low levels typically are associated with the content
elements general information and historical data, which
are specified in the grid (Table 2) as, hardly information
(not explicitly mentioned), ‘reactive translation of require-
ments’, ‘as perceived by stakeholders’, and use of own
Validity Reliability
Specificity of information Knowledge base
Specific information Scientific knowledge
Standard information Expert knowledge
General information Historical data
Method Controllability
Criticising, procedure driven Systematic, fully independent
Analysing, feedback driven Regular, partly independent
Checking, problem driven Ad hoc, not independent
 P.A. Luning et al. / Trends in Food Science & Technology 20 (2009) 300e312 305

Table 2. Grid to assess core assurance activities in a company’s FSMS.

Indicator Assumed mechanism Low level Medium level High level

Grid to assess ‘defining system set-up’
Sophistication of Systematic and precise - translation of external -
translation of external - pro-active translation of ex-
translating external translation of external assurance activities initiated
assurance activities by ternal assurance require-
requirements into requirements will result in by food safety performanceactively acting on changes ments based on systematic
internal FSMS suitable requirements on problems (reactive) as per-
in external assurance analysis of possible changes
system FSMS, which will positively ceived by stakeholders and -
setting (new) requirements in stakeholder requirements
requirements contribute to product safety or due to external directives
with support of external (e.g. new legislation, new
assurance - only necessary changes experts (e.g. consultants) branch demands)
- evaluated on critical aspects
of own food production sys-
tem; well documented
Extent of systematic Systematic use of valid - ad hoc modification of sys- - regular use of standard data - systematic analysis of infor-
use of feedback feedback information will tem initiated by problems from food production system mation from validation and
information to result in appropriate FSMS from own food production (process/product data, final verification reports, transla-
improve FSMS system modifications, which system, complaints product data) tions into concrete FSMS
will positively contribute to - not documented - modifications mainly fo- - modifications are estab-
product safety assurance cused on control activities in lished in clear procedures
production system, not sys- with assigned responsibili-
tematically documented ties; well documented
Grid to assess ‘validation activities’
Sophistication of A scientific evidence-based, - effectiveness preventive - effectiveness preventive - effectiveness preventive mea-
validating systematic, and independent measures validated based on measures validated based on sures validated based on spe-
- preventive equip- validation of effectiveness of historical experience, data, expert knowledge, regula- cific scientific sources (like
ment and facilities selected preventive measure - not explicitly judged by tory documents, and histori- scientific data/literature on
- sanitation and will result in an effective experts, cal results validation studies, predictive
personal hygiene control system, which will - on ad hoc basis, and findings- by (internal) expert modelling), historical results,
programs contribute product safety scarcely (not) reported - on regular basis and after and own experimental trials;
assurance system modifications; - by independent experts,
findings documented - on regular basis and after
(e.g. expert report) system modifications, find-
ings reported and activities
in well-documented
procedures
Sophistication of Similar as for preventive - similar for intervention sys- - similar for intervention - similar for intervention sys-
validating measures tems as for preventive systems as for preventive tems as for preventive
effectiveness measures measures measures
intervention
equipment and
intervention methods
Sophistication of A scientific evidence-based, - validation based on histori- - validation based on com- - validation based on specific
validating systematic, and independent cal experience and/or com- parison with regulatory scientific sources (reviews,
monitoring validation of CCP mon knowledge, documents (like specific historical data on hazards,
systems (CCP and determination and - not explicitly judged by hygiene codes), reports foodborne illnesses,
control points) establishment of control experts, - by expert data on survival or multipli-
circles will result in an - ad hoc basis; findings (not) - on regular basis; findings cation, epidemiology, etc.),
effective FSMS, which will scarcely reported documented (e.g. expert - by independent expert
positively contribute to report) - on regular basis and after
product safety assurance system modifications; find-
ings reported and activities
in well-documented
procedures
(continued)
306 P.A. Luning et al. / Trends in Food Science & Technology 20 (2009) 300e312

Table 2 (continued )
Indicator Assumed mechanism Low level Medium level High level
Grid to assess verification activities
Extent of verifying A specific, systematic, and - verification of procedures - verification of procedures - verification of procedures
people related independent verification of and compliance based on and compliance on analy- and compliance based on
performance procedure characteristics and checking presence of sing procedures (both con- analysing procedures and
- procedure compliance will result in a procedures and records, tent and presence) and records, and observations,
characteristics reliable FSMS, which will on ad hoc basis records, - by independent expert
- procedure positively contribute to - by own people who - by internal expert - with defined frequency and
compliance product safety assurance execute system, - based on regular basis, when system modifications,
- not documented findings in internal report findings reported and activi-
ties in well-documented
procedures
Extent of verifying A specific, systematic, and - verification of performance - verification of performance - verification of performance
equipment and independent verification of based on checking if prod- based on analysing standard based on analysing perfor-
methods equipment and methods’ uct, process parameters are performance records (e.g. mance records, calibration
of related performance will result in a correctly set (e.g. of equip- control charts, records data activities, and confirmation
performance reliable FSMS, which will ment, facilities, measuring, loggers, etc.) and calibration of performance by actual
prevention and contribute to product safety analysis methods), activities, restricted testing (e.g. microbial) testing
intervention assurance - by own people who execute of actual performance - by independent experts
equipment/method system not documented - by internal expert, - with defined frequency and
measuring/analysis - on regular basis, findings when system modifications,
equipment in internal report findings reported and activi-
ties in well-documented
procedures
Grid to assess documentation and record keeping to support food assurance
Appropriateness of An integrated, kept-up-to- - no structured documenta- - structured documentation - structured documentation
documentation date and accessible tion system. system, de-centrally organ- system, centrally organised,
system documentation system will - no access to external ised and - kept-up-to-date with as-
improve effective information sources. - kept-up-to-date, (partly) signed responsibilities, auto-
supply to FSMS, which will automated, available via mated and on-line available
better support validation and specific persons for all,
verification activities, which - access to external sources - with access to external
will positively contribute to not formalized (individual sources of information (li-
product safety assurance contacts) braries, databases)
Appropriateness of A structured, integrated, and - ad hoc registration of - full registration of critical - full registration of critical
record-keeping accessible record-keeping record keeping data product and process data product and process data, in
system system will better support in separated systems (not central integrated system,
validation and verification integrated) - on-line available and acces-
activities, which will positively - accessible via specific sible to all persons
contribute to product safety (authorised) persons
assurance

knowledge (not explicitly mentioned). Low levels are
related to the structure elements checking, problem driven
activities, and ad hoc, not independent, which are specified
in aspects such as ‘problem driven translation’, ‘initiated
by food safety performance problems’, ‘problem driven
modification’, ‘ad hoc modification’, ‘not documented’,
methods not scientific or expert based (not explicitly
mentioned).
When the company’s system set-up is on systematic
translation of assurance requirements and feedback infor-
mation based, then it is a medium or high level. Typical
for the high level is critical evaluation for own production
circumstances and use of verification and validation infor-
mation, and that activities are fully documented.
In practice, companies do face difficulties with the trans-
lation of stakeholder assurance requirements into ones own
x, Yüksel, & Çavus
specific control system set-up (e.g. Bas xoğlu,
2007; Konecka-Matyjek, Turlejska, Pelzner, & Szponar,
2005; Orriss & Whitehead, 2000). Moreover, companies
often have to deal with various stakeholders with
different, sometimes even conflicting, assurance require-
ments, which may complicate the translation into the
company’s specific control system set-up (Cairns et al.,
2005).
Assessment of validation activities
Table 2 also shows the grid to assess the ‘extent of
validation of core control activities’. Like described above,
high levels correspond with the content elements specific
information and scientific knowledge (Table 1), which we
specified for validation activities to typical aspects like
‘specific scientific sources’, and ‘experimental trials for
own production circumstances’. Typical structure elements
are criticising, procedure driven activities, and systematic,
 P.A. Luning et al. / Trends in Food Science & Technology 20 (2009) 300e312
fully independent activities, which are denoted for valida-
tion activities in the typical aspects ‘activities according
to procedures’ ‘regular basis’, ‘after system modifications’,
‘well documented’ (both findings and activities), and ‘inde-
pendent experts’. Medium levels of validation correspond
with content elements standard information and expert
knowledge, which is in the grid reflected in the aspects ‘his-
torical results’, ‘use of regulatory documents’, ‘use of
expert knowledge’. Medium levels correspond with the
structure elements analysing, feedback driven activities,
which is in the grid indicated with ‘comparison with regu-
latory documents’, ‘after system modifications’, whereas
regular, partly independent activities, refer to ‘regular ba-
sis’, ‘experts opinion’, and ‘reporting of findings’. Low
levels are typically associated with content elements gen-
eral information, historical data, in the grid specified as
‘common knowledge’, and ‘use of historical experience’.
Low levels are associated with structure elements checking,
problem driven activities, and ad hoc, not independent,
which are detailed in the grid as ‘data not judged’, ‘ad
hoc basis’, ‘ad hoc activities’, ‘findings not (scarcely) docu-
mented’, and ‘not explicitly judged by experts’ (Table 2).
When in a company’s FSMS core control activities are
validated by experts using expert knowledge and or scientific
sources on a regular basis and findings are reported, then me-
dium or high level. Crucial for the high level is involvement
of independent experts, and that actual effectiveness is with
scientific experimental trials tested and that validation activ-
ities are in well-documented procedures established.
There are no univocal approaches for validation.
Although, the importance of scientific-based validation is
evident from literature, a major source of information to
support validation activities is the use of scientific publica-
tions that documented effectiveness of specific control
measures (CAC, 2008; Jacxsens et al., 2009; Mayes, 1999;
Scott, 2005). These publications maybe specific validation
studies, that evaluate the effect of defined parameters on
the hazards of concern (like, Baker, 2002; Dormedy, Bra-
shears, Cutter, & Burson, 2000; Hanes et al., 2002; Leaper
& Richardson, 1999; Srikaeo & Hourigan, 2002). However,
also scientific information that outlines minimum and max-
imum values of control parameters (such as temperature,
aw, and pH values for growth, inactivation) is useful to un-
derpin validation. Moreover, extensive survey data can
serve as a basis for determining the required level of inac-
tivation or control (Mayes, 1999; Scott, 2005). Panisello,
Rooney, Quantick, & Stanwell-Smith (2000) emphasised
that incorporation of epidemiological data in the documen-
tation of the HACCP system provides assurance that the
system is based on best scientific information available.
Another important approach to validation is to conduct
scientifically valid experimental trials to demonstrate actual
adequacy of control measures, also stated by Codex
Alimentarius (CAC, 2008). Various authors discussed the
importance of microbial data for validation purposes.
This maybe laboratory challenge testing to assess the effect
307
of a control measure, it may be an in-plant challenge test
using surrogate micro-organisms, or in-plant trials, or mi-
crobiological tests to assess effectiveness of control strate-
gies (Brown et al., 2000; CAC, 2008; Kvenberg &
Schwalm, 2000; Martins & Germano, 2008; Scott, 2005;
Swanson & Anderson, 2000). However, there are also
considerations on microbiological tests. For example,
challenge studies using pathogens are usually restricted to
a laboratory environment because real testing in practice
bears the risk of contaminating the process environment.
In addition, selection of non-pathogenic surrogates is not
simple, and challenge tests are often based on worst case
scenarios which is satisfactory in the context of safety but
induce negative impact in terms of quality (Jay, Loessner,
& Golden, 2005; Leaper & Richardson, 1999; Scott, 2005).
A more recent approach to scientifically support valida-
tion is the use of microbiological growth and or inactivation
models (e.g., Pathogen Modelling Program, UK model
FoodMicroModel) to validate whether in-plant processes
are controlling foodborne pathogens effectively (Bull,
Szabo, Cole, & Stewart, 2005; Devlieghere, Francois, De
Meulenaer, & Baert, 2006; McMeekin et al., 2006; Wang,
Amezquita, & Weller, 2006). The above approaches sup-
port a scientific-based validation and such approaches are
typically considered for the high levels (Table 2).
Another source of information for validation purposes is
regulatory documents (CAC, 2008; Jacxsens et al., 2009;
Scott, 2005). Government agencies may develop regulatory
requirements or produce guidance documents (which are
generally based on scientific information) that when fol-
lowed, are considered to be validated control measures. Al-
though such documents are supported by scientific evidence
they have a generic character and need thus to be tested for
the company specific production circumstances to be sure
about effectiveness. An example of such governmental doc-
uments is the USDA lethality guidance (USDA FSIS,
1999), which specifies time and temperature standards for
meat and poultry. Also in Europe, sector organisations are
publishing similar guidance documents to assist their indus-
tries in implementing and validating FSMS. Several guid-
ance documents are as well recognized by European
Commission and available on their website (http://ec.euro-
pa.eu/food/food/biosafety/hygienelegislation/register_national_
guides_en.pdf).
We consider use of historical and or company
knowledge for validation as low level, because it lacks
structured scientific support. According to Scott (2005),
only in some cases historical knowledge alone can serve
as the basis for a validated control measure. For example,
a typical dogma in canning industry is that a pH of 4.6 or
less controls germination and outgrowth of proteolytic
Clostridium botulinum spores, and that a heat process
with an F0 of 3 will reduce spores sufficiently. Moreover,
ILSI (2004) emphasised that food-processing facilities
have been producing safe food for many years before
the advent of the HACCP-based systems. HACCP plans
308 P.A. Luning et al. / Trends in Food Science & Technology 20 (2009) 300e312
should thus not conflict with those systems but effectively
should use practical experience gained over time, histori-
cal results from previous quality systems, on-line QC
monitoring, consumer and customer complaints, and test-
ing of products maybe also used as evidence when evalu-
ating effectiveness of HACCP plans. However, they noted
the importance of quantifiable and objective data, which
underpins the importance of scientific evidence-based
and actual measurement of effectiveness (typical for
high level) (CAC, 2008).
Besides scientific based and structured, independency is
an important aspect for assurance because it prevents that
company interests may interfere with an objective valida-
tion. Validation studies are commonly conducted by con-
tract laboratories, universities, by equipment or ingredient
manufacturers or in-house, and many validation studies
are partnerships between companies and trade associations,
companies and universities, or companies and suppliers
(Jacxsens et al., 2009; Scott, 2005).
In current food businesses, validation activities are often
still based on custom and practice rather than on evidence
and or not (yet) well elaborated (Dzwolack, 2007; Taylor,
2001), although the importance of validation is recognized
(Panisello & Quantick, 2001).
Assessment of verification activities
Verification activities are aimed at judging the FSMS
afterwards. Similar as for validation, the quality of the in-
formation collected and used to judge the system is important.
High levels of verification correspond with content ele-
ments specific information and scientific knowledge (Table 1),
which is for verification activities specified in typical as-
pects like, ‘use of procedures, records, observations, perfor-
mance data’, and ‘confirmation of actual performance by
(microbiological) testing’. Typical structure elements are
criticising, procedure driven activities, and systematic, fully
independent, which are denoted for verification activities in
‘both analysis and observation of procedures and records’,
‘activities according to procedures’, ‘defined frequency’,
‘when system modifications’, ‘well-documented findings
and activities’, and ‘independent experts’ (Table 2). Me-
dium levels of verification correspond with content ele-
ments standard information and expert knowledge, which
are in the grid indicated by ‘standard performance records’,
and ‘use of internal experts’. Medium levels correspond
with structure elements analysing, feedback driven activi-
ties, reflected in ‘analysing presence and content of proce-
dures and records’, ‘restricted testing of actual
performance’, whereas the elements regular, partly inde-
pendent activities, refer to ‘regular basis’, ‘internal test
reports of findings’, and ‘ internal experts’. Low levels of
verification are typically associated with content elements
general information, historical data, which are in the grid
specified as ‘use of records, procedures, and parameter set-
tings’ and ‘own people’. The structure elements of low
levels are associated with checking, problem driven
activities, specified in the grid as ‘just checking of presence
of records, procedures, parameter settings’, whereby the
structure elements ad hoc, not independent, are denoted
as ‘ad hoc basis’, ‘not documented’, and ‘executed by
own people’ (Table 2).
When verification of performance of equipment and
methods is on expert analyses based on records, data,
calibration activities (etc.) on a regular basis and findings
are reported, then medium or high level. Crucial for the
high level is that, actual performance is by observations,
testing and or real measuring confirmed, and that verifi-
cation activities are established in well-documented
procedures.
Similar to validation, there are no univocal approaches
on how to verify critical control activities in an FSMS,
although various checklists and methods can be found in
official documents, internet, and literature (e.g. FDA,
2006; ILSI, 1999, 2004; ISO 22000:2005, 2005; ISO
9001:2008, 2008; Jacxsens et al., 2009). According to
Sperber (1998), important on-line verification activities
are observation and interview of the CCP monitor, and re-
view of CCP. ILSI (1999, 2004) mentions common ways of
gathering information for verification purposes like use of
data recorded at CCPs, results of on-line testing, end-prod-
uct testing, consumer and customer complaints, hygiene
testing, analysis of market place samples, and equipment
calibration checking.
Various studies discussed the importance of microbio-
logical testing for verification purposes. Kvenberg and
Schwalm (2000) argued that besides underpinning validity
of control strategies, microbiological testing is useful in
monitoring actual effectiveness of sanitation standard oper-
ating procedures, compliance of incoming ingredients with
safety criteria, safety of products held for corrective action,
and safety of finished products. Swanson and Anderson
(2000), discussed from an industry perspective, that testing
for pathogens in finished product (as verification) becomes
an ineffective means to assure process control, and sug-
gested that quantitative indicators can provide a much
more effective tool for verifying that FSMS is properly im-
plemented. The choice of appropriate indicators is product
and process specific. However, Cormier et al. (2007) argued
the need to monitor seafood HACCP-based programs by
microbiological testing of final products because informa-
tion of time series of audits and inspections showed to be
more indicative for control system performance than snap-
shot inspections of the final product. All studies underpin
the importance of actual measuring to verify the different
control strategies, as we also defined for the high level
(Table 2).
It should be noticed that new rapid assay methods are
now being developed to replace traditional microbiological
testing (e.g. Cocolin & Rantsiou, 2007; Miller, Keeton,
Acuff, & Prochaska, 2003), but these methods are often
not (yet) internationally approved and acknowledged for
verification purposes.
 P.A. Luning et al. / Trends in Food Science & Technology 20 (2009) 300e312
Independency is an important aspect of the quality of
verification activities similar to validation. Tanner (2000)
underpinned the importance of independency in judging or-
ganisations in achieving food safety and food law compli-
ance. Likewise, Nguyen, Wilcock, and Aung (2004)
emphasised in their explorative study on food safety and
quality systems, the importance of independence in verify-
ing the control system. It removes the bias that could result
if departments were to check their own work.
Verification is commonly perceived by companies as an
important but costly activity (Nguyen et al., 2004; Panisello
& Quantick, 2001). Moreover, in small companies and mi-
crobusiness where the manager is on site all times and has
visual confidence that the system is running according to
plan, they (often) perceive verification activities as useless
double checking exercise (e.g. Taylor, 2001).
Assessment documentation and record keeping
Table 2 also depicts the grid to assess ‘documentation
and record keeping’. As underpinned above, the quality
of collected information and data is crucial for appropriate
evaluation of the FSMS on its principal effectiveness and
proper execution and therewith assuring the compliance
to safety requirements of final products. Documentation
and record-keeping systems that are better able to provide
the right information and data at the right place at the right
moment will better support decision making processes in
the FSMS (Luning & Marcelis, 2009).
High levels of documentation and record keeping are
characterised by content elements specific information and
scientific knowledge, which is in the grid specified as
‘critical product and process data’, and ‘access to external
information sources’. Typical structure elements are criticising,
procedure driven activities, which refer in the grid to
‘kept-up-to-date’, and ‘assigned responsibilities’, whereby
the structure elements systematic, fully independent are in-
dicated as ‘structured’, ‘integrated systems’, and ‘available,
accessible to all’. Medium levels correspond with content
elements standard information, expert knowledge, specified
as ‘critical product and process data’, and formalized ac-
cess to external information sources. Medium levels corre-
spond with structure elements analysing, feedback driven
activities, referring to ‘kept-up-to-date’, and ‘partly auto-
mated’, whereas the elements regular, partly independent
activities are indicated by ‘structured’, ‘separated systems’,
‘restricted availability/accessibility’ (Table 2). Low levels
are typically associated with content elements general in-
formation and historical data, which is in the grid specified
as ‘ad hoc registration’. Low levels are also associated with
structure elements checking, problem driven activities, and
ad hoc, not independent which are detailed in the grid in
‘incomplete’, ‘not kept-up-to-date’, ‘ad hoc registration’,
‘not structured’, and ‘poor availability’ (Table 2).
Various studies have demonstrated and or argued that
documentation and record keeping are important in sup-
porting FSMS performance (e.g. Jacxsens et al., 2009;
309
Nguyen et al., 2004; Walker & Jones, 2002). However,
proper implementation and maintenance of such systems
is often an obstacle for companies in practice (Hielm, Tuo-
minen, Aarnisalo, Raaska, & Maijala, 2006; Nguyen et al.,
2004; Taylor, 2001; Walker et al., 2003). In practice, com-
panies already do have (automated) record-keeping systems
but these are separated systems linked to certain facilities or
equipment, like temperature data loggers of cooling sys-
tems (e.g. Walker et al., 2003). Such systems do support
verification activities but are not part of an integrated
company system collecting all crucial records and they
have restricted accessibility. Moreover, in companies, where
verbal communication plays a major role in management of
their business, they perceive documentation of procedures,
work instructions, (etc.) often as time consuming with a lot
of paperwork. According to Taylor (2001), in such situa-
tions companies should integrate necessary documentation
and record keeping into existing practice with minimal dis-
ruption. Worsfold (2006) concluded in a study in small food
businesses, that better guidance documents will help com-
panies to judge to what extent paperwork and records are
necessary for different sized businesses.
In summary, core activities, which are relevant for assur-
ance of food safety, are described and detailed into different
levels of execution. Literature has supported that these
activities are in practice executed but that companies (espe-
cially the smaller ones) still face difficulties in doing it
effectively and efficiently. The grids may support them in
assessing their current situation and will provide insight
in their assurance activities.
Usefulness of diagnostic instrument and future
perspectives
The dynamic environment wherein agribusinesses and
food companies operate and the high requirements on con-
trol and assurance of food safety require that companies
critically judge the performance of their FSMS and
improve it where necessary. An FSMS of a company is
an interpretation of a combination of various quality assur-
ance guidelines, standards, and legal requirements into
a specific system set-up. To obtain an indication of the sys-
tem performance, companies can audit and or certify (part)
their system against specific food QMS standards such as
BRC and ISO 22000:2005. Such audits and certification
give mainly insight into compliance to the specific stan-
dard. Some methods have been in literature reported to
assess systematically the performance of FSMS or QMS
(e.g. Van der Spiegel, Luning, Ziggers, & Jongen, 2005;
Walker & Jones, 2002). However, these methods focused
specifically on HACCP principles or on quality manage-
ment in a broader sense.
We have developed a diagnostic instrument that enables
assessment of core control and assurance activities in
a company specific FSMS, independent of the implemented
quality assurance guidelines and or standards. In this paper,
we discussed which core assurance activities should be
310 P.A. Luning et al. / Trends in Food Science & Technology 20 (2009) 300e312
included, how they contribute to assurance, and how the
actual assurance activities can be in a company judged.
The practical usefulness of the diagnostic instrument lies
in the following characteristics. The overall diagnostic in-
strument makes a clear distinction between core control
and assurance activities. The comprehensive list to check
core activities, will give a company insight in which type
of activities are addressed in their own FSMS. The grids
provide a nuanced assessment of the different activities,
which provides a company with insight in at what level
they execute their core assurance (and control) activities.
Therewith, it creates a basis for discussion about opportuni-
ties for improvement, which enables companies to take
strategic decisions on their FSMS.
The scientific usefulness is in the possibility to do a sys-
tematic and quantitative analysis of implemented FSMS in
different food sectors, for different business sizes, or differ-
ent chain actors, etc. It will contribute to a further under-
standing of performance of FSMS in practice. Parallel to
this study, a method has been developed to support micro-
biological assessment linked to typical elements of a com-
pany specific FSMS (Jacxsens et al., in press). Coupling
FSMS performance information to data about actual
microbiological product/process performance will give fur-
ther insight in which core activities are critically affecting
effectiveness of FSMS. These insights will form a sound
basis for development of improvement strategies to offer
companies opportunities for upgrading their own specific
system.
Acknowledgement
The research performed has been part of the project
FOOD-CT-2005-007081 (PathogenCombat) supported by
the European Commission through the Sixth Framework
Programme for Research and Development.
References
Ababouch, L. (2006). Assuring fish safety and quality in international
fish trade. Marine Pollution Bulletin, 53(10e12), 561e568.
Baker, D. A. (2002). Use of safety objectives to satisfy the intent of food
safety law. Food Control, 13(6e7), 371e376.
Bas
x, M., Ersun, A. S., & Kıvanç, G. (2006). The evaluation of food
hygiene knowledge, attitudes, and practices of food handlers’ in
food businesses in Turkey. Food Control, 17(4), 317e322.
Bas
x, M., Yüksel, M., & Çavus xoğlu, T. (2007). Difficulties and barriers for
the implementing of HACCP and food safety systems in food
businesses in Turkey. Food Control, 18(2), 124e130.
Beulens, A. J. M., Broens, D. F., Folstar, P., & Hofstede, G. J. (2005).
Food safety and transparency in food chains and networks e
relationships and challenges. Food Control, 16(6), 481e486.
BRC (2008). Global standard for food safety (issue 5). www.brc.org.uk
Brown, M. H., Gill, C. O., Hollingsworth, J., Nickelson, R., Seward, S.,
Sheridan, J. J., et al. (2000). The role of microbiological testing in
systems for assuring the safety of beef. International Journal of Food
Microbiology, 62(1e2), 7e16.
Bull, M. K., Szabo, E. A., Cole, M. B., & Stewart, C. M. (2005). Toward
validation of process criteria for high-pressure processing of orange
juice with predictive models. Journal of Food Protection, 68(5),
949e954.
Cairns, B., Harris, M., Hutchison, R., & Tricker, M. (2005). Improving
performance? The adoption and implementation of quality systems
in U.K. nonprofits. Nonprofit Management & Leadership, 16(2),
135e151.
Cenci-Goga, B. T., Ortenzi, R., Bartocci, E., de Oliveira, A. C.,
Clementi, F., & Vizzani, A. (2005). Effect of the implementation of
HACCP on the microbiological quality of meals at a university
restaurant. Foodborne Pathogens and Disease, 2(2),
138e145.
Cocolin, L., & Rantsiou, K. (2007). Rapid methods for detection and
prediction of the occurrence and virulence of pathogens in meat
and meat products. Meat Technology, 48, 116e122.
Codex Alimentarius Commission (CAC) (2003). Hazard analysis and
critical control point (HACCP) system and guidelines for its appli-
cation. ANNEX to recommended international code of practice/
general principles of food hygiene. CAC/RCP 1e1969, Rev 4. FAO/
WHO Codex Alimentarius Commission.
Codex Alimentarius Commission (CAC) (2008). Guidelines for the
validation of food safety control measures. CAC/GL 69. FAO/WHO
Codex Alimentarius Commission.
Cormier, R. J., Mallet, M., Chiasson, S., Magnusson, H., &
Valdimarsson, G. (2007). Effectiveness and performance of
HACCP-based programs. Food Control, 18(6), 665e671.
Da Cruz, A. G., Cenci, S. A., & Maia, M. C. A. (2006). Quality
assurance requirements in produce processing. Trends in Food
Science & Technology, 17, 406e411.
Devlieghere, F., Francois, K., De Meulenaer, B., & Baert, K. (2006).
Modelling food safety. In P. A. Luning, F. Devlieghere, & T. Verhé
(Eds.), Safety in agri-food chains (pp. 397e438). Wageningen:
Wageningen Academic Publishers.
Doménech, E., Escriche, I., & Martorell, S. (2008). Assessing the
effectiveness of critical control points to guarantee food safety.
Food Control, 19(6), 557e565.
Dormedy, E. S., Brashears, M. M., Cutter, C. N., & Burson, D. E. (2000).
Validation of acid washes as critical control points in hazard
analysis and critical control point systems. Journal of Food
Protection, 63(12), 1676e1680.
Duxbury, D. (2005). Food safety audits. Food Technology, 59(11),
62e65.
Dzwolack, W. (2007). Validating control measures in food safety
management systems. Medycyna Weterynaryjna, 63(9),
1021e1025.
Efstratiadis, M. M., Karirti, A. C., & Arvanitoyannis, I. S. (2000).
Implementation of ISO 9000 to the food industry: an overview.
International Journal of Food Sciences and Nutrition, 51(6),
459e473.
Escriche, I., Doménech, E., & Baert, K. (2006). Design and imple-
mentation of an HACCP system. In P. A. Luning, F. Devlieghere,
& T. Verhé (Eds.), Safety in agri-food chains (pp. 303e354).
Wageningen: Wageningen Academic Publishers.
Evans, J. R., & Lindsay, W. M. (2005). The management and control
of quality, (6th ed.). Ohio: Thomson Corporation South
Western.
Food and Drug Administration USFDA (2006). Managing food safety:
a regulator’s manual for applying HACCP principles to risk-based
retail and food service inspections and evaluating voluntary food
safety management systems. Annex 7-verification inspection
checklist. http://www.cfsan.fda.gov/wdms/hret3-a7.html
GFSI (2007). Global food safety initiative version 5. www.ciesnet.com
Hanes, D. E., Worobo, R. W., Orlandi, P. A., Burr, D. H.,
Miliotis, M. D., Robl, M. G., et al. (2002). Inactivation of Crypto-
sporidium parvum oocysts in fresh apple cider by UV irradiation.
Journal of Food Protection, 68(8), 4168e4172.
Hielm, S., Tuominen, P., Aarnisalo, K., Raaska, L., & Maijala, R.
(2006). Attitudes towards own-checking and HACCP plans among
Finnish food industry employees. Food Control, 17(5),
402e407.
 P.A. Luning et al. / Trends in Food Science & Technology 20 (2009) 300e312
Holt, G., & Henson, S. (2000). Quality assurance management in
small meat manufacturers. Food Control, 11(4), 319e326.
IFS (2007). International featured standards. http://www.food-care.info
ILSI (International Life Sciences Institute) (1999). Validation and
verification of HACCP. http://europe.ilsi.org/publications/Report þ
Series/haccpvalidation.htm
ILSI (International Life Sciences Institute) (2004). A simple guide to
understanding and application of hazard analysis and critical
control points concept, (3rd ed.).. http://europe.ilsi.org/file/
ILSIHACCP3rd.pdf
ISO 9000 (2005). Quality management systems e fundamentals and
vocabulary. http://www.iso.org/iso
ISO 22000:2005 (2005). Food safety management systems e require-
ments for any organization in the food chain. http://www.iso.org/
iso
ISO 9001:2008 (2008). Quality management systems e requirements.
27 pp.. ISO 9001:2008 TC 176/SC 2 quality management systems
e requirements http://www.iso.org/iso. [Accessed 31.10.2008]
Jacxsens, L., Devlieghere, F., & Uyttendaele, M. (2009). Quality
management systems in the food industry. Book in the framework
of Erasmus, ISBN 978-90-5989-275-0.
Jacxsens, L., Kussaga, J., Luning, P. A., Spiegel, M., Uyttendaele, M.,
DeVlieghere, F. A microbial assessment scheme to support micro-
bial performance measurements of food safety management sys-
tems [FoodMicro 2008 special issue]. International Journal of Food
Microbiology, in press, doi:10.1016/j.ijfoodmicro.2009.02.018.
Jay, J. M., Loessner, M. J., & Golden, D. A. (2005). Modern food mi-
crobiology, (7th ed.). Springer.
Khatry, Y., & Collins, R. (2007). Impact and status of HACCP in the
Australian meat industry. British Food Journal, 109(4e5),
343e354.
Konecka-Matyjek, E., Turlejska, H., Pelzner, U., & Szponar, L. (2005).
Actual situation in the area of implementing quality assurance
systems GMP, GHP and HACCP in Polish food production and
processing plants. Food Control, 16(1), 1e9.
Kvenberg, J. E., & Schwalm, D. J. (2000). Use of microbial data for
hazard analysis and critical control point verification e food and
drug administration perspective. Journal of Food Protection, 63(6),
810e814.
Kvenberg, J., Stolfa, P., Stringfellow, D., & Garrett, E. S. (2000). HACCP
development and regulatory assessment in the United States of
America. Food Control, 11(5), 387e401.
Leaper, S., & Richardson, P. (1999). Validation of thermal process
control for the assurance of food safety. Food Control, 10(4e5),
281e283.
Lee, J. A., & Hathaway, S. C. (2000). New Zealand approaches to
HACCP systems. Food Control, 11(5), 373e376.
Legnani, P., Leoni, E., Berveglieri, M., Mirolo, G., & Alvaro, N.
(2004). Hygienic control of mass catering establishments,
microbiological monitoring of food and equipment. Food Control,
15, 205e211.
Luning, P. A., Bango, L., Kussaga, J., Rovira, J., & Marcelis, W. J.
(2008). Comprehensive analysis and differentiated assessment of
food safety control systems: a diagnostic instrument. Trends in
Food Science & Technology. doi:10.1016/j.tifs.2008.03.005.
Luning, P. A., Devlieghere, F., & Verhé, R. (2006). Safety in agri-
food chains. Wageningen: Wageningen Academic Publishers.
681 pp.
Luning, P. A., & Marcelis, W. J. (2006). A techno-managerial approach
to food quality management. Trends in Food Science & Technol-
ogy, 17(7), 378e385.
Luning, P. A., & Marcelis, W. J. (2007). A food quality management
functions model from a techno-managerial perspective. Trends in
Food Science & Technology, 18(3), 159e166.
Luning, P. A., & Marcelis, W. J. (2009). Food quality management:
Techno-managerial principles and practices. Wageningen:
Wageningen Academic Publishers.
311
Luning, P. A., Van der Spiegel, M., & Marcelis, W. J. (2006). Quality
assurance systems and food safety. In P. A. Luning, F. Devlieghere,
& R. Verhé (Eds.), Safety in agri-food chains (pp. 249e302).
Wageningen: Wageningen Academic Publishers.
McMeekin, T. A., Baranyi, J., Bowman, J., Dalgaard, P., Kirk, M.,
Ross, T., et al. (2006). Information systems in food safety manage-
ment. International Journal of Food Microbiology, 112(3),
181e194.
Manning, L., Baines, R. N., & Chadd, S. A. (2006). Food safety man-
agement in broiler meat production. British Food Journal, 108(8),
605e621.
Mantovanelli, A., Marino, M., Comi, G., Vallavanti, W., & Dolzani, L.
(2001). Use of microbial analysis to test HACCP systems in food
industries. Industrie Alimentari, 40(406), 853e865.
Martin, T., Dean, E., Hardy, B., Johnson, T., Jolly, F., Matthews, F., et al.
(2003). A new era for food safety regulation in Australia. Food
Control, 14(6), 429e438.
Martins, E. A., & Germano, P. M. L. (2008). Microbiological indicators
for the assessment of performance in the hazard analysis and crit-
ical control points (HACCP) system in meat lasagne production.
Food Control, 19(8), 764e771.
Mayes, T. (1999). How can the principles of validation and verifi-
cation be applied to hazard analysis. Food Control, 10(4e5),
277e279.
Michaels, B., Keller, C., Blevins, M., Paoli, G., Ruthman, T., Todd, E.,
et al. (2004). Prevention of food worker transmission of foodborne
pathogens: risk assessment and evaluation of effective hygiene
intervention strategies. Food Service Technology, 4(1),
31e49.
Miller, D. R., Keeton, J. T., Acuff, G. R., & Prochaska, J. F. (2003).
Verification of cooking endpoint temperatures in beef by immu-
noassay of lactate dehydrogenase isozyme 5. Journal of Food
Science, 68(6), 2076e2079.
Mitchell, R. T. (1998). Why HACCP fails? Food Control, 9(2e3), 101.
Mortimore, S. (2000). An example of some procedures used to assess
HACCP systems within the food manufacturing industry. Food
Control, 11(5), 403e413.
Mortimore, S. (2001). How to make HACCP really work in practice.
Food Control, 12(4), 209e215.
Motarjemi, Y., & Käferstein, F. (1999). Food safety, hazard analysis and
critical control points and the increase in foodborne disease:
a paradox? Food Control, 10(4e5), 325e333.
Naugle, A. L., Barlow, K. E., Eblen, D. R., Teter, V., & Umholtz, R.
(2006). US Food Safety and Inspection Service testing for Salmo-
nella in selected raw meat and poultry products in the United
States, 1998 through 2003: analysis of set results. Journal of Food
Protection, 69(11), 2607e2614.
NFPA (National Food Processor Association) (2002). Validation
guidelines for automated control of food processing systems used
for the processing and packaging of preserved foods. Washington,
DC: NFPA.
Nguyen, T., Wilcock, A., & Aung, M. (2004). Food safety and quality
systems in Canada. An exploratory study. International Journal of
Quality & Reliability Management, 21(6), 655e671.
Orriss, G. D., & Whitehead, A. J. (2000). Hazard analysis and critical
control point (HACCP) as a part of an overall quality system in
international food trade. Food Control, 11(5), 345e351.
Panisello, P. J., & Quantick, P. C. (2001). Technical barriers to hazard
analysis critical control point (HACCP). Food Control, 12(3),
165e173.
Panisello, P. J., Rooney, R., Quantick, P. C., & Stanwell-Smith, R.
(2000). Application of foodborne disease outbreak data in the
development and maintenance of HACCP systems. International
Journal of Food Microbiology, 59, 221e234.
Ropkins, K., & Beck, A. J. (2000). Evaluation of worldwide approaches
to the use of HACCP to control food safety. Trends in Food Science
& Technology, 11(1), 10e21.
312 P.A. Luning et al. / Trends in Food Science & Technology 20 (2009) 300e312
Scott, V. N. (2005). How does industry validate elements of HACCP
plans? Food Control, 16(6), 497e503.
Sperber, W. H. (1998). Auditing and verification of food safety and
HACCP. Food Control, 9(2e3), 157e162.
Srikaeo, K., & Hourigan, J. A. (2002). The use of statistical process
control (SPC) to enhance validation of critical control points (CCPs)
in shell egg washing. Food Control, 13(4e5), 263e273.
Stecchini, M. L., & Del Torre, M. (2005). The food safety management
system. Veterinary Research Communications, 29(Suppl. 2), 117e121.
Sumner, J., Raven, G., & Givney, R. (2004). Have changes to meat and
poultry food safety regulation in Australia affected the prevalence
of Salmonella or of Salmonellosis. International Journal of Food
Microbiology, 92(2), 199e205.
Sun, Y. M., & Ockerman, H. W. (2005). A review of needs and current
applications of hazard analysis and critical control point (HACCP)
in foodservice areas. Food Control, 16(4), 325e332.
Swanson, K. M. J., & Anderson, J. E. (2000). Industry perspectives on the use
of microbial data for hazard analysis and critical control point valida-
tion and verification. Journal of Food Protection, 63(6), 815e818.
Tanner, B. (2000). Independent assessment by third-party certification
bodies. Food Control, 11(5), 415e417.
Taylor, E. (2001). HACCP in small companies: benefit or burden? Food
Control, 12(4), 217e222.
Taylor, E., & Kane, K. (2005). Reducing the burden of HACCP on SMEs.
Food Control, 16(10), 833e839.
Todd, E. C. D. (2003). Microbiological safety standards and public health
goals to reduce foodborne disease. Meat Science, 66, 33e43.
Trienekens, J., & Zuurbier, P. (2008). Quality and safety standards in
the food industry, developments and challenges. International
Journal of Production Economics, 113, 107e122.
USDA FSIS (United States Department of Agriculture Food Safety and
Inspection Service) (1999). Compliance guidelines for meeting le-
thality performance standards for certain meat and poultry prod-
ucts. www.fsis.usda.gov/OPPDE/rdad/FRPubs/Docs_95-0033f.htm
Van der Meulen, B., & Van der Velde, M. (2004). Food safety law in
the European Union: an introduction. Wageningen: Wageningen
Academic Publishers.
Van der Meulen, B., & Van der Velde, M. (2006). Modern European
food safety law. In P. A. Luning, F. Devlieghere, & R. Verhé (Eds.),
Safety in agri-food chains (pp. 559e618). Wageningen: Wagenin-
gen Academic Publishers.
Van der Spiegel, M., Luning, P. A., de Boer, W. J., Ziggers, G. W., &
Jongen, W. M. F. (2006). Measuring effectiveness of food quality
management in the bakery sector. Total Quality Management and
Business Excellence, 17(6), 1e19.
Van der Spiegel, M., Luning, P. A., Ziggers, G. W., & Jongen, W. M. F.
(2003). Towards a conceptual model to measure effectiveness of
food quality systems. Trends in Food Science, 14(10),
424e431.
Van der Spiegel, M., Luning, P. A., Ziggers, G. W., & Jongen, W. M. F.
(2005). Development of the instrument IMAQE-Food to measure
effectiveness of quality management. International Journal of
Quality and Reliability Management, 22(3), 234e255.
Vela, A. R., & Fernández, J. M. (2003). Barriers for the development
and implementation of HACCP plans: results from a Spanish
regional survey. Food Control, 14(5), 333e337.
Violaris, Y., Bridges, O., & Bridges, J. (2008). Small businesses e big
risks: current status and future direction of HACC in Cyprus. Food
Control, 19, 439e448.
Walker, E., & Jones, N. (2002). An assessment of the value of docu-
menting food safety in small and less developed catering services.
Food Control, 13(4e5), 307e314.
Walker, E., Pritchard, C., & Forsythe, S. (2003). Hazard analysis critical
control point and prerequisite programme implementation in small
and medium size business. Food Control, 14(3), 169e174.
Wallace, C. A., Powell, S. C., & Holyoak, L. (2005). Development of
methods for standardised HACCP assessment. British Food Journal,
107(10), 723e742.
Wang, L. J., Amezquita, A., & Weller, C. L. (2006). A mathematical
model for the validation of safe air-blast chilling of cooked hams.
Transactions of the Asabe, 49(5), 1437e1446.
Worsfold, D. (2006). Cyngor i Fusnesion Bwyd e advice for food
businesses in Wales. Nutrition & Food Science, 36(6),
419e428.