IT-Monitoring 6

IFAC Information Technology Guideline 6
Information April 2002
Technology
Committee
IT Monitoring
Issued by the
International
Federation of
Accountants
This Guideline of the Information Technology Committee was
approved for publication by the International Federation of Accountants
in April 2002.
In this guideline series, the International Federation of Accountants,

through its Information Technology Committee, seeks to promote
executive understanding of the key issues affecting the management of
information and communications. This series of guidelines is written
for management.
This guideline is the sixth of the series and covers Information

Technology Monitoring. In addition to providing an approach to
developing effective information technology plans, it provides an
understanding of the nature and importance of information technology
planning.
IFAC welcomes any comments you may have. Comments should be

sent to:
Technical Director
International Federation of Accountants
535 Fifth Avenue, 26th Floor
New York, NY 10017 USA
Fax: (212) 286-9570
Copies of this paper may be downloaded free of charge from the IFAC
website at http://www.ifac.org.
The approved text of this Guideline is that published in the English

language.
Copyright © April 2002 by the International Federation of Accountants.

All rights reserved. No part of this publication may be reproduced,
stored in a retrieval system, or transmitted in any form or by any
means, electronic, mechanical, photocopying, recording, or otherwise,
without the prior written permission of the International Federation of
Accountants.
ISBN 1-887464-84-0.
Contents
PAGE
Contents..................................................................................................3
PREFACE.............................................................................................. 5
EXECUTIVE SUMMARY................................................................................ 1
WHY MONITOR IT?.............................................................................. 1
WHAT IS IT MONITORING?.....................................................................1
HOW DOES MANAGEMENT MONITOR IT?...................................................2
KEY/CORE PRINCIPLES.............................................................. 2
WHAT TOOLS ARE AVAILABLE?............................................................... 3
WHAT IS THE BEST APPROACH?............................................................... 3
WHEN?...............................................................................................4
WHO?................................................................................................ 4
WHY IS MONITORING OF IT IMPORTANT?................................................... 5
WHAT IS IT MONITORING?........................................................................ 8
HOW DOES MANAGEMENT MONITOR IT?................................................. 13
KEY/CORE PRINCIPLES:........................................................... 13
WHAT IT MONITORING TOOLS ARE AVAILABLE?.......................................14
WHAT IS THE BEST APPROACH TO IMPLEMENT IT MONITORING?...............19
APPROACH - PHASE I: ORIENTATION................................................19
APPROACH - PHASE II: CRITERIA DEFINITION...................................20
APPROACH - PHASE III: ONGOING MONITORING............................... 21

APPROACH - PHASE IV: SEPARATE PERIODIC AND AD HOC MONITORING
........................................................................................................21
APPROACH - PHASE V: SUBSEQUENT ACTIONS..................................21
APPROACH - PHASE VI: REPORTING............................................... 22
WHEN?................................................................................................ 22
WHO?.................................................................................................. 22
Control Objectives for the IT Monitoring Process Domain .................23
M1 – MONITORING THE PROCESSES...........................................................23
HIGH LEVEL CONTROL OBJECTIVE.......................................................... 23
DETAILED CONTROL OBJECTIVES............................................................23
MANAGEMENT GUIDELINES..................................................................24
M2 – ASSESSING INTERNAL CONTROL ADEQUACY........................................ 26
MANAGEMENT GUIDELINES..................................................................27
M3 – OBTAINING INDEPENDENT ASSURANCE............................................... 29
MANAGEMENT GUIDELINES.................................................................. 31
SECURITY AND RISK MONITORING.............................................................33
ADDITIONAL IT MEASURES TO MONITOR.................................................. 35
KEY GOAL INDICATORS (OR OUTCOME MEASURES).................................. 35
KEY PERFORMANCE INDICATORS (PERFORMANCE DRIVERS)..........................36

PREFACE
In a digital world, the management and use of information, information
systems and communications are of crucial importance to the success
of an organization. This arises from the:
increasing dependence on information and the systems and
communications that deliver the information;
scale and cost of current and future investments in information; and
potential of technologies to dramatically change organizations and
business practices, create new opportunities and reduce costs.
Many organizations recognize the potential benefits that technology

can yield. But, with those potential benefits come certain risks. To be
successful, organizations must also understand and manage the risks
associated with implementing new technologies. To provide effective
direction and adequate control, therefore, executive management needs
to have an appreciation of the benefits, risks and constraints of
information technology (IT).
In this series of guidelines, the International Federation of

Accountants’ Information Technology Committee seeks to promote
executive understanding of key issues affecting the management of
information and communications. This series of guidelines is written
for management.
This guideline is the last of the series and covers managing IT

monitoring. In addition to emphasizing the nature and need for IT
monitoring and its impact on IT governance, this guideline provides an
understanding of the main principles on which IT monitoring should be
founded, and a generic approach for implementing effective IT
monitoring.
Executives in various capacities (for example, accountants, financial

controllers, auditors or business managers) are frequently asked to
manage, participate in, assess or comment on the IT monitoring
process. They can do this only if they have a sound knowledge of the
principles and practices required to manage IT monitoring.
IFAC’s Information Technology Committee would like to acknowledge

the support from the Information Systems Audit and Control
Association (ISACA) and thank its various contributors who provided
valuable input for this document:
Susan M. Caldwell, ISACA
Erik Guldentops, IT Governance Institute
John W. Lainhart IV, PricewaterhouseCoopers
Akira Matsuo, Chuo Aoyama Audit Corporation
Ron Saull, Great West Life Assurance Company/London
Life/Investors Group
Michael Schirmbrand, Ernst & Young
Executive Summary
Executive Summary
Why monitor IT?

1. Information, and the systems and communications that deliver
the information, are truly pervasive throughout today’s
organizations. Executive management has a responsibility to
ensure that the organization monitors its use of information
and information technology (IT). IT monitoring is
fundamental to IT governance and part of management’s
stewardship responsibility to make certain that what was
agreed to be done is being done and is being done in line with
directions and policies set by the board. Monitoring is needed
to make sure that those to whom responsibility has been
delegated are doing the right things, are doing them right and
can be held accountable if they do not.
What is IT monitoring?
2. Those responsible for IT governance need first to set
measurable goals, then delegate the execution to executive
management and, finally, regularly verify that performance
matches the goals. If goals and measures are not in line, the
governance body needs to take corrective action, provide
redirection or, possibly, reconsider the original goals.
Monitoring of IT is enabled by the definition of relevant
performance indicators, the systematic and timely reporting of
performance and prompt acting on any deviations identified.
IT monitoring is especially important because of the
complexity and risk involved in IT activities. It has the
business goals of ensuring the delivery of information to help
the organization achieve its objectives and ensuring the
achievement of performance objectives for the IT function.
IT monitoring covers:
 how IT sustains the business with operational processes
and risk and control systems;
 whether IT complies with business strategy, standards and
policy;
1
IT Monitoring
 how IT improves the business with technology, process

and organizational changes; and
 how IT supports enterprise growth through process
knowledge and service capability.
How does management monitor IT?

3. While IT monitoring processes are unique to the needs and
circumstances of each organization, they are generally
developed using seven core principles.
KEY/CORE PRINCIPLES
 COMPREHENSIVENESS— Any monitoring activity has
to be comprehensive, based on simple and consolidated
measures focusing on exceptions.
 RELEVANCE — Any monitoring activity has to be
relevant to the mission, vision, goals and strategy of the
enterprise.
 ACCEPTABILITY — An effective monitoring approach
has to be acceptable to those being monitored. This means
not invading their privacy and not intruding into their
day-to-day responsibilities.
 TIMELINESS — To make correct and expedient
decisions, monitoring data must be available to detect
deviations that need to be reported immediately.
 VERIFIABILITY — Information obtained by the
monitoring process should be verifiable by other means –
thus, it should be accurate and, whenever possible, it
should be based on fact.
 ACTION-ORIENTED — Any form of monitoring must
enable expedient corrective action.
 FLEXIBILITY/ADAPTABILITY — The monitoring system
should be easily adaptable to provide accurate, relevant
and timely information in a changing environment.
2
Executive Summary
What tools are available?

4. While many IT monitoring tools are available, management is
effectively using seven key tools in performing IT governance
responsibilities:
 traffic light reports to follow up on projects and strategic
initiatives;
 performance management through balanced scorecards
(and dashboards);
 benchmarking for decision making relative to IT
investments for risk and control;
 active monitoring of the IT infrastructure;
 brainstorming for risk management and improvements;
 internal and external audit for independent assurance;
 management reporting for executive management review.
What is the best approach?

5. Although monitoring of IT is unique to the organizational
environment, the monitoring process and the underlying
activities are similar. Usually, the process consists of the
following six phases:
Phase I: Orientation. This start-up phase is required to
determine the scope of monitoring and the methodology and
techniques to be applied. In this phase, the resources required
for monitoring IT are mobilized.
Phase II: Criteria Definition. This phase is regularly covered
in the planning/design phase of each IT and business process.
Goals or performance measurement indicators are set up for
monitoring.
Phase III: Ongoing Monitoring. Ongoing monitoring is a
continuous supervisory function over key IT activities and
control processes. Exceptional events have to be identified and
tracked. Performance measures need to be established,
involving both IT and the stakeholders, aligned with the
strategy and reviewed on an ongoing basis.
3
IT Monitoring
Phase IV: Separate Periodic and Ad Hoc Monitoring.

Besides ongoing monitoring, separate periodic and ad hoc
monitoring is vital to ensure the ongoing monitoring and other
control functions operate properly, to periodically review IT-
related risks and opportunities and to obtain comfort relative
to major IT decisions. Periodic monitoring includes internal
audit procedures, external assurance, self-assessments and
brainstorming sessions.
Phase V: Subsequent Actions. Subsequent actions include
corrective actions to redirect IT activities and processes and
bring them back in line with goals, strategy and policy;
minimization of adverse effects; refinement of goals and
measures; changes to strategy, policy and standards; and
initiation of reassessment activities.
Phase VI: Reporting. For monitoring to be able to support
effective IT governance, management reporting about all
phases of the monitoring process, including subsequent
actions and escalation procedures, is an essential element of
the recurring/iterative control cycles.
When?
6. Monitoring is necessary always whenever IT is used within an
organization: from planning and organization, acquisition and
implementation to delivery and support. For effective IT
governance, monitoring is absolutely crucial.
Who?
7. Everyone who has a specific role and/or responsibility for
achieving IT goals and processes must be involved in
monitoring IT. Effective monitoring involves the entire
organization, as information is captured, consolidated and
reported up the various management levels.
4
Why is Monitoring of IT Important?

8. In a global information society, where information travels
through cyberspace on a routine basis, the significance of
information is widely accepted. In addition, information, and
the systems and communications that deliver the information,
are truly pervasive throughout organizations – from the user's
platform to local and wide area networks to servers to
mainframe computers. As such, information technology (IT)
has become an essential part of most enterprises, and
management increasingly needs to address:
 IT’s enabling capacity for new business models and
changing business practices;
 IT’s increasing costs and information’s increasing value;
 the risks of doing business in an interconnected digital
world and the dependence on entities beyond the direct
control of the enterprise;
 IT’s impact on business continuity due to increasing
reliance on information and IT in all aspects of the
enterprise;
 IT’s ability to build and maintain knowledge essential to
sustain and grow the business; and
 the failures of IT increasingly having a major impact on
reputation and enterprise value.
9. Boards and executive management generally expect their

organization’s IT to deliver business value, i.e., provide fast,
secure, high-quality development; generate maximum return
on investment; and move from efficiency and productivity
gains toward value creation and business effectiveness.
10. While some have been successful, many organizations find

that their expectations and reality do not always match.
Unfortunately, too often, boards and executives end up with:
 business losses, reputational damage and a weakened
competitive position;
 the failure of IT initiatives to deliver the innovation and
benefits they promised;
5
IT Monitoring
 inadequate or even obsolete technology;

 un-met deadlines and budget overruns.
11. Boards and management who exercise proper IT monitoring

often uncover and address problems in time to ensure a
successful outcome. Accordingly, executive management has
a responsibility to ensure that the organization monitors its use
of information and IT, to continuously check whether goals are
being achieved, value is being delivered to the business and
risks are mitigated.
12. Increasingly, top management is looking at IT performance.

This is best illustrated by the balanced scorecard, which
translates strategy into action to achieve goals with a
performance measurement system that goes beyond
conventional accounting. Balanced scorecards augment
traditional financial measures, incorporating measures for
those relationships and intangible (e.g., knowledge-based)
assets necessary to compete in the information age: customer
focus, process effectiveness and the ability to learn and grow.
13. Because it is so essential to an enterprise’s operations, IT

needs its own scorecard. Defining clear goals and good
measures that unequivocally reflect the business impact of the
IT goals is a challenge that needs to be resolved in cooperation
among the different governance layers within the enterprise.
6
14. The following illustration depicts some examples of outcome

and performance measures for the different dimensions of the
IT balanced scorecard. (More information on the IT balanced
scorecard is provided in the “Tools” section and an extra set of
possible IT metrics is provided in Appendix 3.)
Financial
# of IT
customers
Cost per IT
customer
Cost-
efficiency
of IT
processes
up
Delivery of
IT value
per
Custome employee
r
Process
Availability
Level of of systems
service and
delivery services
up Developmen
Satisfaction ts on
of existing schedule
customers and
Informati budget
# of new
customers on Throughput
reached and
# of new response
service times
delivery Amount of
channels errors and
rework
Learnin
g
Staff
productivity
and morale
# of staff
trained in
new
technologies
/services
Value delivery
per
employee up
Increased
availability
of 7
knowledge
systems
IT Monitoring
Control
Device
 Comparison
 with standard
Observed (assessor)
information about
what is happening  
(detector)
Entity being 


Controlled          
         
        
   
What is IT Monitoring?
15. Monitoring is fundamental to any control system. Monitoring
is the process of observing what is happening (detection) and
comparing it to a standard that has previously been set
(assessing). While strictly not part of monitoring, the
communication that results with the aim of altering observed
behavior may not be ignored.
16. In the context of IT governance, this means that measurable

goals need to be set first and then delegated to executive
management. Those responsible for IT governance need to
verify regularly that performance matches goals. If goals and
measures are not in line, the IT governance body needs to take
corrective action, provide redirection or, possibly, reconsider
the original goals.
8

 

17.
 



Set Comp 
measur are
result 
able s
goals  
  
  


IT monitoring is fundamental to IT governance; it is part

of management’s stewardship responsibility to make certain
that what was agreed to be done is being done. IT monitoring
is needed to make sure that those to whom responsibility has
been delegated are carrying it out correctly and can be held
accountable if they are not.
18. IT monitoring is enabled by the definition of relevant

performance indicators, the systematic and timely reporting of
performance and prompt acting on deviations.
19. Monitoring of IT is especially important because of the

complexity and risk involved in IT activities. It has the
business goals of ensuring the delivery of information to help
the organization achieve its objectives and ensuring the
achievement of performance objectives for the IT function.
20. The performance objectives are best illustrated by the IT

governance framework proposed by the IT Governance
9
IT Monitoring
Institute. That model implies that the governance entity needs

to monitor:
 whether IT delivers value in supporting the enterprise’s
strategy, in sustaining the enterprise on a day-to-day basis
and in enabling new products and services;
 whether IT’s risks are mitigated; and
 whether IT is in compliance with laws and regulations and
with standards and policies.
Provide
Direction
IT
Set
Activiti
Objectives
IT is aligned es
with the
business,
enables the Increase
business, automation
and (make the
maximizes business
benefits  effective)
IT resources  Decrease
are used cost (make
responsibly the
IT related enterprise
risks are efficient)
managed Manage risks
appropriately (security,
reliability and
compliance)
Measure
Performa
nce
21. Additionally, executives put control frameworks in place to

provide assurance that enterprise objectives are being met and
10
that agreed strategy is executed within established policies.

Hence, monitoring of the proper functioning of internal
control systems, as predicated by Internal Control –
Integrated Framework, issued in 1992 by the Committee of
Sponsoring Organizations of the Treadway Commission
(COSO),1 is an important, but not the sole, subject of IT
monitoring.
22. Management should establish the means for monitoring, either

through independent evaluations or ongoing, structured and
independent process checks:
 compliance with strategic goals;
 the achievement of tactical projects and activities;
 the performance of people, systems and processes;
 the proper functioning of the internal control systems;
 adherence to internal standards and policies; and
 observance of laws and regulations
23. Furthermore, it is essential to monitor the fulfillment of

committed improvements resulting from these monitoring
activities:
 process, systems and people performance improvement
programs;
 self-assessments;
 quality management;
 risk management; and
 internal and external audit.
24. Most monitoring activities have the ultimate goal of:

 sustaining the business;
 ensuring compliance; or
 improving the business.
1
COSO is a voluntary private sector organization dedicated to improving the quality
of financial reporting through business ethics, effective internal controls and corporate
governance.
11
IT Monitoring
25. Each of these goals is illustrated by specific monitoring

processes of the Control Objectives for Information and
related Technology (COBIT) Control Framework, i.e., M1 -
Monitor the Processes, M2 - Assess Internal Control Adequacy
and M3 - Obtain Independent Assurance.
26. A detailed description of the COBIT monitoring processes are

provided in Appendix 1, covering:
 high level control objectives;
 detailed control objectives;
 critical success factors (the most important things to do to
increase the probably of the IT process achieving its
goals);
 outcome measures of key goal indicators or KGIs
(measuring whether the process achieves its goals); and
 performance drivers of key performance indicators or
KPIs (measuring whether the process performs well).
27. These COBIT monitoring processes provide valuable tools for

management in assessing its effectiveness in managing the IT
monitoring process.
12
What IT Monitoring Tools are Available?
How Does Management Monitor IT?

28. While IT monitoring processes are unique to the needs and
circumstances of each organization, they are generally
developed using seven core principles.
KEY/CORE PRINCIPLES:
 COMPREHENSIVENESS — Any monitoring activity has
to be comprehensive, based on simple and consolidated
measures, focusing on exceptions.
 RELEVANCE — Any monitoring activity has to be
relevant to the mission, vision, goals and strategy of the
enterprise. Alignment of the IT strategy to the enterprise
strategy is a critical success factor for successful IT
governance.
 ACCEPTABILITY — An effective monitoring approach
has to be acceptable to those being monitored. This means
not invading their privacy and not intruding into their
day-to-day responsibilities. The “tone at the top” and
maturity level of the internal control systems are essential
to achieving acceptability.
 TIMELINESS — To make correct and expedient
decisions, monitoring data must be available to detect
deviations that need to be reported immediately. The
frequency of monitoring different activities of an
organization should be determined by considering the
risks involved and the frequency and nature of changes
occurring in the operating environment.
 VERIFIABILITY — Information obtained by the
monitoring process should be verifiable by other means.
Thus, it should be accurate and, whenever possible, it
should be based on fact. (It should be noted that obtaining
an opinion/feeling is also part of the governance process –
actually, future predictions to drive the strategy are not
always based on fact.)
 ACTION-ORIENTED — Any form of monitoring must
enable expedient corrective action. Executive
management must ensure that the monitoring function is
13
IT Monitoring
properly defined and structured within the organization to

take the actions needed.
 FLEXIBILITY/ADAPTABILITY — The monitoring system
should be easily adaptable to provide accurate, relevant
and timely information in a changing environment.
Boards of directors, audit committees and executive
management should obtain unbiased information about
line activities for dynamic, flexible and adaptable decision
making.

29. While many IT monitoring tools are available, management is
effectively using seven key tools in performing IT governance
responsibilities:
 Traffic light reports to follow up on projects and
strategic initiatives
Traffic light reports have become a preferred reporting
mechanism for executives and boards. They exemplify the
principle of comprehensiveness and the practice of
exception reporting. They provide a “green” condition
when the committed action is on schedule and on budget.
When issues are known that suggest that the commitment
might go over budget or schedule, or might not achieve
all of its objectives in the future, an “orange” condition is
reported. When budgets or schedules are exceeded, or
when it is clear that goals will not be achieved without
major changes or investments, then a condition “red” is
provided.
These commitments can be of different types, such as:
◦ a project,
◦ an improvement initiative, or
◦ the closure of an audit recommendation.
The improvement initiatives can result from process re-
engineering activities, risk brainstorming sessions, quality
reviews, control self-assessment, etc. It is evident that
orange and red conditions will require a short explanation
of status and corrective actions. It is also good practice for
14
the board and executive and operational management to

have clear agreements on when orange and red conditions
need to be raised and what needs to be reported when they
do occur.
 Performance management through balanced
scorecards (and dashboards)
F in a n c ia l
Me tives
Tar res
ves
Init ts
a su
ge
jec
iati
“ T o s u c c e e d f in a n c ia lly , h o w
Ob
d o e s I T c o n t r ib u te t o th e
o r g a n iz a tio n s s u c c e s s ? ”
C u s to m e r I n te r n a l B u s in e s s P r o c e s s
VV isis io io nnn
Me tives
“ T o a c h ie v e o u r “ T o s a tis f y o u r
Tar res
Me tives
ves
V is io
Tar res
ves
Init s
a su
jec
Init ts
get
aaa nnn ddd

iati
as u
v is io n , h o w s h a r e h o ld e r s a n d
jec
iati
ge
Ob
Ob
s h o u ld IT s u p p o r t c u s to m e rs , w h a t
o u r c u s to m e rs ? ” ssStttrraa ttee ggg yyy IT p r o c e s s e s
m u s t w e e x c e l a t? ”
L e a r n in g a n d G r o w th
M e t i v es
Tar res
ves
s
asu
j ec
get
iati
“ T o a c h ie v e o u r v is io n ,
Ob
Init
h o w w ill w e s u s t a in o u r
a b ilit y t o c h a n g e im p r o v e
o u r I T e n v ir o n m e n t? ”
IT management is increasingly applying the balanced

scorecard method (Kaplan, Norton) and performance
dashboards to the measurement of the value and overall
contribution delivered by IT. In this method, alignment is
achieved between business and IT plans through visible
alignment of IT goal (outcome) measures of IT processes
and the business goals they support. They also identify the
key performance indicators (drivers) of success for each
process.
15
IT Monitoring
Management implements a set of measurement and

monitoring activities to collect information on the
achievement of the outcomes using the goal measures and
on the performance of IT processes using the key
performance indicators. This information, and the
correlation of results over time, enable IT management to
determine whether its IT strategies and approaches are
effective and to decide on corrective or adjusting actions
as appropriate.
It is vital to note that the linkage between the business
balanced scorecard and the IT balanced scorecard is a
strong method of alignment. Many of the outcome
measures of IT influence how well the enterprise is doing
and are, therefore, performance measures for the
enterprise. It is equally vital to stress that the balanced
scorecard should demonstrate the value that IT delivers to
the enterprise.
 Benchmarking for decision making about IT
investments for risk and control
Maturity modeling and benchmarking are other
management practices for monitoring the return of
investment and risk mitigation of IT. Maturity models
provide for measurable, recognizable levels of maturity,
for example, in control maturity, risk management,
operational proficiency, etc. On these scales, an enterprise
can define where it is (As-Is) and where it wants to be
(To-Be). This can then drive strategy and help decision
making on improvement projects. Monitoring these
improvements and regularly reassessing the enterprise’s
maturity level through benchmarking (i.e., comparing it to
where others are) are becoming best practice in
management monitoring in the domain of IT.
 Active monitoring of the IT infrastructure
Concern for IT infrastructure risks at enterprise and
national/international levels has changed how IT security
and risk are being managed. Awareness that the traditional
approach of defining policy, selecting safeguards and
implementing them is too static for a highly volatile
environment has pushed organizations into a more fluid
16
continuous approach of actively monitoring the IT

infrastructure. This consists of continuously monitoring
and performing self-assessments to detect and then fix the
problems identified. Results of this monitoring activity
need to be brought to top management’s attention when
appropriate, and management needs to give guidance on
when notifying management is appropriate and when not.
Such systems will detect and stop unauthorized activity in
systems and networks on a 24/7 basis through constant
information gathering and analysis, looking for attack
signatures, viruses, vulnerabilities, non-compliance with
basic rules and misuse.
These systems need to be supplemented with appropriate
response procedures and, thus, provide for assurance
about the security of the IT infrastructure. They usually
are complemented with intrusion detection exercises,
testing the controls of the infrastructure and resulting in
exception reports to management.
 Brainstorming for risk management and
improvements
Apart from the above techniques, for the most part, only
high-risk enterprises will implement processes and assign
responsibilities to monitor enterprise risks on a
continuous basis. A cost-effective alternative exists in
well-prepared and facilitated risk brainstorming sessions
between top management and those responsible for IT,
security, risk and audit. These brainstorming sessions
might be performed, for example, on an annual basis and
the professionals involved should prepare and document a
list of the most important vulnerabilities and threats for
consideration.
These brainstorming sessions should result in clear
improvement actions and responsibility commitments.
Executive management and boards can then follow up
with traffic light reports (see above). Appendix 2 provides
additional guidance on security and risk monitoring that
should be useful in these brainstorming sessions.
 Internal and external audit for independent assurance
17
IT Monitoring
Internal and external audit reports are key monitoring

tools for executive management. These reports should
include a statement of the audit objectives, a description
of the audit scope and methodology, the period of
coverage and the nature and extent of the audit work
performed. The report should include a full discussion of
the audit findings and conclusions, the cause of the
problem areas noted in the audit and recommendations for
actions to correct the problem areas and improve
operations. The report should include a statement that the
audit was made in accordance with generally accepted
auditing standards and disclose, when applicable,
standards that were not followed. It should include the
pertinent views of responsible officials of the
organization, program, activity or function audited
concerning the auditors' findings, conclusions and
recommendations, and what corrective action is planned.
Although resolution of audit comments rests with
executive management, follow-up by audit staff is a
continuous process to determine if promised corrective
actions actually have been implemented. Auditors should
update information on previous findings, conclusions and
recommendations to determine whether appropriate
actions are being implemented in a timely manner.
 Management reporting
Management reporting is an essential element of IT
governance. Executive management should receive, for
review, reports on the organization’s progress toward
identified goals. It should also receive status reports on
the extent to which planned objectives have been
achieved, deliverables obtained, performance targets met
and risks mitigated. Once the reports have been reviewed,
management should ensure that any required remedial
actions are taken in a timely manner.
18
What is the Best Approach to Implement IT Monitoring?
What is the Best Approach to Implement IT

Monitoring?
30. Although monitoring of IT is unique to the organizational
environment, the monitoring process and the underlying
activities are similar. Usually, the process consists of the six
phases described below.
APPROACH - Phase I: Orientation

This start-up phase is required to determine the scope of
monitoring and the methodology and techniques to be applied.
In this phase the resources required for monitoring IT are
mobilized.
Determine scope: The scope of IT monitoring normally

includes the following key activities:
 determining if monitoring incorporates all business units
or that separate monitoring activities will be developed
for selected business units;
 assessing the quality of the IT monitoring performance
objectives set for each business unit; and
 evaluating the extent of independent assurance
involvement in IT monitoring.
 At the end of this step, the scope for IT monitoring will
have been determined.
Establish methodology/techniques and mobilize resources:

IT monitoring can be a time-consuming process depending on
the size of the organization and the scale of its current or
desired IT dependence. Once the scope has been determined,
the methodology and techniques need to be established, and
the background information and resources necessary for the
planning effort need to be mobilized, including a clear
delineation of reporting lines. Key activities include:
 Gathering necessary background information on the
organization, its IT profile and capabilities and IT
monitoring activities.
19
IT Monitoring
 Selecting a proven methodology to support the IT

monitoring activities. This methodology may be provided
by external consultants, internally developed or acquired
from a third party.
 Determining techniques that will be used for collecting
and analyzing information, including IT monitoring tools
(e.g., performance measurement, balanced scorecard,
benchmarking).
 Establishing an IT monitoring project team. Typically, this
will be a multidisciplinary team, comprising persons with
both IT and business skills. Management should ensure
that the team possesses the technical competence, and
skills and knowledge necessary to perform the needed
analyses in an efficient, effective and economical manner.
Frequently, the team is supplemented by external
consultants with expertise in IT monitoring.
 Formalizing the reporting mechanism for the project
team. Generally, the team reports to a steering committee
headed by the chief executive officer, chief information
officer or another senior business executive and
comprising key business unit managers, the IT manager
and an information systems audit manager.
APPROACH - Phase II: Criteria Definition

This phase is regularly covered in the planning/design phase
of each IT and business process. Goals or performance
measurement indicators are defined and established for IT
monitoring. Periodic re-evaluation of the performance
measures must also be a standard part of the criteria definition
phase, as these will change over time and they must be
updated to incorporate the changes.
For the IT and internal control processes, management should

ensure that relevant performance indicators (e.g., benchmarks)
from both internal and external sources are being defined, and
that data are being collected for the creation of management
information reports and exception reports regarding those
indicators. Controls also should be aimed at validating the
20
What is the Best Approach to Implement IT Monitoring?
propriety and integrity of both organizational and individual

performance measures and indicators.
APPROACH - Phase III: Ongoing Monitoring

Ongoing monitoring is a continuous supervisory function over
key IT activities and control processes. Exceptional events
have to be identified and tracked. Performance measures need
to be established, involving both IT and the stakeholders,
aligned with the strategy and reviewed on an ongoing basis.
IT services should be measured (key performance indicators

and/or critical success factors) by management and be
compared with target levels. Independent assessments and
evaluations of the IT should be performed on a continuous
basis to ensure IT’s continued effectiveness.
APPROACH - Phase IV: Separate Periodic and Ad Hoc

Monitoring
Besides ongoing monitoring, separate periodic and ad hoc
monitoring is essential to (1) ensure that ongoing monitoring
and other control functions operate properly, (2) periodically
review IT related risks and opportunities and (3) obtain
comfort relative to major IT decisions. Periodic monitoring
includes internal audit procedures, external assurance, self-
assessments and brainstorming sessions.
At regular intervals, management should measure customer

satisfaction regarding the IT services delivered to identify
shortfalls in service levels and establish improvement
objectives. Independent assessments and evaluations of IT
processes should be performed on a routine cycle to ensure
IT’s continued effectiveness.
APPROACH - Phase V: Subsequent Actions

Subsequent actions include corrective actions to redirect IT
activities and processes and bring them back in line with
goals, strategy and policy; minimization of adverse effects;
refinement of goals and measures; changes to strategy, policy
and standards; and initiation of reassessment activities.
21
IT Monitoring
Appropriate management action should be initiated to correct

deficiencies and to ensure that effective corrective actions are
taken in a timely manner.
APPROACH - Phase VI: Reporting

If monitoring is to support effective IT governance,
management reporting about all phases of the monitoring
process (including subsequent actions and procedures to alert
top management) is an essential element of recurring/iterative
control cycles.
Executive management should receive, for review, reports on

the organization’s progress toward identified goals.
Management should also receive status reports on the extent to
which planned objectives have been achieved, deliverables
obtained, performance targets met and risks mitigated.
WHEN?
31. Monitoring is necessary always whenever IT is used within an
organization: from planning and organization, acquisition and
implementation to delivery and support. Monitoring is crucial
for effective IT governance and occurs in a planned manner
whenever goals are being verified but also in a continuous and
ad hoc fashion when monitoring for risks, faults or defects.
WHO?
32. Chief executive officers, chief information officers, other
executive management, process owners, users and information
systems auditors all have roles and responsibilities in
monitoring IT’s goals and processes. An effective monitoring
system, like any successful management information system,
involves the whole organization in that monitoring
information is captured, consolidated and reported up, at all
levels.
22
M1 – Monitoring the processes
CONTROL OBJECTIVES FOR THE IT

MONITORING PROCESS DOMAIN
M1 – Monitoring the processes
High level control objective

Control over the IT process of
monitoring the processes
that satisfies the business requirement
to ensure the achievement of the performance objectives set
for the IT
processes
is enabled by
the definition of relevant performance indicators, the
systematic and
timely reporting of performance and prompt acting
on deviations
and takes into consideration
• scorecards with performance drivers
and outcome measures;
• customer satisfaction assessments;
• management reporting;
• knowledge base of historical
performance;
• external benchmarking.
Detailed control objectives
1. Collecting Monitoring Data

For the IT and internal control processes, management should ensure
that relevant performance indicators (e.g., benchmarks) from both
internal and external sources are being defined, and that data are being
collected for the creation of management information reports and
exception reports regarding those indicators. Controls also should be
aimed at validating the propriety and integrity of both organizational
and individual performance measures and indicators.
23
IT Monitoring
2. Assessing Performance
Services to be delivered by the IT function should be measured (key
performance indicators and/or critical success factors) by management
and be compared with target levels. Assessments of the IT function
should be performed on a continuous basis.
3. Assessing Customer Satisfaction

At regular intervals, management should measure customer satisfaction
regarding the services delivered by the IT function to identify shortfalls
in service levels and establish improvement objectives.
4. Management Reporting
Executive management should receive, for review, reports on the
organization’s progress toward identified goals. Management should
also receive status reports on the extent to which planned objectives
have been achieved, deliverables obtained, performance targets met and
risks mitigated. Once management has reviewed the reports, it should
take whatever appropriate action is deemed necessary.
Management Guidelines
Critical Success Factors
 Useful, accurate and timely management reports are available.
 Processes have defined and understandable key goal indicators and
key performance indicators.
 Measurements of IT performance include financial, operational,
customer and organizational learning criteria that ensure alignment
with organization-wide goals and that can be integrated with tools
such as the IT balanced business scorecard.
 There are clearly understood and communicated process
objectives.
 A framework is established for defining and implementing IT
governance reporting requirements.
 A knowledge base of historical performance is established.
24
M3 – Obtaining independent assurance
Key Goal Indicators

 Consistent application of the correct limited number of
performance indicators.
 Increased number of process improvement opportunities detected
and acted on.
 Satisfaction of management and the governance entity with
performance reporting.
 Reduced number of outstanding process deficiencies.
Key Performance Indicators

 Time lag between the process deficiency occurrence and reporting.
 Time lag between the reporting of a deficiency and action initiated.
 Ratio between process deficiencies reported and deficiencies
subsequently accepted as requiring management attention follow-
up (noise index).
 Number of processes monitored.
 Number of cause and effect relations identified and incorporated in
monitoring.
 Number of external benchmarks of process effectiveness.
 Time lag between business changes and any associated changes to
performance indicators.
 Number of changes to the set of performance indicators without
the business goals changing.
25
IT Monitoring
M2 – Assessing internal control adequacy

assessing internal control adequacy
to ensure the achievement of the internal control objectives set
for the
IT processes
is enabled by
the commitment to monitoring internal controls,
assessing their
effectiveness and reporting on them on a regular basis
• responsibilities for internal control;
• ongoing internal control monitoring;
• benchmarks;
• error and exception reporting;
• self-assessments;
• management reporting;
• compliance with legal and regulatory
requirements.
1. Internal Control Monitoring

Management should monitor the effectiveness of internal controls in
the normal course of operations through management and supervisory
activities, comparisons, reconciliations and other routine actions.
Deviations should trigger analysis and corrective action. In addition,
deviations should be communicated to the individual responsible for
the function and also to at least one level of management above that
individual. Serious deviations should be reported to senior
management.
26
2. Timely Operation of Internal Controls

Reliance on internal controls requires that controls operate promptly to
highlight errors and inconsistencies, and that these are corrected before
they have an impact on production and delivery. Information regarding
errors, inconsistencies and exceptions should be kept and
systematically reported to management.
3. Internal Control Level Reporting

Management should report information on internal control levels and
exceptions to the affected parties to ensure the continued effectiveness
of its internal control system. Actions should be taken to identify what
information is needed at a particular level of decision making.
4. Operational Security and Internal Control Assurance

Operational security and internal control assurance should be
established and periodically repeated, with self-assessment or
independent audit to examine whether or not the security and internal
controls are operating according to the stated or implied security and
internal control requirements. Ongoing monitoring activities by
management should look for vulnerabilities and security problems.

 Management clearly defines what components of the processes
need to be controlled.
 Internal control, compliance and internal audit responsibilities are
clearly understood.
 Competence and authority of the internal control compliance
function exist, addressing delegation as appropriate.
 A properly defined IT control process framework is in place.
 A clear process is used for timely reporting of internal control
deficiencies.
 Internal control monitoring data are accurate, complete and timely.
27
IT Monitoring
 There is management commitment to act on internal control

deficiencies.
 There is alignment with risk assessment and security processes.
 A process is in place to support knowledge sharing on internal
control incidents and solutions.
Key Goal Indicators

 Index of senior management satisfaction and comfort with
reporting on internal control monitoring.
 Decreased probability of internal control incidents.
 Positive external qualification and certification reports.
 Number of control improvement initiatives.
 Absence of regulatory or legal non-compliance events.
 Decreased number of security incidents and quality defects.

 Number and coverage of control self-assessments.
 Timeliness between internal control deficiency occurrence and
reporting.
 Number, frequency and coverage of internal compliance reports.
 Number of timely actions on internal control issues.
 Number of control improvements stemming from root cause
analysis.
28

obtaining independent assurance
to increase confidence and trust among the organization,
customers
and third-party providers
is enabled by
independent assurance reviews carried out at regular
intervals
• independent certifications and
accreditation;
• independent effectiveness evaluations;
• independent assurance of compliance
with laws and regulatory requirements;
• independent assurance of compliance
with contractual commitments;
• third-party service provider reviews and
benchmarking;
• performance of assurance reviews by
qualified personnel;
• proactive audit involvement.

1. Independent Security and Internal Control
Certification/Accreditation of IT Services
Management should obtain independent certification/accreditation of
security and internal controls prior to implementing critical new IT
services and re-certification/re-accreditation of these services on a
routine cycle after implementation.
29
IT Monitoring
2. Independent Security and Internal Control

Certification/Accreditation of Third-Party Service Providers
Management should obtain independent certification/accreditation of
security and internal controls prior to using IT service provides and re-
certification/re-accreditation on a routine cycle.
3. Independent Effectiveness Evaluation of IT Services

Management should obtain independent evaluation of the effectiveness
of IT services on a routine cycle.
4. Independent Effectiveness Evaluation of Third-Party Service

Providers
Management should obtain independent evaluation of the effectiveness
of IT service providers on a routine cycle.
5. Independent Assurance of Compliance with Laws and Regulatory

Requirements and Contractual Commitments
Management should obtain independent assurance of the IT function’s
compliance with legal and regulatory requirements and contractual
commitments on a routine cycle.
6. Independent Assurance of Compliance with Laws and Regulatory

Requirements and Contractual Commitments by Third-Party Service
Providers
Management should obtain independent assurance of third-party
service providers’ compliance with legal and regulatory requirements
and contractual commitments on a routine cycle.
7. Competence of Independent Assurance Function

Management should ensure that the independent assurance function
possesses the technical competence, skills and knowledge necessary to
perform such reviews in an effective, efficient and economical manner.
8. Proactive Audit Involvement

IT management should seek audit involvement in a proactive manner
before finalizing IT service solutions.
30

 There is continuous alignment with stakeholder needs.
 The organization has defined processes for IT assurance activities,
especially overall internal control, certification and major
decisions.
 Benchmarking of external service providers is routinely performed.
 Major IT decisions have an up-front requirements analysis for a
third-party assurance opinion.
 Prior to obtaining independent assurance, a high-level risk
assessment is performed with the key stakeholders.
 There is a commitment to leverage independent assurance for
sustainable improvement.
 Assurance activities are performed in accordance with generally
accepted practices, such as SysTrust.
 There is a partnership between auditor and auditee, to encourage
cooperation.
Key Goal Indicators

 Increased number of accepted opinions on the overall system of
internal control for all agreed domains.
 Increased number of quality certifications or accreditations for all
agreed domains.
 Increased number of second opinions reported to the stakeholders
for major IT decisions such as going live, contract negotiations,
joint ventures and major acquisitions.
 Percentage of recommendations closed on time relative to
independent internal control reviews, quality certifications or
accreditations and second opinions.
 Reduced number of failed or reversed major IT decisions.
 Index of confidence and trust of stakeholders.
31
IT Monitoring

 Reduced overhead of obtaining assurance and certifications.
 Timeliness of assurance reporting.
 Timeliness of assurance activities.
 Number of assurance processes initiated.
 Number of iterations before assurance reports are accepted.
 Number of IT decisions requiring assurance where no assurance
was sought.
 Number of IT decisions not requiring assurance where assurance
was sought.
 Reduced number of failed or reversed major IT decisions after a
positive assurance was obtained.
32
Security and Risk Monitoring
Security and Risk Monitoring
Information systems are subject to a wide range of disruptive incidents

of varying degrees of intensity. The business processes that rely on
these systems, and the environment in which both these systems and
processes operate, also are continually subject to change and new risks.
Preventive measures may not always be feasible or cost-effective to

minimize loss, disclosure, damage or disruption. Hence, monitoring
measures need to be established to detect and ensure correction of risk
and security breaches, so that all actual and suspected breaches and risk
exposures are promptly identified, investigated and acted on. This also
will ensure ongoing compliance with policy, standards and minimum
acceptable security and risk practices.
The immediate benefit of instituting risk monitoring measures and

procedures over systems, processes and their environment is to identify
issues promptly, contain damage and expedite recovery. The most
important consequential benefit is that it increases the ability to prevent
future damage and inconvenience, while increasing the predictability of
actions involving failures, risk exposures or breaches of security. An
associated benefit is the deterrence value of effective monitoring
processes.
Actions that may result from monitoring practices are:

 disciplinary or corrective actions;
 minimization and recovery of losses;
 refinement of security levels;
 changes to policy or standards;
 changes to design and implementation of security and risk
management processes;
 initiation of reassessment programs, including root cause and
pattern analysis;
 initiation of intelligent monitoring systems with interactive
feedback; and
 initiation of network or system penetration studies.
33
IT Monitoring
Follow-up of security and other risks is as important as its

implementation, especially in the light of new technological
developments, whether those adopted by the system owner or those
available for use by others. Issues that need to be addressed in
achieving effective monitoring include:
 the appointment of a responsible manager with adequate tools and
resources;
 the performance of independent and objective assessments of
security controls such as those provided by security audits;
 the establishment of clear and expedient investigative procedures;
 the massive amount of management audit trail information from a
large variety of system components that may need to be examined;
 the timeliness of processes to alert management when electronic
transactions are practically instantaneous; and
 the dynamic and ever-changing business and information systems
environment.
It is management’s responsibility to ensure that such processes and

associated responsibilities are embedded into the organization with
clear objectives and accountabilities. Subsequently, management
should monitor whether the processes function well and whether the
results of those processes are appropriate and acted on.
34
Additional IT Measures to Monitor
Additional IT Measures to Monitor
Key Goal Indicators (or outcome measures)

 Enhanced performance and cost management
 Improved return on major IT investments
 Improved time to market
 Increased quality, innovation and risk management
 Benchmarking comparisons of IT’s return on investment, unit cost,
etc.
 Creation of new service delivery channels
 Increased level of service delivery
 Absence of integrity and confidentiality risks
 Cost efficiency of processes and operations
 Confirmation of reliability and effectiveness
 Number of timely changes to processes and systems
 Enhanced performance and cost management
 Measurable contribution from IT to fast introduction of innovative
products and services
 Reaching new and satisfying existing customers
 Meeting stakeholder requirements and expectations on budget and
on time
 Adherence to laws, regulations, industry standards and contractual
commitments
 Transparency on risk taking and adherence to the agreed
organizational risk profile
 Business cases that demonstrate a high potential return on
investment.
 Absence of integrity and confidentiality risks
 Availability of appropriate bandwidth, computing power and IT
delivery mechanisms
35
IT Monitoring
 Confirmation of reliability and effectiveness

 Deviation between estimated and actual costs
 Improved productivity (e.g., delivery of value per employee,
number of customers and cost per customer served)
 Cost efficiency of processes and operations
Key Performance Indicators (performance drivers)

 Improved cost-efficiency of IT processes (costs vs. deliverables)
 Increased number of IT action plans for process improvement
initiatives
 Increased utilization of IT infrastructure
 Increased availability of knowledge and information for managing
the enterprise
 Improved performance as measured by IT balanced scorecards
 System downtime
 Throughput and response times
 Amount of errors and rework
 Number of staff trained in new technology and customer service
skills
 Benchmark comparisons for operational excellence, best practice,
etc.
 Number of non-compliance reportings
 Reduction in development and processing time
 Increased number of enterprise transformation projects enabled by
IT
 Increased satisfaction of IT users and stakeholders (surveys and
number of complaints)
36

IT-Monitoring 6

Hochgeladen von

Dokumentinformationen

Originalbeschreibung:

Originaltitel

Copyright

Verfügbare Formate

Dieses Dokument teilen

Dokument teilen oder einbetten

Freigabeoptionen

Stufen Sie dieses Dokument als nützlich ein?

Sind diese Inhalte unangemessen?

Copyright:

Verfügbare Formate

IT-Monitoring 6

Hochgeladen von

Copyright:

Verfügbare Formate

IFAC Information Technology Guideline 6

Information April 2002

In this guideline series, the International Federation of Accountants,

This guideline is the sixth of the series and covers Information

IFAC welcomes any comments you may have. Comments should be

The approved text of this Guideline is that published in the English

Copyright © April 2002 by the International Federation of Accountants.

WHY MONITOR IT?.............................................................................. 1

HOW DOES MANAGEMENT MONITOR IT?...................................................2

WHAT TOOLS ARE AVAILABLE?............................................................... 3

WHAT IS THE BEST APPROACH?............................................................... 3

WHY IS MONITORING OF IT IMPORTANT?................................................... 5

HOW DOES MANAGEMENT MONITOR IT?................................................. 13

WHAT IT MONITORING TOOLS ARE AVAILABLE?.......................................14

WHAT IS THE BEST APPROACH TO IMPLEMENT IT MONITORING?...............19

APPROACH - PHASE I: ORIENTATION................................................19

APPROACH - PHASE II: CRITERIA DEFINITION...................................20

APPROACH - PHASE III: ONGOING MONITORING............................... 21

APPROACH - PHASE V: SUBSEQUENT ACTIONS..................................21

APPROACH - PHASE VI: REPORTING............................................... 22

Control Objectives for the IT Monitoring Process Domain .................23

M1 – MONITORING THE PROCESSES...........................................................23

HIGH LEVEL CONTROL OBJECTIVE.......................................................... 23

DETAILED CONTROL OBJECTIVES............................................................23

M2 – ASSESSING INTERNAL CONTROL ADEQUACY........................................ 26

HIGH LEVEL CONTROL OBJECTIVE.......................................................... 26

DETAILED CONTROL OBJECTIVES............................................................26

M3 – OBTAINING INDEPENDENT ASSURANCE............................................... 29

HIGH LEVEL CONTROL OBJECTIVE.......................................................... 29

DETAILED CONTROL OBJECTIVES............................................................29

SECURITY AND RISK MONITORING.............................................................33

ADDITIONAL IT MEASURES TO MONITOR.................................................. 35

KEY GOAL INDICATORS (OR OUTCOME MEASURES).................................. 35

KEY PERFORMANCE INDICATORS (PERFORMANCE DRIVERS)..........................36

Many organizations recognize the potential benefits that technology

In this series of guidelines, the International Federation of

This guideline is the last of the series and covers managing IT

Executives in various capacities (for example, accountants, financial

IFAC’s Information Technology Committee would like to acknowledge

Why monitor IT?

 how IT improves the business with technology, process

How does management monitor IT?

What tools are available?

What is the best approach?

Phase IV: Separate Periodic and Ad Hoc Monitoring.

Why is Monitoring of IT Important?

9. Boards and executive management generally expect their

10. While some have been successful, many organizations find

 inadequate or even obsolete technology;

11. Boards and management who exercise proper IT monitoring

12. Increasingly, top management is looking at IT performance.

13. Because it is so essential to an enterprise’s operations, IT

14. The following illustration depicts some examples of outcome

Entity being 

16. In the context of IT governance, this means that measurable

IT monitoring is fundamental to IT governance; it is part

18. IT monitoring is enabled by the definition of relevant

19. Monitoring of IT is especially important because of the

20. The performance objectives are best illustrated by the IT

Institute. That model implies that the governance entity needs

21. Additionally, executives put control frameworks in place to

that agreed strategy is executed within established policies.