Beruflich Dokumente
Kultur Dokumente
Postgraduate Opleiding
IT Audit, Compliance & Advisory
Addressing SIEM
Status Final
Date 30 August 2015
vrije universiteit Amsterdam
PGO IT Audit, Compliance & Advisory
1 Contents
2 Management summary ......................................................................................................................... 4
3 Introduction .............................................................................................................................................. 6
3.1 Background ..................................................................................................................... 6
3.2 Research motivation .................................................................................................... 7
3.3 Scope .................................................................................................................................. 8
3.4 Research questions....................................................................................................... 9
3.4.1 Central research question................................................................................................... 9
3.4.2 Sub question 1: SIEM explained ....................................................................................... 9
3.4.3 Sub question 2: SIEM in IT risk frameworks............................................................... 9
3.4.4 Sub question 3: SIEM case study ...................................................................................... 9
3.5 Research approach .....................................................................................................10
3.6 Related work .................................................................................................................11
3.7 Supervision ....................................................................................................................12
2
vrije universiteit Amsterdam
PGO IT Audit, Compliance & Advisory
7 Synthesis ..................................................................................................................................................50
9 Acknowledgments ................................................................................................................................56
10 Bibliography ...........................................................................................................................................57
11 Appendices ..............................................................................................................................................61
11.1 Mapping of selected frameworks ..........................................................................62
11.2 Interview questions....................................................................................................67
11.3 Mapping of selected organisations .......................................................................68
3
vrije universiteit Amsterdam
PGO IT Audit, Compliance & Advisory
2 Management summary
With hacker attacks and data breaches happening each day, organisations see the need for
employing detective security controls instead of solely relying on preventive controls. One such
detective security control is Security Information and Event Management (hereafter: SIEM). A
SIEM can be used by an organisation to detect and alert on predefined unwanted activities within
its IT environment.
Our research focusses on what areas should be addressed when implementing SIEM, how these
areas are covered within selected IT risk frameworks and to what extend this matches current
SIEM implementations. By means of literary research and field-expert input, a model is created
that covers SIEM focus areas. Each focus area contains minimum requirements that are essential
for a successful and effective SIEM implementation. The formulated focus areas are: organisations
requirements, operational requirements, log management, correlation, alerting, responding and
evaluating. Our model is used to assess the coverage of SIEM within the selected IT risk
frameworks and to determine how financial institutions address SIEM.
Operational
Evaluation
requirements
Alerting Correlation
The graph above shows to what extend the selected IT risk frameworks address our model. It
clearly shows that while the organisational requirements (i.e. general IT risk management) are
addressed by most of the selected frameworks, the operational requirements are not. The
operational requirements focus area covers specific requirements essential to any SIEM
implementation such as the definition of risk scenarios and use cases.
4
vrije universiteit Amsterdam
PGO IT Audit, Compliance & Advisory
During our case study we assessed the SIEM implementation of two financial institutions. For the
first organisation, threat management was the main driver to implement SIEM. The main driver
for the SIEM implementation of the second organisation was compliance with legislative
requirements from DNB. The first organisation addressed most of the requirements specified
within our model. They defined risk scenarios and use cases, whereas the second organisation only
implemented generic predefined rules due to lack of available resources. Both organisations
acknowledge the fact that one needs to define risk scenarios and use cases in order to increase
SIEM effectiveness.
Based on literature research, field-expert input and the performed case studies, it becomes evident
that risk scenarios and use cases are fundamental for SIEM implementations. Selected frameworks
inadequately cover these aspects. Therefore, we conclude that the selected frameworks do not
match current SIEM implementations.
5
vrije universiteit Amsterdam
PGO IT Audit, Compliance & Advisory
3 Introduction
3.1 Background
We are living in a society where technology plays a more important role each year. Everything is
being connected to the Internet: from cars to household appliances. Not just personal devices are
being connected. Organisations are being interconnected over the Internet as well. With
everything being interconnected, the impact of a security breach in an organisations IT
environment becomes bigger as well. Compromise of a single asset, might lead to an infection
spreading throughout the entire organisations network. Where hackers in the early 1970s used
whistles to make free long-distance phone calls (Times, 2000), nowadays they are targeting
organisations for amongst others: their intellectual property, financial gain or to cause
reputational damage.
6
vrije universiteit Amsterdam
PGO IT Audit, Compliance & Advisory
(figure 1) (LockHeed Martin Corporation, 2011). A kill chain, originally described by the American
military, is a structured process that details the components required for an adversary to obtain
the desired results such as espionage or fraud. The cyber kill chain is specifically geared towards
cyber attacks and represents the process that constitutes a successful infiltration. Security
professionals can use the cyber kill chain to identify risks and patterns as well as deploying
appropriate defences for the different stages of an attack more efficiently. Preventing a single
phase of the cyber kill chain disrupts the entire process and causes an attack to fail.
Traditional IT security primarily focuses on the Delivery component of the cyber kill chain by
implementing heavy security measures on the perimeter of the organisations network
(preventive security measures). However, solely focusing on perimeter security is not sufficient
any more. Attackers perform advanced reconnaissance, build specialised weapons, and exploit
zero-day vulnerabilities. Standard anti-virus solutions and Intrusion Detection Systems might not
recognize these specialised weapons, and carefully crafted (spear-) phishing e-mails could pass an
organisations spam filter. As a result, organisations need to implement additional security
measures in order to protect themselves. One such particular (detective) measure has gained
significant popularity over the past years and is named Security Information and Event
Management (hereafter: SIEM).
SIEM allows organisations to monitor status and events from all over their IT landscape. The
collected information can in turn be used to perform detailed analysis and correlation to identify
irregularities and security incidents. Timely identifying security incidents allows for appropriate
measures and actions to be taken. In addition to detection, the information provided by SIEM can
be used to respond to security incidents efficiently. This disrupts the cyber kill chain, stopping
potential breaches to an organisations network. However, implementing an effective SIEM
solution depends on many different aspects and requirements. Both aspects and requirements will
be covered in detail throughout this thesis.
7
vrije universiteit Amsterdam
PGO IT Audit, Compliance & Advisory
Multiple whitepapers state that a correctly implemented SIEM solution is crucial for the success
against cyber threats. However, no holistic and structural methodology exists for implementing
SIEM and its required tooling (PvIB, 2011), (Gartner, 2014), (Schinagl & Schoon, 2014). Several
commercial companies like HP, IBM, and McAfee try filling that gap by offering their own SIEM
related software but mainly end up solely pushing their own products. Papers and blog posts
published by these vendors are rather high-level and vague, containing little to no detail regarding
a holistic SIEM implementation approach (HP, 2014), (IBM, 2014).
Our research focuses on how various IT risk frameworks cover SIEM as a holistic subject and how
this subject is addressed in real-world implementations. We aim to achieve this by first providing
a clear definition of SIEM as well as the components and processes it entails. Secondly, we compile
an overview of focus areas, each containing a minimum set of requirements that should be
addressed when implementing SIEM. IT risk frameworks will be plotted to our model, showing
their coverage of SIEM related aspects. The model will also be used as guidance of our empirical
research in which we assess how SIEM is addressed in practise by multiple financial institutions.
3.3 Scope
The scope of this research entails setting up a model containing relevant focus areas and minimum
requirements that need to be addressed when implementing SIEM for the use of threat
management (i.e. detection of security incidents).
Furthermore, our research entails the assessment of IT risk frameworks listed in section 5.3, based
on our model. Additional IT risk frameworks may exist, although have not been selected due to the
time-boxed approach of this research.
Our research does not provide an exhaustive model that can be used to test the effectiveness of a
SIEM implementation. It can however be used as a guideline for determining success factors of a
SIEM implementation, covering the minimum requirements on an organisational-, operational-,
and technical level.
Empirical research is performed in which we assess how multiple financial institutions address
the topic of SIEM. The assessment shows how selected financial institutions address the
requirements of which our model consists. Performed research does not entail a full in-depth audit
of the organisation and its SIEM environment, but will be used to verify the applicability of our
model.
Also, note that since the concept Security Operations Centre (SOC) is closely related to SIEM, our
research covers crossover domain subjects. However, we do not focus on SOC requirements or any
of its areas, nor incident- and response management.
8
vrije universiteit Amsterdam
PGO IT Audit, Compliance & Advisory
How do IT risk frameworks address SIEM and to what extend does this match current SIEM
implementations?
What are focus areas of SIEM and how do IT risk frameworks address them?
9
vrije universiteit Amsterdam
PGO IT Audit, Compliance & Advisory
4.
Expert
validation
2.
1. 3, 5 & 7. 8. 9.
Literature
Plan Analyse Conclude Document
research
6.
Empirical
research
The plan phase (1) involves gathering background information and determining the final research
questions. Moreover, the research approach is determined and discussed in detail. The planning
phase and the defined research (sub) questions provide the basis for our literature research (2).
During the literature research (2) information is gathered from relevant publications,
whitepapers, testimonials and articles. This phase focusses on two aspects. First, ascertaining a
transparent and comprehensive definition of SIEM. Second, identifying shortcomings, common
issues organisations face and other insights regarding current SIEM implementations. The
information collected during the literature research serves as input for determining key SIEM
focus areas and minimum requirements of which our model comprises (3).
The results of the literature research are analysed (3) and together with our own experience, an
initial version of our model containing key SIEM focus areas and their minimum requirements is
created. Subject matter experts (4) are consulted for rigorously evaluating and validating our
initial model. The feedback provided by the subject matter experts serves as input for the second
round of analysis (5) in which our model is finalised.
With our final model containing SIEM focus areas and minimum requirements, the empirical
research phase (6) commences. This phase consists once again out of two aspects. First, we select
several popular IT risk frameworks that are then mapped to our focus areas. This shows how each
10
vrije universiteit Amsterdam
PGO IT Audit, Compliance & Advisory
selected framework covers our set of minimum requirements that address the implementation of
SIEM. Second, we determine how SIEM is addressed in practice by performing multiple case
studies at financial institutions. The case studies consist out of multiple interviews in which we
match our model to the way each financial institution addresses SIEM.
The results of the empirical research phase (6) are used as input for the final round of analysis (7).
Throughout this phase, we analyse the results of the literature- and empirical research to identify
findings and explanations pertaining to our research questions. We then continue drawing up our
conclusions (8) using the output of the performed analysis.
Finally, the document phase (9) is used for formalizing and documenting our findings and
conclusions.
A recent paper by fellow VU IT Audit, Compliance & Advisory students (Schinagl & Schoon, 2014)
described the importance of an effectively operating SIEM solution for the early detection of
security breaches.
Various papers have been published on subtopics of SIEM. Examples are: The complete guide to
log and event management (Chuvakin, 2010), Effective use case modelling for security information
and event management (SANS, 2009), Attack life cycle use case methodology (HP, 2014). Other
papers are focussed on Security Operations Centre (SOC), and touch upon the subject SIEM (PvIB,
2011).
None of the papers described above provide a holistic view addressing key focus areas of SIEM,
including organisational and operational requirements. In this thesis, we set up a model that
covers focus areas including a minimum set of requirements (focus points) that should be
addressed when implementing SIEM. In addition, we assess multiple IT risk frameworks based on
our model, and look at how financial institutions address the topic of SIEM based on our model.
11
vrije universiteit Amsterdam
PGO IT Audit, Compliance & Advisory
3.7 Supervision
Both our employer KPMG as well as ControlSolutions International and the VU supported and
guided our research, for which we are thankful. The following persons guided us during our
research:
KPMG:
Ing. J.A.M. Hermans RE
Partner, IT Advisory, Risk Consulting, KPMG
Hermans.John@kpmg.nl
ControlSolutions International:
R. Bardoel MSc RE CISA CISSP OSCP
Partner, ControlSolutions International
Ralf.Bardoel@controlsolutions.nl
12
vrije universiteit Amsterdam
PGO IT Audit, Compliance & Advisory
4 SIEM explained
In this chapter, we cover the definition of SIEM and provide insight why organisations require
SIEM. We then move on to discuss the importance for organisations to know and identify what to
monitor (i.e. their crown jewels). Throughout this chapter, we use expert opinions, surveys and
whitepapers to supplement our own experience and provide factual support. During this chapter
we aim at providing a solid foundation for the conclusion to the first research question:
A SIEM collects log files and security information from internal- (i.e. server-, network- and
application logs) and external sources (i.e. threat intelligence sharing). Event correlation is used
to detect and alert on, by the organisation defined, unwanted activities within the network. An
organisation can use the information within the SIEM to effectively respond to detected security
incidents.
13
vrije universiteit Amsterdam
PGO IT Audit, Compliance & Advisory
Figure 3 covers the various aspects of SIEM. The main focus areas that define the fundaments of
every SIEM are: 1) log management, 2) correlation, 3) alerting, 4) responding. These focus areas
will be covered extensively in section 4.4.
There are multiple drivers for an organisation to implement SIEM. Based on our experience in the
field and according to literary study (Netmonastery, 2014), the three main drivers why
organisations implement SIEM are:
1. Threat management (i.e. SIEM as a detective security control);
2. Compliancy;
3. Forensic support.
Each of these drivers will be further discussed in the following sub-sections. In our research, we
focus on organisations that implement SIEM for threat management, as defined in section 3.3.
Using event correlation, relations are established between events from different sources on the
network, such as server logs, IDS/IPS and application. A baseline of normal behaviour is set up for
various events, so that alarms can be raised when deviations on the normal behaviour pattern
occur. Threat intelligence is gathered from various internal and external sources. This enhances
the detection rate of attacks in an ever-changing threat landscape. Incident management functions
are embedded into SIEM to support the IT security department.
Joff Thyer, (Thyer, 2015), believes that in order to successfully defend your network against
hackers and other threats, you need to have the presumption that it is already compromised. This
means that adversaries already found a way around or through existing preventive security
measures and have established a foothold in an organisations environment. He further states that
real-time detective capabilities are required to identify these adversaries and to ensure a holistic
approach for defending an organisations most important assets, also known as an organisations
crown jewels.
14
vrije universiteit Amsterdam
PGO IT Audit, Compliance & Advisory
(DNB, 2014). PCI-DSS is a security standard imposed on all organisations that handle credit card
information. Organisations are required to implement logging and monitoring mechanisms so they
are able to detect and/or minimise the impact of a data compromise (PCI Security Standards
Council, 2013).
However, when talking about SIEM implementations covering compliance needs, we experience
that many organisations implement SIEM only to be compliant with frameworks such as PCI-DSS.
Our experience is that organisations seldom implement or use SIEM for actual compliance
reporting, an observation further corroborated by a survey done by (EiQ Networks, 2013). This
survey states that 35% of interviewees say that they implemented SIEM to make auditors happy
and comply with regulation.
15
vrije universiteit Amsterdam
PGO IT Audit, Compliance & Advisory
processes of a SOC to function correctly. As such, SIEM is an integral part of a SOC but cannot be
considered the SOC itself. Figure 4 illustrates our view on the position of SIEM including the
previously defined aspects and their overlap with a SOC.
It was only after implementing these tools that organisations realized just having the tool installed
is not good enough. According to a survey done by (SC Magazine UK, 2012), organisations are
struggling with effectively managing their SIEM implementations. 59% of interviewees stated that
their implementations failed due to time constraints in monitoring for security incidents and
another 40 percent voiced concern about the time it takes to analyse logs and data.
Another survey by (EiQ Networks, 2013) shows that organisations struggle with the fine-tuning of
their SIEM and that it can take months to obtain useful data. 44% of interviewees state that it took
a few weeks to more than a month to deploy their latest SIEM product. 52% of interviewees require
two or more full-time employees to manage their current SIEM deployment. Another 25% of
interviewees have invested more than a month in professional services since deploying their
current SIEM product.
Netwrix performed a survey (Netwrix, 2014), showing that 61% of interviewees think that their
SIEM reports lack important information. 55% of interviewees even state SIEM reports are hard
to interpret. 75% of interviewees state that SIEM reports contain ample amounts of noise data.
Looking at the results of the surveys, we argue that part of the problem originates in the lack of
adequate preparation by organisations itself. Furthermore, we experience that many
organisations underestimate implementing SIEM. An effective implementation of SIEM requires
an organisation to think about-, and take measures in the areas of: knowing what to protect (i.e.
crown jewels), log management, correlation, alerting, responding and evaluating. We call these
areas the focus areas of SIEM. All focus areas are covered in-depth in the next section.
16
vrije universiteit Amsterdam
PGO IT Audit, Compliance & Advisory
An organisations network easily consists of hundreds, if not thousands of systems running various
services in often-different locations around the world. All these systems and services generate log
files. It is impossible for an organisation to collect log files of all systems and at the same time
perform real-time analysis and correlation. As such, a more scoped approach is required: a type of
approach that inherently raises several important questions. Namely, how does an organisation
know which services or assets to protect? And, what services or assets should be focussed on?
To answer these questions, an organisation needs to know what its crown jewels are. What is the
most important asset or information that is owned by the organisation? Crown jewels can be
identified by performing a risk analysis on organisational level, in other words: an organisations
strategy. Identified risks that have a negative impact on an organisations strategy or primary
processes need to be addressed, as they have impact on an organisations crown jewels.
Crown jewels of organisations can vary per sector. For example, the crown jewels of a bank differ
from the crown jewels of a tech company. The latter most likely has intellectual property and
detailed designs that need to be protected from being stolen by adversaries and hackers. The
crown jewels of a bank are its customers data, plans on future product changes (i.e. changing
interest rates) as well as their image towards the outside world. A breach might result in bad
publicity and customers moving to competitors.
Once an organisation has defined her crown jewels, one knows what to protect. The next step for
an organisation is defining from what to protect her crown jewels. This is done by defining risk
scenarios. Organisational stakeholders, with support of IT security personnel (i.e. SOC employees),
define risk scenarios. Risk scenarios describe undesirable actions to crown jewels and include
common attacks (i.e. DDoS on online services) and attack paths (i.e. reconnaissance using a port
scan). Some risk scenarios might be imposed by legislative requirements depending on the sector.
In addition, organisations can also choose to implement detection of activities within each step of
the cyber kill chain, as covered in section 3.1.
Risk scenarios are translated into use cases. One risk scenario might result into multiple use cases.
Use cases are defined as a set of rules to which logs are correlated to be able to detect a certain
event. According to (Dorigo, 2012), use cases represent a goal or a task.
17
vrije universiteit Amsterdam
PGO IT Audit, Compliance & Advisory
An organisation knows what logs to collect from what devices based on the information required
by use cases and the rules they consist of. Every rule can require different log sources and events.
For SIEM to work correctly, all logs required by use cases and rules should be gathered, normalised
and available to the SIEM tooling. Log normalisation guarantees that identical fields within
different log sources can be correlated to each other. One can hereby think of the same
representation of an IP address for log files gathered from network devices and servers, or the
usage of the same time stamp format for all log entries that are stored within SIEM tooling.
One example of a use case that many regulatory standards require is tracking the activities of
privileged users, those with administrative privileges. However, before you can implement this
SIEM use case, you will need to know who these users are (including all of their accounts), as well
as collecting the logs of the systems they access. For translating this use case into SIEM
components, models such as the one proposed by (SANS, 2009) can be used. In this example case,
we base our use case model as proposed by (SANS, 2009) and can be seen in table 1.
Privilege escalation
Goal Detect unauthorized user access attempts, including privilege escalation.
Data sources Log data from Active Directory, host-based intrusion detection, file
integrity monitoring, and change management tooling.
Relevant systems Hostnames / IP addresses of relevant systems (i.e. crown jewels).
Importance Critical.
Required speed Real-time.
Trigger Trigger on a privilege escalation (i.e. user added to Administrators
conditions group) with no corresponding change request.
Alert Method E-mail and live dashboard update.
18
vrije universiteit Amsterdam
PGO IT Audit, Compliance & Advisory
4.4.3 Correlation
We argue that correlation is an integral
part of SIEM because it is required to
extract sensible information spread
amongst different log entries. Figure 6
shows the relation between multiple log
entries and correlation. Furthermore, the
figure introduces Use cases, which are
required to make meaningful and
structured correlation decisions.
To trigger on use cases, one typically needs to correlate multiple log entries from one or multiple
sources. An example of a use case would be to detect successful brute-force attempts. Hereby one
does not look at one or two single failed login attempts, but multiple failed login attempts on one
19
vrije universiteit Amsterdam
PGO IT Audit, Compliance & Advisory
or more systems by a single user, followed by a successful login attempt. When this happens, it
might indicate that a brute-force attack is in progress. Detecting multiple login attempts on
different systems can only be achieved by correlating log entries from multiple sources.
4.4.4 Alerting
We argue that alerting is an integral part of SIEM because alerting on out of the ordinary actions
(defined in risk scenarios, and implemented by use cases and rules) is the core purpose of a SIEM
focussed on threat management. Figure 7 shows the relation between log files, the correlation of
log entries based on use-case and the resulting alerts that follow.
Two types of alerting exist, namely passive and active alerting. Passive alerting can be seen as
regular reporting where dashboards contain data about specific areas of interest. The SOC team
monitors dashboards for suspicious behaviour. Active alerting is often used by high-impact use
cases that trigger alerts. Hereby an alert is generated and pushed automatically to a SOC employee
(i.e. by a highlighted message in the SIEM tooling dashboard, a pop-up, or an e-mail/SMS message).
20
vrije universiteit Amsterdam
PGO IT Audit, Compliance & Advisory
We argue that responding is an integral part of SIEM as it connects incident information and
alerting with the SOC and its analysts. Most alerts require manual analysis by a SOC analyst. After
analysing the event, the analyst decides the appropriate follow-up for an alert. When an alert is a
true positive, an incident might be created. If an alert is, for instance, generated by a configuration
error, one might choose to register the event as a false positive and link the alert to an existing
change ticket that was created by the employee involved with configuration error.
If analysis by a SOC analyst shows that an alert triggered by a high-impact use case concerns a true
positive, the organisations generic incident management process is followed. This guarantees the
involvement of the right organisational stakeholders as well as the escalation to the appropriate
staff.
Furthermore, we argue that evaluating is also an integral part of SIEM because it allows for lessons-
learned to be fed back into the system. In other words, experiences gained from handling incidents
or false-positives can serve as input for the addition or fine-tuning of use cases and correlation
rules.
21
vrije universiteit Amsterdam
PGO IT Audit, Compliance & Advisory
What are focus areas of SIEM and how do IT risk frameworks address them?
We start with finalising the main focus areas that need to be addressed when implementing SIEM,
as well as minimum requirements for each of the areas. We do this based on literary research, field-
expert opinions as well as our own field experience. This research will result in a model containing
a set of minimum requirements that should be covered when implementing or assessing a SIEM
implementation.
This will be followed by a selection of IT risk frameworks that will be mapped to our model,
assessing to what extent they cover SIEM.
22
vrije universiteit Amsterdam
PGO IT Audit, Compliance & Advisory
Literary research and our own experience was used to create the initial version of our model
containing SIEM focus areas and minimum requirements that need to be addressed by an
organisation when implementing SIEM. Below we explain the relevance of each focus area. Section
5.3 contains an overview of minimum requirements per focus area, as well as reasoning why every
requirement is incorporated.
Organisational requirements
The effective working of SIEM tooling depends on different parts within the organisation, as well
as a variety of organisational processes. An organisation needs a certain maturity before it can
start thinking about implementing a SIEM. One can hereby think of the existence of an information
security policy, (IT) risk management, incident management, as well as processes around the SOC
itself. Requirements in this focus area cover an overview of minimum organisational requirements
that need to be addressed by an organisation for a SIEM to function.
Operational requirements
We define operational requirements as prerequisites that directly link to the implementation and
effective working of SIEM tooling. One can hereby think of setting up requirements that SIEM
tooling needs to satisfy, SIEM tooling selection, and the definition of risk scenarios and use cases
on a logical level.
23
vrije universiteit Amsterdam
PGO IT Audit, Compliance & Advisory
Log Management
As discussed in section 4.4.2, log files are the single greatest source of information for SIEM
implementations. Requirements within this focus area are geared towards determining what to
collect, log integrity, from where to collect, how to store logs and doing it all in a structured and
sustainable way.
Correlation
Correlation of log entries is what makes a SIEM strong in only generating true-positive alerts. As
discussed in section 4.4.3, good Use cases are the fundaments of effective correlation rules.
Requirements in this focus area cover requirements regarding the technical implementation of
logical use cases, as well as the management of the use cases.
Alerting
As discussed in section 4.4.4, the core purpose of a security tailored SIEM is alerting on suspicious
events triggered by Use cases. Requirements in this focus area cover requirements regarding the
technical implementation of triggers and alerts within a SIEM.
Responding
As discussed in section 4.4.5, the SOC responds to alerts that are generated by the SIEM. We do not
cover SOC-specific processes, however, we do include generic requirements regarding responding
to alerts. One can hereby think of the automatic creation of an incident when a high-impact use
case triggers an alert.
Evaluating
This focus area consists of evaluating elements that will take place post-alert. One can hereby think
of a root-cause analysis, lessons learned, the adaptation of use cases based lessons learned, and
reporting requirements.
24
vrije universiteit Amsterdam
PGO IT Audit, Compliance & Advisory
Interviews have been conducted with multiple field-experts. We have discussed the validity of our
model of minimum requirements with the following persons:
All interviewees agreed upon our selection and definition of main SIEM focus areas. Section 5.2
contains our final model built up of the focus areas including minimum requirements that should
be addressed when implementing SIEM. Every requirement contains reasoning why it is
incorporated in our model.
25
vrije universiteit Amsterdam
PGO IT Audit, Compliance & Advisory
26
vrije universiteit Amsterdam
PGO IT Audit, Compliance & Advisory
27
vrije universiteit Amsterdam
PGO IT Audit, Compliance & Advisory
28
vrije universiteit Amsterdam
PGO IT Audit, Compliance & Advisory
an organisation. As such, multiple baselines can exists but should all adhere to
the log policy.
LM1.3 Centralised log requirements are defined
And should at least include: Centralized logging is a key component for supplying your SIEM implementation (Kent &
Centralized logging facilities are implemented with relevant information in a structured and sustainable way. As such, we Souppaya , 2006)
Only authorised users have access to log files recognize the importance of centralized logging facilities and state that it should (Hanley &
Capacity of centralized logging infrastructure is be implemented as described by the log policy. Furthermore, centralized logging Montelibano,
managed helps in securing forensic evidence in case of compromised systems and log 2011)
tampering. (NSA, 2013)
(Lewis, 2014)
29
vrije universiteit Amsterdam
PGO IT Audit, Compliance & Advisory
30
vrije universiteit Amsterdam
PGO IT Audit, Compliance & Advisory
31
vrije universiteit Amsterdam
PGO IT Audit, Compliance & Advisory
32
vrije universiteit Amsterdam
PGO IT Audit, Compliance & Advisory
Framework
2014 Standard of Good Practise for Information Security
COBIT 5
ISO/IEC 27002
NIST Framework for Improving Critical Infrastructure Cybersecurity
PCI DSS 3.0
ISO/IEC 27002
ISO/IEC 27002 is an information security standard that is developed by the International
Organisation for Standardisation (ISO) (ISO, 2013). It provides best practise recommendations on
information security management. It is commonly used within organisations in a wide variety of
sectors. It covers a broad spectrum of controls, including requirements regarding logging and
monitoring.
33
vrije universiteit Amsterdam
PGO IT Audit, Compliance & Advisory
Payment Card Industry Data Security Standard 3.0 (PCI DSS 3.0)
PCI DSS 3.0 (PCI Security Standards Council, 2013) is a standard that is developed by the Payment
Card Industry Security Council. This standard is imposed on all organisations that handle credit
card data. It contains controls that should reduce credit card fraud. It also contains requirements
regarding monitoring.
34
vrije universiteit Amsterdam
PGO IT Audit, Compliance & Advisory
Please note that when multiple frameworks map to a requirement, it does not mean that they cover
the exact same load. This due to the IT risk frameworks having a different level of granularity on
topics. We have included matches where wording may vary, although semantic intent is similar.
Operational
Evaluation
requirements
Alerting Correlation
ISF SoGP covers the most requirements defined in our focus areas. It scores particularly well in
organisational requirements as well as log management, correlation, alerting and evaluation. As
ISF SoGP is a superset of COBIT 5 and provides comprehensive coverage of ISO 27000 series, we
knew beforehand that it would score better than the two separately.
35
vrije universiteit Amsterdam
PGO IT Audit, Compliance & Advisory
The spider graph shows that all frameworks lack coverage of the operational requirements focus
area. This area covers requirements on an operational level specific to the SIEM implementation.
One can hereby think of specific technical SIEM requirements that result from organisational
requirements, as well as the definition of risk scenarios and use cases.
None of the frameworks cover the setup of risk scenarios in corporation with business
stakeholders, based on crown jewels, legislative requirements and/or previous security incidents.
In addition, none of the frameworks cover the definition of use cases based on risk scenarios,
common attack paths or the cyber kill chain phases. The latter can be seen as the base of correlation
rules to be implemented in SIEM, and can hereby be seen as the intelligence of the SIEM solution.
As such, the lack of coverage of these operational requirements could seriously affect the efficiency
of detecting unwanted activities using SIEM.
The following subsections cover the results of mapping the frameworks to the remaining six areas.
Organisational requirements
This focus area covers requirements on an organisational level. One can hereby think of
requirements regarding roles and responsibilities concerning SIEM as well as the organisation
setting up specific requirements that should be met by the SIEM implementation. Requirements
differ depending on an organisations risk appetite, crown jewels, legislative and regulatory
requirements as well as reporting requirements.
All frameworks have good coverage within the organisational requirements focus area, where both
COBIT 5 and ISF SoGP cover all our requirements. One can argue that frameworks cover this focus
area well due to the generic nature of its requirements.
Log management
This focus area covers requirements regarding log management. Next to generic log management
requirements, this focus area covers specific SIEM requirements such as the definition of which
log sources should be on boarded, and which log entries should be collected based on defined use
cases.
ISF SoGP has the best coverage of this focus area. ISO 27002 and NIST Cybersecurity perform well
in this area too, where they mostly cover the generic log management requirements.
Correlation
This focus area covers requirements regarding correlation of events within the SIEM. One can
hereby think of the translation of use cases into correlation rules and the periodic checking on
desired results of the implemented rules.
36
vrije universiteit Amsterdam
PGO IT Audit, Compliance & Advisory
Both ISF SoGP and NIST Cybersecurity cover all our defined requirements. PCI-DSS covers none of
the defined requirements.
Alerting
This focus area covers requirements regarding the implementation of triggers and alerts set within
use cases, as well as the information that alerts contain. This focus area contains two requirements.
As such, covering only one requirement drops the line of an organisation to 50% in the spider
graph immediately.
ISF SoGP covers all requirements defined within this focus area. ISO 27002, COBIT 5 as well as PCI-
DSS cover none of the defined requirements.
Responding
This focus area covers requirements regarding the response to alerts that are generated by the
SIEM. Requirements link to the SOC as well as an organisations incident management process.
This focus area contains two requirements. As such, covering only one requirement drops the line
of an organisation to 50% in the spider graph immediately.
COBIT 5 covers all requirements defined within this focus area. One can argue that this is due to
the requirements being defined on process level
Evaluation
This focus area covers requirements regarding the evaluation of alerts and incidents that are
generated by the SIEM. One can hereby think of the adjustment of use cases if applicable. This focus
area contains two requirements. As such, covering only one requirement drops the line of an
organisation to 50% in the spider graph immediately.
COBIT 5, NIST Cybersecurity, and ISF SoGP cover all requirements defined within this focus area.
PCI-DSS covers none of the defined requirements.
37
vrije universiteit Amsterdam
PGO IT Audit, Compliance & Advisory
To establish a common baseline and obtain comparable results, we used a predefined set of
interview questions as well as our SIEM focus-areas as guidance for the interviews. Each interview
was held with one or more subject-matter experts. The interview questions are listed in Appendix
10.2: Interview questions.
The results of the interviews are detailed per financial institution and focus area. Analysis of the
results is preformed based on a 5-point scoring system as detailed in table 3. Based on the
organisations coverage of a requirement, a score is assigned. For each focus area, the total score
will be the average score of all requirements. The results are used to make a comparison between
the organisations.
-- Requirement is not addressed.
- Requirement is not specifically addressed, however is (partially) covered by
other controls within an organisation.
+/- Requirement is addressed in an ad-hoc fashion. These implementations were
born out of necessity and did not go through a proper design and test phase.
+ Requirement is addressed in a structured manner, going through a design and
test phase. Its implementation was unforeseen however, become apparent
during the SIEM implementation process.
++ Requirement is addressed in a structured manner, going through a design and
test phase. Its coverage is the result of defined requirements and/or foresight.
38
vrije universiteit Amsterdam
PGO IT Audit, Compliance & Advisory
FI1 has a dedicated SOC department that, amongst others, handles security monitoring. The SOC
team consists of seven full-time security professionals. FI1 started using SIEM in 2010, and have
been actively using and developing SIEM and its supporting processes.
6.2.2 Results
This section covers how FI1 addresses SIEM. Results are based on interviews.
General interview
FI1 stated that they did not make use of a framework when implementing their SIEM tooling.
Organisational requirements
FI1s main reason for implementing SIEM is threat management. FI1 wanted to gain better insights
into ongoing attacks and the overall security of the IT infrastructure. Their SIEM has been pitched
to management as a means for threat management. Compliancy and regulatory requirements were
also a factor during the implementation of SIEM.
FI1 decided to initially focus on the network layer and perimeter monitoring. Doing so allowed FI1
to heavily scope the input sources in order to not flood their SIEM tooling with raw data. Over time,
more log sources were enrolled. A process is in place that enables business departments to define
specific security requirements, in collaboration with the security team. This process covers the on
boarding of crown jewels within SIEM.
The following key roles regarding SIEM have been defined and assigned:
Level 1, 2, 3 analyst
Reporting specialist
System administration
SIEM designer (architect)
Logging specialist
Intelligence analyst
39
vrije universiteit Amsterdam
PGO IT Audit, Compliance & Advisory
Operational requirements
The initial operational requirements that led to the implementation of FI1s SIEM tooling are
unclear. A project to select and implement new SIEM tooling is currently ongoing. This project
defines requirements based on gained experiences, future plans and capacity needs.
The business has defined FI1s crown jewels and its main threats. The SIEM designer, in
collaboration with the security team and the business, develops risk scenarios based on these
threats. Each risk scenario is further broken down into use cases. Use cases are defined based on
the phases of the Cyber-kill chain, common attacks, attack paths and threat intelligence.
Furthermore, use cases contain classification, categorisation and prioritizing.
FI1 states that they use three main sources of input for security monitoring. These sources are:
Log files (events).
Intelligence (Threat intelligence, vulnerability scans, netblock information, etc.).
Contextual information.
FI1 states to participate in threat intelligence sharing with other financial institutions, as well as
having a subscription on commercial threat intelligence feeds.
Log management
FI1 designed and implemented log management as part of their SIEM environment. Log
management is not only used to retrieve critical security information but also to perform
availability monitoring and obtain debugging information. FI1 acknowledges that the quality of log
files is of great importance to the successful workings of a SIEM solution.
FI1 states that log management should be well defined and implemented across the IT
environment. The sending of log files should not be interrupted by changes or the reallocation of
systems. As such, log management should be part of operational and technical procedures.
FI1 transmits logs to a central location, from which the SIEM tooling retrieves the log files. The log
files to be retrieved are based on use cases and are specifically selected (i.e. only relevant log files
are retrieved). Logs are automatically normalised by the SIEM tooling.
The SIEM tooling provides mechanisms for checking whether expected log files are still received.
If expected log files are not received, alerts are generated.
Correlation
The SIEM tooling provides authentication and authorisation so that only authorised users are
allowed to modify correlation rules and system settings.
The SIEM designer and specialist are responsible for translating Use cases into specific correlation
rules that can be used in the SIEM tooling. FI1 makes use of a development environment for
designing and testing correlation rules before deploying them into production.
The working of correlation rules is tested by simulating real attacks. Furthermore, periodic red-
and blue teaming exercises are performed to test the overall effectiveness of the rule set.
Alerting
Use cases contain priority, criticality and triggers on when alerts should be generated. The SIEM
tooling provides alerts to the security team, providing all relevant information.
40
vrije universiteit Amsterdam
PGO IT Audit, Compliance & Advisory
Responding
FI1 employs a 3-line responding model
1st line analysts receive alerts and determine the nature of the alert. FI1 states that many
alerts handled by the 1st line analysts are the result of implemented configuration changes.
2nd line analysts take over alerts that cannot be resolved by the 1st line.
3rd line analysts take over alerts that cannot be resolved by the 2nd line. 3rd line analysts
can raise an incident, starting the generic incident management procedure.
Evaluation
The intelligence analyst has the responsibility to update existing use cases if new threats are
identified, new threat-intelligence is received or when scenarios are modified. FI1 states that
updating use cases is an iterative process and takes place throughout the life cycle of a use-case.
FI1 states that use cases are adjusted if applicable. This is mainly done for use cases that generate
a large amount of false positives. No formal process is in place for evaluating the full set of use
cases periodically.
When the business requests to on board a new component in SIEM, reporting requirements are
set up. These reports are then delivered to the business upon agreed interval. No global reporting
requirements have been agreed upon.
It is positive to see that the threat management driver for FI1s SIEM implementation is claimed to
be understood and supported by management. This could be due to previous incidents or
experiences that clearly demonstrate the benefit of a threat management focussed SIEM
implementation. FI1 also indicates that legislative and regulatory requirements enforced by
organisations like the De Nederlandsche Bank also increased management support.
Noteworthy is that FI1 scores average when it comes to the definition of organisational
requirements beforehand. FI1 states that they first want to focus on low hanging fruit (i.e.
infrastructure- and perimeter monitoring), before enrolling crown jewels. FI1 states that after
enrolling low hanging fruit, separate projects have been started to on board crown jewels.
Due to the average score on defining organisational requirements regarding SIEM, we also see low
scorings with regard to the definition of SIEM tooling operational requirements, followed by
market research based upon these requirements.
41
vrije universiteit Amsterdam
PGO IT Audit, Compliance & Advisory
When it comes to the technical preparation of shaping the SIEM tooling, FI1 scores very well. FI1
defines risk scenarios for elements to be monitored. Risk scenarios are then translated into one or
more use cases. FI1 makes use of the cyber kill chain when defining use cases. This shows that the
security team and architects have a good understanding of how hackers work, and what to detect
them in an efficient manner. It is therefore not completely unexpected to see that FI1 scores well
on assigning roles and responsibilities and having a security team with relevant skills. Having a
security team with relevant skills also allows for the good scores we observe in the correlation and
alerting focus areas. These areas require highly specialised skills and dedicated resources in order
to score well.
Most organisations already have a well-defined and implemented log policy for availability and
debugging purposes. However, the security aspects of logging are often not very well defined. This
is reflected in FI1s scores regarding the log management. Here we can see that the log policy itself
scores great but that the security aspects with the policy (such as clock synchronisation) are
mostly defined out of necessity (i.e. due to correlation problems or forensic requirements).
The responding focus area contains only two requirements: the analysis of alerts by analysts, and
the automatic creation of incidents for high-impact incidents. FI1 has an extensive three-line
responding structure, which makes it score very well for the first requirement. No automatic
creation of incidents is configured, for which it scores zero on the second requirement. We
understand that FI1 chose to not implement automatic incident creation since it has an extensive
three-line responding structure that eliminates false positives.
FI1 adjusts use cases if applicable. This is mainly done when use cases generate a large amount of
false positives. Reports that have been agreed upon with specific business stakeholders are
delivered on the agreed interval. There are no global reporting requirements defined. This lowers
the average score for the evaluation focus area.
As a final remark, FI1 scores very well overall. That being said, it is worth to mention that FI1 has
already been working extensively with their SIEM environment and tooling for several years.
42
vrije universiteit Amsterdam
PGO IT Audit, Compliance & Advisory
FI2 does not have a dedicated SOC but instead assigns SOC related roles and responsibilities to the
generic IT security team. In addition, security related tasks and responsibilities are an integral part
of all the different IT departments. The IT security team consists of two dedicated security
professionals. FI2 started using SIEM in 2012 and have been developing their SIEM solution and
supporting processes on a best effort basis.
6.3.2 Results
This section covers how FI2 addresses SIEM. Results are based on interviews.
General interview
FI2 stated that they did not make use of a framework when implementing their SIEM tooling.
Furthermore, FI2 states that the daily security operations and security needs are impacted by
having limited resources available. As such, aspects such as 24/7 monitoring, extensive testing,
dedicated use-case designers and formal documentation/implementation process are not feasible.
However, FI2 does mention that they recognize the tremendous value of these aspects.
Due to the relative small size of the IT security team, a decision was made to collaborate with a
Managed Security Service Provider (MSSP). The SIEM tooling is implemented within the
infrastructure of FI2. Management of the technical configuration and its continuous operation is
done by the MSSP. FI2 is negotiating to outsource more services to the MSSP.
Organisational requirements
FI2s main reason for implementing SIEM is compliance related. The initial DNB self-assessment
revealed the need for a security monitoring solution. This outcome was FI2s driver for
implementing SIEM.
Due to the necessity of SIEM as a result of the DNB self-assessment, management understands and
supports the implementation of SIEM. However, FI2 states that the understanding (and to some
extend support) of management, most likely does not go beyond the original compliance reason.
Due to the compliance driven nature of FI2s SIEM implementation, hardly any organisational
requirements other than compliance and legislative requirements were defined. FI2s strategy was
to have most of their IT infrastructure log everything to their SIEM tooling to filter and sort the
data there.
No real SIEM roles or specific skills are defined and assigned to the members of the IT security
team. However, the IT security team did receive technical training related to the operation and use
of the actual SIEM tooling. Furthermore, the MSSP is responsible for implementing and
43
vrije universiteit Amsterdam
PGO IT Audit, Compliance & Advisory
maintaining the technical SIEM tooling as well as implementing correlation rules and sending out
periodic reports.
Operational requirements
Due to the decision to work with a MSSP, requirements were defined aiming more towards vendor
selection than product selection. FI2s official vendor selection process was used.
FI2 states that no top-down risk scenarios are defined but that to some extend the crown jewels
(on business level) are taken into consideration when determining the objects to monitor. An ad-
hoc approach is used for determining what to monitor. Best practices, own experiences and
previous security incidents are used as input. FI2 states that due to the ad-hoc approach, only a
limited amount of use cases are defined. However, the current approach for determining
monitoring requirements does include, to some extent, common attacks, attack paths and cyber
kill chain phases.
FI2 actively participates in gathering threat intelligence. The MSSP supplies FI2 with a threat
intelligence portal. Based on the gathered threat intelligence new monitoring requirements are
defined by FI2. The rules are implemented by the MSSP.
Log management
FI2 states to have a minimum standards document that contains the minimum requirements for
system settings and configuration options. This document covers logging and monitoring in which
specific logging policies are defined. FI2 states that most, if not all, of our listed log policy
requirements are met and configured.
Specific baselines are created, based on the Microsoft implementation baselines. The baselines
contain the logging requirements as detailed in the minimum standards document.
For the on boarding and modification of devices and systems, the generic change management
process is followed. The change management process includes configuring log settings and
connecting new systems to the central logging facility. In addition, FI2 says that most of the IT
engineers are aware of the logging requirements and configure the correct log policies if missing.
The central SIEM logging facility provides enough capacity for a 6-month retention of all received
log files. Log files are transferred from servers via a dedicated agent (built by the SIEM tooling
vendor). Authorisation rules are defined in the SIEM tooling to prevent unauthorised access to the
system and log files.
No formal decision structure (i.e. based on use cases) was followed in determining which log
sources to monitor. However, a decision was made to first gather IT infrastructure logs (i.e.
network devices, servers, anti-virus, etc.) before collecting application logs. Furthermore, FI2s
strategy to have most of their IT infrastructure log everything to their SIEM did not allow for pre-
emptively selecting which log entries to be collected.
FI2s SIEM tooling has built-in controls for checking if expected log sources are still transmitting
their log files. If anomalies are detected, alarms are raised within the SIEM tooling.
The SIEM tooling normalises received log files based on known log structures. For log files with an
unknown layout, the SIEM tooling provides specific modules for manual interpretation. These
modules are configured and implemented by the MSSP based on the requirements of FI2.
44
vrije universiteit Amsterdam
PGO IT Audit, Compliance & Advisory
Correlation
The MSSP is responsible for the implementation of correlation rules within the SIEM tooling. The
implemented rules are based on the input provided by FI2. However, as stated before, providing
monitoring input is mostly done based on an ad-hoc requirements and only to a small extend based
on pre-defined use cases. In addition, the SIEM tooling provides built-in correlation rules for some
common attacks.
No periodic testing of correlation rules is performed. However, FI2 does state that correlation rules
are checked for the desired result upon initial implementation on an ad-hoc basis.
Authorisation rules allow modification of correlation rules by authorised users only. The MSSP has
full administrative permissions and is allowed to modify system settings and change correlation
rules. FI2 has view privileges within the SIEM tooling and can only view the available information
and alerts. FI2 can view changes made by the MSSP to the system, its correlation rules and the
available data.
Alerting
For the limited set of use cases that is implemented, trigger thresholds and alerting conditions are
defined and implemented within the SIEM tooling. In addition, alerting conditions are configured
for the correlation rules that are the result of the ad-hoc monitoring requirements.
Alerts that result in incident tickets contain detailed information such as log snippets, affected
systems and problem descriptions. Alerts that result in e-mails sent only contain high-level
information such as the problem description. These e-mails do contain a reference to the specific
alert within the SIEM tooling for further information.
Responding
Alerts are viewed and handled by FI2s IT security team. FI2 states that due to resource restraints,
timely responding to alerts is sometimes a challenge. FI2 is looking into a model where it works
together with the MSSP more closely. The MSSP would then assist in handling alerts.
For several high-impact and critical correlation rules, automatic incident tickets are created. These
tickets are than part of the regular incident process.
Evaluation
FI2 and MSSP determine new monitoring requirements and refine current correlation rules in a
bi-weekly tuning call. This call is also used for discussing lessons learned and improvements to be
made. Furthermore, an ad-hoc approach is used for fine-tuning correlation rules based on
identified false positives.
The MSSP sends out daily reports on 6 predefined main topics. Periodic reporting regarding
incidents is covered within the generic incident management process. In addition, the
management team discusses major IT incidents including security incidents.
45
vrije universiteit Amsterdam
PGO IT Audit, Compliance & Advisory
FI2 has a clear primary driver for its SIEM implementation: compliance, passing SIEM
requirements specific to the DNB self-assessment. The management of the bank values being
compliant to the requirements set by De Nederlandsche Bank. Hereby management support and
understanding is covered.
As the results show, only limited organisational requirements were defined prior to the
implementation of SIEM. We believe this could be due the main implementation driver being
compliant to requirements set in the DNB self-assessment. After all, the main focus was on
becoming compliant rather than building a structured and well-thought solution that is an effective
tool for breach detection.
FI2 makes use of services offered by a MSSP. Outsourcing security related tasks brings its
challenges, however, we can clearly see several of our SIEM requirements being covered by the
services offered by the MSSP. Collaborating with a MSSP requires that roles and responsibilities
are well and assigned. Furthermore, it can be expected of a MSSP to have the required skills for
managing the SIEM tooling, its configuration and the implemented correlation rules. FI2
outsourced tasks related to management of the SIEM environment, implementing correlation rules
and periodic reporting to the MSSP.
While FI2 mentions that they see the benefit of defining risk scenarios and use cases, no risk
scenarios are created. We believe this to be the result of not having strong and clear organisational
requirements in combination with limited resources. In turn, the lack of risk scenarios affect the
structured creation of use cases, and thus less effective breach detection. FI2 states that several
use cases have been created on an ad-hoc basis. No process for structurally defining risk scenarios
and use cases is in place. FI2 employs an ad-hoc approach for defining monitoring needs. In spite
of this, we do see that FI2 strongly uses lessons learned as input for defining new monitoring needs.
The gathering of intelligence is actively performed by both FI2 and their MSSP. The MSSP offers a
threat intelligence portal that can be accessed by FI2. The translation of new threat insights into
monitoring needs and ultimately into correlation rules is done using an ad-hoc approach.
Looking at the results within the log management focus area we can observe that FI2 scores quite
well on the technical requirements. Another focus point FI2 scores well on is having the log policy
being part of operating procedures. We believe these results are the product of FI2s strategy of
46
vrije universiteit Amsterdam
PGO IT Audit, Compliance & Advisory
connecting most of their IT infrastructure to the SIEM tooling. Doing this in a structured and
effective way would be impossible without a log policy, baseline and operating procedures.
FI2 does not score well on the requirements covering the determining of log sources and entries
to be collected based on use cases. This is due to the strategy used by FI2: connecting as much log
sources as possible.
FI2 does not check correlation rules on their desired result periodically. FI2 acknowledges that
this is due to the limited resources available. Another reason for this could be due to budget
constrains (i.e. limited budget for penetration tests in combination with monitoring exercises).
FI2 has bi-weekly tuning calls with the MSSP. Amongst other topics, FI2 discusses the need for
adaptation of implemented use cases. Alerts sent out by the SIEM tooling contain detailed
information. High-impact incidents automatically generate incident tickets. The generic incident
management process is used, helping in the structured and timely handling of incidents.
FI2 struggles with making its SIEM implementation more effective due to limited resources. FI2
acknowledges this, and states that it is exploring the possibility to outsource more tasks to the
MSSP. Examples of tasks are the monitoring of changes in the threat landscape as well as the
creation of new risk scenarios and use cases. In our opinion, FI2 should perform a top-down risk
analysis to determine which elements of their IT infrastructure are considered crown jewels. One
can hereby think of business applications and supporting infrastructure such as FI2s Active
Directory environment. Risk scenarios should be defined for the identified crown jewels to bring
the SIEM implementation to a higher maturity level.
47
vrije universiteit Amsterdam
PGO IT Audit, Compliance & Advisory
Operational
Evaluating
requirements
Alerting Correlation
FI1 FI2
The graph clearly shows that FI1 scores higher in all focus areas, except for responding. The focus
area responding only contains two requirements, of which one is the automatic creation of incident
tickets for high-impact use cases (RE1.2). FI1 chose to manually verify all incidents before creating
incident tickets, whereas FI2 does perform automatic incident ticket creation for high-impact use
cases, resulting in a lower score for FI1.
FI1 has a well-prepared approach, starting with understanding and support of management. FI1s
SIEM implementation is focussed on threat detection, where the main implementation driver of
FI2 is compliance. FI1 identified requirements regarding staffing and required knowledge,
resulting in a well-staffed and layered security team.
FI1 performed a risk analysis, defining its crown jewels, prior to the SIEM implementation. Risk
scenarios are defined for identified crown jewels. The risk scenarios are then used as the base for
defining use cases. This approach makes FI1 score well in the focus areas covering organisational-
and operational requirements.
48
vrije universiteit Amsterdam
PGO IT Audit, Compliance & Advisory
FI2s approach is ad-hoc: no risk scenarios are defined prior to the creation of use cases. Use cases
supplied by the MSSP are used. FI2 provides input on what use cases should be implemented in
addition to the standard use cases. Due to this, FI2s scores on the domains organisational- and
operational requirements are significantly lower than FI1s.
The correlation focus area shows the biggest difference between FI1 and FI2. The scores for
requirements CO1.1 and CO1.2 are at opposite ends of the scale. Requirement CO1.2, which states
that correlation rules should be checked for their desired result periodically, requires FTEs and
budget to be performed well. FI1 has dedicated personnel for this, where FI2 works on a best effort
and ad-hoc basis.
The average score of FI1 and FI2 for the log management focus area is almost identical. That being
said, the individual scores shows some interesting differences. For starters, it is unexpected that
FI1 scores lower on LM1.3, which states that the log policy is part of operating procedures. FI2
loses points for the focus areas that require use cases to be defined. Both companies score very
well on LM1.1, which contains an overview of minimum requirements regarding an organisations
log policy. Log management is not only used for SIEM, as it has links with generic IT operations as
well. Next to input for SIEM tooling, log files are used for a variety of generic IT operations tasks
(i.e. troubleshooting, availability monitoring). This might be one reason that both organisations
have log management well covered.
Both companies score reasonably well in the alerting and evaluation focus area. Where FI2 has an
ad-hoc approach for the adjustment of use cases, FI1 has a more structured approach. Both
companies have high-level periodic reporting, where FI1 has specific reports for on-boarded
business crown jewels. Requirements for these reports are set up during the intake procedure.
49
vrije universiteit Amsterdam
PGO IT Audit, Compliance & Advisory
7 Synthesis
Now that we have set up our model and used it to assess multiple IT risk frameworks as well as
two financial institutions, it is time to combine the outcome of our theoretical and empirical
research.
Both financial institutions that were assessed during the case study acknowledge the fact that
attackers using advanced reconnaissance and specialised weapons (i.e. zero-day vulnerabilities)
cannot be stopped by solely implementing preventive measures. Standard anti-virus solutions and
Intrusion Detection Systems might not recognize these specialised weapons, and carefully crafted
(spear-) phishing e-mails could pass an organisations spam filter.
Both theoretical and empirical research have shown that SIEM can be used as a detective measure,
allowing organisations to monitor status and events from all over their IT landscape. Collected
information can be used to perform detailed analysis and correlation to identify irregularities and
security incidents. Timely identifying security incidents allows for appropriate measures and
actions to be taken. In addition to detection, the information provided by SIEM can be used to
respond to security incidents efficiently. This disrupts the cyber kill chain, stopping potential
breaches to an organisations network.
SIEM subject matter experts that assessed our model agreed upon the definition of focus areas.
Both financial institutions that were assessed during the case study also acknowledge that
implementing SIEM requires actions to be taken on various fields, including organisational- and
technical level.
50
vrije universiteit Amsterdam
PGO IT Audit, Compliance & Advisory
Mapping selected IT risk frameworks to our model has shown that coverage greatly differs
between frameworks. ISF SoGP covers the most requirements defined in our focus areas. It scores
particularly well in organisational requirements as well as log management, correlation, alerting
and evaluation. As ISF SoGP is a superset of COBIT 5 and provides comprehensive coverage of ISO
27000 series, we knew beforehand that it would score higher than the two separately.
None of the frameworks covers the setup of risk scenarios in corporation with business
stakeholders, based on crown jewels, legislative requirements and/or previous security incidents.
In addition, none of the frameworks covers the definition of use cases based on risk scenarios,
common attack paths or the cyber kill chain phases.
Both financial institutions assessed during our case study do acknowledge the need for defining
risk scenarios and use cases. Risk scenarios and use cases are crucial in determining what- and
how to monitor, and form the fundament of the intelligence that is built in to the SIEM tooling. As
such, the lack of coverage of these operational requirements could seriously affect the efficiency of
detecting unwanted activities using SIEM.
The case study shows that individual SIEM focus area scores vary greatly; one common point being
log management. Adequate log management shows that organisations understand the need for
efficient and effective SIEM input information management. Both financial institutions
acknowledge that turning this information into relevant alerts and incidents is a challenge. Results
show that organisations defining risk scenarios and use cases know better what to protect and
which information to collect. Not adequately defining risk scenarios and use cases results in
collecting to much data and being overwhelmed with information.
One of the financial institutions that we assessed outsourced parts of their SIEM related activities
to a MSSP. If working with a MSSP it becomes even more crucial to define risk scenarios and use
cases. A MSSP is often unaware of organisation particulars and is dependent on the information
provided by the organisation. Risk scenarios and use cases let the MSSP understand what an
organisations crown jewels are and how the organisation wants to protect them.
51
vrije universiteit Amsterdam
PGO IT Audit, Compliance & Advisory
SIEM stands for Security Information and Event management, as covered in section 4.1. It
combines Security Information Management (SIM), with Security Event Management. (SEM). SIM
focusses on long-term storage of log files, where SEM focusses on real-time monitoring, correlation
of events.
A SIEM collects logs and threat intelligence from internal- (i.e. server-, network- and application
logs) and external sources (i.e. threat intelligence sharing). Event correlation is used to detect and
alert on unwanted activities within the network. Determining what to protect (crown jewels) and
how to monitor this is an integral part of a SIEM solution.
The relevance of SIEM increases by the day as data breaches and company hacks hit the news
regularly. The need for additional security measures grows and focus is shifting from solely
preventive measures to a combination of preventive and detective measures. SIEM is a detective
measure used by an organisations security team (SOC) to detect and respond to security breaches
or compromise of critical assets of (i.e. crown jewels). In some sectors, regulators (i.e. DNB)
enforce detective measures like SIEM. This makes compliance an important driver for SIEM
implementations, next to threat management.
What are focus areas of SIEM and how do IT risk frameworks address them?
We set up a model covering focus areas with minimum requirements that should be addressed
when implementing or assessing a SIEM implementation. The model covers the areas:
organisational requirements, operational requirements, log management, correlation, alerting,
responding and evaluating.
52
vrije universiteit Amsterdam
PGO IT Audit, Compliance & Advisory
We mapped selected IT risk frameworks to our model, assessing to what extent they cover SIEM.
ISF SoGP, being a superset of COBIT 5 and having comprehensive coverage of ISO 27000, has the
best coverage of our focus areas and requirements.
All frameworks lack coverage of SIEM specific requirements such as the ones defined within the
Operational requirements focus area. The most important requirements that are not covered by
the frameworks are the setup of risk scenarios in corporation with business stakeholders, based
on crown jewels, legislative requirements and/or previous security incidents. In addition, none of
the frameworks cover the definition of use cases based on risk scenarios, common attack paths or
the cyber kill chain phases.
Risk scenarios and use cases form the base of the intelligence that is implemented within a SIEM.
As such, the lack of coverage of these operational requirements could seriously impact the
efficiency of detecting unwanted activities using SIEM.
We assessed the SIEM implementation of two financial institutions and conclude that both
institutions address SIEM very differently. The main driver of the first institution is threat
management (detection of- and response to security incidents), where the second institution
implemented SIEM mainly to be compliant to legislative requirements enforced by DNB. Both
financial institutions state that their SIEM implementations are not based on existing frameworks.
Having a different main driver results in the approach of both organisations being very different.
The first institution applies top-down risk management: determining its crown jewels, setting up
risk scenarios and use cases, having a phased infrastructure on boarding approach. It also has a
dedicated security team of seven FTE.
The driver of the second institution is compliance and in contrast to the first institution, it does not
have a dedicated security team. They collaborate with a MSSP who is responsible for management
of the SIEM environment and implementation of rules. Absence of risk scenarios and use cases
result in an ad-hoc approach creating a difficult to maintain SIEM implementation. In turn, affecting
the effectiveness of the SIEM implementation.
53
vrije universiteit Amsterdam
PGO IT Audit, Compliance & Advisory
The size of an organisation determines the shape of several key aspects. Organisations with fewer
FTEs are required to automate or outsource more tasks. With good preparation, that includes the
identification of crown jewels and definition of risk scenarios and use cases, even a small
organisation can achieve a well working SIEM implementation. However, it might need support of
external resources such as a MSSP.
Finally, it does not matter if one outsources tasks to a MSSP or performs all tasks in-house: risk
scenarios and use cases are key for addressing SIEM in practice. It allows for a structured approach
in defining what to monitor and which information to collect. With risk scenarios and use cases as
a solid basis, a SIEM solution can expanded and grow in a sustainable way.
How do IT risk frameworks address SIEM and to what extend does this match current SIEM
implementations?
The selected IT risk frameworks all address SIEM differently. None of the selected frameworks
covers all minimum requirements we deem essential for a successful SIEM implementation. Most
of the selected IT risk frameworks mention determining ones critical assets (i.e. crown jewels). All
frameworks lack coverage of operational requirements such as the definition of risk scenarios and
use cases. Therefore, we conclude that the selected frameworks do not address the very fundament
of SIEM implementations, namely, how crown jewels need to be monitored.
Our field research shows that both financial institutions did not make use of a framework when
implementing their initial SIEM. The large financial institution assessed during our empirical
research used the knowledge of SIEM subject matter experts. This resulted in covering more
requirements of our model than the small financial institution. Specifically when it comes to
operational requirements such as the definition of risk scenarios and use cases.
The small financial institution does acknowledge the need for risk scenarios and use cases to
improve the overall effectiveness of a SIEM for threat management. The usage of one of the selected
frameworks would not have greatly improved their coverage of operational requirements of our
model they now lack.
As such, we conclude that the selected frameworks inadequately address key aspects (i.e. risk
scenarios and use cases) and that as a result, organisations implement their SIEM solution based
on own expertise and that of third parties.
54
vrije universiteit Amsterdam
PGO IT Audit, Compliance & Advisory
A new ISO standard is in the works (ISO 27044) that focusses on SIEM implementations. In our
conclusions, we state that having adequate risk scenarios and use cases are key for successful SIEM
implementations. Based on the subjects in this standard, we see that mainly technical topics are
discussed. As such, it would be interesting to see how the ISO 27044 standard maps to our model.
Furthermore, it would be interesting to see if risk scenarios and use cases are covered and if so, in
how much detail.
One of the financial institutions that was part of our case study partially outsourced SIEM related
tasks to a MSSP. Our model does not cover requirements related to sourcing and MSSPs.
Organisations with only few dedicated security staff might want to partner with a MSSP. Research
regarding MSSP collaboration could provide key insights for specific sourcing requirements,
leading to a more efficient working SIEM.
55
vrije universiteit Amsterdam
PGO IT Audit, Compliance & Advisory
9 Acknowledgments
We would like to express our gratitude to our supervisor Ralf Bardoel whose expertise,
understanding, and patience added considerably to our final result. We appreciate his assistance
and continuous feedback regarding our research and thesis writing. Furthermore, we would like
to thank Ren Matthijsse for the guidance he provided at all levels of the research project. Finally,
we would like to thank Peter Kornelisse to serve as our external reader.
We also acknowledge Erwin Hansen and Stan Hegt of KPMG for providing us with the introductions
at the financial institutions evaluated in this research. Appreciation also goes out to Pieter van
Houten, Jip Hogenboom and Jeroen de Wit for providing valuable and crucial insights regarding
our model of focus areas and minimum requirements.
Finally, we would like to thank all members of the VU faculty staff. Their expertise helped us along
the way and allowed us to perform our research in the first place.
56
vrije universiteit Amsterdam
PGO IT Audit, Compliance & Advisory
10 Bibliography
AccelOps. (2013, February 12). Top 10 SIEM Implementer's Checklist. Retrieved from AccelOps:
http://www.accelops.com/media/1700/Top_10_SIEM_Implementer_Checklist.pdf
Bardoel, R. (2015, June 5). Personal interview. (J. v. Moosdijk, Interviewer)
BBC. (2014). Hack attack causes 'massive damage' at steel works. Retrieved from BBC:
http://www.bbc.com/news/technology-30575104
Chuvakin, A. (2010). The Complete Guide to Log and Event Management. Novell.
Chuvakin, A. (2011, April 22). So you got that SIEM. NOW what do you do? So you got that SIEM.
NOW what do you do? Security Warrior Consulting. Retrieved from
http://www.slideshare.net/anton_chuvakin/so-you-got-that-siem-now-what-do-you-
do-by-dr-anton-chuvakin
Cisco. (2014). 2014 Annual Security Report.
Constantine, C. (2014, March 18). What kind of logs do you need for an effective SIEM
implementation? Retrieved from Alien Vault:
https://www.alienvault.com/blogs/security-essentials/what-kind-of-logs-for-effective-
siem-implementation
DNB. (2014, May 22). Toelichting op Toetsingskader Informatiebeveiliging 2014. Retrieved from
DNB: http://www.toezicht.dnb.nl/binaries/50-230767.pdf
Dorigo, S. (2012). Security Information and Event Management.
EiQ Networks. (2013, March 6). EiQ Networks Survey Reveals Organizations Are Suffering From
SIEM Deployments. Retrieved from EiQ Networks:
http://www.eiqnetworks.com/abouteiqnetworks/news/pressrelease/2013/eIQnetwor
ks-Survey-Reveals-Organizations-Are-Suffering-from-SIEM-Deployments.php
Gartner. (2012, August 9). On People Running SIEM. Retrieved from Gartner:
http://blogs.gartner.com/anton-chuvakin/2012/08/09/on-people-running-siem/
Gartner. (2012, July 30). On SIEM Processes/Practices. Retrieved from Gartner:
http://blogs.gartner.com/anton-chuvakin/2012/07/30/on-siem-processespractices/
Gartner. (2014). 2014 Magic Quadrant for SIEM.
Gartner. (2014, June 30). Evaluation Criteria for Security Information and Event Management.
Retrieved from Gartner: https://www.gartner.com/doc/2785317/evaluation-criteria-
security-information-event
Gartner. (2014, March 26). How to use threat intelligence with your SIEM? Retrieved from
Gartner: http://blogs.gartner.com/anton-chuvakin/2014/03/26/how-to-use-threat-
intelligence-with-your-siem/
Gartner. (2014, September 15). SIEM Technology Assessment and Select Vendor Profiles. Retrieved
from Gartner: https://www.gartner.com/doc/2846817/siem-technology-assessment-
select-vendor
57
vrije universiteit Amsterdam
PGO IT Audit, Compliance & Advisory
Hanley, M., & Montelibano, J. (2011). Insider Threat Control: Using Centralized Logging to Detect
Data Exfiltration Near Insider Termination. Software Engineering Institute.
Harrel, C. (2014, September 1). SIEM Use Case Implementation Mind Map. Retrieved from Journey
Into Incident Response: http://journeyintoir.blogspot.nl/2014/09/siem-use-case-
implementation-mind-map.html
Houten, P. v. (2009). Audit Trail Management. Amsterdam: University of Amsterdam.
HP. (2014, March). HP Attack Life Cycle use case methodology. Retrieved from HP:
http://h20195.www2.hp.com/v2/getpdf.aspx/4aa4-9490enw.pdf
HP. (2014). Security Information & Event Mangement. Retrieved from HP:
http://www8.hp.com/nl/nl/software-solutions/siem-security-information-event-
management/
Huang, D., Emily, G., & Yadron, D. (2014, November 17). Financial Firms Bolster Cybersecurity
Budgets. Retrieved from The Wall Street Journal:
http://www.wsj.com/articles/financial-firms-bolster-cybersecurity-budgets-
1416182536
IBM. (2014). IT Executive Guide to Security Intelligence - Transitioning from SIEM to Total Security
Intelligence.
Infosec Institute. (2014, May 15). Top 6 SIEM Use Cases. Retrieved from Infosec institute:
http://resources.infosecinstitute.com/top-6-seim-use-cases/
InfoSec Nirvana. (2012, June 21). Siem Use Cases - What you need to know? Retrieved from InfoSec
Nirvana: http://infosecnirvana.com/siem-use-cases/
ISACA. (2012). COBIT 5: Business Framework for the Governance and Management of Enterprise IT.
ISACA. (2013, 05 04). COBIT 5 for Information Security. Retrieved from ISACA:
http://www.isaca.org/COBIT/Pages/Information-Security-Product-Page.aspx
ISF. (2014). The Standard of Good Practice for Information Security.
ISO. (2013). ISO/IEC 27001:2013.
Kent, K., & Souppaya , M. (2006). Guide to Computer Security Log Management (SP 800-92). NIST.
Krebs, B. (2014). Sony Breach May Have Exposed Employee Healthcare, Salary Data. Retrieved
from KrebsonSecurity: http://krebsonsecurity.com/2014/12/sony-breach-may-have-
exposed-employee-healthcare-salary-data/
Lewis, R. (2014, September 14). Auvik. Retrieved from Top 7 Success Factors for Setting Up
Centralized Logging: https://www.auvik.com/media/blog/centralized-logging-
checklist/
LockHeed Martin Corporation. (2011). Intelligence-Driven Computer Network Defense Informed
by Analysis of Adversary Campaigns and Intrusion Kill Chains. Annual International
Conference on Information Warfare & Security, (p. 14).
McAfee. (2014). When Minutes Count. Intel Security.
58
vrije universiteit Amsterdam
PGO IT Audit, Compliance & Advisory
Netmonastery. (2014). Key SIEM use cases to ensure you never get compromised.
Netwrix. (2014). SIEM Efficiency Survey Report. Irvine: Netwrix Corporation.
NIST. (2013). Security and Privacy Controls for Federal Information Systems and Organisations (SP
800-53). NIST.
NIST. (2014, 02 12). Framework for Improving Citrical Infrastructure Cybersecurity. Retrieved
from NIST: http://www.nist.gov/cyberframework/upload/cybersecurity-framework-
021214.pdf
NSA. (2013). Spotting the Adversary with Windows Event Log Monitoring. National Security
Agency/Central Security Service.
PCI Security Standards Council. (2013). Payment Card Industry - Data Security Standard 3.0.
PvIB. (2011). Security Operations Center: Een inrichtingsadvies. Retrieved from PvIB:
https://www.pvib.nl/download/?id=17670673
Rorive, K., Beerends, M., Bordewijk, L., Breedijk, F., Cimen, H., Cornelissen, J., . . . Smulders, A.
(2011). Platform voor Informatie Beveiliging, Expertbrief Security Operations Center.
SANS. (2009, September 21). Effective Use Case Modeling for Security Information & Event
Management. Effective Use Case Modeling for Security Information & Event Management.
SANS Institute.
SANS. (2010). Successful SIEM and Log Management Strategies for Audit and Compliance. SANS
Institute InfoSec Reading Room.
SANS. (n.d.). Critical Security Controls fro Effective Cyber Defence. Retrieved from SANS:
https://www.sans.org/critical-security-controls/
SANS, Shackleford. (2012, June). When Breaches Happen: Top Five Questions to Prepare For.
Retrieved from SANS: http://www.sans.org/reading-
room/whitepapers/analyst/breaches-happen-top-questions-prepare-35220
SC Magazine UK. (2012, December 11). Constraints lead to failure to effectively manage SIEM
systems. Retrieved from SC Magazine for IT Security professionals:
http://www.scmagazineuk.com/constraints-lead-to-failure-to-effectively-manage-siem-
systems/article/272068/
Schinagl, S., & Schoon, K. (2014). Security Operations Center: Modelleren en meten van effectiviteit.
Securosis, L.L.C. (2014, February 1). Security Management 2.5: Replacing Your SIEM Yet? Phoenix:
Securosis. Retrieved from mcafee.com: http://www.mcafee.com/nl/resources/white-
papers/wp-replacing-your-siem-yet.pdf
Taschler, S. (2015, April 8). McAffee Community. Retrieved from SIEM Foundations: Verify That
All Data Sources Are Logging as Expected: https://community.mcafee.com/docs/DOC-
6234
Thyer, J. (2015, November 25). Hunt teaming. Let's face it, you are probably compromised. What
next? SANS Institute.
59
vrije universiteit Amsterdam
PGO IT Audit, Compliance & Advisory
60
vrije universiteit Amsterdam
PGO IT Audit, Compliance & Advisory
11 Appendices
This section contains appendices that are referenced within this document.
61
vrije universiteit Amsterdam
PGO IT Audit, Compliance & Advisory
Please note that when multiple frameworks map to a requirement, it does not mean that they cover the exact same load. This due to the IT risk frameworks having
a different level of granularity on topics. We have included matches where wording may vary, although semantic intent is similar.
62
vrije universiteit Amsterdam
PGO IT Audit, Compliance & Advisory
Operational requirements
OPR1.1 SIEM tooling requirements are defined 14.1.1 BAI02.01 - 1,2 DE.DP-2 CF9.1.3
CF10.4.2
CF10.4.4
OPR1.2 SIEM tooling is selected based on market research using defined APO10.02 CF16.1.1
requirements BAI02.02 - 1
OPR1.3 SIEM tooling is implemented BAI03.05 CF8.7.3
CF10.4.8
CF11.1.12
63
vrije universiteit Amsterdam
PGO IT Audit, Compliance & Advisory
Log management
LM1.1 Log policies are defined and should at least include:
- Logs should be accurate 12.4.1 BAI10.01 PR.DS-6 10.5 SR1.4.1
DSS01.03 CF10.4.2
- Logs should be stored in a central and tamper-proof location 12.4.2 PR.DS-1 CF10.4.6
CF10.4.2
- Log retention period per log type (based on needs and 16.1.7 DSS05.07 - 1 PR.PT-1 SR2.1
legislation) CF10.4.10
- Logs are time stamped 12.4.1 PR.PT-1 10.3 CF10.4.5
- The same time source is used for clock synchronisation 12.4.4 PR.PT-1 10.4 CF10.4.5
- Log permissions are restrictive 12.4.2 PR.PT-1 10.4 CF6.1.3, 4
10.5 CF10.4.2
- Log format (i.e. syslog, windows event logs) PR.PT-1 CF10.4.5
- System is not halted through lack of disk space 12.4.2 PR.DS-4 CF10.4.7
- Logs contain originating source 12.4.1 PR.PT-1 10.3 CF10.4.5
LM1.2 Technical baselines are defined and implemented based on log BAI10.02 PR.IP-1 CF10.4.5
policy PR.PT-1
64
vrije universiteit Amsterdam
PGO IT Audit, Compliance & Advisory
Correlation
CO1.1 Use cases are translated into specific correlation rules within DA.AE-3 CF10.4.8
the SIEM tooling
CO1.2 Correlation rules are periodically checked on desired result PR.IP-7 CF10.4.9
DE.DP-5
CO1.3 Modification of correlation rules is limited to authorised users 9.4.1 DSS05.04 - 1 PR.AC-4
only DSS06.03 - 1, 2
65
vrije universiteit Amsterdam
PGO IT Audit, Compliance & Advisory
Responding
RE1.1 Alerts are analysed by the SOC, and sent to incident 16.1.1 DSS02.02 - 1 RS.AN-1 12.5.2 CF10.4.10
management process if applicable DSS05.07 - 3 RS.AN-2 12.5.3
RE1.2 Alerts generated by high impact use cases automatically DSS05.07 - 5
generate an incident
Evaluation
EV1.1 Use cases are adjusted if applicable 16.1.6 MEA02.01 DE.DP-5 CF11.1.7
EV1.2 Reports are defined on the needs of the organisation EDM05.01 - 1, DE.DP-4 SI2.1
2, 3
66
vrije universiteit Amsterdam
PGO IT Audit, Compliance & Advisory
67
vrije universiteit Amsterdam
PGO IT Audit, Compliance & Advisory
Operational requirements
OPR1.1 SIEM tooling requirements are defined +
OPR1.2 SIEM tooling is selected based on market research using defined
requirements
+
OPR1.3 SIEM tooling is implemented ++ ++
OPR1.4 Risk scenarios are defined based on: ++
- Crown jewels +/
- Risk appetite +
- Legislative and regulatory requirements +
- Previous security incidents +
OPR1.5 Use cases are defined based on: ++
- Scenarios ++
- Common attacks and attack paths ++ +/
- Cyber kill chain phases ++ +/
OPR1.6 Threat intelligence (TI) is gathered from (external) sources ++ +/
OPR1.7 TI and changes in threat landscape are used to create/modify use cases
periodically
++ +/
Log management
LM1.1 Log policies are defined and should at least include: ++ ++
- Logs should be accurate + +
- Logs should be stored in a central and tamper-proof location + +
- Log retention period per log type (based on needs and legislation) + +
68
vrije universiteit Amsterdam
PGO IT Audit, Compliance & Advisory
LM1.2 Technical baselines are created and implemented based on log policy ++ ++
LM1.3 Adhering to the log policy is part of operating procedures such as: +/ ++
- On boarding of new devices and systems +/ ++
- Changing devices and systems +/ +
LM1.4 Centralized log requirements are defined ++ ++
- Only authorised users have access to log files ++ ++
- Capacity of centralised logging infrastructure is managed +/ +
LM1.5 Log sources are derived based on use cases +
LM1.6 To be collected log entries are defined based on use cases +
LM1.7 Controls for checking if expected logs are received are implemented ++ ++
Correlation
CO1.1 Use cases are translated into specific correlation rules within the SIEM
tooling
++
CO1.2 Correlation rules are periodically checked on desired result ++
Alerting
AL1.1 Triggers and alerts defined in use cases are implemented ++ +/
AL1.2 Alerts contain relevant information ++ ++
Responding
RE1.1 Alerts are analysed by the SOC, and sent to incident management process if
required
++ +
RE1.2 Alerts generated by high impact use cases automatically generate an
incident
++
Evaluation
EV1.1 Use cases are adjusted if applicable + +/
EV1.2 Reports are defined on the needs of the organisation +/ +/
69