Sie sind auf Seite 1von 10

W h i t e p a p e r

f r o m

P i v o t

957 Rt. 33, Suite 11, Hamilton, NJ 08690 609.581.4600

www.pivotpointsecurity.com

P o i n t

S e c u r i t y

Stop Wasting Money on


Penetration Testing

Contents
Introduction .................................................................................................................................................. 3
Optimally Scoping a Penetration Test........................................................................................................... 4
Examples of Well-Scoped Penetration Tests ................................................................................................ 7
Determining if a Well-Scoped Penetration Test is Sufficient ........................................................................ 8
Final Thoughts ............................................................................................................................................. 10

[2]

Stop Wasting Money on Penetration Testing


Visit us at www.pivotpointsecurity.com

Introduction
It may seem especially odd that we would publish a white
paper entitled Stop Wasting Your Money on Penetration
Testing when 40% of our revenues are generated by
Penetration Testing. To be clear we are not advocating
that penetration testing has no value rather, that
penetration tests are often used in a manner that is
inconsistent with achieving the assurance an organization
seeks. This is exacerbated by a growing (and incorrect)
perception that penetration testing is a commodity and
therefore all penetration tests are the same. In reality, they
are not defined and executed with equal extent and rigor.

To be clear we are not


advocating that Penetration
Testing has no value,
rather, that Penetration
Tests are often used in a
manner that is inconsistent
with achieving the
assurance that the
organization seeks.

We believe there are two predominant issues that cause


organizations to waste money on penetration testing:
Improperly scoping a penetration test in a manner that will reduce the level of assurance that is
achieved or provides assurance in areas not required; and,
Using a penetration test as a cure-all where it will not provide the level of assurance desired
or where another security assessment mechanism would provide greater value at similar cost.
In our experience, organizations that address these two issues:
Significantly reduce the costs associated with penetration testing by scoping the engagement in
a manner specific to the assurance being sought; and,
Achieve notably higher levels of assurance by aligning the testing directly with objectives or
shifting the engagement to other more appropriate activities.

Agreeing on the Definition of a Penetration Test


Unfortunately, there is a significant lack of clarity on how a penetration test is defined, what it entails,
and what it provides. For the purpose of this paper, we will assume that a Penetration Test is
comprised of two phases: a Vulnerability Assessment Phase and a Penetration Testing Phase.
A Vulnerability Assessment is a fully automated process wherein a vulnerability scanner (a
computer loaded with vulnerability assessment software) communicates with the target
environment to identify vulnerabilities relating to improper system configuration and
hardware/software flaws. With the exception of entering the target data, humans are generally
not involved in the Vulnerability Assessment phase.

[3]

Stop Wasting Money on Penetration Testing


Visit us at www.pivotpointsecurity.com

Penetration Testing is the process wherein


appropriately qualified security professionals
leverage the output of the Vulnerability
Assessment in combination with additional
software tools to determine the probability
that the identified vulnerabilities can be
exploited and, if so, the resulting impact to
the organization.

Optimally Scoping a Penetration Test


Optimal scoping of a penetration test relies on a clear
definition of test objectives followed by the
identification of activities that will meet those
objectives. The execution of those activities should
provide the desired level of assurance. The more
accurately the scope and objectives are defined, the
more cost-effective and precise the assessment will
be.
While it certainly sounds easy enough, in practice it
can be a bit challenging - even when you do this for a
living. It is remarkable how frequently the initial
response to the question What are the objectives of
the penetration test? is We want to know how
secure we are. (or a similar variant).

Scoping Good Practices


A few easily defined steps can help to increase the
value of penetration testing:
1. Define the critical risks for the environment in
question;
2. Confirm the business objectives for the
testing;
3. Validate that penetration testing is the
optimal activity to achieve key objectives ;
and,
4. Determine the scope, activities, extent and
rigor of the testing to achieve the business
objectives.
[4]

A Painful Example of a Poorly Scoped


Penetration Test
Early on in my career when I was not as experienced in
helping our clients optimize scope we conducted a nonoptimized test that illustrates how a poorly scoped
penetration test can fail to provide the desired assurance
and waste money. The new CIO of a fairly large
government entity insisted that the only objective of the
test was to provide him an overall measure of his current
security posture. Im new; I need a pen test to get a quick
sense of where we are. I need to know where my biggest
problems are so that I can prioritize where to start. Based
on his guidance we defined a fairly large external and
internal penetration test with an appropriate level of
sampling across the entire organization to ensure that we
could achieve his mandate.
As the CIO had suspected a combination of poor patch,
configuration, and password management practices had
allowed our penetration testing team to garner significant
administrative level access to a number of the
organizational domains, including all of the tax information
for the county. Penetration Test report in hand we met
with the CIO presuming that we had fully met his
objectives. We were dismayed to find that we had not. I
know that you got into all of our tax systems and gained
access to deeds and mortgages but for the most part that
is public information, so its not that important. I would
really like to know if you could have made the same inroads
into the payroll system or the databases that back-end our
Elder Care Services. Seeing is believing, and despite the
fact that it could be confidently assumed that the high level
of Administrative level privilege we had achieved would
have allowed us to gain access to any system in relatively
short order, the CIO wanted irrefutable evidence that those
systems were at risk.
Had we optimally scoped the project by more specifically
targeting the testing we could have reduced the cost of the
engagement by more than 60% and provided the CIO the
specific assurance he actually wanted. This particular
engagement was a painful one for us as we re-tested the
areas at no additional cost to the CIO to ensure that we
achieved his objectives. Fortunately, the story ends well:
the CIO is still our client five years hence and the

environment is now secured in a manner consistent with


prevailing good practice.

Stop Wasting Money on Penetration Testing


Visit us at www.pivotpointsecurity.com

1. Defining Critical Risks


A penetration test is a substantiative test designed to measure the probability that the cumulative
vulnerabilities of the target can be exploited and determine the attacks impact on the organization. In a
sense it is a measure of residual risk.
Determining risk typically involves garnering an understanding of the following aspects of the target
environment:

Business requirements for the environment (applications/systems/information) ;


Existing policies, standards and procedures;
Relevant laws and regulations;
The information technology in use; and,
Critical roles and responsibilities of those who maintain the environment.

2. Confirming Business Objectives


As cited above, the prime business objective of a penetration
test is to determine whether an organizations security can be
compromised by exploiting a targets vulnerabilities.
However, when most people discuss risk in the context of a
penetration test they generally focus on technical risk. In most
penetration tests the risk associated with not achieving
business objectives are actually more critical than the
technical risks.

In most penetration tests


the business risks are
actually more critical than
the technical risks.

For example:
For a bank, the failure to conduct an appropriate penetration test can result in regulatory
sanctions;
For a service provider the inability to provide third party attestation to their security posture can
result in the loss of critical customer contracts; and,
For a federal agency, the failure to conduct appropriate security certification and accreditation
activities (including penetration testing) can result in non-compliance with federal laws and
regulations.
3. Validating that a Penetration Test is the Optimal Tool to Achieve Objectives
While penetration tests are often an effective mechanism to rapidly gain a perspective on the current
security posture it is important to note there are many critical elements of security that are not directly
assessed (e.g. system logging 1, incident response). That is why it is especially dangerous to rely on
penetration testing as the sole form of validation that security controls are appropriately designed, in
place, and operating as intended.
1

System logging can be addressed if a Vulnerability Assessment is run with administrative privilege.
[5]

Stop Wasting Money on Penetration Testing


Visit us at www.pivotpointsecurity.com

When Penetration Testing


Is Not Enough
Several years ago we compromised a
third party hosted fund transfer
application for a major bank during a
penetration test and demonstrated the
ability to move $500M between
accounts. On review, the application
had never been compromised during biannual penetration testing conducted
by three different security firms. What
allowed us to compromise the system
was the use of an easily guessable
password (the companys name) on a
critical server. No other firm had tried
that particular password for the
administrator account. Worse, once we
gained access we found that there was
insufficient segregation between critical
systems and different clients
environments, logging was not enabled
(so breaches would not be detected) ,
and critical systems could directly
access the Internet putting them at risk
to malware infection.

All of these issues would be


easily identified via other
security assessment
activities but cannot be
identified by a penetration
test alone unless another
vulnerability is leveraged
that provides administrative
level access.

[6]

4. Optimizing the Penetration Test Scope


There are several key areas that need to be
properly scoped to ensure success:
Physical Scope The network segment(s),
network infrastructure, and system(s) that
will be tested;
Logical Scope Information, applications,
services, personnel and processes that are
critical to securing the information systems.
(This is often the most challenging portion
to scope due to the transitory nature of
data);
Extent The domains to be tested
(Physical, Human, Wireless, Data Network,
Telecomm);
Activities - The penetration testing
activities allowable within the scope and
context of the testing:
o Reconnaissance activities,
o Social Engineering, and
o Modalities and postures to be
leveraged during the testing; and
Rigor How thorough will the testing be?
What will the sampling rates be? What
percentage of the vulnerabilities will be
tested? How many hours will the test team
spend on the engagement? What is the
level of assurance being sought (i.e. against
a casual hacker with moderate skills that
happens upon the clients infrastructure or
against a disgruntled IT Security Admin with
a deep understanding of the clients
security defenses)?

Stop Wasting Money on Penetration Testing


Visit us at www.pivotpointsecurity.com

Two Examples of Well Scoped Penetration Tests


Understanding what is meant by a well-scoped penetration test is best demonstrated by example.
Below are two examples of well-defined objectives and the activities, extent and rigor intended to
achieve them.

Software As A Service Vendor (SAAS) (Provider of online relationship management software)


Objectives
Ensure that a non-authenticated user attempting to access the systems and applications
comprising the solution cannot garner access to confidential information.
Ensure that an authenticated user of the application cannot escalate their privilege to gain
access to data that is restricted beyond their level of privilege.
Ensure that an authenticated user in one company cannot gain access to the information for
another company and vice versa.
Ensure that the firewall modifications in place support certain companies enhanced database
access requirements and do not allow users to gain database level access beyond that which is
intended.
Ensure that the communications architecture with the credit card processor precludes the SAAS
from PCI-DSS compliance and that the architecture is not vulnerable to a man-in-the-middle
attack or hidden field manipulation.
Resultant Activities
Network Vulnerability Assessment and Penetration Test limited to the exposed architecture of
the solution (firewall, router, application server, web server). Testing was conducted from two
locations to mimic unauthorized access and privileged access. Testing was conducted from a
Grey/Grey perspective.
Application Vulnerability Assessment and Penetration Test limited to the core application.
Testing was conducted from three user modalities (un-privileged, normal user, power user)
across two different companies. Testing was conducted from two locations to mimic
unauthorized access and privileged access. Testing was conducted from a Grey/Grey
perspective.
A Database Vulnerability Assessment was conducted to ensure that the higher level of access
allowed to certain customers did not provide a higher level of access than anticipated or allow
access to another customers data.
A Firewall Rule-Base Review was conducted to substantiate that the rule-base was consistent
with the desired level of access.
Specific tests were conducted to determine if hidden fields could be manipulated to the
detriment of the SAAS vendor or its clients.
A Design Review of the communications architecture with the card processor was conducted to
ensure that it was not vulnerable to a MITM (Man in the Middle) attack.

[7]

Stop Wasting Money on Penetration Testing


Visit us at www.pivotpointsecurity.com

Telecommunications Provider (Provider of private/internet connectivity to companies/individuals)


Objectives
Ensure that the carrier network is appropriately segregated from the corporate network.
Ensure that the current security monitoring mechanisms are effective in detecting and
responding to potential security incidents.
Validate that existing patch management and configuration management systems are effective
in standardizing configurations and minimizing vulnerabilities.
Resultant Activities
Network Discovery and Vulnerability Assessment from four different points of origin (aligned
with predominant customer types). Testing was done from a White Hat/Black Hat perspective
to ensure that Security Monitoring and Incident Response teams were not aware of our
activities. In order to assess the Security Monitoring and Incident Response, testing was
conducted over a week and was initiated from a Secret Ops visibility level where great effort
was made to hide our activities. The testing was then incrementally transitioned to a stealth
visibility where moderate effort was made to conceal our activities. Finally testing moved to a
quiet visibility level that mirrors a normal malicious individuals behavior. The 25%
host/device sampling rate and 50% subnet sampling rates were chosen based on risk, business
unit, and business services with an objective of achieving a 95% confidence interval using the
Central Limit Theorem.
Penetration Testing from the same points of origin and testing visibility noted above.
A secondary Vulnerability Assessment was conducted with administrative privilege across a
sampling of servers and network devices to baseline configuration against corporate standards.
A Network Segregation Review wherein the configuration of all routers and firewalls was baselined against objectives and prevailing good practice. Threat and access maps showing all
combinations of access from all customer vantage points were provided.

Determining if a Well-Scoped Penetration Test is Sufficient


Generally, we find that organizations utilize penetration testing in three distinct ways:
As an inexpensive means to gain some level of independent assurance that can be provided to a
third party (e.g. external auditor, business partner, or customer);
As a mechanism to identify risks in a quick, inexpensive manner and prioritize additional security
assessment activities or remediation efforts; and,
As part of a broader security assessment where the risk and impact are significant. In many
instances this is done as part of a more formal security certification and accreditation. Ideally,
the purpose of the activity is to garner a higher level of assurance.

[8]

Stop Wasting Money on Penetration Testing


Visit us at www.pivotpointsecurity.com

Where risk is low, and the main business objective of the engagement is to garner a third partys opinion
for attestation or independent review, then a stand-alone penetration test may be sufficient.
However, where risk is higher or the objectives of the engagement are more complex, then a more
complete security audit is likely warranted. A full security audit provides a much higher level of
assurance by integrating all three core elements of a security assessment:
Design Assessment activities which evaluate the appropriateness of controls by comparing the
control design against the client's control objectives, industry good practice, laws/regulations,
and/or the auditor's professional judgment (e.g. an Application Architecture Review);
Compliance Assessment activities which validate that the control measures established are
working as designed, consistently, and continuously (e.g. a Password Audit); and,
Substantiative Assessment activities that provide the auditee with assurance that the "net"
control objectives are being achieved, and where they are not, provides a measure of
probability and business impact (e.g. a penetration test).
When executed in combination, an organization receives the highest level of assurance possible, which
includes:
The design of the controls are reasonable and appropriate to the risk and in accordance with
organizational standards and prevailing good practice;
Evidence is gathered to validate that the organization is complying with the controls that have
been evaluated; and,
Appropriate tests are performed to validate that controls are effectively mitigating the risks
specified.
For example, when looking at Network Security we may choose some combination of the following
activities based on the network's complexity, risk profile, and the importance of specific controls to
reducing risk. The order they are presented in may or may not be the logical order that a particular
situation may warrant.

[9]

Stop Wasting Money on Penetration Testing


Visit us at www.pivotpointsecurity.com

Final Thoughts
A penetration test is a lot like a bikini, what it reveals is suggestive but what it doesnt may be vital. That
is, Penetration Tests infer the net effectiveness of the environment by statistically sampling those
control elements that can be readily analyzed from the testing perspective. Because penetration tests
can often tell you a lot about your environment in a relatively short time frame, they can be an excellent
mechanism to provide assurance or rapidly identify control deficiencies. However, appropriately
defining the scope is critical to achieving a reasonable level of assurance.
Where risk is higher, a penetration test probably is not the optimal way to gain assurance and should be
supplemented with other security assessment activities to ensure the testing is reasonable and
appropriate to the risk. Remember:

Dont waste money on penetration testing;

Make sure its what you need; and

Then make sure the bikini fits!

[10]

Stop Wasting Money on Penetration Testing


Visit us at www.pivotpointsecurity.com

Das könnte Ihnen auch gefallen