Exploring "Big data"

Security Analytics: Use

Cases and More
Dave Shackleford, SANS and Voodoo Security
Mark Seward, Sr. Director, Security and Compliance, Splunk
2012 The SANS" Institute - www.sans.org
Recap: Webinar #1

- Security and Big Data: What's all
the Hype About?
- Defined "Big data
- Core use cases
- Incident Response
- Root cause analysis
- Security intelligence
- KPI analytics
2
2012 The SANS" Institute - www.sans.org
The Need for Security Intelligence
- More and more, we need bigger data
sets to analyze
- IT operational data can provide
incredibly useful context and
correlation points
- DB information
- App data
- OS data
- SIEM gives us a
lot of info, but not
enough

2012 The SANS" Institute - www.sans.org
3
3
Security Relevant
Data
All Security
Relevant Data
SIEM
"Ok, Ok, I'm convinced.
Big data and the big
data phenomenon is
real. So what do I do
with it?"
2012 The SANS" Institute - www.sans.org
4
The Big Data Powered Business
Less `Gut Feeling -
More `Evidence based
decisions
Seen as a way to
increase top line
revenues and reduce
expenses
New dependence on
understanding the data
Hurting my business
means messing with,
stealing, or interrupting
the flow of my data and
`changing my decisions
Epsilon leverages big
data and analytics.
Revenues increased by
20%.
IBM

60% potential increase
in operating margins
possible with big data.
McKinsey and Co. June
2011

The Need to 'Think Differently'
6
Creativity
consists of

Convergent
and
Divergent
Thinking
Big-data and Creative
Security Thinking
7
Divergent Thinking:
The Aha moment / Spontaneous epiphany
Remote associative processes
Pattern-based thinking
Convergent Thinking:
About analysis and attention
The act of `un-concealing - chiseling away at a
problem
Write a symphony / poem / solve an algebraic
equation
Stick with a problem till it `cries uncle
Security Intelligence Requires:
'Thinking Like a Criminal'
Application
Data
Netflow
DHCP
DNS
Physical
Security
VPN GPS AD/LDAP
9 Whats the modus operandi of the
attacker?
9 What are the most critical data sets
owned by the business?
9 What physical or virtual assets have
the data?
9 What patterns of weak-signals in
`normal IT activities would represent
`abnormal human or machine
behaviors?
`Normal IT
Services Data
Where are my big data
experts?
Traditional security folk
plus.domain knowledge
Where will `big data
experts be hired
Constituents will be
partners and partners
constituents
`Hub and Spoke design
Fosters data-driven
decision making


Meet your new virtual security team
Finance
Team
Business
Line
Owners
Finance
Team
Legal
Department
IT
Operations
Finance
Team
Development
Business
Service
Providers
Traditional
Security
Team

"Lets define a new thinking
process"
2012 The SANS" Institute - www.sans.org
10
Big Data, Big Thinking, New Process
11
What will cause the
business to stop
functioning?
Whats normal?
Data SMEs from the
business and security
teams figure out - `whats
normal and what would
not be normal
Analysis options
categorized with
combinations of R/T and
historic searches
Support for agile
interpretation and iteration

Adapted fromThe "Human Element" of the Big Data Equation by Steve Durbin ISF , CRM
magazine, November 2012
Copyright 2012, Splunk Inc.
Using the Process -
Example
The Steps The Response
Business Issue Service degradation causes monetary
damage and customer satisfaction
issues.
Construct one of more
hypothesis (team creativity
required)
Unwanted bots can degrade service and
steal content.
Gather data sources and
expertise
What combinations of data would be
considered definitive evidence? What
might be the first signs of trouble? List
all data in which this might be reflected.
Determine the analysis to
be performed
Determine the types of data searches
appropriate
Interpret the results Do the results represent false positives of
false positives or false negatives? Are
there good bots and bad bots?
12
Copyright 2012, Splunk Inc.
Detecting Account Take-over
Statistical analytics and
thresholds
Behavior of logins and password
changes and resets
Analysis of same IP - multiple
password resets
Multiple IPs -- resetting the same
account
How many times people
change their bank information
How many times they change
their credit card information
Does the IP address (location)
match the browser language
or time zone
Unknown Threat Attack
Pattern -- Example
14
Attack Pattern Modeling -
Questions to Ask
15
Is this the first time this
person has received email
from the recipient?
Is the website in the email on
a known list of bad websites?
Are their changes to host
config files closely tied to a
website visit?
If so - import PCAP and
Flowdata
Are there DNS requests to
known bad sites or are the IP
addresses of the DNS URL
request and responses the
same or different?
Monitor port and protocol
usage unusual amounts or
types
Host
based
Analytics
Network
based
Analytics
Is Big Data Changing Security?
Oh yeah.
- Zions Bancorporation presented at
RSA 2012 on how analytics would
change their security model
forever
- The goal? Actionable, real-time
security intelligence over
petabytes of data.
2012 The SANS" Institute - www.sans.org
16
Zion Case Study: Components
- Looked to drive deeper forensics and
build complex stats models
- Needed years of data
- Logs are still centralized
- Using Hadoop and unstructured data
file stores
- Storing:
- DB logs
- FW logs/events
- Antivirus logs
- IDS logs
- Wire ACS transfers
- Credit data
2012 The SANS" Institute - www.sans.org
17
Even More Use Cases
- Fraud Detection
- Patterns of user behavior vs. "other
users
- Intellectual Property Theft
- Data access patterns over long
time periods, with many sources
- Security Monitoring Optimization
- Where are best locations for
sensors and event monitoring?
- What are best/optimal data
sources?
2012 The SANS" Institute - www.sans.org
18
So What is Splunk?
2012 The SANS" Institute - www.sans.org
19
Copyright 2011, Splunk Inc. Listen to your data.
Customer
Facing Data
Outside the
Datacenter
Applications

Web logs

Log4J, JMS, JMX

.NET events

Code and scripts

Networking

Configurations

syslog

SNMP

netflow
Databases

Configurations

Audit/query logs

Tables

Schemas
Virtualization
& Cloud

Hypervisor

Guest OS, Apps

Cloud
Linux/Unix

Configurations

syslog

File system

ps, iostat, top

Configs Messages Traps
Alerts
Metrics Scripts Tickets Changes

Click-stream data

Shopping cart data

Online transaction data

CDRs & IPDRs

Power consumption

RFID data

GPS data
Windows

Registry

Event logs

File system

sysinternals
Logfiles
Splunk Collects and Indexes Any
Machine Data
So What is Splunk?
2012 The SANS" Institute - www.sans.org
21
+
Cluster
Associate
Stats
AVG
Transaction
Addtotals
Delta
Eval
Stddev
Rare
Outlier
Streamstats
Timechart
Time Index Ingestion
Text Base Search
Nested Search
Cross Data-type
Search
cApend
Abstract
Cluster
Bucket
Multikv
Scrub
Join
Rare
Text Based Search Statistical Analysis
Splunk: Big Data Security Intelligence
Platform

2012 The SANS" Institute - www.sans.org
22
22
Statistical Analysis
s
Proactive Monitoring
Search and Investigation
Machine Data
Security Intelligence
for Business
Security Visualizations for
Executives
2012 The SANS" Institute - www.sans.org
23
Enabling IT Risk Scenarios
2
3
Business
Analytics
App
Mgmt
Compliance
IT
Ops
Web
Analytics
Security Relevant Data
Confidentiality / Integrity / Availability
CSO / CIO / CEO
Views
Applying IT Risk
Scenarios
Finding
Abnormal
Behaviors
Open Discussion
- What are the operational
challenges with security big data
analytics?
- Political issues?
2012 The SANS" Institute - www.sans.org
24
Questions?
2012 The SANS" Institute - www.sans.org
25
Contact
Follow-up: q@sans.org

Dave Shackleford
dshackleford@sans.org

Splunk

Mark Seward
mseward@splunk.com
2012 The SANS" Institute - www.sans.org
26