Cops & Robbers

Las Vegas Style

Jeff Jonas, Chief Scientist, IBM Entity Analytics

1
Blogging at www.JeffJonas.TypePad.com
Background

 Founded Systems Research & Development

(SRD) in 1983

 Moved to Las Vegas in early 90’s

 Assisted gaming industry in better
understanding who they were doing business
with (e.g., the MIT Team)

 Acquired by IBM January 2005

 Now Chief Scientist of IBM Entity Analytics

2
My Living Room – December 31, 2005

3
 Cheating Las Vegas

4
The “Cold” Deck : $250,000 Gone in 15 Minutes

[Video Redacted]

5
 More About
Corporate Amnesia

6
Perception Isolation … Produces Corporate Amnesia

Marketing Human Corporate

Department Resources Security
Department Department

Employee Investigations
Prospect
Database Database
Database

Marketing department is mailing offers to a

person currently in jail for stealing from you!
7
 Enterprise Intelligence
Requires
Persistent Context

8
The Brain!
 For Example

9
Consider the Query Against these Observations

Observations Sensors

Marc R Smith Mark Randy Smith

123 Main St 123 Main Street
713 730 5769 713 731 5577
Prospect
The Query Record #A-701 Database

M. Randal Smith
DOB: 06/07/74
713 731 5577

Record #B-9103 Fraud

Database

10
Some Observations are Discoverable

Observations Sensors

Marc R Smith Mark Randy Smith

123 Main St 123 Main Street
713 730 5769 713 731 5577
Prospect
The Query Record #A-701 Database

M. Randal Smith
DOB: 06/07/74
713 731 5577

Record #B-9103 Fraud

Database

11
Other Observables … are Undiscoverable

Observations Sensors

Marc R Smith Mark Randy Smith

123 Main St 123 Main Street
713 730 5769 713 731 5577
Prospect
The Query Record #A-701 Database

M. Randal Smith
DOB: 06/07/74
713 731 5577

Record #B-9103 Fraud

Database

12
If You First Construct Context (Features and Events)

Reconstructed Observations Sensors

Identities
Mark Randy Smith
123 Main Street
713 731 5577
Prospect
Record #A-701 Database

M. Randal Smith
DOB: 06/07/74
FEATURES: 713 731 5577
Mark Randy Smith, M. Randal Smith
123 Main Street, 713 731 5577 Record #B-9103 Fraud
DOB 06/07/74 Database
EVENTS:
Internet Inquiry
13 Arrest
 Accumulating and Persisting this Context

Persistent Observations Sensors

Context
Mark Randy Smith
123 Main Street
713 731 5577
Prospect
Record #A-701 Database

M. Randal Smith
DOB: 06/07/74
713 731 5577
FEATURES:
Mark Randy Smith, M. Randal Smith
123 Main Street
Record #B-9103 Fraud
713 731 5577
DOB 06/07/74
Mark Database

14
Now the Un-discoverable …

Queries

Marc R Smith Mark Randy Smith

123 Main St 123 Main Street
713 730 5769 713 731 5577

Record #A-701

M. Randal Smith
DOB: 06/07/74
713 731 5577

Record #B-9103

15
… After Accumulating and Persisting Context …

Queries Persistent Observations

Context
Marc R Smith Mark Randy Smith
123 Main St 123 Main Street
713 730 5769 713 731 5577

Record #A-701

M. Randal Smith
DOB: 06/07/74
FEATURES:
Mark Randy Smith, Randal Smith 713 731 5577
123 Main Street
713 731 5577 Record #B-9103
DOB 06/07/74

16
Enables Enterprise Discovery

Queries Persistent Observations

Context
Marc R Smith Mark Randy Smith
123 Main St 123 Main Street
713 730 5769 713 731 5577

Record #A-701

M. Randal Smith
DOB: 06/07/74
FEATURES:
Mark Randy Smith, Randal Smith 713 731 5577
123 Main Street
713 731 5577 Record #B-9103
DOB 06/07/74

17
Enables Enterprise Discovery

Queries Persistent Observations

Context
Marc R Smith Mark Randy Smith
123 Main St 123 Main Street
713 730 5769 713 731 5577

Record #A-701

M. Randal Smith
DOB: 06/07/74
FEATURES:
Mark Randy Smith, Randal Smith 713 731 5577
123 Main Street
713 731 5577 Record #B-9103
DOB 06/07/74

18
EXCEPT: Always Treat Data as a Query

Queries
The query could be:
Marc R Smith - A user with a question
123 Main St
713 730 5769
Or, also could be data:
- An account opening
- A new watch list entry
- A background check
- An address change
- A vendor application
- A customer inquiry

19
 1st principle

If you do not process every new

piece of key data (perception)
first like a query … then you will
not know if it matters … until
someone asks.

20
“The Data is a Query” Beats “Boil the Ocean”

Marketing Human Corporate

Department Resources Security
Department Department

Employee Investigations
Prospect
Database Database
Database

Midnight Batch Analytics?

21
And … Any Query can be Treated as Data …

Queries Persistent Observations

Context

Emile Swelter Mark Randy Smith

San Francisco 123 Main Street
12/03/72 713 731 5577

Record #A-701

? M. Randal Smith
DOB: 06/07/74
713 731 5577

Record #B-9103

22
… In Which Case the Query can Stick (Persist)

Queries Persistent Observations

Context

Emile Swelter Mark Randy Smith

San Francisco 123 Main Street
12/03/72 713 731 5577

Record #A-701

M. Randal Smith
DOB: 06/07/74
713 731 5577

Record #B-9103

23
Notable, Stick in the Same Data Space

Persistent
Context

24
Now, New Observations Answer Persistent Queries

Queries Persistent New Observation

Context

Emile Swelter
San Francisco
12/03/72 Emilee Swelter
321 Ovington Place
San Francisco
03/12/72

Question answered
when it becomes true!

25
 2nd principle

Treat queries like data to avoid

asking every question every day.

26
This is Context Construction (A Librarian Function)

Persistent Observations Sensors

Context
Mark Randy Smith
123 Main Street
713 731 5577
Prospect
Record #A-701 Database

M. Randal Smith
DOB: 06/07/74
713 731 5577
FEATURES:
Mark Randy Smith, Randal Smith
123 Main Street
Record #B-9103 Fraud
713 731 5577
Mark
DOB 06/07/74
Database

27
The Ideal Moment for Enterprise Awareness

Persistent Observations Sensors

Context
Mark Randy Smith
123 Main Street
713 731 5577

!
Prospect
Record #A-701 Database

M. Randal Smith
DOB: 06/07/74
713 731 5577
FEATURES:
Mark Randy Smith, Randal Smith
123 Main Street
Record #B-9103 Fraud
713 731 5577
Mark
DOB 06/07/74
Database

28
 3rd principle

Enterprise awareness is
computationally most efficient
when performed at the moment
the observation is perceived.

29
Introducing Perpetual Analytics

The “data finds the data” …

and “relevance finds the user.”

Towards Enterprise Intelligence

30
Real Technology

Scalable to >3B
historical observations
while handling >2,000
real-time perceptions a
second

31
Privacy and Civil Liberties – Policy Think

 What perceptions can or should

be placed into context (in one
brain)?

 What if someone steals the

brain?

What if the librarian is corrupt?

32
Analytics in the Anonymized Data Space

Human
Resources
Department

Mark Randy Smith

Employee
Database

Cd5dced41028cb …

33
The Brain!
The Main Think – Towards Enterprise Intelligence

 Without persistent context you have no

brain

 Treat data and queries with equal rights

 More intelligence possible when thinking on

streaming perceptions

 More or less perceptions, that is the

question

34
Battling Corporate Amnesia is Broadly Useful

 National security

 Financial services

 Health care

 Heavily focused on threat and fraud

intelligence

35
 Cops & Robbers
Las Vegas Style

Jeff Jonas, Chief Scientist, IBM Entity Analytics

36
Blogging at www.JeffJonas.TypePad.com
 Bonus Section!

37
 If a .6%
difference
matters this
much…

… no wonder
traditional
information
systems lack so
much
intelligence!
38
More Observations = Better Context

2 Observations 6 Observations

More Observations More

FEATURES: FEATURES:
Mark Randy Smith, Randal Smith Mark Randy Smith, Randal Smith, Randy Smith
123 Main Street 123 Main Street, Flat 6 20 Lennox Gardens
713 731 5577 713 731 5577, 796 064 03 04
DOB 06/07/74 DOB 06/07/74, Passport: 001003429002

39
Sequence Neutrality is Critical for Context Stability

Reload #11 Reload #12

Percent of Error

Drift
Unstable
(e.g., data warehousing
which requires periodic
reloads to handle data drift)

Stable
(e.g., Analytics with
Sequence Neutrality)

Data Loading Over Time

40
 Cops & Robbers
Las Vegas Style

Jeff Jonas, Chief Scientist, IBM Entity Analytics

41
Blogging at www.JeffJonas.TypePad.com