Module 8

*********
Data
Validation &
Security
Workforce Information
Database Training
Last update November 2006

Module 8
Data Validation & Security

Providing Accurate Data Consistently

Data validation allows us to provide data
with confidence in its accuracy, and we
can consistently provide this data by
implementing thorough security.

Module 8
Data Validation & Security

Data Integrity

Validity, consistency, and accuracy of

the data in a database.
Table-level
Field-level
Relationship-level
Business Rules

Module 8
Data Validation & Security

Data Validation

The process of determining if an

update to a value in a tables data cell
is within a preestablished range or is a
member of a set of allowable values.

Module 8
Data Validation & Security

What are some common data

quality problems that affect
data integrity and validation?

Module 8
Data Validation & Security

Common Data Quality Problems

Illegal values
Violated attribute dependencies
Uniqueness violation
Referential integrity violation
Missing values
Misspellings
Cryptic values
Embedded values
Misfielded values

Module 8
Data Validation & Security

Common Data Quality Problems

continued.

Word transpositions
Duplicate records
Contradicting records
Wrong references
Overlapping data/matching records
Name conflicts
Structural conflicts
Inconsistencies

Module 8
Data Validation & Security

What are some methods for

checking for data validity?

Module 8
Data Validation & Security

Methods for checking

for data validity

Visual/manual
Aggregation
Reviewers guide
Auto data checks
Record counts
Spell checks
Have data provider review

Module 8
Data Validation & Security

Time to Exercise!!

Module 8
Data Validation & Security

ALMIS Database Security Considerations

The contact in my state is:______________

Security Concerns:
Confidentiality
Integrity
Availability

Web security
Application
security
Production vs. test

User level

Physical security
(including backup
& restoration plan)
Network
permissions

Data Storage Access

(Server or PC)
User level
Suppression flags
Data aggregation issues

RDBMS
ODBC

Network
(LAN/WAN)
security
Application
security

Production vs. test

User level

Database security prevents unauthorized person(s) from viewing, destroying or altering data within the database.

Module 8
Data Validation & Security

ALMIS Database Security Considerations

The contact in my state is:______________

Security Concerns:
Confidentiality
Integrity
Availability

Web security
Application
security
Production vs. test

User level

Physical security
(including backup
& restoration plan)
Network
permissions

Data Storage Access

(Server or PC)
User level
Suppression flags
Data aggregation issues

RDBMS
ODBC

Network
(LAN/WAN)
security
Application
security

Production vs. test

User level

Database security prevents unauthorized person(s) from viewing, destroying or altering data within the database.

Module 8
Data Validation & Security

Physical Security
Questions to ask yourself:
Where is your data actually stored?
Are you responsible for physical security?

If you are...

Module 8
Data Validation & Security

Physical Security
What physical security measures do you
have in place?
Do you have the right hardware?
Are you using obsolete hardware that is
prone to crashing or hacking? Do you
have a choice?

Module 8
Data Validation & Security

Physical Security
Do you have a database backup plan?
Do you have remote backup so if a fire
burns the building you dont lose both
your primary and backup data at the
same time?
Do you have a data restoration plan?
Does the backup plan allow for feasible
restoration?

Module 8
Data Validation & Security

Physical Security
Restoration Plan
Does your restoration plan allow for
the restoration of individual tables
or require the entire database?
Do you maintain copies of the tables
on your hard drive?

Module 8
Data Validation & Security

Physical Security
Who has access to the physical storage
location?
Is your database on a PC or server that
someone could easily kick, spill coffee
on, or accidentally reappropriate?

If you are not responsible for physical

security, ask these questions of those
who are!

Module 8
Data Validation & Security

ALMIS Database Security Considerations

The contact in my state is:______________

Security Concerns:
Confidentiality
Integrity
Availability

Web security
Application
security
Production vs. test

User level

Physical security
(including backup
& restoration plan)
Network
permissions

Data Storage Access

(Server or PC)
User level
Suppression flags
Data aggregation issues

RDBMS
ODBC

Network
(LAN/WAN)
security
Application
security

Production vs. test

User level

Database security prevents unauthorized person(s) from viewing, destroying or altering data within the database.

Module 8
Data Validation & Security

Data Storage Access

What software are you using to store the
database?
SQL Server, Oracle, FoxPro, Access?
Survey says:
SQL Server

48%

Oracle

28%

Other

24%

Module 8
Data Validation & Security

Data Storage Access

What can be done at the server level to
provide for security?
Confidentiality, Integrity, Availability

DO apply advanced security to the most

sensitive data, but DONT apply
advanced security to non-sensitive data

Module 8
Data Validation & Security

Data Storage Access

Security at the server and/or database
level
Permissions
What kind of permissions exist?
Read only, SA, etc.
Who sets those permissions?
Who has those permissions?

Module 8
Data Validation & Security

ALMIS Database Security

Security at the data level
Suppression
Suppression flags
Suppression can be handled at the
database level or the application level

Module 8
Data Validation & Security

ALMIS Database Data Security

Two approaches:
1. ALMIS Database contains NO confidential
data (all data available for use without
restriction)
2. ALMIS Database contains confidential
data (data access controlled by security
and/or suppression)

Module 8
Data Validation & Security

ALMIS Database Security

Tables that have suppression flags...
ces

iomatrix

stindprj

indprj

occprj

stoccprj

industry

oeswage

NOTE: tables stfirms and wage have no

suppression flags but may contain confidential data

Module 8
Data Validation & Security

ALMIS Database Data Security

Issues to consider regarding suppression:

If your database doesnt contain

suppressed records, detailed data wont
aggregate to totals (validity checks?)

Without proper suppression, confidential

data can be back-calculated

Module 8
Data Validation & Security

ALMIS Database Data Security

A note on confidential data:
If you are checking 202 data (or any other
potentially confidential data) to see that it
loaded right, make sure your EQUI data
files and/or printouts are always secured
and shred all photocopies when finished.
Check with your local BLS personnel or LMI
administrator for specific confidentiality
policies.

Module 8
Data Validation & Security

ALMIS Database Security Considerations

The contact in my state is:______________

Security Concerns:
Confidentiality
Integrity
Availability

Web security
Application
security
Production vs. test

User level

Physical security
(including backup
& restoration plan)
Network
permissions

Data Storage Access

(Server or PC)
User level
Suppression flags
Data aggregation issues

RDBMS
ODBC

Network
(LAN/WAN)
security
Application
security

Production vs. test

User level

Database security prevents unauthorized person(s) from viewing, destroying or altering data within the database.

Module 8
Data Validation & Security

User Access
Three major types of user access:
PC - direct to database (ODBC or
RDBMS)
Network (LAN/WAN) through
application
Web (passive or active through
application)

Module 8
Data Validation & Security

User Access via the Web

How do Web users access the data from the
database?
Passive (static tables automatically
updated to web pages)
Active (query through application)

Module 8
Data Validation & Security

User Access via the Web

Questions to ask yourself:
What kind of web server are you using?
What are the security considerations with
that choice?
What kind of firewall do you have?

Module 8
Data Validation & Security

User Access via the Web

Does your web interface have
security/confidentiality suppression?
Do you display suppressed data to certain
users? If so, how is access managed?
Passwords? IP address?

Module 8
Data Validation & Security

User Access via a Network

(LAN/WAN)
Questions to ask yourself:
What type of network system do you have?
Who has rights/access to your data via the
network? Through what applications?
Are confidential data (suppressed records)
available?

Module 8
Data Validation & Security

User Access via a Network

(LAN/WAN)
How do you control access to confidential
data? Through user Ids? Through server
access permissions?
DO recognize the important distinction
between network security and data
security.

Module 8
Data Validation & Security

User Access via direct connection

Questions to ask yourself:
Is local access machine password
protected?
Who has access to your machine?
Is there a backup plan for your access
machine?
Is the source data for your database
secure? Backup plan? Restorable?

Module 8
Data Validation & Security

ALMIS Database Data Security

Documentation
It may prove beneficial to keep detailed
records on

How security is done

Where security exists
Who is responsible for security
Who has access/permissions to what
Etc...

Module 8
Data Validation & Security

Data Security
DISCUSSION