Management of Data

Dr. Nityesh Bhatt

nityesh@imnu.ac.in

7.1

The Data Hierarchy

7.2

Figure 7-1

Entities and Attributes

7.3

Figure 7-2

Database Management System (DBMS)

Software for creating and maintaining databases

Acts as interface between application programs and data files Separates logical and design views of data

7.4

Traditional File Processing

7.5

Figure 7-3

Problems with the Traditional File Environment

Data Redundancy Data Inconsistency Program Data Dependence Lack of Flexibility Poor Security Lack of Data Sharing & Availability

7.6

The Contemporary Database Environment

7.7

Figure 7-4

Components of DBMS:
Data definition language: Specifies content and structure of database and defines each data element Data manipulation language: Used to process data in a database Data control language: Used to control data in a database Data dictionary: Stores definitions of data elements and data characteristics

7.8

Types of Databases

Hierarchical and network DBMS Relational DBMS Object-oriented databases

7.9

Relational DBMS:

Represents data as two-dimensional tables called relations Relates data across tables based on common data element Concept of Primary, Foreign, Candidate, Alternate, Composite Key (s) Examples: Oracle, DB2, MS SQL Server
7.10

The Relational Data Model

7.11

Figure 7-7

The Three Basic Operations of a Relational DBMS

7.12

Figure 7-8

Object-Oriented Databases

Stores data and procedures as objects that can be retrieved and shared automatically

7.13

Designing Databases:

Identification of Entities
Data Attributes/ Fields Data Type

Data Size
Constraints Establishing Relationship

Normalisation

7.14

An Unnormalized Relation for ORDER

Figure 7-9
7.15

Normalized Tables Created from ORDER

7.16

Figure 7-10

Centralised Vs. Decentralised Database

7.17

Distributed Databases

7.18

Figure 7-11

Ensuring Data Quality:

Accuracy
Completeness

Relevance
Timeliness

7.19

What is Data Quality Audit, Data Cleansing?

Multidimensional Data Analysis

Online Analytical Processing (OLAP): Multidimensional data analysis (used for BI) Supports manipulation and analysis of large volumes of data from multiple dimensions/ perspectives

7.20

MULTIDIMENSIONAL DATA MODEL

7.21

Data Warehousing

Stores current and historical data

Supports reporting and query tools Consolidates data for management analysis and decision making

Extract, Transform and Load (ETL)

7.22

What is Data Mart/ Data Mining ?

COMPONENTS OF A DATA WAREHOUSE

7.23

Using Databases to Improve Business Performance and Decision Making

Data mining: More discovery driven than OLAP Finds hidden patterns, relationships in large databases and infers rules to predict future behavior E.g., Finding patterns in customer data for one-to-one marketing campaigns or to identify profitable customers. Types of information obtainable from data mining
7.24

Associations Sequences Classification Clustering Forecasting

DATABASE TRENDS

Database Presence on Web Hypermedia Database Big Data

7.25

Big Data
Unexpected Growth in Structured & UnStructured

Data
Exceeds the processing capacity of conventional DBMS (90mn Tweets/Day, Walmart 1 Mn

trans/hour, Facebook 30 bn content)

Characteristics: Volume: doubles every year

Velocity
Variety
7.26

Big Data Software Stack : Hadoop

Managing Data Resources

Establishing an information policy

Firms rules, procedures, roles for sharing, managing, standardizing data

Data administration: Firm function responsible for specific policies and procedures to manage data Data governance: Policies and processes for managing availability, usability, integrity, and security of enterprise data, especially as it relates to government regulations Database administration:
7.27

Defining, organizing, implementing, maintaining database; performed by database design and management group