Learning Diary

Chapter 11- Big Data Analytics Strategy

Big data has gained prominence after the growth of cloud computing.

During recent advent of social media and machine sensors led data generation giving a
new challenge.

It may be defined as a collection of very large and complex data which cannot be
processed with the help of traditional databases and tools. Followings are the
characteristics of Big data

1. Enormous Volume
Big data is predicted to grow enormously an increase of about 44 times which
amounts to growth is increasing exponentially.

2. Variety

Big data has mind blowing variety and heterogeneity as it comes in various format
types and structures such as text, numerical, images, audio, video and social media
data etc.

3. Velocity
The velocity of creation in Big data is very fast and in real time. Shopping mall
sends offer based on your current location.

4. Veracity

It is required for the authentication of data in doubt , spurious data or noisy data.

5. Value
This is the intrinsic value generated out of Big data through statistical modeling
and correlation and event based actions etc.
Sources of Big Data

Followings are the sources of Big Data:

1. Social Media Data

This include data comes from Text, Video, Images and voice post social media
sites like Facebook, LinkedIn, Google Searches, Twitter and YouTube etc.

This data is mostly spontaneous and sometimes it is also planted by canny

marketers and social media activists.

Such data is considered to be a good source to provide the pattern of consumer

behavior and sentiments and is also used by marketing analytics.

2. Machine Sensor Data

This data received from sensor installed at public locations (Traffic signals, Industrial units
and Mobile towers etc) which captures millions of calls routed through the mobile
network.

The volume of this data will grow exponentially as internet is gaining prominence.

3. Transactional Data

This includes data received from traditional transaction like online transactions and digital
payments etc. it generates huge amount of data as the economy is becoming more and
more cashless.

Big Data Analytics:

Big data analytics include methods and tools to examine Big data in order to uncover
hidden patterns and find out the unknown correlations. It uncovered and predict market
trends. It involves the use statistical and predictive modelling tools by data scientists.
Big Data Analytics Technologies and Tools

Traditional data warehouses keep past data that change slowly over time but in the case
of big data, updated frequency is in real time. For collection, processing and analysis the
tools used are different.

This include databases, Hadoop and its companion tools. Some of these tools are:-

1) YARN: It is a key feature of second generation Hadoop following cluster

management technology.

2) MAP REDUCE: It is Java based distributed computing processing technique. This

contains map and reduce as 2 important tasks.

3) SPARK: Run data analytics applications on clustered system.

4) HBASE: It consists of column oriented key/ value data store.

5) HIVE: It can analyse data stored in Hadoop file.

6) KAFKA: A distributed publish subscribe messaging system designed to replace

traditional message brokers.

7) PIG: It offers a mechanism for the parallel programming of map reduce jobs to
be executed on Hadoop clusters.
Types of Big Data Analytics:

Followings are the types of Big Data Analytics

1. Sentiment Analytics
2. Website Traffic Pattern Analytics
3. Market Basket Analysis
4. Behavioral Analytics
5. Weather Analytics

Sentiment Analytics:

It is also known as opinion mining or emotion AI, uses tools like natural language
processing, text analysis to identify, extract and quantify subjective information, It is
widely used as a tool to ascertain. The voice of the customer expressed in product or
service reviews. Examples of Big data, Sentiment analysis tools include Hoot suite
insights, Twitter advance search and Brand watch etc.

Website Traffic Pattern Analytics:

This analysis is used to understand and optimize the web usage by measuring, collecting,
analyzing and reporting data. When a company moves to online, TV Radio or print
advertisement then increased traffic to the company’s website depicts the effectiveness
of these campaign.

Market Basket Analysis:

This analysis helps in analyzing historical inventory, pricing and transactional data to
understand seasonality of products. This help further in arriving at competitive pricing
and target advertising.

Behavioural Analytics:

Behavioural analytics has doubled the revenue in online gaming, Online gaming
companies constantly increase the rate of customer acquisition, retention and
monetization.
Weather Analytics:

Weather analysis help to understand how consumer demand is correlated with weather
pattern. These analytics help the retailers to do accurate inventory planning for items
required as the weather changes.

The weather company can warn people about impending hailstorms, allowing
homeowners to protect their property.

Such advanced information reduces claim pay outs and adds to the goodwill of the
insurance company.

Big Data Analysis Component:

1 Data Loading: This requires a software to load data from multiple data sources. It
should support the distributed nature of Hadoop and the non distributed nature of data
source.

2.Dependency management: There are complex dependencies that must be

managed.

For example: certain data sets have to be loaded before certain jobs in Hadoop can be
run. In dependency management data synchronization, data often needs to be pushed
from Hadoop into a data store.

3.Monitoring API: Every aspect of a big data analytics solution needs to be monitored.
Things that need to monitored include who has access to the system, job health,
performance and data throughout.