1

BIG DATA FOR RETAIL

Abstract
A recent survey at the big data Retail forum summit in Chicago suggests 80% of the retailers are
aware of big data and 67% are either already in the process of building a big data strategy. With
data collection at various touch points ever-growing, and technology enabling data collection and
analysis, companies have raced towards adopting BI (Business Intelligence) and big data
solutions to earn competitive advantage. The term big data refers to solutions that can handle
data that characterizes 4Vs viz., Volume, Velocity, Variety and Veracity. While insights on to
merchandising, procurement was nothing new, customer analytics has unleashed the real big data
capabilities enabling companies to arrive at insights that were earlier close to impossible. In
retail parlance, consumer shopping behavior, preferences etc. are data that was a rarity in itself.
Thanks to changing consumer mindset. Consumers are increasingly ready to share data and also
allow data capturing by retailers; but a recent IBM study concludes that all such acceptance is
with expectations that the retailers would honor their privacy and provide value in return. Big
data solution provides have been able to demonstrate ROI (Return on Investment) for retail firms
that have invested into these technologies. Changes in consumer buying modes, increased
purchasing power in a hypercompetitive industry has coerced the industry to adapt to such
changing patterns. Omnichannel retailing is the norm of the 21
st
century. Retailers are
continuously bothered by this multichannel importance that customers showroom but yet
dynamically change the channel where the ultimate sale is completed. The intrinsic change in
patterns around shopping behavior has been a source of research for retailers and their big data
solution vendors. CRM (Customer Relationship Management), loyalty programs and corporate
data responsibility are some of the key focus areas today.
Keywords: Omnichannel retail, consumer shopping behavior, big data, analytics, ROI, CRM
2
BIG DATA FOR RETAIL

What is big data?
Big data is the label provided to
extreme forms of BI systems that are
constrained by increased frequency of
data capture combined with ever
increasing touch points in the market.
The new channels of distribution
contributes to the complexity on the
variety dimension.

Retailers handle huge volumes of structured and unstructured data. This is inherent nature of the
industry which is transaction intensive. Data from various touch points are collected and stored
in Hadoop clusters (or any other storage systems) [1]. Through technologies like Hive, special
big data analysts help in generating innumerous insights. The need for such analyst capable of
combining industry expertise with technological acumen are in need. Firms that are looking
forward to gaining competitive advantage through big data are already focusing on some the big
data functionalities through hiring analysts in large numbers. Ability of correlate sensibly and

Figure 2- Big data functional view
arrive at insights is the key capability
expected from such analysts. McKinseys
CMAC (Consumers Marketing Analytics
Center) for example has a team that
provides consulting and tools that can
enable firms to generate actionable insights.
Retailers according to various commentaries and discussion in the big data retail forum summits,
have been focusing more on the areas of CRM, loyalty for value creation to retain customers.
Figure 1-Hadoop representation of big data [1]
3
BIG DATA FOR RETAIL

Traditional Business intelligence and big data
Traditional BI has been focusing on operations, consumer expectation, mismatch around value
perceptions and competition as the key areas for optimization [1]. The dimensions that big data
brings in include the predictive analytics, machine learning, and social behavior of consumers
that have become attractive sources of insights. Retailer understand the increased need to
personalize their online and offline store operations for consumers. Informed data collection are
various interactive points and observatory learning systems ([10] application of video analytics)
have been under usage during the last 5 years. The advantage that big data offers is the benefits
of machine learnings to connect individual observations into global insights with great levels of
granularity.








Make or Buy decision
The decision process to creating competitive advantage through big data starts with WHAT are
the areas that can be improved using big data. Once there is a conviction to invest into analytics,
the next decision point is whether to develop a solution using the in-house expertise or procure
an off-the-self, customizable solution from a vendor like say CMAC from McKinsey [14]. Our
study suggests that this decision has to be purely contextual for any firm. Some of the factors that
retail firms need to consider before investing into such a solution are summarized below.
Figure 3 Traditional BI and Big data
Big data adds
Traditional BI
Big data
4
BIG DATA FOR RETAIL


Make decision Factors to consider
If you see it as a Long term strategic asset and core competency
Cost of operation increases due to need in house expertise.
Lead time available to develop
Flexibility and selectivity
Security concerns around the data collected
Fixed cost model
Buy decision Factors to consider
Initial period of ROI Lull
Payment may be for features not required.
Off-the-shelf in most cases; can be setup and be running quickly
Can be less flexible due to constraints with customization
Data security and privacy can be an issue
Variable cost model
Omnichannel retailing
Retail industry has two sectors primarily viz., unorganized and organized. These are called
General trade and modern trade in retail parlance. In developing countries, the proportion of
unorganized retail is usually higher vis--vis developed countries. The adoption of big data is
predominant as expected in the developed countries, where accessibility is not a major concern.
The ability to retain and satisfice consumers is the key challenge that has forced retailers in the
developed countries to focus on CRM, loyalty programs etc. The data collected at various
interaction points with the consumer (cognizant and observatory) are collated and analyzed
5
BIG DATA FOR RETAIL

usually to personalize their experience during shopping at their outlets (online portals as well as
retail stores). The resultant insights are fed to other operations like merchandising (merchandise
assortment and planning (MAP), procurement, inventory forecasting, warehousing and vendor
engagement etc. Changes in lifestyle has created new avenues of convenience and value for
consumers. Brick-and-mortar companies have been forced to now consider online competition
from online retailers. Price parity is a key challenge. Creating the right value to compensate
inherent disparity in cost structures through differentiation is inevitable. There is a challenge for
brick-and-mortar companies with venturing into the online space. They further need investments
into a whole new vertical and traverse through the learning curve to compensate the
disadvantage. Instead, brick-and-mortar companies could embrace other in-store data capture
technologies and develop teams that span the competition and its offerings constantly.
IBM Research Results:
Although a large amount of consumers are willing to share personal data, the key finding is
that they expect a personal experience in return. It is imperative that retailers go about using
data in such a way that it results in customers trust and loyalty - IBM Research [5, 8].
19%
40%
29%
12%
Tech based
buyers and prefer
multiple
channels.
Choose retailers
that offer those
channels
Info search
and buy online
Info Search
But no
purchasing
online
Does not use
technology to
shop

6
BIG DATA FOR RETAIL

Figure 4 Shopper categories based on purchase decision process
Key takeaways from global retailers
Key takeaways from the commentaries of global retailers include the following:
Time sensitive decisions, Monitor emerging trends and taking corrective actions are
critical for generating value out of big data investments

Spend most of your time on WHAT and not HOW [1]
Spend 70% on WHAT to analyze and only 30% on HOW to analyze
Ability of gather relationships across variables in structured data is immense
Build a solution to drive data gathering at various touch points into a common repository.
Consumer Insights
Business Insights
Product Insights &
Social Insights
When you create a Big Data team:
Form teams comprised of right people
Partnership with your IT department is critical
Concrete goals (else leads to analysis paralysis)
Success with Big Data is primarily through
Choosing simple ways to compute insights
Knowing how much to complicate
Insights Decisions Execution
7
BIG DATA FOR RETAIL

Honest data management practices that does not tamper with the privacy promises
made to the consumer
How do you begin with Big Data?
Augment to enhance insights
Do not treat it as a revenue handling system
Improve quality of existing decisions
Improve time to market

Big Data - Retail Use cases
During course of this term paper we conducted a literature review [3] and existing business
solutions from customers case studies to see various places where big data technologies are used
in retail Industry. Apparently, we seem to notice that a lot of multinational retail brands already
kick started their efforts to leverage value out of big data. Through this study, we were able to
bin most of these uses into following categories
CRM & Customer Experience
A customer interacts with company during number of times to buy, use and avail
product/services. Whenever a customer interacts with an organization during various touch
points, it is vital that amount of information available on customer informs and guides process
that will help maximize their experience. For this to happen, retail companies need a 360 degree
view of customers. A 360 degree view of customer includes aspects related to past, present and
future. The following are sources from which retailers can capture such information.
Empowered with large volumes of information relating to customers, retailers can
improvise by personalizing their services. To help customer buy a new smart phone,
8
BIG DATA FOR RETAIL

retailer can analyze data from his previous transactions, their click stream activity, social
media activity, geospatial & demographic information and push highly targeted real time
promotion through search engine and social media advertisements.

Figure 5 - A general overview of various sources from which we can capture customer data.
Ecommerce websites can come up with better recommendation systems to suggest
relevant products that customers might be willing to buy.
Merchandizing, Pricing
Merchandizing deals with retail activity which primarily focuses on promoting sales of goods
especially by presentation in retail outlets. Big data can aid retailers to manage merchandizing by
effectively making use of data aggregated from a variety of sources.
With the profusion of social networking sites and user activity, a lot of social media buzz is
generated online which could be valuable for companies. Using big data technologies, companies
9
BIG DATA FOR RETAIL

can figure out latest trends, fashion styles promoted by public icons & movies and push these
merchandize to storefront.
Using Big Data technologies, retailers can use data captured from news related to local events
and launch promotional buzz. Retail managers can also get information from disparate data
sources and fine tune their pricing strategies accordingly resulting in competitive edge in their
respective industries.
Supply chain & Operations
In addition to improving CRM and merchandizing, big data technologies can also be leveraged
to improve supply chain and regular day to day operations.
Armed with huge volumes of real time data, retailers can use real time tracking options to track
order shipments. In case of any major political, natural or social events, companies can react
spontaneously and by efficiently rerouting their shipments, they can improve their logistics.
On similar lines, big data technology can be harnessed to improvise day to day operations. For
instance, looking at data trends from local news and events and data demand and supply patterns,
companies can work on dynamically staffing people which could result in cost savings.
Marketing
Big data delivers competitive advantage for retailers in marketing activities too. Empowered
with Big data, retailers can indulge more effectively in targeted marketing, micro segmentation,
location based selling and cross selling. In addition to targeting, retailers can also pull out
valuable insights of their product performance or product launches if any from social media.
10
BIG DATA FOR RETAIL

A lot of context specific insights related to customer activity is captured day to day by many
sources. This data can be used for marketing and delivering ads to users based on digital
footprint and context in which user performs digital activities.
Social
With the rise of social networking activity online, social networking sites capture phenomenal
amount of social activity from public. This information could provide valuable insights on
customer perceptions related to various products and services they consume. Companies can
conduct sentiment analysis on social networking sites and get various insights on how customers
feel and perceive about their products.
Visual Analytics
This is another trending domain under big data umbrella. Although in its nascent stages, mining
real time in store closed circuits cameras and other offline video content could provide useful
insights onto how can retailers improve their customer service. For instance, firms like IBM are
coming up with augmented virtual reality shopping application that will automatically deliver
personalized coupons, offers, customer reviews and hidden product details. This would close the
gap between wealth of product information on internet and traditional retail stores [10].
On similar lines, a lot of face matching analytics technology is being developed which can
identify customers walking into store and pass their profiles to sales representatives so that
retailers can deliver personalized customer service.
Some Key Issues in Big Data
Communication & co-ordination of disparate data silos:
11
BIG DATA FOR RETAIL

One of the most critical issues with big data is related to managing sheer quantity of data
produced by various sources. Owing to large volumes and variety of data, organizations build
huge data silos to capture and mine data. Most of the time, coordinating and consolidating these
massive silos could issue potential problems pushing existing infrastructure to its limits.
Technology and research could eventually deal with this problem but however, a lot of
streamlining work needs to be done in this area so that systems can co-ordinate and efficiently
process this information.
Regional, Cultural & linguistic factors
Weather it is Social Network Analysis or Analysis of data from news and newsgroups, there are
always regional, linguistic and cultural factors that kick in and need to be accounted for. Since
Big Data technology deals with a diverse wealth of information, systems that handle and process
this data should be flexible enough to account for all these factors.
Privacy & Regulation issues
Historically, most of the data collected does not attribute to specific individuals. This is more
often de identified information and big data is traditionally used for tracking interests of
customer groups in de-identified form. Information in this form is not regulated across many
nations and companies are free to conduct their business using this. However, with technology
evolution, we have enough analytical tools developed which could track and attribute data to
specific individuals. This resulted in number of governments regularly coming up with
legislative acts like Privacy Act, The Spam Act etc. (Australia).
All these issues complicate environment in which companies operate business with big data
technologies. These factors have to be strictly honored and technology needs to flexibly
encompass all the same.
12
BIG DATA FOR RETAIL

Corporate responsibility on Customer data
Another issue that gets frequently highlighted whenever people talk about big data is related to
the issue of corporate responsibility on customer data. There is a growing concern on to what
extent corporate are responsible in making ethical use of data at their expense [9].
With profusion of data, corporate look for many ways with which they can monetize this. As the
case study The dark side of customer analytics discusses, when a company gets hands on data,
marketers can be tempted to use information in ways that seem attractive but ultimately end up
damaging relationships with customers.
Also, with the amount of data that companies collect, they would be able to profile customers
more accurately without possibly having any direct interaction with them. All these actions call
for an ethical business conduct which is critical when retailers do their business.
Some Radical trends that we could foresee going forward.
Significance of Context Awareness
Going forward, since data will be all around us, the context in which it exists and its relevant
starts to matter more. Empowered with contextual information, companies can produce better
analytical insights into data which is more pertinent to customer. With the increase in number of
devices like the mobiles, tablets and more trending wearable the capability to collect context
specific information becomes easier and companies would use this to obtain better insights from
data [12].
Focus on real problems rather than product selling; Semantic shift
Right now, most of business vendors are in mode of selling big data technology applications and
services. Empowered by these, retailers usually focus on selling their products to individuals.
13
BIG DATA FOR RETAIL

With the evolution of new research areas like semantic technologies, deep data mining etc, and
retailers will be riding on tools with which they will be able to go beyond selling to solving more
real world problems like solving customers needs, proactively reacting to their demands etc.
Visualization driven insights
Visual communication aids in simplifying and understanding large number of insights processed
from data. Companies like Trulia, a real estate website, can use a single visualization to display
many data sets like crime maps, school districts and neighborhood prices etc. Likewise, GE
extensively uses visualization in its strategic information systems to reach its goals because it
makes data comprehension easier for senior executives. Thus, visualization driven insights
would be a commonplace.
Improvements in raising status quo bar of existing technologies
Academia research, Company collaborations and events and annual summits like Big Data &
Analytics for Retail Summit etc would constantly improvise on existing big data and related
technologies thereby raising status quo bar. For instance, the key themes of 2014 summit will be
relating to areas like Customer Engagement, E-Commerce, Consumer Insight, Data Analytics,
Web Analytics, and CRM etc.








14
BIG DATA FOR RETAIL

Conclusion
Retail firms could increase operating margins by up to 60% for those pioneers that maximize
their use of Big Data. McKinsey
The retail industry is different from other industries in that it has intrinsic structure to capture
huge volumes of data. It is important for companies to understand the importance of this data and
apply analytics in a sensible way to achieve positive improvement. Some of the key learning
have created retailers to optimize their cost of operation immensely. While the data analytics has
helped the operations in a positive way, an oversupply of analytical surge has also caused
detrimental results for firms. Most of these failure scenarios are attributed to cognizant or
nonsensical application of correlative relationships to arrive at insights. More problems arise
through reporting of such insights. Key issues dangling around today, discussed above have been
confirmed by multiple retailers, confirming the veracity of these issues.
Our observations indicate that data analytics on the operational front has immense
potential to improve and bring about a structure to the operations. Most retailers in developing
countries that collect/have started collecting information about shopper behavior and personal
information have to consider these insights and escape this curse of analytics.
Big data enables firms to gather finer details about data that they already have. This
includes customer data as well. Usage of customer data has to be in line with the privacy policies
promised to the customer. Any violation can be detrimental to the business [9]. Questions that
retailers face today on customer privacy have to be dealt without prejudice keeping customers at
the center of the business. This must be despite the fact that legal restriction are in place or not.


15
BIG DATA FOR RETAIL

References
1. Big data Calculating ROI
http://www.bigdataretailforum.com/FormDownloadThankYou.aspx?target=http://www.bigd
ataretailforum.com/media/7832/14512.pdf&eventid=7832&m=14512#
2. Connecting consumer experience across all retail touch points
http://www.retailbiz.com.au/2013/03/11/article/Opportunities-along-the-digital-breadcrumb-
trail/GJNDBBKEAL
3. Big data uses for retail
http://www.ngdata.com/solutions/solutions-and-use-cases-for-retail-companies/
http://wikibon.org/wiki/v/Big_Data_in_the_Retail_Industry
4. Retail efficiency using big data
http://cdn2.hubspot.net/hub/173001/docs/boosting_retail_revenue_wpp.pdf?t=0
5. Personalizing in the world of Omnichannel retailing
http://in.teradata.com/retail/?LangType=16393&LangSelect=true
6. http://www.slideshare.net/srinivasbobby/big-data-trends-in-retail-industry-31724077
7. http://www.forbes.com/sites/barbarathau/2014/01/24/why-the-smart-use-of-big-data-will-
transform-the-retail-industry/
8. Result of IBM Retailers survey, originally Published January 13, 2014 on Retail-
Merchandiser.com
9. The dark side of customer analytics : A HBR Case Study
10. Video Analytics for Retail
http://www.securitymagazine.com/ext/resources/2012/September-2012/Retail-Video-
Analytics---feat.doc
11. Volumes of information on big data for retail
www.bigdataretailforum.com
http://theinnovationenterprise.com/summits/big-data-analytics-retail-2014-
chicago#sthash.odh83E3y.dpuf
12. Ideas economy: Finding value in big data
16
BIG DATA FOR RETAIL

http://www.oracle.com/us/technologies/big-data/finding-value-in-big-data-
1991047.pdf?ssSourceSiteId=ocomin
13. Gartner research reports
14. CMAC at McKinsey
http://www.mckinsey.com/client_service/retail/expertise/~/media/mckinsey/dotcom/client_se
rvice/retail/articles/cmac_creating_competitive_advantage_from_big_data.ashx