The tsunami of big data has arrived and the experts and the business fraternity are already

hyped
about how intelligence gathered from big data is going to change the business decision making
process. The cynics are of opinion that big data is going to spell the doom for the good ol market
research. The business environment is changing and with the cut throat increase in competition
there is a growing need for quick decision making based on accurate prediction. Big data comes to
the rescue. 2013 is largely dominated by the possible future of this new phenomenon and how can
the business organisations derive benefit from it. IBM is spending billions of funds in the research of
big data based predictive models which will help the executives in better and accurate decision
making. So is this it? Is this the end of market research? Will machines finally replace the humans?
Before beginning the discussion lets spare a thought on what is Big data and why does it has such an
ominous sign for the market research industry. In laymans term Big Data is the massive volume of
data that is being generated every day from the customers search behaviour, social media,
smartphones, shopping behaviour etc, in a nut shell everywhere a consumer is connected to a digital
system and wherever he is leaving his digital footprint. All this is accumulating into a large pool of
data which has basically has 3 Vs: Volume, Velocity and Variety. One of the common mistake we do
while defining big data is that we look into its size( volume), speed of generation (velocity) and the
different types of data (variety) we capture, but it has no benefits for any business. Big data is useful
data, and to search for this useful data from Exabyte or petabytes of data is the biggest challenge for
the successful implementation of big data for decision making process.
The challenges of big data dont end there. Let list down the big challenges for big data:
1. Big data can find what is happening among the consumers but cannot assess why and
how it is happening
2. Privacy issues regarding big data use
3. Security concerns
4. Is the money invested on Big data predictive analytics worth the value
5. Ethical issues
6. Finding of capable technologists to convert the big data into useful decision making
The biggest challenge with big data is that it cant explain the consumer behaviour. This is a major
drawback for big data. For a proper insight about a marketing problem you need to know why the
consumer is showing certain behaviour. We need to dig deeper into the consumers psychology to
unearth the facts that led to a certain set of behaviour. Social context, aspired culture, etc comes
into play and that can never be explained with the help of big data alone. Good ol market research
comes to the rescue. Traditional market research is indispensable in finding out the behavioural trais
of a consumer. But market research too has its limitations. The biggest problem with the traditional
form of market research is data collection.
In our quest to find the what, i.e. the useful data, market researchers usually goes to the extreme,
resulting in unnecessarily long questionnaires, challenging the respondents patience and time at the
same time. The quality and the accuracy of the data collected become questionable at times. The
problem is more in the healthcare sector, where getting useful data from the busy schedule of the
specialists often pose a huge challenge.
This is a wakeup call. With the growth of technology and the ever dependence of human on
technology needs to leveraged and should be used for the advantage of marketing. The challenges
faced today by market research can be easily averted by the use of big data, where we can get
what the consumer is doing more effectively and accurately. This is very useful in finding about the
details of the target consumer group. Market researchers then can extend their whole energy in
analysing the underlying reasons behind the behaviour by making small and relevant questionnaire
required to make an entity data or a behavioural profile of the consumer. In the next stage big data
can be again used to find out the number of people falling under this behavioural profile. We will get
an exact number of people having a particular behavioural profile. The next stage is to validate the
hypothesis created. From the universe we take a very small sample and test our hypothesis on that
experimental group. If our hypothesis is validated we can get an exact estimate of the number of
consumers for a particular product. Sounds cheeky eh. What is more important is that we can use
this to evolve market research from just giving marketing insight to providing marketing foresight.
This is the future of market research. Big data is the big opportunity which will help market research
to reach its next level, we need to embrace it or else there are ominous signs for the market
research industry.
The big data market research union case study: