Hypothesis Testing for Market Research Analysis

Hypothesis Testing for Market Research Analysis

by Christoph Strehl

copyright © 2005, Christoph Strehl – mail@cstrehl.de – www.christoph-strehl.net Page 1 of 7

 Hypothesis Testing for Market Research Analysis

SUMMARY ........................................................................................................................................................... 3
CONCEPT OF HYPOTHESIS TESTING ......................................................................................................... 3
GENERAL PROCEDURE OF HYPOTHESIS TESTING............................................................................... 4
DEFINE THE BUSINESS OBJECTIVE ....................................................................................................................... 4
STATE THE HYPOTHESIS ...................................................................................................................................... 4
CHOOSE AN ALPHA-VALUE.................................................................................................................................. 4
CALCULATE THE TEST-STATISTIC ........................................................................................................................ 5
DECIDE WHETHER TO ACCEPT OR REJECT THE NULL-HYPOTHESIS ...................................................................... 5
INTERPRET THE DECISION .................................................................................................................................... 5
OVERVIEW OF TEST TYPES........................................................................................................................... 6
PARAMETRIC TESTS ............................................................................................................................................ 6
1-sample T-test .............................................................................................................................................. 6
1-sample Z-Test ............................................................................................................................................. 6
2-sample T-test .............................................................................................................................................. 6
2-sample F-test .............................................................................................................................................. 6
NON-PARAMETRIC TESTS .................................................................................................................................... 6
Chi square-test............................................................................................................................................... 6
Sign Test ........................................................................................................................................................ 6
Wilcoxon Signed Rank test ............................................................................................................................ 7
Mann Whitney U-test ..................................................................................................................................... 7
Kolgomorov-Smirnoff test.............................................................................................................................. 7
CONCLUSION...................................................................................................................................................... 7
BIBLIOGRAPHY ................................................................................................................................................. 7

copyright © 2005, Christoph Strehl – mail@cstrehl.de – www.christoph-strehl.net Page 2 of 7

 Hypothesis Testing for Market Research Analysis

Summary
“Too few researchers understand what statistical hypothesis testing does and doesn't do, and
consequently their results are misinterpreted. Even more commonly, researchers understand
elements of statistical hypothesis testing, but the concept is not integrated into their research.
For example, the influence of sample size on statistical significance may be acknowledged by
a researcher, but this insight is not conveyed when interpreting results in a study with several
thousand subjects.”1

A perfectly conducted market research study is worthless if you don’t know how to interpret
its results. Hypothesis tests can help you to make quantitative and objective conclusions based
on your survey results. So it is important to understand how to choose among different
Hypothesis tests and how to interpret them.

This report concentrates on the general concept of Hypotheses testing and will not go into
details of statistical reasoning or specific tests. Non-statisticians will be able to understand the
basic concepts as well as the power and the caveats of hypotheses tests.

Concept of Hypothesis Testing

Hypothesis Tests are part of the inferential statistics. Inferential statistics are used to draw
inferences about a population from a sample. There are two main methods used in inferential
statistics: estimation and hypothesis testing. In estimation, the sample is used to estimate a
parameter and a confidence interval about the estimate is constructed.2

A hypothesis is a specific statement of prediction, or hypothesis. The important thing to

remember about stating hypotheses is that you formulate your hypothesis, and then you
formulate a second hypothesis that is mutually exclusive of the first and incorporates all
possible alternative outcomes for that case.3

So basically, in Hypothesis testing the researcher states two hypotheses – the null and the
alternative hypothesis - , computes the test and finally decides in favour of one of those
hypothesises.

When deciding to accept or to reject a particular hypothesis, one can never be sure of making
the correct decision. Two types of errors are possible: Accepting a false hypothesis and
rejecting a true hypothesis. With H0 denoting the null hypothesis, the following table lists the
four possible constellations:4

H0 is true H0 is false
Accept H0 No Error Type II error
Reject H0 Type I error No Error

copyright © 2005, Christoph Strehl – mail@cstrehl.de – www.christoph-strehl.net Page 3 of 7

 Hypothesis Testing for Market Research Analysis

General Procedure of Hypothesis Testing

The overall procedure of conducting a hypothesis test can be divided into six steps. They are:

Define the business objective

A hypothesis test can only be as valuable as the definition of the business objective is. Before
taking any further steps one must clearly state what the subject of the test is and what should
to be found out and what should be proven.

State the hypothesis

The second step is translating the business objective into statistical terms and stating the
“Null-Hypothesis” as well as the “Alternative Hypothesis”.

The choice of the right Null-Hypothesis and Alternative-Hypothesis is essential. In order to be

able prove the hypothesis one wants to prove, this hypothesis has to be expressed as the
alternative hypothesis. That’s because if one accepts the Null-Hypothesis for a test he cant’t
be sure about the Type-II-error probability that may have affected that decision, he can only
be sure that there is not enough statistical significance that the sample data contradicts the
Null-Hypothesis. But if one rejects the Null-Hypothesis there is a statistical significance that
the Alternative hypothesis is true and the error-probability is at most as big as the chosen
alpha.

If for example a company the produces soda wanted to analyze a survey and publish an
advertisement that says that more than 75% prefer their soda instead of the competitor’s
product, their Null-hypothesis would be “Less than or equal to 75% prefer our product” and
the alternative hypothesis would be “More than 75% prefer our product”. Of course the
sample data may not support the alternative hypothesis and they have to accept the null-
hypothesis, but this is the only way to be able to gain statistical difference for the hypothesis
the company supports.

Choose an alpha-value
As stated above, there are two types of error that can be made. The Type-I-error is the error if
you reject the Null-Hypothesis although it is in fact true. This error can be limited.
Unfortunately, there proves to be a “trade-off” between the probabilities of both error-types:
Lowering the probability for a type-I-error leads to an increase in the probability of a type-II-
error and vice versa.5

The alpha-value, denoted by α, is the probability of making a type-I-error. The most common
choice for an alpha is 5%. That means if one were to hypothetically perform a test 100 times,
one would make the wrong conclusion in about 5 of these tests. Arriving at the wrong
conclusion in this context means that one has concluded that the results are not due to chance,
when in fact they were.6

But nevertheless the choice of the “right” alpha is a subjective choice that very much depends
on the specific business problem. For example an airline company that tries to make
inferences about if the tires of their planes have to be replaced or not will much likely choose
a very low alpha, because making a mistake in that decision would be disastrous for both the

copyright © 2005, Christoph Strehl – mail@cstrehl.de – www.christoph-strehl.net Page 4 of 7

 Hypothesis Testing for Market Research Analysis

company and the customers. On the other side a company who wants to publish a brand-
awareness result may choose a very high alpha because they don’t care about making a type-I-
error.

Calculate the test-statistic

The test-statistic is a metric that is required to decide about the hypothesis. This step involves
a lot of statistical details and depends very much on the specific test-method. Fortunately
there are lot of statistic tools out there who help you to calculate the test-statistic. Most tools
offer you two different ways of input: Either you enter the complete sample data or you
describe the sample data by naming mean, variance and sample size.

As an output, almost every statistical tool will return you a so called p-value that can be easily
used to decide in favour of one of the two hypotheses. That decision is described in the next
step.

Decide whether to accept or reject the Null-Hypothesis

No matter which type of test – parametric or non-parametric – one conducts, it all comes
down to interpreting the p-value. The p-value is the probability that your sample could have
been drawn from the population(s) being tested (or that a more improbable sample could be
drawn) given the assumption that the null hypothesis is true.7

In order to decide for one of the two hypotheses one has to compare the calculated p-value to
the prespecified alpha: If the p-value is lower than alpha, the Null-Hypothesis has to be
rejected in favour of the alternative hypothesis.

There is one exception to that rule. If the test is a two-sided test, the p-value has to be divided
by two before comparing it to the alpha.

Interpret the decision

If the the p-value leads to the conclusion that the Null-Hypothesis has to be rejected, the
sample provided enough statistical significance for the alternative Hypothesis. Statements like
“We can be 95% confident that the alternative Hypothesis is true” can be concluded. In this
example an alpha of 5% was used.

It gets more tricky if the Null-Hypothesis has to be accepted. In this case the only statement
that can be drawn from that is that there is not enough statistical significance to reject the
Null-Hypothesis. IT DOES NOT mean that there is a statistical significance that the Null-
Hypothesis is true, which is a commonly made mistake.

copyright © 2005, Christoph Strehl – mail@cstrehl.de – www.christoph-strehl.net Page 5 of 7

 Hypothesis Testing for Market Research Analysis

Overview of test types

Tests can generally be divided into two groups: Parametric tests and non-parametric tests.
Non-parametric tests don’t have the same strict restrictions as parametric tests. Specifically,
nonparametric methods were developed to be used in cases when the researcher knows
nothing about the parameters of the variable of interest in the population (hence the name
nonparametric).8
.

Parametric Tests
1-sample T-test
A very commonly used test to make inferences about the mean of a population from a sample
mean. Questions that can be answered with this test might be:

• Is the average money that people spend in a fast food restaurant greater than 4$?
• Is the awareness of my product less than 50%?

1-sample Z-Test
This test is very similar to the 1-sample T-Test, but it requires knowledge about the standard
deviation in the population in order to be computed. Because of this restriction which is
seldom fulfilled this test is not frequently used.

2-sample T-test
This test compares two samples and tries to find out differences about both populations.
Questions for this test might be:

• Does the average satisfaction of my customers in two areas differ?

2-sample F-test
Test to determine whether the two variances are equal.

• Does the deviation of the satisfaction of my customers in two different areas differ?

Non-parametric tests
Chi square-test
This test makes inferences about the relationship of two variables. A possible question is:

• Does the gender of a customer have an impact on the satisfaction level?

Sign Test
A test that can be applied for data with a poor quality. That is data where only positive or
negative changes were measured.

copyright © 2005, Christoph Strehl – mail@cstrehl.de – www.christoph-strehl.net Page 6 of 7

 Hypothesis Testing for Market Research Analysis

Wilcoxon Signed Rank test

This test is the nonparametric analogue to the 1-sample T-test.

Mann Whitney U-test

This test is the nonparametric analogue to the 2-sample T-test.

Kolgomorov-Smirnoff test
This test can make inferences about the population distribution of a sample.

• Is the money someone spends per visit in a fast food restaurant normally distributed?

Conclusion
-There are many different tests out there that can be executed. A good market researcher
doesn’t have to know all of them nor know how to compute them. But he has to understand
the general procedure of hypothesis-testing in order to be able to correctly interpret results
from practical market research.

Bibliography

1
Thompson, Bruce (1994). The concept of statistical significance testing. Practical
Assessment, Research & Evaluation, 4(5). [online] Retrieved March 1, 2005 from
http://PAREonline.net/getvn.asp?v=4&n=5
2
David M. Lane (2003), HyperStat Online Textbook. [online] Retrieved March 1, 2005 from
http://davidmlane.com/hyperstat/A29136.html
3
William M.K. Trochim (2002), Hypotheses [online], Retrieved March 1, 2005 from
http://www.socialresearchmethods.net/kb/hypothes.htm
4
Missong, Martin(2004), MMC-class „Applied Data Analysis“, Lesson 4 „Statistical
Hypothesis Testing“
5
Missong, Martin(2004), MMC-class „Applied Data Analysis“, Lesson 4 „Statistical
Hypothesis Testing“
6
Ed Colet (2002), THE (IN)SIGNIFICANCE OF STATISTICAL SIGNIFICANCE [online],
Retrieved March 1, 2005 from http://www.tgc.com/dsstar/00/1003/102239.html
7
iSixSigma (2003), p-value [online], Retrieved March 1, 2005 from
http://www.isixsigma.com/dictionary/P-Value-301.htm
8
StatSoft Inc (2003), Nonparametric Statistics [online], Retrieved March 1, 2005 from
http://www.statsoft.com/textbook/stnonpar.html

copyright © 2005, Christoph Strehl – mail@cstrehl.de – www.christoph-strehl.net Page 7 of 7