1

Definition
In statistics, a null hypothesis is a statement that one seeks to nullify with
evidence to the contrary. Most commonly it is a statement that the phenomenon
being studied produces no effect or makes no difference. An example of a null
hypothesis is the statement "This diet has no effect on people's weight." Usually,
an experimenter frames a null hypothesis with the intent of rejecting it: that is,
intending to run an experiment which produces data that shows that the
phenomenon under study does make a difference.[2] In some cases there is a
specific alternative hypothesis that is opposed to the null hypothesis, in other
cases the alternative hypothesis is not explicitly stated, or is simply "the null
hypothesis is false" in either event, this is a binary judgment, but the
interpretation differs and is a matter of significant dispute in statistics.
A type I error (or error of the first kind) is the incorrect rejection of a true null
hypothesis. Usually a type I error leads one to conclude that a supposed effect or
relationship exists when in fact it doesn't. Examples of type I errors include a test
that shows a patient to have a disease when in fact the patient does not have the
disease, a fire alarm going on indicating a fire when in fact there is no fire, or an
experiment indicating that a medical treatment should cure a disease when in
fact it does not.
A type II error (or error of the second kind) is the failure to reject a false null
hypothesis. Examples of type II errors would be a blood test failing to detect the
disease it was designed to detect, in a patient who really has the disease; a fire
breaking out and the fire alarm does not ring; or a clinical trial of a medical
treatment failing to show that the treatment works when really it does.[3]
In terms of false positives and false negatives, a positive result corresponds to
rejecting the null hypothesis, while a negative result corresponds to failing to
reject the null hypothesis.
When comparing two means, concluding the means were different when in reality
they were not different would be a Type I error; concluding the means were not
different when in reality they were different would be a Type II error. Various
extensions have been suggested as "Type III errors", though none have wide
use.
All statistical hypothesis tests have a probability of making type I and type II
errors. For example, all blood tests for a disease will falsely detect the disease in
some proportion of people who don't have it, and will fail to detect the disease in
some proportion of people who do have it. A test's probability of making a type I
error is denoted by . A test's probability of making a type II error is denoted by .
These error rates are traded off against each other: for any given sample set, the
effort to reduce one type of error generally results in increasing the other type of
 2

error. For a given test, the only way to reduce both error rates is to increase the
sample size, and this may not be feasible.
These terms are also used in a more general way by social scientists and others
to refer to flaws in reasoning.[4] This article is specifically devoted to the statistical
meanings of those terms and the technical issues of the statistical errors that
those terms describe.

Statistical test theory

In statistical test theory, the notion of statistical error is an integral part
of hypothesis testing. The test requires an unambiguous statement of a null
hypothesis, which usually corresponds to a default "state of nature", for example
"this person is healthy", "this accused is not guilty" or "this product is not broken".
An alternative hypothesis is the negation of null hypothesis, for example, "this
person is not healthy", "this accused is guilty" or "this product is broken". The
result of the test may be negative, relative to the null hypothesis (not healthy,
guilty, broken) or positive (healthy, not guilty, not broken). If the result of the test
corresponds with reality, then a correct decision has been made. However, if the
result of the test does not correspond with reality, then an error has occurred.
Due to the statistical nature of a test, the result is never, except in very rare
cases, free of error. Two types of error are distinguished: type I error and type II
error.
Type I error
A type I error occurs when the null hypothesis (H0) is true, but is rejected. It
is asserting something that is absent, a false hit. A type I error may be likened
to a so-called false positive (a result that indicates that a given condition is
present when it actually is not present).
The type I error rate or significance level is the probability of rejecting the null
hypothesis given that it is true.[5][6] It is denoted by the Greek letter (alpha) and is
also called the alpha level. Often, the significance level is set to 0.05 (5%),
implying that it is acceptable to have a 5% probability of incorrectly rejecting the
null hypothesis.
Type II error
A type II error occurs when the null hypothesis is false, but erroneously fails to
be rejected. It is failing to assert what is present, a miss. A type II error may
be compared with a so-called false negative (where an actual 'hit' was
disregarded by the test and seen as a 'miss') in a test checking for a single
condition with a definitive result of true or false. A Type II error is committed when
we fail to believe a truth.[7] In terms of folk tales, an investigator may fail to see the
wolf ("failing to raise an alarm"). Again, H0: no wolf.
 3

The rate of the type II error is denoted by the Greek letter (beta) and related to
the power of a test (which equals 1).
Table of error types
Tabularised relations between truth/falseness of the null hypothesis and
outcomes of the test:[2]

Null hypothesis (H0) is

Table of error types
True False

Type I error Correct

Reject (False inference
Positive) (True Positive)
Decision About Null
Hypothesis (H0)
Correct
Type II error
Fail to inference
(False
reject (True
Negative)
Negative)

Examples[edit]
Example 1[edit]
Hypothesis: "Adding water to toothpaste protects against cavities."
Null hypothesis: "Adding water to toothpaste has no effect on cavities."
This null hypothesis is tested against experimental data with a view to nullifying it
with evidence to the contrary.
A type I error occurs when detecting an effect (adding water to toothpaste
protects against cavities) that is not present. The null hypothesis is true (i.e., it is
true that adding water to toothpaste has no effect on cavities), but this null
hypothesis is rejected based on bad experimental data.
Example 2
Hypothesis: "Adding fluoride to toothpaste protects against cavities."
Null hypothesis: "Adding fluoride to toothpaste has no effect on cavities."
This null hypothesis is tested against experimental data with a view to nullifying it
with evidence to the contrary.
 4

A type II error occurs when failing to detect an effect (adding fluoride to

toothpaste protects against cavities) that is present. The null hypothesis is false
(i.e., adding fluoride is actually effective against cavities), but the experimental
data is such that the null hypothesis cannot be rejected.
Example 3
Hypothesis: "The evidence produced before the court proves that this man is
guilty."
Null hypothesis (H0): "This man is innocent."
A type I error occurs when convicting an innocent person (a miscarriage of
justice). A type II error occurs when letting a guilty person go free (an error of
impunity).
A positive correct outcome occurs when convicting a guilty person. A negative
correct outcome occurs when letting an innocent person go free.
Example 4
Hypothesis: "A patient's symptoms improve after treatment A more rapidly than
after a placebo treatment."
Null hypothesis (H0): "A patient's symptoms after treatment A are
indistinguishable from a placebo."
A Type I error would falsely indicate that treatment A is more effective than the
placebo, whereas a Type II error would be a failure to demonstrate that treatment
A is more effective than placebo even though it actually is more effective.

Etymology[edit]
In 1928, Jerzy Neyman (18941981) and Egon Pearson (18951980), both
eminent statisticians, discussed the problems associated with "deciding whether
or not a particular sample may be judged as likely to have been randomly drawn
from a certain population"[8]p. 1: and, as Florence Nightingale David remarked, "it is
necessary to remember the adjective 'random' [in the term 'random sample']
should apply to the method of drawing the sample and not to the sample itself".[9]
They identified "two sources of error", namely:
(a) the error of rejecting a hypothesis that should have been accepted, and
(b) the error of accepting a hypothesis that should have been rejected.[8]p.31
In 1930, they elaborated on these two sources of error, remarking that:
...in testing hypotheses two considerations must be kept in view, (1) we
must be able to reduce the chance of rejecting a true hypothesis to as low
a value as desired; (2) the test must be so devised that it will reject the
hypothesis tested when it is likely to be false.[10]
 5

In 1933, they observed that these "problems are rarely presented in

such a form that we can discriminate with certainty between the true
and false hypothesis" (p. 187). They also noted that, in deciding
whether to accept or reject a particular hypothesis amongst a "set of
alternative hypotheses" (p. 201), H1, H2, . . ., it was easy to make an
error:
...[and] these errors will be of two kinds:
(I) we reject H0 [i.e., the hypothesis to be tested] when it is true,
(II) we accept H0 when some alternative hypothesis HA or H1 is true.
[11]p.187
(There are various notations for the alternative).
In all of the papers co-written by Neyman and Pearson the
expression H0 always signifies "the hypothesis to be tested".
In the same paper[11]p. 190 they call these two sources of error, errors
of type I and errors of type II respectively.