Weka Error Statistics

Definitions and
Interpretations

Kappa statistic
Kappa is calculated as the observed agreement beyond chance (75% 50% = 25%)
divided by the maximum agreement beyond chance (100% 50% = 50%).
Example 1

Example 2: > 0
Prevalence class is 5%.
Classifier accuracy is 99%.
Kappa = (99-5)/(100-5) = 0.96
Max kappa is 1
Example 3: = 0
Prevalence class is 90%.
Classifier accuracy is 90%.
No accuracy beyond chance.
Kappa = (90-90)/(100-90) = 0
Example 4: < 0
Prevalence class is 90%.
Classifier accuracy is 10%.
Accuracy worse than chance.
Kappa = (10-90)/(100-90) = -8

Mean absolute error

the average of the difference between predicted (output
value) and actual (target, true) value in all tested cases

1
ti oi

n i 1

Root mean squared error

Similar to mean absolute error but taking squares and
then square root

1 n
ti oi

n i 1

Relative absolute error

the total absolute error made relative to what the error
would have been if the prediction simply had been the
average of the target values.
n

t o
i 1
n

t
i 1

Root relative squared error

Similar to relative absolute error but taking squares and
then square root

t o
i 1
n

t t
i 1

Accuracy by class
Predicted class
True class

positive

negative

positive (#P)

#TP

#FN = #P - #TP

negative (#N)

#FP

#TN = #N - #FP

true positive rate = TP = (#TP)/(#P)

the proportion of examples which were classified as class i, among all
examples which truly have class i
For Zero Rule, TP = 1, i.e., all positive cases are predicted as positive
false positive rate = FP = (#FP)/(#N)
For ZeroR, FP = 1, all negative cases are predicted as positive
Precision = TP rate = #TP / (#TP + #FP)) * 100%
If Zero Rule always predict positive, then precision could be low
It is the proportion of the cases which truly have class i among all those
which were classified as class i
Recall = (#P / #TP) * 100%
Reciprocal of TP
Zero Rule can recall 100% of the positive cases
F-measure = 2* recall * precision / recall + precision)