R-squared

R squared is a statistical measure of how close the data are to the fitted regression line. It is also
known as the coefficient of determination, or the coefficient of multiple determination for
multiple regression. 0% indicates that the model explains none of the variability of the response
data around its mean.

P Values
P values evaluate how well the sample data support the devils advocate argument that the null
hypothesis is true. It measures how compatible your data are with the null hypothesis. How likely
is the effect observed in your sample data if the null hypothesis is true?

High P values: your data are likely with a true null.

Low P values: your data are unlikely with a true null.

A low P value suggests that your sample provides enough evidence that you can reject the null
hypothesis for the entire population.

t-value?
The t-value is a test statistic for t-tests that measures the difference between an observed sample
statistic and its hypothesized population parameter in units of standard error. A t-test compares
the observed t-value to a critical value on the t-distribution with (n-1) degrees of freedom to
determine whether the difference between the estimated and hypothesized values of the
population parameter is statistically significant.
Applications of t-values include:

comparing two sample means

comparing the means of paired observations

determining the significance of a regression coefficient

comparing two regression coefficients

You can also use t-values in a 1-sample t-test. For example, you want to determine whether the
length of a manufactured part meets its target value of 10cm. You take a sample of 50 parts,
conduct a two-sided 1-sample t-test on their mean length with the following hypotheses:

H0: = 0 (the mean length of all parts meets the target value )

H1: 0 (the mean length of all parts does not meet the target value)
The test produces a t-value of 2.5. On the t-distribution with (n-1 = 49) degrees of freedom, this
t-value corresponds to a p-value of 0.0158. For most common significance levels, this result is
statistically significant. Therefore, you reject the null hypothesis that the mean length meets the
target, and conclude that the process needs improvement.

Normality
In statistics, normality tests are used to determine if a data set is well-modeled by a normal
distribution and to compute how likely it is for a random variable underlying the data set to be
normally distributed.
More precisely, the tests are a form of model selection, and can be interpreted several ways,
depending on one's interpretations of probability:

In descriptive statistics terms, one measures a goodness of fit of a normal model to the
data if the fit is poor then the data are not well modeled in that respect by a normal
distribution, without making a judgment on any underlying variable.

Value

The null hypothesis is rejected if the p-value is less than the significance or level. The level is
the probability of rejecting the null hypothesis given that it is true (type I error) and is most often
set at 0.05 (5%).

CFA (Confirmity Factor Analysis)

In statistics, confirmatory factor analysis (CFA) is a special form of factor analysis, most
commonly used in social research.[1] It is used to test whether measures of a construct are
consistent with a researcher's understanding of the nature of that construct (or factor). As such,
the objective of confirmatory factor analysis is to test whether the data fit a hypothesized
measurement model. This hypothesized model is based on theory and/or previous analytic
research.[2] CFA was first developed by Jreskog[3] and has built upon and replaced older methods
of analyzing construct validity such as the MTMM Matrix as described in Campbell & Fiske
(1959).

Chi Square
Chi-square distribution, showing X2 on the x-axis and P-value on the y-axis. A chi-squared test,
also referred to as test (or chi-squaretest), is any statistical hypothesis test in which the sampling
distribution of the test statistic is a chi-square distribution when the null hypothesis is true.

RMSEA
Georg Rasch mentioned chi-square statistics as a way of evaluating fit of data to the model
(Rasch, 1980, p. 25). Ben Wright's Infit and Outfit mean-square statistics are the chi-square
divided by their degrees of freedom. However, large sample sizes have always posed problems
for significance tests based on chi-square statistics. The issue is that, the larger the sample, the
greater the power, and so ever smaller differences are reported as indicating statistically
significant misfit between the data and the model. Thus very large sample sizes can detect
miniscule differences, and with such samples there is almost no need to undertake a chi-square

test as we know that it will be significant (P. Martin-Lf (1974). Indeed, Georg Rasch himself
remarked: "On the whole we should not overlook that since a model is never true, but only more
or less adequate, deficiencies are bound to show, given sufficient data" (Rasch, 1980, p. 92).
Smith et al. (1998) show that the critical interval values for a Type I error (rejection of a true
hypothesis) associated with these statistics varies with sample size. Experience indicates that,
while the value of mean-square tends to increase only slowly with sample size, the critical
interval associated with a 5% significance level shrinks considerably as sample size increases.
Thus a sample of 50 would have a 5% range for Infit of 0.72-1.28, whereas a sample of 500
would have a 5% range of 0.91-1.09. A sample size of 5000 would have a 5% range of 0.97-1.03
(RMT 17:1 p. 918).
In general, large sample sizes will cause most chi-square-based statistics to almost always report
a statistically significant difference between the observed data and model expectations,
suggesting misfit, regardless of the true situation.
One potential mechanism for accommodating large sample sizes may be to use the Root Mean
Square Error of Approximation (RMSEA, Steiger and Lind, 1980) as a supplementary fit. The
RMSEA is widely used in Structural Equation Modeling to provide a mechanism for adjusting
for sample size where chi-square statistics are used.
Consequently, we set out to test the potential of the RMSEA to supplement the chi-square fit
tests reported for Rasch analyses performed by RUMM2030. This investigation focuses on the
"summary fit chi-square" (the item trait interaction statistic). The utility of the RMSEA to
supplement the interpretation of the chi square fit in larger samples was assessed, along with
determination of the level of RMSEA that is consistent with fit to the Rasch model.