MiMa Function Tutorial (version 1.

4) 1

MiMa: An S-Plus/R Function to fit Meta-Analytic Mixed-,

Random-, and Fixed-Effects Models
Wolfgang Viechtbauer
University of Maastricht
http://www.wvbauer.com/

March 8, 2006

Contents
1 Meta-Analytic Models 2
1.1 Mixed-Effects Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2
1.2 Random-Effects Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2
1.3 Fixed-Effects with Moderators Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2
1.4 Fixed-Effects Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3

2 Hypothesis Tests in the Mixed-Effects Model Context 3

3 MiMa Function for Fitting the Meta-Analytic Models 4

4 Example 4

5 Predicted/Estimated Effect Sizes 7

6 Categorical Moderators 8

7 Fitting a Random-Effects Model 8

8 Negative Heterogeneity Estimates 9

9 Possible Problems when Fitting the Mixed-Effects Model 10

10 Fitting Fixed-Effects Models 10

11 How to Cite the Function (APA style) 11

12 References 11
MiMa Function Tutorial (version 1.4) 2

1 Meta-Analytic Models

1.1 Mixed-Effects Model

Given are i = 1, . . . , k independent effect size estimates, each estimating a corresponding effect
size. This can be expressed algebraically by

Yi = θi + i , (1)

where i is the sampling error for the ith study. The sampling errors are assumed to be nor-
mally distributed with mean zero and variance vi . Now assume that p moderator variables are
influencing the effect sizes in a linear fashion. Let Xij denote the value of the j th moderator
variable for the ith effect size. We then write

θi = β0 + β1 Xi1 + . . . + βp Xip + ui , (2)

where ui is assumed to follow a normal distribution with mean zero and variance τ 2 . The
average effect size is given by β0 +β1 Xi1 +. . .+βp Xip , while the study-specific effect size is given
by (2). Therefore, τ 2 denotes the amount of residual heterogeneity, that is, the amount of excess
or unexplained variability in the effect sizes after accounting for the amount of heterogeneity
we would expect based on the moderator variables that are included in the model. Combining
Equations 1 and 2 yields the mixed-effects model, which is given by

Yi = β0 + β1 Xi1 + . . . + βp Xip + ui + i . (3)

Therefore, βj denotes by how much the average effect size changes for a one unit increase in
the corresponding moderator variable when holding all other moderator values constant, while
β0 denotes the average effect size when all moderator variables are equal to zero.

1.2 Random-Effects Model

When no moderators are included in the model, then the mixed-effects model simplifies to

Yi = µ + ui + i , (4)

where β0 has been replaced with µ. According to the random-effects model, all of the hetero-
geneity among the effect sizes is random and cannot be accounted for by moderators. There-
fore, τ 2 in the random-effects model simply denotes the amount of total heterogeneity, that is,
the amount of variability in the effect sizes.

1.3 Fixed-Effects with Moderators Model

When moderators are included in the model and the moderators can account for all of the
heterogeneity among the effect sizes, then the fixed-effects with moderators model applies,
which is given by
Yi = β0 + β1 Xi1 + . . . + βp Xip + i . (5)
Therefore, this model is again just a special case of the mixed-effects model, where τ 2 = 0.
MiMa Function Tutorial (version 1.4) 3

1.4 Fixed-Effects Model

When all of the effect sizes are homogeneous (i.e., no moderators are operating and there is no
random heterogeneity among the effect sizes), then θ1 = θ2 = . . . = θk = θ and the appropriate
model is given by
Yi = θ + i , (6)
using θ in place of β0 .

2 Hypothesis Tests in the Mixed-Effects Model Context

Once estimates of β0 through βp and τ 2 have been obtained (i.e., once the model has been
fitted), we can conduct a variety of hypothesis tests:

1. We can test whether the amount of residual heterogeneity (τ 2 ) is significantly different

from zero (i.e., H0 : τ 2 = 0). For this, we use the QE statistic, which follows a chi-square
distribution under the null hypothesis with degrees of freedom equal to k − p − 1.

2. We can also test whether any of the β1 , . . . , βp parameters in the model are significantly
different from zero, or in other words, whether at least one moderator is influencing the
effect sizes (i.e., H0 : β1 = β2 = . . . = βp = 0). For this, we use the QME statistic, which is
distributed chi-squared with p degrees of freedom (i.e., equal to the number of parameters
tested) under the null hypothesis.

3. We can also test individual parameters, by dividing the parameter estimate by its stan-
dard error. This ratio follows a standard normal distribution under the null hypothesis.
Confidence intervals for each parameter are also easily obtained by inverting these tests.

Notes: (1) The sampling variances (i.e., the vi values) are usually not exactly known. In-
stead, the vi values are replaced with corresponding estimates (v̂i ). Therefore, the distribu-
tions of the test statistics above are only exact when the within-study sample sizes are large
(i.e., when the error in the sampling variance estimates is negligible). (2) Moreover, when fit-
ting a mixed/random-effects model, then τ 2 must be estimated (τ̂ 2 ). The estimate is then
simply treated as a known constant. Therefore, the QME statistic and the individual param-
eter tests assume that k is sufficiently large, such that τ 2 is estimated with high precision
and the error in the estimate is negligible. (3) Finally, most effect size measures are not ex-
actly normally distributed, as assumed under the meta-analytic models. However, the normal
distribution approximation usually becomes more accurate as the within-study sample sizes
increase. Therefore, large within-study sample sizes are needed to be certain that the tests
perform as desired (specifically, that the Type I error rate is controlled). Refined moderator
tests that provide better control of the Type I error rate when these assumptions are violated
have been developed and will be included in the MiMa function later on.
MiMa Function Tutorial (version 1.4) 4

3 MiMa Function for Fitting the Meta-Analytic Models

The S-PLUS/R function “mima” can be used for fitting the various meta-analytic models.
Usage:

mima(yi, vi, mods, method="REML", threshold=0.00001, maxiter=100, alpha=0.05,

verbose="no", digits=4, fe="no", out="no")

Required Arguments:

yi -- a vector of effect size estimates

vi -- a vector of sampling variances
mods -- a vector/matrix of moderator values

Optional Arguments:

method -- method for estimating the amount of (residual) heterogeneity;

possible methods are: "REML", "ML", "HE", "DL", "SH", or "EB";
the default is "REML" (see below for more details)
threshold -- precision of the Fisher scoring algorithm when using either
ML, REML, or EB estimation; the default is 0.00001
maxiter -- maximum number of iterations of the algorithm when using either
ML, REML, or EB estimation; the default is 100
alpha -- to set the width of the confidence interval for individual
parameters (default is .05, which yields a 95% CI)
verbose -- set to "yes" to obtain information on the evolution of the
iterative algorithm when using either ML, REML, or EB estimation
digits -- number of decimals for rounding of results; the default is 4
fe -- set to "yes" to force fitting a fixed-effects model
out -- see below for explanation

Several methods for estimating the amount of (residual) heterogeneity are available. The de-
fault is restricted maximum-likelihood (REML) estimation and this is the recommended method
(Viechtbauer, 2005). Regular maximum-likelihood (ML) estimation is also available, but it is
not recommended, because the estimate of τ 2 tends to be too small (negatively biased). Two
method of moment estimators are also available (HE and DL, respectively); the first is an ex-
tension of the estimator suggested by Hedges (1983, 1989) and the other an extension of the
estimator suggested by DerSimonian and Laird (1986). More details on these two estimators
can be found in Raudenbush (1994). Finally, the estimator suggested by Sidik and Jonkman
(2005a, 2005b) and the empirical Bayes estimator (Berkey, Hoaglin, Mosteller, & Colditz, 1995;
Morris, 1983) are also available (SH and EB, respectively). The ML, REML, and EB estimation
methods make use of the Fisher scoring algorithm, which is robust to poor starting values and
usually converges quickly (Harville, 1977; Jennrich & Sampson, 1976).

4 Example
Table 1 provides the results for k = 16 studies examining the effectiveness of massage therapy
for relieving state anxiety. Listed are the sample sizes of each study (nEi and nC
i ), the effect size
MiMa Function Tutorial (version 1.4) 5

estimate (Yi ), the estimated sampling variance (v̂i ), the minutes per session of massage therapy
provided in each study, whether a fully trained massage therapist or a layperson provided the
therapy, and the mean age of the sample. The data were adopted from Moyer, Rounds, and
Hannum (2004), leaving out three studies with missing data and two studies where the duration
of massage therapy provided was less than 10 minutes.

Table 1
Results for 16 Studies on the Effectiveness of Massage Therapy for Reducing State Anxiety
Sample Sizes Effect Size Sampling Minutes/ Trained Mean
Study nEi nCi Estimate (Yi ) Variance (v̂i ) Session Therapist Age
1 30 30 0.444 0.068 30 0 28
2 46 39 −0.495 0.049 10 0 42
3 15 15 0.195 0.134 20 1 31
4 10 10 0.546 0.207 40 1 39
5 12 12 0.840 0.181 20 1 17
6 10 10 0.105 0.200 30 1 51
7 26 24 0.472 0.082 15 1 26
8 18 14 −0.205 0.128 10 0 64
9 12 12 1.284 0.201 45 1 48
10 12 12 0.068 0.167 30 1 40
11 15 15 0.234 0.134 30 1 52
12 12 12 0.811 0.180 30 1 33
13 15 15 0.204 0.134 30 1 20
14 18 18 1.271 0.134 60 1 27
15 15 15 1.090 0.153 45 1 52
16 43 35 −0.059 0.052 10 1 61
Note. Adopted from Moyer, Rounds, and Hannum (2004). Minutes/Session = minutes of therapy provided per
session; Trained Therapist: 0 for a layperson providing the therapy and 1 for a trained therapist; Mean Age =
mean age of the sample.

The data are available in the S-PLUS dataset file “dat.sdd”. The names of the variables are:
n1, n2, yi, vi, minutes, trained, and meanage. In order to use the mima function, we must
first create a matrix with the moderator values that we want to include in the model. For this,
load the dataset, open up the command window (click on “Window” in the menu bar and choose
“Command Window”), and type:

> attach(dat)
> mods <- cbind(minutes, trained, meanage)

The attach command makes the variables in the dataset available (technically, it adds the
dataset to the S-PLUS search path). The second command creates a matrix, called “mods”,
where the columns are composed of the minutes, trained, and meanage variables.
Next, load the mima function by double-clicking on the file called “mima.ssc” (if asked
whether to overwrite the existing script, then this means that the function is already loaded,
but to be certain that the newest version of the function is available, select ok). Once the script
MiMa Function Tutorial (version 1.4) 6

is loaded, run it (i.e., make it available) by hitting F10 (you can then close the script window).
Finally, to fit the mixed-effects model to the massage therapy data, simply type:

> mima(yi, vi, mods)

The output should then look like this:

Estimate of (Residual) Heterogeneity: 0

Test for (Residual) Heterogeneity:

QE = 9.1278
df = 12
p-value = 0.692

Parameter Estimates:

[,1]
intrcpt -0.2971
minutes 0.0250
trained 0.3013
meanage -0.0059

Variance-Covariance Matrix of Parameter Estimates:

intrcpt minutes trained meanage

intrcpt 0.1218 -0.0013 -0.0112 -0.0018
minutes -0.0013 0.0000 -0.0005 0.0000
trained -0.0112 -0.0005 0.0382 -0.0001
meanage -0.0018 0.0000 -0.0001 0.0000

Omnibus Test of all Moderators:

QME = 28.9176
df = 3
p-value = 0

Individual Moderator Tests:

estimate SE zval pval CI_L CI_U

intrcpt -0.2971 0.3490 -0.8513 0.3946 -0.9811 0.3869
minutes 0.0250 0.0067 3.7172 0.0002 0.0118 0.0381
trained 0.3013 0.1954 1.5422 0.1230 -0.0816 0.6842
meanage -0.0059 0.0063 -0.9488 0.3427 -0.0182 0.0063
MiMa Function Tutorial (version 1.4) 7

The output shows that no residual heterogeneity remains when all three moderators are
included in the model (the estimate of residual heterogeneity is zero). The test for residual
heterogeneity is also far from significant (QE = 9.13, df = 12, p = .692). The QME statistic is
highly significant (QME = 28.92, df = 3, p < .0001), indicating that at least one moderator vari-
able is related to the effect sizes. The individual parameter tests suggest that it is the minutes
moderator that is significant (95% confidence interval bounds for this parameter: (0.01, 0.04),
z = 3.72, p = .0002).
One can also obtain a confidence interval for the amount of (residual) heterogeneity in the
model with the tau2cime.r function available also on the author’s website. Simply open that
script in S-Plus and source it with F10. Then type:

> tau2cime(yi, vi, mods)

95% CI: (0, 0.144)

which shows that the bounds for a 95% confidence interval for τ 2 are given by (0, 0.144). For
more information about this function, see the author’s website.

5 Predicted/Estimated Effect Sizes

Once a model has been fitted, it is easy to obtain predicted/estimated effect sizes from the
output. Recall that the output for the mixed-effects model fitted earlier included the following
parameter estimates:

[,1]
intercept -0.2971
minutes 0.0250
trained 0.3013
meanage -0.0059

Therefore, the predicted/estimated effect size for 30 minutes of therapy provided by a fully
trained therapist to a group with an average age of 40 is:

−0.2971 + 0.0250(30) + 0.3013(1) − 0.0059(40) ≈ 0.52.

We can use S-PLUS/R to carry out these calculations. We first fit the model again, using the
out="yes" option. With this argument set to yes, the estimate of residual heterogeneity, the
parameter estimates, and the variance-covariance matrix of the parameter estimates are given
in list form for further processing. Specifically, if we write:

> res <- mima(yi, vi, mods, out="yes")

then the output is put into the object “res”. Next, we select some moderator values and then
multiply the moderator values with the parameter estimates to obtain a predicted effect size.
For example, for minutes = 30, trained = 1, and meanage = 40, we use:

> mvals <- cbind( c(1, 30, 1, 40) )

> t(mvals) %*% res$b
[,1]
[1,] 0.5157254
MiMa Function Tutorial (version 1.4) 8

The “1” in front of the “30” on the line starting with mvals is needed, because of the intercept
parameter in the model. The variance/standard error of this estimate can also be obtained. To
obtain the variance, we type:

> t(mvals) %*% res$vb %*% mvals

[,1]
[1,] 0.0103143

and for the standard error (the square root of the variance), we use:

> sqrt( t(mvals) %*% res$vb %*% mvals )

[,1]
[1,] 0.1015593

An approximate 95% confidence interval is then given by adding and subtracting 1.96 times the
standard error from the predicted effect size:

0.5157 ± 1.96(0.1016) = (0.32, 0.72).

6 Categorical Moderators
Categorical moderators can be included in the model, in the same way that linear models can
include appropriately coded categorical independent variables (in the example above, the vari-
able specifying whether a layperson or a trained therapist provided the treatment was dummy
coded). See, for example, Neter, Kutner, Nachtsheim, and Wasserman (1996) for more details
on how to code categorical independent variables in the context of linear models.

7 Fitting a Random-Effects Model

One can also fit the random-effects model with the mima function. The random-effects model
is simply a special case of the mixed-effects model, which does not include any moderator
variables. For this, create an empty moderator matrix and then run the mima function as
follows:

> mods <- c()

> mima(yi, vi, mods)

Estimate of (Residual) Heterogeneity: 0.1648

Test for (Residual) Heterogeneity:

QE = 38.0456 df = 15 p-value = 0.0009

MiMa Function Tutorial (version 1.4) 9

Parameter Estimates:

[,1]
intrcpt 0.3772

Variance-Covariance Matrix of Parameter Estimates:

intrcpt
intrcpt 0.0183

Omnibus Test of all Moderators:

QME = NA df = 0 p-value = NA

Individual Moderator Tests:

estimate SE zval pval CI_L CI_U

intrcpt 0.3772 0.1354 2.7855 0.0053 0.1118 0.6425

Note that the estimate of residual heterogeneity now indicates the amount of (total) het-
erogeneity in the effect size estimates over and beyond that which we would expect based on
sampling variability alone. The estimate is quite large (τ̂ 2 = 0.1648). The test for residual
heterogeneity is now equivalent for the usual Q-test for heterogeneity and also reveals that
the effect sizes are heterogeneous (QE = 38.05, df = 15, p = .0009). The QME statistic is
not needed here (since there are no moderators in the model), but we can test whether the
average effect size (i.e., µ) is significantly different from zero, which turns out to be the case
(z = 2.79, p = .0053). The bounds for a 95% confidence interval for µ are (0.11, 0.64). However,
given that at least one moderator appears to be influencing the effect sizes, the results from
the random-effects model are incomplete at best. Again, a confidence interval for τ 2 can also
be obtained with the tau2cime.r function available on the author’s website:

> tau2cime(yi, vi, mods)

95% CI: (0.035, 0.524)

which shows that the bounds for a 95% confidence interval for τ 2 are given by (0.035, 0.524).

8 Negative Heterogeneity Estimates

Except when using the Sidik and Jonkman estimator (SH), it is possible that the heterogeneity
estimate turns out to be negative when using any of the estimation methods. However, negative
estimates of τ 2 are outside of the parameter space. For the method of moments estimators,
negative estimates are therefore simply truncated to zero. For ML, REML, and EB estimation,
the Fisher scoring algorithm used in the mima function was modified using step halving to
guarantee a non-negative estimate (see Harville, 1977, and Jennrich & Sampson, 1976, for more
details on ML and REML estimation and penalty techniques).
MiMa Function Tutorial (version 1.4) 10

9 Possible Problems when Fitting the Mixed-Effects Model

Two problems can occur when fitting the mixed-effects model with the mima function:

1. First of all, there is a chance that the Fisher scoring algorithm used for ML, REML, and
EB estimation does not converge. One can try increasing “maxiter” (the default is 100
iterations), but there are cases where the algorithm will cycle between several estimates
and never converge, no matter how many iterations are run (by setting verbose = "yes",
one can see how the iterative algorithm is progressing). In this case, one can try a different
estimation method thatn the default, which is REML estimation.

2. Another problem that can occur is the following error message when running the function:
“Problem in solve.qr(a): apparently singular matrix”. This problem can occur for at least
three reasons.

(a) When there are linear relationships between the moderator variables (i.e., the values
of one moderator variable can be given as a linear function of the other moderator
variables), then this error will occur. In this case, one must remove moderators
variables from the model until such linear relationships are no longer present.
(b) When some of the moderators are highly correlated, then this error can also occur
(a correlation of exactly ±1 indicates a linear relationship between two moderators
and we are back to the first case). Highly correlated moderators could be argued to
be redundant anyway and therefore one should probably remove one of them.
(c) The error can also occur even when the moderators are not highly correlated, but
the values of the moderators are scaled very differently (e.g., the values of one mod-
erator are between 0 and 1, while the values of another moderator are between 1
and 10,000). Such differential scaling can cause numerical problems with the matrix
inversions used as part of the algorithm (right now, “brute force inversion” is used
in the iterative procedures – in the future, this may be replaced with more elegant
methods). The solution here is to make sure that the scaling is not too dissimilar (it
does not have to be exactly the same). One could try dividing the values of a moder-
ator with very large values by 10 or 100 and re-running the model. For example, to
re-scale the meanage moderator, use:
> meanage <- meanage/10
> mods <- cbind(minutes, trained, meanage)
> mima(yi, vi, mods)
The re-scaling will change the parameter estimate and variance for this moderator,
but the z-value and p-value should remain unchanged. The other parameter esti-
mates should not be affected by the re-scaling.

10 Fitting Fixed-Effects Models

One can also force the mima function to fit a fixed-effects model (with or without moderators).
To do this, set fe="yes". The estimate of (residual) heterogeneity will automatically be set
MiMa Function Tutorial (version 1.4) 11

to zero. This is not recommended, because the resulting moderator tests can have severely
inflated Type I error rates (i.e., moderators that are not actually significant may appear to be
significant). This option is available only for demonstration purposes and for conducting a
sensitivity analysis, but should not be used to actually test the moderators.

11 How to Cite the Function (APA style)

The function and this manual/tutorial should be cited (APA style) as follows:

Viechtbauer, W. (2006). MiMa: An S-Plus/R function to fit meta-analytic mixed-, random-, and

fixed-effects models [Computer software and manual]. Retrieved from http://www.wvbauer.
com/.

12 References
Berkey, C. S., Hoaglin, D. C., Mosteller, F., & Colditz, G. A. (1995). A random-effects regression
model for meta-analysis. Statistics in Medicine, 14, 395-411.
DerSimonian, R., & Laird, N. (1986). Meta-analysis in clinical trials. Controlled Clinical Trials,
7, 177–188.
Harville, D. A. (1977). Maximum likelihood approaches to variance component estimation
and to related problems. Journal of the American Statistical Association, 72, 320–338.
Hedges, L. V. (1983). A random effects model for effect sizes. Psychological Bulletin, 93,
388–395.
Hedges, L. V. (1989). An unbiased correction for sampling error in validity generalization
studies. Journal of Applied Psychology, 74, 469–477.
Jennrich, R. I., & Sampson, P. F. (1976). Newton-Raphson and related algorithms for maxi-
mum likelihood variance component estimation. Technometrics, 18, 11–17.
Morris, C. N. (1983). Parametric Empirical Bayes inference: Theory and applications (with
discussion). Journal of the American Statistical Association, 78, 47-65.
Moyer, C. A., Rounds, J., & Hannum, J. W. (2004). A meta-analysis of massage therapy
research. Psychological Bulletin, 130, 3–18.
Neter, J., Kutner, M. H., Nachtsheim, C. J., & Wasserman, W. (1996). Applied linear statistical
models (4th. ed.). Irwin: Chicago.
Raudenbush, S. W. (1994). Random effects models. In H. M. Cooper & L. V. Hedges (Eds.),
The handbook of research synthesis (pp. 301–321). New York: Russell Sage.
Sidik, K., & Jonkman, J. N. (2005a). A note on variance estimation in random effects meta-
regression. Journal of Biopharmaceutical Statistics, 15, 823-838.
Sidik, K., & Jonkman, J. N. (2005b). Simple heterogeneity variance estimation for meta-
analysis. Journal of the Royal Statistical Society, Series C, 54, 367-384.
Viechtbauer, W. (2005). Bias and efficiency of meta-analytic variance estimators in the
random-effects model. Journal of Educational and Behavioral Statistics, 30, 261-293