Meta-Analysis: Fixed Effect Analysis

Mike W.-L. Cheung

National University of Singapore

This is a pre-print version of the following entry:

Cheung, M. W.-L. (in press). Meta-analysis: Fixed effect analysis. In M. Allen (Ed.), The SAGE
Encyclopedia of Communication Research Methods. Sage.

3000-word entry from the SAGE Encyclopedia of Communication Research Methods

META-ANALYSIS: FIXED-EFFECTS ANALYSIS

Meta-analysis is a de-facto statistical tool to synthesize research findings in educational,

social, behavioral and medical sciences. Applications of meta-analysis have been increased
tremendously in the past few decades. Statistical models for meta-analysis have also been
improved and extended to including univariate, multivariate and three-level meta-analyses,
and meta-analytic structural equation modeling (MASEM). Meta-analysis can now not only
be used to estimate an average effect, but also to test complicated theories involving
structural equation models. To conduct a meta-analysis, researchers first have to extract effect
sizes and study characteristics from the studies. The effect sizes are used to quantify the
effectiveness of the experiments or the associations among the variables of interest. These
effect sizes are combined by either a fixed- or a random-effects meta-analytic model. The
study characteristics may be used as moderators to predict the heterogeneity of the effect
sizes. This entry provides a summary of various fixed-effects models applicable to metaanalysis.
Overview
Primary studies are usually based on a small number of participants. The statistical
power of the analysis in the primary studies has been called into concerns. Significant effects
do not necessary mean that the effects are strong. Results from various studies on the same
topic may also be contradicting with each other. It is difficult to generalize the findings based
on a few studies. The purpose of a meta-analysis is to synthesize the available empirical
studies in order to draw general conclusions.

There are two models in meta-analysis. The fixed-effects model usually, but not
necessary, assumes that the population effect sizes are the same for all studies, which is
known as homogeneity of effect sizes. This model is also known as the common-effects
model. Another model is the random-effects model which assumes that each study may have
its own population effect size. The fixed-effects model is appropriate if the studies are similar
in terms of measures and other sample characteristics while the random-effects model is
appropriate if the studies are assumed a random sample of a larger population. The primary
objective of a fixed-effects analysis is to draw conclusions based on the studies included in
the meta-analysis. On the other hand, the primary objective of a random-effects model is to
generalize the findings beyond the studies included by estimating the average population
effect and its heterogeneity variance. We will mainly focus on the fixed-effects model in this
entry.
Calculations of Effect Sizes and Their Sampling Variances
Effect size (y) and its associated sampling variance (v) are the primary data in a metaanalysis. The effect size summarizes the strength of the effect in the study while the sampling
variance indicates the precision of the effect size. Most meta-analytic models use a weighted
least squares (WLS) approach to weigh the effect sizes. The key idea is that effect sizes with
higher precision (smaller v) carry heavier weight in estimating the common effect size.
The effect sizes can be a Pearson correlation, standardized mean difference (SMD) or
odds ratio. The sampling variances are functions of the sample sizes (n) and possibly
functions of the types of effect sizes. Taking correlation coefficient as an example, the effect
size is still correlation y r = r and its sampling variance can be calculated by
2

vr 1 r 2 n 1 . Since the sampling distribution of a correlation is skewed unless the

sample size is large or the population correlation coefficient is close to zero, some researchers

prefer to convert the correlation to a Fishers z score y z = 0.5log( (1 r ) (1 r ) ) and

v z 1 ( n 3) before conducting the meta-analysis.

While the Pearson correlation is popular in observational studies, a SMD is popular in
experimental studies. The SMD indicates the difference between an experimental and a

control groups in terms of a common standard deviation S pooled =

n1 1 S12 + n2 1 S22
n1 + n2 2

. It

X1 X 2
n + n2
y SMD
3

is defined as y SMD = 1
, and v SMD = 1
, where
+
n1n2
2n1 + n2
4n1 + n2 9 S pooled

X 1 , S12 and n1 are the sample mean, variance, and sample size for group 1, respectively. The

quantities for group 2 are defined similarly as those in group 1. Once the effect sizes and their
sampling variances are calculated, meta-analysis can be applied regardless of the types of
effect sizes.
Fixed-Effects Models
There are three types of fixed-effects meta-analytic models depending on the research
questions and the types of data. They are the univariate meta-analysis, the multivariate metaanalysis, and MASEM. Univariate meta-analysis is used when there is only one effect size
per study, while multivariate meta-analysis is preferred when there are more than one effect
sizes per study. MASEM may be used to fit structural equation models on a pool of
correlation matrices. The following paragraphs summarize these models.
Univariate Meta-Analysis
The most common model is a univariate meta-analysis, which is used when there is
only one effect size defined in a study. For example, a researcher wants to study the
correlation between participants attitudes on a product and their intention to purchase the
product. The researcher extracts k studies on the correlation between participants attitudes on

a product and their intention to purchase the product. We use yi and vi presenting the effect
size and its known sampling variance in the ith study. The model for a fixed-effects
univariate meta-analysis is

yi F ei ,

(1)

where F is the common effect for all k studies, and Var(ei ) vi is the known sampling
variance in the ith study. From the above model, it is clear that the population effect sizes are
assumed equal across all studies, while the observed differences in the sample effect sizes are
only due to the sampling error.
The estimated common effect and its sampling variance can be estimated by:
k

i 1
k

wi yi

and Var( F )

w
i 1 i

1
k

i1 wi

(2)

where wi 1 vi is the weight in estimating the common effect. Since the standard error (SE)
of the parameter estimate SE (F )= Var( F ) is also available, we may test the null
hypothesis H 0 : F 0 on the estimated common effect with a z F SE (F ) statistic. If the
absolute value of the z statistic is larger than 1.96, it is statistically significant at .05 . It
should be noted, however, the estimated SE is under-estimated when there is a large degree of
heterogeneity in the effect sizes.
Researchers may calculate a Q statistic to test the assumption of homogeneity of the
effect sizes. The Q statistic is defined as:
2

Q wi ( yi F ) .
i 1

Under the null hypothesis of the homogeneity of effect sizes, the Q statistic has an
approximate chi-square distribution with (k-1) degrees of freedom. Statistically speaking, if

(3)

the Q statistic is significant, the null hypothesis of the homogeneity of effect sizes is rejected
at .05 . However, it is generally found that the statistical power of the Q statistic is quite
low in detecting the heterogeneity of effect sizes. On the other hand, the Q statistic is likely
significant when there are lots of studies. Therefore, researchers are not advisable to choose
between the fixed- versus the random-effects models by relying on the significance test on
the Q statistic.
When researchers suspect that some study characteristics may be used to explain the
differences on the effect sizes, they may test this hypothesis by using a meta-regression.
Sample study characteristics, such as year of publication, type of interventions, mean age of
the participants, and proportion of female participants, can be used as moderators. The model
is,

yi 0 1 xi ei ,
where xi is a study characteristic in the ith study, 0 and 1 are the intercept and the
regression coefficient, respectively. 1 indicates the expected change on yi when xi
increases one unit. The above model allows studies with different effect sizes. After including
the appropriate study characteristics in the model, the conditional effect sizes are the same.
There are two cautionary notes. First, the conventional explained variance R 2 is not
well defined in Equation (4). It is because Var(ei ) vi is assumed known and is not involved
in calculating the R 2 . The fixed-effects model assumes that the study characteristic explains
all the heterogeneity of the effect sizes, that is, R 2 1 by definition. The R 2 will be more
useful under the mixed-effects model rather than under the fixed-effects model. Another note
is that it is inappropriate to interpret the results as the individual level. Consider the
proportion of female participants as an example. Even though the effect is stronger in studies

(4)

with more (or less) female participants, it does not imply that the effect is stronger in females
(or males). Researchers should avoid committing ecological fallacy when applying metaregression.
Multivariate Meta-Analysis
When the design of the primary studies become more and more sophisticated, one
effect size may not be sufficient to summarize the effect of the study. For example, a
researcher may want to investigate the correlation between participants attitudes on a
product and their intention to purchase the product, and the correlation between the
participants perceived norms and their intention to purchase the product. One nave approach
is to conduct two univariate meta-analyses on the effect sizes. This approach may work.
However, it is not optimal. The main issue is that the effect sizes within the study are usually
correlated. If we conduct separate univariate meta-analyses, the dependence among the effect
sizes would not be properly handled. This issue is similar to the case of conducting several
analysis of variances (ANOVAs) versus conducting one multivariate analysis of variance
(MANOVA)---the MANOVA is always preferred over separate ANOVAs.
A multivariate meta-analysis extends the univariate meta-analytic model to multiple
effect sizes. Suppose that there are two effect sizes per study, the effect sizes are stacked
together, that is, y i y1

y2 i , F 1 2 F , and e i e1 e2 i where x T is the

transpose of x. The model for a multivariate meta-analysis is,

y i F ei ,
v
v
where F is the vector of the common effect for all k studies, and Var( ei ) Vi 11 21
v21 v22 i

is the known sampling variance covariance matrix in the ith study. There are formulas to
calculate the sampling covariance v21 for many types of effect sizes and dependence. Results

(5)

of separate univariate meta-analyses and a multivariate meta-analysis would be similar when

v21 is close to zero; otherwise, a multivariate meta-analysis is generally preferred.

The estimated common effects and its associated sampling covariance matrix can be
estimated similarity as those in the univariate meta-analysis. Since there are multiple effect
sizes, it is more convenient to stack all k studies together, that is, y y1T

y T2 y Tk .

The parameter estimates and their sampling covariance matrix are

F ( X T VF1X ) 1 X T VF1y and Var( F ) ( X T VF1X ) 1 ,

(6)

where X -1 is the inverse of X . The SEs of the parameter estimates are the square roots of the
diagonals in Var( F ) . We may then test the significance of parameter estimates by using

SE (F )= Var( F ) . A Q statistic similar to the one in Equation (3) may also be developed to
test the null hypothesis of homogeneity of effect sizes across all k studies,
Q =( y X F )T VF1 ( y X F ) ,

(7)

which has an approximate chi-square distribution under the null hypothesis of homogeneity
of effect sizes.
When there are hypotheses about how the study characteristics are related to the effect
sizes, a multivariate meta-regression similar to the one in the univariate meta-analysis may
also be conducted. The model is,

y i 0 1 xi ei .
There are two intercepts and two regression coefficients for one moderator xi when there are
two effect sizes per study. This means that the effect of xi on the effect sizes can be different
for different effect sizes y1 and y2 . One advantage, compared to conducting two separating
meta-regression analyses, is that researchers may test whether the regression coefficients are

(7)

the same.
Meta-Analytic Structural Equation Modeling
Neither univariate nor multivariate meta-analyses is an adequate tool when the
primary studies test theoretical models with SEM. Consider the case that the primary studies
compare different numbers of factor structures or test psychological processes with path
model or structural equation models. Researchers may want to synthesize these findings to
draw general conclusions. Suppose that there are a couple of studies testing the Theory of
Planned Behavior model on purchasing a product, researchers may want to combine these
studies and test whether the Theory of Planned Behavior model holds in the these studies.
MASEM is an extension of the multivariate meta-analysis by allowing researchers to
test hypothesized models on the pooled correlation matrix. There are two stages of analyses
in a MASEM analysis. The first stage of analysis tests the homogeneity of correlation
matrices and estimate a pooled correlation matrix, while the second stage of analysis fits and
compares structural equation models on the pooled correlation matrix.
A common approach is to combine the correlation matrices with the univariate metaanalysis in the first stage of analysis. In the second stage of analysis, the pooled correlation
matrix is treated as if it was a covariance matrix in fitting structural equation models. Since
the elements of pooled correlation matrix are based on different sample sizes, a harmonic
mean is usually calculated as the sample size in fitting structural equation models in the
second stage of analysis.
A more statistical justified approach is to apply a two-stage structural equation
modeling approach (TSSEM). Suppose that there are p variables involved in a MASEM, the
data of inputs are correlation matrices, say R i in the ith study. The first stage of the analysis
involves pooling the correlation matrices by using a special case of the confirmatory factor

analytic model. The population covariance matrix i in the ith study is modeled by,

i Di PD
i i,

(8)

where Pi is the population correlation matrix, and Di is a diagonal matrix representing the
estimated standard deviations in the ith study. The diagonal matrix Di is included so that the
statistical theory of SEM is applicable to correlation matrices. We may impose the equality
constraint P1 P2 Pk to estimate the common correlation matrix P . This step can be
easily implemented in most SEM packages. There are several advantages of this step. First,
missing correlation coefficients can be handled by the use of maximum likelihood estimation,
which is generally recommended for handling missing data. Second, a likelihood-ratio
statistic can be used to test the null hypothesis H 0 : P1 P2 Pk . This is similar to the Q
statistics introduced before. Third, an estimated common correlation matrix P and its
asymptotic sampling covariance matrix V are available after the first stage of the analysis.
After the first stage of the analysis, we have the estimated common correlation matrix
k
R P , its asymptotic sampling covariance matrix V , and the total sample size N i 1 ni .

Researchers can fit a structural equation model, say P() , that can be either a path model,
confirmatory factor analytic model or a full structural equation model with latent variables.
The model can be fitted by using the WLS method. The discrepancy function for the
proposed structural model () vechs(P()) , that takes the non-duplicate elements in P() ,
is

F () (r ())T V 1 (r ())T ,
where r vechs( R ) . After fitting the model, chi-square statistic and various goodness-of-fit
indices may be used to evaluate the model fit of the proposed model, while an z statistic using

(9)

the parameter estimate divided by its SE may be used to test the significance of the parameter
estimates. When there are several theoretically competing models, we may compare them by
fitting different models. If these models are nested, chi-square difference test may be used to
compare them. If these models are non-nested, Akaike information criterion (AIC) can be
used to select the best model.
It should be noted that it may not be appropriate to apply the second stage of analysis
if the studies are heterogeneous. Assuming that there are categorical study characteristics, for
example, types of experimental conditions, researchers may group the studies into several
sub-groups and test whether the heterogeneity can be explained by the study characteristics.
This is similar to the meta-regression in Equation (4) with a categorical moderator. If the
studies are homogeneous within each sub-group, researchers may fit the stage two analysis
for each sub-group.
Other Issues
This entry mainly focuses on the fixed-effects models, which are appropriate when the
studies are direct replicates of each other. Both univariate and multivariate models are
available for researchers to synthesize effect sizes. It should be reminded that the fixedeffects models may not be appropriate when the studies are only conceptual replicates of each
other, that is, the studies may be different in terms of measures, design, and samples. The
random-effects models would be more appropriate.
Researchers rarely conduct their meta-analyses manually. Many computer packages,
such as SPSS, SAS and R, have implemented some programs to conduct meta-analysis. There
are also standalone software for meta-analysis. The metaSEM package implemented in R can
be used to fit the fixed-effects (and also the random-effects) meta-analyses introduced in this
entry.

10

Mike W.-L. Cheung

See also Effect Sizes, Meta-Analysis: Estimation of Average Effect, Meta-Analysis:

Literature Search Issues, Meta-Analysis: Model Testing, Meta-Analysis: Random Effects
Analysis, Meta-Analysis: Statistical Conversion to Common Metric, Structural Equation
Modeling

FURTHER READINGS
Borenstein, M., Hedges, L. V., Higgins, J. P. T., & Rothstein, H. R. (2009). Introduction to
meta-analysis. Chichester, West Sussex, U.K.; Hoboken: John Wiley & Sons.
Card, N. A. (2012). Applied meta-analysis for social science research. New York: The
Guilford Press.
Cheung, M. W.-L. (2015). Meta-analysis: A structural equation modeling approach.
Chichester, West Sussex: John Wiley & Sons.
Cooper, H. M., Hedges, L. V., & Valentine, J. C. (2009). The handbook of research synthesis
and meta-analysis (2nd ed.). New York: Russell Sage Foundation.
Hedges, L. V., & Olkin, I. (1985). Statistical methods for meta-analysis. Orlando, FL:
Academic Press.
Schmidt, F. L., & Hunter, J. E. (2015). Methods of meta-analysis: Correcting error and bias
in research findings (3rd ed.). Thousand Oaks, CA: Sage.

11