Controversial Issues Require Clear and Objective Methods to Minimize Biases: Response to

Schumm et al. (2014)

Robert E. Larzelere, Ph.D.
Oklahoma State University
I decided to become a family scientist partly because I thought that children and families deserve
the best science that we can achieve. Science should be used to improve the quality of lives and
relationships as much as it has been used to improve technology. Strongly held values can either
help or hinder achieving that goal. Because we all spend important segments of our lives in some
type of family-like relationships, personal values are both more important and more problematic
in family science than in the natural sciences. For a variety of reasons, our culture often seems
incapable of having civil constructive discussions among people with a diversity of values on
important issues. Instead such discussions usually produce more heat than light, especially on
controversial issues. Family science should be shedding more light than heat than other segments
of society, but we are creatures of our own culture and of our own unique family-related
experiences and values. Although the old ideal of value-free science has been shown to favor
insiders values over under-represented values, it is not clear that we have developed fairer
alternatives. The ideal goal of unbiased objectivity in science can be challenged by the strong
values people have about issues related to family-like relationships. Arguably no issue in family
science provokes more polarized values than issues relevant to marriage and parenting for gays
and lesbians. As a family methodologist, my response therefore focuses on issues about using
our diversity of values to enhance family science as well as critiquing specific aspects of this
particular study.
This paper by Schumm, Landess, and Williams tries to address one major criticism of a
controversial study on parenting by gays and lesbians (Regnerus, 2012). The criticism is that, in
order to maximize sample size, Regnerus (2012) included any fleeting homosexual activity on
the part of a parent as sufficient to qualify for being counted as part of his gay or lesbian
parenting groups. His homosexual parenting groups were therefore biased, partly because
instability of parents intimate relationships was greater in his homosexual parenting groups than
in the comparison groups of heterosexuals. A strength of this new study by Schumm et al. is that
it directly investigates the effect of the stability of parental relationships on outcomes in grown
children within both heterosexual and homosexual parenting groups. Their finding that stability
improved outcomes within both types of parenting couples is an important contribution,
especially now that gay and lesbian marriage is rapidly becoming legal everywhere in the United
States. Children of the most stable parenting partners scored better than less stable parenting
partners on 21 relevant outcomes for heterosexual parents and on 17 relevant outcomes for
lesbian parents. This is remarkably similar given the disadvantages faced by homosexual couples
and their children due to discrimination. It would be less biased to compare effect sizes than to
compare the number of differences to determine whether the stability of parenting couples
matters as much or more among homosexual parents than among heterosexual parents.
Especially on controversial issues, it is crucial to be clear and fair in designing the groups to be
compared, in choice and operationalization of outcome variables, and in reporting the results to
maximize clear interpretations by readers across the entire spectrum of relevant values. Probably
because this paper is a first draft, it seems to fall short of those optimal characteristics at times.

First, the study would be strengthened by making the groups as similar as possible except for the
central independent variables (stability and sexual orientation). Although distinguishing a stable
lesbian group is a major improvement on Regnerus (2012) operationalization of his same-sex
parenting categories, careful matching would have been better (e.g., Wainright & Patterson,
2008). Using 2+ matches for each same-sex parenting couple would improve statistical power
beyond using only one-to-one matches. Although there is a small sample of stable lesbian samesex parents, there is a much larger sample of heterosexual parenting couples from which matched
cases could be drawn to reduce biases in the comparisons being made. It would also be better to
match the groups with the same sexual orientation on irrelevant variables that might bias the
stability results due to confounding if possible. The authors acknowledge that they did not select
an equally stable group from the heterosexual stepfamilies to match the stable lesbian group, but
this is one example of the kind of matching that would improve the group comparisons.
The authors should also specify any important details about outliers, which they mention in the
last section. Especially in small groups, an outlier could dramatically change the results. It would
be helpful to know which conclusions depend upon extreme outliers and which do not. Given the
difficulty of matching closely on multiple variables, some variables could be incorporated as
covariates. For example, the authors could consider controlling for the extent to which the
children and the parents report experiencing discrimination due to homosexuality. Such a
variable might also be a moderator of the effects of the sexual orientation or of parenting stability
on their children. Such a moderator might be useful to predict how much better the children of
homosexual parents could fare in our society if discrimination against them were minimized or
eliminated.
To make the results more clear and transparent, it would be better to report both effect sizes and
statistical significance more consistently and clearly. The reported differences due to stability
within heterosexual families were apparently all significant at the .05 level. The last sentence of
the first paragraph under Results uses the ambiguous term only small differences, without
clarifying whether those are marginally significant or not significant. Instead of limiting the
section on stability effects within heterosexual families to that topic, that section concludes by
comparing same-sex families vs. stable heterosexual families on only one outcome, childhood
sexual abuse. Then it features only one misleading statistic, that the highest rate of childhood
sexual abuse occurred for nonheterosexual children from stable lesbian families (43%). This is
most likely a small-sample outlier, since it is based on the 3 of 7 nonheterosexual children who
reported having been sexually abused by a parent or caretaker. The abusers were most likely to
be males, according to rape statistics available to me. Consistent with other results in this study,
this 43% rate is not significantly higher than the rate for nonheterosexual children of
heterosexual stepfamilies. Furthermore, Schumm et al. omitted the fact that the lowest rate of
childhood sexual abuse was for heterosexual children of stable lesbian parents (0 of 21).
Altogether, the children of stable lesbian parents were not significantly more likely to experience
childhood sexual abuse than were children of intact heterosexual couples, p = .08, Fishers exact
test.
To the authors credit, the next section on the effects of stability for same-sex families
acknowledges the difficulty of detecting significant differences because of the smaller sample

sizes. But the authors never clarify which reported differences were statistically significant.
Comparing effect size differences for the most stable vs. less stable family structures would
provide a fairer test of whether stability effects are equivalent within homosexual and
heterosexual families. Matching on important confounding variables or controlling for them
would make the comparisons less biased as well.
For some reason, this draft includes much less information about the third hypothesis, which
compares children of the most stable lesbian families with the two heterosexual groups. It almost
looks like the authors ran up against limits of pages or time to describe their results carefully and
fairly. Although Table 3 orders the outcome variables by effect size of their differences, there is
no information on statistical significance, although I later discovered that the significance was
reported in Table 2 for continuous outcomes, but that information is lacking for binary outcomes
in Table 1. This could be done via 2 X 2 chi-squared analyses, analogous to pairwise
comparisons following ANOVA, using both liberal and conservative controls for Type I errors.
Although the children of intact heterosexual parents were doing significantly better on several
outcomes, that may be due to the greater stability in their family of upbringing than experienced
by any other group. The initial sentence comparing the children of stable lesbian parents vs.
heterosexual stepfamilies seems misleading. The statement is Second, the extent of differences
between heterosexual stepfamilies and same-sex stable families was much reduced [compared
to differences favoring intact heterosexual families]. It turns out that the only significant
difference on child outcomes between stable lesbian families and heterosexual stepfamilies
favors lesbian families: Their children had significantly higher self-esteem than do children of
heterosexual stepfamilies. But you have to dig quite a bit to figure that out, because that
conclusion is not obvious either in the text or in Table 3. (It can be deduced from Table 2 if the
reader notices the subtitles of the 1st and 2nd data columns, which are presented in parentheses.)
The only outcome on which the children of stable same-sex parents are even marginally worse
than children of heterosexual stepfamilies is physical health. But physical health seems to be an
idiosyncratic outcome, given that it is the only variable for which the children of stable same-sex
parenting couples score worse than children of less-stable same-sex parenting couples.
The interaction effects of sexual orientation of children with sexual orientation of parents are
intriguing, but require replications to determine their reliability.
A careful revision of this study, which incorporates this critique and other critiques, could be an
important contribution to the literature and to the welfare of children who may be affected by
policies related to this topic. Family science has benefited from those whose experiences and
values are under-represented in the dominant values in our discipline (Larzelere & Skeen, 1984).
To take an obvious example, the conclusions made during the days of value-free social science
were biased in favor of the values that dominated social science then. Those biases were never
adequately corrected until under-represented voices were able to contribute the kind of
scholarship necessary to correct those biases. Family science always needs to hear credible
critiques and research from peoples and values that are under-represented in our discipline. Two
overlapping recommendations for using the diversity of family-related values to improve family
methodology are (1) to include insider as well as outsider perspectives at all stages of a research
study and (2) to include the widest possible diversity of relevant values on the research team

(Larzelere & Klein, 1987). No value position should have veto power, but all value positions
should be heard and evaluated fairly.
Family science would benefit in many ways by incorporating a wider diversity of relevant values
in research deliberations and collaborations. Consider, for example, the relevance of attribution
theory for how family science gets perceived by others. Attribution theory is the theory of how
people explain others behavior (Myers, 1983, p. 73). For example, how do other social
scientists and/or the general public explain the behavior of a social scientist who makes a
conclusion that supports parenting by gays and lesbians? According to attribution theorys
discounting principle, people hearing that conclusion will discount its scientific credibility if they
perceive pro-homosexual motivations for that conclusion (Ross & Fletcher, 1985). On the other
hand, the augmentation principle states that the credibility of a conclusion supporting
homosexuals will be strengthened if the social scientist is perceived as having motivations
discouraging such conclusions. That is, to the extent people believe that personal attitudes or
external social pressures favor one conclusion on a controversial issue, they will discount the
perceived scientific objectivity of conclusions supporting the favored conclusion, while they will
augment the scientific credibility of conclusions that disagree with the favored conclusion. The
perceptions of personal biases can arise in several ways, but it is a more viable possibility when a
social scientists conclusions invariably support one side of a controversial issue, especially if the
many judgment calls inherent in science seem to consistently favor making the apparently
preferred conclusions (Schumm, 2012). Note that neither the discounting nor the augmentation
principle proves that a conclusion is incorrect, even though both principles can influence the
perceived credibility of that conclusion. It is dangerous, however, for a social science to be too
quick to dismiss conclusions as biased based on under-represented values, because it is precisely
those under-represented values that can correct blind spots due to the dominant mainstream
biases in that social science. Recall that it was primarily minorities and women who exposed the
White male biases when social science pretended to be value-free.
The discounting and augmentation principles can also be based on perceived external pressures
to support one side of a controversial issue. The external pressures for anti-homosexual or prohomosexual conclusions have changed dramatically over the last 50 years. It used to be
considered hazardous to a professional career to do pro-homosexual research 50 years ago
(Galliher, Brekhus, & Keys, 2004). Galliher et al. (2004) claim that Laud Humphreys was the
first sociologist to come out as gay in 1974. Attribution theory implies that his pro-homosexual
conclusions would have then been seen as more credibly based on science due to external
pressures in the opposite direction, although his coming out as gay made a discounting principle
more apparent to others. Currently, however, conclusions that fail to support homosexual
advocacy are considered dangerous to ones professional career, which decreases the perceived
credibility of pro-homosexual conclusions according to the discounting principle, and increases
the perceived credibility of alternative conclusions. A previous president of the American
Psychological Association, who advocated for homosexual rights during the 1970s and served as
a consultant only for Democratic administrations, says that the American Psychological
Associations credibility as a scientific society has been undermined because its resolutions
invariably support the politically liberal side of any controversial issue (Cummings &
O'Donohue, 2008). From an attribution perspective, this new study by Schumm et al. has the
potential to provide the most convincing evidence to date that children of stable lesbian parents

have outcomes that are at least equivalent to children of heterosexual parents who have also
experienced some disruptions in their family structure. If this study can be improved along the
lines suggested in this critique, this resulting paper can provide even stronger evidence that
stable lesbian parents provide a parenting environment that is at least equivalent to heterosexual
parents, with the important exception of heterosexual parents who remain together throughout
the growing up years of their children.
Of course, this study also has other implications that seem somewhat consistent with other
relevant literature. There is a shortage of adequate studies comparing the children of stable samesex father couples to comparable heterosexual two-parent couples. Societal support for gay
marriage should increase the number and stability of gay couples who decide to include
biological or adopted children in their family. It would be ideal if objective research could
evaluate such parenting as it increases in quantity and quality in the coming years. Will the
stability of gay and lesbian couples catch up with the stability of heterosexual couples? Can these
stabilities be predicted in a way that would improve the welfare of children affected by
placement decisions regardless of the sexual orientation of the prospective parents?
It would also be ideal if well-trained social scientists across a wide range of relevant value
perspectives could collaborate on major studies or meta-analyses on controversial topics such as
this one. The range of value positions among social scientists who collaborate and will defend
their ultimate conclusions can expand the range of the value positions that will perceive the
studies conclusions to be credible. On the other hand, if only a narrow range of dominant value
positions produce conclusions in the name of family science, those conclusions may be viewed
as credible only by an equally narrow range of policy makers. The best examples of experts
trying to incorporate the full range of values on this topic are Loren Marks (2012), David
Blankenhorn (2012, June 22), and Bill Doherty (2013, July 27). Marks sought a wide diversity of
critiques of drafts of his critique of the APA Brief on Lesbian and Gay Parenting, which led him
to be repeatedly humbled leading to a wider scope of vision (p. 784). Blankenhorn and
Doherty have both advocated for strengthening all marriages regardless of sexual orientation.
Daniel Kahneman, who is the only psychologist known to have won a Nobel prize, claimed that
one key to his success was adversarial collaboration (Kahneman, 2003). Many of his
accomplishments emerged from constructive collaborations with adversaries who disagreed with
his initial research interpretations. Such collaborations are admittedly easier to achieve on less
controversial issues, but may be even more important for controversial issues that matter for
children. There are many forces that are increasing the polarization of the modern world on most
important issues. Civil adversarial collaboration has the potential of representing a much-needed
example of using our diversity to bring out the best in all of us rather than using our value
diversity to produce polarized extremes trying to shout each other down.

References
Blankenhorn, D. (2012, June 22). How my view on gay marriage changed. New York Times.
http://www.nytimes.com/2012/06/23/opinion/how-my-view-on-gay-marriage-changed.html?_r=1&
Cummings, N. A., & O'Donohue, W. T. (2008). Eleven blunders that cripple psychotherapy in America. New York:
Routledge.
Doherty, W. (2013, July 27). Marriage comes out of the closet. Minneapolis StarTribune.
http://www.startribune.com/opinion/commentaries/217173411.html
Galliher, J. F., Brekhus, W. H., & Keys, D. P. (2004). Laud Humphreys: Prophet of homosexuality and sociology.
Madison: University of Wisconsin Press.
Kahneman, D. (2003). Experiences of collaborative research. American Psychologist, 58, 723-730. doi:
10.1037/0003-066X.58.9.723
Larzelere, R. E., & Klein, D. M. (1987). Methodology. In M. B. Sussman & S. K. Steinmetz (Eds.), Handbook of
marriage and the family (pp. 125-155). New York: Plenum Press.
Larzelere, R. E., & Skeen, J. H. (1984). The method of multiple hypotheses: A neglected research strategy in family
studies. Journal of Family Issues, 5, 474-492. doi: 10.1177/019251384005004004
Marks, L. (2012). "We see what we seek": A rejoinder to the responses of Amato, Eggebeen, and Osborne. Social
Science Research, 41, 784-785.
Myers, D. G. (1983). Social psychology. New York: McGraw-Hill.
Regnerus, M. (2012). How different are the adult children of parents who have same-sex relationships? Findings
from the New Family Structures Study. Social Science Research, 41, 752-770.
Ross, M., & Fletcher, G. J. O. (1985). Attribution and social perception. In G. Lindzey & E. Aronson (Eds.),
Handbook of social psychology (3rd ed., Vol. 2, pp. 73-122). New York: Random House.
Schumm, W. R. (2012). Methodological decisions and the evaluation of possible effects of different family
structures on children: The new family structures survey (NFSS). Social Science Research, 41(6), 13571366. doi: 10.1016/j.ssresearch.2012.08.011
Wainright, J. L., & Patterson, C. J. (2008). Peer relations among adolescents with female same-sex parents.
Developmental Psychology, 44(1), 117-126. doi: 10.1037/0012-1649.44.1.117