The Counseling Psychologist

The Counseling Psychologist
http://tcp.sagepub.com Use of Theory-Driven Research in Counseling: Investigating Three Counseling Psychology Journals From 1990 to 1999
Carolyn A. Karr and Lisa M. Larson The Counseling Psychologist 2005; 33; 299 DOI: 10.1177/0011000004272257 The online version of this article can be found at: http://tcp.sagepub.com/cgi/content/abstract/33/3/299
Published by:
http://www.sagepublications.com
On behalf of:
Division of Counseling Psychology of the American Psychological Association
Additional services and information for The Counseling Psychologist can be found at: Email Alerts: http://tcp.sagepub.com/cgi/alerts Subscriptions: http://tcp.sagepub.com/subscriptions Reprints: http://www.sagepub.com/journalsReprints.nav Permissions: http://www.sagepub.com/journalsPermissions.nav Citations http://tcp.sagepub.com/cgi/content/refs/33/3/299
Downloaded from http://tcp.sagepub.com at KANSAS STATE UNIV LIBRARIES on May 14, 2009
THE COUNSELING 10.1177/0011000004272257 Karr, Larson / USE OF PSYCHOLOGIST THEORY-DRIVEN / May RESEARCH 2005
Use of Theory-Driven Research in Counseling:

Investigating Three Counseling Psychology Journals From 1990 to 1999
University of MissouriKansas City CCS Assessment Center Iowa State University
Three major journals in counseling psychology were sampled from 1990 to 1999 to assess the percentage of quantitative, empirical articles that were theory driven. Only 43% of the studies utilized a theory or model, and 57% predicted the relation between the variables, with few studies specifying the strength of the relation. Studies sampled in the Journal of Counseling Psychology (63%) and the Journal of Vocational Behavior (65%) reported a significantly higher percentage of theory-driven research than the Journal of Counseling and Development (43%). A higher proportion of Journal of Counseling Psychology studies compared with Journal of Counseling and Development studies anchored findings to theory. Few studies replicated previous studies. This paucity implies that replication is underutilized by the field as a potentially fruitful way to substantiate empirical knowledge. A binary logistic regression was conducted to examine trends over time. No trends emerged. Suggestions for future implementation of theory and implications for counseling psychology as a field are offered.
Carolyn A. Karr Lisa M. Larson
Counseling psychologists have alerted the field that too little theorydriven research is being generated (Harmon, 1982; Heppner, Kivlighan, & Wampold, 1999; Strong, 1984, 1991a, 1991b; Tracey, 1991; Tracey & Glidden-Tracey, 1999). An excellent example of theory-driven research has been the test of John Hollands hexagon of six interests (e.g., Fouad, Harmon, & Borgen, 1997). Recent studies in multicultural (e.g., Prosser, 2003) and vocational psychology (e.g., Blanchard & Lichtenberg, 2003) indirectly suggest a greater need for theory-driven research in these areas as well. A recent review of the process and outcome literature for test interpretation found that only 9% of studies were based on theory (Hanson & Goodyear, 2000). This observation is not limited to counseling psychology, however. Meehl (1978, 1990, 1993) voiced similar criticisms about psychology in general, and social psychology has been concerned with the related areas of meaningful hypothesis construction (e.g., Wallach & Wallach, 1994) and
The authors would like to thank Doug Bonett for his suggestions and guidance with the statistical analyses and Jennifer Swaim for her aid in data compilation.
THE COUNSELING PSYCHOLOGIST, Vol. 33 No. 3, May 2005 299-326 DOI: 10.1177/0011000004272257 2005 by the Society of Counseling Psychology
299
300 THE COUNSELING PSYCHOLOGIST / May 2005
replication of studies (e.g., Neuliep & Crandall, 1991). Their concerns are noteworthy given the usefulness of theory-driven research and the paucity of empirical studies that test established theories. Theories and models are crucial for several reasons. Theories are meant to be organized explications of the dynamics that underlie a given psychological phenomenon (Strong, 1991a). They are fundamental tools to understand the construct being researched. Meehl (1990) articulated a more specific function of theories in empirical studies by advocating that theories should be used to help determine whether the substantive theory, the statistical hypotheses, and the empirical observations are related in terms of logical inference and theoretical causality. The definition of theory used for this study was a general principle formulated to explain a group of related phenomena (Chaplin, 1985, p. 467). For the purposes of this study, a model was construed as a description of the assumed structure of a set of observations (Everitt & Wykes, 1999, p. 119). Although similar, the former utilizes a general tenet to explain related interactions, while the latter describes the expected observable interactions in more detail. By definition, theories and models are similar in function and scope. Forster (2000) stated that the best way to distinguish theories and models is to discuss each in conjunction with predictive hypotheses. In his conceptualization, the three are hierarchically arranged, with theories at the most general level, models applied to concrete systems in the middle, and predictive hypotheses at the lowest level, which result from fitting models to data (Forster, 2000, p. 233). He emphasized that the essential point of this tripartite distinction is that predictive accuracy is a property of predictive hypotheses at the very bottom of the hierarchy, and is traded-off against the truth at the next level upthe level of models (Forster, 2000, p. 233). In this way, both theories1 and models are tested by the utilization of tailored predictive hypotheses. Theory-driven research also provides a conceptual framework that minimizes the chances that researchers efforts will lead to the generation of a multitude of unconnected facts (Strong, 1991a). Strong (1991b) explained the furthering of scientific thought necessarily involves a circular process that begins with theory, carries this theory to the context of testing, is held to specific observations, and is interpreted through the context of discovery. Discovery then follows into further redefinition and redescription of theory. This grounded, circular process ensures that utility is blended with conceptualization. Moreover, use of theory easily guarantees that a meaningful question is asked because of the network of supplementary and predictive information that is part of the theory (Meehl, 1978). Kazdin (1992) noted that much theory-free research is considered meaningful by using the rationale that a
Karr, Larson / USE OF THEORY-DRIVEN RESEARCH 301
particular study has not yet been conducted. He emphasized, however, that literally thousands of variables could be researched together. Conglomerations of variables without links to theoretical networks suggesting the specific and significant reasons why they are chosen are essentially meaningless. The specificity of the hypotheses is also important in conducting theorydriven research. In particular, research questions should be asked in terms of whether the data will fall within a specific meaningful range, based on theories that include components that are likewise relatively narrow and focused (Meehl, 1993). Instead, many hypotheses based on specific theories do not identify the strength of the relations among the theoretical constructs. Meehl (1978, 1990, 1993) has championed the benefits of hypotheses that predict whether experimental observations will fall within a certain, relatively narrow numeric range and whether the observations will be specified a priori. He explains the advantages of numeric hypothesizing over traditional significance testing; numeric hypothesizing allows researchers to ask, does the theory have sufficient verisimilitude to warrant our continuing to test it and amend it rather than, is the theory literally true? (Meehl, 1990). He defines verisimilitude as the relationship between the theory and the real world which the theory speaks about (Meehl, 1990, p. 113). Wampold, Davis, and Good (1990) have further specified the need for hypothesis validity, which is the extent to which research results reflect theoretically derived predictions about the relations between or among constructs (p. 360). In this manner, hypothesis validity ensures that the relation between constructs is informed by theory. Therefore, by utilizing hypotheses that include precise numeric ranges (e.g., effect sizes, variability, and confidence intervals), we can better assess how expectations and resulting observations are linked, while simultaneously evaluating the current theory (Greenwald, Gonzalez, Harris, & Guthrie, 1996; Howard, Curtin, & Johnson, 1991; Levin & Robinson, 1999; Wampold et al., 1990). The magnitude and direction of the relation should be stated a priori because theory identifies more than the important constructs. Theory also should identify the posited strength of the relation among the constructs not the exact statistical value (e.g., .09) but rather small, medium, and large effects and positive and negative relations. This specification provides the template for hypotheses to be formulated. Examining the expected magnitude of the relation a priori, in comparison with the actual magnitude of the relation, contributes important information in determining how variables are related. For example, based on social cognitive theory, if we expect selfefficacy and outcome expectations to be moderately related (r values ranging from .3 to .5) and find instead that the relation across five studies is weaker (i.e., r = .15), then the theory must be adjusted.
Identifying discrepancies can further hone theoretical relations, contributing to advances in theory. By examining expectations of effect magnitude in theory and prior research, and then incorporating these expectancies into the studys hypotheses, potential extra variables or relations are more easily identified, which could lead to new discoveries. In practice, identifying the magnitude of relations a priori is much like the use of effect sizes (Cohen, 1969) in meta-analyses, in that effect size provides a quantified measure of the relation for comparison purposes. Allen (1995) stated that effect size indicates how strongly the independent variable influences the dependent variable, or the strength of the relationship (p. 82). Effect sizes, used in this way, provide a measure of the magnitude of a given relation and are generally more informative than hypotheses and analyses that simply aim to reject the null hypothesis (Cohen, 1994). Effect sizes are also used in replication of research, because strict null hypothesis testing does not include the probability that a result can be replicated (Gigerenzer & Murray, 1987; Lykken, 1968). Replication of research is important in the assessment and reformation of theory, using effect sizes as necessary vehicles to assess relations and further knowledge (Cohen, 1994). Prior research can be useful in determining numeric range estimates for hypothesized relations, as well as informing the direction of these relations. Some researchers question the viability of theory-driven research and even suggest that theory can thwart research progress under certain conditions. Greenwald, Pratkanis, Leippe, and Baumgardner (1986) argued that over-reliance on theory can cause confirmation bias when researchers persevere by revising procedures until obtaining a theory-predicted result. Researchers discovered this bias when they noticed poor preliminary data tended to cloud the interpretation of later data of better quality (Bruner & Potter, 1964; Wyatt & Campbell, 1951). Others have agreed that research is frequently developed by way of confirmation-biased practices (e.g., Popper, 1959). More specifically, changing conditions and procedures until the desired result is obtained is considered theory confirming (Greenwald et al., 1986) or prejudice against the null hypothesis (Greenwald, 1975), which strays from the ideal empirical methods of hypothesis testing and resulting theory disconfirmation. Remedies are offered in the form of utilizing condition-seeking and design approaches, which are used, respectively, to discover conditions on which an existing finding depends and to specify conditions that can produce an unobtainable result. Greenwald et al. (1986; refer for lengthier discussion) stated that the need for theory to guide empirical progress, along with the stimulus to theory that is provided by new findings, rescues the condition-seeking method from being just a means of cultivating empirical trivialities (p. 224). The final step
is then to revise, expand, and/or rearticulate theories to incorporate the new conditions. A related objection to theory-driven research is the contention that theorydriven research necessitates abandoning studies derived from empirical data. In actuality, empirical data can inform future studies and inform/revise theory. The authors are not suggesting that all research should be based on theory. Rather, most studies should be theory-driven, with initially atheoretical data used to specify and refine existing theories. Or new theories can be constructed to explain existing empirical and currently atheoretical data. By utilizing remedies for theory-driven research problems, we ensure the production of meaningful, theory-grounded data. The current study attempts to identify the current proportion of theoretical to atheoretical research. By identifying the theories currently utilized in the sample we will also launch an effort to determine the extent to which theories have been substantiated or further developed by new data. Still, despite these objections, theory-driven research may unify conducted research, especially among findings linked to similar, but different, fields within psychology (Staats, 1983). For example, many objectives of counseling psychology are similar to those of clinical psychology, social work, and school psychology. Yet findings and viewpoints have been largely linked within one field without integrating findings into a larger, unifying framework. Shared theoretical frameworks would serve to unify both psychological research and disciplines (Forsyth & Strong, 1992), benefiting not only the pursuit of science but also professional relationships and activities that can then contribute to a more common scientific ground. In short, the generation of theory-driven research is vital. The purpose of this study was to empirically examine during 10 years (1990 to 1999) the extent to which major counseling psychology journals have published theory-driven research. Specifically, the authors will address two questions. For the first question, the overall proportion of theory-driven research articles to quantitative research articles will be determined in a representative sample of counseling psychology journals. Although the profession has called for more theory-driven research, no studies have been conducted to examine the extent to which major journals in counseling psychology publish theory-driven research. Omer and Dar (1992) examined the research purposes posited in studies appearing in the Journal of Consulting and Clinical Psychology across the 1960s, 1970s, and 1980s. They found that the trend in clinical research has moved away from theoretical bases for research questions and has moved toward pragmatism. Two factors led to the selection of this particular time frame: (a) The aim of the study was to measure trends in utilization of theory for the 10 years
before the new century (2000) to assess how the field should conduct research in the current century, and (b) an active discussion regarding the optimal degree of theory utilization in counseling psychology surfaced just before the 1990s (e.g., Heppner & Claiborn, 1989; Serlin, 1987) and again in the early 1990s (Gelso, 1991; Patton & Jackson, 1991; Strong, 1991a, 1991b), with similar concerns voiced elsewhere in psychology during this time (e.g., Gergen, 1991). Potential trends and/or changes in theory utilization in counseling psychology during the 1990s could then at least be partially attributed to discussions held around this time. The field would benefit if future research addressed these concerns for the first 10 years of the current century (2000 to 2009) for the purpose of comparison. Given the value of theory-driven research, data must be collected to ascertain the extent to which researchers are embedding research questions in theory. For the second question, for those studies that appear to be theory driven, the authors will determine the extent to which the studies report a priori the direction and strength of the relations of certain variables in the theory. Meehl (1978) argued that if research is to inform the broader discipline of psychology, the parameters of the strength of the hypothesized relation must be specified a priori. METHOD Sample: The Studies The three most prominent and primarily quantitative journals in counseling psychology were sampled: Journal of Counseling Psychology (JCP), Journal of Vocational Behavior (JVB), and Journal of Counseling and Development (JCD). The Counseling Psychologist was excluded because its articles are more conceptual. These three journals were selected because they have traditionally been associated with empirical research. For example, when studies have examined the prominence of counseling psychology programs across the United States, the extent to which the faculty publish in these journals (including The Counseling Psychologist) is used to measure the productivity of the counseling psychology faculty. To obtain a representative sample with sufficient power, every seventh article was sampled. A power analysis was conducted to determine adequate sample size. The power analysis was conducted before data collection, and indicated sampling every seventh article would generate a sample size with sufficient statistical power. By utilizing Power and Precision for Windows (Biostat, 2001), an estimated 150 total studies analyzed via c 2 with seven criteria across three journals (effect size = .50; a = .05) yielded a power estimate
of .99. Actual sample size of the studies2 consisted of the following: JCP, n = 70; JVB, n = 63; and JCD, n = 60; N = 193. Because the total number of studies sampled was more than 150, power is considered to be of a sufficient level for this study. Procedures Many decision rules were employed during the selection and analysis of the sampled studies. The sampling began with the first volume in each journal that was published in 1990. If an article contained more than one study, all the studies in that article were used, resulting in a total of 193 studies in JCP, JVB, and JCD. If the seventh article was not data-based, the next article would be sampled. Thus, reviews and conceptual articles were skipped if they were the seventh article. Given that qualitative articles are generally used to build, rather than disconfirm, theory, they also were not sampled. Thus, if the seventh article was qualitative, the next article was sampled. Qualitative studies were excluded for three primary reasons. First, this study was a foray into measuring research trends in this manner in counseling psychology; qualitative studies were not included to keep the decision criteria uniform for the entire sample. Second, qualitative studies are based on the inductive, rather than deductive, approach. This study examined the quantitative research tradition in the field rather than the qualitative, which contains different assumptions. Third, research would benefit the assessment of theory utilization for qualitative studies but was beyond the scope of this study, partly because of the aforementioned factors. Development of the Content of the Questions The seven questions presented in Table 1 were derived from the literature calling for more theory-driven research. Questions 1 through 4 and Question 7 were derived from Tracey and others (Meehl, 1978; Tracey, 1991; Tracey & Glidden-Tracey, 1999), who argued that theory should be used at all stages, from conceptualization of questions and hypotheses to interpretation of results and the discussion of the findings within the field. Question 3 was based on Wallach and Wallach (1998), who argued that useful hypotheses should be specific. For example, Wampold et al. (1990) noted that the following stated purpose of a study is an ambiguous research hypothesis (one identified threat to hypothesis validity):
The present study attempted to determine a) the relation of parental adjustment measures of such variables as depression, marital satisfaction, parenting stress, and other negative life stressors to mothers and fathers perceptions of their
306 Percentages of Sampled Studies Coded Yes or No on the Questions and by Journal and Overall JCP n = 70 Yes 44.3 90.3 87.1 86.7 62.9 17.1 17.1 82.9 11.1 88.9 8.3 82.9 7.9 92.1 6.7 93.3 91.7 13.3 37.1 73.3 65.1 26.7 34.9 59.1 43.3 40.9 56.7 74.4 57.5 10.9 12.4 12.9 73.3 26.7 77.3 22.7 79.5 20.5 25.6 42.5 89.1 87.6 9.7 80.0 20.0 81.8 18.2 84.3 15.7 55.7 47.6 52.4 36.7 63.3 43.0 57.0 .15
a 2
TABLE 1: JVB n = 63 Yes No Yes No Yes No c

2
JCD n = 60 c
2
Overall N = 193
JCP/ JVB
JVB/ JCD
JCP/ JCD c 1.51

2
Question
No
.77 1.29 1.83 1.67 .07 2.52 .99 .03 .11 1.17 5.86* .07 .27 .81 .88 5.15* 4.96* 3.29 2.21
1. Is the study embedded in a theory/model? 2. If so, is the introduction organized around b this theory/model? 3. Of the studies that utilize a theory/model, are the hypotheses drawn directly from theory? 4. Of the studies that utilize a model/theory, are the findings anchored to that model/theory? 5. Is the relation between the variables predicted? 6. Is the predicted degree of the relation between c the variables significant? 7. Is the study a direct extension of one or more previous studies?
NOTE: JCP = Journal of Counseling Psychology; JCD = Journal of Counseling and Development; JVB = Journal of Vocational Behavior. a. Each c 2 equation included one degree of freedom. b. For Questions 2 through 4, a subset of the sample was used (JCP, n = 31; JVB, n = 30; JCD, n = 22). c. No studies in pilot data predicted the strength of the relations between the variables. *p < .05. **p < .01. ***p < .001.
childrens deviant behaviors; b) the relation of teachers independent perceptions of the childrens behaviors to mothers and fathers perceptions; c) the relation of mother, father, and teacher perceptions of child behaviors to observed mother, father, and child behaviors; and d) the relation of parent measures to observed mother, father, and child behaviors. (Webster-Stratton, 1988, pp. 909-910)
Wampold et al. (1990) explained that interpretation of results was difficult because an experimental relation and outcome, with which the results could be compared, were not specified in the hypotheses. For Question 3, the relations were also not informed by a specific theoretical conceptualization. Most critiques of hypothesis formulation and theory building have been theoretical (e.g., Meehl, 1990), with few known empirical studies (e.g., Wallach & Wallach, 1994) investigating specificity of hypotheses utilized in research. Questions 5 and 6 were composed from Meehl (1978), who advocated that the direction and strength of relations between the variables must be articulated within the hypotheses. Some criteria will be subjective because of the nature of the study. Nevertheless, as much objectivity as possible was included in the criteria based on the previously and subsequently named writings in which these factors were concluded to be important for empirical investigation. In the Webster-Stratton (1988) example of an ambiguous research hypothesis, neither the essence of the relation between variables (Question 5) nor the strength of the relation (Question 6) was specified in the stated research hypotheses. For example, one hypothesis might be reworded to meet the Question 5 criterion as follows: Depression will affect fathers perceptions of childrens deviant behaviors. Additionally, the hypothesis might be reworded to meet the Question 6 criterion as follows: Reported depressive symptoms by the father will result in significantly lower reported incidents of childrens deviant behaviors. Although neither hypothesis is ideal in wording or content, each meets the criteria used for this study. The hypotheses in the ambiguous research hypothesis example meet neither Question 5 nor Question 6 criteria. Question 7 identifies several reasons for the importance of replicating research. Despite heavy attempts at persuasion in counseling psychology and areas such as social psychology (Campbell & Jackson, 1979; Kazdin, 1982; Sidman, 1960; Smith, 1970; Sommer & Sommer, 1983), the social sciences have ignored the importance of replicated significant findings by failing to publish them (Mahoney, 1985). Lamal (1991) indicated that replicated studies are crucial for theory disconfirmation because of their epistemological import, providing observations that can either agree or disagree with the theory in question. He also notes that the field is more confident in findings that are reliable. Every study need not be replicated, and Lamal (1991) suggested
that studies with important hypotheses or predictions of the range, position, or multiple values should be prioritized. Finally, a reasonably accurate assessment of existing replication research is needed because what little replication research has been published has typically used a significance level as the primary summary statistic, while it has been suggested that effect size (discussed earlier) is actually a more effective summary statistic for this purpose (Rosenthal, 1991). Decision Rules for Answering the Questions This study is the first venture into empirically assessing the use of theorydriven research in counseling psychology. Consequently, both theories and models were included in the decision criteria to expand the breadth of the investigation. In the identification of theory and model, each generally needed to be named and to have been established in the literature by at least one citation to be included as a theory-driven research article, after which the theory/model was subjected to the studys major decision rules. For clarity, it was identified as a theory/model if it was named and described with reasonable detail in the article. Mere definitions of psychological phenomena were not included. For example, the term stress has been defined as The physiological and psychological response to a condition that threatens or challenges a person and requires some form of adaptation or adjustment (Wood & Wood, 1993, p. G-16). Alternatively, Lazaruss model of stress and coping describes how primary appraisal and secondary appraisal yield the stress response (Lazarus, 1966; Lazarus & Folkman, 1984). Therefore, the former would not be considered a theory/model, while the latter would be considered a theory/model. Finally, the utilized theory had to include descriptions of at least two relations between variables outlined in that theory and had to give rationale regarding the prediction of consequences from the given theoretical antecedents. If the study was based on two theories that were substantively different from each other, the study was considered to be based on two different theories. For example, social cognitive career theory (Lent, Brown, & Hackett, 1994) and social cognitive theory (Bandura, 1977, 1986) are considered in this study to be the same essential theory, because they are both based on a shared root theory. A study was based on more than one theory or model when each was named and described, and both were relatively equal contributors to most hypotheses. In addition, the following decision criteria were employed in determining an affirmative (yes) response and a negative (no) response for each of seven questions listed in Table 1.
1. Is the study embedded in a theory/model? 2. If so, is the introduction organized around this theory/model? 3. Of the studies that utilize a theory/model, are the hypotheses drawn directly from theory? 4. Of the studies that utilize a model/theory, are the findings anchored to that model/theory? 5. Is the relation between the variables predicted? 6. Is the predicted degree of the relation between the variables significant? 7. Is the study a direct extension of one or more previous studies?
Question 1 was coded as yes if a theory or model was explained as part of the rationale in the introduction. The criterion also stipulates that the introduction should be organized around that particular theory. To measure this qualification, the raters were instructed to consider whether a theory was substantially implemented or cursorily mentioned (e.g., mentioned once or twice in the introduction). If organized around a theory, the theorys stated tenets included in the introduction seemed to be leading to logically deduced hypotheses. Additionally, the theory was considered to be clearly identified if the theory was named and described briefly, especially in terms of how the theorys dynamics or processes relate to the studys objectives. A descriptive model was included if it was named and described briefly in the introduction (e.g., especially relevant processes, antecedents, and outcomes) and was also related to most of the hypotheses. In this way, models and theories were conceptualized in a similar manner. Questions 2 through 4 depend on Question 1 being answered affirmatively. In addition, Question 2 was coded as yes if the introduction was organized around that theory or model. If Question 1 was answered affirmatively, Question 3 was coded as yes if the hypotheses were worded so they clearly identified the stated theory or model and if the hypotheses were specific statements delineating the expected results. This latter stipulation included instances in which the theory-driven hypotheses were presented such that a specific pattern of results would falsify the theory. If more than one hypothesis was present and most hypotheses were based on a theory or model, then Question 3 was coded as yes. Question 4 concerned whether the findings were anchored to the theory or model. It was answered affirmatively if Question 1 was answered yes and if the results were interpreted in terms of the theory or model articulated in the introduction. Question 5 was coded as yes if the relations between the hypothesized variables were predicted within the stated hypotheses. Question 6 originally addressed the strength of the relations. Given that pilot data (i.e., a brief review of about 20 studies) yielded no studies in which expected specific parameters of the relation between variables were stated, the question was
altered to be more lenient. Namely, the authors had predicted a significant relation rather than specifying the strength or direction of the relation. For this question to be answered affirmatively, the author(s) had to specify in the hypotheses that significant relations were expected. Questions 5 and 6 were included because theory more than identifies the important constructs. Theory should also identify the posited strength of the relation among the constructsnot specifically (e.g., .09) but rather small, medium, and large effects as well as positive and negative relations. Theoretically based (and also, in part, empirically based) specifications should provide the template for hypotheses to be formulated. Furthermore, examining the expected magnitude of the relation a priori, in comparison with the actual magnitude of the relation, is another important factor in determining how variables are theoretically and empirically related. Identifying these factors can further hone theoretical relations, contributing to advances in theory. By examining expectations of effect magnitude in theory and prior research, and then incorporating expectancies into a studys hypotheses, potential extra variables are more easily identified, which could lead to discoveries. In practice, identifying the magnitude of relations a priori is much like the use of effect sizes (discussed earlier) in analyzing the magnitude of meta-analyses. As shown in Table 1, the researchers were examining whether the study was an extension or replication of a previous study in Question 7. Question 7 was coded as yes if the study was conducted as a replication of a previous study or if the authors indicated that most of the present study was conducted in the same manner as a previous study, perhaps with one extra manipulation or condition for the current study. Additional Questions Additional anecdotal information was obtained from the studies, including the name of the model or theory used, study year, and the number of hypotheses. This information was documented to conduct post hoc analyses in related areas. For instance, the names of the theories were collected to quantify and analyze theories represented in different topic areas in psychology. The study year was utilized to determine potential trends over time. Finally, the number of hypotheses was collected for post hoc analysis as well but was not utilized in the current studys final analyses. Training the Coders Two coders, who were graduate students in a counseling psychology doctoral program, were trained to score the seven questions as yes or no accord-
ing to the criteria outlined above and to code any additional anecdotal information. If the raters were unfamiliar with a theory, its citation was checked using PsychLit and briefly researched by reading other articles that were also based on that model. For training purposes several extra articles were coded but were not included in the sample. Training also included review of a wide range of examples of theories/models that are well-established in the field as well as those that are not yet empirically substantiated. Review of theory included reading articles explaining 8 to 10 different theories and discussing how the relations within each theory have been, and could be, investigated empirically. Discussion within training sessions included how variables could be incorporated and specified in hypotheses, especially with reference to the criteria used in this study. Coders were trained on each of the decision rule criteria until 100% agreement was reached (i.e., both raters agreed when asked to explain how the article met the specified criteria). Five randomly selected articles were coded by both raters as part of the training. During training, disagreements between coders typically involved misinterpretations of the content of theories versus content of supporting information conceptualizing the study in the introduction. Agreement, during training, was reached by reviewing discrepant viewpoints or interpretations of the psychological processes or phenomena in question and discussing each raters perceptions. After discussion, discrepant reasoning was compared with the definition of a theory and model, respectively, with engagement in further discussion until agreement was reached between the raters. They then proceeded to code the 193 studies, dividing the total number between the two of them. Of the sampled studies, every 10th article was coded by both raters to provide agreement statistics (n = 19). The absolute agreement formula used was the number of hits/number of instances, for the agreement subsample of articles, minus articles with missing data (n = 2). The total rater subsample consisted of 17 pairs of observations across seven categories of criteria. Analyses utilizing kappa indicated sufficient agreement between raters (k = .85) across the subsample of pairs of rater observations (n = 119) for the seven criteria. According to Morey and Agresti (1984), levels of agreement of greater than .75 are considered noteworthy. RESULTS Table 1 lists the percentage of studies that were coded yes or no on each of the seven questions, by journal and overall, plus the pairwise c 2 comparison by journal. Results are presented first by addressing the extent to which the studies were theory driven followed by the extent to which the authors of the studies posited the relation between variables and the extent to which the
studies were based on replication. Finally, a comparison of the journals and trends over time are presented. Theory-Driven Nature of the Studies As seen in Table 1, the percentage of studies that were embedded in a model or theory totaled 43%. In other words, most studies were not theory driven. Although 43% of studies were theory driven, and this constitutes a promising effort in utilization of theory, we argue that at least the overwhelming majority of quantitative research should be substantially linked to theory. Moreover, those studies that were theory driven spanned 45 different theories, with 77% of the theories being empirically examined in single studies. Finally, few theory-driven studies specified the magnitude of the relation among constructs. Magnitude of the relation was not counted in the criteria assessing which studies were theory driven; if it had been counted, 43% would have decreased to a much smaller percentage. Of those studies that were considered to be theory driven, most of the introduction sections were organized around a theory or model (Question 2; 84.3%) and specified theory-driven hypotheses (Question 3; 79.5%). Moreover, most theory-driven articles anchored their findings to the theory in the discussion (Question 4; 74.4%). Relation Between Variables Most of the 193 studies predicted the relation between variables (Question 5; 57.5%), as can be seen in Table 1. The remaining studies failed to specify the predicted relations among the variables. An example of failing to specify the predicted relation would occur when a hypothesis indicated that x is expected to be related to y, such as stress is expected to be related to amount of sleep; in this case no direction or magnitude is given. Studies in which the relation was predicted as significant comprised only 10.9% of the entire sample (Question 6). The remainder of the studies (which were theory driven, with the relation specified) failed to specify that the predicted relations would be significant (89.1%). Replication Across Studies A small minority of the 193 studies were direct extensions or replications of previous studies (Question 7; 12.4%), as seen in Table 1.
Post Hoc Comparison Across Journals Pairwise c 2 analyses were conducted (post hoc) between journals (see Table 1). Pairwise comparisons revealed only two questions that were different by journal. Significant differences were found between journals in stating predicted relations in the hypotheses. On Question 5, the pairwise comparisons of JCP and JCD as well as JVB and JCD yielded similar results (see Table 1). Both JCP (62.9%) and JVB (65.1%) had a significantly higher proportion of their studies that had predicted hypothesized relations between the variables compared with studies in JCD (43.3%; p < .03). A significant difference was also found between journals on studies that grounded their results in theory (Question 4). A significantly higher proportion of JCP studies (86.7%) compared with JCD studies (59.1%) anchored their findings in theory. Post Hoc Trends Across Time Table 2 presents the responses to the questions by year. To determine any trends across time, seven binary logistic regressions (SPSS 8.0 for Windows) were conducted in which the criterion variables were the yes/no responses to each of the seven questions in Table 1. The predictor variables were year and year2. The year2 was entered to determine if significant nonlinear patterns existed across time. The results from the binary logistic regressions yielded null findings for all seven questions.3 In short, there appeared to be no significant patterns (e.g., increases or decreases) in the percentage of affirmative responses to the seven questions across time. Additional Post Hoc Analyses To determine which theories were examined empirically, a list was generated by conducting post hoc analyses, of the models or theories in the 83 studies that had identified a model. Because every seventh article was sampled, Table 3 gives a representative frequency count of all of the studies published in those journals. A total of 45 different models were utilized. The most frequently utilized model was Hollands RIASEC (realistic, investigative, artistic, social, enterprising, and conventional) model of career interests (Holland, 1959). The next most frequently utilized theory was social cognitive theory (Bandura, 1977), followed by the racial identity model (Helms, 1984) and the developmental theory of occupational aspirations (Gottfredson, 1981). As can be seen by Table 3, four other models were each examined twice, leaving the remaining 35 models to be examined only once. Compari-
314 Percentages of Sampled Studies Coded Yes or No on the Questions by Year

1990 n 193 83 100.0 0.0 87.5 12.5 90.9 9.1 50.0 50.0 100.0 0.0 87.5 12.5 86.7 13.3 20.8 79.2 42.1 57.9 47.8 52.2 40.0 60.0 33.3 66.7 42.1 57.9 62.5 37.5 61.1 90.9 Yes No Yes No Yes No Yes No Yes No Yes No Yes No Yes No 38.9 9.1 1991 1992 1993 1994 1995 1996 1997 1998 Yes 33.3 66.7 No 66.7 33.3 1999 Yes 50.0 80.0 No 50.0 20.0 83 100.0 0.0 62.5 37.5 90.9 9.1 87.5 12.5 83.3 16.7 87.5 12.5 53.3 46.7 90.9 9.1 66.7 33.3 100.0 0.0 83 193 193 193 16.7 83.3 15.8 84.2 17.4 82.6 15.0 85.0 16.7 83.3 5.3 94.7 13.0 87.0 5.0 95.0 11.1 5.6 45.8 54.2 63.2 36.8 73.9 26.1 70.0 30.0 44.4 55.6 88.9 94.4 60.0 40.0 62.5 37.5 80.0 20.0 50.0 50.0 66.7 33.3 87.5 52.6 5.3 10.5 12.5 47.4 94.7 89.5 80.0 45.8 4.2 16.7 20.0 54.2 95.8 83.3 90.9 72.2 22.2 9.1 27.8 77.8 0.0 100.0 83.3 61.1 16.7 16.7 16.7 38.9 83.3 83.3 60.0 40.0 10.0 40.0 60.0 90.0 0.0 100.0
TABLE 2:
Question
1. Is study embedded in a theory/model? 2. If so, is the introduction organized around this theory or model? 3. Of the studies that utilize a theory/model, are the hypotheses drawn directly from the theory? 4. Of the studies that utilize a model/theory, are the findings anchored to that model/theory? 5. Is the relation between the variables predicted? 6. Is the predicted degree of the relation between the variables significant? 7. Is the study a direct extension of one or more previous studies?
Karr, Larson / USE OF THEORY-DRIVEN RESEARCH 315 TABLE 3: Frequencies of Theories/Models Utilized Frequency
Theory/Model
1. RIASEC model of occupations (Holland, 1959) 17 2. Social cognitive theory (Bandura, 1977, 1986) 8 3. Career circumscription and occupational compromise (Gottfredson, 1981) 5 4. Racial identity model (Helms, 1990) 5 5. Model of treatment acceptability (Conoley, Conoley, Ivey, & Scheel, 1991) 3 6. Spherical model of interests (Tracey & Rounds, 1996) 3 7. Adaptive counseling and therapy (Howard, Nance, & Myers, 1986) 2 8. Black identity development model (Cross, 1970) 2 9. Locus of control (Rotter, 1954) 2 10. Transtheoretical model (Prochaska & DiClemente, 1982) 2 11. Adlerian-based personality priorities (Kefir, 1971) 1 12. Attachment theory (Bowlby, 1969) 1 13. Attribution theory (Weiner, 1974) 1 14. Cognitive theory of depression (Beck, 1967) 1 15. Diathesis-stress-hopelessness model of suicidal behavior (Schotte & Clum, 1982) 1 16. Elaboration likelihood model (Petty & Cacioppo, 1986) 1 17. Information processing model (Srull & Wyer, 1980) 1 18. Information processing of problem solving model (Heppner & Krauskopf, 1987) 1 19. Job-demands control model (Karasek, 1979) 1 20. Levels of program evaluation (Patton, 1978) 1 21. Meta-theory of developmental-contextual career development (Vondracek, 1995) 1 22. Model of attachment (Bartholomew & Horowitz, 1991) 1 23. Model of college womens career choices (Fassinger, 1985) 1 24. Model for conceptualizing group therapy (Yalom, 1970) 1 25. Model of dream interpretation (Hill, 1996) 1 26. Model of employee career decision (Callanan & Greenhaus, 1990) 1 27. Model of family therapy (Minuchin, 1974) 1 28. Model of psychoanalytic counseling (Patton, Meara, & Robbins, 1992) 1 29. Model of racial consciousness (Ponterotto, 1988) 1 30. Model of organizational commitment (Meyer, Bobocel, & Allen, 1991) 1 31. Model of stress and coping (Lazarus & Folkman, 1984) 1 32. Narcissistic injury model (Miller, 1981) 1 33. Process model (Hill & OGrady, 1985) 1 34. Psychology of self (Kohut, 1971) 1 35. Psychosocial theory of development (Erickson, 1980) 1 36. Rogerian theory (Rogers, 1951) 1 37. Root metaphor theory (Pepper, 1942) 1 38. Social exchange theory (Thibault & Kelley, 1959) 1 39. States of mind model (Schwartz & Garamoni, 1986) 1 40. Theory of advanced development (Josselson, 1980) 1 41. Theory of personality types (Millon, 1969) 1 42. Theory of reasoned action (Fishbein & Ajzen, 1975) 1 43. Theory of work adjustment (Dawis & Lofquist, 1984) 1 44. Work role developmental processes (Feij, Whitely, Peiro, & Tavis, 1995) 1 45. Work-family interface (Frone, Yardley, & Markel, 1997) 1 NOTE: Frequency count total is 84; one of the 83 studies was based on two theories. Data are based on a representative sample.
sons between applied journals and journals addressing theory to a greater extent were beyond the scope of the current study and are a promising area for future research. Also generated by post hoc analyses, Table 4 indicates categorization of theories by topic. This thematic look at the models sampled over the decade indicates little agreement of theories used in the field, with several areas lacking in types of theories utilized. The topical areas with the greatest number of theories examined were theories used primarily in therapy, assessment, or treatment programming (n = 17) followed by career development and industrial/organizational theories (n = 11). Among the topical areas with the fewest theories represented were applied cognitive theories (n = 2) and consulting or program evaluation theories (n = 1). Also, only three theories were represented by eight studies in the area of multicultural theories. Topics with the fewest utilized theories represent (a) areas needing research based on the theories named and (b) a greater number of new theories generated to explain pragmatic studies. DISCUSSION The purpose of this article was to examine during 10 years the extent to which the major journals in counseling psychology were adhering to theory in the generation of knowledge. Results indicate that less than half of the sampled empirical quantitative studies published in JCP, JCD, and JVB met that standard and imply that 57% of our quantitative empirical research lacks a theoretical, conceptual framework. If more stringent theoretical criteria regarding the magnitude of the relation were included in this initial determination, the percentage would have been higher. If Strong (1991b) is right that the furthering of scientific thought involves the circular process that begins with theory, then these findings are grave. Meehl (1978) and Kazdin (1992) also were concerned that atheoretical studies may not guarantee that meaningful questions are being asked. Although this study did not ascertain the meaningfulness of the research questions, the hope that almost 60% of our questions are so meaningful that they need no theory is disconcerting. The good news is that the overwhelming majority of the sampled studies (43%) derived from a model did organize the introductions around the specified model (84%), embed their hypotheses in the model (80%), and anchor their findings to the theory (75%). In fact, JCP anchored findings to theory significantly more than JCD (87% vs. 59%). These results are important because they suggest considerable quantitative empirical research is conducted according to many of the guidelines articulated by Strong (1991b), Meehl (1978, 1993), and Kazdin (1992).
Karr, Larson / USE OF THEORY-DRIVEN RESEARCH 317 TABLE 4: Theories/Models Utilized by Topic Area
Theories Used Primarily in Therapy, Assessment, or Treatment Programming (n = 17) Adaptive counseling and therapy (Howard, Nance, & Myers, 1986) Adlerian-based personality priorities (Kefir, 1971) Cognitive theory of depression (Beck, 1967) Diathesis-stress-hopelessness model of suicidal behavior (Schotte & Clum, 1982) Model for conceptualizing group therapy (Yalom, 1970) Model of dream interpretation (Hill, 1996) Model of family therapy (Minuchin, 1974) Model of psychoanalytic counseling (Patton, Meara, & Robbins, 1992) Model of stress and coping (Lazarus & Folkman, 1984) Model of treatment acceptability (Conoley, Conoley, Ivey, & Scheel, 1991) Narcissistic injury model (Miller, 1981) Process model (Hill & OGrady, 1985) Psychology of self (Kohut, 1971) Rogerian theory (Rogers, 1951) Root metaphor theory (Pepper, 1942) Theory of personality types (Millon, 1969) Transtheoretical model (Prochaska & DiClemente, 1982) Career Development and Industrial/Organizational Theories (n = 11) Career circumscription and occupational compromise (Gottfredson, 1981) Job-demands control model (Karasek, 1979) Meta-theory of developmental-contextual career development (Vondracek, 1995) Model of college womens career choices (Fassinger, 1985) Model of employee career decision (Callanan & Greenhaus, 1990) Model of organizational commitment (Meyer, Bobocel, & Allen, 1991) RIASEC model of occupations (Holland, 1959) Spherical model of interests (Tracey & Rounds, 1996) Theory of work adjustment (Dawis & Lofquist, 1984) Work role developmental processes (Feij, Whitely, Peiro, & Tavis, 1995) Work-family interface (Frone, Yardley, & Markel, 1997) Social Psychology Theories (n = 7) Attribution theory (Weiner, 1986) Elaboration likelihood model (Petty & Cacioppo, 1986) Information processing model (Srull & Wyer, 1980) Locus of control (Rotter, 1942) Social cognitive theory (Bandura, 1977, 1986) Social exchange theory (Thibault & Kelley, 1959) Theory of reasoned action (Fishbein & Ajzen, 1975) Developmental Theories (n = 4) Attachment theory (Bowlby, 1969) Model of attachment (Bartholomew & Horowitz, 1991) Psychosocial theory of development (Erickson, 1980) Theory of advanced development (Josselson, 1980) Multicultural Theories (n = 3) Black identity development model (Cross, 1970) Model of racial consciousness (Ponterotto, 1988) Racial identity model (Helms, 1984) (continued)
318 THE COUNSELING PSYCHOLOGIST / May 2005 TABLE 4 (continued) Applied Cognitive Theories (n = 2) Information processing of problem-solving model (Heppner & Krauskopf, 1987) States of mind model (Schwartz & Garamoni, 1986) Consulting or Program Evaluation Theories (n = 1) Levels of program evaluation (Patton, 1978) NOTE: Data are based on a representative sample.
The promising news garnered from this study is that almost 60% of the studies predicted the relations among the variables studied. Moreover, for JCP and JVB, the percentage was significantly higher than for JCD (63% and 65% vs. 43%). In their research design text, Heppner and colleagues (1999) encouraged students to design studies in which important relations are examined. It seems that about 60% of our studies are identifying relations between variables as Heppner and colleagues suggested. Although a less stringent standard than specification of a model, the implication may be that researchers are examining pockets of unspecified models when identifying relations. Perhaps they are forming the foundation for theory or models to follow. Yet without further theoretical articulation of these models, based in part on findings of actual versus expected relation strength, furthering psychological knowledge as a whole is neglected. As expected, few, if any, studies stated the strength of those anticipated relations among the variables. In fact, based on pilot data, the question was coded leniently to state that the predicted relations would be significant, rather than using the more preferable option of stating the anticipated strength based on prior research. Even then, the finding that only 11% of the sampled studies met that criterion was disappointing. Meehl (1978, 1993) has argued persuasively for many years that relations among variables must be specified as falling within a specific range. Without the specification, model building and contingency specification are hampered. One disappointing finding was the low percentage of studies (12%) that were replications or extensions of prior studies. Increasingly, meta-analysis is being used in the field. In meta-analysis the findings of multiple studies are examined statistically to determine the relations among variables across different samples and conditions. Meta-analyses rely on sufficient extension and replications of studies. The findings of this study imply that extensions and replications are being published infrequently. Whether the studies are not being conducted or whether they are being conducted but are not being accepted for publication is unclear. Editors, as well as editorial boards, may not be reinforcing researchers in pursuing extensions or replications (Neuliep & Crandall, 1991). If true, this is unfortunate. Prior literature has advocated that studies should be replicated not only to ensure reliability of
findings and to fully utilize the empirical approach but to hold researchers to describing their studies so findings are significant, believable, and credible (Meehl, 1990). Practical concerns regarding limited journal space could be remedied with increased dialogue, creative solutions, and tenacity. For example, journals could devote special issues to replicated research. Entire journals, books, or book chapters could be created to disseminate these findings, all of which would form tools for enhanced theory articulation. Logistical roadblocks are not valid excuses for hampering scientific inquiry. We were surprised to find that upward trends did not appear in the data across the 10 years. Rhetoric may not have yet filtered into the planning and execution of enough studies, or a consensus may be lacking among researchers and/or editors that theory-driven research is the standard for most quantitative research. Limitations of the study include that, to some extent, the decisions made by editors and reviewers are not necessarily based on empirical knowledge or contribution. For example, preferences for one theory, and availability of grant money in certain theoretical areas, may be shaping which theories are utilized. Educators may not emphasize use of theories in methodology courses, and as a result students may not be instructed that theory-driven research is beneficial. Additionally, new information generated from empirical research may not be used to create new theories, which also limits the extent to which future studies can be based on theory. Further limitations could be tied to sample specific journal selection and the particular studies included in the sample. Finally, other limitations of the study include the possibility of rater bias and that the sample may not represent the entire population of empirical articles in counseling psychology published during this period. For that matter, the degree of theory utilization in published articles may not be similar to the degree of theory utilization in rejected or otherwise nonpublished articles. Furthermore, the number of theories identified in this sampling reflect the nascent state of applied psychology as a whole and counseling psychology as a specialty. Hollands RIASEC model and Banduras social cognitive theory may be the most promising theories for most researchers. However, most identified models were used less than three times in this sample. The proliferation of so many theories that were examined infrequently across a 10-year span may suggest there is little consensus among researchers regarding which theories to advance. As might be expected, the topical areas with the greatest representation were therapeutic/assessment theories and career theories. Yet too few theories are represented in more diverse areas of counseling psychology research. Areas of applied cognitive theory, consultation and program evaluation, and multicultural counseling evidenced the fewest representative theories, while
each could have direct and/or indirect effects on the fields therapeutic activities. The authors suggest greater utilization of available theories, as well as the generation of new theories, to explicate new empirical data. Although the findings support a general diversity of studied topics, one result is a limited number of substantiated theories from which researchers may choose. If theory revision is critical for generating new knowledge in the field, then these results may encourage researchers to reconsider their direction and consider more convergence and less divergence in their grounding of future empirical work. One may wonder why the current study, ironically, is not based on a theory or model. Unfortunately, a substantiated theory is not developed when generating novel methodological hypotheses and approaches. The authors are reasonably certain, after an extensive literature search, that only one likeminded, comprehensive study exists (Omer & Dar, 1992) to date. One may also erroneously presume that the authors are arguing against the production of applied atheoretical research. While applied studies are important, as is the case with the current study, a greater and informed balance must be determined to accelerate the production of theory-driven research. Applied research tends to be conceived without an articulated theory generated for that particular topic, even after several applied studies have been completed in that area (e.g., test feedback). The current study attempts to organize the existing efforts toward theory-driven research to promote comprehensive efforts toward this balance. We call on the field to begin a debate to determine the empirical nature of this optimal balance between empirical and theoretical pursuits, striving for a more integrated approach that utilizes the most useful properties of each endeavor. A Call to Action Counseling psychology researchers, professors, editors, and editorial board members may want to deliberate over the findings of this study. The rhetoric has been clear. The generation of new knowledge in counseling psychology is best served by most quantitative empirical research being comprehensively embedded in theories or models. An adequate, but insufficient, number of the studies are already anchored in theory using rudimentary inclusion criteria. Kazdin (1999) argued that theory is the key to isolate the ways in which the cause exerts influence and how change processes are upheld. Kazdin related that if we can know why it works, then we can know how to intervene and construct tailored, research-based, clinical strategies. Counseling psychology can utilize this suggestion by basing more research on theories that have already been identified and tested in related disciplines, such as social psychology (Strong, Welsh, Corcoran, & Hoyt, 1992). More-
over, professors who teach research design may want to intentionally emphasize the value of theory-driven research (e.g., Larson & Besett-Alesch, 2000). Finally, the authors call on researchers to utilize theory by learning and using the many theories available to the field; specify the direction, magnitude, and significance of the hypothesized relations; and to create new theories on which to base their empirically driven hypotheses and studies. Theory-driven research would hone empirical efforts and would benefit the psychotherapeutic activities within counseling psychology. Utilizing theoretically based counseling interventions eases the work of the practitioner and allows for a flexible structure of interventions. Because theories explain the characteristics and progression of psychological phenomena, as well as appropriate therapeutic responses to phenomena, they provide the practitioner with tools for difficult cases. An excellent example of an empirically validated theoretical framework is dialectical behavior therapy (Linehan, 1991; Shearin & Linehan, 1994). Furthermore, theoretical frameworks can help psychologists who conduct assessments recommend helpful treatments by detailing a more precise therapeutic course. In this way, utilization of theoretical frameworks in therapy may help psychotherapy research become more relevant to practitioners. The gap between psychotherapeutic research and practice has been recognized (e.g., Smith & Grawe, 2003), and some have suggested that this gap exists because psychotherapy research does not answer the questions of the practitioner (Tracey, 1991). Practitioners would likely appreciate empirically validated theories that offer practice options based on progress in therapy, providing empirically validated and theoretically sound contingency plans for the difficult therapy case. Finally, third-party funding sources usually require that empirically validated interventions are utilized and explicated in therapy progress notes. The framework provided by empirically validated theoretical models informs the course of treatment and can provide justification for further needed financial, medical, or social service resources in some cases. In conclusion, Tracey and Glidden-Tracey (1999) proposed that an iterative model of reasoned research should be used when conducting all stages of an empirical study. This model informed the current study in a general sense and should be used as a guideline to direct future research. They listed four components to their model: theory, research design, measurement, and analysis. Each of these components is then dynamically and coherently linked and concomitantly clarified by how the inherent assumptions and logic inform each component. Furthermore, they suggested how the research design should be analyzed to articulate (Abelson, 1995) the studys reasoned argument. Without the orchestrated utilization of these four components, they argued that often the wrong research questions are asked, that there is
poor assumptive specification, and that integration of the components of the conducted research is lacking (Tracey & Glidden-Tracey, 1999). The iterative model should be posited for establishing future studies as the field evaluates where research in the 1990s has fallen in relation to the model. Closely adhering to iterative model guidelines may enhance the viability of research, providing needed integration of theoretical and empirical pursuits. NOTES
1. Although the terms theory and model are not synonyms, in this article when either term is used, theories or models is meant. 2. A list of references for the studies included in the current sample is available from the first or second author. 3. The seven binary logistic regression results are available from the second author.
REFERENCES
Abelson, R. P. (1995). Statistics as principled argument. Hillsdale, NJ: Lawrence Erlbaum. Allen, M. J. (1995). Introduction to psychological research. Itasca, IL: F. E. Peacock. Bandura, A. (1977). Self-efficacy: Toward a unifying theory of behavioral change. Psychological Review, 84, 191-215. Bandura, A. (1986). Social foundations of thought and action: A social cognitive theory. Englewood Cliffs, NJ: Prentice Hall. Bartholomew, K., & Horowitz, L. M. (1991). Attachment styles among young adults: A test of a four-category model. Journal of Personality and Social Psychology, 61, 226-244. Beck, A. T. (1967). Depression: Causes and treatment. Philadelphia: University of Pennsylvania Press. Blanchard, C. A., & Lichtenberg, J. W. (2003). Compromise in career decision making: A test of Gottfredsons theory. Journal of Vocational Behavior, 62, 250-271. Bowlby, J. (1969). Attachment and loss (Vol. 1). New York: Basic Books. Bruner, J. S., & Potter, M. C. (1964). Interference in visual recognition. Science, 144, 424-425. Callanan, G. A., & Greenhaus, J. H. (1990). The career indecision of managers and professionals: Development of a scale and test of a model. Journal of Vocational Behavior, 37, 79-103. Campbell, K. E., & Jackson, T. T. (1979). The role of and need for replication research in social psychology. Replications in Social Psychology, 1, 3-14. Chaplin, J. P. (1985). The dictionary of psychology (2nd rev. ed.). New York: Dell. Cohen, J. (1969). Statistical power analysis for the behavioral sciences. New York: Academic Press. Cohen, J. (1994). The earth is round (p < .05). American Psychologist, 49, 997-1003. Conoley, C. W., Conoley, J. C., Ivey, D. C., & Scheel, M. J. (1991). Enhancing consultation by matching the consultees perspective. Journal of Counseling and Development, 69, 546-549. Cross, W. E., Jr. (1970, April). The Black experience viewed as a process: A crude model for Black self-actualization. Paper presented at the 34th annual meeting of the Association of Social and Behavioral Scientists, Tallahassee, FL.
Karr, Larson / USE OF THEORY-DRIVEN RESEARCH 323 Dawis, R. V., & Lofquist, L. H. (1984). A psychological theory of work adjustment: An individual differences model and its applications. Minneapolis, MN: University of Minneapolis Press. Erickson, E. H. (1980). Identity and the life cycle. New York: Norton. Everitt, B. S., & Wykes, T. (1999). A dictionary of statistics for psychologists. London: Arnold Publishers. Fassinger, R. E. (1985). A causal model of college womens career choice in two samples of college women. Journal of Vocational Behavior, 27, 123-153. Feij, J. A., Whitely, W. T., Peiro, J. M., & Tavis, T. W. (1995). The development of careerenhancing strategies and content innovation: A longitudinal study of new workers. Journal of Vocational Behavior, 46, 231-256. Fishbein, M., & Azjen, A. (1975). Beliefs, attitudes, intentions, and behavior: An introduction to theory and research. Reading, MA: Addison-Wesley. Forster, M. (2000). Hard problems in the philosophy of science: Idealisation and commensurability. In R. Nola & H. Sankey (Eds.), Australasian studies in history and philosophy of science: After Popper, Kuhn, and Feyerabend. Recent issues in theories of scientific method (Vol. 15, pp. 231-250). Dordrecht, Holland: Kluwer Academic. Forsyth, D. R., & Strong, S. R. (1992). The scientific study of counseling and psychotherapy: A unificationist view. In A. E. Kazdin (Ed.), Methodological issues and strategies in clinical research. Washington, DC: American Psychological Association. Fouad, N. A., Harmon, L., & Borgen, F. H. (1997). Structure of interests in employed male and female members of US racial-ethnic minority and nonminority groups. Journal of Counseling Psychology, 44, 339-345. Frone, M. R., Yardley, J. K., & Markel, K. S. (1997). Developing and testing an integrative model of the work-family interface. Journal of Vocational Behavior, 50, 145-167. Gelso, C. J. (1991). Galileo, Aristotle, and science in counseling psychology: To theorize or not to theorize. Journal of Counseling Psychology, 38, 211-213. Gergen, K. J. (1991). Emerging challenges for theory and psychology. Theory & Psychology, 1, 13-35. Gigerenzer, G., & Murray, D. J. (1987). Cognition as intuitive statistics. Hillsdale, NJ: Lawrence Erlbaum. Gottfredson, L. S. (1981). Circumscription and compromise: A developmental theory of occupational aspirations [Monograph]. Journal of Counseling Psychology, 28, 545-579. Greenwald, A. G. (1975). Consequences of prejudice against the null hypothesis. Psychological Bulletin, 82, 1-20. Greenwald, A. G., Gonzalez, R., Harris, R. G., & Guthrie, D. (1996). Effect sizes and p values: What should be reported and what should be replicated? Psychophysiology, 33, 175-183. Greenwald, A. G., Pratkanis, A. R., Leippe, M. R., & Baumgardner, M. H. (1986). Under what conditions does theory obstruct research progress? Psychological Review, 93, 216-229. Hanson, W. E., & Goodyear, R. K. (2000, August). Test interpretation research: A comprehensive review of process and outcome studies. In T. Vacha-Haase (Chair), Improving test interpretation and feedback: Guidelines for practical application. Symposium conducted at the annual meeting of the American Psychological Association, Washington, DC. Harmon, L. (1982). Scientific affairsthe next decade. The Counseling Psychologist, 10, 31-37. Harren, V. A. (1979). A model of career decision making for college students. Journal of Vocational Behavior, 14, 119-133. Helms, J. E. (1984). Toward a theoretical explanation of the effects of race on counseling: A Black and White model. The Counseling Psychologist, 12, 153-165. Heppner, P. P., & Claiborn, C. D. (1989). Social influence research in counseling: A review and critique. Journal of Counseling Psychology, 36, 365-387.
324 THE COUNSELING PSYCHOLOGIST / May 2005 Heppner, P. P., Kivlighan, D. M., & Wampold, B. E. (1999). Research design in counseling. Pacific Grove, CA: Brooks/Cole. Heppner, P. P., & Krauskopf, C. J. (1987). An information-processing approach to personal problem solving. The Counseling Psychologist, 15, 371-447. Hill, C. E. (1996). Working with dreams in psychotherapy. New York: Guilford. Hill, C. E., & OGrady, K. E. (1985). List of therapist intentions illustrated in a case study and with therapists of varying theoretical orientations. Journal of Counseling Psychology, 32, 3-22. Holland, J. (1959). A theory of vocational choice. Journal of Counseling Psychology, 6, 35-45. Howard, G. S., Curtin, T. D., & Johnson, A. J. (1991). Point estimation techniques in psychological research: Studies on the role of meaning in self-determined action. Journal of Counseling Psychology, 38, 219-226. Howard, G. S., Nance, D. W., & Myers, P. (1986). Adaptive counseling and therapy: An integrative eclectic model. The Counseling Psychologist, 11, 363-442. Josselson, R. (1980). Ego development in adolescence. In J. Adelson (Ed.), Handbook of adolescent psychology (pp. 188-210). New York: John Wiley. Karasek, R. A. (1979). Job demands, job decision latitude and mental strain: Implications for job redesign. Administrative Science Quarterly, 24, 285-308. Kazdin, A. E. (1982). Single-case research designs. New York: Oxford University Press. Kazdin, A. E. (1992). Research design in clinical psychology. Boston: Allyn & Bacon. Kazdin, A. E. (1999, August). (Lack of) status of theory in psychotherapy research. Invited address presented at the annual meeting of the American Psychological Association, Boston, MA. Kefir, N. (1971). Priorities: A different approach to lifestyle and neurosis. Paper presented at International Committee of Adlerian Summer Schools and Institutes, Tel-Aviv, Israel. Kohut, H. (1971). The analysis of the self. Madison, CT: International Universities Press. Lamal, P. A. (1991). On the importance of replication. In J. W. Neuliep (Ed.), Replication research in the social sciences (pp. 31-36). Newbury Park, CA: Sage. Larson, L. M., & Besett-Alesch, T. (2000). Bolstering the scientific component in the training of scientist-practitioners: One programs curriculum. The Counseling Psychologist, 28, 873896. Lazarus, R. S. (1966). Psychological stress and the coping process. New York: McGraw-Hill. Lazarus, R. S., & Folkman, S. (1984). Stress, appraisal and coping. New York: Springer. Lent, R. W., Brown, S. D., & Hackett, G. (1994). Toward a unified social cognitive theory of career/academic interest, choice, and performance [Monograph]. Journal of Vocational Behavior, 45, 79-122. Levin, J. R., & Robinson, D. H. (1999). Further reflections on hypothesis testing and editorial policy for primary research journals. Educational Psychology Review, 11, 143-155. Linehan, M. (1991). Asuicidal borderline patients. Archives of General Psychiatry, 48, 10601064. Lykken, D. T. (1968). Statistical significance in psychological research. Psychological Bulletin, 70, 151-159. Mahoney, M. J. (1985). Open exchange and epistemic progress. American Psychologist, 40, 29-39. Meehl, P. (1978). Theoretical risks and tabular asterisks: Sir Karl, Sir Ronald, and the slow progress of soft psychology. Journal of Consulting and Clinical Psychology, 46, 806-834. Meehl, P. (1990). Appraising and amending theories: The strategy of Lakatosian defense and two principles that warrant it. Psychological Inquiry, 1, 108-141. Meehl, P. (1993). Philosophy of science: Help or hindrance? Psychological Reports, 72, 707733.
Karr, Larson / USE OF THEORY-DRIVEN RESEARCH 325 Meyer, J. P., Bobocel, D. R., & Allen, N. J. (1991). Development of organizational commitment during the first year of employment: A longitudinal study of pre- and post-entry influences. Journal of Management, 17, 717-733. Miller, A. (1981). Prisoners of childhood: The drama of the gifted child and the search for the true self. New York: Basic Books. Millon, T. (1969). Modern psychopathology: A biosocial approach to maladaptive learning and functioning. Philadelphia: Saunders. Minuchin, S. (1974). Families and family therapy. Cambridge, MA: Harvard University Press. Morey, L., & Agresti, A. (1984). The measurement of classification agreement: An adjustment to the Rand statistic for chance agreement. Educational and Psychological Measurement, 44, 33-37. Neuliep, J. W., & Crandall, R. (1991). Editorial bias against replication research. In J. W. Neuliep (Ed.), Replication research in the social sciences (pp. 85-90). Newbury Park, CA: Sage. Omer, H., & Dar, R. (1992). Changing trends in three decades of psychotherapy research: The flight from theory into pragmatics. Journal of Consulting and Clinical Psychology, 60, 88-93. Patton, M. J., & Jackson, A. P. (1991). Theory and meaning in counseling research: Comment on Strong (1991). Journal of Counseling Psychology, 38, 214-216. Patton, M. J., Meara, N. M., & Robbins, S. B. (1992). Psychoanalytic counseling. Chichester, West Sussex, England: Wiley. Patton, M. Q. (1978). Utilizationfocused evaluation. Beverly Hills, CA: Sage. Pepper, S. C. (1942). World hypotheses: A study in evidence. Berkeley: University of California Press. Petty, R. E., & Cacioppo, J. T. (1986). The elaboration likelihood model of persuasion. Advances in Experimental Social Psychology, 19, 123205. Ponterotto, J. G. (1988). Racial consciousness development among White counselor trainees: A stage model. Journal of Multicultural Counseling and Development, 16, 146-156. Popper, K. R. (1959). The logic of scientific discovery. New York: Basic Books. Prochaska, J. O., & DiClemente, C. C. (1982). Transtheoretical therapy: Toward a more integrative model of change. Psychotherapy: Theory, Research & Practice, 19, 276-288. Prosser, E. C. (2003). The development and validation of a multicultural model of coping. Dissertation Abstracts International: Section B: The Sciences & Engineering, 64, (2-B), 973. Rogers, C. R. (1951). Client-centered therapy: Its current practice, implications, and theory. Boston: Houghton Mifflin. Rosenthal, R. (1991). Replication in behavioral research. In J. W. Neuliep (Ed.), Replication research in the social sciences (pp. 1-30). Newbury Park, CA: Sage. Rotter, J. D. (1954). Social learning theory and clinical psychology. New York: Prentice Hall. Schotte, D. E., & Clum, G. (1982). Suicide ideation in a college population: A test of a model. Journal of Consulting & Clinical Psychology, 46, 690-696. Schwartz, R. M., & Garamoni, G. L. (1986). Cognitive assessment: A multibehaviormultimethod-multiperspective approach. Journal of Psychotherapy and Behavioral Assessment, 8, 185-197. Serlin, R. C. (1987). Hypothesis testing, theory building, and the philosophy of science. Journal of Counseling Psychology, 34, 365-371. Shearin, E. N., & Linehan, M. M. (1994). Dialectical behavior therapy for borderline personality disorder: Theoretical and empirical foundations. Acta Psychiatrica Scandinavica, 89, 61-68. Sidman, M. (1960). Tactics of scientific research. New York: Basic Books. Smith, E. C., & Grawe, K. (2003). What makes psychotherapy sessions productive? A new approach to bridging the gap between process research and practice. Clinical Psychology & Psychotherapy, 10, 275-285.
326 THE COUNSELING PSYCHOLOGIST / May 2005 Smith, N. C. (1970). Replication studies: A neglected aspect of psychological research. American Psychologist, 25, 970-975. Sommer, R., & Sommer, B. A. (1983). Mystery in Milwaukee: Early intervention, IQ, and psychology textbooks. American Psychologist, 38, 982-985. Srull, T. K., & Wyer, R. S. (1980). Category accessibility and social perception: Some implications for the study of person memory and interpersonal judgments. Journal of Personality and Social Psychology, 38, 841-856. Staats, A. W. (1983). Psychologys crisis of disunity. New York: Praeger Special Studies. Strong, S. R. (1984). Reflections on human nature, science, and progress in counseling psychology. Journal of Counseling Psychology, 31, 470-473. Strong, S. R. (1991a). Theory-driven science and naive empiricism in counseling psychology. Journal of Counseling Psychology, 38, 204-210. Strong, S. R. (1991b). Science in counseling psychology: Reply to Gelso (1991) and Patton and Jackson (1991). Journal of Counseling Psychology, 38, 217-218. Strong, S. R., Welsh, J. A., Corcoran, J. L., & Hoyt, W. T. (1992). Social psychology and counseling psychology: The history, products, and promise of an interface. Journal of Counseling Psychology, 39, 139-157. Thibault, J. W., & Kelley, H. H. (1959). The social psychology of groups. New York: John Wiley. Tracey, T. J. (1991). Counseling research as an applied science. In C. E. Watkins & L. J. Schneider (Eds.), Research in counseling (pp. 3-31). Hillsdale, NJ: Lawrence Erlbaum. Tracey, T. J. G., & Glidden-Tracey, C. E. (1999). Integration of theory, research design, measurement, and analysis: Toward a reasoned argument. The Counseling Psychologist, 27, 299-324. Tracey, T. J. G., & Rounds, J. (1996). The spherical representation of vocational interests. Journal of Vocational Behavior, 48, 3-41. Vondracek, F. W. (1995). Vocational identity across the life-span: A developmental-contextual perspective on achieving self-realization through vocational careers. Man & Work, 6, 85-93. Wallach, L., & Wallach, M. A. (1994). Gergen versus the mainstream: Are hypotheses in social psychology subject to empirical test? Journal of Personality and Social Psychology, 67, 233-242. Wallach, L., & Wallach, M. A. (1998). When experiments serve little purpose: Misguided research in mainstream psychology. Theory & Psychology, 8, 183-194. Wampold, B. E., Davis, B., & Good, R. H., III. (1990). Hypothesis validity of clinical research. Journal of Consulting and Clinical Psychology, 58, 360-367. Webster-Stratton, C. (1988). Mothersand fathersperceptions of child deviance: Roles of parent and child behaviors and parent adjustment. Journal of Consulting and Clinical Psychology, 56, 909-915. Weiner, B. (Ed.). (1974). Achievement motivation and attribution theory. Norristown, NJ: General Learning Press. Weiner, B. (1986). Attribution, emotion, and action. In R. M. Sorrentino & E. T. Higgins (Eds.), Handbook of motivation and cognition: Foundations of social behavior (pp. 281-312). New York: Guilford. Wood, E. R. G., & Wood, S. E. (1993). The world of psychology. Boston: Allyn & Bacon. Wyatt, D. F., & Campbell, D. T. (1951). On the liability of stereotype or hypothesis. Journal of Abnormal and Social Psychology, 46, 496-500. Yalom, I. D. (1970). The theory and practice of group psychotherapy. New York: Basic Books.

The Counseling Psychologist

Hochgeladen von

Dokumentinformationen

Copyright

Verfügbare Formate

Dieses Dokument teilen

Dokument teilen oder einbetten

Freigabeoptionen

Stufen Sie dieses Dokument als nützlich ein?

Sind diese Inhalte unangemessen?

Copyright:

Verfügbare Formate

The Counseling Psychologist

Hochgeladen von

Copyright:

Verfügbare Formate

The Counseling Psychologist

Division of Counseling Psychology of the American Psychological Association

Use of Theory-Driven Research in Counseling:

Carolyn A. Karr Lisa M. Larson

300 THE COUNSELING PSYCHOLOGIST / May 2005

Karr, Larson / USE OF THEORY-DRIVEN RESEARCH 301

302 THE COUNSELING PSYCHOLOGIST / May 2005

Karr, Larson / USE OF THEORY-DRIVEN RESEARCH 303

304 THE COUNSELING PSYCHOLOGIST / May 2005

Karr, Larson / USE OF THEORY-DRIVEN RESEARCH 305

TABLE 1: JVB n = 63 Yes No Yes No Yes No c

JCP/ JCD c 1.51

Karr, Larson / USE OF THEORY-DRIVEN RESEARCH 307

308 THE COUNSELING PSYCHOLOGIST / May 2005

Karr, Larson / USE OF THEORY-DRIVEN RESEARCH 309

310 THE COUNSELING PSYCHOLOGIST / May 2005

Karr, Larson / USE OF THEORY-DRIVEN RESEARCH 311

312 THE COUNSELING PSYCHOLOGIST / May 2005

Karr, Larson / USE OF THEORY-DRIVEN RESEARCH 313

314 Percentages of Sampled Studies Coded Yes or No on the Questions by Year

316 THE COUNSELING PSYCHOLOGIST / May 2005

Karr, Larson / USE OF THEORY-DRIVEN RESEARCH 319

320 THE COUNSELING PSYCHOLOGIST / May 2005

Karr, Larson / USE OF THEORY-DRIVEN RESEARCH 321

322 THE COUNSELING PSYCHOLOGIST / May 2005

Das könnte Ihnen auch gefallen