The current issue and full text archive of this journal is available at
www.emeraldinsight.com/1356-3289.htm
CCIJ
19,2
166
Received 8 October 2012
Revised 27 August 2013
Accepted 13 September
2013
Corporate Communications: An
International Journal
Vol. 19 No. 2, 2014
pp. 166-181
q Emerald Group Publishing Limited
1356-3289
DOI 10.1108/CCIJ-10-2012-0071
Mixed-methods: measurement
and evaluation among investor
relations officers
Matthew W. Ragas
College of Communication, DePaul University, Chicago, Illinois, USA, and
Alexander V. Laskin
Department of Public Relations, Quinnipiac University, Hamden,
Connecticut, USA
Abstract
Purpose – While investor relations have become an established corporate function, research into
how investor relations officers (IROs) practice measurement and evaluation is limited. The purpose of
this paper is to examine which approaches and metrics IROs use to gauge their success.
Design/methodology/approach – To address this gap in the literature, this study surveyed
(n ¼ 384) the corporate membership of the National Investor Relations Institute (NIRI), the world’s
largest investor relations association, on the topic of measurement and evaluation.
Findings – The results indicate that IROs strongly (80 percent) believe that mixed-methods (i.e. both
quantitative and qualitative methods) should be used to measure the success of investor relations.
Mixed-methods advocates place significantly more importance on measurement than IROs that prefer
quantitative- or qualitative-only approaches.
Research limitations/implications – The results of this survey indicate that IROs typically place
the most value on metrics that are qualitative, non-financial and relationship-oriented. These findings
suggest that IROs believe they should be evaluated in large part on their competency at relationship
management.
Practical implications – From a benchmarking perspective, these findings suggest that IROs
looking to align with their peers should use a mix of both quantitative and qualitative evaluation
measures that are non-financial and relationship management-focused.
Originality/value – These findings contribute to recent efforts to explicate a general theory of
investor relations. While investor relations scholarship has grown in recent years, up until this point,
little attention had been paid to measurement and evaluation.
Keywords Measurement, Public relations, Investor relations, Investors
Paper type Research paper
1. Introduction and study purpose
Corporate communications and public relations professionals have long “talked the
talk” regarding the importance of program measurement and evaluation in
demonstrating value and effectiveness (Hutton et al., 2001; Kim, 2001). Recently, the
profession has also started “walking the walk” (Watson, 2012). In 2010, the field
adopted The Barcelona Declaration of Measurement Principles (Manning and
Rockland, 2011). Further, survey research indicates that from 2009 to 2011 US
The authors wish to thank Matthew Brusch and the National Investor Relations Institute for
their assistance on this project.
companies more than doubled the percentage of their communication budgets Mixed-methods
dedicated to measurement (Swerling et al., 2012).
The quest for measurement and evaluation is not a new phenomenon in the field of
corporate communications – Hause (1993), Hon (1998), Lindenmann (2003) and others
– emphasize the importance of measuring the results of programs and activities. Yet,
the focus of the research often was on what to measure (outputs, outcomes and similar)
rather than on how to measure. This study seeks to change this and looks at how 167
practitioners actually measure the results – quantitatively, qualitatively, or using
mixed-methods. Communication scholars (i.e. Lindenmann, 2003, 2006; Goodman and
Hirsch, 2010; Ruler et al., 2008; Stacks, 2010; Stacks and Michaelson, 2010) seem to
suggest that professionals should evaluate their programs using a blend of
quantitative and qualitative methods; this is often called a “mixed-methods”
approach, yet, scholarship that gauges which research paradigm has gained the most
traction in the field remains limited.
Investor relations, a discipline at the intersection of corporate communications,
finance and corporate law (Dolphin, 2004; Penning, 2011), is one such area that talks
about the value of measurement and evaluation (e.g. Metzker, 2006a, 2010), but for
which nationally-representative surveys of investor relations officers (IROs) on how
they approach this topic are almost non-existent. In an effort to fill this gap, the current
study surveyed (n ¼ 384) the corporate membership of the National Investor Relations
Institute (NIRI), the world’s largest investor relations association, to gauge which
research paradigm IROs adopt and which specific measurement criteria are viewed as
the most important. This study also assessed whether this preference is influenced by
the characteristics of the respondent and the company they work for.

2. Literature review
2.1 Investor relations and measurement
In recent decades, investor relations has become an established communication
function within publicly-traded corporations (Dolphin, 2004). Just within the USA,
there are over 5,000 public companies that list their stock on major stock exchanges,
such as the NYSE and the NASDAQ (Lucchetti, 2011). The US-based National Investor
Relations Institute (NIRI) counts more than 3,500 members. The investor relations
function serves as the essential boundary spanner between the publicly held company
and the financial community, communicating corporate strategy and material news to
shareholders and other financial stakeholders, and in turn listening and then sharing
the perceptions and concerns of shareholders with company management and the
board of directors (Ragas, 2011).
As with most departments, investor relations is expected to demonstrate how it
contributes to the attainment of the organization’s goals and objectives, and adds
meaningful value to the organization (Metzker, 2006a, b, 2010; van Riel and Fombrun,
2007). As investor relations have has matured and capital markets have become more
complex, IROs have argued they deserve a larger voice in firm strategy (Metzker,
2006a, b, 2010). One way to advance this goal is through the adoption and effective use
of research methods to improve program performance and demonstrate that IROs are
accountable and delivering value (Zerfass, 2008). However, little is known about how
IROs approach research out in the field and theorization in this regard remains at a
nascent stage, as it largely has across the field (Watson, 2012).
CCIJ 2.2 The quantitative, qualitative and mixed-methods paradigms
19,2 There are three broad categories of research methods for conducting evaluation and
measurement:
(1) qualitative methods;
(2) quantitative methods; and
168 (3) mixed-methods.

The latter combine both qualitative and quantitative approaches. Qualitative methods
consist of collecting open-ended information without pre-set response categories,
thereby yielding non-numeric textual or image-based data; this is commonly called
“soft data” (Babbie, 2007; Creswell and Creswell, 2005; Lindenmann, 2006). Qualitative
methods tend to use small samples to uncover deep, rich insights (i.e. the underlying
qualities). Quantitative research, on the other hand, is focused on quantities
(i.e. counting) and seeks to generalize the closed-ended information collected from a
sample to a larger population through the use of statistics (Creswell and Creswell, 2005;
Lindenmann, 2006). Given that quantitative methods yield numeric data, this
information is often known as “hard data” (Babbie, 2007). Experiments, survey
research and some content analyses would fall under quantitative research, whereas
focus groups, depth interviews, ethnographies and historical analyses would often be
considered qualitative research.
The third major approach to research is mixed-methods. As explained by Creswell
and Creswell (2005), “mixed methods is a research design or methodology for
collecting, analyzing, and mixing both quantitative and qualitative data [. . .] to better
understand research problems” (p. 317). The ideas behind mixed methods date back 50
years (Campbell and Fiske, 1959). Across the social and behavioral sciences, though,
for many years there was what was called “the qualitative-quantitative debate” or
“paradigm wars” (Tashakkori and Teddlie, 1998, p. 3). Over the past 15 years,
mixed-methods has gained increasing acceptance among scholars as the “third
methodological movement” or paradigm (Johnson et al., 2007; Tashakkori and Teddlie,
2010). As argued by Bryant and Cummins (2007), “with the use of multiple methods,
the strengths of one could shore up the deficiencies of another” and “provide a fuller
understanding of the communication process” (p. 10).
With this in mind, communication methodologists generally recommend that
communication professionals use mixed methods to evaluate their research programs
(e.g. Lindenmann, 2003, 2006; Stacks, 2010; Watson and Noble, 2007; Wimmer and
Dominick, 2006). While a specific research question or problem may be best addressed
by using a particular quantitative or qualitative research method, an ongoing general
research program is often best served by employing a mix of both quantitative and
qualitative research tools. As summed up by Lindenmann (2006, p. 12), “a mix of both
qualitative and quantitative research is preferable” (underline in original). A global
survey of communication professionals by Wright et al. (2009) concluded that: “There
are no clear winners when it comes to particular tools. Each one has its adherents.
There is still no clear consensus on measures or methodology” (p. 23).
Within the investor relations field, professionals do not seem wed to the traditional
quantitative-only or qualitative-only research paradigms. Instead, they recommend
and describe using both quantitative and qualitative-oriented metrics. For example, in
terms of qualitative-oriented measures, IROs stress the importance of responsiveness
to investor inquiries, relationships with the financial community, and qualitative Mixed-methods
feedback from their managers and the C-suite (Metzker, 2006a, 2010). Purely
quantitative metrics that IROs may use include company stock price, the valuation of
the stock relative to its peers, and stock liquidity or trading volume (Michaelson and
Gilfeather, 2003).
IROs also engage in many different activities that may be used for assessment
purposes, such as investor conference presentations, individual meetings with the 169
buy-side (i.e. professional money managers), sell-side analyst coverage (i.e. investments
analysts at brokerage firms), meetings with management or the board of directors, and
financial news coverage (Carroll, 2011; Petersen and Martin, 1996; Marston and
Straker, 2001). Investor perception studies, the composition of the shareholder base,
and external recognition/awards are additional potential success criteria (Metzker,
2006a). Finally, social media is a new area for investor relations evaluation (Duckworth
et al., 2009).
It is important to note that many of these aforementioned criteria could be evaluated
either quantitatively or qualitatively depending on the implementation. For example,
an IRO could assess a company’s standing among equity analysts both qualitatively
through their conversations with these analysts and quantitatively by coding the
frequency and tone of coverage in published analyst research notes. The use of
multiple types of metrics is likely to paint a more complete picture and yield a deeper
understanding. In sum, while prior research has not specifically focused on the
preferred research paradigm within investor relations, a review of the literature
suggests that IROs are likely to use a blended approach.
Therefore, the following hypothesis is submitted[1]:
H1. More investor relations officers will evaluate their investor relations program
using the mixed-methods paradigm than a quantitative or qualitative-only
research paradigm.
As reviewed above, IROs may use a wide range of different criteria to measure whether
a program is meeting its goals and objectives. In reviewing the literature, it is clear that
some scholars and professionals advocate for taking a more quantitative approach,
some prefer a more qualitative approach, and others fall somewhere in the middle
(e.g. Goodman and Hirsch, 2010; Michaelson and Gilfeather, 2003; Metzker, 2006a). To
gain a deeper understanding of measurement and evaluation in investor relations, it is
important to understand which specific measurement criteria are perceived as the most
important depending on the research paradigm practiced by the professional. Further,
it is important to see whether this preferred approach also impacts the overall
importance the professional assigns to research and evaluation as a whole.
With this in mind, the following research questions are posed:
RQ1. What are the top individual measurement criteria that are perceived as most
important by IROs choosing the quantitative, qualitative or mixed-methods
research paradigms?
RQ2. What is the relationship between IROs’ selected research paradigm for
measurement and the overall importance they assign to measurement criteria
as a whole?
CCIJ 2.3 Measurement approach and reporting structure
19,2 Corporate communications and public relations scholars (Grunig et al., 2002; Zerfass,
2008), as well as professional bodies that represent the profession (e.g. PRSA in the
USA and CIPR in the UK), frequently define investor relations as one of their
discipline’s sub-functions or affiliated areas. Yet, several studies conclude that IROs
quite commonly report to the Chief Financial Officer (CFO) rather than to the Chief
170 Communication Officer (CCO) or directly to the Chief Executive Officer (CEO) (Petersen
and Martin, 1996). This reporting structure can present a serious issue when it comes
to evaluating investor relations activities.
Ragas (2011) for example, reports that IROs overwhelmingly agree that
relationship-building with the financial community is one of the most important
tasks of investor relations. Yet, relationship-building is a largely intangible criterion
that can be difficult to measure: “Although stock price, trading volume, and analyst
recommendations are readily available, the indicators of relationships between the
company and investors are not” (Ragas, 2011). At the same time, CFOs and other
financial executives tend to focus on quantitative, primarily financial metrics,
preferring them to qualitative measures and attempting to apply them to investor
relations activities as well. Favaro (2001) explains that traditional role of CFO is “to
keep tabs on the money – what came in and what went out” (p. 4). As a result, this
study proposes a hypothesis that IROs reporting to the CFO will have a stronger focus
on quantitative methods in their work in comparison with professionals reporting to
non-CFOs:
H2. IROs reporting to CFOs will be more likely to adopt the quantitative-only
research paradigm for evaluation than IROs reporting to non-CFOs.

2.4 Measurement approach and other constraints

Neither quantitative nor qualitative measures on their own can provide a full picture of
investor relations’ influence on the company’s bottom line. Indeed, no single measure
and no single approach can effectively evaluate the variety of investor relations’
contributions. Michaelson and Gilfeather (2003) describe multiple measures that
investor relations professionals can employ, which combine financial and non-financial
as well as quantitative and qualitative criteria. Ragas (2011)proposes four broad areas
in which investor relations’ contribution may be evaluated:
(1) securities valuation;
(2) stock liquidity;
(3) financial analysts’ coverage; and
(4) relationships with the financial community.

Once again, in each of these areas, a variety of quantitative and qualitative metrics
(i.e. mixed-methods) can and should be employed.
Thus, to better evaluate investor relations’ contribution, a mixed-methods research
approach is the most beneficial. However, due to various limitations, this may not be
the approach actually used by the IRO. For example, company size, budget or
experience constraints can prevent IROs from using a mixed-methods approach and
instead relying on just quantitative or qualitative measures. As a result, this study
submits the following research questions:
 RQ3. What is the influence of market capitalization on IROs’ measurement Mixed-methods
approaches in evaluation?
RQ4. What is the influence of investor relations budget on IROs’ measurement
approaches in evaluation?
RQ5. What is the influence of years of experience on IROs’ measurement
approaches in evaluation? 171

3. Method
This study used a self-administered online questionnaire to gauge how IROs go about
measurement and evaluation. The survey data is based on the responses of the
corporate membership of NIRI, the world’s largest investor relations association.

3.1 Questionnaire and measures

The multi-page online questionnaire consisted of three main sections:
(1) evaluation and measurement;
(2) the specific criteria used to evaluate and measure investor relations; and
(3) respondent demographics.

The measures relevant to the current study are outlined below[2].

Respondents were asked to select which general approach to evaluation research
they use for measuring the success of investor relations. For this closed-ended
question, the three categorical choices provided were qualitative metrics (i.e. based on
non-numerical data), quantitative metrics (i.e. based on numerical data), or a
combination of both metrics.
Using a seven-point Likert-type scale where 1 is “not important at all” and 7 is
“extremely important”, respondents were asked to rate the perceived importance of 17
different criteria in evaluating the success of a company’s investor relations. The
criteria selected for inclusion were based on a careful review of the extant literature on
investor relations activities and measurement (e.g. Dolphin, 2004; Metzker, 2006a, 2010;
Michaelson and Gilfeather, 2003; Penning, 2011; Petersen and Martin, 1996; Marston
and Straker, 2001), and input from NIRI’s research and publications committee
(i.e. composed of practicing IROs). In an effort to be exhaustive, an open-ended
response option allowed respondents to share additional criteria, but tabulation of
these responses did not find any additional criterion that totaled 5 percent of more of
the responses. Therefore, they were excluded from the data analysis.
After indicating consent, respondents were asked a series of closed-ended
demographic questions about the characteristics of themselves and their companies.
This included questions on their reporting structure, the portion of their investor
relations budget spent on and evaluation, their years of experience working in investor
relations, and the market capitalization of their companies. To build statistical power
for data analysis, these variables were collapsed into more parsimonious categories:
reporting (CFO or non-CFO reporting), budget (less than 1 percent, 1-2 percent, 3-4
percent, greater than 5 percent), experience (ten years or less, 11 to 20 years, more than
20 years), and market capitalization (small-cap ¼ less than $2bn in market value,
CCIJ mid-cap ¼ $2bn to less than $10bn in market value, large-cap ¼ $10bn and more in
19,2 market value).

3.2 Sample
The sample frame consisted of the corporate membership list of NIRI as of late
September 2011. This pre-screened list contained the e-mail addresses of all 2,519
172 corporate NIRI members (i.e. only officers that worked in internal investor relations
positions at companies, rather than agencies) as of the time of the survey. The
self-administered questionnaire was open for a two-week period in fall 2011 (September
28-October 13). An e-mail invitation to participate was sent directly to each corporate
member on the list. After taking into account undeliverable e-mails and opt-outs, there
were a total of 2,469 potential respondents. The response rate of 15.5 percent (n ¼ 384)
was satisfactory for online surveys (Kaplowitz et al., 2004), specifically surveys of busy
communication professionals (e.g. Hong and Ki, 2007; Sweetser et al., 2008).
Among the respondents who shared this information, 58 percent (n ¼ 190) reported
having a decade or less of experience in investor relations, 35 percent (n ¼ 116)
reported having 11 to 20 years, while 8 percent (n ¼ 25) indicated having more than 20
years. The market capitalizations of the respondents’ companies were also spread out,
with more than 40 percent (n ¼ 142) working for small-cap companies, 36 percent
(n ¼ 199) with mid-caps, and 21 percent (n ¼ 69) with large-caps. Sixty percent
(n ¼ 201) of respondents’ companies are listed with NYSE AMEX and 35 percent
(n ¼ 117) are listed with NASDAQ OMX. The top industries, based on SIC code, were
services (n ¼ 118, 30 percent), manufacturing (n ¼ 76, 19 percent), finance, insurance
and real estate (n ¼ 58, 15 percent), retail trade (n ¼ 27, 7 percent), and transportation,
communications, electric, gas and sanitary services (n ¼ 21, 5 percent).

3.3 Data analysis

Data analysis was based on a mix of descriptive statistics, one-way ANOVAs, and
Fisher’s exact tests (FET). Given the presence of small and unbalanced cells within the
distribution of the data, FETs, rather than x 2 tests, were appropriate for analyzing the
categorical data. FET calculates p-values based on the true distribution of the test
statistic, rather than the x 2 test’s reliance on asymptotic (i.e. estimated) p-values
(Agresti, 1990). For the ANOVAs, the Welch (1951) F-test correction was employed
instead of the standard F-test whenever Levene’s (1960) test detected heterogeneity of
group variance.

4. Results
H1, which predicted that IROs would adopt the mixed-methods research paradigm for
evaluation, was supported. Eighty percent of respondents (n ¼ 213) favored using a
mix of quantitative and qualitative metrics, while 17 percent (n ¼ 45) favored
exclusively using qualitative metrics, and just 3 percent (n ¼ 9) felt that only
quantitative metrics were appropriate.
Moving onto RQ1, as shown by the limited number of statistically significant
differences found in Table I, there was a high degree of agreement among respondents
adopting quantitative, qualitative or mixed-methods regarding the importance of
individual measurement criteria. For mixed-methods respondents, the most important
criteria were: relationship with financial community (M ¼ 6.09, SD ¼ 1.01), individual
 Preferred type of measurement
Quantitative mean Qualitative mean Mixed mean
Criteria for evaluating success (SD) (SD) (SD) F-test
Social media channels 1.78 1.26 2.27 F(2, 21.605) ¼ 22.2, p ¼ 0.000, h 2 ¼ 0.07
(0.972) (0.657)a (1.41)a
Individual meeting with top shareholders 5.44 5.64 5.98 F(2, 18.4) ¼ 1.697, p ¼ 0.211, h 2 ¼ 0.02
(1.94) (1.09) (0.969)
Presentation(s) to board of directors 4.11 3.41 4.26 F(2, 18.6) ¼ 2.645, p ¼ 0.098, h 2 ¼ 0.03
(2.03) (2.00) (1.62)
Investor conference presentations 5.00 5.00 5.27 F(2, 239) ¼ 0.901, p ¼ 0.408, h 2 ¼ 0.01
(1.80) (1.51) (1.15)
Investor perception studies 4.11 3.58 4.56 F(2, 18.9) ¼ 2.801, p ¼ 0.086, h 2 ¼ 0.03
(1.90) (2.28) (1.88)
Financial news media coverage 2.89 2.62 3.51 F(2, 237) ¼ 4.67, p ¼ 0.01, h 2 ¼ 0.04
(2.03) (1.60) (1.64)
Relationship with the financial community 5.22 6.22 6.09 F(2, 242) ¼ 3.21, p ¼ 0.042, h 2 ¼ 0.03
(1.86) (1.15) (1.01)
Responsiveness to investor inquiries 4.78 5.72 5.84 F(2, 18.5) ¼ 0.920, p ¼ 0.416, h 2 ¼ 0.02
(2.39) (1.45) (1.29)
Composition of shareholder base 4.00 4.14 5.40 F(2, 241) ¼ 16.362, p ¼ 0.000, h 2 ¼ 0.12
(1.87)a (1.53) (1.31)a
Feedback from the financial community 4.67 5.86 5.96 F(2, 18.3) ¼ 1.516, p ¼ 0.246, h 2 ¼ 0.05
(2.24) (1.20) (1.02)
Valuation of company stock relative to peers 5.11 4.08 4.79 F(2, 242) ¼ 2.93, p ¼ 0.055, h 2 ¼ 0.02
(1.83) (1.86) (1.66)
Sell-side analyst coverage quality 5.00 4.81 5.47 F(2, 243) ¼ 3.68, p ¼ 0.027, h 2 ¼ 0.03
(1.66) (1.70) (1.35)
Sell-side analyst coverage quantity 4.33 4.06 4.31 F(2, 241) ¼ 0.370, p ¼ 0.691, h 2 ¼ 0.00
(1.58) (1.76) (1.64)
Liquidity/trading volume in stock 2.11 2.51 4.00 F(2, 240) ¼ 20.1, p ¼ 0.00, h 2 ¼ 0.14
(0.928)a (1.36)b (1.53)a,b
Qualitative assessment by the C-suite 6.11 5.97 5.93 F(2, 240) ¼ 0.099, p ¼ 0.906, h 2 ¼ 0.00
(0.928) (1.51) (1.24)
Change in company stock price 3.67 2.74 3.37 F(2, 238) ¼ 2.27, p ¼ 0.106, h 2 ¼ 0.02
(2.24) (1.65) (1.68)
External recognition/awards 2.22 2.83 3.19 F(2, 236) ¼ 1.84, p ¼ 0.162, h 2 ¼ 0.02
(1.48) (1.93) (1.71)
Overall mean score 4.15 4.17 4.73
(1.12) (0.747) (0.749)
Notes: Respondents were asked to “please rate the importance of . . .” each criterion listed above for evaluation and measurement where 1 is “not important at all” and 7 is “extremely
important”. For the one-way repeated ANOVAs, values are Bonferroni corrected (17/0.05) so values at p , 0.003 are statistically significant. The Welch F-test correction was employed
instead of the standard F-test whenever Levene’s test detected heterogeneity of group variances. a,b ¼ indicate significant differences detected between scores based on the Bonferroni or
Games-Howell post hoc test for painwise comparisons.

based on the respondents’

Mixed-methods

measurement approach
and evaluation criteria
relations measurement
importance of investor
Perceptions of the
173

Table I.
CCIJ meetings with top shareholders (M ¼ 5.98, SD ¼ 0.969), feedback from financial
19,2 community (M ¼ 5.96, SD ¼ 1.02), qualitative assessment by the C-suite (M ¼ 5.93,
SD ¼ 1.24), and responsiveness to investor inquiries (M ¼ 5.84, SD ¼ 1.29).
For qualitative-only respondents, the top criteria were: relationship with financial
community (M ¼ 6.22, SD ¼ 1.15), qualitative assessment by the C-suite (M ¼ 5.97,
SD ¼ 1.51, feedback from financial community (M ¼ 5.86, SD ¼ 1.20), responsiveness
174 to investor inquiries (M ¼ 5.72, SD ¼ 1.45), and individual meetings with top
shareholders (M ¼ 5.64, SD ¼ 1.09).
Finally, for quantitative-only respondents, the top criteria were: qualitative
assessment by the C-suite (M ¼ 6.11, SD ¼ 0.928), individual meetings with top
shareholders (M ¼ 5.44, SD ¼ 1.94), relationship with financial community (M ¼ 5.22,
SD ¼ 1.86), valuation of stock relative to peers (M ¼ 5.11, SD ¼ 1.83), and conference
presentations (M ¼ 5.00, SD ¼ 1.80).
To assess the relationship between the chosen research method paradigm and the
overall importance placed on measurement and evaluation (RQ2), a mean index
(a ¼ 0.843), consisting of the scores for the 17 individual criteria, was created.
Respondents choosing mixed-methods for their evaluation research (M ¼ 4.73,
SD ¼ 0.749) placed significantly more value (F(2, 243) ¼ 9.98, p ¼ 0.000, n 2 ¼ 0.28) on
measurement and evaluation than qualitative-only respondents (M ¼ 4.17, SD ¼ .747).
While measurement research was less valued by quantitative-only respondents
(M ¼ 4.15, SD ¼ 1.12) than mixed-methods respondents, a pairwise comparison
revealed that the difference was only marginally significant ( p , 0.10).
H2 proposed that IROs reporting to the CFO would be more likely to adopt the
quantitative-only paradigm than practitioners not reporting to the CFO. The results
presented in Table II indicate that professionals, no matter of their reporting structure,
prefer mixed methods. Fisher’s exact test did not reveal a statistically significant
difference between those reporting to the CFO versus others (FET ¼ 0.524; p . 0.05).
RQ3 asked whether IROs at higher market capitalization companies would be more
likely to adopt the mixed-methods paradigm. Table III shows that professionals at

Do not report to
Report to CFO CFO Total
Percentage n Percentage n Percentage n
Table II.
Quantitative, qualitative, Quantitative 3.3 6 4.8 3 3.7 9
and mixed methods Qualitative 15.9 29 14.3 9 15.5 38
based on the respondents’ Mixed methods 80.8 147 81 51 80.8 198
reporting relationships Total 100 182 100 63 100 245

Small Mid Large

capitalization capitalization capitalization Total
Percentage n Percentage n Percentage n Percentage n
Table III.
Quantitative, qualitative, Quantitative 5.2 5 3.3 3 1.7 1 3.6 9
and mixed methods Qualitative 18.6 18 13 12 13.6 8 15.3 38
based on companies’ Mixed-methods 76.3 74 83.7 77 84.7 50 81 201
market capitalizations Total 100 97 100 92 100 59 100 248
large market-cap companies do choose to use mixed-methods more (84.7 percent) than Mixed-methods
at mid-caps (83.7 percent) or small-caps (76.3 percent). However, Fisher’s exact test did
not detect significant differences between groups (FET ¼ 2.55; p . 0.05).
RQ4 examined whether IROs with higher measurement and evaluation budgets
favored mixed-methods over using solely quantitative or qualitative methods. The
results shown in Table IV indicate that, no matter the budget percentage, practitioners
rely on mixed-methods the most: 75.2 percent of those with evaluation making up less 175
than 1 percent of the total budget, 86.3 percent of those with budgets of 1-2 percent,
88.9 percent of those with budgets of 3-4 percent, and 85.4 percent of those with
budgets of 5 percent or more. Fisher’s exact test did not find significant differences
(FET ¼ 6.76; p . 0.05).
RQ5 probed whether IROs with more experience would rely more on
mixed-methods for measurement than less experienced colleagues. To evaluate this
question, the respondents were placed into three groups:
(1) ten years or less of experience;
(2) 11-20 years of experience; and
(3) more than 20 years of experience.

Table V shows that practitioners with over 20 years of experience used mixed-methods
more (90 percent) than those with 11-20 years (78 percent) or with ten years or less of
experience (81 percent). However, Fisher’s exact test did not find significant differences
(FET ¼ 2.60; p . 0.05).

5. Discussion
Those looking from the outside in at the investor relations field might assume that
IROs prefer a more quantitative-oriented approach, since they work with
quantitative financial data, often report to financial executives, and often have

Less than 1 More than 5

percent 1-2 percent 3-4 percent percent Total
Percentage n Percentage n Percentage n Percentage n Percentage n

Quantitative 3.8 5 3.9 2 2.8 1 – – 3.1 8 Table IV.

Qualitative 21.1 28 9.8 5 8.3 3 14.6 6 16.1 42 Quantitative, qualitative,
Mixed- and mixed methods
methods 75.2 100 86.3 44 88.9 32 85.4 35 80.8 211 based on the evaluation
Total 100 133 100 51 100 36 100 41 100 261 budget size

Ten or less 11-20 More than 20 Total

Percentage n Percentage n Percentage n Percentage n
Table V.
Quantitative 3.7 5 3.3 3 5 1 3.7 9 Quantitative, qualitative,
Qualitative 14.9 20 18.5 17 5 1 15.4 38 and mixed methods
Mixed-methods 81.3 109 78.3 72 90 18 80.9 199 based on the respondents’
Total 100 134 100 92 100 20 100 246 years of experience
CCIJ training in finance or accounting themselves. However, the results of this study
19,2 demonstrate empirically that IROs see value in a blend of both research
perspectives, preferring to use a combination of quantitative and qualitative metrics
to measure the success of their programs. More specifically, a commanding 80
percent of survey respondents felt that mixed-methods should be used. Further, this
dominant group assigns significantly greater importance overall to measurement
176 than their less flexible colleagues (i.e. the 20 percent that limit themselves to using
just quantitative or qualitative metrics).
Digging more deeply into the data, mixed-methods IROs place the most value on
metrics that are non-financial and relationship-oriented. Said another way,
professionals gravitate towards soft data (i.e. non-numeric information) rather than
hard data (i.e. numeric information), perhaps mirroring a larger trend in the investor
relations field of looking beyond solely the financials (Hoffmann and Fieseler, 2012).
Their relationship with the financial community, individual meetings with top
shareholders, and feedback from the financial community were the three highest rated
criteria among mixed-methods advocates. Even among respondents favoring a
quantitative-only approach, indicators that typically rely more on soft data received
top billing: assessment by the C-suite, individual meetings with top shareholders, and
relationship with the financial community. Quantitative-only respondents did rate the
relative value of company stock, a decidedly financial and tangible metric, in their top
five.
Looking at the highest rated criteria, the results indicate that IROs believe they
should be evaluated in large part on their competency at relationship management and
cultivation (Ragas, 2009, 2011; Kelly et al., 2010), which is similar to their colleagues
working in public relations (Hon, 1998). IROs may place less importance on
quantitative-oriented, financial metrics simply because they feel these indicators are
less under their control. At the end of the day, the vagaries of the economy and
company-specific financial performance, rather than investor relations-specific efforts,
are often the driving forces behind changes in stock price and valuations.
Unexpectedly, this study found that the characteristics of US IROs and their
companies seem to play only a minor role in the measurement approach they take. This
can partially be explained by the strict regulations that govern investor relations.
Further, the uniformity in how US IROs approach measurement and evaluation may be
due to the growing professionalization of the field. Membership and participation in
NIRI, help shape the profession, and help IROs learn and adopt best practices. Investor
relations associations in other parts of the world, such as The Investor Relations
Society in the UK, the Canadian Investor Relations Institute, the Australasian Investor
Relations Association and the Deutscher Investor Relations Verband eV play similar
roles and may also drive uniformity.

5.1 Theoretical and practical implications

These findings contribute to recent efforts to explicate a general theory of investor
relations (Kelly et al., 2010; Ragas, 2009, 2011). While investor relations scholarship
has grown in recent years, up until this point, little attention had been paid to
investor relations measurement and evaluation. Yet, this is an essential piece in
understanding the profession – it highlights not just what professionals do but also
what they think is important. Communication professionals are devoting more
attention to measurement and evaluation (Swerling et al., 2012). Research on Mixed-methods
measurement in investor relations, a discipline that is aligned with corporate
communication (Goodman and Hirsch, 2010), contribute to a broader understanding
of evaluation practices across the field.
In terms of theorization, a growing body of empirical work (e.g. Dolphin, 2004;
Hoffmann and Fieseler, 2012; Kelly et al., 2010; Marston and Straker, 2001) finds
general agreement that IROs place a premium on relationship management and 177
non-financial indicators when developing investor relations strategies, implementing
tactics, and now evaluating their programs. Perhaps this is not surprising, given that
in their day-to-day work, IROs increasingly provide the financial community with
non-financial performance information to complement the company financial
statements they have long provided (Clarke and Murray, 2000; Hoffmann and
Fieseler, 2012). Therefore, these findings suggest that, in order to enhance the validity
of theorization in investor relations, researchers should examine organization-public
relationships (e.g. Ledingham, 2003; Ledingham and Bruning, 1998) in future work.
These findings also have practical implications for investor relations and corporate
communication professionals and academicians. From a benchmarking perspective,
these findings suggest that IROs seeking to stay in step with their peers should use a
mix of both quantitative and qualitative evaluation measures that are non-financial
and relationship management-focused. For scholarly researchers and journal editors,
these findings serve as a reminder that qualitative and mixed-methods are generally
held in high regard among communication professionals (Tashakkori and Teddlie,
2010) and should not receive short shrift in academic journals (Pasadeos et al., 2011;
Sallot et al., 2003).
Of course, there are also trade-offs and a downside to the use of mixed-methods by
scholars and practitioners alike. Proper implementation of mixed-methods research
designs typically take more time, use more resources, and require a broader set of
research skills and proficiencies (Molina-Azorin, 2010). There may also be difficulty in
making sense of the findings and integrating them (Bryman, 2007). For a
communication professional working in a smaller organization or department with
limited resources, as much as they might want to evaluate a program objective using
multiple measurement approaches, this may not be feasible.
Proper training of communication professionals in research methods is likely to
remain a pressing topic for the foreseeable future. Research reveals that professionals
increasingly cite the importance of measurement and evaluation, but that
approximately half of them have not received formal training in research or
measurement (Wright et al., 2009). Colleges and universities can play an important role
in helping close this gap, but so should professional and industry associations like
NIRI, which help set the standards and promote best practices in particular
specializations within corporate communications and related fields.

5.2 Limitations and future research

These findings may not be generalizable to IROs located outside of the USA or investor
relations professionals working for agencies or consultancies. Future research is
needed that explores this topic among non-US IROs and agency-based professionals.
This study chose to survey IROs since it wanted to understand how those
professionals most responsible for the day-to-day operations of an investor relations
CCIJ department approach this topic. However, it is also important to understand how key
19,2 external stakeholders (e.g. analysts and shareholders) and the senior managers that
IROs often report to or interact with perceive measurement and evaluation.
Looking beyond investor relations, the current study should be replicated and
extended in other specialty areas of the field like crisis communication and employee
communication. In the spirit of mixed-methods, more industry surveys are needed, but
178 so is more qualitative work using interviews and focus groups that help answer the
“why” questions that quantitative work alone is rarely able to determine. Theories of
communication measurement and evaluation will not be built overnight (Watson,
2012), but this study lays the groundwork for future scholarship on this topic in
investor relations and beyond.

Notes
1. The authors chose to introduce hypotheses when they determined the literature review had
revealed sufficient prior evidence to warrant predictive statements. When the authors felt
there was insufficient prior evidence, they posed the inquiries as research questions.
2. A copy of the questionnaire is available from the authors upon request.

References
Agresti, A. (1990), Categorical Data Analysis, Wiley, New York, NY.
Babbie, E. (2007), The Practice of Social Research, 11th ed., Thomson Wadsworth, Belmont, CA.
Bryant, J. and Cummins, G. (2007), “Traditions of mass media theory and research”, in Preiss, R.,
Gayle, B., Burrell, N., Allen, M. and Bryant, J. (Eds), Mass Media Effects Research:
Advances Through Meta-Analysis, Lawrence Erlbaum Assocoates, Mahwah, NJ, pp. 1-13.
Bryman, A. (2007), “Barriers to integrating quantitative and qualitative research”, Journal of
Mixed Methods Research, Vol. 1 No. 1, pp. 8-22.
Campbell, D. and Fiske, D. (1959), “Convergent and discriminant validation by the
multitrait-multimethod matrix”, Psychological Bulletin, Vol. 56 No. 2, pp. 81-105.
Carroll, C. (Ed.) (2011), Corporate Reputation and the News Media: Agenda-setting within
Business News Coverage in Developed, Emerging, and Frontier Markets, Routledge, New
York, NY.
Clarke, G. and Murray, L. (2000), “Investor relations: perceptions of the annual statement”,
Corporate Communications: An International Journal, Vol. 5 No. 3, pp. 144-151.
Creswell, J. and Creswell, J. (2005), “Mixed methods research: developments, debates and
dilemmas”, in Swanson, R. and Holton, E. III (Eds), Research in Organizations: Foundations
and Methods of Inquiry, Berrett-Koehler, San Francisco, CA, pp. 313-326.
Dolphin, R. (2004), “The strategic role of investor relations”, Corporate Communications: An
International Journal, Vol. 9 No. 1, pp. 25-42.
Duckworth, A., Golz, J. and Trayner, G. (2009), “Are analysts and investors engaging with new
media?”, available at: http://bit.ly/14qohTn (accessed October 3, 2012).
Favaro, P. (2001), “Beyond bean counting: the CFO’s expanding role”, Strategy & Leadership,
Vol. 29 No. 5, pp. 4-8.
Goodman, M. and Hirsch, P. (2010), Corporate Communication: Strategic Adoption for Global
Practice, Peter Lang, New York, NY.
Grunig, L., Grunig, J. and Dozier, D. (2002), Excellent Public Relations and Effective Organizations: Mixed-methods
A Study of Communication Management in Three Countries, Lawrence Erlbaum
Associates, Mahwah, NJ.
Hause, D. (1993), “Measuring the impact of public relations”, Public Relations Journal, Vol. 4 No. 2,
pp. 14-21.
Hoffmann, C. and Fieseler, C. (2012), “Investor relations beyond financials: non-financial factors
and capital market image building”, Corporate Communications: An International Journal, 179
Vol. 17 No. 2, pp. 138-155.
Hon, L. (1998), “Demonstrating effectiveness in public relations: goals, objectives and
evaluation”, Journal of Public Relations Research, Vol. 10 No. 2, pp. 103-135.
Hong, Y. and Ki, E. (2007), “How do public relations practitioners perceive investor relations?
An exploratory study”, Corporate Communications: An International Journal, Vol. 12 No. 2,
pp. 199-213.
Hutton, J., Goodman, M., Alexander, J. and Genest, C. (2001), “Reputation management: the new
face of corporate public relations?”, Public Relations Review, Vol. 27 No. 3, pp. 247-261.
Johnson, R., Onwuegbuzie, A. and Turner, L. (2007), “Toward a definition of mixed methods
research”, Journal of Mixed Methods Research, Vol. 1 No. 1, pp. 112-133.
Kaplowitz, M., Hadlock, T. and Levine, R. (2004), “A comparison of web and mail survey
response rates”, Public Opinion Quarterly, Vol. 68 No. 1, pp. 94-101.
Kelly, K., Laskin, A. and Rosenstein, G. (2010), “Investor relations: two-way symmetrical
practice”, Journal of Public Relations Research, Vol. 22 No. 2, pp. 182-208.
Kim, Y. (2001), “Measuring the economic value of public relations”, Journal of Public Relations
Research, Vol. 13 No. 1, pp. 3-26.
Ledingham, J. (2003), “Explicating relationship management as a general theory of public
relations”, Journal of Public Relations Research, Vol. 15 No. 2, pp. 181-198.
Ledingham, J. and Bruning, S. (1998), “Relationship management in public relations: dimensions
of an organization-public relationship”, Public Relations Review, Vol. 21 No. 1, pp. 55-65.
Levene, H. (1960), “Robust tests for equality of variances”, in Olkin, I., Ghurye, S., Hoeffding, W.,
Madow, W. and Mann, H. (Eds), Contributions to Probability and Statistics, Stanford
University Press, Stanford, CA, pp. 278-292.
Lindenmann, W. (2003), “Guidelines for measuring the effectiveness of PR programs and
Activities”, Institute for Public Relations, Gainesville, FL.
Lindenmann, W. (2006), “Public relations research for planning and evaluation”, Institute for
Public Relations, Gainesville, FL.
Lucchetti, A. (2011), “US falls behind in stock listings”, Wall Street Journal, May 26, p. A1.
Manning, A. and Rockland, D. (2011), “Understanding the Barcelona Principles”, Public Relations
Strategist, Vol. 17 No. 1, pp. 30-31.
Marston, C. and Straker, M. (2001), “Investor relations: a European survey”, Corporate
Communications: An International Journal, Vol. 6 No. 2, pp. 82-93.
Metzker, C. (2006a), “Measurement and benchmarking for ultimate IR success”, National Investor
Relations Institute, Alexandria, VA.
Metzker, C. (2006b), “The IRO as strategic partner”, IR Update, May, pp. 4-7.
Metzker, C. (2010), “Measure up”, IR Update, June/July, pp. 10-13.
Michaelson, D. and Gilfeather, J. (2003), “What you need to know to measure investor relations”,
Institute for Public Relations, Gainesville, FL.
CCIJ Molina-Azorin, J. (2010), “The use and added value of mixed methods in management research”,
Journal of Mixed Methods Research, Vol. 5 No. 1, pp. 7-24.
19,2
Pasadeos, Y., Lamme, M., Gower, K. and Tian, S. (2011), “A methodological evaluation of public
relations research”, Public Relations Review, Vol. 37 No. 2, pp. 163-165.
Penning, T. (2011), “The value of public relations in investor relations: individual investors’
preferred information types, qualities, and sources”, Journalism & Mass Communication
180 Quarterly, Vol. 88 No. 3, pp. 615-631.
Petersen, B. and Martin, H. (1996), “CEO perceptions of investor relations as a public relations
function”, Journal of Public Relations Research, Vol. 8 No. 3, pp. 173-209.
Ruler, B., Vercic, A. and Vercic, D. (Eds) (2008), Public Relations Metrics: Research and
Evaluation, Routledge, New York, NY.
Sallot, L., Lyon, L., Acosta-Alzuru, C. and Jones, K. (2003), “From aardvark to zebra: a new
millennium analysis of theory development in public relations academic journals”, Journal
of Public Relations Research, Vol. 15 No. 1, pp. 27-90.
Stacks, D. (2010), Primer of Public Relations Research, 2nd ed., Guilford Press, New York, NY.
Stacks, D. and Michaelson, D. (2010), A Practitioner’s Guide to Public Relations Research.
Measurement and Evaluation, Business Expert Press, New York, NY.
Sweetser, K., Porter, L., Chung, D. and Kim, E. (2008), “Credibility and the use of blogs among
professionals in the communication industry”, Journalism & Mass Communication
Quarterly, Vol. 85 No. 1, pp. 169-185.
Swerling, J., Thorson, K., Tenderich, B., Ward, N., O’Tierney, B., Becker, M., Wang, J. and Li, Y.
(2012), “Seventh Communication and Public Relations Generally Accepted Practices
Study”, Strategic Communication & Public Relations Center, University of Southern
California, Los Angleles, CA.
Tashakkori, A. and Teddlie, C. (1998), Mixed Methodology: Combining Qualitative and
Quantitative Approaches, Sage Publications, Thousand Oaks, CA.
Tashakkori, A. and Teddlie, C. (2010), Sage Handbook of Mixed Methods in Social & Behavioral
Research, 2nd ed., Sage Publications, Thousand Oaks, CA.
van Riel, C. and Fombrun, C. (2007), Essentials of Corporate Communication: Implementing
Practices for Effective Reputation Management, Routledge, New York, NY.
Watson, T. (2012), “The evolution of public relations measurement and evaluation”, Public
Relations Review, Vol. 38 No. 3, pp. 390-398.
Watson, T. and Noble, P. (2007), Evaluating Public Relations, 2nd ed., Kogan Page, London.
Welch, B. (1951), “On the comparison of several mean values: an alternative approach”,
Biometrika, Vol. 38 Nos 3/4, pp. 330-336.
Wimmer, R. and Dominick, J. (2006), Mass Media Research: An Introduction, 8th ed., Wadsworth,
Boston, MA.
Wright, D., Gaunt, R., Leggetter, B., Daniels, M. and Zerfass, A. (2009), “Global survey of
communication measurement 2009 – final report”, available at: http://bit.ly/1di7RBi
(accessed April 18, 2013).
Zerfass, A. (2008), “Communication revisited: integrating business strategy and strategic
communication”, in Zerfass, A., van Ruler, B. and Sriramesh, K. (Eds), Public Relations
Research: European and International Perspectives and Innovations, VS Verlag,
Wiesbaden, pp. 65-96.
About the authors Mixed-methods
Matthew W. Ragas, PhD, is an Assistant Professor in the College of Communication at DePaul
University in Chicago, USA. His research focuses on business news and public opinion, investor
relations, and political communication. Matthew W. Ragas is the corresponding author and can
be contacted at: mragas@depaul.edu
Alexander V. Laskin, PhD, is an Associate Professor and Director of Graduate Studies in the
Department of Public Relations at Quinnipiac University, USA. Dr Laskin is an author of over 20
scholarly publications with a predominant focus on investor relations, measurement and 181
evaluation, international communications, and new media.

To purchase reprints of this article please e-mail: reprints@emeraldinsight.com

Or visit our web site for further details: www.emeraldinsight.com/reprints