A Focus on Evidence-Based Management

Leading scholars have promoted EBM as an integrated and

comprehensive approach to help bridge the rigor-relevance
gap and improve the management of organizations
Denise M. Rousseaus 2005 Presidential Address to the
Academy of Management
Jeffrey Pfeffers 2007 Keynote Address to the Society for
Industrial and Organizational Psychology
2007 Conference Board Report, Evidence-Based Human
Resources: A Primer and Summary of Current Literature.

Human Resources, Law and Management Department - September 2009

The Origins: Evidence-Based Medicine

the conscientious, explicit and judicious use

of current best medical/clinical research
evidence in making decisions about the care
of individual patients.
Dr. David Sackett (McMaster University)

Human Resources, Law and Management Department - September 2009

Evidence-Based Management
Ignorance is arguably higher in management than in
medicine
If doctors practiced medicine like many companies
practice management, there would be more
unnecessarily sick or dead patients and many more
doctors in jail or suffering other penalties for
malpractice
(Pfeffer & Sutton, 2006, p1)

Pfeffer, J. & Sutton, R. I. (2006). Evidence-based management. Harvard Business Review, January, pp1-12.
Confer also Rousseau, D. M. (2006). Is there such a thing as "evidence-based management"? Academy of Management Review. 31, pp 256-269
Human Resources, Law and Management Department - September 2009

What is Evidence-Based Management?

A way of thinking about organizational decisions in a
systematic manner
Being committed to doing things based critically on the best
available scientific evidence
Not solely based upon unsystematic experience, instincts,
intuition, tradition, untested beliefs, trends and fashions etc.
Translating the best scientific evidence into organizational
practices (Big E Evidence)
Systematically gathering local data to inform decision-making
(little e evidence)
Learning about human behavior within organizations and using
that knowledge to enhance managerial practice.

Human Resources, Law and Management Department - September 2009

What is Evidence-Based Management?

Evidence-based management is about making decisions through the
conscientious, explicit, and judicious use of 4 sources of information:

practitioner expertise and judgment

evidence from the local context
a critical evaluation of the best available research evidence
the perspectives of those people who might be affected by the decision

Briner, R. B., Denyer, D., & Rousseau, D. M. (2009). Evidence-Based Management: Concept Cleanup Time? Academy of Management
Perspectives. 23, pp19-32.

Using evidence does not mean slavishly following it, acting only when
there is good evidence, or doing nothing if there is none.
Briner, R. B. and Rousseau, D. M. (2011). Evidence-Based IO Psychology: Not There Yet. Industrial and Organizational Psychology.4,
pp322.

Human Resources, Law and Management Department - September 2009

What EBM is !
Something managers and practitioners do
Something practitioners already do to some extent
About the practice of management
A family of related approaches to decision making
A way of thinking about how to make decisions
About using a wide range of different kinds of research evidence
depending on the problem
Practitioners using research as only one source of information to inform
decisions
A means of getting existing management research out to practitioners
Likely to help both the process and outcome of practitioner decisionmaking
About questioning ideas such as best practice
Human Resources, Law and Management Department - September 2009

What EBM is NOT!

1.
2.
3.
4.
5.
6.

NOT Something management scholars do

NOT A brand-new way of making decisions
NOT About conducting particular types of research
NOT A single decision-making method
NOT A rigid, one-size-fits all decision-making formula
NOT About using only certain types of research evidence
irrespective of the problem
7. NOT Scholars or researchers telling practitioners what to do
8. NOT About conducting research only about management practice
9. NOT The solution to all management problems
10. NOT About identifying and promoting best practice
Human Resources, Law and Management Department - September 2009

EBM QUIZ
Rynes, S. L., Colbert, A. E., & Brown, K. G. (2002). HR professionals beliefs about effective human resource practices: Correspondence between research and practice. Human Resources Management,
41(2), pp149-174.
Rynes, S. L., Giluk, T. L., & Brown, K. G. (2007). The very separate worlds of academic and practitioner periodicals in human resource management: Implications for evidence-based management.
Academy of Management Journal, 50(5), pp987-1008.

TRUE or FALSE?
General mental ability is the strongest, or one of the strongest, predictors
of job performance
TRUE
Setting goals and providing feedback is a highly effective motivational
practice
TRUE
Unstructured employment interviews are more valid than structured
interviews
FALSE
Personality is related to job performance
TRUE
Companies with vision statements perform better than those without
TRUE
Human Resources, Law and Management Department - September 2009

EBM QUIZ
TRUE or FALSE?
On average, encouraging employees to participate in decision making is more
effective for improving organizational performance than is setting performance goals
FALSE
Most people under-evaluate how well they perform on the job
FALSE
Training for simple skills will be more effective if it is presented in one concentrated
session than if it is presented in several sessions over time
FALSE
Integrity tests designed to predict whether someone will steal, be absent, etc. can
predict those types of counterproductive behaviors
TRUE
Graphology (handwriting analysis) is able to predict job performance effectively
because it is difficult to fake
FALSE
Human Resources, Law and Management Department - September 2009

Theory?

An organized framework of concepts aimed at illustrating, interpreting and/or

predicting reality

It is the theory that decides (determines) what we can observe

A Frame or an Abstraction of Reality Albert Einstein (via Gregory, 1990)

There is nothing more practical than a good theory Kurt Lewin (1951,p169)
No theory is going to be inviolate. Let me put it clearly. The only kind of theory
that can be proposed and ever will be proposed that absolutely will remain
inviolate for decades, certainly centuries, is a theory that is not testable. If a
theory is at all testable, it will not remain unchanged. It has to change. All
theories are wrong. One doesnt ask about theories, can I show that they are
wrong or can I show that they are right, but rather one asks, how much of the
empirical realm can it handle and how must it be modified and changed as it
matures Leon Festinger (1987)
Human Resources, Law and Management Department - September 2009

Independent Variable (Predictor) &

Dependent Variable (Criterion)
a theoretical model establishes a causal relationship between 2
variables which is then tested in research studies
the independent variable (IV) can be either qualitative or
quantitative and is typically the variable being manipulated or
changed in the study What is being changed
the dependent variable (DP) can be either qualitative or
quantitative and is the observed result of the independent
variable being manipulated or changed What is being
observed?

Be aware that finding a relation between the IV & the DP doesn't

establish the direction of causality

Causality or which is the cause and which is the effect can only
be established by the type of the research design
Human Resources, Law and Management Department - September 2009

2 Types of Research Design

Concurrent or Cross-Sectional Research Design: both the independent
and the dependent variable are measured in the same point in time
(roughly simultaneously) A Cross-Section in the Time-Continuum
Predictive and/or Longitudinal Research Design: the independent
variable is measured at an earlier point in time than the dependent
variable and/or repeatedly for a long time period

Only Predictive and/or Longitudinal Research Designs can establish

Causality and consequently they are considered scientifically superior

However in the position of IV you ought to find that variable which is at

least theoretically assumed to be the cause of an effect in the DP, with
the former being treated as the predictor and the later as the criterion
Human Resources, Law and Management Department - September 2009

Reliability
reliability: the estimated consistency of the measurement for a given
instrument/method (e.g. test) employed with itself (internal validity),
across time (test-retest) and within/across raters (intra/inter-rater)
the repeatability of the measurement

the level of reliability puts an upper limit to the possible level of validity
high reliability is necessary but not sufficient for high validity
internal reliability: coefficient alpha 0.80(at least 0.70) for a measure
to have an acceptable degree of reliability
Inter-rater agreement: minimum level 75% or higher
test-retest reliability: concerned with time stability of measurement;
level of r should range between r = .50 to .90
Intra-rater agreement: for short time intervals between measures, a
fairly high relationship is expected; r = .80 to .90

Human Resources, Law and Management Department - September 2009

Validity
conclusion or criterion related validity: the degree to which the
independent variable measures or is correlated to the dependent
variable (correlation coefficient: r)

concurrent validity under a concurrent or cross-sectional research

design
predictive validity under a predictive research design
the square of the correlation coefficient states the amount of variance
in the DV explained by the IV
internal validity: an overall inductive estimation of the degree of
established causality between IV & DV
external validity: an estimation of the degree of generalizability of our
conclusions at the population level

Human Resources, Law and Management Department - September 2009

Intervening Variables: Moderator

intervening variables: variables which interact with the IV
affecting its relationship with the DV and which when
incorporated in our models upgrade theoretical rigor

moderator: a variable that interacts with the IV affecting

the direction (positive or negative) and/or strength of its
relation with the DV

a type of multiplicative statistical interaction

be aware that the moderator stands at the same point of causal

precedence or supremacy with the IV

the moderator can be either a qualitative (e.g. gender, race) or a

quantitative (e.g. amount of pay) variable

Human Resources, Law and Management Department - September 2009

Intervening Variables: Mediator

mediator: a variable through which the IV relates with the
DV either partially or fully

a mediation is not a statistical interaction

when the IV is related to the DV only through the mediator then we

have a full mediation

the mediator is causally inferior to the IV but causally superior to the

DV

a mediator can be either a qualitative or a quantitative variable

when the independent variable is related to the DV both indirectly
through the mediator and directly we have a partial mediation

Human Resources, Law and Management Department - September 2009

Control Variables
control variables: variables known from research to be having an
effect on the DV and for that reason are kept constant to prevent
their influence on the effect of the IV on the DV alternative
causes (predictors) of the dependent variable (criterion)
What other variables have to be held constant in order to isolate
the net effect of the IV to the DV?

The more control variables employed in a study the more

rigorous its methodologyand thus the more we can rely in its
conclusions

Human Resources, Law and Management Department - September 2009