- a new way to Design and to Evaluate Occupational Health and Safety Interventions

Louise M. Pedersen, MSc, ph.d.-fellow Department of Sociology and Social Work, University of Aalborg, Denmark EAWOP Small Group Meeting, 24th May 2012, Nottingham, United Kingdom

Realistic Evaluation

Companies involved in my PhD

Controlled design
Company A (intervention):

Company B (control):

Lost Time Injury rate: 15 High level of technical prevention 125 production employees 3 production managers 4 team leaders 2 serious accidents during the Summer 2008 plus minor incidents afterwards => are highly motivated for the project

Lost Time Injury rate: 3 High level of technical prevention 105 production employees 2 production managers (+1) 1 manager assistent Very few accidents since 2000 and downsized with 15 production employees => are less motivated for the project

Importance of qualitative data to measure cultural aspects

Production worker:
In my group you are considered to be a chicken if you report a near-miss accident.

But you are a man if you go to work with a broken leg.

4 www.regionmidtjylland.dk

Realistic evaluation

Pawson and Tilley (1997): Realistic evaluation Key question: What works for whom, when, under what circumstances in what respects and how? Incorporates context in reviews within safety science while still adhering to the basic principles of the Cochrane criteria. Instead of controlling for real-world phenomena such as change and diversity, it tries to learn from them.

The realistic evaluation circle

Program theory (CMO) Context (C) Mechanisms (M) Outcome (O) Program specification What works for whom, when and under what circumstances ? Observations Multiple methods, Analysis of Context, Mechanisms and output/outcome Hypotheses What is expected to work for whom and under what circumstances?

Pawson og Tilley 1997: 85

CMO-figurations
Context

Mechanisms

X Intervention

Y Output/ outcome

R.E. applied to health and safety science

Context Formal structures with substantial influence on outcome, i.e. global (finansial crises), national (laws), and local (level of health and/or safety at baseline, organizational change, production pressure) Mechanisms Personal characteristics of or interpersonal relations between key actors which are expected to affect outcome.

Output

Positive or negative, expected and unexpected immediate intervention results. Traditionally named Y.
All the effects the end-user experiences caused by the intervention or other factors

Outcome

Advantages of the realistic evaluation model

Process evaluation and effect evaluation in one analysis Causal relationships in a social-constructivistic sence Systematic inclusion of context and mechanisms in the program theory Distinction between implementation failure and theory failure Quantitative AND qualitative data Interaction between researchers and practit ioners Learn from intervention success AND failures
Louise M. Pedersen, 2012 9

Advantages of the REVISED realistic evaluation model

Additional

Clear definitions of context and mechanisms Examples of context and mechanisms which can be included in occupational safety interventions

E.g. organizational changes, production pressure,

and informal rules and norms, motivation

Possibility to include control(s) Cross sectional or longitudinal design Distinction between output and outcome measures
Louise M. Pedersen, 2012 10

Questions

Distinction between output and outcome? Output measures based on qualitative data? Link between data on intervention processes and output/outcome? When are the controls too different to be used for comparison?

Louise M. Pedersen, 2012

11

Questions?
Louise Mller Pedersen E-mail: Lmpd@id.aau.dk
http://www.linkedin.com/pub/ louise-m-pedersen/5/407/ab7

Pedersen, L.M., Nielsen, K.J. & Kines, P. (2012): Realistic evaluation as a new way to design and evaluate occupational safety interventions. Safety Science, Vol. 50, Nr. 1, p. 48-54.
Louise M. Pedersen, 2012 12