Framework to Monitor and Evaluate

the Outcomes of the Introduction of

Domestic Violence Clauses

For the Australian Domestic and Family Violence
Clearinghouse







Social Policy Research Centre

University of New South Wales
May 2011



DRAFT Framework to Monitor and Evaluate the Outcomes of the Introduction of Domestic Violence Clauses
Social Policy Research Centre i
Contents
1 Introduction ......................................................................................................... 1
2 Evaluation framework and program logic ........................................................ 1
3 Data sources ......................................................................................................... 3
3.1 Existing data sources ............................................................................................... 4
4 Project-specific evaluation data ......................................................................... 5
4.1 Establishment of baseline data and instrument development .................................. 5
4.2 Workplace survey .................................................................................................... 6
4.3 Union survey ........................................................................................................... 7
5 Scope and Sampling ............................................................................................ 7
References ...................................................................................................................... 8


DRAFT Framework to Monitor and Evaluate the Outcomes of the Introduction of Domestic Violence Clauses
Social Policy Research Centre 1
1 Introduction
The Domestic Violence Workplace Rights and Entitlements project is being
conducted by the Australian Domestic and Family Violence Clearinghouse and
funded by the Commonwealth Department of Education, Employment and Workplace
Relations (DEEWR).
The goal of the project, which runs from July 2010 to December 2011, is to reduce the
impact of family violence on working women by achieving better workplace rights
that support them to stay safely in their jobs and in their homes. This is to be achieved
by:
Briefing a broad range of unions and employers nationally on the issue of family
violence as a workplace issue, the adoption of family violence provisions in
enterprise agreements and other workplace instruments
Developing with unions and employers a set of model workplace information and
training resources for general staff, human resources personnel, union delegates
and supervisors.
Producing model policies and safety plans to assist in the informed introduction of
family violence clauses.
Surveying union members to provide essential information on the impact of
family violence, and enabling future monitoring of the outcomes of introducing
family violence clauses and other instruments.
The first Australian family violence clauses have been incorporated into the enterprise
agreement between the Surf Coast Shire Council (Victoria) and the Australian
Services Union, Victorian Authorities and Services Branch (September 2010)
providing worlds best practice with up to twenty day extra paid leave for reasons of
family violence.
This document proposes an evaluation and monitoring framework for the introduction
of domestic violence clauses into industrial awards and agreements and the Domestic
Violence Workplace Rights and Entitlements project.
2 Evaluation framework and program logic
A program logic is a systematic, visual representation of the underlying assumptions
of a planned program. It illustrates why and how a program is presumed to work
(AIFS and SPRC, 2010).
Figure 2.1 is a program logic for the implementation and impact of the introduction of
Domestic Violence clauses into industrial agreements, based on the anticipated
activities, and short-, medium- and long-term outcomes.
The model should be read from bottom to top. Inputs are at the bottom. These are
anticipated to drive changes in the content of industrial agreements, and in the
development of workplace initiatives including training. These changes are
anticipated in turn to lead to improvements in key measures related to domestic and
family violence.
DRAFT Framework to Monitor and Evaluate the Outcomes of the Introduction of Domestic Violence Clauses
Social Policy Research Centre 2
A program logic sets out the hypothesised results of an intervention, and evaluation
and monitoring activities should ascertain whether or not they occurred. This requires
measuring if and to what extent changes have occurred as anticipated. Possible
measures and data sources for monitoring whether the project has been implemented
as intended (inputs) are identified in Table 2.1. No one data set is available to monitor
the outcomes of the project. The ABS Conceptual Framework for Measuring Family
and Domestic Violence (2009) identifies a range of data and definitional needs that
are require to improve data on victims, perpetrators and responses. Possible measures
and data sources for monitoring whether the anticipated outcomes have occurred are
identified in Table 2.2.
It should be noted that evaluation frameworks such as this one alone are insufficient
for the attribution of any changes to the introduction of the clauses and associated
initiatives. The relationship between family and domestic violence prevalence and
reporting is generally agreed to be characterised by under-reporting and under-
recording. The prevalence of family and domestic violence is therefore likely to be
much higher than reported rates, and changes in reported prevalence could mean
improvements to reporting, or actual changes in prevalence, or both. Moreover, even
if it were possible to be confident that the anticipated outcomes had really happened,
this framework does not allow for any calculation of the impact of the project on these
changes. It is possible that other industrial, social, economic or cultural changes could
bring about changes to workplace cultures and the prevalence of domestic violence;
and this framework cannot disentangle those changes from the impact of the project.
Notwithstanding these limitations, the framework should provide a robust, relatively
straightforward, mechanism for monitoring the implementation of the clauses.

DRAFT Framework to Monitor and Evaluate the Outcomes of the Introduction of Domestic Violence Clauses
Social Policy Research Centre 3
Figure 2.1: A program logic for evaluating the implementation and impact of Domestic Violence clauses into industrial agreements.
Incorporation of domestic violence clauses
into industrial instruments.
Introduction of training and other significant
workplace initiatives related to implementing the
clauses.
Uptake of entitlements and related
safety policies and procedures in
workplaces where these are
available, by employees
experiencing domestic violence
1. A reported reduction in the rates and types of domestic violence
experienced by workers, impacting on their work performance, and
manifesting in the workplace (e.g. abusive phone calls/e-mails; performance
issues).
2. Reduction in adverse action experienced by women at work (loss of job,
performance management, disciplinary action) as a result of domestic
violence.
3. The effectiveness of safety plans and measures such as telephone and e-
mail screening.
4.The uptake of protection orders and their effectiveness.
5. Increased activity by the workplace on community related domestic violence
prevention.
6. Development of workplace cultures which are supportive of employees
experiencing domestic violence.
Reform area
Medium term
outcomes
Long term
outcomes
1. Development of model domestic violence entitlements
clauses appropriate for inclusion in enterprise agreements
2. Meeting with and presenting issues relating to the impact of
domestic violence on the workplace and the use of the clauses
to minimise this to interested unions and employers across
the country.
3. Development of model policies and procedures to assist
workplaces operationalise domestic violence clauses
effectively
1. Development of fact sheets with information about domestic
violence and its impact on employees, employers and union
representatives
2. Development of a generic training module which can be
customised to particular employers
3. Customising the training module for the partner unions and
employers
4. Development of a website providing information and
resources for employers (including model domestic violence
clauses), and an online training tool
Inputs
Incorporation of domestic
violence clauses into industrial
instruments.
Introduction of training and
other significant workplace
initiatives related to
implementing the clauses.
Short term
outcomes

DRAFT Framework to Monitor and Evaluate the Outcomes of the Introduction of Domestic Violence Clauses
Social Policy Research Centre 1
Table 2.1: Possible measures and data sources: inputs
Measure Source
Clauses
Development of model domestic
violence entitlements clauses
appropriate for inclusion in
enterprise agreements
Number of clauses
developed
ADFCV
Meeting with and presenting issues
relating to the impact of domestic
violence on the workplace and the
use of the clauses to minimise this to
interested unions and employers
across the country.
Number of meetings,
workshops, forums
attended
ADFCV
Development of model policies and
procedures to assist workplaces
operationalise domestic violence
clauses effectively
Number of policies and
procedures developed,
distributed
ADFCV
Training
Development of fact sheets with
information about domestic
violence and its impact on
employees, employers and union
representatives
Fact sheets developed ADFCV
Development of a generic training
module which can be customised to
particular employers
Training models
developed
ADFCV
Customising the training module for
the partner unions and employers
Number of customised
training models
ADFCV
Development of a website
providing information and
resources for employers (including
model domestic violence clauses),
and an online training tool
Website developed ADFCV

DRAFT Framework to Monitor and Evaluate the Outcomes of the Introduction of Domestic Violence Clauses
Social Policy Research Centre 2
Table 2.2: Possible measures and data sources: short-, medium and long-term outcomes
Outcome Measure Source
Short term
Incorporation of domestic violence
clauses into industrial instruments.
Number of negotiations
commenced to insert clauses
into industrial instruments
Number of industrial
instruments which
incorporate clauses
Union survey
(possible long-term
source: ABS
SEARS/TUS)
Introduction of training and other
significant workplace initiatives related
to implementing the clauses.
Number of workplaces
which introduce training and
other initiatives
Take-up and effectiveness of
training
Workplace survey
Union survey

Medium term
Uptake of entitlements and related
safety policies and procedures in
workplaces where these are available by
employees experiencing domestic
violence.
Rates of leave and other
entitlements used


Workplace survey
(possible long-
term source: ABS
SEARS/TUS)
Long term
A reported reduction in the rates and
types of domestic violence experienced
by workers, impacting on their work
performance, and manifesting in the
workplace (e.g. abusive phone calls and
e-mails; performance issues).
Number of workers
reporting domestic violence
at work
Reported impact of domestic
violence on work
performance and workplace

Workplace survey
Union survey

Reduction in adverse action
experienced by women at work (loss of
job, performance management,
disciplinary action) as a result of
domestic violence.
Reported impact of domestic
violence on work
performance and workplace

Focus groups
Workplace survey
Union survey

The effectiveness of safety plans and
measures such as telephone and e-mail
screening.
Experience and perceived
impact of safety plans
Satisfaction ratings
Focus groups
Workplace survey
Union survey

The uptake of protection orders and
their effectiveness.
Number of protection
orders
Outcome of protection
orders
Specific research
required. Victoria is
the only
jurisdiction with a
family violence
database.
Individual
jurisdictions have
some data available
on protection
orders, (e.g. NSW
BOCSAR) but this
DRAFT Framework to Monitor and Evaluate the Outcomes of the Introduction of Domestic Violence Clauses
Social Policy Research Centre 3
would require
further analysis.
Most research on
protection orders is
conducted via case
analysis or
recruitment from
court hearings.
Increased activity by the workplace on
community related domestic violence
prevention.
Focus groups
Workplace survey
Union survey

Development of workplace cultures
which are supportive of employees
experiencing domestic violence
Focus groups
Workplace survey
Union survey


3 Data sources
Based on the program logic, which indicates that the anticipated outcomes of the clauses are
short-term, medium-term and long-term, a mixed method approach to data collection and
analysis is suggested. This should include:
Exploitation of data already routinely collected
Modifying and expanding existing instruments used for routine data collection
Collecting and analysing project-specific data on implementation and impact
For most evaluations, these options present a trade-off between precision and efficiency. In
general, analysis of data that collected by a third-party agency for different purposes, for
example ABS survey or census data, has fewer costs than the project or evaluation
researchers collecting the data themselves. This is also true of analysis of administrative data
that projects collect in the course of doing business: for example, number of staff employed,
number and type of services delivered. The disadvantage of this first approach, however, is
that these data sources can only provide, in most cases, proxies for what the evaluation is
actually trying to measure, and proxies vary in how well they fit the intended outcome.
The second method involves adding items to existing surveys and reports. This process
involves identifying existing instruments that could be adapted, and negotiating with the
agency responsible to (normally) add new questions. This can be done with large-scale
surveys such as ABS surveys, and with routinely collected project data. The advantage of this
method is that it combines better efficiency and precision than simply working with existing
data sources, with fewer costs than a totally new survey. However, it is dependent both on the
identification of surveys that could be augmented to fit the evaluation, and on the capacity
and willingness of the relevant agencies to make these changes.
A third method is for projectsor independent evaluatorsto design and implement
evaluation instruments themselves, and ensure that these instruments collect data that does fit
very closely with what the evaluation is trying to measure. However, this demands resources,
from design to implantation to analysis of the evaluation instruments, and cannot be used for
DRAFT Framework to Monitor and Evaluate the Outcomes of the Introduction of Domestic Violence Clauses
Social Policy Research Centre 4
ongoing evaluations with confidence, unless it is likely that the resources will also be
ongoing.
Using a combination of methods may therefore address the disadvantages, and utilise the
benefits, of each. The following section describes possible existing data sources that may be
used and/or augmented, and options for developing project-specific instruments.
3.1 Existing data sources
ABS SEARS and TUS
The ABS Survey of Employment Arrangements, Retirement and Superannuation (SEARS) is
conducted every six years. The next survey will be in 2013 (see box). It collects detailed
information about:
the diversity of employment arrangements in Australia, including the types of
employment people have, aspects of job stability and flexibility, and working patterns;
the working arrangements that people use, or would like to use to balance their work and
caring responsibilities;
plans that people aged 45 years and over have for retirement, including transitions to
retirement, expected sources of income at retirement and factors that might influence the
decision to retire;
characteristics of retirement including age at retirement, reasons for retiring and
retirement income; and
the superannuation coverage of individuals, including superannuation contributions and
account balances.
The Australian Bureau of Statistics (ABS) conducted the Survey of Employment
Arrangements, Retirement and Superannuation (SEARS) in 2007 and the Time Use Survey
(TUS) was last conducted in 2006. SEARS 2007 and TUS 2006 will be combined and
components updated to form SEARS/TUS 2013. SEARS/TUS 2013 will be comprised of a
SEARS questionnaire to be completed by the entire sample of households, and a Time Use
module and Time Use Diary to be completed by a proportion of households within that
sample.
SEARS/TUS 2013 has a target sample size of 13,750 households from all States and
Territories. Information will be collected via Computer Assisted Personal Interview (CAPI).
The Time Use component has a target sample size of 4,000 responding households within
the overall sample. Households in the TUS subsample will also be required to complete a
time use module and fill in a Time Use Diary in which they will record their activity over a 48
hour period.
(From the ABS Discussion Paper: Development of the 2013 Survey of Employment
Arrangements, Retirement and Superannuation and Time Use Survey)

Items in the survey relevant to monitoring the impact of the Domestic Violence clauses
include employment arrangements and work, caring and family balance.
DRAFT Framework to Monitor and Evaluate the Outcomes of the Introduction of Domestic Violence Clauses
Social Policy Research Centre 5
Whether entitled to paid sick leave in job
Whether entitled to paid sick leave in job
Whether entitled to paid holiday leave in job
Leave entitlements in job
Whether entitled to paid carers leave in job
Whether had time off work to provide care last week
Time absent from work due to providing care
Whether used working arrangements to provide care last week
Type of working arrangements used to provide care last week
Whether wanted to make more use of working arrangements to care for someone
Type of working arrangements wanted to make more use of
Main reason did not make more use of working arrangements to care for someone
Whether would like to have made use of working arrangement to care but didn't
Type of working arrangement would like to have used but did not
Main reason did not make use of working arrangement would like to have used
Ideally, a question specifically on access to domestic violence workplace entitlements could
be added to this survey.
4 Project-specific evaluation data
4.1 Establishment of baseline data and instrument development
The development of effective instruments to monitor the effects of the introduction of the
clauses, their effects, and their interactions with other workplace initiatives, should be
established by the use of broad-ranging, qualitative research. This type of research cannot
provide baseline data on either the prevalence and workplace effects of domestic violence, or
the take-up of domestic violence clauses in industrial agreements (no one data source has
been identified that could provide this quantitative data). However, given that the anticipated
outcomes do not relate only to the introduction of the clauses, but also their role in changing
workplace cultures, broader outcome measures are needed.
Research indicates that the relationship between family violence and supportive workplaces
is complex. Disclosure by victims of family violence to supervisors can be beneficial, but
different kinds of support are desired and warranted (Yragui et al., 2011). For example, has
been hypothesised that women who are closer to ending an abusive relationship will want
more comprehensive assistance from workplaces, whereas those in a pre-contemplation
phase will be interested in specific, limited and confidential support (Perrin et al., 2010). The
introduction of policies and initiatives to encourage disclosure, and support victims, can have
unanticipated consequences (O'Leary-Kelly et al., 2008). The impact of the introduction of
clauses into industrial agreements will vary between individual workplaces, and be affected
by the existing workplace culture and systems already in place to protect safety and
confidentiality (for example: screening of phone calls, security systems and measures, and
employee assistance programs) (Swanberg et al., 2006; Swanberg and Ojha, 2010).
DRAFT Framework to Monitor and Evaluate the Outcomes of the Introduction of Domestic Violence Clauses
Social Policy Research Centre 6
Therefore, in addition to monitoring anticipated changes, the evaluation should also attempt
to identify unanticipated consequences, both positive and negative. Qualitative approaches
are most appropriate, and could include open-ended questions on surveys (see below) and
periodic focus groups and interviews with employers, employees and unions.
4.2 Focus groups and interviews
Focus groups with key stakeholders should be conducted prior to the development of survey
instruments. As an alternative to focus groups, or as an additional method, individual
interviews (face-to-face, in person, email or phone) could also be used. These baseline
activities will be important to:
Identify workplace- and industry-specific issues likely to affect implementation or
outcomes, so that these can be included in the surveys
Identify common concerns and anticipated benefits, so that these can be included in the
surveys
Gather baseline information on existing safety plans, workplace cultures and community
activity around domestic violence.
Focus groups should then be conducted throughout the implementation of the project to
identify unanticipated outcomes, positive and negative, and the interaction between different
elements. Ideally, the second round of focus groups should be held around 12 months from
commencement. The focus groups should also gather qualitative information on:
Experiences of negotiating the insertion of clauses into industrial agreement
Factors which facilitate and hinder introduction of clauses into industrial agreements
Perceived impact of safety plans, and barriers and facilitators to implementation
Perceptions of workplace culture and changes over time as a result of the project
Facilitators and barriers training, awareness programs and other initiatives (employer- and
union-initiated
Perceived effectiveness of workplace activity on community related domestic violence
prevention
4.3 Workplace survey
Online surveys for employees and employers should be administered to monitor changes in
key outcome areas, and include items on the following topics:
Knowledge of training, awareness programs and other initiatives
Take-up and effectiveness of training (satisfaction ratings, qualitative assessment)
Access to and take-up of leave (DV, carers, unpaid etc)
Incidence of experiencing family and domestic violence at work
Reported impact of domestic violence on work performance and workplace
Knowledge of safety plans
Perceived impact of safety plans (satisfaction ratings, qualitative assessment)
DRAFT Framework to Monitor and Evaluate the Outcomes of the Introduction of Domestic Violence Clauses
Social Policy Research Centre 7
Knowledge and perceived effectiveness of workplace activity on community related
domestic violence prevention
Perceptions of workplace culture
4.4 Union survey
Online surveys for union stakeholders (local delegates and state/national offices) should also
be administered, to monitor changes in key outcome areas, and include items on the
following topics:
Introduction of clauses into industrial agreements
Factors which facilitate and hinder introduction of clauses into industrial agreements
Facilitators and barriers training, awareness programs and other initiatives (employer- and
union-initiated)
Disclosure and reporting of family and domestic violence at work
Reported impact of domestic violence on work performance and workplace
Introduction and perceived effectiveness of safety plans
Knowledge and perceived effectiveness of workplace activity on community related
domestic violence prevention
5 Scope and Sampling
Ideally, surveys would be administered in all workplaces (a census), but resource and other
constraints will likely make this impracticable. An alternative evaluation strategy is to
administer surveys in a selection of workplaces (a sample survey), and conduct qualitative
research with all or some of those workplaces participating in the sample survey. Six unions,
representing large memberships, are participating in the survey being conducted for the
Domestic Violence Workplace Rights and Entitlements project, and should be invited to
participate in later surveys and qualitative data collection. The sampling strategy should be
designed to ensure that a broad representation of workplace characteristics are covered, as
well as other key variables (jurisdiction, unionisation rates, risk factors for family violence
victimisation etc).
DRAFT Framework to Monitor and Evaluate the Outcomes of the Introduction of Domestic Violence Clauses
Social Policy Research Centre 8
References
Australian Bureau of Statistics (ABS) (2009), Conceptual Framework for Measuring Family
and Domestic Violence, Cat. No 4529.0, Australian Bureau of Statistics, Canberra.
Australian Institute of Family Studies (AIFS) and Social Policy Research Centre (SPRC)
(2010), Keep Them Safe: Evaluation framework, Prepared for the NSW Department
of Premier and Cabinet. Unpublished, Sydney.
O'Leary-Kelly, A., Lean, E., Reeves, C. and Randel, J. (2008), 'Coming into the light:
Intimate partner violence and its effects at work', The Academy of Management
Perspectives (formerly The Academy of Management Executive)(AMP), 22(2), 57-72.
Perrin, N. A., Yragui, N. L., Hanson, G. C. and Glass, N. (2010). Patterns of Workplace
Supervisor Support Desired by Abused Women [electronic version]. Journal of
Interpersonal Violence. Retrieved from
http://jiv.sagepub.com/content/early/2010/09/17/0886260510383025.abstract. DOI:
10.1177/0886260510383025
Swanberg, J. E., Caroline Macke and Logan, T. (2006), ' Intimate Partner Violence, Women,
and Work: Coping on the Job. Violence and Victims', Violence and Victims,, 21(5),
561-578.
Swanberg, J. E. and Ojha, M., U. (2010), State Employment Protection Statutes for Victims of
Domestic Violence as an Employment Matter, University of Kentucky: Centre for
Poverty Research, Lexington. http://www.ukcpr.org/Publications/DP2010-11.pdf.
Yragui, N., Mankowski, E., Perrin, N. and Glass, N. (2011), 'Dimensions of Support Among
Abused Women in the Workplace', American Journal of Community Psychology, 1-
12.