Training Effectiveness and Transfer: A Mixed Methods Investigation

Dr Mary C. Cowman
Department of Health, Sport and Exercise Science
Waterford Institute of Technology
Waterford
IRELAND
Email: mcowman@wit.ie





Dr Alma M. McCarthy
Department of Management
J.E Cairnes Graduate School of Business & Public Policy
National University of Ireland, Galway
Galway
IRELAND
Email: alma.mccarthy@nuigalway.ie


Stream: HRD, Evaluation and Learning











Abstract

The effectiveness of workplace training and development is largely determined by
the extent to which training transfer occurs and is sustained over time. However,
there are gaps in our understanding of the training transfer process (Burke and
Hutchins, 2008; Holton, Chen and Naquin, 2003; Martin, 2010; Spitzer, 2005). Using
both quantitative and qualitative data, this paper investigates the impact of a training
and development intervention at individual and organisational level in a health care
context in Ireland. Furthermore, it examines organisational system factors affecting
training transfer post training.





Keywords

Training Transfer; evaluation; training effectiveness; training impact;, mixed
methods; organisational system factors
Training Effectiveness and Transfer: A Mixed Methods Investigation

1.0 Introduction

The effectiveness of workplace training and development is largely determined by
the extent to which training transfer occurs and is sustained over time. However,
there are gaps in our undertanding of the training transfer process (Burke and
Hutchins, 2008; Holton, Chen and Naquin, 2003; Martin, 2010; Spitzer, 2005).
Ruona, Leimbach, Holton and Bates (2002) suggest that methods to evaluate the
impact of training are inadequate for answering many of the questions regarding the
effectiveness of training and development efforts. These factors have implications
for the extent to which training transfer research is contributing to our understanding
of the process of training transfer and effectiveness. Using both quantitative and
qualitative data, this paper investigates the impact of a training and development
intervention at individual and organisational level in a health care context in Ireland.
Furthermore, it examines organisational system factors affecting training transfer
post training.

2.0 Training Evaluation and Effectiveness

Measuring and evaluating the effectiveness of training and development
programmes is one of the most critical components of SHRD (Horwitz 1999).
However, according to Hutchins and Burke (2007), studies have estimated that 85%
of training resources are dedicated to designing and delivering training with the
remaining 15% divided between front-end analysis and evaluation activities. Where
measurement of training effectiveness is taking place, Spitzer (2005) argues that
HRD is failing to demonstrate that investment in training and development
programmes is producing results at organisational level.

Defining training effectiveness is complex which has implications for the
development of strategies to measure training effectiveness. Alvarez, Salas and
Garofano (2004) distinguish between the term training evaluation and training
effectiveness. The former is described as a measurement technique to determine if
training goals have been met. The latter is a theoretical approach used to study the
individual, training and organisational variables that are likely to influence training
outcomes.

Previous training evaluation studies have been criticised for the lack of information
provided on the impact of the training intervention at both the individual (level three)
and the organisational levels (level four) (Cheng & Ho 2001), for the failure to identify
sub-categories of learning (Alliger et al. 1997) or, to measure the horizontal impact
of training at individual level (Kearns 2005) such as the impact on the team or the
unit. Wang and Sun (2009) suggest that the purpose of investing in employees is to
enhance their current and future productivity for the organisation. For example, Mayo
(2000) suggests that HRD can contribute to employee added value through
increased motivation, commitment, efficiency and competence.


Techniques for measuring training outcomes proposed by Kearns (2005) and
Sptizers (2005) are reflective of a training effectiveness rather than a training
evaluation approach to investigating the impact of training interventions. These
arguments demonstrate the importance of adopting an approach for measuring
training effectiveness that is capable of establishing different forms of training
transfer, the impact of the training intervention at different levels of investigation and
the manner in which different factors have facilitated or inhibited the transfer
process. According to Wang and Spitzer (2005), the most recent stage in the
evolution of measurement and evaluation is characterised by research oriented,
practice based comprehensive methodologies which are rooted in existing theories
and which seek to develop more comprehensive, robust evaluation techniques in line
with the effectiveness approach. Tan, Hall and Boyce (2003) argue that the most
developed and used models focus on defining different training effectiveness criteria
and their organisational implications which reflect the definition of training
effectiveness proposed by Alvarez et al (2004).

Using a mixed methods approach, this paper investigates training effectiveness in
the healthcare sector in Ireland. Mixed methods research (the combined use of
quantitative and qualitative methods in the same study) is becoming an increasingly
popular approach in the fields of psychology, education, sociology, management and
health sciences (Greene, Caracelli and Graham, 1989; OCathain, 2009). The central
premise for the use of mixed methods is that the use of quantitative and qualitative
approaches in combination provides greater understanding of research problems
than monomethod designs (Cresswell and Plano Clark, 2007; Tashakkori and
Teddlie, 2003). The model developed for the purpose of this study builds on the
evaluation and effectiveness frameworks presented by Rouiller and Goldstein
(1993), Cannon-Bowers, Salas, Tannenbaum and Mathieu (1995), Alvarez et al.
(2004), Holton (1996 and 2000), and Spitzer (2005). In this study, we explore
training transfer as am important dimension of training effectiveness in a health care
sector. The next section discusses the concept of training transfer and how it can be
operationalised.



3.0 Organisation Transfer System and Training Transfer
Transfer of training is the application of learned knowledge, skills and attitudes to the
job and subsequent maintenance over time (Cheng & Ho 2001) for the purposes of
improving the job performance (Velada & Caetano 2007). Geilen (cited in Van der
Klink, Gielen, and Nauta 2001) identified three dimensions of transfer: the direction;
the level of complexity; and the distance. The direction of training transfer refers to
either positive transfer where training leads to desired performance or negative
transfer, where it fails to produce intended job performance. Lateral transfer refers to
the learner being able to achieve a task at the same complexity level as the task
already mastered whereas vertical transfer refers to the ability to apply learning to
similar or more complex skills. Regarding distance, near transfer refers to training in
tasks that are similar or equal to the learners job tasks. Far transfer is where there
is a lack of similarity to the job tasks and training focuses on understanding and the
application of principles or rules. For the purposes of this study, the direction of
training transfer and the influence of organisational factors on the direction of training
transfer were explored as this is the most relevant dimension for the training
intervention under investigation here.

Baldwin and Ford (1988) identified three categories of influences on training transfer;
individual characteristics, training design factors and organisational factors. A review
of literature has demonstrated that research exploring the relationship between
organisational, or the transfer system, factors and training transfer has been
investigated less frequently than training design or indivdual characteristics (Valada
et al. 2007; Saks & Belcourt, 2006; Chen, Holton & Bates 2006). The transfer system
refers to all the person, training, and organisational related factors that have the
potential to influence transfer of learning to job performance (Holton, Bates and
Rouna, 2000). The complex events such as outcomes of a training intervention
cannot be understood by analysing them in isolation because learned skills at the
individual level are embedded in a wider context.
Noe (2000) suggests that an understanding of organisational factors affecting
transfer will make a greater contribution to HRD practitioners wishing to optimise the
effectiveness of training and development programmes. Holton et al. (2000) stated
that an understanding of what constitutes an organizational transfer climate is
unclear and there is no clear consensus on the network of factors affecting transfer
of learning in the workplace. This is evident from the different explanations of the
work environment. Lim and Johnson (2002) however, stated the work environment
factors can be separated into two subcategories: factors that relate to the work
system and people related factors. The following are the organisational transfer
factors investigated in the current study:

3.1 Lim and Johnson (2002) found that the most important organisational system
factor affecting transfer of training at the individual level is supervisor support and
involvement. However, Nijman, Nijhof, Wognum and Veldkamp (2006) argued
that empirical research does not unambiguously confirm this relationship and
provides contradictory results. We explore this factor in the current study.

3.2 Another level of support shown to influence transfer of training is peer
support. Nijman et al. (2006) described peer support as the extent to which peers
behaviour optimises trainees implementation of learning on the job. Peer support
affects motivation to transfer (Noe 1986) and has been shown to predict the
perceived opportunities to use learning (Quinones, Ford, Sego & Smith, 1995).
The health care setting where this study is situated is characterised by staff
shortages and increased workloads which may influence peer support.

3.3 An open communication climate was identified by Lim and Johnson (2002)
as a work system factor found to influence training transfer. The manner in which
health care organization departments and units are organised can add to staff
pressures (Clarke, 2007), which can affect the level of support for training
transfer received. It may influence trainees opportunity to perform or, it may
affect communication between departments (Kupritz, 2002). Therefore, the
variable communication between departments was included in the study.

3.4 According to Holton et al. (1997), opportunity to use is influenced by
organizational factors (such as department goals and values), individual factors
(confidence to use new skills) and contextual factors (pace of work). It is also
influenced by the provision of adequate resources and carrying out tasks that
enable use of new skills. Furthermore, the opportunity to practice skills on
immediate return to job can impact on skill retention.

3.5 Lack of adequate resources and failure to provide staff cover or reduce
workload while attending or implementing training can impact on motivation to
transfer training. Gregoire (cited in Clarke 2002), identified a lack of time and
resources and daily demands of the job as major work-related impediments to the
use of training. Therefore the variable resources were included in the study.

3.6 Huczynski and Lewis (cited in Clarke 2002), found work overload and crisis
work significantly impeded transfer. With regard to task constraints however,
Clarke (2002) stated there have been mixed results in studies. However, this
study was conducted in the health care sector where work overload and crisis
work is a feature and where, according to Clarke (2007) there has been a lack of
transfer research. Consequently, this study included the organisational variable of
competing priorities as part of the investigation of factors affecting transfer.

3.7 The final organisational transfer system variable considered for its influence
on training transfer is trainee level of interest because, according to Elangovan
and Karakowsky (1999) and Santos and Stuart (2003), training transfer will occur
when trainees have both the ability and motivation to acquire and apply new
skills.

In sum, the training evaluation and effectiveness literature indicates that HRD
researchers should engage in more comprehensive studies of the organisational
factors that can affect training transfer to further our understanding of training
effectiveness. This study explores how seven work-related and person-related
organisational transfer system factors impact on training transfer. The study also
explores how training transfer impacts on a number of employee level (job
satisfaction, conference, efficiency) and organisation-level outcomes (perception of
resident outcomes and quality of service delivered). Figure 1 presents an organising
framework for the study.



Figure 1: Organising Framework for the Study





























4.0 Methodology

4.1 Method of inquiry
A cross sectional survey was used to gather data from a range of subjects with
different demographic and situational characteristics. A mixed methods or
triangulation approach was also utilised. Both qualitative and quantitative methods
of inquiry were utilised which involved the use of questionnaires with open and
closed questions and semi structured interviews. Finally, data was gathered from
dual perspectives; participants who took part in the intervention and their
management.

4.2 Training intervention under investigation
The intervention utilised for the purposes of the study was a 60 hour training
programme designed specifically for the heath sector and the subsector (elder care)
being investigated. The Activity in Care Training (ACT) programme provides course
participants with the knowledge and skills to develop chair-based physical activity
programmes appropriate to needs of residents in long stay facilities for older adults,
the setting in which this investigation takes place.
Organisation
transfer system
Work related
(Collective) Level
of interest
Opportunity to use
Competing
priorities
Communication
between
departments
Resources
(management
only)
Person related
Peer support
Supervisor
support



Transfer of training
Delivering
Regular delivery





Outcomes

Employee
Job Satisfaction
Confidence
Efficiency

Organisation
Perceptions of
resident
outcomes
Services offered
Quality
Satisfaction
Employee value


4.3 Sample and Procedure
The sample groups used for this study were the 204 public sector employees who
completed the training intervention across 15 courses and 102 facilities plus their line
managers. Questionnaires containing both quantitative and qualitative items were
mailed to the 204 participants/employees some time after completing their training
programme. A total of 124 usable surveys were returned representing a response
rate of 61%.

A non-probability approach for the selection of management respondents was
adopted; management personnel were identified by the participating employees to
ensure the appropriate person was targeted for the study. The management sample
population was 103. A total of 43 usable questionnaires were returned from
management respondents representing 42% of the sample population.

Interviews were conducted using a non-random, convenience sample with ten
employees and five managers.

4.4 Measures

4.4.1 Training outcomes
The training outcome variables investigated were grouped into employee and
organisational categories. In the healthcare industry, the outcomes of this training
intervention are expected to be largely intangible or difficult to quantify. Thus, open
ended or part open ended questions were provided to seek further clarification on
intangible outcomes.

Outcomes - Employee

Job satisfaction. Employee job satisfaction was measured using a three-scale item.
Items included personal satisfaction with your job and attitude to the organisation.
The Cronbachs alpha co-efficient for the three-item scale was .77. Management
perceptions were also measured using a three item scale with similar items and the
Cronbachs alpha co-efficient was .85.

Confidence. Employee confidence was measured using a three-scale item. Items
included confidence to do physical activities with residents and confidence in
performing other work related activities with residents. The Cronbachs alpha co-
efficient was .87. Management perceptions were measured using a four-item scale
and the Cronbachs alpha co-efficient was .90.

Efficiency. Employee efficiency was measured using a four-scale item including
attention to safety performing other duties with residents and ability to respond to
residents needs. The Cronbachs alpha co-efficient was .87. Management
perceptions were measured using a five item scale including level of supervision
required to do activity sessions. The Cronbachs alpha co-efficient was .80.


Outcomes Organisation

Benefits to Residents: Employee perceptions of benefits to residents as a
consequence of participating in the activity sessions were measured by two nominal
items and four sets of scales representing a total of 14 items. The four scale
categories comprised of social benefits, cognitive benefits, psychological benefits
and physical benefits. The Cronbachs alpha co-efficient for the four combined
scales was .84. Measurement items were generally similar for management with
some modifications. Each of the four scales had alpha scores above .8.

Quality of service offered. Quality of services offered was measured by a six-item
scale. Sample items include changes in quality of services offered, number of
residents attending the session and time allocated to activities overall. The
Cronbachs alpha co-efficient was .92. The management perspective was measured
by three nominal items presented in closed question format such as improvements in
the quality of the chair based activity session offered and the number of residents
catered for.

Satisfaction with quality of service. Satisfaction levels with quality of services offered
were measured by a three-item scale including resident satisfaction with the
activities offered and satisfaction of residents relatives with the overall activity
programmes offered. The Cronbachs alpha co-efficient was .88. Management
satisfaction levels were measured using one nominal item looking at resident
satisfaction.

4.4.2 Organisation factors affecting transfer

Level of interest. Level of interest was measured using a two-item scale: I enjoy
delivering the session and the residents enjoy taking part in the sessions. The
Cronbachs alpha co-efficient was .69. The management perspective was measured
using a three-item scale including the motivation of the relevant staff members, and
management policy to offer regular physical activity sessions. The Cronbachs
alpha co-efficient was .79.

Opportunity to use. Opportunity to use was measured using a two-item scale
including delivering activity sessions is included in my job specification and the
activity sessions are allocated a specific time. The Cronbachs alpha co-efficient
was.72. The management perspective was measured using a two-item scale. The
Cronbachs alpha co-efficient was .86.

Competing priorities. Competing priorities was measured using a four-item scale
including it is too busy on my ward/unit to have regular sessions, and time is
always made on my unit/ward for the activity sessions. The Cronbachs alpha co-
efficient was .78. The management perspective was measured using a three-item
scale including time remaining after taking care of the nursing and caring duties.
The Cronbachs alpha co-efficient was .80.

Supervisor Support. Supervisor support was measured using a three-item scale
including I developed a plan with my line manager to set up the activity sessions,
and my line manager is not aware of how the activities are going. The Cronbachs
alpha co-efficient was .73. The management perspective was measured using a
four-item scale including personally setting time aside to discuss the progress of the
sessions with the ACT leader. The Cronbachs alpha co-efficient was .79.

Peer support. Peer support was measured using a three-item scale including the
other staff encourage me to do the activities. The Cronbachs alpha co-efficient was
.75. The management perspective was measured using a two-item scale including
the co-operation of other staff getting residents ready for the sessions. The
Cronbachs alpha co-efficient was .89.

Communication between departments. Communication was measured using a two-
item scale including it can be difficult to get the information I need about residents
from other units. The Cronbachs alpha co-efficient was .72. Due to the subjectivity
and experiential nature of this construct, management responses were not sought.

Resources
The resources variable was measured using a three-item scale including staff
shortages make it difficult to do the sessions on a regular basis. However, as the
Cronbachs alpha co-efficient was .26, this variable was eliminated from the
employee analysis due to its lack of reliability. The management perspective was
measured using a four-item scale including the ability of the ACT leaders to set up
the session without assistance and the availability of a dedicated room for the
activities. The Cronbachs alpha co-efficient was .82.


3.4.3 Transfer of Training - Direction

Delivery training program learning
This variable sought to establish baseline data with regard to the implementation of
training within respondents organizations. Employee responses were measured by
five items at a nominal level which included questions such as are you currently
delivering chair based physical activity sessions within your organization.
Management responses were measured using a two item scale such as to your
knowledge, are the (ACT leaders) currently leading chair based physical activity
sessions within your organisation.

Delivering regularly
Regular delivery was defined as delivering a minimum of one session per week. This
variable was established by the researcher from the recoding of responses relating
to the item on the frequency with which the chair based physical activities are
implemented.


4.5 Data Analysis
The quantitative data enabled the production of descriptive statistics and qualitative
data provided explanations of constructs, actions and contexts observed by the
respondents. Qualitative data was gathered through the open ended questions on
the questionnaires and the interviews. In both instances, respondents comments
were reviewed and content analysed prior to grouping them into specific categories
or themes according to what Scanlan et al. (cited in Biddle, Markland, Gilbourne,
Chatzisarantis and Sparkes (2001) refer to as a deductive approach to content
analysis. However, this was not feasible in all cases and therefore, any themes or
quotes that did not fit into these categories were organised into new categories.
This data was used to support the quantitative data.


5.0 Findings

Employee respondents ages ranged between 27 and 69 years. The mean age was
49 (S.D. 7.9). Length of employment ranged from 3 to 37 years with the mean
number of years 13.1 (S.D. 7.6). Management respondents were responsible for
between one and eight employees who had completed the intervention. The mean
number of employees for whom respondents were accountable was 2.3 (S.D. 1.3)
with 11.4% responsible for four or more employees. The mean length of time they
have been line manager to the trained personnel varied from 6.9 years for the first
employee to 5.4 years for the fourth employee. The average number of older
residents per organization ranged from eight to 420 residents with the mean number
58 (S.D. 64.6).
5.1 Transfer of Training

5.1.1 Delivery of Chair Based Physical Activity Sessions (training program content)
& Regular Delivery

Sixty eight percent of employee respondents were delivering chair based physical
activity sessions at the time of completing the questionnaire. Forty eight per cent (n.
84) of employee respondents were delivering the sessions on a regular basis (at
least once per week), while 26% did not indicate the frequency of delivery. The mean
number of sessions delivered per week was 2.37 ( S.D. 1.42).

Of the 32% employees who are no longer delivering sessions, 50% either never
implemented them or stopped shortly after completing the intervention and 31% do
not expect to resume delivering sessions in the future. Reasons cited for not
implementing the sessions were: inability to release staff from other duties (46%),
lack of employee interest (17%), employee appointed to a new role (14%), the
activities being delivered by other staff members (11%) and change in employee
circumstances (11%).

These results provide critical evidence that training has been transferred at some
level.

5.2 Impact of the training and development programme

5.2.1 Employee outcomes

The mean scores for perceptions of the employee outcomes on job satisfaction,
confidence and efficiency from both employees and management perspectives are
presented in Table 1 below using the likert scale where 1 represents strongly
disagree and 5 represents strongly agree. These results indicate that both sets of
respondents agree employees work-related self efficacy and work efficiency has
improved as well as their job satisfaction. Furthermore, whilst other factors may
have contributed to these changes, the results also show respondents agree that
they are also linked to participation in the training intervention. These results are
presented in table 1 below.

Table 1: Mean scores for perceptions of intervention impact on employees attitude
and behaviour
Dependent Variable Employee Management
n. Mean S.D n. Mean S.D
Job Satisfaction 102 4.2 .86 42 4.0 .67
Confidence 107 4.3 .86 42 4.2 .73
Efficiency 103 4.4 .79 40 4.1 .72
Extent to which the
intervention was
responsible
105 3.7 .88 43 3.4 .90
1 = Not responsible 5 = fully responsible


5.2.2 Organisational Outcomes
Ninety seven percent of employees believe some residents have benefitted to some
degree as a consequence of taking part in the activity sessions. The mean score for
the combined scales of changes is presented in table 2 below with 1 representing
almost never notice these changes and 5 representing almost always notice.
Management results supported employee perceptions.

The mean scores for employee perceptions of the degree of positive change in
relation to the quality of services offered, satisfaction with the quality of service
offered and perceptions of employee added value to the organisation or team are
presented in Table 2 below with 1 representing no change and 5 representing a
complete change. Management results generally indicated support for employee
perceptions with 98% believing there was an improvement in the quality of
programmes offered, 90% indicating there was an improvement in resident
satisfaction with the sessions provided and 78% believing there was an improvement
in the number of residents attending the programme. Table 2 provides a summary of
these results.


Table 2: Organisational outcomes summary of means and frequencies
Variable
Employee Management
n. Mean S.D n. Mean S.D
Improvement in residents -
combined scales
79 3.7 .61 38 3.5 .7
Quality of services offered scale 60 3.5 1.02
Satisfaction with quality of services
offered scale
99 3.1 1.14
Staff versatility/value scale 105 4.3 .87
N % Yes %
No
Don
t
kno
w
Improvement in quality of chair
based physical activity programmes
offered
41 90. 7.3 2.4
Improvement in number of CBPA
offered
40 75 22.
5
2.5
Improvement in number of
residents attending
40 77.5 20 2.5
Improvement in residents
satisfaction with the CBPA
38 89.5 5.3 5.3
Improved staff teamwork 39 76.9 15.
4
7.7

Qualitative Data
The employee questionnaire had an open ended section inviting respondents to
elaborate on any positive changes or personal benefits experienced which they
believed was a consequence of participating in the intervention. The top six rated
experiences presented in table 3 below are ranked according to the frequency of
citation. Whilst the emerging topics or items largely reflect the constructs explored
within the study, the reference to a recognised qualification presented a new theme
to the findings.
When all comments were collated into themes, job satisfaction was the only personal
outcome cited. However, the comments provided more insight into why and how
respondents perceive they have benefited personally from attending the intervention.
For example, job satisfaction had improved because they have developed a better
relationship with the residents as a result of providing activity sessions and because
it highlights the social aspect of our care. One respondent indicated that it made
me stay in the older adult ward. Reference to increased awareness was made by
several respondents such as knowing more (now) about illness and frailty and
being more aware of health and safety issues with residents.

Regarding organisational changes, employees were asked to explain what they
believed was the most important change to have taken place within their
organisation or unit as a result of the intervention. The most frequently cited
outcomes were resident-related benefits. For example, being more open and
expressing themselves or interacting more with other residents and staff.
Interestingly, a change to the attitudes and support from other staff members was
the second most frequently cited category of outcomes by respondents. For
example, respondents referred to an increase in staff awareness of residents
capabilities increased and management recognition of the improvement in quality of
life through activities (increased).

Specific employee qualitative data shed greater light on the significance of those
changes for the organisation. For example, before the course, we never had an
activity programme on long stay wards and, we now have protected time allocated
in order to perform activities as well as we now have a dedicated physical activity
team with a good skill mix. In terms of changing the approach to resident care,
employees indicated there is a new way of living opened up to residents. From a
management perspective, comments such as (the training programme) has
broadened out into other activities for residents and there is more (emphasis) on
psychological care or, that trained staff are perceived to be spending increased
quality time with residents and showing more in-depth understanding of patients as
individuals reflect an increasing focus on health promoting and quality of life
services for residents which constitute level 4 results

In relation to the most important organisational changes to take place as a
consequence of the training intervention, qualitative comments provided information
with regard to how services for older adults had improved. There is now a dedicated
team and not just one person providing the activities while another indicated that
activities are becoming more important to our clients overall healthcare. A greater
variety and number of activities offered were mentioned, while another stated the
activity programmes offered were more appropriate because they take into account
the mobility and mental well being of the resident.

Table 3: Summary of qualitative data on training intervention outcomes
employee/participant perspective
Question Area of change Number of
times cited
Positive changes experienced
as a consequence of
participating in the intervention
Awareness or insight into the
importance of PA for older adults
11
Increased confidence to do the
activities and to do the job
8
Greater understanding of
residents needs in terms of
activities
7
Improved job satisfaction 4
Better skills to do the job 4
More valued member of the team
Have a recognised qualification
Applying new learning to other
groups
2
2
2
Personal rewards or benefits as
a consequence of participating
in the intervention
Job satisfaction

19
What is the most important
change to take place in the
organisation or unit as a result of
the training intervention?
Resident outcomes 26
Staff attitudes and input 14
Management attitudes and action 11
Programmes offered 10

Table 4 below provides a summary of managements perceptions with regard to
personal benefits received by employees.


Table 4: Summary of qualitative comments on training intervention outcomes-
management perspective
Question Area of change Number of
times cited
Positive changes
observed in employees
Confidence in ability to do activities 6
Enhanced skills and learning 4
Improved relationship with residents 3
Other outcomes Employee value to the organisation 5
Organisational benefits (regarding
services/programmes offered)
5
Benefits to residents 3
Personal rewards or
benefits to staff
member(s)
Confidence 8
Job Satisfaction
6


Interviews with employees cited increased confidence as the most important
personal outcome followed by enjoyment (job satisfaction). Two respondents
described how learning had been applied to different contexts beyond the workplace.
An extension in the number and range of activities offered together with
improvements in the quality of the activities and the numbers of residents able to
access the activities were also cited.

Feedback from the interviews with management concurred with the employee
comments. Regarding improvements in confidence, indicators included increased
self esteem and assertiveness and they work more autonomously as members of
the hospital recreation group.


5.3 The extent to which the organisation transfer system influences transfer of
training in public sector hospitals.

Only 51% of management respondents were satisfied with the level of activity
sessions offered despite indicating almost 87% of trained employees are providing
them for residents at some level.
The employee and management mean scores for the extent to which they perceive
each of the seven variables influence training transfer are presented in table 5
below. In this instance, 1 represents strongly disagree, 3 represents neither agree
nor disagree and 5 represents strongly agree (see Table 5)

Table 5: Mean scores for perceptions of the effect of organisation transfer system
variables on training transfer.
Variable
Employees
Management
n. Mean S.D n. Mean S.D
Level of Interest Scale 108 4.6 .64 41 4. .736
Opportunity to use Scale 106 3.6 1.44 41 4. .952
Line manager support scale 99 3.5 1.19 41 3.7 .763
Peer support Scale 104 3.2 1.15 41 3.4 1.21
Competing priority Scale 92 3.1 1.21 38 3.6 .962
Communication Scale 97 2.5 1.27
Resources Scale 40 3.4 .950

According to the mean scores, both management and employees concur with regard
to the top three influencers on training transfer from the seven organisation transfer
system variables investigated. The level of interest variable is perceived to be the
greatest influence on the extent to which the chair based physical activity sessions
are conducted followed by opportunity to use, support of line manager
respectively. The following three variables peer support, competing priorities and
communication between departments were ranked in a different order by both sets
of respondents as illustrated in table 5 above.

5.3.1 Qualitative Data on Organisational Transfer Factors

Employee Data
The employee questionnaire contained an open ended item asking respondents to
indicate what they believe had been the biggest influence for the extent to which they
were transferring training. Table 6 below summarises the comment themes in rank
order. In terms of positive influencers, the top four factors were manager or line
manager support, resident interest, support from other staff and allocation of
dedicated time. For example, in terms of line manager and peer support,
respondents cited the backing of line managers or the fact that managers specifically
wanting to see the activities implemented as important influencers. Other positive
influencers included the assistance of peers in getting residents ready for the
activities, or being allocated dedicated time to carry out the activities and having a
defined role as an activity person within the organisation. With regard to detractors
from transferring training, staff shortages, insufficient time, resident (health or
cognitive) status and lack of support from other staff were the four top cited factors.
Qualitative feedback also indicated that employees working in some departments
may be allocated to other activities rendering them unable to do the group activities.

While staff shortages, staff support, time and completing other duties feature highly
as disablers to transferring training, resident status is cited as second highest factor.
For example, one respondent indicated that the residents dependency has
increased. For this respondent, it had implications for being released from work or
for getting cover to do the activities because of the increased workload on staff due
to the higher dependency of residents. For other respondents, it was felt that the
change in resident status made them unsuitable for the activities.

Table 6: Summary of qualitative comments on organisational system factors
influencing training transfer employee perspective

Rank Positive
Influencers
Number
of
citations
Rank Negative influencers Number
of
citations
1 Management or
line manager
support
12 1 Staff shortages 12
2 Resident Interest 11 Time 12
3 Staff support 10 3 Residents (health or
cognitive)status
9
4 Dedicated time 9 4 Staff support 7
5 Part of job 8 5 Assigned other duties 5
6 Personal interest 6 6 Where working 3
7 (Unit) where
working
4 7 Changed job role 2
(Attending) the
training intervention
4 Lack of Resources 2
9 Current position
and level of
autonomy
3 8 Other 4
Being part of an
activity team
3


Management data

With regard to the positive influencers, management respondents felt the motivation
and dedication of the trained employees exceeded all other factors whereas staff
shortages was the most frequently cited negative influence. Specific activities which
management believe positively impacted on transfer include assigning protected
time (to do the activities), including the activities in (their) job description and
having a committed , dedicated activity person in place to undertake the activities.
It is interesting to note that the characteristics of the trained personnel are both the
highest ranked enabler and third ranked negative influence according to
management comments. For example, one respondent indicated staff say they
have no time but we have dedicated time twice weekly for activities whilst another
indicated that on occasions, I have to prompt the leaders to do the sessions. Table
7 below presents a summary of management comment themes in rank order.

Table 7: Summary of qualitative comments on organisational system factors
influencing training transfer management perspective
Rank Positive
Influencers
Number
of
citations
Rank Negative influencers Number
of
citations
1 Motivation and
dedication of trained
staff
9 1 Staff shortages 6
2 Staff support 3 2 Residents (health or
cognitive) status
3
Client interest 3 3 Leader interest 2
Dedicated time 3 4 Too busy 1
Part of job 3 5 Staff teamwork 1
Management
support
3 6 Not enough people
trained
1




6. Discussion and Conclusions

This study adopted a mixed method approach to measure the impact of a training
and development intervention at individual and organisational level across 77 long
stay hospital settings within the Irish health care sector. It also investigated the
effect of organisational factors on training transfer post training.

With regards to measuring training outcomes, the primary concern relates more to
demonstrating a link between the training intervention and results observed,
especially in the case of intangible outcomes which, according to Spitzer (2005),
most evaluation models fail to accommodate. This was particularly relevant for this
study where outcomes were largely intangible and value measured in terms of cost-
effectiveness would not be easily demonstrated. Results demonstrated a high
percentage of respondents had experienced intangible personal rewards such as
improved job satisfaction. Furthermore, qualitative data reports that employees
demonstrated how perceptions of improvements in their efficiency and confidence
are linked to their perceived job satisfaction and secondly, how they were motivated
by satisfaction in their work rather than extrinsic rewards.

With regards to the organisational outcomes explored for in this study, all
respondents agreed that almost all residents had experienced positive benefits from
participating in the activities delivered as a consequence of the training intervention.
What is evident from these comments is that while training has impacted on one
aspect of patient care, it has done so at multiple levels; health benefits, patient
access, quality control, dedicated time for activities and the expansion of a
recreational and therapeutic programme. However, it was both the dual perspective
and qualitative methodology that detected these outcomes.

Qualitative feedback from both management and employees rank staff shortages as
the top inhibitor to transferring training which is not surprising in a sector
experiencing staff cutbacks and moratoriums on staff recruitment.


5.2 Implications for Practice & Research
The use of qualitative research methods enabled a deeper understanding of the
intangible outcomes of the training intervention at employee and organisational level
which was particularly important in a sector where training outcomes were expected
to be predominantly intangible. This has implications for the design of research
instruments, for the expertise required to design and administer such instruments
and the expertise for analysing and interpreting the results. The use of qualitative
research methods have created a greater understanding of the organisational
transfer system factors perceived by respondents to contribute to the extent of
training transfer. It enabled the distinction between how the variables investigated
acted as both enhancers and inhibitors which quantitative methods did not do. The
qualitative feedback also indicated that different elements within some of the
variables were perceived to have different levels of impact on training transfer by
management and employees. This provides further support for the use of dual
perspectives and triangulation of methods in evaluation research

5.3 Limitations of the study

Although a cross sectional approach was used to investigate 77 hospitals within the
Irish healthcare sector, health care systems are very complex with powerful
subcultures. While the findings of this study contributed to our understanding of the
transfer process in this sector, it cannot be assumed that they will generalise to all
settings and sectors. Nonetheless, the integrative model and processes used to
operationalise the study can be adapted for use in other sectors. Another limitation
of the study was the small number of management respondents. To make the study
operational, the number of organisation system constructs investigated for effect on
training transfer was limited to seven which did not capture all the potential
influencing variables on training transfer.

References
Alliger, G. M., Tannenbaum, S. I., Bennett Jr, W., Traver, H.and Shotland, A. 1997.
A meta analysis of the relations among training criteria. Personnel Psychology. 50
(2), pp. 341-358.

Alvarez, K., Salas, E. and Garofano, C., 2004. An integrated model of training
evaluation and effectiveness, Human Resource Development Review, 3(4), pp. 385-
416.

Baldwin, T. T. and Ford, J. K. 1988. Transfer of training: a review and directions for
future research. Personnel Psychology, 41, pp. 63-105.

Broad, M. L. 2005. Beyond Transfer of Training. Engaging Systems to Improve
Performance, USA: Pfeiffer.

Burke, L. A. and Hutchins, H. M. 2008. A study of best practices in training transfer
and proposed model of transfer. Human Resource Development Quarterly, 19 (2),
pp. 107-128.
Cannon-Bowers, J. A., Salas, E., Tannenbaum, S.I. and Mathieu, J. E. 1995. Toward
theoretically based principles of training effectiveness: A model and initial empirical
investigation. Military Psychology, (7), pp. 141-164.

Chen, G. and Klimoski, R. J. 2007. Training and development of human resources at
work: is the state of our science strong? Human Resource Management Review,
17(2), pp.180-190.

Chen, Hsin-Chih, Holton lll, E. and Bates, R. A. 2006. Situational and demographic
influences on transfer system characteristics in organisations. Performance
improvement quarterly, 19 (3), pp. 7 26.

Cheng, E. W. L., and Ho, D.C. K. 2001. A review of transfer of training studies in the
past decade. Personnel Review, 30 (1), pp. 102-118.

Clarke, N. 2002. Job/work environment factors influencing training transfer within a
human service agency: some indicative support for Baldwin and Fords transfer
climate construct. International Journal of Training and Development, 6 (3) pp. 146-
162.

Clarke, N. 2007. The limitations of in-service training in social services. In: S.
Sambrook, & J. Stewart (Eds.), Human resource development in the public sector.
The case of health and social care). London: Routledge, pp. 315-340

Elangovan, A. R.and Karakowsky, L. 1999. The role of trainee and environmental
factors in transfer of training: an exploratory framework. Leadership and
Organisation Development Journal, 20, (5), pp.268-276.

Ford, J. K. and Kraiger, K. 1995. The application of cognitive constructs to the
instructional systems model of training: implications for needs assessment, design
and transfer. In: C. L. Cooper, and I. T. Robertson, International Review of
Industrial and Organisational Psychology. Chichester: Wiley, pp.1-48

Greene, J. and Caracelli, V. (eds.) 1997. Advances in Mixed-Method Evaluation:
The Challenges and Benefits of Integrating Diverse Paradigms. Jossey-Bass, San
Francisco.

Holton III, E. F. 1996. The flawed four level evaluation model. Human Resource
Development Quarterly 7, pp. 5-21.

Holton lll, E. F., Bates, R. A., Seyler, D. L. and Carvalho, M.B.1997. Toward
construct validation of a transfer climate instrument. Human Resource Development
Quarterly, 8 (2), pp. 95-113.

Holton lll, , E. F., Bates, R. A. and Ruona, W.E.A. 2000. Development of a
generalized learning transfer system inventory. Human Resource Development
Quarterly, 11 (4), pp. 336-360.

Holton III, E. F., Chen, Hsin-Chih and Naquin, S. S. 2003. An examination of
learning transfer system characteristics across organisational settings. Human
Resources Development Quarterly, 14 (4), pp. 459.

Hutchins, H. M. (2009). In the trainers voice: a study of training transfer practices.
Performance Improvement Quarterly, 22 (1), 69-93.

Hutchins, H. M. and Burke, L. A. 2007. Identifying trainers knowledge of training
transfer research findings closing the gap between research and practice.
International Journal of Training and Development, 11 (4), pp. 236-264.

Kearns, P. 2005. From return on investment to added value evaluation: the
foundation for organisational learning. Advances in Developing Human Resources, 7
(1), pp. 135-146.

Kupritz, V. W. 2002. The relative impact of workplace design on training transfer
Human Resource Development Quarterly, 13 (4), pp. 427-448.

Lim, D. H. and Johnson, S. D. 2002. Trainee perceptions of factors that influence
learning transfer. International Journal of Training and Development, 6 (1), pp. 36-
48.

Martin, H. J. 2010. Workplace climate and peer support as determinants of training
transfer. Human Resource Development Quarterly , 21 (1), pp. 87-104.

Mayo, A 2000. The role of employee development in the growth of intellectual
capital. Personnel Review, 29 (4) pp. 521-533.

Nijman, Derk-Jan J. M., Nijhof, W. J., Wognum, A.A. M. and Veldkamp, B. P 2006.
Exploring differential effects of supervisor support on transfer of training. Journal of
European Industrial Training, 30 ( 7) pp. 529-549.

Noe, R. A. 1986. Trainees attributes and attitudes: neglected influences on training
effectiveness. Academy of Management Review , 11 (4), pp. 736-749.

Noe, R. A. 2000. Invited reaction: development of a generalized learning transfer
system inventory. Human Resource Development Quarterly, 11 (4), 361-366.

Onwuegbuzie, A. J., Burke Johnson, R., and Collins, K. M. T. 2009. Call for Mixed
Analysis: A Philosphical Framework for Combining Qualitative and Quantitative
Approaches. nternational Journal of Multiple Research Approaches. 3 (2), pp. 114-
140.

Quinones, M. A., Ford, J.K., Sego, D.J. and Smith, E.M. 1995. The effects of
individual and transfer environment characteristics on the opportunity to perform
trained tasks. Training Research Journal, 1, pp. 29-48.

Rouiller, J. Z. and Goldstein, I. L. 1993. The relationship between organisational
transfer climate and positive transfer of training. Human Resource Development
Quarterly. 4 (4), pp. 377-399.

Ruona, W.E.A., Leimbach, M, Holton lll, E.F. and Bates, R. A. 2002. The relationship
between learner utility reactions and predicted learning transfer among trainee.
International Journal of Training and Development, 6 (4), pp.218-228.

Saks, A. M. and Belcourt, M. 2006. An investigation of training activities and transfer
of training in organisations. Human Resource Management, 45 (4), pp. 629-648.

Salas, E. and Cannon-Bowers, J. A. 2001. The science of training: a
decade of progress. Annual Review of Psychology, 52, pp. 471-499.

Santos, A. and Stuart, M. 2003. Employee perceptions and their influence on
training effectiveness. Human Resource Management, 13 (1), pp. 27-45.

Spitzer, D. R. (2005). Learning effectiveness measurement: a new approach for
measuring and managing learning to achieve business results. Advances in
Developing Human Resources, 7 (1), pp. 55-71.

Tan, J., Hall, R. J. and Boyce, C. (2003) The role of employee reactions in
predicting training effectiveness, Human Resource Development Quarterly Vol. 14:,
No. 4. Pp. 283-296.

Tannenbaum, S. I. and Woods, S. B. 1992. Determining a strategy for evaluating
training: operating within organisational constraints. UHR. Human Resource
Planning, 15 (2), pp. 63-82.

Tashakkori, A. and Teddlie, C. (Eds.) 2003. Handbook of Mixed Methods in Social &
Behavioral Research, Sage, Thousand Oaks.

Van der Klink, M. R., Gielen, E. E. M., and Nauta, C. 2001. Supervisory support as a
major condition to enhance transfer. International Journal of Training and
Development, 5 (1), pp. 52-63.

Wang, G. G. and Spitzer, D. R. (2005). Human resource development measurement
and evaluation: looking back and moving forward. Advances in Developing Human
Resources. 7 (1), pp. 5-16.


Wang, G. and Sun, J. Y. (2009). Perspectives on theory. Clarifying the boundaries of
human resource development. Human Resource Development International, 2 (1),
pp. 93-102.