Sie sind auf Seite 1von 26
Journal of Human Resources in Hospitality & Tourism ISSN: 1533-2845 (Print) 1533-2853 (Online) Journal homepage:
Journal of Human Resources in Hospitality & Tourism ISSN: 1533-2845 (Print) 1533-2853 (Online) Journal homepage:

Journal of Human Resources in Hospitality & Tourism

ISSN: 1533-2845 (Print) 1533-2853 (Online) Journal homepage: http://www.tandfonline.com/loi/whrh20

Exploration of hotel managers' training evaluation practices and perceptions utilizing Kirkpatrick's and Phillips's models

Anh D. D. Ho, Susan W. Arendt, Tianshu Zheng & Kathy A. Hanisch

To cite this article: Anh D. D. Ho, Susan W. Arendt, Tianshu Zheng & Kathy A. Hanisch (2016) Exploration of hotel managers' training evaluation practices and perceptions utilizing Kirkpatrick's and Phillips's models, Journal of Human Resources in Hospitality & Tourism, 15:2, 184-208, DOI:

Full Terms & Conditions of access and use can be found at

JOURNAL OF HUMAN RESOURCES IN HOSPITALITY & TOURISM 2016, VOL. 15, NO. 2, 184 208

RESOURCES IN HOSPITALITY & TOURISM 2016, VOL. 15, NO. 2, 184 208 http://dx.doi.org/10.1080/15332845.2016.1084861
RESOURCES IN HOSPITALITY & TOURISM 2016, VOL. 15, NO. 2, 184 208 http://dx.doi.org/10.1080/15332845.2016.1084861

Downloaded by [University of Glasgow] at 08:44 27 December 2017

Exploration of hotel managerstraining evaluation practices and perceptions utilizing Kirkpatrick s and Phillips s models

Anh D. D. Ho a , Susan W. Arendt a , Tianshu Zheng a , and Kathy A. Hanisch b

a Department of Apparel, Events, and Hospitality Management, Iowa State University, Ames, Iowa, USA; b Department of Psychology, Iowa State University, Ames, Iowa, USA

ABSTRACT

Training is one human resource development practice found in most organizations, however, studies showed that little attention is given to the importance of training evaluation in real life practices. This study is an exploration of the practices and perceptions of hotel managers in training evaluation using Kirkpatrick s and Phillips s models. In-depth interviews were conducted with six hotel managers and paper-based questionnaires were sent out to managers of hotels with more than 30 rooms in a Midwestern state. The ndings indicated that hotel managers viewed training evaluation activities as important and observation was rated the most important and the most frequently employed method for managers in evaluating training. The study s ndings contribute to the literature by providing researchers with more insights into how hotel managers evaluate their training and what they believed a practical process should possess. It also gives researchers a brief understanding of the perceptions of managers from different hotel sizes.

KEYWORDS

Evaluation; hotel managers; Kirkpatricks model; training

Introduction

Human resource (HR) practitioners tend to focus their efforts on costs and pro- cesses rather than measuring the value added from HR practices (Ramlall, 2003). Because cost effectiveness is essential for business success, it is important for HR managers to establish the value of HR practices to the organization in ways that are easy for top-level managers to understand. Traditional ways to show the value of HR departments have included demonstrating nancial results of HR practices in relationship to the organization protability (Ulrich, 1997; Ramlall, 2003). Training is one HR development practice found in most organizations, but deter- mining measurable nancial results of training is often a complicated process. With the focus on evaluating and demonstrating training effectiveness, studies have been done on training evaluation concepts, models, and applications of these

CONTACT Anh D. D. Ho

Iowa State University, 31 MacKay, Ames, IA 50010-1121.

© 2016 Taylor & Francis Group, LLC

Ames, IA 50010-1121. © 2016 Taylor & Francis Group, LLC anhho@iastate.edu Department of Apparel, Events, and

anhho@iastate.edu

© 2016 Taylor & Francis Group, LLC anhho@iastate.edu Department of Apparel, Events, and Hospitality Management,

Department of Apparel, Events, and Hospitality Management,

Downloaded by [University of Glasgow] at 08:44 27 December 2017

JOURNAL OF HUMAN RESOURCES IN HOSPITALITY & TOURISM

2017 JOURNAL OF HUMAN RESOURCES IN HOSPITALITY & TOURISM 185 in different industries (Bartel, 2000 ;

185

in different industries (Bartel, 2000; Chang, 2010; Holton, 1996; Kearns, 2005; Kirkpatrick, 1959a , 1959b, 1960a, 1960b; Kline & Harris, 2008; Phillips, 1996; Pine & Tingley, 1993; Russ-Eft & Preskill, 2005). Despite the vast amount of research done on training evaluation, Bersin (2006) noted that managers and practitioners had given little attention to the impact of training on business and overall return on investment (ROI). Similar results were found in a research of American Society for Training & Development (ASTD) in 2009. Swanson (2005) indicated that there was a gap between the literature and HR development evaluation (including train- ing) as a result of HR practitionersand researchersdifferent perspectives. Under- standing the perceptions of both groups, especially practitioners, will contribute to the development of a more practical, usable training evaluation model. In a fast- paced industry with a high level of interaction with customers, hotel managers often place strong emphasis on their employee training programs. At the same time, few known studies have been conducted on hotel industry training evaluation (Chang, 2010; Kline & Harris, 2008). The purpose of this study was to examine training evaluation processes and practices currently used by hotel managers and ascertain input for a practical evaluation process. To explore practices and percep- tions of hotel managers related to training, the research objectives were as follows:

determine how hotel managers perceive and use training evaluation activities in their hotels; evaluate how hotel managersperceptions of training evaluation activities affect their usage of those activities; identify reasons hotel managers apply or do not apply training evaluation models or processes; and ascertain what hotel managers perceive as practical characteristics in a train- ing evaluation process.

Literature review

Green and McGill (2011), in a report for the ASTD, estimated that U.S. organiza- tions spent about $171.5 billion on learning and development for employees in 2010. Customer service training is an essential part of the hotel industry, consider- ing the people-focused nature of the business. With high investments in training, it is evident that hotel managers should evaluate the effectiveness of their customer service training. Although hotels have different methods of operating, employeesservice skills are universally paramount.

Theoretical framework

The theoretical framework for this study was the training evaluation model origi- nally developed by Kirkpatrick (1959a ) and later expanded by Phillips ( 1996). These two models were selected because of their signi cance in the literature and application to training evaluation. Kirkpatrick s model divided training evaluation into four steps: reaction, learning, behavior, and results (Kirkpatrick, 1959a, 1959b,

Downloaded by [University of Glasgow] at 08:44 27 December 2017

186

by [University of Glasgow] at 08:44 27 December 2017 186 A. D. D. HO ET AL.

A. D. D. HO ET AL.

1960a , 1960b). In the rst step, reaction, Kirkpatrick (1959a) recommended train- ing directors evaluate participant reaction with emphasis on anonymity (to garner honest feedback) and the possibility of quantifying collected feedback. The second step, learning, refers to the amount of knowledge participants obtain from training (Kirkpatrick, 1959b); while behavior, the third step, is focused on how participants apply the new knowledge in their jobs (Kirkpatrick, 1960a ). Kirkpatrick recom- mended the use of before-and-after measurement, a control group, and statistical analysis for both the second and third steps. Paper-and-pencil tests or classroom performance could be used to measure learning, while for behavior, the perfor- mance assessment should be done by others (e.g., superiors, subordinates, peers) rather than by the training participants themselves (Kirkpatrick, 1959b, 1960a ). For the last step, Kirkpatrick (1960b) indicated that business results are the best way to evaluate a training program but are also complicated to measure. Kirkpatrick s four-step model popularized training evaluation concepts and has been used as the foundation for many later models (e.g., Kearns, 2005; Phillips, 1996; Wang & Spitzer, 2005). Kirkpatrick s model has been acknowledged as a standard for practice in the eld and has been used by researchers and organiza- tions in a wide range of industries (Bartel, 2000; Chang, 2010; Kline & Harris, 2008; Pine & Tingley, 1993; Russ-Eft & Preskill, 2005). Phillipss model added a fth level to Kirkpatrick s model (Phillips, 1996). This fth level of evaluation focuses on ROI and compares the training bene ts, as expressed in nancial results, with training costs. Phillips assumed that there were also intangible benets associated with training, and that those could be converted into easy to understand nancial values. He believed that the benecial effects of training may drop off after the rst year and therefore a conservative approach to evaluation would not consider long-term benets as part of the calculation (Phil- lips, 1996).

The gap between theoretical models and practices

Despite the large body of research during the long history of training measurement and evaluation, studies show that a gap still exists between expected and actual training measurement (Bersin, 2006; Kline, 2008; Phillips, 1996). Bersin (2006) conducted a survey with training managers at more than 140 companies (all sizes and types) about their training measurements. The most commonly measured fac- tors and the percentage of participating companies measuring these factors were as follows: completion (88%), enrollment (86%), and training participants satisfac- tion (81%). In contrast, training managers viewed completion, enrollment, and sat- isfaction factors as less valuable compared to other factors. Seventy-six percent of the managers surveyed valued job impact, 72% valued business impact, and 65% valued business metrics. However, those valued measurements were used mini- mally within the surveyed companies, with 14% and 10% measuring job impact and business impact, respectively; whereas ROI was even less, with only 5% of the

Downloaded by [University of Glasgow] at 08:44 27 December 2017

JOURNAL OF HUMAN RESOURCES IN HOSPITALITY & TOURISM

2017 JOURNAL OF HUMAN RESOURCES IN HOSPITALITY & TOURISM 187 companies measuring and using ROI. Organizations

187

companies measuring and using ROI. Organizations were spending about 2.6% of their training budget on measurement and 82% of the managers thought their organizations should allocate more funds to training and measurement. Similarly, a recent ASTD research report (2009) indicated that the percentage of participating companies measuring the third level of Kirkpatrick s model were signi cantly lower than that of level 1 and 2 (54.5% compared to 91.6% and 80.8%). Only 36.9% of the companies evaluated training s impacts on business results and evalu- ation of ROI had the lowest percentage of 17.9%. Conversely, three fourths of the participating companies that practiced training evaluation stated that evaluation of level 3 (behavior) and 4 (business results) had high or very high value. The percen- tages of managers that rated level 1 (participant reactions), 2 (learning), and 5 (ROI) high to very high value were 35.9%, 54.9%, and 59.4%, respectively.

Training evaluation in the hotel industry

In 2011 the hotel industry had total revenues of $137.5 billion and employed 1.8 million workers (American Hotel & Lodging Association, 2012), thereby con- tributing signicantly to the U.S. economy. With a high level of customer interac- tion, training is an essential tool to ensure hotel employees provide excellent customer service. Heide and Grønhaug (2009) assessed key factors in hotel guestsperceptions and found hospitability a main determinant contributing to guestssat- isfaction and loyalty. To ensure a hospitable and welcoming environment for guests, it is recommended that hoteliers focus, not only on guestsneeds, but also on employee training. Meanwhile, in a case study in Kenya, researchers found that only one fourth of participating hotel employees received some form of perfor- mance evaluation. In the same study, evaluation through the form of performance appraisals was found to have a positive effect on employee satisfaction, which was known to have a signicant contribution to on-the-job performance (Onyango & Okech, 2008). Saks and Burke (2012) learned that organizations were more likely to evaluate the rst two steps of Kirkpatrick s model, however only the third and fourth steps of the model had signicant effects on training transfer. In other words, organizations that conducted Kirkpatricks third- and fourth-step evalua- tions more frequently reported higher transfer rates compared to the organizations that did not do these steps as frequently. Particularly, for the hotel industry, Frash, Antun, Kline, and Almanza (2010) determined the work environment, including performance expectation, feedback/coaching, and capacity to transfer, as one of the main factors with strong impact on employee training transfer. Thus, hotel training managers were recommended to clearly express their performance expect- ations and include on-the-job feedback opportunities for performance after training. Kline and Harris (2008) examined the approaches used by leading hotel organi- zations to measure the costs and outcomes of training. They conducted in-depth, semi-structured interviews with individuals responsible for training at six award-

Downloaded by [University of Glasgow] at 08:44 27 December 2017

188

by [University of Glasgow] at 08:44 27 December 2017 188 A. D. D. HO ET AL.

A. D. D. HO ET AL.

winning hotel companies to determine whether ROI was used. Interview questions were based on Kirkpatrick s and Phillipss models. Two themes emerged from the data analysis: training needs were considered prior to training budget preparation and training budget was reviewed by the top management level. Regarding training benets and ROI measurements, business impacts were measured by easy to acquire data (e.g., employee turnover) and other informal methods such as employee feedback sessions. Overall, hotel managers believed that measuring train- ing benets still needed improvement; and ROI was not formally measured. The researchers concluded that, although controlling costs and tracking training invest- ments were important, most trainers did not commit to developing or using a sys- tem to measure these criteria. It was suggested that managers needed to be equipped with techniques and suitable tools for measuring training ROI (Kline & Harris, 2008). Chang ( 2010) studied the effectiveness of Kirkpatrick s model in hotel sales training evaluation and found that the model could be employed effectively to assess training in hotels. She recommended that hotels collect performance data on both individual and organizational levels to enable comprehensive training evaluation. Kirkpatrick discussed various methods to evaluate training at each level, however limited literature could be found on speci c activities hoteliers apply in their properties. Therefore, this study attempted to identify the popularly used activities to evaluate training and how managers view those activities.

Reasons for the gap between theories and practices

There were several reasons why organizations failed to carry out systematic evalua- tion of training. One of the reasons was because training professionals did not believe in evaluation or thought that evaluation was too dif cult to conduct (Swan- son, 2005). Another reason was the lack of condence that training programs could add value to the organization (Spitzer, 1999 ). Salas and Kosarzycki (2003) noted that the simplistic views and misconceptions of organizations prevented them from paying attention to research ndings on training. Moreover, many companies lacked the resources or processes needed to conduct training evalua- tion. In other cases, it was hard for companies to acquire meaningful business measures when there was little alignment between learning acquired from training and job performance (Bersin, 2006). Last but not least, failure of researchers in communicating research ndings and making them accessible to practitioners con- tributed to the reasons why organizations did not incorporate academic ndings in the eld (Salas & Kosarzycki, 2003). Despite the availability of other models, managers and practitioners were still using only the rst step of Kirkpatricks model, reaction, to evaluate training (Spit- zer, 1999). The different mindsets and interests of HR researchers and practitioners contributed to widening the gap between theory and practice. While researchers were motivated to study and explore new understandings and explanations,

Downloaded by [University of Glasgow] at 08:44 27 December 2017

JOURNAL OF HUMAN RESOURCES IN HOSPITALITY & TOURISM

2017 JOURNAL OF HUMAN RESOURCES IN HOSPITALITY & TOURISM 189 practitioners were focused on ensuring organizational

189

practitioners were focused on ensuring organizational processes and outcomes (Swanson, 2005). Understanding the perceptions of both groups, especially practi- tioners who will be applying the models, will contribute to the development of a more practical, usable training evaluation model or process.

Customer services in the hotel industry

Customer service is paramount in the hospitality industry because hospitality busi- ness involves multiple and repeated interactions with customers; hotels are a major part of the hospitality industry. Researchers have been studying the relationship between customer service and customer satisfaction for years. In 1992, Cronin and Taylor conducted a study of service industries and found empirical evidence that

perceived service quality could lead to customer satisfaction. Additionally, the scale

to

evaluate hotel service quality might differ by industry, depending on the amount

of

service involved in each industry.

Oh ( 1999 ) conducted a study on customers of two luxury hotels to test a model of service quality, customer value and customer satisfaction. The researcher found that service quality and customer value in combination may completely mediate perceptions towards customer satisfaction (Oh, 1999 , p.78). Furthermore, Kandampully and Suhartanto ( 2000 ) found that customer satisfaction and hotel image played important roles in establishing customer loyalty. Considering the demonstrable effect of service quality on customer satisfaction, and eventually cu stomer loyalty, it is essential for hotel managers to emphasize customer-service training for hotel employees.

Methodology

A mixed methods approach was employed for this study, including two parts: in-

depth interviews and a questionnaire. The purpose of the in-depth interviews was to elicit valuable information about hotel managersperceptions, awareness, and reasons for using or not using training evaluation models. The interview data were used to develop a questionnaire suitable for collecting empirical data from hotel

managers in the Midwest, thereby determining the importance and usage of each aspect involved in training evaluation. Another purpose of the questionnaire was

to determine managersopinions about characteristics of a practical training evalu-

ation process.

In-depth interviews Participant selection

Purposive was employed for the in-d epth interviews. Based on personal contacts and referrals, six managers (either general managers or HR manag- ers) from two hotels of 30 to 100 room s, three hotels of 100 to 300 rooms, and one hotel of more than 300 rooms, in one Midwestern state were

Downloaded by [University of Glasgow] at 08:44 27 December 2017

190

by [University of Glasgow] at 08:44 27 December 2017 190 A. D. D. HO ET AL.

A. D. D. HO ET AL.

recruited for the interviews. The par ticipants were selected because they were managers with training responsi bilities in hotels with an evaluation process for customer service training. Their responsibilities and evaluation processes were con rmed after the initial e-mail invitation. Various size hotels were initially selected to study the similarities and differences between the process and perception of their managers. However, this com- parison was only possible for the interview data. The comparison was not possible for questionnaire data due to the low response rate from managers at hotels with more than 200 rooms. The interviews were used to extract more information from hotel managers in addition to the available knowl- edge from current literature.

Research procedure and data analysis

In-depth interviews were conducted with six hotel managers to determine the approaches they used in evaluating customer service training and their awareness and perceptions of the Kirkpatrick s and Phillipss models. Open-ended questions were developed and customized using information from these two models, includ- ing the four steps of Kirkpatrick s (1959a) model (reaction, learning, behaviors, and business results) and the fth step (ROI calculation) from Phillipss (1996) model. Model names were not mentioned during the interview but model steps served as training process elements in interview questions. To avoid confusion, model steps were described in laymans terms and model-specic terminology was avoided in questions. Examples of interview questions were as follows: How do you evaluate the reactions of employees toward the customer service training? (step 1 of Kirkpatrick s model), How do you evaluate whether employees apply what they have learned from the customer service training to their job?(step 3 of Kirkpatrick s model), or How do you link customer service training results with monetary outcomes?(step 5 from Phillipss model). Interview questions were reviewed and approved by three researchers with expertise in training and the hos- pitality industry. Prior to the interviews, participants were contacted via an invitation e-mail and a follow-up phone call (within 1 week after the e-mail). Interviews were conducted in person at the managershotels, audio recorded for transcribing purposes, and lasted 50 to 80 minutes. The interview data were transcribed and hand coded by the primary researcher. After transcribing and coding, 416 codes were used and 31 categories arose from those codes. With further analysis, two major themes emerged and are dis- cussed in this article. The coding, categorizations, and interpretation were indepen- dently reviewed by another qualitative researcher. The interview transcripts were e- mailed to all six managers who were then asked to give transcript accuracy feedback; this process is referred to as member checking and is used to enhance trustworthiness of data (Creswell, 2007). After the e-mail and a follow-up phone call, four managers conrmed transcript accuracy and two did not reply.

Downloaded by [University of Glasgow] at 08:44 27 December 2017

JOURNAL OF HUMAN RESOURCES IN HOSPITALITY & TOURISM

2017 JOURNAL OF HUMAN RESOURCES IN HOSPITALITY & TOURISM 191 Questionnaire Participant selection Hotels from the

191

Questionnaire Participant selection

Hotels from the same Midwestern state were selected from the American Automo- bile Association (AAA) TourBook and all hotels meeting the criterion of more than 30 rooms were included (AAA, 2011 ). This criterion is based on the assump- tion that with 30 rooms or less, the hotel is less likely to have formal customer ser- vice training for employees. Jameson ( 2000) found that small hospitality rms, those with less than 50 employees, conducted training on more of an informal basis. In addition, many hotels having less than 30 rooms were bed and breakfasts managed by an owner rather than a general manager (AAA, 2011); hence, it is less likely bed and breakfasts have formal employee training. The questionnaires were sent to the manager in charge of training evaluation (either the general manager or HR manager) at 361 hotels that met the criterion. Managers that participated in the in-depth interviews were excluded from this sample.

Research procedure and data analysis

The questionnaire was developed based on information obtained from the litera- ture (Kirkpatrick, 1959a, 1959b, 1960a , 1960b; Phillips, 1996) and supported by qualitative data from the six in-depth interviews. Few similar studies (Chang, 2010; Kline & Harris, 2008) have been done in hotels, so knowledge gained from the interviews was used to customize the questionnaire to better t the hotel industry. There were two sections in the questionnaire. The rst section, demographics, included hotel information and personal information questions. The second sec- tion consisted of questions about training evaluation. There were ve main ques- tions in the second section related to importance rating of evaluation activities (e.g., How important is an evaluation form in assessing the amount of knowledge employees gained from the training? ), usage rating of evaluation activities (e.g., How often do you use observation in determining whether employees apply what they have learned from the training? ), reasons for not using evaluation activities, practical characteristics (e.g., How important is each of the following characteris- tics [e.g., easy to conduct, standardized] in developing a practical and applicable training evaluation process? ), and information sources for a training evaluation process (e.g., How important are the following information sources [e.g., personal experience, colleagues, conferences] to you when you develop a customer service training evaluation process? ). At the end of the questionnaire, participants could choose to enter a drawing for two $50 gift cards. Before questionnaire distribution, a pilot test was conducted with four hotel managers who participated in the interviews and ve graduate students in hospital- ity management. The questionnaire was also reviewed by three professionals in the eld. Based on their feedback, only a few minor wording changes were made.

Downloaded by [University of Glasgow] at 08:44 27 December 2017

192

by [University of Glasgow] at 08:44 27 December 2017 192 A. D. D. HO ET AL.

A. D. D. HO ET AL.

After pilot testing, the paper-based questionnaire was distributed through U.S. mail with paid return postage included. Dif culties in acquiring hotel managers e-mail addresses could possibly low er response rate, therefore mail distribution was chosen over an online questionnaire. Mailing addresses of the hotels were obtained from the AAA TourBook (AAA, 2011 ). Distributed questionnaires were coded for follow-up purposes and the codes were kept separate from the responses, preventing participants answers from being linked to identifying information. Before mailing the questionnaires, phone calls were randomly made to approxi- mately 200 hotels to obtain managersnames (either the general manager or the manager responsible for training evaluation) so envelopes could be personally addressed. An additional 161 questionnaires were sent to hotels that were not called or hotels that refused to give out information and these envelopes were addressed generically to the General Manager. Reminder postcards were mailed to managers 2 weeks after the questionnaires were sent. Follow up phone calls were made to 70 randomly selected hotels 1 week after reminder postcards were sent to increase response rate. This process, to maximize response rates, was consistent with that recommended by Dillman, Smyth, and Christian (2009). The Statistical Package for the Social Sciences (SPSS) program (version 20.0) was used for questionnaire data analysis. Data coding, data entry, and data valida- tion were completed using the procedures recommended by Groves, Fowler, Couper, Lepkowski, Singer, and Tourangeau ( 2004). Descriptive statistics were run on all variables, and Cronbachs alpha was used to measure questionnaire reliabil- ity. The Cronbachs alpha coef cients for the importance rating, usage rating, prac- tical characteristic, and information source ranged from 0.71 to 0.83, indicating a relatively high internal consistency (McMillan & Schumacher, 2010). Regression analysis was employed to determine the relationship between perceived impor- tance and usage of training evaluation activities. Although it was not the main pur- pose of this study, a t-test was used to examine differences in importance and evaluation-usage ratings between different hotel sizes.

Results and discussions

Participantspro les Interview participants

All six interview participants had at least 1.5 years of experience in the hotel indus- try and three had been in the industry for more than 10 years. They held the title of either General Manager, Hotel Manager, or HR Manager. Four participants had been in their positions for less than 2 years, while the other two had been in their positions for more than 5 years. Four managers had more than 10 years of training experience; the other two had at least 6 months experience. All demographic infor- mation for interview participants is presented in Table 1.

Downloaded by [University of Glasgow] at 08:44 27 December 2017

JOURNAL OF HUMAN RESOURCES IN HOSPITALITY & TOURISM

2017 JOURNAL OF HUMAN RESOURCES IN HOSPITALITY & TOURISM 193 Table 1. Demographic characteristics of hotel

193

Table 1. Demographic characteristics of hotel managers.

Characteristic

Questionnaire a

Frequency ( n )

Percent (%)

Interviews b

Frequency ( n)

Percent (%)

Hotel information Number of rooms < 100 rooms

39

65.0

2

33.3

100 200 rooms

17

28.3

2

33.3

201 300 rooms

4

6.7

1

16.7

> 300 rooms Hotel type

0

0

1

16.7

Independent hotel

10

16.9

0

0

Chain hotel

10

16.9

3

50.0

Franchised hotel

39

66.1

3

50.0

Personal information Gender Male

26

44.1

2

33.3

Female

33

55.9

4

66.7

Age c < 30 years old

9

15.3

30 40 years old

19

32.2

41 50 years old

13

22.0

51 60 years old

13

22.0

> 60 years old

5

8.5

Ethnicity d Asian/Pacic Islander

4

6.8

Black/African-American

0

0

Hispanic/Latino

1

1.7

Native American Indian

0

0

White/Caucasian

54

91.5

Education e High school

15

25

Associates degree

11

18.3

Bachelors degree

27

45

Masters degree

5

8.3

Unknown education Years working in the hotel industry

2

3.4

< 1 year

3

5.1

0

0

1 2 years

2

3.4

1

16.7

> 2 years, < 5 years

7

11.9

0

0

5 years, < 10 years

16

27.1

2

33.3

10 years, <15 years

12

20.3

1

16.7

15years

19

32.2

2

33.3

Years working in the company < 1 year

8

13.8

1

16.7

1 2 years

10

17.2

2

33.3

> 2 years, < 5 years

11

19.0

0

0

5 years, < 10 years

12

20.7

0

0

10 years, <15 years

10

17.2

1

16.7

15 years Years working in the hotel

7

12.1

2

33.3

< 1 year

10

16.9

2

33.3

1 2 years

11

18.6

2

33.3

> 2 years, < 5 years

15

25.4

0

0

5 years, < 10 years

14

23.7

1

16.7

10 years, <15 years

6

10.2

0

0

15 years

3

5.1

1

16.7

Years working with training < 1 year

1

1.7

1

16.7

1 2 years

2

3.4

1

16.7

> 2 years, < 5 years

11

18.6

0

0

5 years, < 10 years

11

18.6

0

0

10 years, <15 years

9

15.3

1

16.7

15 years

25

42.4

3

50

a n D 58 60. b n D 6. c,d,e Age, ethnicity, and education were not obtained for interview participants.

Downloaded by [University of Glasgow] at 08:44 27 December 2017

194

by [University of Glasgow] at 08:44 27 December 2017 194 A. D. D. HO ET AL.

A. D. D. HO ET AL.

Questionnaire participants

A total of 361 questionnaires were mailed, 61 of which were returned. One ques-

tionnaire was returned with less than half of the questions answered and was omit- ted from the analysis. With 60 usable questionnaires, the response rate was 16.6%, consistent with a similar study by Paez and Arendt (2014). No replacement meth- ods were used for missing values; therefore, choices with no response were not included in the analyses. Among the 60 returned and analyzed questionnaires, 39

(65%) were from managers of hotels with less than 100 rooms, 17 (28.3%) were

from managers of hotels with 100 to 200 rooms, and 4 (6.7%) were from managers

of

hotels with 201 to 300 rooms. The two latter categories were grouped together

as

one group for data analysis due to the small number of participants representing

large hotels. Therefore, two groups are presented for the t-test analysis: small-sized hotels with less than 100 rooms ( n D 39) and medium-sized hotels with between 100 and 300 rooms (n D 21). Twenty-six males (44.1%) and 33 females (55.9%) completed the questionnaire; one respondent did not complete the gender question. Most of the participants were White/Caucasians (91.5%), and more than half had at least a Bachelor s degree (53.3%). Almost 80% of the participants had at least 5 years of hotel indus- try experience and 75% had at least 5 years of training experience. All demographic information for questionnaire participants is presented in Table 1.

Interview ndings Theme 1: Linking training and business results

The rst three steps of Kirkpatrick s model (reactions, learning, behaviors) were practiced formally and informally among the six interviewed hotel managers. For the rst step, reaction, four out of six managers used observation to evaluate their employees reactions toward training; one manager used a paper-based survey. A general manager from a medium-sized hotel said he did not evaluate the effect of training, providing this rationale:

I think it is unbene cial to ask them what they feel about the training because millions and trillions of time and energy and money has already gone into the training and it s proven that it works.

For step two, learning, some managers used observation or corporate-developed tests to evaluate what employees learned from the training. As in step one, observa- tion was usually employed to evaluate the application of training knowledge on the job. In addition, guest surveys (surveys sent to guests who stayed at the hotels) were used by all managers to evaluate their training processes, usually to evaluate knowledge (step two), behavior (step three), or as an indicator of business results (step four). One HR manager from a large-sized hotel used a corporate process similar to the one Kirkpatrick (1959a , 1959b, 1960a , 1960b) proposed in his model,

Downloaded by [University of Glasgow] at 08:44 27 December 2017

JOURNAL OF HUMAN RESOURCES IN HOSPITALITY & TOURISM

2017 JOURNAL OF HUMAN RESOURCES IN HOSPITALITY & TOURISM 195 suggesting the model was practical and

195

suggesting the model was practical and managers were practicing steps similar to those described by Kirkpatrick, even when they were not aware of the model itself. All interviewed managers employed formal or informal observations during most of their evaluation processes to provide timely feedback. One general man- ager of a small-size hotel had the following to say about observations:

I really don t have an evaluation that I really evaluate them on. I guess it s more, really watching them in training, watching them role-playing, when they work with, with the guests and everything. I guess it s more of what I see them doing, what I hear them doing, more than evaluating them on a piece of paper.

When interviewed participants were asked why they did not link training with business results and monetary outcomes, one general manager of a medium-size hotel stated:

anytime you invest money into training your employees and making them more effec- tive at their tasks you are always, somehow down the road, directly link better nancial performance for the hotel because people are going to want to come back more and more and more but to ask, Is there a speci c tool of measurement that would link those two that we would directly ll out?No. No there isnt anything like that. And I m not even aware of anything like that.

He further con rmed that he did not have the tools and know-how to quantify training results, suggesting he was not aware of the tools available for linking train- ing to business results and ROI, and believed it was dif cult to quantify training results into monetary gures. Another reason that managers did not establish a link between training and business results was skepticism about the value of training evaluations. One partic- ipant, a general manager of a small-sized hotel, had this to say:

How it relates economically, it doesn t make sense to me that if you already have the high customer service scores told to you by your customers that I can come up with any other way to measure it, that I can link economically that isnt just made up. And I m not into made up analysis to try to prove a point one way or the other.

Such skepticism is supported by Swanson s (2005) statement suggesting that some training professionals did not believe in evaluation, which explains why they did not conduct them. It is reasonable for managers to disregard the potential of correlating training results with business results if their current processes are work- ing well. It would take additional resources, time, and money to perform more evaluations, while the value of the endeavor could go unrecognized. It is the researcher s task to ensure that training evaluations value is clearly demonstrated to managers. There were other reasons for not linking training with business results, such as corporate not having procedures in place for evaluation, and managers too busy to establish an evaluation process. One HR manager from a medium-sized hotel stated that she did not have enough knowledge of other departments to set up a

Downloaded by [University of Glasgow] at 08:44 27 December 2017

196

by [University of Glasgow] at 08:44 27 December 2017 196 A. D. D. HO ET AL.

A. D. D. HO ET AL.

system to link their department training to business results, and she was too busy to work with other managers to establish that link. Different from previous literature (Spitzer, 1999), interviewed managers acknowledged the value of having a training evaluation process; they based this claim on either their experiences or industrial practices. One interview participant who was a general manager of a small-sized hotel made the following comment:

There s probably no formal way to link it [employees behavior to business results]. Infor- mal way, I know that good customer service is essential for repeat business number one, to get customer to repeat. It s my understanding within the industry it s one of the big- gest reasons, if not the biggest why hotel guests choose not to either return to a speci c hotel, or return to a brand is their perception of customer service.

All interviewed managers had the same opinion about training costs: Training was an expensive process, costly in the short term, but protable to the organiza- tion in the long term. In addition, it would be less costly if managers could train employees well and retain them, compared to the high cost of hiring new employees.

Theme 2: Training evaluationhotel size comparisons

According to the general manager of a small-sized hotel, the characteristics of an evaluation model depend somewhat on the size of the organization. Hotels of different sizes might require different types of evaluation for their training. One interview participant, who was a general manager of a small-sized hotel, discussed the benets proximity allowed, due to the smaller size of the hotel where he worked:

Because I work with these employees everyday, I sat 10 feet away from them when they re actually doing their job. If I were in an organization where my of ce was down the hall- way, and my job was to evaluate them, I would have to be walking up here, observing a lot more. I would have to have a lot more processes in place to gather the information necessary to evaluate so it kind of depends on the size, it personally depends on the size of the organization, the organizational structure.

In small and medium sized hotels it was easier for managers to interact directly with employees they train and evaluate. However, HR managers of the two larger hotels indicated they usually observed from afar or let the employeesdirect man- ager (e.g., front-desk manager) observe and evaluate employees. In all cases, the employees direct managers, whether the general manager or front-desk manager, had close interactions with their employees and directly trained them. However, the person evaluating the training differed from one hotel to another. For small and medium hotels, trainers worked directly with the employees they evaluated. For larger hotels, the trainer was not necessarily the person who evaluated and sometimes did not work in the same division of the organization. In each case, the

Downloaded by [University of Glasgow] at 08:44 27 December 2017

JOURNAL OF HUMAN RESOURCES IN HOSPITALITY & TOURISM

2017 JOURNAL OF HUMAN RESOURCES IN HOSPITALITY & TOURISM 197 process needed to be different to

197

process needed to be different to accommodate the working relationships between trainer, trainee, and evaluator. Some similarities were found among various sized hotels despite process differ- ences. Almost all interviewees emphasized the need for practical standardization and consistency of process characteristics. It varied from consistency in following up with areas of improvement to standardization in the training and evaluating of various jobs. There are many potential factors in the hotel industry affecting the quality of customer service and training is one of those. Employees training other employ- ees produce inconsistent levels of service quality, depending on the knowledge, experience, or simply the personality of the trainer. Therefore, having standard- ized training and training evaluation processes will help managers ensure consis- tency in guest service. In addition, standardized training record keeping makes it easier to compare individual employees performances, and link the results to training.

Questionnaire ndings Hotel manager s perceptions and usage of training evaluation activities

Managers were asked to rate the importance of different training evaluation activi- ties on a 7-point scale (1 D not at all important, 2 D very unimportant, 3 D some- what unimportant, 4 D neutral, 5 D somewhat important, 6 D very important, 7 D extremely important). The results indicated managers found most training eval- uation activities somewhat important to very important. The results showed high- est ratings for perceived importance of observation, discussion with employees, and guest comment cards/surveys. Particularly, managers rated discussion with their employees (M D 6.48, SD D 0.85) as the most important method to evaluate train- ees reactions to training, while evaluation form ( M D 4.68, SD D 1.60) was per- ceived as the least important method to evaluate traineesreactions. Observation was one of the most important methods used by managers to evaluate traineesreactions ( M D 6.32, SD D 0.98), learning (M D 6.63, SD D 0.64), and behaviors on the job ( M D 6.60, SD D 0.62). Kirkpatrick (1959b) suggested test after training as one of the most effective methods to evaluate learning acquired from training, however in the present study it was rated as the least important method, with a mean score of 4.70 (SD D 1.61). This nding could be attributed to the nature of the hotel industry that requires a large amount of on-the-job training (Clements & Josiam, 1995). The level of importance rating could be related to each methods usage. For customer service training, the training is primarily conducted on the job rather than in a classroom setting, making it dif cult to use paper-based or com- puter-based methods (e.g., an evaluation form or test). On the other hand, a man- ager can conduct formal or informal observational evaluations without interrupting a trainee s work, which may explain why participating managers regarded this as a highly important method.

Downloaded by [University of Glasgow] at 08:44 27 December 2017

198

by [University of Glasgow] at 08:44 27 December 2017 198 A. D. D. HO ET AL.

A. D. D. HO ET AL.

Table 2. Importance and usage ratings of training evaluation activities.

Importance rating a

Usage rating b

Evaluation category Training evaluation activities

Mean

SD

Mean

SD

Reaction Observation Discussion with employees Evaluation form Learning Observation Guest comment cards/surveys Test after training Behaviors Observation Guestsdirect feedback Guest comment cards/surveys Business results Monetary results Outcomes compared to cost Follow-up actions

6.32

0.98

6.25

1.20

6.48

0.85

6.40

0.72

4.68

1.60

3.53

1.72

6.63

0.64

6.71

0.56

6.07

1.18

5.82

1.32

4.70

1.61

3.92

1.92

6.60

0.62

6.50

1.00

6.40

0.83

6.15

1.01

6.18

0.95

5.90

1.23

5.88

1.15

5.08

1.39

5.63

1.16

4.82

1.63

5.28

1.42

4.43

1.90

6.28

0.94

5.87

1.13

a n D 60; scale: 1 D not at all important, 2 D very unimportant, 3 D somewhat unimportant, 4 D neutral, 5 D some- what important, 6 D very important, 7 D extremely important. b n D 60; scale: 1 D never, 2 D rarely, 3 D occasionally, 4 D sometimes, 5 D frequently, 6 D usually, 7 D every time.

Guest opinions, elicited through comment cards, surveys, or direct (verbal or e- mail) feedback, also constituted an important method, with a mean score of more than 6.0 (descriptive measures are presented in Table 2 for all variables; further de nitions of variables can be found in Table 3). Considering the customer-

Table 3. Summary of variables for regression analyses.

Descriptions of dependent variables

Descriptions of independent variables

Reactionevaluating trainees reaction toward training Usage of observation Usage of discussion with employees

Usage of evaluation form Learningevaluating trainees learning acquired from training Usage of observation Usage of guest comment cards/surveys

Usage of test after training Behaviorsevaluating trainees behaviors acquired from training Usage of observation Usage of guestsdirect feedback

Usage of guest comment cards/surveys

How often the managers link employees behaviors to business results How often the managers link training results with monetary results How often the managers analyze outcomes of training compared to the costs of that training

How often the managers take follow-up actions based on training evaluation results

Perceived importance of observation Perceived importance of discussion with employees Perceived importance of evaluation form

Perceived importance of observation Perceived importance of guest comment cards/ surveys Perceived importance of test after training

Perceived importance of observation Perceived importance of guestsdirect feedback Perceived importance of guest comment cards/ surveys How important it is for the managers to link employees behaviors to business results How important it is for the managers to link training results with monetary results How important it is for the managers to analyze outcomes of training compared to the costs of that training How important it is for the managers take follow-up actions based on training evaluation results

Downloaded by [University of Glasgow] at 08:44 27 December 2017

JOURNAL OF HUMAN RESOURCES IN HOSPITALITY & TOURISM

2017 JOURNAL OF HUMAN RESOURCES IN HOSPITALITY & TOURISM 199 oriented nature of the industry, it

199

oriented nature of the industry, it is logical for hotel managers to pay close atten- tion to their guestsfeedback and to use these comments as tools to evaluate train- ing methods. Managers considered follow-up actions based on training evaluation results ( M D 6.28, SD D 0.94) a very important part of the evaluation process. They thought it was relatively important to link behaviors occurring from training to business results (M D 5.88, SD D 1.15), to express training results in monetary terms (M D 5.63, SD D 1.16), and to analyze the outcomes of training against the costs of that training (M D 5.28, SD D 1.42). These results are consistent with Bersin s nding ( 2006) that managers regard business results one of the most important measures of training. The questionnaire asked managers to rate the frequency of usage for each of the evaluation activities on a 7-point scale (1 D never, 2 D rarely, 3 D occasionally, 4 D sometimes, 5 D frequently, 6 D usually, 7 D every time). Regarding observa- tions, these were usually used to evaluate participants reactions (M D 6.25, SD D 1.20), learning (M D 6.71, SD D 0.56), and on-the-job behaviors ( M D 6.50, SD D 1.0). Discussions with employees and feedback from guests were also frequently used to evaluate training effectiveness. Managers viewed evaluation form and test after training as less important ways to evaluate training; they also did not utilize these methods often in their hotels, with the mean rating scores of 3.53 (SD D 1.72) and 3.92 (SD D 1.92), respectively. The reason these two methods were not utilized more often can be attributed to the nature of the industry, involv- ing on-the-job training and a non-classroom learning environment. It could also explain the high usage of observation by managers, a method that could be con- ducted without affecting the trainee s work. The descriptive measures of activity usages are presented in Table 2. Questionnaire analysis results indicated that managers frequently linked behav- iors acquired from training to business results, and results from training to mone- tary results, with mean scores of 5.08 (SD D 1.39) and 4.82 ( SD D 1.63), respectively. This nding differed from the results of Bersin s study ( 2006) that showed only a small number of companies conducting evaluations of business results. The ndings from interviews could be utilized as an attempt to explain this difference. When given the same questions, interviewed managers initially indi- cated they used some nancial measurements to evaluate hotel performance, as well as individual employees performance (e.g., turnover rate, guest survey scores). However, when probed further, interviewed managers stated they did not directly link their customer service training program with business results and ROI (stated as monetary outcomes in both the interview and questionnaire questions), which was similar to previous ndings that managers were not practicing evaluation on Kirkpatrick s fourth and fth levels (Bersin, 2006; Kline & Harris, 2008). The same rationale could apply to questionnaire results. The managers might use business or some other nancial measures to evaluate traineesperformance; however, it is unknown whether those results were directly linked to the training given.

Downloaded by [University of Glasgow] at 08:44 27 December 2017

200

by [University of Glasgow] at 08:44 27 December 2017 200 A. D. D. HO ET AL.

A. D. D. HO ET AL.

An independent samples t -test indicated importance-rating mean scores for small-sized hotels (less than 100 rooms) and middle-sized hotels (100 300 rooms) differed signi cantly ( p < 0.05) on three of the 13 listed activities. The activities that had signi cantly different importance-rating mean scores between the two groups were evaluation form in evaluating participants reaction (t(58) D ¡ 2.037, p D 0.046), guests direct feedback in evaluating on-the-job behaviors ( t(57) D ¡ 3.167, p D 0.002), and observation in evaluating on-the-job behaviors ( t(58) D ¡ 2.944, p D 0.005). The middle-sized group had higher mean rating scores in these three categories compared to the small-sized group, which could be explained by the difference in sizes. In smaller hotels, the direct manager is usually the one who trains and evaluates employees, whereas in larger hotels, the person who con- ducts training and training evaluations may not have a reporting relationship with the employees. Because of the working relationship, it could be more important for training evaluation practitioners in larger hotels to employ a more formalized method, such as forms, to evaluate trainee s reactions.

Relationship between hotel managers perceptions of training evaluation activities and their usage of those activities

To address the second research objective, determine how hotel managerspercep- tions of training evaluation activities affect their usage of those activities, additional analysis was performed. Regression analysis was conducted for each pair of varia- bles as an attempt to examine the relationship between perceived importance of training evaluation activities and usage in hotels. There were a total of 13 simple regression analyses conducted for 13 pairs of variables. The independent variables were perceived importance of training evaluation activities and the dependent vari- ables were usages of training evaluation activities. Similar to the theory of planned behavior, in which a persons attitude toward a behavior affects his/her intention to perform the behavior, which in turn affects the chance of the behavior being car- ried out (Ajzen, 1991), perceived importance was viewed as the independent vari- able explaining the usage of evaluation activity. However, the usage of a training evaluation activity could be used to explain the perceived importance of that activ- ity, exemplied by a manager in a corporate setting who receives procedures from the corporate of ce and therefore perceives the procedures as important. Descrip- tions of all variable pairs are presented in Table 3. The results ( Table 4 ) indicated that managersperceived importance of training evaluations frequently have a signi cant positive relationship with evaluation activity usage. The ndings demonstrate that managersperception of training evaluation could be used to effectively explain the usage of training evaluation activities. Guest comment cards/surveys to evaluate behaviors (adjusted R 2 D 0.50), observation to evaluate reaction (adjusted R 2 D 0.43), and business results (adjusted R 2 D 0.41) are the evaluation activities in which the managersperceived perception contributes signi cantly to activity usage.

Downloaded by [University of Glasgow] at 08:44 27 December 2017

 

JOURNAL OF HUMAN RESOURCES IN HOSPITALITY & TOURISM

JOURNAL OF HUMAN RESOURCES IN HOSPITALITY & TOURISM

201

Table 4. Results of simple regression analyses.

 
 

Model

Variables a

Constant

bT

 

F

Signicance

R 2

Adjusted R 2

Reaction Observation Discussion with employees Evaluation form Learning Observation Guest comment cards/surveys Test after training Behaviors Observation Guestsdirect feedback Guest comment cards/surveys Business results Monetary results Analyzing outcomes compared to cost Follow-up actions

1.12

0.81

6.76

45.63

0.00

0.44

0.43

4.83

0.24

2.29

5.23

0.03

0.08

0.07

1.05

0.53

4.28

18.34

0.00

0.24

0.23

4.30

0.36

3.47

12.02

0.00

0.17

0.16

2.20

0.60

4.78

22.80

0.00

0.28

0.27

0.67

0.69

5.43

29.45

0.00

0.34

0.33

0.61

0.89

5.02

25.17

0.00

0.30

0.29

1.33

0.75

6.01

36.08

0.00

0.38

0.37

0.17

0.93

7.75

60.12

0.00

0.51

0.50

0.45

0.79

6.53

42.58

0.00

0.42

0.41

¡0.04

0.86

5.95

35.41

0.00

0.38

0.37

1.46

0.56

3.53

12.44

0.00

0.18

0.16

2.70

0.50

3.53

12.43

0.00

0.18

0.16

a Dependent variables: usages of training evaluation activities. Independent variables: perceived importance of training evaluation activities. p < 0.05; p < 0.01.

Correlation analysis was also conducted for the perceived importance of all 13 training evaluation activities (the rst main question of the second section). In the reaction and learning category, ve out of six correlations were insigni cant, indi- cating that participating managers who viewed one training evaluation activity as important did not necessarily view another activity within the same category as important. For example, how important managers perceive observation in evaluat- ing trainees reactions does not correlate with how important managers perceive evaluation forms are in evaluating reactions. There were signi cant correlations between three evaluation activities used to evaluate behaviors, possibly explained by the activitiessimilarity. There was also a high correlation (r D 0.76, p < 0.01) between the evaluation of business results and monetary results, suggesting a rela- tively strong relationship between the two variables.

Reasons hotel managers apply or do not apply training evaluation activities

Not all questionnaire participants provided reasons for not evaluating training because it was an optional question (only answer if not using any of the listed activities). Among the reasons for not evaluating training, time consuming was chosen most frequently ( n D 7). No program provided by the corporate of ce was the second most often chosen reason ( n D 5), followed by high cost for training evaluation (n D 3). Both the interview and questionnaire ndings were consistent with those reported in Bersins study (2006): Managers were not assessing ROI for training. As discussed in the previous section, interviewed managers provided the following two reasons for not assessing ROI: (1) they did not have the tools to quantify train- ing results, and (2) the corporate ofce did not establish procedures for that type of

Downloaded by [University of Glasgow] at 08:44 27 December 2017

202

by [University of Glasgow] at 08:44 27 December 2017 202 A. D. D. HO ET AL.

A. D. D. HO ET AL.

assessment. These reasons were similar to those found in Bersin s (2006) study:

that many companies lacked the resources or processes needed to conduct evalua- tions. Kline and Harris (2008) also suggested that hotel managers needed to be equipped with techniques and suitable tools for measuring training ROI. The liter- ature discusses several other reasons organizations fail to carry out systematic eval- uations, including the decisions of training professionals who do not believe in evaluations, training managers who think evaluations are too dif cult to conduct, and lack of condence that training programs add value to the organization (Spit- zer, 1999; Swanson, 2005). Both the present study and Swanson s (2005) found that a reason systematic training evaluations were not used was because practi- tioners believed they were difcult to conduct. However, in contrary to Spitzer s ( 1999) nding that managers were not con dent training is of value, all managers from the present interviews implied their training processes were effective and added value to their business. It should be noted that changes in the industry and advances in training evaluations have changed since the Spitzer ( 1999) study. Despite many models available in the literature, Spitzer ( 1999) stated that most managers and practitioners used only the rst step of Kirkpatrick s model, reac- tion, to evaluate their training. However, the present studys results indicated that hotel managers carried out more than just the rst step of Kirkpatricks model. They also employed training evaluation activities in their process similar to the sec- ond and third steps of the model.

What hotel managers perceived as practical characteristics in a training evaluation process

The questionnaire data indicated that timely feedback (M D 6.14, SD D 0.78), ease in conducting ( M D 6.03, SD D 1.17), and cost effectiveness (M D 5.97, SD D 1.05) were the three most important characteristics to questionnaire partici- pants. Anonymous evaluation from outside source (M D 4.60, SD D 1.56) was the characteristic with lowest mean score. Although standardization was not on top of the list, it was considered an important characteristic to the participating manag- ers, rendering a mean score of 5.64 ( SD D 1.24). When asked to rate the impor- tance of several information sources in developing a customer service training evaluation process, the participants considered personal experience (M D 6.16, SD D 0.96) the most important one. Other sources that were considered relatively important were colleagues (M D 5.89, SD D 0.88), corporate of ce ( M D 5.30, SD D 1.66), conferences (M D 4.77, SD D 1.27), and the Internet ( M D 4.56, SD D 1.64). The mean scores for all characteristics and information sources are displayed in Table 5. Observation, discussions with employees, and guests feedback played an impor- tant role in evaluating hotel service training, suggesting it is important that researchers incorporate those factors into evaluation models or processes for hotels. Each method of training evaluation should also offer managers or practi- tioners exibility and apply to different evaluation levels. For example, one given

Downloaded by [University of Glasgow] at 08:44 27 December 2017

JOURNAL OF HUMAN RESOURCES IN HOSPITALITY & TOURISM

2017 JOURNAL OF HUMAN RESOURCES IN HOSPITALITY & TOURISM 203 Table 5. Importance ratings a of

203

Table 5. Importance ratings a of characteristics and information sources.

 

Mean

SD

Characteristics b Provides timely feedback Easy to conduct Cost effective Provides consistent evaluation Contains practical analysis Helps to continuously evaluate training results Standardized in evaluation Linked to business results Consists of outsider evaluation Information sources c Personal experiences Colleagues Headquarter/corporate ofce Conferences Internet Business magazines/newspapers Books

6.14

0.78

6.03

1.17

5.97

1.05

5.95

1.13

5.85

1.19

5.69

1.21

5.64

1.24

5.61

1.27

4.60

1.56

6.16

0.96

5.89

0.88

5.30

1.66

4.77

1.27

4.56

1.64

3.98

1.40

3.95

1.38

a Scale: 1 D not at all important, 2 D very unimportant, 3 D somewhat unimportant, 4 D neutral, 5 D somewhat important, 6 D very important, 7 D extremely important. b n D 58 59. c n D 56 57.

observational method should have the inherent exibility to be used to evaluate trainee s reactions, their knowledge acquired from training, or their behaviors. Researchers should ensure this high level of design exibility when creating train- ing evaluation tools. Standardization is important when developing a process but ensuring standardization for different job positions can be dif cult. Various jobs have distinctive functions requiring different types of evaluation, therefore it is necessary for researchers to consider job functions when designing training evaluations. Managers, especially top managers, do not have time to carry out a process requiring a multitude of st eps, therefore it should be uncompli- cated and relatively easy to administer. Perhaps a comprehensive training evaluation process incorporating all possible methods of Kirkpatrick s evalua- tion steps could be developed. Hotels have different methods of training and operation, and depending on their training approach, managers could select and customize the most appropriate evaluation methods from a core model, developing optimum evaluation processes for different training needs or job descriptions. Managersperceptions of the importance of training evaluation activities could be used to effectively explain their usage for the same activities, suggesting researchers should put more emphasis on raising practitioners awareness of what evaluation tools have been designed for them. Furthermore, personal experiences and colleagues were important information sources. Therefore, researchers should develop easily accessed and understood processes or model sources for managers and practitioners.

Downloaded by [University of Glasgow] at 08:44 27 December 2017

204

by [University of Glasgow] at 08:44 27 December 2017 204 A. D. D. HO ET AL.

A. D. D. HO ET AL.

Conclusions and implications

Implications for industry practitioners

The study results indicated that hotel managers valued and practiced the evalua- tion steps proposed in Kirkpatrick s and Phillipss models even when they might not be aware of the model itself. This provides evidence that Kirkpatricks model has come close to being a practiced standard at the studied hotels. Each industry has distinctive characteristics requiring some degree of specializa- tion for training and training evaluation. The fast-paced working environment with high-level customer interaction of the hotel industry demands timely feed- back and continuous training evaluations involving observations and guest feed- back. A training evaluation process customized for hotel managers should incorporate characteristics that help them manage their operations most ef ciently and effectively. Integration of on-the-job observations and guest feedback for eval- uation should be aligned with the training itself and evaluations should be simple enough for managers to apply on an ongoing daily basis. An example might be a simple checklist integrated on electronic devices (e.g., smart phones, tablets) to assist managers with on-the-job observations. The checklist can help managers conveniently and systematically record their employees behaviors. An observation checklist can be created anytime, anywhere, and for any employees; upon comple- tion, it can be sent to the managers mailbox for storage. Consistency is another important element to consider in developing a training evaluation process for hotels. Although evaluation methods may vary according to the job (e.g., due to their different levels of customer inter- action), the evaluation results should be consistent over time and consistent with training goals. Lastly, the cost of evaluating training should be taken into consideration for training evaluation processes implementation. The possibly high cost of conducting training evaluations on a frequent basis will raise concerns for managers and practitioners and may prevent them from acknowledging and using available tools. Especially for larger hotel chains, the corporate of ce should clearly communicate the bene ts and ROI (e.g., costs versus bene ts) of training evaluation activities in order to encourage the usage of those activities. The managers in this study believed evaluations on all ve levels were impor- tant, although they practiced evaluation focused mainly on the rst three levels. Interviewed managers indicated that they have neither the tools nor the knowledge to conduct training evaluation above level 3 (behaviors) of Kirkpatrick s model. According to current literature, this situation occurred not only in the hotel indus- try (Bersin, 2006). Hotels, or organizations in general, need to make sure their trainers are able to conduct all necessary evaluation activities. In other words, they need to train the trainerson how to evaluate their training effectiveness at all lev- els, ranging from how the trainees feel about the training to how the training bene- ts the business nancially. Having the skills to evaluate training effectiveness will

Downloaded by [University of Glasgow] at 08:44 27 December 2017

JOURNAL OF HUMAN RESOURCES IN HOSPITALITY & TOURISM

2017 JOURNAL OF HUMAN RESOURCES IN HOSPITALITY & TOURISM 205 help trainers to associate their training

205

help trainers to associate their training outcomes with business results and improve it accordingly, thus beneting their trainees and the organization. To ensure their trainers are capable of evaluation at all levels, an organization needs to implement a standardized training evaluation system. Many large hotel chains have their own standardized training programs and training evaluation sys- tems. In this study, the researchers were only able to examine the training evalua- tion system of six hotels. The managers con rmed that they did not have the appropriate tools to evaluate above level 3 of Kirkpatrick s model. Possibly due to the industry nature, participants in this study resorted to less standardized evalua- tion activities such as unsystematic self-observation or guest feedback, while neglecting the more systematic activities like test or evaluation form. In most hotel chains, managers depend on corporate of ces to provide appropriate knowledge and tools. Without a well-developed training and training evaluation system, prop- erty managers will depend on their personal experiences to assess their employees. This will lead to different levels of standards applied across properties of the same hotel chain, resulting in unstable performance. Managers should incorporate the use of standardized test and evaluation forms into the system, even for more sub- jective activities, such as observation. Development and application of a standard- ized evaluation system throughout the whole organization (from corporate to property level) will help hotel managers ensure the delivery of promised services and maintain that level of service consistently. With the increasing use of smart phones and tablets, the development of an application could simplify the training evaluation process for hotel managers, as they can conduct evaluation activities anytime and anywhere at their convenience. In addition, managers and trainers could benet from professional development opportunities geared toward training evaluation activities. A standardized training evaluation system alone is not sufcient; it needs to be implemented and then rein- forced. The use of training evaluation activities should also be encouraged through organizational culture and a rewards system. It is a tough, but not impossible, task to change the evaluation behaviors in any organization, but it can be done more thor- oughly with the support from all levels of that organization. Organizations can rein- force the use of a training evaluation system by demonstrating its practical and monetary benets. Clearly displaying the benets and making the system easy to use will encourage managers and practitioners to use them more often, thus making eval- uation practices a part of the organization culture.

Implications for researchers

This study provided researchers with insights into how hotel managers evaluate the training of their employees and what they believe a practical evaluation process should possess. Recognizing what managers require from a process will help researchers develop practical training evaluation models. This will, in turn, benet hotel managers, practitioners, and the industry in general. The ndings provide a

Downloaded by [University of Glasgow] at 08:44 27 December 2017

206

by [University of Glasgow] at 08:44 27 December 2017 206 A. D. D. HO ET AL.

A. D. D. HO ET AL.

better understanding of the perceptions held by managers representing a variety of hotel sizes. Interviewed managers indicated hotel size was an important aspect affecting the characteristics of a training evaluation model. Personal interactions between evaluator and trainee occur more often in smaller-size hotels, while more formalized and comprehensive processes are often the norm in middle-size and large-size hotels, and models must take this into consideration. Further research is necessary to distinguish the differences. It is suggested more customized models or processes be developed speci c to various-sized hotels. The ndings also showed that managers were depending more on process passed down from corporate of ce and on their own experiences when conducting training evaluation. On the other hand, most studies done on training evaluation were targeted toward other researchers and educational audiences. Those research ndings are usually presented at academic conferences or on scholastic journals, which are not commonly approached by practitioners. One primary task of researchers is to communicate their research ndings in language understandable to practitioners and through channels accessible to them. In that way, researchers will be able to contribute more to practical development of training evaluation, and practitioners can provide feedback to help further develop existing theories. Additionally, as the process is passed down from higher levels, it is also important for researchers to approach upper level managers with the appropriate tools.

Limitations

Further larger scale investigations are needed to gain a broader and more compre- hensive understanding, and to explore the extent to which these study results can be generalized, e.g., conducting research on managers of major-size hotels or in other geographic areas. In addition, because the link between perceived importance and usage is undetermined, further research is necessary to determine the relation- ship between the two variables. There were two groups (large- and major-sized hotels) not presented in the questionnaire sample because of the limited number of large-size hotels in the area chosen to conduct the research. Due to this limitation, differences between hotels of different sizes could only be examined based on interview data. Greater hotel- size diversity will be benecial for future researchers to obtain quantitative data on a larger scale.

References

Aizen, I. (1991). The theory of planned behavior. Organizational Behavior and Human Decision Processes , 50 , 179 211. American Automobile Association (AAA). (2011). North central Iowa, Minnesota, Nebraska, North Dakota, South Dakota tour book . Heathrow, FL: AAA Publishing. American Hotel & Lodging Association. (2012). 2012 lodging industry pro le . Retrieved from http://www.ahla.com/content.aspx?id D34706

Downloaded by [University of Glasgow] at 08:44 27 December 2017

JOURNAL OF HUMAN RESOURCES IN HOSPITALITY & TOURISM

2017 JOURNAL OF HUMAN RESOURCES IN HOSPITALITY & TOURISM 207 American Society for Training and Development

207

American Society for Training and Development (ASTD). (2009). The value of training: Making training evaluations more effective . Alexandria, VA: ASTD Press. Bartel, A. P. (2000). Measuring the employer s return on investments in training: Evi- dence from the literature. Industrial Relations , 39 , 502 524. doi:10.1111/0019-

Bersin, J. (2006). Companies still struggle to tie training to business goals. Training , 43 (10), 22. Chang, Y.-H. E. (2010). An empirical study of Kirkpatrick s evaluation model in the hospitality industry. FIU electronic, theses and dissertations (Paper 325). Retrieved from http://digital

commons. u.edu/etd/325 Clements, C. J., & Josiam, B. M. (1995). Training: Quantifying the nancial bene ts. Interna- tional Journal of Contemporary Hospitality Management , 7, 10 15. Creswell, J. W. (2007). Qualitative inquiry and research design: Choosing among ve traditions (2nd ed.). Thousand Oaks, CA: Sage. Cronin, J. J., & Taylor, S. A. (1992). Measuring service quality: A re-examination and extension. Journal of Marketing, 56(3), 55 68. Dillman, D. A., Smyth, J. D., & Christian, L. M. (2009). Internet, mail, and mixed-mode surveys:

The tailored design method. Hoboken, NJ: John Wiley & Sons. Frash, R., Antun, J., Kline, S., & Almanza, B. (2010). Like it! Learn it! Use it? A eld study of hotel training. Cornell Hospitality Quarterly , 51 , 398 414. Green, M., & McGill, E. (2011). The 2011 state of the industry. T C D, 65(11), 44 50. Groves, R. M., Fowler, F. J. J., Couper, M. P., Lepkowski, J. M., Singer, E., & Tourangeau, R. (2004). Survey methodology . Hoboken, NJ: Wiley-Interscience. Heide, M., & Grønhaug, K. (2009). Key factors in guests perception of hotel atmosphere. Cor- nell Hospitality Quarterly , 50 , 29 43. doi: 10.1177/1938965508328420 Holton, E. F. III. (1996). The awed four-level evaluation model. Human Resource Development Quarterly , 7 , 5 21. doi:10.1002/hrdq.3920070103 Jameson, S. M. (2000). Recruitment and training in small rms. Journal of European Industrial Training , 24 , 43 49. doi:10.1108/03090590010308255 Kandampully, J., & Suhartanto, D. (2000). Customer loyalty in the hotel industry: The role of customer satisfaction and image. International Journal of Contemporary Hospitality Manage- ment , 12 , 346 351. doi:10.1108/09596110010342559 Kearns, P. (2005). From return on investment to added value evaluation: The foundation for organizational learning. Advances in Developing Human Resources, 7 , 135 145. doi:10.1177/

Kirkpatrick, D. L. (1959a). Techniques for evaluation training programs. Journal of the Ameri- can Society for Training and Development , 13(11), 3 9. Kirkpatrick, D. L. (1959b). Techniques for evaluation training programs: Part 2 learning. Jour- nal of the American Society for Training and Development , 13 (12), 21 26. Kirkpatrick, D. L. (1960a). Techniques for evaluation training programs: Part 3 behavior. Jour- nal of the American Society for Training and Development , 14 (1), 13 18. Kirkpatrick, D. L. (1960b). Techniques for evaluation training programs: Part 4 results. Jour- nal of the American Society for Training and Development , 14 (2), 28 32. Kline, S., & Harris, K. (2008). ROI is MIA: Why are hoteliers failing to demand the ROI of train- ing? International Journal of Contemporary Hospitality Management , 20 (1), 45 59.

McMillan, J. H., & Schumacher, S. (2010). Research in education: Evidence-based inquiry. Bos- ton, MA: Pearson Education. Oh, H. (1999). Service quality, customer satisfaction, and customer value: A holistic perspective. Hospitality Management , 18, 67 82. doi:10.1016/S0278-4319(98)00047-4

Downloaded by [University of Glasgow] at 08:44 27 December 2017

208

by [University of Glasgow] at 08:44 27 December 2017 208 A. D. D. HO ET AL.

A. D. D. HO ET AL.

Onyango, F. E., & Okech, R. N. (2008). Human resource systems in Kenya: A case study of hotel human resources performance. Cornell Hospitality Quarterly , 49, 413 427. doi:10.1177/

Paez, P., & Arendt, S. W. (2014). Managers attitudes towards employees with disabilities in the hospitality industry. International Journal of Hospitality & Tourism Administration , 15(2), 172 190. doi:10.1080/15256480.2014.901065 Phillips, J. J. (1996). ROI: The search for best practices. Training & Development , 50(2), 42 47. Pine, J., & Tingley, J. C. (1993). ROI of soft-skills training. Training , 30(2), 55 60. Ramlall, S. J. (2003). Measuring human resource management s effectiveness in improving per- formance. Human Resource Planning, 26(1), 51 63. Russ-Eft, D., & Preskill, H. (2005). In search of the Holy Grail: Return on investment evaluation in human resource development. Advances in Developing Human Resources , 7, 71 85.

Saks, A. M., & Burke, L. A. (2012). An investigation into the relationship between training eval- uation and the transfer of training. International Journal of Training and Development , 16 (2), 118 127. doi:10.1111/j.1468-2419.2011.00397.x Salas, E. & Kosarzycki, M. P. (2003). Why don t organizations pay attention to (and use) nd- ings from the science of training? Human Resource Development Quarterly , 14 , 487 491.

Spitzer, D. R. (1999). Embracing evaluation. Training, 36(6), 42 47. Swanson, R. A. (2005). Evaluation, a state of mind. Advances in Developing Human Resources , 7, 16 21. doi:10.1177/1523422304272078 Ulrich, D. (1997). Measuring human resources: An overview of practice and a prescription for results. Human Resource Management , 36 , 303 320. doi:10.1002/(SICI)1099-050X(199723) 36:3< 303::AID-HRM3 >3.0.CO;2-#

Wang, G. G., & Spitzer, D. (2005). Human resource development measurement and evaluation:

Looking back and moving forward. Advances in Developing Human Resources , 7 , 5 15.