Beruflich Dokumente
Kultur Dokumente
The Development of a Pay-for-Performance Appraisal System for Public Park and Recreation Agencies: A Case Study
Michael A. Mulvaney William R. McKinney Richard Grodsky
EXECUTIVE SUMMARY: Well-designed employee performance appraisals provide agencies with information that can guide administrative and developmental decision-making about their most important assettheir human resources. Administratively, performance appraisals serve as the formal evaluation tool used by managers when making decisions about distribution of pay increases. Developmentally, performance appraisals assist agencies in identifying issues such as employee training needs, selection, and promotional opportunities (Milkovich & Newman, 2005). Despite its importance, an agencys performance appraisal system is often viewed by employees and management as a frustrating and unfair process. These frustrations are largely attributed to performance appraisal instruments that are not job related, have confusing or unclear rating levels, and are viewed as subjective and biased by staff (Mathis & Jackson, 2006). A study was undertaken to respond to this issue by identifying the steps involved in creating a pay-for-performance system for a public park and recreation agency. This case study (1) identied a systematic procedure for creating performance appraisal instruments, (2) described the appropriate training necessary for those conducting a performance appraisal interview, (3) implemented the performance reviews using the developed instruments and the results of the appraisal training, and (4) determined how much of a merit increase is to be given for different levels of performance. Emergent themes from focus group interviews with employees identied differences between employee attitudes toward the original payfor-performance system and the newly developed system, highlighting the value of employee participation in the development of the appraisal system. Results of the case study are analyzed and discussed. Authors: Michael. A. Mulvaney is with the Department of Recreation Administration, Eastern Illinois University, 600 Lincoln Avenue, Charleston, IL 61920.Email: mamulvaney@eiu.edu. Phone: (217) 581-6589. William
127
McKinney is with the Department of Recreation, Sport, and Tourism at the University of Illinois. Richard Grodsky is the Executive Director for the Elmhurst Park District.
Human
Resource
Performance appraisal has become a general description for a variety of activities through which agencies provide feedback to their employees, develop competencies, enhance performance, and distribute rewards (Grote, 2000). An agencys performance appraisal system impacts individual and organizational operations by guiding decisions about compensation and merit pay increases, training and development opportunities, performance improvement, termination, promotion, organizational climate, and nancial management. Social psychologists using experimental methods have found that performancebased pay enhances effort and improves work quality (Landau & Leventhal, 1976; Mitra, Gupta, & Jenkins, 1997). Economic theorists have argued that performancecontingent rewards motivate effort and attract talent when effort and talent are not easily increased or advanced (Holstrom, 1979; Zenger, 1992). Authors studying organizational management have also cited the benets of distributing rewards based on performance (Milkovich & Newman, 2005). The results of these ndings have worked their way into professional practice: employer surveys indicate that almost 90% of U.S. agencies use a pay-for-performance system to award pay increases (Cascio, 2006). Despite the many uses of a performance appraisal system, the process is often seen by park and recreation administration and staff as a painful annual exercise in which the manager rates the performance of subordinates over the past 12 months and uses that information for administrative and developmental decisions. Furthermore, poorly designed and administered pay-for-performance systems are likely to be demotivating and generate anxiety among employees (Bassett, 1994). Dolan (1996) identied ve characteristics of decient pay-for-performance systems: (1) the performance measure(s) are unclear or not job specic, (2) supervisors may not be adequately trained to evaluate employee performance, (3) the incentive value of the reward offered may be too low, (4) the link between performance levels and rewards may be weak or unnoticeable, and (5) the agency failed to address the issue of redlining (i.e., employees moving out of their jobs pay range). Recognizing that one of the major difculties with performance appraisal stems from the variety of objectives the appraisal is supposed to address (development, promotion, termination, staff training, etc.) and that salary decisions account for nearly 80% of appraisal uses (Mathis & Jackson, 2006; Thomas & Bretz, 1994; Smith, Hornsby, & Shirmeyer, 1996), this study provides an overview of the collaborative steps involved in creating a performance appraisal system used for merit pay increase decisions within a park and recreation agency. The purpose of this study was to describe the development process and procedures undertaken for a public park and recreation agencys performance appraisal instrument, illustrating it with a representative case study. The value of employee participation throughout the appraisal systems development is also highlighted.
130
buy-in of their performance appraisal system. Specically, managers wanted to improve employee perceptions of the fairness, accuracy, and overall satisfaction of the agencys performance appraisal system. Third, managers expressed concern about the need for employees to become and remain aware of the highly demanding nature of todays park and recreation users. From increasing utilization of new technologies to agency restructuring to meet residents leisure needs, Elmhurst Park District employees have been required to learn and adapt at a rapid pace. A recognized method to address these issues is establishment of a valid pay-for-performance plan that rewards highperforming employees while not rewarding employees whose performance falls below an established standard. This case study explored the process of developing pay-for-performance instruments, conducting performance reviews, and implementing a model for the distribution of merit pay increases. Once the pay-for-performance system was implemented, the study assessed employee attitudes about the newly developed pay-forperformance system compared to the agencys previous system. Employee satisfaction with the performance appraisal system and their perceptions of the procedural and distributive justice of the system were assessed. Specically, this study used focus group interview data to address the following questions: 1.) What specic steps are involved in developing a pay-for-performance instrument for a public park and recreation agency? 2.) What procedures are effective for conducting the performance appraisal review? 3.) What steps are involved in developing a model for distributing merit pay increases for a public park and recreation agency? 4.) How do employee attitudes about the new pay-for-performance appraisal instrument differ compared to the previous pay-for-performance appraisal system? 5.) How do employee attitudes about the new performance review session differ compared to the previous performance review session? 6.) How do employee perceptions of the procedural justice of the new pay-forperformance system differ compared to the previous performance review system? 7.) How do employee perceptions of the distributive justice of the new pay-forperformance system differ compared to the previous appraisal system? Although a case study with a limited number of respondents cannot be widely generalized, the results should be of interest to management researchers and directors of park and recreation agencies who think critically about how to increase employee performance and improve management of pay-for-performance dollars available within the agencys operating budget. The need to develop a pay-for-performance appraisal system that motivates staff, is cost effective, and assists the agency in meeting its goals is arguably a problem, or opportunity, that has applications across the eld of parks and recreation.
Qualitative data was collected to compare employee attitudes between the original performance appraisal system and the newly developed system. Specically, focus group interviews with full-time employees of the Elmhurst Park District were conducted. Such interviews are useful in encouraging members of the focus group to state feelings, perceptions, attitudes, and beliefs that they might not express if interviewed individually, for fear of employer retaliation. The focus groups for this study were chosen based on their position in the agency. Specically, every employee who supervised and evaluated full-time employees was included in the focus groups. Based upon the supervisory criteria, 17 employees were
132
included in the focus groups. To reduce employee anxiety associated with having the supervisor and subordinate in the same focus group sharing sensitive information, and in an effort to increase the accuracy of the interview data, employees were divided into two focus groups. The rst focus group contained eight director-level positions (recreation director, park director, nance director, information services director, organizational planning director, marketing and communication director, project development director, and a senior accountant). Employees in this focus group had from two to 17 years of experience within their respective job titles. Supervisory responsibility in these positions varied from supervising one employee to a position overseeing seven employees. The second focus group consisted of nine mid-level management positions (ofce manager, facility manager, recreation manager, parks manager, administrative services manager, tness/racquet sports manager, operations manager, human resources manager, and a golf course manager). Job experience within this group ranged from less than a year to seven years of experience. Supervisory responsibility also varied, with one manager responsible for just one full-time employee while three managers supervised more than ve employees. Data from the interviews was collected by note-taking procedures. Note taking was selected over tape recording for the data-collection method because the content discussed in the interviews was sensitive. To help ensure validity of the note-taking procedures, a triangulated approach was adopted: three investigators recorded the interview responses, and comparisons among the three investigators were made to check for accuracy. Consistent with Lincoln and Gubas (1985) approach to ensuring the quality or trustworthiness of the qualitative data, member checks were used to solicit feedback on the interview ndings from the respondents. Specically, the focus group members were asked to review their recorded responses at the conclusion of each interview session. In addition, the interviewees reviewed the nal research report for accuracy and completeness.
Data Analysis
The focus group interview data was examined using an inductive thematic analysis approach. The raw interview data was reviewed to identify patterns that described and organized employee reactions to the original system and newly developed performance appraisal system. To identify themes or patterns generated from focus group interviews, coding was used (Boyatzis, 1998) to break down the data into manageable segments, leading to the identication of themes (Schwandt, 2001; Strauss & Corbin, 1998). Two steps were employed to enhance the accuracy and reliability of the data analysis procedures. First, a code book was developed by the authors that included a list of themes and their translations. This code book provided the authors an ongoing reference and assisted in consistently identifying emerging patterns in the interview data. Second, adopting a procedure recommended by Schwandt (2001) to improve the reliability of qualitative data analysis, the authors separately coded the interview data and later reviewed it to identify any discrepancies.
Job analysis
Job analyses are a systematic way to gather and analyze information about the content of jobs (Mathis & Jackson, 2006). The job analysis process should identify the jobs under review, the participants involved in conducting the job analysis, a review of existing documentation including current job descriptions, the identication of the major job content domains contained within the job, and a list of tasks to be fullled under each domain. (Mathis & Jackson, 2006). Job descriptions often summarize the elements identied in a job analysis. In completing job analyses, research has indicated that involvement of employees at all levels facilitates acceptance of the system and increases cooperation (Levy & Williams, 1998; Longenecker & Fink, 1999, 2003). When employees are allowed to participate in the development of the appraisal system, their sense of ownership increases. Employee involvement in the performance appraisal and development process is critical. Longenecker and Fink (1999) caution against attempts to save time by bypassing employee and manager input because it can lower the systems credibility. If managers, acting alone, produce a system that does not meet staff needs, it damages the perceived connection between pay and performance and loses the performanceenhancing effects of employee commitment to agency goals. Lack of employee involvement creates a domino effect; researchers have found if either the managers or employees are dissatised with the system, the effectiveness of the overall appraisal and feedback process is diminished (Gosselin & Murphy, 1994; Petit & Haines, 1994; Tziner & Kopelman, 2002; Wiersma & Latham, 1986).
134
Subscribing to the approach to dening job performance suggested by Viswesvaran and Ones (2000), job analyses were completed for every full-time job title in the Elmhurst Park District. To complete each job analysis, a meeting between the employee(s) and the supervisor was conducted for every full-time position. In conducting the job analyses, the employee(s) and supervisor collectively reviewed the current job description. Job descriptions of similar positions from other agencies were also reviewed to guide employees in brainstorming a list of job domains and tasks performed. During this meeting, the employee(s) and supervisor collectively identied between six and ten job domains that represented the major content areas of responsibility for the job (programming, human resource management, nance, etc.). Once the general content domains were identied, the employee(s) and supervisor discussed and identied a list of specic tasks within each job domain. On average, 10 to 15 tasks were identied for each job domain. Consistent with previous research, the task descriptions (1) began with an action verb, (2) included only one specic task, and (3) stated what the employee did (Brademas, Lowrey, Gress, & Bostrom, 1981; Drauden & Peterson, 1974; Grant, 1988; Mathis & Jackson, 2006).
135
are often divided into two general formats: ranking and rating. Ranking formats require the rater to compare employees against each other on a performance measure. Rating formats have two elements: (1) they require raters to evaluate employees on some absolute standard rather than relative to other employees, and (2) each performance standard is measured on a scale on which appraisers can check the point that best represents the employees performance level (Milkovich & Newman, 2005). In deciding which appraisal format is most appropriate for an agency Keeley (1978) and Tziner and Kopelman (2002) suggest that an understanding of the type of tasks being performed is needed. Because the task statements developed during the job analysis phase included written statements of what the employee does, the Elmhurst Park District incorporated an anchored rating format. An anchored rating format describes performance variation along a continuum from good to bad (Milkovich & Newman, 2005). The type and number of descriptors used in anchoring the continuum provide the major differences in rating scales. Organizational research has shown that the reliability of a performance appraisal instrument is strongest when using between three and seven descriptive anchors (Arvey & Murphy, 1998). The Elmhurst Park District selected a three-anchor approach. As a result, each task was evaluated against three anchors on the performance continuum: (1) below standard, (2) meets standard, and (3) exceeds standard. A not applicable rating was created to address tasks that were not observed or performed during the evaluation period.
136
rating employees, thus giving them some rm ideas about what level of performance is required for each task to be completed by the employee (see Jackson, Schuller, & Rivero, 1989; Milkovich & Newman, 2005).
Rater training
The next step in the construction of the performance appraisal system is to understand how and where the raters make mistakes (Milkovich & Newman, 2005) and how to reduce those errors. One approach to limiting errors in the performance appraisal process is through appraiser training (Bernadin & Pence, 1980; Wilson & Western, 2001; Zedeck & Cascio, 1982). Surprisingly, managers frequently report that they receive very little training beyond a description of the rating form (Longenecker & Fink, 1999). An effective formal performance appraisal system cannot exist without ongoing education for all key appraisers in the appraisal process (Longenecker & Fink, 1999). Developing the skills necessary to conduct effective performance appraisals, including issues of psychometric errors, can be accomplished through appraiser training (Schweiger & Sumners, 1994). Training sessions subscribing to the previously mentioned principles were conducted for employees of the Elmhurst Park District. Employees were provided with a 3-hour session that focused on three categories: (1) rater-error training, (2) performance dimension training, and (3) performance standard training. During the rater-error training session, employees were introduced to several psychometric errors (e.g., leniency, halo effect, recency) and offered suggestions to address those problematic areas. The performance dimension training session involved a collective discussion on performance dimensions between raters and ratees. Finally, the performance standard training provided raters with a standard of comparison or frame of reference for making appraisal decisions. Because the Elmhurst Park District adopted a three-anchored approach, a discussion on how below standard, meets standard, and exceeds standard are dened within the agency was conducted.
137
increase decisions. The development of the new appraisal system occurred during the last six months of the 12-month review period. Thus, formally implementing the new appraisal system as the evaluation system for the entire 12-month period would have been unfair to the employees because the performance standards created in the new system were not identical to the performance standards from the previous system. The trial run began with the employee and supervisor independently completing the appraisal instrument. To assist in the nal calculations, the appraisal instruments were created in a Microsoft Excel format. Paper copies of the spreadsheet-formatted instrument were printed and provided to the employee and supervisor. Once the employee and supervisor had independently completed the appraisal instrument, they met and discussed the ratings. During that time, the employee and supervisor agreed on a rating for each task statement. The appraisal was then completed and signed by both the employee and supervisor. The Elmhurst Park District Human Resource Specialist applied the previously established weights for each task statement to determine a nal percentile score for each employee once the nal ratings had been determined (Figure 1). Statistically, this procedure involved (1) multiplying the score of the task (i.e., 1 = below standard, 2 = meets standard, 3 = exceeds standard) by the weight of each task (tasks receiving a not applicable rating were voided from the computations), (2) determining the total points possible (e.g., the sum of each tasks weight multiplied by 3 = exceeds standard), and (3) dividing the total possible points by the total points earned by the employee to obtain a nal percentile score. For example, in Figure 1, the Division Managers nal percentile score was 68.97% .
Pay differentials
Pay differentials represent the varying levels of merit increases that are based on an employees job performance (Milkovich & Newman, 2005). An agency should consider the pay differentials it uses when rewarding various levels of job performance. An agency must ask, How much do we pay the higher performers compared to the lower performers? (Martocchio, 2001). For example, how much more does an employee receive if he or she exceeds standard compared to meets standard? This amount needs to appropriately distinguish the various levels of performance in order for the pay-for-performance system to correctly serve as a motivator for staff. In looking at this issue, researchers have suggested that an approximately 3% pay increase is needed between the levels of performance as perceived by the employees (see Arvey & Murphy, 1998).
138
Personnel Management
x 1 30
30
90
Recruit, hire and train staff Manage and evaluate staff Provide ongoing direction, foresight and motivation to staff Prepare for and conduct staff meetings and trainings as needed Adhere to and enforce staff compliance with organizational policies and procedures, and the strategic plan Ensure that employees are provided necessary training in usage of tools, equipment, machinery and personal protective equipment Assess future division stafng requirements Financial Management Not Below Applicable Standard x x x x 3 2 3 Score 1 1 1 3 36 25 6 12 5 Weight 36 49 x Meets Standard Exceeds Standard 2 Score 10 Weight 20 Weighted Score 36 75 18 24 30 Total Possible 108 75 18 36
Prepare and monitor division budget Coordinate purchase of division supplies, materials and equipment Develop and implement program pricing policies Manage internal and external assistance payments and delinquent accounts Manage bid specs for major purchases Organizational Planning Not Below Applicable Standard x x Meets Standard x Exceeds Standard
15 Weighted Score 36 49
Establish and monitor division goals and objectives Serve as member of the Parks and Recreation Services Management Team and other appointed committee assignments
x x
Attend staff meetings Prepare annual t-shirt bid Develop and implement customer service standards Communications/Customer Service Not Below Applicable Standard 2 49 Meets Standard x x Exceeds Standard Score Weight Weighted Score 98 Total Possible 147
2 1 3
30 15 25
60 15 75
90 45 75
36
72 32 48 27
108 48 48 27
Develop and maintain cooperative relationships and effective oral and written communications with internal and external customers Promote District programs to patrons, guests and staff Prepare written and verbal bullet points, updates and reports as required. Act as a liaison to afliates, community groups and governmental units Design and distribute information for public distribution Coordinate seasonal brochure productions Safety Not Below Applicable Standard Meets Standard x 60 Weighted Score 38
x x
2 2
21 49
42 98
63 147
Encourage and demonstrate safe work habits through use of established safety program guidelines Serve as member of the Districts Crisis Management Team Maintain CPR and AED certication, and ensure that all staff within supervision do the same
139
140
Programming
Develop programs to create lifetime users Develop and implement a consistent customer feedback program Ensure safe and appropriate program facilities Manager program inclusion process Registration Not Below Applicable Standard x x 3 2 3 36 25 Meets Standard x x x Exceeds Standard 72 108 Weighted Score 72 75 48 1895 2748 68.97
Manage program registration process Manage program cancellation/expansion/addition process Manage permit and rental registrations
Response Scale: 0 = Not Applicable 1 = Below Standard 2 = Meets Standard 3 = Exceeds Standard The task is not performed/not observed The performance is below standards The performance meets standards The performance exceeds standards
2748
141
In addressing pay differentials, researchers have also introduced the just noticeable differences (JND) concept (Heneman & Ellis, 1982; Katkowski, Medsker, & Pritchard, 2002; Krefting & Mahoney, 1977; Mitra, Gupta, & Jenkins, 1997). JND refers to the minimum pay increase that employees will perceive as making a meaningful change in compensation and thereby enhance their motivation to improve job performance (Heneman & Ellis, 1982). JND as a concept also has been represented as smallest meaningful pay increase (Krefting & Mahoney, 1977). Rooted in psychophysical research that centered on identifying JND in sounds, weights, and other environmental stimuli, the JND concept has been used in agency compensation programs to investigate appropriate levels of merit pay increase amounts based on performance (Katkowski, Medsker, & Pritchard, 2002). In an attempt to further understand JND, Katkowski, Medsker, and Pritchard (2002) empirically reviewed 11 JND studies. When looking at the pay increase percentage corresponding to a JND, Katkowski et al. (2002) drew from nine of the 11 studies (only nine studies provided enough information to be included in the analysis) and generated an overall JND range of 6% to 8%. Thus, based on those ndings, approximately 6% to 8% is recommended for employees who exceed performance standards, to adequately address employee perceptions of the effort/input to outcome ratio. Guided by the research on pay differentials and JND, a preliminary pay increase model was developed for the Elmhurst Park District. The model divided performance levels into three categories: below standard, meets standard, and exceeds standard. Using Katkowski et al.s (2002) meta-analysis ndings, an initial list of merit pay percentage increases was proposed to the park district. The proposal recommended that employees within the exceeds standard category receive an 8% pay increase, employees within the meets standard category receive a 4% pay increase, and employees within the below standard category receive no increase. Discussions with the Elmhurst Park District staff prompted an adjustment to the preliminary model. Specically, the district did not feel that employees within the below standard category should receive no merit pay increase. Recognizing the almost constant increase in the cost of living index and that all employees of the district were recognized as the best available when hired, employees felt that a 1% increase should be awarded to the lowest performers. This view led to the revision of the preliminary model. Thus, the exceeds standard category was reduced to a 7% merit raise and the below standard category was increased to 1%. These percentages served as the foundation for the remaining steps in the development of the districts merit pay increase distribution model.
Forced distribution
An agency must consider and identify an approach to determining placement of employees into various levels of performance. A widely used method to complete this task is forced distribution, which enables evaluators to assign a predetermined percentage of employees to each of the various performance categories (Martocchio, 2001). The distribution of employee performance is represented in a bell-shaped curve (Mathis & Jackson, 2006) that allows pay increase amounts to be statistically assigned to various points on the curve.
142
The rationale for forced distribution is that it eliminates clustering among employees at the top of the distribution (rater leniency), at the bottom of the distribution (rater severity), or in the middle (central tendency) (Cascio, 2006). Because the distribution of ratings/scores is forced into a bell-shaped curve, a relatively small portion of employees will be identied as truly outstanding, a relatively small portion as unsatisfactory, and the remaining portion will be somewhere in the middle. Guralnik, Rozmarin, and So (2004) suggest that this process is particularly helpful when pay increase budgets are limited for an agency. Forced distribution also makes managers identify high, average, and low performers. Thus, high performers can be rewarded and developed while low performers can be encouraged to improve or asked to leave. A few criteria made the forced distribution technique appealing to the pay-forperformance system for the Elmhurst Park District. First, forced distribution is most effective in an agency with more than 50 employees (Martochhio, 2001). The Elmhurst Park District has almost 60 full-time employees. Next, companies that benet from forced distribution typically have growth and performance-driven cultures that belong in a highly demanding industry. This criterion is particularly tting for most public park and recreation agencies (including the Elmhurst Park District) who are regularly confronted with increasing and ever-changing needs of the publics they serve. From population shifts (i.e., demographics such as an increase in the communitys aging population) to budgetary constraints, public park and recreation agencies must be concerned with the entire publics needs while being increasingly faced with demands to identify and establish creative revenue sources. In an effort to remain consistent with the performance levels from the appraisal instruments, three levels of performance (below standard, meets standard, and exceeds standard) were identied in the distribution. Scores below 1.0 (standardized Z scores) were categorized as below standard, scores falling between +/ 1.0 were categorized as meeting standard, and scores above +1.0 were categorized as exceeding standard. Using the performance appraisal scores from the Elmhurst Park Districts trial run, employees were placed on the distribution curve. Six employees fell into the below standard category, 1 employees into the meets standard category, and seven into the exceeds standard category. Upon reviewing these results, the park district moved the cutoff point for exceeds standard to +0.50 Z score. The move was based on the park districts assertion that it hired above average employees. It was believed that some of the employees at the Elmhurst Park District who were forced into the meets standard category would likely exceeded the standard at another agency and were, in fact, performing above standards. Therefore, the cutoff point was lowered, which resulted in sixteen employees falling into the exceeds standard category and 35 employees in the meets standard category. Results of this distribution are provided in Figure 2. Once the employees were plotted on the distribution curve, a series of distribution analyses was conducted to determine the average merit pay increase for the agency. The results of these analyses determined the average merit pay increase to be over 4.5%. Unfortunately, the Elmhurst Park Districts merit pay increase pool was restricted to 3.5% to 4%. Like most public park and recreation agencies, the Elmhurst Park Districts merit pay increase resource pool was determined by the districts board of commissioners and was based on change in the local/national consumer price index and the overall nancial health of the agency. As a result of this problem, the Elmhurst Park District established a pay adjustment matrix for their agency.
143 Figure 2. Elmhurst Park Districts Forced Distribution of Peformance Appraisal Scores
n = 33 0.50 72.21
n = 16 1.0 74.05
144
and receiving a 3.5% increase for the same performance would receive a $1,925 raise. When comparing the two raises over the course of a year, the difference comes to less than 50 cents per day. Thus, although the percentage is signicantly less, the absolute amount is approximately the same. The compa-ratio suggests that higher performance is to be expected of more senior employees and that a smaller percentage increase is available to those reaching the top of their pay range for their job title (Mathis & Jackson, 2006). Guided by the compensation literature and in response to the Elmhurst Park Districts need to monitor and control merit pay costs, a pay adjustment matrix was developed. The pay adjustment matrix had three performance levels and four pay quartiles. The JND and pay differential research served as the foundation to the matrix and were used to establish the percentages in the rst quartile. In particular, an employee with a compa-ratio of .80 was eligible to receive a 1% (below standard), 4% (meets standard), or a 7% (exceeds standard) merit pay increase. Percentages were inserted into the remaining performance levels and quartiles based on the Elmhurst Park Districts pay ranges and merit pay resource pool while adequately maintaining just noticeable differences in pay increases based on performance. The nalized merit pay increase model is provided in Figure 3.
Figure 4 shows the results of the revised pay-for-performance plan of the Elmhurst Park District. Column 1 shows the percentile score, arranged from highest to lowest, of each employee on the performance appraisal. The percentile score was determined by multiplying the actual performance score (1 = below standard, 2 = meets standard, 3 = exceeds standard) by the weight given to each performance criteria or task statement contained in the job-specic performance appraisal instrument. The maximum score was three times the weight given for each task statement. The score earned by the employee was divided by the maximum score possible for the appraisal instrument to obtain the nalized percentile score. Column 2 is the percentile score converted to standardized Z scores and divided according to the forced distribution model adopted by the Elmhurst Park District. Thus, six scores fell below the 1.0 Z score, 33 scores were within the range of 1 to .50 Z score, and 15 scores had a Z score greater than .50.
145
Column 3 shows the compa-ratio for the performance appraisal rating, which signies each staff persons merit pay increase. As previously identied, the actual pay increase received by each staff person was based not only on performance rating but also on that employees position in the pay range for his or her job title; that is, an employee who exceeded performance standards and was at the top of the pay range received a smaller percentage pay increase than an employee who merely met performance standards but was at the midpoint of his or her pay range. The percentage increase may be less for an employee at the top of his or her pay range, but as discussed in the section about compa-ratios, the actual dollar amount might be greater. The new model, using the forced distribution and compa-ratios, resulted in an overall pay increase of 3.67%an acceptable range for the Elmhurst Park District.
147
staff in the dark creates skepticism and distrust in the results. (Female, mid-level facility manager) I dread it. I dont know what to expect going in [to the appraisal interview]. Of course I want a good evaluation for obvious reasons, but it seems like regardless of what I feel my performance was for the year, the interview session throws me an unexpected curve ball. (Male, mid-level operations manager) In contrast, when asked about the newly developed instrument, a majority of employees felt that the interview session and system were greatly improved. A common theme emerging from both focus groups was the participative process of the new appraisal instruments. Consider the following statement: Since the employee was involved in creating the instrument, it has become much easier and less confrontational to comment on specic [performance] ratings since they [the employees] were part of the process to identify what was included on the performance appraisal. (Female, mid-level recreation manager)
148
Despite the majority of employees expressing positive comments about the procedures used in the new pay-for-performance instrument, some staff expressed concern. Consider the comment made by an upper-level ofce manager: The proof is in the pudding, and it is still too early to tell. It does not seem realistic to ding them on one particular task, but we are. We have delineated multiple tasks into multiple statements. Our previous instrument was [broader]. (Female, upper-level accountant) When looking at the way supervisors conducted the performance review and rated their employees, an interesting nding emerged. Some supervisors felt that their subordinates rated themselves more critically than they did. For example, one director stated, I was surprised at how some of my staff self-rated themselves compared to the scores I gave themmine were actually higher. This made for an interesting [performance appraisal interview] discussion. I was pleasantly surprisedI thought the discussion was valuable in coming to an agreement. (Male, facilities director)
149
Upon completing the trial run of the new performance appraisal instrument, interviewees were shown how their ratings would be applied to the merit pay increase and compa-ratio distribution model. Next, interview data on employee reactions and distributive justice perceptions of the newly developed system was examined. Several interesting patterns in the new systems distributive justice emerged. A majority of the coded themes centered on the three elements of the new distribution model. For example, a director stated, I agree with the pay adjustment matrix idea. I believe it is a necessity that must be present for the system to work correctly. Everyone cannot make the same [percentage raise], and certain people may get mad, but they are going to have to deal with it. (Male, upper-level director) In contrast, an upper-level administrator was cautious about the implementation of compa-ratios and pay adjustment matrices: Implementing the compa-ratio will be difcult. Staff dont like it [in the original model] now, and I dont think they will like it in the new one. I understand its importance, but I think staff will have a hard time understanding it. A lot will depend on how fair this [the new system] is and how it gets communicated to staff. (Male, upper-level parks manager) When asked why employees did not understanding the agencys current and/or new models, the upper-level administrator suggested that the problem was a result of the current models inability to adequately control merit pay increases. In addition to Elmhursts pay ranges failing to keep pace with the top raise amounts for employees above the midpoint, the previous system failed to adopt a forced distribution approach. Consequently, the park district had difculty linking the appropriate merit pay raise increase to the performance level. Several employees therefore disregarded the existing distribution model or attributed their confusion about the link between the model and their merit raise to not completely understanding the system. The upper-level administrators conclusions were supported by several employees who liked the forced distribution concept. The employees interviewed felt that forced distribution would help discriminate among performance levels. Consider the following statement: Forced distribution is an excellent tool to adjust pay in an objective and quantiable manner. We clearly meet the criteria by having over 50 employees in our appraisal system. I also think the new performance instruments make our performance evaluations much more objective than before. Forced distribution is what we need to make fair pay decisions, and I think it helps make our evaluation process easier to understand and use. (Female, accountant staff)
150 Discussion
Guided by a strategic human resource management framework asserting that agencies should align their pay system to support their strategic objectives, this study sought to provide an overview of the steps involved in creating job-specic, contentvalid, pay-for-performance appraisal instruments, an appraisal review process, and a distribution model for merit pay increases. As suggested by Milkovich and Newman (2005), a valid pay-for-performance plan requires (1) clearly dened, job-specic criteria that enable creation of an appraisal instrument that describes different performance levels from low to high, (2) a well-dened performance appraisal interview process, and (3) equitable decisions about how much of a merit increase will be given for different levels of performance. Based on research ndings that employee participation leads to elevated levels of employee acceptance and on research suggesting that fairly distributed merit pay increases improve staff and agency performance, this study adopted a participatory approach in creating the Elmhurst Park Districts pay-for-performance appraisal system. In addition, focus group interviews and observations were used to analyze employee satisfaction with the appraisal interviews/sessions and perceptions about fairness of the appraisal system. Results from the focus group interviews suggested that, overall, employees were satised with the new system, distribution model, and its fairness as an evaluation tool. For example, several employees indicated that the previous system lacked one or more of the three elements associated with the new merit pay distribution model (i.e., pay differentials, forced distribution, or pay adjustment matrices). In contrast, focus group results suggested that employees had a more positive and slightly cautious attitude toward the new system. The new system and model allowed for merit pay increases that were recognizable by staff and clearly tied to the performance ratings. Employees believed that the new system greatly improved on the subjective nature of the previous instrument but failed to completely remove all subjectivity in the process. However, the use of forced distribution as a method of distributing employees across performance levels was well received as an approach to further removing subjectivity from merit pay distribution decisions. In addition to the positive role of forced distribution, the reduction from ve performance levels to three was well received by the employees. Employee performance appraisal results indicated a tight cluster of scores near the mean, and research has advocated for the use of fewer performance-based pay distinctions in these situations (Martocchio, 2001; Zenger, 1992). In particular, research has found an increase in perceived inequity comparisons by employees when more than three performance distinctions are made in an organization with minimal score variance (Zenger, 1992). These inequity perceptions can be troublesome for the agency: studies have identied several costly outcomes to the employer, including low effort, turnover, job dissatisfaction, poor interpersonal relations with colleagues, and sabotage (Arvey & Murphy, 1998; Levy & Williams, 2004). In summary, the overall results of the focus group interviews suggested that employees were pleased with the new system but were concerned about some of the systems implications. Specically, concerns were expressed about the pay adjustment
151
matrix and pay differentials. However, despite these concerns, interviewees believed that the system was a positive addition to the Elmhurst Park District and its operations. Results suggested that employees of the park district were optimistic about the motivational aspects of the new system, particularly when recognizing the obvious need to maintain pay equity within the agency while adhering to the agencys merit increase pool restrictions.
2)
3)
These ndings are further supported by a comment from one of the supervisors: It was somewhat difcult to tell Employee A that he is a 1 [below standard rating], but it was worth it. I was surprisedEmployee A showed little disagreement with the rating [during the performance review session], and I saw immediate and continued improvement in Employee As attitude and performance. The supervisors feedback identied specic job-decient areas where the employee could improve. The employees participation in the appraisal process and the clearly stated areas for improvement led to more positive perceptions about fairness and to acceptance of the supervisors ratings. These high levels of employee acceptance and understanding will likely enhance the employees motivation and job performance. This case study of the Elmhurst Park District sought to provide an empirically grounded overview of the initial steps involved in developing a pay-for-performance system for public parks and recreation agencies. Future research with different and larger samples is needed to further understand pay-for-performance appraisal practices
152
in the leisure service eld. Although park districts represent the norm in the state of Illinois, public park and recreation departments housed within municipal or county government represent a predominant type of leisure service organization in the United States. As a case study, this research was limited to performance appraisal practices in a park district within the state of Illinois. Additional research examining the development of a pay-for-performance system within a municipal or county park and recreation department is needed. In addition, research that examines the social context of performance appraisal development in public park and recreation agencies could provide additional insight into the role of employee participation. As research in the management eld has suggested (see Levy & Williams, 2004), research examining the effects of the social context of the agency, such as feedback culture, group dynamics, politics, impression management, and other environmental variables, is needed and can help provide a richer understanding of this important management issue. The trial-run approach of the performance appraisal process was limitation of the study. Although the performance appraisal process was live, it was without consequences (i.e., results were not used to make merit pay increase decisions). Employees were aware of the lack of consequences in the trial run. Future studies will attempt to address the success and continual acceptance of the performance appraisal system adopted by the Elmhurst Park District. This study adopted a two-pronged approach to further understanding performance appraisal systems in public park and recreation agencies, using the Elmhurst Park District as a case study. First, the study sought to identify the steps needed to develop a pay-for-performance appraisal system for a public park and recreation agency. Next, the study examined employee reactions to a system that adopted these steps. The study identied positive employee reactions to a performance appraisal system that incorporated the steps, and strong evidence was found for increased satisfaction and improved perceptions of procedural justice in the new system.
References
Arvey, R.D., & Murphy, K.R. (1998). Performance evaluation in work settings. Annual Review of Psychology, 49, 141-168. Bartlett, K. R., & McKinney, W. R. (2004). A study of the role of professional development, job attitudes, and turnover among public park and recreation employees. Journal of Park and Recreation Administration, 24(4), 63-81. Bassett, G. (1994). Merit pay increases are a mistake. Compensation and Benets Review, 26(2), 20-22. Bergmann, T. J., & Scarpello, V. G. (2001). Compensation decision making (4th edition). Orlando, FL: Harcourt College Publishers. Bernadin, H. J., & Pence, E. C. (1980). Effects of rater training: Creating new response sets and decreasing accuracy. Journal of Applied Psychology, 6, 60-66. Bommer, W. H., Johnson, J. L., Rich, G. A., Podsakoff, P. M., & MacKenzie, S. B. (1995). On the interchangeability of objective and subjective measures of employee performance: A meta-analysis. Personnel Psychology, 48, 587-605.
153
Borman, W. C., & Motowidlo, S. J. (1993). Expanding the criterion domain to include elements of contextual performance. In: Schmidt, N., & Borman, W. C. (Eds.), Personnel selection in organizations, 71-98. Boyatzis, R.E. (1998). Thematic analysis and code development: Transforming qualitative information. Thousand Oaks, CA: Sage Publications. Brademas, D. J., Lowrey, G. A., Gress, K., & Bostrom, D. (1981). A model job analysis procedure for the park and recreation profession. Urbana-Champaign, IL: Ofce of Recreation and Park Resources. Campbell, D.J., Campbell, K.M., & Chia, H.B. (1998). Merit pay, performance appraisal, and individual motivation: An analysis and alternative. Human Resource Management, 37, 2, 131-146. Cardy, R. L., & Dobbins, G. H. (1994). Performance appraisal: alternative perspectives. Cincinnati, OH: South-Western Publishing. Cascio, W. F. (2006). Managing human resources: Productivity, quality of work life, prots. Boston, MA: McGraw-Hill Irwin. Chelladurai, P. (1999). Human resource management in sport and recreation. Champaign, IL. Human Kinetics. Chen, H.M. & Min, K.J (2004). The role of human capital cost in accounting. Journal of Intellectual Capital, 5, 1, 116-131. Dolan, K. A. (1996). When money isnt enough. Forbes, 11/18, 164-170. Drauden, G., & Peterson, H. (1974). A domain sampling approach to job analysis. St. Paul, MN: Test Validation Center. Dyer, L., & Reeves, T. (1995). Human resource strategies and rm performance: What do we know and where do we need to go? International Journal of Human Resource Management, 6(3), 656-670 Edginton, C. R., Hudson, S. D., & Lankford, S. V. (2001). Managing recreation, parks, and leisure services: An introduction. Champaign, IL: Sagamore Publishing. Edginton, C. R., & Williams, J. G. (1978). Productive management of leisure service organizations: A behavioral approach. New York, NY: John Wiley & Sons. Giles, W. F., & Mossholder, K. W. (1990). Employee reactions to contextual and session components of performance appraisal. Journal of Applied Psychology, 75, 371-377. Gosselin, A., & Murphy, K.R. (1994). LEchec de levaluation de la performance. Gestion, 17-26. Grant, P. C. (1988). What use is a job description? Personnel Journal, 67(2), 44-54. Greller, M. M. (1978). The nature of subordinate participation in the appraisal interview. Academy of Management Journal, 21, 646-658. Grote, D. (2000). Public sector organizations. Public Personnel Management, 29, 1, 1-20. Guralnik, O., Rozmarin, E., & So, A. (2004). Forced distribution: Is it right for you? Human Resource Development Quarterly, 15(3), 339-345. Harris, M. M., & Schaubroeck, J. (1988). A meta analysis of self-supervisor, self-peer and peer-supervisor ratings. Personnel Psychology, 4, 43-62. Heneman, R. L. (1986). The relationship between supervisory and results-oriented measures of performance: A meta-analysis. Personnel Psychology, 39, 811-826.
154
Heneman, H. G., & Ellis, R. A. (1982). Correlates of just noticeable differences in pay increases. Labor Law Journal, 33, 533-538. Holstrom, B. (1979). Moral hazard and observability. Bell Journal of Economics, 10(1), 74-91. Jackson, S. E., Schuller, R. S., & Rivero, J. C. (1989). Organizational characteristics as predictors of personnel practices. Personnel Psychology, 42, 727-786. Katkowski, D. A., Medsker, G. J., & Pritchard, K. H. (2002). Literature review of acceptable or just noticeably different pay increases. Human Resources. Keeley, M. (1978). A contingency framework for performance evaluation. Academy of Management Review, 3, 428-438. Keeping, L.M., & Levy, P.E. (2000). Performance appraisal reactions: Measurements, modeling, and method bias. Journal of Applied Psychology, 85, 5, 708-724. Krefting, L. A., & Mahoney, T. A. (1977). Determining the size of a meaningful pay increase. Industrial Relations, 16(1), 83-93. Lance, C. E., Teachout, M. S., & Donnelly, T. M. (1992). Specication of the criterion construct Space: An application of hierarchial conrmatory factor analysis. Journal of Applied Psychology, 77, 437-452. Landau, S. B., & Leventhal, G. S. (1976). A simulation study of administrators behavior toward employees who receive job offers. Journal of Applied Social Psychology, 6(4), 291-306. Landy, F. S., & Farr, J. L. (1980). Performance rating. Psychological Bulletin, 87, 72-107. Levy, P.E., & Williams, J.R. (1998). The role of perceived system knowledge in predicting appraisal reactions, job satisfaction, and organizational commitment. Journal of Organizational Behavior, 19, 53-65. Levy, P. E., & Williams, J. R. (2004). The social context of performance appraisal: A review and framework for the future. Journal of Management, 20(6), 881-905. Lincoln, Y.S., & Guba, E.G. (1985). Naturalistic inquiry. Beverly Hills, CA: Sage Publications. Longenecker, C.O., & Fink, L.S. (1999). Creating effective performance appraisals. Industrial Management, 41, 5, 18-24. Longenecker, C.O., & Fink, L.S. (2003). Benchmarks for effective performance rating instruments. Journal of Compensation and Benets, 19, 2, 24-31. Martocchio, J.J. (2001). Strategic compensation: A human resource management approach. Upper Saddle River, NJ: Pearson Prentice Hall. Mathis, R.L., & Jackson, J.H. (2006). Human resource management (11th ed.). Thomas Learning, Mason, OH. McKinney, W. R., & Yen, T. H. (1989). Personnel management in large U. S. park and recreation organizations. Journal of Park and Recreation Administration, 7(2), 1-25. Milkovich, G. T., & Newman, J. (2005). Compensation (8th Ed.). Boston, MA: Irwin McGraw-Hill Companies, Inc. Mitra, A., Gupta, N., & Jenkins, G. D. (1997). A drop in the bucket: When is a pay raise a payraise? Journal of Organizational Behavior, 18(2), 117-137.
155
Moss, S.E., & Martinko, M.J. (1998). The effects of performance attributions and outcome dependence on leader feedback behavior following poor subordinate performance. Journal of Organizational Behavior, 19, 3, 259-274. Murphy, K. R., & Cleveland, J. N. (1995). Understanding performance appraisal: social, organizational, and goal-based perspectives. Thousand Oaks, CA: Sage Publications. Nadler, D.A. (1979). The effects of feedback on task group behavior: A review of the experimental research. Organizational Behavior and Human Performance, 25, 123-128. Ones, D. S., Schmidt, F. L., & Viswesvaran, C. (1996). Comparative analysis of the reliability of job performance ratings. Journal of Applied Psychology, 81(5), 557574. Petit, A., & Haines, V. (1994). Trois instruments devaluation du rendement. Gestion, 59-68. Podsakoff, P.M., & Farh, J.L. (1989). Effects of feedback sign and credibility on goal setting and task performance. Organizational Behavior and Human Performance, 23, 309-338. Roberts, G. E. (2003). Employee performance appraisal system participation: A technique that works. Public Personnel Management, 32(1), 89-99. Rotundo, M., & Sackett, P. R. (2002). The relative importance of task, citizenship, and counterproductive performance to global ratings of job performance: A policycapturing approach. Journal of Applied Psychology, 87, 66-80. Schmidt, F. L., & Kaplan, L. B. (1971). Managerial talent, pay, and age. Journal of Vocational Behavior, 1, 129-135. Schwandt, T.A (2001). Dictionary of qualitative inquiry (2nd Ed.). Thousand Oaks, CA: Sage Publications. Schweiger, I., & Sumners, G.E. (1994). Optimizing the value of performance appraisals. Managerial Auditing Journal, 9, 8, 3-7 Scullen, S. E., Goff, M., & Mount, M. K. (2000). Understanding the latent structure of job performance ratings. Journal of Applied Psychology, 24, 419-434. Smith, B.N., Hornsby, J.S., & Shirmeyer, R. (1996). Current trends in performance appraisal: An examination of managerial practice. SAM Advanced Management Journal, 1, 20. Smither, J. W. (1998). Lessons learned: Research implications for performance appraisal and management practice. IN: J. W. Smither (Ed.), Performance appraisal: State of the art in practice. San Francisco, CA: Jossey-Bass. Strauss, A., & Corbin, J. (1998). Basics of qualitative research: Techniques and procedures for developing grounded theory (2nd Ed.). Thousand Oaks, CA: Sage Publications. Thomas, S.L., & Bretz, R.D. (1994). Research and practice in performance appraisal: Evaluating employee performance in Americas largest companies. SAM Advanced Management Journal, 59, 2, 28-34. Tompkins, J. (2002). Strategic human resources management in government: Unresolved issues. Public Personnel Management, 31, 1, 95-111.
156
Tziner, A., & Kopelman, R.E. (2002). Is there a preferred performance rating format? A non-psychometric perspective. Applied Psychology: An International Review, 51, 3, 479-503. Van Scotter, J. R., Motowidlo, S. J., & Cross, T. C. (2000). Effects of task performance and contextual performance on systematic rewards. Journal of Applied Psychology, 85, 526-535. Vance, R. J., MacCallum, R. C., Coovert, M. D., & Hedge, J. W. (1988). Construct validity of multiple job performance measures using conrmatory factor analysis. Journal of Applied Psychology, 73, 74-80. Viswesvaran, C., & Ones, D. S. (2000). Perspectives on models of job performance. International Journal of Selection and Assessment, 8(4), 216-226. Wiersma, V.C., & Latham, G.P. (1986). The practicality of behavioral observation scales, expectation scales and trait scales. Personnel Psychology, 39, 619-628. Wilson, J.P., & Western. S. (2001). Performance appraisal: An obstacle to training and development? Career Development International, 6, 2/3, 93-102. Wojcik, J. (2000). Focus on performance. Business Insurance, July, 20. Wright, P. M., & McMahan, G. C. (1992). Theoretical perspectives for strategic human resource management. Journal of Management, 18(2), 295-320 Zedeck, S., & Cascio, W. (1982). Performance appraisal decision as a function of rater training and purpose of the appraisal. Journal of Applied Psychology, 67(6), 752758. Zenger, T. R. (1992). Why do employees only reward extreme performance? Examining the relationships among performance, pay and turnover. Administrative Science Quarterly, 37(2), 198-219.