INTERNATIONAL JOURNAL OF SELECTION AND ASSESSMENT

VOLUME 14 NUMBER 1 MARCH 2006

360 Degree Feedback and Developmental Outcomes: The Role of Feedback Characteristics, Self-Efficacy and Importance of Feedback Dimensions to Focal Managers Current Role
Caroline Bailey* and Michelle Austin
Queensland University of Technology

This longitudinal study investigates whether developmental changes following 360 degree feedback are predicted by the favourability of ratings received, and moderated by focal individuals self-efficacy and perceived importance of feedback. Five developmental criteria are investigated longitudinally: (i) self-assessments, (ii) line managers ratings, (iii) amount of developmental activity, (iv) global self-efficacy and (iv) self-efficacy for development. Feedback ratings from certain rater groups predicted changes in ratings, but not changes in self-efficacy or amount of developmental activity. Self-efficacy significantly moderated the feedbackperformance association for certain rater groups, but feedback importance did not. Contrary to expectations, the focal individuals initial self-assessment predicted changes in self-efficacy, over the favourability of ratings received. The implications of these findings for organizations using 360 degree feedback for developmental purposes are discussed.

urveys in Britain and the United States have established that organizations use of multiple-source multiplerater (MSMR) feedback systems such as 360 degree feedback for developmental purposes has increased steadily since their introduction (Fletcher, 2001). In utilizing such systems for employee development, organizations are tacitly endorsing an assumption of many learning theories that providing feedback on performance will result in improvements in individuals subsequent performance. Also referred to as knowledge of results, feedback has been described as a key element in the processes of skill acquisition, goal attainment and behaviour change (Anderson, 1982, 1983; Locke, 1996). Of primary concern to this study was that empirical evidence suggests the link between providing feedback and improvements in employees subsequent performance ratings are not uniform (Kluger & DeNisi, 1996). Secondly, explanations for this variability in outcomes are currently quite limited; studies investigating outcomes generally accord a central role to the characteristics of feedback received, and have not considered the contribution of

Address correspondence to Caroline Bailey, School of Management, Queensland University of Technology, Brisbane, QLD 4001, Australia. E-mail: c2.bailey@qut.edu.au

situational factors and/or individual difference variables (Maurer, Mitchell, & Barbeite, 2002). The current research seeks to address these issues, positing that participation in feedback can lead to positive or negative outcomes for focal individuals, and that this variability can be attributed in part to three factors: the characteristics of feedback received (favourability, according to rating scale descriptors), the focal individuals selfefficacy (both global self-efficacy and specific self-efficacy for developmental activity) and perceived importance of the feedback behaviours to the individuals current role. The impacts of feedback were represented in the current study by five criterion variables, obtained 3 months after initial participation. These were repeated measures of (a) self-assessments, (b) line managers ratings, (c) global selfefficacy and (d) self-efficacy for development. The fifth criterion was the amount of developmental activity focal individuals reportedly engaged in following feedback. Changes in self-assessments and line managers assessments were of interest, because of the links suggested by previous research between these ratings and individuals subsequent performance and progression within an organization (Ashford, 1989; Fletcher, 2001). The criterion measures of developmental activity, global self-efficacy and self-efficacy for development were included to extend the

r 2006 The Authors Journal compilation r 2006 Blackwell Publishing Ltd, 9600 Garsington Road, Oxford, OX4 2DQ, UK and 350 Main St, Malden, MA 02148, USA

51

52

CAROLINE BAILEY AND MICHELLE AUSTIN

scope of 360-feedback impact research beyond the criterion measures (i.e. subsequent feedback ratings) that currently dominate the literature. With the exception of a few studies (Atkins & Wood, 2002; Atwater, Ostroff, Yammarino, & Fleenor, 1998; Beehr, Ivanitskaya, Hansen, Erofeev, & Gudanowski, 2001; Hazucha, Hezlett, & Schneider, 1993; Maurer et al., 2002), investigations of feedback impact to date have largely examined impact by statistical comparison of successive feedback ratings over time (Smither, London, Flautt, Vargas, & Kucine, 2002). In addition, this study aimed for further understanding of the processes between feedback and outcomes, by investigating how individuals self-efficacy and perceptions of the importance of feedback before receiving feedback as well as the amount of developmental activity engaged in following feedback affect the feedbackperformance association. To the authors knowledge only four longitudinal investigations of the feedbackperformance association have been published. Hazucha et al. (1993) examined focal individuals subsequent engagement in developmental activities and changes in 360-feedback ratings over time. Brutus, London, and Martineau (1999) investigated the relationship between 360-feedback ratings and subsequent selection of developmental goals. Maurer et al. (2002) investigated focal individuals attitudes towards the feedback system and subsequent involvement in on-the-job and off-the-job develop activities 10 months after initial participation. Most recently, Smither, London, Flautt,

Vargas, and Kucine (2003) investigated the association between 360-feedback and participation in executive coaching and impact upon subsequent 360-feedback ratings. All found variability in feedback outcomes, which could be attributed to certain individual differences and/or situational variables. This study contributes to the literature, by investigating over time the role of self-efficacy and perceptions of the importance of feedback. Finally, this study investigates developmental outcomes within the context of 360-feedback, rather than just a single rater source program (i.e. upward feedback) which have been the focus of most previous research (Smither et al., 2002). There is empirical evidence that individuals reactions to feedback are sensitive to rater dynamics (Bailey & Fletcher, 2002; Bastos & Fletcher, 1995). In particular, Bailey and Fletcher (2002) found that focal individuals ratings of intent to change behaviour in response to feedback received was determined in part by the interaction of ratings from different sources, and that focal individuals regarded some feedback sources as significantly more accurate and important than others. Aside from this and a few longitudinal studies of 360-feedback, little else is known about how rater source dynamics relate to developmental outcomes. The current study investigated how ratings from five different sources related to developmental criteria. The extent of the current design is shown in Figure 1; the theoretical rationale and empirical evidence underpinning this design is as follows.

Self-Assessment (T1)

Perceived importance of feedback

Initial Self-Efficacy (Global S.Eff and S.Eff Devel.)

Key
H

Main effect Moderator Mediator Amt. Developmental activity

H

H H

Favourability of fbk ratings (line mgr, peers, sub., other) T1

H H H H ,H H ,H

Change in SelfAssessment

Change in SelfEfficacy

H H

Amt. Developmental activity

H

Change in line mgr rating

Favourability of fbk ratings (peers, sub., other) T1

H H H

Perceived importance of feedback

Initial Self-Efficacy (Global S.Eff and S.Eff Devel.)

Figure 1. Summary of hypothesized relationships between predictors, moderators, mediators and dependent variables.

International Journal of Selection and Assessment

r 2006 The Authors Journal compilation r Blackwell Publishing Ltd. 2006

360 DEGREE FEEDBACK AND DEVELOPMENTAL OUTCOMES

53

The FeedbackPerformance Association

In their meta-analysis of feedback intervention research (predominantly conducted in laboratory settings) Kluger and DeNisi (1996) concluded that while the majority of feedback interventions do lead to improved performance, in over a third of feedback interventions, recipients average performance declined following feedback. A similar picture emerges from studies of developmental feedback within an organizational context. In a review of 13 longitudinal investigations of mainly upward feedback, Smither et al. (2002) concluded that ratings improved over time for some, but not all focal individuals. It was anticipated from these previous investigations of impact that feedback outcomes would vary across participants; this study investigated whether variability may be accounted for by the following factors.

Predictors of Feedback Impact

Favourability of Feedback Received. Developmental feedback intervention literature to date regards the characteristics of feedback received, rather than any individual differences or situational variables, as the primary determinant of feedback impact. In their model of 360-feedback impact, London and Smither (1995) propose that individuals compare the favourability of their self-assessments to others ratings, and evaluate the extent to which others ratings are in agreement with each other, revising their self-image accordingly. The interaction between the self-assessment and others ratings to feedback outcomes is echoed in many other theoretical perspectives discussed in feedback literature such as prospect theory (Kahneman & Tversky, 1979), control theory (Carver & Scheier, 1981) and feedback intervention theory (Kluger & DeNisi, 1996). All these approaches suggest that when individuals receive negative feedback (ratings which indicate a shortfall in performance, relative to some standard), they will be motivated to reduce the discrepancy, and make efforts to improve their performance. Empirical evidence from applied settings supports the view that receiving negative feedback does motivate the individual to reduce the discrepancy (Brutus et al., 1999; Maurer et al., 2002). On this basis, it was anticipated there would be a negative association between the favourability of feedback from external raters, and amount of developmental activity engaged in following feedback (Hypothesis 1). Individuals receiving feedback from their line manager, peers, subordinates and other raters which indicates a shortfall in their performance were expected to engage in more developmental activity than those individuals who received favourable ratings, as the latter arguably have less incentive to change (Hazucha et al., 1993; Maurer et al., 2002).

Similarly, and in line with the tenets of London and Smithers model (1995), a negative association was expected between both the favourability of initial feedback from external raters, and increments in the favourability of the focal individuals subsequent self-assessment (Hypothesis 2a) and subsequent line managers rating (Hypothesis 2b). As before, those who initially received negative ratings from others were expected to show greater performance improvements over time (relative to their initial level of performance), than those who received more favourable feedback, who arguably have less need to improve performance. Given that engaging in developmental activities should enhance ones performance over time, exploratory analyses also investigated whether amount of developmental activity had a mediational effect upon the association between feedback received and changes in selfassessments (Hypothesis 2c) and line managers ratings (Hypothesis 2d). Exploratory analyses were also conducted to investigate the association between the favourability of the initial self-assessment and changes in line managers evaluations (Hypothesis 2e), as well as amount of developmental activity engaged in following initial participation in feedback (Hypothesis 2f). The impact of feedback favourability on focal individuals self-efficacy was also investigated. Broadly defined as ones judgement of perceived capability for performance (Bandura, 1986) self-efficacy can be considered as both a global construct (i.e. an individuals perceptions of their general capability to perform any activity, Schwarzer, 1992) as well as task-specific (i.e. perceptions of mastery for specific activities; Maurer et al., 2002). The current study assessed both focal managers global self-efficacy and their task-specific efficacy for applying what they have learned from feedback to their role, subsequently referred to as self-efficacy for development (Maurer et al., 2002). Investigations of self-efficacy in other contexts suggest receiving favourable feedback may enhance feelings of personal mastery, whereas receiving unfavourable feedback may cause self-efficacy to be undermined (Bandura, 1997). Thus, the favourability of feedback from line managers, peers, subordinates and other raters was hypothesized to be positively associated to changes in focal individuals global self-efficacy (Hypothesis 3a) and selfefficacy for development (Hypothesis 3b). Regression analyses were also conducted, to investigate exploratory (non-directional) hypotheses that the favourability of focal individuals initial self-assessments predict changes in global self-efficacy (Hypothesis 3c) and self-efficacy for development (Hypothesis 3d). Focal Individuals Self-Efficacy. The second objective of this study was to examine the extent to which focal individuals self-efficacy before participation in feedback accounted for variability in developmental activity and performance outcomes (i.e. self-efficacy as a predictor as opposed to outcome variable, as described in the previous section).

r 2006 The Authors Journal compilation r Blackwell Publishing Ltd. 2006

Volume 14 Number 1 March 2006

54

CAROLINE BAILEY AND MICHELLE AUSTIN

Self-efficacy has been identified as a fundamental predictor in theories concerned with performance outcomes generally (e.g. Social Learning Theory, Bandura, 1977; Goal Setting Theory, Locke, 1996) as well as those within specific organizational contexts (e.g. coping with difficult career related tasks, Stumpf, Brief, & Hartman, 1987; simulated air traffic control task training, Eyring, Johnson, & Francis, 1993). It is consistently observed across different contexts that individuals with high selfefficacy have stronger intent to achieve goals, show more persistence and generally have more positive thoughts and attitudes towards the task than individuals with low selfefficacy. As a result, they tend to achieve higher performance outcomes (Bandura, 1997). Despite the theoretical and empirical links between selfefficacy and performance, there is comparatively little discussion in the 360-feedback literature as to the role self-efficacy plays in determining performance outcomes following feedback. At present, detailed conceptual discussion is limited to London and Smither (1995), who outline the role of self-efficacy in moderating individuals goal setting behaviours following receipt of feedback, and Yammarino and Atwater (1997) who discuss the role of self-efficacy in performance outcomes for those individuals who hold a negative view of themselves, underestimating their performance relative to others perceptions. Even less empirical research is available. In a review of the literature, only one study was found that has looked at the association between self-efficacy and 360-feedback impact, that by Maurer et al. (2002). They investigated developmental outcomes of 360-feedback, and the extent to which outcomes were predicted by three factors (a) the favourability of feedback ratings, (b) individual difference variables (performance goal vs. learning goal emphasis, implicit theory of ability, self-efficacy and age) and (c) work context variables (perceived organizational development, perceived support for development and perceived resource availability). Maurer et al. (2002) concluded that certain individual differences (belief in skill malleability and selfefficacy for development) and work context variables (social support for development) are just as important, if not more important than the feedback received, as determinants of developmental activity. Focusing specifically on their findings for self-efficacy, they concluded that self-efficacy has a moderate, positive association to individuals attitudes towards the feedback system (b 5 .26, po.05) and involvement in off-the-job developmental activities (b 5 .21, po.05). On the basis of the theoretical and empirical rationale for self-efficacy offered by perspectives such as social learning theory (Bandura, 1977), goal setting theory (Locke, 1996) and feedback impact (London & Smither, 1995; Maurer et al., 2002; Yammarino & Atwater, 1997) it was anticipated that both global self-efficacy and selfefficacy for development have a positive association to the amount of developmental activity (Hypothesis 4). Further-

more both forms of self-efficacy were expected to moderate the relationship between characteristics of feedback received and amount of developmental activity (Hypothesis 5). Self-efficacy beliefs were also expected to moderate the association between feedback received, and subsequent self-assessments (Hypothesis 6a) and ratings by the line manager over time (Hypothesis 6b).

Importance of Feedback Dimensions to Current Role. In their model of feedback impact, London and Smither (1995) suggest that the extent to which the favourability of feedback has an impact upon the individual, will be determined in part by its salience. They discuss a number of factors influencing the salience of feedback, one of which is the extent to which the individual perceives the feedback behaviours to be relevant to their role; they propose a positive association between feedback salience and impact of feedback on self-image, development and subsequent performance. In the current study, feedback dimensions that are rated of low importance indicate that those particular behaviours, skills or attributes are not central to individuals overall effectiveness in a particular role. Following the reasoning of London and Smither (1995) it was anticipated that focal individuals perceptions of feedback importance moderates the association between the favourability of feedback received and amount of developmental activity (Hypothesis 7) and changes in self-assessments (Hypothesis 8a) and line managers ratings (Hypothesis 8b) over time. Finally, Ashford (1989) notes how many self-image theories posit that individuals self-efficacy will only be affected by feedback that is directly relevant to them. Thus, it was expected that the focal individuals perceptions of feedback importance also moderates the association between the favourability of feedback received, and subsequent changes in global self-efficacy (Hypothesis 9a) and self-efficacy for development (Hypothesis 9b).

Applications of Research
Having a better understanding of how feedback characteristics impact upon participants, and exploring the role of self-efficacy and perceived importance of feedback in the feedbackperformance association, has value from both theoretical and practical perspectives. To date only one theoretical model of 360-feedback impact has been published (London & Smither, 1995). While much research has focused upon the central processes they describe (i.e. the association between initial feedback and subsequent ratings), there is much less research investigating the individual differences and situational variables also included in the model. By investigating the role of selfefficacy and perceptions of feedback importance, the current study attempts to address this gap in the literature. From a practical standpoint, investigating the role of self-efficacy and perceived importance of feedback may increase our understanding of how two people with very

International Journal of Selection and Assessment

r 2006 The Authors Journal compilation r Blackwell Publishing Ltd. 2006

360 DEGREE FEEDBACK AND DEVELOPMENTAL OUTCOMES

55

similar feedback profiles can have completely different reactions to their feedback, and subsequent performance outcomes. There are also implications for the development of feedback tools, system support and overall utility of developmental activities based on their output. Briefing processes may be impacted, as may feedback facilitation interviews and personal development planning activities with the emphasis given to certain feedback characteristics, as well as individuals self-efficacy before, and following participation in the feedback system.

Method
Design and Operationalization of Feedback Outcomes
This study was a field-based, correlational study of U.K. financial sector managers reactions to a developmental 360-feedback system. Predictor variables obtained for each focal manager were (i) mean item rating (i.e. feedback favourability) for the self-assessment and each of the four external rater groups (the focal individuals line manager, peers, reports and other raters who represented their internal clients); (ii) initial global self-efficacy; (iii) initial self-efficacy for development; and (iv) the mean rating of perceived importance of this feedback dimension to the individuals role, from the focal individual. Each of the four external rater groups also provided ratings of feedback importance, but this data was not included in the current research design. Criterion measures were obtained 3 months after initial participation in feedback. Repeated measures were obtained for the mean item rating on the 360-feedback questionnaire by the (i) focal individual and (ii) line manager; a repeated measure of (iii) global selfefficacy and (iv) self-efficacy for development. A fifth criterion variable, the amount of developmental activity engaged in following feedback was obtained by self-report from the focal individual. Before regression analyses all of these measures were transformed to be mean centred, to minimize multi-collinearity between regression terms (Cronbach, 1990). The Condition Index (a collinearity diagnostic statistic) in all regression analyses was less than 15, indicating that centred variables did not suffer from multi-collinearity. It is of value at this juncture to discuss the operationalization of impact in the current research design (i.e. comparison of pre- and post-measures, in a one group design), given the controversy surrounding its appropriate statistical representation. Historically, researchers relied on various forms of difference score (i.e. T2 minus T1 values) to operationalize impact. However, the inadequacies of such an approach are now well documented (see Edwards, 1995, 2001 for a detailed critique), and researchers have sought alternative analytical approaches that avoid the substantial problems inherent in difference scores, such

as repeated measure regression or ANOVA-based approaches. In the current investigation, the relationship between the favourability of external raters feedback and developmental activity (Hypothesis 1) was relatively straightforward, being investigated by multiple linear regression (enter method). Feedback terms for each external rating constituency (line manager, peers, reports, others) were entered in a single step as predictors of amount of developmental activity. The analysis of change in self-assessments (Hypothesis 2a) and line managers ratings (Hypothesis 2b) and change in global self-efficacy (Hypothesis 3a) and selfefficacy for development (Hypothesis 3b) was conducted using repeated measures regression. For these analyses, individuals initial self-assessment, line manager rating or initial self-efficacy measure (as appropriate) were entered in the first block in a hierarchical regression equation. This serves to act as a control for the initial (T1) level of this attribute in the criterion (i.e. T2 measure). Predictors (feedback favourability terms) were then entered in the second block as predictors of the T2 criterion measure. The increment in R2 change and standardized b coefficients provided indication of the variance accounted for by predictor variables. Note: For all regression analyses, missing values were replaced with the grand mean, to maintain consistent df values across equations.

Participants
Participants were obtained from a population of managers working in a U.K.-based financial institution. Participation in 360-feedback was a compulsory process occurring once every 2 years for all managerial staff within the organization; with average tenure over 10 years, the majority of participants were expected to have experienced 360feedback before participation in this study. However, the format of the 360-feedback questionnaire used in this study had not been used previously in this organization. Four hundred and twenty-seven focal managers (the entire population of managers within the organization scheduled to participate in the feedback program that year) were invited to participate in the study; of these, 104 managers agreed to participate. These individuals agreed to take part in the study before their completing self-assessments or receiving feedback from colleagues, thereby ensuring the decision to participate was not influenced by the characteristics of the feedback received. Each focal manager nominated between six and 12 feedback providers who were categorized according to their relationship to the feedback recipient as their line manager, peer, subordinate or other (internal clients or previous colleagues). Ratings on the 360-feedback questionnaire were provided anonymously by multi-rater groups (as a single rater source, line managers ratings were the only non-anonymous ratings from an external source). As is sometimes the case for 360-feedback research, exact

r 2006 The Authors Journal compilation r Blackwell Publishing Ltd. 2006

Volume 14 Number 1 March 2006

56

CAROLINE BAILEY AND MICHELLE AUSTIN

numbers of external raters could not be determined, because of the organizations utilization of aggregate ratings (rater group averages) for ratings from multiple sources (peers, subordinates and others). However, as the organizations 360-feedback system software would not generate a report until a focal manager received ratings from a minimum of six other raters, the minimum sample size was 728 raters (104 self-assessments, 624 assessments from external raters). Three months after initial participation in feedback, focal individuals and their line managers were invited to provide a second assessment (using the same 360-feedback measure), as criterion measures of performance. Ratings were received by 78 focal managers and 67 line managers; in total, complete datasets were obtained at both time points for 67 focal managers (64.42% of the initial sample). Note: The vast majority (75%) of focal managers participated at both time points, however it is possible that they may have differed systematically to a minority (25%) who did not repeat participation. As only complete cases (i.e. those with T1 and T2 feedback ratings) were made available to the authors for analysis, it was not possible to test for statistical differences. All participants were employed by one organization, and the majority (over 70%) were based in the U.K. and of white European descent. 30.9% were female and 67.6% male (1.5% missing data), with ages ranging from 24 to 59 years (mean 38.46 years) and an average organizational tenure of 10.8 years (range 6 months31 years).

r 5 .51 (mean r self-assessment 5 .50, SD 5 .16; mean r line managers ratings 5 .46, SD 5 .12; mean r peers ratings 5 .52, SD 5 .14, mean r subordinates ratings 5 .54, SD 5 .16, mean r others ratings 5 .54, SD 5 .14). Given the high intercorrelation between competencies, it was decided that analyses should focus on the mean profile rating, rather than at the dimension level, for each rater group. Rater group descriptives and the correlation of mean profile ratings between rater sources (e.g. self to peers) and over time (e.g. line manager rating T1T2) are provided in Table 1, operationalized as the favourability of feedback rating. Unfortunately, the 360-feedback system database archived records of the rater group average for multiple rater sources (i.e. peers, subordinates and others) rather than individual ratings within each rater group. This prevented analysis of ratings at the individual level, including establishing the interrater agreement of aggregate ratings from a particular source. As well as ascribing ratings to behavioural indicators, raters were invited to provide free comments which were reported verbatim in the written feedback report. These data were not analyzed.

Measures
Developmental 360-Feedback Questionnaire
Feedback Ratings. The feedback questionnaire had been designed around the organizations competency framework, and was being used for the first time in electronic format, with questionnaires distributed via the organizations intranet. Feedback ratings were ascribed for 55 behaviours, corresponding to 12 competencies designed to assess focal individuals managerial skills (e.g. leadership, interpersonal skills, technical and professional competence). Each competency was comprised of four to six behavioural indicators, which were rated by a five-point scale (ranging from 0, the participant never behaves like this to 4 the participant behaves like this all the time is a role model for this behaviour). A response category of N indicated that the rater could not comment upon this behaviour and such responses were recorded as missing data. Alpha coefficients for competencies were high (ranging from .76 to .91), indicating good internal consistency. To determine whether analyses should be focused at the mean rating for the whole profile, or mean rating for each competency, Pearsons correlations investigated the intercorrelation of competencies for each rater group. The average intercorrelation (across all rater groups) was

Importance of Feedback Behaviour to the Focal Individuals Role. As part of the 360-feedback questionnaire, Importance Ratings (a rating of the extent to which this competency is relevant to the feedback recipients role) were provided for each of the 12 competencies by each rater. A five-point rating scale was used from unimportant (coded 0) to extremely important (coded 4); a response category of dont know was also provided and coded as missing data. Participants made a selfassessment of feedback importance and were provided with the average rating of importance from each rater group, for each competency, as part of the feedback report. The current study investigated the role of focal individuals perceptions of feedback importance only; data on feedback importance from external rater sources was not explored. The mean intercorrelation between competency ratings for focal individuals ratings of feedback importance was r 5 .40. Given this degree of intercorrelation, and the fact that feedback favourability was being investigated at the profile level, the predictor variable importance of feedback was also investigated at the profile rather than competency level. This was computed as the mean of importance ratings made for each competency, by the focal individual. Note: descriptive data is also reported for feedback importance from external raters (Table 1), but hypotheses and analyses focus on the role of feedback importance from the focal individuals perspective only.

Global Self-Efficacy
Defined as an individuals belief in their capability to perform any task, global self-efficacy was measured by

International Journal of Selection and Assessment

r 2006 The Authors Journal compilation r Blackwell Publishing Ltd. 2006

Table 1. Correlation matrix of time one and time two measures SD 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16

r 2006 The Authors Journal compilation r Blackwell Publishing Ltd. 2006

Time one measures Favourability of feedback 1. Self 2.66 2. Line manager 2.53 3. Peer 2.60 4. Subordinate 2.88 5. Other 2.57 .49 .46 .49** .40 .52** .28* .36 .26 .57** .44 .50** .47** .31* .54** .45* .07 .19 .14 .77** .50** .49** .47** .62** .41* .65** .55** .10 .06 .53** .52** .39** .56** .55** .59** .29 .82** .57** .44** .30** .77* .34* .50**

360 DEGREE FEEDBACK AND DEVELOPMENTAL OUTCOMES

Perceived importance of feedback dimensions 6. Self 2.76 .55 .65** .41** 7. Line manager 2.64 .55 .30** .64** 8. Peer 2.64 .48 .40** .27* 9. Subordinate 2.94 .39 .32* .60** 10. Other 2.58 .52 .43** .48** .15 .15

Initial self-efficacy 11. Global 3.80 .62 .15 12. For development 3.62 .63 .09 .03 .13 .04 .08 .17 .08 .18 .08

.07 .28 .21* .22

.08 .05

.68**

Time two measures Favourability of feedback 13. Self 2.80 .44 .72** .50** 14. Line manager 2.68 .47 .49** .44** .63** .49** .08 .21* .06 .14 .29 .21 .03 .14 .05 .47** .41* .54** .51** .51** .36** .18 .32** .15

.38** .43** .14 .14 .06

.47** .47** .03 .14 .04

.51** .38* .23 .14 .05

.49** .14 .55** .08 .07 .10 .04 .50** .16 .13

.02 .06 .52** .24* .39** .29** .24* .41** .36** .58** .21* .20 .10 .47** .47**

Subsequent self-efficacy 15. Global 3.80 .58 .29** .25* 16. For development 3.56 .83 .33** .22*

Developmental activity 17. Amount 3.39 1.18 .21* .07

Volume 14 Number 1 March 2006

*po.05, **po.01.

57

58

CAROLINE BAILEY AND MICHELLE AUSTIN

focal managers self-report on a four-item measure (taken from Schwarzer, 1992). An example item is I am positive that I could deal efficiently with unexpected events. Ratings were made along a five-point rating scale from strongly disagree (coded 1) to strongly agree (coded 5). The same measure was utilized before participation in feedback (as a predictor variable) and 3 months after participation (as an outcome measure). Time one a 5 .76, time Two a 5 .72 suggesting measures had acceptable internal consistency (Nunnally, 1978). The scale average was computed for each focal individual and used in regression analyses.

meet with their line manager, and create a development plan on the basis of the 360-feedback received. Measures of self-efficacy were not included in this feedback report. Three months after receiving the feedback report, focal individuals and their line managers were asked to complete a second assessment of the focal individual, using the same 360-feedback questionnaire. Focal individuals also completed a second assessment of their global self-efficacy and self-efficacy for development, as well as indicating the amount of developmental activity engaged in since initial participation in feedback. Responses to all these measures were returned directly to the researcher.

Self-Efficacy for Development

Defined as an individuals belief in their capability to apply what they learn from 360-feedback, self-efficacy for development was measured by focal managers selfreport on a three-item measure (written specifically for the current study), and rated on a five-point rating scale (as for global self-efficacy). An example item is I am confident in my ability to apply what I learn from 360-feedback to my job. As with global self-efficacy, the same measure of selfefficacy for development was utilized before participation in feedback as a predictor variable, and 3 months after participation as an outcome measure. Time one a 5 .58, time two a 5 .76. Although internal consistency at time one was lower than the usual criterion level recommended (Rust & Golombok, 1997), as only a three-item scale, the authors considered the internal consistency of this scale to be acceptable.

Results
Means, standard deviations and correlations between T1 and T2 variables are shown in Table 1.

Group Trends in Feedback

As shown in Table 1, mean values for self- and externalraters feedback ratings indicate that typically focal individuals received feedback that was favourable, according to rating scale descriptors. Similarly, on average, selfand external-raters regarded feedback to be about behaviours that were important to the focal individuals role. Although sample means indicated some differences in the favourability of feedback ascribed by different rater groups, repeated measures ANOVA indicated these were not significant differences (F4, 19 5 2.84, p4.05). Similarly, there were small, but non-significant differences in mean ratings of feedback importance between rater groups; these were not statistically significant (F4, 19 5 .54, p4.05). This suggests that typically focal individuals were receiving a relatively consistent message from their feedback providers, with regard to their current level of effectiveness and the importance of feedback dimensions to their role. As shown in Table 1, descriptives for self-assessments and line managers ratings indicate that ratings increased in favourability following participation in feedback; paired sample t-tests indicated a significant difference (t 5 3.39, df 5 65, po.01 and t 5 2.48, df 5 65, po.05, respectively). On an average, focal managers ratings of amount of developmental activity was above the mid-point on the rating scale (indicating at least a moderate amount of developmental activity was undertaken following participation in the programme). However, as also shown in Table 1, standard deviation statistics for these variables indicate not all managers attain such positive outcomes; focal individuals more than 1.5 standard deviations below the mean in these measures did not attain favourable ratings. The focal individuals average level of global self-efficacy remained unchanged over time. Paired sample t-tests indicated average self-efficacy for development decreased from T1 to T2 but not significantly (t 5 .76, df 5 65,

Self-Report of Developmental Activity

As part of the survey containing self-efficacy items at time two, participants were also asked to indicate How much developmental activity have you engaged in since your first feedback?. This item was rated on a five-point scale from 0 (none) to 4 (a great deal).

Procedure
Having volunteered to participate in the study, focal managers completed a self-assessment and nominated raters to also provide assessments (ratings of effectiveness and the importance of competency dimensions to the individuals role), as part of the organizations 360feedback process. At this point, focal managers also provided measures of global self-efficacy and self-efficacy for development. Feedback ratings and importance of feedback ratings were provided anonymously by external raters, and returned directly to system administrators who then compiled ratings into a feedback report, copies of which were given to both the focal individual and their line manager. The focal individual was then encouraged to

International Journal of Selection and Assessment

r 2006 The Authors Journal compilation r Blackwell Publishing Ltd. 2006

360 DEGREE FEEDBACK AND DEVELOPMENTAL OUTCOMES

59

p4.05). The sample mean indicated individuals were still efficacious following participation in feedback, as it was above the scale mid-point. Thus, as anticipated, mixed responses to participation in feedback were observed in focal individuals. Factors contributing to this variability were examined through regression analyses.

The Favourability of Feedback as a Predictor of Criterion Measures

As shown in Table 1, small, non-significant correlations were observed between the favourability of ratings from all external rater sources, and the amount of developmental activity the focal individual engaged in. Simultaneous regression examined the extent to which the favourability of external raters feedback (line manager, peers, subordinates and others) predicted variance in developmental activity. The effect size was small and not statistically significant (adjusted R2 5 .04, F 5 .24, p4.05). It was concluded data did not support Hypothesis 1. As shown in Table 2, changes in focal individuals selfassessments were predicted by the favourability of feedback from their line manager and peers. However, the association between predictors and criterion was in the

opposite direction to that anticipated by Hypothesis 2a; individuals who received favourable feedback from external rater sources subsequently had greater changes in self-assessments. The only significant predictor of changes in line managers ratings was the favourability of peers feedback. Again, contrary to expectations, greater changes in line managers ratings were predicted by individuals receiving favourable (rather than unfavourable) feedback from their peers. In combination, it was concluded data offered very limited support for Hypothesis 2b. An examination of correlation coefficients in Table 1 and regression analyses (as previously described under Hypothesis 1) indicated whether the amount of developmental activity engaged in mediated the association between the favourability of feedback received and changes in ratings over time (Hypotheses 2c and 2d). Baron and Kenny (1986) describe mediation as a causal chain (p. 1176) with a mediator reflecting how or why there is an association between an IV and DV. They outline the test of mediation as a three-part process. Firstly, the IV must be a significant predictor of the mediator; secondly, the mediator must be a significant predictor of the DV. Finally, the IV must be significantly predictive of the DV. As shown

Table 2. Hierarchical regression analyses: favourability of external raters feedback as predictors of criterion measures Step Variable(s) entered R2 .51*** .63*** R2 change .12*** Standardized b .72*** .18*** .32*** .05 .03 .44*** .32** .11 .19 .50*** .14 .09 .18 .12 .24* .10 .21 .07 .01 t 9.02 2.01 3.65 .67 .33 4.35 2.94 1.07 1.62 5.10 1.16 .80 1.72 .99 2.15 .78 1.60 .61 .05

Dependent variable: self-assessment (T2) 1. Self-assessment (T1) 2. Line manager (T1) Peer Subordinate Other Dependent variable: line manager rating (T2) 1. Line manager rating (T1) 2. Peer Subordinate Other Dependent variable: global self-efficacy (T2) 1. Global self-efficacy (T1) 2. Line manager (T1) Peer Subordinate Other

.19*** .37***

.18***

.25*** .32***

.07

Dependent variable: self-efficacy for development (self-efficacy, development) (T2) 1. Self-efficacy development (T1) .05* 2. Line manager (T1) .13 .08 Peer Subordinate Other
*po.05, **po.01, ***po.001.

r 2006 The Authors Journal compilation r Blackwell Publishing Ltd. 2006

Volume 14 Number 1 March 2006

60

CAROLINE BAILEY AND MICHELLE AUSTIN

Table 3. Hierarchical regression analyses: initial self-assessment and favourability of feedback received as predictors of change in line managers ratings, amount of development, global self-efficacy and self-efficacy for development Block Variable(s) entered R2 .19*** .29*** .39*** R2 change Standardized b .44*** .36** .25* .11 .16 .21 .05 .07 .10 .04 .50*** .22* .08 .01 .18 .16 .24* .31** .04 .11 .08 .05 t 4.35 3.36 2.18 1.14 1.34 1.91 .37 .47 .79 .30 5.10 2.28 .65 .05 1.77 1.32 2.15 2.94 .30 .83 .66 .37

Dependent variable: line manager rating (T2) 1. Line manager rating (T1) 2. Self-assessment (T1) 3. Peer Subordinate Other Dependent variable: amount of development 1. Self-assessment (T1) 2. Line manager (T1) Peer Subordinate Other Dependent variable: global self-efficacy (T2) 1. Global self-efficacy (T1) 2. Self-assessment (T1) 3. Line manager (T1) Peer Subordinate Other

.10** .10*

.03 .00

.01

.24*** .29*** .35***

.05* .06

Dependent variable: self-efficacy for development (T2) 1. Self-efficacy for development (T1) .04* 2. Self-assessment (T1) .14** 3. Line manager (T1) .16* Peer Subordinate Other
*po.05, **po.01, ***po.001.

.10** .02

in Table 1, correlation analyses indicated that the favourability of feedback received was not a significant correlate of amount of developmental activity. Hierarchical regression analyses were used to investigate whether amount of developmental activity was significantly associated to changes in self-assessments or line managers ratings. The self-assessment or line managers rating (as appropriate) at time one was entered in the first block and amount of development on the second block, as predictor of time two ratings. R2 change statistics were not significant for either changes in the self-assessment or line managers ratings (R2 change 5 .00, p4.05 and R2 change 5 .01, p4.05, respectively. It was concluded data did not support Hypotheses 2c or 2d. Changes in self-efficacy were not accounted for by the favourability of feedback from line manager, peers, subordinates or other raters. In both equations (one for each form of self-efficacy), R2 change values were small and not statistically significant (see Table 2). It was concluded that data did not support Hypotheses 3a or 3b.

Further to these hypotheses, four exploratory hierarchical regression analyses were conducted to ascertain the role of the initial self-assessment in (i) changes in the line managers rating (Hypothesis 2e), (ii) amount of developmental activity (Hypothesis 2f), (iii) changes in global selfefficacy (Hypothesis 3c) and (iv) self-efficacy for development (Hypothesis 3d) over time, in comparison with favourability of feedback received. The dependent variable in all equations was the T2 measure. In the first step, the T1 measure of the criterion variable was entered as a control for initial values. The focal individuals initial self-assessment was entered in the second step and favourability of feedback from external raters entered on the third step. Significant findings are presented in Table 3. The self-assessment was a significant predictor of three criterion variables changes in line managers ratings, global self-efficacy and self-efficacy for development (neither the self-assessment nor favourability of feedback were significant predictors of amount of developmental activity). In two out of three significant equations, R2

International Journal of Selection and Assessment

r 2006 The Authors Journal compilation r Blackwell Publishing Ltd. 2006

360 DEGREE FEEDBACK AND DEVELOPMENTAL OUTCOMES

61

change values indicated the self-assessment was a better predictor than the favourability of feedback received. The only exception to this was the prediction of line managers ratings where the favourability of peers ratings significantly contributed to variance, over and above the initial self-assessment. It was concluded that data supported Hypotheses 2e, 3c, 3d, but not Hypothesis 2f.

Self-Efficacy as a Moderator of Feedback Impact

As shown in Table 1, initial self-efficacy for development had a small, positive association to developmental activity. However, when examined by simple linear regression, it was not found to be a significant predictor (adjusted R2 5 .03, F 5 2.84, p 5 .10, standardized b 5 .21). Initial global self-efficacy was not a significant correlate or predictor of developmental activity. It was concluded data did not support Hypothesis 4. It had been hypothesized that self-efficacy may moderate the association between favourability of feedback and amount of developmental activity (Hypothesis 5). However, as shown in Table 1, near zero, non-significant correlations were observed between the favourability of feedback from all external rater sources and amount of developmental activity. Hence, analyses for moderation effects under Hypothesis 5 were not conducted. However, as the favourability of feedback from certain rater sources had been found to predict changes in selfassessments and line managers ratings, the extent to which initial self-efficacy moderated these associations was explored. Defined operationally by Baron and Kenny (1986) as a variable that affects the direction and/or strength of the relation between an independent or predictor variable and a dependent or criterion variable (p. 1174), a moderator effect is indicated if the regression term representing the interaction between the predictor and hypothesized moderator is significant. They note that ideally the moderator variable does not correlate with either the predictor variable or the criterion; as shown in Table 1, both forms of self-efficacy fulfil this condition. Regression terms representing the interaction between self-efficacy and feedback favourability were computed for each rater source, for both forms of self-efficacy. Changes in criterion measures were analyzed as before, with the T1 measure of the criterion entered in the first block as a control, the predictor (favourability of feedback) and moderator (either global self-efficacy or self-efficacy for development) entered as separate terms on the second step and the interaction between the predictor and moderator entered on the third step, with the T2 measure as the dependent variable. The extent to which self-efficacy significantly moderated the feedback-criterion association was indicated by R2 change between steps 2 and 3. For prediction of changes in self-assessments, eight equations were tested: 2 self-efficacy 4 rater combinations (line manager, peer, subordinate or other). For changes in line

managers ratings six equations were tested: 2 selfefficacy 3 rater combinations (peer, subordinate or other). The results of analyses with significant findings are presented in Table 4. The number of significant effects exceeds that expected by chance. As shown in Table 4, both forms of self-efficacy were found to significantly moderate aspects of feedback impact. The nature of the moderator effects appears to vary across rater sources. Analyses conducted under Hypothesis 6a (moderator effects for changes in selfassessments) indicated that when global self-efficacy is low, there is a stronger association between the favourability of line managers ratings and changes in the self-assessment, than when it is high (see Figure 2). Conversely, for peers ratings, high global self-efficacy increased the strength of the association between favourability of peers ratings and changes in self-assessments (see Figure 3). Similarly a stronger association was observed between others ratings and changes in line managers ratings when global selfefficacy was high (see Figure 5). Finally, analyses of moderator effects under Hypothesis 6b (changes in line managers ratings) found self-efficacy for development significantly moderated the association between the favourability of line managers ratings and changes in the self-assessment (see Table 4 and Figure 4). When selfefficacy for development is high, there is a moderate negative association between line managers ratings and changes in the self-assessment; when it is low, this association is much weaker. It was concluded data provided some support for Hypothesis 6a and limited support for Hypothesis 6b.

Perceived Importance of Feedback as a Moderator of Feedback Impact

It was hypothesized that focal individuals perceptions of feedback importance may moderate the association between the favourability of feedback received and amount of developmental activity (Hypothesis 7), changes in selfassessments (Hypothesis 8a) and line managers ratings (Hypothesis 8b), and changes in global self-efficacy (Hypothesis 9a) and self-efficacy for development (Hypothesis 9b). Moderator effects were again explored by hierarchical regression analyses, as described previously (replacing selfefficacy terms with importance of feedback). In total, 20 equations were analyzed (5 DVs 4 feedback sources). Despite sufficient statistical power for regression analyses, perceived importance of feedback did not significantly moderate the association between feedback favourability (from any source) and amount of developmental activity or changes in self-assessments, line managers ratings, global self-efficacy or changes in self-efficacy for development. It was concluded data did not support hypotheses 7, 8a, 8b, 9a or 9b.

r 2006 The Authors Journal compilation r Blackwell Publishing Ltd. 2006

Volume 14 Number 1 March 2006

62

CAROLINE BAILEY AND MICHELLE AUSTIN

Table 4. Significant findings for hierarchical regression analyses: moderator effects of initial self-efficacy Step Variable(s) entered R2 .51*** .55*** .59*** .51*** .60*** .65*** .51*** .55*** .65*** .19*** .28* .32* R2 change .04* .04* Standardized b .72*** .22** .01 .20* .72*** .36*** .07 .24** .72*** .23** .07 .33*** .44*** .34** .03 .20* t 9.02 2.53 .09 2.59 9.02 4.21 .91 3.09 9.02 2.65 .88 4.66 4.35 3.09 .30 2.02

Dependent variable: self-assessment (T2) 1. Self-assessment (T1) 2. Line manager (T1) Global self-efficacy (T1) 3. Line manager global self-efficacy Dependent variable: self-assessment (T2) 1. Self-assessment (T1) 2. Peer Global self-efficacy (T1) 3. Peer global self-efficacy Dependent variable: self-assessment (T2) 1. Self-assessment (T1) 2. Line manager (T1) Self-efficacy development (T1) 3. Line manager self-efficacy development Dependent variable: line manager rating (T2) 1. Line manager (T1) 2. Other Self-efficacy development (T1) 3. Other self-efficacy development
*po.05, **po01, ***po.001.
Global Self-Efficacy Moderator Effects Change in SelfAssessment (sd)

.09*** .05**

.04* .10***

.09* .04*

Self-Efficacy for Development Moderator Effects Change in SelfAssessment (sd) +1.0

+1.0 Global S.Eff low 1.0 1.0 +1.0 Global S.Eff high

S.Effdev low S.Effdev high 1.0 1.0 +1.0 Favourability Line Mgr (sd)

Favourability Line Mgr (sd)

Figure 2. Moderator effects of global self-efficacy: favourability of line managers feedback as predictor of change in self-assessments.
Global Self-Efficacy Moderator Effects Change in SelfAssessment (sd) +1.0 Global S.Eff low Global S.Eff high 1.0 1.0 +1.0 Favourability Peer (sd)

Figure 4. Moderator effects of self-efficacy for development: favourability of line managers feedback as predictor of change in self-assessments.
Global Self-Efficacy Moderator Effects Change in Line Mgr rating (sd) +1.0
Global S.Eff low Gobal S.Eff high

1.0 1.0 +1.0 Favourability Others (sd)

Figure 3. Moderator effects of global self-efficacy: favourability of peers feedback as predictor of change in selfassessments.

Figure 5. Moderator effects of global self-efficacy: favourability of others feedback as predictor of change in line managers ratings.

International Journal of Selection and Assessment

r 2006 The Authors Journal compilation r Blackwell Publishing Ltd. 2006

360 DEGREE FEEDBACK AND DEVELOPMENTAL OUTCOMES

63

Discussion
This study sought to identify whether the developmental outcomes observed with participation in a 360-feedback program were predicted by the favourability of feedback alone or a combination of factors (specifically, the favourability of feedback, perceived importance of feedback and/or individuals self-efficacy beliefs). The key finding of this investigation was that while favourability of feedback (from certain rater sources) was predictive of criterion measures, focal individuals initial self-assessments and self-efficacy before participation in feedback were also implicated in the feedbackperformance association. As such, this study supports Maurer et al. (2002), providing further empirical evidence of individual differences that moderate the feedbackperformance association within 360-feedback, and aspects of the theoretical work of Ashford (1989) regarding self-image re-evaluation. Examining first central hypotheses, the nature of feedback from certain rater sources (line managers and peers) was predictive of changes in self-assessments and to a lesser extent line managers successive ratings (peers ratings only). Note that the design of this study does not enable us to determine whether the changes in ratings reflect a change in actual performance or change in raters perceptions (as a result of seeing others ratings) or some combination of the two. As such, findings are discussed in terms of changes in ratings, without detailed assumption of what this represents in terms of feedback recipients behaviours and/or raters perceptions. A priority for future 360feedback research is to address this fundamental research question what do changes in ratings represent?. Contrary to much feedback literature and the central tenet of London and Smithers (1995) model (that individuals will respond to a negative discrepancy), there was a positive association between feedback from certain rater sources and changes in self- and line-managers ratings. In the current sample, when controlling for the initial favourability of assessments, focal individuals subsequent ratings became more favourable when receiving favourable, rather than unfavourable feedback ratings; receiving unfavourable ratings was associated to smaller increments in ratings over time. This highlights an important issue previously discussed by critics of 360-feedback processes that receiving negative ratings may not necessarily result in constructive reactions. Individuals who receive negative feedback are by definition those that have development needs, and those arguably of most concern to the organization (in introducing a developmental 360-feedback system, it is presumed the organizations objective is primarily for improving under-performers, rather than recognition of good performance, which may be achieved through more formal reward systems). The feedbackperformance association observed in the current study would suggest the 360-

feedback process may provide approbation for those that are performing (resulting in enhancement of perceptions and/or positive behavioural change), but has less positive consequences for those individuals receiving negative feedback. How to manage reactions appropriately is a complex issue; this study would suggest that greatest care is needed when facilitating feedback that is negative; it cannot be assumed that individuals will have constructive reactions to negative ratings. Additionally, it seems not all rater sources were attended to equally. The predictive value of line managers ratings over other sources has previously been found in 360feedback research (e.g. Bailey & Fletcher 2002; Gregarus, Ford & Brutus, 2003; Smither et al., 2003). However, it is interesting to note that subordinate ratings were not found to be significant predictors at all. This fits with the findings of Brett and Atwater (2001) who concluded from their investigation of perceptions of 360-feedback usefulness, that feedback from direct reports appeared not to influence reactions as much as those from bosses and peers (p. 940). However, in the main, this goes against a growing body of U.S. literature which has found subordinate ratings to be one of the best predictors of feedback impact (e.g. Brutus, London, & Martineau, 1999) and attitudes to the feedback process (e.g. Maurer et al., 2002). Whatever the cause, this finding supports the common practice of providing a feedback report to the focal individual that has a breakdown of ratings from each rater source. Further research is needed that examines the credibility of different rater sources, and factors influencing focal individuals attentiveness to particular sources. At present, findings are limited to one study by Bailey and Fletcher (2002). As hypothesized, both global self-efficacy and selfefficacy for development were found to significantly moderate the association between feedback received from certain sources and changes in ratings (line managers ratings and/or changes in the self-assessment). However, the nature of moderator effects on the feedbackperformance association was not constant, varying as a function of rater source and type of self-efficacy. Focal individuals who are generally confident in their capabilities (i.e. high global self-efficacy) respond more positively to favourable ratings from their peers and others, whereas focal individuals who are not confident in their capabilities respond more positively to favourable ratings from their line manager. Thus, it is tentatively suggested that having confidence in ones ability appears to relate to who is perceived as a useful source of feedback. In particular, individuals who are confident in their own capabilities rely on peers and others ratings as perhaps a fine tuning of performance, whereas those with low selfefficacy rely more on line managers, as this individual has direct authority and supervisory responsibility. Again, the plausibility of this can only be determined by further research.

r 2006 The Authors Journal compilation r Blackwell Publishing Ltd. 2006

Volume 14 Number 1 March 2006

64

CAROLINE BAILEY AND MICHELLE AUSTIN

Interestingly, moderator effects for self-efficacy for development were in the opposite direction to that observed for global self-efficacy. A negative feedback performance association was observed, with high selfefficacy for development increasing the association between line managers ratings and changes in the selfassessment. Individuals with high self-efficacy for development changed self-assessments more in response to negative ratings from the line manager. It is possible that this reflects the interplay between the line manager suggesting development needs and the role of line managers in supporting development. If the individual believes they have that support (i.e. likely to have high self-efficacy for development), it logically follows that these individuals will show greater changes than those who do not have such strong capability beliefs (or receive relatively favourable ratings from the line manager). Whatever the cause for differential moderator effects, from a practical perspective, it suggests that feedback ratings from different sources may be interpreted very differently, according to the individuals level of self-efficacy (rather than actual favourability of ratings ascribed). To ensure the individuals response to feedback is constructive, organizations have a responsibility to ensure that sufficient support is given. In particular, it may be useful to explore issues around participants current capability beliefs before receiving feedback, as well as following feedback, to help them understand and manage their reactions to the feedback process and feedback from different sources. Given the predictive value of line managers ratings, and the moderator effect observed with low global self-efficacy, organizations may need to be wary of using 360-feedback as a developmental process with those employees who may be generally less confident in their level of ability (i.e. inexperienced new hires). More support may be needed in interpretation and personal development planning for these individuals than for more experienced employees. The favourability of initial self-assessments (before receiving ratings from external raters) predicted changes in ratings by the line manager. Individuals with a positive self-image (favourable self-assessments) were perceived to improve over time, or at least were rated more favourably at time two. This supports Ashford (1989), who argues that self-image is strongly associated to an individuals performance and career progression within an organization. In addition, the initial self-assessment was also found to be a better predictor of changes in global self-efficacy and selfefficacy for development over time than the favourability of feedback received from external raters. Given the links between self-efficacy and concepts such as motivation and goal setting (Erez & Judge, 2001; Locke, 1996), and performance and job satisfaction (Judge & Bono, 2001), the self-assessment seems as crucial to effective development as feedback ratings. Moreover, as the self-assessment predicted changes in global self-efficacy (with favourable self-assessments associated to increments

in global self-efficacy following feedback), it seems likely there are implications for employee behaviours beyond the context of developmental activities arising from 360feedback. While feedback literature does acknowledge the role of the individuals self-image, it is discussed largely in relation to the impact of external raters ratings. The current study suggests that the self-assessment alone is also of major developmental importance. So, to truly understand the impact of feedback upon an individual, these findings suggest that researchers and feedback counsellors should give as much attention to the self-assessment as the ratings received from external raters. In sum, the predictive value of the initial self-assessment and the significant moderating effects of initial self-efficacy to criterion measures reinforces the need for organizations to support the focal individual by facilitating both the feedback and development process. If organizations can enhance individuals capability beliefs about responding to feedback, findings here suggest this will promote a positive self-image. Following the tenets of Judge, Locke, and Durhams (1997) theory of core self-evaluations, this in turn should lead to performance improvement and so long-term benefits for the individual, their team and the organization as a whole. There are a number of limitations to the current study. As with most research conducted in organizations, it was not possible to compare findings for this sample to a control group of managers who did not participate in feedback. Instead, this study relies on a comparison of preand post-participation measures, where there is a risk of response-scale shift (Martineau, 2004) and/or contamination because of common method variance and recall effects. The organizations 360-feedback process includes providing each focal managers line managers a copy of the feedback report containing comparative ratings made by other rater sources. This could be posited as a possible influence on self-competency and line managers competency ratings at time two, thereby clouding whether these criterion measures represent actual changes in performance or simply changes in raters perceptions, as a result of seeing others perceptions of the individual. In addition, the rating context changed from time one (for development) to time two (for research). Furthermore, the final sample size available is somewhat smaller than that considered suitable for regression analyses (Tabachnick & Fidell, 2001). The relatively short time between initial participation and outcome measures (3 months) means conclusions cannot be offered for anything other than the most immediate short-term impacts of feedback. It may be with a longer time frame, that focal individuals have more opportunity to engage in developmental activity and/or change their behaviour, and hence more significant changes could be observed in time. Despite sufficient statistical power for regression analyses, the extent to which the focal individual perceived feedback to be about behaviours that are important to their role, was not found to moderate feedback impact. This may reflect a

International Journal of Selection and Assessment

r 2006 The Authors Journal compilation r Blackwell Publishing Ltd. 2006

360 DEGREE FEEDBACK AND DEVELOPMENTAL OUTCOMES

65

lack of variance in ratings of feedback importance (descriptive data indicated that focal individuals generally regarded behaviours identified in their feedback reports as important). Finally, there appear to be problems in the criterion measure of developmental activity; unlike other criteria it was found to be unrelated to any of the predictor variables. In addition it was a broad-brush self-report of developmental activities, susceptible to subjective interpretation (i.e. what constitutes a developmental activity?). Following the differences in developmental outcomes found by Maurer et al. (2002) using a measure that enables a fine grain analysis of developmental activities (i.e. on-thejob vs. off-the-job training) could provide greater insights into the effects of variables examined here. In conclusion, this study provides further evidence that there are factors beyond the characteristics of feedback operating within the feedback-performance association of developmental 360-feedback systems. In order to promote developmental activity, findings suggest organizations need also to consider the focal individuals initial self-image and self-efficacy beliefs before participation. As such, it seems the briefing and preparation for feedback and subsequent support for development may be just as important determinants of outcomes as the characteristics of feedback received.

Acknowledgements
The authors are indebted to Professor Jeff Edwards, and his advice regarding the use of regression analyses to operationalize change.

References
Anderson, J.R. (1982) Acquisition of cognitive skill. Psychological Review, 89(4), 369406. Anderson, J.R. (1983) Skill acquisition: Compilation of weak method problem solutions. Psychological Review, 94(2), 192210. Ashford, S.J. (1989) Self-assessment in organizations: A literature review and integrative model. Research in Organizational Behavior, 11, 133174. Atkins, P.W.B. and Wood, R.E. (2002) Self- versus others ratings as predictors of assessment center ratings: Validation evidence for 360-degree feedback programs. Personnel Psychology, 55, 871904. Atwater, L.E., Ostroff, C.E., Yammarino, F.J. and Fleenor, J.W. (1998) Selfother agreement: Does it really matter? Personnel Psychology, 51(3), 577598. Bailey, C. and Fletcher, C. (2002). When do other peoples opinions matter? The credibility of feedback from co-workers. British Psychological Society Annual Occupational Psychology Conference 2002 Blackpool, U.K., Book of Proceedings, pp. 109113. Bandura, A. (1977) Self-efficacy: Toward a unifying theory of behavioral change. Psychological Review, 84(2), 191215. Bandura, A. (1986) Social foundations of thought and action: A social cognitive theory. Englewood Cliffs, NJ: Prentice Hall. Bandura, A. (1997) Self-efficacy: The exercise of control. New York: Freeman. Baron, R.M. and Kenny, D.A. (1986) The moderatormediator variable distinction in social psychological research: Conceptual,

strategic, and statistical considerations. Journal of Personality & Social Psychology, 51(6), 11731182. Bastos, M. and Fletcher, C. (1995) Exploring the individuals perceptions of sources and credibility of feedback in the work environment. International Journal of Selection and Assessment, 3(1), 2940. Beehr, T., Ivanitskaya, L., Hansen, C., Erofeev, D. and Gudanowski, D. (2001) Evaluation of 360 degree feedback ratings: Relationships with each other and with performance and selection predictors. Journal of Organizational Behavior, 22, 775788. Brett, J. and Atwater, L. (2001) 360 degree feedback: accuracy, reactions and perceptions of usefulness. Journal of Applied Psychology, 85(5), 930942. Brutus, S., London, M. and Martineau, J. (1999) The impact of 360degree feedback on planning for career development. Journal of Management Development, 18(8), 676693. Carver, C.S. and Scheier, M.F. (1981) Attention and self-regulation: A control-theory approach to human behavior. New York: Springer. Cronbach, L.J. (1990) Essentials of psychological testing. New York: Longman. Edwards, J.R. (1995) Alternatives to difference scores as dependent variables in the study of congruence in organizational research. Organizational Behavior & Human Decision Processes, 64(3), 307324. Edwards, J.R. (2001) Ten difference score myths. Organizational Research Methods, 4(3), 265287. Erez, A. and Judge, T.A. (2001) Relationship of core self-evaluations to goal setting, motivation, and performance. Journal of Applied Psychology, 86(6), 12701279. Eyring, J., Johnson, D. and Francis, D. (1993) A cross-level units-ofanalysis approach to individual differences in skill acquisition. Journal of Applied Psychology, 78, 805814. Fletcher, C. (2001) Performance appraisal and management: The developing research agenda. Journal of Occupational and Organizational Psychology, 74(4), 473488. Greguras, G., Ford, J. and Brutus, S. (2003) Manager attention to multisource feedback. Journal of Management Development, 22(4), 345361. Hazucha, J.F., Hezlett, S.A. and Schneider, R.J. (1993) The impact of 360-degree feedback on management skills development. Human Resource Management, 32(23), 325351. Judge, T.A. and Bono, J.E. (2001) Relationship of core selfevaluations traits self-esteem, generalized self-efficacy, locus of control, and emotional stability with job satisfaction and job performance: A meta-analysis. Journal of Applied Psychology., 86(1), 8092. Judge, T.A., Locke, E.A. and Durham, C.C. (1997) The dispositional causes of job satisfaction: A core evaluations approach. Research in Organizational Behavior, 19, 151188. Kahneman, D. and Tversky, A. (1979) Prospect theory: An analysis of decision under risk. Econometrica, 47, 263291. Kluger, A.N. and DeNisi, A. (1996) The effects of feedback interventions on performance: Historical review, a meta-analysis and a preliminary feedback intervention theory. Psychological Bulletin, 119, 254284. Locke, E.A. (1996) Motivation through conscious goal setting. Applied and Preventative Psychology, 5, 117124. London, M. and Smither, J.W. (1995) Can multi-source feedback change perceptions of goal accomplishment, self-evaluations and performance related outcomes? Theory based applications and directions for research. Personnel Psychology, 48, 803839. Martineau, J. (2004) The effects of construct shift on growth and accountability models. Dissertation Abstracts International, Section A: Humanities and Social Sciences, 65(4-A), 1335.

r 2006 The Authors Journal compilation r Blackwell Publishing Ltd. 2006

Volume 14 Number 1 March 2006

66

CAROLINE BAILEY AND MICHELLE AUSTIN

Maurer, T., Mitchell, D. and Barbeite, F. (2002) Predictors of attitudes toward a 360-degree feedback system and involvement in post-feedback management development activity. Journal of Occupational and Organizational Psychology, 75, 87107. Nunnally, J. (1978) Psychometric theory (2nd Edn.). New York: McGraw-Hill. Rust, J. and Golombok, S. (1997) Modern psychometrics. UK: Routledge. Schwarzer, R. (Ed). (1992) Self-efficacy: Thought-control of action. Washington, DC: Hemisphere Publishing Meta-Analysis Corporation. Smither, J.W., London, M., Flautt, R., Vargas, Y. and Kucine, I. (2002, April) Does discussing multisource feedback with raters enhance performance improvement? Paper presented at the 17th

Annual Conference of the Society of Industrial and Organizational Psychology, Toronto, ON. Smither, J.W., London, M., Flautt, R., Vargas, Y. and Kucine, I. (2003) Can working with an executive coach improve multisource feedback ratings over time? A quasi-experimental field study. Personnel Psychology, 56(1), 2344. Stumpf, S.A., Brief, A.P. and Hartman, K. (1987) Self-efficacy expectations and coping with career related events. Journal of Vocational Behavior, 31, 91108. Tabachnick, B. and Fidell, L. (2001) Using multivariate statistics. Allyn & Bacon: Boston, MA. Yammarino, F. and Atwater, L.E. (1997) Do managers see themselves as others see them? Implications for self-other rating agreement for human resource management. Organizational Dynamics, Spring, 1997, 3544.

International Journal of Selection and Assessment

r 2006 The Authors Journal compilation r Blackwell Publishing Ltd. 2006