The Determinants and Performance Effects of Supervisor Bias
Jasmijn C. Bol
University of Illinois at Urbana-Champaign [email protected]
September 2007
Abstract This paper examines the determinants and performance effects of leniency and centrality bias. An empirical analysis of a compensation plan for low-level employees with both objective and subjective performance measures leads to two key results. First, the causes of supervisor bias include: employee performance, the difference between the organizational level of the supervisor and the employee, the financial position of the firm, the duration of the employee-supervisor relationship, and supervisor characteristics. This indicates that supervisors take their own personal preferences into account when appraising employee performance. Second, supervisor bias affects future employee incentives. Contrary to previous assumptions, the results show that biased performance ratings can have both positive and negative effects on incentives. Leniency bias positively affects performance improvement, while centrality bias has a negative effect on performance. Acknowledgements: I want to thank Stan Baiman, Antonio Dávila, Henri Dekker, Chris Ittner, Frank Moers, Fernando Peñalva, Joan Enric Ricart, Stan Veuger and seminar participants at the 2006 Global Management Accounting Research Symposium in Copenhagen, the 2006 AAA Annual Meeting in Washington D.C., the University of Pennsylvania, the London School of Economics, the University of Colorado at Boulder, the University of Illinois at Urbana-Champaign, the London Business School and the Harvard Business School for their comments and suggestions.
1
INTRODUCTION
This paper provides empirical evidence on the determinants and performance effects of
supervisor bias. Knowledge of supervisor bias is important for managers in charge of incentive
design. Understanding the causes of supervisor bias, and the consequences of supervisor bias on
future performance, can help compensation system designers make better-informed decisions
about how much supervisor discretion to allow for in performance-based compensation plans.
In this paper, I focus on two well-known forms of supervisor bias: leniency bias and
centrality bias (Prendergast, 1999). Leniency bias is the tendency to provide employees with
inflated subjective performance ratings (Bretz, Milkovich, & Read, 1992), while centrality bias is
the tendency to compress performance ratings, creating less variance in performance ratings than
in actual performance. Although there is considerable empirical evidence indicating that the use
of subjective performance measures leads to lenient and compressed ratings (Landy & Farr,
1980; Murphy & Cleveland, 1991; Bretz et al., 1992; Prendergast & Topel, 1993; Jawahar &
Williams, 1997; Prendergast, 1999; Moers, 2005), empirical evidence on how supervisor bias
actually influences employee behavior is almost entirely absent. This paper extends the current
literature by focusing both on what causes a supervisor to bias performance ratings and on how
biased ratings affect the incentive effects of performance-based compensation contracts.
I study a compensation plan for low-level employees of a financial service provider. The
incentive system includes both objective and subjective performance measures and gives
supervisors some freedom to make ex post modifications to financial rewards. I use contract
design and performance data from five branch offices for 2003 and 2004. I begin my analysis by
establishing that the performance ratings on the subjective dimension are subject to leniency and
centrality bias. I then examine why the performance ratings of some employees are subject to
2
more leniency and centrality bias than those of others. Finally, I examine how these biases affect
performance change between 2003 and 2004.
The results indicate that supervisors do not behave as pure contract-driven robots when
appraising performance. I find that differences in organizational level between employee and
supervisor, and the financial performance of the company, are determinants of leniency and
centrality bias. This is consistent with supervisors taking information-gathering costs and
attitudes towards rating accuracy into consideration when rating behavior. Supervisor
characteristics, the duration of the employee-supervisor relationship, and employee performance
influence the extent of leniency bias, suggesting that supervisors consider the cost of
communicating evaluations and how employee ratings reflect on their own management
capabilities.
Finally, performance ratings influence incentive provision. Leniency bias has a positive
effect on performance improvement, while centrality bias has a negative effect, for both above-
average and below-average performers.
The study makes several important contributions to the performance evaluation and
compensation literature. This is the first paper that empirically investigates how biased
performance ratings influence the effectiveness of a performance-based compensation contract.
Prior studies have established the existence of leniency and centrality bias, but not the
consequences of these biases on future performance. Moreover, this paper does not only present
evidence indicating that the use of subjective performance measures leads to biased performance
ratings; it also provides insights into the determinants of supervisor bias. Finally, this paper
offers a detailed description of the salient features of an incentive system for low-level
employees and of the supervisors’ and employees’ reactions to this system. This gives us insight
3
into the compensation practice for low-level employees, an area in which detailed information is
scarce (Indjejikian, 1999).
This paper consists of five sections. Section II reviews research on supervisor bias in
compensation contracts and develops the hypotheses. The research design is presented in section
III and in section IV the results are analyzed. The final section summarizes the results and
discusses future research possibilities.
I. HYPOTHESES
Most papers dealing with subjectivity in compensation contracts focus on the
determinants of subjectivity. These studies examine the role of subjectivity in incentive systems
and indicate how introducing subjectivity improves contracts by mitigating incentive distortions
or reducing risk (Baker, Gibbons, & Murphy, 1994; Baiman & Rajan, 1995; Hayes & Schaefer,
2000; Ittner, Larcker, & Meyer, 2003; Gibbs, Merchant, Van der Stede, & Vargus, 2004).
However, the fact that performance evaluation is subject to supervisor discretion can also give
rise to a number of problems, the most prominent of which is rating inaccuracy.1
Since the correctness of subjective performance evaluations cannot be assessed by
outside parties, supervisors can take their own preferences into consideration when rating
employee performance. One documented consequence of allowing supervisors to apply
discretion is biased performance ratings (Landy & Farr, 1980). A long line of research, mainly in
human resources management, investigates performance rating accuracy in subjective
performance appraisal and presents considerable evidence on biased performance ratings (e.g.,
1 Inaccurate performance ratings are clearly not the only potential negative consequence of introducing subjectivity into explicit compensation contracts. Other concerns, such as unclear measurements criteria, are also likely to influence the effectiveness of the compensation contract. In this paper, I limit the discussion to examining the effects of biased performance ratings.
4
Murphy & Cleveland, 1991; Bretz et al., 1992; Jawahar & Williams, 1997; Moers, 2005). These
papers find that supervisors tend to rate leniently and that they do not differentiate strongly
between employees when evaluating performance.
The extent to which ratings are biased is not the same for different supervisors or for
different employees evaluated by the same supervisor. Supervisors are predicted to vary the
amount of bias applied to the performance ratings depending on how rating bias influences their
own utility.2 In determining the extent of bias, supervisors consider the potential negative
consequences of communicating performance ratings. Communicating harsh but accurate ratings
to employees will likely damage personal relationships and lead to discussions and criticism. By
offering employees lenient and compressed ratings, supervisors can reduce the real and
psychological cost of communicating evaluations (McGregor, 1957; Bernardin & Buckley,
1981). The extent of bias that supervisors apply to performance ratings is also influenced by the
time and effort needed to gather the appropriate performance information. High information-
gathering costs will make supervisors less willing to invest the required time in information
collection (Harris, 1994). This is predicted to lead to leniency and centrality bias, because
biasing imprecise performance evaluations reduces the chance that resulting imprecise ratings
lead to painful discussions and dissatisfied employees.
Since supervisors’ compensation and promotion possibilities are often linked to employee
performance, supervisors are also expected to take the effects of their rating decisions on
employee incentives into account (Prendergast & Topel, 1993), as well as how the performance
ratings reflect on the supervisors’ own management capabilities (Longenecker, Sims, & Gioia,
2 In this paper I focus on incentive-consistent bias by supervisors. However, supervisors can also unconsciously bias performance ratings due to their cognitive limitations. See, for example, Lipe and Salterio (2000) and O’Connor, Deng and Shields (2006).
5
1987). By rating poorly performing employees more leniently, supervisors may try to hide
departmental problems in order to come across as more capable managers.
Supervisors are also predicted to take their superiors’ attitude towards rating accuracy
into account. Supervisors adjust their biasing behavior based on the expected rewards for
accurate ratings and/or the expected consequences of being perceived as a biased rater by their
own superiors (Longenecker et al., 1987).
Finally, rating behavior is influenced by the utility supervisors get out of (dis)favoring
employees (Prendergast & Topel, 1996). Favoring and disfavoring certain employees leads to
biased ratings, although favoritism does not necessarily lead to leniency and centrality bias.
If supervisors take their own utility into account when rating performance, I expect to
find more leniency and centrality bias in situations where bias is predicted to increase rater
utility. In particular, I predict that the costs of communicating evaluations, information-gathering
costs, employee performance, and the attitude of the supervisor’s superiors towards rating
accuracy influence the extent of leniency and centrality bias present in performance ratings. This
results in the following general hypothesis:
Hypothesis 1: The extent of bias applied to subjective performance ratings is influenced by the costs of communicating evaluations, information-gathering costs, employee performance, and the attitude of the supervisor’s superiors towards rating accuracy.
Employee Incentives and Supervisor Bias
Performance ratings are an important element of incentive contracts because they
summarize the outcome of the performance evaluation process and link performance to pay by
indicating how much compensation the employee will receive. Bias in performance ratings is
likely to influence the incentives created by the compensation contract. In this paper I identify
6
three ways in which bias in performance ratings is expected to influence employee incentives: 1)
through the effect bias has on the link between pay and performance, 2) through the effect bias
has on the perceived fairness of the compensation system, and 3) through the effect bias has on
the performance information received by the employees.
The Link between Pay and Performance
One of the main objectives of using a performance-based compensation system is
motivating employees to supply effort on the right job dimensions.3 According to agency theory,
linking pay to performance motivates employees to exert increased effort to improve
performance, because increased performance results in increased pay (Holmstrom, 1979).
Performance measures play a crucial role in this process because employees direct their attention
to those actions that are measured.
However, the incentive effect of a compensation plan does not depend solely on what is
measured; it also depends on how these actions are measured. Employees will not be motivated
to increase effort unless improved performance is actually expected to translate into more
compensation.4 When performance is assessed subjectively, this may not take place consistently.
Bias clouds the link between pay and performance, affecting the incentive provision of the
compensation plan.
The Perceived Fairness of the Compensation System
Employee incentives are not solely determined by the relation between pay and
performance. The extent to which employees are motivated by a compensation system is also
3 Another important objective of performance measurement is to differentiate between highly skilled and less skilled employees. Although the effect of less than optimal personnel decisions might be severe, the aim of this paper is not to investigate the effect of bias on selection issues. 4 In expectancy theory (Vroom, 1964; Heneman & Schwab, 1972) this is referred to as the degree to which performance is instrumental for the attainment of certain outcomes, and research shows that it is important in motivating individuals.
7
influenced by the perceived fairness of the compensation plan (Akerlof & Yellen, 1988; Blinder
& Choi, 1990; Colquitt, Conlon, Wesson, Porter, & Yee Ng, 2001). Employees not only care
about how their rating compares to their performance, but also about how the received
performance rating compares to their expectations and to the ratings received by others. In the
organizational justice literature, two types of subjective perceptions of fairness are distinguished:
the fairness of the outcome distributions, or distributive justice, and the fairness of the
procedures used to determine these outcome distributions, or procedural justice (Greenberg,
1990). Inaccuracies in performance ratings caused by supervisor bias are predicted to influence
the perceived fairness of the compensation system as bias changes the outcome distribution of
the compensation plan.
Performance Information
Compensation systems also influence employee incentives through their effect on self-
perception. Performance ratings provide employees with information about how their
performance is perceived by the supervisor. This information influences employee incentives by
affecting the employee’s self-perceived marginal productivity of effort (Fang & Moscarini,
2002). Higher self-confidence enhances motivation by affecting the expected result of effort,
which provides supervisors with clear incentives to provide employees with performance ratings
that increase or maintain their self-esteem (Bénabou & Tirole, 2002; 2003).
The Performance Effects of Leniency and Centrality Bias
Supervisor bias is studied extensively in the HR literature in the hope of finding ways to
reduce it, under the maintained assumption that inaccurate ratings per se are bad for
compensation contracting (see Landy & Farr, 1980; Rynes, Gerhart, & Parks, 2005). In this
8
paper, I argue that this assumption may be incorrect. There are several ways in which bias is
expected to influence the functioning of performance-based compensation contracts and they are
not all negative. In the following sections I discuss how the two biases considered in this paper,
leniency and centrality bias, are predicted to influence employee incentives.
Leniency Bias
Higher ratings can affect incentive provision positively by increasing congruence
between performance rating expectations and received performance ratings. Individuals have a
tendency to overestimate themselves5 and therefore to rate themselves higher than their
supervisors do (McFarlane Shore & Thornton, 1986; Harris & Schaubroeck, 1988). Employees
who believe they have received a lower performance rating (and consequently less
compensation) than they deserve, are expected to lower their performance in order to restore a
feeling of equity (Akerlof & Yellen, 1988; Colquitt et al., 2001). More lenient performance
ratings may minimize these feelings of unfairness and the resulting negative effects on
performance.
Moreover, more lenient performance ratings can maintain or increase the employee’s
confidence in his ability and efficacy, which can be valuable to the firm because it increases the
employee’s motivation to undertake ambitious projects and persist even when faced with
adversity (Bénabou & Tirole, 2002).
On the other hand, since performance ratings give employees a signal about how their
performance is perceived by their supervisor, lenient ratings may lead employees to falsely
conclude that they are performing their tasks as desired by their supervisor. Providing employees
5 There exists abundant evidence in the psychology literature that most people overestimate their abilities and past achievements (e.g. Larwood & Whittaker, 1977; Arkin, Cooper, & Kolditz, 1980), that they tend to recall their success more than their failures (e.g. Mischel, Ebbesen, & Zeiss, 1976), and that they have the tendency to be unrealistically optimistic about their future life events (e.g. Weinstein, 1980).
9
with this mistaken impression will have a negative effect on performance because it motivates
them to continue to take wrong/suboptimal actions. Hence, leniency bias is predicted to have a
positive effect on effort, but this will only lead to improved performance if the behavior that is
stimulated is desirable. Since supervisors are expected to be cautious about stimulating
undesirable actions, the overall effect of leniency bias is expected to be positive. I state the
hypothesis as follows:
Hypothesis 2: Leniency bias has a positive effect on the effectiveness of a compensation contract as an incentive provider.
Centrality Bias6
The lack of distinction between performance ratings of different employees is expected to
influence employee incentives in several ways. First, above-average performers are likely to feel
disenchanted when employees who perform worse are rewarded almost equally. This is expected
to negatively affect above-average employees’ future performance (Lazear, 1991). Moreover,
compression reduces the probability that the value of a marginal performance rating increase will
outweigh the cost of the extra effort needed to improve performance sufficiently to receive this
marginal performance rating increase. Centrality bias is thus predicted to have a negative effect
on the incentives of above-average performers. This results in the following hypothesis:
Hypothesis 3: Centrality bias has a negative effect on the effectiveness of a compensation contract as an incentive provider for above-average performers.
The situation is different for below-average performers, as compression influences their
ratings in a positive way. Because of centrality bias, their performance seems similar to that of
top performers. Since most employees consider their own performance to be above-average
6 Although both biases will move up ratings for below-average performance, centrality bias also decreases the variance in reported employee performance.
10
(Meyer, 1975), providing below-average employees with their true comparative position will be
a deflating experience for most of them, which can have a disruptive effect on their performance
(Pearce & Porter, 1986; Gabris & Mitchell, 1988). Compressed ratings might therefore have a
positive effect on the incentives of below-average performers as they protect them from
information that will lower their self-perception and the perceived prospects from providing
effort in the future (Prendergast & Topel, 1993; Fang & Moscarini, 2002).
On the other hand, just as with above-average performers, the lack of variance in
performance ratings might also negatively affect the motivation of below-average performers as
their marginal cost to improve performance ratings may be higher than the marginal benefit they
receive from the performance rating improvement.
Since I cannot predict which of these arguments explains the relation between centrality
bias and the performance change of below-average performers, their relative importance must be
empirically determined.
II. RESEARCH DESIGN
Research Setting
In this paper I use data provided by a Dutch financial service provider (FSP). FSP is one
of the main financial service providers in the Netherlands, and it has a considerable stake in the
European market. It serves over nine million customers and its assets have a total value of some
€500 billion (in 2005).
Recently, competition in the financial sector has become more intense. More specialized
firms have managed to increase their marketshare substantially, jeopardizing FSP’s market
leading position in the Dutch market. In order to defend its market position, FSP decided to
11
move to more results-oriented management, to strengthen the relationships with current
customers and to acquire new customers. To accomplish these objectives FSP restructured its
front-offices, introduced a more pro-active way of approaching potential new customers and
designed a variety of new management control systems. As part of this initiative, FSP introduced
a new incentive system for its branch offices in 2003. The compensation plan covers all branch
office employees, except for the local management teams.
The New Compensation Contract
The new compensation system consists of a fixed salary and a bonus, where the fixed
salary depends on the employee’s job and the periodic fixed salary increase (up to a stated
maximum), and the bonus on the employee’s overall performance rating.7 In the incentive
system, performance is not only objectively measured but also subjectively assessed. More
specifically, half of the overall rating is determined by an output-based performance rating and
half by a competence-based performance rating.
The output-based section of the system consists of three to six subjective or objective
performance measures that measure specific employee output. Examples of objective
performance measures are: ‘the number of insurance policies sold,’ ‘the percentage increase in
portfolio growth,’ or ‘the number of appointments made.’ Examples of subjective outcome
performance measures are ‘the value of the front-office support,’ ‘the quality of the management
information reports,’ or ‘timely preparation of customer reports.’ The designers of the system
7 Although the performance ratings are taken into account when making promotion decisions, the system was designed to determine yearly compensation, not promotions.
12
have provided the supervisors with examples of performance measures, but the supervisors
decide wich performance measures to use. 8
Although the intention of the new compensation system is to improve output, the
designers of the system wanted to avoid a blind focus on improving measured performance. The
compensation system therefore includes the subjective assessment of competencies that are
considered essential to performing the employee’s tasks in an efficient and correct manner. The
supervisor must choose between one and four competence measures, which are subjective
assessments of specific behavior that the employee must demonstrate. Hence, the system not
only indicates what type of output improvement is expected, it also informs the employees how
output improvements are expected to be accomplished. The competencies ‘cooperative behavior’
and ‘customer focus’ are included in all performance documents (see Figure 1).
The weight on each performance measure is determined by the number of measures
chosen within each section. Each section determines half of the overall rating and the weights of
the measures within each section are equal. The supervisor also sets specific targets and margins
for each of the selected measures.9 At the end of the year, the supervisor evaluates performance
according to these measures and targets, and assigns a rating between one and five (not
necessarily integers) to each measure. This leads to an overall rating which determines a)
whether the employee receives the periodic fixed salary increase, and b) the bonus percentage
range. The employee only receives the periodic fixed salary increase if the overall rating is above
1.5.10
8 Although supervisors individually select the performance measures and set the targets, before presenting the contracts to the employees, all supervisors and management meet to discuss and compare the chosen performance measures and targets. In this way FSP hopes to increase the consistency in the use of the system. 9 Margins indicate when a performance target is not met, almost met, met, surpassed and generously surpassed indicated by the scores ‘bad,’ ‘regular,’ ‘good,’ ‘very good’ and ‘excellent,’ respectively. 10 This is only true for employees who have not yet reached the maximum of their salary scale.
13
Additional Discretion in the Compensation System
The use of subjective performance measures is not the only way in which the FSP has
introduced supervisor discretion into its compensation system. Supervisors also have discretion
over the awarded bonus percentage. The overall rating is linked only to a bonus percentage
range,11 not to a concrete percentage and, if deemed necessary, supervisors are allowed to
provide employees with a bonus percentage outside the stated range. These discretionary bonus
adjustments are intended to ensure that important actions that are not foreseen ex ante can still be
rewarded.
In sum, the incentive system links pay to performance and puts part of the compensation
at risk. It does not rely exclusively on objective measures, but combines objective performance
measurement with subjective performance appraisal. Nevertheless, the incentive system gives the
supervisor a relatively wide scope for discretion.
The New versus the Old Compensation System
Although both the new and the old system consist of a fixed salary and possibly a bonus
and/or a periodic fixed salary increase, the new compensation system is different in two
important ways. First, the old system did not use any objective performance measures; only
subjective assessments were used to evaluate performance. Second, in the old system a higher
performance rating did not consistently translate into a higher bonus percentage. In theory, there
was a link between pay and performance, where a higher performance rating should result in a
higher bonus. In practice, however, supervisors seemed to simply increase the bonus percentage
with one or two percentage points when performance was acceptable. This resulted in a situation
11 A ‘bad’ rating (0-1.5) corresponds to a bonus of 0%, a ‘regular’ rating (1.5-2.5) to 1-3%, a ‘good’ rating (2.5-3.5) to 4-8%, a ‘very good’ rating (3.5-4.5) to 9-12% and an ‘excellent’ rating (4.5-5) to 13-15%. See Figure 1.
14
where some employees would, for example, get a bonus of four percent when rated ‘excellent’,
while others received fourteen percent when rated ‘good’.
The lack of a clear link between pay and performanc was the main reason for FSP to
abandon the old system. By creating a clear link, the new system is expected to increase
employee incentives and focus on output. 12 The new system was not introduced to reduce
compensation costs: FSP increased employee compensation so that no employee would receive
less compensation after the introduction of the new system.
Introduction and Implementation of the New Compensation System
FSP took the introduction of the new compensation system very seriously. All employees
and supervisors received information (through presentations, folders, letters, and internet sites)
on the functioning of the system, its formal steps, and the timelines of its implementation. The
directors of the branch offices also wrote a formal letter to all employees indicating top
management’s support for the new system. Supervisors were provided with workshops which
explained how to select the performance measures, how to set the performance targets, and how
to rate performance. In order not to destroy the incentive effect of the system, as had happened
with the old system, supervisors where explicitly asked to rate performance accurately and to
uphold the link between pay and performance.13 Moreover, empirical evidence (e.g.
Longenecker et al., 1987; Burney, Henle, & Widener, 2006) indicates that attaching importance
to the implementation and use of a compensation system reduces the raters’ tendency to bias
performance ratings. The setting of this study is therefore biased against finding the existence of
supervisor bias, and therefore their determinants and consequences.
12 An internal study that examined the consequences of the introduction of the new compensation system finds that both supervisors and employees think that employee effort and focus on output have increased after the introduction of the system. 13 Although the supervisors were asked to rate accurately, the concepts of leniency and centrality bias were not explained and supervisors were not explicitly asked to refrain from applying these biases.
15
Data
The analyses employ two years of proprietary archival data on the incentive plans for
employees of five offices of FSP. I limit my study to employees who were employed by the bank
in both 2003 and 2004 and to departments that made significant use of objective performance
measures (at least 25%). This results in 396 complete performance documents, 198 from 2003
and 198 from 2004.14
The documents (see Figure 1) provide information about the chosen performance
measures, the targets, the ratings per measure, the total rating, and the old and new total
compensation (salary scale, periodic salary increase and bonus). Furthermore, the documents
indicate the employee’s job, department and supervisor, and an extensive description of the
chosen performance measures. Based on these descriptions I classify each output-based
performance measure as being either objective or subjective, while the competencies are by
definition subjective. This results in two performance dimensions: objective and subjective.15 I
also obtained personal characteristics of the supervisors and employees such as gender and age,
and I interviewed several employees, supervisors, and designers of the system to get a better
understanding of the compensation system and the company background. Finally, I used data
provided by an internal study that examined employee and supervisor reactions to the
compensation system.
14 The data gathered does not cover all the employees meeting these conditions because some of the performance documents were incomplete or were never received by the human resource department. I have checked whether certain departments were underrepresented in the sample but found no such bias. 15 For 63 percent of the employees in the sample the subjective dimension solely consists of competencies, while for the remaining employees only one or two subjective output-based measures were chosen. I examine whether the results presented in this paper are influenced by the type of subjective measures that are used (only competencies or also subjective output-based measures), by re-estimating all models for only those employees that have no subjective output-based measures and by re-estimating all models while excluding the subjective output-based measures. In both cases the results are similar to those presented.
16
Summary statistics for the main variables used in this paper (see Tables 1, 2 and 3)
indicate that supervisors in 2003, on average, provided higher ratings on the subjective than on
the objective dimension. Both the mean (2.94 versus 2.80) and the median (3.00 versus 2.75)
were significantly higher in 2003 (p < 0.005 two-tailed). This relationship reversed in 2004; both
the mean (3.14 versus 3.28) and the median (3.13 versus 3.33) subjective rating were
significantly lower (p < 0.005 two-tailed). The descriptive statistics also show that the variance
in the objective ratings was significantly larger (p < 0.001 two-tailed) than the variance in the
subjective ratings in both 2003 and 2004. As predicted, the subjective ratings seem to be more
lenient (although only in 2003) and more compressed, suggesting the presence of leniency and
centrality bias. In the next section, I examine the existence of leniency and centrality bias more
formally.
Leniency and Centrality Bias
In order to examine the determinants and performance effects of supervisor bias, I first
examine whether the performance ratings are subject to leniency and centrality bias, as predicted
by previous research. To detect leniency, I test whether performance ratings on the subjective
dimension are higher, on average, than performance ratings on the objective dimension, after
controlling for other relevant influences. Similar to Moers (2005), a higher score on the
subjective dimension is considered to be evidence consistent with more lenient ratings. Leniency
bias can only be adequately captured in this way if the performance ratings on the objective
dimension are unbiased or at least significantly closer to the true performance value than the
subjective ratings. Another underlying assumption is that employees have similar ability levels
on the two dimensions. After controlling for effort allocation effects, actual employee
17
performance on the objective and subjective dimension (as opposed to supervisors’ potentially
biased assessments) is therefore assumed to be very similar. Thus, considering the assumption
that the objective performance rating is a good benchmark of actual performance, there should
be, on average, no difference between the ratings on the different dimensions unless the ratings
are biased, after controlling for other relevant influences.
As the dependent variable, I use the employees’ performance ratings on the objective and
subjective dimensions (RATING). This means that each employee is included twice for each
year, once with his objective rating and once with his subjective rating. The influence of
discretion on the ratings is examined by including a dummy variable that equals one if the
observation is a subjective rating and zero otherwise (DSubjectivity). To account for differences in
contract design that may cause different effort allocations, I control for the contractual weight on
the specific dimension (WEIGHT), as a higher weight is expected to lead to higher effort and
consequently to higher ratings. I also include the number of performance measures (NR_PM), as
a larger number of performance measures can lead to incentive dilution. To control for year
effects, I include a year dummy (Y2003) that is one if the observation relates to 2003 and zero
otherwise.
In order to determine whether leniency bias is specific to a certain group of employees, or
is a more general consequence of using subjective performance measures, I include several
additional control variables. First, supervisors are not expected to behave in identical ways when
it comes to rating performance. I include supervisor dummies (DSupervisor) to control for these
differences. Since rating behavior might also be dissimilar for different types of employees, I
also control for three employee variables: age (AGE), gender (SEX), and number of designated
contract hours per week (HOURS). Finally, the way the system is used (e.g., the type of
18
measures that are chosen) and/or the importance attached to the system (e.g., the time that is
provided for instructions and evaluations) might not be identical for different departments and
offices. To control for these differences, office (DOffice) and department dummies (DDepartment) are
included.16
To control for likely correlation of regression model errors for a given employee, I use a
random effects model with robust standard errors. This leads to the following specification:
0 1 2 3 4 2003 5 6
48 53 60
78 49 54
_
J k l
it Subjectivity it it t it i
it j Supervisor k Office l Department i itj k l
RATING D WEIGHT NR PM Y AGE SEX
HOURS D D D
α α α α α α α
α α α α µ ε= = =
= + + + + + +
+ + + + + +∑ ∑ ∑
(1)
where i indicates performance ratings (i= 1, …, 792), t time, j supervisors, k offices, and l
departments. I use random effects17 to obtain consistent estimates of all parameters, including
coefficients of time invariant regressors (Greene, 1993).
I investigate whether the use of subjective measures leads to centrality bias, by
calculating the ratio between employees’ ratings on the objective (subjective) dimension and the
mean rating on the objective (subjective) dimension (R_RATING = Max ((rating/mean rating),
(mean rating/rating)). To justify this method, I assume that the variance in performance is similar
over the two dimensions and that the variance in the objective performance ratings is similar to
the performance variance in the sample. All other variables are the same as in model 1, and a
random effects model with robust standard errors is used, leading to the following empirical
specification:
16 Employees have different tasks and duties that are logically translated into different performance measures. I partly control for these differences by adding supervisor, office, and department dummies. The chosen performance measures per supervisor are, however, also not identical per se. To examine the influence of performance measure differences within reference groups, I redo the analyses for only those observations where the chosen performance measures in the reference group are at least 90% the same in the two consecutive years. The results show similar inferences as those in Table 5, implying that the results are not driven by differences in performance measures within reference groups. 17 The Hausman test indicates that random effects estimation is adequate for models 1, 2, 3 and 4.
19
0 1 2 3 4 2003 5 6
48 53 60
78 49 54
_ _
J k l
it Subjectivity it it t it i
i t j Supervisor k Office l Department i itj k l
R RATING D WEIGHT NR PM Y AGE SEX
HOURS D D D
α α α α α α α
α α α α µ ε= = =
= + + + + + +
+ + + + + +∑ ∑ ∑
(2)
Since the descriptive statistics show different patterns when comparing the objective and
subjective ratings for the two consecutive years, I examine the existence of leniency and
centrality on a yearly basis, in addition to the pooled sample.
The Determinants of Bias
In this section, I examine the determinants of leniency and centrality bias. Although all
the compensation contracts are subject to supervisor discretion, the extent to which ratings are
biased is expected to vary with supervisor preferences. In Table 4, descriptive statistics on
leniency and centrality bias and the discretionary bonus adjustment are presented. They show
that the average amount of leniency bias was larger in 2003 than in 2004, while the average
amount of centrality bias was very similar in the two consecutive years. Supervisors also took
advantage of their ability to make discretionary adjustments, especially in 2003. None of these
adjustments was negative, and the average adjustment was 2.7% (1.5%) in 2003 (2004).
I measure the extent of leniency bias in the subjective rating of a specific employee by
comparing the objective rating to the subjective rating of that employee. As differences between
the objective and subjective rating can be caused by differences in effort allocation on the
different dimensions, I control for both the contractual weight and the number of performance
measures. Moreover, the difference between the objective and subjective rating captures both
leniency and compression applied to the rating (even after controlling for effort allocation
effects). In order to control for the effort allocation effect, and to separate leniency bias from
centrality bias, I use a residual model. I regress the ratio between subjective and objective ratings
20
on the weights, the number of performance measures and on the centrality bias (explained
below), and use the residuals of this regression as a measure of leniency bias (LEN_BIAS).18
Several variables are predicted to affect the extent of leniency bias applied by
supervisors. Since supervisors prefer to avoid negative consequences related to rating
performance, and painful confrontations are more likely to occur when performance is
inadequate, employee performance is predicted to have a negative effect on the extent of
leniency bias. Supervisors might also rate more leniently, especially when employee
performance is poor, to hide departmental problems. I capture employee performance by
including the objective performance ratings (OBJ_R) (see Table 6 for variable definitions).
Confronting employees with harsh but accurate ratings might be less costly to the
supervisor when the personal relationship with the employee is not that strong. I therefore predict
that ratings are less lenient when the employee and supervisor have only started working together
recently. Moreover, since supervisors care about the performance of their department, they are
expected to reduce leniency when rating leniently might give employees the mistaken idea that
they are performing their duties correctly when they are not. Since misunderstandings on how
tasks should be preformed are more likely to occur when the employee and supervisor have
worked together for a short period of time, supervisors are also expected to be less lenient in the
initial stage of the relationship to avoid stimulating dysfunctional behavior. To capture the
influence of the duration of the employee-supervisor relationship, I include a dummy variable
that equals one if the employee joined the company within the last three years, if the employee
recently changed jobs within the company or if a new supervisor was assigned, and zero
otherwise (NEW_R).
18 A problem with this measure is that it assumes leniency bias and centrality bias to be independent. However, if I use the simple ratio between subjective and objective ratings and control for the extent of centrality bias in the regression, I get similar results.
21
The extent of leniency bias is also influenced by the time and effort the supervisor must
invest in collecting performance information. High information-gathering costs are predicted to
result in less precise evaluations, which are expected to lead to more lenient ratings because
leniency decreases the possibility that imprecise ratings lead to problems. I capture information-
gathering costs using the supervisor’s position in the organizational hierarchy. More specifically,
I proxy for information-gathering costs (INFO_C) using the difference between the
organizational level of the supervisor and the employee.19
The effect of the attitude of the supervisor’s superiors towards rating accuracy is
analyzed by examining the financial performance of the offices, as financial conditions are
expected to affect attitudes towards leniency bias. More specifically, poor financial performance
is predicted to put pressure on the office to keep compensation costs down, making the superior
less tolerant of lenient ratings (Longenecker et al., 1987). I measure the financial position of the
offices by including their profit growth (GROWTH) and the difference between their budgeted
and actual profit (BUDGET).
Supervisor and employee characteristics are also expected to influence supervisor bias as
they influence the supervisor’s preferences. I include supervisor gender (SUP_SEX) and
employee gender (SEX) as independent variables because some evidence in the psychology
literature indicates that female supervisors rate more leniently (Tsui & O'Reilly, 1989), and that
female employees are rated lower than male colleagues (Rosen & Jerdee, 1974). Finally, the age
difference between the supervisor and employee (DIF_AGE) is included because research on
relational demography shows that individuals with comparative demographic characteristics are
more likely to develop a close relationship (Tsui & O'Reilly, 1989). A closer relationship is
19 The influence of the supervisor’s position in the organizational hierarchy on rating behavior has been analyzed in the psychology literature, and several studies indicate little interrater agreement on performance ratings provided by supervisors from different organizational levels (Berry, Nelson, & McNally, 1966; DeCotiis & Petit, 1978).
22
expected to have a positive effect on leniency bias (Varma, DeNisi, & Peters, 1996), because it
can increase the utility received from favoring the employee, and/or increase the pain suffered
when the employee needs to be confronted.
Discretionary bonus adjustments provide supervisors with an additional way to influence
compensation. To control for a possible substitution effect, I include the number of percentage
points that the bonus is adjusted beyond the stated range (DIS_ADJ). Finally, I include year
(Y2003) and department dummies (DDepartment) as additional controls.20,.21 A random effects model
at the employee level22 with robust standard errors is used to test the following empirical
specification:
0 1 2 3 4 5
6 7 8 9 10 2003
17
11
_ _ _ _
_ _ _
l
it it it it it it
i i it it t
l Department i itl
LEN BIAS OBJ R NEW R INFO C GROWTH BUDGET
SUP SEX SEX DIF AGE DIS ADJ Y
D
α α α α α αα α α α α
α µ ε=
= + + + + ++ + + + +
+ + +∑
where i relates to employee ratings (i= 1, …, 396), t to time and l to departments. (3)
In order to measure centrality bias, I calculate the ratio between the standard deviation of
all ratings on the objective dimension and the standard deviation of all ratings on the subjective
dimension for each reference group23 (CEN_BIAS). This ratio is the same for all employees in a
reference group, and a high ratio indicates that the subjective ratings are compressed relative to
the objective ratings.
20 Due to high correlation between the variables measuring office performance and the office dummies, the office dummies are not included in the model. 21 The introduction of additional variables that control for the number of employees evaluated by the supervisor and contract choices made by the supervisor, such as the contractual weight placed on subjective measures, the number of subjective measures included and the consistency in the chosen subjective measures within the reference group, leads to similar results. 22 Random effects at the supervisor level lead to similar results except for the age difference between the supervisor and the employee, which in this specification no longer has a negative effect on the extent of leniency bias. 23 A reference group consists of all employees who have a similar function and who are evaluated by the same supervisor.
23
Since the extent of centrality bias is measured per reference group, I examine the
determinants of centrality bias at the supervisor level. I include the same explanatory variables as
in model 3 except that I use the median objective performance rating of the reference group
instead of the individual performance ratings (OBJ_R), and I include supervisor age (SUP_AGE)
instead of the age difference between supervisor and employee (DIF_AGE). Employee gender
(SEX) is also excluded. The predicted signs of the explanatory variables are similar to those in
model 3 apart from the expected sign on office performance. Pressure to keep compensation
costs down is predicted to lead to more, instead of less, compression. Furthermore, the standard
deviation of all objective performance ratings per reference group (REF_SD) is included as an
additional control variable, since the extent to which ratings can be compressed depends on
existing performance variation in the reference group. This leads to the following specification
that is tested with a random effects model with robust standard errors:24, 25
0 1 2 3 4 5
6 7 8 9 10 2003
_ _ _ _
_ _ _ _it it it it it it
i it it it t i it
CEN BIAS OBJ R NEW R INFO C GROWTH BUDGET
SUP SEX SUP AGE DIS ADJ REF SD Y
α α α α α αα α α α α µ ε
= + + + + ++ + + + + + +
where i relates to supervisor rating decisions (i= 1, …, 67), and t to time. (4)
Performance Effects
In this part of the analysis, I investigate how leniency and centrality bias affect the
effectiveness of the compensation system as an incentive provider. I capture the differences in
incentives provided by the compensation system by examining the change in employee
performance following exposure to supervisor discretion. I measure the change between 2003
and 2004 as the supervisors’ influence on incentive provision becomes apparent after the 24 Estimating the model at the employee level with random effects at the supervisor level leads to similar results. 25 As with employee performance, I use the median bonus adjustments per reference group instead of the individual bonus adjustments (DIS_ADJ). Moreover, because of the limited number of observations at the supervisor level (n = 67) the department dummies are not included.
24
evaluation of 2003. Changes in objective performance (∆PERF_O), subjective performance
(∆PERF_S), and total performance (∆PERF_T) are examined because bias is hypothesized to
have an overall influence on employee incentives. Performance change is expected on both the
objective and the subjective dimensions. However, since prior analyses indicate that the
subjective (and therefore the total) ratings are biased, they must be interpreted with care.
Existing variation in the biases provides an opportunity to examine the influence of these
biases on future performance.26 Leniency (LEN_BIAS) and centrality bias (CEN_BIAS) are
measured as before. However, in order to analyze the separate effects of centrality bias on above-
average and below-average performers, I split CEN_BIAS into two variables: CEN_BIASA and
CEN_BIASB. CEN_BIASA (CEN_BIASB) takes the value of the ratio when the employee
performs above (below) the average of his reference group on the objective dimension, and zero
otherwise.
I control for several variables that might influence the change in employees’ performance
ratings. First, discretionary bonus adjustments are expected to have a positive effect on employee
incentives. By adjusting ‘incorrect’ ratings, the supervisor signals that providing employees with
a fair reward is considered important. This reinforces employees’ beliefs that improving
performance will result in higher rewards, which is essential in motivating employees to enhance
performance (Dubinsky & Levy, 1989). Moreover, discretionary adjustments are expected to
have a positive effect on incentives by creating a feeling of reciprocity.27 I control for the effect
26 When developing hypotheses on the influence of centrality bias on incentive effectiveness, I assume that the employees had at least some knowledge of the rating distribution of their reference group. During the interviews, I asked several employees about this matter and they confirmed that they had some, though not full, knowledge of the performance ratings of their peers. Moreover, through the internal study conducted by the company (see page 15), employees confidentially provided information on the perceived fairness of the evaluation procedures and the outcome distributions of the new compensation system, which implies that they possessed knowledge of how rewards were distributed. 27 The employee receives more from the company than the required minimum, making the employee willing to provide more than the minimum effort required (Hannan, Kagel, & Moser, 2002).
25
of discretionary adjustments using the DIS_ADJ variable defined earlier. Discretionary
adjustments are not only expected to influence the motivation of the employees who receive
them, but also the motivation of those who do not but are confronted by their existence.
Discretionary bonus adjustments create a lack of consistency in rewarding procedures. For some
employees actions not included in the compensation contract are taken into account, while for
others they are not. This lack of consistency influences incentives by influencing employee
perceptions of procedural justice within their organization (Parker & Kohlmeyer, 2005). To
control for the influence of discretionary bonus adjustments on employees who do not receive
them, I use a dummy variable that equals one if an adjustment has taken place within the
reference group and zero otherwise (REF_DISADJ).
Third, I control for the maximum amount the performance of the employee can still
improve (the difference between the maximum performance rating (5) and the obtained
performance rating in 2003, MGROWTH), for changes in the contractual incentive weight on the
objective dimension (DIF_WO) and for changes in the number of performance measures
(DIF_NR).
I also include several personal conditions that might make an employee less sensitive to
the incentives provided by the compensation contract. First, employees who have reached the
upper limit of their salary scale might be less receptive to changes in provided incentives as
performance improvement can only lead to a bonus increase, not to a periodic salary increase. I
also include contract hours and age, as I suspect that part-time and older employees are not
motivated in the same way (e.g., they are expected to be less concerned about their future
career). Some employees have occupied a different position and/or were appraised by a different
supervisor in 2004. To ensure this does not drive the results, two dummy variables are included
26
(DDIF_FUN & DDIF_SUP) that equal one if the employee’s function and supervisor, respectively,
have changed, and zero otherwise. Finally, I add office (DOffice) and department dummies
(DDepartment) to control for office-specific and job-specific elements, such as local market
conditions, that might influence employee performance. This leads to the following empirical
specification:28
0 1 2 3 4 5
6 7 8 9 _ 10 _ 11
18 25
12 1314 19
_ _ _ _ _
_ _ _
.
i i
k l
i i i i i i
i i i DIF SUP DIF JOB i
i i k Office l Departmentk l
PERF LEN BIAS CEN BIASA CEN BIASB DIS ADJ REF DISADJ
MGROWTH DIF WO DIF NR D D MAX SC
HOURS AGE D D
β β β β β ββ β β β β β
β β β β ε= =
∆ = + + + + +
+ + + + + +
+ + + + +∑ ∑ i
(5)
where i relates to employees (i= 1, …, 198) and ∆PERF can be ∆PERF_T, ∆PERF_O or ∆PERF_S.
III. RESULTS
Leniency and Centrality Bias
As a first step in my analysis, I examine whether the performance ratings are, on average,
subject to leniency and centrality bias (Table 5). The results show that the ratings on the
subjective dimension are higher than the ratings on the objective dimension, after controlling for
relevant influences, implying that subjective performance ratings are subject to leniency bias.
However, the results of the annual analyses show that the subjective performance ratings are not
more lenient in 2004. Subjectivity even has a marginally significant negative effect on the
ratings. These results indicate that supervisors in general use the leeway they have in subjective
performance appraisal to bias performance ratings, but that this does not necessarily result in
more lenient ratings.
28 I have chosen to model the relationship between centrality and leniency bias and performance change as a linear relationship because I expect the effect of more leniency and centrality bias to be relatively constant over the limited range found in this setting.
27
Consistent with earlier empirical work, I find that the ratings on the subjective
performance dimensions are more compressed than the rating on the objective dimension. The
number of performance measures has a negative effect on the ratings, suggesting that a higher
number of measures leads to lower performance ratings, consistent with the dilution argument.
The Determinants of Bias
Determinants of leniency and centrality bias are investigated in Table 6. Employee
performance is found to be one of the main determinants of leniency bias. The objective
performance ratings negatively affect the extent of leniency bias, which indicates that
supervisors rate more leniently when employee performance is low. The performance of the
reference group has no effect on centrality bias.
Ratings are found to be less lenient when the employee and supervisor only recently
started to work together, indicating that supervisors are more willing to be accurate when
personal ties are not that strong, and when lenient ratings are more likely to provide a wrong
signal on the correctness of employee actions. Although the lack of a strong personal relationship
was predicted to make supervisors more willing to differentiate between employees, ratings are
not found not to be less, but more compressed in the initial period of the employee-supervisor
relationship. A possible explanation for this is that supervisors refrain from differentiating
strongly until they have more information on employee performance.
The proxy for information-gathering costs positively affects the extent of leniency and
centrality bias, indicating that supervisors provide more lenient and compressed ratings when
gathering information is more costly. The evidence also indicates that the office’s financial
position affects supervisor bias. As predicted, both financial performance variables have a
28
positive effect on the extent of leniency bias and a negative effect on the extent of centrality bias.
This indicates that supervisors become more lenient and compress less when the office is
performing better financially.
Supervisor gender is also found to influence rating behavior. I find that male supervisors
are more lenient, not female supervisors, as predicted by previous studies. A speculative
explanation for this finding is that female supervisors are aware of the stereotype and
consequently exercise more caution when rating. Employee gender was not found to influence
the extent of bias.29 Finally, age differences between supervisors and employees negatively affect
the extent of leniency bias, suggesting that it is more costly for supervisors to rate accurately
when they have a strong relationship with the employees.
In sum, the results indicate that supervisors vary the amount of bias they apply to
performance ratings, and that more bias is applied when biased performance ratings increase the
utility the supervisor receives out of rating employee performance.
Performance Effects
The performance effects of leniency and centrality bias are examined in Table 7. The
results30 indicate that supervisor bias has both positive and negative effects on future
performance.31 First, leniency bias has a positive effect on objective and total performance
29 Most studies that find that female employees receive lower ratings than male employees are conducted in settings where the occupation would likely be perceived as masculine (see Landy & Farr, 1980), while this setting has both masculine- and feminine-type positions. However, even after interacting the position’s stereotype with employee gender, I find no evidence of employee gender affecting the extent of bias applied to the ratings. 30 The Breusch-Pagan and White's tests for heteroskedasticity indicate that the data does not suffer from severe heteroskedasticity problems and the Variance Inflation Factors (VIF) indicate the absence of multicollinearity problems. 31 I also examine the performance effects on a more aggregate level. More specifically, I investigate whether the average performance change of all employees who were appraised by the same supervisor is larger (smaller) when that supervisor has a stronger tendency to apply leniency (centrality) bias. To test this I use model 5, but measure the
29
changes, indicating that employees who receive more lenient ratings on the subjective dimension
show more performance improvement on the objective and total dimensions in the following
year. The results also show that centrality bias, for above- and below-average performers, has a
negative effect on performance change on all dimensions, indicating that more compressed
ratings have a negative influence on the performance of all employees.
Regarding the impact of discretionary bonus adjustments, I find that discretionary
adjustments have a positive effect on performance change, while discretionary adjustments made
in the reference group have no effect. This implies that receiving a discretionary adjustment
increases future performance. An alternative explanation is that the ratings in 2004 now portray
the true level of overall performance. Instead of needing an ex post adjustment to give the
employee the fair amount of compensation, the true value is now directly captured by the
measures, resulting in a higher rating. However, even if we cannot be sure that incentives will
improve after employees have received an adjustment, the results show that employees who do
not receive an adjustment do not show a significant negative performance reaction, thereby
taking away an important reason for not adjusting rewards.
The results also show that the employee’s growth potential (MGROWTH) has a positive
effect on performance change. This shows that employees with more room for improvement
improve their ratings to a greater extent. Furthermore, I find that changes in weights and number
of performance measures influence performance change. Being evaluated by a different
supervisor is also found to influence performance; it negatively affects performance changes.
Finally, the results show that age has a negative effect on performance change, implying that
younger employees are more sensitive to incentives.
performance changes and the biases at the supervisor level. The results show that the average extent of leniency (centrality) bias is positively (negatively) related to the average performance change.
30
Robustness Checks
The results show that performance change on the subjective dimension is not
significantly affected by leniency bias. A plausible explanation for this finding is that the
motivational effect caused by the leniency bias in 2003 is offset by the less lenient average
subjective performance ratings provided in 2004. An alternative explanation is that employees
who score higher on the subjective dimension than on the objective dimension (the majority of
the employees) reallocate their effort to the objective dimension because improving a high rating
even further requires more than a linear effort increase. I examine this possibility by including
the quadratic term of the maximum growth potential (MGROWTH) to control for the possible
non-linear effort requirements. The results (not reported) show that the quadratic terms have no
significant effect on performance change and all other inferences remain the same.
I also investigate the influence of effort reallocation on performance change. I do this by
examining the group of employees who are most likely to reallocate their effort from the
subjective to the objective dimension because of non-linear effort requirements; i.e., employees
scoring high on the subjective dimension (top 20%) and scoring higher on the subjective than on
the objective dimension. I use model 5 and replace the leniency bias variable by a dummy
variable that indicates one if the employee belongs to this group and zero otherwise. The results
(not reported) show no relation between the dummy variable and performance change on the
objective dimension, suggesting that the positive relationship between leniency bias and
performance change on the objective dimension is not driven by the reallocation of effort from
the subjective to the objective dimension.
Previously, I implicitly assumed that the performance ratings are exactly proportional to
the bonus percentage. The ratings are, however, linked to a bonus range, not to a concrete
31
percentage. The actual bonus might therefore not be exactly proportional to the rating, even if no
discretionary bonus adjustment has occurred. To ensure that this is not driving the results, I
calculate the difference between the actual bonus percentage and the bonus percentage that
would have been paid if the relation between the rating and the bonus percentage had been
completely proportional. After replacing the discretionary adjustment variable (DIS_ADJ) with
this newly constructed difference variable, I re-estimate specification 5. The results (not
reported) are similar to those in Table 7.32
Additional Analysis
In order to capture the incentive effect, I measure the performance change by comparing
the performance rating of 2003 with the rating of 2004. However, the subjective and therefore
the total rating are biased and the amount of bias applied by the supervisor is not necessarily
consistent over the years. The measurements of the performance change on the subjective and
total dimension are therefore clouded.
To address this limitation, I preform an additional analysis with data collected through a
questionnaire that was part of an internal study performed by FSP.33 In this analysis, instead of
the change in performance, I use the change in effort to capture the incentive effects. The change
in effort is measured by using the responses to a survey question that asked employees how their
effort was affected by the new incentive system (see Table 8 for variable definition). The
questionnaire also asked the employees to provide details on the design of their performance
document and the outcome of the performance measurement process, which allows me to
measure both leniency and centrality bias. The information from the questionnaire is less precise
32 The results of model 3 and 4 also remain unchanged when DIS_ADJ is replaced by this new difference variable. 33 The questionnaire was conducted in august of 2004 and administrated by the designers of the system. I had no influence on the design or execution of the questionnaire.
32
and elaborate than the archival data (the questionnaire was kept anonymous so I was unable to
link the questionnaire to the archival data). The calculations of the biases are therefore not
identical to those performed for the main analysis.34 Finally, the questionnaire provides
information on some of the control variables included in the main model. This leads to the
following specification that is tested with an ordered probit regression model:
0 1 2 3 4 5
6 7
_ _ _ _i i i i i i
i i
EFFORT LEN BIAS CEN BIASA CEN BIASB MAX SC FULLTIME
AGE FEMALE
β β β β β ββ β ε
= + + + + +
+ + + +
(6)
where i relates to employees (i= 1, …, 70).
The results, presented in Table 8, confirm the findings of the main analysis. Leniency
bias has the hypothesized positive effect on effort, while centrality bias has the expected negative
effect. In summary, I find strong support for the prediction that leniency and centrality bias affect
the effectiveness of the compensation system as an incentive provider.
IV. CONCLUSIONS AND FUTURE RESEARCH
Rating inaccuracy caused by supervisor bias is perceived to be one of the main problems
from introducing subjectivity into compensation contracts. This is due to its assumed negative
effect on the compensation system’s ability to motivate employees and provide valuable
information for personnel decisions. Our knowledge of the influence of supervisor bias on the
effectiveness of compensation contracting is, however, extremely limited. Empirical studies
examining the consequences of biased performance ratings are especially lacking.
34 For the calculation of leniency bias in the main analysis I used the percentage of objective and subjective performance measures. Since in the questionnaire there is no information on the performance measures that allows me to distinguish between objective and subjective output-based measures (competences are by definition subjective), I only include those employees that indicated that their output-based measures were very measurable (hence, assuming that for these employees all output-based performance measures were objective). The way centrality bias is calculated is identical but since the questionnaire was only completed by a limited number of employees, the information on the performance of the reference group is very limited.
33
This paper contributes to the literature by investigating what causes supervisors to bias
performance ratings and by analyzing how biased ratings affect employee incentives. The results
indicate that the use of subjective measures leads to more lenient and compressed ratings and
that supervisors bias ratings to a greater extent when bias enhances the utility received out of
rating performance. The results also provide strong support for the prediction that supervisor bias
not only affects current performance ratings, but also future employee incentives. I find that
leniency bias has a positive effect on performance improvement, while centrality bias has a
negative performance effect for both above- and below-average performers.
The empirical finding that supervisor bias can have a positive influence on employee
incentives provides an explanation for earlier empirical findings that show that supervisors in
general do not receive rewards for rating accurately (e.g., Napier & Latham, 1986). Companies
seem to be interested in the effectiveness of performance-based compensation contracts in
increasing employees’ future performance, not necessarily in the accuracy of the performance
appraisal as such (at least not for incentive purposes). Motivation to manage the performance
appraisal process in an efficient manner seems to be provided by linking the supervisors’
compensation and promotion possibilities to their unit’s performance, not to the accuracy of their
unit’s performance ratings.
The findings from this study are not without limitations. First, although there are no
theoretical reasons to expect that the results would not extend to other settings, its
generalizability is reduced by relying on data from one firm. Moreover, I was only able to obtain
information for two consecutive years. As a consequence, I was unable to examine whether the
documented effects of leniency and centrality bias persist over the years. More extensive time-
series data from multiple organizations would remove these limitations. Second, the focus of this
34
study has been on intentional bias. Supervisors can also unintentionally bias performance ratings
due to their cognitive limitations. I was unable to distinguish the extent of distortion caused by
these cognitive limitations and therefore unable to investigate its effect.
Another opportunity for future research is to examine the effect of inaccurate ratings on
other functions of performance ratings. I have limited the study to incentive provision, but
performance ratings are also likely to influence training and promotion decisions, among others.
Finally, this study focuses only on the effects of biased performance ratings on future
performance. Subjectivity in compensation contracting is likely to give rise to additional
concerns, e.g., uncertainty about performance criteria. Investigating how these other concerns, in
combination with biased ratings, influence the effectiveness of compensation contracts would
make an important contribution to the literature.
35
REFERENCES
Akerlof, G. A. & J. L. Yellen, 1988, Fairness and Unemployment, The American Economic Review, 78(2): 44-49. Arkin, R., H. Cooper, & T. Kolditz, 1980, A Statistical Review of the Literature Concerning the Self-Serving Attribution Bias in Interpersonal Influence Situations, Journal of Personality, 48: 435-48. Baiman, S. & M. V. Rajan, 1995, The Informational Advantages of Discretionary Bonus Schemes, The Accounting Review, 70(4): 557-79. Baker, G. P., R. Gibbons, & K. J. Murphy, 1994, Subjective Performance Measures in Optimal Incentive Contracts, Quarterly Journal of Economics, 109(4): 1125-56. Bénabou, R. & J. Tirole, 2003, Intrinsic and Extrinsic Motivation, Review of Economic Studies, 70(3): 489-520. Bénabou, R. & J. Tirole, 2002, Self-Confidence and Personal Motivation, The Quarterly Journal of Economics, 117(3): 871-915. Bernardin, J. H. & R. M. Buckley, 1981, Strategies in Rater Training, Academy of Management Review, 6(2): 205-12. Berry, N. H., P. D. Nelson, & M. McNally, 1966, A Note on Supervisor Ratings, Personnel Psychology, 19: 423-26. Blinder, A. S. & D. H. Choi, 1990, A Shred of Evidence on Theories of Wage Stickiness, The Quarterly Journal of Economics, 105(4): 1003-15. Bretz, R. D., G. T. Milkovich, & W. Read, 1992, The Current State of Performance Appraisal Research and Practice: Concerns, Directions, and Implications, Journal of Management, 18(2): 321-52. Burney, L. L., C. A. Henle, & S. K. Widener, 2006, Do Characteristics of Strategic Performance Measurement Systems Used in Incentives Enhance Organizational Fairness?, Working Paper: 1-41. Colquitt, J. A., D. E. Conlon, M. J. Wesson, C. O. L. H. Porter, & K. Yee Ng, 2001, Justice at the Millennium: A Meta-Analytic Review of 25 Years of Organizational Justice Research, Journal of Applied Psychology, 86(3): 425-45. DeCotiis, T. & A. Petit, 1978, The Performance Appraisal Process: A Model and Some Testable Propositions, Academy of Management Review, 3(3): 635-46.
36
Dubinsky, A. J. & M. Levy, 1989, Influence of Organizational Fairness on Work Outcomes of Retail Salespeople, Journal of Retailing, 65(2): 221-52. Fang, H. & G. Moscarini, 2002, Overconfidence, Morale and Wage-Setting Policies: Mimeo, Yale University. Gabris, G. T. & K. Mitchell, 1988, The Impact of Merit Raise Scores on Employee Attitudes: The Matthew Effect of Performance Appraisal, Public Personnel Management, 17(4): 369–89. Gibbs, M., K. A. Merchant, W. A. Van der Stede, & M. E. Vargus, 2004, Determinants and Effects of Subjectivity in Incentives, The Accounting Review, 79(2): 409-36. Greenberg, J., 1990, Organizational Justice: Yesterday, Today, and Tomorrow, Journal of Management, 16(2): 399-432. Greene, W. H., 1993, Econometric Analysis, 5 ed, New Jersey: Pearson Education, Inc. Hannan, L. R., J. H. Kagel, & D. V. Moser, 2002, Partial Gift Exchange in an Experimental Labor Market: Impact of Subject Population Differences, Productivity Differences, and Effort Requests on Behavior, Journal of Labor Economics, 20(4): 923-51. Harris, M. M., 1994, Rater Motivation in the Performance Appraisal Context: A Theoretical Framework, Journal of Management, 20(4): 737-56. Harris, M. M. & J. Schaubroeck, 1988, A Meta-Analysis of Self-Supervisor, Self-Peer, and Peer-Supervisor Ratings, Personnel Psychology, 41: 43 - 62. Hayes, R. M. & S. Schaefer, 2000, Implicit Contracts and the Explanatory Power of Top Executive Compensation for Future Performance, Rand Journal of Economics, 31(2): 273-93. Heneman, H. G. & D. P. Schwab, 1972, Evaluation of Research on Expectancy Theory Predictions of Employee Performance, Psychological Bulletin, 78(1): 1-9. Holmstrom, B., 1979, Moral Hazard and Observability, Bell Journal of Economics, 10(1): 74-91. Indjejikian, R. J., 1999, Performance Evaluation and Compensation Research: An Agency Perspective, Accounting Horizons, 13(2): 147-57. Ittner, C. D., D. F. Larcker, & M. W. Meyer, 2003, Subjectivity and the Weighting of Performance Measures: Evidence from a Balanced Scorecard, The Accounting Review, 78(3): 725-58. Jawahar, I. M. & C. R. Williams, 1997, Where All the Children Are above Average: The Performance Appraisal Purpose Effect, Personnel Psychology, 50: 905-26. Landy, F. J. & J. L. Farr, 1980, Performance Rating, Psychological Bulletin, 87(1): 72-107.
37
Larwood, L. & W. Whittaker, 1977, Managerial Myopia: Self-Serving Biases in Organizational Planning, Journal of Applied Psychology, 62(2): 194–98. Lazear, E. P., 1991, Labor Economics and the Psychology of Organizations, Journal of Economic Perspectives, 5(2): 89-110. Lipe, M. G. & S. E. Salterio, 2000, The Balanced Scorecard: Judgmental Effects of Common and Unique Performance Measures, The Accounting Review, 75(3): 283-98. Longenecker, C. O., H. P. Sims, & D. A. Gioia, 1987, Behind the Mask: The Politic of Employee Appraisal, The Academy of Management Executive, 1(3): 183-93. McFarlane Shore, L. & G. C. Thornton, 1986, Effects of Gender on Self-and Supervisory Ratings, Academy of Management Journal, 29(1): 115 - 29. McGregor, D. M., 1957, An Uneasy Look at Performance Appraisal, Harvard Business Review, 35(3): 89-94. Meyer, H. H., 1975, The Pay-for-Performance Dilemma, Organizational Dynamics, 3: 39-50. Mischel, W., E. B. Ebbesen, & A. M. Zeiss, 1976, Determinants of Selective Memory About the Self, Journal of Consulting and Clinical Psychology, 44(1): 92-103. Moers, F., 2005, Discretion and Bias in Performance Evaluation: The Impact of Diversity and Subjectivity, Accounting, Organizations and Society, 30(1): 67-80. Murphy, K. R. & J. Cleveland, 1991, Performance Appraisal: An Organizational Perspective, Boston: Allyn and Bacon. Napier, N. K. & G. P. Latham, 1986, Outcome Expectancies of People Who Conduct Performance Appraisals, Personnel Psychology, 39(4): 827-37. O'Connor, N., F. J. Deng, & M. D. Shields, 2006, Determinants of the Subjective Performance Measurement of Managerial Behavior, Working Paper: 1-33. Parker, R. J. & J. M. Kohlmeyer, 2005, Organizational Justice and Turnover in Public Accounting Firms: A Research Note, Accounting, Organizations and Society, 30: 357-69. Pearce, J. L. & L. W. Porter, 1986, Employee Responses to Formal Performance Appraisal Feedback, Journal of Applied Psychology, 71(2): 211-18. Prendergast, C., 1999, The Provision of Incentives in Firms, Journal of Economic Literature, 37(1): 7-63. Prendergast, C. & R. H. Topel, 1993, Discretion and Bias in Performance Evaluation, European Economic Review, 37(2-3): 355-65.
38
Prendergast, C. & R. H. Topel, 1996, Favoritism in Organizations, Journal of Political Economy, 104(5): 958-78. Rosen, B. & T. H. Jerdee, 1974, Influence of Sex Role Stereotypes on Personnel Decisions, Journal of Applied Psychology, 59(1): 9-14. Rynes, S. L., B. Gerhart, & L. Parks, 2005, Personnel Psychology: Performance Evaluation and Pay for Performance, Annual Review of Psychology, 56: 571-600. Tsui, A. S. & C. A. O'Reilly, 1989, Beyond Simple Demographic Effects: The Importance of Relational Demography in Superior-Subordinate Dyads, The Academy of Management Journal, 32(2): 402-23. Varma, A., A. S. DeNisi, & L. H. Peters, 1996, Interpersonal Affect and Performance Appraisals: A Field Study, Personnel Psychology, 49: 341-60. Vroom, V. H., 1964, Work and Motivation: Wiley New York. Weinstein, N. D., 1980, Unrealistic Optimism About Future Life Events, Journal of Personality and Social Psychology, 39(5): 806-20.
39
FIGURE 1 The performance measurement document
Name:Department:Position:Supervisor:
EvaluationBad Regular Good Very Good Excellent
1) Ο Ο Ο Ο Ο
2) Ο Ο Ο Ο Ο
3) Ο Ο Ο Ο Ο
4) Ο Ο Ο Ο Ο
5) Ο Ο Ο Ο Ο
6) Ο Ο Ο Ο Ο
Comments
EvaluationBad Regular Good Very Good Excellent
1) Ο Ο Ο Ο Ο
2) Ο Ο Ο Ο Ο
3) Ο Ο Ο Ο Ο
4) Ο Ο Ο Ο Ο
5) Ο Ο Ο Ο Ο
6) Ο Ο Ο Ο Ο
Comments
Total rating output dimension 50%
Total rating competence dimension 50%
Total score
Salary Scale Growth % Bonus
Salary Scale Growth % Bonus
PERFORMANCE DOCUMENT
Salary Determination
Good Very Good
Output performance measures
Competence performance measures
Current Salary
New Salary
Excellent
0% 1-3% 4-8% 9-12% 13-15%
Bad Regular
40
TA
BL
E 1
D
escr
ipti
ve s
tati
stic
s: P
art
I
20
03
2004
V
aria
ble
Mea
n
SD
M
edia
n R
ang
e
n
M
ean
S
D
Med
ian
Ra
nge
n O
vera
ll p
erfo
rma
nce
ratin
g
2.
87
0.
52
2.
86
1.6
– 4.
7
198
3.
20
0.
50
3.
23
1.6
– 4.
4
198
Obj
ectiv
e pe
rfo
rma
nce
ratin
g
2.80
0.76
2.75
1.0
– 5.
0
198
3.
28
0.
74
3.
33
1.0
– 5.
0 1
98
Sub
ject
ive
perf
orm
anc
e ra
ting
2.94
0.46
3.00
1.
5 –
4.3
19
8
3.14
0.45
3.13
2.
1 –
4.6
198
T
otal
# o
f per
form
anc
e m
easu
res
9.70
1.77
1
0.00
5 –
12
198
10.6
8
1.49
1
1.00
5
– 1
2
198
# o
f obj
ectiv
e pe
rfo
rma
nce
mea
sure
s
4.
04
1.3
5
4.00
1
– 6
198
4.
48
1.
30
5.
00
2
– 6
198
# o
f su
bjec
tive
perf
orm
anc
e m
easu
res
5.
66
1.
36
6.
00
2
– 9
198
6.
20
1.
06
6.
00
3
– 8
198
% o
f tot
al r
atin
g o
bjec
tive
ly d
eter
min
ed
44.3
3
8.7
2
50.
00
25 –
50
19
8
4
4.37
7
.57
50
.00
25
– 50
19
8 %
of t
otal
rat
ing
subj
ectiv
ely
det
erm
ined
55.6
7
8.7
2
50
.00
50
– 75
19
8
5
5.63
7
.57
5
0.00
50
– 7
5
198
41
TABLE 2 Descriptive statistics: Part II
Variable Mean SD Median Range n
Age 38.39 8.60 37 23 – 60 198 Supervisor age 39.76 7.54 37 28 – 57 41 Designated contract hours in 2003 32.68 6.52 36 12 – 40 198 Designated contract hours in 2004 32.27 6.80 36 12 – 40 198 Profit growth in 2003a 1.09 2.62 1.60 -6.2 – 11.9 198 Profit growth in 2004a 5.68 18.21 1.27 -20.3 – 67.2 198
Budgeted versus actual profit in 2003b 3.61 3.56 1.64 1.6 – 12.6 198 Budgeted versus actual profit in 2004b 4.14 4.05 4.24 -4.2 – 7.81 198
Variable 2003 2004
% of employees at their salary scale maximum 26.26 31.31 % of female employees 56.06 56.06 Number of supervisors 35 32 % of female supervisors 28.57 40.63
a The percentage difference between the offices’ profit this year and the offices’ profit last year. b The percentage difference between the offices’ budgeted profit growth and the actual profit growth.
42
TA
BL
E 3
C
orre
lati
on b
etw
een
vari
able
s
198
obse
rvat
ions
for
200
3
1
2
3
4
5
6
7
8
9
10
1.
Ove
rall
per
form
anc
e ra
ting
2.
Obj
ectiv
e pe
rfo
rma
nce
ratin
g
0.
90***
3.
S
ubje
ctiv
e pe
rfo
rma
nce
ratin
g
0.80***
0.47
***
4.
Tot
al #
of p
erfo
rma
nce
m.
-0.
14**
-0
.11
-0.
13*
5.
#
of o
bjec
tive
perf
orm
anc
e m
.
-0.
12*
-0.1
7**
-0.
02
0.6
5***
6.
# o
f su
bjec
tive
perf
orm
anc
e m
.
-0.
06
0
.03
-0.
15**
0
.66*
**
-0.
15**
7.
%
obj
ectiv
ely
det
erm
ined
-
0.10
-0.1
6**
0.02
0
.10
0
.52***
-0
.39*
**
8.
Em
plo
yee
age
-0.
02
-0
.03
-0.
03
-
0.04
-0.
02
-
0.03
-
0.03
9.
E
mp
loye
e co
ntra
ct h
our
s
0.
18**
0
.19**
*
0.1
2*
0.
30**
*
0.1
3*
0.26
***
0.04
-0.0
4 10
. Pro
fit g
row
th
-0.
16**
-0
.15*
*
-0.
12
-
0.12*
-0
.05
-0.
10
-0.
01
0.
04
-0
.17
**
11. B
udge
ted
vers
us a
ctua
l pro
fit
0.
24***
0
.12*
0.
35**
*
-0.0
6
0
.17**
-
0.25
***
0.09
-
0.16*
*
0.02
-0.1
1 19
8 ob
serv
atio
ns f
or 2
004
1.
Ove
rall
per
form
anc
e ra
ting
2.
O
bjec
tive
perf
orm
anc
e ra
ting
0.
88***
3.
Sub
ject
ive
perf
orm
anc
e ra
ting
0.
78***
0.41
***
4.
T
otal
# o
f per
form
anc
e m
.
-
0.08
-
0.08
-0
.05
5.
# o
f obj
ectiv
e pe
rfo
rma
nce
m.
0.02
-
0.03
0.07
0
.72*
**
6.
# o
f su
bjec
tive
perf
orm
anc
e m
.
-0.
14**
-0.
08
-0.1
6**
0
.53*
**
-0.2
1***
7.
% o
bjec
tive
ly d
eter
min
ed
0.
18***
0.10
0.19*
**
-0.0
5
0.5
7***
-
0.77
***
8.
E
mp
loye
e ag
e
-0.
22***
-0.
25**
*
-0.
09
0.0
4
-0.0
2
0.0
8
-
0.06
9.
E
mp
loye
e co
ntra
ct h
our
s
0.
14**
0.
15**
0.11
0
.10
0
.08
0
.04
-0.
01
0.
00
10. P
rofit
gro
wth
-
0.08
-0.
09
-0.0
4
-0.2
8***
-0
.19*
**
-0.
16**
-0.1
0
-0.0
1
0.08
11
. Bud
gete
d ve
rsus
act
ual p
rofit
0.12
0.13
*
0.03
0
.36**
*
0.1
0
0.3
9***
-0
.02
0.07
0.
14*
-0.3
0***
**
*, *
*, *
is s
tatis
tica
lly s
igni
fica
nt a
t res
pect
ivel
y th
e 1%
, 5%
, an
d 10
% le
vel (
two-
taile
d).
43
TABLE 4 Descriptive statistics: Part III
Variable Mean SD Median Range n
2003 Leniency biasa 0.06 0.32 0.00 -0.44 – 1.75 198 Centrality biasb 2.18 1.41 1.92 0.32 – 5.95 198 Leniency bias per supervisorc 0.06 0.22 0.03 -0.29 – 0.80 35 Centrality bias per supervisor 2.22 1.36 2.17 0.40 – 5.95 35 2004 Leniency bias -0.06 0.27 -0.11 -0.50 – 1.61 198 Centrality bias 2.20 1.15 1.92 0.52 – 6.05 198 Leniency bias per supervisor -0.06 0.14 -0.10 -0.25 – 0.40 32 Centrality bias per supervisor 2.10 1.00 1.88 0.68 – 5.10 32
Variable 2003 2004
Discretionary bonus adjustment 35% 19% Average bonus adjustmentd 2.7% 1.5%
a Leniency bias is captured by regressing the ratio between the subjective and objective ratings on the weights, the number of performance measures and on the centrality bias, and using the residuals of this regression as a measure of leniency bias. b Centrality bias is measured as the ratio between the standard deviation of all performance ratings on the objective dimension and the standard deviation of all performance ratings on the subjective dimension for each reference group. c The average amount of bias applied per supervisor. d The average bonus adjustment once an adjustment is made. All adjustments were of a positive nature.
44
TA
BL
E 5
The
impa
ct o
f su
bjec
tivi
ty o
n th
e pe
rfor
man
ce r
atin
g an
d on
its
vari
atio
n
2003
2004
Inde
pend
ent
vari
able
s a
RA
TIN
G
R_R
AT
ING
R
AT
ING
R
_RA
TIN
G
RA
TIN
G
R_R
AT
ING
D
Su
bje
ctiv
ity
0.10
**
(2.1
1)
-0.1
4***
(-
7.35
) 0.
29**
* (4
.80)
-0
.15*
**
(-5.
70)
-0.0
9 (-
1.44
) -0
.11*
**
(-4.
82)
WE
IGH
T
0.00
(0
.20)
0.
00
(1.3
3)
-0.0
0 (-
0.86
) 0.
01**
(2
.04)
0.
01
(0.8
8)
0.00
(1
.54)
NR
_P
M
-0.0
6***
(-2
.76)
-0
.00
(-0.
25)
-0.0
6**
(-2.
31)
-0.0
1 (-
1.27
) -0
.07
(-1.
19)
-0.0
2 (-
1.32
)
Y 20
03
-0.4
0***
(-9
.80)
0.
05**
* (3
.28)
AG
E
-0.
01**
(-2
.47)
0.
01*
(1.7
6)
-0.0
1 (-
1.34
) -0
.00
(-0.
20)
-0.0
1***
(-
2.69
) 0.
01**
* (2
.66)
SE
X
-0.1
2
(-1
.38)
0.
05*
(1.8
1)
-0.1
0 (-
1.14
) 0.
04
(1.1
7)
-0.1
7 (-
1.55
) 0.
07*
(1.8
7)
HO
UR
S
0.0
1
(1
.43)
-0
.00
(-1.
14)
0.01
(1
.50)
-0
.00
(-1.
12)
0.01
(1
.08)
-0
.00
(-1.
05)
R2 W
ithin
B
etw
een
O
vera
ll
0.
22
0.41
0.
31
0.14
0.
32
0.20
0.09
0.
58
0.43
0.22
0.
42
0.
33
0.05
0.
37
0.26
0.14
0.
32
0.24
***,
**,
* in
dic
ate
stat
istic
al s
igni
fica
nce
at r
espe
ctiv
ely
the
1%
, 5%
and
10%
leve
l (tw
o-t
aile
d).
Z-v
alu
es a
re i
n pa
rent
hese
s an
d n
= 7
92 fo
r th
e fir
st t
wo
regr
essi
ons
and
n =
396
for
the
last
four
.
(Co
ntin
ued
on
next
pa
ge)
45
T
AB
LE
5 (
cont
inue
d)
RA
TIN
G
Per
form
anc
e ra
ting,
on
the
obj
ectiv
e o
r th
e su
bje
ctiv
e d
ime
nsio
n R
_R
AT
ING
Per
form
anc
e ra
tings
var
iatio
n, m
easu
red
as
the
ratio
bet
wee
n e
mp
loye
es’
per
form
anc
e ra
tings
on
the
obj
ectiv
e
(sub
ject
ive)
dim
ens
ion
and
the
m
ean
perf
orm
anc
e ra
ting
on
the
obj
ect
ive
(su
bjec
tive)
dim
ens
ion:
Ma
x ((
ratin
g/m
ean
ratin
g),
(mea
n ra
ting/
ratin
g))
DS
ub
ject
ivity
D
um
my
varia
ble
that
equ
als
1 if
the
obs
erva
tion
refe
rs t
o a
sub
ject
ive
perf
orm
anc
e ra
ting
and
0 o
ther
wis
e
WE
IGH
T
Co
ntra
ctua
l ince
ntiv
e w
eig
ht o
n th
e d
ime
nsio
n (o
bjec
tive
or
subj
ec
tive)
to
whi
ch t
he o
bser
vatio
n re
late
s N
R_
PM
N
um
ber
of p
erfo
rma
nce
mea
sure
s o
n th
e d
ime
nsio
n (o
bjec
tiv
e o
r su
bjec
tive)
to
whi
ch t
he o
bser
vatio
n re
late
s Y 2
00
3 D
um
my
varia
ble t
hat
equa
ls 1
if t
he o
bser
vatio
n re
late
s to
200
3 an
d 0
othe
rwi
se
AG
E
Em
plo
yee
age
S
EX
E
mp
loye
e ge
nder
(0
for
fem
ale
and
1 fo
r m
ale
) H
OU
RS
E
mp
loye
e de
sig
nate
d co
ntra
ct h
our
s pe
r w
eek
a
An
inte
rcep
t, a
s w
ell a
s su
perv
iso
r, d
epar
tme
nt a
nd o
ffice
du
mm
ies
are
incl
uded
but
not
rep
ort
ed.
46
TABLE 6
The determinants of leniency and centrality bias
Independent variables a
LEN_BIAS
CEN_BIAS
OBJ_R b -0.33***
(-16.07) 0.19
(0.52)
NEW_R -0.05** (-2.12)
0.64* (1.78)
INFO_C 0.04*
(1.78) 0.48*
(1.69)
GROWTH 0.02** (2.08)
-0.02** (-2.54)
BUDGET 0.01***
(3.44) -0.04
(-1.54)
SUP_SEX 0.11*** (4.05)
-0.08 (-0.23)
SEX 0.03
(1.26)
DIF_AGE -0.01*
(-1.73)
SUP_AGE -0.01*
(-1.73) 0.02
(1.38) DIS_ADJ b 0.00
(0.44) -0.23
(-1.33) Y2003 -0.06***
(-3.22) 0.58*
(1.67) REF_SD 2.30***
(3.71)
R2 Within Between
Overall
0.73 0.63 0.67
0.55 0.29 0.33
***, **, * indicate statistical significance at respectively the 1%, 5% and 10% level (two-tailed). Z-values are in parentheses and n = 396 for the first and n = 67 for the second regression .
(Continued on next page)
47
TABLE 6 (continued)
LEN_BIAS Leniency bias, measured as the residuals of the regression of the ratio
between the subjective performance rating and the objective performance rating on the contractual weights, the number of performance measures and the centrality bias
CEN_BIAS Centrality bias, measured as the ratio between the standard deviation of all performance ratings on the objective dimension and the standard deviation of all performance ratings on the subjective dimension for each reference group
OBJ_R Objective performance rating NEW_R Dummy variable that equals 1 if the employee joined the company in the
last three years, if the employee recently changed jobs within the company or if a new supervisor was assigned, and 0 otherwise
INFO_C Dummy variable that is 0 if the supervisor and the employee have the same organizational level and 1 if the supervisor has a higher organizational level
GROWTH Percentage difference between the profit of this year and the profit of last year calculated per office per year
BUDGET Percentage difference between budgeted profit growth and actual profit growth calculated per office per year
SUP_SEX Supervisor gender (0 for female and 1 for male) SEX Employee gender (0 for female and 1 for male) DIF_AGE Absolute difference between supervisor age and employee age SUP_AGE Supervisor age DIS_ADJ Difference between the rewarded bonus percentage and the maximum
of the stated bonus range Y2003 Dummy variable that equals 1 if the observation relates to 2003 and 0
otherwise REF_SD Standard deviation of all objective performance ratings within the same
reference group a An intercept and department dummies are included but not reported. b Since the determinants of centrality bias are examined at the supervisor level the median objective performance rating per reference group and the median bonus adjustment per reference group are used in the second regression.
48
TABLE 7
Regression of the effects of supervisor discretion on incentive provision (I)
Independent variables a
∆PERF_O
∆PERF_S
∆PERF_T
LEN_BIAS 0.73*** (3.00)
-0.10 (-0.94)
0.22* (1.89)
CEN_BIASA -0.14*** (-3.12)
-0.04* (-1.72)
-0.08*** (-3.08)
CEN_BIASB -0.13*** (-3.15)
-0.06*** (-2.69)
-0.09*** (-3.69)
DIS_ADJ 0.06* (1.86)
0.07*** (4.01)
0.07*** (3.65)
REF_DISADJ 0.01 (0.11)
0.07 (0.93)
0.02 (0.27)
MGROWTHb 0.27** (2.26)
0.45*** (6.78)
0.36*** (4.66)
DIF_WO 0.01** (2.07)
0.00 (0.67)
0.01** (2.36)
DIF_NRc -0.02 (-0.53)
-0.06* (-1.79)
-0.02 (-1.34)
DDIF_ SUP -0.18 (-1.46)
-0.13* (-1.91)
-0.16** (-2.14)
DDIF_JOB 0.17 (0.85)
-0.04 (-0.32)
0.07 (0.56)
MAX_SC -0.18 (-1.42)
0.05 (0.68)
-0.06 (-0.80)
HOURS
-0.01 (-0.73)
0.00 (0.10)
-0.00
(-0.42)
AGE
-0.02*** (-2.64)
-0.01** (-2.27)
-0.01*** (-3.18)
R2-ADJ
0.47
0.46
0.51
***, **, * indicate statistical significance at respectively the 1%, 5%, and 10% (two-tailed). T-values are in parentheses and n = 198 for all three regressions.
(Continued on next page)
49
TABLE 7 (continued)
∆PERF_O Difference between the objective performance rating of 2003 and the objective performance rating of 2004 ∆PERF_S Difference between the subjective performance rating of 2003 and the subjective performance rating of 2004 ∆PERF_T Difference between the total performance rating of 2003 and the total performance rating of 2004 LEN_BIAS Leniency bias, measured as the residuals of the regression of the ratio
between the subjective performance rating and the objective performance rating on the contractual weights, the number of performance measures and the centrality bias
CEN_BIASA Variable that equals CEN_BIAS if the observation relates to an above-average performer and 0 otherwise CEN_BIASB Variable that equals CEN_BIAS if the observation relates to a below- average performer and 0 otherwise DIS_ADJ Difference between the rewarded bonus percentage and the maximum
of the stated bonus range REF_DISADJ Dummy variable that equals 1 if a bonus adjustment was made within the reference group and 0 otherwise MGROWTH Growth potential, measured as the difference between the maximum
performance rating (5) and the obtained performance rating in 2003 DIF_WO Difference between the contractual incentive weight on the objective dimension in 2003 and in 2004 DIF_NR Difference between the number of performance measures included in
2003 and the number included in 2004 DDIF_SUP Dummy variable that equals 1 if the employee’s supervisor is different in 2004 than in 2003 and 0 otherwise DDIF_JOB Dummy variable that equals 1 if the employee’s function is different in 2004 than in 2003 and 0 otherwise MAX_SC Dummy variable that equals 1 if the employee has reached the maximum of his salary scale and 0 otherwise HOURS Employee designated contract hours per week AGE Employee age a An intercept, as well as department and office dummies are included but not reported. b For regressions with the dependent variable ∆PERF_O, ∆PERF_S and ∆PERF_T I use the objective rating, the subjective rating and the total rating of 2003, respectively, to calculate MGROWTH. c For regressions with the dependent variable ∆PERF_O, ∆PERF_S and ∆PERF_T I use the number of objective, subjective and total performance measures, respectively, to calculate DIF_NR.
50
TABLE 8
Regression of the effects of supervisor discretion on incentive provision (II)
Independent variables a
EFFORT
LEN_BIAS 0.39* (1.65)
CEN_BIASA -0.27† (-1.44)
CEN_BIASB -0.72** (-2.24)
MAX_SC -0.58 (-1.43)
FULLTIME 0.39 (0.97)
AGE -0.02 (-1.19)
FEMALE 0.29 (0.81)
McFadden’s Pseudo R2
0.10
***, **, *, † indicate statistical significance at respectively the 1%, 5%, 10% and 15% (two-tailed). T-values are in parentheses and n = 70.
EFFORT Change in employee effort. The change is measured by using the
employee’s assessment (captured on a 5-item Likert scale) of how the new incentive system has affected his effort.
LEN_BIAS Leniency bias, measured as the residuals of the regression of the ratio between the performance rating on the competence dimension and the performance rating on the output-based dimension on the number of performance measures and the centrality bias, for all those employees who have indicated in the questionnaire that their output-based performance measures were very measurable
CEN_BIASA Variable that equals CEN_BIAS if the observation relates to an above-average performer and 0 otherwise CEN_BIASB Variable that equals CEN_BIAS if the observation relates to a below- average performer and 0 otherwise
(Continued on next page)
51
TABLE 8 (continued)
MAX_SC Dummy variable that equals 1 if the employee has reached the maximum of his salary scale and 0 otherwise FULLTIME Dummy variable that equals 1 if the employee works fulltime and 0
otherwise AGE Employee age FEMALE Dummy variable that equals 1 if the employee is female and 0 otherwise a The intercept is included, but not separately reported.
Top Related